2024-09-16T01:20:35.509 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T01:20:35.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T01:20:35.624 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906474 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906474' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 8921c8ef947807b717db041d85f0761b7914d5fc subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi139.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMCYcLNvPY2lbyVVABgkEjhLPasfPEsVutQGHHkDJAn01t4bINN4T1zbdcc1IDXHml840NEjb2NWTw0yXAkYhVg= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKAA64hsEQwGjTl5fmsPKn7Ze88POVrn1y83LjjG/kv/YtrGCUnBtwmlDJ44EJdEHRb/x2dBFnXjqtvA1mrEetI= smithi176.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF09CYB5EHjINFiXTTByuk83p1VB+wYkmTLBlbZNOxFh+X1yZAv7bcdef6UHPWIDzsH2za4jWYfqZojVM0Y+fno= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T01:20:35.624 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-16T01:20:35.625 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-16T01:20:35.625 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T01:20:35.626 INFO:teuthology.task.internal:Checking packages... 2024-09-16T01:20:35.648 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-16T01:20:35.648 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T01:20:35.648 INFO:teuthology.packaging:ref: None 2024-09-16T01:20:35.648 INFO:teuthology.packaging:tag: None 2024-09-16T01:20:35.648 INFO:teuthology.packaging:branch: squid 2024-09-16T01:20:35.648 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:20:35.649 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-16T01:20:35.847 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-328.gad35db06 2024-09-16T01:20:35.848 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T01:20:35.856 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T01:20:35.856 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T01:20:35.876 INFO:teuthology.task.internal:Saving configuration 2024-09-16T01:20:35.897 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T01:20:35.903 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T01:20:35.929 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906474', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 01:15:22.331931', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMCYcLNvPY2lbyVVABgkEjhLPasfPEsVutQGHHkDJAn01t4bINN4T1zbdcc1IDXHml840NEjb2NWTw0yXAkYhVg='} 2024-09-16T01:20:35.950 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906474', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 01:15:22.333284', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKAA64hsEQwGjTl5fmsPKn7Ze88POVrn1y83LjjG/kv/YtrGCUnBtwmlDJ44EJdEHRb/x2dBFnXjqtvA1mrEetI='} 2024-09-16T01:20:35.974 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi176.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906474', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 01:15:22.334415', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF09CYB5EHjINFiXTTByuk83p1VB+wYkmTLBlbZNOxFh+X1yZAv7bcdef6UHPWIDzsH2za4jWYfqZojVM0Y+fno='} 2024-09-16T01:20:35.974 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T01:20:35.981 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-16T01:20:35.981 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-16T01:20:35.981 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-16T01:20:35.981 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T01:20:36.115 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2414ddbbe0>, signals=[15]) 2024-09-16T01:20:36.115 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T01:20:36.122 INFO:teuthology.task.internal:Opening connections... 2024-09-16T01:20:36.122 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T01:20:36.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:20:36.204 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T01:20:36.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:20:36.278 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi176.front.sepia.ceph.com 2024-09-16T01:20:36.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:20:36.354 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T01:20:36.361 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2024-09-16T01:20:36.380 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2024-09-16T01:20:36.380 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2024-09-16T01:20:36.438 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2024-09-16T01:20:36.438 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2024-09-16T01:20:36.438 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2024-09-16T01:20:36.438 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2024-09-16T01:20:36.438 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2024-09-16T01:20:36.438 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2024-09-16T01:20:36.438 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T01:20:36.438 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2024-09-16T01:20:36.438 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2024-09-16T01:20:36.439 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T01:20:36.439 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2024-09-16T01:20:36.439 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T01:20:36.439 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T01:20:36.439 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T01:20:36.439 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2024-09-16T01:20:36.458 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-09-16T01:20:36.477 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-09-16T01:20:36.477 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-09-16T01:20:36.535 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-09-16T01:20:36.535 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-09-16T01:20:36.535 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-09-16T01:20:36.535 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-09-16T01:20:36.536 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-09-16T01:20:36.536 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-09-16T01:20:36.536 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T01:20:36.536 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-09-16T01:20:36.536 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-09-16T01:20:36.536 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T01:20:36.536 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-09-16T01:20:36.536 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T01:20:36.536 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T01:20:36.536 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T01:20:36.537 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-09-16T01:20:36.559 DEBUG:teuthology.orchestra.run.smithi176:> uname -m 2024-09-16T01:20:36.577 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2024-09-16T01:20:36.578 DEBUG:teuthology.orchestra.run.smithi176:> cat /etc/os-release 2024-09-16T01:20:36.636 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="CentOS Stream" 2024-09-16T01:20:36.636 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="9" 2024-09-16T01:20:36.636 INFO:teuthology.orchestra.run.smithi176.stdout:ID="centos" 2024-09-16T01:20:36.636 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE="rhel fedora" 2024-09-16T01:20:36.636 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="9" 2024-09-16T01:20:36.636 INFO:teuthology.orchestra.run.smithi176.stdout:PLATFORM_ID="platform:el9" 2024-09-16T01:20:36.636 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T01:20:36.637 INFO:teuthology.orchestra.run.smithi176.stdout:ANSI_COLOR="0;31" 2024-09-16T01:20:36.637 INFO:teuthology.orchestra.run.smithi176.stdout:LOGO="fedora-logo-icon" 2024-09-16T01:20:36.637 INFO:teuthology.orchestra.run.smithi176.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T01:20:36.637 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://centos.org/" 2024-09-16T01:20:36.637 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T01:20:36.637 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T01:20:36.637 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T01:20:36.637 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2024-09-16T01:20:36.667 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T01:20:36.678 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T01:20:36.684 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T01:20:36.684 DEBUG:teuthology.orchestra.run.smithi139:> test '!' -e /home/ubuntu/cephtest 2024-09-16T01:20:36.686 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-09-16T01:20:36.688 DEBUG:teuthology.orchestra.run.smithi176:> test '!' -e /home/ubuntu/cephtest 2024-09-16T01:20:36.704 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T01:20:36.710 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T01:20:36.710 DEBUG:teuthology.orchestra.run.smithi139:> test -z $(ls -A /var/lib/ceph) 2024-09-16T01:20:36.744 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-09-16T01:20:36.746 DEBUG:teuthology.orchestra.run.smithi176:> test -z $(ls -A /var/lib/ceph) 2024-09-16T01:20:36.992 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T01:20:37.067 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T01:20:37.081 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-16T01:20:37.081 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-16T01:20:37.081 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-16T01:20:37.082 DEBUG:teuthology.orchestra.run.smithi139:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T01:20:37.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:20:37.099 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-09-16T01:20:37.158 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-503.el9.x86_64 2024-09-16T01:20:37.158 INFO:teuthology.task.kernel:Running kernel on smithi139: 5.14.0-503.el9.x86_64 2024-09-16T01:20:37.158 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum install -y kernel 2024-09-16T01:20:39.547 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-16T01:20:41.423 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-16T01:20:46.024 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 6.5 MB 00:00 2024-09-16T01:20:47.524 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-16T01:20:48.445 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-16T01:20:53.817 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-16T01:20:55.865 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T01:20:55.865 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T01:20:55.941 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T01:20:55.950 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T01:20:55.950 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-09-16T01:20:55.950 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T01:20:55.950 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T01:20:55.950 INFO:teuthology.orchestra.run.smithi139.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T01:20:55.950 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-16T01:20:55.950 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T01:20:55.951 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T01:20:55.951 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T01:20:55.951 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:20:55.951 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T01:20:55.951 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-16T01:20:55.951 INFO:teuthology.orchestra.run.smithi139.stdout:Install 4 Packages 2024-09-16T01:20:55.951 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:20:55.951 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 79 M 2024-09-16T01:20:55.952 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 126 M 2024-09-16T01:20:55.952 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T01:20:56.168 INFO:teuthology.orchestra.run.smithi139.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 192 kB/s | 23 kB 00:00 2024-09-16T01:20:57.094 INFO:teuthology.orchestra.run.smithi139.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-09-16T01:20:57.710 INFO:teuthology.orchestra.run.smithi139.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-09-16T01:20:58.019 INFO:teuthology.orchestra.run.smithi139.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 15 MB/s | 29 MB 00:01 2024-09-16T01:20:58.019 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:20:58.019 INFO:teuthology.orchestra.run.smithi139.stdout:Total 38 MB/s | 79 MB 00:02 2024-09-16T01:20:58.700 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T01:20:59.050 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T01:20:59.051 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T01:21:00.000 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T01:21:00.000 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T01:21:02.463 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T01:21:03.193 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T01:21:03.285 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T01:21:06.097 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T01:21:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T01:21:11.060 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T01:21:11.305 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:21:16.141 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:21:54.655 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:21:54.704 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:21:55.078 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:21:55.078 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T01:21:55.078 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T01:21:55.078 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T01:21:55.926 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T01:21:55.926 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:21:55.926 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T01:21:55.926 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T01:21:55.926 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T01:21:55.926 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T01:21:55.927 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T01:21:55.927 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:21:55.927 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T01:21:56.285 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T01:21:56.372 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T01:21:56.373 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T01:21:56.373 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T01:21:56.373 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T01:21:56.373 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T01:21:56.373 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-16T01:21:56.373 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi139.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T01:21:56.373 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T01:21:56.433 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T01:21:56.433 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T01:21:56.433 DEBUG:teuthology.orchestra.run.smithi139:> sudo rpm -qi grub2-tools 2024-09-16T01:21:56.473 INFO:teuthology.orchestra.run.smithi139.stdout:Name : grub2-tools 2024-09-16T01:21:56.473 INFO:teuthology.orchestra.run.smithi139.stdout:Epoch : 1 2024-09-16T01:21:56.473 INFO:teuthology.orchestra.run.smithi139.stdout:Version : 2.06 2024-09-16T01:21:56.473 INFO:teuthology.orchestra.run.smithi139.stdout:Release : 82.el9 2024-09-16T01:21:56.473 INFO:teuthology.orchestra.run.smithi139.stdout:Architecture: x86_64 2024-09-16T01:21:56.473 INFO:teuthology.orchestra.run.smithi139.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T01:21:56.474 INFO:teuthology.orchestra.run.smithi139.stdout:Group : Unspecified 2024-09-16T01:21:56.474 INFO:teuthology.orchestra.run.smithi139.stdout:Size : 8274670 2024-09-16T01:21:56.474 INFO:teuthology.orchestra.run.smithi139.stdout:License : GPLv3+ 2024-09-16T01:21:56.474 INFO:teuthology.orchestra.run.smithi139.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T01:21:56.474 INFO:teuthology.orchestra.run.smithi139.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T01:21:56.474 INFO:teuthology.orchestra.run.smithi139.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T01:21:56.474 INFO:teuthology.orchestra.run.smithi139.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T01:21:56.474 INFO:teuthology.orchestra.run.smithi139.stdout:Packager : builder@centos.org 2024-09-16T01:21:56.474 INFO:teuthology.orchestra.run.smithi139.stdout:Vendor : CentOS 2024-09-16T01:21:56.474 INFO:teuthology.orchestra.run.smithi139.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T01:21:56.474 INFO:teuthology.orchestra.run.smithi139.stdout:Summary : Support tools for GRUB. 2024-09-16T01:21:56.474 INFO:teuthology.orchestra.run.smithi139.stdout:Description : 2024-09-16T01:21:56.474 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:21:56.474 INFO:teuthology.orchestra.run.smithi139.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T01:21:56.475 INFO:teuthology.orchestra.run.smithi139.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T01:21:56.475 INFO:teuthology.orchestra.run.smithi139.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T01:21:56.475 INFO:teuthology.orchestra.run.smithi139.stdout:hardware devices. 2024-09-16T01:21:56.475 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:21:56.475 INFO:teuthology.orchestra.run.smithi139.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T01:21:56.477 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T01:21:56.477 INFO:teuthology.task.kernel:Updating grub on smithi139 to boot 5.14.0-505.el9.x86_64 2024-09-16T01:21:56.477 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T01:21:57.150 INFO:teuthology.orchestra.run.smithi139.stderr:Generating grub configuration file ... 2024-09-16T01:21:58.473 INFO:teuthology.orchestra.run.smithi139.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T01:21:58.504 INFO:teuthology.orchestra.run.smithi139.stderr:done 2024-09-16T01:21:58.506 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-09-16T01:21:58.542 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.UE5sbpRNb9 2024-09-16T01:21:58.542 DEBUG:teuthology.orchestra.run.smithi139:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UE5sbpRNb9 2024-09-16T01:21:58.609 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.UE5sbpRNb9 2024-09-16T01:21:58.747 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.UE5sbpRNb9 is 6KB 2024-09-16T01:21:58.798 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.UE5sbpRNb9 2024-09-16T01:21:58.813 DEBUG:teuthology.orchestra.run.smithi139:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T01:21:58.876 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T01:21:58.876 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T01:21:58.876 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T01:21:58.876 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T01:21:58.877 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T01:21:59.013 DEBUG:teuthology.orchestra.run.smithi139:> sudo shutdown -r now 2024-09-16T01:22:29.015 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-16T01:22:29.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T01:22:29.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:22:47.485 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.139 2024-09-16T01:22:56.486 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T01:22:56.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:22:59.553 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.139 2024-09-16T01:23:11.553 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T01:23:11.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:23:43.327 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-09-16T01:23:43.675 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T01:23:43.675 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-16T01:23:43.676 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-09-16T01:23:43.690 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-505.el9.x86_64 2024-09-16T01:23:43.690 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T01:23:43.690 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T01:23:43.690 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T01:23:44.692 DEBUG:teuthology.parallel:result is None 2024-09-16T01:23:44.692 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T01:23:44.698 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T01:23:44.698 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T01:23:44.701 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T01:23:44.704 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T01:23:44.722 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T01:23:44.731 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T01:23:44.738 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T01:23:44.739 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T01:23:44.760 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T01:23:44.762 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T01:23:44.931 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T01:23:44.938 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T01:23:44.938 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T01:23:44.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:23:44.955 DEBUG:teuthology.orchestra.run.smithi176:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T01:23:44.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:23:44.973 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T01:23:44.975 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T01:23:44.999 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T01:23:45.048 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:23:45.054 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:23:45.057 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:23:45.067 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:23:45.078 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:23:45.088 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T01:23:45.093 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T01:23:45.105 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T01:23:45.106 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T01:23:45.108 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T01:23:45.111 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T01:23:45.140 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T01:23:45.149 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T01:23:45.149 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T01:23:45.173 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T01:23:45.179 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T01:23:45.197 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:23:45.271 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:23:45.332 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:23:45.415 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:23:45.455 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:23:45.455 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T01:23:45.523 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:23:45.568 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:23:45.622 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:23:45.707 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:23:45.750 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T01:23:45.750 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T01:23:45.817 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:23:45.859 DEBUG:teuthology.orchestra.run.smithi176:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T01:23:45.919 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:23:45.999 DEBUG:teuthology.orchestra.run.smithi176:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T01:23:46.047 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-16T01:23:46.047 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T01:23:46.113 DEBUG:teuthology.orchestra.run.smithi139:> sudo service rsyslog restart 2024-09-16T01:23:46.116 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-09-16T01:23:46.119 DEBUG:teuthology.orchestra.run.smithi176:> sudo service rsyslog restart 2024-09-16T01:23:46.159 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T01:23:46.170 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T01:23:46.192 INFO:teuthology.orchestra.run.smithi176.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T01:23:46.589 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T01:23:46.597 INFO:teuthology.task.internal:Starting timer... 2024-09-16T01:23:46.600 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T01:23:46.616 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T01:23:46.626 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-16T01:23:46.626 DEBUG:teuthology.orchestra.run.smithi139:> sudo service auditd rotate 2024-09-16T01:23:46.757 INFO:teuthology.orchestra.run.smithi139.stdout:Rotating logs: 2024-09-16T01:23:46.761 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-09-16T01:23:46.822 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-09-16T01:23:46.824 DEBUG:teuthology.orchestra.run.smithi176:> sudo service auditd rotate 2024-09-16T01:23:46.886 INFO:teuthology.orchestra.run.smithi176.stdout:Rotating logs: 2024-09-16T01:23:46.888 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T01:23:46.888 DEBUG:teuthology.orchestra.run.smithi139:> /usr/sbin/getenforce 2024-09-16T01:23:46.937 INFO:teuthology.orchestra.run.smithi139.stdout:Permissive 2024-09-16T01:23:46.937 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-09-16T01:23:46.972 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-09-16T01:23:46.972 DEBUG:teuthology.orchestra.run.smithi176:> /usr/sbin/getenforce 2024-09-16T01:23:46.998 INFO:teuthology.orchestra.run.smithi176.stdout:Permissive 2024-09-16T01:23:46.999 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi139.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi176.front.sepia.ceph.com': 'permissive'} 2024-09-16T01:23:46.999 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T01:23:47.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:23:47.072 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T01:23:47.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:23:47.103 DEBUG:teuthology.orchestra.run.smithi176:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T01:23:47.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:23:47.130 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T01:23:47.130 DEBUG:teuthology.orchestra.run.smithi139:> sudo /usr/sbin/setenforce permissive 2024-09-16T01:23:47.440 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-09-16T01:23:47.480 DEBUG:teuthology.orchestra.run.smithi176:> sudo /usr/sbin/setenforce permissive 2024-09-16T01:23:47.512 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T01:23:47.521 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T01:23:47.532 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T01:23:47.533 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi139.front.sepia.ceph.com,smithi164.front.sepia.ceph.com,smithi176.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T01:27:49.373 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi139.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com'), Remote(name='ubuntu@smithi176.front.sepia.ceph.com')] 2024-09-16T01:27:49.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T01:27:49.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:27:49.461 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-09-16T01:27:49.540 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T01:27:49.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T01:27:49.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:27:49.620 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-16T01:27:49.710 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T01:27:49.710 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-16T01:27:49.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T01:27:49.794 DEBUG:teuthology.orchestra.run.smithi176:> true 2024-09-16T01:27:49.877 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-16T01:27:49.878 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T01:27:49.888 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T01:27:49.888 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T01:27:49.888 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T01:27:49.891 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T01:27:49.891 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T01:27:49.893 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T01:27:49.893 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T01:27:49.922 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T01:27:49.925 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T01:27:49.937 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T01:27:49.941 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T01:27:49.947 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T01:27:49.962 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T01:27:49.964 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: ntpd: command not found 2024-09-16T01:27:49.967 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-09-16T01:27:49.975 INFO:teuthology.orchestra.run.smithi139.stdout:506 Cannot talk to daemon 2024-09-16T01:27:49.979 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-09-16T01:27:49.987 INFO:teuthology.orchestra.run.smithi176.stderr:sudo: ntpd: command not found 2024-09-16T01:27:49.990 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T01:27:49.995 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T01:27:49.998 INFO:teuthology.orchestra.run.smithi176.stdout:506 Cannot talk to daemon 2024-09-16T01:27:50.005 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T01:27:50.009 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T01:27:50.013 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T01:27:50.027 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T01:27:50.059 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-09-16T01:27:50.063 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T01:27:50.064 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-09-16T01:27:50.064 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:27:50.064 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:27:50.064 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:27:50.064 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:27:50.076 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-09-16T01:27:50.079 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T01:27:50.080 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-09-16T01:27:50.080 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:27:50.080 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:27:50.080 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:27:50.080 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:27:50.084 INFO:teuthology.orchestra.run.smithi176.stderr:bash: line 1: ntpq: command not found 2024-09-16T01:27:50.088 INFO:teuthology.orchestra.run.smithi176.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T01:27:50.088 INFO:teuthology.orchestra.run.smithi176.stdout:=============================================================================== 2024-09-16T01:27:50.088 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:27:50.088 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:27:50.088 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:27:50.088 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:27:50.089 INFO:teuthology.run_tasks:Running task install... 2024-09-16T01:27:50.098 DEBUG:teuthology.task.install:project ceph 2024-09-16T01:27:50.098 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-16T01:27:50.098 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-16T01:27:50.099 INFO:teuthology.task.install:Using flavor: default 2024-09-16T01:27:50.108 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T01:27:50.108 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-16T01:27:50.108 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T01:27:50.109 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:27:50.111 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T01:27:50.111 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:27:50.112 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T01:27:50.112 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:27:50.255 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T01:27:50.255 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T01:27:50.260 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T01:27:50.260 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T01:27:50.266 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T01:27:50.266 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T01:27:50.363 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T01:27:50.364 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-16T01:27:50.364 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T01:27:50.371 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T01:27:50.371 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:27:50.372 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T01:27:50.390 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T01:27:50.390 DEBUG:teuthology.orchestra.run.smithi176:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T01:27:50.399 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T01:27:50.399 DEBUG:teuthology.orchestra.run.smithi139:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T01:27:50.440 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T01:27:50.441 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T01:27:50.441 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T01:27:50.457 DEBUG:teuthology.orchestra.run.smithi176:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T01:27:50.467 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T01:27:50.467 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T01:27:50.468 DEBUG:teuthology.orchestra.run.smithi139:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T01:27:50.532 DEBUG:teuthology.orchestra.run.smithi176:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T01:27:50.533 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T01:27:50.544 DEBUG:teuthology.orchestra.run.smithi139:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T01:27:50.596 INFO:teuthology.orchestra.run.smithi176.stdout:check_obsoletes = 1 2024-09-16T01:27:50.597 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum clean all 2024-09-16T01:27:50.608 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T01:27:50.609 INFO:teuthology.orchestra.run.smithi139.stdout:check_obsoletes = 1 2024-09-16T01:27:50.610 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum clean all 2024-09-16T01:27:50.673 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-09-16T01:27:50.674 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-09-16T01:27:50.823 INFO:teuthology.orchestra.run.smithi176.stdout:45 files removed 2024-09-16T01:27:50.845 INFO:teuthology.orchestra.run.smithi139.stdout:45 files removed 2024-09-16T01:27:50.850 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T01:27:50.872 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T01:27:50.899 INFO:teuthology.orchestra.run.smithi164.stdout:45 files removed 2024-09-16T01:27:50.926 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T01:27:51.413 INFO:teuthology.orchestra.run.smithi176.stdout:ceph packages for x86_64 291 kB/s | 83 kB 00:00 2024-09-16T01:27:51.435 INFO:teuthology.orchestra.run.smithi139.stdout:ceph packages for x86_64 301 kB/s | 83 kB 00:00 2024-09-16T01:27:51.471 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 311 kB/s | 83 kB 00:00 2024-09-16T01:27:51.688 INFO:teuthology.orchestra.run.smithi176.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-16T01:27:51.727 INFO:teuthology.orchestra.run.smithi139.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-16T01:27:51.756 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-16T01:27:51.955 INFO:teuthology.orchestra.run.smithi176.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-16T01:27:51.986 INFO:teuthology.orchestra.run.smithi139.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-16T01:27:52.031 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-16T01:27:52.555 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-16T01:27:52.557 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-16T01:27:52.719 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-16T01:27:55.229 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 14 MB/s | 20 MB 00:01 2024-09-16T01:27:57.125 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 20 MB 00:03 2024-09-16T01:27:58.565 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - AppStream 4.1 MB/s | 20 MB 00:04 2024-09-16T01:27:59.588 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-16T01:28:02.155 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 19 kB 00:01 2024-09-16T01:28:02.951 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-16T01:28:03.483 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-16T01:28:04.740 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.5 MB 00:02 2024-09-16T01:28:05.192 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 19 kB 00:00 2024-09-16T01:28:06.022 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-16T01:28:06.416 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-16T01:28:07.295 INFO:teuthology.orchestra.run.smithi176.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-16T01:28:08.523 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-16T01:28:10.892 INFO:teuthology.orchestra.run.smithi139.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:28:10.892 INFO:teuthology.orchestra.run.smithi139.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:28:10.904 INFO:teuthology.orchestra.run.smithi139.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T01:28:10.905 INFO:teuthology.orchestra.run.smithi139.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T01:28:10.906 INFO:teuthology.orchestra.run.smithi139.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T01:28:10.907 INFO:teuthology.orchestra.run.smithi139.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T01:28:10.908 INFO:teuthology.orchestra.run.smithi139.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T01:28:10.911 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T01:28:10.912 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T01:28:10.974 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T01:28:11.002 INFO:teuthology.orchestra.run.smithi139.stdout:====================================================================================== 2024-09-16T01:28:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-16T01:28:11.002 INFO:teuthology.orchestra.run.smithi139.stdout:====================================================================================== 2024-09-16T01:28:11.002 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T01:28:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T01:28:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T01:28:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T01:28:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T01:28:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T01:28:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T01:28:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T01:28:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T01:28:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T01:28:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T01:28:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T01:28:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T01:28:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T01:28:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T01:28:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T01:28:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T01:28:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T01:28:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T01:28:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T01:28:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T01:28:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T01:28:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T01:28:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T01:28:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T01:28:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T01:28:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T01:28:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T01:28:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T01:28:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T01:28:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T01:28:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T01:28:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T01:28:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T01:28:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T01:28:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T01:28:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T01:28:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T01:28:11.005 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrading: 2024-09-16T01:28:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T01:28:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T01:28:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T01:28:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T01:28:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T01:28:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T01:28:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T01:28:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T01:28:11.006 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-16T01:28:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T01:28:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T01:28:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T01:28:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T01:28:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T01:28:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T01:28:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T01:28:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T01:28:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T01:28:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T01:28:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T01:28:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T01:28:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T01:28:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T01:28:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T01:28:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T01:28:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T01:28:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T01:28:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T01:28:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T01:28:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T01:28:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T01:28:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T01:28:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T01:28:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T01:28:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T01:28:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T01:28:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T01:28:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T01:28:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T01:28:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T01:28:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T01:28:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T01:28:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T01:28:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T01:28:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T01:28:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T01:28:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T01:28:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T01:28:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T01:28:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T01:28:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T01:28:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T01:28:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T01:28:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T01:28:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T01:28:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T01:28:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T01:28:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T01:28:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T01:28:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T01:28:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T01:28:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T01:28:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T01:28:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T01:28:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T01:28:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T01:28:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T01:28:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T01:28:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T01:28:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T01:28:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T01:28:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T01:28:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T01:28:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T01:28:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T01:28:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T01:28:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T01:28:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T01:28:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T01:28:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T01:28:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T01:28:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T01:28:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T01:28:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T01:28:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T01:28:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T01:28:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T01:28:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T01:28:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T01:28:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T01:28:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T01:28:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T01:28:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T01:28:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T01:28:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T01:28:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T01:28:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T01:28:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T01:28:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T01:28:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T01:28:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T01:28:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T01:28:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T01:28:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T01:28:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T01:28:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T01:28:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T01:28:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T01:28:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T01:28:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T01:28:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T01:28:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T01:28:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T01:28:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T01:28:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T01:28:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T01:28:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T01:28:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T01:28:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T01:28:11.014 INFO:teuthology.orchestra.run.smithi139.stdout:Installing weak dependencies: 2024-09-16T01:28:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T01:28:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:28:11.014 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T01:28:11.014 INFO:teuthology.orchestra.run.smithi139.stdout:====================================================================================== 2024-09-16T01:28:11.015 INFO:teuthology.orchestra.run.smithi139.stdout:Install 148 Packages 2024-09-16T01:28:11.015 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrade 8 Packages 2024-09-16T01:28:11.015 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:28:11.015 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 226 M 2024-09-16T01:28:11.015 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T01:28:11.444 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-16T01:28:11.599 INFO:teuthology.orchestra.run.smithi139.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-16T01:28:11.933 INFO:teuthology.orchestra.run.smithi139.stdout:(2/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-16T01:28:11.966 INFO:teuthology.orchestra.run.smithi139.stdout:(3/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 2.4 MB/s | 896 kB 00:00 2024-09-16T01:28:11.992 INFO:teuthology.orchestra.run.smithi139.stdout:(4/156): ceph-immutable-object-cache-19.1.1-326 2.4 MB/s | 145 kB 00:00 2024-09-16T01:28:12.150 INFO:teuthology.orchestra.run.smithi139.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 9.6 MB/s | 1.5 MB 00:00 2024-09-16T01:28:12.442 INFO:teuthology.orchestra.run.smithi139.stdout:(6/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 16 MB/s | 4.7 MB 00:00 2024-09-16T01:28:12.492 INFO:teuthology.orchestra.run.smithi139.stdout:(7/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 4.5 MB/s | 2.4 MB 00:00 2024-09-16T01:28:12.676 INFO:teuthology.orchestra.run.smithi176.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-09-16T01:28:12.692 INFO:teuthology.orchestra.run.smithi139.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 17 MB/s | 21 MB 00:01 2024-09-16T01:28:12.734 INFO:teuthology.orchestra.run.smithi139.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 608 kB/s | 25 kB 00:00 2024-09-16T01:28:13.226 INFO:teuthology.orchestra.run.smithi139.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 21 MB/s | 17 MB 00:00 2024-09-16T01:28:13.276 INFO:teuthology.orchestra.run.smithi139.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 4.5 MB/s | 228 kB 00:00 2024-09-16T01:28:13.343 INFO:teuthology.orchestra.run.smithi139.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 406 kB/s | 27 kB 00:00 2024-09-16T01:28:13.419 INFO:teuthology.orchestra.run.smithi139.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 423 kB/s | 32 kB 00:00 2024-09-16T01:28:13.519 INFO:teuthology.orchestra.run.smithi139.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 7.3 MB/s | 751 kB 00:00 2024-09-16T01:28:13.636 INFO:teuthology.orchestra.run.smithi139.stdout:(15/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 9.3 MB/s | 11 MB 00:01 2024-09-16T01:28:13.661 INFO:teuthology.orchestra.run.smithi139.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 1.1 MB/s | 163 kB 00:00 2024-09-16T01:28:13.687 INFO:teuthology.orchestra.run.smithi139.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 2.5 MB/s | 127 kB 00:00 2024-09-16T01:28:13.729 INFO:teuthology.orchestra.run.smithi139.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 7.4 MB/s | 506 kB 00:00 2024-09-16T01:28:13.732 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:28:13.733 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:28:13.746 INFO:teuthology.orchestra.run.smithi164.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T01:28:13.747 INFO:teuthology.orchestra.run.smithi164.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T01:28:13.748 INFO:teuthology.orchestra.run.smithi164.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T01:28:13.749 INFO:teuthology.orchestra.run.smithi164.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T01:28:13.751 INFO:teuthology.orchestra.run.smithi164.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T01:28:13.754 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T01:28:13.755 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T01:28:13.770 INFO:teuthology.orchestra.run.smithi139.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 1.1 MB/s | 45 kB 00:00 2024-09-16T01:28:13.818 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-16T01:28:13.842 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-16T01:28:13.842 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-16T01:28:13.842 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-16T01:28:13.842 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-16T01:28:13.842 INFO:teuthology.orchestra.run.smithi164.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T01:28:13.842 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T01:28:13.842 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T01:28:13.842 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T01:28:13.842 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T01:28:13.842 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T01:28:13.842 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T01:28:13.843 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T01:28:13.843 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T01:28:13.843 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T01:28:13.843 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T01:28:13.843 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T01:28:13.843 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T01:28:13.843 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T01:28:13.843 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T01:28:13.843 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T01:28:13.843 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T01:28:13.843 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T01:28:13.843 INFO:teuthology.orchestra.run.smithi164.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T01:28:13.843 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T01:28:13.843 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T01:28:13.844 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T01:28:13.844 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T01:28:13.844 INFO:teuthology.orchestra.run.smithi164.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T01:28:13.844 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T01:28:13.844 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T01:28:13.844 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T01:28:13.844 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T01:28:13.844 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T01:28:13.844 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T01:28:13.844 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T01:28:13.844 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T01:28:13.844 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T01:28:13.844 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T01:28:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T01:28:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T01:28:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T01:28:13.845 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-09-16T01:28:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T01:28:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T01:28:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T01:28:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T01:28:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T01:28:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T01:28:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T01:28:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T01:28:13.846 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-16T01:28:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T01:28:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T01:28:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T01:28:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T01:28:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T01:28:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T01:28:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T01:28:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T01:28:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T01:28:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T01:28:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T01:28:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T01:28:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T01:28:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T01:28:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T01:28:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T01:28:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T01:28:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T01:28:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T01:28:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T01:28:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T01:28:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T01:28:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T01:28:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T01:28:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T01:28:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T01:28:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T01:28:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T01:28:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T01:28:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T01:28:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T01:28:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T01:28:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T01:28:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T01:28:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T01:28:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T01:28:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T01:28:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T01:28:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T01:28:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T01:28:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T01:28:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T01:28:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T01:28:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T01:28:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T01:28:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T01:28:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T01:28:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T01:28:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T01:28:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T01:28:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T01:28:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T01:28:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T01:28:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T01:28:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T01:28:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T01:28:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T01:28:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T01:28:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T01:28:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T01:28:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T01:28:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T01:28:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T01:28:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T01:28:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T01:28:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T01:28:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T01:28:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T01:28:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T01:28:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T01:28:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T01:28:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T01:28:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T01:28:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T01:28:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T01:28:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T01:28:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T01:28:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T01:28:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T01:28:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T01:28:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T01:28:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T01:28:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T01:28:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T01:28:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T01:28:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T01:28:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T01:28:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T01:28:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T01:28:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T01:28:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T01:28:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T01:28:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T01:28:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T01:28:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T01:28:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T01:28:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T01:28:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T01:28:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T01:28:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T01:28:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T01:28:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T01:28:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T01:28:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T01:28:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T01:28:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T01:28:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T01:28:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T01:28:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T01:28:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T01:28:13.854 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-09-16T01:28:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T01:28:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:28:13.854 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-16T01:28:13.855 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-16T01:28:13.855 INFO:teuthology.orchestra.run.smithi164.stdout:Install 148 Packages 2024-09-16T01:28:13.855 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 8 Packages 2024-09-16T01:28:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:28:13.855 INFO:teuthology.orchestra.run.smithi139.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.6 MB/s | 136 kB 00:00 2024-09-16T01:28:13.856 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 226 M 2024-09-16T01:28:13.856 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-16T01:28:13.938 INFO:teuthology.orchestra.run.smithi139.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.9 MB/s | 161 kB 00:00 2024-09-16T01:28:14.046 INFO:teuthology.orchestra.run.smithi139.stdout:(22/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-16T01:28:14.072 INFO:teuthology.orchestra.run.smithi139.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 2.3 MB/s | 320 kB 00:00 2024-09-16T01:28:14.105 INFO:teuthology.orchestra.run.smithi139.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.0 MB/s | 300 kB 00:00 2024-09-16T01:28:14.131 INFO:teuthology.orchestra.run.smithi139.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.7 MB/s | 99 kB 00:00 2024-09-16T01:28:14.156 INFO:teuthology.orchestra.run.smithi139.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-16T01:28:14.223 INFO:teuthology.orchestra.run.smithi139.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.5 MB/s | 170 kB 00:00 2024-09-16T01:28:14.281 INFO:teuthology.orchestra.run.smithi139.stdout:(28/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 21 MB/s | 3.1 MB 00:00 2024-09-16T01:28:14.307 INFO:teuthology.orchestra.run.smithi139.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 382 kB/s | 32 kB 00:00 2024-09-16T01:28:14.332 INFO:teuthology.orchestra.run.smithi139.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.9 MB/s | 146 kB 00:00 2024-09-16T01:28:14.546 INFO:teuthology.orchestra.run.smithi164.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 27 kB/s | 6.5 kB 00:00 2024-09-16T01:28:14.658 INFO:teuthology.orchestra.run.smithi139.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 10 MB/s | 3.6 MB 00:00 2024-09-16T01:28:14.766 INFO:teuthology.orchestra.run.smithi139.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-16T01:28:14.791 INFO:teuthology.orchestra.run.smithi139.stdout:(33/156): ceph-mgr-modules-core-19.1.1-326.g892 1.8 MB/s | 249 kB 00:00 2024-09-16T01:28:14.888 INFO:teuthology.orchestra.run.smithi164.stdout:(2/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 9.2 MB/s | 5.4 MB 00:00 2024-09-16T01:28:14.921 INFO:teuthology.orchestra.run.smithi164.stdout:(3/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 2.3 MB/s | 896 kB 00:00 2024-09-16T01:28:14.946 INFO:teuthology.orchestra.run.smithi164.stdout:(4/156): ceph-immutable-object-cache-19.1.1-326 2.4 MB/s | 145 kB 00:00 2024-09-16T01:28:14.961 INFO:teuthology.orchestra.run.smithi176.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:28:14.962 INFO:teuthology.orchestra.run.smithi176.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T01:28:14.974 INFO:teuthology.orchestra.run.smithi176.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T01:28:14.974 INFO:teuthology.orchestra.run.smithi176.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T01:28:14.975 INFO:teuthology.orchestra.run.smithi176.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T01:28:14.976 INFO:teuthology.orchestra.run.smithi176.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T01:28:14.978 INFO:teuthology.orchestra.run.smithi176.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T01:28:14.980 INFO:teuthology.orchestra.run.smithi176.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T01:28:14.981 INFO:teuthology.orchestra.run.smithi176.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T01:28:15.044 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-09-16T01:28:15.055 INFO:teuthology.orchestra.run.smithi164.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 14 MB/s | 1.5 MB 00:00 2024-09-16T01:28:15.062 INFO:teuthology.orchestra.run.smithi176.stdout:====================================================================================== 2024-09-16T01:28:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-09-16T01:28:15.062 INFO:teuthology.orchestra.run.smithi176.stdout:====================================================================================== 2024-09-16T01:28:15.062 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-09-16T01:28:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T01:28:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T01:28:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T01:28:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T01:28:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T01:28:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T01:28:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T01:28:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T01:28:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T01:28:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T01:28:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T01:28:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T01:28:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T01:28:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T01:28:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T01:28:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T01:28:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T01:28:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T01:28:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T01:28:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T01:28:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T01:28:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T01:28:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T01:28:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T01:28:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T01:28:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T01:28:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T01:28:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T01:28:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T01:28:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T01:28:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T01:28:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T01:28:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T01:28:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T01:28:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T01:28:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T01:28:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T01:28:15.065 INFO:teuthology.orchestra.run.smithi176.stdout:Upgrading: 2024-09-16T01:28:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T01:28:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T01:28:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T01:28:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T01:28:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T01:28:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T01:28:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T01:28:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T01:28:15.066 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-09-16T01:28:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T01:28:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T01:28:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T01:28:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T01:28:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T01:28:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T01:28:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T01:28:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T01:28:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T01:28:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T01:28:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T01:28:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T01:28:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T01:28:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T01:28:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T01:28:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T01:28:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T01:28:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T01:28:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T01:28:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T01:28:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T01:28:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T01:28:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T01:28:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T01:28:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T01:28:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T01:28:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T01:28:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T01:28:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T01:28:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T01:28:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T01:28:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T01:28:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T01:28:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T01:28:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T01:28:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T01:28:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T01:28:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T01:28:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T01:28:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T01:28:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T01:28:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T01:28:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T01:28:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T01:28:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T01:28:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T01:28:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T01:28:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T01:28:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T01:28:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T01:28:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T01:28:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T01:28:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T01:28:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T01:28:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T01:28:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T01:28:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T01:28:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T01:28:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T01:28:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T01:28:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T01:28:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T01:28:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T01:28:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T01:28:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T01:28:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T01:28:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T01:28:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T01:28:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T01:28:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T01:28:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T01:28:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T01:28:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T01:28:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T01:28:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T01:28:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T01:28:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T01:28:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T01:28:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T01:28:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T01:28:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T01:28:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T01:28:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T01:28:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T01:28:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T01:28:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T01:28:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T01:28:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T01:28:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T01:28:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T01:28:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T01:28:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T01:28:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T01:28:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T01:28:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T01:28:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T01:28:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T01:28:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T01:28:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T01:28:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T01:28:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T01:28:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T01:28:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T01:28:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T01:28:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T01:28:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T01:28:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T01:28:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T01:28:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T01:28:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T01:28:15.074 INFO:teuthology.orchestra.run.smithi176.stdout:Installing weak dependencies: 2024-09-16T01:28:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T01:28:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:28:15.074 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-09-16T01:28:15.074 INFO:teuthology.orchestra.run.smithi176.stdout:====================================================================================== 2024-09-16T01:28:15.074 INFO:teuthology.orchestra.run.smithi176.stdout:Install 148 Packages 2024-09-16T01:28:15.074 INFO:teuthology.orchestra.run.smithi176.stdout:Upgrade 8 Packages 2024-09-16T01:28:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:28:15.075 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 226 M 2024-09-16T01:28:15.075 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-09-16T01:28:15.217 INFO:teuthology.orchestra.run.smithi139.stdout:(34/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 20 MB/s | 49 MB 00:02 2024-09-16T01:28:15.242 INFO:teuthology.orchestra.run.smithi139.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 36 kB/s | 16 kB 00:00 2024-09-16T01:28:15.267 INFO:teuthology.orchestra.run.smithi139.stdout:(36/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 98 kB/s | 49 kB 00:00 2024-09-16T01:28:15.301 INFO:teuthology.orchestra.run.smithi139.stdout:(37/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 3.3 MB/s | 285 kB 00:00 2024-09-16T01:28:15.326 INFO:teuthology.orchestra.run.smithi139.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 312 kB/s | 18 kB 00:00 2024-09-16T01:28:15.359 INFO:teuthology.orchestra.run.smithi139.stdout:(39/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 6.4 MB/s | 765 kB 00:00 2024-09-16T01:28:15.364 INFO:teuthology.orchestra.run.smithi164.stdout:(6/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 15 MB/s | 4.7 MB 00:00 2024-09-16T01:28:15.418 INFO:teuthology.orchestra.run.smithi139.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 682 kB/s | 80 kB 00:00 2024-09-16T01:28:15.472 INFO:teuthology.orchestra.run.smithi164.stdout:(7/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 4.3 MB/s | 2.4 MB 00:00 2024-09-16T01:28:15.485 INFO:teuthology.orchestra.run.smithi139.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-16T01:28:15.519 INFO:teuthology.orchestra.run.smithi139.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-16T01:28:15.552 INFO:teuthology.orchestra.run.smithi139.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-16T01:28:15.577 INFO:teuthology.orchestra.run.smithi176.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 26 kB/s | 6.5 kB 00:00 2024-09-16T01:28:15.577 INFO:teuthology.orchestra.run.smithi139.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-16T01:28:15.603 INFO:teuthology.orchestra.run.smithi139.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-16T01:28:15.628 INFO:teuthology.orchestra.run.smithi139.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-16T01:28:15.653 INFO:teuthology.orchestra.run.smithi139.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-16T01:28:15.687 INFO:teuthology.orchestra.run.smithi139.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-09-16T01:28:15.802 INFO:teuthology.orchestra.run.smithi176.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 3.9 MB/s | 896 kB 00:00 2024-09-16T01:28:15.829 INFO:teuthology.orchestra.run.smithi139.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 755 kB/s | 107 kB 00:00 2024-09-16T01:28:15.834 INFO:teuthology.orchestra.run.smithi164.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 14 MB/s | 21 MB 00:01 2024-09-16T01:28:15.852 INFO:teuthology.orchestra.run.smithi176.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 2.8 MB/s | 145 kB 00:00 2024-09-16T01:28:15.854 INFO:teuthology.orchestra.run.smithi139.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-16T01:28:15.879 INFO:teuthology.orchestra.run.smithi139.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-16T01:28:15.881 INFO:teuthology.orchestra.run.smithi164.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 545 kB/s | 25 kB 00:00 2024-09-16T01:28:15.905 INFO:teuthology.orchestra.run.smithi139.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-16T01:28:15.930 INFO:teuthology.orchestra.run.smithi139.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 649 kB/s | 33 kB 00:00 2024-09-16T01:28:15.964 INFO:teuthology.orchestra.run.smithi139.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-09-16T01:28:15.989 INFO:teuthology.orchestra.run.smithi139.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-09-16T01:28:16.014 INFO:teuthology.orchestra.run.smithi139.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-16T01:28:16.040 INFO:teuthology.orchestra.run.smithi139.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-16T01:28:16.065 INFO:teuthology.orchestra.run.smithi139.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-09-16T01:28:16.099 INFO:teuthology.orchestra.run.smithi139.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-16T01:28:16.165 INFO:teuthology.orchestra.run.smithi139.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-09-16T01:28:16.191 INFO:teuthology.orchestra.run.smithi139.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 267 kB/s | 24 kB 00:00 2024-09-16T01:28:16.216 INFO:teuthology.orchestra.run.smithi139.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-16T01:28:16.240 INFO:teuthology.orchestra.run.smithi164.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 19 MB/s | 17 MB 00:00 2024-09-16T01:28:16.241 INFO:teuthology.orchestra.run.smithi139.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-16T01:28:16.261 INFO:teuthology.orchestra.run.smithi176.stdout:(4/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 5.8 MB/s | 2.4 MB 00:00 2024-09-16T01:28:16.266 INFO:teuthology.orchestra.run.smithi139.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 2.8 MB/s | 581 kB 00:00 2024-09-16T01:28:16.290 INFO:teuthology.orchestra.run.smithi164.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 4.5 MB/s | 228 kB 00:00 2024-09-16T01:28:16.291 INFO:teuthology.orchestra.run.smithi139.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-16T01:28:16.317 INFO:teuthology.orchestra.run.smithi139.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-16T01:28:16.365 INFO:teuthology.orchestra.run.smithi164.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 361 kB/s | 27 kB 00:00 2024-09-16T01:28:16.424 INFO:teuthology.orchestra.run.smithi164.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 546 kB/s | 32 kB 00:00 2024-09-16T01:28:16.425 INFO:teuthology.orchestra.run.smithi139.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-09-16T01:28:16.428 INFO:teuthology.orchestra.run.smithi176.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 9.1 MB/s | 1.5 MB 00:00 2024-09-16T01:28:16.468 INFO:teuthology.orchestra.run.smithi139.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-16T01:28:16.499 INFO:teuthology.orchestra.run.smithi164.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 9.8 MB/s | 751 kB 00:00 2024-09-16T01:28:16.534 INFO:teuthology.orchestra.run.smithi139.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 18 MB/s | 4.9 MB 00:00 2024-09-16T01:28:16.568 INFO:teuthology.orchestra.run.smithi139.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-16T01:28:16.610 INFO:teuthology.orchestra.run.smithi139.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.8 MB/s | 910 kB 00:00 2024-09-16T01:28:16.649 INFO:teuthology.orchestra.run.smithi164.stdout:(15/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 9.0 MB/s | 11 MB 00:01 2024-09-16T01:28:16.652 INFO:teuthology.orchestra.run.smithi139.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-16T01:28:16.675 INFO:teuthology.orchestra.run.smithi164.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 933 kB/s | 163 kB 00:00 2024-09-16T01:28:16.677 INFO:teuthology.orchestra.run.smithi139.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-16T01:28:16.700 INFO:teuthology.orchestra.run.smithi164.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 2.5 MB/s | 127 kB 00:00 2024-09-16T01:28:16.729 INFO:teuthology.orchestra.run.smithi176.stdout:(6/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 16 MB/s | 4.7 MB 00:00 2024-09-16T01:28:16.733 INFO:teuthology.orchestra.run.smithi164.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 8.5 MB/s | 506 kB 00:00 2024-09-16T01:28:16.761 INFO:teuthology.orchestra.run.smithi139.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-09-16T01:28:16.786 INFO:teuthology.orchestra.run.smithi139.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 565 kB/s | 61 kB 00:00 2024-09-16T01:28:16.809 INFO:teuthology.orchestra.run.smithi164.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 604 kB/s | 45 kB 00:00 2024-09-16T01:28:16.811 INFO:teuthology.orchestra.run.smithi139.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-16T01:28:16.836 INFO:teuthology.orchestra.run.smithi139.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-16T01:28:16.862 INFO:teuthology.orchestra.run.smithi139.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-16T01:28:16.884 INFO:teuthology.orchestra.run.smithi164.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.8 MB/s | 136 kB 00:00 2024-09-16T01:28:16.887 INFO:teuthology.orchestra.run.smithi139.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-16T01:28:16.921 INFO:teuthology.orchestra.run.smithi139.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-16T01:28:16.954 INFO:teuthology.orchestra.run.smithi139.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 519 kB/s | 35 kB 00:00 2024-09-16T01:28:16.976 INFO:teuthology.orchestra.run.smithi164.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.7 MB/s | 161 kB 00:00 2024-09-16T01:28:17.005 INFO:teuthology.orchestra.run.smithi139.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-16T01:28:17.060 INFO:teuthology.orchestra.run.smithi164.stdout:(22/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-16T01:28:17.072 INFO:teuthology.orchestra.run.smithi139.stdout:(83/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-16T01:28:17.085 INFO:teuthology.orchestra.run.smithi164.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 2.9 MB/s | 320 kB 00:00 2024-09-16T01:28:17.106 INFO:teuthology.orchestra.run.smithi139.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-16T01:28:17.127 INFO:teuthology.orchestra.run.smithi164.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 4.4 MB/s | 300 kB 00:00 2024-09-16T01:28:17.131 INFO:teuthology.orchestra.run.smithi139.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-16T01:28:17.147 INFO:teuthology.orchestra.run.smithi139.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-16T01:28:17.152 INFO:teuthology.orchestra.run.smithi164.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.5 MB/s | 99 kB 00:00 2024-09-16T01:28:17.194 INFO:teuthology.orchestra.run.smithi164.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-16T01:28:17.214 INFO:teuthology.orchestra.run.smithi139.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-16T01:28:17.248 INFO:teuthology.orchestra.run.smithi139.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-09-16T01:28:17.273 INFO:teuthology.orchestra.run.smithi139.stdout:(89/156): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-09-16T01:28:17.278 INFO:teuthology.orchestra.run.smithi164.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.0 MB/s | 170 kB 00:00 2024-09-16T01:28:17.298 INFO:teuthology.orchestra.run.smithi139.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-16T01:28:17.324 INFO:teuthology.orchestra.run.smithi139.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-16T01:28:17.349 INFO:teuthology.orchestra.run.smithi139.stdout:(92/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-09-16T01:28:17.374 INFO:teuthology.orchestra.run.smithi139.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 837 kB/s | 42 kB 00:00 2024-09-16T01:28:17.378 INFO:teuthology.orchestra.run.smithi164.stdout:(28/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 14 MB/s | 3.1 MB 00:00 2024-09-16T01:28:17.399 INFO:teuthology.orchestra.run.smithi139.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-09-16T01:28:17.406 INFO:teuthology.orchestra.run.smithi164.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 249 kB/s | 32 kB 00:00 2024-09-16T01:28:17.433 INFO:teuthology.orchestra.run.smithi139.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-16T01:28:17.448 INFO:teuthology.orchestra.run.smithi164.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.1 MB/s | 146 kB 00:00 2024-09-16T01:28:17.458 INFO:teuthology.orchestra.run.smithi139.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-09-16T01:28:17.484 INFO:teuthology.orchestra.run.smithi139.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-16T01:28:17.517 INFO:teuthology.orchestra.run.smithi139.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-16T01:28:17.567 INFO:teuthology.orchestra.run.smithi139.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 693 kB/s | 58 kB 00:00 2024-09-16T01:28:17.593 INFO:teuthology.orchestra.run.smithi139.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-16T01:28:17.618 INFO:teuthology.orchestra.run.smithi139.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T01:28:17.643 INFO:teuthology.orchestra.run.smithi139.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-16T01:28:17.711 INFO:teuthology.orchestra.run.smithi139.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-09-16T01:28:17.715 INFO:teuthology.orchestra.run.smithi164.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 12 MB/s | 3.6 MB 00:00 2024-09-16T01:28:17.769 INFO:teuthology.orchestra.run.smithi139.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.3 MB/s | 548 kB 00:00 2024-09-16T01:28:17.790 INFO:teuthology.orchestra.run.smithi164.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 3.3 MB/s | 249 kB 00:00 2024-09-16T01:28:17.811 INFO:teuthology.orchestra.run.smithi139.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-16T01:28:17.812 INFO:teuthology.orchestra.run.smithi176.stdout:(7/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 2.2 MB/s | 5.4 MB 00:02 2024-09-16T01:28:17.836 INFO:teuthology.orchestra.run.smithi139.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-16T01:28:17.857 INFO:teuthology.orchestra.run.smithi164.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 741 kB/s | 49 kB 00:00 2024-09-16T01:28:17.870 INFO:teuthology.orchestra.run.smithi139.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-09-16T01:28:17.920 INFO:teuthology.orchestra.run.smithi139.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 522 kB/s | 26 kB 00:00 2024-09-16T01:28:17.957 INFO:teuthology.orchestra.run.smithi164.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-16T01:28:17.971 INFO:teuthology.orchestra.run.smithi176.stdout:(8/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86_ 14 MB/s | 17 MB 00:01 2024-09-16T01:28:17.982 INFO:teuthology.orchestra.run.smithi164.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 131 kB/s | 16 kB 00:00 2024-09-16T01:28:18.021 INFO:teuthology.orchestra.run.smithi176.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 507 kB/s | 25 kB 00:00 2024-09-16T01:28:18.033 INFO:teuthology.orchestra.run.smithi164.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 3.7 MB/s | 285 kB 00:00 2024-09-16T01:28:18.067 INFO:teuthology.orchestra.run.smithi164.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 8.9 MB/s | 765 kB 00:00 2024-09-16T01:28:18.079 INFO:teuthology.orchestra.run.smithi139.stdout:(109/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-09-16T01:28:18.100 INFO:teuthology.orchestra.run.smithi164.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 272 kB/s | 18 kB 00:00 2024-09-16T01:28:18.113 INFO:teuthology.orchestra.run.smithi139.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 253 kB/s | 49 kB 00:00 2024-09-16T01:28:18.146 INFO:teuthology.orchestra.run.smithi139.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-16T01:28:18.171 INFO:teuthology.orchestra.run.smithi139.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-09-16T01:28:18.217 INFO:teuthology.orchestra.run.smithi164.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 532 kB/s | 80 kB 00:00 2024-09-16T01:28:18.230 INFO:teuthology.orchestra.run.smithi139.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-16T01:28:18.255 INFO:teuthology.orchestra.run.smithi139.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T01:28:18.280 INFO:teuthology.orchestra.run.smithi139.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-09-16T01:28:18.306 INFO:teuthology.orchestra.run.smithi139.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-09-16T01:28:18.326 INFO:teuthology.orchestra.run.smithi164.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-09-16T01:28:18.331 INFO:teuthology.orchestra.run.smithi139.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-16T01:28:18.357 INFO:teuthology.orchestra.run.smithi139.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-16T01:28:18.398 INFO:teuthology.orchestra.run.smithi139.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-16T01:28:18.457 INFO:teuthology.orchestra.run.smithi139.stdout:(120/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-09-16T01:28:18.463 INFO:teuthology.orchestra.run.smithi176.stdout:(10/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 16 MB/s | 11 MB 00:00 2024-09-16T01:28:18.482 INFO:teuthology.orchestra.run.smithi139.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-09-16T01:28:18.508 INFO:teuthology.orchestra.run.smithi139.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.0 MB/s | 220 kB 00:00 2024-09-16T01:28:18.513 INFO:teuthology.orchestra.run.smithi176.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 4.4 MB/s | 228 kB 00:00 2024-09-16T01:28:18.533 INFO:teuthology.orchestra.run.smithi139.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-16T01:28:18.555 INFO:teuthology.orchestra.run.smithi176.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 651 kB/s | 27 kB 00:00 2024-09-16T01:28:18.558 INFO:teuthology.orchestra.run.smithi139.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-16T01:28:18.583 INFO:teuthology.orchestra.run.smithi139.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-16T01:28:18.597 INFO:teuthology.orchestra.run.smithi176.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 761 kB/s | 32 kB 00:00 2024-09-16T01:28:18.608 INFO:teuthology.orchestra.run.smithi139.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-16T01:28:18.634 INFO:teuthology.orchestra.run.smithi139.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-16T01:28:18.656 INFO:teuthology.orchestra.run.smithi176.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 13 MB/s | 751 kB 00:00 2024-09-16T01:28:18.659 INFO:teuthology.orchestra.run.smithi139.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-16T01:28:18.684 INFO:teuthology.orchestra.run.smithi139.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T01:28:18.697 INFO:teuthology.orchestra.run.smithi176.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 3.8 MB/s | 163 kB 00:00 2024-09-16T01:28:18.710 INFO:teuthology.orchestra.run.smithi139.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-16T01:28:18.735 INFO:teuthology.orchestra.run.smithi139.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-16T01:28:18.748 INFO:teuthology.orchestra.run.smithi176.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 2.5 MB/s | 127 kB 00:00 2024-09-16T01:28:18.759 INFO:teuthology.orchestra.run.smithi164.stdout:(41/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 17 MB/s | 49 MB 00:02 2024-09-16T01:28:18.769 INFO:teuthology.orchestra.run.smithi139.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-16T01:28:18.785 INFO:teuthology.orchestra.run.smithi164.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 552 kB/s | 253 kB 00:00 2024-09-16T01:28:18.798 INFO:teuthology.orchestra.run.smithi176.stdout:(17/156): libradosstriper1-19.1.1-326.g8921c8ef 9.9 MB/s | 506 kB 00:00 2024-09-16T01:28:18.802 INFO:teuthology.orchestra.run.smithi139.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-16T01:28:18.827 INFO:teuthology.orchestra.run.smithi139.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-09-16T01:28:18.835 INFO:teuthology.orchestra.run.smithi164.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-16T01:28:18.853 INFO:teuthology.orchestra.run.smithi139.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-16T01:28:18.860 INFO:teuthology.orchestra.run.smithi164.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-16T01:28:18.878 INFO:teuthology.orchestra.run.smithi139.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-16T01:28:18.911 INFO:teuthology.orchestra.run.smithi164.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-16T01:28:18.911 INFO:teuthology.orchestra.run.smithi139.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 530 kB/s | 31 kB 00:00 2024-09-16T01:28:18.936 INFO:teuthology.orchestra.run.smithi164.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-16T01:28:18.936 INFO:teuthology.orchestra.run.smithi139.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T01:28:18.961 INFO:teuthology.orchestra.run.smithi139.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-16T01:28:18.987 INFO:teuthology.orchestra.run.smithi139.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-16T01:28:18.995 INFO:teuthology.orchestra.run.smithi164.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-16T01:28:19.007 INFO:teuthology.orchestra.run.smithi176.stdout:(18/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 26 MB/s | 5.3 MB 00:00 2024-09-16T01:28:19.012 INFO:teuthology.orchestra.run.smithi139.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-16T01:28:19.037 INFO:teuthology.orchestra.run.smithi139.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-16T01:28:19.048 INFO:teuthology.orchestra.run.smithi176.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 1.1 MB/s | 45 kB 00:00 2024-09-16T01:28:19.080 INFO:teuthology.orchestra.run.smithi139.stdout:(143/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-09-16T01:28:19.099 INFO:teuthology.orchestra.run.smithi176.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 2.7 MB/s | 136 kB 00:00 2024-09-16T01:28:19.104 INFO:teuthology.orchestra.run.smithi139.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-16T01:28:19.112 INFO:teuthology.orchestra.run.smithi164.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-09-16T01:28:19.130 INFO:teuthology.orchestra.run.smithi139.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-16T01:28:19.149 INFO:teuthology.orchestra.run.smithi176.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 3.1 MB/s | 161 kB 00:00 2024-09-16T01:28:19.155 INFO:teuthology.orchestra.run.smithi139.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T01:28:19.170 INFO:teuthology.orchestra.run.smithi164.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-16T01:28:19.189 INFO:teuthology.orchestra.run.smithi139.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-16T01:28:19.199 INFO:teuthology.orchestra.run.smithi176.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 6.3 MB/s | 320 kB 00:00 2024-09-16T01:28:19.204 INFO:teuthology.orchestra.run.smithi164.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-09-16T01:28:19.229 INFO:teuthology.orchestra.run.smithi164.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-16T01:28:19.239 INFO:teuthology.orchestra.run.smithi139.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-16T01:28:19.250 INFO:teuthology.orchestra.run.smithi176.stdout:(23/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.9 MB/s | 300 kB 00:00 2024-09-16T01:28:19.262 INFO:teuthology.orchestra.run.smithi164.stdout:(52/156): python3-cryptography-36.0.1-4.el9.x86 2.4 MB/s | 1.2 MB 00:00 2024-09-16T01:28:19.288 INFO:teuthology.orchestra.run.smithi164.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-16T01:28:19.300 INFO:teuthology.orchestra.run.smithi176.stdout:(24/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.9 MB/s | 99 kB 00:00 2024-09-16T01:28:19.313 INFO:teuthology.orchestra.run.smithi164.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-16T01:28:19.338 INFO:teuthology.orchestra.run.smithi164.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-16T01:28:19.350 INFO:teuthology.orchestra.run.smithi176.stdout:(25/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-16T01:28:19.364 INFO:teuthology.orchestra.run.smithi164.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-16T01:28:19.406 INFO:teuthology.orchestra.run.smithi164.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-16T01:28:19.406 INFO:teuthology.orchestra.run.smithi139.stdout:(149/156): librados2-19.1.1-326.g8921c8ef.el9.x 14 MB/s | 3.4 MB 00:00 2024-09-16T01:28:19.431 INFO:teuthology.orchestra.run.smithi164.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-16T01:28:19.465 INFO:teuthology.orchestra.run.smithi139.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-09-16T01:28:19.492 INFO:teuthology.orchestra.run.smithi176.stdout:(26/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 22 MB/s | 3.1 MB 00:00 2024-09-16T01:28:19.507 INFO:teuthology.orchestra.run.smithi139.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 984 kB/s | 41 kB 00:00 2024-09-16T01:28:19.542 INFO:teuthology.orchestra.run.smithi176.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 3.3 MB/s | 170 kB 00:00 2024-09-16T01:28:19.549 INFO:teuthology.orchestra.run.smithi139.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 5.8 MB/s | 249 kB 00:00 2024-09-16T01:28:19.584 INFO:teuthology.orchestra.run.smithi176.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 765 kB/s | 32 kB 00:00 2024-09-16T01:28:19.599 INFO:teuthology.orchestra.run.smithi139.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 9.4 MB/s | 1.8 MB 00:00 2024-09-16T01:28:19.606 INFO:teuthology.orchestra.run.smithi164.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-16T01:28:19.635 INFO:teuthology.orchestra.run.smithi176.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.9 MB/s | 146 kB 00:00 2024-09-16T01:28:19.640 INFO:teuthology.orchestra.run.smithi164.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 2.7 MB/s | 581 kB 00:00 2024-09-16T01:28:19.657 INFO:teuthology.orchestra.run.smithi139.stdout:(154/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 6.7 MB/s | 3.2 MB 00:00 2024-09-16T01:28:19.668 INFO:teuthology.orchestra.run.smithi164.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-16T01:28:19.690 INFO:teuthology.orchestra.run.smithi164.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-16T01:28:19.699 INFO:teuthology.orchestra.run.smithi139.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 1.4 MB 00:00 2024-09-16T01:28:19.715 INFO:teuthology.orchestra.run.smithi164.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-16T01:28:19.749 INFO:teuthology.orchestra.run.smithi139.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-09-16T01:28:19.756 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:28:19.756 INFO:teuthology.orchestra.run.smithi139.stdout:Total 26 MB/s | 226 MB 00:08 2024-09-16T01:28:19.758 INFO:teuthology.orchestra.run.smithi164.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 924 kB/s | 38 kB 00:00 2024-09-16T01:28:19.783 INFO:teuthology.orchestra.run.smithi164.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-09-16T01:28:19.793 INFO:teuthology.orchestra.run.smithi176.stdout:(30/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 23 MB/s | 3.6 MB 00:00 2024-09-16T01:28:20.075 INFO:teuthology.orchestra.run.smithi164.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.7 MB/s | 3.0 MB 00:00 2024-09-16T01:28:20.094 INFO:teuthology.orchestra.run.smithi176.stdout:(31/156): ceph-common-19.1.1-326.g8921c8ef.el9. 4.4 MB/s | 21 MB 00:04 2024-09-16T01:28:20.210 INFO:teuthology.orchestra.run.smithi176.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 18 MB/s | 7.4 MB 00:00 2024-09-16T01:28:20.245 INFO:teuthology.orchestra.run.smithi176.stdout:(33/156): ceph-mgr-modules-core-19.1.1-326.g892 1.6 MB/s | 249 kB 00:00 2024-09-16T01:28:20.270 INFO:teuthology.orchestra.run.smithi176.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 834 kB/s | 49 kB 00:00 2024-09-16T01:28:20.295 INFO:teuthology.orchestra.run.smithi176.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 327 kB/s | 16 kB 00:00 2024-09-16T01:28:20.321 INFO:teuthology.orchestra.run.smithi176.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 5.6 MB/s | 285 kB 00:00 2024-09-16T01:28:20.362 INFO:teuthology.orchestra.run.smithi176.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 441 kB/s | 18 kB 00:00 2024-09-16T01:28:20.421 INFO:teuthology.orchestra.run.smithi176.stdout:(38/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 6.0 MB/s | 765 kB 00:00 2024-09-16T01:28:20.509 INFO:teuthology.orchestra.run.smithi164.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-09-16T01:28:20.530 INFO:teuthology.orchestra.run.smithi176.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-16T01:28:20.655 INFO:teuthology.orchestra.run.smithi176.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-16T01:28:20.667 INFO:teuthology.orchestra.run.smithi164.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 5.4 MB/s | 4.9 MB 00:00 2024-09-16T01:28:20.709 INFO:teuthology.orchestra.run.smithi164.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 4.4 MB/s | 910 kB 00:00 2024-09-16T01:28:20.734 INFO:teuthology.orchestra.run.smithi164.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-16T01:28:20.739 INFO:teuthology.orchestra.run.smithi176.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-16T01:28:20.764 INFO:teuthology.orchestra.run.smithi176.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-16T01:28:20.793 INFO:teuthology.orchestra.run.smithi164.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-16T01:28:20.814 INFO:teuthology.orchestra.run.smithi176.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-16T01:28:20.835 INFO:teuthology.orchestra.run.smithi164.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-09-16T01:28:20.865 INFO:teuthology.orchestra.run.smithi176.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-16T01:28:20.940 INFO:teuthology.orchestra.run.smithi176.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-16T01:28:20.974 INFO:teuthology.orchestra.run.smithi176.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-09-16T01:28:20.977 INFO:teuthology.orchestra.run.smithi164.stdout:(73/156): python3-babel-2.9.1-2.el9.noarch.rpm 42 MB/s | 6.0 MB 00:00 2024-09-16T01:28:20.981 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T01:28:20.999 INFO:teuthology.orchestra.run.smithi176.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-16T01:28:21.044 INFO:teuthology.orchestra.run.smithi164.stdout:(74/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-09-16T01:28:21.069 INFO:teuthology.orchestra.run.smithi164.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 666 kB/s | 61 kB 00:00 2024-09-16T01:28:21.111 INFO:teuthology.orchestra.run.smithi164.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-09-16T01:28:21.130 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T01:28:21.130 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T01:28:21.153 INFO:teuthology.orchestra.run.smithi164.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-09-16T01:28:21.178 INFO:teuthology.orchestra.run.smithi164.stdout:(78/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-16T01:28:21.182 INFO:teuthology.orchestra.run.smithi176.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-09-16T01:28:21.203 INFO:teuthology.orchestra.run.smithi164.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-16T01:28:21.216 INFO:teuthology.orchestra.run.smithi176.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-09-16T01:28:21.241 INFO:teuthology.orchestra.run.smithi176.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-16T01:28:21.245 INFO:teuthology.orchestra.run.smithi164.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 838 kB/s | 35 kB 00:00 2024-09-16T01:28:21.267 INFO:teuthology.orchestra.run.smithi176.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-16T01:28:21.287 INFO:teuthology.orchestra.run.smithi164.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-16T01:28:21.292 INFO:teuthology.orchestra.run.smithi176.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-16T01:28:21.317 INFO:teuthology.orchestra.run.smithi176.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-09-16T01:28:21.359 INFO:teuthology.orchestra.run.smithi176.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-09-16T01:28:21.379 INFO:teuthology.orchestra.run.smithi164.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-09-16T01:28:21.401 INFO:teuthology.orchestra.run.smithi176.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-09-16T01:28:21.421 INFO:teuthology.orchestra.run.smithi164.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-09-16T01:28:21.446 INFO:teuthology.orchestra.run.smithi164.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-09-16T01:28:21.468 INFO:teuthology.orchestra.run.smithi176.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-09-16T01:28:21.488 INFO:teuthology.orchestra.run.smithi164.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-16T01:28:21.493 INFO:teuthology.orchestra.run.smithi176.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-16T01:28:21.518 INFO:teuthology.orchestra.run.smithi176.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-09-16T01:28:21.552 INFO:teuthology.orchestra.run.smithi176.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-16T01:28:21.565 INFO:teuthology.orchestra.run.smithi164.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 358 kB/s | 42 kB 00:00 2024-09-16T01:28:21.586 INFO:teuthology.orchestra.run.smithi176.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-16T01:28:21.611 INFO:teuthology.orchestra.run.smithi176.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-16T01:28:21.622 INFO:teuthology.orchestra.run.smithi164.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.8 MB/s | 269 kB 00:00 2024-09-16T01:28:21.636 INFO:teuthology.orchestra.run.smithi176.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-16T01:28:21.662 INFO:teuthology.orchestra.run.smithi176.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-16T01:28:21.664 INFO:teuthology.orchestra.run.smithi164.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-09-16T01:28:21.687 INFO:teuthology.orchestra.run.smithi176.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-16T01:28:21.690 INFO:teuthology.orchestra.run.smithi164.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-09-16T01:28:21.712 INFO:teuthology.orchestra.run.smithi176.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-16T01:28:21.715 INFO:teuthology.orchestra.run.smithi164.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-16T01:28:21.740 INFO:teuthology.orchestra.run.smithi164.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-16T01:28:21.774 INFO:teuthology.orchestra.run.smithi164.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-09-16T01:28:21.804 INFO:teuthology.orchestra.run.smithi176.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 42 MB/s | 4.9 MB 00:00 2024-09-16T01:28:21.807 INFO:teuthology.orchestra.run.smithi164.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-16T01:28:21.849 INFO:teuthology.orchestra.run.smithi164.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-09-16T01:28:21.883 INFO:teuthology.orchestra.run.smithi164.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-09-16T01:28:21.896 INFO:teuthology.orchestra.run.smithi176.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-09-16T01:28:21.916 INFO:teuthology.orchestra.run.smithi164.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-16T01:28:21.938 INFO:teuthology.orchestra.run.smithi176.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-16T01:28:21.967 INFO:teuthology.orchestra.run.smithi164.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-16T01:28:21.972 INFO:teuthology.orchestra.run.smithi176.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-16T01:28:21.988 INFO:teuthology.orchestra.run.smithi176.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.7 MB/s | 137 kB 00:00 2024-09-16T01:28:22.000 INFO:teuthology.orchestra.run.smithi164.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2024-09-16T01:28:22.042 INFO:teuthology.orchestra.run.smithi164.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 2.2 MB/s | 96 kB 00:00 2024-09-16T01:28:22.047 INFO:teuthology.orchestra.run.smithi176.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-16T01:28:22.076 INFO:teuthology.orchestra.run.smithi164.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-09-16T01:28:22.110 INFO:teuthology.orchestra.run.smithi164.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-16T01:28:22.124 INFO:teuthology.orchestra.run.smithi176.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 38 MB/s | 5.8 MB 00:00 2024-09-16T01:28:22.148 INFO:teuthology.orchestra.run.smithi176.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-09-16T01:28:22.190 INFO:teuthology.orchestra.run.smithi176.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-16T01:28:22.265 INFO:teuthology.orchestra.run.smithi176.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 42 MB/s | 6.0 MB 00:00 2024-09-16T01:28:22.290 INFO:teuthology.orchestra.run.smithi176.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-16T01:28:22.316 INFO:teuthology.orchestra.run.smithi176.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-16T01:28:22.341 INFO:teuthology.orchestra.run.smithi176.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-16T01:28:22.423 INFO:teuthology.orchestra.run.smithi164.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-09-16T01:28:22.423 INFO:teuthology.orchestra.run.smithi176.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-16T01:28:22.423 INFO:teuthology.orchestra.run.smithi176.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-16T01:28:22.425 INFO:teuthology.orchestra.run.smithi176.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-09-16T01:28:22.476 INFO:teuthology.orchestra.run.smithi176.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-16T01:28:22.551 INFO:teuthology.orchestra.run.smithi176.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-09-16T01:28:22.576 INFO:teuthology.orchestra.run.smithi176.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-16T01:28:22.601 INFO:teuthology.orchestra.run.smithi176.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-16T01:28:22.626 INFO:teuthology.orchestra.run.smithi176.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-16T01:28:22.652 INFO:teuthology.orchestra.run.smithi176.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-16T01:28:22.677 INFO:teuthology.orchestra.run.smithi176.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-16T01:28:22.702 INFO:teuthology.orchestra.run.smithi176.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-16T01:28:22.728 INFO:teuthology.orchestra.run.smithi176.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-16T01:28:22.753 INFO:teuthology.orchestra.run.smithi176.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-16T01:28:22.786 INFO:teuthology.orchestra.run.smithi164.stdout:(103/156): postgresql-server-13.16-1.el9.x86_64 2.8 MB/s | 5.8 MB 00:02 2024-09-16T01:28:22.803 INFO:teuthology.orchestra.run.smithi176.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 836 kB/s | 42 kB 00:00 2024-09-16T01:28:22.844 INFO:teuthology.orchestra.run.smithi164.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-16T01:28:22.845 INFO:teuthology.orchestra.run.smithi176.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-16T01:28:22.878 INFO:teuthology.orchestra.run.smithi164.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-16T01:28:22.896 INFO:teuthology.orchestra.run.smithi176.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-16T01:28:22.903 INFO:teuthology.orchestra.run.smithi164.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 775 kB/s | 19 kB 00:00 2024-09-16T01:28:22.946 INFO:teuthology.orchestra.run.smithi176.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 722 kB/s | 36 kB 00:00 2024-09-16T01:28:23.012 INFO:teuthology.orchestra.run.smithi164.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-16T01:28:23.037 INFO:teuthology.orchestra.run.smithi164.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T01:28:23.062 INFO:teuthology.orchestra.run.smithi164.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-16T01:28:23.087 INFO:teuthology.orchestra.run.smithi164.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T01:28:23.104 INFO:teuthology.orchestra.run.smithi176.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-09-16T01:28:23.113 INFO:teuthology.orchestra.run.smithi164.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-16T01:28:23.130 INFO:teuthology.orchestra.run.smithi176.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 258 kB/s | 47 kB 00:00 2024-09-16T01:28:23.148 INFO:teuthology.orchestra.run.smithi164.stdout:(112/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-09-16T01:28:23.155 INFO:teuthology.orchestra.run.smithi176.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 6.0 MB/s | 304 kB 00:00 2024-09-16T01:28:23.180 INFO:teuthology.orchestra.run.smithi176.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T01:28:23.197 INFO:teuthology.orchestra.run.smithi164.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-16T01:28:23.205 INFO:teuthology.orchestra.run.smithi176.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-16T01:28:23.230 INFO:teuthology.orchestra.run.smithi164.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.4 MB/s | 43 kB 00:00 2024-09-16T01:28:23.231 INFO:teuthology.orchestra.run.smithi176.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T01:28:23.255 INFO:teuthology.orchestra.run.smithi164.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T01:28:23.265 INFO:teuthology.orchestra.run.smithi176.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 761 kB/s | 45 kB 00:00 2024-09-16T01:28:23.281 INFO:teuthology.orchestra.run.smithi164.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-16T01:28:23.306 INFO:teuthology.orchestra.run.smithi164.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-16T01:28:23.314 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T01:28:23.314 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T01:28:23.331 INFO:teuthology.orchestra.run.smithi164.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-09-16T01:28:23.332 INFO:teuthology.orchestra.run.smithi176.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-16T01:28:23.365 INFO:teuthology.orchestra.run.smithi164.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-16T01:28:23.390 INFO:teuthology.orchestra.run.smithi176.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-16T01:28:23.391 INFO:teuthology.orchestra.run.smithi164.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-16T01:28:23.424 INFO:teuthology.orchestra.run.smithi176.stdout:(105/156): protobuf-compiler-3.14.0-13.el9.x86_ 5.3 MB/s | 863 kB 00:00 2024-09-16T01:28:23.424 INFO:teuthology.orchestra.run.smithi164.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-16T01:28:23.457 INFO:teuthology.orchestra.run.smithi176.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 4.5 MB/s | 308 kB 00:00 2024-09-16T01:28:23.458 INFO:teuthology.orchestra.run.smithi164.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-16T01:28:23.483 INFO:teuthology.orchestra.run.smithi176.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-09-16T01:28:23.483 INFO:teuthology.orchestra.run.smithi164.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-16T01:28:23.500 INFO:teuthology.orchestra.run.smithi176.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-16T01:28:23.508 INFO:teuthology.orchestra.run.smithi164.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-16T01:28:23.525 INFO:teuthology.orchestra.run.smithi176.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-16T01:28:23.534 INFO:teuthology.orchestra.run.smithi164.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 921 kB/s | 23 kB 00:00 2024-09-16T01:28:23.550 INFO:teuthology.orchestra.run.smithi176.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-16T01:28:23.560 INFO:teuthology.orchestra.run.smithi164.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-09-16T01:28:23.575 INFO:teuthology.orchestra.run.smithi176.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-16T01:28:23.601 INFO:teuthology.orchestra.run.smithi164.stdout:(127/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.7 MB/s | 2.0 MB 00:00 2024-09-16T01:28:23.626 INFO:teuthology.orchestra.run.smithi176.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-16T01:28:23.626 INFO:teuthology.orchestra.run.smithi164.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 293 kB/s | 20 kB 00:00 2024-09-16T01:28:23.651 INFO:teuthology.orchestra.run.smithi164.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T01:28:23.668 INFO:teuthology.orchestra.run.smithi176.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-16T01:28:23.668 INFO:teuthology.orchestra.run.smithi164.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 4.6 MB/s | 79 kB 00:00 2024-09-16T01:28:23.693 INFO:teuthology.orchestra.run.smithi176.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T01:28:23.693 INFO:teuthology.orchestra.run.smithi164.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-09-16T01:28:23.718 INFO:teuthology.orchestra.run.smithi176.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T01:28:23.719 INFO:teuthology.orchestra.run.smithi164.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-16T01:28:23.744 INFO:teuthology.orchestra.run.smithi176.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-16T01:28:23.744 INFO:teuthology.orchestra.run.smithi164.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-09-16T01:28:23.769 INFO:teuthology.orchestra.run.smithi176.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-16T01:28:23.770 INFO:teuthology.orchestra.run.smithi164.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-09-16T01:28:23.795 INFO:teuthology.orchestra.run.smithi164.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-16T01:28:23.821 INFO:teuthology.orchestra.run.smithi164.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 560 kB/s | 28 kB 00:00 2024-09-16T01:28:23.836 INFO:teuthology.orchestra.run.smithi176.stdout:(118/156): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-16T01:28:23.846 INFO:teuthology.orchestra.run.smithi164.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-16T01:28:23.861 INFO:teuthology.orchestra.run.smithi176.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-16T01:28:23.886 INFO:teuthology.orchestra.run.smithi176.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-16T01:28:23.887 INFO:teuthology.orchestra.run.smithi164.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-16T01:28:23.912 INFO:teuthology.orchestra.run.smithi176.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-16T01:28:23.913 INFO:teuthology.orchestra.run.smithi164.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-09-16T01:28:23.938 INFO:teuthology.orchestra.run.smithi164.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-09-16T01:28:23.945 INFO:teuthology.orchestra.run.smithi176.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-16T01:28:23.963 INFO:teuthology.orchestra.run.smithi164.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-16T01:28:23.970 INFO:teuthology.orchestra.run.smithi176.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-16T01:28:23.989 INFO:teuthology.orchestra.run.smithi164.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-16T01:28:23.996 INFO:teuthology.orchestra.run.smithi176.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-16T01:28:24.021 INFO:teuthology.orchestra.run.smithi176.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 925 kB/s | 23 kB 00:00 2024-09-16T01:28:24.022 INFO:teuthology.orchestra.run.smithi164.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-16T01:28:24.047 INFO:teuthology.orchestra.run.smithi176.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-09-16T01:28:24.047 INFO:teuthology.orchestra.run.smithi164.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 808 kB/s | 20 kB 00:00 2024-09-16T01:28:24.071 INFO:teuthology.orchestra.run.smithi176.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 796 kB/s | 20 kB 00:00 2024-09-16T01:28:24.073 INFO:teuthology.orchestra.run.smithi164.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-16T01:28:24.098 INFO:teuthology.orchestra.run.smithi164.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T01:28:24.122 INFO:teuthology.orchestra.run.smithi176.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-16T01:28:24.147 INFO:teuthology.orchestra.run.smithi176.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T01:28:24.148 INFO:teuthology.orchestra.run.smithi164.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-16T01:28:24.172 INFO:teuthology.orchestra.run.smithi176.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-16T01:28:24.206 INFO:teuthology.orchestra.run.smithi176.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-16T01:28:24.257 INFO:teuthology.orchestra.run.smithi176.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-09-16T01:28:24.282 INFO:teuthology.orchestra.run.smithi176.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-16T01:28:24.307 INFO:teuthology.orchestra.run.smithi176.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-09-16T01:28:24.333 INFO:teuthology.orchestra.run.smithi176.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-16T01:28:24.357 INFO:teuthology.orchestra.run.smithi164.stdout:(148/156): librados2-19.1.1-326.g8921c8ef.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-16T01:28:24.358 INFO:teuthology.orchestra.run.smithi176.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-16T01:28:24.383 INFO:teuthology.orchestra.run.smithi176.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-16T01:28:24.408 INFO:teuthology.orchestra.run.smithi176.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T01:28:24.424 INFO:teuthology.orchestra.run.smithi164.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-16T01:28:24.433 INFO:teuthology.orchestra.run.smithi176.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-16T01:28:24.459 INFO:teuthology.orchestra.run.smithi176.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-16T01:28:24.484 INFO:teuthology.orchestra.run.smithi176.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-16T01:28:24.510 INFO:teuthology.orchestra.run.smithi176.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-16T01:28:24.543 INFO:teuthology.orchestra.run.smithi176.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-16T01:28:24.560 INFO:teuthology.orchestra.run.smithi176.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-16T01:28:24.594 INFO:teuthology.orchestra.run.smithi176.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-16T01:28:24.619 INFO:teuthology.orchestra.run.smithi176.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-16T01:28:24.669 INFO:teuthology.orchestra.run.smithi176.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-16T01:28:24.758 INFO:teuthology.orchestra.run.smithi164.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 5.8 MB/s | 2.3 MB 00:00 2024-09-16T01:28:24.825 INFO:teuthology.orchestra.run.smithi164.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 4.5 MB/s | 1.8 MB 00:00 2024-09-16T01:28:24.850 INFO:teuthology.orchestra.run.smithi164.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 444 kB/s | 41 kB 00:00 2024-09-16T01:28:24.900 INFO:teuthology.orchestra.run.smithi164.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 3.2 MB/s | 249 kB 00:00 2024-09-16T01:28:24.911 INFO:teuthology.orchestra.run.smithi176.stdout:(148/156): librados2-19.1.1-326.g8921c8ef.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-16T01:28:25.026 INFO:teuthology.orchestra.run.smithi164.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 7.7 MB/s | 1.4 MB 00:00 2024-09-16T01:28:25.159 INFO:teuthology.orchestra.run.smithi164.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.2 MB/s | 2.4 MB 00:00 2024-09-16T01:28:25.411 INFO:teuthology.orchestra.run.smithi176.stdout:(149/156): ceph-test-19.1.1-326.g8921c8ef.el9.x 6.6 MB/s | 49 MB 00:07 2024-09-16T01:28:25.470 INFO:teuthology.orchestra.run.smithi176.stdout:(150/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 3.9 MB/s | 3.2 MB 00:00 2024-09-16T01:28:25.604 INFO:teuthology.orchestra.run.smithi176.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 3.4 MB/s | 2.3 MB 00:00 2024-09-16T01:28:25.629 INFO:teuthology.orchestra.run.smithi176.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 257 kB/s | 41 kB 00:00 2024-09-16T01:28:25.662 INFO:teuthology.orchestra.run.smithi176.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 4.2 MB/s | 249 kB 00:00 2024-09-16T01:28:25.754 INFO:teuthology.orchestra.run.smithi176.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 5.3 MB/s | 1.8 MB 00:00 2024-09-16T01:28:25.829 INFO:teuthology.orchestra.run.smithi176.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 6.8 MB/s | 1.4 MB 00:00 2024-09-16T01:28:25.963 INFO:teuthology.orchestra.run.smithi176.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 8.0 MB/s | 2.4 MB 00:00 2024-09-16T01:28:25.969 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:28:25.969 INFO:teuthology.orchestra.run.smithi176.stdout:Total 21 MB/s | 226 MB 00:10 2024-09-16T01:28:27.076 INFO:teuthology.orchestra.run.smithi164.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.0 MB/s | 16 MB 00:05 2024-09-16T01:28:27.082 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:28:27.082 INFO:teuthology.orchestra.run.smithi164.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-16T01:28:27.145 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-09-16T01:28:27.296 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-09-16T01:28:27.297 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-09-16T01:28:28.315 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-16T01:28:28.467 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-16T01:28:28.467 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-16T01:28:29.096 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-09-16T01:28:29.096 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-09-16T01:28:29.723 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T01:28:29.854 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T01:28:29.895 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T01:28:29.962 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T01:28:29.995 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T01:28:30.021 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T01:28:30.079 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T01:28:30.320 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-16T01:28:30.320 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-16T01:28:30.427 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T01:28:30.530 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:28:30.654 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:28:30.690 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:28:30.782 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:28:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:28:31.002 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T01:28:31.066 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T01:28:31.093 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:28:31.199 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:28:31.227 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:28:31.307 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:28:31.337 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:28:31.356 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:28:31.573 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:28:31.736 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:28:31.852 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T01:28:31.955 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:28:32.028 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:28:32.244 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:28:32.357 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T01:28:32.465 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T01:28:32.500 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T01:28:32.544 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T01:28:32.626 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T01:28:32.678 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T01:28:32.718 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T01:28:32.761 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T01:28:32.794 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T01:28:32.821 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T01:28:32.891 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T01:28:32.944 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T01:28:32.988 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T01:28:33.044 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T01:28:33.122 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T01:28:33.154 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T01:28:33.197 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T01:28:33.226 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T01:28:33.295 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T01:28:33.340 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T01:28:33.414 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T01:28:33.556 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T01:28:33.609 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T01:28:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T01:28:34.221 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T01:28:34.259 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T01:28:34.302 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T01:28:34.340 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T01:28:34.383 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T01:28:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T01:28:34.601 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T01:28:34.655 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T01:28:34.729 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T01:28:34.768 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T01:28:34.804 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T01:28:34.852 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T01:28:34.888 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T01:28:34.931 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T01:28:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T01:28:35.327 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T01:28:35.374 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T01:28:35.415 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T01:28:35.501 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-09-16T01:28:35.520 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T01:28:35.553 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T01:28:35.593 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T01:28:35.625 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T01:28:35.634 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T01:28:35.659 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T01:28:35.663 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T01:28:35.725 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T01:28:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T01:28:35.759 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T01:28:35.784 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T01:28:35.827 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T01:28:35.885 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T01:28:35.924 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T01:28:35.969 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T01:28:36.026 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T01:28:36.067 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T01:28:36.099 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T01:28:36.129 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T01:28:36.148 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T01:28:36.164 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T01:28:36.197 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T01:28:36.238 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T01:28:36.252 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:28:36.273 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T01:28:36.360 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T01:28:36.375 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:28:36.403 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:28:36.411 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T01:28:36.452 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T01:28:36.494 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:28:36.524 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:28:36.553 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T01:28:36.602 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T01:28:36.670 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T01:28:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-16T01:28:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T01:28:36.757 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:28:36.829 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T01:28:36.861 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:28:36.868 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T01:28:36.883 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:28:36.938 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T01:28:36.945 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:28:36.968 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:28:36.972 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T01:28:36.989 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:28:36.997 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T01:28:37.040 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T01:28:37.318 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T01:28:37.366 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T01:28:37.373 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T01:28:37.415 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T01:28:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:28:37.487 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:28:37.514 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T01:28:37.578 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T01:28:37.604 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:28:37.623 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T01:28:37.625 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:28:37.632 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:28:37.711 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T01:28:37.716 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:28:37.751 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T01:28:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:28:37.766 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T01:28:37.867 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:28:37.911 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T01:28:37.934 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:28:37.935 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T01:28:37.965 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T01:28:37.968 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T01:28:37.996 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:28:38.043 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T01:28:38.091 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:28:38.113 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:28:38.160 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T01:28:38.168 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:28:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:28:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T01:28:38.272 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T01:28:38.368 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T01:28:38.392 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:28:38.397 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T01:28:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T01:28:38.501 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T01:28:38.538 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T01:28:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T01:28:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T01:28:38.637 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T01:28:38.672 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T01:28:38.677 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T01:28:38.696 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T01:28:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T01:28:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T01:28:38.774 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:28:38.813 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T01:28:38.827 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T01:28:38.858 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T01:28:38.865 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:28:38.901 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T01:28:38.978 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T01:28:39.013 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T01:28:39.056 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T01:28:39.073 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T01:28:39.093 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T01:28:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T01:28:39.178 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T01:28:39.216 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T01:28:39.282 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T01:28:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T01:28:39.315 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T01:28:39.367 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T01:28:39.422 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T01:28:39.431 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T01:28:39.476 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T01:28:39.476 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T01:28:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T01:28:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T01:28:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T01:28:39.612 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T01:28:39.681 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T01:28:39.743 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T01:28:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T01:28:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T01:28:39.902 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T01:28:39.936 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T01:28:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T01:28:39.998 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T01:28:40.033 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T01:28:40.071 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T01:28:40.080 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T01:28:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T01:28:40.109 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T01:28:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T01:28:40.152 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T01:28:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T01:28:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T01:28:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T01:28:40.294 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T01:28:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T01:28:40.380 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T01:28:40.386 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T01:28:40.414 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T01:28:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T01:28:40.444 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T01:28:40.499 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T01:28:40.571 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T01:28:40.603 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T01:28:40.639 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T01:28:40.686 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T01:28:40.730 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T01:28:40.740 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T01:28:40.782 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T01:28:40.784 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:28:40.824 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T01:28:40.876 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:28:40.906 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T01:28:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T01:28:40.952 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T01:28:40.960 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T01:28:41.009 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T01:28:41.030 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T01:28:41.050 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T01:28:41.059 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T01:28:41.102 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T01:28:41.140 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T01:28:41.181 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T01:28:41.344 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T01:28:41.382 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T01:28:41.395 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T01:28:41.426 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T01:28:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T01:28:41.450 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T01:28:41.466 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T01:28:41.478 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:28:41.491 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T01:28:41.516 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T01:28:41.523 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T01:28:41.550 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:28:41.555 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T01:28:41.580 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T01:28:41.591 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T01:28:41.599 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T01:28:41.646 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T01:28:41.682 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T01:28:41.716 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T01:28:41.725 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T01:28:41.756 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T01:28:41.767 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T01:28:41.793 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T01:28:41.835 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T01:28:41.874 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T01:28:41.904 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T01:28:41.914 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T01:28:41.944 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T01:28:41.945 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T01:28:41.979 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T01:28:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T01:28:42.013 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T01:28:42.054 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T01:28:42.088 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T01:28:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T01:28:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T01:28:42.239 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T01:28:42.267 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T01:28:42.345 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T01:28:42.370 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T01:28:42.384 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T01:28:42.410 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T01:28:42.425 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T01:28:42.434 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T01:28:42.510 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T01:28:42.850 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T01:28:42.892 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T01:28:42.920 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T01:28:42.955 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T01:28:42.994 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T01:28:43.033 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T01:28:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T01:28:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T01:28:43.131 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T01:28:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T01:28:43.173 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T01:28:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T01:28:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T01:28:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T01:28:43.297 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T01:28:43.339 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T01:28:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T01:28:43.379 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T01:28:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T01:28:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T01:28:43.490 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T01:28:43.527 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T01:28:43.559 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T01:28:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T01:28:43.743 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T01:28:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T01:28:43.876 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T01:28:44.212 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:28:44.252 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:28:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T01:28:44.345 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T01:28:44.378 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T01:28:44.478 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T01:28:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T01:28:44.541 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T01:28:44.586 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T01:28:44.639 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T01:28:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T01:28:44.706 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T01:28:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:28:44.846 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:28:44.892 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T01:28:44.920 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T01:28:44.942 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:28:44.942 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T01:28:44.942 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T01:28:44.942 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:28:44.983 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:28:44.997 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T01:28:45.645 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T01:28:45.752 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T01:28:45.851 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T01:28:45.930 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T01:28:45.957 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T01:28:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T01:28:46.300 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T01:28:46.345 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:28:46.436 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:28:46.467 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T01:28:46.503 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T01:28:46.982 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T01:28:46.985 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T01:28:47.023 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:28:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T01:28:47.094 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:28:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T01:28:47.117 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T01:28:47.144 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T01:28:47.455 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T01:28:47.499 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:28:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T01:28:47.649 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T01:28:47.690 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T01:28:48.173 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T01:28:48.210 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:28:48.265 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T01:28:48.295 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T01:28:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:28:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:28:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:28:50.026 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:28:50.105 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:28:50.105 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T01:28:50.105 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T01:28:50.105 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:28:50.144 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:28:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:28:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:28:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T01:28:51.195 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:28:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T01:28:51.289 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T01:28:51.289 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T01:28:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:28:51.322 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:28:56.651 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:28:56.651 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /sys 2024-09-16T01:28:56.651 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /proc 2024-09-16T01:28:56.651 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /mnt 2024-09-16T01:28:56.651 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /var/tmp 2024-09-16T01:28:56.651 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /home 2024-09-16T01:28:56.651 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /root 2024-09-16T01:28:56.651 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /tmp 2024-09-16T01:28:56.651 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:28:56.826 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:28:56.843 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:28:56.843 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:28:56.843 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T01:28:56.843 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:28:58.362 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:28:58.389 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:28:58.389 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:28:58.389 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T01:28:58.389 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:28:58.390 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:28:58.390 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:28:58.621 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:28:58.645 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:28:58.645 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:28:58.645 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T01:28:58.645 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:28:58.646 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:28:58.646 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:28:59.088 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:28:59.112 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:28:59.112 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:28:59.112 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T01:28:59.112 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:28:59.113 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:28:59.113 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:28:59.218 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T01:28:59.275 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T01:28:59.330 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T01:28:59.515 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T01:28:59.564 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T01:28:59.649 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T01:28:59.755 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T01:28:59.902 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T01:28:59.967 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T01:29:00.004 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T01:29:00.031 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T01:29:00.445 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:29:00.472 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:29:00.503 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T01:29:00.565 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T01:29:00.595 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:29:01.647 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:29:01.688 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:29:01.878 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:29:01.878 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /sys 2024-09-16T01:29:01.878 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /proc 2024-09-16T01:29:01.878 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /mnt 2024-09-16T01:29:01.878 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /var/tmp 2024-09-16T01:29:01.878 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /home 2024-09-16T01:29:01.878 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /root 2024-09-16T01:29:01.879 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /tmp 2024-09-16T01:29:01.879 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:29:02.054 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:29:02.069 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:29:02.069 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:02.069 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T01:29:02.069 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:29:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:29:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:29:02.800 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:29:02.980 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T01:29:03.008 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T01:29:03.008 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-09-16T01:29:03.008 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-09-16T01:29:03.008 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-09-16T01:29:03.008 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-09-16T01:29:03.008 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-09-16T01:29:03.008 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-09-16T01:29:03.008 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-09-16T01:29:03.008 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:29:03.016 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:29:03.041 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:29:03.041 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:03.041 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T01:29:03.041 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:29:03.041 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:29:03.041 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:29:03.091 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:29:03.103 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:29:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:29:03.200 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T01:29:03.200 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:03.200 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T01:29:03.200 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:29:03.204 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T01:29:03.264 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T01:29:03.565 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:29:03.592 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:29:03.592 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:03.592 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T01:29:03.592 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:29:03.592 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:29:03.593 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:29:03.825 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:29:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:29:03.848 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:03.848 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T01:29:03.848 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:29:03.848 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:29:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:29:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T01:29:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:29:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:29:04.259 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:04.259 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T01:29:04.259 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:29:04.260 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:29:04.260 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:29:04.293 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:29:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:29:04.316 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:29:04.316 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:04.316 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T01:29:04.316 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:29:04.317 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:29:04.317 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:29:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:29:04.326 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:04.326 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T01:29:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:29:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:29:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:29:04.402 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:04.402 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T01:29:04.402 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:29:04.402 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:29:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:29:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T01:29:04.479 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T01:29:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T01:29:04.672 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:29:04.698 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T01:29:04.699 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:04.699 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T01:29:04.699 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:29:04.699 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T01:29:04.699 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:29:04.719 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T01:29:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:29:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:29:04.745 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:04.745 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T01:29:04.745 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:29:04.745 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:29:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:29:04.768 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T01:29:04.846 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T01:29:04.932 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:29:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T01:29:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T01:29:04.954 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:04.954 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T01:29:04.954 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:29:04.954 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T01:29:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:29:05.073 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T01:29:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T01:29:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T01:29:05.178 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T01:29:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:29:05.413 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T01:29:05.413 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:05.413 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T01:29:05.414 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:29:05.414 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T01:29:05.414 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:29:05.528 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T01:29:05.582 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:29:05.585 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T01:29:05.610 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:29:05.640 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T01:29:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T01:29:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T01:29:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:29:05.836 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T01:29:05.882 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T01:29:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T01:29:06.048 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T01:29:06.180 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T01:29:06.233 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T01:29:06.264 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T01:29:06.292 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T01:29:06.696 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:29:06.724 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T01:29:06.755 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T01:29:06.774 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:29:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T01:29:06.818 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:29:06.847 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:29:07.776 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:29:07.802 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:29:07.889 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T01:29:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:29:07.924 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:29:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T01:29:08.095 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:29:08.119 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:29:08.120 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:08.120 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T01:29:08.120 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:29:08.120 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:29:08.120 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:29:08.171 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:29:08.182 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:29:08.280 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T01:29:08.335 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T01:29:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T01:29:08.908 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:29:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T01:29:09.169 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T01:29:09.201 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:29:09.226 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T01:29:09.226 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:09.226 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T01:29:09.226 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:29:09.226 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T01:29:09.226 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:29:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T01:29:09.277 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:29:09.289 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T01:29:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:29:09.339 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:29:09.339 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:09.340 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T01:29:09.340 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:29:09.340 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:29:09.340 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:29:09.382 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:29:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T01:29:09.405 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:29:09.405 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:09.405 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T01:29:09.405 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:29:09.450 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:29:09.450 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T01:29:09.457 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T01:29:09.473 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:29:09.473 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:09.473 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T01:29:09.473 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:29:09.473 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:29:09.473 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:29:09.499 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T01:29:09.540 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T01:29:09.572 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T01:29:09.599 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T01:29:09.776 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:29:09.791 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T01:29:09.798 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:29:09.798 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:09.798 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T01:29:09.798 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:29:09.798 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:29:09.799 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:29:09.837 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T01:29:09.867 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T01:29:09.897 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T01:29:09.935 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T01:29:10.379 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T01:29:10.413 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:29:10.438 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T01:29:10.438 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:10.438 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T01:29:10.438 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:29:10.438 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T01:29:10.438 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:29:10.480 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:29:10.504 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T01:29:10.504 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:10.504 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T01:29:10.504 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:29:10.537 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:29:10.556 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:29:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T01:29:10.580 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:10.580 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T01:29:10.580 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:29:10.580 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T01:29:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:29:10.582 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:29:10.603 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:29:10.890 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:29:10.914 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T01:29:10.914 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T01:29:10.914 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T01:29:10.914 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:29:10.914 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T01:29:10.914 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:29:11.438 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T01:29:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T01:29:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T01:29:11.665 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T01:29:11.698 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T01:29:11.723 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T01:29:11.758 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T01:29:11.758 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:29:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:29:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:29:11.934 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:29:11.959 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T01:29:11.965 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:29:11.965 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:29:12.153 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:29:12.153 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:29:12.237 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:29:12.301 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:29:14.304 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T01:29:14.345 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T01:29:14.385 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T01:29:14.418 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T01:29:14.445 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T01:29:14.618 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T01:29:14.659 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T01:29:14.689 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T01:29:14.718 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T01:29:14.756 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T01:29:15.388 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:29:15.420 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:29:15.441 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:29:15.443 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T01:29:15.484 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T01:29:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T01:29:15.558 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T01:29:15.585 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T01:29:15.768 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T01:29:15.816 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T01:29:15.845 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T01:29:15.876 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T01:29:15.913 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T01:29:16.268 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T01:29:16.358 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T01:29:16.399 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T01:29:16.481 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T01:29:16.487 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:29:16.511 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T01:29:16.527 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:29:16.536 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T01:29:16.548 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T01:29:16.563 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T01:29:16.563 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:29:16.638 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:29:16.638 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:29:16.722 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:29:16.756 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T01:29:16.760 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:29:16.761 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:29:16.933 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:29:16.933 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:29:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:29:17.079 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:29:17.388 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T01:29:17.475 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T01:29:17.516 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T01:29:17.597 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T01:29:17.636 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T01:29:17.661 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T01:29:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T01:29:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:29:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T01:29:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:29:17.864 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:29:17.864 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T01:29:17.864 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T01:29:17.864 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T01:29:17.864 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T01:29:17.865 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T01:29:17.865 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T01:29:17.865 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T01:29:17.865 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:29:17.865 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:29:17.865 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:29:17.865 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T01:29:17.865 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T01:29:17.865 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:29:17.865 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T01:29:17.866 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T01:29:17.866 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T01:29:17.866 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T01:29:17.866 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T01:29:17.866 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T01:29:17.866 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T01:29:17.866 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T01:29:17.866 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T01:29:17.866 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T01:29:17.866 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T01:29:17.867 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T01:29:17.867 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T01:29:17.867 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T01:29:17.867 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T01:29:17.867 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T01:29:17.867 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T01:29:17.867 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T01:29:17.867 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T01:29:17.867 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T01:29:17.867 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T01:29:17.867 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T01:29:17.867 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T01:29:17.868 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T01:29:17.868 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T01:29:17.868 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T01:29:17.868 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T01:29:17.868 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T01:29:17.868 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T01:29:17.868 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T01:29:17.868 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T01:29:17.868 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T01:29:17.868 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T01:29:17.869 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T01:29:17.869 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T01:29:17.869 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T01:29:17.869 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T01:29:17.869 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T01:29:17.869 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T01:29:17.869 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T01:29:17.869 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T01:29:17.869 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T01:29:17.869 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T01:29:17.869 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T01:29:17.869 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T01:29:17.869 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T01:29:17.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T01:29:17.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T01:29:17.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T01:29:17.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T01:29:17.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T01:29:17.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T01:29:17.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T01:29:17.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T01:29:17.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T01:29:17.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T01:29:17.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T01:29:17.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T01:29:17.870 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T01:29:17.871 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T01:29:17.871 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T01:29:17.871 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T01:29:17.871 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T01:29:17.871 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T01:29:17.871 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T01:29:17.871 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T01:29:17.871 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T01:29:17.871 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T01:29:17.871 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T01:29:17.871 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T01:29:17.872 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T01:29:17.872 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T01:29:17.872 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T01:29:17.872 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T01:29:17.872 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T01:29:17.872 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T01:29:17.872 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T01:29:17.872 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T01:29:17.872 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T01:29:17.872 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T01:29:17.872 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T01:29:17.872 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T01:29:17.873 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T01:29:17.873 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T01:29:17.873 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T01:29:17.873 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T01:29:17.873 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T01:29:17.873 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T01:29:17.873 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T01:29:17.873 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T01:29:17.873 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T01:29:17.873 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T01:29:17.873 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T01:29:17.873 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T01:29:17.874 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T01:29:17.874 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T01:29:17.874 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T01:29:17.874 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T01:29:17.874 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T01:29:17.874 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T01:29:17.874 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T01:29:17.874 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T01:29:17.874 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T01:29:17.874 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T01:29:17.874 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T01:29:17.875 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T01:29:17.875 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T01:29:17.875 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T01:29:17.875 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T01:29:17.875 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T01:29:17.875 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T01:29:17.875 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T01:29:17.875 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T01:29:17.875 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T01:29:17.875 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T01:29:17.875 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T01:29:17.875 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T01:29:17.875 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T01:29:17.876 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T01:29:17.876 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T01:29:17.876 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T01:29:17.876 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T01:29:17.876 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T01:29:17.876 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T01:29:17.876 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T01:29:17.876 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T01:29:17.876 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T01:29:17.876 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T01:29:17.876 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T01:29:17.876 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T01:29:17.877 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T01:29:17.877 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T01:29:17.877 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T01:29:17.877 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T01:29:17.877 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T01:29:17.877 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T01:29:17.877 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T01:29:17.877 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T01:29:17.877 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T01:29:17.877 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T01:29:17.877 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T01:29:17.877 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T01:29:17.877 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T01:29:17.878 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T01:29:17.878 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T01:29:17.878 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T01:29:17.878 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T01:29:17.878 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T01:29:17.878 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T01:29:17.878 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T01:29:17.878 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T01:29:17.889 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T01:29:17.895 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:29:17.895 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:29:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T01:29:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:29:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T01:29:18.213 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:29:20.330 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:29:20.330 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:29:20.331 INFO:teuthology.orchestra.run.smithi139.stdout:Upgraded: 2024-09-16T01:29:20.331 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T01:29:20.331 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T01:29:20.331 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T01:29:20.331 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T01:29:20.331 INFO:teuthology.orchestra.run.smithi139.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.331 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.331 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T01:29:20.331 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T01:29:20.331 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T01:29:20.331 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T01:29:20.331 INFO:teuthology.orchestra.run.smithi139.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T01:29:20.331 INFO:teuthology.orchestra.run.smithi139.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T01:29:20.332 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T01:29:20.332 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.332 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.332 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.332 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.332 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:20.332 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.332 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.332 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.332 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:20.332 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:20.332 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:20.332 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:20.333 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:20.333 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.333 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.333 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:20.333 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.333 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.333 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.333 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:20.333 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:20.333 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.333 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.333 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:20.333 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T01:29:20.333 INFO:teuthology.orchestra.run.smithi139.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T01:29:20.334 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T01:29:20.334 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T01:29:20.334 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T01:29:20.334 INFO:teuthology.orchestra.run.smithi139.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T01:29:20.334 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T01:29:20.334 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T01:29:20.334 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T01:29:20.334 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T01:29:20.334 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T01:29:20.334 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.334 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.334 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.334 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T01:29:20.334 INFO:teuthology.orchestra.run.smithi139.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T01:29:20.335 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T01:29:20.335 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T01:29:20.335 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T01:29:20.335 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.335 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.335 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T01:29:20.335 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.335 INFO:teuthology.orchestra.run.smithi139.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T01:29:20.335 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T01:29:20.335 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T01:29:20.335 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T01:29:20.335 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T01:29:20.335 INFO:teuthology.orchestra.run.smithi139.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T01:29:20.335 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T01:29:20.336 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T01:29:20.336 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T01:29:20.336 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T01:29:20.336 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T01:29:20.336 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T01:29:20.336 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T01:29:20.336 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T01:29:20.336 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T01:29:20.336 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T01:29:20.336 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T01:29:20.336 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T01:29:20.336 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T01:29:20.336 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T01:29:20.336 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T01:29:20.337 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T01:29:20.337 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T01:29:20.337 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.337 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.337 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.337 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T01:29:20.337 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T01:29:20.337 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T01:29:20.337 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T01:29:20.337 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T01:29:20.337 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T01:29:20.337 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T01:29:20.337 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T01:29:20.338 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T01:29:20.338 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T01:29:20.338 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T01:29:20.338 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T01:29:20.338 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T01:29:20.338 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T01:29:20.338 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T01:29:20.338 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T01:29:20.338 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T01:29:20.338 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T01:29:20.338 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T01:29:20.338 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T01:29:20.339 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T01:29:20.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T01:29:20.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T01:29:20.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T01:29:20.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T01:29:20.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T01:29:20.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T01:29:20.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T01:29:20.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T01:29:20.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T01:29:20.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.341 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.342 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:20.342 INFO:teuthology.orchestra.run.smithi139.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T01:29:20.342 INFO:teuthology.orchestra.run.smithi139.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T01:29:20.342 INFO:teuthology.orchestra.run.smithi139.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T01:29:20.342 INFO:teuthology.orchestra.run.smithi139.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T01:29:20.342 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T01:29:20.342 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T01:29:20.342 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T01:29:20.342 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:29:20.342 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T01:29:20.749 DEBUG:teuthology.parallel:result is None 2024-09-16T01:29:22.583 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:29:22.583 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T01:29:22.584 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T01:29:22.584 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T01:29:22.584 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T01:29:22.584 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T01:29:22.584 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T01:29:22.584 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T01:29:22.584 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:29:22.584 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:29:22.584 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:29:22.584 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T01:29:22.584 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T01:29:22.584 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:29:22.585 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T01:29:22.585 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T01:29:22.585 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T01:29:22.585 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T01:29:22.585 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T01:29:22.585 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T01:29:22.585 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T01:29:22.585 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T01:29:22.585 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T01:29:22.585 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T01:29:22.585 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T01:29:22.585 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T01:29:22.586 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T01:29:22.586 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T01:29:22.586 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T01:29:22.586 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T01:29:22.586 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T01:29:22.586 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T01:29:22.586 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T01:29:22.586 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T01:29:22.586 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T01:29:22.586 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T01:29:22.586 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T01:29:22.586 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T01:29:22.586 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T01:29:22.586 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T01:29:22.587 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T01:29:22.587 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T01:29:22.587 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T01:29:22.587 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T01:29:22.587 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T01:29:22.587 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T01:29:22.587 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T01:29:22.587 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T01:29:22.587 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T01:29:22.587 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T01:29:22.587 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T01:29:22.587 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T01:29:22.587 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T01:29:22.588 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T01:29:22.588 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T01:29:22.588 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T01:29:22.588 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T01:29:22.588 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T01:29:22.588 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T01:29:22.588 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T01:29:22.588 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T01:29:22.589 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T01:29:22.589 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T01:29:22.589 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T01:29:22.589 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T01:29:22.589 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T01:29:22.589 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T01:29:22.589 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T01:29:22.589 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T01:29:22.589 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T01:29:22.589 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T01:29:22.589 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T01:29:22.589 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T01:29:22.589 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T01:29:22.590 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T01:29:22.590 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T01:29:22.590 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T01:29:22.590 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T01:29:22.590 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T01:29:22.590 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T01:29:22.590 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T01:29:22.590 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T01:29:22.590 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T01:29:22.590 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T01:29:22.590 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T01:29:22.591 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T01:29:22.591 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T01:29:22.591 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T01:29:22.591 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T01:29:22.591 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T01:29:22.591 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T01:29:22.591 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T01:29:22.591 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T01:29:22.591 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T01:29:22.591 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T01:29:22.591 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T01:29:22.591 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T01:29:22.591 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T01:29:22.591 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T01:29:22.592 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T01:29:22.592 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T01:29:22.592 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T01:29:22.592 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T01:29:22.592 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T01:29:22.592 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T01:29:22.592 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T01:29:22.592 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T01:29:22.592 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T01:29:22.592 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T01:29:22.592 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T01:29:22.592 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T01:29:22.592 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T01:29:22.593 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T01:29:22.593 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T01:29:22.593 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T01:29:22.593 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T01:29:22.593 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T01:29:22.593 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T01:29:22.593 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T01:29:22.593 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T01:29:22.593 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T01:29:22.593 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T01:29:22.593 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T01:29:22.593 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T01:29:22.593 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T01:29:22.594 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T01:29:22.594 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T01:29:22.594 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T01:29:22.594 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T01:29:22.594 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T01:29:22.594 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T01:29:22.594 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T01:29:22.594 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T01:29:22.594 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T01:29:22.594 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T01:29:22.594 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T01:29:22.594 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T01:29:22.594 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T01:29:22.595 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T01:29:22.595 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T01:29:22.595 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T01:29:22.595 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T01:29:22.595 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T01:29:22.595 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T01:29:22.595 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T01:29:22.595 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T01:29:22.595 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T01:29:22.595 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T01:29:22.595 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T01:29:22.595 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T01:29:22.595 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T01:29:22.595 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T01:29:22.596 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T01:29:22.596 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T01:29:22.596 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T01:29:22.596 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T01:29:22.596 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T01:29:22.596 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T01:29:22.596 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T01:29:22.596 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T01:29:22.596 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T01:29:22.596 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T01:29:22.596 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T01:29:22.596 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T01:29:23.743 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:29:23.743 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T01:29:23.743 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T01:29:23.744 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T01:29:23.744 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T01:29:23.744 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T01:29:23.744 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T01:29:23.744 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T01:29:23.744 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T01:29:23.744 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T01:29:23.744 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T01:29:23.744 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T01:29:23.744 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T01:29:23.744 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T01:29:23.744 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T01:29:23.744 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T01:29:23.745 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T01:29:23.745 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T01:29:23.745 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T01:29:23.745 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T01:29:23.745 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T01:29:23.745 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T01:29:23.745 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T01:29:23.745 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T01:29:23.745 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T01:29:23.745 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T01:29:23.745 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T01:29:23.745 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T01:29:23.745 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T01:29:23.745 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T01:29:23.758 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T01:29:23.758 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T01:29:23.758 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T01:29:23.759 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T01:29:23.759 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T01:29:23.759 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T01:29:23.759 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T01:29:23.759 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T01:29:23.759 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T01:29:23.759 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T01:29:23.759 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T01:29:23.759 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T01:29:23.759 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T01:29:23.759 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T01:29:23.760 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T01:29:23.760 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T01:29:23.760 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T01:29:23.760 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T01:29:23.760 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T01:29:23.760 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T01:29:23.760 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T01:29:23.760 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T01:29:23.760 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T01:29:23.760 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T01:29:23.761 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T01:29:23.761 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T01:29:23.761 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T01:29:23.761 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T01:29:23.761 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T01:29:23.761 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T01:29:23.761 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T01:29:23.761 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T01:29:23.761 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T01:29:23.761 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T01:29:23.761 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T01:29:23.761 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T01:29:23.761 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T01:29:23.761 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T01:29:23.762 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T01:29:23.762 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T01:29:23.762 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T01:29:23.762 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T01:29:23.762 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T01:29:23.762 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T01:29:23.762 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T01:29:23.762 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T01:29:23.762 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T01:29:23.762 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T01:29:23.762 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T01:29:23.763 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T01:29:23.763 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T01:29:23.763 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T01:29:23.763 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T01:29:23.763 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T01:29:23.763 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T01:29:23.763 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T01:29:23.763 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T01:29:23.763 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T01:29:23.764 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T01:29:23.764 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T01:29:23.764 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T01:29:23.764 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T01:29:23.764 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T01:29:23.764 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T01:29:23.764 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T01:29:23.764 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T01:29:23.764 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T01:29:23.765 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T01:29:23.765 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T01:29:23.765 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T01:29:23.765 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T01:29:23.765 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T01:29:23.765 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T01:29:23.765 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T01:29:23.765 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T01:29:23.765 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T01:29:23.765 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T01:29:23.765 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T01:29:23.765 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T01:29:23.765 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T01:29:23.765 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T01:29:23.766 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T01:29:23.766 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T01:29:23.766 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T01:29:23.766 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T01:29:23.766 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T01:29:23.766 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T01:29:23.766 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T01:29:23.766 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T01:29:23.766 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T01:29:23.766 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T01:29:23.766 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T01:29:23.766 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T01:29:23.766 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T01:29:23.766 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T01:29:23.767 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T01:29:23.767 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T01:29:23.767 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T01:29:23.767 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T01:29:23.767 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T01:29:23.767 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T01:29:23.767 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T01:29:23.768 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T01:29:23.768 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T01:29:23.768 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T01:29:23.768 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T01:29:23.768 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T01:29:23.768 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T01:29:23.768 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T01:29:23.768 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T01:29:23.768 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T01:29:23.768 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T01:29:23.768 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T01:29:23.768 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T01:29:23.769 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T01:29:23.769 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T01:29:23.769 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T01:29:23.769 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T01:29:23.769 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T01:29:23.769 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T01:29:23.769 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T01:29:23.769 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T01:29:23.769 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T01:29:23.769 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T01:29:23.769 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T01:29:23.769 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T01:29:23.770 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T01:29:23.770 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T01:29:23.770 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T01:29:23.770 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T01:29:23.770 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T01:29:23.770 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T01:29:23.770 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T01:29:23.770 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T01:29:25.166 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:29:25.166 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:29:25.166 INFO:teuthology.orchestra.run.smithi176.stdout:Upgraded: 2024-09-16T01:29:25.166 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T01:29:25.166 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T01:29:25.166 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T01:29:25.166 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T01:29:25.166 INFO:teuthology.orchestra.run.smithi176.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.166 INFO:teuthology.orchestra.run.smithi176.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.166 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T01:29:25.167 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T01:29:25.167 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-09-16T01:29:25.167 INFO:teuthology.orchestra.run.smithi176.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T01:29:25.167 INFO:teuthology.orchestra.run.smithi176.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T01:29:25.167 INFO:teuthology.orchestra.run.smithi176.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T01:29:25.167 INFO:teuthology.orchestra.run.smithi176.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T01:29:25.167 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.167 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.167 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.167 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.167 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:25.167 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.168 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.168 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.168 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:25.168 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:25.168 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:25.168 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:25.168 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:25.168 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.168 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.168 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:25.168 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.168 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.169 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.169 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:25.169 INFO:teuthology.orchestra.run.smithi176.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:25.169 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.169 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.169 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:25.169 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T01:29:25.169 INFO:teuthology.orchestra.run.smithi176.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T01:29:25.169 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T01:29:25.169 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T01:29:25.169 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T01:29:25.169 INFO:teuthology.orchestra.run.smithi176.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T01:29:25.170 INFO:teuthology.orchestra.run.smithi176.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T01:29:25.170 INFO:teuthology.orchestra.run.smithi176.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T01:29:25.170 INFO:teuthology.orchestra.run.smithi176.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T01:29:25.170 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T01:29:25.170 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T01:29:25.170 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.170 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.170 INFO:teuthology.orchestra.run.smithi176.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.170 INFO:teuthology.orchestra.run.smithi176.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T01:29:25.170 INFO:teuthology.orchestra.run.smithi176.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T01:29:25.170 INFO:teuthology.orchestra.run.smithi176.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T01:29:25.170 INFO:teuthology.orchestra.run.smithi176.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T01:29:25.170 INFO:teuthology.orchestra.run.smithi176.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T01:29:25.171 INFO:teuthology.orchestra.run.smithi176.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.171 INFO:teuthology.orchestra.run.smithi176.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.171 INFO:teuthology.orchestra.run.smithi176.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T01:29:25.171 INFO:teuthology.orchestra.run.smithi176.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.171 INFO:teuthology.orchestra.run.smithi176.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T01:29:25.171 INFO:teuthology.orchestra.run.smithi176.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T01:29:25.171 INFO:teuthology.orchestra.run.smithi176.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T01:29:25.171 INFO:teuthology.orchestra.run.smithi176.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T01:29:25.171 INFO:teuthology.orchestra.run.smithi176.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T01:29:25.172 INFO:teuthology.orchestra.run.smithi176.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T01:29:25.172 INFO:teuthology.orchestra.run.smithi176.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T01:29:25.172 INFO:teuthology.orchestra.run.smithi176.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T01:29:25.172 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T01:29:25.172 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T01:29:25.172 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T01:29:25.173 INFO:teuthology.orchestra.run.smithi176.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T01:29:25.173 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T01:29:25.173 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T01:29:25.173 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T01:29:25.173 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T01:29:25.173 INFO:teuthology.orchestra.run.smithi176.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T01:29:25.173 INFO:teuthology.orchestra.run.smithi176.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T01:29:25.174 INFO:teuthology.orchestra.run.smithi176.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T01:29:25.174 INFO:teuthology.orchestra.run.smithi176.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T01:29:25.174 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T01:29:25.174 INFO:teuthology.orchestra.run.smithi176.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T01:29:25.174 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T01:29:25.174 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.174 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.175 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.175 INFO:teuthology.orchestra.run.smithi176.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T01:29:25.175 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T01:29:25.175 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T01:29:25.175 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T01:29:25.176 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T01:29:25.176 INFO:teuthology.orchestra.run.smithi176.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T01:29:25.176 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T01:29:25.176 INFO:teuthology.orchestra.run.smithi176.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T01:29:25.176 INFO:teuthology.orchestra.run.smithi176.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T01:29:25.176 INFO:teuthology.orchestra.run.smithi176.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T01:29:25.176 INFO:teuthology.orchestra.run.smithi176.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T01:29:25.177 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T01:29:25.177 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T01:29:25.177 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T01:29:25.177 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T01:29:25.177 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T01:29:25.178 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T01:29:25.178 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T01:29:25.178 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T01:29:25.178 INFO:teuthology.orchestra.run.smithi176.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T01:29:25.178 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T01:29:25.178 INFO:teuthology.orchestra.run.smithi176.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T01:29:25.178 INFO:teuthology.orchestra.run.smithi176.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T01:29:25.179 INFO:teuthology.orchestra.run.smithi176.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T01:29:25.179 INFO:teuthology.orchestra.run.smithi176.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T01:29:25.179 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T01:29:25.179 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T01:29:25.179 INFO:teuthology.orchestra.run.smithi176.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T01:29:25.179 INFO:teuthology.orchestra.run.smithi176.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T01:29:25.179 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T01:29:25.180 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T01:29:25.180 INFO:teuthology.orchestra.run.smithi176.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T01:29:25.180 INFO:teuthology.orchestra.run.smithi176.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T01:29:25.180 INFO:teuthology.orchestra.run.smithi176.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T01:29:25.180 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T01:29:25.180 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T01:29:25.180 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T01:29:25.181 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T01:29:25.181 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T01:29:25.181 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T01:29:25.181 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.181 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.181 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T01:29:25.181 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T01:29:25.181 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T01:29:25.182 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.182 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T01:29:25.182 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T01:29:25.182 INFO:teuthology.orchestra.run.smithi176.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T01:29:25.182 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T01:29:25.182 INFO:teuthology.orchestra.run.smithi176.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T01:29:25.182 INFO:teuthology.orchestra.run.smithi176.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T01:29:25.182 INFO:teuthology.orchestra.run.smithi176.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T01:29:25.182 INFO:teuthology.orchestra.run.smithi176.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T01:29:25.182 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T01:29:25.182 INFO:teuthology.orchestra.run.smithi176.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T01:29:25.182 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T01:29:25.182 INFO:teuthology.orchestra.run.smithi176.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T01:29:25.183 INFO:teuthology.orchestra.run.smithi176.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T01:29:25.183 INFO:teuthology.orchestra.run.smithi176.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T01:29:25.183 INFO:teuthology.orchestra.run.smithi176.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T01:29:25.183 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.183 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.183 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:25.183 INFO:teuthology.orchestra.run.smithi176.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T01:29:25.183 INFO:teuthology.orchestra.run.smithi176.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T01:29:25.183 INFO:teuthology.orchestra.run.smithi176.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T01:29:25.183 INFO:teuthology.orchestra.run.smithi176.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T01:29:25.183 INFO:teuthology.orchestra.run.smithi176.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T01:29:25.183 INFO:teuthology.orchestra.run.smithi176.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T01:29:25.184 INFO:teuthology.orchestra.run.smithi176.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T01:29:25.184 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:29:25.184 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-09-16T01:29:25.574 DEBUG:teuthology.parallel:result is None 2024-09-16T01:29:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T01:29:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:29:26.342 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-09-16T01:29:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T01:29:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T01:29:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T01:29:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T01:29:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T01:29:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T01:29:26.344 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-16T01:29:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T01:29:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T01:29:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T01:29:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T01:29:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T01:29:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T01:29:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T01:29:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T01:29:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T01:29:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T01:29:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T01:29:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T01:29:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T01:29:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T01:29:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T01:29:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T01:29:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T01:29:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T01:29:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T01:29:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T01:29:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T01:29:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T01:29:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.354 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T01:29:26.354 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T01:29:26.354 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T01:29:26.354 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T01:29:26.354 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T01:29:26.354 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T01:29:26.355 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T01:29:26.355 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T01:29:26.355 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T01:29:26.355 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T01:29:26.355 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T01:29:26.356 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T01:29:26.356 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T01:29:26.356 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T01:29:26.356 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T01:29:26.356 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T01:29:26.356 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T01:29:26.357 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T01:29:26.357 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T01:29:26.357 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T01:29:26.357 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T01:29:26.357 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T01:29:26.358 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T01:29:26.358 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.358 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.358 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.358 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T01:29:26.358 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T01:29:26.359 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T01:29:26.359 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T01:29:26.359 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T01:29:26.359 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T01:29:26.359 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T01:29:26.360 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T01:29:26.360 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T01:29:26.360 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T01:29:26.360 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T01:29:26.360 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T01:29:26.360 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T01:29:26.361 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T01:29:26.361 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T01:29:26.361 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T01:29:26.361 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T01:29:26.361 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T01:29:26.362 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T01:29:26.362 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T01:29:26.362 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T01:29:26.362 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T01:29:26.362 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T01:29:26.363 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T01:29:26.363 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T01:29:26.363 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T01:29:26.363 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T01:29:26.363 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T01:29:26.364 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T01:29:26.364 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T01:29:26.364 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T01:29:26.364 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T01:29:26.364 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T01:29:26.364 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T01:29:26.364 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T01:29:26.364 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T01:29:26.364 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T01:29:26.364 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T01:29:26.365 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T01:29:26.365 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T01:29:26.365 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.365 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.365 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T01:29:26.365 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T01:29:26.365 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T01:29:26.365 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.365 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T01:29:26.365 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T01:29:26.365 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T01:29:26.365 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T01:29:26.366 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T01:29:26.366 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T01:29:26.366 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T01:29:26.366 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T01:29:26.366 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T01:29:26.366 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T01:29:26.366 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T01:29:26.366 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T01:29:26.366 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T01:29:26.366 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T01:29:26.366 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T01:29:26.367 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.367 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.367 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T01:29:26.367 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T01:29:26.367 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T01:29:26.367 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T01:29:26.367 INFO:teuthology.orchestra.run.smithi164.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T01:29:26.367 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T01:29:26.367 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T01:29:26.368 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T01:29:26.368 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:29:26.368 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-16T01:29:26.756 DEBUG:teuthology.parallel:result is None 2024-09-16T01:29:26.757 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:29:26.905 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T01:29:26.931 INFO:teuthology.orchestra.run.smithi139.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T01:29:26.932 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T01:29:26.932 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T01:29:26.934 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:29:27.077 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T01:29:27.100 INFO:teuthology.orchestra.run.smithi164.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T01:29:27.100 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T01:29:27.100 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T01:29:27.102 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:29:27.253 DEBUG:teuthology.orchestra.run.smithi176:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T01:29:27.279 INFO:teuthology.orchestra.run.smithi176.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T01:29:27.279 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T01:29:27.280 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T01:29:27.281 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T01:29:27.282 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:29:27.282 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T01:29:27.314 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T01:29:27.314 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T01:29:27.342 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-16T01:29:27.343 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T01:29:27.374 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T01:29:27.374 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:29:27.374 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T01:29:27.404 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T01:29:27.474 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T01:29:27.474 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T01:29:27.499 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T01:29:27.565 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-16T01:29:27.566 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T01:29:27.596 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T01:29:27.665 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T01:29:27.666 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:29:27.666 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T01:29:27.695 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T01:29:27.763 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T01:29:27.764 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T01:29:27.790 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T01:29:27.856 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-16T01:29:27.856 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T01:29:27.886 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T01:29:27.954 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T01:29:27.954 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:29:27.955 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T01:29:27.984 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T01:29:28.053 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T01:29:28.054 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T01:29:28.079 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T01:29:28.145 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-16T01:29:28.145 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T01:29:28.175 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T01:29:28.244 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T01:29:28.335 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-16T01:29:28.335 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:29:28.336 INFO:tasks.cephadm:Cluster fsid is 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:29:28.336 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T01:29:28.336 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.139', 'mon.b': '172.21.15.164', 'mon.c': '172.21.15.176'} 2024-09-16T01:29:28.336 INFO:tasks.cephadm:First mon is mon.a on smithi139 2024-09-16T01:29:28.336 INFO:tasks.cephadm:First mgr is x 2024-09-16T01:29:28.337 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T01:29:28.337 DEBUG:teuthology.orchestra.run.smithi139:> sudo hostname $(hostname -s) 2024-09-16T01:29:28.367 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname $(hostname -s) 2024-09-16T01:29:28.393 DEBUG:teuthology.orchestra.run.smithi176:> sudo hostname $(hostname -s) 2024-09-16T01:29:28.422 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-16T01:29:28.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:29:28.578 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56081/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-326-g8921c8ef', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-326.g8921c8ef'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-15 16:06:42.385367', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-16T01:29:28.725 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref squid, sha1 8921c8ef947807b717db041d85f0761b7914d5fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:29:28.727 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-16T01:29:28.727 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-16T01:29:28.727 DEBUG:teuthology.orchestra.run.smithi139:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T01:29:29.125 INFO:teuthology.orchestra.run.smithi139.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 01:29 /home/ubuntu/cephtest/cephadm 2024-09-16T01:29:29.125 DEBUG:teuthology.orchestra.run.smithi164:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T01:29:29.521 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 01:29 /home/ubuntu/cephtest/cephadm 2024-09-16T01:29:29.521 DEBUG:teuthology.orchestra.run.smithi176:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T01:29:29.902 INFO:teuthology.orchestra.run.smithi176.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 01:29 /home/ubuntu/cephtest/cephadm 2024-09-16T01:29:29.903 DEBUG:teuthology.orchestra.run.smithi139:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T01:29:29.924 DEBUG:teuthology.orchestra.run.smithi164:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T01:29:29.944 DEBUG:teuthology.orchestra.run.smithi176:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T01:29:29.983 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc on all hosts... 2024-09-16T01:29:29.983 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T01:29:29.986 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T01:29:29.989 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T01:29:31.463 INFO:teuthology.orchestra.run.smithi164.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:29:31.573 INFO:teuthology.orchestra.run.smithi139.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:29:31.575 INFO:teuthology.orchestra.run.smithi176.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:29:58.210 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:29:58.211 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T01:29:58.211 INFO:teuthology.orchestra.run.smithi139.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T01:29:58.211 INFO:teuthology.orchestra.run.smithi139.stdout: "repo_digests": [ 2024-09-16T01:29:58.211 INFO:teuthology.orchestra.run.smithi139.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T01:29:58.211 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T01:29:58.211 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:30:02.950 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-16T01:30:02.950 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T01:30:02.950 INFO:teuthology.orchestra.run.smithi176.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T01:30:02.950 INFO:teuthology.orchestra.run.smithi176.stdout: "repo_digests": [ 2024-09-16T01:30:02.950 INFO:teuthology.orchestra.run.smithi176.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T01:30:02.950 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-16T01:30:02.951 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-16T01:30:03.657 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T01:30:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T01:30:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T01:30:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: "repo_digests": [ 2024-09-16T01:30:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T01:30:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T01:30:03.657 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T01:30:03.684 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph 2024-09-16T01:30:03.719 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph 2024-09-16T01:30:03.747 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /etc/ceph 2024-09-16T01:30:03.782 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 777 /etc/ceph 2024-09-16T01:30:03.810 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /etc/ceph 2024-09-16T01:30:03.835 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 777 /etc/ceph 2024-09-16T01:30:03.864 INFO:tasks.cephadm:Writing seed config... 2024-09-16T01:30:03.865 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-16T01:30:03.865 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-16T01:30:03.866 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-16T01:30:03.866 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-16T01:30:03.866 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-16T01:30:03.866 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-16T01:30:03.866 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-16T01:30:03.866 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-16T01:30:03.866 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-16T01:30:03.866 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-16T01:30:03.866 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-16T01:30:03.866 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-16T01:30:03.866 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-16T01:30:03.866 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-16T01:30:03.866 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T01:30:03.867 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-16T01:30:03.868 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-16T01:30:03.868 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-16T01:30:03.868 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-16T01:30:03.868 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-16T01:30:03.868 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-16T01:30:03.868 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-16T01:30:03.868 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-16T01:30:03.868 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T01:30:03.868 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T01:30:03.868 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-16T01:30:03.868 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-16T01:30:03.868 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-16T01:30:03.868 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-16T01:30:03.869 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-16T01:30:03.869 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-16T01:30:03.869 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-16T01:30:03.869 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-16T01:30:03.869 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-16T01:30:03.870 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:30:03.870 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T01:30:03.886 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1d90dd3a-73cb-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-16T01:30:03.886 DEBUG:teuthology.orchestra.run.smithi139:mon.a> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.a.service 2024-09-16T01:30:03.929 DEBUG:teuthology.orchestra.run.smithi139:mgr.x> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.x.service 2024-09-16T01:30:03.971 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T01:30:03.971 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc -v bootstrap --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.139 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T01:30:04.173 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T01:30:04.173 INFO:teuthology.orchestra.run.smithi139.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc', '-v', 'bootstrap', '--fsid', '1d90dd3a-73cb-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.139', '--skip-admin-label'] 2024-09-16T01:30:04.174 INFO:teuthology.orchestra.run.smithi139.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T01:30:04.174 INFO:teuthology.orchestra.run.smithi139.stdout:Verifying podman|docker is present... 2024-09-16T01:30:04.198 INFO:teuthology.orchestra.run.smithi139.stdout:/bin/podman: stdout 5.2.2 2024-09-16T01:30:04.198 INFO:teuthology.orchestra.run.smithi139.stdout:Verifying lvm2 is present... 2024-09-16T01:30:04.198 INFO:teuthology.orchestra.run.smithi139.stdout:Verifying time synchronization is in place... 2024-09-16T01:30:04.205 INFO:teuthology.orchestra.run.smithi139.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T01:30:04.206 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T01:30:04.212 INFO:teuthology.orchestra.run.smithi139.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T01:30:04.212 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stdout inactive 2024-09-16T01:30:04.218 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stdout enabled 2024-09-16T01:30:04.225 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stdout active 2024-09-16T01:30:04.225 INFO:teuthology.orchestra.run.smithi139.stdout:Unit chronyd.service is enabled and running 2024-09-16T01:30:04.225 INFO:teuthology.orchestra.run.smithi139.stdout:Repeating the final host check... 2024-09-16T01:30:04.257 INFO:teuthology.orchestra.run.smithi139.stdout:/bin/podman: stdout 5.2.2 2024-09-16T01:30:04.257 INFO:teuthology.orchestra.run.smithi139.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T01:30:04.257 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl is present 2024-09-16T01:30:04.257 INFO:teuthology.orchestra.run.smithi139.stdout:lvcreate is present 2024-09-16T01:30:04.263 INFO:teuthology.orchestra.run.smithi139.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T01:30:04.263 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T01:30:04.269 INFO:teuthology.orchestra.run.smithi139.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T01:30:04.269 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stdout inactive 2024-09-16T01:30:04.276 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stdout enabled 2024-09-16T01:30:04.282 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stdout active 2024-09-16T01:30:04.283 INFO:teuthology.orchestra.run.smithi139.stdout:Unit chronyd.service is enabled and running 2024-09-16T01:30:04.283 INFO:teuthology.orchestra.run.smithi139.stdout:Host looks OK 2024-09-16T01:30:04.283 INFO:teuthology.orchestra.run.smithi139.stdout:Cluster fsid: 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:30:04.283 INFO:teuthology.orchestra.run.smithi139.stdout:Acquiring lock 139911695254528 on /run/cephadm/1d90dd3a-73cb-11ef-bceb-c7b262605968.lock 2024-09-16T01:30:04.283 INFO:teuthology.orchestra.run.smithi139.stdout:Lock 139911695254528 acquired on /run/cephadm/1d90dd3a-73cb-11ef-bceb-c7b262605968.lock 2024-09-16T01:30:04.283 INFO:teuthology.orchestra.run.smithi139.stdout:Verifying IP 172.21.15.139 port 3300 ... 2024-09-16T01:30:04.284 INFO:teuthology.orchestra.run.smithi139.stdout:Verifying IP 172.21.15.139 port 6789 ... 2024-09-16T01:30:04.284 INFO:teuthology.orchestra.run.smithi139.stdout:Base mon IP(s) is [172.21.15.139:3300, 172.21.15.139:6789], mon addrv is [v2:172.21.15.139:3300,v1:172.21.15.139:6789] 2024-09-16T01:30:04.287 INFO:teuthology.orchestra.run.smithi139.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.139 metric 100 2024-09-16T01:30:04.287 INFO:teuthology.orchestra.run.smithi139.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.139 metric 100 2024-09-16T01:30:04.290 INFO:teuthology.orchestra.run.smithi139.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T01:30:04.290 INFO:teuthology.orchestra.run.smithi139.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-16T01:30:04.290 INFO:teuthology.orchestra.run.smithi139.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-16T01:30:04.294 INFO:teuthology.orchestra.run.smithi139.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T01:30:04.295 INFO:teuthology.orchestra.run.smithi139.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T01:30:04.295 INFO:teuthology.orchestra.run.smithi139.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T01:30:04.295 INFO:teuthology.orchestra.run.smithi139.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-16T01:30:04.295 INFO:teuthology.orchestra.run.smithi139.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7f59/64 scope link noprefixroute 2024-09-16T01:30:04.295 INFO:teuthology.orchestra.run.smithi139.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T01:30:04.295 INFO:teuthology.orchestra.run.smithi139.stdout:Mon IP `172.21.15.139` is in CIDR network `172.21.0.0/20` 2024-09-16T01:30:04.295 INFO:teuthology.orchestra.run.smithi139.stdout:Mon IP `172.21.15.139` is in CIDR network `172.21.0.0/20` 2024-09-16T01:30:04.296 INFO:teuthology.orchestra.run.smithi139.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T01:30:04.296 INFO:teuthology.orchestra.run.smithi139.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T01:30:04.296 INFO:teuthology.orchestra.run.smithi139.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:30:04.754 INFO:teuthology.orchestra.run.smithi139.stdout:/bin/podman: stdout f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-16T01:30:04.755 INFO:teuthology.orchestra.run.smithi139.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T01:30:04.755 INFO:teuthology.orchestra.run.smithi139.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T01:30:04.755 INFO:teuthology.orchestra.run.smithi139.stdout:/bin/podman: stderr Copying blob sha256:c0574da0c3174f3dc49daa274908f97d06456bd6a3cfab87e94022c1369a83a6 2024-09-16T01:30:04.755 INFO:teuthology.orchestra.run.smithi139.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T01:30:04.755 INFO:teuthology.orchestra.run.smithi139.stdout:/bin/podman: stderr Copying config sha256:f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-16T01:30:04.755 INFO:teuthology.orchestra.run.smithi139.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T01:30:05.667 INFO:teuthology.orchestra.run.smithi139.stdout:ceph: stdout ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-16T01:30:05.668 INFO:teuthology.orchestra.run.smithi139.stdout:Ceph version: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-16T01:30:05.668 INFO:teuthology.orchestra.run.smithi139.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T01:30:06.982 INFO:teuthology.orchestra.run.smithi139.stdout:stat: stdout 167 167 2024-09-16T01:30:06.982 INFO:teuthology.orchestra.run.smithi139.stdout:Creating initial keys... 2024-09-16T01:30:07.912 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph-authtool: stdout AQAfiudmjJgyExAArJ1JVGUVczCywmTrV4D5Fg== 2024-09-16T01:30:08.876 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph-authtool: stdout AQAgiudmNXyVEBAADsI0CxHiWR6X2QtkcNm4dw== 2024-09-16T01:30:09.774 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph-authtool: stdout AQAhiudm0ObsCBAAruGZh6VNPotKWyzyeqUItg== 2024-09-16T01:30:09.775 INFO:teuthology.orchestra.run.smithi139.stdout:Creating initial monmap... 2024-09-16T01:30:10.712 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T01:30:10.713 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-16T01:30:10.713 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:30:10.713 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T01:30:10.713 INFO:teuthology.orchestra.run.smithi139.stdout:monmaptool for a [v2:172.21.15.139:3300,v1:172.21.15.139:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T01:30:10.713 INFO:teuthology.orchestra.run.smithi139.stdout:setting min_mon_release = quincy 2024-09-16T01:30:10.713 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/monmaptool: set fsid to 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:30:10.713 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T01:30:10.713 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:30:10.713 INFO:teuthology.orchestra.run.smithi139.stdout:Creating mon... 2024-09-16T01:30:12.111 INFO:teuthology.orchestra.run.smithi139.stdout:create mon.a on 2024-09-16T01:30:12.611 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-16T01:30:12.847 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T01:30:13.362 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968.target. 2024-09-16T01:30:13.363 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968.target. 2024-09-16T01:30:14.240 INFO:teuthology.orchestra.run.smithi139.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.a 2024-09-16T01:30:14.240 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-16T01:30:14.654 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968.target.wants/ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@.service. 2024-09-16T01:30:15.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:14 smithi139 podman[30787]: 2024-09-16 01:30:14.789248189 +0000 UTC m=+0.015912437 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:30:16.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:15 smithi139 podman[30787]: 2024-09-16 01:30:15.85953604 +0000 UTC m=+1.086200330 container create 674bf0b163433811b280541b71368859d714ac5aafd495a280ef08f1cc9ca0cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-16T01:30:18.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:17 smithi139 podman[30787]: 2024-09-16 01:30:17.723518126 +0000 UTC m=+2.950182375 container init 674bf0b163433811b280541b71368859d714ac5aafd495a280ef08f1cc9ca0cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T01:30:18.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:17 smithi139 podman[30787]: 2024-09-16 01:30:17.727563846 +0000 UTC m=+2.954228095 container start 674bf0b163433811b280541b71368859d714ac5aafd495a280ef08f1cc9ca0cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-16T01:30:18.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:17 smithi139 ceph-mon[30807]: mkfs 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:30:18.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:17 smithi139 ceph-mon[30807]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T01:30:18.597 INFO:teuthology.orchestra.run.smithi139.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T01:30:18.597 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stdout disabled 2024-09-16T01:30:18.606 INFO:teuthology.orchestra.run.smithi139.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T01:30:18.606 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stdout inactive 2024-09-16T01:30:18.606 INFO:teuthology.orchestra.run.smithi139.stdout:firewalld.service is not enabled 2024-09-16T01:30:18.606 INFO:teuthology.orchestra.run.smithi139.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T01:30:18.606 INFO:teuthology.orchestra.run.smithi139.stdout:Waiting for mon to start... 2024-09-16T01:30:18.606 INFO:teuthology.orchestra.run.smithi139.stdout:Waiting for mon... 2024-09-16T01:30:18.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:18 smithi139 bash[30787]: 674bf0b163433811b280541b71368859d714ac5aafd495a280ef08f1cc9ca0cb 2024-09-16T01:30:18.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:18 smithi139 systemd[1]: Started Ceph mon.a for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:30:18.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:18 smithi139 ceph-mon[30807]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T01:30:18.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:18 smithi139 ceph-mon[30807]: monmap epoch 1 2024-09-16T01:30:18.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:18 smithi139 ceph-mon[30807]: fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:30:18.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:18 smithi139 ceph-mon[30807]: last_changed 2024-09-16T01:30:10.119520+0000 2024-09-16T01:30:18.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:18 smithi139 ceph-mon[30807]: created 2024-09-16T01:30:10.119520+0000 2024-09-16T01:30:18.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:18 smithi139 ceph-mon[30807]: min_mon_release 19 (squid) 2024-09-16T01:30:18.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:18 smithi139 ceph-mon[30807]: election_strategy: 1 2024-09-16T01:30:18.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:18 smithi139 ceph-mon[30807]: 0: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.a 2024-09-16T01:30:18.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:18 smithi139 ceph-mon[30807]: fsmap 2024-09-16T01:30:18.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:18 smithi139 ceph-mon[30807]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T01:30:18.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:18 smithi139 ceph-mon[30807]: mgrmap e1: no daemons active 2024-09-16T01:30:21.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:21 smithi139 ceph-mon[30807]: from='client.? 172.21.15.139:0/271212511' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-16T01:30:21.768 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T01:30:21.768 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout id: 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:30:21.768 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T01:30:21.768 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 2024-09-16T01:30:21.768 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout services: 2024-09-16T01:30:21.768 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 3s) 2024-09-16T01:30:21.768 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T01:30:21.768 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T01:30:21.768 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 2024-09-16T01:30:21.769 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout data: 2024-09-16T01:30:21.769 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T01:30:21.769 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T01:30:21.769 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T01:30:21.769 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T01:30:21.769 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 2024-09-16T01:30:21.769 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.059+0000 7ff3569d6640 1 Processor -- start 2024-09-16T01:30:21.769 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.059+0000 7ff3569d6640 1 -- start start 2024-09-16T01:30:21.769 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.059+0000 7ff3569d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3501065f0 0x7ff3501069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:21.769 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.059+0000 7ff3569d6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff350106fc0 con 0x7ff3501065f0 2024-09-16T01:30:21.769 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.059+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3501065f0 0x7ff3501069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:21.769 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.060+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3501065f0 0x7ff3501069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48518/0 (socket says 172.21.15.139:48518) 2024-09-16T01:30:21.770 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.060+0000 7ff34ffff640 1 -- 172.21.15.139:0/2661595876 learned_addr learned my addr 172.21.15.139:0/2661595876 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:21.770 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.060+0000 7ff34ffff640 1 -- 172.21.15.139:0/2661595876 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3501077f0 con 0x7ff3501065f0 2024-09-16T01:30:21.770 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.062+0000 7ff34ffff640 1 --2- 172.21.15.139:0/2661595876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3501065f0 0x7ff3501069f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff340009820 tx=0x7ff34002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba750e4bcc29d26b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:21.770 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.062+0000 7ff34f7fe640 1 -- 172.21.15.139:0/2661595876 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff340037070 con 0x7ff3501065f0 2024-09-16T01:30:21.770 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.062+0000 7ff34f7fe640 1 -- 172.21.15.139:0/2661595876 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff340032440 con 0x7ff3501065f0 2024-09-16T01:30:21.770 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.062+0000 7ff34f7fe640 1 -- 172.21.15.139:0/2661595876 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff340032740 con 0x7ff3501065f0 2024-09-16T01:30:21.770 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.063+0000 7ff3569d6640 1 -- 172.21.15.139:0/2661595876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3501065f0 msgr2=0x7ff3501069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:21.770 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.063+0000 7ff3569d6640 1 --2- 172.21.15.139:0/2661595876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3501065f0 0x7ff3501069f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff340009820 tx=0x7ff34002a270 comp rx=0 tx=0).stop 2024-09-16T01:30:21.770 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.063+0000 7ff3569d6640 1 -- 172.21.15.139:0/2661595876 shutdown_connections 2024-09-16T01:30:21.770 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.063+0000 7ff3569d6640 1 --2- 172.21.15.139:0/2661595876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3501065f0 0x7ff3501069f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:21.770 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.063+0000 7ff3569d6640 1 -- 172.21.15.139:0/2661595876 >> 172.21.15.139:0/2661595876 conn(0x7ff350101e20 msgr2=0x7ff350104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:21.770 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.063+0000 7ff3569d6640 1 -- 172.21.15.139:0/2661595876 shutdown_connections 2024-09-16T01:30:21.771 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.063+0000 7ff3569d6640 1 -- 172.21.15.139:0/2661595876 wait complete. 2024-09-16T01:30:21.771 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.064+0000 7ff3569d6640 1 Processor -- start 2024-09-16T01:30:21.771 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.064+0000 7ff3569d6640 1 -- start start 2024-09-16T01:30:21.771 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.064+0000 7ff3569d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3501065f0 0x7ff350199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:21.771 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.064+0000 7ff3569d6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff350107d90 con 0x7ff3501065f0 2024-09-16T01:30:21.771 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.064+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3501065f0 0x7ff350199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:21.771 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.064+0000 7ff34ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3501065f0 0x7ff350199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48526/0 (socket says 172.21.15.139:48526) 2024-09-16T01:30:21.771 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.064+0000 7ff34ffff640 1 -- 172.21.15.139:0/271212511 learned_addr learned my addr 172.21.15.139:0/271212511 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:21.771 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.065+0000 7ff34ffff640 1 -- 172.21.15.139:0/271212511 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35019a2a0 con 0x7ff3501065f0 2024-09-16T01:30:21.771 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.065+0000 7ff34ffff640 1 --2- 172.21.15.139:0/271212511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3501065f0 0x7ff350199d60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff3400090b0 tx=0x7ff3400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:21.771 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.065+0000 7ff34dffb640 1 -- 172.21.15.139:0/271212511 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff340041070 con 0x7ff3501065f0 2024-09-16T01:30:21.771 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.065+0000 7ff34dffb640 1 -- 172.21.15.139:0/271212511 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff34003c7a0 con 0x7ff3501065f0 2024-09-16T01:30:21.771 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.065+0000 7ff3569d6640 1 -- 172.21.15.139:0/271212511 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35019a530 con 0x7ff3501065f0 2024-09-16T01:30:21.771 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.065+0000 7ff34dffb640 1 -- 172.21.15.139:0/271212511 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff340037040 con 0x7ff3501065f0 2024-09-16T01:30:21.772 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.065+0000 7ff3569d6640 1 -- 172.21.15.139:0/271212511 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff35019d220 con 0x7ff3501065f0 2024-09-16T01:30:21.772 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.065+0000 7ff34dffb640 1 -- 172.21.15.139:0/271212511 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff340044070 con 0x7ff3501065f0 2024-09-16T01:30:21.772 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.065+0000 7ff34dffb640 1 -- 172.21.15.139:0/271212511 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff34003d3f0 con 0x7ff3501065f0 2024-09-16T01:30:21.772 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.066+0000 7ff3569d6640 1 -- 172.21.15.139:0/271212511 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3501083e0 con 0x7ff3501065f0 2024-09-16T01:30:21.772 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.068+0000 7ff34dffb640 1 -- 172.21.15.139:0/271212511 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff340046450 con 0x7ff3501065f0 2024-09-16T01:30:21.772 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.127+0000 7ff3569d6640 1 -- 172.21.15.139:0/271212511 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7ff35019d790 con 0x7ff3501065f0 2024-09-16T01:30:21.772 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.127+0000 7ff34dffb640 1 -- 172.21.15.139:0/271212511 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+310 (secure 0 0 0) 0x7ff340046630 con 0x7ff3501065f0 2024-09-16T01:30:21.772 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.128+0000 7ff3569d6640 1 -- 172.21.15.139:0/271212511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3501065f0 msgr2=0x7ff350199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:21.772 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.128+0000 7ff3569d6640 1 --2- 172.21.15.139:0/271212511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3501065f0 0x7ff350199d60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff3400090b0 tx=0x7ff3400046c0 comp rx=0 tx=0).stop 2024-09-16T01:30:21.772 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.128+0000 7ff3569d6640 1 -- 172.21.15.139:0/271212511 shutdown_connections 2024-09-16T01:30:21.772 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.128+0000 7ff3569d6640 1 --2- 172.21.15.139:0/271212511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3501065f0 0x7ff350199d60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:21.772 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.128+0000 7ff3569d6640 1 -- 172.21.15.139:0/271212511 >> 172.21.15.139:0/271212511 conn(0x7ff350101e20 msgr2=0x7ff35018e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:21.772 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.128+0000 7ff3569d6640 1 -- 172.21.15.139:0/271212511 shutdown_connections 2024-09-16T01:30:21.773 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:21.128+0000 7ff3569d6640 1 -- 172.21.15.139:0/271212511 wait complete. 2024-09-16T01:30:21.773 INFO:teuthology.orchestra.run.smithi139.stdout:mon is available 2024-09-16T01:30:21.773 INFO:teuthology.orchestra.run.smithi139.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T01:30:22.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:22 smithi139 ceph-mon[30807]: from='client.? 172.21.15.139:0/108811175' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T01:30:22.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:22 smithi139 ceph-mon[30807]: from='client.? 172.21.15.139:0/108811175' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T01:30:22.865 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 2024-09-16T01:30:22.865 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T01:30:22.865 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout fsid = 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:30:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.139:3300,v1:172.21.15.139:6789] 2024-09-16T01:30:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T01:30:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T01:30:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T01:30:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T01:30:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 2024-09-16T01:30:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T01:30:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T01:30:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 2024-09-16T01:30:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T01:30:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T01:30:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T01:30:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T01:30:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T01:30:22.867 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T01:30:22.867 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.223+0000 7f1723be3640 1 Processor -- start 2024-09-16T01:30:22.867 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.223+0000 7f1723be3640 1 -- start start 2024-09-16T01:30:22.867 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.223+0000 7f1723be3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f171c1076f0 0x7f171c107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:22.867 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.223+0000 7f1723be3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f171c1080c0 con 0x7f171c1076f0 2024-09-16T01:30:22.867 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.223+0000 7f172197f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f171c1076f0 0x7f171c107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:22.867 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.223+0000 7f172197f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f171c1076f0 0x7f171c107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48530/0 (socket says 172.21.15.139:48530) 2024-09-16T01:30:22.867 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.223+0000 7f172197f640 1 -- 172.21.15.139:0/1928990245 learned_addr learned my addr 172.21.15.139:0/1928990245 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:22.867 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.223+0000 7f172197f640 1 -- 172.21.15.139:0/1928990245 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c1088f0 con 0x7f171c1076f0 2024-09-16T01:30:22.867 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.224+0000 7f172197f640 1 --2- 172.21.15.139:0/1928990245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f171c1076f0 0x7f171c107af0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1718009820 tx=0x7f171802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58200e1fe07fd63e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:22.867 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.224+0000 7f172097d640 1 -- 172.21.15.139:0/1928990245 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1718037070 con 0x7f171c1076f0 2024-09-16T01:30:22.867 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.224+0000 7f172097d640 1 -- 172.21.15.139:0/1928990245 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1718032440 con 0x7f171c1076f0 2024-09-16T01:30:22.867 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.224+0000 7f172097d640 1 -- 172.21.15.139:0/1928990245 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1718032740 con 0x7f171c1076f0 2024-09-16T01:30:22.868 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.225+0000 7f1723be3640 1 -- 172.21.15.139:0/1928990245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f171c1076f0 msgr2=0x7f171c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:22.868 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.225+0000 7f1723be3640 1 --2- 172.21.15.139:0/1928990245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f171c1076f0 0x7f171c107af0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1718009820 tx=0x7f171802a270 comp rx=0 tx=0).stop 2024-09-16T01:30:22.868 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.225+0000 7f1723be3640 1 -- 172.21.15.139:0/1928990245 shutdown_connections 2024-09-16T01:30:22.868 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.225+0000 7f1723be3640 1 --2- 172.21.15.139:0/1928990245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f171c1076f0 0x7f171c107af0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:22.868 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.225+0000 7f1723be3640 1 -- 172.21.15.139:0/1928990245 >> 172.21.15.139:0/1928990245 conn(0x7f171c102ec0 msgr2=0x7f171c1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:22.868 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.225+0000 7f1723be3640 1 -- 172.21.15.139:0/1928990245 shutdown_connections 2024-09-16T01:30:22.868 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.225+0000 7f1723be3640 1 -- 172.21.15.139:0/1928990245 wait complete. 2024-09-16T01:30:22.868 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.226+0000 7f1723be3640 1 Processor -- start 2024-09-16T01:30:22.868 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.226+0000 7f1723be3640 1 -- start start 2024-09-16T01:30:22.868 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.226+0000 7f1723be3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f171c1076f0 0x7f171c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:22.868 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.226+0000 7f1723be3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f171c108e90 con 0x7f171c1076f0 2024-09-16T01:30:22.868 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.226+0000 7f172197f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f171c1076f0 0x7f171c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:22.869 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.226+0000 7f172197f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f171c1076f0 0x7f171c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48538/0 (socket says 172.21.15.139:48538) 2024-09-16T01:30:22.869 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.226+0000 7f172197f640 1 -- 172.21.15.139:0/108811175 learned_addr learned my addr 172.21.15.139:0/108811175 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:22.869 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.226+0000 7f172197f640 1 -- 172.21.15.139:0/108811175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c07e0b0 con 0x7f171c1076f0 2024-09-16T01:30:22.869 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.226+0000 7f172197f640 1 --2- 172.21.15.139:0/108811175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f171c1076f0 0x7f171c07db70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f17180090b0 tx=0x7f17180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:22.869 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.227+0000 7f1712ffd640 1 -- 172.21.15.139:0/108811175 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1718041070 con 0x7f171c1076f0 2024-09-16T01:30:22.869 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.227+0000 7f1712ffd640 1 -- 172.21.15.139:0/108811175 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f171803c7a0 con 0x7f171c1076f0 2024-09-16T01:30:22.869 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.227+0000 7f1723be3640 1 -- 172.21.15.139:0/108811175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171c07c2a0 con 0x7f171c1076f0 2024-09-16T01:30:22.869 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.227+0000 7f1723be3640 1 -- 172.21.15.139:0/108811175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f171c07c7e0 con 0x7f171c1076f0 2024-09-16T01:30:22.869 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.227+0000 7f1712ffd640 1 -- 172.21.15.139:0/108811175 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1718037040 con 0x7f171c1076f0 2024-09-16T01:30:22.869 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.227+0000 7f1712ffd640 1 -- 172.21.15.139:0/108811175 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1718044070 con 0x7f171c1076f0 2024-09-16T01:30:22.869 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.227+0000 7f1712ffd640 1 -- 172.21.15.139:0/108811175 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f171803d3f0 con 0x7f171c1076f0 2024-09-16T01:30:22.869 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.228+0000 7f1723be3640 1 -- 172.21.15.139:0/108811175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17000052c0 con 0x7f171c1076f0 2024-09-16T01:30:22.869 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.230+0000 7f1712ffd640 1 -- 172.21.15.139:0/108811175 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1718046450 con 0x7f171c1076f0 2024-09-16T01:30:22.870 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.286+0000 7f1723be3640 1 -- 172.21.15.139:0/108811175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f1700002420 con 0x7f171c1076f0 2024-09-16T01:30:22.870 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.290+0000 7f1712ffd640 1 -- 172.21.15.139:0/108811175 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f1718046630 con 0x7f171c1076f0 2024-09-16T01:30:22.870 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.291+0000 7f1712ffd640 1 -- 172.21.15.139:0/108811175 <== mon.0 v2:172.21.15.139:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1718056070 con 0x7f171c1076f0 2024-09-16T01:30:22.870 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.292+0000 7f1723be3640 1 -- 172.21.15.139:0/108811175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f171c1076f0 msgr2=0x7f171c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:22.870 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.292+0000 7f1723be3640 1 --2- 172.21.15.139:0/108811175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f171c1076f0 0x7f171c07db70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f17180090b0 tx=0x7f17180046c0 comp rx=0 tx=0).stop 2024-09-16T01:30:22.870 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.292+0000 7f1723be3640 1 -- 172.21.15.139:0/108811175 shutdown_connections 2024-09-16T01:30:22.870 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.292+0000 7f1723be3640 1 --2- 172.21.15.139:0/108811175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f171c1076f0 0x7f171c07db70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:22.870 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.292+0000 7f1723be3640 1 -- 172.21.15.139:0/108811175 >> 172.21.15.139:0/108811175 conn(0x7f171c102ec0 msgr2=0x7f171c18f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:22.870 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.292+0000 7f1723be3640 1 -- 172.21.15.139:0/108811175 shutdown_connections 2024-09-16T01:30:22.870 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:22.292+0000 7f1723be3640 1 -- 172.21.15.139:0/108811175 wait complete. 2024-09-16T01:30:22.870 INFO:teuthology.orchestra.run.smithi139.stdout:Generating new minimal ceph.conf... 2024-09-16T01:30:23.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:23 smithi139 ceph-mon[30807]: from='client.? 172.21.15.139:0/3432760166' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:30:23.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.234+0000 7f6456d29640 1 Processor -- start 2024-09-16T01:30:23.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.234+0000 7f6456d29640 1 -- start start 2024-09-16T01:30:23.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.234+0000 7f6456d29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64501076f0 0x7f6450107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:23.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.234+0000 7f6456d29640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f64501080c0 con 0x7f64501076f0 2024-09-16T01:30:23.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.235+0000 7f6454ac5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64501076f0 0x7f6450107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:23.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.235+0000 7f6454ac5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64501076f0 0x7f6450107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39732/0 (socket says 172.21.15.139:39732) 2024-09-16T01:30:23.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.235+0000 7f6454ac5640 1 -- 172.21.15.139:0/2002358736 learned_addr learned my addr 172.21.15.139:0/2002358736 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:23.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.235+0000 7f6454ac5640 1 -- 172.21.15.139:0/2002358736 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64501088f0 con 0x7f64501076f0 2024-09-16T01:30:23.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.236+0000 7f6454ac5640 1 --2- 172.21.15.139:0/2002358736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64501076f0 0x7f6450107af0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6444009820 tx=0x7f644402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f0f1d0e3659aa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:23.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.236+0000 7f644f7fe640 1 -- 172.21.15.139:0/2002358736 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6444037070 con 0x7f64501076f0 2024-09-16T01:30:23.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.236+0000 7f644f7fe640 1 -- 172.21.15.139:0/2002358736 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6444032900 con 0x7f64501076f0 2024-09-16T01:30:23.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.236+0000 7f644f7fe640 1 -- 172.21.15.139:0/2002358736 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6444030850 con 0x7f64501076f0 2024-09-16T01:30:23.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.236+0000 7f6456d29640 1 -- 172.21.15.139:0/2002358736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64501076f0 msgr2=0x7f6450107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:23.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.236+0000 7f6456d29640 1 --2- 172.21.15.139:0/2002358736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64501076f0 0x7f6450107af0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6444009820 tx=0x7f644402a270 comp rx=0 tx=0).stop 2024-09-16T01:30:23.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.236+0000 7f6456d29640 1 -- 172.21.15.139:0/2002358736 shutdown_connections 2024-09-16T01:30:23.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.236+0000 7f6456d29640 1 --2- 172.21.15.139:0/2002358736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64501076f0 0x7f6450107af0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:23.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.236+0000 7f6456d29640 1 -- 172.21.15.139:0/2002358736 >> 172.21.15.139:0/2002358736 conn(0x7f6450102ec0 msgr2=0x7f64501052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:23.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.237+0000 7f6456d29640 1 -- 172.21.15.139:0/2002358736 shutdown_connections 2024-09-16T01:30:23.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.237+0000 7f6456d29640 1 -- 172.21.15.139:0/2002358736 wait complete. 2024-09-16T01:30:23.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.237+0000 7f6456d29640 1 Processor -- start 2024-09-16T01:30:23.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.237+0000 7f6456d29640 1 -- start start 2024-09-16T01:30:23.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.237+0000 7f6456d29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64501076f0 0x7f645019b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:23.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.237+0000 7f6456d29640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f645010bca0 con 0x7f64501076f0 2024-09-16T01:30:23.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.237+0000 7f6454ac5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64501076f0 0x7f645019b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:23.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.237+0000 7f6454ac5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64501076f0 0x7f645019b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39742/0 (socket says 172.21.15.139:39742) 2024-09-16T01:30:23.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.237+0000 7f6454ac5640 1 -- 172.21.15.139:0/3432760166 learned_addr learned my addr 172.21.15.139:0/3432760166 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:23.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.238+0000 7f6454ac5640 1 -- 172.21.15.139:0/3432760166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645019bc20 con 0x7f64501076f0 2024-09-16T01:30:23.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.238+0000 7f6454ac5640 1 --2- 172.21.15.139:0/3432760166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64501076f0 0x7f645019b6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f64440097f0 tx=0x7f6444030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:23.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.238+0000 7f644dffb640 1 -- 172.21.15.139:0/3432760166 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6444037040 con 0x7f64501076f0 2024-09-16T01:30:23.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.238+0000 7f644dffb640 1 -- 172.21.15.139:0/3432760166 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6444040070 con 0x7f64501076f0 2024-09-16T01:30:23.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.238+0000 7f6456d29640 1 -- 172.21.15.139:0/3432760166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645019beb0 con 0x7f64501076f0 2024-09-16T01:30:23.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.238+0000 7f644dffb640 1 -- 172.21.15.139:0/3432760166 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f644403a390 con 0x7f64501076f0 2024-09-16T01:30:23.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.238+0000 7f6456d29640 1 -- 172.21.15.139:0/3432760166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645019eba0 con 0x7f64501076f0 2024-09-16T01:30:23.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.238+0000 7f644dffb640 1 -- 172.21.15.139:0/3432760166 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f644403a5e0 con 0x7f64501076f0 2024-09-16T01:30:23.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.238+0000 7f644dffb640 1 -- 172.21.15.139:0/3432760166 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6444045950 con 0x7f64501076f0 2024-09-16T01:30:23.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.239+0000 7f6456d29640 1 -- 172.21.15.139:0/3432760166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6450107b70 con 0x7f64501076f0 2024-09-16T01:30:23.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.241+0000 7f644dffb640 1 -- 172.21.15.139:0/3432760166 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6444045460 con 0x7f64501076f0 2024-09-16T01:30:23.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.299+0000 7f6456d29640 1 -- 172.21.15.139:0/3432760166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f645019ed70 con 0x7f64501076f0 2024-09-16T01:30:23.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.300+0000 7f644dffb640 1 -- 172.21.15.139:0/3432760166 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f644403b430 con 0x7f64501076f0 2024-09-16T01:30:23.907 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.300+0000 7f6456d29640 1 -- 172.21.15.139:0/3432760166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64501076f0 msgr2=0x7f645019b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:23.907 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.300+0000 7f6456d29640 1 --2- 172.21.15.139:0/3432760166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64501076f0 0x7f645019b6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f64440097f0 tx=0x7f6444030d80 comp rx=0 tx=0).stop 2024-09-16T01:30:23.907 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.300+0000 7f6456d29640 1 -- 172.21.15.139:0/3432760166 shutdown_connections 2024-09-16T01:30:23.907 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.300+0000 7f6456d29640 1 --2- 172.21.15.139:0/3432760166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64501076f0 0x7f645019b6e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:23.907 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.300+0000 7f6456d29640 1 -- 172.21.15.139:0/3432760166 >> 172.21.15.139:0/3432760166 conn(0x7f6450102ec0 msgr2=0x7f64501039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:23.907 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.301+0000 7f6456d29640 1 -- 172.21.15.139:0/3432760166 shutdown_connections 2024-09-16T01:30:23.907 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:23.301+0000 7f6456d29640 1 -- 172.21.15.139:0/3432760166 wait complete. 2024-09-16T01:30:23.907 INFO:teuthology.orchestra.run.smithi139.stdout:Restarting the monitor... 2024-09-16T01:30:24.209 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:23 smithi139 systemd[1]: Stopping Ceph mon.a for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:30:24.474 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a[30803]: 2024-09-16T01:30:24.207+0000 7f3c495cb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T01:30:24.475 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a[30803]: 2024-09-16T01:30:24.207+0000 7f3c495cb640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T01:30:24.791 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:24 smithi139 podman[31050]: 2024-09-16 01:30:24.474074587 +0000 UTC m=+0.393812793 container died 674bf0b163433811b280541b71368859d714ac5aafd495a280ef08f1cc9ca0cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T01:30:24.791 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:24 smithi139 podman[31050]: 2024-09-16 01:30:24.79111191 +0000 UTC m=+0.710850125 container cleanup 674bf0b163433811b280541b71368859d714ac5aafd495a280ef08f1cc9ca0cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T01:30:25.141 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:24 smithi139 bash[31050]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a 2024-09-16T01:30:25.141 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:24 smithi139 podman[31064]: 2024-09-16 01:30:24.950526432 +0000 UTC m=+0.477403913 container remove 674bf0b163433811b280541b71368859d714ac5aafd495a280ef08f1cc9ca0cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T01:30:25.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 systemd[1]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T01:30:25.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 systemd[1]: Stopped Ceph mon.a for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:30:25.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 systemd[1]: Starting Ceph mon.a for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:30:25.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 podman[31156]: 2024-09-16 01:30:25.274757297 +0000 UTC m=+0.015763071 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:30:25.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 podman[31156]: 2024-09-16 01:30:25.39277243 +0000 UTC m=+0.133778200 container create 9a9750ef87b6c31e7f3a3ff1954a7c7ff21f7c5016735b7b1e79246d0e55a621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-16T01:30:25.709 INFO:teuthology.orchestra.run.smithi139.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T01:30:25.831 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 podman[31156]: 2024-09-16 01:30:25.620082637 +0000 UTC m=+0.361088404 container init 9a9750ef87b6c31e7f3a3ff1954a7c7ff21f7c5016735b7b1e79246d0e55a621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T01:30:25.831 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 podman[31156]: 2024-09-16 01:30:25.623219607 +0000 UTC m=+0.364225375 container start 9a9750ef87b6c31e7f3a3ff1954a7c7ff21f7c5016735b7b1e79246d0e55a621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T01:30:25.831 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T01:30:25.832 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-16T01:30:25.832 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: pidfile_write: ignore empty --pid-file 2024-09-16T01:30:25.832 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: load: jerasure load: lrc 2024-09-16T01:30:25.832 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: RocksDB version: 7.9.2 2024-09-16T01:30:25.832 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Git sha 0 2024-09-16T01:30:25.832 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-16T01:30:25.832 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: DB SUMMARY 2024-09-16T01:30:25.832 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: DB Session ID: YHVBFXCLG7A7DYRH9GZ0 2024-09-16T01:30:25.832 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: CURRENT file: CURRENT 2024-09-16T01:30:25.832 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T01:30:25.832 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T01:30:25.832 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T01:30:25.833 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106115 ; 2024-09-16T01:30:25.833 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.error_if_exists: 0 2024-09-16T01:30:25.833 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.create_if_missing: 0 2024-09-16T01:30:25.833 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.paranoid_checks: 1 2024-09-16T01:30:25.833 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T01:30:25.833 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T01:30:25.833 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T01:30:25.833 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.env: 0x5633b789c940 2024-09-16T01:30:25.833 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T01:30:25.833 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.info_log: 0x5633b8eef6e0 2024-09-16T01:30:25.833 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T01:30:25.833 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.statistics: (nil) 2024-09-16T01:30:25.833 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.use_fsync: 0 2024-09-16T01:30:25.834 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_log_file_size: 0 2024-09-16T01:30:25.834 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T01:30:25.834 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T01:30:25.834 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T01:30:25.834 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T01:30:25.834 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.allow_fallocate: 1 2024-09-16T01:30:25.834 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T01:30:25.834 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T01:30:25.834 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.use_direct_reads: 0 2024-09-16T01:30:25.834 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T01:30:25.834 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T01:30:25.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.db_log_dir: 2024-09-16T01:30:25.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.wal_dir: 2024-09-16T01:30:25.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T01:30:25.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T01:30:25.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T01:30:25.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T01:30:25.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T01:30:25.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T01:30:25.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T01:30:25.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T01:30:25.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.write_buffer_manager: 0x5633b8efedc0 2024-09-16T01:30:25.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T01:30:25.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T01:30:25.836 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T01:30:25.836 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T01:30:25.836 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T01:30:25.836 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T01:30:25.836 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T01:30:25.836 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T01:30:25.836 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.unordered_write: 0 2024-09-16T01:30:25.836 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T01:30:25.836 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T01:30:25.836 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T01:30:25.836 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T01:30:25.836 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.row_cache: None 2024-09-16T01:30:25.837 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.wal_filter: None 2024-09-16T01:30:25.837 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T01:30:25.838 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T01:30:25.838 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.two_write_queues: 0 2024-09-16T01:30:25.838 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T01:30:25.838 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.wal_compression: 0 2024-09-16T01:30:25.838 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.atomic_flush: 0 2024-09-16T01:30:25.838 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T01:30:25.838 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T01:30:25.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T01:30:25.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.log_readahead_size: 0 2024-09-16T01:30:25.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T01:30:25.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T01:30:25.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T01:30:25.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T01:30:25.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T01:30:25.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T01:30:25.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T01:30:25.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_background_jobs: 2 2024-09-16T01:30:25.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_background_compactions: -1 2024-09-16T01:30:25.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_subcompactions: 1 2024-09-16T01:30:25.840 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T01:30:25.840 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T01:30:25.840 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T01:30:25.840 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T01:30:25.840 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T01:30:25.840 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T01:30:25.840 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T01:30:25.840 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T01:30:25.840 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_open_files: -1 2024-09-16T01:30:25.840 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T01:30:25.840 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T01:30:25.840 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T01:30:25.841 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T01:30:25.841 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_background_flushes: -1 2024-09-16T01:30:25.841 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Compression algorithms supported: 2024-09-16T01:30:25.841 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: kZSTD supported: 0 2024-09-16T01:30:25.841 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: kXpressCompression supported: 0 2024-09-16T01:30:25.841 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: kBZip2Compression supported: 0 2024-09-16T01:30:25.841 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T01:30:25.841 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: kLZ4Compression supported: 1 2024-09-16T01:30:25.841 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: kZlibCompression supported: 1 2024-09-16T01:30:25.841 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T01:30:25.841 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: kSnappyCompression supported: 1 2024-09-16T01:30:25.841 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T01:30:25.841 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T01:30:25.842 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-16T01:30:25.842 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T01:30:25.842 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T01:30:25.842 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.merge_operator: 2024-09-16T01:30:25.842 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compaction_filter: None 2024-09-16T01:30:25.842 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T01:30:25.842 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T01:30:25.842 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T01:30:25.842 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T01:30:25.842 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5633b8eef320) 2024-09-16T01:30:25.842 INFO:journalctl@ceph.mon.a.smithi139.stdout: cache_index_and_filter_blocks: 1 2024-09-16T01:30:25.842 INFO:journalctl@ceph.mon.a.smithi139.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T01:30:25.842 INFO:journalctl@ceph.mon.a.smithi139.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T01:30:25.843 INFO:journalctl@ceph.mon.a.smithi139.stdout: pin_top_level_index_and_filter: 1 2024-09-16T01:30:25.843 INFO:journalctl@ceph.mon.a.smithi139.stdout: index_type: 0 2024-09-16T01:30:25.843 INFO:journalctl@ceph.mon.a.smithi139.stdout: data_block_index_type: 0 2024-09-16T01:30:25.843 INFO:journalctl@ceph.mon.a.smithi139.stdout: index_shortening: 1 2024-09-16T01:30:25.843 INFO:journalctl@ceph.mon.a.smithi139.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T01:30:25.843 INFO:journalctl@ceph.mon.a.smithi139.stdout: checksum: 4 2024-09-16T01:30:25.843 INFO:journalctl@ceph.mon.a.smithi139.stdout: no_block_cache: 0 2024-09-16T01:30:25.843 INFO:journalctl@ceph.mon.a.smithi139.stdout: block_cache: 0x5633b8ee31f0 2024-09-16T01:30:25.843 INFO:journalctl@ceph.mon.a.smithi139.stdout: block_cache_name: BinnedLRUCache 2024-09-16T01:30:25.843 INFO:journalctl@ceph.mon.a.smithi139.stdout: block_cache_options: 2024-09-16T01:30:25.844 INFO:journalctl@ceph.mon.a.smithi139.stdout: capacity : 536870912 2024-09-16T01:30:25.844 INFO:journalctl@ceph.mon.a.smithi139.stdout: num_shard_bits : 4 2024-09-16T01:30:25.844 INFO:journalctl@ceph.mon.a.smithi139.stdout: strict_capacity_limit : 0 2024-09-16T01:30:25.844 INFO:journalctl@ceph.mon.a.smithi139.stdout: high_pri_pool_ratio: 0.000 2024-09-16T01:30:25.844 INFO:journalctl@ceph.mon.a.smithi139.stdout: block_cache_compressed: (nil) 2024-09-16T01:30:25.844 INFO:journalctl@ceph.mon.a.smithi139.stdout: persistent_cache: (nil) 2024-09-16T01:30:25.844 INFO:journalctl@ceph.mon.a.smithi139.stdout: block_size: 4096 2024-09-16T01:30:25.844 INFO:journalctl@ceph.mon.a.smithi139.stdout: block_size_deviation: 10 2024-09-16T01:30:25.844 INFO:journalctl@ceph.mon.a.smithi139.stdout: block_restart_interval: 16 2024-09-16T01:30:25.844 INFO:journalctl@ceph.mon.a.smithi139.stdout: index_block_restart_interval: 1 2024-09-16T01:30:25.844 INFO:journalctl@ceph.mon.a.smithi139.stdout: metadata_block_size: 4096 2024-09-16T01:30:25.844 INFO:journalctl@ceph.mon.a.smithi139.stdout: partition_filters: 0 2024-09-16T01:30:25.844 INFO:journalctl@ceph.mon.a.smithi139.stdout: use_delta_encoding: 1 2024-09-16T01:30:25.844 INFO:journalctl@ceph.mon.a.smithi139.stdout: filter_policy: bloomfilter 2024-09-16T01:30:25.845 INFO:journalctl@ceph.mon.a.smithi139.stdout: whole_key_filtering: 1 2024-09-16T01:30:25.845 INFO:journalctl@ceph.mon.a.smithi139.stdout: verify_compression: 0 2024-09-16T01:30:25.845 INFO:journalctl@ceph.mon.a.smithi139.stdout: read_amp_bytes_per_bit: 0 2024-09-16T01:30:25.845 INFO:journalctl@ceph.mon.a.smithi139.stdout: format_version: 5 2024-09-16T01:30:25.845 INFO:journalctl@ceph.mon.a.smithi139.stdout: enable_index_compression: 1 2024-09-16T01:30:25.845 INFO:journalctl@ceph.mon.a.smithi139.stdout: block_align: 0 2024-09-16T01:30:25.845 INFO:journalctl@ceph.mon.a.smithi139.stdout: max_auto_readahead_size: 262144 2024-09-16T01:30:25.845 INFO:journalctl@ceph.mon.a.smithi139.stdout: prepopulate_block_cache: 0 2024-09-16T01:30:25.845 INFO:journalctl@ceph.mon.a.smithi139.stdout: initial_auto_readahead_size: 8192 2024-09-16T01:30:25.845 INFO:journalctl@ceph.mon.a.smithi139.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T01:30:25.845 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T01:30:25.845 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T01:30:25.845 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compression: NoCompression 2024-09-16T01:30:25.845 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T01:30:25.846 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T01:30:25.846 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T01:30:25.846 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.num_levels: 7 2024-09-16T01:30:25.846 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T01:30:25.846 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T01:30:25.846 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T01:30:25.846 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T01:30:25.846 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T01:30:25.846 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T01:30:25.846 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T01:30:25.846 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T01:30:25.846 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T01:30:25.846 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T01:30:25.846 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T01:30:25.847 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T01:30:25.847 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T01:30:25.847 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T01:30:25.847 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T01:30:25.847 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T01:30:25.847 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T01:30:25.847 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T01:30:25.847 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T01:30:25.847 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T01:30:25.847 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T01:30:25.847 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T01:30:25.848 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T01:30:25.848 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T01:30:25.848 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T01:30:25.848 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T01:30:25.848 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T01:30:25.848 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T01:30:25.848 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T01:30:25.848 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T01:30:25.848 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T01:30:25.848 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T01:30:25.848 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T01:30:25.848 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T01:30:25.848 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T01:30:25.848 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T01:30:25.849 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T01:30:25.849 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T01:30:25.849 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T01:30:25.849 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T01:30:25.849 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T01:30:25.849 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T01:30:25.849 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T01:30:25.849 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T01:30:25.849 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T01:30:25.849 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T01:30:25.849 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T01:30:25.849 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T01:30:25.849 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T01:30:25.849 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T01:30:25.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T01:30:25.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T01:30:25.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T01:30:25.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T01:30:25.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.inplace_update_support: 0 2024-09-16T01:30:25.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T01:30:25.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T01:30:25.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T01:30:25.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T01:30:25.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.bloom_locality: 0 2024-09-16T01:30:25.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.max_successive_merges: 0 2024-09-16T01:30:25.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T01:30:25.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T01:30:25.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.ttl: 2592000 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.enable_blob_files: false 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.min_blob_size: 0 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T01:30:25.851 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2961c875-ac44-4942-a3da-94ff57ed4c6f 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726450225649831, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726450225650874, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100978, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 336, "table_properties": {"data_size": 98863, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14453, "raw_average_key_size": 49, "raw_value_size": 91043, "raw_average_value_size": 312, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726450225, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2961c875-ac44-4942-a3da-94ff57ed4c6f", "db_session_id": "YHVBFXCLG7A7DYRH9GZ0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726450225650968, "job": 1, "event": "recovery_finished"} 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5633b8f10e00 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: DB pointer 0x5633b900c000 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout: ** DB Stats ** 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T01:30:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: ** Compaction Stats [default] ** 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: L0 2/0 100.47 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: Sum 2/0 100.47 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: ** Compaction Stats [default] ** 2024-09-16T01:30:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: Cumulative compaction: 0.00 GB write, 31.03 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: Interval compaction: 0.00 GB write, 31.03 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T01:30:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout: Block cache BinnedLRUCache@0x5633b8ee31f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout: 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: starting mon.a rank 0 at public addrs [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] at bind addrs [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: mon.a@-1(???) e1 preinit fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: mon.a@-1(???).mds e1 new map 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: mon.a@-1(???).mds e1 print_map 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout: e1 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout: btime 2024-09-16T01:30:17:770097+0000 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout: legacy client fscid: -1 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout: 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout: No filesystems configured 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T01:30:25.855 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:30:25.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:30:25.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:30:25.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T01:30:25.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 bash[31156]: 9a9750ef87b6c31e7f3a3ff1954a7c7ff21f7c5016735b7b1e79246d0e55a621 2024-09-16T01:30:25.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 systemd[1]: Started Ceph mon.a for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:30:25.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T01:30:25.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: monmap epoch 1 2024-09-16T01:30:25.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:30:25.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: last_changed 2024-09-16T01:30:10.119520+0000 2024-09-16T01:30:25.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: created 2024-09-16T01:30:10.119520+0000 2024-09-16T01:30:25.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: min_mon_release 19 (squid) 2024-09-16T01:30:25.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: election_strategy: 1 2024-09-16T01:30:25.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: 0: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.a 2024-09-16T01:30:25.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: fsmap 2024-09-16T01:30:25.857 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T01:30:25.857 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:25 smithi139 ceph-mon[31175]: mgrmap e1: no daemons active 2024-09-16T01:30:28.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:27 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1237242531' entity='client.admin' 2024-09-16T01:30:29.230 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.923+0000 7fdac3c2a640 1 Processor -- start 2024-09-16T01:30:29.235 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.923+0000 7fdac3c2a640 1 -- start start 2024-09-16T01:30:29.236 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.923+0000 7fdac3c2a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdabc1076f0 0x7fdabc107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:29.236 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.923+0000 7fdac3c2a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc1080c0 con 0x7fdabc1076f0 2024-09-16T01:30:29.236 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.924+0000 7fdac19c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdabc1076f0 0x7fdabc107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:29.236 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.924+0000 7fdac19c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdabc1076f0 0x7fdabc107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39746/0 (socket says 172.21.15.139:39746) 2024-09-16T01:30:29.236 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.924+0000 7fdac19c6640 1 -- 172.21.15.139:0/2651572317 learned_addr learned my addr 172.21.15.139:0/2651572317 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:29.236 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.924+0000 7fdac19c6640 1 -- 172.21.15.139:0/2651572317 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdabc1088f0 con 0x7fdabc1076f0 2024-09-16T01:30:29.236 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.926+0000 7fdac19c6640 1 --2- 172.21.15.139:0/2651572317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdabc1076f0 0x7fdabc107af0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009820 tx=0x7fdab802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdafbd7462c1ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:29.236 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.926+0000 7fdac09c4640 1 -- 172.21.15.139:0/2651572317 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab8037070 con 0x7fdabc1076f0 2024-09-16T01:30:29.236 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.926+0000 7fdac09c4640 1 -- 172.21.15.139:0/2651572317 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdab8032900 con 0x7fdabc1076f0 2024-09-16T01:30:29.236 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.926+0000 7fdac09c4640 1 -- 172.21.15.139:0/2651572317 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab8030850 con 0x7fdabc1076f0 2024-09-16T01:30:29.236 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.927+0000 7fdac3c2a640 1 -- 172.21.15.139:0/2651572317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdabc1076f0 msgr2=0x7fdabc107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:29.236 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.927+0000 7fdac3c2a640 1 --2- 172.21.15.139:0/2651572317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdabc1076f0 0x7fdabc107af0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009820 tx=0x7fdab802a270 comp rx=0 tx=0).stop 2024-09-16T01:30:29.237 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.927+0000 7fdac3c2a640 1 -- 172.21.15.139:0/2651572317 shutdown_connections 2024-09-16T01:30:29.237 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.927+0000 7fdac3c2a640 1 --2- 172.21.15.139:0/2651572317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdabc1076f0 0x7fdabc107af0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:29.237 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.927+0000 7fdac3c2a640 1 -- 172.21.15.139:0/2651572317 >> 172.21.15.139:0/2651572317 conn(0x7fdabc102ec0 msgr2=0x7fdabc1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:29.237 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.927+0000 7fdac3c2a640 1 -- 172.21.15.139:0/2651572317 shutdown_connections 2024-09-16T01:30:29.237 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.927+0000 7fdac3c2a640 1 -- 172.21.15.139:0/2651572317 wait complete. 2024-09-16T01:30:29.237 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.927+0000 7fdac3c2a640 1 Processor -- start 2024-09-16T01:30:29.237 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.927+0000 7fdac3c2a640 1 -- start start 2024-09-16T01:30:29.237 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.928+0000 7fdac3c2a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdabc1076f0 0x7fdabc07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:29.237 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.928+0000 7fdac3c2a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc10bca0 con 0x7fdabc1076f0 2024-09-16T01:30:29.237 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.928+0000 7fdac19c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdabc1076f0 0x7fdabc07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:29.237 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.928+0000 7fdac19c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdabc1076f0 0x7fdabc07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39762/0 (socket says 172.21.15.139:39762) 2024-09-16T01:30:29.237 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.928+0000 7fdac19c6640 1 -- 172.21.15.139:0/1237242531 learned_addr learned my addr 172.21.15.139:0/1237242531 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.928+0000 7fdac19c6640 1 -- 172.21.15.139:0/1237242531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdabc07e0b0 con 0x7fdabc1076f0 2024-09-16T01:30:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.928+0000 7fdac19c6640 1 --2- 172.21.15.139:0/1237242531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdabc1076f0 0x7fdabc07db70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009940 tx=0x7fdab8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.928+0000 7fdab2ffd640 1 -- 172.21.15.139:0/1237242531 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab8042070 con 0x7fdabc1076f0 2024-09-16T01:30:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.928+0000 7fdac3c2a640 1 -- 172.21.15.139:0/1237242531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdabc07c2a0 con 0x7fdabc1076f0 2024-09-16T01:30:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.928+0000 7fdab2ffd640 1 -- 172.21.15.139:0/1237242531 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdab80308c0 con 0x7fdabc1076f0 2024-09-16T01:30:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.928+0000 7fdab2ffd640 1 -- 172.21.15.139:0/1237242531 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab8037040 con 0x7fdabc1076f0 2024-09-16T01:30:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.928+0000 7fdac3c2a640 1 -- 172.21.15.139:0/1237242531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdabc07c7e0 con 0x7fdabc1076f0 2024-09-16T01:30:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.928+0000 7fdab2ffd640 1 -- 172.21.15.139:0/1237242531 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdab8045070 con 0x7fdabc1076f0 2024-09-16T01:30:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.929+0000 7fdab2ffd640 1 -- 172.21.15.139:0/1237242531 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdab8047bc0 con 0x7fdabc1076f0 2024-09-16T01:30:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.929+0000 7fdac3c2a640 1 -- 172.21.15.139:0/1237242531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdabc10be20 con 0x7fdabc1076f0 2024-09-16T01:30:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.932+0000 7fdab2ffd640 1 -- 172.21.15.139:0/1237242531 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdab80327b0 con 0x7fdabc1076f0 2024-09-16T01:30:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.988+0000 7fdac3c2a640 1 -- 172.21.15.139:0/1237242531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fdabc07ca60 con 0x7fdabc1076f0 2024-09-16T01:30:29.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.990+0000 7fdab2ffd640 1 -- 172.21.15.139:0/1237242531 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fdabc07ca60 con 0x7fdabc1076f0 2024-09-16T01:30:29.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.991+0000 7fdac3c2a640 1 -- 172.21.15.139:0/1237242531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdabc1076f0 msgr2=0x7fdabc07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:29.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.991+0000 7fdac3c2a640 1 --2- 172.21.15.139:0/1237242531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdabc1076f0 0x7fdabc07db70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009940 tx=0x7fdab8032ce0 comp rx=0 tx=0).stop 2024-09-16T01:30:29.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.991+0000 7fdac3c2a640 1 -- 172.21.15.139:0/1237242531 shutdown_connections 2024-09-16T01:30:29.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.991+0000 7fdac3c2a640 1 --2- 172.21.15.139:0/1237242531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdabc1076f0 0x7fdabc07db70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:29.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.991+0000 7fdac3c2a640 1 -- 172.21.15.139:0/1237242531 >> 172.21.15.139:0/1237242531 conn(0x7fdabc102ec0 msgr2=0x7fdabc1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:29.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.991+0000 7fdac3c2a640 1 -- 172.21.15.139:0/1237242531 shutdown_connections 2024-09-16T01:30:29.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:26.991+0000 7fdac3c2a640 1 -- 172.21.15.139:0/1237242531 wait complete. 2024-09-16T01:30:29.240 INFO:teuthology.orchestra.run.smithi139.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T01:30:29.260 INFO:teuthology.orchestra.run.smithi139.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T01:30:29.260 INFO:teuthology.orchestra.run.smithi139.stdout:Creating mgr... 2024-09-16T01:30:29.260 INFO:teuthology.orchestra.run.smithi139.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T01:30:29.261 INFO:teuthology.orchestra.run.smithi139.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T01:30:29.516 INFO:teuthology.orchestra.run.smithi139.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.x 2024-09-16T01:30:29.516 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-16T01:30:29.661 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968.target.wants/ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@.service. 2024-09-16T01:30:29.871 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:29 smithi139 systemd[1]: Starting Ceph mgr.x for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:30:30.140 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:29 smithi139 podman[31387]: 2024-09-16 01:30:29.870279049 +0000 UTC m=+0.093508252 container create 6c955bee72d84911e2eb34d0b00c6092197a9e04c81d0312472d14fddfd8685d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T01:30:30.140 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:29 smithi139 podman[31387]: 2024-09-16 01:30:29.792425408 +0000 UTC m=+0.015654578 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T01:30:30.141 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:30 smithi139 podman[31387]: 2024-09-16 01:30:30.064339647 +0000 UTC m=+0.287568809 container init 6c955bee72d84911e2eb34d0b00c6092197a9e04c81d0312472d14fddfd8685d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-16T01:30:30.141 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:30 smithi139 podman[31387]: 2024-09-16 01:30:30.067151472 +0000 UTC m=+0.290380634 container start 6c955bee72d84911e2eb34d0b00c6092197a9e04c81d0312472d14fddfd8685d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux ) 2024-09-16T01:30:30.141 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:30 smithi139 bash[31387]: 6c955bee72d84911e2eb34d0b00c6092197a9e04c81d0312472d14fddfd8685d 2024-09-16T01:30:30.154 INFO:teuthology.orchestra.run.smithi139.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T01:30:30.154 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stdout disabled 2024-09-16T01:30:30.162 INFO:teuthology.orchestra.run.smithi139.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T01:30:30.162 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stdout inactive 2024-09-16T01:30:30.162 INFO:teuthology.orchestra.run.smithi139.stdout:firewalld.service is not enabled 2024-09-16T01:30:30.162 INFO:teuthology.orchestra.run.smithi139.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T01:30:30.171 INFO:teuthology.orchestra.run.smithi139.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T01:30:30.171 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stdout disabled 2024-09-16T01:30:30.181 INFO:teuthology.orchestra.run.smithi139.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T01:30:30.181 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stdout inactive 2024-09-16T01:30:30.181 INFO:teuthology.orchestra.run.smithi139.stdout:firewalld.service is not enabled 2024-09-16T01:30:30.181 INFO:teuthology.orchestra.run.smithi139.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-16T01:30:30.181 INFO:teuthology.orchestra.run.smithi139.stdout:Waiting for mgr to start... 2024-09-16T01:30:30.181 INFO:teuthology.orchestra.run.smithi139.stdout:Waiting for mgr... 2024-09-16T01:30:30.403 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:30 smithi139 systemd[1]: Started Ceph mgr.x for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:30:30.403 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:30 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:30.189+0000 7f32aeade040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:30:30.403 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:30 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:30.250+0000 7f32aeade040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:30:30.403 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:30 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:30.402+0000 7f32aeade040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:30:30.667 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:30 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:30.471+0000 7f32aeade040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:30:30.667 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:30 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:30.625+0000 7f32aeade040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:30:30.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:30 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/910884166' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T01:30:30.923 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:30 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:30.776+0000 7f32aeade040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:30:31.288 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:31 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:31.094+0000 7f32aeade040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:30:31.288 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:31 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:31.154+0000 7f32aeade040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:30:31.291 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 2024-09-16T01:30:31.302 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:30:31.302 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:30:31.302 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T01:30:31.302 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T01:30:31.302 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T01:30:31.302 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T01:30:31.302 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:31.302 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T01:30:31.302 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T01:30:31.302 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 0 2024-09-16T01:30:31.302 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:30:31.303 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T01:30:31.303 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T01:30:31.303 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:30:31.303 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-16T01:30:31.303 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T01:30:31.303 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:30:31.303 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T01:30:31.303 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T01:30:31.303 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:31.303 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T01:30:31.303 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:30:31.303 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T01:30:31.303 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T01:30:31.304 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T01:30:31.304 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T01:30:31.304 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T01:30:31.304 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T01:30:31.304 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:31.304 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T01:30:31.304 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T01:30:31.304 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T01:30:31.304 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T01:30:31.304 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T01:30:31.304 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T01:30:31.304 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T01:30:31.304 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T01:30:31.305 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T01:30:31.305 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:31.305 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T01:30:31.305 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:30:31.305 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T01:30:17:770097+0000", 2024-09-16T01:30:31.305 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T01:30:31.305 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T01:30:31.305 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:31.305 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T01:30:31.305 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T01:30:31.305 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T01:30:31.305 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T01:30:31.305 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T01:30:31.306 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T01:30:31.306 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T01:30:31.306 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:30:31.306 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:30:31.306 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:31.306 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T01:30:31.306 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:30:31.306 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T01:30:17.770853+0000", 2024-09-16T01:30:31.306 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:30:31.306 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:31.306 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T01:30:31.307 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:30:31.307 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.595+0000 7fb9caa95640 1 Processor -- start 2024-09-16T01:30:31.307 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.595+0000 7fb9caa95640 1 -- start start 2024-09-16T01:30:31.307 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.596+0000 7fb9caa95640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9c41081a0 0x7fb9c410a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:31.307 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.596+0000 7fb9caa95640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c407be70 con 0x7fb9c41081a0 2024-09-16T01:30:31.307 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.596+0000 7fb9c8831640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9c41081a0 0x7fb9c410a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:31.307 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.596+0000 7fb9c8831640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9c41081a0 0x7fb9c410a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39782/0 (socket says 172.21.15.139:39782) 2024-09-16T01:30:31.307 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.596+0000 7fb9c8831640 1 -- 172.21.15.139:0/1833648741 learned_addr learned my addr 172.21.15.139:0/1833648741 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:31.307 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.596+0000 7fb9c8831640 1 -- 172.21.15.139:0/1833648741 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c410aad0 con 0x7fb9c41081a0 2024-09-16T01:30:31.307 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.597+0000 7fb9c8831640 1 --2- 172.21.15.139:0/1833648741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9c41081a0 0x7fb9c410a590 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009820 tx=0x7fb9b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837e440e1f552878 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:31.307 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.597+0000 7fb9c37fe640 1 -- 172.21.15.139:0/1833648741 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9b8037070 con 0x7fb9c41081a0 2024-09-16T01:30:31.307 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.597+0000 7fb9c37fe640 1 -- 172.21.15.139:0/1833648741 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb9b8032900 con 0x7fb9c41081a0 2024-09-16T01:30:31.307 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.597+0000 7fb9c37fe640 1 -- 172.21.15.139:0/1833648741 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9b8030850 con 0x7fb9c41081a0 2024-09-16T01:30:31.308 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.598+0000 7fb9caa95640 1 -- 172.21.15.139:0/1833648741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9c41081a0 msgr2=0x7fb9c410a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:31.308 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.598+0000 7fb9caa95640 1 --2- 172.21.15.139:0/1833648741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9c41081a0 0x7fb9c410a590 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009820 tx=0x7fb9b802a270 comp rx=0 tx=0).stop 2024-09-16T01:30:31.308 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.598+0000 7fb9caa95640 1 -- 172.21.15.139:0/1833648741 shutdown_connections 2024-09-16T01:30:31.308 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.598+0000 7fb9caa95640 1 --2- 172.21.15.139:0/1833648741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9c41081a0 0x7fb9c410a590 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:31.308 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.598+0000 7fb9caa95640 1 -- 172.21.15.139:0/1833648741 >> 172.21.15.139:0/1833648741 conn(0x7fb9c4101e20 msgr2=0x7fb9c4104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:31.308 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.598+0000 7fb9caa95640 1 -- 172.21.15.139:0/1833648741 shutdown_connections 2024-09-16T01:30:31.308 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.598+0000 7fb9caa95640 1 -- 172.21.15.139:0/1833648741 wait complete. 2024-09-16T01:30:31.308 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.598+0000 7fb9caa95640 1 Processor -- start 2024-09-16T01:30:31.308 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.598+0000 7fb9caa95640 1 -- start start 2024-09-16T01:30:31.308 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.599+0000 7fb9caa95640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9c41081a0 0x7fb9c419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:31.308 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.599+0000 7fb9caa95640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c410bb10 con 0x7fb9c41081a0 2024-09-16T01:30:31.308 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.599+0000 7fb9c8831640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9c41081a0 0x7fb9c419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:31.308 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.599+0000 7fb9c8831640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9c41081a0 0x7fb9c419b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39788/0 (socket says 172.21.15.139:39788) 2024-09-16T01:30:31.309 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.599+0000 7fb9c8831640 1 -- 172.21.15.139:0/910884166 learned_addr learned my addr 172.21.15.139:0/910884166 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:31.309 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.599+0000 7fb9c8831640 1 -- 172.21.15.139:0/910884166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c419bc30 con 0x7fb9c41081a0 2024-09-16T01:30:31.309 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.599+0000 7fb9c8831640 1 --2- 172.21.15.139:0/910884166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9c41081a0 0x7fb9c419b6f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8031000 tx=0x7fb9b80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:31.309 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.599+0000 7fb9c1ffb640 1 -- 172.21.15.139:0/910884166 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9b8042070 con 0x7fb9c41081a0 2024-09-16T01:30:31.309 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.599+0000 7fb9c1ffb640 1 -- 172.21.15.139:0/910884166 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb9b8032be0 con 0x7fb9c41081a0 2024-09-16T01:30:31.309 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.599+0000 7fb9caa95640 1 -- 172.21.15.139:0/910884166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c419bec0 con 0x7fb9c41081a0 2024-09-16T01:30:31.309 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.599+0000 7fb9c1ffb640 1 -- 172.21.15.139:0/910884166 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9b8037040 con 0x7fb9c41081a0 2024-09-16T01:30:31.309 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.600+0000 7fb9caa95640 1 -- 172.21.15.139:0/910884166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c419ebb0 con 0x7fb9c41081a0 2024-09-16T01:30:31.309 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.600+0000 7fb9c1ffb640 1 -- 172.21.15.139:0/910884166 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb9b8045070 con 0x7fb9c41081a0 2024-09-16T01:30:31.309 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.600+0000 7fb9c1ffb640 1 -- 172.21.15.139:0/910884166 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb9b8047b20 con 0x7fb9c41081a0 2024-09-16T01:30:31.309 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.601+0000 7fb9caa95640 1 -- 172.21.15.139:0/910884166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c4108600 con 0x7fb9c41081a0 2024-09-16T01:30:31.310 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.604+0000 7fb9c1ffb640 1 -- 172.21.15.139:0/910884166 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb9b804c030 con 0x7fb9c41081a0 2024-09-16T01:30:31.310 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.666+0000 7fb9caa95640 1 -- 172.21.15.139:0/910884166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb9c419ed80 con 0x7fb9c41081a0 2024-09-16T01:30:31.310 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.667+0000 7fb9c1ffb640 1 -- 172.21.15.139:0/910884166 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fb9b80328c0 con 0x7fb9c41081a0 2024-09-16T01:30:31.310 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.668+0000 7fb9caa95640 1 -- 172.21.15.139:0/910884166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9c41081a0 msgr2=0x7fb9c419b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:31.310 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.668+0000 7fb9caa95640 1 --2- 172.21.15.139:0/910884166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9c41081a0 0x7fb9c419b6f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8031000 tx=0x7fb9b80326e0 comp rx=0 tx=0).stop 2024-09-16T01:30:31.310 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.668+0000 7fb9caa95640 1 -- 172.21.15.139:0/910884166 shutdown_connections 2024-09-16T01:30:31.310 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.668+0000 7fb9caa95640 1 --2- 172.21.15.139:0/910884166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9c41081a0 0x7fb9c419b6f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:31.310 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.668+0000 7fb9caa95640 1 -- 172.21.15.139:0/910884166 >> 172.21.15.139:0/910884166 conn(0x7fb9c4101e20 msgr2=0x7fb9c4102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:31.310 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.668+0000 7fb9caa95640 1 -- 172.21.15.139:0/910884166 shutdown_connections 2024-09-16T01:30:31.310 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:30.668+0000 7fb9caa95640 1 -- 172.21.15.139:0/910884166 wait complete. 2024-09-16T01:30:31.310 INFO:teuthology.orchestra.run.smithi139.stdout:mgr not available, waiting (1/15)... 2024-09-16T01:30:31.672 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:31 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:31.341+0000 7f32aeade040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:30:32.672 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:32 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:32.214+0000 7f32aeade040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:30:32.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:32 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:32.281+0000 7f32aeade040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:30:33.239 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:32 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:32.985+0000 7f32aeade040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:30:33.239 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:33 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:33.098+0000 7f32aeade040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:30:33.239 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:33 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:33.238+0000 7f32aeade040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:30:33.509 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:33 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:33.302+0000 7f32aeade040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:30:33.509 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:33 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:33.461+0000 7f32aeade040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:30:33.774 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:33 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:33.537+0000 7f32aeade040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:30:33.774 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:33 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:33.600+0000 7f32aeade040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:30:33.774 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:33 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:33.772+0000 7f32aeade040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:30:34.061 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:33 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2925700820' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T01:30:34.062 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:33 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:33.845+0000 7f32aeade040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:30:34.062 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:33 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:33.914+0000 7f32aeade040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:30:34.062 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:34.000+0000 7f32aeade040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:30:34.062 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:34.060+0000 7f32aeade040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:30:34.416 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:34.121+0000 7f32aeade040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:30:34.633 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 2024-09-16T01:30:34.633 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:30:34.633 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 0 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T01:30:34.634 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:30:34.635 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T01:30:34.635 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T01:30:34.635 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:34.635 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T01:30:34.635 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:30:34.635 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T01:30:34.635 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T01:30:34.635 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T01:30:34.643 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T01:30:34.643 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T01:30:34.643 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T01:30:34.643 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:34.643 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T01:30:34.643 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T01:30:34.643 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T01:30:34.643 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T01:30:34.643 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T01:30:34.643 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T01:30:34.643 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T01:30:34.643 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T01:30:34.644 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T01:30:34.644 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:34.644 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T01:30:34.644 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:30:34.644 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T01:30:17:770097+0000", 2024-09-16T01:30:34.644 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T01:30:34.644 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T01:30:34.644 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:34.644 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T01:30:34.644 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T01:30:34.644 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T01:30:34.644 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T01:30:34.645 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T01:30:34.645 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T01:30:34.645 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T01:30:34.645 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:30:34.645 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:30:34.645 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:34.645 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T01:30:34.645 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:30:34.645 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T01:30:17.770853+0000", 2024-09-16T01:30:34.645 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:30:34.645 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:34.645 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T01:30:34.645 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:30:34.645 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.722+0000 7fe0fa184640 1 Processor -- start 2024-09-16T01:30:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.722+0000 7fe0fa184640 1 -- start start 2024-09-16T01:30:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.722+0000 7fe0fa184640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f41081e0 0x7fe0f410a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.722+0000 7fe0fa184640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f407be30 con 0x7fe0f41081e0 2024-09-16T01:30:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.723+0000 7fe0f37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f41081e0 0x7fe0f410a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.723+0000 7fe0f37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f41081e0 0x7fe0f410a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45620/0 (socket says 172.21.15.139:45620) 2024-09-16T01:30:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.723+0000 7fe0f37fe640 1 -- 172.21.15.139:0/2271789738 learned_addr learned my addr 172.21.15.139:0/2271789738 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.723+0000 7fe0f37fe640 1 -- 172.21.15.139:0/2271789738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f410ab10 con 0x7fe0f41081e0 2024-09-16T01:30:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.724+0000 7fe0f37fe640 1 --2- 172.21.15.139:0/2271789738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f41081e0 0x7fe0f410a5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009820 tx=0x7fe0e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f9609d6a89e3a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.724+0000 7fe0f27fc640 1 -- 172.21.15.139:0/2271789738 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0e4037070 con 0x7fe0f41081e0 2024-09-16T01:30:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.724+0000 7fe0f27fc640 1 -- 172.21.15.139:0/2271789738 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe0e4032900 con 0x7fe0f41081e0 2024-09-16T01:30:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.724+0000 7fe0f27fc640 1 -- 172.21.15.139:0/2271789738 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0e4030850 con 0x7fe0f41081e0 2024-09-16T01:30:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.725+0000 7fe0fa184640 1 -- 172.21.15.139:0/2271789738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f41081e0 msgr2=0x7fe0f410a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.725+0000 7fe0fa184640 1 --2- 172.21.15.139:0/2271789738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f41081e0 0x7fe0f410a5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009820 tx=0x7fe0e402a270 comp rx=0 tx=0).stop 2024-09-16T01:30:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.725+0000 7fe0fa184640 1 -- 172.21.15.139:0/2271789738 shutdown_connections 2024-09-16T01:30:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.725+0000 7fe0fa184640 1 --2- 172.21.15.139:0/2271789738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f41081e0 0x7fe0f410a5d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.725+0000 7fe0fa184640 1 -- 172.21.15.139:0/2271789738 >> 172.21.15.139:0/2271789738 conn(0x7fe0f4101e20 msgr2=0x7fe0f4104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.725+0000 7fe0fa184640 1 -- 172.21.15.139:0/2271789738 shutdown_connections 2024-09-16T01:30:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.725+0000 7fe0fa184640 1 -- 172.21.15.139:0/2271789738 wait complete. 2024-09-16T01:30:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.725+0000 7fe0fa184640 1 Processor -- start 2024-09-16T01:30:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.726+0000 7fe0fa184640 1 -- start start 2024-09-16T01:30:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.726+0000 7fe0fa184640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f41081e0 0x7fe0f419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.726+0000 7fe0fa184640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f410baf0 con 0x7fe0f41081e0 2024-09-16T01:30:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.726+0000 7fe0f37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f41081e0 0x7fe0f419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.726+0000 7fe0f37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f41081e0 0x7fe0f419b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45628/0 (socket says 172.21.15.139:45628) 2024-09-16T01:30:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.726+0000 7fe0f37fe640 1 -- 172.21.15.139:0/2925700820 learned_addr learned my addr 172.21.15.139:0/2925700820 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.726+0000 7fe0f37fe640 1 -- 172.21.15.139:0/2925700820 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f419bc30 con 0x7fe0f41081e0 2024-09-16T01:30:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.726+0000 7fe0f37fe640 1 --2- 172.21.15.139:0/2925700820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f41081e0 0x7fe0f419b6f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40097f0 tx=0x7fe0e4030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.727+0000 7fe0f0ff9640 1 -- 172.21.15.139:0/2925700820 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0e4042070 con 0x7fe0f41081e0 2024-09-16T01:30:34.648 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.727+0000 7fe0fa184640 1 -- 172.21.15.139:0/2925700820 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f419bec0 con 0x7fe0f41081e0 2024-09-16T01:30:34.648 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.727+0000 7fe0f0ff9640 1 -- 172.21.15.139:0/2925700820 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe0e4032ca0 con 0x7fe0f41081e0 2024-09-16T01:30:34.648 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.727+0000 7fe0f0ff9640 1 -- 172.21.15.139:0/2925700820 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0e4037040 con 0x7fe0f41081e0 2024-09-16T01:30:34.648 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.727+0000 7fe0fa184640 1 -- 172.21.15.139:0/2925700820 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f419c320 con 0x7fe0f41081e0 2024-09-16T01:30:34.648 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.727+0000 7fe0f0ff9640 1 -- 172.21.15.139:0/2925700820 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe0e4045070 con 0x7fe0f41081e0 2024-09-16T01:30:34.648 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.727+0000 7fe0f0ff9640 1 -- 172.21.15.139:0/2925700820 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe0e4047a90 con 0x7fe0f41081e0 2024-09-16T01:30:34.648 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.728+0000 7fe0fa184640 1 -- 172.21.15.139:0/2925700820 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f4108640 con 0x7fe0f41081e0 2024-09-16T01:30:34.648 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.732+0000 7fe0f0ff9640 1 -- 172.21.15.139:0/2925700820 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe0e403db10 con 0x7fe0f41081e0 2024-09-16T01:30:34.648 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.793+0000 7fe0fa184640 1 -- 172.21.15.139:0/2925700820 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe0f4106030 con 0x7fe0f41081e0 2024-09-16T01:30:34.648 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.794+0000 7fe0f0ff9640 1 -- 172.21.15.139:0/2925700820 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fe0e403d430 con 0x7fe0f41081e0 2024-09-16T01:30:34.648 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.795+0000 7fe0fa184640 1 -- 172.21.15.139:0/2925700820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f41081e0 msgr2=0x7fe0f419b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:34.648 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.795+0000 7fe0fa184640 1 --2- 172.21.15.139:0/2925700820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f41081e0 0x7fe0f419b6f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40097f0 tx=0x7fe0e4030ec0 comp rx=0 tx=0).stop 2024-09-16T01:30:34.648 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.795+0000 7fe0fa184640 1 -- 172.21.15.139:0/2925700820 shutdown_connections 2024-09-16T01:30:34.649 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.795+0000 7fe0fa184640 1 --2- 172.21.15.139:0/2925700820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f41081e0 0x7fe0f419b6f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:34.649 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.795+0000 7fe0fa184640 1 -- 172.21.15.139:0/2925700820 >> 172.21.15.139:0/2925700820 conn(0x7fe0f4101e20 msgr2=0x7fe0f4102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:34.649 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.795+0000 7fe0fa184640 1 -- 172.21.15.139:0/2925700820 shutdown_connections 2024-09-16T01:30:34.649 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:33.795+0000 7fe0fa184640 1 -- 172.21.15.139:0/2925700820 wait complete. 2024-09-16T01:30:34.649 INFO:teuthology.orchestra.run.smithi139.stdout:mgr not available, waiting (2/15)... 2024-09-16T01:30:35.423 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:34.923+0000 7f32aeade040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:30:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-mon[31175]: Activating manager daemon x 2024-09-16T01:30:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-mon[31175]: mgrmap e2: x(active, starting, since 0.00317825s) 2024-09-16T01:30:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-mon[31175]: from='mgr.14100 172.21.15.139:0/4167275237' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T01:30:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-mon[31175]: from='mgr.14100 172.21.15.139:0/4167275237' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T01:30:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-mon[31175]: from='mgr.14100 172.21.15.139:0/4167275237' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T01:30:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-mon[31175]: from='mgr.14100 172.21.15.139:0/4167275237' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:30:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-mon[31175]: from='mgr.14100 172.21.15.139:0/4167275237' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T01:30:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-mon[31175]: Manager daemon x is now available 2024-09-16T01:30:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-mon[31175]: from='mgr.14100 172.21.15.139:0/4167275237' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T01:30:35.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-mon[31175]: from='mgr.14100 172.21.15.139:0/4167275237' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T01:30:35.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-mon[31175]: from='mgr.14100 172.21.15.139:0/4167275237' entity='mgr.x' 2024-09-16T01:30:35.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-mon[31175]: from='mgr.14100 172.21.15.139:0/4167275237' entity='mgr.x' 2024-09-16T01:30:35.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:34 smithi139 ceph-mon[31175]: from='mgr.14100 172.21.15.139:0/4167275237' entity='mgr.x' 2024-09-16T01:30:37.036 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:36 smithi139 ceph-mon[31175]: mgrmap e3: x(active, since 1.00719s) 2024-09-16T01:30:37.886 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 2024-09-16T01:30:37.889 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:30:37.889 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:30:37.889 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T01:30:37.889 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T01:30:37.889 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T01:30:37.889 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T01:30:37.889 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:37.889 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T01:30:37.889 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T01:30:37.889 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 0 2024-09-16T01:30:37.889 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:30:37.890 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T01:30:37.890 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T01:30:37.890 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:30:37.890 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-09-16T01:30:37.890 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T01:30:37.890 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:30:37.890 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T01:30:37.890 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T01:30:37.890 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:37.890 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T01:30:37.890 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:30:37.891 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T01:30:37.891 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T01:30:37.891 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T01:30:37.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T01:30:37.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T01:30:37.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T01:30:37.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:37.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T01:30:37.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T01:30:37.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T01:30:37.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T01:30:37.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T01:30:37.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T01:30:37.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T01:30:37.900 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T01:30:37.900 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T01:30:37.900 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:37.900 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T01:30:37.900 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:30:37.900 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T01:30:17:770097+0000", 2024-09-16T01:30:37.900 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T01:30:37.900 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T01:30:37.900 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:37.901 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T01:30:37.901 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T01:30:37.901 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T01:30:37.901 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T01:30:37.901 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T01:30:37.901 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T01:30:37.901 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T01:30:37.901 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout ], 2024-09-16T01:30:37.901 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:30:37.901 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:37.901 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T01:30:37.901 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T01:30:37.902 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T01:30:17.770853+0000", 2024-09-16T01:30:37.902 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T01:30:37.902 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout }, 2024-09-16T01:30:37.902 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T01:30:37.902 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:30:37.902 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.035+0000 7f808fdcc640 1 Processor -- start 2024-09-16T01:30:37.902 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.035+0000 7f808fdcc640 1 -- start start 2024-09-16T01:30:37.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.035+0000 7f808fdcc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80881059f0 0x7f8088105df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:37.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.035+0000 7f808fdcc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f80881063c0 con 0x7f80881059f0 2024-09-16T01:30:37.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.035+0000 7f808db68640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80881059f0 0x7f8088105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:37.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.036+0000 7f808db68640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80881059f0 0x7f8088105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45706/0 (socket says 172.21.15.139:45706) 2024-09-16T01:30:37.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.036+0000 7f808db68640 1 -- 172.21.15.139:0/2179597889 learned_addr learned my addr 172.21.15.139:0/2179597889 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:37.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.036+0000 7f808db68640 1 -- 172.21.15.139:0/2179597889 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80881089d0 con 0x7f80881059f0 2024-09-16T01:30:37.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.037+0000 7f808db68640 1 --2- 172.21.15.139:0/2179597889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80881059f0 0x7f8088105df0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8084009820 tx=0x7f808402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64f2ab215bf2bd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:37.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.037+0000 7f808cb66640 1 -- 172.21.15.139:0/2179597889 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8084037070 con 0x7f80881059f0 2024-09-16T01:30:37.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.037+0000 7f808cb66640 1 -- 172.21.15.139:0/2179597889 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8084032900 con 0x7f80881059f0 2024-09-16T01:30:37.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.037+0000 7f808cb66640 1 -- 172.21.15.139:0/2179597889 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8084030850 con 0x7f80881059f0 2024-09-16T01:30:37.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.038+0000 7f808fdcc640 1 -- 172.21.15.139:0/2179597889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80881059f0 msgr2=0x7f8088105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:37.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.038+0000 7f808fdcc640 1 --2- 172.21.15.139:0/2179597889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80881059f0 0x7f8088105df0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8084009820 tx=0x7f808402a270 comp rx=0 tx=0).stop 2024-09-16T01:30:37.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.038+0000 7f808fdcc640 1 -- 172.21.15.139:0/2179597889 shutdown_connections 2024-09-16T01:30:37.903 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.038+0000 7f808fdcc640 1 --2- 172.21.15.139:0/2179597889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80881059f0 0x7f8088105df0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:37.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.038+0000 7f808fdcc640 1 -- 172.21.15.139:0/2179597889 >> 172.21.15.139:0/2179597889 conn(0x7f8088101560 msgr2=0x7f8088103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:37.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.038+0000 7f808fdcc640 1 -- 172.21.15.139:0/2179597889 shutdown_connections 2024-09-16T01:30:37.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.038+0000 7f808fdcc640 1 -- 172.21.15.139:0/2179597889 wait complete. 2024-09-16T01:30:37.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.038+0000 7f808fdcc640 1 Processor -- start 2024-09-16T01:30:37.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.038+0000 7f808fdcc640 1 -- start start 2024-09-16T01:30:37.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.039+0000 7f808fdcc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80881059f0 0x7f808819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:37.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.039+0000 7f808fdcc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f808810bc00 con 0x7f80881059f0 2024-09-16T01:30:37.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.039+0000 7f808db68640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80881059f0 0x7f808819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:37.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.039+0000 7f808db68640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80881059f0 0x7f808819b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45722/0 (socket says 172.21.15.139:45722) 2024-09-16T01:30:37.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.039+0000 7f808db68640 1 -- 172.21.15.139:0/4046335964 learned_addr learned my addr 172.21.15.139:0/4046335964 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:37.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.039+0000 7f808db68640 1 -- 172.21.15.139:0/4046335964 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808819bc80 con 0x7f80881059f0 2024-09-16T01:30:37.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.039+0000 7f808db68640 1 --2- 172.21.15.139:0/4046335964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80881059f0 0x7f808819b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8084009940 tx=0x7f80840326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:37.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.039+0000 7f807effd640 1 -- 172.21.15.139:0/4046335964 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8084042070 con 0x7f80881059f0 2024-09-16T01:30:37.904 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.039+0000 7f807effd640 1 -- 172.21.15.139:0/4046335964 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8084032be0 con 0x7f80881059f0 2024-09-16T01:30:37.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.039+0000 7f808fdcc640 1 -- 172.21.15.139:0/4046335964 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808819bf10 con 0x7f80881059f0 2024-09-16T01:30:37.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.039+0000 7f807effd640 1 -- 172.21.15.139:0/4046335964 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8084037040 con 0x7f80881059f0 2024-09-16T01:30:37.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.039+0000 7f808fdcc640 1 -- 172.21.15.139:0/4046335964 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808819ec00 con 0x7f80881059f0 2024-09-16T01:30:37.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.040+0000 7f807effd640 1 -- 172.21.15.139:0/4046335964 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f8084045070 con 0x7f80881059f0 2024-09-16T01:30:37.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.040+0000 7f807effd640 1 --2- 172.21.15.139:0/4046335964 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f8074039160 0x7f807403b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:37.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.040+0000 7f807effd640 1 -- 172.21.15.139:0/4046335964 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f808406c0f0 con 0x7f80881059f0 2024-09-16T01:30:37.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.040+0000 7f808fdcc640 1 -- 172.21.15.139:0/4046335964 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8088105e70 con 0x7f80881059f0 2024-09-16T01:30:37.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.041+0000 7f808d367640 1 --2- 172.21.15.139:0/4046335964 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f8074039160 0x7f807403b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:37.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.041+0000 7f808d367640 1 --2- 172.21.15.139:0/4046335964 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f8074039160 0x7f807403b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f80780098c0 tx=0x7f8078006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:37.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.046+0000 7f807effd640 1 -- 172.21.15.139:0/4046335964 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8084032d90 con 0x7f80881059f0 2024-09-16T01:30:37.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.272+0000 7f808fdcc640 1 -- 172.21.15.139:0/4046335964 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f808819f170 con 0x7f80881059f0 2024-09-16T01:30:37.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.273+0000 7f807effd640 1 -- 172.21.15.139:0/4046335964 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f808402ed90 con 0x7f80881059f0 2024-09-16T01:30:37.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.275+0000 7f808fdcc640 1 -- 172.21.15.139:0/4046335964 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f8074039160 msgr2=0x7f807403b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:37.905 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.275+0000 7f808fdcc640 1 --2- 172.21.15.139:0/4046335964 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f8074039160 0x7f807403b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f80780098c0 tx=0x7f8078006db0 comp rx=0 tx=0).stop 2024-09-16T01:30:37.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.275+0000 7f808fdcc640 1 -- 172.21.15.139:0/4046335964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80881059f0 msgr2=0x7f808819b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:37.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.275+0000 7f808fdcc640 1 --2- 172.21.15.139:0/4046335964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80881059f0 0x7f808819b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8084009940 tx=0x7f80840326e0 comp rx=0 tx=0).stop 2024-09-16T01:30:37.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.275+0000 7f808fdcc640 1 -- 172.21.15.139:0/4046335964 shutdown_connections 2024-09-16T01:30:37.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.275+0000 7f808fdcc640 1 --2- 172.21.15.139:0/4046335964 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f8074039160 0x7f807403b620 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:37.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.275+0000 7f808fdcc640 1 --2- 172.21.15.139:0/4046335964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80881059f0 0x7f808819b740 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:37.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.275+0000 7f808fdcc640 1 -- 172.21.15.139:0/4046335964 >> 172.21.15.139:0/4046335964 conn(0x7f8088101560 msgr2=0x7f8088102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:37.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.275+0000 7f808fdcc640 1 -- 172.21.15.139:0/4046335964 shutdown_connections 2024-09-16T01:30:37.906 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:37.275+0000 7f808fdcc640 1 -- 172.21.15.139:0/4046335964 wait complete. 2024-09-16T01:30:37.906 INFO:teuthology.orchestra.run.smithi139.stdout:mgr is available 2024-09-16T01:30:38.138 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:37 smithi139 ceph-mon[31175]: mgrmap e4: x(active, since 2s) 2024-09-16T01:30:38.138 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:37 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/4046335964' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T01:30:39.040 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 2024-09-16T01:30:39.041 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T01:30:39.041 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout fsid = 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:30:39.041 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.139:3300,v1:172.21.15.139:6789] 2024-09-16T01:30:39.041 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T01:30:39.041 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T01:30:39.041 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T01:30:39.041 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T01:30:39.041 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 2024-09-16T01:30:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T01:30:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T01:30:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 2024-09-16T01:30:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T01:30:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T01:30:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T01:30:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T01:30:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T01:30:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T01:30:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.322+0000 7f26757d8640 1 Processor -- start 2024-09-16T01:30:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.322+0000 7f26757d8640 1 -- start start 2024-09-16T01:30:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.322+0000 7f26757d8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26701076c0 0x7f2670107ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.322+0000 7f26757d8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2670108090 con 0x7f26701076c0 2024-09-16T01:30:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.322+0000 7f266effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26701076c0 0x7f2670107ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:39.043 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.322+0000 7f266effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26701076c0 0x7f2670107ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45738/0 (socket says 172.21.15.139:45738) 2024-09-16T01:30:39.043 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.322+0000 7f266effd640 1 -- 172.21.15.139:0/2671453617 learned_addr learned my addr 172.21.15.139:0/2671453617 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:39.043 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.322+0000 7f266effd640 1 -- 172.21.15.139:0/2671453617 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26701088c0 con 0x7f26701076c0 2024-09-16T01:30:39.043 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.323+0000 7f266effd640 1 --2- 172.21.15.139:0/2671453617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26701076c0 0x7f2670107ac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2660009820 tx=0x7f266002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19da2000176d9c2b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:39.043 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.323+0000 7f266dffb640 1 -- 172.21.15.139:0/2671453617 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2660037070 con 0x7f26701076c0 2024-09-16T01:30:39.043 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.323+0000 7f266dffb640 1 -- 172.21.15.139:0/2671453617 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2660032900 con 0x7f26701076c0 2024-09-16T01:30:39.043 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.323+0000 7f266dffb640 1 -- 172.21.15.139:0/2671453617 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2660030850 con 0x7f26701076c0 2024-09-16T01:30:39.043 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.324+0000 7f26757d8640 1 -- 172.21.15.139:0/2671453617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26701076c0 msgr2=0x7f2670107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:39.043 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.324+0000 7f26757d8640 1 --2- 172.21.15.139:0/2671453617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26701076c0 0x7f2670107ac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2660009820 tx=0x7f266002a270 comp rx=0 tx=0).stop 2024-09-16T01:30:39.043 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.324+0000 7f26757d8640 1 -- 172.21.15.139:0/2671453617 shutdown_connections 2024-09-16T01:30:39.043 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.324+0000 7f26757d8640 1 --2- 172.21.15.139:0/2671453617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26701076c0 0x7f2670107ac0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:39.043 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.324+0000 7f26757d8640 1 -- 172.21.15.139:0/2671453617 >> 172.21.15.139:0/2671453617 conn(0x7f2670102e50 msgr2=0x7f2670105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:39.043 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.324+0000 7f26757d8640 1 -- 172.21.15.139:0/2671453617 shutdown_connections 2024-09-16T01:30:39.043 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.324+0000 7f26757d8640 1 -- 172.21.15.139:0/2671453617 wait complete. 2024-09-16T01:30:39.044 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.324+0000 7f26757d8640 1 Processor -- start 2024-09-16T01:30:39.044 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.324+0000 7f26757d8640 1 -- start start 2024-09-16T01:30:39.044 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.325+0000 7f26757d8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26701076c0 0x7f267019b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:39.044 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.325+0000 7f26757d8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f267010bc70 con 0x7f26701076c0 2024-09-16T01:30:39.044 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.325+0000 7f266effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26701076c0 0x7f267019b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:39.044 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.325+0000 7f266effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26701076c0 0x7f267019b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45754/0 (socket says 172.21.15.139:45754) 2024-09-16T01:30:39.044 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.325+0000 7f266effd640 1 -- 172.21.15.139:0/2170312120 learned_addr learned my addr 172.21.15.139:0/2170312120 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:39.044 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.325+0000 7f266effd640 1 -- 172.21.15.139:0/2170312120 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267019bbf0 con 0x7f26701076c0 2024-09-16T01:30:39.044 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.325+0000 7f266effd640 1 --2- 172.21.15.139:0/2170312120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26701076c0 0x7f267019b6b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2660009940 tx=0x7f2660032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:39.044 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.325+0000 7f2657fff640 1 -- 172.21.15.139:0/2170312120 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2660042070 con 0x7f26701076c0 2024-09-16T01:30:39.044 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.325+0000 7f2657fff640 1 -- 172.21.15.139:0/2170312120 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26600308c0 con 0x7f26701076c0 2024-09-16T01:30:39.045 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.325+0000 7f2657fff640 1 -- 172.21.15.139:0/2170312120 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2660037040 con 0x7f26701076c0 2024-09-16T01:30:39.045 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.325+0000 7f26757d8640 1 -- 172.21.15.139:0/2170312120 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267019be80 con 0x7f26701076c0 2024-09-16T01:30:39.045 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.325+0000 7f26757d8640 1 -- 172.21.15.139:0/2170312120 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267019eb70 con 0x7f26701076c0 2024-09-16T01:30:39.045 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.326+0000 7f2657fff640 1 -- 172.21.15.139:0/2170312120 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f2660045070 con 0x7f26701076c0 2024-09-16T01:30:39.045 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.326+0000 7f2657fff640 1 --2- 172.21.15.139:0/2170312120 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2658039160 0x7f265803b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:39.045 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.326+0000 7f2657fff640 1 -- 172.21.15.139:0/2170312120 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f266006c430 con 0x7f26701076c0 2024-09-16T01:30:39.045 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.326+0000 7f26757d8640 1 -- 172.21.15.139:0/2170312120 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267010bdf0 con 0x7f26701076c0 2024-09-16T01:30:39.045 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.326+0000 7f266e7fc640 1 --2- 172.21.15.139:0/2170312120 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2658039160 0x7f265803b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:39.045 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.327+0000 7f266e7fc640 1 --2- 172.21.15.139:0/2170312120 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2658039160 0x7f265803b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f26640098c0 tx=0x7f2664006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:39.045 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.331+0000 7f2657fff640 1 -- 172.21.15.139:0/2170312120 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f266006be50 con 0x7f26701076c0 2024-09-16T01:30:39.045 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.496+0000 7f26757d8640 1 -- 172.21.15.139:0/2170312120 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f2670107b40 con 0x7f26701076c0 2024-09-16T01:30:39.045 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.497+0000 7f2657fff640 1 -- 172.21.15.139:0/2170312120 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f266003cd90 con 0x7f26701076c0 2024-09-16T01:30:39.046 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.500+0000 7f26757d8640 1 -- 172.21.15.139:0/2170312120 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2658039160 msgr2=0x7f265803b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:39.046 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.500+0000 7f26757d8640 1 --2- 172.21.15.139:0/2170312120 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2658039160 0x7f265803b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f26640098c0 tx=0x7f2664006db0 comp rx=0 tx=0).stop 2024-09-16T01:30:39.046 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.500+0000 7f26757d8640 1 -- 172.21.15.139:0/2170312120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26701076c0 msgr2=0x7f267019b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:39.046 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.500+0000 7f26757d8640 1 --2- 172.21.15.139:0/2170312120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26701076c0 0x7f267019b6b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2660009940 tx=0x7f2660032ce0 comp rx=0 tx=0).stop 2024-09-16T01:30:39.046 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.500+0000 7f26757d8640 1 -- 172.21.15.139:0/2170312120 shutdown_connections 2024-09-16T01:30:39.046 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.500+0000 7f26757d8640 1 --2- 172.21.15.139:0/2170312120 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2658039160 0x7f265803b620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:39.046 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.500+0000 7f26757d8640 1 --2- 172.21.15.139:0/2170312120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26701076c0 0x7f267019b6b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:39.046 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.500+0000 7f26757d8640 1 -- 172.21.15.139:0/2170312120 >> 172.21.15.139:0/2170312120 conn(0x7f2670102e50 msgr2=0x7f2670103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:39.046 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.500+0000 7f26757d8640 1 -- 172.21.15.139:0/2170312120 shutdown_connections 2024-09-16T01:30:39.046 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:38.500+0000 7f26757d8640 1 -- 172.21.15.139:0/2170312120 wait complete. 2024-09-16T01:30:39.046 INFO:teuthology.orchestra.run.smithi139.stdout:Enabling cephadm module... 2024-09-16T01:30:39.255 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:38 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2170312120' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T01:30:40.274 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:39 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1124815848' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-16T01:30:40.274 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:39 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: ignoring --setuser ceph since I am not root 2024-09-16T01:30:40.274 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:39 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: ignoring --setgroup ceph since I am not root 2024-09-16T01:30:40.274 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:40 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:40.071+0000 7f50884b1040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:30:40.275 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:40 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:40.127+0000 7f50884b1040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:30:40.275 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:40 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:40.273+0000 7f50884b1040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:30:40.589 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:40 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:40.341+0000 7f50884b1040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:30:40.589 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:40 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:40.485+0000 7f50884b1040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:30:40.596 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.430+0000 7fb588bf2640 1 Processor -- start 2024-09-16T01:30:40.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.430+0000 7fb588bf2640 1 -- start start 2024-09-16T01:30:40.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.430+0000 7fb588bf2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5841065f0 0x7fb5841069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:40.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.430+0000 7fb588bf2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb584106fc0 con 0x7fb5841065f0 2024-09-16T01:30:40.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.431+0000 7fb58259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5841065f0 0x7fb5841069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:40.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.431+0000 7fb58259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5841065f0 0x7fb5841069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45760/0 (socket says 172.21.15.139:45760) 2024-09-16T01:30:40.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.431+0000 7fb58259c640 1 -- 172.21.15.139:0/2005678971 learned_addr learned my addr 172.21.15.139:0/2005678971 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:40.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.431+0000 7fb58259c640 1 -- 172.21.15.139:0/2005678971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5841077f0 con 0x7fb5841065f0 2024-09-16T01:30:40.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.432+0000 7fb58259c640 1 --2- 172.21.15.139:0/2005678971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5841065f0 0x7fb5841069f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb574009820 tx=0x7fb57402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767f18c74a6d01f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:40.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.432+0000 7fb58159a640 1 -- 172.21.15.139:0/2005678971 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb574037070 con 0x7fb5841065f0 2024-09-16T01:30:40.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.432+0000 7fb58159a640 1 -- 172.21.15.139:0/2005678971 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb574032900 con 0x7fb5841065f0 2024-09-16T01:30:40.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.432+0000 7fb58159a640 1 -- 172.21.15.139:0/2005678971 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb574030850 con 0x7fb5841065f0 2024-09-16T01:30:40.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.432+0000 7fb588bf2640 1 -- 172.21.15.139:0/2005678971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5841065f0 msgr2=0x7fb5841069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:40.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.432+0000 7fb588bf2640 1 --2- 172.21.15.139:0/2005678971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5841065f0 0x7fb5841069f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb574009820 tx=0x7fb57402a270 comp rx=0 tx=0).stop 2024-09-16T01:30:40.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.432+0000 7fb588bf2640 1 -- 172.21.15.139:0/2005678971 shutdown_connections 2024-09-16T01:30:40.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.432+0000 7fb588bf2640 1 --2- 172.21.15.139:0/2005678971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5841065f0 0x7fb5841069f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:40.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.432+0000 7fb588bf2640 1 -- 172.21.15.139:0/2005678971 >> 172.21.15.139:0/2005678971 conn(0x7fb584101e20 msgr2=0x7fb584104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:40.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.432+0000 7fb588bf2640 1 -- 172.21.15.139:0/2005678971 shutdown_connections 2024-09-16T01:30:40.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.432+0000 7fb588bf2640 1 -- 172.21.15.139:0/2005678971 wait complete. 2024-09-16T01:30:40.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.433+0000 7fb588bf2640 1 Processor -- start 2024-09-16T01:30:40.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.433+0000 7fb588bf2640 1 -- start start 2024-09-16T01:30:40.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.433+0000 7fb588bf2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5841065f0 0x7fb584191820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:40.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.433+0000 7fb588bf2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb58410aba0 con 0x7fb5841065f0 2024-09-16T01:30:40.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.433+0000 7fb58259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5841065f0 0x7fb584191820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:40.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.433+0000 7fb58259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5841065f0 0x7fb584191820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45764/0 (socket says 172.21.15.139:45764) 2024-09-16T01:30:40.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.433+0000 7fb58259c640 1 -- 172.21.15.139:0/1124815848 learned_addr learned my addr 172.21.15.139:0/1124815848 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:40.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.434+0000 7fb58259c640 1 -- 172.21.15.139:0/1124815848 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb584191d60 con 0x7fb5841065f0 2024-09-16T01:30:40.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.434+0000 7fb58259c640 1 --2- 172.21.15.139:0/1124815848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5841065f0 0x7fb584191820 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb574009940 tx=0x7fb5740326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:40.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.434+0000 7fb5737fe640 1 -- 172.21.15.139:0/1124815848 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb574042070 con 0x7fb5841065f0 2024-09-16T01:30:40.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.434+0000 7fb5737fe640 1 -- 172.21.15.139:0/1124815848 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb574032be0 con 0x7fb5841065f0 2024-09-16T01:30:40.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.434+0000 7fb588bf2640 1 -- 172.21.15.139:0/1124815848 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58418ff50 con 0x7fb5841065f0 2024-09-16T01:30:40.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.434+0000 7fb5737fe640 1 -- 172.21.15.139:0/1124815848 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb574037040 con 0x7fb5841065f0 2024-09-16T01:30:40.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.434+0000 7fb588bf2640 1 -- 172.21.15.139:0/1124815848 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb584190490 con 0x7fb5841065f0 2024-09-16T01:30:40.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.435+0000 7fb5737fe640 1 -- 172.21.15.139:0/1124815848 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fb574045070 con 0x7fb5841065f0 2024-09-16T01:30:40.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.435+0000 7fb588bf2640 1 -- 172.21.15.139:0/1124815848 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb58410ad20 con 0x7fb5841065f0 2024-09-16T01:30:40.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.435+0000 7fb5737fe640 1 --2- 172.21.15.139:0/1124815848 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7fb56c039160 0x7fb56c03b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:40.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.435+0000 7fb5737fe640 1 -- 172.21.15.139:0/1124815848 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb57406c1f0 con 0x7fb5841065f0 2024-09-16T01:30:40.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.435+0000 7fb581d9b640 1 --2- 172.21.15.139:0/1124815848 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7fb56c039160 0x7fb56c03b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:40.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.436+0000 7fb581d9b640 1 --2- 172.21.15.139:0/1124815848 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7fb56c039160 0x7fb56c03b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb5780098c0 tx=0x7fb578006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:40.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.440+0000 7fb5737fe640 1 -- 172.21.15.139:0/1124815848 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb57403be10 con 0x7fb5841065f0 2024-09-16T01:30:40.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.650+0000 7fb588bf2640 1 -- 172.21.15.139:0/1124815848 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fb5840008d0 con 0x7fb5841065f0 2024-09-16T01:30:40.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.935+0000 7fb5737fe640 1 -- 172.21.15.139:0/1124815848 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fb57403cd20 con 0x7fb5841065f0 2024-09-16T01:30:40.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.937+0000 7fb5737fe640 1 -- 172.21.15.139:0/1124815848 <== mon.0 v2:172.21.15.139:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fb57403b510 con 0x7fb5841065f0 2024-09-16T01:30:40.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.938+0000 7fb588bf2640 1 -- 172.21.15.139:0/1124815848 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7fb56c039160 msgr2=0x7fb56c03b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:40.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.938+0000 7fb588bf2640 1 --2- 172.21.15.139:0/1124815848 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7fb56c039160 0x7fb56c03b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb5780098c0 tx=0x7fb578006db0 comp rx=0 tx=0).stop 2024-09-16T01:30:40.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.938+0000 7fb588bf2640 1 -- 172.21.15.139:0/1124815848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5841065f0 msgr2=0x7fb584191820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:40.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.938+0000 7fb588bf2640 1 --2- 172.21.15.139:0/1124815848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5841065f0 0x7fb584191820 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb574009940 tx=0x7fb5740326e0 comp rx=0 tx=0).stop 2024-09-16T01:30:40.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.938+0000 7fb588bf2640 1 -- 172.21.15.139:0/1124815848 shutdown_connections 2024-09-16T01:30:40.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.939+0000 7fb588bf2640 1 --2- 172.21.15.139:0/1124815848 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7fb56c039160 0x7fb56c03b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:40.601 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.939+0000 7fb588bf2640 1 --2- 172.21.15.139:0/1124815848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5841065f0 0x7fb584191820 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:40.601 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.939+0000 7fb588bf2640 1 -- 172.21.15.139:0/1124815848 >> 172.21.15.139:0/1124815848 conn(0x7fb584101e20 msgr2=0x7fb584102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:40.601 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.939+0000 7fb588bf2640 1 -- 172.21.15.139:0/1124815848 shutdown_connections 2024-09-16T01:30:40.601 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:39.939+0000 7fb588bf2640 1 -- 172.21.15.139:0/1124815848 wait complete. 2024-09-16T01:30:40.916 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:40 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:40.629+0000 7f50884b1040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:30:41.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:40 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1124815848' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T01:30:41.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:40 smithi139 ceph-mon[31175]: mgrmap e5: x(active, since 5s) 2024-09-16T01:30:41.173 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:40 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:40.948+0000 7f50884b1040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:30:41.173 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:41 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:41.013+0000 7f50884b1040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:30:41.586 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:41 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:41.204+0000 7f50884b1040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:30:41.895 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:30:41.895 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T01:30:41.895 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T01:30:41.895 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-16T01:30:41.895 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T01:30:41.895 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:30:41.895 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.993+0000 7f2457849640 1 Processor -- start 2024-09-16T01:30:41.895 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.994+0000 7f2457849640 1 -- start start 2024-09-16T01:30:41.895 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.994+0000 7f2457849640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2450106610 0x7f2450106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:41.895 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.994+0000 7f2457849640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2450106fe0 con 0x7f2450106610 2024-09-16T01:30:41.895 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.994+0000 7f24555e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2450106610 0x7f2450106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:41.896 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.994+0000 7f24555e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2450106610 0x7f2450106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45802/0 (socket says 172.21.15.139:45802) 2024-09-16T01:30:41.896 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.994+0000 7f24555e5640 1 -- 172.21.15.139:0/630112762 learned_addr learned my addr 172.21.15.139:0/630112762 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:41.896 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.994+0000 7f24555e5640 1 -- 172.21.15.139:0/630112762 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2450107810 con 0x7f2450106610 2024-09-16T01:30:41.896 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.995+0000 7f24555e5640 1 --2- 172.21.15.139:0/630112762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2450106610 0x7f2450106a10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f244c009820 tx=0x7f244c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2d8988328b5bfc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:41.896 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.995+0000 7f2447fff640 1 -- 172.21.15.139:0/630112762 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f244c037070 con 0x7f2450106610 2024-09-16T01:30:41.896 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.995+0000 7f2447fff640 1 -- 172.21.15.139:0/630112762 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f244c032900 con 0x7f2450106610 2024-09-16T01:30:41.896 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.995+0000 7f2447fff640 1 -- 172.21.15.139:0/630112762 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f244c030850 con 0x7f2450106610 2024-09-16T01:30:41.896 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.996+0000 7f2457849640 1 -- 172.21.15.139:0/630112762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2450106610 msgr2=0x7f2450106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:41.896 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.996+0000 7f2457849640 1 --2- 172.21.15.139:0/630112762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2450106610 0x7f2450106a10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f244c009820 tx=0x7f244c02a270 comp rx=0 tx=0).stop 2024-09-16T01:30:41.896 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.996+0000 7f2457849640 1 -- 172.21.15.139:0/630112762 shutdown_connections 2024-09-16T01:30:41.896 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.996+0000 7f2457849640 1 --2- 172.21.15.139:0/630112762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2450106610 0x7f2450106a10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:41.897 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.996+0000 7f2457849640 1 -- 172.21.15.139:0/630112762 >> 172.21.15.139:0/630112762 conn(0x7f2450101da0 msgr2=0x7f24501041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:41.897 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.996+0000 7f2457849640 1 -- 172.21.15.139:0/630112762 shutdown_connections 2024-09-16T01:30:41.897 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.996+0000 7f2457849640 1 -- 172.21.15.139:0/630112762 wait complete. 2024-09-16T01:30:41.897 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.996+0000 7f2457849640 1 Processor -- start 2024-09-16T01:30:41.897 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.996+0000 7f2457849640 1 -- start start 2024-09-16T01:30:41.897 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.997+0000 7f2457849640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2450106610 0x7f245019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:41.897 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.997+0000 7f2457849640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f245010abc0 con 0x7f2450106610 2024-09-16T01:30:41.897 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.997+0000 7f24555e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2450106610 0x7f245019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:41.897 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.997+0000 7f24555e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2450106610 0x7f245019b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45812/0 (socket says 172.21.15.139:45812) 2024-09-16T01:30:41.897 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.997+0000 7f24555e5640 1 -- 172.21.15.139:0/2086790996 learned_addr learned my addr 172.21.15.139:0/2086790996 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:41.897 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.997+0000 7f24555e5640 1 -- 172.21.15.139:0/2086790996 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245019bcb0 con 0x7f2450106610 2024-09-16T01:30:41.897 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.997+0000 7f24555e5640 1 --2- 172.21.15.139:0/2086790996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2450106610 0x7f245019b770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f244c009940 tx=0x7f244c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:41.897 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.997+0000 7f24467fc640 1 -- 172.21.15.139:0/2086790996 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f244c042070 con 0x7f2450106610 2024-09-16T01:30:41.898 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.997+0000 7f2457849640 1 -- 172.21.15.139:0/2086790996 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245019bf40 con 0x7f2450106610 2024-09-16T01:30:41.898 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.997+0000 7f24467fc640 1 -- 172.21.15.139:0/2086790996 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f244c032be0 con 0x7f2450106610 2024-09-16T01:30:41.898 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.997+0000 7f24467fc640 1 -- 172.21.15.139:0/2086790996 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f244c037040 con 0x7f2450106610 2024-09-16T01:30:41.898 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.997+0000 7f2457849640 1 -- 172.21.15.139:0/2086790996 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245019ec30 con 0x7f2450106610 2024-09-16T01:30:41.898 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.998+0000 7f24467fc640 1 -- 172.21.15.139:0/2086790996 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f244c045070 con 0x7f2450106610 2024-09-16T01:30:41.898 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.998+0000 7f2457849640 1 -- 172.21.15.139:0/2086790996 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245010ad40 con 0x7f2450106610 2024-09-16T01:30:41.898 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.999+0000 7f24467fc640 1 --2- 172.21.15.139:0/2086790996 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f243c0391b0 0x7f243c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:41.898 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.999+0000 7f24467fc640 1 -- 172.21.15.139:0/2086790996 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f244c06c290 con 0x7f2450106610 2024-09-16T01:30:41.898 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.999+0000 7f2454de4640 1 -- 172.21.15.139:0/2086790996 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f243c0391b0 msgr2=0x7f243c03b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/2853057507 2024-09-16T01:30:41.898 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:40.999+0000 7f2454de4640 1 --2- 172.21.15.139:0/2086790996 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f243c0391b0 0x7f243c03b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:30:41.898 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.003+0000 7f24467fc640 1 -- 172.21.15.139:0/2086790996 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f244c045e10 con 0x7f2450106610 2024-09-16T01:30:41.898 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.200+0000 7f2454de4640 1 -- 172.21.15.139:0/2086790996 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f243c0391b0 msgr2=0x7f243c03b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/2853057507 2024-09-16T01:30:41.898 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.200+0000 7f2454de4640 1 --2- 172.21.15.139:0/2086790996 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f243c0391b0 0x7f243c03b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:30:41.898 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.222+0000 7f2457849640 1 -- 172.21.15.139:0/2086790996 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f245002da30 con 0x7f2450106610 2024-09-16T01:30:41.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.222+0000 7f24467fc640 1 -- 172.21.15.139:0/2086790996 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f244c03b330 con 0x7f2450106610 2024-09-16T01:30:41.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.225+0000 7f2457849640 1 -- 172.21.15.139:0/2086790996 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f243c0391b0 msgr2=0x7f243c03b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:30:41.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.225+0000 7f2457849640 1 --2- 172.21.15.139:0/2086790996 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f243c0391b0 0x7f243c03b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:41.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.225+0000 7f2457849640 1 -- 172.21.15.139:0/2086790996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2450106610 msgr2=0x7f245019b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:41.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.225+0000 7f2457849640 1 --2- 172.21.15.139:0/2086790996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2450106610 0x7f245019b770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f244c009940 tx=0x7f244c0326e0 comp rx=0 tx=0).stop 2024-09-16T01:30:41.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.225+0000 7f2457849640 1 -- 172.21.15.139:0/2086790996 shutdown_connections 2024-09-16T01:30:41.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.225+0000 7f2457849640 1 --2- 172.21.15.139:0/2086790996 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f243c0391b0 0x7f243c03b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:41.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.225+0000 7f2457849640 1 --2- 172.21.15.139:0/2086790996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2450106610 0x7f245019b770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:41.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.225+0000 7f2457849640 1 -- 172.21.15.139:0/2086790996 >> 172.21.15.139:0/2086790996 conn(0x7f2450101da0 msgr2=0x7f2450102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:41.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.225+0000 7f2457849640 1 -- 172.21.15.139:0/2086790996 shutdown_connections 2024-09-16T01:30:41.899 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:41.225+0000 7f2457849640 1 -- 172.21.15.139:0/2086790996 wait complete. 2024-09-16T01:30:41.899 INFO:teuthology.orchestra.run.smithi139.stdout:Waiting for the mgr to restart... 2024-09-16T01:30:41.899 INFO:teuthology.orchestra.run.smithi139.stdout:Waiting for mgr epoch 5... 2024-09-16T01:30:42.170 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:41 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2086790996' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T01:30:42.171 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:42.097+0000 7f50884b1040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:30:42.171 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:42.169+0000 7f50884b1040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:30:43.132 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:42.874+0000 7f50884b1040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:30:43.132 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:42.990+0000 7f50884b1040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:30:43.406 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:43.129+0000 7f50884b1040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:30:43.406 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:43.189+0000 7f50884b1040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:30:43.406 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:43.337+0000 7f50884b1040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:30:43.406 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:43.404+0000 7f50884b1040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:30:43.672 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:43.463+0000 7f50884b1040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:30:43.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:43.617+0000 7f50884b1040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:30:43.947 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:43.682+0000 7f50884b1040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:30:43.947 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:43.748+0000 7f50884b1040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:30:43.947 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:43.829+0000 7f50884b1040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:30:43.948 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:43.887+0000 7f50884b1040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:30:43.948 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:43.945+0000 7f50884b1040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:30:44.922 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:44.748+0000 7f50884b1040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:30:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: Active manager daemon x restarted 2024-09-16T01:30:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: Activating manager daemon x 2024-09-16T01:30:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T01:30:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: mgrmap e6: x(active, starting, since 0.00346387s) 2024-09-16T01:30:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:30:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T01:30:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T01:30:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T01:30:44.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T01:30:44.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: Manager daemon x is now available 2024-09-16T01:30:44.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' 2024-09-16T01:30:44.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' 2024-09-16T01:30:44.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:30:44.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:30:44.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T01:30:44.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:44 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T01:30:46.067 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:45 smithi139 ceph-mon[31175]: Found migration_current of "None". Setting to last migration. 2024-09-16T01:30:46.067 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:45 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' 2024-09-16T01:30:46.067 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:45 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' 2024-09-16T01:30:46.068 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:45 smithi139 ceph-mon[31175]: mgrmap e7: x(active, since 1.00563s) 2024-09-16T01:30:46.343 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:30:46.343 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-16T01:30:46.343 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T01:30:46.343 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:30:46.343 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.308+0000 7f2cacdee640 1 Processor -- start 2024-09-16T01:30:46.343 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.308+0000 7f2cacdee640 1 -- start start 2024-09-16T01:30:46.343 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.308+0000 7f2cacdee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ca8105fc0 0x7f2ca81063c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:46.344 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.308+0000 7f2cacdee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca807be20 con 0x7f2ca8105fc0 2024-09-16T01:30:46.344 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.308+0000 7f2ca659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ca8105fc0 0x7f2ca81063c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:46.344 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.308+0000 7f2ca659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ca8105fc0 0x7f2ca81063c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53690/0 (socket says 172.21.15.139:53690) 2024-09-16T01:30:46.344 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.308+0000 7f2ca659c640 1 -- 172.21.15.139:0/1735878399 learned_addr learned my addr 172.21.15.139:0/1735878399 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:46.344 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.309+0000 7f2ca659c640 1 -- 172.21.15.139:0/1735878399 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca8106900 con 0x7f2ca8105fc0 2024-09-16T01:30:46.344 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.310+0000 7f2ca659c640 1 --2- 172.21.15.139:0/1735878399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ca8105fc0 0x7f2ca81063c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009820 tx=0x7f2c9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3034ebc5041ec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:46.344 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.310+0000 7f2ca559a640 1 -- 172.21.15.139:0/1735878399 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c98037070 con 0x7f2ca8105fc0 2024-09-16T01:30:46.344 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.310+0000 7f2ca559a640 1 -- 172.21.15.139:0/1735878399 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c98032900 con 0x7f2ca8105fc0 2024-09-16T01:30:46.344 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.310+0000 7f2ca559a640 1 -- 172.21.15.139:0/1735878399 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c98030850 con 0x7f2ca8105fc0 2024-09-16T01:30:46.344 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.310+0000 7f2cacdee640 1 -- 172.21.15.139:0/1735878399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ca8105fc0 msgr2=0x7f2ca81063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:46.344 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.310+0000 7f2cacdee640 1 --2- 172.21.15.139:0/1735878399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ca8105fc0 0x7f2ca81063c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009820 tx=0x7f2c9802a270 comp rx=0 tx=0).stop 2024-09-16T01:30:46.344 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.311+0000 7f2cacdee640 1 -- 172.21.15.139:0/1735878399 shutdown_connections 2024-09-16T01:30:46.345 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.311+0000 7f2cacdee640 1 --2- 172.21.15.139:0/1735878399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ca8105fc0 0x7f2ca81063c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:46.345 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.311+0000 7f2cacdee640 1 -- 172.21.15.139:0/1735878399 >> 172.21.15.139:0/1735878399 conn(0x7f2ca8101db0 msgr2=0x7f2ca81041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:46.345 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.311+0000 7f2cacdee640 1 -- 172.21.15.139:0/1735878399 shutdown_connections 2024-09-16T01:30:46.345 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.311+0000 7f2cacdee640 1 -- 172.21.15.139:0/1735878399 wait complete. 2024-09-16T01:30:46.345 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.311+0000 7f2cacdee640 1 Processor -- start 2024-09-16T01:30:46.345 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.311+0000 7f2cacdee640 1 -- start start 2024-09-16T01:30:46.345 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.312+0000 7f2cacdee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ca8105fc0 0x7f2ca819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:46.345 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.312+0000 7f2cacdee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8107a00 con 0x7f2ca8105fc0 2024-09-16T01:30:46.345 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.312+0000 7f2ca659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ca8105fc0 0x7f2ca819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:46.345 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.312+0000 7f2ca659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ca8105fc0 0x7f2ca819d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53696/0 (socket says 172.21.15.139:53696) 2024-09-16T01:30:46.345 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.312+0000 7f2ca659c640 1 -- 172.21.15.139:0/3508373267 learned_addr learned my addr 172.21.15.139:0/3508373267 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:46.345 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.312+0000 7f2ca659c640 1 -- 172.21.15.139:0/3508373267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca819de20 con 0x7f2ca8105fc0 2024-09-16T01:30:46.345 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.312+0000 7f2ca659c640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ca8105fc0 0x7f2ca819d8e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2c98031000 tx=0x7f2c980326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:46.345 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.312+0000 7f2c977fe640 1 -- 172.21.15.139:0/3508373267 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c98042070 con 0x7f2ca8105fc0 2024-09-16T01:30:46.346 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.312+0000 7f2c977fe640 1 -- 172.21.15.139:0/3508373267 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c98032be0 con 0x7f2ca8105fc0 2024-09-16T01:30:46.346 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.312+0000 7f2cacdee640 1 -- 172.21.15.139:0/3508373267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca819e0b0 con 0x7f2ca8105fc0 2024-09-16T01:30:46.346 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.312+0000 7f2c977fe640 1 -- 172.21.15.139:0/3508373267 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c98037040 con 0x7f2ca8105fc0 2024-09-16T01:30:46.346 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.312+0000 7f2cacdee640 1 -- 172.21.15.139:0/3508373267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca81a0da0 con 0x7f2ca8105fc0 2024-09-16T01:30:46.346 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.313+0000 7f2c977fe640 1 -- 172.21.15.139:0/3508373267 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f2c98045070 con 0x7f2ca8105fc0 2024-09-16T01:30:46.346 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.314+0000 7f2c977fe640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2c900391b0 0x7f2c9003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:46.346 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.314+0000 7f2c977fe640 1 -- 172.21.15.139:0/3508373267 --> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2c9003bd80 con 0x7f2c900391b0 2024-09-16T01:30:46.346 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.314+0000 7f2c977fe640 1 -- 172.21.15.139:0/3508373267 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2c9806c240 con 0x7f2ca8105fc0 2024-09-16T01:30:46.346 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.314+0000 7f2ca5d9b640 1 -- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2c900391b0 msgr2=0x7f2c9003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/2853057507 2024-09-16T01:30:46.346 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.314+0000 7f2ca5d9b640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2c900391b0 0x7f2c9003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:30:46.346 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.515+0000 7f2ca5d9b640 1 -- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2c900391b0 msgr2=0x7f2c9003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/2853057507 2024-09-16T01:30:46.346 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.515+0000 7f2ca5d9b640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2c900391b0 0x7f2c9003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:30:46.346 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.916+0000 7f2ca5d9b640 1 -- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2c900391b0 msgr2=0x7f2c9003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/2853057507 2024-09-16T01:30:46.347 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:42.916+0000 7f2ca5d9b640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2c900391b0 0x7f2c9003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T01:30:46.347 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:43.717+0000 7f2ca5d9b640 1 -- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2c900391b0 msgr2=0x7f2c9003b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/2853057507 2024-09-16T01:30:46.347 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:43.717+0000 7f2ca5d9b640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2c900391b0 0x7f2c9003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T01:30:46.347 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:44.752+0000 7f2c977fe640 1 -- 172.21.15.139:0/3508373267 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f2c9806adc0 con 0x7f2ca8105fc0 2024-09-16T01:30:46.347 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:44.752+0000 7f2c977fe640 1 -- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2c900391b0 msgr2=0x7f2c9003b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:30:46.347 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:44.752+0000 7f2c977fe640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2c900391b0 0x7f2c9003b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:46.347 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.753+0000 7f2c977fe640 1 -- 172.21.15.139:0/3508373267 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f2c9802ed80 con 0x7f2ca8105fc0 2024-09-16T01:30:46.347 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.753+0000 7f2c977fe640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f2c9003cae0 0x7f2c9003eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:46.347 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.753+0000 7f2c977fe640 1 -- 172.21.15.139:0/3508373267 --> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2c9802bba0 con 0x7f2c9003cae0 2024-09-16T01:30:46.347 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.754+0000 7f2ca5d9b640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f2c9003cae0 0x7f2c9003eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:46.347 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.754+0000 7f2ca5d9b640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f2c9003cae0 0x7f2c9003eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c003980 tx=0x7f2c9c006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:46.347 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.755+0000 7f2c977fe640 1 -- 172.21.15.139:0/3508373267 <== mgr.14118 v2:172.21.15.139:6800/1485008061 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f2c9802bba0 con 0x7f2c9003cae0 2024-09-16T01:30:46.347 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.767+0000 7f2cacdee640 1 -- 172.21.15.139:0/3508373267 --> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f2ca81063c0 con 0x7f2c9003cae0 2024-09-16T01:30:46.347 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.767+0000 7f2c977fe640 1 -- 172.21.15.139:0/3508373267 <== mgr.14118 v2:172.21.15.139:6800/1485008061 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f2ca81063c0 con 0x7f2c9003cae0 2024-09-16T01:30:46.348 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.767+0000 7f2cacdee640 1 -- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f2c9003cae0 msgr2=0x7f2c9003eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:46.348 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.767+0000 7f2cacdee640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f2c9003cae0 0x7f2c9003eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c003980 tx=0x7f2c9c006e10 comp rx=0 tx=0).stop 2024-09-16T01:30:46.348 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.767+0000 7f2cacdee640 1 -- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ca8105fc0 msgr2=0x7f2ca819d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:46.348 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.767+0000 7f2cacdee640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ca8105fc0 0x7f2ca819d8e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2c98031000 tx=0x7f2c980326e0 comp rx=0 tx=0).stop 2024-09-16T01:30:46.348 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.768+0000 7f2cacdee640 1 -- 172.21.15.139:0/3508373267 shutdown_connections 2024-09-16T01:30:46.348 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.768+0000 7f2cacdee640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/2853057507,v1:172.21.15.139:6801/2853057507] conn(0x7f2c900391b0 0x7f2c9003b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:46.348 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.768+0000 7f2cacdee640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f2c9003cae0 0x7f2c9003eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:46.348 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.768+0000 7f2cacdee640 1 --2- 172.21.15.139:0/3508373267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ca8105fc0 0x7f2ca819d8e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:46.348 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.768+0000 7f2cacdee640 1 -- 172.21.15.139:0/3508373267 >> 172.21.15.139:0/3508373267 conn(0x7f2ca8101db0 msgr2=0x7f2ca8191be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:46.348 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.768+0000 7f2cacdee640 1 -- 172.21.15.139:0/3508373267 shutdown_connections 2024-09-16T01:30:46.348 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:45.768+0000 7f2cacdee640 1 -- 172.21.15.139:0/3508373267 wait complete. 2024-09-16T01:30:46.348 INFO:teuthology.orchestra.run.smithi139.stdout:mgr epoch 5 is available 2024-09-16T01:30:46.348 INFO:teuthology.orchestra.run.smithi139.stdout:Setting orchestrator backend to cephadm... 2024-09-16T01:30:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:46 smithi139 ceph-mon[31175]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T01:30:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:46 smithi139 ceph-mon[31175]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T01:30:47.549 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:47.448+0000 7f4ff4fd6640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:01:30:47] ENGINE Error in HTTPServer.serve 2024-09-16T01:30:47.550 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: Traceback (most recent call last): 2024-09-16T01:30:47.550 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:30:47.550 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self._connections.run(self.expiration_interval) 2024-09-16T01:30:47.550 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:30:47.550 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self._run(expiration_interval) 2024-09-16T01:30:47.550 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:30:47.550 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:30:47.550 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:30:47.550 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:30:47.550 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:30:47.550 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: s = self.context.wrap_socket( 2024-09-16T01:30:47.550 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:30:47.550 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: return self.sslsocket_class._create( 2024-09-16T01:30:47.551 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:30:47.551 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self.do_handshake() 2024-09-16T01:30:47.551 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:30:47.551 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self._sslobj.do_handshake() 2024-09-16T01:30:47.551 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:30:47.551 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:30:47.556 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.746+0000 7f0bb752c640 1 Processor -- start 2024-09-16T01:30:47.556 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.746+0000 7f0bb752c640 1 -- start start 2024-09-16T01:30:47.556 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.747+0000 7f0bb752c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bb01081e0 0x7f0bb010a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:47.557 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.747+0000 7f0bb752c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb007be30 con 0x7f0bb01081e0 2024-09-16T01:30:47.557 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.747+0000 7f0bb52c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bb01081e0 0x7f0bb010a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:47.557 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.747+0000 7f0bb52c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bb01081e0 0x7f0bb010a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53782/0 (socket says 172.21.15.139:53782) 2024-09-16T01:30:47.557 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.747+0000 7f0bb52c8640 1 -- 172.21.15.139:0/604193090 learned_addr learned my addr 172.21.15.139:0/604193090 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:47.557 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.747+0000 7f0bb52c8640 1 -- 172.21.15.139:0/604193090 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb010ab10 con 0x7f0bb01081e0 2024-09-16T01:30:47.557 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.748+0000 7f0bb52c8640 1 --2- 172.21.15.139:0/604193090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bb01081e0 0x7f0bb010a5d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009820 tx=0x7f0bac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e130920cf2ee3d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:47.557 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.748+0000 7f0ba7fff640 1 -- 172.21.15.139:0/604193090 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bac037070 con 0x7f0bb01081e0 2024-09-16T01:30:47.557 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.748+0000 7f0ba7fff640 1 -- 172.21.15.139:0/604193090 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0bac032900 con 0x7f0bb01081e0 2024-09-16T01:30:47.557 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.748+0000 7f0ba7fff640 1 -- 172.21.15.139:0/604193090 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bac030850 con 0x7f0bb01081e0 2024-09-16T01:30:47.557 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.748+0000 7f0bb752c640 1 -- 172.21.15.139:0/604193090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bb01081e0 msgr2=0x7f0bb010a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:47.557 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.748+0000 7f0bb752c640 1 --2- 172.21.15.139:0/604193090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bb01081e0 0x7f0bb010a5d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009820 tx=0x7f0bac02a270 comp rx=0 tx=0).stop 2024-09-16T01:30:47.557 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.748+0000 7f0bb752c640 1 -- 172.21.15.139:0/604193090 shutdown_connections 2024-09-16T01:30:47.558 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.748+0000 7f0bb752c640 1 --2- 172.21.15.139:0/604193090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bb01081e0 0x7f0bb010a5d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:47.558 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.749+0000 7f0bb752c640 1 -- 172.21.15.139:0/604193090 >> 172.21.15.139:0/604193090 conn(0x7f0bb0101e20 msgr2=0x7f0bb0104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:47.558 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.749+0000 7f0bb752c640 1 -- 172.21.15.139:0/604193090 shutdown_connections 2024-09-16T01:30:47.558 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.749+0000 7f0bb752c640 1 -- 172.21.15.139:0/604193090 wait complete. 2024-09-16T01:30:47.558 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.749+0000 7f0bb752c640 1 Processor -- start 2024-09-16T01:30:47.558 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.749+0000 7f0bb752c640 1 -- start start 2024-09-16T01:30:47.558 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.749+0000 7f0bb752c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bb01081e0 0x7f0bb0107a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:47.558 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.749+0000 7f0bb752c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb010baf0 con 0x7f0bb01081e0 2024-09-16T01:30:47.558 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.749+0000 7f0bb52c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bb01081e0 0x7f0bb0107a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:47.558 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.749+0000 7f0bb52c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bb01081e0 0x7f0bb0107a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53784/0 (socket says 172.21.15.139:53784) 2024-09-16T01:30:47.558 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.749+0000 7f0bb52c8640 1 -- 172.21.15.139:0/2914753049 learned_addr learned my addr 172.21.15.139:0/2914753049 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:47.558 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.750+0000 7f0bb52c8640 1 -- 172.21.15.139:0/2914753049 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb0107fa0 con 0x7f0bb01081e0 2024-09-16T01:30:47.558 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.750+0000 7f0bb52c8640 1 --2- 172.21.15.139:0/2914753049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bb01081e0 0x7f0bb0107a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0bac031000 tx=0x7f0bac0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:47.558 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.750+0000 7f0ba67fc640 1 -- 172.21.15.139:0/2914753049 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bac042070 con 0x7f0bb01081e0 2024-09-16T01:30:47.559 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.750+0000 7f0bb752c640 1 -- 172.21.15.139:0/2914753049 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb0106190 con 0x7f0bb01081e0 2024-09-16T01:30:47.559 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.750+0000 7f0ba67fc640 1 -- 172.21.15.139:0/2914753049 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0bac032be0 con 0x7f0bb01081e0 2024-09-16T01:30:47.559 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.750+0000 7f0ba67fc640 1 -- 172.21.15.139:0/2914753049 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bac037040 con 0x7f0bb01081e0 2024-09-16T01:30:47.559 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.750+0000 7f0bb752c640 1 -- 172.21.15.139:0/2914753049 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb01066d0 con 0x7f0bb01081e0 2024-09-16T01:30:47.559 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.751+0000 7f0bb752c640 1 -- 172.21.15.139:0/2914753049 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb0108640 con 0x7f0bb01081e0 2024-09-16T01:30:47.559 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.752+0000 7f0ba67fc640 1 -- 172.21.15.139:0/2914753049 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f0bac045070 con 0x7f0bb01081e0 2024-09-16T01:30:47.559 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.752+0000 7f0ba67fc640 1 --2- 172.21.15.139:0/2914753049 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f0b9c0390a0 0x7f0b9c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:47.559 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.752+0000 7f0ba67fc640 1 -- 172.21.15.139:0/2914753049 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0bac06cbc0 con 0x7f0bb01081e0 2024-09-16T01:30:47.559 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.752+0000 7f0bb4ac7640 1 --2- 172.21.15.139:0/2914753049 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f0b9c0390a0 0x7f0b9c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:47.559 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.753+0000 7f0bb4ac7640 1 --2- 172.21.15.139:0/2914753049 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f0b9c0390a0 0x7f0b9c03b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00098c0 tx=0x7f0ba0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:47.559 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.757+0000 7f0ba67fc640 1 -- 172.21.15.139:0/2914753049 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0bac04c030 con 0x7f0bb01081e0 2024-09-16T01:30:47.559 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.804+0000 7f0ba67fc640 1 -- 172.21.15.139:0/2914753049 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f0bac06b7d0 con 0x7f0bb01081e0 2024-09-16T01:30:47.559 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.944+0000 7f0bb752c640 1 -- 172.21.15.139:0/2914753049 --> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f0bb0106f20 con 0x7f0b9c0390a0 2024-09-16T01:30:47.559 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.949+0000 7f0ba67fc640 1 -- 172.21.15.139:0/2914753049 <== mgr.14118 v2:172.21.15.139:6800/1485008061 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0bb0106f20 con 0x7f0b9c0390a0 2024-09-16T01:30:47.560 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.951+0000 7f0bb752c640 1 -- 172.21.15.139:0/2914753049 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f0b9c0390a0 msgr2=0x7f0b9c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:47.560 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.951+0000 7f0bb752c640 1 --2- 172.21.15.139:0/2914753049 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f0b9c0390a0 0x7f0b9c03b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00098c0 tx=0x7f0ba0006db0 comp rx=0 tx=0).stop 2024-09-16T01:30:47.560 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.951+0000 7f0bb752c640 1 -- 172.21.15.139:0/2914753049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bb01081e0 msgr2=0x7f0bb0107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:47.560 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.951+0000 7f0bb752c640 1 --2- 172.21.15.139:0/2914753049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bb01081e0 0x7f0bb0107a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0bac031000 tx=0x7f0bac0326e0 comp rx=0 tx=0).stop 2024-09-16T01:30:47.560 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.951+0000 7f0bb752c640 1 -- 172.21.15.139:0/2914753049 shutdown_connections 2024-09-16T01:30:47.560 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.951+0000 7f0bb752c640 1 --2- 172.21.15.139:0/2914753049 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f0b9c0390a0 0x7f0b9c03b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:47.560 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.951+0000 7f0bb752c640 1 --2- 172.21.15.139:0/2914753049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bb01081e0 0x7f0bb0107a60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:47.560 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.951+0000 7f0bb752c640 1 -- 172.21.15.139:0/2914753049 >> 172.21.15.139:0/2914753049 conn(0x7f0bb0101e20 msgr2=0x7f0bb0102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:47.560 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.951+0000 7f0bb752c640 1 -- 172.21.15.139:0/2914753049 shutdown_connections 2024-09-16T01:30:47.560 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:46.952+0000 7f0bb752c640 1 -- 172.21.15.139:0/2914753049 wait complete. 2024-09-16T01:30:48.146 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-mon[31175]: mgrmap e8: x(active, since 2s) 2024-09-16T01:30:48.146 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-mon[31175]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:30:48.146 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' 2024-09-16T01:30:48.146 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:30:48.146 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:47 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:30:48.714 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T01:30:48.714 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.968+0000 7fd2462c1640 1 Processor -- start 2024-09-16T01:30:48.714 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.969+0000 7fd2462c1640 1 -- start start 2024-09-16T01:30:48.714 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.969+0000 7fd2462c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd240107740 0x7fd240107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:48.715 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.969+0000 7fd2462c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd240108080 con 0x7fd240107740 2024-09-16T01:30:48.715 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.969+0000 7fd23ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd240107740 0x7fd240107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:48.715 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.969+0000 7fd23ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd240107740 0x7fd240107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53790/0 (socket says 172.21.15.139:53790) 2024-09-16T01:30:48.715 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.969+0000 7fd23ffff640 1 -- 172.21.15.139:0/2941021161 learned_addr learned my addr 172.21.15.139:0/2941021161 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:48.715 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.969+0000 7fd23ffff640 1 -- 172.21.15.139:0/2941021161 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd240108200 con 0x7fd240107740 2024-09-16T01:30:48.715 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.970+0000 7fd23ffff640 1 --2- 172.21.15.139:0/2941021161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd240107740 0x7fd240107b40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd230009820 tx=0x7fd23002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b175e93ca4fb31 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:48.715 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.970+0000 7fd23effd640 1 -- 172.21.15.139:0/2941021161 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd230037070 con 0x7fd240107740 2024-09-16T01:30:48.715 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.970+0000 7fd23effd640 1 -- 172.21.15.139:0/2941021161 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd230032900 con 0x7fd240107740 2024-09-16T01:30:48.715 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.970+0000 7fd23effd640 1 -- 172.21.15.139:0/2941021161 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd230030850 con 0x7fd240107740 2024-09-16T01:30:48.715 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.971+0000 7fd2462c1640 1 -- 172.21.15.139:0/2941021161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd240107740 msgr2=0x7fd240107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:48.715 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.971+0000 7fd2462c1640 1 --2- 172.21.15.139:0/2941021161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd240107740 0x7fd240107b40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd230009820 tx=0x7fd23002a270 comp rx=0 tx=0).stop 2024-09-16T01:30:48.715 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.971+0000 7fd2462c1640 1 -- 172.21.15.139:0/2941021161 shutdown_connections 2024-09-16T01:30:48.715 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.971+0000 7fd2462c1640 1 --2- 172.21.15.139:0/2941021161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd240107740 0x7fd240107b40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:48.716 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.971+0000 7fd2462c1640 1 -- 172.21.15.139:0/2941021161 >> 172.21.15.139:0/2941021161 conn(0x7fd240102f50 msgr2=0x7fd2401053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:48.716 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.971+0000 7fd2462c1640 1 -- 172.21.15.139:0/2941021161 shutdown_connections 2024-09-16T01:30:48.716 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.971+0000 7fd2462c1640 1 -- 172.21.15.139:0/2941021161 wait complete. 2024-09-16T01:30:48.716 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.971+0000 7fd2462c1640 1 Processor -- start 2024-09-16T01:30:48.716 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.971+0000 7fd2462c1640 1 -- start start 2024-09-16T01:30:48.716 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.972+0000 7fd2462c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd240107740 0x7fd24019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:48.716 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.972+0000 7fd2462c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd24010bbb0 con 0x7fd240107740 2024-09-16T01:30:48.716 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.972+0000 7fd23ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd240107740 0x7fd24019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:48.716 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.972+0000 7fd23ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd240107740 0x7fd24019d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53798/0 (socket says 172.21.15.139:53798) 2024-09-16T01:30:48.716 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.972+0000 7fd23ffff640 1 -- 172.21.15.139:0/1970089486 learned_addr learned my addr 172.21.15.139:0/1970089486 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:48.716 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.972+0000 7fd23ffff640 1 -- 172.21.15.139:0/1970089486 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24019de60 con 0x7fd240107740 2024-09-16T01:30:48.717 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.972+0000 7fd23ffff640 1 --2- 172.21.15.139:0/1970089486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd240107740 0x7fd24019d920 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd230031000 tx=0x7fd2300326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:48.717 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.972+0000 7fd23d7fa640 1 -- 172.21.15.139:0/1970089486 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd230042070 con 0x7fd240107740 2024-09-16T01:30:48.717 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.972+0000 7fd23d7fa640 1 -- 172.21.15.139:0/1970089486 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd230032be0 con 0x7fd240107740 2024-09-16T01:30:48.717 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.972+0000 7fd2462c1640 1 -- 172.21.15.139:0/1970089486 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24019e0f0 con 0x7fd240107740 2024-09-16T01:30:48.717 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.972+0000 7fd2462c1640 1 -- 172.21.15.139:0/1970089486 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2401a0de0 con 0x7fd240107740 2024-09-16T01:30:48.717 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.972+0000 7fd23d7fa640 1 -- 172.21.15.139:0/1970089486 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd230037040 con 0x7fd240107740 2024-09-16T01:30:48.717 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.973+0000 7fd23d7fa640 1 -- 172.21.15.139:0/1970089486 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fd230045070 con 0x7fd240107740 2024-09-16T01:30:48.717 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.973+0000 7fd2462c1640 1 -- 172.21.15.139:0/1970089486 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2200052c0 con 0x7fd240107740 2024-09-16T01:30:48.717 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.973+0000 7fd23d7fa640 1 --2- 172.21.15.139:0/1970089486 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fd22c039200 0x7fd22c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:48.717 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.973+0000 7fd23d7fa640 1 -- 172.21.15.139:0/1970089486 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd23006c290 con 0x7fd240107740 2024-09-16T01:30:48.717 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.974+0000 7fd23f7fe640 1 --2- 172.21.15.139:0/1970089486 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fd22c039200 0x7fd22c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:48.717 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.974+0000 7fd23f7fe640 1 --2- 172.21.15.139:0/1970089486 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fd22c039200 0x7fd22c03b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd2280098c0 tx=0x7fd228006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:48.717 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:47.978+0000 7fd23d7fa640 1 -- 172.21.15.139:0/1970089486 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd230045360 con 0x7fd240107740 2024-09-16T01:30:48.717 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:48.144+0000 7fd2462c1640 1 -- 172.21.15.139:0/1970089486 --> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fd220000bf0 con 0x7fd22c039200 2024-09-16T01:30:48.718 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:48.144+0000 7fd23d7fa640 1 -- 172.21.15.139:0/1970089486 <== mgr.14118 v2:172.21.15.139:6800/1485008061 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fd220000bf0 con 0x7fd22c039200 2024-09-16T01:30:48.718 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:48.146+0000 7fd2462c1640 1 -- 172.21.15.139:0/1970089486 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fd22c039200 msgr2=0x7fd22c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:48.718 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:48.146+0000 7fd2462c1640 1 --2- 172.21.15.139:0/1970089486 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fd22c039200 0x7fd22c03b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd2280098c0 tx=0x7fd228006db0 comp rx=0 tx=0).stop 2024-09-16T01:30:48.718 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:48.146+0000 7fd2462c1640 1 -- 172.21.15.139:0/1970089486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd240107740 msgr2=0x7fd24019d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:48.718 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:48.146+0000 7fd2462c1640 1 --2- 172.21.15.139:0/1970089486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd240107740 0x7fd24019d920 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd230031000 tx=0x7fd2300326e0 comp rx=0 tx=0).stop 2024-09-16T01:30:48.718 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:48.147+0000 7fd2462c1640 1 -- 172.21.15.139:0/1970089486 shutdown_connections 2024-09-16T01:30:48.718 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:48.147+0000 7fd2462c1640 1 --2- 172.21.15.139:0/1970089486 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fd22c039200 0x7fd22c03b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:48.718 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:48.147+0000 7fd2462c1640 1 --2- 172.21.15.139:0/1970089486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd240107740 0x7fd24019d920 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:48.718 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:48.147+0000 7fd2462c1640 1 -- 172.21.15.139:0/1970089486 >> 172.21.15.139:0/1970089486 conn(0x7fd240102f50 msgr2=0x7fd240103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:48.718 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:48.147+0000 7fd2462c1640 1 -- 172.21.15.139:0/1970089486 shutdown_connections 2024-09-16T01:30:48.718 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:48.147+0000 7fd2462c1640 1 -- 172.21.15.139:0/1970089486 wait complete. 2024-09-16T01:30:48.718 INFO:teuthology.orchestra.run.smithi139.stdout:Generating ssh key... 2024-09-16T01:30:49.083 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:48 smithi139 ceph-mon[31175]: [16/Sep/2024:01:30:47] ENGINE Bus STARTING 2024-09-16T01:30:49.083 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:48 smithi139 ceph-mon[31175]: [16/Sep/2024:01:30:47] ENGINE Serving on https://172.21.15.139:7150 2024-09-16T01:30:49.083 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:48 smithi139 ceph-mon[31175]: [16/Sep/2024:01:30:47] ENGINE Error in HTTPServer.serve 2024-09-16T01:30:49.084 INFO:journalctl@ceph.mon.a.smithi139.stdout: Traceback (most recent call last): 2024-09-16T01:30:49.084 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:30:49.084 INFO:journalctl@ceph.mon.a.smithi139.stdout: self._connections.run(self.expiration_interval) 2024-09-16T01:30:49.084 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:30:49.084 INFO:journalctl@ceph.mon.a.smithi139.stdout: self._run(expiration_interval) 2024-09-16T01:30:49.084 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:30:49.084 INFO:journalctl@ceph.mon.a.smithi139.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:30:49.084 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:30:49.084 INFO:journalctl@ceph.mon.a.smithi139.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:30:49.084 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:30:49.084 INFO:journalctl@ceph.mon.a.smithi139.stdout: s = self.context.wrap_socket( 2024-09-16T01:30:49.084 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:30:49.084 INFO:journalctl@ceph.mon.a.smithi139.stdout: return self.sslsocket_class._create( 2024-09-16T01:30:49.085 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:30:49.085 INFO:journalctl@ceph.mon.a.smithi139.stdout: self.do_handshake() 2024-09-16T01:30:49.085 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:30:49.085 INFO:journalctl@ceph.mon.a.smithi139.stdout: self._sslobj.do_handshake() 2024-09-16T01:30:49.085 INFO:journalctl@ceph.mon.a.smithi139.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:30:49.085 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:48 smithi139 ceph-mon[31175]: [16/Sep/2024:01:30:47] ENGINE Serving on http://172.21.15.139:8765 2024-09-16T01:30:49.085 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:48 smithi139 ceph-mon[31175]: [16/Sep/2024:01:30:47] ENGINE Bus STARTED 2024-09-16T01:30:49.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: Generating public/private rsa key pair. 2024-09-16T01:30:49.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: Your identification has been saved in /tmp/tmpdzlap_yf/key 2024-09-16T01:30:49.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: Your public key has been saved in /tmp/tmpdzlap_yf/key.pub 2024-09-16T01:30:49.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: The key fingerprint is: 2024-09-16T01:30:49.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: SHA256:TX2lCEUWwr88B3qkjumpaZt8L1Wy0JA5HzL7n2M+GTE ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:30:49.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: The key's randomart image is: 2024-09-16T01:30:49.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: +---[RSA 3072]----+ 2024-09-16T01:30:49.674 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: | +oo=. .| 2024-09-16T01:30:49.674 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: | B o= . o | 2024-09-16T01:30:49.674 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: | O.oo o | 2024-09-16T01:30:49.674 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: | oo+ E. | 2024-09-16T01:30:49.674 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: | So.O = | 2024-09-16T01:30:49.674 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: | * * . | 2024-09-16T01:30:49.674 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: | = o * | 2024-09-16T01:30:49.674 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: | ..o=.. B | 2024-09-16T01:30:49.674 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: | .*=oo.o.o | 2024-09-16T01:30:49.674 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: +----[SHA256]-----+ 2024-09-16T01:30:50.112 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-mon[31175]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:30:50.113 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' 2024-09-16T01:30:50.113 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:49 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' 2024-09-16T01:30:50.119 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.162+0000 7fd8cdca3640 1 Processor -- start 2024-09-16T01:30:50.119 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.162+0000 7fd8cdca3640 1 -- start start 2024-09-16T01:30:50.119 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.162+0000 7fd8cdca3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8c8108160 0x7fd8c810a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:50.119 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.162+0000 7fd8cdca3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c807beb0 con 0x7fd8c8108160 2024-09-16T01:30:50.119 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.162+0000 7fd8c77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8c8108160 0x7fd8c810a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:50.119 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.162+0000 7fd8c77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8c8108160 0x7fd8c810a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53812/0 (socket says 172.21.15.139:53812) 2024-09-16T01:30:50.119 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.162+0000 7fd8c77fe640 1 -- 172.21.15.139:0/1522348147 learned_addr learned my addr 172.21.15.139:0/1522348147 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:50.119 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.162+0000 7fd8c77fe640 1 -- 172.21.15.139:0/1522348147 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c810aa90 con 0x7fd8c8108160 2024-09-16T01:30:50.120 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.163+0000 7fd8c77fe640 1 --2- 172.21.15.139:0/1522348147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8c8108160 0x7fd8c810a550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009820 tx=0x7fd8b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279afbb7419c13d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:50.120 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.163+0000 7fd8c67fc640 1 -- 172.21.15.139:0/1522348147 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8b8037070 con 0x7fd8c8108160 2024-09-16T01:30:50.120 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.163+0000 7fd8c67fc640 1 -- 172.21.15.139:0/1522348147 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd8b8032900 con 0x7fd8c8108160 2024-09-16T01:30:50.120 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.163+0000 7fd8c67fc640 1 -- 172.21.15.139:0/1522348147 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8b8030850 con 0x7fd8c8108160 2024-09-16T01:30:50.120 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.164+0000 7fd8cdca3640 1 -- 172.21.15.139:0/1522348147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8c8108160 msgr2=0x7fd8c810a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:50.120 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.164+0000 7fd8cdca3640 1 --2- 172.21.15.139:0/1522348147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8c8108160 0x7fd8c810a550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009820 tx=0x7fd8b802a270 comp rx=0 tx=0).stop 2024-09-16T01:30:50.120 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.164+0000 7fd8cdca3640 1 -- 172.21.15.139:0/1522348147 shutdown_connections 2024-09-16T01:30:50.120 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.164+0000 7fd8cdca3640 1 --2- 172.21.15.139:0/1522348147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8c8108160 0x7fd8c810a550 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:50.120 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.164+0000 7fd8cdca3640 1 -- 172.21.15.139:0/1522348147 >> 172.21.15.139:0/1522348147 conn(0x7fd8c8101d60 msgr2=0x7fd8c81041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:50.120 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.164+0000 7fd8cdca3640 1 -- 172.21.15.139:0/1522348147 shutdown_connections 2024-09-16T01:30:50.120 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.164+0000 7fd8cdca3640 1 -- 172.21.15.139:0/1522348147 wait complete. 2024-09-16T01:30:50.120 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.164+0000 7fd8cdca3640 1 Processor -- start 2024-09-16T01:30:50.120 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.165+0000 7fd8cdca3640 1 -- start start 2024-09-16T01:30:50.121 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.165+0000 7fd8cdca3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8c8108160 0x7fd8c819b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:50.121 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.165+0000 7fd8cdca3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c810bb30 con 0x7fd8c8108160 2024-09-16T01:30:50.121 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.165+0000 7fd8c77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8c8108160 0x7fd8c819b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:50.121 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.165+0000 7fd8c77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8c8108160 0x7fd8c819b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53818/0 (socket says 172.21.15.139:53818) 2024-09-16T01:30:50.121 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.165+0000 7fd8c77fe640 1 -- 172.21.15.139:0/706634906 learned_addr learned my addr 172.21.15.139:0/706634906 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:50.121 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.165+0000 7fd8c77fe640 1 -- 172.21.15.139:0/706634906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c819bc40 con 0x7fd8c8108160 2024-09-16T01:30:50.121 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.165+0000 7fd8c77fe640 1 --2- 172.21.15.139:0/706634906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8c8108160 0x7fd8c819b700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8031000 tx=0x7fd8b80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:50.121 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.165+0000 7fd8c4ff9640 1 -- 172.21.15.139:0/706634906 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8b8042070 con 0x7fd8c8108160 2024-09-16T01:30:50.121 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.165+0000 7fd8cdca3640 1 -- 172.21.15.139:0/706634906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c819bed0 con 0x7fd8c8108160 2024-09-16T01:30:50.121 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.165+0000 7fd8c4ff9640 1 -- 172.21.15.139:0/706634906 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd8b8032be0 con 0x7fd8c8108160 2024-09-16T01:30:50.121 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.165+0000 7fd8c4ff9640 1 -- 172.21.15.139:0/706634906 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8b8037040 con 0x7fd8c8108160 2024-09-16T01:30:50.121 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.165+0000 7fd8cdca3640 1 -- 172.21.15.139:0/706634906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c819ebc0 con 0x7fd8c8108160 2024-09-16T01:30:50.121 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.166+0000 7fd8c4ff9640 1 -- 172.21.15.139:0/706634906 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fd8b8045070 con 0x7fd8c8108160 2024-09-16T01:30:50.121 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.166+0000 7fd8cdca3640 1 -- 172.21.15.139:0/706634906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c81085c0 con 0x7fd8c8108160 2024-09-16T01:30:50.122 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.166+0000 7fd8c4ff9640 1 --2- 172.21.15.139:0/706634906 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fd8b0039200 0x7fd8b003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:50.122 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.167+0000 7fd8c4ff9640 1 -- 172.21.15.139:0/706634906 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd8b806c400 con 0x7fd8c8108160 2024-09-16T01:30:50.122 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.167+0000 7fd8c6ffd640 1 --2- 172.21.15.139:0/706634906 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fd8b0039200 0x7fd8b003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:50.122 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.167+0000 7fd8c6ffd640 1 --2- 172.21.15.139:0/706634906 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fd8b0039200 0x7fd8b003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00ad30 tx=0x7fd8bc0093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:50.122 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.171+0000 7fd8c4ff9640 1 -- 172.21.15.139:0/706634906 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd8b803a780 con 0x7fd8c8108160 2024-09-16T01:30:50.122 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.340+0000 7fd8cdca3640 1 -- 172.21.15.139:0/706634906 --> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fd8c8103060 con 0x7fd8b0039200 2024-09-16T01:30:50.122 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.478+0000 7fd8c4ff9640 1 -- 172.21.15.139:0/706634906 <== mgr.14118 v2:172.21.15.139:6800/1485008061 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd8c8103060 con 0x7fd8b0039200 2024-09-16T01:30:50.122 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.480+0000 7fd8cdca3640 1 -- 172.21.15.139:0/706634906 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fd8b0039200 msgr2=0x7fd8b003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:50.122 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.480+0000 7fd8cdca3640 1 --2- 172.21.15.139:0/706634906 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fd8b0039200 0x7fd8b003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00ad30 tx=0x7fd8bc0093f0 comp rx=0 tx=0).stop 2024-09-16T01:30:50.122 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.480+0000 7fd8cdca3640 1 -- 172.21.15.139:0/706634906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8c8108160 msgr2=0x7fd8c819b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:50.122 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.480+0000 7fd8cdca3640 1 --2- 172.21.15.139:0/706634906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8c8108160 0x7fd8c819b700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8031000 tx=0x7fd8b80326e0 comp rx=0 tx=0).stop 2024-09-16T01:30:50.122 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.480+0000 7fd8cdca3640 1 -- 172.21.15.139:0/706634906 shutdown_connections 2024-09-16T01:30:50.122 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.480+0000 7fd8cdca3640 1 --2- 172.21.15.139:0/706634906 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fd8b0039200 0x7fd8b003b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:50.123 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.480+0000 7fd8cdca3640 1 --2- 172.21.15.139:0/706634906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8c8108160 0x7fd8c819b700 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:50.123 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.480+0000 7fd8cdca3640 1 -- 172.21.15.139:0/706634906 >> 172.21.15.139:0/706634906 conn(0x7fd8c8101d60 msgr2=0x7fd8c8102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:50.123 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.481+0000 7fd8cdca3640 1 -- 172.21.15.139:0/706634906 shutdown_connections 2024-09-16T01:30:50.123 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:49.481+0000 7fd8cdca3640 1 -- 172.21.15.139:0/706634906 wait complete. 2024-09-16T01:30:51.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:50 smithi139 ceph-mon[31175]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:30:51.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:50 smithi139 ceph-mon[31175]: Generating ssh key... 2024-09-16T01:30:51.276 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmmVDWTdIdKh1gGfDeECBzCSn6c+0f3hMa5U0wtD+Jo4hrtk+pjW7K31f4+N1jVzpo/U4/0DVQSeehvkxwAn3j4P81HbJUVCWb9AhLmEr7NWFf1M0I1UEd62D+nl6rmVA24mxlPEd6TAgXsBt8WphUIA98en4V2NVd9zEadqLh0HA2JX/YK+LXjOo4YpYHywqC2NBM8ceeIJr9LCNB9RmS1SkPecrbG0jqQTx9avQMB07kGtIVFNUp53m1648Y35wgl9yRFQBHQClchRLs3c9mBITbM7WYxgSlExu5OEcwXAKPCqr9CO96Dy8uqhqTjnNBj8w4zQohQy3K7wSOEXbC71+qBg0s4DIMqw3qxm+8iNhUmaKRRKetkJpXMgeEeM6AOY9BEIl45GRcQCz2ooMz1Vvg9wqp4+kQv0W1lzf4oh/4WzeP+0DO889uqfKAeAZe3UE7Y7l2VYTjBvZAErDg56TqiyDZEIeuhnOFx4FsyitmMg8reG/lF0vlnm6ie3s= ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:30:51.276 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.496+0000 7fdafe9fa640 1 Processor -- start 2024-09-16T01:30:51.276 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.497+0000 7fdafe9fa640 1 -- start start 2024-09-16T01:30:51.276 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.497+0000 7fdafe9fa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaf81076f0 0x7fdaf8107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:51.276 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.497+0000 7fdafe9fa640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81080c0 con 0x7fdaf81076f0 2024-09-16T01:30:51.276 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.497+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaf81076f0 0x7fdaf8107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:51.277 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.497+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaf81076f0 0x7fdaf8107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53834/0 (socket says 172.21.15.139:53834) 2024-09-16T01:30:51.277 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.497+0000 7fdaf7fff640 1 -- 172.21.15.139:0/285161933 learned_addr learned my addr 172.21.15.139:0/285161933 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:51.277 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.497+0000 7fdaf7fff640 1 -- 172.21.15.139:0/285161933 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf81088f0 con 0x7fdaf81076f0 2024-09-16T01:30:51.277 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.498+0000 7fdaf7fff640 1 --2- 172.21.15.139:0/285161933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaf81076f0 0x7fdaf8107af0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009820 tx=0x7fdae802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfbe6c467eb0786 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:51.277 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.498+0000 7fdaf6ffd640 1 -- 172.21.15.139:0/285161933 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdae8037070 con 0x7fdaf81076f0 2024-09-16T01:30:51.277 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.498+0000 7fdaf6ffd640 1 -- 172.21.15.139:0/285161933 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdae8032900 con 0x7fdaf81076f0 2024-09-16T01:30:51.277 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.498+0000 7fdaf6ffd640 1 -- 172.21.15.139:0/285161933 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdae8030850 con 0x7fdaf81076f0 2024-09-16T01:30:51.277 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.499+0000 7fdafe9fa640 1 -- 172.21.15.139:0/285161933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaf81076f0 msgr2=0x7fdaf8107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:51.277 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.499+0000 7fdafe9fa640 1 --2- 172.21.15.139:0/285161933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaf81076f0 0x7fdaf8107af0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009820 tx=0x7fdae802a270 comp rx=0 tx=0).stop 2024-09-16T01:30:51.277 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.499+0000 7fdafe9fa640 1 -- 172.21.15.139:0/285161933 shutdown_connections 2024-09-16T01:30:51.277 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.499+0000 7fdafe9fa640 1 --2- 172.21.15.139:0/285161933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaf81076f0 0x7fdaf8107af0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:51.277 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.499+0000 7fdafe9fa640 1 -- 172.21.15.139:0/285161933 >> 172.21.15.139:0/285161933 conn(0x7fdaf8102ec0 msgr2=0x7fdaf81052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:51.277 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.499+0000 7fdafe9fa640 1 -- 172.21.15.139:0/285161933 shutdown_connections 2024-09-16T01:30:51.278 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.499+0000 7fdafe9fa640 1 -- 172.21.15.139:0/285161933 wait complete. 2024-09-16T01:30:51.278 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.499+0000 7fdafe9fa640 1 Processor -- start 2024-09-16T01:30:51.278 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.499+0000 7fdafe9fa640 1 -- start start 2024-09-16T01:30:51.278 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.500+0000 7fdafe9fa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaf81076f0 0x7fdaf807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:51.278 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.500+0000 7fdafe9fa640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf810bca0 con 0x7fdaf81076f0 2024-09-16T01:30:51.278 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.500+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaf81076f0 0x7fdaf807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:51.278 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.500+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaf81076f0 0x7fdaf807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53844/0 (socket says 172.21.15.139:53844) 2024-09-16T01:30:51.278 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.500+0000 7fdaf7fff640 1 -- 172.21.15.139:0/997761616 learned_addr learned my addr 172.21.15.139:0/997761616 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:51.278 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.500+0000 7fdaf7fff640 1 -- 172.21.15.139:0/997761616 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf807e0b0 con 0x7fdaf81076f0 2024-09-16T01:30:51.278 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.500+0000 7fdaf7fff640 1 --2- 172.21.15.139:0/997761616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaf81076f0 0x7fdaf807db70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009940 tx=0x7fdae8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:51.278 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.500+0000 7fdaf57fa640 1 -- 172.21.15.139:0/997761616 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdae8042070 con 0x7fdaf81076f0 2024-09-16T01:30:51.278 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.500+0000 7fdaf57fa640 1 -- 172.21.15.139:0/997761616 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdae80308c0 con 0x7fdaf81076f0 2024-09-16T01:30:51.278 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.500+0000 7fdafe9fa640 1 -- 172.21.15.139:0/997761616 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf807c2a0 con 0x7fdaf81076f0 2024-09-16T01:30:51.279 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.500+0000 7fdaf57fa640 1 -- 172.21.15.139:0/997761616 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdae8037040 con 0x7fdaf81076f0 2024-09-16T01:30:51.279 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.500+0000 7fdafe9fa640 1 -- 172.21.15.139:0/997761616 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf807c7e0 con 0x7fdaf81076f0 2024-09-16T01:30:51.279 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.501+0000 7fdaf57fa640 1 -- 172.21.15.139:0/997761616 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fdae8045070 con 0x7fdaf81076f0 2024-09-16T01:30:51.279 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.501+0000 7fdaf57fa640 1 --2- 172.21.15.139:0/997761616 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fdae4039200 0x7fdae403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:51.279 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.501+0000 7fdaf57fa640 1 -- 172.21.15.139:0/997761616 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdae806c390 con 0x7fdaf81076f0 2024-09-16T01:30:51.279 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.501+0000 7fdafe9fa640 1 -- 172.21.15.139:0/997761616 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf8107b70 con 0x7fdaf81076f0 2024-09-16T01:30:51.279 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.501+0000 7fdaf77fe640 1 --2- 172.21.15.139:0/997761616 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fdae4039200 0x7fdae403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:51.279 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.502+0000 7fdaf77fe640 1 --2- 172.21.15.139:0/997761616 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fdae4039200 0x7fdae403b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdae00098c0 tx=0x7fdae0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:51.279 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.506+0000 7fdaf57fa640 1 -- 172.21.15.139:0/997761616 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdae804b170 con 0x7fdaf81076f0 2024-09-16T01:30:51.279 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.675+0000 7fdafe9fa640 1 -- 172.21.15.139:0/997761616 --> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fdaf81040e0 con 0x7fdae4039200 2024-09-16T01:30:51.279 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.676+0000 7fdaf57fa640 1 -- 172.21.15.139:0/997761616 <== mgr.14118 v2:172.21.15.139:6800/1485008061 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fdaf81040e0 con 0x7fdae4039200 2024-09-16T01:30:51.279 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.678+0000 7fdafe9fa640 1 -- 172.21.15.139:0/997761616 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fdae4039200 msgr2=0x7fdae403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:51.279 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.678+0000 7fdafe9fa640 1 --2- 172.21.15.139:0/997761616 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fdae4039200 0x7fdae403b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdae00098c0 tx=0x7fdae0006db0 comp rx=0 tx=0).stop 2024-09-16T01:30:51.279 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.678+0000 7fdafe9fa640 1 -- 172.21.15.139:0/997761616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaf81076f0 msgr2=0x7fdaf807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:51.280 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.678+0000 7fdafe9fa640 1 --2- 172.21.15.139:0/997761616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaf81076f0 0x7fdaf807db70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009940 tx=0x7fdae8032ce0 comp rx=0 tx=0).stop 2024-09-16T01:30:51.280 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.678+0000 7fdafe9fa640 1 -- 172.21.15.139:0/997761616 shutdown_connections 2024-09-16T01:30:51.280 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.678+0000 7fdafe9fa640 1 --2- 172.21.15.139:0/997761616 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fdae4039200 0x7fdae403b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:51.280 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.678+0000 7fdafe9fa640 1 --2- 172.21.15.139:0/997761616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaf81076f0 0x7fdaf807db70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:51.280 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.678+0000 7fdafe9fa640 1 -- 172.21.15.139:0/997761616 >> 172.21.15.139:0/997761616 conn(0x7fdaf8102ec0 msgr2=0x7fdaf81039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:51.280 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.678+0000 7fdafe9fa640 1 -- 172.21.15.139:0/997761616 shutdown_connections 2024-09-16T01:30:51.280 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:50.679+0000 7fdafe9fa640 1 -- 172.21.15.139:0/997761616 wait complete. 2024-09-16T01:30:51.280 INFO:teuthology.orchestra.run.smithi139.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T01:30:51.280 INFO:teuthology.orchestra.run.smithi139.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T01:30:51.280 INFO:teuthology.orchestra.run.smithi139.stdout:Adding host smithi139... 2024-09-16T01:30:52.067 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:51 smithi139 ceph-mon[31175]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:30:52.898 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:52 smithi139 ceph-mon[31175]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi139", "addr": "172.21.15.139", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:30:54.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:53 smithi139 ceph-mon[31175]: Deploying cephadm binary to smithi139 2024-09-16T01:30:55.658 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout Added host 'smithi139' with addr '172.21.15.139' 2024-09-16T01:30:55.658 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.689+0000 7f104f97e640 1 Processor -- start 2024-09-16T01:30:55.658 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.689+0000 7f104f97e640 1 -- start start 2024-09-16T01:30:55.658 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.689+0000 7f104f97e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10481076f0 0x7f1048107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:55.658 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.689+0000 7f104f97e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f10481080c0 con 0x7f10481076f0 2024-09-16T01:30:55.659 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.689+0000 7f104d71a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10481076f0 0x7f1048107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:55.659 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.689+0000 7f104d71a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10481076f0 0x7f1048107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53850/0 (socket says 172.21.15.139:53850) 2024-09-16T01:30:55.659 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.689+0000 7f104d71a640 1 -- 172.21.15.139:0/3228822154 learned_addr learned my addr 172.21.15.139:0/3228822154 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:55.659 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.689+0000 7f104d71a640 1 -- 172.21.15.139:0/3228822154 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10481088f0 con 0x7f10481076f0 2024-09-16T01:30:55.659 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.690+0000 7f104d71a640 1 --2- 172.21.15.139:0/3228822154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10481076f0 0x7f1048107af0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f103c009820 tx=0x7f103c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b600a9fa732f19ca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:55.659 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.690+0000 7f103bfff640 1 -- 172.21.15.139:0/3228822154 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f103c037070 con 0x7f10481076f0 2024-09-16T01:30:55.659 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.690+0000 7f103bfff640 1 -- 172.21.15.139:0/3228822154 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f103c032900 con 0x7f10481076f0 2024-09-16T01:30:55.659 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.690+0000 7f103bfff640 1 -- 172.21.15.139:0/3228822154 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f103c030850 con 0x7f10481076f0 2024-09-16T01:30:55.659 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.691+0000 7f104f97e640 1 -- 172.21.15.139:0/3228822154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10481076f0 msgr2=0x7f1048107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:55.659 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.691+0000 7f104f97e640 1 --2- 172.21.15.139:0/3228822154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10481076f0 0x7f1048107af0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f103c009820 tx=0x7f103c02a270 comp rx=0 tx=0).stop 2024-09-16T01:30:55.659 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.691+0000 7f104f97e640 1 -- 172.21.15.139:0/3228822154 shutdown_connections 2024-09-16T01:30:55.659 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.691+0000 7f104f97e640 1 --2- 172.21.15.139:0/3228822154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10481076f0 0x7f1048107af0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:55.659 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.691+0000 7f104f97e640 1 -- 172.21.15.139:0/3228822154 >> 172.21.15.139:0/3228822154 conn(0x7f1048102ec0 msgr2=0x7f10481052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.691+0000 7f104f97e640 1 -- 172.21.15.139:0/3228822154 shutdown_connections 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.691+0000 7f104f97e640 1 -- 172.21.15.139:0/3228822154 wait complete. 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.691+0000 7f104f97e640 1 Processor -- start 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.691+0000 7f104f97e640 1 -- start start 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.692+0000 7f104f97e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10481076f0 0x7f104819b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.692+0000 7f104f97e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f104810bca0 con 0x7f10481076f0 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.692+0000 7f104d71a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10481076f0 0x7f104819b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.692+0000 7f104d71a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10481076f0 0x7f104819b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53856/0 (socket says 172.21.15.139:53856) 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.692+0000 7f104d71a640 1 -- 172.21.15.139:0/1539348977 learned_addr learned my addr 172.21.15.139:0/1539348977 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.692+0000 7f104d71a640 1 -- 172.21.15.139:0/1539348977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104819bc20 con 0x7f10481076f0 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.692+0000 7f104d71a640 1 --2- 172.21.15.139:0/1539348977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10481076f0 0x7f104819b6e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f103c009940 tx=0x7f103c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.692+0000 7f103a7fc640 1 -- 172.21.15.139:0/1539348977 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f103c042070 con 0x7f10481076f0 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.692+0000 7f103a7fc640 1 -- 172.21.15.139:0/1539348977 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f103c0308c0 con 0x7f10481076f0 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.692+0000 7f104f97e640 1 -- 172.21.15.139:0/1539348977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104819beb0 con 0x7f10481076f0 2024-09-16T01:30:55.660 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.692+0000 7f103a7fc640 1 -- 172.21.15.139:0/1539348977 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f103c037040 con 0x7f10481076f0 2024-09-16T01:30:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.692+0000 7f104f97e640 1 -- 172.21.15.139:0/1539348977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104819eba0 con 0x7f10481076f0 2024-09-16T01:30:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.693+0000 7f103a7fc640 1 -- 172.21.15.139:0/1539348977 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f103c045070 con 0x7f10481076f0 2024-09-16T01:30:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.693+0000 7f104f97e640 1 -- 172.21.15.139:0/1539348977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f104810be20 con 0x7f10481076f0 2024-09-16T01:30:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.693+0000 7f103a7fc640 1 --2- 172.21.15.139:0/1539348977 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f10340391b0 0x7f103403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.693+0000 7f103a7fc640 1 -- 172.21.15.139:0/1539348977 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f103c06c330 con 0x7f10481076f0 2024-09-16T01:30:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.693+0000 7f104cf19640 1 --2- 172.21.15.139:0/1539348977 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f10340391b0 0x7f103403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.694+0000 7f104cf19640 1 --2- 172.21.15.139:0/1539348977 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f10340391b0 0x7f103403b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f10440098c0 tx=0x7f1044006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.698+0000 7f103a7fc640 1 -- 172.21.15.139:0/1539348977 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f103c03cd90 con 0x7f10481076f0 2024-09-16T01:30:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:51.868+0000 7f104f97e640 1 -- 172.21.15.139:0/1539348977 --> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi139", "addr": "172.21.15.139", "target": ["mon-mgr", ""]}) -- 0x7f10481040e0 con 0x7f10340391b0 2024-09-16T01:30:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:55.053+0000 7f103a7fc640 1 -- 172.21.15.139:0/1539348977 <== mgr.14118 v2:172.21.15.139:6800/1485008061 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f10481040e0 con 0x7f10340391b0 2024-09-16T01:30:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:55.056+0000 7f104f97e640 1 -- 172.21.15.139:0/1539348977 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f10340391b0 msgr2=0x7f103403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:55.056+0000 7f104f97e640 1 --2- 172.21.15.139:0/1539348977 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f10340391b0 0x7f103403b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f10440098c0 tx=0x7f1044006db0 comp rx=0 tx=0).stop 2024-09-16T01:30:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:55.056+0000 7f104f97e640 1 -- 172.21.15.139:0/1539348977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10481076f0 msgr2=0x7f104819b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:55.056+0000 7f104f97e640 1 --2- 172.21.15.139:0/1539348977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10481076f0 0x7f104819b6e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f103c009940 tx=0x7f103c032ce0 comp rx=0 tx=0).stop 2024-09-16T01:30:55.662 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:55.056+0000 7f104f97e640 1 -- 172.21.15.139:0/1539348977 shutdown_connections 2024-09-16T01:30:55.662 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:55.056+0000 7f104f97e640 1 --2- 172.21.15.139:0/1539348977 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f10340391b0 0x7f103403b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:55.662 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:55.056+0000 7f104f97e640 1 --2- 172.21.15.139:0/1539348977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10481076f0 0x7f104819b6e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:55.662 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:55.056+0000 7f104f97e640 1 -- 172.21.15.139:0/1539348977 >> 172.21.15.139:0/1539348977 conn(0x7f1048102ec0 msgr2=0x7f10481039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:55.662 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:55.056+0000 7f104f97e640 1 -- 172.21.15.139:0/1539348977 shutdown_connections 2024-09-16T01:30:55.662 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:55.056+0000 7f104f97e640 1 -- 172.21.15.139:0/1539348977 wait complete. 2024-09-16T01:30:55.662 INFO:teuthology.orchestra.run.smithi139.stdout:Deploying unmanaged mon service... 2024-09-16T01:30:56.309 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:56 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' 2024-09-16T01:30:56.310 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:56 smithi139 ceph-mon[31175]: Added host smithi139 2024-09-16T01:30:56.310 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:56 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:30:57.168 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T01:30:57.168 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.092+0000 7f779192d640 1 Processor -- start 2024-09-16T01:30:57.168 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.092+0000 7f779192d640 1 -- start start 2024-09-16T01:30:57.168 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.092+0000 7f779192d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778c1065f0 0x7f778c1069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:57.168 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.092+0000 7f779192d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f778c106fc0 con 0x7f778c1065f0 2024-09-16T01:30:57.168 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.093+0000 7f778affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778c1065f0 0x7f778c1069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:57.168 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.093+0000 7f778affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778c1065f0 0x7f778c1069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39728/0 (socket says 172.21.15.139:39728) 2024-09-16T01:30:57.168 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.093+0000 7f778affd640 1 -- 172.21.15.139:0/2090559584 learned_addr learned my addr 172.21.15.139:0/2090559584 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:57.168 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.093+0000 7f778affd640 1 -- 172.21.15.139:0/2090559584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778c1077f0 con 0x7f778c1065f0 2024-09-16T01:30:57.168 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.094+0000 7f778affd640 1 --2- 172.21.15.139:0/2090559584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778c1065f0 0x7f778c1069f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f777c009e80 tx=0x7f777c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa427e393dd3ed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:57.169 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.094+0000 7f7789ffb640 1 -- 172.21.15.139:0/2090559584 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f777c036070 con 0x7f778c1065f0 2024-09-16T01:30:57.169 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.094+0000 7f7789ffb640 1 -- 172.21.15.139:0/2090559584 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f777c0318c0 con 0x7f778c1065f0 2024-09-16T01:30:57.169 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.094+0000 7f7789ffb640 1 -- 172.21.15.139:0/2090559584 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f777c007ab0 con 0x7f778c1065f0 2024-09-16T01:30:57.169 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.094+0000 7f779192d640 1 -- 172.21.15.139:0/2090559584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778c1065f0 msgr2=0x7f778c1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:57.169 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.094+0000 7f779192d640 1 --2- 172.21.15.139:0/2090559584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778c1065f0 0x7f778c1069f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f777c009e80 tx=0x7f777c02a7a0 comp rx=0 tx=0).stop 2024-09-16T01:30:57.169 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.094+0000 7f779192d640 1 -- 172.21.15.139:0/2090559584 shutdown_connections 2024-09-16T01:30:57.169 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.094+0000 7f779192d640 1 --2- 172.21.15.139:0/2090559584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778c1065f0 0x7f778c1069f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:57.169 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.094+0000 7f779192d640 1 -- 172.21.15.139:0/2090559584 >> 172.21.15.139:0/2090559584 conn(0x7f778c101e20 msgr2=0x7f778c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:57.169 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.094+0000 7f779192d640 1 -- 172.21.15.139:0/2090559584 shutdown_connections 2024-09-16T01:30:57.169 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.094+0000 7f779192d640 1 -- 172.21.15.139:0/2090559584 wait complete. 2024-09-16T01:30:57.169 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.095+0000 7f779192d640 1 Processor -- start 2024-09-16T01:30:57.169 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.095+0000 7f779192d640 1 -- start start 2024-09-16T01:30:57.169 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.095+0000 7f779192d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778c1065f0 0x7f778c07ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:57.170 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.095+0000 7f779192d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f778c10aba0 con 0x7f778c1065f0 2024-09-16T01:30:57.170 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.095+0000 7f778affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778c1065f0 0x7f778c07ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:57.170 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.095+0000 7f778affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778c1065f0 0x7f778c07ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39734/0 (socket says 172.21.15.139:39734) 2024-09-16T01:30:57.170 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.095+0000 7f778affd640 1 -- 172.21.15.139:0/3767700789 learned_addr learned my addr 172.21.15.139:0/3767700789 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:57.170 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.095+0000 7f778affd640 1 -- 172.21.15.139:0/3767700789 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778c19e470 con 0x7f778c1065f0 2024-09-16T01:30:57.170 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.096+0000 7f778affd640 1 --2- 172.21.15.139:0/3767700789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778c1065f0 0x7f778c07ba50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f777c007ce0 tx=0x7f777c031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:57.170 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.096+0000 7f779092b640 1 -- 172.21.15.139:0/3767700789 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f777c036050 con 0x7f778c1065f0 2024-09-16T01:30:57.170 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.096+0000 7f779092b640 1 -- 172.21.15.139:0/3767700789 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f777c042070 con 0x7f778c1065f0 2024-09-16T01:30:57.170 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.096+0000 7f779192d640 1 -- 172.21.15.139:0/3767700789 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778c19e640 con 0x7f778c1065f0 2024-09-16T01:30:57.170 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.096+0000 7f779092b640 1 -- 172.21.15.139:0/3767700789 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f777c03d8a0 con 0x7f778c1065f0 2024-09-16T01:30:57.170 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.096+0000 7f779192d640 1 -- 172.21.15.139:0/3767700789 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f778c19e9e0 con 0x7f778c1065f0 2024-09-16T01:30:57.170 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.097+0000 7f779092b640 1 -- 172.21.15.139:0/3767700789 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f777c03db50 con 0x7f778c1065f0 2024-09-16T01:30:57.170 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.097+0000 7f779192d640 1 -- 172.21.15.139:0/3767700789 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f778c106a70 con 0x7f778c1065f0 2024-09-16T01:30:57.171 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.097+0000 7f779092b640 1 --2- 172.21.15.139:0/3767700789 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f7778039200 0x7f777803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:57.171 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.097+0000 7f779092b640 1 -- 172.21.15.139:0/3767700789 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f777c007e00 con 0x7f778c1065f0 2024-09-16T01:30:57.171 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.097+0000 7f778a7fc640 1 --2- 172.21.15.139:0/3767700789 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f7778039200 0x7f777803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:57.171 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.097+0000 7f778a7fc640 1 --2- 172.21.15.139:0/3767700789 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f7778039200 0x7f777803b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f77740098c0 tx=0x7f7774006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:57.171 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.102+0000 7f779092b640 1 -- 172.21.15.139:0/3767700789 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f777c044310 con 0x7f778c1065f0 2024-09-16T01:30:57.171 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.308+0000 7f779192d640 1 -- 172.21.15.139:0/3767700789 --> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f778c103080 con 0x7f7778039200 2024-09-16T01:30:57.171 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.315+0000 7f779092b640 1 -- 172.21.15.139:0/3767700789 <== mgr.14118 v2:172.21.15.139:6800/1485008061 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f778c103080 con 0x7f7778039200 2024-09-16T01:30:57.171 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.317+0000 7f779192d640 1 -- 172.21.15.139:0/3767700789 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f7778039200 msgr2=0x7f777803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:57.171 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.317+0000 7f779192d640 1 --2- 172.21.15.139:0/3767700789 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f7778039200 0x7f777803b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f77740098c0 tx=0x7f7774006db0 comp rx=0 tx=0).stop 2024-09-16T01:30:57.171 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.317+0000 7f779192d640 1 -- 172.21.15.139:0/3767700789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778c1065f0 msgr2=0x7f778c07ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:57.171 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.317+0000 7f779192d640 1 --2- 172.21.15.139:0/3767700789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778c1065f0 0x7f778c07ba50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f777c007ce0 tx=0x7f777c031ca0 comp rx=0 tx=0).stop 2024-09-16T01:30:57.171 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.317+0000 7f779192d640 1 -- 172.21.15.139:0/3767700789 shutdown_connections 2024-09-16T01:30:57.172 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.317+0000 7f779192d640 1 --2- 172.21.15.139:0/3767700789 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f7778039200 0x7f777803b6c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:57.172 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.317+0000 7f779192d640 1 --2- 172.21.15.139:0/3767700789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778c1065f0 0x7f778c07ba50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:57.172 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.317+0000 7f779192d640 1 -- 172.21.15.139:0/3767700789 >> 172.21.15.139:0/3767700789 conn(0x7f778c101e20 msgr2=0x7f778c102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:57.172 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.317+0000 7f779192d640 1 -- 172.21.15.139:0/3767700789 shutdown_connections 2024-09-16T01:30:57.172 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:56.317+0000 7f779192d640 1 -- 172.21.15.139:0/3767700789 wait complete. 2024-09-16T01:30:57.172 INFO:teuthology.orchestra.run.smithi139.stdout:Deploying unmanaged mgr service... 2024-09-16T01:30:57.421 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:57 smithi139 ceph-mon[31175]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:30:57.421 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:57 smithi139 ceph-mon[31175]: Saving service mon spec with placement count:5 2024-09-16T01:30:57.421 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:57 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' 2024-09-16T01:30:58.434 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T01:30:58.434 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.632+0000 7f47c352a640 1 Processor -- start 2024-09-16T01:30:58.434 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.632+0000 7f47c352a640 1 -- start start 2024-09-16T01:30:58.435 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.632+0000 7f47c352a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc1076f0 0x7f47bc107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:58.435 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.632+0000 7f47c352a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1080c0 con 0x7f47bc1076f0 2024-09-16T01:30:58.435 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.632+0000 7f47c12c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc1076f0 0x7f47bc107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:58.435 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.632+0000 7f47c12c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc1076f0 0x7f47bc107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39736/0 (socket says 172.21.15.139:39736) 2024-09-16T01:30:58.435 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.632+0000 7f47c12c6640 1 -- 172.21.15.139:0/1903188537 learned_addr learned my addr 172.21.15.139:0/1903188537 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:58.435 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.633+0000 7f47c12c6640 1 -- 172.21.15.139:0/1903188537 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc1088f0 con 0x7f47bc1076f0 2024-09-16T01:30:58.435 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.633+0000 7f47c12c6640 1 --2- 172.21.15.139:0/1903188537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc1076f0 0x7f47bc107af0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009820 tx=0x7f47b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d37b8ffd5e27b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:58.435 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.633+0000 7f47b3fff640 1 -- 172.21.15.139:0/1903188537 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47b8037070 con 0x7f47bc1076f0 2024-09-16T01:30:58.435 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.633+0000 7f47b3fff640 1 -- 172.21.15.139:0/1903188537 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f47b8032900 con 0x7f47bc1076f0 2024-09-16T01:30:58.435 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.633+0000 7f47b3fff640 1 -- 172.21.15.139:0/1903188537 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47b8030850 con 0x7f47bc1076f0 2024-09-16T01:30:58.435 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.634+0000 7f47c352a640 1 -- 172.21.15.139:0/1903188537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc1076f0 msgr2=0x7f47bc107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:58.435 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.634+0000 7f47c352a640 1 --2- 172.21.15.139:0/1903188537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc1076f0 0x7f47bc107af0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009820 tx=0x7f47b802a270 comp rx=0 tx=0).stop 2024-09-16T01:30:58.435 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.634+0000 7f47c352a640 1 -- 172.21.15.139:0/1903188537 shutdown_connections 2024-09-16T01:30:58.436 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.634+0000 7f47c352a640 1 --2- 172.21.15.139:0/1903188537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc1076f0 0x7f47bc107af0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:58.436 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.634+0000 7f47c352a640 1 -- 172.21.15.139:0/1903188537 >> 172.21.15.139:0/1903188537 conn(0x7f47bc102ec0 msgr2=0x7f47bc1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:58.436 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.634+0000 7f47c352a640 1 -- 172.21.15.139:0/1903188537 shutdown_connections 2024-09-16T01:30:58.436 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.634+0000 7f47c352a640 1 -- 172.21.15.139:0/1903188537 wait complete. 2024-09-16T01:30:58.436 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.635+0000 7f47c352a640 1 Processor -- start 2024-09-16T01:30:58.436 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.635+0000 7f47c352a640 1 -- start start 2024-09-16T01:30:58.436 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.635+0000 7f47c352a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc1076f0 0x7f47bc19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:58.436 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.635+0000 7f47c352a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc10bca0 con 0x7f47bc1076f0 2024-09-16T01:30:58.436 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.635+0000 7f47c12c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc1076f0 0x7f47bc19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:58.436 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.635+0000 7f47c12c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc1076f0 0x7f47bc19b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39740/0 (socket says 172.21.15.139:39740) 2024-09-16T01:30:58.436 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.635+0000 7f47c12c6640 1 -- 172.21.15.139:0/2449182441 learned_addr learned my addr 172.21.15.139:0/2449182441 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:30:58.436 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.635+0000 7f47c12c6640 1 -- 172.21.15.139:0/2449182441 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc19bc20 con 0x7f47bc1076f0 2024-09-16T01:30:58.436 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.635+0000 7f47c12c6640 1 --2- 172.21.15.139:0/2449182441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc1076f0 0x7f47bc19b6e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009940 tx=0x7f47b8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:58.437 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.635+0000 7f47b27fc640 1 -- 172.21.15.139:0/2449182441 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47b8042070 con 0x7f47bc1076f0 2024-09-16T01:30:58.437 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.636+0000 7f47b27fc640 1 -- 172.21.15.139:0/2449182441 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f47b80308c0 con 0x7f47bc1076f0 2024-09-16T01:30:58.437 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.636+0000 7f47c352a640 1 -- 172.21.15.139:0/2449182441 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47bc19beb0 con 0x7f47bc1076f0 2024-09-16T01:30:58.437 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.636+0000 7f47b27fc640 1 -- 172.21.15.139:0/2449182441 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47b8037040 con 0x7f47bc1076f0 2024-09-16T01:30:58.437 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.636+0000 7f47c352a640 1 -- 172.21.15.139:0/2449182441 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47bc19eba0 con 0x7f47bc1076f0 2024-09-16T01:30:58.437 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.636+0000 7f47b27fc640 1 -- 172.21.15.139:0/2449182441 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f47b8045070 con 0x7f47bc1076f0 2024-09-16T01:30:58.437 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.636+0000 7f47c352a640 1 -- 172.21.15.139:0/2449182441 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47bc10be20 con 0x7f47bc1076f0 2024-09-16T01:30:58.437 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.637+0000 7f47b27fc640 1 --2- 172.21.15.139:0/2449182441 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f47a8039200 0x7f47a803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:30:58.437 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.637+0000 7f47b27fc640 1 -- 172.21.15.139:0/2449182441 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f47b806c440 con 0x7f47bc1076f0 2024-09-16T01:30:58.437 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.637+0000 7f47c0ac5640 1 --2- 172.21.15.139:0/2449182441 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f47a8039200 0x7f47a803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:30:58.437 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.637+0000 7f47c0ac5640 1 --2- 172.21.15.139:0/2449182441 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f47a8039200 0x7f47a803b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0098c0 tx=0x7f47ac006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:30:58.437 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.641+0000 7f47b27fc640 1 -- 172.21.15.139:0/2449182441 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47b8045360 con 0x7f47bc1076f0 2024-09-16T01:30:58.437 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.827+0000 7f47c352a640 1 -- 172.21.15.139:0/2449182441 --> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f47bc1040e0 con 0x7f47a8039200 2024-09-16T01:30:58.438 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.831+0000 7f47b27fc640 1 -- 172.21.15.139:0/2449182441 <== mgr.14118 v2:172.21.15.139:6800/1485008061 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f47bc1040e0 con 0x7f47a8039200 2024-09-16T01:30:58.438 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.833+0000 7f47c352a640 1 -- 172.21.15.139:0/2449182441 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f47a8039200 msgr2=0x7f47a803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:58.438 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.833+0000 7f47c352a640 1 --2- 172.21.15.139:0/2449182441 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f47a8039200 0x7f47a803b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0098c0 tx=0x7f47ac006db0 comp rx=0 tx=0).stop 2024-09-16T01:30:58.438 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.833+0000 7f47c352a640 1 -- 172.21.15.139:0/2449182441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc1076f0 msgr2=0x7f47bc19b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:30:58.438 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.833+0000 7f47c352a640 1 --2- 172.21.15.139:0/2449182441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc1076f0 0x7f47bc19b6e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009940 tx=0x7f47b8032ce0 comp rx=0 tx=0).stop 2024-09-16T01:30:58.438 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.833+0000 7f47c352a640 1 -- 172.21.15.139:0/2449182441 shutdown_connections 2024-09-16T01:30:58.438 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.833+0000 7f47c352a640 1 --2- 172.21.15.139:0/2449182441 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f47a8039200 0x7f47a803b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:58.438 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.833+0000 7f47c352a640 1 --2- 172.21.15.139:0/2449182441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc1076f0 0x7f47bc19b6e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:30:58.438 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.833+0000 7f47c352a640 1 -- 172.21.15.139:0/2449182441 >> 172.21.15.139:0/2449182441 conn(0x7f47bc102ec0 msgr2=0x7f47bc1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:30:58.438 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.834+0000 7f47c352a640 1 -- 172.21.15.139:0/2449182441 shutdown_connections 2024-09-16T01:30:58.438 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:57.834+0000 7f47c352a640 1 -- 172.21.15.139:0/2449182441 wait complete. 2024-09-16T01:30:59.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:58 smithi139 ceph-mon[31175]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:30:59.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:58 smithi139 ceph-mon[31175]: Saving service mgr spec with placement count:2 2024-09-16T01:30:59.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:30:58 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' 2024-09-16T01:31:00.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:00 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1231319941' entity='client.admin' 2024-09-16T01:31:00.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:00 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' 2024-09-16T01:31:00.213 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.851+0000 7ff20db56640 1 Processor -- start 2024-09-16T01:31:00.213 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.851+0000 7ff20db56640 1 -- start start 2024-09-16T01:31:00.213 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.851+0000 7ff20db56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff208107720 0x7ff208107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:00.213 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.851+0000 7ff20db56640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2081080f0 con 0x7ff208107720 2024-09-16T01:31:00.214 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.852+0000 7ff2077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff208107720 0x7ff208107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:00.214 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.852+0000 7ff2077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff208107720 0x7ff208107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39748/0 (socket says 172.21.15.139:39748) 2024-09-16T01:31:00.214 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.852+0000 7ff2077fe640 1 -- 172.21.15.139:0/4133577078 learned_addr learned my addr 172.21.15.139:0/4133577078 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:00.214 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.852+0000 7ff2077fe640 1 -- 172.21.15.139:0/4133577078 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2081088d0 con 0x7ff208107720 2024-09-16T01:31:00.214 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.853+0000 7ff2077fe640 1 --2- 172.21.15.139:0/4133577078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff208107720 0x7ff208107b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009820 tx=0x7ff1f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f690274d09cb924e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:00.214 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.853+0000 7ff206ffd640 1 -- 172.21.15.139:0/4133577078 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1f8037070 con 0x7ff208107720 2024-09-16T01:31:00.214 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.853+0000 7ff206ffd640 1 -- 172.21.15.139:0/4133577078 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff1f8032900 con 0x7ff208107720 2024-09-16T01:31:00.214 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.853+0000 7ff206ffd640 1 -- 172.21.15.139:0/4133577078 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1f8030850 con 0x7ff208107720 2024-09-16T01:31:00.214 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.853+0000 7ff20db56640 1 -- 172.21.15.139:0/4133577078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff208107720 msgr2=0x7ff208107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:00.214 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.853+0000 7ff20db56640 1 --2- 172.21.15.139:0/4133577078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff208107720 0x7ff208107b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009820 tx=0x7ff1f802a270 comp rx=0 tx=0).stop 2024-09-16T01:31:00.214 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.853+0000 7ff20db56640 1 -- 172.21.15.139:0/4133577078 shutdown_connections 2024-09-16T01:31:00.214 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.853+0000 7ff20db56640 1 --2- 172.21.15.139:0/4133577078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff208107720 0x7ff208107b20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:00.215 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.853+0000 7ff20db56640 1 -- 172.21.15.139:0/4133577078 >> 172.21.15.139:0/4133577078 conn(0x7ff208102ef0 msgr2=0x7ff208105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:00.215 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.853+0000 7ff20db56640 1 -- 172.21.15.139:0/4133577078 shutdown_connections 2024-09-16T01:31:00.215 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.853+0000 7ff20db56640 1 -- 172.21.15.139:0/4133577078 wait complete. 2024-09-16T01:31:00.215 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.854+0000 7ff20db56640 1 Processor -- start 2024-09-16T01:31:00.215 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.854+0000 7ff20db56640 1 -- start start 2024-09-16T01:31:00.215 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.854+0000 7ff20db56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff208107720 0x7ff20819b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:00.215 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.854+0000 7ff20db56640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff20810bc80 con 0x7ff208107720 2024-09-16T01:31:00.215 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.854+0000 7ff2077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff208107720 0x7ff20819b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:00.215 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.854+0000 7ff2077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff208107720 0x7ff20819b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39760/0 (socket says 172.21.15.139:39760) 2024-09-16T01:31:00.215 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.854+0000 7ff2077fe640 1 -- 172.21.15.139:0/1231319941 learned_addr learned my addr 172.21.15.139:0/1231319941 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:00.215 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.854+0000 7ff2077fe640 1 -- 172.21.15.139:0/1231319941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20819bc00 con 0x7ff208107720 2024-09-16T01:31:00.215 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.855+0000 7ff2077fe640 1 --2- 172.21.15.139:0/1231319941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff208107720 0x7ff20819b6c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009940 tx=0x7ff1f8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:00.215 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.855+0000 7ff2057fa640 1 -- 172.21.15.139:0/1231319941 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1f8042070 con 0x7ff208107720 2024-09-16T01:31:00.216 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.855+0000 7ff2057fa640 1 -- 172.21.15.139:0/1231319941 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff1f80308c0 con 0x7ff208107720 2024-09-16T01:31:00.216 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.855+0000 7ff2057fa640 1 -- 172.21.15.139:0/1231319941 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1f8037040 con 0x7ff208107720 2024-09-16T01:31:00.216 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.855+0000 7ff20db56640 1 -- 172.21.15.139:0/1231319941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff20819be90 con 0x7ff208107720 2024-09-16T01:31:00.216 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.855+0000 7ff20db56640 1 -- 172.21.15.139:0/1231319941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff20819eb80 con 0x7ff208107720 2024-09-16T01:31:00.216 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.856+0000 7ff2057fa640 1 -- 172.21.15.139:0/1231319941 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7ff1f8045070 con 0x7ff208107720 2024-09-16T01:31:00.216 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.856+0000 7ff2057fa640 1 --2- 172.21.15.139:0/1231319941 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff1f4039200 0x7ff1f403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:00.216 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.856+0000 7ff2057fa640 1 -- 172.21.15.139:0/1231319941 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff1f806c4a0 con 0x7ff208107720 2024-09-16T01:31:00.216 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.856+0000 7ff20db56640 1 -- 172.21.15.139:0/1231319941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff20810be00 con 0x7ff208107720 2024-09-16T01:31:00.216 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.856+0000 7ff1fffff640 1 --2- 172.21.15.139:0/1231319941 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff1f4039200 0x7ff1f403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:00.216 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.856+0000 7ff1fffff640 1 --2- 172.21.15.139:0/1231319941 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff1f4039200 0x7ff1f403b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00098c0 tx=0x7ff1f0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:00.216 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:58.861+0000 7ff2057fa640 1 -- 172.21.15.139:0/1231319941 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff1f8045360 con 0x7ff208107720 2024-09-16T01:31:00.216 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:59.029+0000 7ff20db56640 1 -- 172.21.15.139:0/1231319941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7ff208107ba0 con 0x7ff208107720 2024-09-16T01:31:00.217 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:59.034+0000 7ff2057fa640 1 -- 172.21.15.139:0/1231319941 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7ff1f803c760 con 0x7ff208107720 2024-09-16T01:31:00.217 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:59.037+0000 7ff20db56640 1 -- 172.21.15.139:0/1231319941 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff1f4039200 msgr2=0x7ff1f403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:00.217 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:59.037+0000 7ff20db56640 1 --2- 172.21.15.139:0/1231319941 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff1f4039200 0x7ff1f403b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00098c0 tx=0x7ff1f0006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:00.217 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:59.037+0000 7ff20db56640 1 -- 172.21.15.139:0/1231319941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff208107720 msgr2=0x7ff20819b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:00.217 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:59.037+0000 7ff20db56640 1 --2- 172.21.15.139:0/1231319941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff208107720 0x7ff20819b6c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009940 tx=0x7ff1f8032ce0 comp rx=0 tx=0).stop 2024-09-16T01:31:00.217 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:59.037+0000 7ff20db56640 1 -- 172.21.15.139:0/1231319941 shutdown_connections 2024-09-16T01:31:00.217 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:59.037+0000 7ff20db56640 1 --2- 172.21.15.139:0/1231319941 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff1f4039200 0x7ff1f403b6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:00.217 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:59.037+0000 7ff20db56640 1 --2- 172.21.15.139:0/1231319941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff208107720 0x7ff20819b6c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:00.217 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:59.037+0000 7ff20db56640 1 -- 172.21.15.139:0/1231319941 >> 172.21.15.139:0/1231319941 conn(0x7ff208102ef0 msgr2=0x7ff208103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:00.217 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:59.037+0000 7ff20db56640 1 -- 172.21.15.139:0/1231319941 shutdown_connections 2024-09-16T01:31:00.217 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:30:59.037+0000 7ff20db56640 1 -- 172.21.15.139:0/1231319941 wait complete. 2024-09-16T01:31:01.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:01 smithi139 ceph-mon[31175]: from='mgr.14118 172.21.15.139:0/1295454937' entity='mgr.x' 2024-09-16T01:31:01.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:01 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2753597787' entity='client.admin' 2024-09-16T01:31:01.688 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.618+0000 7f4aaca85640 1 Processor -- start 2024-09-16T01:31:01.688 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.619+0000 7f4aaca85640 1 -- start start 2024-09-16T01:31:01.688 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.619+0000 7f4aaca85640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa8107740 0x7f4aa8107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:01.688 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.619+0000 7f4aaca85640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa8108110 con 0x7f4aa8107740 2024-09-16T01:31:01.688 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.619+0000 7f4aa659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa8107740 0x7f4aa8107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:01.688 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.619+0000 7f4aa659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa8107740 0x7f4aa8107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39770/0 (socket says 172.21.15.139:39770) 2024-09-16T01:31:01.688 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.619+0000 7f4aa659c640 1 -- 172.21.15.139:0/3701116435 learned_addr learned my addr 172.21.15.139:0/3701116435 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:01.689 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.619+0000 7f4aa659c640 1 -- 172.21.15.139:0/3701116435 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa81088f0 con 0x7f4aa8107740 2024-09-16T01:31:01.689 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.620+0000 7f4aa659c640 1 --2- 172.21.15.139:0/3701116435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa8107740 0x7f4aa8107b40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009820 tx=0x7f4aa002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdac6208aa2b5cd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:01.689 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.620+0000 7f4aa559a640 1 -- 172.21.15.139:0/3701116435 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4aa0037070 con 0x7f4aa8107740 2024-09-16T01:31:01.689 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.620+0000 7f4aa559a640 1 -- 172.21.15.139:0/3701116435 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa0032920 con 0x7f4aa8107740 2024-09-16T01:31:01.689 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.620+0000 7f4aa559a640 1 -- 172.21.15.139:0/3701116435 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4aa0030810 con 0x7f4aa8107740 2024-09-16T01:31:01.689 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.621+0000 7f4aaca85640 1 -- 172.21.15.139:0/3701116435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa8107740 msgr2=0x7f4aa8107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:01.689 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.621+0000 7f4aaca85640 1 --2- 172.21.15.139:0/3701116435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa8107740 0x7f4aa8107b40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009820 tx=0x7f4aa002a270 comp rx=0 tx=0).stop 2024-09-16T01:31:01.689 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.621+0000 7f4aaca85640 1 -- 172.21.15.139:0/3701116435 shutdown_connections 2024-09-16T01:31:01.689 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.621+0000 7f4aaca85640 1 --2- 172.21.15.139:0/3701116435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa8107740 0x7f4aa8107b40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:01.689 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.621+0000 7f4aaca85640 1 -- 172.21.15.139:0/3701116435 >> 172.21.15.139:0/3701116435 conn(0x7f4aa8102ef0 msgr2=0x7f4aa8105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:01.689 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.621+0000 7f4aaca85640 1 -- 172.21.15.139:0/3701116435 shutdown_connections 2024-09-16T01:31:01.689 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.621+0000 7f4aaca85640 1 -- 172.21.15.139:0/3701116435 wait complete. 2024-09-16T01:31:01.689 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.622+0000 7f4aaca85640 1 Processor -- start 2024-09-16T01:31:01.690 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.622+0000 7f4aaca85640 1 -- start start 2024-09-16T01:31:01.690 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.622+0000 7f4aaca85640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa8107740 0x7f4aa819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:01.690 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.622+0000 7f4aaca85640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa810bca0 con 0x7f4aa8107740 2024-09-16T01:31:01.690 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.622+0000 7f4aa659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa8107740 0x7f4aa819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:01.690 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.622+0000 7f4aa659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa8107740 0x7f4aa819b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39786/0 (socket says 172.21.15.139:39786) 2024-09-16T01:31:01.690 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.622+0000 7f4aa659c640 1 -- 172.21.15.139:0/2753597787 learned_addr learned my addr 172.21.15.139:0/2753597787 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:01.690 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.622+0000 7f4aa659c640 1 -- 172.21.15.139:0/2753597787 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa819bcc0 con 0x7f4aa8107740 2024-09-16T01:31:01.690 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.622+0000 7f4aa659c640 1 --2- 172.21.15.139:0/2753597787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa8107740 0x7f4aa819b780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00097f0 tx=0x7f4aa00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:01.690 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.622+0000 7f4a9f7fe640 1 -- 172.21.15.139:0/2753597787 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4aa0037040 con 0x7f4aa8107740 2024-09-16T01:31:01.690 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.622+0000 7f4aaca85640 1 -- 172.21.15.139:0/2753597787 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa819bf50 con 0x7f4aa8107740 2024-09-16T01:31:01.690 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.622+0000 7f4a9f7fe640 1 -- 172.21.15.139:0/2753597787 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa0030c40 con 0x7f4aa8107740 2024-09-16T01:31:01.690 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.622+0000 7f4a9f7fe640 1 -- 172.21.15.139:0/2753597787 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4aa003a5b0 con 0x7f4aa8107740 2024-09-16T01:31:01.691 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.622+0000 7f4aaca85640 1 -- 172.21.15.139:0/2753597787 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa819ec40 con 0x7f4aa8107740 2024-09-16T01:31:01.691 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.623+0000 7f4a9f7fe640 1 -- 172.21.15.139:0/2753597787 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f4aa003a7f0 con 0x7f4aa8107740 2024-09-16T01:31:01.691 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.623+0000 7f4aaca85640 1 -- 172.21.15.139:0/2753597787 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa810be20 con 0x7f4aa8107740 2024-09-16T01:31:01.691 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.624+0000 7f4a9f7fe640 1 --2- 172.21.15.139:0/2753597787 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f4a940391b0 0x7f4a9403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:01.691 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.624+0000 7f4a9f7fe640 1 -- 172.21.15.139:0/2753597787 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4aa006c800 con 0x7f4aa8107740 2024-09-16T01:31:01.691 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.624+0000 7f4aa5d9b640 1 --2- 172.21.15.139:0/2753597787 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f4a940391b0 0x7f4a9403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:01.691 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.624+0000 7f4aa5d9b640 1 --2- 172.21.15.139:0/2753597787 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f4a940391b0 0x7f4a9403b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4a900098c0 tx=0x7f4a90006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:01.691 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.628+0000 7f4a9f7fe640 1 -- 172.21.15.139:0/2753597787 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4aa0031c00 con 0x7f4aa8107740 2024-09-16T01:31:01.691 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.819+0000 7f4aaca85640 1 -- 172.21.15.139:0/2753597787 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f4aa8107bc0 con 0x7f4aa8107740 2024-09-16T01:31:01.691 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.821+0000 7f4a9f7fe640 1 -- 172.21.15.139:0/2753597787 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f4aa0044920 con 0x7f4aa8107740 2024-09-16T01:31:01.691 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.824+0000 7f4aaca85640 1 -- 172.21.15.139:0/2753597787 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f4a940391b0 msgr2=0x7f4a9403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:01.691 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.824+0000 7f4aaca85640 1 --2- 172.21.15.139:0/2753597787 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f4a940391b0 0x7f4a9403b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4a900098c0 tx=0x7f4a90006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:01.691 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.824+0000 7f4aaca85640 1 -- 172.21.15.139:0/2753597787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa8107740 msgr2=0x7f4aa819b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:01.692 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.824+0000 7f4aaca85640 1 --2- 172.21.15.139:0/2753597787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa8107740 0x7f4aa819b780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00097f0 tx=0x7f4aa00327b0 comp rx=0 tx=0).stop 2024-09-16T01:31:01.692 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.824+0000 7f4aaca85640 1 -- 172.21.15.139:0/2753597787 shutdown_connections 2024-09-16T01:31:01.692 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.824+0000 7f4aaca85640 1 --2- 172.21.15.139:0/2753597787 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7f4a940391b0 0x7f4a9403b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:01.692 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.824+0000 7f4aaca85640 1 --2- 172.21.15.139:0/2753597787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa8107740 0x7f4aa819b780 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:01.692 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.824+0000 7f4aaca85640 1 -- 172.21.15.139:0/2753597787 >> 172.21.15.139:0/2753597787 conn(0x7f4aa8102ef0 msgr2=0x7f4aa8103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:01.692 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.824+0000 7f4aaca85640 1 -- 172.21.15.139:0/2753597787 shutdown_connections 2024-09-16T01:31:01.692 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:00.824+0000 7f4aaca85640 1 -- 172.21.15.139:0/2753597787 wait complete. 2024-09-16T01:31:01.692 INFO:teuthology.orchestra.run.smithi139.stdout:Enabling the dashboard module... 2024-09-16T01:31:02.638 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:02 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1290370140' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-16T01:31:02.638 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:02 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: ignoring --setuser ceph since I am not root 2024-09-16T01:31:02.638 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:02 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: ignoring --setgroup ceph since I am not root 2024-09-16T01:31:02.638 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:02 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:02.636+0000 7f17d2860040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:31:02.917 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:02 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:02.693+0000 7f17d2860040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:31:02.917 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:02 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:02.848+0000 7f17d2860040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:31:02.917 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:02 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:02.915+0000 7f17d2860040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:31:03.172 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:03 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:03.063+0000 7f17d2860040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:31:03.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.206+0000 7fed655c6640 1 Processor -- start 2024-09-16T01:31:03.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.206+0000 7fed655c6640 1 -- start start 2024-09-16T01:31:03.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.206+0000 7fed655c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed60105fd0 0x7fed601063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:03.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.206+0000 7fed655c6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed6007be50 con 0x7fed60105fd0 2024-09-16T01:31:03.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.206+0000 7fed5effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed60105fd0 0x7fed601063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:03.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.206+0000 7fed5effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed60105fd0 0x7fed601063d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39798/0 (socket says 172.21.15.139:39798) 2024-09-16T01:31:03.239 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.206+0000 7fed5effd640 1 -- 172.21.15.139:0/1309455537 learned_addr learned my addr 172.21.15.139:0/1309455537 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:03.240 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.206+0000 7fed5effd640 1 -- 172.21.15.139:0/1309455537 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed60106910 con 0x7fed60105fd0 2024-09-16T01:31:03.240 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.207+0000 7fed5effd640 1 --2- 172.21.15.139:0/1309455537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed60105fd0 0x7fed601063d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fed50009820 tx=0x7fed5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8b94b1c3722a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:03.240 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.207+0000 7fed5e7fc640 1 -- 172.21.15.139:0/1309455537 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed50037070 con 0x7fed60105fd0 2024-09-16T01:31:03.240 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.207+0000 7fed5e7fc640 1 -- 172.21.15.139:0/1309455537 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed50032920 con 0x7fed60105fd0 2024-09-16T01:31:03.240 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.207+0000 7fed5e7fc640 1 -- 172.21.15.139:0/1309455537 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed50030810 con 0x7fed60105fd0 2024-09-16T01:31:03.240 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.208+0000 7fed655c6640 1 -- 172.21.15.139:0/1309455537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed60105fd0 msgr2=0x7fed601063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:03.240 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.208+0000 7fed655c6640 1 --2- 172.21.15.139:0/1309455537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed60105fd0 0x7fed601063d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fed50009820 tx=0x7fed5002a270 comp rx=0 tx=0).stop 2024-09-16T01:31:03.240 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.208+0000 7fed655c6640 1 -- 172.21.15.139:0/1309455537 shutdown_connections 2024-09-16T01:31:03.240 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.208+0000 7fed655c6640 1 --2- 172.21.15.139:0/1309455537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed60105fd0 0x7fed601063d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:03.240 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.208+0000 7fed655c6640 1 -- 172.21.15.139:0/1309455537 >> 172.21.15.139:0/1309455537 conn(0x7fed60101da0 msgr2=0x7fed601041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:03.240 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.208+0000 7fed655c6640 1 -- 172.21.15.139:0/1309455537 shutdown_connections 2024-09-16T01:31:03.240 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.208+0000 7fed655c6640 1 -- 172.21.15.139:0/1309455537 wait complete. 2024-09-16T01:31:03.240 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.209+0000 7fed655c6640 1 Processor -- start 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.209+0000 7fed655c6640 1 -- start start 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.209+0000 7fed655c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed60105fd0 0x7fed6019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.209+0000 7fed655c6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed60107a50 con 0x7fed60105fd0 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.209+0000 7fed5effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed60105fd0 0x7fed6019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.209+0000 7fed5effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed60105fd0 0x7fed6019d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39808/0 (socket says 172.21.15.139:39808) 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.209+0000 7fed5effd640 1 -- 172.21.15.139:0/1290370140 learned_addr learned my addr 172.21.15.139:0/1290370140 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.209+0000 7fed5effd640 1 -- 172.21.15.139:0/1290370140 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6019de50 con 0x7fed60105fd0 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.209+0000 7fed5effd640 1 --2- 172.21.15.139:0/1290370140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed60105fd0 0x7fed6019d910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fed50030420 tx=0x7fed50032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.210+0000 7fed5cff9640 1 -- 172.21.15.139:0/1290370140 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed50037040 con 0x7fed60105fd0 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.210+0000 7fed5cff9640 1 -- 172.21.15.139:0/1290370140 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed50040070 con 0x7fed60105fd0 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.210+0000 7fed655c6640 1 -- 172.21.15.139:0/1290370140 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6019e0e0 con 0x7fed60105fd0 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.210+0000 7fed5cff9640 1 -- 172.21.15.139:0/1290370140 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed5003a6e0 con 0x7fed60105fd0 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.210+0000 7fed655c6640 1 -- 172.21.15.139:0/1290370140 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed601a0dd0 con 0x7fed60105fd0 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.211+0000 7fed5cff9640 1 -- 172.21.15.139:0/1290370140 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fed5003a990 con 0x7fed60105fd0 2024-09-16T01:31:03.241 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.211+0000 7fed655c6640 1 -- 172.21.15.139:0/1290370140 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed601063d0 con 0x7fed60105fd0 2024-09-16T01:31:03.242 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.211+0000 7fed5cff9640 1 --2- 172.21.15.139:0/1290370140 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fed4c039200 0x7fed4c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:03.242 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.211+0000 7fed5cff9640 1 -- 172.21.15.139:0/1290370140 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fed5006c960 con 0x7fed60105fd0 2024-09-16T01:31:03.242 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.211+0000 7fed57fff640 1 --2- 172.21.15.139:0/1290370140 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fed4c039200 0x7fed4c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:03.242 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.211+0000 7fed57fff640 1 --2- 172.21.15.139:0/1290370140 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fed4c039200 0x7fed4c03b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fed480098c0 tx=0x7fed48006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:03.242 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.216+0000 7fed5cff9640 1 -- 172.21.15.139:0/1290370140 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed50044c70 con 0x7fed60105fd0 2024-09-16T01:31:03.242 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.426+0000 7fed655c6640 1 -- 172.21.15.139:0/1290370140 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fed60196d80 con 0x7fed60105fd0 2024-09-16T01:31:03.242 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.478+0000 7fed5cff9640 1 -- 172.21.15.139:0/1290370140 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fed500439b0 con 0x7fed60105fd0 2024-09-16T01:31:03.242 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.479+0000 7fed5cff9640 1 -- 172.21.15.139:0/1290370140 <== mon.0 v2:172.21.15.139:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fed50045980 con 0x7fed60105fd0 2024-09-16T01:31:03.242 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.481+0000 7fed655c6640 1 -- 172.21.15.139:0/1290370140 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fed4c039200 msgr2=0x7fed4c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:03.242 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.481+0000 7fed655c6640 1 --2- 172.21.15.139:0/1290370140 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fed4c039200 0x7fed4c03b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fed480098c0 tx=0x7fed48006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:03.242 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.481+0000 7fed655c6640 1 -- 172.21.15.139:0/1290370140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed60105fd0 msgr2=0x7fed6019d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:03.242 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.481+0000 7fed655c6640 1 --2- 172.21.15.139:0/1290370140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed60105fd0 0x7fed6019d910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fed50030420 tx=0x7fed50032670 comp rx=0 tx=0).stop 2024-09-16T01:31:03.242 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.481+0000 7fed655c6640 1 -- 172.21.15.139:0/1290370140 shutdown_connections 2024-09-16T01:31:03.243 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.481+0000 7fed655c6640 1 --2- 172.21.15.139:0/1290370140 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7fed4c039200 0x7fed4c03b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:03.243 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.481+0000 7fed655c6640 1 --2- 172.21.15.139:0/1290370140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed60105fd0 0x7fed6019d910 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:03.243 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.481+0000 7fed655c6640 1 -- 172.21.15.139:0/1290370140 >> 172.21.15.139:0/1290370140 conn(0x7fed60101da0 msgr2=0x7fed60191c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:03.243 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.481+0000 7fed655c6640 1 -- 172.21.15.139:0/1290370140 shutdown_connections 2024-09-16T01:31:03.243 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:02.481+0000 7fed655c6640 1 -- 172.21.15.139:0/1290370140 wait complete. 2024-09-16T01:31:03.467 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:03 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:03.203+0000 7f17d2860040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:31:03.777 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:03 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1290370140' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T01:31:03.777 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:03 smithi139 ceph-mon[31175]: mgrmap e9: x(active, since 17s) 2024-09-16T01:31:03.777 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:03 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:03.523+0000 7f17d2860040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:31:03.777 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:03 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:03.583+0000 7f17d2860040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:31:03.777 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:03 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:03.775+0000 7f17d2860040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:31:04.595 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:31:04.595 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T01:31:04.596 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T01:31:04.596 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-16T01:31:04.596 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T01:31:04.596 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:31:04.596 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.660+0000 7ff3adacf640 1 Processor -- start 2024-09-16T01:31:04.596 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.660+0000 7ff3adacf640 1 -- start start 2024-09-16T01:31:04.596 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.660+0000 7ff3adacf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a8105990 0x7ff3a8105d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:04.596 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.660+0000 7ff3adacf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a8106360 con 0x7ff3a8105990 2024-09-16T01:31:04.596 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.660+0000 7ff3a77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a8105990 0x7ff3a8105d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:04.596 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.661+0000 7ff3a77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a8105990 0x7ff3a8105d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60458/0 (socket says 172.21.15.139:60458) 2024-09-16T01:31:04.596 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.661+0000 7ff3a77fe640 1 -- 172.21.15.139:0/1747402050 learned_addr learned my addr 172.21.15.139:0/1747402050 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:04.596 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.661+0000 7ff3a77fe640 1 -- 172.21.15.139:0/1747402050 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a8108970 con 0x7ff3a8105990 2024-09-16T01:31:04.596 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.661+0000 7ff3a77fe640 1 --2- 172.21.15.139:0/1747402050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a8105990 0x7ff3a8105d90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff398009820 tx=0x7ff39802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90e92a41343ae21 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:04.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.662+0000 7ff3a67fc640 1 -- 172.21.15.139:0/1747402050 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff398037070 con 0x7ff3a8105990 2024-09-16T01:31:04.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.662+0000 7ff3a67fc640 1 -- 172.21.15.139:0/1747402050 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff398032920 con 0x7ff3a8105990 2024-09-16T01:31:04.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.662+0000 7ff3a67fc640 1 -- 172.21.15.139:0/1747402050 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff398030810 con 0x7ff3a8105990 2024-09-16T01:31:04.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.662+0000 7ff3adacf640 1 -- 172.21.15.139:0/1747402050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a8105990 msgr2=0x7ff3a8105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:04.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.662+0000 7ff3adacf640 1 --2- 172.21.15.139:0/1747402050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a8105990 0x7ff3a8105d90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff398009820 tx=0x7ff39802a270 comp rx=0 tx=0).stop 2024-09-16T01:31:04.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.662+0000 7ff3adacf640 1 -- 172.21.15.139:0/1747402050 shutdown_connections 2024-09-16T01:31:04.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.662+0000 7ff3adacf640 1 --2- 172.21.15.139:0/1747402050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a8105990 0x7ff3a8105d90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:04.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.662+0000 7ff3adacf640 1 -- 172.21.15.139:0/1747402050 >> 172.21.15.139:0/1747402050 conn(0x7ff3a81014e0 msgr2=0x7ff3a8103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:04.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.662+0000 7ff3adacf640 1 -- 172.21.15.139:0/1747402050 shutdown_connections 2024-09-16T01:31:04.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.662+0000 7ff3adacf640 1 -- 172.21.15.139:0/1747402050 wait complete. 2024-09-16T01:31:04.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.663+0000 7ff3adacf640 1 Processor -- start 2024-09-16T01:31:04.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.663+0000 7ff3adacf640 1 -- start start 2024-09-16T01:31:04.597 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.663+0000 7ff3adacf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a8105990 0x7ff3a819b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:04.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.663+0000 7ff3adacf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a810bba0 con 0x7ff3a8105990 2024-09-16T01:31:04.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.663+0000 7ff3a77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a8105990 0x7ff3a819b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:04.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.663+0000 7ff3a77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a8105990 0x7ff3a819b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60474/0 (socket says 172.21.15.139:60474) 2024-09-16T01:31:04.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.663+0000 7ff3a77fe640 1 -- 172.21.15.139:0/1634936676 learned_addr learned my addr 172.21.15.139:0/1634936676 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:04.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.664+0000 7ff3a77fe640 1 -- 172.21.15.139:0/1634936676 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a819bca0 con 0x7ff3a8105990 2024-09-16T01:31:04.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.664+0000 7ff3a77fe640 1 --2- 172.21.15.139:0/1634936676 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a8105990 0x7ff3a819b760 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff398030560 tx=0x7ff398032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:04.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.664+0000 7ff3a4ff9640 1 -- 172.21.15.139:0/1634936676 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff398037040 con 0x7ff3a8105990 2024-09-16T01:31:04.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.664+0000 7ff3adacf640 1 -- 172.21.15.139:0/1634936676 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a819bf30 con 0x7ff3a8105990 2024-09-16T01:31:04.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.664+0000 7ff3a4ff9640 1 -- 172.21.15.139:0/1634936676 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff398040070 con 0x7ff3a8105990 2024-09-16T01:31:04.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.664+0000 7ff3adacf640 1 -- 172.21.15.139:0/1634936676 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a819ec20 con 0x7ff3a8105990 2024-09-16T01:31:04.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.664+0000 7ff3a4ff9640 1 -- 172.21.15.139:0/1634936676 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff39803a6d0 con 0x7ff3a8105990 2024-09-16T01:31:04.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.665+0000 7ff3a4ff9640 1 -- 172.21.15.139:0/1634936676 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7ff39803a870 con 0x7ff3a8105990 2024-09-16T01:31:04.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.665+0000 7ff3adacf640 1 -- 172.21.15.139:0/1634936676 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a8105e10 con 0x7ff3a8105990 2024-09-16T01:31:04.598 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.665+0000 7ff3a4ff9640 1 --2- 172.21.15.139:0/1634936676 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff390039260 0x7ff39003b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.665+0000 7ff3a4ff9640 1 -- 172.21.15.139:0/1634936676 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff39806ca90 con 0x7ff3a8105990 2024-09-16T01:31:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.665+0000 7ff3a6ffd640 1 -- 172.21.15.139:0/1634936676 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff390039260 msgr2=0x7ff39003b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/1485008061 2024-09-16T01:31:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.665+0000 7ff3a6ffd640 1 --2- 172.21.15.139:0/1634936676 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff390039260 0x7ff39003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:31:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.670+0000 7ff3a4ff9640 1 -- 172.21.15.139:0/1634936676 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff39806c3e0 con 0x7ff3a8105990 2024-09-16T01:31:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.866+0000 7ff3a6ffd640 1 -- 172.21.15.139:0/1634936676 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff390039260 msgr2=0x7ff39003b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/1485008061 2024-09-16T01:31:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.866+0000 7ff3a6ffd640 1 --2- 172.21.15.139:0/1634936676 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff390039260 0x7ff39003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:31:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.884+0000 7ff3adacf640 1 -- 172.21.15.139:0/1634936676 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff3a819edf0 con 0x7ff3a8105990 2024-09-16T01:31:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.884+0000 7ff3a4ff9640 1 -- 172.21.15.139:0/1634936676 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7ff3a819edf0 con 0x7ff3a8105990 2024-09-16T01:31:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.887+0000 7ff3adacf640 1 -- 172.21.15.139:0/1634936676 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff390039260 msgr2=0x7ff39003b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:31:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.887+0000 7ff3adacf640 1 --2- 172.21.15.139:0/1634936676 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff390039260 0x7ff39003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.887+0000 7ff3adacf640 1 -- 172.21.15.139:0/1634936676 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a8105990 msgr2=0x7ff3a819b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.887+0000 7ff3adacf640 1 --2- 172.21.15.139:0/1634936676 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a8105990 0x7ff3a819b760 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff398030560 tx=0x7ff398032670 comp rx=0 tx=0).stop 2024-09-16T01:31:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.887+0000 7ff3adacf640 1 -- 172.21.15.139:0/1634936676 shutdown_connections 2024-09-16T01:31:04.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.887+0000 7ff3adacf640 1 --2- 172.21.15.139:0/1634936676 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff390039260 0x7ff39003b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:04.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.887+0000 7ff3adacf640 1 --2- 172.21.15.139:0/1634936676 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a8105990 0x7ff3a819b760 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:04.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.887+0000 7ff3adacf640 1 -- 172.21.15.139:0/1634936676 >> 172.21.15.139:0/1634936676 conn(0x7ff3a81014e0 msgr2=0x7ff3a81020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:04.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.887+0000 7ff3adacf640 1 -- 172.21.15.139:0/1634936676 shutdown_connections 2024-09-16T01:31:04.600 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:03.887+0000 7ff3adacf640 1 -- 172.21.15.139:0/1634936676 wait complete. 2024-09-16T01:31:04.600 INFO:teuthology.orchestra.run.smithi139.stdout:Waiting for the mgr to restart... 2024-09-16T01:31:04.600 INFO:teuthology.orchestra.run.smithi139.stdout:Waiting for mgr epoch 9... 2024-09-16T01:31:04.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:04 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1634936676' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T01:31:04.979 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:04 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:04.676+0000 7f17d2860040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:31:04.979 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:04 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:04.744+0000 7f17d2860040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:31:05.587 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:05 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:05.483+0000 7f17d2860040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:31:05.923 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:05 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:05.601+0000 7f17d2860040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:31:05.923 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:05 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:05.746+0000 7f17d2860040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:31:05.923 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:05 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:05.807+0000 7f17d2860040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:31:06.245 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:05 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:05.957+0000 7f17d2860040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:31:06.245 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:06 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:06.026+0000 7f17d2860040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:31:06.245 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:06 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:06.088+0000 7f17d2860040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:31:06.245 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:06 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:06.243+0000 7f17d2860040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:31:06.513 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:06 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:06.307+0000 7f17d2860040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:31:06.513 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:06 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:06.372+0000 7f17d2860040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:31:06.513 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:06 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:06.453+0000 7f17d2860040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:31:06.513 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:06 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:06.511+0000 7f17d2860040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:31:06.922 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:06 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:06.569+0000 7f17d2860040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:31:07.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:07 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:07.341+0000 7f17d2860040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:31:07.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:07 smithi139 ceph-mon[31175]: Active manager daemon x restarted 2024-09-16T01:31:07.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:07 smithi139 ceph-mon[31175]: Activating manager daemon x 2024-09-16T01:31:07.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:07 smithi139 ceph-mon[31175]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T01:31:07.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:07 smithi139 ceph-mon[31175]: mgrmap e10: x(active, starting, since 0.00325332s) 2024-09-16T01:31:07.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:31:07.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T01:31:07.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T01:31:07.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T01:31:07.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T01:31:07.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:07 smithi139 ceph-mon[31175]: Manager daemon x is now available 2024-09-16T01:31:07.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:31:07.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T01:31:07.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T01:31:09.139 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:08.852+0000 7f173db74640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:01:31:08] ENGINE Error in HTTPServer.serve 2024-09-16T01:31:09.139 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: Traceback (most recent call last): 2024-09-16T01:31:09.139 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:31:09.139 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self._connections.run(self.expiration_interval) 2024-09-16T01:31:09.139 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:31:09.139 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self._run(expiration_interval) 2024-09-16T01:31:09.139 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:31:09.139 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:31:09.140 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:31:09.140 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:31:09.140 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:31:09.140 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: s = self.context.wrap_socket( 2024-09-16T01:31:09.140 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:31:09.140 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: return self.sslsocket_class._create( 2024-09-16T01:31:09.140 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:31:09.140 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self.do_handshake() 2024-09-16T01:31:09.140 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:31:09.141 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self._sslobj.do_handshake() 2024-09-16T01:31:09.141 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:31:09.141 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:09.147 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout { 2024-09-16T01:31:09.148 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T01:31:09.148 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T01:31:09.148 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout } 2024-09-16T01:31:09.148 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.977+0000 7ff92a532640 1 Processor -- start 2024-09-16T01:31:09.148 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.977+0000 7ff92a532640 1 -- start start 2024-09-16T01:31:09.148 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.977+0000 7ff92a532640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff92407bcf0 0x7ff92410a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:09.148 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.977+0000 7ff92a532640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff92410a850 con 0x7ff92407bcf0 2024-09-16T01:31:09.148 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.978+0000 7ff923fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff92407bcf0 0x7ff92410a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:09.148 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.978+0000 7ff923fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff92407bcf0 0x7ff92410a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60478/0 (socket says 172.21.15.139:60478) 2024-09-16T01:31:09.148 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.978+0000 7ff923fff640 1 -- 172.21.15.139:0/2136017812 learned_addr learned my addr 172.21.15.139:0/2136017812 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:09.148 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.978+0000 7ff923fff640 1 -- 172.21.15.139:0/2136017812 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92410a9d0 con 0x7ff92407bcf0 2024-09-16T01:31:09.148 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.979+0000 7ff923fff640 1 --2- 172.21.15.139:0/2136017812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff92407bcf0 0x7ff92410a280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff914009820 tx=0x7ff91402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6dd516c7ca6c737 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:09.149 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.979+0000 7ff922ffd640 1 -- 172.21.15.139:0/2136017812 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff914037070 con 0x7ff92407bcf0 2024-09-16T01:31:09.149 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.979+0000 7ff922ffd640 1 -- 172.21.15.139:0/2136017812 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff914032920 con 0x7ff92407bcf0 2024-09-16T01:31:09.149 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.979+0000 7ff922ffd640 1 -- 172.21.15.139:0/2136017812 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff914030810 con 0x7ff92407bcf0 2024-09-16T01:31:09.149 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.980+0000 7ff92a532640 1 -- 172.21.15.139:0/2136017812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff92407bcf0 msgr2=0x7ff92410a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:09.149 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.980+0000 7ff92a532640 1 --2- 172.21.15.139:0/2136017812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff92407bcf0 0x7ff92410a280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff914009820 tx=0x7ff91402a270 comp rx=0 tx=0).stop 2024-09-16T01:31:09.149 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.980+0000 7ff92a532640 1 -- 172.21.15.139:0/2136017812 shutdown_connections 2024-09-16T01:31:09.149 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.980+0000 7ff92a532640 1 --2- 172.21.15.139:0/2136017812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff92407bcf0 0x7ff92410a280 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:09.149 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.980+0000 7ff92a532640 1 -- 172.21.15.139:0/2136017812 >> 172.21.15.139:0/2136017812 conn(0x7ff924101db0 msgr2=0x7ff9241041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:09.149 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.980+0000 7ff92a532640 1 -- 172.21.15.139:0/2136017812 shutdown_connections 2024-09-16T01:31:09.149 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.980+0000 7ff92a532640 1 -- 172.21.15.139:0/2136017812 wait complete. 2024-09-16T01:31:09.149 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.981+0000 7ff92a532640 1 Processor -- start 2024-09-16T01:31:09.149 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.981+0000 7ff92a532640 1 -- start start 2024-09-16T01:31:09.149 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.981+0000 7ff92a532640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff924107ab0 0x7ff924106220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:09.150 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.981+0000 7ff92a532640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff92410bb10 con 0x7ff924107ab0 2024-09-16T01:31:09.150 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.982+0000 7ff923fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff924107ab0 0x7ff924106220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:09.150 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.982+0000 7ff923fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff924107ab0 0x7ff924106220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60494/0 (socket says 172.21.15.139:60494) 2024-09-16T01:31:09.150 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.982+0000 7ff923fff640 1 -- 172.21.15.139:0/352454751 learned_addr learned my addr 172.21.15.139:0/352454751 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:09.150 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.982+0000 7ff923fff640 1 -- 172.21.15.139:0/352454751 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff924107ed0 con 0x7ff924107ab0 2024-09-16T01:31:09.150 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.982+0000 7ff923fff640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff924107ab0 0x7ff924106220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff9140097f0 tx=0x7ff9140327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:09.150 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.982+0000 7ff9217fa640 1 -- 172.21.15.139:0/352454751 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff914037040 con 0x7ff924107ab0 2024-09-16T01:31:09.150 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.982+0000 7ff92a532640 1 -- 172.21.15.139:0/352454751 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff924106760 con 0x7ff924107ab0 2024-09-16T01:31:09.150 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.982+0000 7ff9217fa640 1 -- 172.21.15.139:0/352454751 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff914030c40 con 0x7ff924107ab0 2024-09-16T01:31:09.150 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.982+0000 7ff9217fa640 1 -- 172.21.15.139:0/352454751 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff91403a5d0 con 0x7ff924107ab0 2024-09-16T01:31:09.150 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.983+0000 7ff92a532640 1 -- 172.21.15.139:0/352454751 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff924106c40 con 0x7ff924107ab0 2024-09-16T01:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.983+0000 7ff9217fa640 1 -- 172.21.15.139:0/352454751 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7ff91403a7c0 con 0x7ff924107ab0 2024-09-16T01:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.984+0000 7ff9217fa640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff90c039260 0x7ff90c03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.984+0000 7ff9217fa640 1 -- 172.21.15.139:0/352454751 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff914030df0 con 0x7ff924107ab0 2024-09-16T01:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.984+0000 7ff9237fe640 1 -- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff90c039260 msgr2=0x7ff90c03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/1485008061 2024-09-16T01:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.984+0000 7ff9237fe640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff90c039260 0x7ff90c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:04.984+0000 7ff92a532640 1 -- 172.21.15.139:0/352454751 --> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff9241082f0 con 0x7ff90c039260 2024-09-16T01:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:05.185+0000 7ff9237fe640 1 -- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff90c039260 msgr2=0x7ff90c03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/1485008061 2024-09-16T01:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:05.185+0000 7ff9237fe640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff90c039260 0x7ff90c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:05.586+0000 7ff9237fe640 1 -- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff90c039260 msgr2=0x7ff90c03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/1485008061 2024-09-16T01:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:05.586+0000 7ff9237fe640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff90c039260 0x7ff90c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T01:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:06.387+0000 7ff9237fe640 1 -- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff90c039260 msgr2=0x7ff90c03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/1485008061 2024-09-16T01:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:06.387+0000 7ff9237fe640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff90c039260 0x7ff90c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T01:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:07.345+0000 7ff9217fa640 1 -- 172.21.15.139:0/352454751 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7ff914044a10 con 0x7ff924107ab0 2024-09-16T01:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:07.345+0000 7ff9217fa640 1 -- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff90c039260 msgr2=0x7ff90c03b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:31:09.152 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:07.345+0000 7ff9217fa640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff90c039260 0x7ff90c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:09.152 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.346+0000 7ff9217fa640 1 -- 172.21.15.139:0/352454751 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7ff914040210 con 0x7ff924107ab0 2024-09-16T01:31:09.152 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.346+0000 7ff9217fa640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff90c03cb30 0x7ff90c03ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:09.152 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.347+0000 7ff9217fa640 1 -- 172.21.15.139:0/352454751 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff91406c650 con 0x7ff90c03cb30 2024-09-16T01:31:09.152 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.347+0000 7ff9237fe640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff90c03cb30 0x7ff90c03ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:09.152 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.348+0000 7ff9237fe640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff90c03cb30 0x7ff90c03ef20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff918003980 tx=0x7ff9180092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:09.152 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.348+0000 7ff9217fa640 1 -- 172.21.15.139:0/352454751 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ff91406c650 con 0x7ff90c03cb30 2024-09-16T01:31:09.152 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.355+0000 7ff92a532640 1 -- 172.21.15.139:0/352454751 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff924106ec0 con 0x7ff90c03cb30 2024-09-16T01:31:09.152 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.355+0000 7ff9217fa640 1 -- 172.21.15.139:0/352454751 <== mgr.14150 v2:172.21.15.139:6800/331514141 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ff924106ec0 con 0x7ff90c03cb30 2024-09-16T01:31:09.152 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.356+0000 7ff92a532640 1 -- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff90c03cb30 msgr2=0x7ff90c03ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:09.152 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.356+0000 7ff92a532640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff90c03cb30 0x7ff90c03ef20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff918003980 tx=0x7ff9180092b0 comp rx=0 tx=0).stop 2024-09-16T01:31:09.152 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.356+0000 7ff92a532640 1 -- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff924107ab0 msgr2=0x7ff924106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:09.152 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.356+0000 7ff92a532640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff924107ab0 0x7ff924106220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff9140097f0 tx=0x7ff9140327b0 comp rx=0 tx=0).stop 2024-09-16T01:31:09.153 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.356+0000 7ff92a532640 1 -- 172.21.15.139:0/352454751 shutdown_connections 2024-09-16T01:31:09.153 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.356+0000 7ff92a532640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff90c03cb30 0x7ff90c03ef20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:09.153 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.356+0000 7ff92a532640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:6800/1485008061,v1:172.21.15.139:6801/1485008061] conn(0x7ff90c039260 0x7ff90c03b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:09.153 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.356+0000 7ff92a532640 1 --2- 172.21.15.139:0/352454751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff924107ab0 0x7ff924106220 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:09.153 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.356+0000 7ff92a532640 1 -- 172.21.15.139:0/352454751 >> 172.21.15.139:0/352454751 conn(0x7ff924101db0 msgr2=0x7ff9241029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:09.153 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.356+0000 7ff92a532640 1 -- 172.21.15.139:0/352454751 shutdown_connections 2024-09-16T01:31:09.153 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:08.356+0000 7ff92a532640 1 -- 172.21.15.139:0/352454751 wait complete. 2024-09-16T01:31:09.153 INFO:teuthology.orchestra.run.smithi139.stdout:mgr epoch 9 is available 2024-09-16T01:31:09.153 INFO:teuthology.orchestra.run.smithi139.stdout:Generating a dashboard self-signed certificate... 2024-09-16T01:31:09.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:09 smithi139 ceph-mon[31175]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T01:31:09.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:09 smithi139 ceph-mon[31175]: mgrmap e11: x(active, since 1.00602s) 2024-09-16T01:31:09.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:09 smithi139 ceph-mon[31175]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T01:31:09.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:10.477 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-16T01:31:10.478 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.521+0000 7f90eb30c640 1 Processor -- start 2024-09-16T01:31:10.478 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.522+0000 7f90eb30c640 1 -- start start 2024-09-16T01:31:10.478 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.522+0000 7f90eb30c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e4107ba0 0x7f90e4107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:10.478 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.522+0000 7f90eb30c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f90e4108570 con 0x7f90e4107ba0 2024-09-16T01:31:10.478 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.522+0000 7f90e90a8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e4107ba0 0x7f90e4107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:10.478 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.522+0000 7f90e90a8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e4107ba0 0x7f90e4107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60558/0 (socket says 172.21.15.139:60558) 2024-09-16T01:31:10.478 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.522+0000 7f90e90a8640 1 -- 172.21.15.139:0/1465555507 learned_addr learned my addr 172.21.15.139:0/1465555507 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:10.478 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.522+0000 7f90e90a8640 1 -- 172.21.15.139:0/1465555507 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e4106800 con 0x7f90e4107ba0 2024-09-16T01:31:10.478 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.523+0000 7f90e90a8640 1 --2- 172.21.15.139:0/1465555507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e4107ba0 0x7f90e4107fa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009820 tx=0x7f90e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16711d08de8e5846 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:10.478 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.523+0000 7f90dbfff640 1 -- 172.21.15.139:0/1465555507 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90e0037070 con 0x7f90e4107ba0 2024-09-16T01:31:10.478 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.523+0000 7f90dbfff640 1 -- 172.21.15.139:0/1465555507 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e0032920 con 0x7f90e4107ba0 2024-09-16T01:31:10.478 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.523+0000 7f90dbfff640 1 -- 172.21.15.139:0/1465555507 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90e0030810 con 0x7f90e4107ba0 2024-09-16T01:31:10.479 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.524+0000 7f90eb30c640 1 -- 172.21.15.139:0/1465555507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e4107ba0 msgr2=0x7f90e4107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:10.479 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.524+0000 7f90eb30c640 1 --2- 172.21.15.139:0/1465555507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e4107ba0 0x7f90e4107fa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009820 tx=0x7f90e002a270 comp rx=0 tx=0).stop 2024-09-16T01:31:10.479 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.524+0000 7f90eb30c640 1 -- 172.21.15.139:0/1465555507 shutdown_connections 2024-09-16T01:31:10.479 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.524+0000 7f90eb30c640 1 --2- 172.21.15.139:0/1465555507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e4107ba0 0x7f90e4107fa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:10.479 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.524+0000 7f90eb30c640 1 -- 172.21.15.139:0/1465555507 >> 172.21.15.139:0/1465555507 conn(0x7f90e41014e0 msgr2=0x7f90e4103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:10.479 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.524+0000 7f90eb30c640 1 -- 172.21.15.139:0/1465555507 shutdown_connections 2024-09-16T01:31:10.479 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.524+0000 7f90eb30c640 1 -- 172.21.15.139:0/1465555507 wait complete. 2024-09-16T01:31:10.479 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.525+0000 7f90eb30c640 1 Processor -- start 2024-09-16T01:31:10.479 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.525+0000 7f90eb30c640 1 -- start start 2024-09-16T01:31:10.479 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.525+0000 7f90eb30c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e4107ba0 0x7f90e419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:10.479 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.525+0000 7f90eb30c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f90e410abc0 con 0x7f90e4107ba0 2024-09-16T01:31:10.479 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.525+0000 7f90e90a8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e4107ba0 0x7f90e419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:10.479 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.525+0000 7f90e90a8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e4107ba0 0x7f90e419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60568/0 (socket says 172.21.15.139:60568) 2024-09-16T01:31:10.479 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.525+0000 7f90e90a8640 1 -- 172.21.15.139:0/2836390009 learned_addr learned my addr 172.21.15.139:0/2836390009 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:10.480 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.525+0000 7f90e90a8640 1 -- 172.21.15.139:0/2836390009 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e419ab80 con 0x7f90e4107ba0 2024-09-16T01:31:10.480 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.525+0000 7f90e90a8640 1 --2- 172.21.15.139:0/2836390009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e4107ba0 0x7f90e419a640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f90e0030560 tx=0x7f90e0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:10.480 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.526+0000 7f90da7fc640 1 -- 172.21.15.139:0/2836390009 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90e0037040 con 0x7f90e4107ba0 2024-09-16T01:31:10.480 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.526+0000 7f90da7fc640 1 -- 172.21.15.139:0/2836390009 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e0040070 con 0x7f90e4107ba0 2024-09-16T01:31:10.480 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.526+0000 7f90eb30c640 1 -- 172.21.15.139:0/2836390009 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e419ae10 con 0x7f90e4107ba0 2024-09-16T01:31:10.480 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.526+0000 7f90da7fc640 1 -- 172.21.15.139:0/2836390009 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90e003a6d0 con 0x7f90e4107ba0 2024-09-16T01:31:10.480 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.526+0000 7f90eb30c640 1 -- 172.21.15.139:0/2836390009 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e419b270 con 0x7f90e4107ba0 2024-09-16T01:31:10.480 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.527+0000 7f90da7fc640 1 -- 172.21.15.139:0/2836390009 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f90e003b420 con 0x7f90e4107ba0 2024-09-16T01:31:10.480 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.527+0000 7f90da7fc640 1 --2- 172.21.15.139:0/2836390009 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f90d003b350 0x7f90d003d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:10.480 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.527+0000 7f90eb30c640 1 -- 172.21.15.139:0/2836390009 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e4107fa0 con 0x7f90e4107ba0 2024-09-16T01:31:10.480 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.527+0000 7f90da7fc640 1 -- 172.21.15.139:0/2836390009 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f90e006c730 con 0x7f90e4107ba0 2024-09-16T01:31:10.480 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.527+0000 7f90e88a7640 1 --2- 172.21.15.139:0/2836390009 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f90d003b350 0x7f90d003d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:10.480 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.527+0000 7f90e88a7640 1 --2- 172.21.15.139:0/2836390009 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f90d003b350 0x7f90d003d810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f90d40098c0 tx=0x7f90d4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:10.481 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.532+0000 7f90da7fc640 1 -- 172.21.15.139:0/2836390009 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90e004a200 con 0x7f90e4107ba0 2024-09-16T01:31:10.481 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.715+0000 7f90eb30c640 1 -- 172.21.15.139:0/2836390009 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f90e41027a0 con 0x7f90d003b350 2024-09-16T01:31:10.481 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.904+0000 7f90da7fc640 1 -- 172.21.15.139:0/2836390009 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f90e41027a0 con 0x7f90d003b350 2024-09-16T01:31:10.481 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.906+0000 7f90eb30c640 1 -- 172.21.15.139:0/2836390009 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f90d003b350 msgr2=0x7f90d003d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:10.481 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.907+0000 7f90eb30c640 1 --2- 172.21.15.139:0/2836390009 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f90d003b350 0x7f90d003d810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f90d40098c0 tx=0x7f90d4006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:10.481 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.907+0000 7f90eb30c640 1 -- 172.21.15.139:0/2836390009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e4107ba0 msgr2=0x7f90e419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:10.481 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.907+0000 7f90eb30c640 1 --2- 172.21.15.139:0/2836390009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e4107ba0 0x7f90e419a640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f90e0030560 tx=0x7f90e0032670 comp rx=0 tx=0).stop 2024-09-16T01:31:10.481 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.907+0000 7f90eb30c640 1 -- 172.21.15.139:0/2836390009 shutdown_connections 2024-09-16T01:31:10.481 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.907+0000 7f90eb30c640 1 --2- 172.21.15.139:0/2836390009 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f90d003b350 0x7f90d003d810 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:10.481 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.907+0000 7f90eb30c640 1 --2- 172.21.15.139:0/2836390009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e4107ba0 0x7f90e419a640 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:10.481 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.907+0000 7f90eb30c640 1 -- 172.21.15.139:0/2836390009 >> 172.21.15.139:0/2836390009 conn(0x7f90e41014e0 msgr2=0x7f90e4102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:10.481 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.907+0000 7f90eb30c640 1 -- 172.21.15.139:0/2836390009 shutdown_connections 2024-09-16T01:31:10.481 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:09.907+0000 7f90eb30c640 1 -- 172.21.15.139:0/2836390009 wait complete. 2024-09-16T01:31:10.481 INFO:teuthology.orchestra.run.smithi139.stdout:Creating initial admin user... 2024-09-16T01:31:10.588 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:10 smithi139 ceph-mon[31175]: [16/Sep/2024:01:31:08] ENGINE Bus STARTING 2024-09-16T01:31:10.588 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:10 smithi139 ceph-mon[31175]: [16/Sep/2024:01:31:08] ENGINE Serving on https://172.21.15.139:7150 2024-09-16T01:31:10.588 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:10 smithi139 ceph-mon[31175]: [16/Sep/2024:01:31:08] ENGINE Error in HTTPServer.serve 2024-09-16T01:31:10.588 INFO:journalctl@ceph.mon.a.smithi139.stdout: Traceback (most recent call last): 2024-09-16T01:31:10.588 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:31:10.588 INFO:journalctl@ceph.mon.a.smithi139.stdout: self._connections.run(self.expiration_interval) 2024-09-16T01:31:10.588 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:31:10.588 INFO:journalctl@ceph.mon.a.smithi139.stdout: self._run(expiration_interval) 2024-09-16T01:31:10.588 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:31:10.588 INFO:journalctl@ceph.mon.a.smithi139.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:31:10.588 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:31:10.588 INFO:journalctl@ceph.mon.a.smithi139.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:31:10.589 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:31:10.589 INFO:journalctl@ceph.mon.a.smithi139.stdout: s = self.context.wrap_socket( 2024-09-16T01:31:10.589 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:31:10.589 INFO:journalctl@ceph.mon.a.smithi139.stdout: return self.sslsocket_class._create( 2024-09-16T01:31:10.589 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:31:10.589 INFO:journalctl@ceph.mon.a.smithi139.stdout: self.do_handshake() 2024-09-16T01:31:10.589 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:31:10.589 INFO:journalctl@ceph.mon.a.smithi139.stdout: self._sslobj.do_handshake() 2024-09-16T01:31:10.589 INFO:journalctl@ceph.mon.a.smithi139.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:31:10.589 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:10 smithi139 ceph-mon[31175]: [16/Sep/2024:01:31:08] ENGINE Serving on http://172.21.15.139:8765 2024-09-16T01:31:10.589 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:10 smithi139 ceph-mon[31175]: [16/Sep/2024:01:31:08] ENGINE Bus STARTED 2024-09-16T01:31:10.589 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:10 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:10.589 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:10 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:10.590 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:10 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:10.590 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:10 smithi139 ceph-mon[31175]: mgrmap e12: x(active, since 2s) 2024-09-16T01:31:11.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:11 smithi139 ceph-mon[31175]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:31:11.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:11 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:12.507 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:12 smithi139 ceph-mon[31175]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:31:12.515 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$xpxzzUTeNIGzItUA7sUhl.EC4aM0goASig4EKil4N0YDyi863Ersy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726450271, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T01:31:12.515 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.871+0000 7fdeef5c8640 1 Processor -- start 2024-09-16T01:31:12.515 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.871+0000 7fdeef5c8640 1 -- start start 2024-09-16T01:31:12.515 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.871+0000 7fdeef5c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee8108180 0x7fdee810a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:12.515 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.871+0000 7fdeef5c8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdee807beb0 con 0x7fdee8108180 2024-09-16T01:31:12.515 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.871+0000 7fdeed364640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee8108180 0x7fdee810a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:12.515 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.871+0000 7fdeed364640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee8108180 0x7fdee810a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60582/0 (socket says 172.21.15.139:60582) 2024-09-16T01:31:12.515 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.871+0000 7fdeed364640 1 -- 172.21.15.139:0/748160629 learned_addr learned my addr 172.21.15.139:0/748160629 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:12.515 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.872+0000 7fdeed364640 1 -- 172.21.15.139:0/748160629 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee810aab0 con 0x7fdee8108180 2024-09-16T01:31:12.515 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.872+0000 7fdeed364640 1 --2- 172.21.15.139:0/748160629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee8108180 0x7fdee810a570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009820 tx=0x7fdee402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b58e41d646e6d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:12.515 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.873+0000 7fdedffff640 1 -- 172.21.15.139:0/748160629 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdee4037070 con 0x7fdee8108180 2024-09-16T01:31:12.516 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.873+0000 7fdedffff640 1 -- 172.21.15.139:0/748160629 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee4032920 con 0x7fdee8108180 2024-09-16T01:31:12.516 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.873+0000 7fdedffff640 1 -- 172.21.15.139:0/748160629 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdee4030810 con 0x7fdee8108180 2024-09-16T01:31:12.516 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.873+0000 7fdeef5c8640 1 -- 172.21.15.139:0/748160629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee8108180 msgr2=0x7fdee810a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:12.516 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.873+0000 7fdeef5c8640 1 --2- 172.21.15.139:0/748160629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee8108180 0x7fdee810a570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009820 tx=0x7fdee402a270 comp rx=0 tx=0).stop 2024-09-16T01:31:12.516 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.873+0000 7fdeef5c8640 1 -- 172.21.15.139:0/748160629 shutdown_connections 2024-09-16T01:31:12.516 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.873+0000 7fdeef5c8640 1 --2- 172.21.15.139:0/748160629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee8108180 0x7fdee810a570 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:12.516 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.873+0000 7fdeef5c8640 1 -- 172.21.15.139:0/748160629 >> 172.21.15.139:0/748160629 conn(0x7fdee8101da0 msgr2=0x7fdee81041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:12.516 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.873+0000 7fdeef5c8640 1 -- 172.21.15.139:0/748160629 shutdown_connections 2024-09-16T01:31:12.516 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.873+0000 7fdeef5c8640 1 -- 172.21.15.139:0/748160629 wait complete. 2024-09-16T01:31:12.516 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.874+0000 7fdeef5c8640 1 Processor -- start 2024-09-16T01:31:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.874+0000 7fdeef5c8640 1 -- start start 2024-09-16T01:31:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.874+0000 7fdeef5c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee8108180 0x7fdee819b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.874+0000 7fdeef5c8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdee810bb90 con 0x7fdee8108180 2024-09-16T01:31:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.874+0000 7fdeed364640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee8108180 0x7fdee819b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.874+0000 7fdeed364640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee8108180 0x7fdee819b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60586/0 (socket says 172.21.15.139:60586) 2024-09-16T01:31:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.874+0000 7fdeed364640 1 -- 172.21.15.139:0/705954812 learned_addr learned my addr 172.21.15.139:0/705954812 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.875+0000 7fdeed364640 1 -- 172.21.15.139:0/705954812 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee819bc70 con 0x7fdee8108180 2024-09-16T01:31:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.875+0000 7fdeed364640 1 --2- 172.21.15.139:0/705954812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee8108180 0x7fdee819b730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdee4030420 tx=0x7fdee4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.875+0000 7fdede7fc640 1 -- 172.21.15.139:0/705954812 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdee4037040 con 0x7fdee8108180 2024-09-16T01:31:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.875+0000 7fdeef5c8640 1 -- 172.21.15.139:0/705954812 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee819bf00 con 0x7fdee8108180 2024-09-16T01:31:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.875+0000 7fdede7fc640 1 -- 172.21.15.139:0/705954812 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee4040070 con 0x7fdee8108180 2024-09-16T01:31:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.875+0000 7fdede7fc640 1 -- 172.21.15.139:0/705954812 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdee403a6e0 con 0x7fdee8108180 2024-09-16T01:31:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.875+0000 7fdeef5c8640 1 -- 172.21.15.139:0/705954812 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee819ebf0 con 0x7fdee8108180 2024-09-16T01:31:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.876+0000 7fdede7fc640 1 -- 172.21.15.139:0/705954812 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fdee403a9e0 con 0x7fdee8108180 2024-09-16T01:31:12.518 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.876+0000 7fdede7fc640 1 --2- 172.21.15.139:0/705954812 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fded4039200 0x7fded403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:12.518 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.877+0000 7fdede7fc640 1 -- 172.21.15.139:0/705954812 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdee406ca40 con 0x7fdee8108180 2024-09-16T01:31:12.518 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.876+0000 7fdeef5c8640 1 -- 172.21.15.139:0/705954812 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee81085e0 con 0x7fdee8108180 2024-09-16T01:31:12.518 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.877+0000 7fdeecb63640 1 --2- 172.21.15.139:0/705954812 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fded4039200 0x7fded403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:12.518 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.877+0000 7fdeecb63640 1 --2- 172.21.15.139:0/705954812 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fded4039200 0x7fded403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fded80098c0 tx=0x7fded8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:12.518 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:10.882+0000 7fdede7fc640 1 -- 172.21.15.139:0/705954812 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdee4042e10 con 0x7fdee8108180 2024-09-16T01:31:12.518 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:11.072+0000 7fdeef5c8640 1 -- 172.21.15.139:0/705954812 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fdee8103010 con 0x7fded4039200 2024-09-16T01:31:12.518 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:11.298+0000 7fdede7fc640 1 -- 172.21.15.139:0/705954812 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fdee8103010 con 0x7fded4039200 2024-09-16T01:31:12.518 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:11.300+0000 7fdeef5c8640 1 -- 172.21.15.139:0/705954812 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fded4039200 msgr2=0x7fded403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:12.519 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:11.300+0000 7fdeef5c8640 1 --2- 172.21.15.139:0/705954812 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fded4039200 0x7fded403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fded80098c0 tx=0x7fded8006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:12.519 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:11.300+0000 7fdeef5c8640 1 -- 172.21.15.139:0/705954812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee8108180 msgr2=0x7fdee819b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:12.519 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:11.300+0000 7fdeef5c8640 1 --2- 172.21.15.139:0/705954812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee8108180 0x7fdee819b730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdee4030420 tx=0x7fdee4032670 comp rx=0 tx=0).stop 2024-09-16T01:31:12.519 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:11.300+0000 7fdeef5c8640 1 -- 172.21.15.139:0/705954812 shutdown_connections 2024-09-16T01:31:12.519 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:11.300+0000 7fdeef5c8640 1 --2- 172.21.15.139:0/705954812 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fded4039200 0x7fded403b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:12.519 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:11.300+0000 7fdeef5c8640 1 --2- 172.21.15.139:0/705954812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee8108180 0x7fdee819b730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:12.519 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:11.300+0000 7fdeef5c8640 1 -- 172.21.15.139:0/705954812 >> 172.21.15.139:0/705954812 conn(0x7fdee8101da0 msgr2=0x7fdee8102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:12.519 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:11.301+0000 7fdeef5c8640 1 -- 172.21.15.139:0/705954812 shutdown_connections 2024-09-16T01:31:12.519 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:11.301+0000 7fdeef5c8640 1 -- 172.21.15.139:0/705954812 wait complete. 2024-09-16T01:31:12.519 INFO:teuthology.orchestra.run.smithi139.stdout:Fetching dashboard port number... 2024-09-16T01:31:13.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:13 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3477619811' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-16T01:31:13.788 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T01:31:13.788 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.988+0000 7f0d7f658640 1 Processor -- start 2024-09-16T01:31:13.788 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.988+0000 7f0d7f658640 1 -- start start 2024-09-16T01:31:13.788 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.988+0000 7f0d7f658640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d781065f0 0x7f0d781069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:13.788 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.988+0000 7f0d7f658640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78106fc0 con 0x7f0d781065f0 2024-09-16T01:31:13.788 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.988+0000 7f0d7d3f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d781065f0 0x7f0d781069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:13.788 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.988+0000 7f0d7d3f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d781065f0 0x7f0d781069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34686/0 (socket says 172.21.15.139:34686) 2024-09-16T01:31:13.789 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.988+0000 7f0d7d3f4640 1 -- 172.21.15.139:0/3527804115 learned_addr learned my addr 172.21.15.139:0/3527804115 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:13.789 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.988+0000 7f0d7d3f4640 1 -- 172.21.15.139:0/3527804115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d781077f0 con 0x7f0d781065f0 2024-09-16T01:31:13.789 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.989+0000 7f0d7d3f4640 1 --2- 172.21.15.139:0/3527804115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d781065f0 0x7f0d781069f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009820 tx=0x7f0d7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57da1717fedd55a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:13.789 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.989+0000 7f0d6ffff640 1 -- 172.21.15.139:0/3527804115 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d74037070 con 0x7f0d781065f0 2024-09-16T01:31:13.789 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.989+0000 7f0d6ffff640 1 -- 172.21.15.139:0/3527804115 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d74032920 con 0x7f0d781065f0 2024-09-16T01:31:13.789 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.989+0000 7f0d6ffff640 1 -- 172.21.15.139:0/3527804115 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d74030810 con 0x7f0d781065f0 2024-09-16T01:31:13.789 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.990+0000 7f0d7f658640 1 -- 172.21.15.139:0/3527804115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d781065f0 msgr2=0x7f0d781069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:13.789 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.990+0000 7f0d7f658640 1 --2- 172.21.15.139:0/3527804115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d781065f0 0x7f0d781069f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009820 tx=0x7f0d7402a270 comp rx=0 tx=0).stop 2024-09-16T01:31:13.789 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.990+0000 7f0d7f658640 1 -- 172.21.15.139:0/3527804115 shutdown_connections 2024-09-16T01:31:13.789 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.990+0000 7f0d7f658640 1 --2- 172.21.15.139:0/3527804115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d781065f0 0x7f0d781069f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:13.789 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.990+0000 7f0d7f658640 1 -- 172.21.15.139:0/3527804115 >> 172.21.15.139:0/3527804115 conn(0x7f0d78101e20 msgr2=0x7f0d78104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:13.789 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.990+0000 7f0d7f658640 1 -- 172.21.15.139:0/3527804115 shutdown_connections 2024-09-16T01:31:13.789 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.990+0000 7f0d7f658640 1 -- 172.21.15.139:0/3527804115 wait complete. 2024-09-16T01:31:13.790 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.991+0000 7f0d7f658640 1 Processor -- start 2024-09-16T01:31:13.790 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.991+0000 7f0d7f658640 1 -- start start 2024-09-16T01:31:13.790 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.991+0000 7f0d7f658640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d781065f0 0x7f0d7819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:13.790 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.991+0000 7f0d7f658640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7810aba0 con 0x7f0d781065f0 2024-09-16T01:31:13.790 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.991+0000 7f0d7d3f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d781065f0 0x7f0d7819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:13.790 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.991+0000 7f0d7d3f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d781065f0 0x7f0d7819b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34694/0 (socket says 172.21.15.139:34694) 2024-09-16T01:31:13.790 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.991+0000 7f0d7d3f4640 1 -- 172.21.15.139:0/3477619811 learned_addr learned my addr 172.21.15.139:0/3477619811 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:13.790 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.991+0000 7f0d7d3f4640 1 -- 172.21.15.139:0/3477619811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7819bcd0 con 0x7f0d781065f0 2024-09-16T01:31:13.790 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.991+0000 7f0d7d3f4640 1 --2- 172.21.15.139:0/3477619811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d781065f0 0x7f0d7819b790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d74030560 tx=0x7f0d74032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:13.790 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.991+0000 7f0d6e7fc640 1 -- 172.21.15.139:0/3477619811 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d74037040 con 0x7f0d781065f0 2024-09-16T01:31:13.790 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.991+0000 7f0d6e7fc640 1 -- 172.21.15.139:0/3477619811 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d74040070 con 0x7f0d781065f0 2024-09-16T01:31:13.790 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.991+0000 7f0d7f658640 1 -- 172.21.15.139:0/3477619811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7819bf60 con 0x7f0d781065f0 2024-09-16T01:31:13.790 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.992+0000 7f0d6e7fc640 1 -- 172.21.15.139:0/3477619811 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d7403a6d0 con 0x7f0d781065f0 2024-09-16T01:31:13.790 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.992+0000 7f0d7f658640 1 -- 172.21.15.139:0/3477619811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7819ec50 con 0x7f0d781065f0 2024-09-16T01:31:13.791 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.992+0000 7f0d6e7fc640 1 -- 172.21.15.139:0/3477619811 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f0d7403a870 con 0x7f0d781065f0 2024-09-16T01:31:13.791 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.993+0000 7f0d7f658640 1 -- 172.21.15.139:0/3477619811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7810ad20 con 0x7f0d781065f0 2024-09-16T01:31:13.791 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.993+0000 7f0d6e7fc640 1 --2- 172.21.15.139:0/3477619811 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0d64039260 0x7f0d6403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:13.791 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.993+0000 7f0d6e7fc640 1 -- 172.21.15.139:0/3477619811 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0d7406c970 con 0x7f0d781065f0 2024-09-16T01:31:13.791 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.993+0000 7f0d7cbf3640 1 --2- 172.21.15.139:0/3477619811 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0d64039260 0x7f0d6403b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:13.791 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.993+0000 7f0d7cbf3640 1 --2- 172.21.15.139:0/3477619811 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0d64039260 0x7f0d6403b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0d680098c0 tx=0x7f0d68006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:13.791 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:12.997+0000 7f0d6e7fc640 1 -- 172.21.15.139:0/3477619811 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d7404a200 con 0x7f0d781065f0 2024-09-16T01:31:13.791 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:13.161+0000 7f0d7f658640 1 -- 172.21.15.139:0/3477619811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f0d7802d9b0 con 0x7f0d781065f0 2024-09-16T01:31:13.791 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:13.161+0000 7f0d6e7fc640 1 -- 172.21.15.139:0/3477619811 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f0d74031e80 con 0x7f0d781065f0 2024-09-16T01:31:13.791 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:13.163+0000 7f0d7f658640 1 -- 172.21.15.139:0/3477619811 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0d64039260 msgr2=0x7f0d6403b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:13.791 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:13.163+0000 7f0d7f658640 1 --2- 172.21.15.139:0/3477619811 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0d64039260 0x7f0d6403b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0d680098c0 tx=0x7f0d68006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:13.791 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:13.163+0000 7f0d7f658640 1 -- 172.21.15.139:0/3477619811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d781065f0 msgr2=0x7f0d7819b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:13.791 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:13.163+0000 7f0d7f658640 1 --2- 172.21.15.139:0/3477619811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d781065f0 0x7f0d7819b790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d74030560 tx=0x7f0d74032670 comp rx=0 tx=0).stop 2024-09-16T01:31:13.792 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:13.163+0000 7f0d7f658640 1 -- 172.21.15.139:0/3477619811 shutdown_connections 2024-09-16T01:31:13.792 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:13.163+0000 7f0d7f658640 1 --2- 172.21.15.139:0/3477619811 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0d64039260 0x7f0d6403b720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:13.792 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:13.163+0000 7f0d7f658640 1 --2- 172.21.15.139:0/3477619811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d781065f0 0x7f0d7819b790 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:13.792 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:13.163+0000 7f0d7f658640 1 -- 172.21.15.139:0/3477619811 >> 172.21.15.139:0/3477619811 conn(0x7f0d78101e20 msgr2=0x7f0d78102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:13.792 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:13.163+0000 7f0d7f658640 1 -- 172.21.15.139:0/3477619811 shutdown_connections 2024-09-16T01:31:13.792 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:13.163+0000 7f0d7f658640 1 -- 172.21.15.139:0/3477619811 wait complete. 2024-09-16T01:31:13.796 INFO:teuthology.orchestra.run.smithi139.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T01:31:13.796 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stdout disabled 2024-09-16T01:31:13.803 INFO:teuthology.orchestra.run.smithi139.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T01:31:13.803 INFO:teuthology.orchestra.run.smithi139.stdout:systemctl: stdout inactive 2024-09-16T01:31:13.803 INFO:teuthology.orchestra.run.smithi139.stdout:firewalld.service is not enabled 2024-09-16T01:31:13.803 INFO:teuthology.orchestra.run.smithi139.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T01:31:13.806 INFO:teuthology.orchestra.run.smithi139.stdout:Ceph Dashboard is now available at: 2024-09-16T01:31:13.806 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:31:13.806 INFO:teuthology.orchestra.run.smithi139.stdout: URL: https://smithi139.front.sepia.ceph.com:8443/ 2024-09-16T01:31:13.806 INFO:teuthology.orchestra.run.smithi139.stdout: User: admin 2024-09-16T01:31:13.806 INFO:teuthology.orchestra.run.smithi139.stdout: Password: 3umivwwsok 2024-09-16T01:31:13.806 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:31:13.806 INFO:teuthology.orchestra.run.smithi139.stdout:Saving cluster configuration to /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config directory 2024-09-16T01:31:14.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:14 smithi139 ceph-mon[31175]: mgrmap e13: x(active, since 6s) 2024-09-16T01:31:15.061 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.212+0000 7f8fcff8e640 1 Processor -- start 2024-09-16T01:31:15.061 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.212+0000 7f8fcff8e640 1 -- start start 2024-09-16T01:31:15.061 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.212+0000 7f8fcff8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fc81076f0 0x7f8fc8107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:15.061 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.212+0000 7f8fcff8e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc81080c0 con 0x7f8fc81076f0 2024-09-16T01:31:15.061 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.212+0000 7f8fcdd2a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fc81076f0 0x7f8fc8107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:15.061 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.212+0000 7f8fcdd2a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fc81076f0 0x7f8fc8107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34710/0 (socket says 172.21.15.139:34710) 2024-09-16T01:31:15.061 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.212+0000 7f8fcdd2a640 1 -- 172.21.15.139:0/1175459783 learned_addr learned my addr 172.21.15.139:0/1175459783 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:15.061 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.213+0000 7f8fcdd2a640 1 -- 172.21.15.139:0/1175459783 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc81088f0 con 0x7f8fc81076f0 2024-09-16T01:31:15.062 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.213+0000 7f8fcdd2a640 1 --2- 172.21.15.139:0/1175459783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fc81076f0 0x7f8fc8107af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009820 tx=0x7f8fc402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd8ee2a2295fea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:15.062 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.214+0000 7f8fccd28640 1 -- 172.21.15.139:0/1175459783 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fc4037070 con 0x7f8fc81076f0 2024-09-16T01:31:15.062 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.214+0000 7f8fccd28640 1 -- 172.21.15.139:0/1175459783 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc4032920 con 0x7f8fc81076f0 2024-09-16T01:31:15.062 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.214+0000 7f8fccd28640 1 -- 172.21.15.139:0/1175459783 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fc4030810 con 0x7f8fc81076f0 2024-09-16T01:31:15.062 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.214+0000 7f8fcff8e640 1 -- 172.21.15.139:0/1175459783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fc81076f0 msgr2=0x7f8fc8107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:15.062 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.214+0000 7f8fcff8e640 1 --2- 172.21.15.139:0/1175459783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fc81076f0 0x7f8fc8107af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009820 tx=0x7f8fc402a270 comp rx=0 tx=0).stop 2024-09-16T01:31:15.062 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.214+0000 7f8fcff8e640 1 -- 172.21.15.139:0/1175459783 shutdown_connections 2024-09-16T01:31:15.062 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.214+0000 7f8fcff8e640 1 --2- 172.21.15.139:0/1175459783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fc81076f0 0x7f8fc8107af0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:15.062 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.214+0000 7f8fcff8e640 1 -- 172.21.15.139:0/1175459783 >> 172.21.15.139:0/1175459783 conn(0x7f8fc8102ec0 msgr2=0x7f8fc81052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:15.062 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.214+0000 7f8fcff8e640 1 -- 172.21.15.139:0/1175459783 shutdown_connections 2024-09-16T01:31:15.062 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.214+0000 7f8fcff8e640 1 -- 172.21.15.139:0/1175459783 wait complete. 2024-09-16T01:31:15.062 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.215+0000 7f8fcff8e640 1 Processor -- start 2024-09-16T01:31:15.062 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.215+0000 7f8fcff8e640 1 -- start start 2024-09-16T01:31:15.062 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.215+0000 7f8fcff8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fc81076f0 0x7f8fc819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:15.063 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.215+0000 7f8fcff8e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc810bca0 con 0x7f8fc81076f0 2024-09-16T01:31:15.063 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.215+0000 7f8fcdd2a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fc81076f0 0x7f8fc819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:15.063 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.215+0000 7f8fcdd2a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fc81076f0 0x7f8fc819b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34716/0 (socket says 172.21.15.139:34716) 2024-09-16T01:31:15.063 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.215+0000 7f8fcdd2a640 1 -- 172.21.15.139:0/3183739715 learned_addr learned my addr 172.21.15.139:0/3183739715 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:15.063 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.215+0000 7f8fcdd2a640 1 -- 172.21.15.139:0/3183739715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc819bc30 con 0x7f8fc81076f0 2024-09-16T01:31:15.063 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.216+0000 7f8fcdd2a640 1 --2- 172.21.15.139:0/3183739715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fc81076f0 0x7f8fc819b6f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40097f0 tx=0x7f8fc40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:15.063 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.216+0000 7f8fbeffd640 1 -- 172.21.15.139:0/3183739715 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fc4037040 con 0x7f8fc81076f0 2024-09-16T01:31:15.063 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.216+0000 7f8fcff8e640 1 -- 172.21.15.139:0/3183739715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc819bec0 con 0x7f8fc81076f0 2024-09-16T01:31:15.063 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.216+0000 7f8fbeffd640 1 -- 172.21.15.139:0/3183739715 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc4030cc0 con 0x7f8fc81076f0 2024-09-16T01:31:15.063 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.216+0000 7f8fbeffd640 1 -- 172.21.15.139:0/3183739715 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fc403a640 con 0x7f8fc81076f0 2024-09-16T01:31:15.063 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.216+0000 7f8fcff8e640 1 -- 172.21.15.139:0/3183739715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc819ebb0 con 0x7f8fc81076f0 2024-09-16T01:31:15.063 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.217+0000 7f8fbeffd640 1 -- 172.21.15.139:0/3183739715 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f8fc403a940 con 0x7f8fc81076f0 2024-09-16T01:31:15.063 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.217+0000 7f8fcff8e640 1 -- 172.21.15.139:0/3183739715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc810be20 con 0x7f8fc81076f0 2024-09-16T01:31:15.063 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.217+0000 7f8fbeffd640 1 --2- 172.21.15.139:0/3183739715 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8fb40392a0 0x7f8fb403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:15.064 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.217+0000 7f8fbeffd640 1 -- 172.21.15.139:0/3183739715 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8fc406cb60 con 0x7f8fc81076f0 2024-09-16T01:31:15.064 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.217+0000 7f8fcd529640 1 --2- 172.21.15.139:0/3183739715 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8fb40392a0 0x7f8fb403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:15.064 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.217+0000 7f8fcd529640 1 --2- 172.21.15.139:0/3183739715 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8fb40392a0 0x7f8fb403b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80098c0 tx=0x7f8fb8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:15.064 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.221+0000 7f8fbeffd640 1 -- 172.21.15.139:0/3183739715 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8fc4039740 con 0x7f8fc81076f0 2024-09-16T01:31:15.064 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.473+0000 7f8fcff8e640 1 -- 172.21.15.139:0/3183739715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f8fc819ed80 con 0x7f8fc81076f0 2024-09-16T01:31:15.064 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.475+0000 7f8fbeffd640 1 -- 172.21.15.139:0/3183739715 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7f8fc403ba40 con 0x7f8fc81076f0 2024-09-16T01:31:15.064 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T01:31:15.064 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.477+0000 7f8fcff8e640 1 -- 172.21.15.139:0/3183739715 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8fb40392a0 msgr2=0x7f8fb403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:15.064 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.477+0000 7f8fcff8e640 1 --2- 172.21.15.139:0/3183739715 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8fb40392a0 0x7f8fb403b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80098c0 tx=0x7f8fb8006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:15.064 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.477+0000 7f8fcff8e640 1 -- 172.21.15.139:0/3183739715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fc81076f0 msgr2=0x7f8fc819b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:15.064 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.477+0000 7f8fcff8e640 1 --2- 172.21.15.139:0/3183739715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fc81076f0 0x7f8fc819b6f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40097f0 tx=0x7f8fc40327b0 comp rx=0 tx=0).stop 2024-09-16T01:31:15.064 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.478+0000 7f8fcff8e640 1 -- 172.21.15.139:0/3183739715 shutdown_connections 2024-09-16T01:31:15.064 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.478+0000 7f8fcff8e640 1 --2- 172.21.15.139:0/3183739715 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8fb40392a0 0x7f8fb403b760 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:15.065 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.478+0000 7f8fcff8e640 1 --2- 172.21.15.139:0/3183739715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fc81076f0 0x7f8fc819b6f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:15.065 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.478+0000 7f8fcff8e640 1 -- 172.21.15.139:0/3183739715 >> 172.21.15.139:0/3183739715 conn(0x7f8fc8102ec0 msgr2=0x7f8fc81039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:15.065 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.478+0000 7f8fcff8e640 1 -- 172.21.15.139:0/3183739715 shutdown_connections 2024-09-16T01:31:15.065 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/bin/ceph: stderr 2024-09-16T01:31:14.478+0000 7f8fcff8e640 1 -- 172.21.15.139:0/3183739715 wait complete. 2024-09-16T01:31:15.065 INFO:teuthology.orchestra.run.smithi139.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T01:31:15.065 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:31:15.065 INFO:teuthology.orchestra.run.smithi139.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T01:31:15.065 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:31:15.065 INFO:teuthology.orchestra.run.smithi139.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T01:31:15.065 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:31:15.066 INFO:teuthology.orchestra.run.smithi139.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T01:31:15.066 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:31:15.066 INFO:teuthology.orchestra.run.smithi139.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T01:31:15.066 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:31:15.066 INFO:teuthology.orchestra.run.smithi139.stdout: ceph telemetry on 2024-09-16T01:31:15.066 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:31:15.066 INFO:teuthology.orchestra.run.smithi139.stdout:For more information see: 2024-09-16T01:31:15.066 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:31:15.066 INFO:teuthology.orchestra.run.smithi139.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T01:31:15.066 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:31:15.066 INFO:teuthology.orchestra.run.smithi139.stdout:Bootstrap complete. 2024-09-16T01:31:15.099 INFO:tasks.cephadm:Fetching config... 2024-09-16T01:31:15.099 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:31:15.100 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T01:31:15.117 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T01:31:15.117 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:31:15.117 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T01:31:15.172 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T01:31:15.172 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:31:15.173 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-16T01:31:15.237 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T01:31:15.237 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:31:15.237 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T01:31:15.294 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T01:31:15.294 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmmVDWTdIdKh1gGfDeECBzCSn6c+0f3hMa5U0wtD+Jo4hrtk+pjW7K31f4+N1jVzpo/U4/0DVQSeehvkxwAn3j4P81HbJUVCWb9AhLmEr7NWFf1M0I1UEd62D+nl6rmVA24mxlPEd6TAgXsBt8WphUIA98en4V2NVd9zEadqLh0HA2JX/YK+LXjOo4YpYHywqC2NBM8ceeIJr9LCNB9RmS1SkPecrbG0jqQTx9avQMB07kGtIVFNUp53m1648Y35wgl9yRFQBHQClchRLs3c9mBITbM7WYxgSlExu5OEcwXAKPCqr9CO96Dy8uqhqTjnNBj8w4zQohQy3K7wSOEXbC71+qBg0s4DIMqw3qxm+8iNhUmaKRRKetkJpXMgeEeM6AOY9BEIl45GRcQCz2ooMz1Vvg9wqp4+kQv0W1lzf4oh/4WzeP+0DO889uqfKAeAZe3UE7Y7l2VYTjBvZAErDg56TqiyDZEIeuhnOFx4FsyitmMg8reG/lF0vlnm6ie3s= ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T01:31:15.368 INFO:teuthology.orchestra.run.smithi139.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmmVDWTdIdKh1gGfDeECBzCSn6c+0f3hMa5U0wtD+Jo4hrtk+pjW7K31f4+N1jVzpo/U4/0DVQSeehvkxwAn3j4P81HbJUVCWb9AhLmEr7NWFf1M0I1UEd62D+nl6rmVA24mxlPEd6TAgXsBt8WphUIA98en4V2NVd9zEadqLh0HA2JX/YK+LXjOo4YpYHywqC2NBM8ceeIJr9LCNB9RmS1SkPecrbG0jqQTx9avQMB07kGtIVFNUp53m1648Y35wgl9yRFQBHQClchRLs3c9mBITbM7WYxgSlExu5OEcwXAKPCqr9CO96Dy8uqhqTjnNBj8w4zQohQy3K7wSOEXbC71+qBg0s4DIMqw3qxm+8iNhUmaKRRKetkJpXMgeEeM6AOY9BEIl45GRcQCz2ooMz1Vvg9wqp4+kQv0W1lzf4oh/4WzeP+0DO889uqfKAeAZe3UE7Y7l2VYTjBvZAErDg56TqiyDZEIeuhnOFx4FsyitmMg8reG/lF0vlnm6ie3s= ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:15.385 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmmVDWTdIdKh1gGfDeECBzCSn6c+0f3hMa5U0wtD+Jo4hrtk+pjW7K31f4+N1jVzpo/U4/0DVQSeehvkxwAn3j4P81HbJUVCWb9AhLmEr7NWFf1M0I1UEd62D+nl6rmVA24mxlPEd6TAgXsBt8WphUIA98en4V2NVd9zEadqLh0HA2JX/YK+LXjOo4YpYHywqC2NBM8ceeIJr9LCNB9RmS1SkPecrbG0jqQTx9avQMB07kGtIVFNUp53m1648Y35wgl9yRFQBHQClchRLs3c9mBITbM7WYxgSlExu5OEcwXAKPCqr9CO96Dy8uqhqTjnNBj8w4zQohQy3K7wSOEXbC71+qBg0s4DIMqw3qxm+8iNhUmaKRRKetkJpXMgeEeM6AOY9BEIl45GRcQCz2ooMz1Vvg9wqp4+kQv0W1lzf4oh/4WzeP+0DO889uqfKAeAZe3UE7Y7l2VYTjBvZAErDg56TqiyDZEIeuhnOFx4FsyitmMg8reG/lF0vlnm6ie3s= ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T01:31:15.427 INFO:teuthology.orchestra.run.smithi164.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmmVDWTdIdKh1gGfDeECBzCSn6c+0f3hMa5U0wtD+Jo4hrtk+pjW7K31f4+N1jVzpo/U4/0DVQSeehvkxwAn3j4P81HbJUVCWb9AhLmEr7NWFf1M0I1UEd62D+nl6rmVA24mxlPEd6TAgXsBt8WphUIA98en4V2NVd9zEadqLh0HA2JX/YK+LXjOo4YpYHywqC2NBM8ceeIJr9LCNB9RmS1SkPecrbG0jqQTx9avQMB07kGtIVFNUp53m1648Y35wgl9yRFQBHQClchRLs3c9mBITbM7WYxgSlExu5OEcwXAKPCqr9CO96Dy8uqhqTjnNBj8w4zQohQy3K7wSOEXbC71+qBg0s4DIMqw3qxm+8iNhUmaKRRKetkJpXMgeEeM6AOY9BEIl45GRcQCz2ooMz1Vvg9wqp4+kQv0W1lzf4oh/4WzeP+0DO889uqfKAeAZe3UE7Y7l2VYTjBvZAErDg56TqiyDZEIeuhnOFx4FsyitmMg8reG/lF0vlnm6ie3s= ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:15.440 DEBUG:teuthology.orchestra.run.smithi176:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmmVDWTdIdKh1gGfDeECBzCSn6c+0f3hMa5U0wtD+Jo4hrtk+pjW7K31f4+N1jVzpo/U4/0DVQSeehvkxwAn3j4P81HbJUVCWb9AhLmEr7NWFf1M0I1UEd62D+nl6rmVA24mxlPEd6TAgXsBt8WphUIA98en4V2NVd9zEadqLh0HA2JX/YK+LXjOo4YpYHywqC2NBM8ceeIJr9LCNB9RmS1SkPecrbG0jqQTx9avQMB07kGtIVFNUp53m1648Y35wgl9yRFQBHQClchRLs3c9mBITbM7WYxgSlExu5OEcwXAKPCqr9CO96Dy8uqhqTjnNBj8w4zQohQy3K7wSOEXbC71+qBg0s4DIMqw3qxm+8iNhUmaKRRKetkJpXMgeEeM6AOY9BEIl45GRcQCz2ooMz1Vvg9wqp4+kQv0W1lzf4oh/4WzeP+0DO889uqfKAeAZe3UE7Y7l2VYTjBvZAErDg56TqiyDZEIeuhnOFx4FsyitmMg8reG/lF0vlnm6ie3s= ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T01:31:15.481 INFO:teuthology.orchestra.run.smithi176.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmmVDWTdIdKh1gGfDeECBzCSn6c+0f3hMa5U0wtD+Jo4hrtk+pjW7K31f4+N1jVzpo/U4/0DVQSeehvkxwAn3j4P81HbJUVCWb9AhLmEr7NWFf1M0I1UEd62D+nl6rmVA24mxlPEd6TAgXsBt8WphUIA98en4V2NVd9zEadqLh0HA2JX/YK+LXjOo4YpYHywqC2NBM8ceeIJr9LCNB9RmS1SkPecrbG0jqQTx9avQMB07kGtIVFNUp53m1648Y35wgl9yRFQBHQClchRLs3c9mBITbM7WYxgSlExu5OEcwXAKPCqr9CO96Dy8uqhqTjnNBj8w4zQohQy3K7wSOEXbC71+qBg0s4DIMqw3qxm+8iNhUmaKRRKetkJpXMgeEeM6AOY9BEIl45GRcQCz2ooMz1Vvg9wqp4+kQv0W1lzf4oh/4WzeP+0DO889uqfKAeAZe3UE7Y7l2VYTjBvZAErDg56TqiyDZEIeuhnOFx4FsyitmMg8reG/lF0vlnm6ie3s= ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:15.493 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T01:31:15.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:15 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3183739715' entity='client.admin' 2024-09-16T01:31:15.744 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:31:16.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.312+0000 7f982de44640 1 -- 172.21.15.139:0/1679159800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98281020c0 msgr2=0x7f98281044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:16.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.312+0000 7f982de44640 1 --2- 172.21.15.139:0/1679159800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98281020c0 0x7f98281044b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9818009c20 tx=0x7f981802aa90 comp rx=0 tx=0).stop 2024-09-16T01:31:16.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.312+0000 7f982de44640 1 -- 172.21.15.139:0/1679159800 shutdown_connections 2024-09-16T01:31:16.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.312+0000 7f982de44640 1 --2- 172.21.15.139:0/1679159800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98281020c0 0x7f98281044b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:16.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.312+0000 7f982de44640 1 -- 172.21.15.139:0/1679159800 >> 172.21.15.139:0/1679159800 conn(0x7f98280fbca0 msgr2=0x7f98280fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:16.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.312+0000 7f982de44640 1 -- 172.21.15.139:0/1679159800 shutdown_connections 2024-09-16T01:31:16.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.312+0000 7f982de44640 1 -- 172.21.15.139:0/1679159800 wait complete. 2024-09-16T01:31:16.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.313+0000 7f982de44640 1 Processor -- start 2024-09-16T01:31:16.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.313+0000 7f982de44640 1 -- start start 2024-09-16T01:31:16.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.314+0000 7f982de44640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98281020c0 0x7f9828196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:16.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.314+0000 7f982de44640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f98281076d0 con 0x7f98281020c0 2024-09-16T01:31:16.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.314+0000 7f98277fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98281020c0 0x7f9828196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:16.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.314+0000 7f98277fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98281020c0 0x7f9828196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34744/0 (socket says 172.21.15.139:34744) 2024-09-16T01:31:16.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.314+0000 7f98277fe640 1 -- 172.21.15.139:0/1923070871 learned_addr learned my addr 172.21.15.139:0/1923070871 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:16.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.314+0000 7f98277fe640 1 -- 172.21.15.139:0/1923070871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9828197230 con 0x7f98281020c0 2024-09-16T01:31:16.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.315+0000 7f98277fe640 1 --2- 172.21.15.139:0/1923070871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98281020c0 0x7f9828196cf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9818006010 tx=0x7f981802e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:16.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.315+0000 7f9824ff9640 1 -- 172.21.15.139:0/1923070871 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9818047070 con 0x7f98281020c0 2024-09-16T01:31:16.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.315+0000 7f9824ff9640 1 -- 172.21.15.139:0/1923070871 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9818003e50 con 0x7f98281020c0 2024-09-16T01:31:16.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.316+0000 7f9824ff9640 1 -- 172.21.15.139:0/1923070871 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9818033050 con 0x7f98281020c0 2024-09-16T01:31:16.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.316+0000 7f982de44640 1 -- 172.21.15.139:0/1923070871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9828197520 con 0x7f98281020c0 2024-09-16T01:31:16.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.316+0000 7f982de44640 1 -- 172.21.15.139:0/1923070871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982819a1a0 con 0x7f98281020c0 2024-09-16T01:31:16.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.316+0000 7f9824ff9640 1 -- 172.21.15.139:0/1923070871 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f9818036070 con 0x7f98281020c0 2024-09-16T01:31:16.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.317+0000 7f9824ff9640 1 --2- 172.21.15.139:0/1923070871 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9810039300 0x7f981003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:16.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.317+0000 7f982de44640 1 -- 172.21.15.139:0/1923070871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9828107e50 con 0x7f98281020c0 2024-09-16T01:31:16.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.317+0000 7f9826ffd640 1 --2- 172.21.15.139:0/1923070871 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9810039300 0x7f981003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:16.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.317+0000 7f9824ff9640 1 -- 172.21.15.139:0/1923070871 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f981803e030 con 0x7f98281020c0 2024-09-16T01:31:16.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.317+0000 7f9826ffd640 1 --2- 172.21.15.139:0/1923070871 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9810039300 0x7f981003b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f981c0098c0 tx=0x7f981c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:16.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.321+0000 7f9824ff9640 1 -- 172.21.15.139:0/1923070871 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f981802c360 con 0x7f98281020c0 2024-09-16T01:31:16.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.495+0000 7f982de44640 1 -- 172.21.15.139:0/1923070871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f982802d970 con 0x7f98281020c0 2024-09-16T01:31:16.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.498+0000 7f9824ff9640 1 -- 172.21.15.139:0/1923070871 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f9818038450 con 0x7f98281020c0 2024-09-16T01:31:16.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.501+0000 7f982de44640 1 -- 172.21.15.139:0/1923070871 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9810039300 msgr2=0x7f981003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:16.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.501+0000 7f982de44640 1 --2- 172.21.15.139:0/1923070871 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9810039300 0x7f981003b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f981c0098c0 tx=0x7f981c006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:16.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.501+0000 7f982de44640 1 -- 172.21.15.139:0/1923070871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98281020c0 msgr2=0x7f9828196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:16.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.501+0000 7f982de44640 1 --2- 172.21.15.139:0/1923070871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98281020c0 0x7f9828196cf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9818006010 tx=0x7f981802e3c0 comp rx=0 tx=0).stop 2024-09-16T01:31:16.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.501+0000 7f982de44640 1 -- 172.21.15.139:0/1923070871 shutdown_connections 2024-09-16T01:31:16.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.501+0000 7f982de44640 1 --2- 172.21.15.139:0/1923070871 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9810039300 0x7f981003b7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:16.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.501+0000 7f982de44640 1 --2- 172.21.15.139:0/1923070871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98281020c0 0x7f9828196cf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:16.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.501+0000 7f982de44640 1 -- 172.21.15.139:0/1923070871 >> 172.21.15.139:0/1923070871 conn(0x7f98280fbca0 msgr2=0x7f98280fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:16.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.501+0000 7f982de44640 1 -- 172.21.15.139:0/1923070871 shutdown_connections 2024-09-16T01:31:16.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:16.501+0000 7f982de44640 1 -- 172.21.15.139:0/1923070871 wait complete. 2024-09-16T01:31:17.570 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T01:31:17.571 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T01:31:17.811 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:31:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:17 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1923070871' entity='client.admin' 2024-09-16T01:31:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:31:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:31:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:18.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.270+0000 7f0a83094640 1 -- 172.21.15.139:0/2813195861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a7c101640 msgr2=0x7f0a7c101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:18.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.270+0000 7f0a83094640 1 --2- 172.21.15.139:0/2813195861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a7c101640 0x7f0a7c101a40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009880 tx=0x7f0a7002a5d0 comp rx=0 tx=0).stop 2024-09-16T01:31:18.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.270+0000 7f0a83094640 1 -- 172.21.15.139:0/2813195861 shutdown_connections 2024-09-16T01:31:18.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.270+0000 7f0a83094640 1 --2- 172.21.15.139:0/2813195861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a7c101640 0x7f0a7c101a40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:18.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.270+0000 7f0a83094640 1 -- 172.21.15.139:0/2813195861 >> 172.21.15.139:0/2813195861 conn(0x7f0a7c0fcdf0 msgr2=0x7f0a7c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:18.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.270+0000 7f0a83094640 1 -- 172.21.15.139:0/2813195861 shutdown_connections 2024-09-16T01:31:18.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.270+0000 7f0a83094640 1 -- 172.21.15.139:0/2813195861 wait complete. 2024-09-16T01:31:18.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.270+0000 7f0a83094640 1 Processor -- start 2024-09-16T01:31:18.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.271+0000 7f0a83094640 1 -- start start 2024-09-16T01:31:18.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.271+0000 7f0a83094640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a7c101640 0x7f0a7c196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:18.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.271+0000 7f0a83094640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c107750 con 0x7f0a7c101640 2024-09-16T01:31:18.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.271+0000 7f0a80e30640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a7c101640 0x7f0a7c196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:18.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.271+0000 7f0a80e30640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a7c101640 0x7f0a7c196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34760/0 (socket says 172.21.15.139:34760) 2024-09-16T01:31:18.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.271+0000 7f0a80e30640 1 -- 172.21.15.139:0/3515388577 learned_addr learned my addr 172.21.15.139:0/3515388577 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:18.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.271+0000 7f0a80e30640 1 -- 172.21.15.139:0/3515388577 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c197370 con 0x7f0a7c101640 2024-09-16T01:31:18.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.271+0000 7f0a80e30640 1 --2- 172.21.15.139:0/3515388577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a7c101640 0x7f0a7c196e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009230 tx=0x7f0a700048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:18.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.271+0000 7f0a79ffb640 1 -- 172.21.15.139:0/3515388577 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a7003f070 con 0x7f0a7c101640 2024-09-16T01:31:18.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.272+0000 7f0a79ffb640 1 -- 172.21.15.139:0/3515388577 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a70003ad0 con 0x7f0a7c101640 2024-09-16T01:31:18.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.272+0000 7f0a79ffb640 1 -- 172.21.15.139:0/3515388577 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a70035030 con 0x7f0a7c101640 2024-09-16T01:31:18.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.272+0000 7f0a83094640 1 -- 172.21.15.139:0/3515388577 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7c197600 con 0x7f0a7c101640 2024-09-16T01:31:18.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.272+0000 7f0a83094640 1 -- 172.21.15.139:0/3515388577 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7c19a2f0 con 0x7f0a7c101640 2024-09-16T01:31:18.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.273+0000 7f0a79ffb640 1 -- 172.21.15.139:0/3515388577 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f0a70042070 con 0x7f0a7c101640 2024-09-16T01:31:18.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.273+0000 7f0a83094640 1 -- 172.21.15.139:0/3515388577 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a7c107ed0 con 0x7f0a7c101640 2024-09-16T01:31:18.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.273+0000 7f0a79ffb640 1 --2- 172.21.15.139:0/3515388577 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0a6c039300 0x7f0a6c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:18.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.273+0000 7f0a79ffb640 1 -- 172.21.15.139:0/3515388577 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0a7006cf90 con 0x7f0a7c101640 2024-09-16T01:31:18.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.273+0000 7f0a7bfff640 1 --2- 172.21.15.139:0/3515388577 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0a6c039300 0x7f0a6c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:18.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.274+0000 7f0a7bfff640 1 --2- 172.21.15.139:0/3515388577 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0a6c039300 0x7f0a6c03b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0a680098c0 tx=0x7f0a68006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:18.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.278+0000 7f0a79ffb640 1 -- 172.21.15.139:0/3515388577 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a70040260 con 0x7f0a7c101640 2024-09-16T01:31:18.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.446+0000 7f0a83094640 1 -- 172.21.15.139:0/3515388577 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f0a7c02d830 con 0x7f0a6c039300 2024-09-16T01:31:18.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.449+0000 7f0a79ffb640 1 -- 172.21.15.139:0/3515388577 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0a7c02d830 con 0x7f0a6c039300 2024-09-16T01:31:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.452+0000 7f0a83094640 1 -- 172.21.15.139:0/3515388577 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0a6c039300 msgr2=0x7f0a6c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.452+0000 7f0a83094640 1 --2- 172.21.15.139:0/3515388577 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0a6c039300 0x7f0a6c03b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0a680098c0 tx=0x7f0a68006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.452+0000 7f0a83094640 1 -- 172.21.15.139:0/3515388577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a7c101640 msgr2=0x7f0a7c196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.452+0000 7f0a83094640 1 --2- 172.21.15.139:0/3515388577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a7c101640 0x7f0a7c196e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009230 tx=0x7f0a700048f0 comp rx=0 tx=0).stop 2024-09-16T01:31:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.452+0000 7f0a83094640 1 -- 172.21.15.139:0/3515388577 shutdown_connections 2024-09-16T01:31:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.452+0000 7f0a83094640 1 --2- 172.21.15.139:0/3515388577 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0a6c039300 0x7f0a6c03b7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.452+0000 7f0a83094640 1 --2- 172.21.15.139:0/3515388577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a7c101640 0x7f0a7c196e30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.452+0000 7f0a83094640 1 -- 172.21.15.139:0/3515388577 >> 172.21.15.139:0/3515388577 conn(0x7f0a7c0fcdf0 msgr2=0x7f0a7c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.452+0000 7f0a83094640 1 -- 172.21.15.139:0/3515388577 shutdown_connections 2024-09-16T01:31:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:18.452+0000 7f0a83094640 1 -- 172.21.15.139:0/3515388577 wait complete. 2024-09-16T01:31:19.215 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi164 2024-09-16T01:31:19.215 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T01:31:19.215 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.conf 2024-09-16T01:31:19.235 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T01:31:19.235 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:31:19.293 INFO:tasks.cephadm:Adding host smithi164 to orchestrator... 2024-09-16T01:31:19.293 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch host add smithi164 2024-09-16T01:31:19.533 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:31:19.653 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:19 smithi139 ceph-mon[31175]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:31:19.653 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:19.653 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:31:19.653 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:19.653 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:31:19.653 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:19 smithi139 ceph-mon[31175]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-16T01:31:19.653 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:19.654 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:19.654 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:19.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.961+0000 7fa75b710640 1 -- 172.21.15.139:0/4106714831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa754106590 msgr2=0x7fa754106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:19.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.961+0000 7fa75b710640 1 --2- 172.21.15.139:0/4106714831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa754106590 0x7fa754106970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009c20 tx=0x7fa74c02aa90 comp rx=0 tx=0).stop 2024-09-16T01:31:19.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.961+0000 7fa75b710640 1 -- 172.21.15.139:0/4106714831 shutdown_connections 2024-09-16T01:31:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.961+0000 7fa75b710640 1 --2- 172.21.15.139:0/4106714831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa754106590 0x7fa754106970 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.961+0000 7fa75b710640 1 -- 172.21.15.139:0/4106714831 >> 172.21.15.139:0/4106714831 conn(0x7fa7540fc1a0 msgr2=0x7fa7540fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.961+0000 7fa75b710640 1 -- 172.21.15.139:0/4106714831 shutdown_connections 2024-09-16T01:31:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.961+0000 7fa75b710640 1 -- 172.21.15.139:0/4106714831 wait complete. 2024-09-16T01:31:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.961+0000 7fa75b710640 1 Processor -- start 2024-09-16T01:31:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.961+0000 7fa75b710640 1 -- start start 2024-09-16T01:31:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.961+0000 7fa75b710640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa754106590 0x7fa7541992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.961+0000 7fa75b710640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa75410abc0 con 0x7fa754106590 2024-09-16T01:31:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.962+0000 7fa7594ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa754106590 0x7fa7541992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.962+0000 7fa7594ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa754106590 0x7fa7541992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34772/0 (socket says 172.21.15.139:34772) 2024-09-16T01:31:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.962+0000 7fa7594ac640 1 -- 172.21.15.139:0/4205302444 learned_addr learned my addr 172.21.15.139:0/4205302444 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.962+0000 7fa7594ac640 1 -- 172.21.15.139:0/4205302444 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa754199820 con 0x7fa754106590 2024-09-16T01:31:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.962+0000 7fa7594ac640 1 --2- 172.21.15.139:0/4205302444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa754106590 0x7fa7541992e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa74c005e30 tx=0x7fa74c02fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.962+0000 7fa7527fc640 1 -- 172.21.15.139:0/4205302444 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa74c047070 con 0x7fa754106590 2024-09-16T01:31:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.962+0000 7fa7527fc640 1 -- 172.21.15.139:0/4205302444 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74c003df0 con 0x7fa754106590 2024-09-16T01:31:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.962+0000 7fa75b710640 1 -- 172.21.15.139:0/4205302444 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa754199ab0 con 0x7fa754106590 2024-09-16T01:31:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.962+0000 7fa75b710640 1 -- 172.21.15.139:0/4205302444 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa75419dbb0 con 0x7fa754106590 2024-09-16T01:31:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.962+0000 7fa7527fc640 1 -- 172.21.15.139:0/4205302444 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa74c033050 con 0x7fa754106590 2024-09-16T01:31:19.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.963+0000 7fa7527fc640 1 -- 172.21.15.139:0/4205302444 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fa74c04a070 con 0x7fa754106590 2024-09-16T01:31:19.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.963+0000 7fa7527fc640 1 --2- 172.21.15.139:0/4205302444 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa73c039300 0x7fa73c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:19.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.963+0000 7fa75b710640 1 -- 172.21.15.139:0/4205302444 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa740005290 con 0x7fa754106590 2024-09-16T01:31:19.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.963+0000 7fa7527fc640 1 -- 172.21.15.139:0/4205302444 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa74c03e030 con 0x7fa754106590 2024-09-16T01:31:19.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.963+0000 7fa758cab640 1 --2- 172.21.15.139:0/4205302444 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa73c039300 0x7fa73c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:19.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.964+0000 7fa758cab640 1 --2- 172.21.15.139:0/4205302444 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa73c039300 0x7fa73c03b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7480098c0 tx=0x7fa748006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:19.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:19.968+0000 7fa7527fc640 1 -- 172.21.15.139:0/4205302444 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa74c051080 con 0x7fa754106590 2024-09-16T01:31:20.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:20.135+0000 7fa75b710640 1 -- 172.21.15.139:0/4205302444 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}) -- 0x7fa740000bc0 con 0x7fa73c039300 2024-09-16T01:31:20.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:20 smithi139 ceph-mon[31175]: Updating smithi139:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:20 smithi139 ceph-mon[31175]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:31:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:20 smithi139 ceph-mon[31175]: Updating smithi139:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:31:21.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:21 smithi139 ceph-mon[31175]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:31:22.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:22 smithi139 ceph-mon[31175]: Deploying cephadm binary to smithi164 2024-09-16T01:31:23.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:23.419+0000 7fa7527fc640 1 -- 172.21.15.139:0/4205302444 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fa740000bc0 con 0x7fa73c039300 2024-09-16T01:31:23.420 INFO:teuthology.orchestra.run.smithi139.stdout:Added host 'smithi164' with addr '172.21.15.164' 2024-09-16T01:31:23.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:23.422+0000 7fa75b710640 1 -- 172.21.15.139:0/4205302444 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa73c039300 msgr2=0x7fa73c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:23.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:23.422+0000 7fa75b710640 1 --2- 172.21.15.139:0/4205302444 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa73c039300 0x7fa73c03b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7480098c0 tx=0x7fa748006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:23.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:23.422+0000 7fa75b710640 1 -- 172.21.15.139:0/4205302444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa754106590 msgr2=0x7fa7541992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:23.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:23.422+0000 7fa75b710640 1 --2- 172.21.15.139:0/4205302444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa754106590 0x7fa7541992e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa74c005e30 tx=0x7fa74c02fc10 comp rx=0 tx=0).stop 2024-09-16T01:31:23.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:23.422+0000 7fa75b710640 1 -- 172.21.15.139:0/4205302444 shutdown_connections 2024-09-16T01:31:23.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:23.422+0000 7fa75b710640 1 --2- 172.21.15.139:0/4205302444 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa73c039300 0x7fa73c03b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:23.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:23.422+0000 7fa75b710640 1 --2- 172.21.15.139:0/4205302444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa754106590 0x7fa7541992e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:23.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:23.422+0000 7fa75b710640 1 -- 172.21.15.139:0/4205302444 >> 172.21.15.139:0/4205302444 conn(0x7fa7540fc1a0 msgr2=0x7fa7540fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:23.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:23.422+0000 7fa75b710640 1 -- 172.21.15.139:0/4205302444 shutdown_connections 2024-09-16T01:31:23.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:23.422+0000 7fa75b710640 1 -- 172.21.15.139:0/4205302444 wait complete. 2024-09-16T01:31:24.062 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T01:31:24.301 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:31:24.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:24.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:24 smithi139 ceph-mon[31175]: Added host smithi164 2024-09-16T01:31:24.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:31:24.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:24.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.774+0000 7fcd1c211640 1 -- 172.21.15.139:0/2042541633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd141027e0 msgr2=0x7fcd14102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.774+0000 7fcd1c211640 1 --2- 172.21.15.139:0/2042541633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd141027e0 0x7fcd14102bc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009880 tx=0x7fcd1002a670 comp rx=0 tx=0).stop 2024-09-16T01:31:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.774+0000 7fcd1c211640 1 -- 172.21.15.139:0/2042541633 shutdown_connections 2024-09-16T01:31:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.774+0000 7fcd1c211640 1 --2- 172.21.15.139:0/2042541633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd141027e0 0x7fcd14102bc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.774+0000 7fcd1c211640 1 -- 172.21.15.139:0/2042541633 >> 172.21.15.139:0/2042541633 conn(0x7fcd140fc1a0 msgr2=0x7fcd140fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.774+0000 7fcd1c211640 1 -- 172.21.15.139:0/2042541633 shutdown_connections 2024-09-16T01:31:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.774+0000 7fcd1c211640 1 -- 172.21.15.139:0/2042541633 wait complete. 2024-09-16T01:31:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.774+0000 7fcd1c211640 1 Processor -- start 2024-09-16T01:31:24.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.775+0000 7fcd1c211640 1 -- start start 2024-09-16T01:31:24.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.775+0000 7fcd1c211640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd141027e0 0x7fcd14197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:24.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.775+0000 7fcd1c211640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1410bce0 con 0x7fcd141027e0 2024-09-16T01:31:24.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.775+0000 7fcd19fad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd141027e0 0x7fcd14197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:24.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.775+0000 7fcd19fad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd141027e0 0x7fcd14197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34876/0 (socket says 172.21.15.139:34876) 2024-09-16T01:31:24.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.775+0000 7fcd19fad640 1 -- 172.21.15.139:0/785237030 learned_addr learned my addr 172.21.15.139:0/785237030 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:24.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.775+0000 7fcd19fad640 1 -- 172.21.15.139:0/785237030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd141976c0 con 0x7fcd141027e0 2024-09-16T01:31:24.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.775+0000 7fcd19fad640 1 --2- 172.21.15.139:0/785237030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd141027e0 0x7fcd14197180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009230 tx=0x7fcd10003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:24.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.775+0000 7fcd0affd640 1 -- 172.21.15.139:0/785237030 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd1003f070 con 0x7fcd141027e0 2024-09-16T01:31:24.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.775+0000 7fcd1c211640 1 -- 172.21.15.139:0/785237030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd14197950 con 0x7fcd141027e0 2024-09-16T01:31:24.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.775+0000 7fcd0affd640 1 -- 172.21.15.139:0/785237030 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd1003bd40 con 0x7fcd141027e0 2024-09-16T01:31:24.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.775+0000 7fcd1c211640 1 -- 172.21.15.139:0/785237030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd14197db0 con 0x7fcd141027e0 2024-09-16T01:31:24.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.775+0000 7fcd0affd640 1 -- 172.21.15.139:0/785237030 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd10049060 con 0x7fcd141027e0 2024-09-16T01:31:24.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.776+0000 7fcd0affd640 1 -- 172.21.15.139:0/785237030 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fcd10042070 con 0x7fcd141027e0 2024-09-16T01:31:24.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.776+0000 7fcd1c211640 1 -- 172.21.15.139:0/785237030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1406a760 con 0x7fcd141027e0 2024-09-16T01:31:24.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.776+0000 7fcd0affd640 1 --2- 172.21.15.139:0/785237030 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcd000392a0 0x7fcd0003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:24.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.777+0000 7fcd0affd640 1 -- 172.21.15.139:0/785237030 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcd1006d2e0 con 0x7fcd141027e0 2024-09-16T01:31:24.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.777+0000 7fcd197ac640 1 --2- 172.21.15.139:0/785237030 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcd000392a0 0x7fcd0003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:24.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.777+0000 7fcd197ac640 1 --2- 172.21.15.139:0/785237030 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcd000392a0 0x7fcd0003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcd040098c0 tx=0x7fcd04006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:24.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.781+0000 7fcd0affd640 1 -- 172.21.15.139:0/785237030 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd1003ac70 con 0x7fcd141027e0 2024-09-16T01:31:24.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.953+0000 7fcd1c211640 1 -- 172.21.15.139:0/785237030 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd14198540 con 0x7fcd000392a0 2024-09-16T01:31:24.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.955+0000 7fcd0affd640 1 -- 172.21.15.139:0/785237030 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fcd14198540 con 0x7fcd000392a0 2024-09-16T01:31:24.955 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:31:24.955 INFO:teuthology.orchestra.run.smithi139.stdout:[{"addr": "172.21.15.139", "hostname": "smithi139", "labels": [], "status": ""}, {"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}] 2024-09-16T01:31:24.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.956+0000 7fcd1c211640 1 -- 172.21.15.139:0/785237030 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcd000392a0 msgr2=0x7fcd0003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:24.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.956+0000 7fcd1c211640 1 --2- 172.21.15.139:0/785237030 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcd000392a0 0x7fcd0003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcd040098c0 tx=0x7fcd04006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:24.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.956+0000 7fcd1c211640 1 -- 172.21.15.139:0/785237030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd141027e0 msgr2=0x7fcd14197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:24.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.956+0000 7fcd1c211640 1 --2- 172.21.15.139:0/785237030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd141027e0 0x7fcd14197180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009230 tx=0x7fcd10003d80 comp rx=0 tx=0).stop 2024-09-16T01:31:24.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.957+0000 7fcd1c211640 1 -- 172.21.15.139:0/785237030 shutdown_connections 2024-09-16T01:31:24.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.957+0000 7fcd1c211640 1 --2- 172.21.15.139:0/785237030 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcd000392a0 0x7fcd0003b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:24.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.957+0000 7fcd1c211640 1 --2- 172.21.15.139:0/785237030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd141027e0 0x7fcd14197180 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:24.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.957+0000 7fcd1c211640 1 -- 172.21.15.139:0/785237030 >> 172.21.15.139:0/785237030 conn(0x7fcd140fc1a0 msgr2=0x7fcd14103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:24.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.957+0000 7fcd1c211640 1 -- 172.21.15.139:0/785237030 shutdown_connections 2024-09-16T01:31:24.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:24.957+0000 7fcd1c211640 1 -- 172.21.15.139:0/785237030 wait complete. 2024-09-16T01:31:25.536 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi176 2024-09-16T01:31:25.536 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-16T01:31:25.536 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/etc/ceph/ceph.conf 2024-09-16T01:31:25.557 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-16T01:31:25.557 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:31:25.614 INFO:tasks.cephadm:Adding host smithi176 to orchestrator... 2024-09-16T01:31:25.615 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch host add smithi176 2024-09-16T01:31:25.856 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:31:26.083 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:26.084 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:25 smithi139 ceph-mon[31175]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:31:26.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.290+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1364405862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf8107690 msgr2=0x7f7cf8107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:26.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.290+0000 7f7cfe7ce640 1 --2- 172.21.15.139:0/1364405862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf8107690 0x7f7cf8107a70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009880 tx=0x7f7ce802a5d0 comp rx=0 tx=0).stop 2024-09-16T01:31:26.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.290+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1364405862 shutdown_connections 2024-09-16T01:31:26.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.290+0000 7f7cfe7ce640 1 --2- 172.21.15.139:0/1364405862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf8107690 0x7f7cf8107a70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:26.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.290+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1364405862 >> 172.21.15.139:0/1364405862 conn(0x7f7cf8076250 msgr2=0x7f7cf8076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:26.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.290+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1364405862 shutdown_connections 2024-09-16T01:31:26.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.290+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1364405862 wait complete. 2024-09-16T01:31:26.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.291+0000 7f7cfe7ce640 1 Processor -- start 2024-09-16T01:31:26.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.291+0000 7f7cfe7ce640 1 -- start start 2024-09-16T01:31:26.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.291+0000 7f7cfe7ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf8107690 0x7f7cf8197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:26.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.291+0000 7f7cfe7ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf810bce0 con 0x7f7cf8107690 2024-09-16T01:31:26.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.291+0000 7f7cf7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf8107690 0x7f7cf8197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:26.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.291+0000 7f7cf7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf8107690 0x7f7cf8197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34890/0 (socket says 172.21.15.139:34890) 2024-09-16T01:31:26.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.291+0000 7f7cf7fff640 1 -- 172.21.15.139:0/1891364683 learned_addr learned my addr 172.21.15.139:0/1891364683 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:26.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.291+0000 7f7cf7fff640 1 -- 172.21.15.139:0/1891364683 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf8197650 con 0x7f7cf8107690 2024-09-16T01:31:26.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.292+0000 7f7cf7fff640 1 --2- 172.21.15.139:0/1891364683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf8107690 0x7f7cf8197110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009230 tx=0x7f7ce8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:26.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.292+0000 7f7cf5ffb640 1 -- 172.21.15.139:0/1891364683 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ce803f070 con 0x7f7cf8107690 2024-09-16T01:31:26.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.292+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1891364683 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf81978e0 con 0x7f7cf8107690 2024-09-16T01:31:26.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.292+0000 7f7cf5ffb640 1 -- 172.21.15.139:0/1891364683 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce803bc80 con 0x7f7cf8107690 2024-09-16T01:31:26.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.292+0000 7f7cf5ffb640 1 -- 172.21.15.139:0/1891364683 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ce8035030 con 0x7f7cf8107690 2024-09-16T01:31:26.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.292+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1891364683 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf819b9e0 con 0x7f7cf8107690 2024-09-16T01:31:26.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.293+0000 7f7cf5ffb640 1 -- 172.21.15.139:0/1891364683 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f7ce8042070 con 0x7f7cf8107690 2024-09-16T01:31:26.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.293+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1891364683 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cf8102d30 con 0x7f7cf8107690 2024-09-16T01:31:26.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.293+0000 7f7cf5ffb640 1 --2- 172.21.15.139:0/1891364683 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7ce40392a0 0x7f7ce403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:26.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.293+0000 7f7cf5ffb640 1 -- 172.21.15.139:0/1891364683 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7ce806cfb0 con 0x7f7cf8107690 2024-09-16T01:31:26.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.293+0000 7f7ceffff640 1 --2- 172.21.15.139:0/1891364683 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7ce40392a0 0x7f7ce403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:26.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.294+0000 7f7ceffff640 1 --2- 172.21.15.139:0/1891364683 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7ce40392a0 0x7f7ce403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00098c0 tx=0x7f7ce0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:26.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.298+0000 7f7cf5ffb640 1 -- 172.21.15.139:0/1891364683 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ce804c350 con 0x7f7cf8107690 2024-09-16T01:31:26.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:26.471+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1891364683 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi176", "target": ["mon-mgr", ""]}) -- 0x7f7cf8198470 con 0x7f7ce40392a0 2024-09-16T01:31:27.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:26 smithi139 ceph-mon[31175]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi176", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:31:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:28 smithi139 ceph-mon[31175]: Deploying cephadm binary to smithi176 2024-09-16T01:31:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:28 smithi139 ceph-mon[31175]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:31:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:31:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:28 smithi139 ceph-mon[31175]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-16T01:31:29.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:29.581+0000 7f7cf5ffb640 1 -- 172.21.15.139:0/1891364683 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f7cf8198470 con 0x7f7ce40392a0 2024-09-16T01:31:29.584 INFO:teuthology.orchestra.run.smithi139.stdout:Added host 'smithi176' with addr '172.21.15.176' 2024-09-16T01:31:29.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:29.583+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1891364683 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7ce40392a0 msgr2=0x7f7ce403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:29.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:29.583+0000 7f7cfe7ce640 1 --2- 172.21.15.139:0/1891364683 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7ce40392a0 0x7f7ce403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00098c0 tx=0x7f7ce0006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:29.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:29.583+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1891364683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf8107690 msgr2=0x7f7cf8197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:29.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:29.583+0000 7f7cfe7ce640 1 --2- 172.21.15.139:0/1891364683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf8107690 0x7f7cf8197110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009230 tx=0x7f7ce8003d40 comp rx=0 tx=0).stop 2024-09-16T01:31:29.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:29.584+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1891364683 shutdown_connections 2024-09-16T01:31:29.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:29.584+0000 7f7cfe7ce640 1 --2- 172.21.15.139:0/1891364683 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7ce40392a0 0x7f7ce403b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:29.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:29.584+0000 7f7cfe7ce640 1 --2- 172.21.15.139:0/1891364683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf8107690 0x7f7cf8197110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:29.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:29.584+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1891364683 >> 172.21.15.139:0/1891364683 conn(0x7f7cf8076250 msgr2=0x7f7cf80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:29.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:29.584+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1891364683 shutdown_connections 2024-09-16T01:31:29.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:29.584+0000 7f7cfe7ce640 1 -- 172.21.15.139:0/1891364683 wait complete. 2024-09-16T01:31:29.878 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:29 smithi139 ceph-mon[31175]: Updating smithi164:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:29.878 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:29 smithi139 ceph-mon[31175]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:31:29.878 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:29 smithi139 ceph-mon[31175]: Updating smithi164:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:31:29.878 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:29.879 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:29.879 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:30.174 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T01:31:30.412 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:31:30.790 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:30 smithi139 ceph-mon[31175]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:30.790 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:30 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:30.790 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:30 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:31:30.790 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:30 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:30.790 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:30 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:30.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.868+0000 7f50204ff640 1 -- 172.21.15.139:0/3088994526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5018106590 msgr2=0x7f5018106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:30.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.868+0000 7f50204ff640 1 --2- 172.21.15.139:0/3088994526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5018106590 0x7f5018106970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5014009880 tx=0x7f501402a5d0 comp rx=0 tx=0).stop 2024-09-16T01:31:30.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.868+0000 7f50204ff640 1 -- 172.21.15.139:0/3088994526 shutdown_connections 2024-09-16T01:31:30.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.868+0000 7f50204ff640 1 --2- 172.21.15.139:0/3088994526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5018106590 0x7f5018106970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:30.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.868+0000 7f50204ff640 1 -- 172.21.15.139:0/3088994526 >> 172.21.15.139:0/3088994526 conn(0x7f50180fc1a0 msgr2=0x7f50180fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:30.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.868+0000 7f50204ff640 1 -- 172.21.15.139:0/3088994526 shutdown_connections 2024-09-16T01:31:30.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.868+0000 7f50204ff640 1 -- 172.21.15.139:0/3088994526 wait complete. 2024-09-16T01:31:30.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.869+0000 7f50204ff640 1 Processor -- start 2024-09-16T01:31:30.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.869+0000 7f50204ff640 1 -- start start 2024-09-16T01:31:30.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.869+0000 7f50204ff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5018106590 0x7f50180fc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:30.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.869+0000 7f50204ff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f501810abc0 con 0x7f5018106590 2024-09-16T01:31:30.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.869+0000 7f501e29b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5018106590 0x7f50180fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:30.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.869+0000 7f501e29b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5018106590 0x7f50180fc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34914/0 (socket says 172.21.15.139:34914) 2024-09-16T01:31:30.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.869+0000 7f501e29b640 1 -- 172.21.15.139:0/3314194365 learned_addr learned my addr 172.21.15.139:0/3314194365 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:30.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.869+0000 7f501e29b640 1 -- 172.21.15.139:0/3314194365 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50180fe540 con 0x7f5018106590 2024-09-16T01:31:30.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.869+0000 7f501e29b640 1 --2- 172.21.15.139:0/3314194365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5018106590 0x7f50180fc650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5014009230 tx=0x7f5014003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:30.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.869+0000 7f500f7fe640 1 -- 172.21.15.139:0/3314194365 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f501403f070 con 0x7f5018106590 2024-09-16T01:31:30.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.870+0000 7f50204ff640 1 -- 172.21.15.139:0/3314194365 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50180fcb90 con 0x7f5018106590 2024-09-16T01:31:30.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.870+0000 7f500f7fe640 1 -- 172.21.15.139:0/3314194365 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f501403bc80 con 0x7f5018106590 2024-09-16T01:31:30.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.870+0000 7f500f7fe640 1 -- 172.21.15.139:0/3314194365 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5014035030 con 0x7f5018106590 2024-09-16T01:31:30.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.870+0000 7f50204ff640 1 -- 172.21.15.139:0/3314194365 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50180fd050 con 0x7f5018106590 2024-09-16T01:31:30.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.870+0000 7f500f7fe640 1 -- 172.21.15.139:0/3314194365 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f5014042070 con 0x7f5018106590 2024-09-16T01:31:30.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.871+0000 7f50204ff640 1 -- 172.21.15.139:0/3314194365 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5018101c00 con 0x7f5018106590 2024-09-16T01:31:30.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.871+0000 7f500f7fe640 1 --2- 172.21.15.139:0/3314194365 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f50040392a0 0x7f500403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:30.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.871+0000 7f500f7fe640 1 -- 172.21.15.139:0/3314194365 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f501406cfd0 con 0x7f5018106590 2024-09-16T01:31:30.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.871+0000 7f501da9a640 1 --2- 172.21.15.139:0/3314194365 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f50040392a0 0x7f500403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:30.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.871+0000 7f501da9a640 1 --2- 172.21.15.139:0/3314194365 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f50040392a0 0x7f500403b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f50080098c0 tx=0x7f5008006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:30.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:30.876+0000 7f500f7fe640 1 -- 172.21.15.139:0/3314194365 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50140372d0 con 0x7f5018106590 2024-09-16T01:31:31.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:31.049+0000 7f50204ff640 1 -- 172.21.15.139:0/3314194365 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f50180fdb80 con 0x7f50040392a0 2024-09-16T01:31:31.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:31.050+0000 7f500f7fe640 1 -- 172.21.15.139:0/3314194365 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f50180fdb80 con 0x7f50040392a0 2024-09-16T01:31:31.052 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:31:31.052 INFO:teuthology.orchestra.run.smithi139.stdout:[{"addr": "172.21.15.139", "hostname": "smithi139", "labels": [], "status": ""}, {"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}, {"addr": "172.21.15.176", "hostname": "smithi176", "labels": [], "status": ""}] 2024-09-16T01:31:31.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:31.052+0000 7f50204ff640 1 -- 172.21.15.139:0/3314194365 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f50040392a0 msgr2=0x7f500403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:31.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:31.052+0000 7f50204ff640 1 --2- 172.21.15.139:0/3314194365 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f50040392a0 0x7f500403b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f50080098c0 tx=0x7f5008006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:31.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:31.052+0000 7f50204ff640 1 -- 172.21.15.139:0/3314194365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5018106590 msgr2=0x7f50180fc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:31.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:31.052+0000 7f50204ff640 1 --2- 172.21.15.139:0/3314194365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5018106590 0x7f50180fc650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5014009230 tx=0x7f5014003d40 comp rx=0 tx=0).stop 2024-09-16T01:31:31.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:31.053+0000 7f50204ff640 1 -- 172.21.15.139:0/3314194365 shutdown_connections 2024-09-16T01:31:31.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:31.053+0000 7f50204ff640 1 --2- 172.21.15.139:0/3314194365 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f50040392a0 0x7f500403b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:31.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:31.053+0000 7f50204ff640 1 --2- 172.21.15.139:0/3314194365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5018106590 0x7f50180fc650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:31.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:31.053+0000 7f50204ff640 1 -- 172.21.15.139:0/3314194365 >> 172.21.15.139:0/3314194365 conn(0x7f50180fc1a0 msgr2=0x7f5018069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:31.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:31.053+0000 7f50204ff640 1 -- 172.21.15.139:0/3314194365 shutdown_connections 2024-09-16T01:31:31.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:31.053+0000 7f50204ff640 1 -- 172.21.15.139:0/3314194365 wait complete. 2024-09-16T01:31:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:31 smithi139 ceph-mon[31175]: Added host smithi176 2024-09-16T01:31:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:31 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:31.743 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T01:31:31.744 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T01:31:31.987 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:31:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.392+0000 7f7868192640 1 -- 172.21.15.139:0/3594390099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7860100570 msgr2=0x7f7860100950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:32.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.392+0000 7f7868192640 1 --2- 172.21.15.139:0/3594390099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7860100570 0x7f7860100950 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f785c009880 tx=0x7f785c02a670 comp rx=0 tx=0).stop 2024-09-16T01:31:32.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.392+0000 7f7868192640 1 -- 172.21.15.139:0/3594390099 shutdown_connections 2024-09-16T01:31:32.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.392+0000 7f7868192640 1 --2- 172.21.15.139:0/3594390099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7860100570 0x7f7860100950 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:32.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.392+0000 7f7868192640 1 -- 172.21.15.139:0/3594390099 >> 172.21.15.139:0/3594390099 conn(0x7f78600fc1a0 msgr2=0x7f78600fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:32.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.392+0000 7f7868192640 1 -- 172.21.15.139:0/3594390099 shutdown_connections 2024-09-16T01:31:32.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.392+0000 7f7868192640 1 -- 172.21.15.139:0/3594390099 wait complete. 2024-09-16T01:31:32.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.393+0000 7f7868192640 1 Processor -- start 2024-09-16T01:31:32.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.393+0000 7f7868192640 1 -- start start 2024-09-16T01:31:32.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.393+0000 7f7868192640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7860100570 0x7f78601a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:32.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.393+0000 7f7868192640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7860112300 con 0x7f7860100570 2024-09-16T01:31:32.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.393+0000 7f7865f2e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7860100570 0x7f78601a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:32.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.393+0000 7f7865f2e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7860100570 0x7f78601a33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38210/0 (socket says 172.21.15.139:38210) 2024-09-16T01:31:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.393+0000 7f7865f2e640 1 -- 172.21.15.139:0/1728058196 learned_addr learned my addr 172.21.15.139:0/1728058196 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.394+0000 7f7865f2e640 1 -- 172.21.15.139:0/1728058196 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78601a3920 con 0x7f7860100570 2024-09-16T01:31:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.394+0000 7f7865f2e640 1 --2- 172.21.15.139:0/1728058196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7860100570 0x7f78601a33e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f785c009230 tx=0x7f785c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.394+0000 7f7856ffd640 1 -- 172.21.15.139:0/1728058196 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f785c03f070 con 0x7f7860100570 2024-09-16T01:31:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.394+0000 7f7868192640 1 -- 172.21.15.139:0/1728058196 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78601a3bb0 con 0x7f7860100570 2024-09-16T01:31:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.394+0000 7f7856ffd640 1 -- 172.21.15.139:0/1728058196 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f785c03bd40 con 0x7f7860100570 2024-09-16T01:31:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.394+0000 7f7856ffd640 1 -- 172.21.15.139:0/1728058196 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f785c035030 con 0x7f7860100570 2024-09-16T01:31:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.394+0000 7f7868192640 1 -- 172.21.15.139:0/1728058196 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f786019d5a0 con 0x7f7860100570 2024-09-16T01:31:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.395+0000 7f7856ffd640 1 -- 172.21.15.139:0/1728058196 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f785c042070 con 0x7f7860100570 2024-09-16T01:31:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.395+0000 7f7868192640 1 -- 172.21.15.139:0/1728058196 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f786006a7c0 con 0x7f7860100570 2024-09-16T01:31:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.395+0000 7f7856ffd640 1 --2- 172.21.15.139:0/1728058196 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f784c0392a0 0x7f784c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.395+0000 7f7856ffd640 1 -- 172.21.15.139:0/1728058196 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f785c06d010 con 0x7f7860100570 2024-09-16T01:31:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.395+0000 7f786572d640 1 --2- 172.21.15.139:0/1728058196 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f784c0392a0 0x7f784c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.396+0000 7f786572d640 1 --2- 172.21.15.139:0/1728058196 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f784c0392a0 0x7f784c03b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f78500098c0 tx=0x7f7850006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.400+0000 7f7856ffd640 1 -- 172.21.15.139:0/1728058196 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f785c031de0 con 0x7f7860100570 2024-09-16T01:31:32.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:32.571+0000 7f7868192640 1 -- 172.21.15.139:0/1728058196 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f78601009d0 con 0x7f7860100570 2024-09-16T01:31:32.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:32 smithi139 ceph-mon[31175]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:31:32.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:32 smithi139 ceph-mon[31175]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:33.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:33.508+0000 7f7856ffd640 1 -- 172.21.15.139:0/1728058196 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f785c03a950 con 0x7f7860100570 2024-09-16T01:31:33.510 INFO:teuthology.orchestra.run.smithi139.stderr:adjusted tunables profile to default 2024-09-16T01:31:33.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:33.511+0000 7f7868192640 1 -- 172.21.15.139:0/1728058196 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f784c0392a0 msgr2=0x7f784c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:33.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:33.511+0000 7f7868192640 1 --2- 172.21.15.139:0/1728058196 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f784c0392a0 0x7f784c03b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f78500098c0 tx=0x7f7850006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:33.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:33.511+0000 7f7868192640 1 -- 172.21.15.139:0/1728058196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7860100570 msgr2=0x7f78601a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:33.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:33.511+0000 7f7868192640 1 --2- 172.21.15.139:0/1728058196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7860100570 0x7f78601a33e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f785c009230 tx=0x7f785c003d80 comp rx=0 tx=0).stop 2024-09-16T01:31:33.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:33.511+0000 7f7868192640 1 -- 172.21.15.139:0/1728058196 shutdown_connections 2024-09-16T01:31:33.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:33.511+0000 7f7868192640 1 --2- 172.21.15.139:0/1728058196 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f784c0392a0 0x7f784c03b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:33.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:33.511+0000 7f7868192640 1 --2- 172.21.15.139:0/1728058196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7860100570 0x7f78601a33e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:33.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:33.511+0000 7f7868192640 1 -- 172.21.15.139:0/1728058196 >> 172.21.15.139:0/1728058196 conn(0x7f78600fc1a0 msgr2=0x7f78600fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:33.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:33.511+0000 7f7868192640 1 -- 172.21.15.139:0/1728058196 shutdown_connections 2024-09-16T01:31:33.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:33.511+0000 7f7868192640 1 -- 172.21.15.139:0/1728058196 wait complete. 2024-09-16T01:31:33.900 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:33 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1728058196' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T01:31:34.213 INFO:tasks.cephadm:Adding mon.a on smithi139 2024-09-16T01:31:34.213 INFO:tasks.cephadm:Adding mon.b on smithi164 2024-09-16T01:31:34.213 INFO:tasks.cephadm:Adding mon.c on smithi176 2024-09-16T01:31:34.213 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi139:172.21.15.139=a;smithi164:172.21.15.164=b;smithi176:172.21.15.176=c' 2024-09-16T01:31:34.456 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:34.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:34 smithi139 ceph-mon[31175]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:34.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:34 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1728058196' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T01:31:34.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:34 smithi139 ceph-mon[31175]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:31:34.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:34.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:34.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:34.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:34.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi176", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:31:34.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:34.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:31:34.933 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.921+0000 7f1b19513640 1 -- 172.21.15.176:0/4098806683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b14107660 msgr2=0x7f1b14107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:34.933 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.921+0000 7f1b19513640 1 --2- 172.21.15.176:0/4098806683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b14107660 0x7f1b14107a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009880 tx=0x7f1afc02a5d0 comp rx=0 tx=0).stop 2024-09-16T01:31:34.933 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.921+0000 7f1b19513640 1 -- 172.21.15.176:0/4098806683 shutdown_connections 2024-09-16T01:31:34.933 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.921+0000 7f1b19513640 1 --2- 172.21.15.176:0/4098806683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b14107660 0x7f1b14107a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.921+0000 7f1b19513640 1 -- 172.21.15.176:0/4098806683 >> 172.21.15.176:0/4098806683 conn(0x7f1b140fd2b0 msgr2=0x7f1b140ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.921+0000 7f1b19513640 1 -- 172.21.15.176:0/4098806683 shutdown_connections 2024-09-16T01:31:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.922+0000 7f1b19513640 1 -- 172.21.15.176:0/4098806683 wait complete. 2024-09-16T01:31:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.922+0000 7f1b19513640 1 Processor -- start 2024-09-16T01:31:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.922+0000 7f1b19513640 1 -- start start 2024-09-16T01:31:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.922+0000 7f1b19513640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b14107660 0x7f1b141970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.922+0000 7f1b19513640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1410bc70 con 0x7f1b14107660 2024-09-16T01:31:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.923+0000 7f1b12ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b14107660 0x7f1b141970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.923+0000 7f1b12ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b14107660 0x7f1b141970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.176:60878/0 (socket says 172.21.15.176:60878) 2024-09-16T01:31:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.923+0000 7f1b12ffd640 1 -- 172.21.15.176:0/21031560 learned_addr learned my addr 172.21.15.176:0/21031560 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-16T01:31:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.923+0000 7f1b12ffd640 1 -- 172.21.15.176:0/21031560 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b141975e0 con 0x7f1b14107660 2024-09-16T01:31:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.923+0000 7f1b12ffd640 1 --2- 172.21.15.176:0/21031560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b14107660 0x7f1b141970a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009230 tx=0x7f1afc003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:34.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.923+0000 7f1b0bfff640 1 -- 172.21.15.176:0/21031560 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1afc03f070 con 0x7f1b14107660 2024-09-16T01:31:34.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.923+0000 7f1b19513640 1 -- 172.21.15.176:0/21031560 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b141978d0 con 0x7f1b14107660 2024-09-16T01:31:34.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.923+0000 7f1b0bfff640 1 -- 172.21.15.176:0/21031560 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1afc03bc80 con 0x7f1b14107660 2024-09-16T01:31:34.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.923+0000 7f1b0bfff640 1 -- 172.21.15.176:0/21031560 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1afc035030 con 0x7f1b14107660 2024-09-16T01:31:34.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.923+0000 7f1b19513640 1 -- 172.21.15.176:0/21031560 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1419b970 con 0x7f1b14107660 2024-09-16T01:31:34.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.924+0000 7f1b0bfff640 1 -- 172.21.15.176:0/21031560 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f1afc042070 con 0x7f1b14107660 2024-09-16T01:31:34.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.924+0000 7f1b19513640 1 -- 172.21.15.176:0/21031560 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b14102d10 con 0x7f1b14107660 2024-09-16T01:31:34.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.924+0000 7f1b0bfff640 1 --2- 172.21.15.176:0/21031560 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1b00039300 0x7f1b0003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:34.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.924+0000 7f1b0bfff640 1 -- 172.21.15.176:0/21031560 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1afc06d0c0 con 0x7f1b14107660 2024-09-16T01:31:34.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.925+0000 7f1b127fc640 1 --2- 172.21.15.176:0/21031560 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1b00039300 0x7f1b0003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:34.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.925+0000 7f1b127fc640 1 --2- 172.21.15.176:0/21031560 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1b00039300 0x7f1b0003b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1b040098c0 tx=0x7f1b04006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:34.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:34.929+0000 7f1b0bfff640 1 -- 172.21.15.176:0/21031560 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1afc031e20 con 0x7f1b14107660 2024-09-16T01:31:35.110 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:35.108+0000 7f1b19513640 1 -- 172.21.15.176:0/21031560 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi139:172.21.15.139=a;smithi164:172.21.15.164=b;smithi176:172.21.15.176=c", "target": ["mon-mgr", ""]}) -- 0x7f1b14198400 con 0x7f1b00039300 2024-09-16T01:31:35.115 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:35.113+0000 7f1b0bfff640 1 -- 172.21.15.176:0/21031560 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1b14198400 con 0x7f1b00039300 2024-09-16T01:31:35.115 INFO:teuthology.orchestra.run.smithi176.stdout:Scheduled mon update... 2024-09-16T01:31:35.118 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:35.116+0000 7f1b19513640 1 -- 172.21.15.176:0/21031560 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1b00039300 msgr2=0x7f1b0003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:35.118 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:35.116+0000 7f1b19513640 1 --2- 172.21.15.176:0/21031560 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1b00039300 0x7f1b0003b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1b040098c0 tx=0x7f1b04006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:35.118 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:35.116+0000 7f1b19513640 1 -- 172.21.15.176:0/21031560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b14107660 msgr2=0x7f1b141970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:35.118 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:35.116+0000 7f1b19513640 1 --2- 172.21.15.176:0/21031560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b14107660 0x7f1b141970a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009230 tx=0x7f1afc003d40 comp rx=0 tx=0).stop 2024-09-16T01:31:35.118 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:35.116+0000 7f1b19513640 1 -- 172.21.15.176:0/21031560 shutdown_connections 2024-09-16T01:31:35.118 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:35.116+0000 7f1b19513640 1 --2- 172.21.15.176:0/21031560 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1b00039300 0x7f1b0003b7c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:35.118 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:35.116+0000 7f1b19513640 1 --2- 172.21.15.176:0/21031560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b14107660 0x7f1b141970a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:35.119 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:35.116+0000 7f1b19513640 1 -- 172.21.15.176:0/21031560 >> 172.21.15.176:0/21031560 conn(0x7f1b140fd2b0 msgr2=0x7f1b140feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:35.119 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:35.116+0000 7f1b19513640 1 -- 172.21.15.176:0/21031560 shutdown_connections 2024-09-16T01:31:35.119 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:35.116+0000 7f1b19513640 1 -- 172.21.15.176:0/21031560 wait complete. 2024-09-16T01:31:35.724 DEBUG:teuthology.orchestra.run.smithi164:mon.b> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.b.service 2024-09-16T01:31:35.727 DEBUG:teuthology.orchestra.run.smithi176:mon.c> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.c.service 2024-09-16T01:31:35.729 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T01:31:35.729 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T01:31:35.875 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:35 smithi139 ceph-mon[31175]: Updating smithi176:/etc/ceph/ceph.conf 2024-09-16T01:31:35.875 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:35 smithi139 ceph-mon[31175]: Updating smithi176:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:35.875 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:35 smithi139 ceph-mon[31175]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:31:35.875 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:35 smithi139 ceph-mon[31175]: Updating smithi176:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:31:35.875 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:35.875 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:35.876 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:35.876 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:35.876 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:31:35.876 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:35.876 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:31:35.876 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:35.876 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:31:35.876 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:36.825 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:36 smithi139 ceph-mon[31175]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi139:172.21.15.139=a;smithi164:172.21.15.164=b;smithi176:172.21.15.176=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:31:36.825 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:36 smithi139 ceph-mon[31175]: Saving service mon spec with placement smithi139:172.21.15.139=a;smithi164:172.21.15.164=b;smithi176:172.21.15.176=c;count:3 2024-09-16T01:31:36.825 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:36 smithi139 ceph-mon[31175]: Deploying daemon mon.c on smithi176 2024-09-16T01:31:36.825 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:36 smithi139 ceph-mon[31175]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:36.890 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:37.786 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.783+0000 7f5465dc2640 1 -- 172.21.15.176:0/680356577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5460107690 msgr2=0x7f5460107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:37.970 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.784+0000 7f5465dc2640 1 --2- 172.21.15.176:0/680356577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5460107690 0x7f5460107a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5450009880 tx=0x7f545002a5d0 comp rx=0 tx=0).stop 2024-09-16T01:31:37.971 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.784+0000 7f5465dc2640 1 -- 172.21.15.176:0/680356577 shutdown_connections 2024-09-16T01:31:37.971 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.784+0000 7f5465dc2640 1 --2- 172.21.15.176:0/680356577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5460107690 0x7f5460107a70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:37.971 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.784+0000 7f5465dc2640 1 -- 172.21.15.176:0/680356577 >> 172.21.15.176:0/680356577 conn(0x7f5460076250 msgr2=0x7f5460076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:37.971 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.784+0000 7f5465dc2640 1 -- 172.21.15.176:0/680356577 shutdown_connections 2024-09-16T01:31:37.971 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.784+0000 7f5465dc2640 1 -- 172.21.15.176:0/680356577 wait complete. 2024-09-16T01:31:37.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.784+0000 7f5465dc2640 1 Processor -- start 2024-09-16T01:31:37.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.784+0000 7f5465dc2640 1 -- start start 2024-09-16T01:31:37.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.785+0000 7f5465dc2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5460107690 0x7f5460197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:37.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.785+0000 7f5465dc2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f546010bce0 con 0x7f5460107690 2024-09-16T01:31:37.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.785+0000 7f545f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5460107690 0x7f5460197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:37.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.785+0000 7f545f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5460107690 0x7f5460197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.176:60894/0 (socket says 172.21.15.176:60894) 2024-09-16T01:31:37.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.785+0000 7f545f7fe640 1 -- 172.21.15.176:0/3803509298 learned_addr learned my addr 172.21.15.176:0/3803509298 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-16T01:31:37.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.785+0000 7f545f7fe640 1 -- 172.21.15.176:0/3803509298 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5460197650 con 0x7f5460107690 2024-09-16T01:31:37.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.786+0000 7f545f7fe640 1 --2- 172.21.15.176:0/3803509298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5460107690 0x7f5460197110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5450009230 tx=0x7f5450003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:37.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.786+0000 7f545d7fa640 1 -- 172.21.15.176:0/3803509298 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f545003f070 con 0x7f5460107690 2024-09-16T01:31:37.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.786+0000 7f5465dc2640 1 -- 172.21.15.176:0/3803509298 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54601978e0 con 0x7f5460107690 2024-09-16T01:31:37.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.786+0000 7f545d7fa640 1 -- 172.21.15.176:0/3803509298 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f545003bc80 con 0x7f5460107690 2024-09-16T01:31:37.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.786+0000 7f545d7fa640 1 -- 172.21.15.176:0/3803509298 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5450035030 con 0x7f5460107690 2024-09-16T01:31:37.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.786+0000 7f5465dc2640 1 -- 172.21.15.176:0/3803509298 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546019b9e0 con 0x7f5460107690 2024-09-16T01:31:37.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.787+0000 7f545d7fa640 1 -- 172.21.15.176:0/3803509298 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f5450042070 con 0x7f5460107690 2024-09-16T01:31:37.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.787+0000 7f5465dc2640 1 -- 172.21.15.176:0/3803509298 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5460102d30 con 0x7f5460107690 2024-09-16T01:31:37.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.787+0000 7f545d7fa640 1 --2- 172.21.15.176:0/3803509298 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f544c0392a0 0x7f544c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:37.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.787+0000 7f545d7fa640 1 -- 172.21.15.176:0/3803509298 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f545006d0b0 con 0x7f5460107690 2024-09-16T01:31:37.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.787+0000 7f5457fff640 1 --2- 172.21.15.176:0/3803509298 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f544c0392a0 0x7f544c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:37.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.788+0000 7f5457fff640 1 --2- 172.21.15.176:0/3803509298 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f544c0392a0 0x7f544c03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f54480098c0 tx=0x7f5448006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:37.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:37.793+0000 7f545d7fa640 1 -- 172.21.15.176:0/3803509298 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5450031e20 con 0x7f5460107690 2024-09-16T01:31:38.025 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:38.023+0000 7f5465dc2640 1 -- 172.21.15.176:0/3803509298 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5460198470 con 0x7f5460107690 2024-09-16T01:31:38.025 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:38.023+0000 7f545d7fa640 1 -- 172.21.15.176:0/3803509298 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f5450031570 con 0x7f5460107690 2024-09-16T01:31:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:31:38.026 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","modified":"2024-09-16T01:30:10.119520Z","created":"2024-09-16T01:30:10.119520Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:3300","nonce":0},{"type":"v1","addr":"172.21.15.139:6789","nonce":0}]},"addr":"172.21.15.139:6789/0","public_addr":"172.21.15.139:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T01:31:38.026 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-09-16T01:31:38.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:38.026+0000 7f5465dc2640 1 -- 172.21.15.176:0/3803509298 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f544c0392a0 msgr2=0x7f544c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:38.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:38.026+0000 7f5465dc2640 1 --2- 172.21.15.176:0/3803509298 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f544c0392a0 0x7f544c03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f54480098c0 tx=0x7f5448006db0 comp rx=0 tx=0).stop 2024-09-16T01:31:38.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:38.026+0000 7f5465dc2640 1 -- 172.21.15.176:0/3803509298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5460107690 msgr2=0x7f5460197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:38.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:38.026+0000 7f5465dc2640 1 --2- 172.21.15.176:0/3803509298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5460107690 0x7f5460197110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5450009230 tx=0x7f5450003d40 comp rx=0 tx=0).stop 2024-09-16T01:31:38.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:38.026+0000 7f5465dc2640 1 -- 172.21.15.176:0/3803509298 shutdown_connections 2024-09-16T01:31:38.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:38.026+0000 7f5465dc2640 1 --2- 172.21.15.176:0/3803509298 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f544c0392a0 0x7f544c03b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:38.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:38.026+0000 7f5465dc2640 1 --2- 172.21.15.176:0/3803509298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5460107690 0x7f5460197110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:38.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:38.026+0000 7f5465dc2640 1 -- 172.21.15.176:0/3803509298 >> 172.21.15.176:0/3803509298 conn(0x7f5460076250 msgr2=0x7f54600fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:38.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:38.026+0000 7f5465dc2640 1 -- 172.21.15.176:0/3803509298 shutdown_connections 2024-09-16T01:31:38.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:38.026+0000 7f5465dc2640 1 -- 172.21.15.176:0/3803509298 wait complete. 2024-09-16T01:31:38.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:38 smithi139 ceph-mon[31175]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:38.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:38 smithi139 ceph-mon[31175]: from='client.? 172.21.15.176:0/3803509298' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:31:39.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:39 smithi176 systemd[1]: Started Ceph mon.c for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:31:39.952 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T01:31:39.952 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T01:31:40.210 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:31:45.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.012+0000 7f8682d21640 1 -- 172.21.15.176:0/3034793369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f866c003530 msgr2=0x7f866c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:45.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.012+0000 7f8682d21640 1 --2- 172.21.15.176:0/3034793369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f866c003530 0x7f866c0059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f867c06aa60 tx=0x7f867002c490 comp rx=0 tx=0).stop 2024-09-16T01:31:45.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.012+0000 7f8682d21640 1 -- 172.21.15.176:0/3034793369 shutdown_connections 2024-09-16T01:31:45.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.012+0000 7f8682d21640 1 --2- 172.21.15.176:0/3034793369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f866c003530 0x7f866c0059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:45.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.012+0000 7f8682d21640 1 -- 172.21.15.176:0/3034793369 >> 172.21.15.176:0/3034793369 conn(0x7f867c0fc220 msgr2=0x7f867c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:45.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.012+0000 7f8682d21640 1 -- 172.21.15.176:0/3034793369 shutdown_connections 2024-09-16T01:31:45.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.012+0000 7f8682d21640 1 -- 172.21.15.176:0/3034793369 wait complete. 2024-09-16T01:31:45.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.013+0000 7f8682d21640 1 Processor -- start 2024-09-16T01:31:45.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.013+0000 7f8682d21640 1 -- start start 2024-09-16T01:31:45.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.013+0000 7f8682d21640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f866c003530 0x7f867c197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:45.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.013+0000 7f8682d21640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f867c1978c0 0x7f867c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:45.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.013+0000 7f8682d21640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f867c109780 con 0x7f867c1978c0 2024-09-16T01:31:45.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.013+0000 7f8682d21640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f867c109600 con 0x7f866c003530 2024-09-16T01:31:45.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.013+0000 7f8680abd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f866c003530 0x7f867c197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:45.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.013+0000 7f8680abd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f866c003530 0x7f867c197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:45334/0 (socket says 172.21.15.176:45334) 2024-09-16T01:31:45.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.013+0000 7f8680abd640 1 -- 172.21.15.176:0/984535915 learned_addr learned my addr 172.21.15.176:0/984535915 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-16T01:31:45.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.014+0000 7f867bfff640 1 --2- 172.21.15.176:0/984535915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f867c1978c0 0x7f867c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:45.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.014+0000 7f8680abd640 1 -- 172.21.15.176:0/984535915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f866c003530 msgr2=0x7f867c197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-16T01:31:45.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.014+0000 7f8680abd640 1 -- 172.21.15.176:0/984535915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f866c003530 msgr2=0x7f867c197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T01:31:45.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.014+0000 7f8680abd640 1 --2- 172.21.15.176:0/984535915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f866c003530 0x7f867c197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T01:31:45.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.014+0000 7f8680abd640 1 --2- 172.21.15.176:0/984535915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f866c003530 0x7f867c197380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:31:45.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.014+0000 7f867bfff640 1 -- 172.21.15.176:0/984535915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f866c003530 msgr2=0x7f867c197380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:31:45.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.014+0000 7f867bfff640 1 --2- 172.21.15.176:0/984535915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f866c003530 0x7f867c197380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:45.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.014+0000 7f867bfff640 1 -- 172.21.15.176:0/984535915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c197ee0 con 0x7f867c1978c0 2024-09-16T01:31:45.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.014+0000 7f867bfff640 1 --2- 172.21.15.176:0/984535915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f867c1978c0 0x7f867c19bc00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f866800b3d0 tx=0x7f866800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:45.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.014+0000 7f8679ffb640 1 -- 172.21.15.176:0/984535915 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8668011630 con 0x7f867c1978c0 2024-09-16T01:31:45.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.014+0000 7f8679ffb640 1 -- 172.21.15.176:0/984535915 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8668011cb0 con 0x7f867c1978c0 2024-09-16T01:31:45.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.014+0000 7f8682d21640 1 -- 172.21.15.176:0/984535915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867c19c140 con 0x7f867c1978c0 2024-09-16T01:31:45.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.014+0000 7f8679ffb640 1 -- 172.21.15.176:0/984535915 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8668010df0 con 0x7f867c1978c0 2024-09-16T01:31:45.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.015+0000 7f8682d21640 1 -- 172.21.15.176:0/984535915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867c19c680 con 0x7f867c1978c0 2024-09-16T01:31:45.019 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.015+0000 7f8679ffb640 1 -- 172.21.15.176:0/984535915 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f8668018020 con 0x7f867c1978c0 2024-09-16T01:31:45.019 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.016+0000 7f8679ffb640 1 --2- 172.21.15.176:0/984535915 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8660039090 0x7f866003b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:45.019 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.016+0000 7f8679ffb640 1 -- 172.21.15.176:0/984535915 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f866804beb0 con 0x7f867c1978c0 2024-09-16T01:31:45.019 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.016+0000 7f8680abd640 1 --2- 172.21.15.176:0/984535915 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8660039090 0x7f866003b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:45.020 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.016+0000 7f8682d21640 1 -- 172.21.15.176:0/984535915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867c06ad10 con 0x7f867c1978c0 2024-09-16T01:31:45.020 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.017+0000 7f8680abd640 1 --2- 172.21.15.176:0/984535915 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8660039090 0x7f866003b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8670012c10 tx=0x7f86700323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:45.022 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.021+0000 7f8679ffb640 1 -- 172.21.15.176:0/984535915 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f866801f020 con 0x7f867c1978c0 2024-09-16T01:31:45.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:31:45.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: mon.a calling monitor election 2024-09-16T01:31:45.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:45.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: mon.c calling monitor election 2024-09-16T01:31:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T01:31:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: monmap epoch 2 2024-09-16T01:31:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: last_changed 2024-09-16T01:31:39.723501+0000 2024-09-16T01:31:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: created 2024-09-16T01:30:10.119520+0000 2024-09-16T01:31:45.175 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: min_mon_release 19 (squid) 2024-09-16T01:31:45.175 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: election_strategy: 1 2024-09-16T01:31:45.175 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: 0: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.a 2024-09-16T01:31:45.175 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.c 2024-09-16T01:31:45.175 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: fsmap 2024-09-16T01:31:45.175 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:31:45.175 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: mgrmap e13: x(active, since 37s) 2024-09-16T01:31:45.175 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: overall HEALTH_OK 2024-09-16T01:31:45.175 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:45.175 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:45.175 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:45.175 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:31:45.175 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:31:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: mon.a calling monitor election 2024-09-16T01:31:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: mon.c calling monitor election 2024-09-16T01:31:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T01:31:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: monmap epoch 2 2024-09-16T01:31:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: last_changed 2024-09-16T01:31:39.723501+0000 2024-09-16T01:31:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: created 2024-09-16T01:30:10.119520+0000 2024-09-16T01:31:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: min_mon_release 19 (squid) 2024-09-16T01:31:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: election_strategy: 1 2024-09-16T01:31:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: 0: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.a 2024-09-16T01:31:45.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.c 2024-09-16T01:31:45.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: fsmap 2024-09-16T01:31:45.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:31:45.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: mgrmap e13: x(active, since 37s) 2024-09-16T01:31:45.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: overall HEALTH_OK 2024-09-16T01:31:45.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:45.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:45.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:45.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:31:45.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:45.256 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.254+0000 7f8682d21640 1 -- 172.21.15.176:0/984535915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f867c19c9c0 con 0x7f867c1978c0 2024-09-16T01:31:45.256 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.255+0000 7f8679ffb640 1 -- 172.21.15.176:0/984535915 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f8668018310 con 0x7f867c1978c0 2024-09-16T01:31:45.256 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:31:45.257 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":2,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","modified":"2024-09-16T01:31:39.723501Z","created":"2024-09-16T01:30:10.119520Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:3300","nonce":0},{"type":"v1","addr":"172.21.15.139:6789","nonce":0}]},"addr":"172.21.15.139:6789/0","public_addr":"172.21.15.139:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T01:31:45.257 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 2 2024-09-16T01:31:45.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.257+0000 7f8682d21640 1 -- 172.21.15.176:0/984535915 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8660039090 msgr2=0x7f866003b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:45.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.257+0000 7f8682d21640 1 --2- 172.21.15.176:0/984535915 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8660039090 0x7f866003b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8670012c10 tx=0x7f86700323a0 comp rx=0 tx=0).stop 2024-09-16T01:31:45.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.257+0000 7f8682d21640 1 -- 172.21.15.176:0/984535915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f867c1978c0 msgr2=0x7f867c19bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:45.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.257+0000 7f8682d21640 1 --2- 172.21.15.176:0/984535915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f867c1978c0 0x7f867c19bc00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f866800b3d0 tx=0x7f866800b890 comp rx=0 tx=0).stop 2024-09-16T01:31:45.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.257+0000 7f8682d21640 1 -- 172.21.15.176:0/984535915 shutdown_connections 2024-09-16T01:31:45.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.257+0000 7f8682d21640 1 --2- 172.21.15.176:0/984535915 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8660039090 0x7f866003b550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:45.259 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.257+0000 7f8682d21640 1 --2- 172.21.15.176:0/984535915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f867c1978c0 0x7f867c19bc00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:45.259 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.257+0000 7f8682d21640 1 --2- 172.21.15.176:0/984535915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f866c003530 0x7f867c197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:45.259 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.257+0000 7f8682d21640 1 -- 172.21.15.176:0/984535915 >> 172.21.15.176:0/984535915 conn(0x7f867c0fc220 msgr2=0x7f867c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:45.259 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.257+0000 7f8682d21640 1 -- 172.21.15.176:0/984535915 shutdown_connections 2024-09-16T01:31:45.259 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:45.257+0000 7f8682d21640 1 -- 172.21.15.176:0/984535915 wait complete. 2024-09-16T01:31:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:45 smithi176 ceph-mon[35324]: Deploying daemon mon.b on smithi164 2024-09-16T01:31:45.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:45 smithi176 ceph-mon[35324]: from='client.? 172.21.15.176:0/984535915' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:31:45.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:45 smithi176 ceph-mon[35324]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:45.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:45 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:46.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:45 smithi139 ceph-mon[31175]: Deploying daemon mon.b on smithi164 2024-09-16T01:31:46.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:45 smithi139 ceph-mon[31175]: from='client.? 172.21.15.176:0/984535915' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:31:46.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:45 smithi139 ceph-mon[31175]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:46.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:45 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:46.853 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T01:31:46.853 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T01:31:47.104 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:31:47.456 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.454+0000 7f08127e3640 1 -- 172.21.15.176:0/3844340917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f080c106630 msgr2=0x7f080c106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:47.456 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.454+0000 7f08127e3640 1 --2- 172.21.15.176:0/3844340917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f080c106630 0x7f080c106a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009880 tx=0x7f07fc02a5d0 comp rx=0 tx=0).stop 2024-09-16T01:31:47.456 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.454+0000 7f08127e3640 1 -- 172.21.15.176:0/3844340917 shutdown_connections 2024-09-16T01:31:47.456 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.454+0000 7f08127e3640 1 --2- 172.21.15.176:0/3844340917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f080c106630 0x7f080c106a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:47.456 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.454+0000 7f08127e3640 1 -- 172.21.15.176:0/3844340917 >> 172.21.15.176:0/3844340917 conn(0x7f080c0fc220 msgr2=0x7f080c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:47.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.454+0000 7f08127e3640 1 -- 172.21.15.176:0/3844340917 shutdown_connections 2024-09-16T01:31:47.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.454+0000 7f08127e3640 1 -- 172.21.15.176:0/3844340917 wait complete. 2024-09-16T01:31:47.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.455+0000 7f08127e3640 1 Processor -- start 2024-09-16T01:31:47.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.455+0000 7f08127e3640 1 -- start start 2024-09-16T01:31:47.459 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.455+0000 7f08127e3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f080c106630 0x7f080c0fc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:47.459 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.455+0000 7f08127e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f080c0fcc10 0x7f080c0fd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:47.459 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.455+0000 7f08127e3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f080c10ac60 con 0x7f080c0fcc10 2024-09-16T01:31:47.459 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.455+0000 7f08127e3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f080c10aae0 con 0x7f080c106630 2024-09-16T01:31:47.459 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f080bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f080c106630 0x7f080c0fc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:47.459 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f080b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f080c0fcc10 0x7f080c0fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:47.459 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f080bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f080c106630 0x7f080c0fc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:45346/0 (socket says 172.21.15.176:45346) 2024-09-16T01:31:47.459 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f080bfff640 1 -- 172.21.15.176:0/4092407941 learned_addr learned my addr 172.21.15.176:0/4092407941 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-16T01:31:47.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f080bfff640 1 -- 172.21.15.176:0/4092407941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f080c106630 msgr2=0x7f080c0fc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-16T01:31:47.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f080bfff640 1 -- 172.21.15.176:0/4092407941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f080c106630 msgr2=0x7f080c0fc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T01:31:47.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f080bfff640 1 --2- 172.21.15.176:0/4092407941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f080c106630 0x7f080c0fc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T01:31:47.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f080bfff640 1 --2- 172.21.15.176:0/4092407941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f080c106630 0x7f080c0fc6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:31:47.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f080b7fe640 1 -- 172.21.15.176:0/4092407941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f080c106630 msgr2=0x7f080c0fc6d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:31:47.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f080b7fe640 1 --2- 172.21.15.176:0/4092407941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f080c106630 0x7f080c0fc6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:47.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f080b7fe640 1 -- 172.21.15.176:0/4092407941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c0fe5c0 con 0x7f080c0fcc10 2024-09-16T01:31:47.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f080b7fe640 1 --2- 172.21.15.176:0/4092407941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f080c0fcc10 0x7f080c0fd070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f080000b400 tx=0x7f080000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:47.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f08097fa640 1 -- 172.21.15.176:0/4092407941 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0800011630 con 0x7f080c0fcc10 2024-09-16T01:31:47.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f08127e3640 1 -- 172.21.15.176:0/4092407941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080c1a36b0 con 0x7f080c0fcc10 2024-09-16T01:31:47.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f08097fa640 1 -- 172.21.15.176:0/4092407941 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0800011cb0 con 0x7f080c0fcc10 2024-09-16T01:31:47.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f08097fa640 1 -- 172.21.15.176:0/4092407941 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0800010d80 con 0x7f080c0fcc10 2024-09-16T01:31:47.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.456+0000 7f08127e3640 1 -- 172.21.15.176:0/4092407941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080c1a3c40 con 0x7f080c0fcc10 2024-09-16T01:31:47.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.458+0000 7f08097fa640 1 -- 172.21.15.176:0/4092407941 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f0800018020 con 0x7f080c0fcc10 2024-09-16T01:31:47.461 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.458+0000 7f08127e3640 1 -- 172.21.15.176:0/4092407941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080c101ca0 con 0x7f080c0fcc10 2024-09-16T01:31:47.461 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.458+0000 7f08097fa640 1 --2- 172.21.15.176:0/4092407941 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f07f4039030 0x7f07f403b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:47.461 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.458+0000 7f08097fa640 1 -- 172.21.15.176:0/4092407941 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f080004bfc0 con 0x7f080c0fcc10 2024-09-16T01:31:47.461 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.458+0000 7f080bfff640 1 --2- 172.21.15.176:0/4092407941 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f07f4039030 0x7f07f403b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:47.461 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.458+0000 7f080bfff640 1 --2- 172.21.15.176:0/4092407941 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f07f4039030 0x7f07f403b4f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0096c0 tx=0x7f07fc005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:47.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.463+0000 7f08097fa640 1 -- 172.21.15.176:0/4092407941 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f080001f020 con 0x7f080c0fcc10 2024-09-16T01:31:47.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.692+0000 7f08127e3640 1 -- 172.21.15.176:0/4092407941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f080c1a3f50 con 0x7f080c0fcc10 2024-09-16T01:31:47.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.693+0000 7f08097fa640 1 -- 172.21.15.176:0/4092407941 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f0800018310 con 0x7f080c0fcc10 2024-09-16T01:31:47.694 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:31:47.695 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":2,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","modified":"2024-09-16T01:31:39.723501Z","created":"2024-09-16T01:30:10.119520Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:3300","nonce":0},{"type":"v1","addr":"172.21.15.139:6789","nonce":0}]},"addr":"172.21.15.139:6789/0","public_addr":"172.21.15.139:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T01:31:47.695 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 2 2024-09-16T01:31:47.696 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.695+0000 7f08127e3640 1 -- 172.21.15.176:0/4092407941 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f07f4039030 msgr2=0x7f07f403b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:47.696 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.695+0000 7f08127e3640 1 --2- 172.21.15.176:0/4092407941 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f07f4039030 0x7f07f403b4f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0096c0 tx=0x7f07fc005a40 comp rx=0 tx=0).stop 2024-09-16T01:31:47.697 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.695+0000 7f08127e3640 1 -- 172.21.15.176:0/4092407941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f080c0fcc10 msgr2=0x7f080c0fd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:47.697 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.695+0000 7f08127e3640 1 --2- 172.21.15.176:0/4092407941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f080c0fcc10 0x7f080c0fd070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f080000b400 tx=0x7f080000b8c0 comp rx=0 tx=0).stop 2024-09-16T01:31:47.697 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.695+0000 7f08127e3640 1 -- 172.21.15.176:0/4092407941 shutdown_connections 2024-09-16T01:31:47.697 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.695+0000 7f08127e3640 1 --2- 172.21.15.176:0/4092407941 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f07f4039030 0x7f07f403b4f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:47.697 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.695+0000 7f08127e3640 1 --2- 172.21.15.176:0/4092407941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f080c0fcc10 0x7f080c0fd070 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:47.697 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.695+0000 7f08127e3640 1 --2- 172.21.15.176:0/4092407941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f080c106630 0x7f080c0fc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:47.697 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.695+0000 7f08127e3640 1 -- 172.21.15.176:0/4092407941 >> 172.21.15.176:0/4092407941 conn(0x7f080c0fc220 msgr2=0x7f080c1900f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:47.697 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.695+0000 7f08127e3640 1 -- 172.21.15.176:0/4092407941 shutdown_connections 2024-09-16T01:31:47.697 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:47.695+0000 7f08127e3640 1 -- 172.21.15.176:0/4092407941 wait complete. 2024-09-16T01:31:48.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 podman[35103]: 2024-09-16 01:31:48.168985476 +0000 UTC m=+0.016268439 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:31:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:48 smithi139 ceph-mon[31175]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:48 smithi139 ceph-mon[31175]: from='client.? 172.21.15.176:0/4092407941' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:31:48.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:48 smithi176 ceph-mon[35324]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:48.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:48 smithi176 ceph-mon[35324]: from='client.? 172.21.15.176:0/4092407941' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:31:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 podman[35103]: 2024-09-16 01:31:48.418893219 +0000 UTC m=+0.266176168 container init 05e60772d0f771f6f268869d9f840cb4bdd749bab5418a5d5fb45e0aad263ff2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T01:31:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 podman[35103]: 2024-09-16 01:31:48.422138225 +0000 UTC m=+0.269421167 container start 05e60772d0f771f6f268869d9f840cb4bdd749bab5418a5d5fb45e0aad263ff2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T01:31:48.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T01:31:48.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-16T01:31:48.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: pidfile_write: ignore empty --pid-file 2024-09-16T01:31:48.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: load: jerasure load: lrc 2024-09-16T01:31:48.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: RocksDB version: 7.9.2 2024-09-16T01:31:48.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Git sha 0 2024-09-16T01:31:48.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-16T01:31:48.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: DB SUMMARY 2024-09-16T01:31:48.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: DB Session ID: EX3ZRNOWJZPJ37T1UANL 2024-09-16T01:31:48.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: CURRENT file: CURRENT 2024-09-16T01:31:48.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T01:31:48.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-16T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-16T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.error_if_exists: 0 2024-09-16T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.create_if_missing: 0 2024-09-16T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.paranoid_checks: 1 2024-09-16T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.env: 0x5555ee180940 2024-09-16T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.info_log: 0x5555eee8f6e0 2024-09-16T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T01:31:48.799 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.statistics: (nil) 2024-09-16T01:31:48.799 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.use_fsync: 0 2024-09-16T01:31:48.799 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_log_file_size: 0 2024-09-16T01:31:48.800 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T01:31:48.800 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T01:31:48.800 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T01:31:48.800 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T01:31:48.800 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.allow_fallocate: 1 2024-09-16T01:31:48.800 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T01:31:48.800 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T01:31:48.800 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.use_direct_reads: 0 2024-09-16T01:31:48.800 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T01:31:48.800 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T01:31:48.800 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.db_log_dir: 2024-09-16T01:31:48.800 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.wal_dir: 2024-09-16T01:31:48.800 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T01:31:48.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T01:31:48.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T01:31:48.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T01:31:48.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T01:31:48.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T01:31:48.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T01:31:48.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T01:31:48.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.write_buffer_manager: 0x5555eee9edc0 2024-09-16T01:31:48.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T01:31:48.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T01:31:48.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T01:31:48.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T01:31:48.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T01:31:48.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T01:31:48.802 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T01:31:48.802 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T01:31:48.802 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.unordered_write: 0 2024-09-16T01:31:48.802 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T01:31:48.802 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T01:31:48.802 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T01:31:48.802 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T01:31:48.802 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.row_cache: None 2024-09-16T01:31:48.802 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.wal_filter: None 2024-09-16T01:31:48.802 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T01:31:48.802 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T01:31:48.802 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.two_write_queues: 0 2024-09-16T01:31:48.802 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T01:31:48.802 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.wal_compression: 0 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.atomic_flush: 0 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.log_readahead_size: 0 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_background_jobs: 2 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_background_compactions: -1 2024-09-16T01:31:48.803 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_subcompactions: 1 2024-09-16T01:31:48.804 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T01:31:48.804 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T01:31:48.804 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T01:31:48.804 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T01:31:48.804 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T01:31:48.804 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T01:31:48.804 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T01:31:48.804 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T01:31:48.804 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_open_files: -1 2024-09-16T01:31:48.804 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T01:31:48.804 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T01:31:48.804 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T01:31:48.804 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T01:31:48.804 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_background_flushes: -1 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Compression algorithms supported: 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: kZSTD supported: 0 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: kXpressCompression supported: 0 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: kBZip2Compression supported: 0 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: kLZ4Compression supported: 1 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: kZlibCompression supported: 1 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: kSnappyCompression supported: 1 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T01:31:48.805 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.merge_operator: 2024-09-16T01:31:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compaction_filter: None 2024-09-16T01:31:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T01:31:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T01:31:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T01:31:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T01:31:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5555eee8f320) 2024-09-16T01:31:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout: cache_index_and_filter_blocks: 1 2024-09-16T01:31:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T01:31:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T01:31:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout: pin_top_level_index_and_filter: 1 2024-09-16T01:31:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout: index_type: 0 2024-09-16T01:31:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout: data_block_index_type: 0 2024-09-16T01:31:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout: index_shortening: 1 2024-09-16T01:31:48.806 INFO:journalctl@ceph.mon.b.smithi164.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T01:31:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout: checksum: 4 2024-09-16T01:31:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout: no_block_cache: 0 2024-09-16T01:31:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_cache: 0x5555eee831f0 2024-09-16T01:31:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_cache_name: BinnedLRUCache 2024-09-16T01:31:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_cache_options: 2024-09-16T01:31:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout: capacity : 536870912 2024-09-16T01:31:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout: num_shard_bits : 4 2024-09-16T01:31:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout: strict_capacity_limit : 0 2024-09-16T01:31:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout: high_pri_pool_ratio: 0.000 2024-09-16T01:31:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_cache_compressed: (nil) 2024-09-16T01:31:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout: persistent_cache: (nil) 2024-09-16T01:31:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_size: 4096 2024-09-16T01:31:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_size_deviation: 10 2024-09-16T01:31:48.807 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_restart_interval: 16 2024-09-16T01:31:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout: index_block_restart_interval: 1 2024-09-16T01:31:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout: metadata_block_size: 4096 2024-09-16T01:31:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout: partition_filters: 0 2024-09-16T01:31:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout: use_delta_encoding: 1 2024-09-16T01:31:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout: filter_policy: bloomfilter 2024-09-16T01:31:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout: whole_key_filtering: 1 2024-09-16T01:31:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout: verify_compression: 0 2024-09-16T01:31:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout: read_amp_bytes_per_bit: 0 2024-09-16T01:31:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout: format_version: 5 2024-09-16T01:31:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout: enable_index_compression: 1 2024-09-16T01:31:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_align: 0 2024-09-16T01:31:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout: max_auto_readahead_size: 262144 2024-09-16T01:31:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout: prepopulate_block_cache: 0 2024-09-16T01:31:48.808 INFO:journalctl@ceph.mon.b.smithi164.stdout: initial_auto_readahead_size: 8192 2024-09-16T01:31:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T01:31:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T01:31:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T01:31:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compression: NoCompression 2024-09-16T01:31:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T01:31:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T01:31:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T01:31:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.num_levels: 7 2024-09-16T01:31:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T01:31:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T01:31:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T01:31:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T01:31:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T01:31:48.809 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T01:31:48.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T01:31:48.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T01:31:48.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T01:31:48.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T01:31:48.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T01:31:48.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T01:31:48.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T01:31:48.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T01:31:48.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T01:31:48.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T01:31:48.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T01:31:48.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T01:31:48.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T01:31:48.810 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T01:31:48.811 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T01:31:48.811 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T01:31:48.811 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T01:31:48.811 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T01:31:48.811 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T01:31:48.811 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T01:31:48.811 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T01:31:48.811 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T01:31:48.811 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T01:31:48.811 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T01:31:48.811 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T01:31:48.811 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T01:31:48.811 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T01:31:48.811 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T01:31:48.812 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T01:31:48.812 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T01:31:48.812 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T01:31:48.812 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T01:31:48.812 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T01:31:48.812 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T01:31:48.812 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T01:31:48.812 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T01:31:48.812 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T01:31:48.812 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T01:31:48.812 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T01:31:48.812 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T01:31:48.812 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T01:31:48.812 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T01:31:48.813 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T01:31:48.813 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T01:31:48.813 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T01:31:48.813 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T01:31:48.813 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T01:31:48.813 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T01:31:48.813 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.inplace_update_support: 0 2024-09-16T01:31:48.813 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T01:31:48.813 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T01:31:48.813 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T01:31:48.813 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T01:31:48.813 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.bloom_locality: 0 2024-09-16T01:31:48.813 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.max_successive_merges: 0 2024-09-16T01:31:48.814 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T01:31:48.814 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T01:31:48.814 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T01:31:48.814 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T01:31:48.814 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.ttl: 2592000 2024-09-16T01:31:48.814 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T01:31:48.814 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T01:31:48.814 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T01:31:48.814 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.enable_blob_files: false 2024-09-16T01:31:48.814 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.min_blob_size: 0 2024-09-16T01:31:48.814 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T01:31:48.814 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T01:31:48.815 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T01:31:48.815 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T01:31:48.815 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T01:31:48.815 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T01:31:48.815 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T01:31:48.815 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T01:31:48.815 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T01:31:48.815 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T01:31:48.815 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: eebb822f-589a-4a4d-8735-5c14b222c644 2024-09-16T01:31:48.815 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726450308449047, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T01:31:48.815 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T01:31:48.815 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726450308449509, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726450308, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "eebb822f-589a-4a4d-8735-5c14b222c644", "db_session_id": "EX3ZRNOWJZPJ37T1UANL", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T01:31:48.815 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726450308449602, "job": 1, "event": "recovery_finished"} 2024-09-16T01:31:48.815 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T01:31:48.816 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T01:31:48.816 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5555eeeb0e00 2024-09-16T01:31:48.816 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: DB pointer 0x5555eefac000 2024-09-16T01:31:48.816 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T01:31:48.816 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T01:31:48.816 INFO:journalctl@ceph.mon.b.smithi164.stdout: ** DB Stats ** 2024-09-16T01:31:48.816 INFO:journalctl@ceph.mon.b.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T01:31:48.816 INFO:journalctl@ceph.mon.b.smithi164.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T01:31:48.816 INFO:journalctl@ceph.mon.b.smithi164.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T01:31:48.816 INFO:journalctl@ceph.mon.b.smithi164.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T01:31:48.816 INFO:journalctl@ceph.mon.b.smithi164.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T01:31:48.816 INFO:journalctl@ceph.mon.b.smithi164.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T01:31:48.816 INFO:journalctl@ceph.mon.b.smithi164.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T01:31:48.816 INFO:journalctl@ceph.mon.b.smithi164.stdout: 2024-09-16T01:31:48.817 INFO:journalctl@ceph.mon.b.smithi164.stdout: ** Compaction Stats [default] ** 2024-09-16T01:31:48.817 INFO:journalctl@ceph.mon.b.smithi164.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T01:31:48.817 INFO:journalctl@ceph.mon.b.smithi164.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T01:31:48.817 INFO:journalctl@ceph.mon.b.smithi164.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T01:31:48.817 INFO:journalctl@ceph.mon.b.smithi164.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T01:31:48.817 INFO:journalctl@ceph.mon.b.smithi164.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T01:31:48.817 INFO:journalctl@ceph.mon.b.smithi164.stdout: 2024-09-16T01:31:48.817 INFO:journalctl@ceph.mon.b.smithi164.stdout: ** Compaction Stats [default] ** 2024-09-16T01:31:48.817 INFO:journalctl@ceph.mon.b.smithi164.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T01:31:48.817 INFO:journalctl@ceph.mon.b.smithi164.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T01:31:48.817 INFO:journalctl@ceph.mon.b.smithi164.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T01:31:48.817 INFO:journalctl@ceph.mon.b.smithi164.stdout: 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: Block cache BinnedLRUCache@0x5555eee831f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: 2024-09-16T01:31:48.818 INFO:journalctl@ceph.mon.b.smithi164.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T01:31:48.819 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-16T01:31:48.819 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: using public_addr v2:172.21.15.164:0/0 -> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-09-16T01:31:48.819 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: starting mon.b rank -1 at public addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] at bind addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:48.819 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b@-1(???) e0 preinit fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:48.819 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b@-1(synchronizing).mds e1 new map 2024-09-16T01:31:48.819 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-16T01:31:48.819 INFO:journalctl@ceph.mon.b.smithi164.stdout: e1 2024-09-16T01:31:48.819 INFO:journalctl@ceph.mon.b.smithi164.stdout: btime 2024-09-16T01:30:17:770097+0000 2024-09-16T01:31:48.819 INFO:journalctl@ceph.mon.b.smithi164.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T01:31:48.819 INFO:journalctl@ceph.mon.b.smithi164.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T01:31:48.819 INFO:journalctl@ceph.mon.b.smithi164.stdout: legacy client fscid: -1 2024-09-16T01:31:48.819 INFO:journalctl@ceph.mon.b.smithi164.stdout: 2024-09-16T01:31:48.819 INFO:journalctl@ceph.mon.b.smithi164.stdout: No filesystems configured 2024-09-16T01:31:48.820 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T01:31:48.820 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T01:31:48.820 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T01:31:48.820 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T01:31:48.820 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T01:31:48.820 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T01:31:48.820 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T01:31:48.820 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:31:48.820 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:31:48.820 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T01:31:48.820 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: Deploying cephadm binary to smithi176 2024-09-16T01:31:48.820 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:48.820 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.820 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.821 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.821 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.821 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:31:48.821 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:48.821 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:31:48.821 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-16T01:31:48.821 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: Updating smithi164:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:48.821 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:31:48.821 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: Updating smithi164:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:31:48.821 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.821 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.821 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.821 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:48.821 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.822 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:31:48.822 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.822 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.822 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: Added host smithi176 2024-09-16T01:31:48.822 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.822 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:31:48.822 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:48.822 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1728058196' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T01:31:48.822 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:48.822 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1728058196' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T01:31:48.822 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:31:48.822 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.822 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.822 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi176", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: Updating smithi176:/etc/ceph/ceph.conf 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: Updating smithi176:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: Updating smithi176:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:48.823 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:31:48.824 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.824 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:31:48.824 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:48.824 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi139:172.21.15.139=a;smithi164:172.21.15.164=b;smithi176:172.21.15.176=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:31:48.824 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: Saving service mon spec with placement smithi139:172.21.15.139=a;smithi164:172.21.15.164=b;smithi176:172.21.15.176=c;count:3 2024-09-16T01:31:48.824 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: Deploying daemon mon.c on smithi176 2024-09-16T01:31:48.824 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:48.824 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:48.824 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='client.? 172.21.15.176:0/3803509298' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:31:48.824 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:31:48.824 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.a calling monitor election 2024-09-16T01:31:48.824 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:48.824 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:48.825 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:48.825 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:48.825 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.c calling monitor election 2024-09-16T01:31:48.825 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:48.825 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:48.825 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:48.825 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:48.825 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T01:31:48.825 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: monmap epoch 2 2024-09-16T01:31:48.825 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:48.825 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: last_changed 2024-09-16T01:31:39.723501+0000 2024-09-16T01:31:48.825 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: created 2024-09-16T01:30:10.119520+0000 2024-09-16T01:31:48.825 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: min_mon_release 19 (squid) 2024-09-16T01:31:48.825 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: election_strategy: 1 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: 0: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.a 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.c 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: fsmap 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mgrmap e13: x(active, since 37s) 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: overall HEALTH_OK 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: Deploying daemon mon.b on smithi164 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='client.? 172.21.15.176:0/984535915' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:48.826 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:48.827 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:48.827 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: from='client.? 172.21.15.176:0/4092407941' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:31:48.827 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 ceph-mon[35121]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T01:31:48.827 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 bash[35103]: 05e60772d0f771f6f268869d9f840cb4bdd749bab5418a5d5fb45e0aad263ff2 2024-09-16T01:31:48.827 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:48 smithi164 systemd[1]: Started Ceph mon.b for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:31:49.276 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T01:31:49.276 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T01:31:49.527 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:31:53.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.545+0000 7f533e2b4640 1 -- 172.21.15.176:0/3966447294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5320003530 msgr2=0x7f53200059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:53.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.545+0000 7f533e2b4640 1 --2- 172.21.15.176:0/3966447294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5320003530 0x7f53200059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f53381016c0 tx=0x7f532802c3f0 comp rx=0 tx=0).stop 2024-09-16T01:31:53.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.545+0000 7f533e2b4640 1 -- 172.21.15.176:0/3966447294 shutdown_connections 2024-09-16T01:31:53.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.545+0000 7f533e2b4640 1 --2- 172.21.15.176:0/3966447294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5320003530 0x7f53200059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:53.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.545+0000 7f533e2b4640 1 -- 172.21.15.176:0/3966447294 >> 172.21.15.176:0/3966447294 conn(0x7f53380fc280 msgr2=0x7f53380fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:53.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.545+0000 7f533e2b4640 1 -- 172.21.15.176:0/3966447294 shutdown_connections 2024-09-16T01:31:53.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.545+0000 7f533e2b4640 1 -- 172.21.15.176:0/3966447294 wait complete. 2024-09-16T01:31:53.549 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.546+0000 7f533e2b4640 1 Processor -- start 2024-09-16T01:31:53.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.546+0000 7f533e2b4640 1 -- start start 2024-09-16T01:31:53.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f533e2b4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5320003530 0x7f5338197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:53.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f533e2b4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5338197880 0x7f533819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:53.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f533e2b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f533819c150 0x7f533819c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:53.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f533e2b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5338108690 con 0x7f533819c150 2024-09-16T01:31:53.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f533e2b4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5338108510 con 0x7f5320003530 2024-09-16T01:31:53.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f533e2b4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5338108810 con 0x7f5338197880 2024-09-16T01:31:53.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f53377fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5338197880 0x7f533819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:53.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f53377fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5338197880 0x7f533819bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:51430/0 (socket says 172.21.15.176:51430) 2024-09-16T01:31:53.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f53377fe640 1 -- 172.21.15.176:0/3237643216 learned_addr learned my addr 172.21.15.176:0/3237643216 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-16T01:31:53.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f533c851640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f533819c150 0x7f533819c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:53.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f5337fff640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5320003530 0x7f5338197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:53.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f53377fe640 1 -- 172.21.15.176:0/3237643216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5338197880 msgr2=0x7f533819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-16T01:31:53.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f53377fe640 1 -- 172.21.15.176:0/3237643216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5338197880 msgr2=0x7f533819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T01:31:53.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f53377fe640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5338197880 0x7f533819bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T01:31:53.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f5337fff640 1 -- 172.21.15.176:0/3237643216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5320003530 msgr2=0x7f5338197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-16T01:31:53.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f5337fff640 1 -- 172.21.15.176:0/3237643216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5320003530 msgr2=0x7f5338197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T01:31:53.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f5337fff640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5320003530 0x7f5338197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T01:31:53.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f53377fe640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5338197880 0x7f533819bc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:31:53.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f5337fff640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5320003530 0x7f5338197340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:31:53.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f533c851640 1 -- 172.21.15.176:0/3237643216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5338197880 msgr2=0x7f533819bc10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:31:53.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f533c851640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5338197880 0x7f533819bc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:53.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f533c851640 1 -- 172.21.15.176:0/3237643216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5320003530 msgr2=0x7f5338197340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:31:53.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f533c851640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5320003530 0x7f5338197340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:53.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.547+0000 7f533c851640 1 -- 172.21.15.176:0/3237643216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53381a3ec0 con 0x7f533819c150 2024-09-16T01:31:53.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.548+0000 7f533c851640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f533819c150 0x7f533819c600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f533000ff40 tx=0x7f533000ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:53.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.548+0000 7f53357fa640 1 -- 172.21.15.176:0/3237643216 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533000d0d0 con 0x7f533819c150 2024-09-16T01:31:53.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.548+0000 7f53357fa640 1 -- 172.21.15.176:0/3237643216 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533000bb80 con 0x7f533819c150 2024-09-16T01:31:53.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.548+0000 7f533e2b4640 1 -- 172.21.15.176:0/3237643216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5338102140 con 0x7f533819c150 2024-09-16T01:31:53.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.548+0000 7f53357fa640 1 -- 172.21.15.176:0/3237643216 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330018870 con 0x7f533819c150 2024-09-16T01:31:53.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.548+0000 7f533e2b4640 1 -- 172.21.15.176:0/3237643216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5338102700 con 0x7f533819c150 2024-09-16T01:31:53.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.549+0000 7f53357fa640 1 -- 172.21.15.176:0/3237643216 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f5330018a50 con 0x7f533819c150 2024-09-16T01:31:53.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.549+0000 7f53357fa640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5324039730 0x7f532403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:53.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.549+0000 7f533e2b4640 1 -- 172.21.15.176:0/3237643216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53381097c0 con 0x7f533819c150 2024-09-16T01:31:53.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.549+0000 7f53357fa640 1 -- 172.21.15.176:0/3237643216 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f53300500b0 con 0x7f533819c150 2024-09-16T01:31:53.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.549+0000 7f5337fff640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5324039730 0x7f532403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:53.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.550+0000 7f5337fff640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5324039730 0x7f532403bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5328014cc0 tx=0x7f5328005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:53.555 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.554+0000 7f53357fa640 1 -- 172.21.15.176:0/3237643216 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5330020de0 con 0x7f533819c150 2024-09-16T01:31:53.798 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.797+0000 7f533e2b4640 1 -- 172.21.15.176:0/3237643216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5338102960 con 0x7f533819c150 2024-09-16T01:31:53.798 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:31:53.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:53.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: mon.a calling monitor election 2024-09-16T01:31:53.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: mon.c calling monitor election 2024-09-16T01:31:53.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:53.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:53.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:53.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:53.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:53.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:53.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:53.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:53.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T01:31:53.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: monmap epoch 3 2024-09-16T01:31:53.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:53.800 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: last_changed 2024-09-16T01:31:48.473391+0000 2024-09-16T01:31:53.800 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: created 2024-09-16T01:30:10.119520+0000 2024-09-16T01:31:53.800 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: min_mon_release 19 (squid) 2024-09-16T01:31:53.800 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: election_strategy: 1 2024-09-16T01:31:53.800 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: 0: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.a 2024-09-16T01:31:53.800 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.c 2024-09-16T01:31:53.800 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-09-16T01:31:53.800 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: fsmap 2024-09-16T01:31:53.800 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:31:53.800 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: mgrmap e13: x(active, since 46s) 2024-09-16T01:31:53.800 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: overall HEALTH_OK 2024-09-16T01:31:53.801 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:53.801 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:53.801 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:53.801 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:53.801 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.797+0000 7f53357fa640 1 -- 172.21.15.176:0/3237643216 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f533001b120 con 0x7f533819c150 2024-09-16T01:31:53.801 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:31:53.801 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":3,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","modified":"2024-09-16T01:31:48.473391Z","created":"2024-09-16T01:30:10.119520Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:3300","nonce":0},{"type":"v1","addr":"172.21.15.139:6789","nonce":0}]},"addr":"172.21.15.139:6789/0","public_addr":"172.21.15.139:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T01:31:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 3 2024-09-16T01:31:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.800+0000 7f533e2b4640 1 -- 172.21.15.176:0/3237643216 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5324039730 msgr2=0x7f532403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.800+0000 7f533e2b4640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5324039730 0x7f532403bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5328014cc0 tx=0x7f5328005f60 comp rx=0 tx=0).stop 2024-09-16T01:31:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.800+0000 7f533e2b4640 1 -- 172.21.15.176:0/3237643216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f533819c150 msgr2=0x7f533819c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.800+0000 7f533e2b4640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f533819c150 0x7f533819c600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f533000ff40 tx=0x7f533000ff70 comp rx=0 tx=0).stop 2024-09-16T01:31:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.800+0000 7f533e2b4640 1 -- 172.21.15.176:0/3237643216 shutdown_connections 2024-09-16T01:31:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.800+0000 7f533e2b4640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5324039730 0x7f532403bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.800+0000 7f533e2b4640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f533819c150 0x7f533819c600 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.800+0000 7f533e2b4640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5338197880 0x7f533819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.800+0000 7f533e2b4640 1 --2- 172.21.15.176:0/3237643216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5320003530 0x7f5338197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.800+0000 7f533e2b4640 1 -- 172.21.15.176:0/3237643216 >> 172.21.15.176:0/3237643216 conn(0x7f53380fc280 msgr2=0x7f533806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.800+0000 7f533e2b4640 1 -- 172.21.15.176:0/3237643216 shutdown_connections 2024-09-16T01:31:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:53.800+0000 7f533e2b4640 1 -- 172.21.15.176:0/3237643216 wait complete. 2024-09-16T01:31:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:31:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: mon.a calling monitor election 2024-09-16T01:31:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: mon.c calling monitor election 2024-09-16T01:31:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:53.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:53.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:53.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:53.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:53.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T01:31:53.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: monmap epoch 3 2024-09-16T01:31:53.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:53.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: last_changed 2024-09-16T01:31:48.473391+0000 2024-09-16T01:31:53.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: created 2024-09-16T01:30:10.119520+0000 2024-09-16T01:31:53.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: min_mon_release 19 (squid) 2024-09-16T01:31:53.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: election_strategy: 1 2024-09-16T01:31:53.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: 0: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.a 2024-09-16T01:31:53.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.c 2024-09-16T01:31:53.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-09-16T01:31:53.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: fsmap 2024-09-16T01:31:53.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:31:53.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: mgrmap e13: x(active, since 46s) 2024-09-16T01:31:53.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: overall HEALTH_OK 2024-09-16T01:31:53.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:53.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:53.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:53.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:54.353 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T01:31:54.353 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T01:31:54.614 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:31:54.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:54 smithi139 ceph-mon[31175]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:54.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:54 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:31:54.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:54 smithi139 ceph-mon[31175]: from='client.? 172.21.15.176:0/3237643216' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:31:54.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:54 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:54.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:54 smithi176 ceph-mon[35324]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:54.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:54 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:31:54.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:54 smithi176 ceph-mon[35324]: from='client.? 172.21.15.176:0/3237643216' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:31:54.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:54 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:55.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.056+0000 7f503a422640 1 -- 172.21.15.139:0/353682908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5034107660 msgr2=0x7f5034107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:55.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.056+0000 7f503a422640 1 --2- 172.21.15.139:0/353682908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5034107660 0x7f5034107a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5024009880 tx=0x7f502402a5d0 comp rx=0 tx=0).stop 2024-09-16T01:31:55.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.056+0000 7f503a422640 1 -- 172.21.15.139:0/353682908 shutdown_connections 2024-09-16T01:31:55.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.056+0000 7f503a422640 1 --2- 172.21.15.139:0/353682908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5034107660 0x7f5034107a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:55.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.056+0000 7f503a422640 1 -- 172.21.15.139:0/353682908 >> 172.21.15.139:0/353682908 conn(0x7f50340fd2b0 msgr2=0x7f50340ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:55.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.056+0000 7f503a422640 1 -- 172.21.15.139:0/353682908 shutdown_connections 2024-09-16T01:31:55.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.056+0000 7f503a422640 1 -- 172.21.15.139:0/353682908 wait complete. 2024-09-16T01:31:55.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.057+0000 7f503a422640 1 Processor -- start 2024-09-16T01:31:55.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.057+0000 7f503a422640 1 -- start start 2024-09-16T01:31:55.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.057+0000 7f503a422640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5034107660 0x7f50341972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:55.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.057+0000 7f503a422640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5034197800 0x7f503419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:55.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.057+0000 7f503a422640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f503419c0d0 0x7f503419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:55.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.057+0000 7f503a422640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f503410bc70 con 0x7f5034197800 2024-09-16T01:31:55.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.057+0000 7f503a422640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f503410baf0 con 0x7f503419c0d0 2024-09-16T01:31:55.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.057+0000 7f503a422640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f503410bdf0 con 0x7f5034107660 2024-09-16T01:31:55.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5034197800 0x7f503419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:55.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50389bf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f503419c0d0 0x7f503419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:55.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5034197800 0x7f503419bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56930/0 (socket says 172.21.15.139:56930) 2024-09-16T01:31:55.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50337fe640 1 -- 172.21.15.139:0/162654253 learned_addr learned my addr 172.21.15.139:0/162654253 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:31:55.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f5033fff640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5034107660 0x7f50341972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:55.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50389bf640 1 -- 172.21.15.139:0/162654253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f503419c0d0 msgr2=0x7f503419c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-16T01:31:55.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50389bf640 1 -- 172.21.15.139:0/162654253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f503419c0d0 msgr2=0x7f503419c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T01:31:55.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50389bf640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f503419c0d0 0x7f503419c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T01:31:55.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50389bf640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f503419c0d0 0x7f503419c580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:31:55.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50337fe640 1 -- 172.21.15.139:0/162654253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5034107660 msgr2=0x7f50341972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:55.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50337fe640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5034107660 0x7f50341972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:55.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50337fe640 1 -- 172.21.15.139:0/162654253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f503419c0d0 msgr2=0x7f503419c580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:31:55.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50337fe640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f503419c0d0 0x7f503419c580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:55.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50337fe640 1 -- 172.21.15.139:0/162654253 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5034074420 con 0x7f5034197800 2024-09-16T01:31:55.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50337fe640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5034197800 0x7f503419bb90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f502800b520 tx=0x7f502800b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:55.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50317fa640 1 -- 172.21.15.139:0/162654253 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028004230 con 0x7f5034197800 2024-09-16T01:31:55.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50317fa640 1 -- 172.21.15.139:0/162654253 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50280043d0 con 0x7f5034197800 2024-09-16T01:31:55.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f503a422640 1 -- 172.21.15.139:0/162654253 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5034074710 con 0x7f5034197800 2024-09-16T01:31:55.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f50317fa640 1 -- 172.21.15.139:0/162654253 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028003d60 con 0x7f5034197800 2024-09-16T01:31:55.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.058+0000 7f503a422640 1 -- 172.21.15.139:0/162654253 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5034074cd0 con 0x7f5034197800 2024-09-16T01:31:55.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.059+0000 7f50317fa640 1 -- 172.21.15.139:0/162654253 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f502800e070 con 0x7f5034197800 2024-09-16T01:31:55.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.059+0000 7f50317fa640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f501c039730 0x7f501c03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:55.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.059+0000 7f50317fa640 1 -- 172.21.15.139:0/162654253 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f502804bd80 con 0x7f5034197800 2024-09-16T01:31:55.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.060+0000 7f5033fff640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f501c039730 0x7f501c03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:55.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.060+0000 7f503a422640 1 -- 172.21.15.139:0/162654253 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5034107ac0 con 0x7f5034197800 2024-09-16T01:31:55.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.060+0000 7f5033fff640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f501c039730 0x7f501c03bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5024000c00 tx=0x7f5024005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:55.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.064+0000 7f50317fa640 1 -- 172.21.15.139:0/162654253 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5028014020 con 0x7f5034197800 2024-09-16T01:31:55.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.243+0000 7f503a422640 1 -- 172.21.15.139:0/162654253 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f5034074f90 con 0x7f5034197800 2024-09-16T01:31:55.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.243+0000 7f50317fa640 1 -- 172.21.15.139:0/162654253 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f5028020350 con 0x7f5034197800 2024-09-16T01:31:55.245 INFO:teuthology.orchestra.run.smithi139.stdout:# minimal ceph.conf for 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:55.246 INFO:teuthology.orchestra.run.smithi139.stdout:[global] 2024-09-16T01:31:55.246 INFO:teuthology.orchestra.run.smithi139.stdout: fsid = 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:55.246 INFO:teuthology.orchestra.run.smithi139.stdout: mon_host = [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] 2024-09-16T01:31:55.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.247+0000 7f503a422640 1 -- 172.21.15.139:0/162654253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f501c039730 msgr2=0x7f501c03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:55.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.247+0000 7f503a422640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f501c039730 0x7f501c03bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5024000c00 tx=0x7f5024005a20 comp rx=0 tx=0).stop 2024-09-16T01:31:55.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.247+0000 7f503a422640 1 -- 172.21.15.139:0/162654253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5034197800 msgr2=0x7f503419bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:55.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.247+0000 7f503a422640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5034197800 0x7f503419bb90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f502800b520 tx=0x7f502800b9e0 comp rx=0 tx=0).stop 2024-09-16T01:31:55.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.247+0000 7f503a422640 1 -- 172.21.15.139:0/162654253 shutdown_connections 2024-09-16T01:31:55.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.247+0000 7f503a422640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f501c039730 0x7f501c03bbf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:55.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.247+0000 7f503a422640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f503419c0d0 0x7f503419c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:55.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.247+0000 7f503a422640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5034197800 0x7f503419bb90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:55.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.247+0000 7f503a422640 1 --2- 172.21.15.139:0/162654253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5034107660 0x7f50341972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:55.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.247+0000 7f503a422640 1 -- 172.21.15.139:0/162654253 >> 172.21.15.139:0/162654253 conn(0x7f50340fd2b0 msgr2=0x7f50340fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:55.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.247+0000 7f503a422640 1 -- 172.21.15.139:0/162654253 shutdown_connections 2024-09-16T01:31:55.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:31:55.247+0000 7f503a422640 1 -- 172.21.15.139:0/162654253 wait complete. 2024-09-16T01:31:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:31:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: mon.a calling monitor election 2024-09-16T01:31:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: mon.c calling monitor election 2024-09-16T01:31:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:31:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T01:31:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: monmap epoch 3 2024-09-16T01:31:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: last_changed 2024-09-16T01:31:48.473391+0000 2024-09-16T01:31:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: created 2024-09-16T01:30:10.119520+0000 2024-09-16T01:31:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: min_mon_release 19 (squid) 2024-09-16T01:31:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: election_strategy: 1 2024-09-16T01:31:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: 0: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.a 2024-09-16T01:31:55.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.c 2024-09-16T01:31:55.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-09-16T01:31:55.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: fsmap 2024-09-16T01:31:55.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:31:55.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: mgrmap e13: x(active, since 46s) 2024-09-16T01:31:55.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: overall HEALTH_OK 2024-09-16T01:31:55.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:55.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:55.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:55.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:55.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:55.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:31:55.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='client.? 172.21.15.176:0/3237643216' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T01:31:55.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:56.030 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T01:31:56.030 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:31:56.031 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T01:31:56.061 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:31:56.061 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:31:56.127 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T01:31:56.127 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T01:31:56.156 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T01:31:56.156 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:31:56.221 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-16T01:31:56.221 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T01:31:56.249 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-16T01:31:56.249 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:31:56.311 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: mon.b calling monitor election 2024-09-16T01:31:56.312 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-16T01:31:56.312 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-16T01:31:56.312 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: Updating smithi176:/etc/ceph/ceph.conf 2024-09-16T01:31:56.312 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:56.312 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: Updating smithi139:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:56.312 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: Updating smithi164:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:56.312 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: Updating smithi176:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:56.312 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: mon.b calling monitor election 2024-09-16T01:31:56.312 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: mon.c calling monitor election 2024-09-16T01:31:56.312 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: mon.a calling monitor election 2024-09-16T01:31:56.312 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T01:31:56.312 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: monmap epoch 3 2024-09-16T01:31:56.313 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:56.313 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: last_changed 2024-09-16T01:31:48.473391+0000 2024-09-16T01:31:56.313 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: created 2024-09-16T01:30:10.119520+0000 2024-09-16T01:31:56.313 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: min_mon_release 19 (squid) 2024-09-16T01:31:56.313 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: election_strategy: 1 2024-09-16T01:31:56.313 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: 0: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.a 2024-09-16T01:31:56.313 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.c 2024-09-16T01:31:56.313 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-09-16T01:31:56.313 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: fsmap 2024-09-16T01:31:56.313 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:31:56.313 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: mgrmap e13: x(active, since 48s) 2024-09-16T01:31:56.313 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: overall HEALTH_OK 2024-09-16T01:31:56.313 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.313 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.314 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.314 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.314 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.314 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.314 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.314 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.314 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.314 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.314 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.314 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:31:56.314 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:31:56.314 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:56.315 INFO:tasks.cephadm:Adding mgr.x on smithi139 2024-09-16T01:31:56.315 INFO:tasks.cephadm:Adding mgr.y on smithi164 2024-09-16T01:31:56.315 INFO:tasks.cephadm:Adding mgr.z on smithi176 2024-09-16T01:31:56.316 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi139=x;smithi164=y;smithi176=z' 2024-09-16T01:31:56.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: mon.b calling monitor election 2024-09-16T01:31:56.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-16T01:31:56.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-16T01:31:56.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: Updating smithi176:/etc/ceph/ceph.conf 2024-09-16T01:31:56.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:56.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: Updating smithi139:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:56.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: Updating smithi164:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:56.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: Updating smithi176:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:56.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: mon.b calling monitor election 2024-09-16T01:31:56.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: mon.c calling monitor election 2024-09-16T01:31:56.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: mon.a calling monitor election 2024-09-16T01:31:56.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T01:31:56.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: monmap epoch 3 2024-09-16T01:31:56.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:56.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: last_changed 2024-09-16T01:31:48.473391+0000 2024-09-16T01:31:56.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: created 2024-09-16T01:30:10.119520+0000 2024-09-16T01:31:56.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: min_mon_release 19 (squid) 2024-09-16T01:31:56.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: election_strategy: 1 2024-09-16T01:31:56.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: 0: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.a 2024-09-16T01:31:56.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.c 2024-09-16T01:31:56.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-09-16T01:31:56.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: fsmap 2024-09-16T01:31:56.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:31:56.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: mgrmap e13: x(active, since 48s) 2024-09-16T01:31:56.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: overall HEALTH_OK 2024-09-16T01:31:56.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.440 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.440 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:31:56.440 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:31:56.440 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:56.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: mon.b calling monitor election 2024-09-16T01:31:56.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-16T01:31:56.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-16T01:31:56.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: Updating smithi176:/etc/ceph/ceph.conf 2024-09-16T01:31:56.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:56.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: Updating smithi139:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:56.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: Updating smithi164:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:56.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: Updating smithi176:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:31:56.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: mon.b calling monitor election 2024-09-16T01:31:56.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: mon.c calling monitor election 2024-09-16T01:31:56.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: mon.a calling monitor election 2024-09-16T01:31:56.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T01:31:56.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: monmap epoch 3 2024-09-16T01:31:56.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:31:56.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: last_changed 2024-09-16T01:31:48.473391+0000 2024-09-16T01:31:56.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: created 2024-09-16T01:30:10.119520+0000 2024-09-16T01:31:56.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: min_mon_release 19 (squid) 2024-09-16T01:31:56.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: election_strategy: 1 2024-09-16T01:31:56.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: 0: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.a 2024-09-16T01:31:56.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.c 2024-09-16T01:31:56.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: fsmap 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: mgrmap e13: x(active, since 48s) 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: overall HEALTH_OK 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:56.548 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:31:56.548 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:31:56.548 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:56.592 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:31:57.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.003+0000 7f9b671e2640 1 -- 172.21.15.176:0/1987667255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b601005f0 msgr2=0x7f9b601009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:57.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.003+0000 7f9b671e2640 1 --2- 172.21.15.176:0/1987667255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b601005f0 0x7f9b601009f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009880 tx=0x7f9b5c02a5d0 comp rx=0 tx=0).stop 2024-09-16T01:31:57.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.003+0000 7f9b671e2640 1 -- 172.21.15.176:0/1987667255 shutdown_connections 2024-09-16T01:31:57.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.003+0000 7f9b671e2640 1 --2- 172.21.15.176:0/1987667255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b601005f0 0x7f9b601009f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:57.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.003+0000 7f9b671e2640 1 -- 172.21.15.176:0/1987667255 >> 172.21.15.176:0/1987667255 conn(0x7f9b600fbda0 msgr2=0x7f9b600fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:57.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.003+0000 7f9b671e2640 1 -- 172.21.15.176:0/1987667255 shutdown_connections 2024-09-16T01:31:57.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.003+0000 7f9b671e2640 1 -- 172.21.15.176:0/1987667255 wait complete. 2024-09-16T01:31:57.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.003+0000 7f9b671e2640 1 Processor -- start 2024-09-16T01:31:57.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.003+0000 7f9b671e2640 1 -- start start 2024-09-16T01:31:57.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.003+0000 7f9b671e2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b601005f0 0x7f9b60197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:57.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b671e2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b60197560 0x7f9b6019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:57.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b671e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b601979e0 0x7f9b6019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:57.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b671e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9b601066e0 con 0x7f9b601979e0 2024-09-16T01:31:57.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b671e2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60106560 con 0x7f9b601005f0 2024-09-16T01:31:57.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b671e2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60106860 con 0x7f9b60197560 2024-09-16T01:31:57.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b57fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b60197560 0x7f9b6019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:57.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b57fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b60197560 0x7f9b6019c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:51434/0 (socket says 172.21.15.176:51434) 2024-09-16T01:31:57.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b64f7e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b601005f0 0x7f9b60197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:57.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b57fff640 1 -- 172.21.15.176:0/208650238 learned_addr learned my addr 172.21.15.176:0/208650238 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-16T01:31:57.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b6577f640 1 --2- 172.21.15.176:0/208650238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b601979e0 0x7f9b6019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:57.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b57fff640 1 -- 172.21.15.176:0/208650238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b601005f0 msgr2=0x7f9b60197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:57.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b57fff640 1 --2- 172.21.15.176:0/208650238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b601005f0 0x7f9b60197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:57.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b57fff640 1 -- 172.21.15.176:0/208650238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b601979e0 msgr2=0x7f9b6019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:57.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b57fff640 1 --2- 172.21.15.176:0/208650238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b601979e0 0x7f9b6019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:57.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b57fff640 1 -- 172.21.15.176:0/208650238 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6019f420 con 0x7f9b60197560 2024-09-16T01:31:57.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b57fff640 1 --2- 172.21.15.176:0/208650238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b60197560 0x7f9b6019c5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000b520 tx=0x7f9b5000b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:57.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b55ffb640 1 -- 172.21.15.176:0/208650238 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50004230 con 0x7f9b60197560 2024-09-16T01:31:57.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b55ffb640 1 -- 172.21.15.176:0/208650238 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b500043d0 con 0x7f9b60197560 2024-09-16T01:31:57.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b671e2640 1 -- 172.21.15.176:0/208650238 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6019f710 con 0x7f9b60197560 2024-09-16T01:31:57.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.004+0000 7f9b55ffb640 1 -- 172.21.15.176:0/208650238 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50003d60 con 0x7f9b60197560 2024-09-16T01:31:57.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.005+0000 7f9b671e2640 1 -- 172.21.15.176:0/208650238 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b6019fd20 con 0x7f9b60197560 2024-09-16T01:31:57.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.005+0000 7f9b55ffb640 1 -- 172.21.15.176:0/208650238 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f9b5000e070 con 0x7f9b60197560 2024-09-16T01:31:57.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.006+0000 7f9b55ffb640 1 --2- 172.21.15.176:0/208650238 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9b48039730 0x7f9b4803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:31:57.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.006+0000 7f9b55ffb640 1 -- 172.21.15.176:0/208650238 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9b5004bc50 con 0x7f9b60197560 2024-09-16T01:31:57.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.006+0000 7f9b671e2640 1 -- 172.21.15.176:0/208650238 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b6019ffe0 con 0x7f9b60197560 2024-09-16T01:31:57.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.006+0000 7f9b64f7e640 1 --2- 172.21.15.176:0/208650238 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9b48039730 0x7f9b4803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:31:57.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.006+0000 7f9b64f7e640 1 --2- 172.21.15.176:0/208650238 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9b48039730 0x7f9b4803bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009230 tx=0x7f9b5c0315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:31:57.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.011+0000 7f9b55ffb640 1 -- 172.21.15.176:0/208650238 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b50014020 con 0x7f9b60197560 2024-09-16T01:31:57.188 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.187+0000 7f9b671e2640 1 -- 172.21.15.176:0/208650238 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi139=x;smithi164=y;smithi176=z", "target": ["mon-mgr", ""]}) -- 0x7f9b60062630 con 0x7f9b48039730 2024-09-16T01:31:57.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.193+0000 7f9b55ffb640 1 -- 172.21.15.176:0/208650238 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9b60062630 con 0x7f9b48039730 2024-09-16T01:31:57.194 INFO:teuthology.orchestra.run.smithi176.stdout:Scheduled mgr update... 2024-09-16T01:31:57.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.195+0000 7f9b671e2640 1 -- 172.21.15.176:0/208650238 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9b48039730 msgr2=0x7f9b4803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:57.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.195+0000 7f9b671e2640 1 --2- 172.21.15.176:0/208650238 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9b48039730 0x7f9b4803bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009230 tx=0x7f9b5c0315d0 comp rx=0 tx=0).stop 2024-09-16T01:31:57.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.195+0000 7f9b671e2640 1 -- 172.21.15.176:0/208650238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b60197560 msgr2=0x7f9b6019c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:31:57.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.195+0000 7f9b671e2640 1 --2- 172.21.15.176:0/208650238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b60197560 0x7f9b6019c5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000b520 tx=0x7f9b5000b9e0 comp rx=0 tx=0).stop 2024-09-16T01:31:57.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.195+0000 7f9b671e2640 1 -- 172.21.15.176:0/208650238 shutdown_connections 2024-09-16T01:31:57.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.195+0000 7f9b671e2640 1 --2- 172.21.15.176:0/208650238 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9b48039730 0x7f9b4803bbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:57.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.195+0000 7f9b671e2640 1 --2- 172.21.15.176:0/208650238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b601979e0 0x7f9b6019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:57.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.195+0000 7f9b671e2640 1 --2- 172.21.15.176:0/208650238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b60197560 0x7f9b6019c5d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:57.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.195+0000 7f9b671e2640 1 --2- 172.21.15.176:0/208650238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b601005f0 0x7f9b60197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:31:57.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.195+0000 7f9b671e2640 1 -- 172.21.15.176:0/208650238 >> 172.21.15.176:0/208650238 conn(0x7f9b600fbda0 msgr2=0x7f9b6006acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:31:57.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.195+0000 7f9b671e2640 1 -- 172.21.15.176:0/208650238 shutdown_connections 2024-09-16T01:31:57.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:31:57.195+0000 7f9b671e2640 1 -- 172.21.15.176:0/208650238 wait complete. 2024-09-16T01:31:57.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:57 smithi139 ceph-mon[31175]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T01:31:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:57 smithi139 ceph-mon[31175]: Reconfiguring daemon mon.a on smithi139 2024-09-16T01:31:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:57 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:57.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:57 smithi176 ceph-mon[35324]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T01:31:57.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:57 smithi176 ceph-mon[35324]: Reconfiguring daemon mon.a on smithi139 2024-09-16T01:31:57.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:57 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:57 smithi164 ceph-mon[35121]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T01:31:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:57 smithi164 ceph-mon[35121]: Reconfiguring daemon mon.a on smithi139 2024-09-16T01:31:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:57 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:31:57.722 DEBUG:teuthology.orchestra.run.smithi164:mgr.y> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.y.service 2024-09-16T01:31:57.725 DEBUG:teuthology.orchestra.run.smithi176:mgr.z> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.z.service 2024-09-16T01:31:57.727 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T01:31:57.727 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:31:57.728 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T01:31:57.745 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T01:31:57.745 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_1 2024-09-16T01:31:57.749 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:31:57 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:31:57.472+0000 7f1771569640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T01:31:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T01:31:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:31:57.765 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-16T01:31:57.765 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:31:57.765 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:31:57.766 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-16 01:31:15.926244373 +0000 2024-09-16T01:31:57.766 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-16 01:27:11.018331121 +0000 2024-09-16T01:31:57.766 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-16 01:27:11.018331121 +0000 2024-09-16T01:31:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-16 01:27:11.018331121 +0000 2024-09-16T01:31:57.766 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T01:31:57.830 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-16T01:31:57.831 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-16T01:31:57.831 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000190674 s, 2.7 MB/s 2024-09-16T01:31:57.832 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T01:31:57.889 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_2 2024-09-16T01:31:57.945 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T01:31:57.945 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:31:57.945 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-16T01:31:57.945 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:31:57.945 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:31:57.945 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-16 01:31:15.952243819 +0000 2024-09-16T01:31:57.945 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-16 01:27:11.346324074 +0000 2024-09-16T01:31:57.945 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-16 01:27:11.346324074 +0000 2024-09-16T01:31:57.945 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-16 01:27:11.346324074 +0000 2024-09-16T01:31:57.946 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T01:31:58.009 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-16T01:31:58.009 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-16T01:31:58.009 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000210832 s, 2.4 MB/s 2024-09-16T01:31:58.010 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T01:31:58.068 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_3 2024-09-16T01:31:58.124 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T01:31:58.124 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:31:58.124 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-16T01:31:58.124 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:31:58.125 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:31:58.125 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-16 01:31:15.977243287 +0000 2024-09-16T01:31:58.125 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-16 01:27:11.667317177 +0000 2024-09-16T01:31:58.125 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-16 01:27:11.667317177 +0000 2024-09-16T01:31:58.126 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-16 01:27:11.667317177 +0000 2024-09-16T01:31:58.126 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T01:31:58.187 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-16T01:31:58.187 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-16T01:31:58.187 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000201476 s, 2.5 MB/s 2024-09-16T01:31:58.188 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T01:31:58.245 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_4 2024-09-16T01:31:58.300 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T01:31:58.301 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:31:58.301 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-16T01:31:58.301 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:31:58.301 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:31:58.301 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-16 01:31:16.004242712 +0000 2024-09-16T01:31:58.301 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-16 01:27:11.995310130 +0000 2024-09-16T01:31:58.301 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-16 01:27:11.995310130 +0000 2024-09-16T01:31:58.301 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-16 01:27:11.995310130 +0000 2024-09-16T01:31:58.301 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T01:31:58.361 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:58 smithi139 ceph-mon[31175]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi139=x;smithi164=y;smithi176=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:31:58.362 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:58 smithi139 ceph-mon[31175]: Saving service mgr spec with placement smithi139=x;smithi164=y;smithi176=z;count:3 2024-09-16T01:31:58.362 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:58 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:58.362 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:58 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:58.362 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:58 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:58.362 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:58 smithi139 ceph-mon[31175]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T01:31:58.362 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:58 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:31:58.362 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:58 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:31:58.362 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:58 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:58.363 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:58 smithi139 ceph-mon[31175]: Reconfiguring daemon mgr.x on smithi139 2024-09-16T01:31:58.363 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:58 smithi139 ceph-mon[31175]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:58.363 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:58 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:58.364 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-16T01:31:58.364 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-16T01:31:58.364 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000191895 s, 2.7 MB/s 2024-09-16T01:31:58.365 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T01:31:58.421 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T01:31:58.422 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T01:31:58.439 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T01:31:58.439 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-09-16T01:31:58.496 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T01:31:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:31:58.497 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-16T01:31:58.497 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:31:58.497 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:31:58.497 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 01:31:47.967922113 +0000 2024-09-16T01:31:58.497 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 01:27:29.885180226 +0000 2024-09-16T01:31:58.497 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 01:27:29.885180226 +0000 2024-09-16T01:31:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 01:27:29.885180226 +0000 2024-09-16T01:31:58.498 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T01:31:58.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:58 smithi164 ceph-mon[35121]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi139=x;smithi164=y;smithi176=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:31:58.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:58 smithi164 ceph-mon[35121]: Saving service mgr spec with placement smithi139=x;smithi164=y;smithi176=z;count:3 2024-09-16T01:31:58.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:58 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:58.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:58 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:58.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:58 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:58.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:58 smithi164 ceph-mon[35121]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T01:31:58.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:58 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:31:58.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:58 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:31:58.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:58 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:58.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:58 smithi164 ceph-mon[35121]: Reconfiguring daemon mgr.x on smithi139 2024-09-16T01:31:58.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:58 smithi164 ceph-mon[35121]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:58.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:58 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:58.562 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T01:31:58.562 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T01:31:58.562 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000278292 s, 1.8 MB/s 2024-09-16T01:31:58.563 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T01:31:58.619 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-09-16T01:31:58.677 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T01:31:58.678 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:31:58.678 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-16T01:31:58.678 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:31:58.678 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:31:58.678 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 01:31:47.967922113 +0000 2024-09-16T01:31:58.678 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 01:27:30.214174968 +0000 2024-09-16T01:31:58.678 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 01:27:30.214174968 +0000 2024-09-16T01:31:58.678 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 01:27:30.214174968 +0000 2024-09-16T01:31:58.678 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T01:31:58.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:58 smithi176 ceph-mon[35324]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi139=x;smithi164=y;smithi176=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:31:58.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:58 smithi176 ceph-mon[35324]: Saving service mgr spec with placement smithi139=x;smithi164=y;smithi176=z;count:3 2024-09-16T01:31:58.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:58 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:58.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:58 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:58.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:58 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:58.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:58 smithi176 ceph-mon[35324]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T01:31:58.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:58 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:31:58.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:58 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:31:58.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:58 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:58.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:58 smithi176 ceph-mon[35324]: Reconfiguring daemon mgr.x on smithi139 2024-09-16T01:31:58.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:58 smithi176 ceph-mon[35324]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:58.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:58 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:58.743 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T01:31:58.743 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T01:31:58.743 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000180953 s, 2.8 MB/s 2024-09-16T01:31:58.744 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T01:31:58.800 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-09-16T01:31:58.857 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T01:31:58.857 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:31:58.857 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-16T01:31:58.857 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:31:58.857 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:31:58.857 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 01:31:47.968922097 +0000 2024-09-16T01:31:58.857 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 01:27:30.534169852 +0000 2024-09-16T01:31:58.857 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 01:27:30.534169852 +0000 2024-09-16T01:31:58.857 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 01:27:30.534169852 +0000 2024-09-16T01:31:58.858 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T01:31:58.921 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T01:31:58.921 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T01:31:58.921 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000190323 s, 2.7 MB/s 2024-09-16T01:31:58.922 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T01:31:58.978 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-09-16T01:31:59.033 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T01:31:59.034 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:31:59.034 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-16T01:31:59.034 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:31:59.034 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:31:59.034 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 01:31:47.968922097 +0000 2024-09-16T01:31:59.034 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 01:27:30.866164546 +0000 2024-09-16T01:31:59.034 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 01:27:30.866164546 +0000 2024-09-16T01:31:59.034 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 01:27:30.866164546 +0000 2024-09-16T01:31:59.034 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T01:31:59.100 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T01:31:59.100 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T01:31:59.100 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000173553 s, 3.0 MB/s 2024-09-16T01:31:59.101 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T01:31:59.157 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-16T01:31:59.157 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T01:31:59.177 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T01:31:59.177 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_1 2024-09-16T01:31:59.236 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T01:31:59.236 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:31:59.236 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T01:31:59.237 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:31:59.237 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:31:59.237 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-16 01:31:39.165610058 +0000 2024-09-16T01:31:59.237 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-16 01:27:30.341793852 +0000 2024-09-16T01:31:59.237 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-16 01:27:30.341793852 +0000 2024-09-16T01:31:59.237 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-09-16 01:27:30.341793852 +0000 2024-09-16T01:31:59.237 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T01:31:59.306 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-16T01:31:59.307 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-16T01:31:59.307 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000285053 s, 1.8 MB/s 2024-09-16T01:31:59.307 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T01:31:59.366 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_2 2024-09-16T01:31:59.424 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T01:31:59.424 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:31:59.424 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-16T01:31:59.425 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:31:59.425 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:31:59.425 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-16 01:31:39.165610058 +0000 2024-09-16T01:31:59.425 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-16 01:27:30.653790836 +0000 2024-09-16T01:31:59.425 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-16 01:27:30.653790836 +0000 2024-09-16T01:31:59.425 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-09-16 01:27:30.653790836 +0000 2024-09-16T01:31:59.425 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T01:31:59.491 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-16T01:31:59.491 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-16T01:31:59.491 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000210113 s, 2.4 MB/s 2024-09-16T01:31:59.493 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T01:31:59.550 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_3 2024-09-16T01:31:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T01:31:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:31:59.607 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-16T01:31:59.607 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:31:59.608 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:31:59.608 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-16 01:31:39.166610048 +0000 2024-09-16T01:31:59.608 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-16 01:27:30.982787656 +0000 2024-09-16T01:31:59.608 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-16 01:27:30.982787656 +0000 2024-09-16T01:31:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-09-16 01:27:30.982787656 +0000 2024-09-16T01:31:59.608 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T01:31:59.673 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-16T01:31:59.673 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-16T01:31:59.674 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000209472 s, 2.4 MB/s 2024-09-16T01:31:59.675 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T01:31:59.733 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_4 2024-09-16T01:31:59.790 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T01:31:59.790 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T01:31:59.791 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-16T01:31:59.791 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:31:59.791 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T01:31:59.791 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-16 01:31:39.166610048 +0000 2024-09-16T01:31:59.791 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-16 01:27:31.315784437 +0000 2024-09-16T01:31:59.791 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-16 01:27:31.315784437 +0000 2024-09-16T01:31:59.791 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-09-16 01:27:31.315784437 +0000 2024-09-16T01:31:59.791 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T01:31:59.853 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:59.853 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:59.853 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:59 smithi176 ceph-mon[35324]: Reconfiguring mon.b (monmap changed)... 2024-09-16T01:31:59.853 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:31:59.853 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:31:59.853 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:59.853 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:59 smithi176 ceph-mon[35324]: Reconfiguring daemon mon.b on smithi164 2024-09-16T01:31:59.853 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:31:59 smithi176 ceph-mon[35324]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:59.855 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-16T01:31:59.855 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-16T01:31:59.855 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000228813 s, 2.2 MB/s 2024-09-16T01:31:59.856 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T01:31:59.913 INFO:tasks.cephadm:Deploying osd.0 on smithi139 with /dev/vg_nvme/lv_4... 2024-09-16T01:31:59.913 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T01:31:59.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:59.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:59.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:59 smithi139 ceph-mon[31175]: Reconfiguring mon.b (monmap changed)... 2024-09-16T01:31:59.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:31:59.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:31:59.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:59.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:59 smithi139 ceph-mon[31175]: Reconfiguring daemon mon.b on smithi164 2024-09-16T01:31:59.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:31:59 smithi139 ceph-mon[31175]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:31:59.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:59.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:31:59.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:59 smithi164 ceph-mon[35121]: Reconfiguring mon.b (monmap changed)... 2024-09-16T01:31:59.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:31:59.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:31:59.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:31:59.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:59 smithi164 ceph-mon[35121]: Reconfiguring daemon mon.b on smithi164 2024-09-16T01:31:59.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:31:59 smithi164 ceph-mon[35121]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:00.165 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:32:00.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:00.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:00.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:00 smithi139 ceph-mon[31175]: Reconfiguring mon.c (monmap changed)... 2024-09-16T01:32:00.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:32:00.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:32:00.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:00.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:00 smithi139 ceph-mon[31175]: Reconfiguring daemon mon.c on smithi176 2024-09-16T01:32:01.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:01.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:01.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:00 smithi176 ceph-mon[35324]: Reconfiguring mon.c (monmap changed)... 2024-09-16T01:32:01.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:32:01.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:32:01.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:01.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:00 smithi176 ceph-mon[35324]: Reconfiguring daemon mon.c on smithi176 2024-09-16T01:32:01.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:01.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:01.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:00 smithi164 ceph-mon[35121]: Reconfiguring mon.c (monmap changed)... 2024-09-16T01:32:01.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T01:32:01.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T01:32:01.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:01.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:00 smithi164 ceph-mon[35121]: Reconfiguring daemon mon.c on smithi176 2024-09-16T01:32:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:32:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:32:02.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:32:02.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:02.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:02 smithi139 ceph-mon[31175]: Deploying daemon mgr.z on smithi176 2024-09-16T01:32:02.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:02 smithi139 ceph-mon[31175]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:02.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:02.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:02.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:02.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:02.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:02.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:02.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:32:02.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:32:02.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:32:02.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:02.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:02 smithi176 ceph-mon[35324]: Deploying daemon mgr.z on smithi176 2024-09-16T01:32:02.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:02 smithi176 ceph-mon[35324]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:02.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:02.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:02.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:02.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:02.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:02.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:02.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:32:02.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:32:02.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:32:02.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:02.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:02 smithi164 ceph-mon[35121]: Deploying daemon mgr.z on smithi176 2024-09-16T01:32:02.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:02 smithi164 ceph-mon[35121]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:02.603 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:32:02.630 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_4 2024-09-16T01:32:02.911 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:32:03.124 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:02 smithi176 systemd[1]: Starting Ceph mgr.z for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:32:03.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.352+0000 7f556ee95640 1 -- 172.21.15.139:0/1841294300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5568100b10 msgr2=0x7f5568100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:32:03.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.352+0000 7f556ee95640 1 --2- 172.21.15.139:0/1841294300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5568100b10 0x7f5568100f10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f555c009900 tx=0x7f555c02a410 comp rx=0 tx=0).stop 2024-09-16T01:32:03.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.352+0000 7f556ee95640 1 -- 172.21.15.139:0/1841294300 shutdown_connections 2024-09-16T01:32:03.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.352+0000 7f556ee95640 1 --2- 172.21.15.139:0/1841294300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55681026d0 0x7f5568108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:03.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.352+0000 7f556ee95640 1 --2- 172.21.15.139:0/1841294300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5568101d10 0x7f5568102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:03.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.352+0000 7f556ee95640 1 --2- 172.21.15.139:0/1841294300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5568100b10 0x7f5568100f10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:03.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.352+0000 7f556ee95640 1 -- 172.21.15.139:0/1841294300 >> 172.21.15.139:0/1841294300 conn(0x7f55680fc2a0 msgr2=0x7f55680fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:32:03.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.352+0000 7f556ee95640 1 -- 172.21.15.139:0/1841294300 shutdown_connections 2024-09-16T01:32:03.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.352+0000 7f556ee95640 1 -- 172.21.15.139:0/1841294300 wait complete. 2024-09-16T01:32:03.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.352+0000 7f556ee95640 1 Processor -- start 2024-09-16T01:32:03.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556ee95640 1 -- start start 2024-09-16T01:32:03.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556ee95640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5568100b10 0x7f5568197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:32:03.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556ee95640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5568101d10 0x7f55681984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:32:03.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556ee95640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55681026d0 0x7f556819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:32:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556ee95640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f556810c840 con 0x7f55681026d0 2024-09-16T01:32:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556ee95640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f556810c6c0 con 0x7f5568101d10 2024-09-16T01:32:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556ee95640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f556810c9c0 con 0x7f5568100b10 2024-09-16T01:32:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556cc31640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5568100b10 0x7f5568197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:32:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556cc31640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5568100b10 0x7f5568197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:56554/0 (socket says 172.21.15.139:56554) 2024-09-16T01:32:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556cc31640 1 -- 172.21.15.139:0/2742702803 learned_addr learned my addr 172.21.15.139:0/2742702803 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:32:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556d432640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55681026d0 0x7f556819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:32:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f5567fff640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5568101d10 0x7f55681984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:32:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556d432640 1 -- 172.21.15.139:0/2742702803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5568100b10 msgr2=0x7f5568197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:32:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556d432640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5568100b10 0x7f5568197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556d432640 1 -- 172.21.15.139:0/2742702803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5568101d10 msgr2=0x7f55681984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:32:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556d432640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5568101d10 0x7f55681984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556d432640 1 -- 172.21.15.139:0/2742702803 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556819fbe0 con 0x7f55681026d0 2024-09-16T01:32:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.353+0000 7f556cc31640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5568100b10 0x7f5568197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:32:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.354+0000 7f556d432640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55681026d0 0x7f556819f570 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f556000e940 tx=0x7f556000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:32:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.354+0000 7f5565ffb640 1 -- 172.21.15.139:0/2742702803 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560004d20 con 0x7f55681026d0 2024-09-16T01:32:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.354+0000 7f5565ffb640 1 -- 172.21.15.139:0/2742702803 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5560005020 con 0x7f55681026d0 2024-09-16T01:32:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.354+0000 7f556ee95640 1 -- 172.21.15.139:0/2742702803 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556819fed0 con 0x7f55681026d0 2024-09-16T01:32:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.354+0000 7f5565ffb640 1 -- 172.21.15.139:0/2742702803 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55600116b0 con 0x7f55681026d0 2024-09-16T01:32:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.354+0000 7f556ee95640 1 -- 172.21.15.139:0/2742702803 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55681a03b0 con 0x7f55681026d0 2024-09-16T01:32:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.355+0000 7f5565ffb640 1 -- 172.21.15.139:0/2742702803 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f5560011850 con 0x7f55681026d0 2024-09-16T01:32:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.355+0000 7f5565ffb640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5558039500 0x7f555803b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:32:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.355+0000 7f5565ffb640 1 -- 172.21.15.139:0/2742702803 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f556004cbc0 con 0x7f55681026d0 2024-09-16T01:32:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.355+0000 7f556cc31640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5558039500 0x7f555803b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:32:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.355+0000 7f556ee95640 1 -- 172.21.15.139:0/2742702803 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55681a0670 con 0x7f55681026d0 2024-09-16T01:32:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.356+0000 7f556cc31640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5558039500 0x7f555803b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f555c000c00 tx=0x7f555c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:32:03.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.360+0000 7f5565ffb640 1 -- 172.21.15.139:0/2742702803 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5560016030 con 0x7f55681026d0 2024-09-16T01:32:03.376 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:03 smithi176 podman[37114]: 2024-09-16 01:32:03.123331566 +0000 UTC m=+0.105938900 container create 48120aef7454b98b17b09007388a9ce1c933390176259b14108b578fc47b8e3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-16T01:32:03.376 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:03 smithi176 podman[37114]: 2024-09-16 01:32:03.033624364 +0000 UTC m=+0.016231666 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:32:03.376 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:03 smithi176 podman[37114]: 2024-09-16 01:32:03.300447248 +0000 UTC m=+0.283054543 container init 48120aef7454b98b17b09007388a9ce1c933390176259b14108b578fc47b8e3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T01:32:03.376 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:03 smithi176 podman[37114]: 2024-09-16 01:32:03.30307902 +0000 UTC m=+0.285686313 container start 48120aef7454b98b17b09007388a9ce1c933390176259b14108b578fc47b8e3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T01:32:03.376 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:03 smithi176 ceph-mgr[37132]: -- 172.21.15.176:0/3154611785 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559cbd4610e0 con 0x559cbd43f400 2024-09-16T01:32:03.376 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:03 smithi176 bash[37114]: 48120aef7454b98b17b09007388a9ce1c933390176259b14108b578fc47b8e3d 2024-09-16T01:32:03.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:03.542+0000 7f556ee95640 1 -- 172.21.15.139:0/2742702803 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f556806af10 con 0x7f5558039500 2024-09-16T01:32:03.633 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:03 smithi176 systemd[1]: Started Ceph mgr.z for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:32:03.633 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:03 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:03.426+0000 7fcc89a0e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:32:03.633 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:03 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:03.486+0000 7fcc89a0e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:32:03.633 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:03 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:03.632+0000 7fcc89a0e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:32:03.936 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:03 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:03.697+0000 7fcc89a0e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:32:03.937 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:03 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:03.842+0000 7fcc89a0e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:32:04.289 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:03 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:03.980+0000 7fcc89a0e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:32:04.289 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:04.288+0000 7fcc89a0e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:32:04.591 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:04 smithi139 ceph-mon[31175]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:04.591 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:04.591 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:04.591 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:04.591 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:32:04.591 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:32:04.591 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:32:04.591 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:04.591 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:04 smithi139 ceph-mon[31175]: Deploying daemon mgr.y on smithi164 2024-09-16T01:32:04.592 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:04 smithi139 ceph-mon[31175]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:32:04.592 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:32:04.592 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:32:04.592 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:04.686 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:04.349+0000 7fcc89a0e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:32:04.687 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:04.532+0000 7fcc89a0e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:32:04.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-mon[35324]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:04.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:04.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:04.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:04.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:32:04.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:32:04.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:32:04.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:04.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-mon[35324]: Deploying daemon mgr.y on smithi164 2024-09-16T01:32:04.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-mon[35324]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:32:04.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:32:04.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:32:04.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:04.742 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:04 smithi164 ceph-mon[35121]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:04.743 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:04.743 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:04.743 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:04.743 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T01:32:04.743 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T01:32:04.743 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T01:32:04.743 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:04.743 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:04 smithi164 ceph-mon[35121]: Deploying daemon mgr.y on smithi164 2024-09-16T01:32:04.743 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:04 smithi164 ceph-mon[35121]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:32:04.743 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:32:04.743 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:32:04.743 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:05.295 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:05 smithi164 systemd[1]: Starting Ceph mgr.y for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:32:05.672 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:05 smithi164 podman[36469]: 2024-09-16 01:32:05.295578999 +0000 UTC m=+0.015942472 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:32:05.672 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:05 smithi164 podman[36469]: 2024-09-16 01:32:05.393999763 +0000 UTC m=+0.114363275 container create bc3233dc290976ce0651d10712423cc1d83908ecf1e13ca1d04f3dff44a5ce38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD) 2024-09-16T01:32:05.672 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:05 smithi164 podman[36469]: 2024-09-16 01:32:05.596532016 +0000 UTC m=+0.316895498 container init bc3233dc290976ce0651d10712423cc1d83908ecf1e13ca1d04f3dff44a5ce38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T01:32:05.672 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:05 smithi164 podman[36469]: 2024-09-16 01:32:05.599820847 +0000 UTC m=+0.320184321 container start bc3233dc290976ce0651d10712423cc1d83908ecf1e13ca1d04f3dff44a5ce38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-16T01:32:05.672 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:05 smithi164 ceph-mgr[36488]: -- 172.21.15.164:0/1321078197 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55616788d0e0 con 0x55616786b000 2024-09-16T01:32:05.686 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:05 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:05.407+0000 7fcc89a0e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:32:05.686 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:05 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:05.474+0000 7fcc89a0e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:32:05.939 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:05 smithi164 bash[36469]: bc3233dc290976ce0651d10712423cc1d83908ecf1e13ca1d04f3dff44a5ce38 2024-09-16T01:32:05.939 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:05 smithi164 systemd[1]: Started Ceph mgr.y for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:32:05.939 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:05 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:05.722+0000 7f05c1a0d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:32:05.939 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:05 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:05.781+0000 7f05c1a0d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:32:05.939 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:05 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:05.937+0000 7f05c1a0d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:32:06.295 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:06.006+0000 7f05c1a0d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:32:06.295 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:06.167+0000 7f05c1a0d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:32:06.494 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:06.226+0000 7fcc89a0e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:32:06.494 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:06.345+0000 7fcc89a0e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:32:06.494 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:06.492+0000 7fcc89a0e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:32:06.494 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-mon[35324]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:06.494 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:06.494 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:06.494 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:06.494 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:06.495 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/973815313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b66696da-ef66-4f25-9527-bbcc9ded2cee"}]: dispatch 2024-09-16T01:32:06.495 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/973815313' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b66696da-ef66-4f25-9527-bbcc9ded2cee"}]': finished 2024-09-16T01:32:06.495 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-mon[35324]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T01:32:06.495 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:06.495 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:06.495 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/848327648' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:32:06.625 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:06.323+0000 7f05c1a0d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:32:06.625 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-mon[35121]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:06.625 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:06.625 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:06.625 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:06.625 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:06.625 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/973815313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b66696da-ef66-4f25-9527-bbcc9ded2cee"}]: dispatch 2024-09-16T01:32:06.626 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/973815313' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b66696da-ef66-4f25-9527-bbcc9ded2cee"}]': finished 2024-09-16T01:32:06.626 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-mon[35121]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T01:32:06.626 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:06.626 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:06.626 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/848327648' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:32:06.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:06 smithi139 ceph-mon[31175]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:06.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:06 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:06.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:06 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:06.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:06 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:06.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:06 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:06.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:06 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/973815313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b66696da-ef66-4f25-9527-bbcc9ded2cee"}]: dispatch 2024-09-16T01:32:06.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:06 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/973815313' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b66696da-ef66-4f25-9527-bbcc9ded2cee"}]': finished 2024-09-16T01:32:06.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:06 smithi139 ceph-mon[31175]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T01:32:06.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:06 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:06.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:06 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:06.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:06 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/848327648' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:32:06.783 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:06.559+0000 7fcc89a0e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:32:06.783 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:06.712+0000 7fcc89a0e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:32:06.783 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:06.782+0000 7fcc89a0e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:32:06.906 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:06.651+0000 7f05c1a0d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:32:06.907 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:06.712+0000 7f05c1a0d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:32:07.083 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:06 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:06.842+0000 7fcc89a0e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:32:07.083 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:07.012+0000 7fcc89a0e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:32:07.083 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:07.082+0000 7fcc89a0e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:32:07.295 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:06 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:06.905+0000 7f05c1a0d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:32:07.351 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:07.151+0000 7fcc89a0e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:32:07.351 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:07.234+0000 7fcc89a0e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:32:07.351 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:07.292+0000 7fcc89a0e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:32:07.352 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:07.350+0000 7fcc89a0e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:32:08.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:08.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:08.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:08.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:08.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:08.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:07 smithi139 ceph-mon[31175]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:08.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.186 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:32:08 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:32:08.130+0000 7fcc89a0e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:32:08.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-mon[35324]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.295 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:07.797+0000 7f05c1a0d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:32:08.295 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:07.863+0000 7f05c1a0d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:32:08.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:08.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:08.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:08.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:08.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:08.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-mon[35121]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:08.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:08.841 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:08 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:08.582+0000 7f05c1a0d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:32:08.841 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:08 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:08.697+0000 7f05c1a0d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:32:08.841 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:08 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:08.839+0000 7f05c1a0d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:32:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:09.080+0000 7f5565ffb640 1 -- 172.21.15.139:0/2742702803 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mgrmap(e 14) ==== 92409+0+0 (secure 0 0 0) 0x7f55600108f0 con 0x7f55681026d0 2024-09-16T01:32:09.119 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:09 smithi164 ceph-mon[35121]: Standby manager daemon z started 2024-09-16T01:32:09.120 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:09 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.176:0/296159523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T01:32:09.120 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:09 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.176:0/296159523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:32:09.120 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:09 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.176:0/296159523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T01:32:09.120 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:09 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.176:0/296159523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:32:09.120 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:08 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:08.901+0000 7f05c1a0d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:32:09.120 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:09 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:09.049+0000 7f05c1a0d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:32:09.120 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:09 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:09.118+0000 7f05c1a0d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:32:09.401 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:09 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:09.178+0000 7f05c1a0d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:32:09.402 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:09 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:09.334+0000 7f05c1a0d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:32:09.402 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:09 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:09.400+0000 7f05c1a0d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:32:09.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:09 smithi139 ceph-mon[31175]: Standby manager daemon z started 2024-09-16T01:32:09.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:09 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.176:0/296159523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T01:32:09.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:09 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.176:0/296159523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:32:09.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:09 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.176:0/296159523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T01:32:09.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:09 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.176:0/296159523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:32:09.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:09 smithi176 ceph-mon[35324]: Standby manager daemon z started 2024-09-16T01:32:09.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:09 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.176:0/296159523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T01:32:09.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:09 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.176:0/296159523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:32:09.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:09 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.176:0/296159523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T01:32:09.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:09 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.176:0/296159523' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:32:09.669 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:09 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:09.467+0000 7f05c1a0d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:32:09.669 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:09 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:09.549+0000 7f05c1a0d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:32:09.669 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:09 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:09.608+0000 7f05c1a0d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:32:09.669 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:09 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:09.667+0000 7f05c1a0d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:32:10.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:10 smithi139 ceph-mon[31175]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-16T01:32:10.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:10 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T01:32:10.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:10 smithi139 ceph-mon[31175]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:10.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:10 smithi176 ceph-mon[35324]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-16T01:32:10.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:10 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T01:32:10.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:10 smithi176 ceph-mon[35324]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:10.457 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:10 smithi164 ceph-mon[35121]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-16T01:32:10.457 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:10 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T01:32:10.457 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:10 smithi164 ceph-mon[35121]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:10.795 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:32:10 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:32:10.456+0000 7f05c1a0d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:32:11.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:11.094+0000 7f5565ffb640 1 -- 172.21.15.139:0/2742702803 <== mon.0 v2:172.21.15.139:3300/0 8 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f55600103d0 con 0x7f55681026d0 2024-09-16T01:32:11.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:11 smithi139 ceph-mon[31175]: Standby manager daemon y started 2024-09-16T01:32:11.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:11 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.164:0/285540938' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T01:32:11.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:11 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.164:0/285540938' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:32:11.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:11 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.164:0/285540938' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T01:32:11.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:11 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.164:0/285540938' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:32:11.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:11 smithi176 ceph-mon[35324]: Standby manager daemon y started 2024-09-16T01:32:11.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:11 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.164:0/285540938' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T01:32:11.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:11 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.164:0/285540938' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:32:11.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:11 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.164:0/285540938' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T01:32:11.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:11 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.164:0/285540938' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:32:11.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:11 smithi164 ceph-mon[35121]: Standby manager daemon y started 2024-09-16T01:32:11.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:11 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.164:0/285540938' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T01:32:11.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:11 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.164:0/285540938' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:32:11.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:11 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.164:0/285540938' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T01:32:11.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:11 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.164:0/285540938' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:32:12.352 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:12 smithi139 ceph-mon[31175]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-16T01:32:12.352 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:12 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T01:32:12.352 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:12 smithi139 ceph-mon[31175]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:12.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:12 smithi176 ceph-mon[35324]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-16T01:32:12.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:12 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T01:32:12.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:12 smithi176 ceph-mon[35324]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:12 smithi164 ceph-mon[35121]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-16T01:32:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:12 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T01:32:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:12 smithi164 ceph-mon[35121]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:14.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:14 smithi139 ceph-mon[31175]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:14.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:14 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T01:32:14.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:14 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:14.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:14 smithi176 ceph-mon[35324]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:14.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:14 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T01:32:14.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:14 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:14.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:14 smithi164 ceph-mon[35121]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:14.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:14 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T01:32:14.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:14 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:15.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:15 smithi139 ceph-mon[31175]: Deploying daemon osd.0 on smithi139 2024-09-16T01:32:15.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:15 smithi176 ceph-mon[35324]: Deploying daemon osd.0 on smithi139 2024-09-16T01:32:15.696 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:15 smithi164 ceph-mon[35121]: Deploying daemon osd.0 on smithi139 2024-09-16T01:32:16.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:16 smithi176 ceph-mon[35324]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:16.773 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:16 smithi139 ceph-mon[31175]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:16.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:16 smithi164 ceph-mon[35121]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:18.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:18 smithi176 ceph-mon[35324]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:18.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:18 smithi164 ceph-mon[35121]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:18.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:18 smithi139 ceph-mon[31175]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:20.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:20 smithi176 ceph-mon[35324]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:20.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:20 smithi164 ceph-mon[35121]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:20.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:20 smithi139 ceph-mon[31175]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:21.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:21 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:21.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:21 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:21.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:21 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:21.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:21 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:21.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:21 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:21.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:21 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:21.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:21 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:21.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:21 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:21.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:21 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:22.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:22 smithi139 ceph-mon[31175]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:22.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:22 smithi176 ceph-mon[35324]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:22.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:22 smithi164 ceph-mon[35121]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:24.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:23 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:24.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:23 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:24.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:23 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:24.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:23 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:24.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:23 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:24.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:23 smithi164 ceph-mon[35121]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:23 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:23 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:23 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:23 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:23 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:23 smithi139 ceph-mon[31175]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:24.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:23 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:23 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:23 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:23 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:23 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:23 smithi176 ceph-mon[35324]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:24.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:24.870+0000 7f5565ffb640 1 -- 172.21.15.139:0/2742702803 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f556806af10 con 0x7f5558039500 2024-09-16T01:32:24.873 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 0 on host 'smithi139' 2024-09-16T01:32:24.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:24.873+0000 7f556ee95640 1 -- 172.21.15.139:0/2742702803 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5558039500 msgr2=0x7f555803b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:32:24.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:24.873+0000 7f556ee95640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5558039500 0x7f555803b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f555c000c00 tx=0x7f555c0315b0 comp rx=0 tx=0).stop 2024-09-16T01:32:24.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:24.873+0000 7f556ee95640 1 -- 172.21.15.139:0/2742702803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55681026d0 msgr2=0x7f556819f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:32:24.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:24.873+0000 7f556ee95640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55681026d0 0x7f556819f570 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f556000e940 tx=0x7f556000ee00 comp rx=0 tx=0).stop 2024-09-16T01:32:24.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:24.873+0000 7f556ee95640 1 -- 172.21.15.139:0/2742702803 shutdown_connections 2024-09-16T01:32:24.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:24.873+0000 7f556ee95640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5558039500 0x7f555803b9c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:24.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:24.873+0000 7f556ee95640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55681026d0 0x7f556819f570 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:24.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:24.873+0000 7f556ee95640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5568101d10 0x7f55681984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:24.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:24.873+0000 7f556ee95640 1 --2- 172.21.15.139:0/2742702803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5568100b10 0x7f5568197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:24.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:24.873+0000 7f556ee95640 1 -- 172.21.15.139:0/2742702803 >> 172.21.15.139:0/2742702803 conn(0x7f55680fc2a0 msgr2=0x7f556806a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:32:24.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:24.873+0000 7f556ee95640 1 -- 172.21.15.139:0/2742702803 shutdown_connections 2024-09-16T01:32:24.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:32:24.873+0000 7f556ee95640 1 -- 172.21.15.139:0/2742702803 wait complete. 2024-09-16T01:32:25.462 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:25.462 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:25.462 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:25.489 DEBUG:teuthology.orchestra.run.smithi139:osd.0> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T01:32:25.491 INFO:tasks.cephadm:Deploying osd.1 on smithi164 with /dev/vg_nvme/lv_4... 2024-09-16T01:32:25.492 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T01:32:25.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:25.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:25.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:25.731 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:32:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:26.430 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:26 smithi164 ceph-mon[35121]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:26.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:26 smithi139 ceph-mon[31175]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:26.673 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 01:32:26 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0[42159]: 2024-09-16T01:32:26.406+0000 7fa1c24a9640 -1 osd.0 0 log_to_monitors true 2024-09-16T01:32:26.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:26 smithi176 ceph-mon[35324]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:27.572 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:27 smithi139 ceph-mon[31175]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T01:32:27.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:27 smithi176 ceph-mon[35324]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T01:32:27.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:27 smithi164 ceph-mon[35121]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T01:32:28.057 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:32:28.082 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_4 2024-09-16T01:32:28.343 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:32:28.604 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:28 smithi139 ceph-mon[31175]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T01:32:28.605 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:28 smithi139 ceph-mon[31175]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T01:32:28.605 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:28 smithi139 ceph-mon[31175]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-16T01:32:28.605 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:28.605 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:28 smithi139 ceph-mon[31175]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:28.682 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:28 smithi164 ceph-mon[35121]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T01:32:28.682 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:28 smithi164 ceph-mon[35121]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T01:32:28.682 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:28 smithi164 ceph-mon[35121]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-16T01:32:28.682 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:28 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:28.682 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:28 smithi164 ceph-mon[35121]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:28.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:28 smithi176 ceph-mon[35324]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T01:32:28.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:28 smithi176 ceph-mon[35324]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T01:32:28.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:28 smithi176 ceph-mon[35324]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-16T01:32:28.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:28 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:28.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:28 smithi176 ceph-mon[35324]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:28.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.768+0000 7f1b481f2640 1 -- 172.21.15.164:0/2951552497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b40100450 msgr2=0x7f1b40102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:32:28.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.768+0000 7f1b481f2640 1 --2- 172.21.15.164:0/2951552497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b40100450 0x7f1b40102860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009ca0 tx=0x7f1b3002a8c0 comp rx=0 tx=0).stop 2024-09-16T01:32:28.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.768+0000 7f1b481f2640 1 -- 172.21.15.164:0/2951552497 shutdown_connections 2024-09-16T01:32:28.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.768+0000 7f1b481f2640 1 --2- 172.21.15.164:0/2951552497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b40105750 0x7f1b40105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:28.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.768+0000 7f1b481f2640 1 --2- 172.21.15.164:0/2951552497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b40102da0 0x7f1b40105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:28.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.768+0000 7f1b481f2640 1 --2- 172.21.15.164:0/2951552497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b40100450 0x7f1b40102860 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:28.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.768+0000 7f1b481f2640 1 -- 172.21.15.164:0/2951552497 >> 172.21.15.164:0/2951552497 conn(0x7f1b400fc280 msgr2=0x7f1b400fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:32:28.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.768+0000 7f1b481f2640 1 -- 172.21.15.164:0/2951552497 shutdown_connections 2024-09-16T01:32:28.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.768+0000 7f1b481f2640 1 -- 172.21.15.164:0/2951552497 wait complete. 2024-09-16T01:32:28.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.769+0000 7f1b481f2640 1 Processor -- start 2024-09-16T01:32:28.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.769+0000 7f1b481f2640 1 -- start start 2024-09-16T01:32:28.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.769+0000 7f1b481f2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b40100450 0x7f1b4019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:32:28.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.769+0000 7f1b481f2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b40102da0 0x7f1b4019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:32:28.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.769+0000 7f1b481f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b40105750 0x7f1b401a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:32:28.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.769+0000 7f1b481f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40113eb0 con 0x7f1b40105750 2024-09-16T01:32:28.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.769+0000 7f1b481f2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40113d30 con 0x7f1b40102da0 2024-09-16T01:32:28.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.769+0000 7f1b481f2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40114030 con 0x7f1b40100450 2024-09-16T01:32:28.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.769+0000 7f1b4578d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b40102da0 0x7f1b4019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:32:28.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.769+0000 7f1b4578d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b40102da0 0x7f1b4019ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50272/0 (socket says 172.21.15.164:50272) 2024-09-16T01:32:28.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.769+0000 7f1b4578d640 1 -- 172.21.15.164:0/2586480593 learned_addr learned my addr 172.21.15.164:0/2586480593 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T01:32:28.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.769+0000 7f1b4678f640 1 --2- 172.21.15.164:0/2586480593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b40105750 0x7f1b401a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:32:28.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.769+0000 7f1b45f8e640 1 --2- 172.21.15.164:0/2586480593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b40100450 0x7f1b4019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:32:28.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.770+0000 7f1b4578d640 1 -- 172.21.15.164:0/2586480593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b40100450 msgr2=0x7f1b4019e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:32:28.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.770+0000 7f1b4578d640 1 --2- 172.21.15.164:0/2586480593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b40100450 0x7f1b4019e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:28.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.770+0000 7f1b4578d640 1 -- 172.21.15.164:0/2586480593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b40105750 msgr2=0x7f1b401a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:32:28.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.770+0000 7f1b4578d640 1 --2- 172.21.15.164:0/2586480593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b40105750 0x7f1b401a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:28.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.770+0000 7f1b4578d640 1 -- 172.21.15.164:0/2586480593 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b401a6120 con 0x7f1b40102da0 2024-09-16T01:32:28.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.770+0000 7f1b4578d640 1 --2- 172.21.15.164:0/2586480593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b40102da0 0x7f1b4019ea30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400e9f0 tx=0x7f1b3400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:32:28.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.770+0000 7f1b3effd640 1 -- 172.21.15.164:0/2586480593 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3400cd80 con 0x7f1b40102da0 2024-09-16T01:32:28.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.770+0000 7f1b481f2640 1 -- 172.21.15.164:0/2586480593 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b401a6410 con 0x7f1b40102da0 2024-09-16T01:32:28.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.770+0000 7f1b3effd640 1 -- 172.21.15.164:0/2586480593 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b34005500 con 0x7f1b40102da0 2024-09-16T01:32:28.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.770+0000 7f1b3effd640 1 -- 172.21.15.164:0/2586480593 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34010690 con 0x7f1b40102da0 2024-09-16T01:32:28.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.770+0000 7f1b481f2640 1 -- 172.21.15.164:0/2586480593 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b401a6900 con 0x7f1b40102da0 2024-09-16T01:32:28.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.772+0000 7f1b481f2640 1 -- 172.21.15.164:0/2586480593 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b400fdd70 con 0x7f1b40102da0 2024-09-16T01:32:28.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.772+0000 7f1b3effd640 1 -- 172.21.15.164:0/2586480593 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f1b340046c0 con 0x7f1b40102da0 2024-09-16T01:32:28.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.773+0000 7f1b3effd640 1 --2- 172.21.15.164:0/2586480593 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1b280a31e0 0x7f1b280a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:32:28.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.773+0000 7f1b3effd640 1 -- 172.21.15.164:0/2586480593 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f1b340ce120 con 0x7f1b40102da0 2024-09-16T01:32:28.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.773+0000 7f1b45f8e640 1 --2- 172.21.15.164:0/2586480593 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1b280a31e0 0x7f1b280a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:32:28.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.773+0000 7f1b45f8e640 1 --2- 172.21.15.164:0/2586480593 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1b280a31e0 0x7f1b280a56a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b30005e30 tx=0x7f1b30005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:32:28.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.777+0000 7f1b3effd640 1 -- 172.21.15.164:0/2586480593 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b3409c9d0 con 0x7f1b40102da0 2024-09-16T01:32:28.923 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 01:32:28 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0[42159]: 2024-09-16T01:32:28.762+0000 7fa1bec40640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T01:32:28.923 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 01:32:28 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0[42159]: 2024-09-16T01:32:28.765+0000 7fa1b624c640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:32:28.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:28.959+0000 7f1b481f2640 1 -- 172.21.15.164:0/2586480593 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1b40062630 con 0x7f1b280a31e0 2024-09-16T01:32:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-16T01:32:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T01:32:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:32:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:29.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:29.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:29.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:29.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' 2024-09-16T01:32:29.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:32:29.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:32:29.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:29.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-16T01:32:29.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T01:32:29.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:29.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:29.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:29.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:29.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:32:29.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:29.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:29.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:29.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:29.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' 2024-09-16T01:32:29.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:32:29.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:32:29.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:29.693 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-16T01:32:29.693 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T01:32:29.693 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:29.693 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:29.693 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:29.693 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:29.693 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:32:29.693 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:29.693 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:29.693 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:29.694 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:29.694 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: from='osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133]' entity='osd.0' 2024-09-16T01:32:29.694 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:32:29.694 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:32:29.694 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:30.605 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:30 smithi164 ceph-mon[35121]: purged_snaps scrub starts 2024-09-16T01:32:30.605 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:30 smithi164 ceph-mon[35121]: purged_snaps scrub ok 2024-09-16T01:32:30.606 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:30 smithi164 ceph-mon[35121]: Detected new or changed devices on smithi139 2024-09-16T01:32:30.606 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:30 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi139 to 17088M 2024-09-16T01:32:30.606 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:30 smithi164 ceph-mon[35121]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:32:30.606 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:30 smithi164 ceph-mon[35121]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:30.606 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:30 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:30.606 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:30 smithi164 ceph-mon[35121]: osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133] boot 2024-09-16T01:32:30.606 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:30 smithi164 ceph-mon[35121]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T01:32:30.606 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:30 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:30.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:30 smithi139 ceph-mon[31175]: purged_snaps scrub starts 2024-09-16T01:32:30.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:30 smithi139 ceph-mon[31175]: purged_snaps scrub ok 2024-09-16T01:32:30.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:30 smithi139 ceph-mon[31175]: Detected new or changed devices on smithi139 2024-09-16T01:32:30.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:30 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi139 to 17088M 2024-09-16T01:32:30.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:30 smithi139 ceph-mon[31175]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:32:30.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:30 smithi139 ceph-mon[31175]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:30.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:30 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:30.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:30 smithi139 ceph-mon[31175]: osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133] boot 2024-09-16T01:32:30.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:30 smithi139 ceph-mon[31175]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T01:32:30.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:30 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:30.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:30 smithi176 ceph-mon[35324]: purged_snaps scrub starts 2024-09-16T01:32:30.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:30 smithi176 ceph-mon[35324]: purged_snaps scrub ok 2024-09-16T01:32:30.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:30 smithi176 ceph-mon[35324]: Detected new or changed devices on smithi139 2024-09-16T01:32:30.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:30 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi139 to 17088M 2024-09-16T01:32:30.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:30 smithi176 ceph-mon[35324]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:32:30.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:30 smithi176 ceph-mon[35324]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T01:32:30.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:30 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:30.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:30 smithi176 ceph-mon[35324]: osd.0 [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133] boot 2024-09-16T01:32:30.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:30 smithi176 ceph-mon[35324]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T01:32:30.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:30 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:32:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:31 smithi139 ceph-mon[31175]: from='client.? 172.21.15.164:0/4114131717' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccad17f2-0750-425e-9810-49a13a929ac3"}]: dispatch 2024-09-16T01:32:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:31 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccad17f2-0750-425e-9810-49a13a929ac3"}]: dispatch 2024-09-16T01:32:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:31 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccad17f2-0750-425e-9810-49a13a929ac3"}]': finished 2024-09-16T01:32:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:31 smithi139 ceph-mon[31175]: osdmap e9: 2 total, 1 up, 2 in 2024-09-16T01:32:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:31 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:31.689 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:31 smithi176 ceph-mon[35324]: from='client.? 172.21.15.164:0/4114131717' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccad17f2-0750-425e-9810-49a13a929ac3"}]: dispatch 2024-09-16T01:32:31.689 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:31 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccad17f2-0750-425e-9810-49a13a929ac3"}]: dispatch 2024-09-16T01:32:31.689 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:31 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccad17f2-0750-425e-9810-49a13a929ac3"}]': finished 2024-09-16T01:32:31.689 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:31 smithi176 ceph-mon[35324]: osdmap e9: 2 total, 1 up, 2 in 2024-09-16T01:32:31.689 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:31 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:31.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:31 smithi164 ceph-mon[35121]: from='client.? 172.21.15.164:0/4114131717' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccad17f2-0750-425e-9810-49a13a929ac3"}]: dispatch 2024-09-16T01:32:31.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:31 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccad17f2-0750-425e-9810-49a13a929ac3"}]: dispatch 2024-09-16T01:32:31.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:31 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccad17f2-0750-425e-9810-49a13a929ac3"}]': finished 2024-09-16T01:32:31.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:31 smithi164 ceph-mon[35121]: osdmap e9: 2 total, 1 up, 2 in 2024-09-16T01:32:31.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:31 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:32.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:32 smithi139 ceph-mon[31175]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:32.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:32 smithi139 ceph-mon[31175]: from='client.? 172.21.15.164:0/2881048123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:32:32.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:32 smithi176 ceph-mon[35324]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:32.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:32 smithi176 ceph-mon[35324]: from='client.? 172.21.15.164:0/2881048123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:32:32.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:32 smithi164 ceph-mon[35121]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:32.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:32 smithi164 ceph-mon[35121]: from='client.? 172.21.15.164:0/2881048123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:32:34.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:34 smithi139 ceph-mon[31175]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:34.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:34 smithi176 ceph-mon[35324]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:34.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:34 smithi164 ceph-mon[35121]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:36.495 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:36 smithi164 ceph-mon[35121]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:36.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:36 smithi139 ceph-mon[31175]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:36.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:36 smithi176 ceph-mon[35324]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:38.404 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:38 smithi164 ceph-mon[35121]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:38.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:38 smithi139 ceph-mon[31175]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:38.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:38 smithi176 ceph-mon[35324]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:39 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T01:32:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:39 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:39.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:39 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T01:32:39.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:39 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:39 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T01:32:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:39 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:40.381 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:40 smithi164 ceph-mon[35121]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:40.382 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:40 smithi164 ceph-mon[35121]: Deploying daemon osd.1 on smithi164 2024-09-16T01:32:40.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:40 smithi139 ceph-mon[31175]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:40.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:40 smithi139 ceph-mon[31175]: Deploying daemon osd.1 on smithi164 2024-09-16T01:32:40.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:40 smithi176 ceph-mon[35324]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:40.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:40 smithi176 ceph-mon[35324]: Deploying daemon osd.1 on smithi164 2024-09-16T01:32:42.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:42 smithi164 ceph-mon[35121]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:42.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:42 smithi139 ceph-mon[31175]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:42.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:42 smithi176 ceph-mon[35324]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:44.523 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:44 smithi164 ceph-mon[35121]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:44.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:44 smithi139 ceph-mon[31175]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:44.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:44 smithi176 ceph-mon[35324]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:46.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:46 smithi139 ceph-mon[31175]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:46.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:46 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:46.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:46 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:46.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:46 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:46.678 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:46 smithi164 ceph-mon[35121]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:46.678 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:46 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:46.678 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:46 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:46.678 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:46 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:46.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:46 smithi176 ceph-mon[35324]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:46.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:46 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:46.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:46 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:46.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:46 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:48 smithi139 ceph-mon[31175]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:48 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:48 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:48 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:48 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:48 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:48.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:48 smithi176 ceph-mon[35324]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:48.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:48 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:48.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:48 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:48.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:48 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:48.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:48 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:48.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:48 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:48 smithi164 ceph-mon[35121]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:48.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:49.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:49.733+0000 7f1b3effd640 1 -- 172.21.15.164:0/2586480593 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1b40062630 con 0x7f1b280a31e0 2024-09-16T01:32:49.736 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 1 on host 'smithi164' 2024-09-16T01:32:49.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:49.736+0000 7f1b481f2640 1 -- 172.21.15.164:0/2586480593 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1b280a31e0 msgr2=0x7f1b280a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:32:49.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:49.736+0000 7f1b481f2640 1 --2- 172.21.15.164:0/2586480593 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1b280a31e0 0x7f1b280a56a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b30005e30 tx=0x7f1b30005f50 comp rx=0 tx=0).stop 2024-09-16T01:32:49.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:49.736+0000 7f1b481f2640 1 -- 172.21.15.164:0/2586480593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b40102da0 msgr2=0x7f1b4019ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:32:49.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:49.736+0000 7f1b481f2640 1 --2- 172.21.15.164:0/2586480593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b40102da0 0x7f1b4019ea30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400e9f0 tx=0x7f1b3400edf0 comp rx=0 tx=0).stop 2024-09-16T01:32:49.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:49.736+0000 7f1b481f2640 1 -- 172.21.15.164:0/2586480593 shutdown_connections 2024-09-16T01:32:49.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:49.736+0000 7f1b481f2640 1 --2- 172.21.15.164:0/2586480593 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1b280a31e0 0x7f1b280a56a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:49.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:49.736+0000 7f1b481f2640 1 --2- 172.21.15.164:0/2586480593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b40105750 0x7f1b401a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:49.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:49.736+0000 7f1b481f2640 1 --2- 172.21.15.164:0/2586480593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b40102da0 0x7f1b4019ea30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:49.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:49.736+0000 7f1b481f2640 1 --2- 172.21.15.164:0/2586480593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b40100450 0x7f1b4019e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:49.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:49.736+0000 7f1b481f2640 1 -- 172.21.15.164:0/2586480593 >> 172.21.15.164:0/2586480593 conn(0x7f1b400fc280 msgr2=0x7f1b401036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:32:49.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:49.736+0000 7f1b481f2640 1 -- 172.21.15.164:0/2586480593 shutdown_connections 2024-09-16T01:32:49.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:32:49.736+0000 7f1b481f2640 1 -- 172.21.15.164:0/2586480593 wait complete. 2024-09-16T01:32:50.345 DEBUG:teuthology.orchestra.run.smithi164:osd.1> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T01:32:50.347 INFO:tasks.cephadm:Deploying osd.2 on smithi176 with /dev/vg_nvme/lv_4... 2024-09-16T01:32:50.347 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T01:32:50.569 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:50 smithi164 ceph-mon[35121]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:50.569 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:50 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:50.569 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:50 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:50.569 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:50 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:50.601 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:32:50.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:50 smithi139 ceph-mon[31175]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:50.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:50 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:50.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:50 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:50.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:50 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:50.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:50 smithi176 ceph-mon[35324]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:50.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:50 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:32:50.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:50 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:50.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:50 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:51.786 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 01:32:51 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1[39585]: 2024-09-16T01:32:51.658+0000 7f6fabf2b640 -1 osd.1 0 log_to_monitors true 2024-09-16T01:32:52.604 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:52 smithi176 ceph-mon[35324]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:52.604 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:52 smithi176 ceph-mon[35324]: from='osd.1 [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:32:52.604 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:52 smithi176 ceph-mon[35324]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:32:52.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:52 smithi139 ceph-mon[31175]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:52.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:52 smithi139 ceph-mon[31175]: from='osd.1 [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:32:52.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:52 smithi139 ceph-mon[31175]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:32:52.755 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:52 smithi164 ceph-mon[35121]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:52.755 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:52 smithi164 ceph-mon[35121]: from='osd.1 [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:32:52.755 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:52 smithi164 ceph-mon[35121]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T01:32:53.082 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:32:53.111 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_4 2024-09-16T01:32:53.363 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:32:53.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T01:32:53.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: from='osd.1 [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:32:53.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T01:32:53.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:53.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:32:53.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:53.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:53.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:32:53.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:53.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:53.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:53.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:53.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T01:32:53.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T01:32:53.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T01:32:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: from='osd.1 [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:32:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T01:32:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:32:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:32:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:53.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:53.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T01:32:53.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T01:32:53.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T01:32:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: from='osd.1 [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:32:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T01:32:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:32:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:32:53.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:53.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:53.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:32:53.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:32:53.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T01:32:53.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T01:32:53.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.797+0000 7f952d9a0640 1 -- 172.21.15.176:0/3640305651 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528107940 msgr2=0x7f9528109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:32:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.797+0000 7f952d9a0640 1 --2- 172.21.15.176:0/3640305651 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528107940 0x7f9528109d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9520009ca0 tx=0x7f952002a8c0 comp rx=0 tx=0).stop 2024-09-16T01:32:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.797+0000 7f952d9a0640 1 -- 172.21.15.176:0/3640305651 shutdown_connections 2024-09-16T01:32:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.797+0000 7f952d9a0640 1 --2- 172.21.15.176:0/3640305651 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528107940 0x7f9528109d30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.797+0000 7f952d9a0640 1 --2- 172.21.15.176:0/3640305651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9528105010 0x7f9528107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.797+0000 7f952d9a0640 1 --2- 172.21.15.176:0/3640305651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95281026e0 0x7f9528104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.797+0000 7f952d9a0640 1 -- 172.21.15.176:0/3640305651 >> 172.21.15.176:0/3640305651 conn(0x7f95280fc280 msgr2=0x7f95280fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:32:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.797+0000 7f952d9a0640 1 -- 172.21.15.176:0/3640305651 shutdown_connections 2024-09-16T01:32:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.797+0000 7f952d9a0640 1 -- 172.21.15.176:0/3640305651 wait complete. 2024-09-16T01:32:53.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.797+0000 7f952d9a0640 1 Processor -- start 2024-09-16T01:32:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.797+0000 7f952d9a0640 1 -- start start 2024-09-16T01:32:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f952d9a0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95281026e0 0x7f9528197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:32:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f952d9a0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9528105010 0x7f9528198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:32:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f952d9a0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528107940 0x7f952819f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:32:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f952d9a0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f952810d880 con 0x7f95281026e0 2024-09-16T01:32:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f952d9a0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f952810d700 con 0x7f9528105010 2024-09-16T01:32:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f952d9a0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f952810da00 con 0x7f9528107940 2024-09-16T01:32:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f95277fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528107940 0x7f952819f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:32:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f95277fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528107940 0x7f952819f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:41108/0 (socket says 172.21.15.176:41108) 2024-09-16T01:32:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f951e7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9528105010 0x7f9528198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:32:53.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f95277fe640 1 -- 172.21.15.176:0/2245063627 learned_addr learned my addr 172.21.15.176:0/2245063627 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-16T01:32:53.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f9526ffd640 1 --2- 172.21.15.176:0/2245063627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95281026e0 0x7f9528197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:32:53.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f95277fe640 1 -- 172.21.15.176:0/2245063627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9528105010 msgr2=0x7f9528198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:32:53.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f95277fe640 1 --2- 172.21.15.176:0/2245063627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9528105010 0x7f9528198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:53.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f95277fe640 1 -- 172.21.15.176:0/2245063627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95281026e0 msgr2=0x7f9528197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:32:53.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f95277fe640 1 --2- 172.21.15.176:0/2245063627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95281026e0 0x7f9528197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:32:53.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f95277fe640 1 -- 172.21.15.176:0/2245063627 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952819fb00 con 0x7f9528107940 2024-09-16T01:32:53.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f951e7fc640 1 --2- 172.21.15.176:0/2245063627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9528105010 0x7f9528198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:32:53.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f95277fe640 1 --2- 172.21.15.176:0/2245063627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528107940 0x7f952819f490 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9520031ae0 tx=0x7f9520031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:32:53.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.798+0000 7f952c99e640 1 -- 172.21.15.176:0/2245063627 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520003c30 con 0x7f9528107940 2024-09-16T01:32:53.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.799+0000 7f952c99e640 1 -- 172.21.15.176:0/2245063627 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9520003dd0 con 0x7f9528107940 2024-09-16T01:32:53.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.799+0000 7f952d9a0640 1 -- 172.21.15.176:0/2245063627 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952819fd90 con 0x7f9528107940 2024-09-16T01:32:53.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.799+0000 7f952c99e640 1 -- 172.21.15.176:0/2245063627 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952002fcb0 con 0x7f9528107940 2024-09-16T01:32:53.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.799+0000 7f952d9a0640 1 -- 172.21.15.176:0/2245063627 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95281a0270 con 0x7f9528107940 2024-09-16T01:32:53.805 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.800+0000 7f952d9a0640 1 -- 172.21.15.176:0/2245063627 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95280fdd70 con 0x7f9528107940 2024-09-16T01:32:53.805 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.801+0000 7f952c99e640 1 -- 172.21.15.176:0/2245063627 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f9520004050 con 0x7f9528107940 2024-09-16T01:32:53.805 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.801+0000 7f952c99e640 1 --2- 172.21.15.176:0/2245063627 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f95140a31e0 0x7f95140a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:32:53.805 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.802+0000 7f952c99e640 1 -- 172.21.15.176:0/2245063627 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2029+0+0 (secure 0 0 0) 0x7f95200ef160 con 0x7f9528107940 2024-09-16T01:32:53.806 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.802+0000 7f9526ffd640 1 --2- 172.21.15.176:0/2245063627 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f95140a31e0 0x7f95140a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:32:53.806 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.802+0000 7f9526ffd640 1 --2- 172.21.15.176:0/2245063627 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f95140a31e0 0x7f95140a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9518009bd0 tx=0x7f9518005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:32:53.808 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.806+0000 7f952c99e640 1 -- 172.21.15.176:0/2245063627 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95200bd910 con 0x7f9528107940 2024-09-16T01:32:53.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:32:53.980+0000 7f952d9a0640 1 -- 172.21.15.176:0/2245063627 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9528062630 con 0x7f95140a31e0 2024-09-16T01:32:54.295 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1[39585]: 2024-09-16T01:32:53.811+0000 7f6fa7eae640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T01:32:54.295 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 01:32:53 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1[39585]: 2024-09-16T01:32:53.814+0000 7f6f9fcce640 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:32:54.660 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:54 smithi176 ceph-mon[35324]: Detected new or changed devices on smithi164 2024-09-16T01:32:54.660 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:54 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi164 to 17088M 2024-09-16T01:32:54.660 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:54 smithi176 ceph-mon[35324]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:54.661 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:54 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:54.661 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:54 smithi176 ceph-mon[35324]: from='osd.1 ' entity='osd.1' 2024-09-16T01:32:54.661 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:54 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:32:54.661 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:54 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:32:54.661 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:54 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:54.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:54 smithi139 ceph-mon[31175]: Detected new or changed devices on smithi164 2024-09-16T01:32:54.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:54 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi164 to 17088M 2024-09-16T01:32:54.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:54 smithi139 ceph-mon[31175]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:54.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:54 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:54.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:54 smithi139 ceph-mon[31175]: from='osd.1 ' entity='osd.1' 2024-09-16T01:32:54.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:54 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:32:54.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:54 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:32:54.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:54 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:54 smithi164 ceph-mon[35121]: Detected new or changed devices on smithi164 2024-09-16T01:32:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:54 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi164 to 17088M 2024-09-16T01:32:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:54 smithi164 ceph-mon[35121]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:54 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:54 smithi164 ceph-mon[35121]: from='osd.1 ' entity='osd.1' 2024-09-16T01:32:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:54 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:32:54.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:54 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:32:54.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:54 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:32:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:55 smithi139 ceph-mon[31175]: purged_snaps scrub starts 2024-09-16T01:32:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:55 smithi139 ceph-mon[31175]: purged_snaps scrub ok 2024-09-16T01:32:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:55 smithi139 ceph-mon[31175]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:32:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:55 smithi139 ceph-mon[31175]: osd.1 [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330] boot 2024-09-16T01:32:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:55 smithi139 ceph-mon[31175]: osdmap e12: 2 total, 2 up, 2 in 2024-09-16T01:32:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:55.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:55 smithi176 ceph-mon[35324]: purged_snaps scrub starts 2024-09-16T01:32:55.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:55 smithi176 ceph-mon[35324]: purged_snaps scrub ok 2024-09-16T01:32:55.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:55 smithi176 ceph-mon[35324]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:32:55.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:55.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:55 smithi176 ceph-mon[35324]: osd.1 [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330] boot 2024-09-16T01:32:55.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:55 smithi176 ceph-mon[35324]: osdmap e12: 2 total, 2 up, 2 in 2024-09-16T01:32:55.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:55 smithi164 ceph-mon[35121]: purged_snaps scrub starts 2024-09-16T01:32:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:55 smithi164 ceph-mon[35121]: purged_snaps scrub ok 2024-09-16T01:32:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:55 smithi164 ceph-mon[35121]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:32:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:55 smithi164 ceph-mon[35121]: osd.1 [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330] boot 2024-09-16T01:32:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:55 smithi164 ceph-mon[35121]: osdmap e12: 2 total, 2 up, 2 in 2024-09-16T01:32:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:32:56.653 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:56 smithi176 ceph-mon[35324]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:56.653 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:56 smithi176 ceph-mon[35324]: from='client.? 172.21.15.176:0/1145871994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baf0ba73-8faf-4e51-a171-c47898800638"}]: dispatch 2024-09-16T01:32:56.653 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:56 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baf0ba73-8faf-4e51-a171-c47898800638"}]: dispatch 2024-09-16T01:32:56.653 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:56 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baf0ba73-8faf-4e51-a171-c47898800638"}]': finished 2024-09-16T01:32:56.653 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:56 smithi176 ceph-mon[35324]: osdmap e13: 3 total, 2 up, 3 in 2024-09-16T01:32:56.653 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:32:56.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:56 smithi139 ceph-mon[31175]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:56.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:56 smithi139 ceph-mon[31175]: from='client.? 172.21.15.176:0/1145871994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baf0ba73-8faf-4e51-a171-c47898800638"}]: dispatch 2024-09-16T01:32:56.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:56 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baf0ba73-8faf-4e51-a171-c47898800638"}]: dispatch 2024-09-16T01:32:56.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:56 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baf0ba73-8faf-4e51-a171-c47898800638"}]': finished 2024-09-16T01:32:56.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:56 smithi139 ceph-mon[31175]: osdmap e13: 3 total, 2 up, 3 in 2024-09-16T01:32:56.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:32:56.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:56 smithi164 ceph-mon[35121]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T01:32:56.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:56 smithi164 ceph-mon[35121]: from='client.? 172.21.15.176:0/1145871994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baf0ba73-8faf-4e51-a171-c47898800638"}]: dispatch 2024-09-16T01:32:56.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:56 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baf0ba73-8faf-4e51-a171-c47898800638"}]: dispatch 2024-09-16T01:32:56.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:56 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baf0ba73-8faf-4e51-a171-c47898800638"}]': finished 2024-09-16T01:32:56.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:56 smithi164 ceph-mon[35121]: osdmap e13: 3 total, 2 up, 3 in 2024-09-16T01:32:56.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:32:57.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:57 smithi139 ceph-mon[31175]: from='client.? 172.21.15.176:0/1425790798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:32:57.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:57 smithi176 ceph-mon[35324]: from='client.? 172.21.15.176:0/1425790798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:32:57.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:57 smithi164 ceph-mon[35121]: from='client.? 172.21.15.176:0/1425790798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:32:58.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:32:58 smithi139 ceph-mon[31175]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:32:58.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:32:58 smithi176 ceph-mon[35324]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:32:58.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:32:58 smithi164 ceph-mon[35121]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:00.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:00 smithi139 ceph-mon[31175]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:00.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:00 smithi176 ceph-mon[35324]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:00.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:00 smithi164 ceph-mon[35121]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:02.473 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:02 smithi176 ceph-mon[35324]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:02 smithi164 ceph-mon[35121]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:02.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:02 smithi139 ceph-mon[31175]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:04.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:04 smithi176 ceph-mon[35324]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:04.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T01:33:04.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:04.725 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:04 smithi139 ceph-mon[31175]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:04.725 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T01:33:04.725 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:04 smithi164 ceph-mon[35121]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T01:33:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:05.674 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:05 smithi176 ceph-mon[35324]: Deploying daemon osd.2 on smithi176 2024-09-16T01:33:05.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:05 smithi164 ceph-mon[35121]: Deploying daemon osd.2 on smithi176 2024-09-16T01:33:05.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:05 smithi139 ceph-mon[31175]: Deploying daemon osd.2 on smithi176 2024-09-16T01:33:06.610 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:06 smithi176 ceph-mon[35324]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:06.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:06 smithi164 ceph-mon[35121]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:06.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:06 smithi139 ceph-mon[31175]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:08.746 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:08 smithi176 ceph-mon[35324]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:08.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:08 smithi164 ceph-mon[35121]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:08.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:08 smithi139 ceph-mon[31175]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:10.743 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:10 smithi176 ceph-mon[35324]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:10.743 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:10 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:33:10.743 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:10 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:10.743 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:10 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:10 smithi164 ceph-mon[35121]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:10 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:33:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:10 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:10 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:10.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:10 smithi139 ceph-mon[31175]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:10.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:10 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:33:10.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:10 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:10.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:10 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:12.534 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:12 smithi176 ceph-mon[35324]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:12.535 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:12 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:12.535 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:12 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:12.535 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:12 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:12.535 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:12 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:33:12.535 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:12 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:12 smithi164 ceph-mon[35121]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:12 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:12 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:12 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:12 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:33:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:12 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:12 smithi139 ceph-mon[31175]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:12 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:12 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:12 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:12 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:33:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:12 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:14.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:33:14.135+0000 7f952c99e640 1 -- 172.21.15.176:0/2245063627 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9528062630 con 0x7f95140a31e0 2024-09-16T01:33:14.137 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 2 on host 'smithi176' 2024-09-16T01:33:14.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:33:14.137+0000 7f952d9a0640 1 -- 172.21.15.176:0/2245063627 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f95140a31e0 msgr2=0x7f95140a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:33:14.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:33:14.137+0000 7f952d9a0640 1 --2- 172.21.15.176:0/2245063627 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f95140a31e0 0x7f95140a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9518009bd0 tx=0x7f9518005f90 comp rx=0 tx=0).stop 2024-09-16T01:33:14.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:33:14.137+0000 7f952d9a0640 1 -- 172.21.15.176:0/2245063627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528107940 msgr2=0x7f952819f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:33:14.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:33:14.137+0000 7f952d9a0640 1 --2- 172.21.15.176:0/2245063627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528107940 0x7f952819f490 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9520031ae0 tx=0x7f9520031ee0 comp rx=0 tx=0).stop 2024-09-16T01:33:14.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:33:14.137+0000 7f952d9a0640 1 -- 172.21.15.176:0/2245063627 shutdown_connections 2024-09-16T01:33:14.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:33:14.137+0000 7f952d9a0640 1 --2- 172.21.15.176:0/2245063627 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f95140a31e0 0x7f95140a56a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:14.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:33:14.137+0000 7f952d9a0640 1 --2- 172.21.15.176:0/2245063627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528107940 0x7f952819f490 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:14.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:33:14.137+0000 7f952d9a0640 1 --2- 172.21.15.176:0/2245063627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9528105010 0x7f9528198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:14.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:33:14.137+0000 7f952d9a0640 1 --2- 172.21.15.176:0/2245063627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95281026e0 0x7f9528197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:14.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:33:14.137+0000 7f952d9a0640 1 -- 172.21.15.176:0/2245063627 >> 172.21.15.176:0/2245063627 conn(0x7f95280fc280 msgr2=0x7f9528102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:33:14.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:33:14.137+0000 7f952d9a0640 1 -- 172.21.15.176:0/2245063627 shutdown_connections 2024-09-16T01:33:14.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:33:14.137+0000 7f952d9a0640 1 -- 172.21.15.176:0/2245063627 wait complete. 2024-09-16T01:33:14.748 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:14 smithi176 ceph-mon[35324]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:14.748 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:14 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:33:14.748 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:14 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:14.748 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:14 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:14.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:14 smithi164 ceph-mon[35121]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:14.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:14 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:33:14.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:14 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:14.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:14 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:14.843 DEBUG:teuthology.orchestra.run.smithi176:osd.2> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T01:33:14.845 INFO:tasks.cephadm:Deploying osd.3 on smithi139 with /dev/vg_nvme/lv_3... 2024-09-16T01:33:14.845 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T01:33:14.873 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:14 smithi139 ceph-mon[31175]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:14.874 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:14 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:33:14.874 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:14 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:14.874 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:14 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:15.099 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:33:16.333 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 01:33:16 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2[40288]: 2024-09-16T01:33:16.233+0000 7fe3bbfe0640 -1 osd.2 0 log_to_monitors true 2024-09-16T01:33:16.492 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:16 smithi139 ceph-mon[31175]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:16.492 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:16 smithi139 ceph-mon[31175]: from='osd.2 [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:33:16.492 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:16 smithi139 ceph-mon[31175]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:33:16.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:16 smithi176 ceph-mon[35324]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:16.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:16 smithi176 ceph-mon[35324]: from='osd.2 [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:33:16.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:16 smithi176 ceph-mon[35324]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:33:16.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:16 smithi164 ceph-mon[35121]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:16.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:16 smithi164 ceph-mon[35121]: from='osd.2 [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:33:16.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:16 smithi164 ceph-mon[35121]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T01:33:17.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:17 smithi139 ceph-mon[31175]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T01:33:17.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:17 smithi139 ceph-mon[31175]: osdmap e14: 3 total, 2 up, 3 in 2024-09-16T01:33:17.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:17.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:17 smithi139 ceph-mon[31175]: from='osd.2 [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:33:17.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:17 smithi139 ceph-mon[31175]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:33:17.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:17.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:17.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:33:17.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:17.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:17.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:33:17.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:17.702 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:33:17.730 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_3 2024-09-16T01:33:17.749 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:17 smithi176 ceph-mon[35324]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T01:33:17.749 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:17 smithi176 ceph-mon[35324]: osdmap e14: 3 total, 2 up, 3 in 2024-09-16T01:33:17.749 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:17.749 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:17 smithi176 ceph-mon[35324]: from='osd.2 [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:33:17.749 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:17 smithi176 ceph-mon[35324]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:33:17.749 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:17.749 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:17.750 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:33:17.750 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:17.750 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:17.750 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:33:17.750 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:17 smithi164 ceph-mon[35121]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T01:33:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:17 smithi164 ceph-mon[35121]: osdmap e14: 3 total, 2 up, 3 in 2024-09-16T01:33:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:17 smithi164 ceph-mon[35121]: from='osd.2 [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:33:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:17 smithi164 ceph-mon[35121]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:33:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:33:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:33:17.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:18.022 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:33:18.186 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 01:33:17 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2[40288]: 2024-09-16T01:33:17.886+0000 7fe3b7f63640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T01:33:18.187 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 01:33:17 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2[40288]: 2024-09-16T01:33:17.890+0000 7fe3afd83640 -1 osd.2 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:33:18.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.474+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3259529341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ec101d10 msgr2=0x7fa6ec102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:33:18.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.474+0000 7fa6f0cac640 1 --2- 172.21.15.139:0/3259529341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ec101d10 0x7fa6ec102190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009900 tx=0x7fa6e002a410 comp rx=0 tx=0).stop 2024-09-16T01:33:18.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.474+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3259529341 shutdown_connections 2024-09-16T01:33:18.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.474+0000 7fa6f0cac640 1 --2- 172.21.15.139:0/3259529341 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6ec1026d0 0x7fa6ec108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:18.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.474+0000 7fa6f0cac640 1 --2- 172.21.15.139:0/3259529341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ec101d10 0x7fa6ec102190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:18.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.474+0000 7fa6f0cac640 1 --2- 172.21.15.139:0/3259529341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ec100b10 0x7fa6ec100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:18.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.474+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3259529341 >> 172.21.15.139:0/3259529341 conn(0x7fa6ec0fc2a0 msgr2=0x7fa6ec0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:33:18.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.474+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3259529341 shutdown_connections 2024-09-16T01:33:18.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.474+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3259529341 wait complete. 2024-09-16T01:33:18.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.475+0000 7fa6f0cac640 1 Processor -- start 2024-09-16T01:33:18.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.475+0000 7fa6f0cac640 1 -- start start 2024-09-16T01:33:18.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.475+0000 7fa6f0cac640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6ec100b10 0x7fa6ec197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:33:18.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.475+0000 7fa6f0cac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ec101d10 0x7fa6ec198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:33:18.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.475+0000 7fa6f0cac640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ec1026d0 0x7fa6ec19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:33:18.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.475+0000 7fa6f0cac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec10c790 con 0x7fa6ec101d10 2024-09-16T01:33:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.475+0000 7fa6f0cac640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec10c610 con 0x7fa6ec1026d0 2024-09-16T01:33:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.475+0000 7fa6f0cac640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec10c910 con 0x7fa6ec100b10 2024-09-16T01:33:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.475+0000 7fa6e9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ec101d10 0x7fa6ec198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:33:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6ea59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6ec100b10 0x7fa6ec197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:33:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6e9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ec101d10 0x7fa6ec198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42368/0 (socket says 172.21.15.139:42368) 2024-09-16T01:33:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6e9d9b640 1 -- 172.21.15.139:0/3529718719 learned_addr learned my addr 172.21.15.139:0/3529718719 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:33:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6ead9d640 1 --2- 172.21.15.139:0/3529718719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ec1026d0 0x7fa6ec19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:33:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6e9d9b640 1 -- 172.21.15.139:0/3529718719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6ec100b10 msgr2=0x7fa6ec197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:33:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6e9d9b640 1 --2- 172.21.15.139:0/3529718719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6ec100b10 0x7fa6ec197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6e9d9b640 1 -- 172.21.15.139:0/3529718719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ec1026d0 msgr2=0x7fa6ec19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:33:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6e9d9b640 1 --2- 172.21.15.139:0/3529718719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ec1026d0 0x7fa6ec19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6e9d9b640 1 -- 172.21.15.139:0/3529718719 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec19fb30 con 0x7fa6ec101d10 2024-09-16T01:33:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6ea59c640 1 --2- 172.21.15.139:0/3529718719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6ec100b10 0x7fa6ec197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:33:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6e9d9b640 1 --2- 172.21.15.139:0/3529718719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ec101d10 0x7fa6ec198440 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009230 tx=0x7fa6e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:33:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6db7fe640 1 -- 172.21.15.139:0/3529718719 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e003f030 con 0x7fa6ec101d10 2024-09-16T01:33:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6db7fe640 1 -- 172.21.15.139:0/3529718719 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6e0003ce0 con 0x7fa6ec101d10 2024-09-16T01:33:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3529718719 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ec19fdc0 con 0x7fa6ec101d10 2024-09-16T01:33:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6db7fe640 1 -- 172.21.15.139:0/3529718719 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0035070 con 0x7fa6ec101d10 2024-09-16T01:33:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.476+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3529718719 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ec1a0240 con 0x7fa6ec101d10 2024-09-16T01:33:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.477+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3529718719 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ec1a0530 con 0x7fa6ec101d10 2024-09-16T01:33:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.478+0000 7fa6db7fe640 1 -- 172.21.15.139:0/3529718719 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7fa6e0042030 con 0x7fa6ec101d10 2024-09-16T01:33:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.479+0000 7fa6db7fe640 1 --2- 172.21.15.139:0/3529718719 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa6d40a31e0 0x7fa6d40a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:33:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.479+0000 7fa6ea59c640 1 --2- 172.21.15.139:0/3529718719 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa6d40a31e0 0x7fa6d40a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:33:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.479+0000 7fa6db7fe640 1 -- 172.21.15.139:0/3529718719 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2571+0+0 (secure 0 0 0) 0x7fa6e00ee5c0 con 0x7fa6ec101d10 2024-09-16T01:33:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.479+0000 7fa6ea59c640 1 --2- 172.21.15.139:0/3529718719 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa6d40a31e0 0x7fa6d40a56a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec101b70 tx=0x7fa6dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:33:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.483+0000 7fa6db7fe640 1 -- 172.21.15.139:0/3529718719 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6e00bc3e0 con 0x7fa6ec101d10 2024-09-16T01:33:18.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:18.660+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3529718719 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa6ec06aed0 con 0x7fa6d40a31e0 2024-09-16T01:33:18.662 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:18 smithi139 ceph-mon[31175]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:18.662 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:18 smithi139 ceph-mon[31175]: Detected new or changed devices on smithi176 2024-09-16T01:33:18.663 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:18 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi176 to 17088M 2024-09-16T01:33:18.663 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:18 smithi139 ceph-mon[31175]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-16T01:33:18.663 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:18 smithi139 ceph-mon[31175]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T01:33:18.663 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:18 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:18.663 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:18 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:18.663 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:18 smithi139 ceph-mon[31175]: from='osd.2 ' entity='osd.2' 2024-09-16T01:33:18.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:18 smithi164 ceph-mon[35121]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:18.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:18 smithi164 ceph-mon[35121]: Detected new or changed devices on smithi176 2024-09-16T01:33:18.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:18 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi176 to 17088M 2024-09-16T01:33:18.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:18 smithi164 ceph-mon[35121]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-16T01:33:18.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:18 smithi164 ceph-mon[35121]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T01:33:18.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:18 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:18.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:18 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:18.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:18 smithi164 ceph-mon[35121]: from='osd.2 ' entity='osd.2' 2024-09-16T01:33:18.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:18 smithi176 ceph-mon[35324]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:18.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:18 smithi176 ceph-mon[35324]: Detected new or changed devices on smithi176 2024-09-16T01:33:18.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:18 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi176 to 17088M 2024-09-16T01:33:18.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:18 smithi176 ceph-mon[35324]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-16T01:33:18.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:18 smithi176 ceph-mon[35324]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T01:33:18.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:18 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:18.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:18 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:18.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:18 smithi176 ceph-mon[35324]: from='osd.2 ' entity='osd.2' 2024-09-16T01:33:19.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:19 smithi139 ceph-mon[31175]: purged_snaps scrub starts 2024-09-16T01:33:19.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:19 smithi139 ceph-mon[31175]: purged_snaps scrub ok 2024-09-16T01:33:19.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:19.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:33:19.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:33:19.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:19.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:19 smithi139 ceph-mon[31175]: osd.2 [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646] boot 2024-09-16T01:33:19.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:19 smithi139 ceph-mon[31175]: osdmap e16: 3 total, 3 up, 3 in 2024-09-16T01:33:19.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:19.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:19 smithi176 ceph-mon[35324]: purged_snaps scrub starts 2024-09-16T01:33:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:19 smithi176 ceph-mon[35324]: purged_snaps scrub ok 2024-09-16T01:33:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:19 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:19 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:33:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:19 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:33:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:19 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:19 smithi176 ceph-mon[35324]: osd.2 [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646] boot 2024-09-16T01:33:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:19 smithi176 ceph-mon[35324]: osdmap e16: 3 total, 3 up, 3 in 2024-09-16T01:33:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:19 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:20.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:19 smithi164 ceph-mon[35121]: purged_snaps scrub starts 2024-09-16T01:33:20.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:19 smithi164 ceph-mon[35121]: purged_snaps scrub ok 2024-09-16T01:33:20.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:19 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:20.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:19 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:33:20.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:19 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:33:20.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:19 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:20.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:19 smithi164 ceph-mon[35121]: osd.2 [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646] boot 2024-09-16T01:33:20.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:19 smithi164 ceph-mon[35121]: osdmap e16: 3 total, 3 up, 3 in 2024-09-16T01:33:20.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:19 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:33:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:20 smithi139 ceph-mon[31175]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:33:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:20 smithi139 ceph-mon[31175]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:20 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:33:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:20 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T01:33:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:20 smithi139 ceph-mon[31175]: osdmap e17: 3 total, 3 up, 3 in 2024-09-16T01:33:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:20 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:33:20.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:20 smithi176 ceph-mon[35324]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:33:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:20 smithi176 ceph-mon[35324]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:20 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:33:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:20 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T01:33:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:20 smithi176 ceph-mon[35324]: osdmap e17: 3 total, 3 up, 3 in 2024-09-16T01:33:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:20 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:33:21.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:20 smithi164 ceph-mon[35121]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:33:21.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:20 smithi164 ceph-mon[35121]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-16T01:33:21.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:20 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:33:21.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:20 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T01:33:21.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:20 smithi164 ceph-mon[35121]: osdmap e17: 3 total, 3 up, 3 in 2024-09-16T01:33:21.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:20 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:33:21.516 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 01:33:21 smithi139 sudo[46544]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:33:21.516 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 01:33:21 smithi139 sudo[46544]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:33:21.516 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 01:33:21 smithi139 sudo[46544]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:33:21.516 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 01:33:21 smithi139 sudo[46544]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:33:21.900 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 01:33:21 smithi139 sudo[46572]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:33:21.900 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 01:33:21 smithi139 sudo[46572]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:33:21.901 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 01:33:21 smithi139 sudo[46572]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:33:21.901 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 01:33:21 smithi139 sudo[46572]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:33:22.045 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 01:33:21 smithi164 sudo[43288]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:33:22.045 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 01:33:21 smithi164 sudo[43288]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:33:22.045 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 01:33:21 smithi164 sudo[43288]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:33:22.045 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 01:33:21 smithi164 sudo[43288]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:33:22.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:21 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T01:33:22.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:21 smithi164 ceph-mon[35121]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T01:33:22.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:21 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/692629743' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98957bf1-67bb-4092-9ea4-b263a5706f8e"}]: dispatch 2024-09-16T01:33:22.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:21 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/692629743' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98957bf1-67bb-4092-9ea4-b263a5706f8e"}]': finished 2024-09-16T01:33:22.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:21 smithi164 ceph-mon[35121]: osdmap e19: 4 total, 3 up, 4 in 2024-09-16T01:33:22.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:21 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:22.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:21 smithi164 ceph-mon[35121]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:22.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:21 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:22.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:21 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2949853665' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:33:22.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:22.084+0000 7fa6db7fe640 1 -- 172.21.15.139:0/3529718719 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa6e003b460 con 0x7fa6ec101d10 2024-09-16T01:33:22.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:21 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T01:33:22.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:21 smithi139 ceph-mon[31175]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T01:33:22.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:21 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/692629743' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98957bf1-67bb-4092-9ea4-b263a5706f8e"}]: dispatch 2024-09-16T01:33:22.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:21 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/692629743' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98957bf1-67bb-4092-9ea4-b263a5706f8e"}]': finished 2024-09-16T01:33:22.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:21 smithi139 ceph-mon[31175]: osdmap e19: 4 total, 3 up, 4 in 2024-09-16T01:33:22.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:21 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:22.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:21 smithi139 ceph-mon[31175]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:22.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:21 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:22.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:21 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2949853665' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:33:22.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:21 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T01:33:22.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:21 smithi176 ceph-mon[35324]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T01:33:22.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:21 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/692629743' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98957bf1-67bb-4092-9ea4-b263a5706f8e"}]: dispatch 2024-09-16T01:33:22.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:21 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/692629743' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98957bf1-67bb-4092-9ea4-b263a5706f8e"}]': finished 2024-09-16T01:33:22.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:21 smithi176 ceph-mon[35324]: osdmap e19: 4 total, 3 up, 4 in 2024-09-16T01:33:22.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:21 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:22.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:21 smithi176 ceph-mon[35324]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:22.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:21 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:22.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:21 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2949853665' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:33:22.545 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 01:33:22 smithi164 sudo[43291]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:33:22.545 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 01:33:22 smithi164 sudo[43291]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:33:22.545 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 01:33:22 smithi164 sudo[43291]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:33:22.545 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 01:33:22 smithi164 sudo[43291]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:33:22.573 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 01:33:22 smithi176 sudo[43920]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:33:22.573 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 01:33:22 smithi176 sudo[43920]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:33:22.573 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 01:33:22 smithi176 sudo[43920]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:33:22.573 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 01:33:22 smithi176 sudo[43920]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:33:22.936 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 01:33:22 smithi176 sudo[43923]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:33:22.937 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 01:33:22 smithi176 sudo[43923]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:33:22.937 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 01:33:22 smithi176 sudo[43923]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:33:22.937 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 01:33:22 smithi176 sudo[43923]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:33:23.105 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:22 smithi139 sudo[46807]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:33:23.105 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:22 smithi139 sudo[46807]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:33:23.106 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:22 smithi139 sudo[46807]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:33:23.106 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:22 smithi139 sudo[46807]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:33:23.368 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:23 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:23.368 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:23 smithi176 ceph-mon[35324]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T01:33:23.368 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:23 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:23.368 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:23 smithi176 ceph-mon[35324]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:33:23.369 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:23 smithi176 sudo[43926]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:33:23.369 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:23 smithi176 sudo[43926]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:33:23.369 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:23 smithi176 sudo[43926]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:33:23.369 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:23 smithi176 sudo[43926]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:33:23.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:23 smithi139 sudo[46993]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:33:23.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:23 smithi139 sudo[46993]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:33:23.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:23 smithi139 sudo[46993]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:33:23.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:23 smithi139 sudo[46993]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:33:23.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:23 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:23.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:23 smithi139 ceph-mon[31175]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T01:33:23.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:23 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:23.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:23 smithi139 ceph-mon[31175]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:33:23.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:23 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:23.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:23 smithi164 ceph-mon[35121]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T01:33:23.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:23 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:23.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:23 smithi164 ceph-mon[35121]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:33:23.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:23 smithi176 sudo[43929]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:33:23.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:23 smithi176 sudo[43929]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:33:23.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:23 smithi176 sudo[43929]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:33:23.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:23 smithi176 sudo[43929]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:33:24.106 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:23 smithi164 sudo[43294]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T01:33:24.106 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:23 smithi164 sudo[43294]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:33:24.106 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:23 smithi164 sudo[43294]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:33:24.106 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:23 smithi164 sudo[43294]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:33:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:33:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:33:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:33:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:33:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:33:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:33:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:33:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:33:24.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:24.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:33:24.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:33:24.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:33:24.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:33:24.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:33:24.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:24 smithi139 ceph-mon[31175]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:33:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:33:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:33:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:33:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:33:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:33:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:33:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:33:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:33:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:33:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:33:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:33:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:33:24.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:33:24.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:24 smithi176 ceph-mon[35324]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:33:24.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 sudo[43297]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T01:33:24.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 sudo[43297]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T01:33:24.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 sudo[43297]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T01:33:24.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 sudo[43297]: pam_unix(sudo:session): session closed for user root 2024-09-16T01:33:24.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:33:24.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:33:24.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:33:24.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:33:24.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:33:24.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:33:24.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:33:24.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:33:24.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:24.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:33:24.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T01:33:24.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:33:24.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:33:24.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:33:24.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:24 smithi164 ceph-mon[35121]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T01:33:26.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:26 smithi139 ceph-mon[31175]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:26.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:26 smithi176 ceph-mon[35324]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:26 smithi164 ceph-mon[35121]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:28.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:28 smithi176 ceph-mon[35324]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:28.708 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:28 smithi139 ceph-mon[31175]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:28 smithi164 ceph-mon[35121]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:30.660 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:30 smithi139 ceph-mon[31175]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:30.660 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:30 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T01:33:30.660 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:30 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:30.660 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:30 smithi139 ceph-mon[31175]: Deploying daemon osd.3 on smithi139 2024-09-16T01:33:30.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:30 smithi176 ceph-mon[35324]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:30.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:30 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T01:33:30.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:30 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:30.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:30 smithi176 ceph-mon[35324]: Deploying daemon osd.3 on smithi139 2024-09-16T01:33:30.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:30 smithi164 ceph-mon[35121]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:30.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:30 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T01:33:30.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:30 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:30.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:30 smithi164 ceph-mon[35121]: Deploying daemon osd.3 on smithi139 2024-09-16T01:33:32.664 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:32 smithi139 ceph-mon[31175]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:32.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:32 smithi176 ceph-mon[35324]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:32.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:32 smithi164 ceph-mon[35121]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:34.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:34 smithi176 ceph-mon[35324]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:34.765 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:34 smithi139 ceph-mon[31175]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:34.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:34 smithi164 ceph-mon[35121]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:36.732 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:36 smithi139 ceph-mon[31175]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:36.732 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:36 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:33:36.732 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:36 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:36.732 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:36 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:36.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:36 smithi164 ceph-mon[35121]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:36.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:36 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:33:36.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:36 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:36.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:36 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:36.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:36 smithi176 ceph-mon[35324]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:36.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:36 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:33:36.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:36 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:36.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:36 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:38 smithi164 ceph-mon[35121]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:38.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:38 smithi139 ceph-mon[31175]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:38.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:38 smithi176 ceph-mon[35324]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:40.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:39 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:40.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:39 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:40.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:39 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:40.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:39 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:33:40.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:39 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:40.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:39 smithi164 ceph-mon[35121]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:40.079 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:39 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:40.079 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:39 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:40.079 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:39 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:40.079 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:39 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:33:40.079 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:39 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:40.079 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:39 smithi139 ceph-mon[31175]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:40.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:40.117+0000 7fa6db7fe640 1 -- 172.21.15.139:0/3529718719 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa6ec06aed0 con 0x7fa6d40a31e0 2024-09-16T01:33:40.118 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 3 on host 'smithi139' 2024-09-16T01:33:40.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:40.119+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3529718719 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa6d40a31e0 msgr2=0x7fa6d40a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:33:40.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:40.119+0000 7fa6f0cac640 1 --2- 172.21.15.139:0/3529718719 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa6d40a31e0 0x7fa6d40a56a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec101b70 tx=0x7fa6dc005f90 comp rx=0 tx=0).stop 2024-09-16T01:33:40.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:40.119+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3529718719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ec101d10 msgr2=0x7fa6ec198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:33:40.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:40.119+0000 7fa6f0cac640 1 --2- 172.21.15.139:0/3529718719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ec101d10 0x7fa6ec198440 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009230 tx=0x7fa6e00042f0 comp rx=0 tx=0).stop 2024-09-16T01:33:40.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:40.119+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3529718719 shutdown_connections 2024-09-16T01:33:40.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:40.119+0000 7fa6f0cac640 1 --2- 172.21.15.139:0/3529718719 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa6d40a31e0 0x7fa6d40a56a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:40.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:40.119+0000 7fa6f0cac640 1 --2- 172.21.15.139:0/3529718719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ec1026d0 0x7fa6ec19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:40.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:40.119+0000 7fa6f0cac640 1 --2- 172.21.15.139:0/3529718719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ec101d10 0x7fa6ec198440 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:40.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:40.119+0000 7fa6f0cac640 1 --2- 172.21.15.139:0/3529718719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6ec100b10 0x7fa6ec197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:40.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:40.119+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3529718719 >> 172.21.15.139:0/3529718719 conn(0x7fa6ec0fc2a0 msgr2=0x7fa6ec06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:33:40.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:40.119+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3529718719 shutdown_connections 2024-09-16T01:33:40.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:33:40.119+0000 7fa6f0cac640 1 -- 172.21.15.139:0/3529718719 wait complete. 2024-09-16T01:33:40.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:39 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:40.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:39 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:40.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:39 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:40.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:39 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:33:40.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:39 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:40.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:39 smithi176 ceph-mon[35324]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:40.842 DEBUG:teuthology.orchestra.run.smithi139:osd.3> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T01:33:40.844 INFO:tasks.cephadm:Deploying osd.4 on smithi164 with /dev/vg_nvme/lv_3... 2024-09-16T01:33:40.844 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T01:33:41.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:40 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:33:41.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:40 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:41.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:40 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:41.105 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:33:41.116 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:40 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:33:41.116 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:40 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:41.116 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:40 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:41.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:40 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:33:41.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:40 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:41.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:40 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:42.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:41 smithi164 ceph-mon[35121]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:42.165 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:41 smithi139 ceph-mon[31175]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:42.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:41 smithi176 ceph-mon[35324]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:42.423 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 01:33:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3[48588]: 2024-09-16T01:33:42.357+0000 7f69ba24b640 -1 osd.3 0 log_to_monitors true 2024-09-16T01:33:42.991 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:42 smithi164 ceph-mon[35121]: from='osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T01:33:43.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:42 smithi139 ceph-mon[31175]: from='osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T01:33:43.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:42 smithi176 ceph-mon[35324]: from='osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T01:33:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:33:43.630 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_3 2024-09-16T01:33:43.883 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:33:43.907 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:43 smithi164 ceph-mon[35121]: from='osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T01:33:43.907 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:43 smithi164 ceph-mon[35121]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T01:33:43.907 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:43 smithi164 ceph-mon[35121]: from='osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-16T01:33:43.908 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:43 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:43.908 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:43 smithi164 ceph-mon[35121]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:44.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:43 smithi139 ceph-mon[31175]: from='osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T01:33:44.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:43 smithi139 ceph-mon[31175]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T01:33:44.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:43 smithi139 ceph-mon[31175]: from='osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-16T01:33:44.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:43 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:44.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:43 smithi139 ceph-mon[31175]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:44.173 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 01:33:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3[48588]: 2024-09-16T01:33:43.808+0000 7f69b61ce640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T01:33:44.173 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 01:33:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3[48588]: 2024-09-16T01:33:43.816+0000 7f69b1ff6640 -1 osd.3 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:33:44.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:43 smithi176 ceph-mon[35324]: from='osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T01:33:44.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:43 smithi176 ceph-mon[35324]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T01:33:44.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:43 smithi176 ceph-mon[35324]: from='osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-16T01:33:44.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:43 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:44.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:43 smithi176 ceph-mon[35324]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:44.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.321+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/1308109900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bc8104fd0 msgr2=0x7f2bc81073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:33:44.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.321+0000 7f2bcfc4f640 1 --2- 172.21.15.164:0/1308109900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bc8104fd0 0x7f2bc81073c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009900 tx=0x7f2bb802a410 comp rx=0 tx=0).stop 2024-09-16T01:33:44.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.321+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/1308109900 shutdown_connections 2024-09-16T01:33:44.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.321+0000 7f2bcfc4f640 1 --2- 172.21.15.164:0/1308109900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bc8107900 0x7f2bc8109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:44.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.321+0000 7f2bcfc4f640 1 --2- 172.21.15.164:0/1308109900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bc8104fd0 0x7f2bc81073c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:44.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.321+0000 7f2bcfc4f640 1 --2- 172.21.15.164:0/1308109900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bc81026a0 0x7f2bc8104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:44.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.321+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/1308109900 >> 172.21.15.164:0/1308109900 conn(0x7f2bc80fc280 msgr2=0x7f2bc80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:33:44.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.321+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/1308109900 shutdown_connections 2024-09-16T01:33:44.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.321+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/1308109900 wait complete. 2024-09-16T01:33:44.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.322+0000 7f2bcfc4f640 1 Processor -- start 2024-09-16T01:33:44.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.322+0000 7f2bcfc4f640 1 -- start start 2024-09-16T01:33:44.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.322+0000 7f2bcfc4f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bc81026a0 0x7f2bc8197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:33:44.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.322+0000 7f2bcfc4f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bc8104fd0 0x7f2bc81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:33:44.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.322+0000 7f2bcfc4f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bc8107900 0x7f2bc819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:33:44.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.322+0000 7f2bcfc4f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc810d860 con 0x7f2bc81026a0 2024-09-16T01:33:44.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.322+0000 7f2bcfc4f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc810d6e0 con 0x7f2bc8104fd0 2024-09-16T01:33:44.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.322+0000 7f2bcfc4f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc810d9e0 con 0x7f2bc8107900 2024-09-16T01:33:44.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bcd1ea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bc8104fd0 0x7f2bc81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:33:44.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bcd1ea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bc8104fd0 0x7f2bc81984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58066/0 (socket says 172.21.15.164:58066) 2024-09-16T01:33:44.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bcd1ea640 1 -- 172.21.15.164:0/4031982551 learned_addr learned my addr 172.21.15.164:0/4031982551 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T01:33:44.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bcd9eb640 1 --2- 172.21.15.164:0/4031982551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bc81026a0 0x7f2bc8197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:33:44.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bcd1ea640 1 -- 172.21.15.164:0/4031982551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bc8107900 msgr2=0x7f2bc819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:33:44.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bcd1ea640 1 --2- 172.21.15.164:0/4031982551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bc8107900 0x7f2bc819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:44.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bcd1ea640 1 -- 172.21.15.164:0/4031982551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bc81026a0 msgr2=0x7f2bc8197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:33:44.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bcd1ea640 1 --2- 172.21.15.164:0/4031982551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bc81026a0 0x7f2bc8197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:33:44.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bcd1ea640 1 -- 172.21.15.164:0/4031982551 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc819fbb0 con 0x7f2bc8104fd0 2024-09-16T01:33:44.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bcd1ea640 1 --2- 172.21.15.164:0/4031982551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bc8104fd0 0x7f2bc81984c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8031610 tx=0x7f2bb8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:33:44.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bbeffd640 1 -- 172.21.15.164:0/4031982551 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb803f030 con 0x7f2bc8104fd0 2024-09-16T01:33:44.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bbeffd640 1 -- 172.21.15.164:0/4031982551 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb8003ad0 con 0x7f2bc8104fd0 2024-09-16T01:33:44.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/4031982551 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc819fe40 con 0x7f2bc8104fd0 2024-09-16T01:33:44.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bbeffd640 1 -- 172.21.15.164:0/4031982551 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8035070 con 0x7f2bc8104fd0 2024-09-16T01:33:44.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.323+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/4031982551 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc81a0290 con 0x7f2bc8104fd0 2024-09-16T01:33:44.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.324+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/4031982551 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc80fdd70 con 0x7f2bc8104fd0 2024-09-16T01:33:44.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.325+0000 7f2bbeffd640 1 -- 172.21.15.164:0/4031982551 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2bb8042030 con 0x7f2bc8104fd0 2024-09-16T01:33:44.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.326+0000 7f2bbeffd640 1 --2- 172.21.15.164:0/4031982551 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2bb40a3270 0x7f2bb40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:33:44.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.326+0000 7f2bbeffd640 1 -- 172.21.15.164:0/4031982551 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3389+0+0 (secure 0 0 0) 0x7f2bb80eeaa0 con 0x7f2bc8104fd0 2024-09-16T01:33:44.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.326+0000 7f2bcd9eb640 1 --2- 172.21.15.164:0/4031982551 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2bb40a3270 0x7f2bb40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:33:44.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.327+0000 7f2bcd9eb640 1 --2- 172.21.15.164:0/4031982551 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2bb40a3270 0x7f2bb40a5730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009bd0 tx=0x7f2bc4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:33:44.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.331+0000 7f2bbeffd640 1 -- 172.21.15.164:0/4031982551 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2bb80bcb00 con 0x7f2bc8104fd0 2024-09-16T01:33:44.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:33:44.514+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/4031982551 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2bc8062630 con 0x7f2bb40a3270 2024-09-16T01:33:45.062 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-16T01:33:45.062 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T01:33:45.062 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:45.062 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:45.063 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: Detected new or changed devices on smithi139 2024-09-16T01:33:45.063 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:45.063 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:45.063 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:33:45.063 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:33:45.063 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi139 to 8544M 2024-09-16T01:33:45.063 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:45.063 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:45.063 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:33:45.063 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:45.063 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:33:45.063 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:33:45.063 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:33:45.063 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:45.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-16T01:33:45.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T01:33:45.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:45.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:45.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: Detected new or changed devices on smithi139 2024-09-16T01:33:45.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:45.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:45.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:33:45.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:33:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi139 to 8544M 2024-09-16T01:33:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:33:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:33:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:33:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:33:45.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:45.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-16T01:33:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T01:33:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: Detected new or changed devices on smithi139 2024-09-16T01:33:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:33:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:33:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi139 to 8544M 2024-09-16T01:33:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:45.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:33:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:33:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:33:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:33:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:33:45.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:45.901 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:45 smithi164 ceph-mon[35121]: purged_snaps scrub starts 2024-09-16T01:33:45.901 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:45 smithi164 ceph-mon[35121]: purged_snaps scrub ok 2024-09-16T01:33:45.902 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:45 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:45.902 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:45 smithi164 ceph-mon[35121]: osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071] boot 2024-09-16T01:33:45.902 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:45 smithi164 ceph-mon[35121]: osdmap e22: 4 total, 4 up, 4 in 2024-09-16T01:33:45.902 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:45 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:45.902 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:45 smithi164 ceph-mon[35121]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:45.902 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:45 smithi164 ceph-mon[35121]: osdmap e23: 4 total, 4 up, 4 in 2024-09-16T01:33:46.088 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:45 smithi176 ceph-mon[35324]: purged_snaps scrub starts 2024-09-16T01:33:46.089 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:45 smithi176 ceph-mon[35324]: purged_snaps scrub ok 2024-09-16T01:33:46.089 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:45 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:46.089 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:45 smithi176 ceph-mon[35324]: osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071] boot 2024-09-16T01:33:46.089 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:45 smithi176 ceph-mon[35324]: osdmap e22: 4 total, 4 up, 4 in 2024-09-16T01:33:46.089 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:45 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:46.089 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:45 smithi176 ceph-mon[35324]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:46.089 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:45 smithi176 ceph-mon[35324]: osdmap e23: 4 total, 4 up, 4 in 2024-09-16T01:33:46.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:45 smithi139 ceph-mon[31175]: purged_snaps scrub starts 2024-09-16T01:33:46.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:45 smithi139 ceph-mon[31175]: purged_snaps scrub ok 2024-09-16T01:33:46.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:45 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:46.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:45 smithi139 ceph-mon[31175]: osd.3 [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071] boot 2024-09-16T01:33:46.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:45 smithi139 ceph-mon[31175]: osdmap e22: 4 total, 4 up, 4 in 2024-09-16T01:33:46.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:45 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:33:46.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:45 smithi139 ceph-mon[31175]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-16T01:33:46.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:45 smithi139 ceph-mon[31175]: osdmap e23: 4 total, 4 up, 4 in 2024-09-16T01:33:47.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:47 smithi164 ceph-mon[35121]: osdmap e24: 4 total, 4 up, 4 in 2024-09-16T01:33:47.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:47 smithi164 ceph-mon[35121]: from='client.? 172.21.15.164:0/600870207' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cd76696-8161-4e82-a571-29e98c11d5a0"}]: dispatch 2024-09-16T01:33:47.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:47 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cd76696-8161-4e82-a571-29e98c11d5a0"}]: dispatch 2024-09-16T01:33:47.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:47 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cd76696-8161-4e82-a571-29e98c11d5a0"}]': finished 2024-09-16T01:33:47.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:47 smithi164 ceph-mon[35121]: osdmap e25: 5 total, 4 up, 5 in 2024-09-16T01:33:47.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:47 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:33:47.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:47 smithi164 ceph-mon[35121]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:33:47.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:47 smithi164 ceph-mon[35121]: from='client.? 172.21.15.164:0/3543198882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:33:48.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:47 smithi139 ceph-mon[31175]: osdmap e24: 4 total, 4 up, 4 in 2024-09-16T01:33:48.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:47 smithi139 ceph-mon[31175]: from='client.? 172.21.15.164:0/600870207' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cd76696-8161-4e82-a571-29e98c11d5a0"}]: dispatch 2024-09-16T01:33:48.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:47 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cd76696-8161-4e82-a571-29e98c11d5a0"}]: dispatch 2024-09-16T01:33:48.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:47 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cd76696-8161-4e82-a571-29e98c11d5a0"}]': finished 2024-09-16T01:33:48.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:47 smithi139 ceph-mon[31175]: osdmap e25: 5 total, 4 up, 5 in 2024-09-16T01:33:48.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:47 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:33:48.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:47 smithi139 ceph-mon[31175]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:33:48.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:47 smithi139 ceph-mon[31175]: from='client.? 172.21.15.164:0/3543198882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:33:48.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:47 smithi176 ceph-mon[35324]: osdmap e24: 4 total, 4 up, 4 in 2024-09-16T01:33:48.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:47 smithi176 ceph-mon[35324]: from='client.? 172.21.15.164:0/600870207' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cd76696-8161-4e82-a571-29e98c11d5a0"}]: dispatch 2024-09-16T01:33:48.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:47 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cd76696-8161-4e82-a571-29e98c11d5a0"}]: dispatch 2024-09-16T01:33:48.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:47 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cd76696-8161-4e82-a571-29e98c11d5a0"}]': finished 2024-09-16T01:33:48.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:47 smithi176 ceph-mon[35324]: osdmap e25: 5 total, 4 up, 5 in 2024-09-16T01:33:48.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:47 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:33:48.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:47 smithi176 ceph-mon[35324]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:33:48.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:47 smithi176 ceph-mon[35324]: from='client.? 172.21.15.164:0/3543198882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:33:49.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:48 smithi139 ceph-mon[31175]: osdmap e26: 5 total, 4 up, 5 in 2024-09-16T01:33:49.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:48 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:33:49.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:48 smithi176 ceph-mon[35324]: osdmap e26: 5 total, 4 up, 5 in 2024-09-16T01:33:49.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:48 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:33:49.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:48 smithi164 ceph-mon[35121]: osdmap e26: 5 total, 4 up, 5 in 2024-09-16T01:33:49.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:33:50.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:49 smithi139 ceph-mon[31175]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:33:50.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:49 smithi176 ceph-mon[35324]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:33:50.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:49 smithi164 ceph-mon[35121]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:33:52.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:52 smithi176 ceph-mon[35324]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T01:33:52.726 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:52 smithi164 ceph-mon[35121]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T01:33:52.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:52 smithi139 ceph-mon[31175]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T01:33:54.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:54 smithi176 ceph-mon[35324]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-16T01:33:54.730 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:54 smithi164 ceph-mon[35121]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-16T01:33:54.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:54 smithi139 ceph-mon[31175]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-16T01:33:55.767 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T01:33:55.768 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:55.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T01:33:55.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:55.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T01:33:55.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:33:56.775 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:56 smithi164 ceph-mon[35121]: Deploying daemon osd.4 on smithi164 2024-09-16T01:33:56.776 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:56 smithi164 ceph-mon[35121]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T01:33:56.830 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:56 smithi176 ceph-mon[35324]: Deploying daemon osd.4 on smithi164 2024-09-16T01:33:56.830 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:56 smithi176 ceph-mon[35324]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T01:33:56.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:56 smithi139 ceph-mon[31175]: Deploying daemon osd.4 on smithi164 2024-09-16T01:33:56.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:56 smithi139 ceph-mon[31175]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T01:33:58.714 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:33:58 smithi164 ceph-mon[35121]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:33:58.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:33:58 smithi139 ceph-mon[31175]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:33:58.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:33:58 smithi176 ceph-mon[35324]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:34:00.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:00 smithi164 ceph-mon[35121]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-16T01:34:00.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:00 smithi139 ceph-mon[31175]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-16T01:34:00.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:00 smithi176 ceph-mon[35324]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-16T01:34:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:01 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:01 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:01 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:01.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:01 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:01.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:01 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:01.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:01 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:01.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:01 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:01.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:01 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:01.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:01 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:02 smithi164 ceph-mon[35121]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:34:02.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:02 smithi139 ceph-mon[31175]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:34:02.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:02 smithi176 ceph-mon[35324]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:34:04.725 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:04 smithi139 ceph-mon[31175]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:34:04.725 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:04.725 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:04.725 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:04.725 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:04.725 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:04 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:04 smithi164 ceph-mon[35121]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:34:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:04 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:04.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:04 smithi176 ceph-mon[35324]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:34:04.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:04.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:04.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:04.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:04.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:04 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:05.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:34:05.327+0000 7f2bbeffd640 1 -- 172.21.15.164:0/4031982551 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2bc8062630 con 0x7f2bb40a3270 2024-09-16T01:34:05.329 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 4 on host 'smithi164' 2024-09-16T01:34:05.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:34:05.329+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/4031982551 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2bb40a3270 msgr2=0x7f2bb40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:34:05.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:34:05.329+0000 7f2bcfc4f640 1 --2- 172.21.15.164:0/4031982551 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2bb40a3270 0x7f2bb40a5730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009bd0 tx=0x7f2bc4005f90 comp rx=0 tx=0).stop 2024-09-16T01:34:05.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:34:05.329+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/4031982551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bc8104fd0 msgr2=0x7f2bc81984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:34:05.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:34:05.329+0000 7f2bcfc4f640 1 --2- 172.21.15.164:0/4031982551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bc8104fd0 0x7f2bc81984c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8031610 tx=0x7f2bb8004270 comp rx=0 tx=0).stop 2024-09-16T01:34:05.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:34:05.329+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/4031982551 shutdown_connections 2024-09-16T01:34:05.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:34:05.329+0000 7f2bcfc4f640 1 --2- 172.21.15.164:0/4031982551 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2bb40a3270 0x7f2bb40a5730 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:05.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:34:05.329+0000 7f2bcfc4f640 1 --2- 172.21.15.164:0/4031982551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bc8107900 0x7f2bc819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:05.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:34:05.329+0000 7f2bcfc4f640 1 --2- 172.21.15.164:0/4031982551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2bc8104fd0 0x7f2bc81984c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:05.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:34:05.329+0000 7f2bcfc4f640 1 --2- 172.21.15.164:0/4031982551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bc81026a0 0x7f2bc8197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:05.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:34:05.329+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/4031982551 >> 172.21.15.164:0/4031982551 conn(0x7f2bc80fc280 msgr2=0x7f2bc8103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:34:05.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:34:05.329+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/4031982551 shutdown_connections 2024-09-16T01:34:05.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:34:05.329+0000 7f2bcfc4f640 1 -- 172.21.15.164:0/4031982551 wait complete. 2024-09-16T01:34:05.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:05 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:05.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:05 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:05.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:05 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:05.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:05 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:05.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:05 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:05.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:05 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:05.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:05 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:05.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:05 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:05.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:05 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:05.961 DEBUG:teuthology.orchestra.run.smithi164:osd.4> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T01:34:05.964 INFO:tasks.cephadm:Deploying osd.5 on smithi176 with /dev/vg_nvme/lv_3... 2024-09-16T01:34:05.964 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T01:34:06.220 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:34:06.644 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:06 smithi176 ceph-mon[35324]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:34:06.774 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:06 smithi164 ceph-mon[35121]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:34:06.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:06 smithi139 ceph-mon[31175]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:34:07.045 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 01:34:06 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4[46048]: 2024-09-16T01:34:06.994+0000 7f56cfa44640 -1 osd.4 0 log_to_monitors true 2024-09-16T01:34:07.679 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:07 smithi176 ceph-mon[35324]: from='osd.4 [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:34:07.679 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:07 smithi176 ceph-mon[35324]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:34:07.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:07 smithi164 ceph-mon[35121]: from='osd.4 [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:34:07.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:07 smithi164 ceph-mon[35121]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:34:07.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:07 smithi139 ceph-mon[31175]: from='osd.4 [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:34:07.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:07 smithi139 ceph-mon[31175]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T01:34:08.608 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:08 smithi164 ceph-mon[35121]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:34:08.608 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:08 smithi164 ceph-mon[35121]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T01:34:08.608 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:08 smithi164 ceph-mon[35121]: from='osd.4 [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:34:08.608 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:08 smithi164 ceph-mon[35121]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T01:34:08.608 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:08 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:08.608 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:08 smithi164 ceph-mon[35121]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:34:08.739 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:08 smithi176 ceph-mon[35324]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:34:08.739 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:08 smithi176 ceph-mon[35324]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T01:34:08.739 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:08 smithi176 ceph-mon[35324]: from='osd.4 [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:34:08.739 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:08 smithi176 ceph-mon[35324]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T01:34:08.740 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:08 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:08.740 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:08 smithi176 ceph-mon[35324]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:34:08.742 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:34:08.772 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_3 2024-09-16T01:34:08.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:08 smithi139 ceph-mon[31175]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:34:08.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:08 smithi139 ceph-mon[31175]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T01:34:08.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:08 smithi139 ceph-mon[31175]: from='osd.4 [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:34:08.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:08 smithi139 ceph-mon[31175]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T01:34:08.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:08 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:08.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:08 smithi139 ceph-mon[31175]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:34:09.027 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:34:09.295 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 01:34:08 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4[46048]: 2024-09-16T01:34:08.872+0000 7f56cb9c7640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T01:34:09.295 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 01:34:08 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4[46048]: 2024-09-16T01:34:08.877+0000 7f56c37e7640 -1 osd.4 28 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:34:09.508 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.484+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/4007387590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ae0102df0 msgr2=0x7f1ae0103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:34:09.508 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.484+0000 7f1ae6a9d640 1 --2- 172.21.15.176:0/4007387590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ae0102df0 0x7f1ae0103270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009900 tx=0x7f1acc02a410 comp rx=0 tx=0).stop 2024-09-16T01:34:09.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.484+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/4007387590 shutdown_connections 2024-09-16T01:34:09.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.484+0000 7f1ae6a9d640 1 --2- 172.21.15.176:0/4007387590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ae01037b0 0x7f1ae010a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:09.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.484+0000 7f1ae6a9d640 1 --2- 172.21.15.176:0/4007387590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ae0102df0 0x7f1ae0103270 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:09.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.484+0000 7f1ae6a9d640 1 --2- 172.21.15.176:0/4007387590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ae0101bf0 0x7f1ae0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:09.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.484+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/4007387590 >> 172.21.15.176:0/4007387590 conn(0x7f1ae00fd3c0 msgr2=0x7f1ae00ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:34:09.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.484+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/4007387590 shutdown_connections 2024-09-16T01:34:09.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.484+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/4007387590 wait complete. 2024-09-16T01:34:09.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae6a9d640 1 Processor -- start 2024-09-16T01:34:09.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae6a9d640 1 -- start start 2024-09-16T01:34:09.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae6a9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ae0101bf0 0x7f1ae0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:34:09.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae6a9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ae0102df0 0x7f1ae00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:34:09.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae6a9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ae01037b0 0x7f1ae0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:34:09.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae6a9d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae010d920 con 0x7f1ae0102df0 2024-09-16T01:34:09.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae6a9d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae010d7a0 con 0x7f1ae0101bf0 2024-09-16T01:34:09.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae6a9d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae010daa0 con 0x7f1ae01037b0 2024-09-16T01:34:09.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae503a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ae01037b0 0x7f1ae0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:34:09.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae503a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ae01037b0 0x7f1ae0074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:48026/0 (socket says 172.21.15.176:48026) 2024-09-16T01:34:09.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae503a640 1 -- 172.21.15.176:0/1697949123 learned_addr learned my addr 172.21.15.176:0/1697949123 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-16T01:34:09.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1adffff640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ae0102df0 0x7f1ae00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:34:09.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae4839640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ae0101bf0 0x7f1ae0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:34:09.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae503a640 1 -- 172.21.15.176:0/1697949123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ae0101bf0 msgr2=0x7f1ae0075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:34:09.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae503a640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ae0101bf0 0x7f1ae0075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:09.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae503a640 1 -- 172.21.15.176:0/1697949123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ae0102df0 msgr2=0x7f1ae00745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:34:09.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae503a640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ae0102df0 0x7f1ae00745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:09.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.485+0000 7f1ae503a640 1 -- 172.21.15.176:0/1697949123 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae00753e0 con 0x7f1ae01037b0 2024-09-16T01:34:09.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.486+0000 7f1ae4839640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ae0101bf0 0x7f1ae0075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:34:09.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.486+0000 7f1ae503a640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ae01037b0 0x7f1ae0074b20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800b970 tx=0x7f1ad800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:34:09.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.486+0000 7f1addffb640 1 -- 172.21.15.176:0/1697949123 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8019070 con 0x7f1ae01037b0 2024-09-16T01:34:09.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.486+0000 7f1addffb640 1 -- 172.21.15.176:0/1697949123 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad800cb40 con 0x7f1ae01037b0 2024-09-16T01:34:09.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.486+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/1697949123 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae01a20a0 con 0x7f1ae01037b0 2024-09-16T01:34:09.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.486+0000 7f1addffb640 1 -- 172.21.15.176:0/1697949123 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8007650 con 0x7f1ae01037b0 2024-09-16T01:34:09.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.486+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/1697949123 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae01a25e0 con 0x7f1ae01037b0 2024-09-16T01:34:09.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.487+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/1697949123 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae0108fa0 con 0x7f1ae01037b0 2024-09-16T01:34:09.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.488+0000 7f1addffb640 1 -- 172.21.15.176:0/1697949123 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f1ad801c070 con 0x7f1ae01037b0 2024-09-16T01:34:09.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.488+0000 7f1addffb640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1ad00a3240 0x7f1ad00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:34:09.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.489+0000 7f1addffb640 1 -- 172.21.15.176:0/1697949123 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3821+0+0 (secure 0 0 0) 0x7f1ad80cdd60 con 0x7f1ae01037b0 2024-09-16T01:34:09.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.489+0000 7f1ae4839640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1ad00a3240 0x7f1ad00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:34:09.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.489+0000 7f1ae4839640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1ad00a3240 0x7f1ad00a5700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0102c50 tx=0x7f1ad4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:34:09.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.493+0000 7f1addffb640 1 -- 172.21.15.176:0/1697949123 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ad809b5c0 con 0x7f1ae01037b0 2024-09-16T01:34:09.669 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:09.668+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/1697949123 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1ae0062630 con 0x7f1ad00a3240 2024-09-16T01:34:09.669 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:09 smithi176 ceph-mon[35324]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T01:34:09.670 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:09 smithi176 ceph-mon[35324]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T01:34:09.670 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:09.670 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:09.670 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:09.670 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:09.670 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:09.670 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:09.670 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:09.670 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:09.670 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:09.670 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:09.670 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:09 smithi176 ceph-mon[35324]: from='osd.4 ' entity='osd.4' 2024-09-16T01:34:09.670 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:09 smithi164 ceph-mon[35121]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T01:34:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:09 smithi164 ceph-mon[35121]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T01:34:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:09.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:09.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:09.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:09.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:09.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:09.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:09.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:09.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:09 smithi164 ceph-mon[35121]: from='osd.4 ' entity='osd.4' 2024-09-16T01:34:09.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:09 smithi139 ceph-mon[31175]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T01:34:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:09 smithi139 ceph-mon[31175]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T01:34:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:09.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:09.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:09.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:09 smithi139 ceph-mon[31175]: from='osd.4 ' entity='osd.4' 2024-09-16T01:34:09.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:10.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:10 smithi176 ceph-mon[35324]: purged_snaps scrub starts 2024-09-16T01:34:10.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:10 smithi176 ceph-mon[35324]: purged_snaps scrub ok 2024-09-16T01:34:10.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:10 smithi176 ceph-mon[35324]: Detected new or changed devices on smithi164 2024-09-16T01:34:10.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:10 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi164 to 8544M 2024-09-16T01:34:10.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:10 smithi176 ceph-mon[35324]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:34:10.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:10 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:34:10.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:10 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:34:10.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:10 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:10.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:10 smithi176 ceph-mon[35324]: osd.4 [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792] boot 2024-09-16T01:34:10.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:10 smithi176 ceph-mon[35324]: osdmap e29: 5 total, 5 up, 5 in 2024-09-16T01:34:10.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:10 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:10 smithi164 ceph-mon[35121]: purged_snaps scrub starts 2024-09-16T01:34:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:10 smithi164 ceph-mon[35121]: purged_snaps scrub ok 2024-09-16T01:34:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:10 smithi164 ceph-mon[35121]: Detected new or changed devices on smithi164 2024-09-16T01:34:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:10 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi164 to 8544M 2024-09-16T01:34:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:10 smithi164 ceph-mon[35121]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:34:10.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:10 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:34:10.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:10 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:34:10.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:10 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:10.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:10 smithi164 ceph-mon[35121]: osd.4 [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792] boot 2024-09-16T01:34:10.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:10 smithi164 ceph-mon[35121]: osdmap e29: 5 total, 5 up, 5 in 2024-09-16T01:34:10.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:10 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:10.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:10 smithi139 ceph-mon[31175]: purged_snaps scrub starts 2024-09-16T01:34:10.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:10 smithi139 ceph-mon[31175]: purged_snaps scrub ok 2024-09-16T01:34:10.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:10 smithi139 ceph-mon[31175]: Detected new or changed devices on smithi164 2024-09-16T01:34:10.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:10 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi164 to 8544M 2024-09-16T01:34:10.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:10 smithi139 ceph-mon[31175]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-16T01:34:10.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:10 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:34:10.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:10 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:34:10.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:10 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:10.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:10 smithi139 ceph-mon[31175]: osd.4 [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792] boot 2024-09-16T01:34:10.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:10 smithi139 ceph-mon[31175]: osdmap e29: 5 total, 5 up, 5 in 2024-09-16T01:34:10.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:10 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:34:11.640 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:11 smithi176 ceph-mon[35324]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:34:11.640 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:11 smithi176 ceph-mon[35324]: osdmap e30: 5 total, 5 up, 5 in 2024-09-16T01:34:11.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:11 smithi164 ceph-mon[35121]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:34:11.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:11 smithi164 ceph-mon[35121]: osdmap e30: 5 total, 5 up, 5 in 2024-09-16T01:34:11.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:11 smithi139 ceph-mon[31175]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:34:11.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:11 smithi139 ceph-mon[31175]: osdmap e30: 5 total, 5 up, 5 in 2024-09-16T01:34:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:12 smithi164 ceph-mon[35121]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:12 smithi164 ceph-mon[35121]: from='client.? 172.21.15.176:0/2106092922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccc17a29-9a58-40ec-b453-5c38bee86f25"}]: dispatch 2024-09-16T01:34:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:12 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccc17a29-9a58-40ec-b453-5c38bee86f25"}]: dispatch 2024-09-16T01:34:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:12 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccc17a29-9a58-40ec-b453-5c38bee86f25"}]': finished 2024-09-16T01:34:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:12 smithi164 ceph-mon[35121]: osdmap e31: 6 total, 5 up, 6 in 2024-09-16T01:34:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:12 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:12 smithi139 ceph-mon[31175]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:12 smithi139 ceph-mon[31175]: from='client.? 172.21.15.176:0/2106092922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccc17a29-9a58-40ec-b453-5c38bee86f25"}]: dispatch 2024-09-16T01:34:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:12 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccc17a29-9a58-40ec-b453-5c38bee86f25"}]: dispatch 2024-09-16T01:34:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:12 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccc17a29-9a58-40ec-b453-5c38bee86f25"}]': finished 2024-09-16T01:34:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:12 smithi139 ceph-mon[31175]: osdmap e31: 6 total, 5 up, 6 in 2024-09-16T01:34:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:12 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:12.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:12 smithi176 ceph-mon[35324]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:12.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:12 smithi176 ceph-mon[35324]: from='client.? 172.21.15.176:0/2106092922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccc17a29-9a58-40ec-b453-5c38bee86f25"}]: dispatch 2024-09-16T01:34:12.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:12 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccc17a29-9a58-40ec-b453-5c38bee86f25"}]: dispatch 2024-09-16T01:34:12.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:12 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccc17a29-9a58-40ec-b453-5c38bee86f25"}]': finished 2024-09-16T01:34:12.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:12 smithi176 ceph-mon[35324]: osdmap e31: 6 total, 5 up, 6 in 2024-09-16T01:34:12.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:12 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:13.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:13 smithi164 ceph-mon[35121]: from='client.? 172.21.15.176:0/2815281399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:34:13.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:13 smithi139 ceph-mon[31175]: from='client.? 172.21.15.176:0/2815281399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:34:13.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:13 smithi176 ceph-mon[35324]: from='client.? 172.21.15.176:0/2815281399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:34:14.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:14 smithi164 ceph-mon[35121]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:14.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:14 smithi139 ceph-mon[31175]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:14.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:14 smithi176 ceph-mon[35324]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:16.750 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:16 smithi176 ceph-mon[35324]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:16.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:16 smithi164 ceph-mon[35121]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:16.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:16 smithi139 ceph-mon[31175]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:18.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:18 smithi176 ceph-mon[35324]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:18.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:18 smithi164 ceph-mon[35121]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:18.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:18 smithi139 ceph-mon[31175]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:20.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:20 smithi164 ceph-mon[35121]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:20.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:20 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T01:34:20.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:20 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:20.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:20 smithi176 ceph-mon[35324]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:20.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:20 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T01:34:20.799 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:20 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:20 smithi139 ceph-mon[31175]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:20 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T01:34:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:20 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:21.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:21 smithi164 ceph-mon[35121]: Deploying daemon osd.5 on smithi176 2024-09-16T01:34:21.852 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:21 smithi176 ceph-mon[35324]: Deploying daemon osd.5 on smithi176 2024-09-16T01:34:21.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:21 smithi139 ceph-mon[31175]: Deploying daemon osd.5 on smithi176 2024-09-16T01:34:22.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:22 smithi164 ceph-mon[35121]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:22.805 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:22 smithi176 ceph-mon[35324]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:22.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:22 smithi139 ceph-mon[31175]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:24 smithi164 ceph-mon[35121]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:24.888 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:24 smithi176 ceph-mon[35324]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:24.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:24 smithi139 ceph-mon[31175]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:26.635 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:26 smithi176 ceph-mon[35324]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:26.635 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:26 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:26.635 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:26 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:26.635 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:26 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:26 smithi164 ceph-mon[35121]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:26 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:26 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:26 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:26.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:26 smithi139 ceph-mon[31175]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:26.962 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:26 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:26.962 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:26 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:26.962 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:26 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:28 smithi164 ceph-mon[35121]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:28.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:28 smithi139 ceph-mon[31175]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:28.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:28 smithi176 ceph-mon[35324]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:30.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:30.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:30.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:30.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:30.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:30.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:29 smithi139 ceph-mon[31175]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:30.173 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:30.174 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:30.174 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:30.174 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:30.174 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:30.174 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:29 smithi176 ceph-mon[35324]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:30.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:30.214+0000 7f1addffb640 1 -- 172.21.15.176:0/1697949123 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1ae0062630 con 0x7f1ad00a3240 2024-09-16T01:34:30.216 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 5 on host 'smithi176' 2024-09-16T01:34:30.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:30.216+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/1697949123 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1ad00a3240 msgr2=0x7f1ad00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:34:30.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:30.216+0000 7f1ae6a9d640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1ad00a3240 0x7f1ad00a5700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0102c50 tx=0x7f1ad4005f90 comp rx=0 tx=0).stop 2024-09-16T01:34:30.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:30.216+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/1697949123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ae01037b0 msgr2=0x7f1ae0074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:34:30.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:30.216+0000 7f1ae6a9d640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ae01037b0 0x7f1ae0074b20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800b970 tx=0x7f1ad800be30 comp rx=0 tx=0).stop 2024-09-16T01:34:30.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:30.216+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/1697949123 shutdown_connections 2024-09-16T01:34:30.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:30.216+0000 7f1ae6a9d640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1ad00a3240 0x7f1ad00a5700 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:30.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:30.216+0000 7f1ae6a9d640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ae01037b0 0x7f1ae0074b20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:30.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:30.216+0000 7f1ae6a9d640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ae0102df0 0x7f1ae00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:30.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:30.216+0000 7f1ae6a9d640 1 --2- 172.21.15.176:0/1697949123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ae0101bf0 0x7f1ae0075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:30.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:30.216+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/1697949123 >> 172.21.15.176:0/1697949123 conn(0x7f1ae00fd3c0 msgr2=0x7f1ae0106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:34:30.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:30.216+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/1697949123 shutdown_connections 2024-09-16T01:34:30.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:34:30.217+0000 7f1ae6a9d640 1 -- 172.21.15.176:0/1697949123 wait complete. 2024-09-16T01:34:30.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:30.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:30.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:30.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:30.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:30.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:29 smithi164 ceph-mon[35121]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:30.795 DEBUG:teuthology.orchestra.run.smithi176:osd.5> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T01:34:30.798 INFO:tasks.cephadm:Deploying osd.6 on smithi139 with /dev/vg_nvme/lv_2... 2024-09-16T01:34:30.798 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T01:34:31.057 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:34:31.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:30 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:31.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:30 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:31.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:30 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:31.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:30 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:31.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:30 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:31.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:30 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:31.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:30 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:31.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:30 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:31.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:30 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:32.035 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:31 smithi176 ceph-mon[35324]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:32.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:31 smithi139 ceph-mon[31175]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:32.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:31 smithi164 ceph-mon[35121]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:33.081 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:32 smithi176 ceph-mon[35324]: from='osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T01:34:33.081 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 01:34:32 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T01:34:32.805+0000 7f493c5d8640 -1 osd.5 0 log_to_monitors true 2024-09-16T01:34:33.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:32 smithi139 ceph-mon[31175]: from='osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T01:34:33.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:32 smithi164 ceph-mon[35121]: from='osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T01:34:33.595 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:34:33.622 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_2 2024-09-16T01:34:33.875 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:34:34.184 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: from='osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T01:34:34.184 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: osdmap e32: 6 total, 5 up, 6 in 2024-09-16T01:34:34.184 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: from='osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:34:34.184 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:34.184 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:34.185 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: Detected new or changed devices on smithi176 2024-09-16T01:34:34.185 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:34.185 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:34.185 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:34.185 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:34.185 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi176 to 8544M 2024-09-16T01:34:34.185 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:34.185 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:34.185 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:34.185 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:33 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:34.186 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T01:34:33.926+0000 7f493855b640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T01:34:34.186 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T01:34:33.933+0000 7f4934383640 -1 osd.5 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:34:34.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: from='osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T01:34:34.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: osdmap e32: 6 total, 5 up, 6 in 2024-09-16T01:34:34.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: from='osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:34:34.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:34.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:34.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: Detected new or changed devices on smithi176 2024-09-16T01:34:34.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:34.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:34.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:34.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:34.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi176 to 8544M 2024-09-16T01:34:34.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:34.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:34.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:34.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:33 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:34.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: from='osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T01:34:34.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: osdmap e32: 6 total, 5 up, 6 in 2024-09-16T01:34:34.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: from='osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:34:34.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:34.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:34.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: Detected new or changed devices on smithi176 2024-09-16T01:34:34.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:34.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:34.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:34.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:34.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi176 to 8544M 2024-09-16T01:34:34.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:34.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:34.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:34.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:33 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:34.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.395+0000 7f2d7c812640 1 -- 172.21.15.139:0/1113128934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d781037c0 msgr2=0x7f2d7810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:34:34.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.395+0000 7f2d7c812640 1 --2- 172.21.15.139:0/1113128934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d781037c0 0x7f2d7810a050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009ca0 tx=0x7f2d7002a8c0 comp rx=0 tx=0).stop 2024-09-16T01:34:34.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.395+0000 7f2d7c812640 1 -- 172.21.15.139:0/1113128934 shutdown_connections 2024-09-16T01:34:34.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.395+0000 7f2d7c812640 1 --2- 172.21.15.139:0/1113128934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d781037c0 0x7f2d7810a050 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:34.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.395+0000 7f2d7c812640 1 --2- 172.21.15.139:0/1113128934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d78102e00 0x7f2d78103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:34.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.395+0000 7f2d7c812640 1 --2- 172.21.15.139:0/1113128934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d78101c00 0x7f2d78102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:34.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.395+0000 7f2d7c812640 1 -- 172.21.15.139:0/1113128934 >> 172.21.15.139:0/1113128934 conn(0x7f2d780fd390 msgr2=0x7f2d780ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:34:34.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.395+0000 7f2d7c812640 1 -- 172.21.15.139:0/1113128934 shutdown_connections 2024-09-16T01:34:34.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.396+0000 7f2d7c812640 1 -- 172.21.15.139:0/1113128934 wait complete. 2024-09-16T01:34:34.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.396+0000 7f2d7c812640 1 Processor -- start 2024-09-16T01:34:34.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.396+0000 7f2d7c812640 1 -- start start 2024-09-16T01:34:34.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.396+0000 7f2d7c812640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d78101c00 0x7f2d78075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:34:34.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.396+0000 7f2d7c812640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d78102e00 0x7f2d780745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:34:34.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.396+0000 7f2d7c812640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d781037c0 0x7f2d78074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:34:34.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d7c812640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7810d980 con 0x7f2d78102e00 2024-09-16T01:34:34.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d7c812640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7810d800 con 0x7f2d78101c00 2024-09-16T01:34:34.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d7c812640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7810db00 con 0x7f2d781037c0 2024-09-16T01:34:34.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d75d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d78102e00 0x7f2d780745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:34:34.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d75d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d78102e00 0x7f2d780745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45460/0 (socket says 172.21.15.139:45460) 2024-09-16T01:34:34.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d75d9b640 1 -- 172.21.15.139:0/2761775524 learned_addr learned my addr 172.21.15.139:0/2761775524 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:34:34.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d7659c640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d78101c00 0x7f2d78075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:34:34.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d75d9b640 1 -- 172.21.15.139:0/2761775524 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d781037c0 msgr2=0x7f2d78074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:34:34.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d76d9d640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d781037c0 0x7f2d78074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:34:34.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d75d9b640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d781037c0 0x7f2d78074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:34.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d75d9b640 1 -- 172.21.15.139:0/2761775524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d78101c00 msgr2=0x7f2d78075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:34:34.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d75d9b640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d78101c00 0x7f2d78075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:34.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d75d9b640 1 -- 172.21.15.139:0/2761775524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d780753e0 con 0x7f2d78102e00 2024-09-16T01:34:34.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d7659c640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d78101c00 0x7f2d78075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:34:34.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d76d9d640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d781037c0 0x7f2d78074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:34:34.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d75d9b640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d78102e00 0x7f2d780745e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800b4a0 tx=0x7f2d6800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:34:34.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d6f7fe640 1 -- 172.21.15.139:0/2761775524 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68014070 con 0x7f2d78102e00 2024-09-16T01:34:34.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d6f7fe640 1 -- 172.21.15.139:0/2761775524 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d68010af0 con 0x7f2d78102e00 2024-09-16T01:34:34.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d7c812640 1 -- 172.21.15.139:0/2761775524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d781a21b0 con 0x7f2d78102e00 2024-09-16T01:34:34.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d6f7fe640 1 -- 172.21.15.139:0/2761775524 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68017d10 con 0x7f2d78102e00 2024-09-16T01:34:34.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.397+0000 7f2d7c812640 1 -- 172.21.15.139:0/2761775524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d781a2740 con 0x7f2d78102e00 2024-09-16T01:34:34.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.399+0000 7f2d7c812640 1 -- 172.21.15.139:0/2761775524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d78109000 con 0x7f2d78102e00 2024-09-16T01:34:34.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.400+0000 7f2d6f7fe640 1 -- 172.21.15.139:0/2761775524 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2d68017070 con 0x7f2d78102e00 2024-09-16T01:34:34.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.400+0000 7f2d6f7fe640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d640a3240 0x7f2d640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:34:34.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.400+0000 7f2d7659c640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d640a3240 0x7f2d640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:34:34.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.401+0000 7f2d6f7fe640 1 -- 172.21.15.139:0/2761775524 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4253+0+0 (secure 0 0 0) 0x7f2d680cdd40 con 0x7f2d78102e00 2024-09-16T01:34:34.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.401+0000 7f2d7659c640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d640a3240 0x7f2d640a5700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d60006fd0 tx=0x7f2d60008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:34:34.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.405+0000 7f2d6f7fe640 1 -- 172.21.15.139:0/2761775524 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d6809ba40 con 0x7f2d78102e00 2024-09-16T01:34:34.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:34.578+0000 7f2d7c812640 1 -- 172.21.15.139:0/2761775524 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2d78062630 con 0x7f2d640a3240 2024-09-16T01:34:35.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:34 smithi176 ceph-mon[35324]: from='osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-16T01:34:35.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:34 smithi176 ceph-mon[35324]: osdmap e33: 6 total, 5 up, 6 in 2024-09-16T01:34:35.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:34 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:35.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:34 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:35.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:34 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:34:35.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:34 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:34:35.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:34 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:35.240 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:34 smithi139 ceph-mon[31175]: from='osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-16T01:34:35.240 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:34 smithi139 ceph-mon[31175]: osdmap e33: 6 total, 5 up, 6 in 2024-09-16T01:34:35.241 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:35.241 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:35.241 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:34:35.241 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:34:35.241 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:35.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:34 smithi164 ceph-mon[35121]: from='osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-16T01:34:35.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:34 smithi164 ceph-mon[35121]: osdmap e33: 6 total, 5 up, 6 in 2024-09-16T01:34:35.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:34 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:35.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:34 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:35.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:34 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:34:35.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:34 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:34:35.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:34 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:36.128 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:35 smithi139 ceph-mon[31175]: purged_snaps scrub starts 2024-09-16T01:34:36.128 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:35 smithi139 ceph-mon[31175]: purged_snaps scrub ok 2024-09-16T01:34:36.128 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:35 smithi139 ceph-mon[31175]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:34:36.128 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:36.128 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:35 smithi139 ceph-mon[31175]: osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602] boot 2024-09-16T01:34:36.128 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:35 smithi139 ceph-mon[31175]: osdmap e34: 6 total, 6 up, 6 in 2024-09-16T01:34:36.128 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:36.128 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:35 smithi139 ceph-mon[31175]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:36.129 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:35 smithi139 ceph-mon[31175]: osdmap e35: 6 total, 6 up, 6 in 2024-09-16T01:34:36.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:35 smithi176 ceph-mon[35324]: purged_snaps scrub starts 2024-09-16T01:34:36.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:35 smithi176 ceph-mon[35324]: purged_snaps scrub ok 2024-09-16T01:34:36.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:35 smithi176 ceph-mon[35324]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:34:36.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:35 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:36.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:35 smithi176 ceph-mon[35324]: osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602] boot 2024-09-16T01:34:36.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:35 smithi176 ceph-mon[35324]: osdmap e34: 6 total, 6 up, 6 in 2024-09-16T01:34:36.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:35 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:36.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:35 smithi176 ceph-mon[35324]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:36.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:35 smithi176 ceph-mon[35324]: osdmap e35: 6 total, 6 up, 6 in 2024-09-16T01:34:36.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:35 smithi164 ceph-mon[35121]: purged_snaps scrub starts 2024-09-16T01:34:36.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:35 smithi164 ceph-mon[35121]: purged_snaps scrub ok 2024-09-16T01:34:36.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:35 smithi164 ceph-mon[35121]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:34:36.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:35 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:36.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:35 smithi164 ceph-mon[35121]: osd.5 [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602] boot 2024-09-16T01:34:36.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:35 smithi164 ceph-mon[35121]: osdmap e34: 6 total, 6 up, 6 in 2024-09-16T01:34:36.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:35 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:34:36.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:35 smithi164 ceph-mon[35121]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-16T01:34:36.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:35 smithi164 ceph-mon[35121]: osdmap e35: 6 total, 6 up, 6 in 2024-09-16T01:34:38.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:37 smithi164 ceph-mon[35121]: osdmap e36: 6 total, 6 up, 6 in 2024-09-16T01:34:38.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:37 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3575203028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5ee7212-10c6-4a05-992e-622a14a5a6b5"}]: dispatch 2024-09-16T01:34:38.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:37 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3575203028' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e5ee7212-10c6-4a05-992e-622a14a5a6b5"}]': finished 2024-09-16T01:34:38.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:37 smithi164 ceph-mon[35121]: osdmap e37: 7 total, 6 up, 7 in 2024-09-16T01:34:38.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:37 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:34:38.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:37 smithi164 ceph-mon[35121]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:38.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:37 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3536879279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:34:38.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:37 smithi139 ceph-mon[31175]: osdmap e36: 6 total, 6 up, 6 in 2024-09-16T01:34:38.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:37 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3575203028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5ee7212-10c6-4a05-992e-622a14a5a6b5"}]: dispatch 2024-09-16T01:34:38.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:37 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3575203028' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e5ee7212-10c6-4a05-992e-622a14a5a6b5"}]': finished 2024-09-16T01:34:38.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:37 smithi139 ceph-mon[31175]: osdmap e37: 7 total, 6 up, 7 in 2024-09-16T01:34:38.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:37 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:34:38.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:37 smithi139 ceph-mon[31175]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:38.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:37 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3536879279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:34:38.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:37 smithi176 ceph-mon[35324]: osdmap e36: 6 total, 6 up, 6 in 2024-09-16T01:34:38.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:37 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3575203028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5ee7212-10c6-4a05-992e-622a14a5a6b5"}]: dispatch 2024-09-16T01:34:38.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:37 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3575203028' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e5ee7212-10c6-4a05-992e-622a14a5a6b5"}]': finished 2024-09-16T01:34:38.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:37 smithi176 ceph-mon[35324]: osdmap e37: 7 total, 6 up, 7 in 2024-09-16T01:34:38.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:37 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:34:38.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:37 smithi176 ceph-mon[35324]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:38.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:37 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3536879279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:34:39.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:39 smithi139 ceph-mon[31175]: osdmap e38: 7 total, 6 up, 7 in 2024-09-16T01:34:39.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:39 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:34:39.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:39 smithi176 ceph-mon[35324]: osdmap e38: 7 total, 6 up, 7 in 2024-09-16T01:34:39.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:39 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:34:39.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:39 smithi164 ceph-mon[35121]: osdmap e38: 7 total, 6 up, 7 in 2024-09-16T01:34:39.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:39 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:34:40.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:40 smithi139 ceph-mon[31175]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:40.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:40 smithi176 ceph-mon[35324]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:40.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:40 smithi164 ceph-mon[35121]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:42.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:42 smithi139 ceph-mon[31175]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T01:34:42.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:42 smithi164 ceph-mon[35121]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T01:34:42.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:42 smithi176 ceph-mon[35324]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T01:34:44.700 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:44 smithi139 ceph-mon[31175]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-16T01:34:44.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:44 smithi164 ceph-mon[35121]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-16T01:34:44.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:44 smithi176 ceph-mon[35324]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-16T01:34:46.540 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:46 smithi139 ceph-mon[31175]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-16T01:34:46.541 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:46 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T01:34:46.541 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:46 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:46.739 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:46 smithi176 ceph-mon[35324]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-16T01:34:46.739 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:46 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T01:34:46.740 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:46 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:46 smithi164 ceph-mon[35121]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-16T01:34:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:46 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T01:34:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:46 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:47.668 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:47 smithi139 ceph-mon[31175]: Deploying daemon osd.6 on smithi139 2024-09-16T01:34:47.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:47 smithi164 ceph-mon[35121]: Deploying daemon osd.6 on smithi139 2024-09-16T01:34:47.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:47 smithi176 ceph-mon[35324]: Deploying daemon osd.6 on smithi139 2024-09-16T01:34:48.572 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:48 smithi139 ceph-mon[31175]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:34:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:48 smithi164 ceph-mon[35121]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:34:48.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:48 smithi176 ceph-mon[35324]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T01:34:50.725 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:50 smithi139 ceph-mon[31175]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-16T01:34:50.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:50 smithi164 ceph-mon[35121]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-16T01:34:50.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:50 smithi176 ceph-mon[35324]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-16T01:34:52.721 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:52 smithi139 ceph-mon[31175]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:34:52.721 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:52 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:52.721 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:52 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:52.721 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:52 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:52 smithi164 ceph-mon[35121]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:34:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:52 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:52 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:52 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:52.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:52 smithi176 ceph-mon[35324]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:34:52.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:52 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:52.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:52 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:52.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:52 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:54 smithi164 ceph-mon[35121]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:54.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:54 smithi139 ceph-mon[31175]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:54.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:54 smithi176 ceph-mon[35324]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:55.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:55.809+0000 7f2d6f7fe640 1 -- 172.21.15.139:0/2761775524 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2d78062630 con 0x7f2d640a3240 2024-09-16T01:34:55.811 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 6 on host 'smithi139' 2024-09-16T01:34:55.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:55.811+0000 7f2d7c812640 1 -- 172.21.15.139:0/2761775524 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d640a3240 msgr2=0x7f2d640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:34:55.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:55.811+0000 7f2d7c812640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d640a3240 0x7f2d640a5700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d60006fd0 tx=0x7f2d60008040 comp rx=0 tx=0).stop 2024-09-16T01:34:55.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:55.811+0000 7f2d7c812640 1 -- 172.21.15.139:0/2761775524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d78102e00 msgr2=0x7f2d780745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:34:55.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:55.811+0000 7f2d7c812640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d78102e00 0x7f2d780745e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800b4a0 tx=0x7f2d6800b960 comp rx=0 tx=0).stop 2024-09-16T01:34:55.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:55.811+0000 7f2d7c812640 1 -- 172.21.15.139:0/2761775524 shutdown_connections 2024-09-16T01:34:55.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:55.811+0000 7f2d7c812640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d640a3240 0x7f2d640a5700 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:55.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:55.811+0000 7f2d7c812640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d781037c0 0x7f2d78074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:55.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:55.811+0000 7f2d7c812640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d78102e00 0x7f2d780745e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:55.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:55.811+0000 7f2d7c812640 1 --2- 172.21.15.139:0/2761775524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d78101c00 0x7f2d78075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:34:55.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:55.811+0000 7f2d7c812640 1 -- 172.21.15.139:0/2761775524 >> 172.21.15.139:0/2761775524 conn(0x7f2d780fd390 msgr2=0x7f2d781061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:34:55.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:55.811+0000 7f2d7c812640 1 -- 172.21.15.139:0/2761775524 shutdown_connections 2024-09-16T01:34:55.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:34:55.811+0000 7f2d7c812640 1 -- 172.21.15.139:0/2761775524 wait complete. 2024-09-16T01:34:56.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:56.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:56.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:56.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:56.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:56.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:55 smithi139 ceph-mon[31175]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:56.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:56.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:56.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:56.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:56.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:56.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:55 smithi176 ceph-mon[35324]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:56.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:56.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:56.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:56.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:56.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:56.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:55 smithi164 ceph-mon[35121]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:56.525 DEBUG:teuthology.orchestra.run.smithi139:osd.6> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.6.service 2024-09-16T01:34:56.527 INFO:tasks.cephadm:Deploying osd.7 on smithi164 with /dev/vg_nvme/lv_2... 2024-09-16T01:34:56.528 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T01:34:56.784 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:34:56.905 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:56.905 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:56.905 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:56 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:57.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:57.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:57.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:56 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:57.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:34:57.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:57.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:56 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:57.887 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:57 smithi164 ceph-mon[35121]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:58.052 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:57 smithi139 ceph-mon[31175]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:58.052 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 01:34:57 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T01:34:57.848+0000 7f37341cc640 -1 osd.6 0 log_to_monitors true 2024-09-16T01:34:58.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:57 smithi176 ceph-mon[35324]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:59.102 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:58 smithi139 ceph-mon[31175]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T01:34:59.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:58 smithi176 ceph-mon[35324]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T01:34:59.285 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:58 smithi164 ceph-mon[35121]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T01:34:59.288 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:34:59.315 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_2 2024-09-16T01:34:59.572 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:34:59.947 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T01:34:59.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: osdmap e39: 7 total, 6 up, 7 in 2024-09-16T01:34:59.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-16T01:34:59.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:34:59.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:34:59.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: Detected new or changed devices on smithi139 2024-09-16T01:34:59.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:59.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:59.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:59.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:59.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:34:59.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi139 to 5696M 2024-09-16T01:34:59.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:34:59.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:34:59.949 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:34:59.949 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:34:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:00.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.031+0000 7efd1c61e640 1 -- 172.21.15.164:0/2441371590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd14102130 msgr2=0x7efd14102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:35:00.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.031+0000 7efd1c61e640 1 --2- 172.21.15.164:0/2441371590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd14102130 0x7efd14102530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efd10009900 tx=0x7efd1002a410 comp rx=0 tx=0).stop 2024-09-16T01:35:00.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.031+0000 7efd1c61e640 1 -- 172.21.15.164:0/2441371590 shutdown_connections 2024-09-16T01:35:00.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.031+0000 7efd1c61e640 1 --2- 172.21.15.164:0/2441371590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd141017a0 0x7efd14109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:00.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.031+0000 7efd1c61e640 1 --2- 172.21.15.164:0/2441371590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd14100de0 0x7efd14101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:00.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.031+0000 7efd1c61e640 1 --2- 172.21.15.164:0/2441371590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd14102130 0x7efd14102530 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:00.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.031+0000 7efd1c61e640 1 -- 172.21.15.164:0/2441371590 >> 172.21.15.164:0/2441371590 conn(0x7efd140fba80 msgr2=0x7efd140fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:35:00.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.031+0000 7efd1c61e640 1 -- 172.21.15.164:0/2441371590 shutdown_connections 2024-09-16T01:35:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.031+0000 7efd1c61e640 1 -- 172.21.15.164:0/2441371590 wait complete. 2024-09-16T01:35:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1c61e640 1 Processor -- start 2024-09-16T01:35:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1c61e640 1 -- start start 2024-09-16T01:35:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1c61e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd14100de0 0x7efd14100870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:35:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1c61e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd141017a0 0x7efd1419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:35:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1c61e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd14102130 0x7efd1419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:35:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1c61e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efd1410c7e0 con 0x7efd141017a0 2024-09-16T01:35:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1c61e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efd1410c660 con 0x7efd14100de0 2024-09-16T01:35:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1c61e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efd1410c960 con 0x7efd14102130 2024-09-16T01:35:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1a3ba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd14100de0 0x7efd14100870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:35:00.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1a3ba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd14100de0 0x7efd14100870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43362/0 (socket says 172.21.15.164:43362) 2024-09-16T01:35:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1a3ba640 1 -- 172.21.15.164:0/652185080 learned_addr learned my addr 172.21.15.164:0/652185080 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T01:35:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd19bb9640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd141017a0 0x7efd1419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:35:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1abbb640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd14102130 0x7efd1419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:35:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1a3ba640 1 -- 172.21.15.164:0/652185080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd14102130 msgr2=0x7efd1419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:35:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1a3ba640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd14102130 0x7efd1419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1a3ba640 1 -- 172.21.15.164:0/652185080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd141017a0 msgr2=0x7efd1419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:35:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1a3ba640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd141017a0 0x7efd1419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1a3ba640 1 -- 172.21.15.164:0/652185080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1419fbf0 con 0x7efd14100de0 2024-09-16T01:35:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1abbb640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd14102130 0x7efd1419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:35:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.032+0000 7efd1a3ba640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd14100de0 0x7efd14100870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efd10009230 tx=0x7efd10032c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:35:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.033+0000 7efd0b7fe640 1 -- 172.21.15.164:0/652185080 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1003f030 con 0x7efd14100de0 2024-09-16T01:35:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.033+0000 7efd0b7fe640 1 -- 172.21.15.164:0/652185080 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd10003ce0 con 0x7efd14100de0 2024-09-16T01:35:00.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.033+0000 7efd1c61e640 1 -- 172.21.15.164:0/652185080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1419fe80 con 0x7efd14100de0 2024-09-16T01:35:00.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.033+0000 7efd0b7fe640 1 -- 172.21.15.164:0/652185080 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10035070 con 0x7efd14100de0 2024-09-16T01:35:00.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.033+0000 7efd1c61e640 1 -- 172.21.15.164:0/652185080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd141a0280 con 0x7efd14100de0 2024-09-16T01:35:00.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.034+0000 7efd1c61e640 1 -- 172.21.15.164:0/652185080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd14108090 con 0x7efd14100de0 2024-09-16T01:35:00.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.035+0000 7efd0b7fe640 1 -- 172.21.15.164:0/652185080 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7efd10042030 con 0x7efd14100de0 2024-09-16T01:35:00.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.035+0000 7efd0b7fe640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7efd000a3240 0x7efd000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:35:00.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.036+0000 7efd0b7fe640 1 -- 172.21.15.164:0/652185080 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4685+0+0 (secure 0 0 0) 0x7efd100ef110 con 0x7efd14100de0 2024-09-16T01:35:00.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.036+0000 7efd19bb9640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7efd000a3240 0x7efd000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:35:00.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.036+0000 7efd19bb9640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7efd000a3240 0x7efd000a5700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd141005f0 tx=0x7efd04009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:35:00.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.044+0000 7efd0b7fe640 1 -- 172.21.15.164:0/652185080 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd100bcc60 con 0x7efd14100de0 2024-09-16T01:35:00.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T01:35:00.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: osdmap e39: 7 total, 6 up, 7 in 2024-09-16T01:35:00.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-16T01:35:00.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:35:00.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:35:00.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: Detected new or changed devices on smithi139 2024-09-16T01:35:00.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:00.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:00.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:00.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:00.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:00.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi139 to 5696M 2024-09-16T01:35:00.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:00.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:00.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:35:00.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:34:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:00.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T01:35:00.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: osdmap e39: 7 total, 6 up, 7 in 2024-09-16T01:35:00.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-16T01:35:00.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:35:00.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:35:00.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: Detected new or changed devices on smithi139 2024-09-16T01:35:00.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:00.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:00.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:00.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:00.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:00.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi139 to 5696M 2024-09-16T01:35:00.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:00.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:00.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:35:00.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:34:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:00.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:00.226+0000 7efd1c61e640 1 -- 172.21.15.164:0/652185080 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7efd14062630 con 0x7efd000a3240 2024-09-16T01:35:00.673 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 01:35:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T01:35:00.225+0000 7f3730963640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T01:35:00.673 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 01:35:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T01:35:00.234+0000 7f3727f6f640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:35:00.933 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:00 smithi164 ceph-mon[35121]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-16T01:35:00.933 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:00 smithi164 ceph-mon[35121]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T01:35:00.934 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:35:00.934 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:35:00.934 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:00 smithi164 ceph-mon[35121]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:35:00.934 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:00 smithi164 ceph-mon[35121]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' 2024-09-16T01:35:00.934 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:35:00.934 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:35:00.934 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:01.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:00 smithi139 ceph-mon[31175]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-16T01:35:01.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:00 smithi139 ceph-mon[31175]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T01:35:01.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:35:01.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:35:01.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:00 smithi139 ceph-mon[31175]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:35:01.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:00 smithi139 ceph-mon[31175]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' 2024-09-16T01:35:01.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:35:01.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:35:01.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:01.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:00 smithi176 ceph-mon[35324]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-16T01:35:01.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:00 smithi176 ceph-mon[35324]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T01:35:01.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:35:01.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:35:01.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:00 smithi176 ceph-mon[35324]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:35:01.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:00 smithi176 ceph-mon[35324]: from='osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164]' entity='osd.6' 2024-09-16T01:35:01.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:35:01.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:35:01.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:01.865 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:01 smithi164 ceph-mon[35121]: purged_snaps scrub starts 2024-09-16T01:35:01.865 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:01 smithi164 ceph-mon[35121]: purged_snaps scrub ok 2024-09-16T01:35:01.866 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:01 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:35:01.866 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:01 smithi164 ceph-mon[35121]: osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164] boot 2024-09-16T01:35:01.866 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:01 smithi164 ceph-mon[35121]: osdmap e41: 7 total, 7 up, 7 in 2024-09-16T01:35:01.866 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:01 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:35:01.866 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:01 smithi164 ceph-mon[35121]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:35:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:01 smithi139 ceph-mon[31175]: purged_snaps scrub starts 2024-09-16T01:35:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:01 smithi139 ceph-mon[31175]: purged_snaps scrub ok 2024-09-16T01:35:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:01 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:35:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:01 smithi139 ceph-mon[31175]: osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164] boot 2024-09-16T01:35:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:01 smithi139 ceph-mon[31175]: osdmap e41: 7 total, 7 up, 7 in 2024-09-16T01:35:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:01 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:35:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:01 smithi139 ceph-mon[31175]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:35:02.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:01 smithi176 ceph-mon[35324]: purged_snaps scrub starts 2024-09-16T01:35:02.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:01 smithi176 ceph-mon[35324]: purged_snaps scrub ok 2024-09-16T01:35:02.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:01 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:35:02.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:01 smithi176 ceph-mon[35324]: osd.6 [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164] boot 2024-09-16T01:35:02.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:01 smithi176 ceph-mon[35324]: osdmap e41: 7 total, 7 up, 7 in 2024-09-16T01:35:02.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:01 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:35:02.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:01 smithi176 ceph-mon[35324]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-16T01:35:03.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:03 smithi164 ceph-mon[35121]: from='client.? 172.21.15.164:0/3805901726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23143c15-0839-4fd5-9d0a-19ae891000f0"}]: dispatch 2024-09-16T01:35:03.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:03 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23143c15-0839-4fd5-9d0a-19ae891000f0"}]: dispatch 2024-09-16T01:35:03.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:03 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23143c15-0839-4fd5-9d0a-19ae891000f0"}]': finished 2024-09-16T01:35:03.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:03 smithi164 ceph-mon[35121]: osdmap e42: 8 total, 7 up, 8 in 2024-09-16T01:35:03.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:03 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:03.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:03 smithi164 ceph-mon[35121]: from='client.? 172.21.15.164:0/1059452926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:35:03.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:03 smithi139 ceph-mon[31175]: from='client.? 172.21.15.164:0/3805901726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23143c15-0839-4fd5-9d0a-19ae891000f0"}]: dispatch 2024-09-16T01:35:03.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:03 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23143c15-0839-4fd5-9d0a-19ae891000f0"}]: dispatch 2024-09-16T01:35:03.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:03 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23143c15-0839-4fd5-9d0a-19ae891000f0"}]': finished 2024-09-16T01:35:03.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:03 smithi139 ceph-mon[31175]: osdmap e42: 8 total, 7 up, 8 in 2024-09-16T01:35:03.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:03 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:03.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:03 smithi139 ceph-mon[31175]: from='client.? 172.21.15.164:0/1059452926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:35:03.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:03 smithi176 ceph-mon[35324]: from='client.? 172.21.15.164:0/3805901726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23143c15-0839-4fd5-9d0a-19ae891000f0"}]: dispatch 2024-09-16T01:35:03.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:03 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23143c15-0839-4fd5-9d0a-19ae891000f0"}]: dispatch 2024-09-16T01:35:03.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:03 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23143c15-0839-4fd5-9d0a-19ae891000f0"}]': finished 2024-09-16T01:35:03.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:03 smithi176 ceph-mon[35324]: osdmap e42: 8 total, 7 up, 8 in 2024-09-16T01:35:03.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:03 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:03.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:03 smithi176 ceph-mon[35324]: from='client.? 172.21.15.164:0/1059452926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:35:04.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:04 smithi164 ceph-mon[35121]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:04.574 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:04 smithi139 ceph-mon[31175]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:04.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:04 smithi176 ceph-mon[35324]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:06.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:06 smithi164 ceph-mon[35121]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:06.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:06 smithi139 ceph-mon[31175]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:06.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:06 smithi176 ceph-mon[35324]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:08.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:08 smithi164 ceph-mon[35121]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:08.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:08 smithi139 ceph-mon[31175]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:08.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:08 smithi176 ceph-mon[35324]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:10 smithi164 ceph-mon[35121]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:10.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:10 smithi139 ceph-mon[31175]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:10.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:10 smithi176 ceph-mon[35324]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:11.721 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:11 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T01:35:11.721 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:11 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:11.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:11 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T01:35:11.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:11 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:11.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:11 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T01:35:11.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:11 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:12.776 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:12 smithi164 ceph-mon[35121]: Deploying daemon osd.7 on smithi164 2024-09-16T01:35:12.776 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:12 smithi164 ceph-mon[35121]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:12 smithi139 ceph-mon[31175]: Deploying daemon osd.7 on smithi164 2024-09-16T01:35:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:12 smithi139 ceph-mon[31175]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:12.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:12 smithi176 ceph-mon[35324]: Deploying daemon osd.7 on smithi164 2024-09-16T01:35:12.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:12 smithi176 ceph-mon[35324]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:14.628 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:14 smithi164 ceph-mon[35121]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:14.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:14 smithi139 ceph-mon[31175]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:14.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:14 smithi176 ceph-mon[35324]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:16.696 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:16 smithi164 ceph-mon[35121]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:16.739 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:16 smithi176 ceph-mon[35324]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:16.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:16 smithi139 ceph-mon[31175]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:17.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:35:17.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:17.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:35:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:17.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:35:17.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:17.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:18.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:18 smithi164 ceph-mon[35121]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:18.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:18 smithi139 ceph-mon[31175]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:18.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:18 smithi176 ceph-mon[35324]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:20.687 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:20 smithi164 ceph-mon[35121]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:20.687 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:20 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:20.687 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:20 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:20.687 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:20 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:20.687 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:20 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:35:20.688 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:20 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:20.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:20.727+0000 7efd0b7fe640 1 -- 172.21.15.164:0/652185080 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efd14062630 con 0x7efd000a3240 2024-09-16T01:35:20.729 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 7 on host 'smithi164' 2024-09-16T01:35:20.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:20.730+0000 7efd1c61e640 1 -- 172.21.15.164:0/652185080 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7efd000a3240 msgr2=0x7efd000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:35:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:20.730+0000 7efd1c61e640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7efd000a3240 0x7efd000a5700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd141005f0 tx=0x7efd04009110 comp rx=0 tx=0).stop 2024-09-16T01:35:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:20.730+0000 7efd1c61e640 1 -- 172.21.15.164:0/652185080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd14100de0 msgr2=0x7efd14100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:35:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:20.730+0000 7efd1c61e640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd14100de0 0x7efd14100870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efd10009230 tx=0x7efd10032c70 comp rx=0 tx=0).stop 2024-09-16T01:35:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:20.730+0000 7efd1c61e640 1 -- 172.21.15.164:0/652185080 shutdown_connections 2024-09-16T01:35:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:20.730+0000 7efd1c61e640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7efd000a3240 0x7efd000a5700 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:20.730+0000 7efd1c61e640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd14102130 0x7efd1419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:20.730+0000 7efd1c61e640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd141017a0 0x7efd1419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:20.730+0000 7efd1c61e640 1 --2- 172.21.15.164:0/652185080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd14100de0 0x7efd14100870 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:20.730+0000 7efd1c61e640 1 -- 172.21.15.164:0/652185080 >> 172.21.15.164:0/652185080 conn(0x7efd140fba80 msgr2=0x7efd14105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:35:20.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:20.730+0000 7efd1c61e640 1 -- 172.21.15.164:0/652185080 shutdown_connections 2024-09-16T01:35:20.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:35:20.730+0000 7efd1c61e640 1 -- 172.21.15.164:0/652185080 wait complete. 2024-09-16T01:35:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:20 smithi139 ceph-mon[31175]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:20 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:20 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:20 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:20 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:35:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:20 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:20.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:20 smithi176 ceph-mon[35324]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:20 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:20 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:20 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:20 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:35:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:20 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:21.395 DEBUG:teuthology.orchestra.run.smithi164:osd.7> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.7.service 2024-09-16T01:35:21.397 INFO:tasks.cephadm:Deploying osd.8 on smithi176 with /dev/vg_nvme/lv_2... 2024-09-16T01:35:21.397 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T01:35:21.652 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:35:21.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:21 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:35:21.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:21 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:21.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:21 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:21.698 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:21 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:35:21.698 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:21 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:21.698 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:21 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:21.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:21 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:35:21.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:21 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:21.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:21 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:22.734 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:22 smithi164 ceph-mon[35121]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:22.893 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:22 smithi176 ceph-mon[35324]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:22.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:22 smithi139 ceph-mon[31175]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:23.045 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 01:35:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T01:35:22.797+0000 7f2e2f2ce640 -1 osd.7 0 log_to_monitors true 2024-09-16T01:35:23.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:23 smithi164 ceph-mon[35121]: from='osd.7 [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T01:35:23.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:23 smithi164 ceph-mon[35121]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T01:35:23.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:23 smithi139 ceph-mon[31175]: from='osd.7 [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T01:35:23.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:23 smithi139 ceph-mon[31175]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T01:35:23.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:23 smithi176 ceph-mon[35324]: from='osd.7 [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T01:35:23.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:23 smithi176 ceph-mon[35324]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T01:35:24.312 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:35:24.345 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_2 2024-09-16T01:35:24.592 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:35:24.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T01:35:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: osdmap e43: 8 total, 7 up, 8 in 2024-09-16T01:35:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: from='osd.7 [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:35:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:35:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:35:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:24 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T01:35:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: osdmap e43: 8 total, 7 up, 8 in 2024-09-16T01:35:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: from='osd.7 [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:35:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:35:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:35:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:24.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:24.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T01:35:24.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: osdmap e43: 8 total, 7 up, 8 in 2024-09-16T01:35:24.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:24.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: from='osd.7 [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:35:24.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:35:24.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:24.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:24.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:24.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:24.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:24.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:24.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:24.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:35:24.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:24 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:25.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.094+0000 7f66192a2640 1 -- 172.21.15.176:0/1927897798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6614102e00 msgr2=0x7f6614103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:35:25.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.094+0000 7f66192a2640 1 --2- 172.21.15.176:0/1927897798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6614102e00 0x7f6614103280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009900 tx=0x7f65fc02a410 comp rx=0 tx=0).stop 2024-09-16T01:35:25.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.094+0000 7f66192a2640 1 -- 172.21.15.176:0/1927897798 shutdown_connections 2024-09-16T01:35:25.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.094+0000 7f66192a2640 1 --2- 172.21.15.176:0/1927897798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66141037c0 0x7f661410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:25.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.094+0000 7f66192a2640 1 --2- 172.21.15.176:0/1927897798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6614102e00 0x7f6614103280 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:25.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.094+0000 7f66192a2640 1 --2- 172.21.15.176:0/1927897798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6614101c00 0x7f6614102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:25.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.094+0000 7f66192a2640 1 -- 172.21.15.176:0/1927897798 >> 172.21.15.176:0/1927897798 conn(0x7f66140fd390 msgr2=0x7f66140ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:35:25.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.094+0000 7f66192a2640 1 -- 172.21.15.176:0/1927897798 shutdown_connections 2024-09-16T01:35:25.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.094+0000 7f66192a2640 1 -- 172.21.15.176:0/1927897798 wait complete. 2024-09-16T01:35:25.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.094+0000 7f66192a2640 1 Processor -- start 2024-09-16T01:35:25.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66192a2640 1 -- start start 2024-09-16T01:35:25.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66192a2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6614101c00 0x7f6614197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:35:25.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66192a2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6614102e00 0x7f6614198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:35:25.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66192a2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66141037c0 0x7f661419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:35:25.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66192a2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f661410d880 con 0x7f6614101c00 2024-09-16T01:35:25.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66192a2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f661410d700 con 0x7f6614102e00 2024-09-16T01:35:25.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66192a2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f661410da00 con 0x7f66141037c0 2024-09-16T01:35:25.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66137fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66141037c0 0x7f661419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:35:25.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66137fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66141037c0 0x7f661419f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54558/0 (socket says 172.21.15.176:54558) 2024-09-16T01:35:25.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66137fe640 1 -- 172.21.15.176:0/35298593 learned_addr learned my addr 172.21.15.176:0/35298593 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-16T01:35:25.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f660a7fc640 1 --2- 172.21.15.176:0/35298593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6614102e00 0x7f6614198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:35:25.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f6612ffd640 1 --2- 172.21.15.176:0/35298593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6614101c00 0x7f6614197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:35:25.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66137fe640 1 -- 172.21.15.176:0/35298593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6614102e00 msgr2=0x7f6614198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:35:25.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66137fe640 1 --2- 172.21.15.176:0/35298593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6614102e00 0x7f6614198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:25.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66137fe640 1 -- 172.21.15.176:0/35298593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6614101c00 msgr2=0x7f6614197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:35:25.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66137fe640 1 --2- 172.21.15.176:0/35298593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6614101c00 0x7f6614197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:25.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.095+0000 7f66137fe640 1 -- 172.21.15.176:0/35298593 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661419fb10 con 0x7f66141037c0 2024-09-16T01:35:25.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.096+0000 7f66137fe640 1 --2- 172.21.15.176:0/35298593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66141037c0 0x7f661419f4a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f660c00eb10 tx=0x7f660c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:35:25.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.096+0000 7f6610ff9640 1 -- 172.21.15.176:0/35298593 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c004b20 con 0x7f66141037c0 2024-09-16T01:35:25.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.096+0000 7f66192a2640 1 -- 172.21.15.176:0/35298593 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661419fe00 con 0x7f66141037c0 2024-09-16T01:35:25.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.096+0000 7f6610ff9640 1 -- 172.21.15.176:0/35298593 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f660c004cc0 con 0x7f66141037c0 2024-09-16T01:35:25.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.096+0000 7f6610ff9640 1 -- 172.21.15.176:0/35298593 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c010760 con 0x7f66141037c0 2024-09-16T01:35:25.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.096+0000 7f66192a2640 1 -- 172.21.15.176:0/35298593 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66141a02b0 con 0x7f66141037c0 2024-09-16T01:35:25.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.097+0000 7f66192a2640 1 -- 172.21.15.176:0/35298593 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6614109020 con 0x7f66141037c0 2024-09-16T01:35:25.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.098+0000 7f6610ff9640 1 -- 172.21.15.176:0/35298593 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f660c004e70 con 0x7f66141037c0 2024-09-16T01:35:25.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.098+0000 7f6610ff9640 1 --2- 172.21.15.176:0/35298593 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f66000a3270 0x7f66000a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:35:25.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.099+0000 7f6610ff9640 1 -- 172.21.15.176:0/35298593 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5117+0+0 (secure 0 0 0) 0x7f660c0cf3e0 con 0x7f66141037c0 2024-09-16T01:35:25.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.099+0000 7f6612ffd640 1 --2- 172.21.15.176:0/35298593 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f66000a3270 0x7f66000a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:35:25.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.099+0000 7f6612ffd640 1 --2- 172.21.15.176:0/35298593 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f66000a3270 0x7f66000a5730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6604005db0 tx=0x7f6604005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:35:25.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.103+0000 7f6610ff9640 1 -- 172.21.15.176:0/35298593 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f660c09cd80 con 0x7f66141037c0 2024-09-16T01:35:25.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:25.276+0000 7f66192a2640 1 -- 172.21.15.176:0/35298593 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6614062630 con 0x7f66000a3270 2024-09-16T01:35:25.294 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T01:35:24.885+0000 7f2e2ba65640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T01:35:25.295 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 01:35:24 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T01:35:24.895+0000 7f2e23071640 -1 osd.7 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:35:25.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:25 smithi176 ceph-mon[35324]: Detected new or changed devices on smithi164 2024-09-16T01:35:25.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:25 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi164 to 5696M 2024-09-16T01:35:25.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:25 smithi176 ceph-mon[35324]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T01:35:25.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:25 smithi176 ceph-mon[35324]: osdmap e44: 8 total, 7 up, 8 in 2024-09-16T01:35:25.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:25.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:25.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:25 smithi176 ceph-mon[35324]: from='osd.7 ' entity='osd.7' 2024-09-16T01:35:25.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:35:25.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:35:25.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:25 smithi164 ceph-mon[35121]: Detected new or changed devices on smithi164 2024-09-16T01:35:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:25 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi164 to 5696M 2024-09-16T01:35:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:25 smithi164 ceph-mon[35121]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T01:35:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:25 smithi164 ceph-mon[35121]: osdmap e44: 8 total, 7 up, 8 in 2024-09-16T01:35:25.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:25.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:25.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:25 smithi164 ceph-mon[35121]: from='osd.7 ' entity='osd.7' 2024-09-16T01:35:25.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:35:25.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:35:25.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:25.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:25 smithi139 ceph-mon[31175]: Detected new or changed devices on smithi164 2024-09-16T01:35:25.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:25 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi164 to 5696M 2024-09-16T01:35:25.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:25 smithi139 ceph-mon[31175]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T01:35:25.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:25 smithi139 ceph-mon[31175]: osdmap e44: 8 total, 7 up, 8 in 2024-09-16T01:35:25.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:25.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:25.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:25 smithi139 ceph-mon[31175]: from='osd.7 ' entity='osd.7' 2024-09-16T01:35:25.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:35:25.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:35:25.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:26.752 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:26 smithi176 ceph-mon[35324]: purged_snaps scrub starts 2024-09-16T01:35:26.752 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:26 smithi176 ceph-mon[35324]: purged_snaps scrub ok 2024-09-16T01:35:26.752 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:26 smithi176 ceph-mon[35324]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:35:26.752 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:26 smithi176 ceph-mon[35324]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:26.752 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:26 smithi176 ceph-mon[35324]: osd.7 [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693] boot 2024-09-16T01:35:26.752 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:26 smithi176 ceph-mon[35324]: osdmap e45: 8 total, 8 up, 8 in 2024-09-16T01:35:26.752 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:26 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:26.752 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:26 smithi176 ceph-mon[35324]: osdmap e46: 8 total, 8 up, 8 in 2024-09-16T01:35:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:26 smithi164 ceph-mon[35121]: purged_snaps scrub starts 2024-09-16T01:35:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:26 smithi164 ceph-mon[35121]: purged_snaps scrub ok 2024-09-16T01:35:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:26 smithi164 ceph-mon[35121]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:35:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:26 smithi164 ceph-mon[35121]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:26 smithi164 ceph-mon[35121]: osd.7 [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693] boot 2024-09-16T01:35:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:26 smithi164 ceph-mon[35121]: osdmap e45: 8 total, 8 up, 8 in 2024-09-16T01:35:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:26 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:26 smithi164 ceph-mon[35121]: osdmap e46: 8 total, 8 up, 8 in 2024-09-16T01:35:26.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:26 smithi139 ceph-mon[31175]: purged_snaps scrub starts 2024-09-16T01:35:26.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:26 smithi139 ceph-mon[31175]: purged_snaps scrub ok 2024-09-16T01:35:26.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:26 smithi139 ceph-mon[31175]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:35:26.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:26 smithi139 ceph-mon[31175]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-16T01:35:26.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:26 smithi139 ceph-mon[31175]: osd.7 [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693] boot 2024-09-16T01:35:26.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:26 smithi139 ceph-mon[31175]: osdmap e45: 8 total, 8 up, 8 in 2024-09-16T01:35:26.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:26 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:35:26.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:26 smithi139 ceph-mon[31175]: osdmap e46: 8 total, 8 up, 8 in 2024-09-16T01:35:28.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:27 smithi164 ceph-mon[35121]: osdmap e47: 8 total, 8 up, 8 in 2024-09-16T01:35:28.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:27 smithi164 ceph-mon[35121]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:28.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:27 smithi164 ceph-mon[35121]: from='client.? 172.21.15.176:0/2230561553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5"}]: dispatch 2024-09-16T01:35:28.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:27 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5"}]: dispatch 2024-09-16T01:35:28.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:27 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5"}]': finished 2024-09-16T01:35:28.050 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:27 smithi176 ceph-mon[35324]: osdmap e47: 8 total, 8 up, 8 in 2024-09-16T01:35:28.050 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:27 smithi176 ceph-mon[35324]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:28.050 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:27 smithi176 ceph-mon[35324]: from='client.? 172.21.15.176:0/2230561553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5"}]: dispatch 2024-09-16T01:35:28.050 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:27 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5"}]: dispatch 2024-09-16T01:35:28.051 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:27 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5"}]': finished 2024-09-16T01:35:28.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:27 smithi139 ceph-mon[31175]: osdmap e47: 8 total, 8 up, 8 in 2024-09-16T01:35:28.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:27 smithi139 ceph-mon[31175]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:28.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:27 smithi139 ceph-mon[31175]: from='client.? 172.21.15.176:0/2230561553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5"}]: dispatch 2024-09-16T01:35:28.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:27 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5"}]: dispatch 2024-09-16T01:35:28.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:27 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5"}]': finished 2024-09-16T01:35:28.735 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:28 smithi164 ceph-mon[35121]: osdmap e48: 9 total, 8 up, 9 in 2024-09-16T01:35:28.735 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:28 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:28.735 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:28 smithi164 ceph-mon[35121]: from='client.? 172.21.15.176:0/1554339348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:35:28.736 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:28 smithi164 ceph-mon[35121]: osdmap e49: 9 total, 8 up, 9 in 2024-09-16T01:35:28.736 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:28 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:29.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:28 smithi139 ceph-mon[31175]: osdmap e48: 9 total, 8 up, 9 in 2024-09-16T01:35:29.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:29.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:28 smithi139 ceph-mon[31175]: from='client.? 172.21.15.176:0/1554339348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:35:29.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:28 smithi139 ceph-mon[31175]: osdmap e49: 9 total, 8 up, 9 in 2024-09-16T01:35:29.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:29.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:28 smithi176 ceph-mon[35324]: osdmap e48: 9 total, 8 up, 9 in 2024-09-16T01:35:29.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:28 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:29.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:28 smithi176 ceph-mon[35324]: from='client.? 172.21.15.176:0/1554339348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:35:29.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:28 smithi176 ceph-mon[35324]: osdmap e49: 9 total, 8 up, 9 in 2024-09-16T01:35:29.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:28 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:30.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:29 smithi164 ceph-mon[35121]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:30.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:29 smithi139 ceph-mon[31175]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:30.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:29 smithi176 ceph-mon[35324]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:32.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:32 smithi164 ceph-mon[35121]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:32.872 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:32 smithi176 ceph-mon[35324]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:32.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:32 smithi139 ceph-mon[31175]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:34.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:34 smithi176 ceph-mon[35324]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:34.725 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:34 smithi139 ceph-mon[31175]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:34.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:34 smithi164 ceph-mon[35121]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:36.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:36 smithi176 ceph-mon[35324]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:36.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:36 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T01:35:36.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:36 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:36.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:36 smithi164 ceph-mon[35121]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:36.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:36 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T01:35:36.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:36 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:36.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:36 smithi139 ceph-mon[31175]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:36.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:36 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T01:35:36.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:36 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:37.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:37 smithi164 ceph-mon[35121]: Deploying daemon osd.8 on smithi176 2024-09-16T01:35:37.813 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:37 smithi176 ceph-mon[35324]: Deploying daemon osd.8 on smithi176 2024-09-16T01:35:37.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:37 smithi139 ceph-mon[31175]: Deploying daemon osd.8 on smithi176 2024-09-16T01:35:38.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:38 smithi176 ceph-mon[35324]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:38 smithi164 ceph-mon[35121]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:38.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:38 smithi139 ceph-mon[31175]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:40.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:40 smithi164 ceph-mon[35121]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:40.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:40 smithi139 ceph-mon[31175]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:40.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:40 smithi176 ceph-mon[35324]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:42.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:42 smithi164 ceph-mon[35121]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:42.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:42 smithi139 ceph-mon[31175]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:42.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:42 smithi176 ceph-mon[35324]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:43.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:43 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:35:43.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:43 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:43.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:43 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:43 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:35:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:43 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:43 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:43.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:43 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:35:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:43 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:43 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:44.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:44 smithi176 ceph-mon[35324]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:44.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:44 smithi164 ceph-mon[35121]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:44.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:44 smithi139 ceph-mon[31175]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:46.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:46.329+0000 7f6610ff9640 1 -- 172.21.15.176:0/35298593 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6614062630 con 0x7f66000a3270 2024-09-16T01:35:46.331 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 8 on host 'smithi176' 2024-09-16T01:35:46.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:46.331+0000 7f66192a2640 1 -- 172.21.15.176:0/35298593 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f66000a3270 msgr2=0x7f66000a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:35:46.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:46.331+0000 7f66192a2640 1 --2- 172.21.15.176:0/35298593 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f66000a3270 0x7f66000a5730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6604005db0 tx=0x7f6604005d40 comp rx=0 tx=0).stop 2024-09-16T01:35:46.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:46.332+0000 7f66192a2640 1 -- 172.21.15.176:0/35298593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66141037c0 msgr2=0x7f661419f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:35:46.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:46.332+0000 7f66192a2640 1 --2- 172.21.15.176:0/35298593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66141037c0 0x7f661419f4a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f660c00eb10 tx=0x7f660c00c2d0 comp rx=0 tx=0).stop 2024-09-16T01:35:46.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:46.332+0000 7f66192a2640 1 -- 172.21.15.176:0/35298593 shutdown_connections 2024-09-16T01:35:46.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:46.332+0000 7f66192a2640 1 --2- 172.21.15.176:0/35298593 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f66000a3270 0x7f66000a5730 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:46.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:46.332+0000 7f66192a2640 1 --2- 172.21.15.176:0/35298593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66141037c0 0x7f661419f4a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:46.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:46.332+0000 7f66192a2640 1 --2- 172.21.15.176:0/35298593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6614102e00 0x7f6614198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:46.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:46.332+0000 7f66192a2640 1 --2- 172.21.15.176:0/35298593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6614101c00 0x7f6614197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:46.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:46.332+0000 7f66192a2640 1 -- 172.21.15.176:0/35298593 >> 172.21.15.176:0/35298593 conn(0x7f66140fd390 msgr2=0x7f66141061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:35:46.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:46.332+0000 7f66192a2640 1 -- 172.21.15.176:0/35298593 shutdown_connections 2024-09-16T01:35:46.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:35:46.332+0000 7f66192a2640 1 -- 172.21.15.176:0/35298593 wait complete. 2024-09-16T01:35:46.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:46 smithi176 ceph-mon[35324]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:46.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:46 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:46.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:46 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:46.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:46 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:46.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:46 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:35:46.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:46 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:46.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:46 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:35:46.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:46 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:46.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:46 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:46 smithi164 ceph-mon[35121]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:46 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:46 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:46 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:46 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:35:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:46 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:46 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:35:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:46 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:46 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:46 smithi139 ceph-mon[31175]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:46 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:46 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:46 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:46 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:35:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:46 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:46 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:35:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:46 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:46 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:47.058 DEBUG:teuthology.orchestra.run.smithi176:osd.8> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.8.service 2024-09-16T01:35:47.060 INFO:tasks.cephadm:Deploying osd.9 on smithi139 with /dev/vg_nvme/lv_1... 2024-09-16T01:35:47.060 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T01:35:47.320 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:35:48.705 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:48 smithi176 ceph-mon[35324]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:48.706 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 01:35:48 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T01:35:48.543+0000 7fc28e008640 -1 osd.8 0 log_to_monitors true 2024-09-16T01:35:48.740 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:48 smithi139 ceph-mon[31175]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:48 smithi164 ceph-mon[35121]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:49.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:49 smithi176 ceph-mon[35324]: from='osd.8 [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T01:35:49.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:49 smithi176 ceph-mon[35324]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T01:35:49.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:49 smithi139 ceph-mon[31175]: from='osd.8 [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T01:35:49.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:49 smithi139 ceph-mon[31175]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T01:35:49.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:49 smithi164 ceph-mon[35121]: from='osd.8 [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T01:35:49.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:49 smithi164 ceph-mon[35121]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T01:35:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:35:50.161 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_1 2024-09-16T01:35:50.410 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:35:50.715 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:50.716 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T01:35:50.716 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: osdmap e50: 9 total, 8 up, 9 in 2024-09-16T01:35:50.716 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:50.716 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: from='osd.8 [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:35:50.716 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:35:50.716 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:50.716 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:50.716 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:50.716 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:50.716 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:50.716 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:50.717 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:50.717 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:35:50.717 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:50 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:50.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:50.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T01:35:50.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: osdmap e50: 9 total, 8 up, 9 in 2024-09-16T01:35:50.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:50.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: from='osd.8 [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:35:50.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:35:50.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:50.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:50.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:50.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:50.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:50.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:50.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:50.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:35:50.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:50 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:50.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.918+0000 7fdedb8f0640 1 -- 172.21.15.139:0/2847502776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded4105030 msgr2=0x7fded4107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:35:50.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.918+0000 7fdedb8f0640 1 --2- 172.21.15.139:0/2847502776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded4105030 0x7fded4107420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009900 tx=0x7fdec802a410 comp rx=0 tx=0).stop 2024-09-16T01:35:50.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.918+0000 7fdedb8f0640 1 -- 172.21.15.139:0/2847502776 shutdown_connections 2024-09-16T01:35:50.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.918+0000 7fdedb8f0640 1 --2- 172.21.15.139:0/2847502776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fded4107960 0x7fded4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:50.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.918+0000 7fdedb8f0640 1 --2- 172.21.15.139:0/2847502776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded4105030 0x7fded4107420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:50.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.918+0000 7fdedb8f0640 1 --2- 172.21.15.139:0/2847502776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded4102700 0x7fded4104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:50.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.918+0000 7fdedb8f0640 1 -- 172.21.15.139:0/2847502776 >> 172.21.15.139:0/2847502776 conn(0x7fded40fc280 msgr2=0x7fded40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:35:50.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.918+0000 7fdedb8f0640 1 -- 172.21.15.139:0/2847502776 shutdown_connections 2024-09-16T01:35:50.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.918+0000 7fdedb8f0640 1 -- 172.21.15.139:0/2847502776 wait complete. 2024-09-16T01:35:50.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.918+0000 7fdedb8f0640 1 Processor -- start 2024-09-16T01:35:50.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.918+0000 7fdedb8f0640 1 -- start start 2024-09-16T01:35:50.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fdedb8f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded4102700 0x7fded4197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:35:50.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fdedb8f0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fded4105030 0x7fded4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:35:50.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fdedb8f0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded4107960 0x7fded419f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:35:50.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fdedb8f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fded410d830 con 0x7fded4102700 2024-09-16T01:35:50.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fdedb8f0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fded410d6b0 con 0x7fded4105030 2024-09-16T01:35:50.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fdedb8f0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fded410d9b0 con 0x7fded4107960 2024-09-16T01:35:50.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fded968c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded4102700 0x7fded4197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:35:50.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fded968c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded4102700 0x7fded4197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55084/0 (socket says 172.21.15.139:55084) 2024-09-16T01:35:50.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fded968c640 1 -- 172.21.15.139:0/1608708084 learned_addr learned my addr 172.21.15.139:0/1608708084 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:35:50.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fded9e8d640 1 --2- 172.21.15.139:0/1608708084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded4107960 0x7fded419f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:35:50.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fded8e8b640 1 --2- 172.21.15.139:0/1608708084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fded4105030 0x7fded4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:35:50.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fded968c640 1 -- 172.21.15.139:0/1608708084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded4107960 msgr2=0x7fded419f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:35:50.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fded968c640 1 --2- 172.21.15.139:0/1608708084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded4107960 0x7fded419f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:50.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fded968c640 1 -- 172.21.15.139:0/1608708084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fded4105030 msgr2=0x7fded4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:35:50.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fded968c640 1 --2- 172.21.15.139:0/1608708084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fded4105030 0x7fded4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:35:50.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fded968c640 1 -- 172.21.15.139:0/1608708084 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded419fb70 con 0x7fded4102700 2024-09-16T01:35:50.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fded968c640 1 --2- 172.21.15.139:0/1608708084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded4102700 0x7fded4197f40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdec400eab0 tx=0x7fdec400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:35:50.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fded27fc640 1 -- 172.21.15.139:0/1608708084 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec400ce40 con 0x7fded4102700 2024-09-16T01:35:50.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.919+0000 7fded27fc640 1 -- 172.21.15.139:0/1608708084 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdec4011950 con 0x7fded4102700 2024-09-16T01:35:50.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.920+0000 7fdedb8f0640 1 -- 172.21.15.139:0/1608708084 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded419fe60 con 0x7fded4102700 2024-09-16T01:35:50.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.920+0000 7fded27fc640 1 -- 172.21.15.139:0/1608708084 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4010ae0 con 0x7fded4102700 2024-09-16T01:35:50.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.920+0000 7fdedb8f0640 1 -- 172.21.15.139:0/1608708084 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded41a02b0 con 0x7fded4102700 2024-09-16T01:35:50.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.920+0000 7fdedb8f0640 1 -- 172.21.15.139:0/1608708084 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded40fdd70 con 0x7fded4102700 2024-09-16T01:35:50.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.922+0000 7fded27fc640 1 -- 172.21.15.139:0/1608708084 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fdec4011b00 con 0x7fded4102700 2024-09-16T01:35:50.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.922+0000 7fded27fc640 1 --2- 172.21.15.139:0/1608708084 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fdebc0a3270 0x7fdebc0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:35:50.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.922+0000 7fded27fc640 1 -- 172.21.15.139:0/1608708084 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5862+0+0 (secure 0 0 0) 0x7fdec40cf620 con 0x7fded4102700 2024-09-16T01:35:50.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.922+0000 7fded8e8b640 1 --2- 172.21.15.139:0/1608708084 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fdebc0a3270 0x7fdebc0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:35:50.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.923+0000 7fded8e8b640 1 --2- 172.21.15.139:0/1608708084 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fdebc0a3270 0x7fdebc0a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdec8006010 tx=0x7fdec8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:35:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:50.927+0000 7fded27fc640 1 -- 172.21.15.139:0/1608708084 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdec4016030 con 0x7fded4102700 2024-09-16T01:35:50.936 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T01:35:50.454+0000 7fc289f8b640 -1 osd.8 0 waiting for initial osdmap 2024-09-16T01:35:50.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T01:35:50.462+0000 7fc285db3640 -1 osd.8 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:35:50.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:50.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T01:35:50.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: osdmap e50: 9 total, 8 up, 9 in 2024-09-16T01:35:50.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:50.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: from='osd.8 [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:35:50.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:35:50.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:50.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:50.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:50.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:50.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:35:50.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:50.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:50.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:35:50.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:50 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:35:51.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:35:51.099+0000 7fdedb8f0640 1 -- 172.21.15.139:0/1608708084 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fded4062630 con 0x7fdebc0a3270 2024-09-16T01:35:51.757 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:51 smithi139 ceph-mon[31175]: Detected new or changed devices on smithi176 2024-09-16T01:35:51.757 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:51 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi176 to 5696M 2024-09-16T01:35:51.757 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:51 smithi139 ceph-mon[31175]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-16T01:35:51.757 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:51 smithi139 ceph-mon[31175]: osdmap e51: 9 total, 8 up, 9 in 2024-09-16T01:35:51.757 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:51 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:51.757 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:51 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:51.757 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:51 smithi139 ceph-mon[31175]: osd.8 [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361] boot 2024-09-16T01:35:51.757 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:51 smithi139 ceph-mon[31175]: osdmap e52: 9 total, 9 up, 9 in 2024-09-16T01:35:51.757 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:51 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:51.757 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:51 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:35:51.757 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:51 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:35:51.758 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:51 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:51 smithi164 ceph-mon[35121]: Detected new or changed devices on smithi176 2024-09-16T01:35:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:51 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi176 to 5696M 2024-09-16T01:35:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:51 smithi164 ceph-mon[35121]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-16T01:35:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:51 smithi164 ceph-mon[35121]: osdmap e51: 9 total, 8 up, 9 in 2024-09-16T01:35:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:51 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:51 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:51 smithi164 ceph-mon[35121]: osd.8 [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361] boot 2024-09-16T01:35:51.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:51 smithi164 ceph-mon[35121]: osdmap e52: 9 total, 9 up, 9 in 2024-09-16T01:35:51.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:51 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:51.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:51 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:35:51.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:51 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:35:51.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:51 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:51.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:51 smithi176 ceph-mon[35324]: Detected new or changed devices on smithi176 2024-09-16T01:35:51.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:51 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi176 to 5696M 2024-09-16T01:35:51.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:51 smithi176 ceph-mon[35324]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-16T01:35:51.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:51 smithi176 ceph-mon[35324]: osdmap e51: 9 total, 8 up, 9 in 2024-09-16T01:35:51.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:51 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:51.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:51 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:51.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:51 smithi176 ceph-mon[35324]: osd.8 [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361] boot 2024-09-16T01:35:51.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:51 smithi176 ceph-mon[35324]: osdmap e52: 9 total, 9 up, 9 in 2024-09-16T01:35:51.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:51 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:35:51.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:51 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:35:51.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:51 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:35:51.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:51 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:35:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:52 smithi164 ceph-mon[35121]: purged_snaps scrub starts 2024-09-16T01:35:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:52 smithi164 ceph-mon[35121]: purged_snaps scrub ok 2024-09-16T01:35:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:52 smithi164 ceph-mon[35121]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:35:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:52 smithi164 ceph-mon[35121]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:52 smithi164 ceph-mon[35121]: osdmap e53: 9 total, 9 up, 9 in 2024-09-16T01:35:52.829 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:52 smithi139 ceph-mon[31175]: purged_snaps scrub starts 2024-09-16T01:35:52.829 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:52 smithi139 ceph-mon[31175]: purged_snaps scrub ok 2024-09-16T01:35:52.829 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:52 smithi139 ceph-mon[31175]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:35:52.829 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:52 smithi139 ceph-mon[31175]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:52.829 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:52 smithi139 ceph-mon[31175]: osdmap e53: 9 total, 9 up, 9 in 2024-09-16T01:35:52.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:52 smithi176 ceph-mon[35324]: purged_snaps scrub starts 2024-09-16T01:35:52.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:52 smithi176 ceph-mon[35324]: purged_snaps scrub ok 2024-09-16T01:35:52.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:52 smithi176 ceph-mon[35324]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:35:52.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:52 smithi176 ceph-mon[35324]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-16T01:35:52.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:52 smithi176 ceph-mon[35324]: osdmap e53: 9 total, 9 up, 9 in 2024-09-16T01:35:53.992 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:53 smithi139 ceph-mon[31175]: osdmap e54: 9 total, 9 up, 9 in 2024-09-16T01:35:53.992 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:53 smithi139 ceph-mon[31175]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:35:53.992 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:53 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2430565393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d25a3b-1e66-480e-9433-ceef7f0b9279"}]: dispatch 2024-09-16T01:35:53.992 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:53 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2430565393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09d25a3b-1e66-480e-9433-ceef7f0b9279"}]': finished 2024-09-16T01:35:53.992 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:53 smithi139 ceph-mon[31175]: osdmap e55: 10 total, 9 up, 10 in 2024-09-16T01:35:53.992 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:35:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:53 smithi164 ceph-mon[35121]: osdmap e54: 9 total, 9 up, 9 in 2024-09-16T01:35:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:53 smithi164 ceph-mon[35121]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:35:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:53 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2430565393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d25a3b-1e66-480e-9433-ceef7f0b9279"}]: dispatch 2024-09-16T01:35:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:53 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2430565393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09d25a3b-1e66-480e-9433-ceef7f0b9279"}]': finished 2024-09-16T01:35:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:53 smithi164 ceph-mon[35121]: osdmap e55: 10 total, 9 up, 10 in 2024-09-16T01:35:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:35:54.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:53 smithi176 ceph-mon[35324]: osdmap e54: 9 total, 9 up, 9 in 2024-09-16T01:35:54.214 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:53 smithi176 ceph-mon[35324]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:35:54.214 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:53 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2430565393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d25a3b-1e66-480e-9433-ceef7f0b9279"}]: dispatch 2024-09-16T01:35:54.214 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:53 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2430565393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09d25a3b-1e66-480e-9433-ceef7f0b9279"}]': finished 2024-09-16T01:35:54.214 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:53 smithi176 ceph-mon[35324]: osdmap e55: 10 total, 9 up, 10 in 2024-09-16T01:35:54.215 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:35:55.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:54 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1198656708' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:35:55.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:54 smithi164 ceph-mon[35121]: osdmap e56: 10 total, 9 up, 10 in 2024-09-16T01:35:55.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:54 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:35:55.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:54 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1198656708' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:35:55.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:54 smithi139 ceph-mon[31175]: osdmap e56: 10 total, 9 up, 10 in 2024-09-16T01:35:55.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:54 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:35:55.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:54 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1198656708' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:35:55.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:54 smithi176 ceph-mon[35324]: osdmap e56: 10 total, 9 up, 10 in 2024-09-16T01:35:55.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:54 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:35:56.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:55 smithi164 ceph-mon[35121]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:35:56.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:55 smithi139 ceph-mon[31175]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:35:56.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:55 smithi176 ceph-mon[35324]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:35:58.558 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:35:58 smithi139 ceph-mon[31175]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T01:35:58.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:35:58 smithi176 ceph-mon[35324]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T01:35:58.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:35:58 smithi164 ceph-mon[35121]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T01:36:00.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:00 smithi164 ceph-mon[35121]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-16T01:36:00.859 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:00 smithi139 ceph-mon[31175]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-16T01:36:00.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:00 smithi176 ceph-mon[35324]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-16T01:36:02.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:02 smithi139 ceph-mon[31175]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T01:36:02.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T01:36:02.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:02 smithi164 ceph-mon[35121]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T01:36:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T01:36:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:02.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:02 smithi176 ceph-mon[35324]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T01:36:02.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T01:36:02.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:03.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:03 smithi164 ceph-mon[35121]: Deploying daemon osd.9 on smithi139 2024-09-16T01:36:03.916 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:03 smithi139 ceph-mon[31175]: Deploying daemon osd.9 on smithi139 2024-09-16T01:36:03.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:03 smithi176 ceph-mon[35324]: Deploying daemon osd.9 on smithi139 2024-09-16T01:36:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:04 smithi164 ceph-mon[35121]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-16T01:36:04.920 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:04 smithi139 ceph-mon[31175]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-16T01:36:04.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:04 smithi176 ceph-mon[35324]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-16T01:36:06.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:06 smithi139 ceph-mon[31175]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-16T01:36:06.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:06 smithi164 ceph-mon[35121]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-16T01:36:06.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:06 smithi176 ceph-mon[35324]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-16T01:36:08.534 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:08 smithi139 ceph-mon[31175]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:36:08.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:08 smithi164 ceph-mon[35121]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:36:08.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:08 smithi176 ceph-mon[35324]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:36:09.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:36:09.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:09.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:36:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:09.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:36:09.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:09.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:10.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:10 smithi139 ceph-mon[31175]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:10 smithi164 ceph-mon[35121]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:10.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:10 smithi176 ceph-mon[35324]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:12.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:36:12.559+0000 7fded27fc640 1 -- 172.21.15.139:0/1608708084 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fded4062630 con 0x7fdebc0a3270 2024-09-16T01:36:12.561 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 9 on host 'smithi139' 2024-09-16T01:36:12.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:36:12.562+0000 7fdedb8f0640 1 -- 172.21.15.139:0/1608708084 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fdebc0a3270 msgr2=0x7fdebc0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:36:12.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:36:12.562+0000 7fdedb8f0640 1 --2- 172.21.15.139:0/1608708084 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fdebc0a3270 0x7fdebc0a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdec8006010 tx=0x7fdec8000bc0 comp rx=0 tx=0).stop 2024-09-16T01:36:12.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:36:12.562+0000 7fdedb8f0640 1 -- 172.21.15.139:0/1608708084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded4102700 msgr2=0x7fded4197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:36:12.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:36:12.562+0000 7fdedb8f0640 1 --2- 172.21.15.139:0/1608708084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded4102700 0x7fded4197f40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdec400eab0 tx=0x7fdec400ef70 comp rx=0 tx=0).stop 2024-09-16T01:36:12.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:36:12.562+0000 7fdedb8f0640 1 -- 172.21.15.139:0/1608708084 shutdown_connections 2024-09-16T01:36:12.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:36:12.562+0000 7fdedb8f0640 1 --2- 172.21.15.139:0/1608708084 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fdebc0a3270 0x7fdebc0a5730 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:12.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:36:12.562+0000 7fdedb8f0640 1 --2- 172.21.15.139:0/1608708084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded4107960 0x7fded419f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:12.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:36:12.562+0000 7fdedb8f0640 1 --2- 172.21.15.139:0/1608708084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fded4105030 0x7fded4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:12.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:36:12.562+0000 7fdedb8f0640 1 --2- 172.21.15.139:0/1608708084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded4102700 0x7fded4197f40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:12.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:36:12.562+0000 7fdedb8f0640 1 -- 172.21.15.139:0/1608708084 >> 172.21.15.139:0/1608708084 conn(0x7fded40fc280 msgr2=0x7fded4103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:36:12.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:36:12.562+0000 7fdedb8f0640 1 -- 172.21.15.139:0/1608708084 shutdown_connections 2024-09-16T01:36:12.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:36:12.562+0000 7fdedb8f0640 1 -- 172.21.15.139:0/1608708084 wait complete. 2024-09-16T01:36:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:12 smithi164 ceph-mon[35121]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:12 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:12 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:12 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:12 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:36:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:12 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:12 smithi139 ceph-mon[31175]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:12 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:12 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:12 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:12 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:36:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:12 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:12.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:12 smithi176 ceph-mon[35324]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:12.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:12 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:12.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:12 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:12.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:12 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:12.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:12 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:36:12.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:12 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:13.398 DEBUG:teuthology.orchestra.run.smithi139:osd.9> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.9.service 2024-09-16T01:36:13.400 INFO:tasks.cephadm:Deploying osd.10 on smithi164 with /dev/vg_nvme/lv_1... 2024-09-16T01:36:13.400 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T01:36:13.623 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:13 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:36:13.623 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:13 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:13.623 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:13 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:13.660 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:36:13.781 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:13 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:36:13.782 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:13 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:13.782 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:13 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:13.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:13 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:36:13.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:13 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:13.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:13 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:14.745 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:14 smithi164 ceph-mon[35121]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:14.745 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:14 smithi164 ceph-mon[35121]: from='osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T01:36:14.806 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:14 smithi139 ceph-mon[31175]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:14.806 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:14 smithi139 ceph-mon[31175]: from='osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T01:36:14.807 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 01:36:14 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T01:36:14.496+0000 7fcf599a2640 -1 osd.9 0 log_to_monitors true 2024-09-16T01:36:14.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:14 smithi176 ceph-mon[35324]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:14.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:14 smithi176 ceph-mon[35324]: from='osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T01:36:15.709 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:15 smithi164 ceph-mon[35121]: from='osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T01:36:15.709 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:15 smithi164 ceph-mon[35121]: osdmap e57: 10 total, 9 up, 10 in 2024-09-16T01:36:15.709 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:15 smithi164 ceph-mon[35121]: from='osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-16T01:36:15.709 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:15 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:15.709 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:15 smithi164 ceph-mon[35121]: from='osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-16T01:36:15.709 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:15 smithi164 ceph-mon[35121]: osdmap e58: 10 total, 9 up, 10 in 2024-09-16T01:36:15.709 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:15 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:15.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 01:36:15 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T01:36:15.570+0000 7fcf55925640 -1 osd.9 0 waiting for initial osdmap 2024-09-16T01:36:15.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 01:36:15 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T01:36:15.582+0000 7fcf5174d640 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:36:15.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:15 smithi139 ceph-mon[31175]: from='osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T01:36:15.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:15 smithi139 ceph-mon[31175]: osdmap e57: 10 total, 9 up, 10 in 2024-09-16T01:36:15.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:15 smithi139 ceph-mon[31175]: from='osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-16T01:36:15.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:15 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:15.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:15 smithi139 ceph-mon[31175]: from='osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-16T01:36:15.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:15 smithi139 ceph-mon[31175]: osdmap e58: 10 total, 9 up, 10 in 2024-09-16T01:36:15.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:15 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:15.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:15 smithi176 ceph-mon[35324]: from='osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T01:36:15.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:15 smithi176 ceph-mon[35324]: osdmap e57: 10 total, 9 up, 10 in 2024-09-16T01:36:15.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:15 smithi176 ceph-mon[35324]: from='osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-09-16T01:36:15.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:15 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:15.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:15 smithi176 ceph-mon[35324]: from='osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-09-16T01:36:15.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:15 smithi176 ceph-mon[35324]: osdmap e58: 10 total, 9 up, 10 in 2024-09-16T01:36:15.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:15 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:16.473 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T01:36:16.499 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_1 2024-09-16T01:36:16.758 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:36:16.783 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:16 smithi164 ceph-mon[35121]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:16.783 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:16.783 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:16.783 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:16.783 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:16.783 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:16.783 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:16.783 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:16.784 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:16.784 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:16.784 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:36:16.784 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:16.784 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:16.838 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:16 smithi176 ceph-mon[35324]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:16.838 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:16.838 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:16.838 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:16.838 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:16.838 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:16.839 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:16.839 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:16.839 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:16.839 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:16.839 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:36:16.839 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:16.839 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:16.912 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:16 smithi139 ceph-mon[31175]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:16.913 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:16.913 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:16.913 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:16.913 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:16.913 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:16.913 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:16.913 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:16.913 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:16.913 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:16.913 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:36:16.913 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:16.913 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:17.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.250+0000 7f2d4f606640 1 -- 172.21.15.164:0/4072410327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d48107920 msgr2=0x7f2d48109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:36:17.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.250+0000 7f2d4f606640 1 --2- 172.21.15.164:0/4072410327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d48107920 0x7f2d48109d10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009ca0 tx=0x7f2d4002a8c0 comp rx=0 tx=0).stop 2024-09-16T01:36:17.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.250+0000 7f2d4f606640 1 -- 172.21.15.164:0/4072410327 shutdown_connections 2024-09-16T01:36:17.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.250+0000 7f2d4f606640 1 --2- 172.21.15.164:0/4072410327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d48107920 0x7f2d48109d10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:17.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.250+0000 7f2d4f606640 1 --2- 172.21.15.164:0/4072410327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d48104ff0 0x7f2d481073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:17.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.250+0000 7f2d4f606640 1 --2- 172.21.15.164:0/4072410327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d481026c0 0x7f2d48104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:17.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.250+0000 7f2d4f606640 1 -- 172.21.15.164:0/4072410327 >> 172.21.15.164:0/4072410327 conn(0x7f2d480fc280 msgr2=0x7f2d480fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:36:17.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.251+0000 7f2d4f606640 1 -- 172.21.15.164:0/4072410327 shutdown_connections 2024-09-16T01:36:17.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.251+0000 7f2d4f606640 1 -- 172.21.15.164:0/4072410327 wait complete. 2024-09-16T01:36:17.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.251+0000 7f2d4f606640 1 Processor -- start 2024-09-16T01:36:17.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.251+0000 7f2d4f606640 1 -- start start 2024-09-16T01:36:17.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.251+0000 7f2d4f606640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d481026c0 0x7f2d48197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:36:17.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.251+0000 7f2d4f606640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d48104ff0 0x7f2d48198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:36:17.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.251+0000 7f2d4f606640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d48107920 0x7f2d4819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:36:17.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.251+0000 7f2d4f606640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4810d860 con 0x7f2d48107920 2024-09-16T01:36:17.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.251+0000 7f2d4f606640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4810d6e0 con 0x7f2d481026c0 2024-09-16T01:36:17.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.251+0000 7f2d4f606640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4810d9e0 con 0x7f2d48104ff0 2024-09-16T01:36:17.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.251+0000 7f2d4d3a2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d481026c0 0x7f2d48197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:36:17.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.251+0000 7f2d4d3a2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d481026c0 0x7f2d48197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44148/0 (socket says 172.21.15.164:44148) 2024-09-16T01:36:17.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.251+0000 7f2d4d3a2640 1 -- 172.21.15.164:0/1952354255 learned_addr learned my addr 172.21.15.164:0/1952354255 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T01:36:17.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.251+0000 7f2d4cba1640 1 --2- 172.21.15.164:0/1952354255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d48104ff0 0x7f2d48198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:36:17.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.252+0000 7f2d4dba3640 1 --2- 172.21.15.164:0/1952354255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d48107920 0x7f2d4819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:36:17.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.252+0000 7f2d4d3a2640 1 -- 172.21.15.164:0/1952354255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d48104ff0 msgr2=0x7f2d48198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:36:17.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.252+0000 7f2d4d3a2640 1 --2- 172.21.15.164:0/1952354255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d48104ff0 0x7f2d48198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:17.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.252+0000 7f2d4d3a2640 1 -- 172.21.15.164:0/1952354255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d48107920 msgr2=0x7f2d4819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:36:17.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.252+0000 7f2d4d3a2640 1 --2- 172.21.15.164:0/1952354255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d48107920 0x7f2d4819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:17.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.252+0000 7f2d4d3a2640 1 -- 172.21.15.164:0/1952354255 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4819faf0 con 0x7f2d481026c0 2024-09-16T01:36:17.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.252+0000 7f2d4d3a2640 1 --2- 172.21.15.164:0/1952354255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d481026c0 0x7f2d48197ec0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400d950 tx=0x7f2d4400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:36:17.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.252+0000 7f2d3e7fc640 1 -- 172.21.15.164:0/1952354255 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44017070 con 0x7f2d481026c0 2024-09-16T01:36:17.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.252+0000 7f2d4f606640 1 -- 172.21.15.164:0/1952354255 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4819fde0 con 0x7f2d481026c0 2024-09-16T01:36:17.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.252+0000 7f2d3e7fc640 1 -- 172.21.15.164:0/1952354255 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d44009c30 con 0x7f2d481026c0 2024-09-16T01:36:17.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.252+0000 7f2d3e7fc640 1 -- 172.21.15.164:0/1952354255 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4400f6b0 con 0x7f2d481026c0 2024-09-16T01:36:17.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.252+0000 7f2d4f606640 1 -- 172.21.15.164:0/1952354255 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d481a0230 con 0x7f2d481026c0 2024-09-16T01:36:17.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.253+0000 7f2d4f606640 1 -- 172.21.15.164:0/1952354255 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d480fdd70 con 0x7f2d481026c0 2024-09-16T01:36:17.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.255+0000 7f2d3e7fc640 1 -- 172.21.15.164:0/1952354255 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2d440109d0 con 0x7f2d481026c0 2024-09-16T01:36:17.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.255+0000 7f2d3e7fc640 1 --2- 172.21.15.164:0/1952354255 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d340a3270 0x7f2d340a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:36:17.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.255+0000 7f2d3e7fc640 1 -- 172.21.15.164:0/1952354255 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6261+0+0 (secure 0 0 0) 0x7f2d440cf080 con 0x7f2d481026c0 2024-09-16T01:36:17.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.255+0000 7f2d4cba1640 1 --2- 172.21.15.164:0/1952354255 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d340a3270 0x7f2d340a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:36:17.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.256+0000 7f2d4cba1640 1 --2- 172.21.15.164:0/1952354255 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d340a3270 0x7f2d340a5730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2d481993e0 tx=0x7f2d3800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:36:17.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.259+0000 7f2d3e7fc640 1 -- 172.21.15.164:0/1952354255 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d4409c5a0 con 0x7f2d481026c0 2024-09-16T01:36:17.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:17.443+0000 7f2d4f606640 1 -- 172.21.15.164:0/1952354255 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2d48062630 con 0x7f2d340a3270 2024-09-16T01:36:17.788 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:17 smithi164 ceph-mon[35121]: purged_snaps scrub starts 2024-09-16T01:36:17.788 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:17 smithi164 ceph-mon[35121]: purged_snaps scrub ok 2024-09-16T01:36:17.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:17 smithi164 ceph-mon[35121]: Detected new or changed devices on smithi139 2024-09-16T01:36:17.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:17 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi139 to 4272M 2024-09-16T01:36:17.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:17 smithi164 ceph-mon[35121]: osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108] boot 2024-09-16T01:36:17.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:17 smithi164 ceph-mon[35121]: osdmap e59: 10 total, 10 up, 10 in 2024-09-16T01:36:17.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:17.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:36:17.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:36:17.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:17 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:17 smithi139 ceph-mon[31175]: purged_snaps scrub starts 2024-09-16T01:36:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:17 smithi139 ceph-mon[31175]: purged_snaps scrub ok 2024-09-16T01:36:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:17 smithi139 ceph-mon[31175]: Detected new or changed devices on smithi139 2024-09-16T01:36:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:17 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi139 to 4272M 2024-09-16T01:36:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:17 smithi139 ceph-mon[31175]: osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108] boot 2024-09-16T01:36:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:17 smithi139 ceph-mon[31175]: osdmap e59: 10 total, 10 up, 10 in 2024-09-16T01:36:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:36:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:36:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:17 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:17.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:17 smithi176 ceph-mon[35324]: purged_snaps scrub starts 2024-09-16T01:36:17.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:17 smithi176 ceph-mon[35324]: purged_snaps scrub ok 2024-09-16T01:36:17.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:17 smithi176 ceph-mon[35324]: Detected new or changed devices on smithi139 2024-09-16T01:36:17.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:17 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi139 to 4272M 2024-09-16T01:36:17.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:17 smithi176 ceph-mon[35324]: osd.9 [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108] boot 2024-09-16T01:36:17.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:17 smithi176 ceph-mon[35324]: osdmap e59: 10 total, 10 up, 10 in 2024-09-16T01:36:17.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:36:17.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:36:17.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:36:17.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:17 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:18.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:18 smithi139 ceph-mon[31175]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:18.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:18 smithi139 ceph-mon[31175]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:36:18.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:18 smithi139 ceph-mon[31175]: osdmap e60: 10 total, 10 up, 10 in 2024-09-16T01:36:18.927 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:18 smithi164 ceph-mon[35121]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:18.927 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:18 smithi164 ceph-mon[35121]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:36:18.927 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:18 smithi164 ceph-mon[35121]: osdmap e60: 10 total, 10 up, 10 in 2024-09-16T01:36:18.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:18 smithi176 ceph-mon[35324]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-16T01:36:18.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:18 smithi176 ceph-mon[35324]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:36:18.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:18 smithi176 ceph-mon[35324]: osdmap e60: 10 total, 10 up, 10 in 2024-09-16T01:36:19.730 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:19 smithi164 ceph-mon[35121]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:19 smithi139 ceph-mon[31175]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:19 smithi176 ceph-mon[35324]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:20 smithi139 ceph-mon[31175]: from='client.? 172.21.15.164:0/1441111703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32b4d8ce-b644-42fa-80cc-a15a1f5f3a42"}]: dispatch 2024-09-16T01:36:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:20 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32b4d8ce-b644-42fa-80cc-a15a1f5f3a42"}]: dispatch 2024-09-16T01:36:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:20 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32b4d8ce-b644-42fa-80cc-a15a1f5f3a42"}]': finished 2024-09-16T01:36:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:20 smithi139 ceph-mon[31175]: osdmap e61: 11 total, 10 up, 11 in 2024-09-16T01:36:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:20 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:20.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:20 smithi176 ceph-mon[35324]: from='client.? 172.21.15.164:0/1441111703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32b4d8ce-b644-42fa-80cc-a15a1f5f3a42"}]: dispatch 2024-09-16T01:36:20.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:20 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32b4d8ce-b644-42fa-80cc-a15a1f5f3a42"}]: dispatch 2024-09-16T01:36:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:20 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32b4d8ce-b644-42fa-80cc-a15a1f5f3a42"}]': finished 2024-09-16T01:36:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:20 smithi176 ceph-mon[35324]: osdmap e61: 11 total, 10 up, 11 in 2024-09-16T01:36:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:20 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:20.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:20 smithi164 ceph-mon[35121]: from='client.? 172.21.15.164:0/1441111703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32b4d8ce-b644-42fa-80cc-a15a1f5f3a42"}]: dispatch 2024-09-16T01:36:20.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:20 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32b4d8ce-b644-42fa-80cc-a15a1f5f3a42"}]: dispatch 2024-09-16T01:36:20.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:20 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32b4d8ce-b644-42fa-80cc-a15a1f5f3a42"}]': finished 2024-09-16T01:36:20.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:20 smithi164 ceph-mon[35121]: osdmap e61: 11 total, 10 up, 11 in 2024-09-16T01:36:20.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:20 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:21.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:21 smithi139 ceph-mon[31175]: from='client.? 172.21.15.164:0/4188965674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:36:21.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:21 smithi139 ceph-mon[31175]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:21.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:21 smithi176 ceph-mon[35324]: from='client.? 172.21.15.164:0/4188965674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:36:21.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:21 smithi176 ceph-mon[35324]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:22.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:21 smithi164 ceph-mon[35121]: from='client.? 172.21.15.164:0/4188965674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:36:22.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:21 smithi164 ceph-mon[35121]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:24 smithi164 ceph-mon[35121]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:24.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:24 smithi139 ceph-mon[31175]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:24.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:24 smithi176 ceph-mon[35324]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:26.711 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:26 smithi164 ceph-mon[35121]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:26.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:26 smithi139 ceph-mon[31175]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:26.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:26 smithi176 ceph-mon[35324]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:28.774 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:28 smithi164 ceph-mon[35121]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:28 smithi139 ceph-mon[31175]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:28.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:28 smithi176 ceph-mon[35324]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:29.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T01:36:29.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:29.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T01:36:29.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:29.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T01:36:29.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:30.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:30 smithi164 ceph-mon[35121]: Deploying daemon osd.10 on smithi164 2024-09-16T01:36:30.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:30 smithi164 ceph-mon[35121]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:30.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:30 smithi139 ceph-mon[31175]: Deploying daemon osd.10 on smithi164 2024-09-16T01:36:30.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:30 smithi139 ceph-mon[31175]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:30.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:30 smithi176 ceph-mon[35324]: Deploying daemon osd.10 on smithi164 2024-09-16T01:36:30.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:30 smithi176 ceph-mon[35324]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:32.577 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:32 smithi164 ceph-mon[35121]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:32.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:32 smithi139 ceph-mon[31175]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:32.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:32 smithi176 ceph-mon[35324]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:34.748 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:34 smithi139 ceph-mon[31175]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:34.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:34 smithi164 ceph-mon[35121]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:34.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:34 smithi176 ceph-mon[35324]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:35.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:35 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:36:35.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:35 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:35.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:35 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:35.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:36:35.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:35.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:35 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:35.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:35 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:36:35.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:35 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:35.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:35 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:36.622 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:36 smithi164 ceph-mon[35121]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:36.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:36 smithi139 ceph-mon[31175]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:36.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:36 smithi176 ceph-mon[35324]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:38.257 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 10 on host 'smithi164' 2024-09-16T01:36:38.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:38.255+0000 7f2d3e7fc640 1 -- 172.21.15.164:0/1952354255 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2d48062630 con 0x7f2d340a3270 2024-09-16T01:36:38.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:38.258+0000 7f2d4f606640 1 -- 172.21.15.164:0/1952354255 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d340a3270 msgr2=0x7f2d340a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:36:38.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:38.258+0000 7f2d4f606640 1 --2- 172.21.15.164:0/1952354255 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d340a3270 0x7f2d340a5730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2d481993e0 tx=0x7f2d3800b410 comp rx=0 tx=0).stop 2024-09-16T01:36:38.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:38.258+0000 7f2d4f606640 1 -- 172.21.15.164:0/1952354255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d481026c0 msgr2=0x7f2d48197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:36:38.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:38.258+0000 7f2d4f606640 1 --2- 172.21.15.164:0/1952354255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d481026c0 0x7f2d48197ec0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400d950 tx=0x7f2d4400de10 comp rx=0 tx=0).stop 2024-09-16T01:36:38.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:38.258+0000 7f2d4f606640 1 -- 172.21.15.164:0/1952354255 shutdown_connections 2024-09-16T01:36:38.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:38.258+0000 7f2d4f606640 1 --2- 172.21.15.164:0/1952354255 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d340a3270 0x7f2d340a5730 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:38.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:38.258+0000 7f2d4f606640 1 --2- 172.21.15.164:0/1952354255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d48107920 0x7f2d4819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:38.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:38.258+0000 7f2d4f606640 1 --2- 172.21.15.164:0/1952354255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d48104ff0 0x7f2d48198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:38.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:38.258+0000 7f2d4f606640 1 --2- 172.21.15.164:0/1952354255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d481026c0 0x7f2d48197ec0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:38.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:38.258+0000 7f2d4f606640 1 -- 172.21.15.164:0/1952354255 >> 172.21.15.164:0/1952354255 conn(0x7f2d480fc280 msgr2=0x7f2d48102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:36:38.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:38.258+0000 7f2d4f606640 1 -- 172.21.15.164:0/1952354255 shutdown_connections 2024-09-16T01:36:38.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T01:36:38.258+0000 7f2d4f606640 1 -- 172.21.15.164:0/1952354255 wait complete. 2024-09-16T01:36:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:38 smithi164 ceph-mon[35121]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:38 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:38 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:38 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:38 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:36:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:38 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:38 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:36:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:38 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:38 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:38.868 DEBUG:teuthology.orchestra.run.smithi164:osd.10> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.10.service 2024-09-16T01:36:38.871 INFO:tasks.cephadm:Deploying osd.11 on smithi176 with /dev/vg_nvme/lv_1... 2024-09-16T01:36:38.871 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T01:36:38.899 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:38 smithi176 ceph-mon[35324]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:38.899 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:38 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:38.899 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:38 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:38.900 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:38 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:38.900 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:38 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:36:38.900 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:38 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:38.900 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:38 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:36:38.900 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:38 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:38.900 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:38 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:38.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:38 smithi139 ceph-mon[31175]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:38.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:38 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:38.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:38 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:38.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:38 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:38.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:38 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:36:38.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:38 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:38.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:38 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:36:38.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:38 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:38.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:38 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:39.129 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:36:40.614 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:40 smithi164 ceph-mon[35121]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:40.659 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:40 smithi176 ceph-mon[35324]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:40.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:40 smithi139 ceph-mon[31175]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:41.045 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 01:36:40 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T01:36:40.612+0000 7f781f87e640 -1 osd.10 0 log_to_monitors true 2024-09-16T01:36:41.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:41 smithi176 ceph-mon[35324]: from='osd.10 [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T01:36:41.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:41 smithi176 ceph-mon[35324]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T01:36:41.768 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-16T01:36:41.783 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:41 smithi164 ceph-mon[35121]: from='osd.10 [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T01:36:41.783 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:41 smithi164 ceph-mon[35121]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T01:36:41.794 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_1 2024-09-16T01:36:41.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:41 smithi139 ceph-mon[31175]: from='osd.10 [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T01:36:41.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:41 smithi139 ceph-mon[31175]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T01:36:42.088 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:36:42.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.575+0000 7f3a6b228640 1 -- 172.21.15.176:0/395497766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a64101d10 msgr2=0x7f3a64102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:36:42.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.575+0000 7f3a6b228640 1 --2- 172.21.15.176:0/395497766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a64101d10 0x7f3a64102190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009900 tx=0x7f3a5402a410 comp rx=0 tx=0).stop 2024-09-16T01:36:42.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.576+0000 7f3a6b228640 1 -- 172.21.15.176:0/395497766 shutdown_connections 2024-09-16T01:36:42.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.576+0000 7f3a6b228640 1 --2- 172.21.15.176:0/395497766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a641026d0 0x7f3a64108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:42.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.576+0000 7f3a6b228640 1 --2- 172.21.15.176:0/395497766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a64101d10 0x7f3a64102190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:42.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.576+0000 7f3a6b228640 1 --2- 172.21.15.176:0/395497766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a64100b10 0x7f3a64100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:42.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.576+0000 7f3a6b228640 1 -- 172.21.15.176:0/395497766 >> 172.21.15.176:0/395497766 conn(0x7f3a640fc2a0 msgr2=0x7f3a640fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:36:42.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.576+0000 7f3a6b228640 1 -- 172.21.15.176:0/395497766 shutdown_connections 2024-09-16T01:36:42.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.576+0000 7f3a6b228640 1 -- 172.21.15.176:0/395497766 wait complete. 2024-09-16T01:36:42.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.576+0000 7f3a6b228640 1 Processor -- start 2024-09-16T01:36:42.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.576+0000 7f3a6b228640 1 -- start start 2024-09-16T01:36:42.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a6b228640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a64100b10 0x7f3a64196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:36:42.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a6b228640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a64101d10 0x7f3a64197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:36:42.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a6b228640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a641026d0 0x7f3a6419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:36:42.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a6b228640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6410c790 con 0x7f3a64100b10 2024-09-16T01:36:42.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a6b228640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6410c610 con 0x7f3a64101d10 2024-09-16T01:36:42.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a6b228640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6410c910 con 0x7f3a641026d0 2024-09-16T01:36:42.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a697c5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a641026d0 0x7f3a6419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:36:42.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a697c5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a641026d0 0x7f3a6419e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37406/0 (socket says 172.21.15.176:37406) 2024-09-16T01:36:42.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a697c5640 1 -- 172.21.15.176:0/2724148436 learned_addr learned my addr 172.21.15.176:0/2724148436 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-16T01:36:42.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a5bfff640 1 --2- 172.21.15.176:0/2724148436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a64101d10 0x7f3a64197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:36:42.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a68fc4640 1 --2- 172.21.15.176:0/2724148436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a64100b10 0x7f3a64196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:36:42.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a697c5640 1 -- 172.21.15.176:0/2724148436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a64101d10 msgr2=0x7f3a64197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:36:42.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a697c5640 1 --2- 172.21.15.176:0/2724148436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a64101d10 0x7f3a64197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:42.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a697c5640 1 -- 172.21.15.176:0/2724148436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a64100b10 msgr2=0x7f3a64196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:36:42.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a697c5640 1 --2- 172.21.15.176:0/2724148436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a64100b10 0x7f3a64196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:36:42.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a697c5640 1 -- 172.21.15.176:0/2724148436 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6406a810 con 0x7f3a641026d0 2024-09-16T01:36:42.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a697c5640 1 --2- 172.21.15.176:0/2724148436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a641026d0 0x7f3a6419e400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009fd0 tx=0x7f3a5c00b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:36:42.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a59ffb640 1 -- 172.21.15.176:0/2724148436 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c017070 con 0x7f3a641026d0 2024-09-16T01:36:42.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a6b228640 1 -- 172.21.15.176:0/2724148436 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6406ab00 con 0x7f3a641026d0 2024-09-16T01:36:42.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a59ffb640 1 -- 172.21.15.176:0/2724148436 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a5c003e50 con 0x7f3a641026d0 2024-09-16T01:36:42.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.577+0000 7f3a59ffb640 1 -- 172.21.15.176:0/2724148436 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c0106c0 con 0x7f3a641026d0 2024-09-16T01:36:42.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.578+0000 7f3a6b228640 1 -- 172.21.15.176:0/2724148436 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a6406afc0 con 0x7f3a641026d0 2024-09-16T01:36:42.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.578+0000 7f3a6b228640 1 -- 172.21.15.176:0/2724148436 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a64107f60 con 0x7f3a641026d0 2024-09-16T01:36:42.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.579+0000 7f3a59ffb640 1 -- 172.21.15.176:0/2724148436 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3a5c003970 con 0x7f3a641026d0 2024-09-16T01:36:42.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.580+0000 7f3a59ffb640 1 --2- 172.21.15.176:0/2724148436 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3a4c0a3270 0x7f3a4c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:36:42.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.580+0000 7f3a59ffb640 1 -- 172.21.15.176:0/2724148436 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6414+0+0 (secure 0 0 0) 0x7f3a5c0cee50 con 0x7f3a641026d0 2024-09-16T01:36:42.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.580+0000 7f3a68fc4640 1 --2- 172.21.15.176:0/2724148436 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3a4c0a3270 0x7f3a4c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:36:42.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.581+0000 7f3a68fc4640 1 --2- 172.21.15.176:0/2724148436 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3a4c0a3270 0x7f3a4c0a5730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3a60005fd0 tx=0x7f3a6000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:36:42.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.584+0000 7f3a59ffb640 1 -- 172.21.15.176:0/2724148436 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a5c09bc90 con 0x7f3a641026d0 2024-09-16T01:36:42.761 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:36:42.758+0000 7f3a6b228640 1 -- 172.21.15.176:0/2724148436 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3a6406b290 con 0x7f3a4c0a3270 2024-09-16T01:36:42.761 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:42.761 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T01:36:42.761 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: from='osd.10 [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:36:42.761 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: osdmap e62: 11 total, 10 up, 11 in 2024-09-16T01:36:42.761 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:42.761 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:36:42.761 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:42.762 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:42.762 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:42.762 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:42.762 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:42.762 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:42.762 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:42.762 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:42.762 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:36:42.762 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:42 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:42.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:42.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T01:36:42.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: from='osd.10 [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:36:42.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: osdmap e62: 11 total, 10 up, 11 in 2024-09-16T01:36:42.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:42.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:36:42.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:42.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:42.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:42.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:42.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:42.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:42.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:42.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:42.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:36:42.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:42.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:42.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T01:36:42.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: from='osd.10 [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:36:42.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: osdmap e62: 11 total, 10 up, 11 in 2024-09-16T01:36:42.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:42.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T01:36:42.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:42.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:42.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:42.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:42.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:42.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:36:42.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:42.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:42.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:36:42.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:42 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:36:43.295 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T01:36:42.940+0000 7f781b801640 -1 osd.10 0 waiting for initial osdmap 2024-09-16T01:36:43.295 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 01:36:42 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T01:36:42.950+0000 7f7813621640 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:36:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:43 smithi164 ceph-mon[35121]: Detected new or changed devices on smithi164 2024-09-16T01:36:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:43 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi164 to 4272M 2024-09-16T01:36:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:43 smithi164 ceph-mon[35121]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T01:36:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:43 smithi164 ceph-mon[35121]: osdmap e63: 11 total, 10 up, 11 in 2024-09-16T01:36:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:43 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:43 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:43 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:36:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:43 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:36:43.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:43 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:43.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:43 smithi164 ceph-mon[35121]: from='osd.10 ' entity='osd.10' 2024-09-16T01:36:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:43 smithi139 ceph-mon[31175]: Detected new or changed devices on smithi164 2024-09-16T01:36:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:43 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi164 to 4272M 2024-09-16T01:36:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:43 smithi139 ceph-mon[31175]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T01:36:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:43 smithi139 ceph-mon[31175]: osdmap e63: 11 total, 10 up, 11 in 2024-09-16T01:36:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:43 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:43 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:43 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:36:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:43 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:36:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:43 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:43 smithi139 ceph-mon[31175]: from='osd.10 ' entity='osd.10' 2024-09-16T01:36:43.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:43 smithi176 ceph-mon[35324]: Detected new or changed devices on smithi164 2024-09-16T01:36:43.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:43 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi164 to 4272M 2024-09-16T01:36:43.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:43 smithi176 ceph-mon[35324]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T01:36:43.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:43 smithi176 ceph-mon[35324]: osdmap e63: 11 total, 10 up, 11 in 2024-09-16T01:36:43.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:43 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:43.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:43 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:43.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:43 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T01:36:43.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:43 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T01:36:43.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:43 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:43.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:43 smithi176 ceph-mon[35324]: from='osd.10 ' entity='osd.10' 2024-09-16T01:36:44.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:44 smithi164 ceph-mon[35121]: purged_snaps scrub starts 2024-09-16T01:36:44.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:44 smithi164 ceph-mon[35121]: purged_snaps scrub ok 2024-09-16T01:36:44.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:44 smithi164 ceph-mon[35121]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:36:44.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:44 smithi164 ceph-mon[35121]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:44.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:44.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:44 smithi164 ceph-mon[35121]: osd.10 [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828] boot 2024-09-16T01:36:44.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:44 smithi164 ceph-mon[35121]: osdmap e64: 11 total, 11 up, 11 in 2024-09-16T01:36:44.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:44 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:44.905 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:44 smithi176 ceph-mon[35324]: purged_snaps scrub starts 2024-09-16T01:36:44.905 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:44 smithi176 ceph-mon[35324]: purged_snaps scrub ok 2024-09-16T01:36:44.905 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:44 smithi176 ceph-mon[35324]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:36:44.905 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:44 smithi176 ceph-mon[35324]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:44.905 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:44.905 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:44 smithi176 ceph-mon[35324]: osd.10 [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828] boot 2024-09-16T01:36:44.905 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:44 smithi176 ceph-mon[35324]: osdmap e64: 11 total, 11 up, 11 in 2024-09-16T01:36:44.905 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:44 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:44 smithi139 ceph-mon[31175]: purged_snaps scrub starts 2024-09-16T01:36:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:44 smithi139 ceph-mon[31175]: purged_snaps scrub ok 2024-09-16T01:36:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:44 smithi139 ceph-mon[31175]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:36:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:44 smithi139 ceph-mon[31175]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-16T01:36:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:44 smithi139 ceph-mon[31175]: osd.10 [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828] boot 2024-09-16T01:36:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:44 smithi139 ceph-mon[31175]: osdmap e64: 11 total, 11 up, 11 in 2024-09-16T01:36:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:44 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:36:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:45 smithi164 ceph-mon[35121]: osdmap e65: 11 total, 11 up, 11 in 2024-09-16T01:36:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:45 smithi164 ceph-mon[35121]: from='client.? 172.21.15.176:0/171317' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49098bdc-58a9-4a74-8045-d408af7711ef"}]: dispatch 2024-09-16T01:36:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:45 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49098bdc-58a9-4a74-8045-d408af7711ef"}]: dispatch 2024-09-16T01:36:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:45 smithi164 ceph-mon[35121]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49098bdc-58a9-4a74-8045-d408af7711ef"}]': finished 2024-09-16T01:36:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:45 smithi164 ceph-mon[35121]: osdmap e66: 12 total, 11 up, 12 in 2024-09-16T01:36:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:45 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:36:45.867 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:45 smithi176 ceph-mon[35324]: osdmap e65: 11 total, 11 up, 11 in 2024-09-16T01:36:45.867 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:45 smithi176 ceph-mon[35324]: from='client.? 172.21.15.176:0/171317' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49098bdc-58a9-4a74-8045-d408af7711ef"}]: dispatch 2024-09-16T01:36:45.867 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:45 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49098bdc-58a9-4a74-8045-d408af7711ef"}]: dispatch 2024-09-16T01:36:45.867 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:45 smithi176 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49098bdc-58a9-4a74-8045-d408af7711ef"}]': finished 2024-09-16T01:36:45.867 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:45 smithi176 ceph-mon[35324]: osdmap e66: 12 total, 11 up, 12 in 2024-09-16T01:36:45.868 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:45 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:36:45.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:45 smithi139 ceph-mon[31175]: osdmap e65: 11 total, 11 up, 11 in 2024-09-16T01:36:45.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:45 smithi139 ceph-mon[31175]: from='client.? 172.21.15.176:0/171317' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49098bdc-58a9-4a74-8045-d408af7711ef"}]: dispatch 2024-09-16T01:36:45.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:45 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49098bdc-58a9-4a74-8045-d408af7711ef"}]: dispatch 2024-09-16T01:36:45.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:45 smithi139 ceph-mon[31175]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49098bdc-58a9-4a74-8045-d408af7711ef"}]': finished 2024-09-16T01:36:45.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:45 smithi139 ceph-mon[31175]: osdmap e66: 12 total, 11 up, 12 in 2024-09-16T01:36:45.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:45 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:36:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:46 smithi164 ceph-mon[35121]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:36:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:46 smithi164 ceph-mon[35121]: osdmap e67: 12 total, 11 up, 12 in 2024-09-16T01:36:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:46 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:36:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:46 smithi164 ceph-mon[35121]: from='client.? 172.21.15.176:0/2253503737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:36:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:46 smithi139 ceph-mon[31175]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:36:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:46 smithi139 ceph-mon[31175]: osdmap e67: 12 total, 11 up, 12 in 2024-09-16T01:36:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:46 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:36:46.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:46 smithi139 ceph-mon[31175]: from='client.? 172.21.15.176:0/2253503737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:36:46.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:46 smithi176 ceph-mon[35324]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:36:46.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:46 smithi176 ceph-mon[35324]: osdmap e67: 12 total, 11 up, 12 in 2024-09-16T01:36:46.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:46 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:36:46.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:46 smithi176 ceph-mon[35324]: from='client.? 172.21.15.176:0/2253503737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T01:36:48.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:47 smithi164 ceph-mon[35121]: osdmap e68: 12 total, 11 up, 12 in 2024-09-16T01:36:48.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:47 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:36:48.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:47 smithi164 ceph-mon[35121]: pgmap v228: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-16T01:36:48.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:47 smithi139 ceph-mon[31175]: osdmap e68: 12 total, 11 up, 12 in 2024-09-16T01:36:48.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:47 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:36:48.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:47 smithi139 ceph-mon[31175]: pgmap v228: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-16T01:36:48.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:47 smithi176 ceph-mon[35324]: osdmap e68: 12 total, 11 up, 12 in 2024-09-16T01:36:48.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:47 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:36:48.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:47 smithi176 ceph-mon[35324]: pgmap v228: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-16T01:36:49.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:48 smithi164 ceph-mon[35121]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-16T01:36:49.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:48 smithi139 ceph-mon[31175]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-16T01:36:49.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:48 smithi176 ceph-mon[35324]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-16T01:36:50.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:49 smithi164 ceph-mon[35121]: pgmap v229: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-16T01:36:50.092 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:49 smithi176 ceph-mon[35324]: pgmap v229: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-16T01:36:50.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:49 smithi139 ceph-mon[31175]: pgmap v229: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-16T01:36:51.769 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:51 smithi176 ceph-mon[35324]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-16T01:36:51.769 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:51 smithi176 ceph-mon[35324]: Cluster is now healthy 2024-09-16T01:36:51.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:51 smithi164 ceph-mon[35121]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-16T01:36:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:51 smithi164 ceph-mon[35121]: Cluster is now healthy 2024-09-16T01:36:51.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:51 smithi139 ceph-mon[31175]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-16T01:36:51.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:51 smithi139 ceph-mon[31175]: Cluster is now healthy 2024-09-16T01:36:52.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:52 smithi176 ceph-mon[35324]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-16T01:36:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:52 smithi164 ceph-mon[35121]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-16T01:36:52.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:52 smithi139 ceph-mon[31175]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-16T01:36:54.762 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:54 smithi176 ceph-mon[35324]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T01:36:54.762 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:54 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T01:36:54.762 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:54 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:54 smithi164 ceph-mon[35121]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T01:36:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:54 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T01:36:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:54 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:54.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:54 smithi139 ceph-mon[31175]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T01:36:54.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:54 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T01:36:54.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:54 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:36:55.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:55 smithi176 ceph-mon[35324]: Deploying daemon osd.11 on smithi176 2024-09-16T01:36:55.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:55 smithi164 ceph-mon[35121]: Deploying daemon osd.11 on smithi176 2024-09-16T01:36:55.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:55 smithi139 ceph-mon[31175]: Deploying daemon osd.11 on smithi176 2024-09-16T01:36:56.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:56 smithi164 ceph-mon[35121]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-16T01:36:56.856 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:56 smithi176 ceph-mon[35324]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-16T01:36:56.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:56 smithi139 ceph-mon[31175]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-16T01:36:58.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:36:58 smithi164 ceph-mon[35121]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-16T01:36:58.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:36:58 smithi139 ceph-mon[31175]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-16T01:36:58.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:36:58 smithi176 ceph-mon[35324]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-16T01:37:00.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:00 smithi164 ceph-mon[35121]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:37:00.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:00.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:00.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:00.839 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:00 smithi176 ceph-mon[35324]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:37:00.839 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:00.840 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:00.840 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:00.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:00 smithi139 ceph-mon[31175]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:37:00.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:00.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:00.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:02 smithi164 ceph-mon[35121]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:37:02.867 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:02 smithi176 ceph-mon[35324]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:37:02.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:02 smithi139 ceph-mon[31175]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T01:37:03.573 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 11 on host 'smithi176' 2024-09-16T01:37:03.573 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:03.572+0000 7f3a59ffb640 1 -- 172.21.15.176:0/2724148436 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f3a6406b290 con 0x7f3a4c0a3270 2024-09-16T01:37:03.575 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:03.574+0000 7f3a6b228640 1 -- 172.21.15.176:0/2724148436 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3a4c0a3270 msgr2=0x7f3a4c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:03.575 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:03.574+0000 7f3a6b228640 1 --2- 172.21.15.176:0/2724148436 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3a4c0a3270 0x7f3a4c0a5730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3a60005fd0 tx=0x7f3a6000b390 comp rx=0 tx=0).stop 2024-09-16T01:37:03.576 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:03.574+0000 7f3a6b228640 1 -- 172.21.15.176:0/2724148436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a641026d0 msgr2=0x7f3a6419e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:03.576 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:03.574+0000 7f3a6b228640 1 --2- 172.21.15.176:0/2724148436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a641026d0 0x7f3a6419e400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009fd0 tx=0x7f3a5c00b260 comp rx=0 tx=0).stop 2024-09-16T01:37:03.576 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:03.574+0000 7f3a6b228640 1 -- 172.21.15.176:0/2724148436 shutdown_connections 2024-09-16T01:37:03.576 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:03.574+0000 7f3a6b228640 1 --2- 172.21.15.176:0/2724148436 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3a4c0a3270 0x7f3a4c0a5730 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:03.576 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:03.575+0000 7f3a6b228640 1 --2- 172.21.15.176:0/2724148436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a641026d0 0x7f3a6419e400 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:03.576 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:03.575+0000 7f3a6b228640 1 --2- 172.21.15.176:0/2724148436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a64101d10 0x7f3a64197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:03.576 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:03.575+0000 7f3a6b228640 1 --2- 172.21.15.176:0/2724148436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a64100b10 0x7f3a64196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:03.576 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:03.575+0000 7f3a6b228640 1 -- 172.21.15.176:0/2724148436 >> 172.21.15.176:0/2724148436 conn(0x7f3a640fc2a0 msgr2=0x7f3a64105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:03.576 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:03.575+0000 7f3a6b228640 1 -- 172.21.15.176:0/2724148436 shutdown_connections 2024-09-16T01:37:03.576 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:03.575+0000 7f3a6b228640 1 -- 172.21.15.176:0/2724148436 wait complete. 2024-09-16T01:37:04.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:03 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:03 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:03 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:04.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:03 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:04.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:03 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:03 smithi139 ceph-mon[31175]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:37:04.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:03 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:04.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:03 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:03 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:03 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:03 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:03 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:04.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:03 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:04.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:03 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:03 smithi176 ceph-mon[35324]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:37:04.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:03 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:04.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:03 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:03 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.246 DEBUG:teuthology.orchestra.run.smithi176:osd.11> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.11.service 2024-09-16T01:37:04.249 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-16T01:37:04.249 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T01:37:04.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:03 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:03 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:03 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:04.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:03 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:04.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:03 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:03 smithi164 ceph-mon[35121]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:37:04.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:03 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:04.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:03 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:03 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:04.510 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:05.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.992+0000 7f890fcee640 1 -- 172.21.15.139:0/3305504763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8908101c00 msgr2=0x7f8908102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:05.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.992+0000 7f890fcee640 1 --2- 172.21.15.139:0/3305504763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8908101c00 0x7f8908102000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009900 tx=0x7f88f802a410 comp rx=0 tx=0).stop 2024-09-16T01:37:05.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.992+0000 7f890fcee640 1 -- 172.21.15.139:0/3305504763 shutdown_connections 2024-09-16T01:37:05.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.992+0000 7f890fcee640 1 --2- 172.21.15.139:0/3305504763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89081037c0 0x7f890810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:05.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.992+0000 7f890fcee640 1 --2- 172.21.15.139:0/3305504763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8908102e00 0x7f8908103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:05.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.992+0000 7f890fcee640 1 --2- 172.21.15.139:0/3305504763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8908101c00 0x7f8908102000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:05.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.992+0000 7f890fcee640 1 -- 172.21.15.139:0/3305504763 >> 172.21.15.139:0/3305504763 conn(0x7f89080fd390 msgr2=0x7f89080ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:05.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.993+0000 7f890fcee640 1 -- 172.21.15.139:0/3305504763 shutdown_connections 2024-09-16T01:37:05.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.993+0000 7f890fcee640 1 -- 172.21.15.139:0/3305504763 wait complete. 2024-09-16T01:37:05.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.993+0000 7f890fcee640 1 Processor -- start 2024-09-16T01:37:05.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.993+0000 7f890fcee640 1 -- start start 2024-09-16T01:37:05.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.993+0000 7f890fcee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8908101c00 0x7f8908197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:05.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.993+0000 7f890fcee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8908102e00 0x7f89081984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:05.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.993+0000 7f890fcee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89081037c0 0x7f890819f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:05.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.993+0000 7f890fcee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f890810d8e0 con 0x7f89081037c0 2024-09-16T01:37:05.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.993+0000 7f890fcee640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f890810d760 con 0x7f8908102e00 2024-09-16T01:37:05.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.993+0000 7f890fcee640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f890810da60 con 0x7f8908101c00 2024-09-16T01:37:05.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.993+0000 7f890e28b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89081037c0 0x7f890819f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:05.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.993+0000 7f890e28b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89081037c0 0x7f890819f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51542/0 (socket says 172.21.15.139:51542) 2024-09-16T01:37:05.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.993+0000 7f890e28b640 1 -- 172.21.15.139:0/4048774275 learned_addr learned my addr 172.21.15.139:0/4048774275 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:05.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.993+0000 7f890da8a640 1 --2- 172.21.15.139:0/4048774275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8908101c00 0x7f8908197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:05.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.994+0000 7f890d289640 1 --2- 172.21.15.139:0/4048774275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8908102e00 0x7f89081984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:05.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.994+0000 7f890e28b640 1 -- 172.21.15.139:0/4048774275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8908101c00 msgr2=0x7f8908197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:05.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.994+0000 7f890e28b640 1 --2- 172.21.15.139:0/4048774275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8908101c00 0x7f8908197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:05.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.994+0000 7f890e28b640 1 -- 172.21.15.139:0/4048774275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8908102e00 msgr2=0x7f89081984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:05.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.994+0000 7f890e28b640 1 --2- 172.21.15.139:0/4048774275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8908102e00 0x7f89081984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:05.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.994+0000 7f890e28b640 1 -- 172.21.15.139:0/4048774275 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890819fb90 con 0x7f89081037c0 2024-09-16T01:37:05.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.994+0000 7f890e28b640 1 --2- 172.21.15.139:0/4048774275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89081037c0 0x7f890819f520 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f890000eaa0 tx=0x7f890000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:05.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.994+0000 7f88feffd640 1 -- 172.21.15.139:0/4048774275 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890000cdb0 con 0x7f89081037c0 2024-09-16T01:37:05.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.994+0000 7f88feffd640 1 -- 172.21.15.139:0/4048774275 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8900005020 con 0x7f89081037c0 2024-09-16T01:37:05.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.994+0000 7f890fcee640 1 -- 172.21.15.139:0/4048774275 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890819fe80 con 0x7f89081037c0 2024-09-16T01:37:05.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.994+0000 7f88feffd640 1 -- 172.21.15.139:0/4048774275 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89000116b0 con 0x7f89081037c0 2024-09-16T01:37:05.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.994+0000 7f890fcee640 1 -- 172.21.15.139:0/4048774275 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89081a0360 con 0x7f89081037c0 2024-09-16T01:37:05.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.995+0000 7f890fcee640 1 -- 172.21.15.139:0/4048774275 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8908109070 con 0x7f89081037c0 2024-09-16T01:37:05.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.996+0000 7f88feffd640 1 -- 172.21.15.139:0/4048774275 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8900005560 con 0x7f89081037c0 2024-09-16T01:37:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.997+0000 7f88feffd640 1 --2- 172.21.15.139:0/4048774275 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f88f40a3270 0x7f88f40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.997+0000 7f890da8a640 1 --2- 172.21.15.139:0/4048774275 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f88f40a3270 0x7f88f40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.997+0000 7f88feffd640 1 -- 172.21.15.139:0/4048774275 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6809+0+0 (secure 0 0 0) 0x7f89000cf560 con 0x7f89081037c0 2024-09-16T01:37:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:04.997+0000 7f890da8a640 1 --2- 172.21.15.139:0/4048774275 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f88f40a3270 0x7f88f40a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f88f8005ab0 tx=0x7f88f8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.001+0000 7f88feffd640 1 -- 172.21.15.139:0/4048774275 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8900016030 con 0x7f89081037c0 2024-09-16T01:37:05.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.171+0000 7f890fcee640 1 -- 172.21.15.139:0/4048774275 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f890802da10 con 0x7f89081037c0 2024-09-16T01:37:05.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.171+0000 7f88feffd640 1 -- 172.21.15.139:0/4048774275 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f890009c860 con 0x7f89081037c0 2024-09-16T01:37:05.173 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:05.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.173+0000 7f890fcee640 1 -- 172.21.15.139:0/4048774275 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f88f40a3270 msgr2=0x7f88f40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:05.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.173+0000 7f890fcee640 1 --2- 172.21.15.139:0/4048774275 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f88f40a3270 0x7f88f40a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f88f8005ab0 tx=0x7f88f8000bc0 comp rx=0 tx=0).stop 2024-09-16T01:37:05.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.173+0000 7f890fcee640 1 -- 172.21.15.139:0/4048774275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89081037c0 msgr2=0x7f890819f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:05.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.173+0000 7f890fcee640 1 --2- 172.21.15.139:0/4048774275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89081037c0 0x7f890819f520 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f890000eaa0 tx=0x7f890000ef60 comp rx=0 tx=0).stop 2024-09-16T01:37:05.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.173+0000 7f890fcee640 1 -- 172.21.15.139:0/4048774275 shutdown_connections 2024-09-16T01:37:05.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.173+0000 7f890fcee640 1 --2- 172.21.15.139:0/4048774275 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f88f40a3270 0x7f88f40a5730 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:05.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.173+0000 7f890fcee640 1 --2- 172.21.15.139:0/4048774275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89081037c0 0x7f890819f520 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:05.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.173+0000 7f890fcee640 1 --2- 172.21.15.139:0/4048774275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8908102e00 0x7f89081984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:05.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.173+0000 7f890fcee640 1 --2- 172.21.15.139:0/4048774275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8908101c00 0x7f8908197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:05.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.173+0000 7f890fcee640 1 -- 172.21.15.139:0/4048774275 >> 172.21.15.139:0/4048774275 conn(0x7f89080fd390 msgr2=0x7f8908106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:05.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.173+0000 7f890fcee640 1 -- 172.21.15.139:0/4048774275 shutdown_connections 2024-09-16T01:37:05.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:05.173+0000 7f890fcee640 1 -- 172.21.15.139:0/4048774275 wait complete. 2024-09-16T01:37:05.416 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:05 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/4048774275' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:37:05.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:05 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/4048774275' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:37:05.656 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:05 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/4048774275' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:37:05.983 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726450603,"num_in_osds":12,"osd_in_since":1726450605,"num_remapped_pgs":0} 2024-09-16T01:37:06.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:06 smithi176 ceph-mon[35324]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:37:06.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:06 smithi176 ceph-mon[35324]: from='osd.11 [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T01:37:06.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:06 smithi176 ceph-mon[35324]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T01:37:06.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 01:37:06 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T01:37:06.129+0000 7fefd7e4d640 -1 osd.11 0 log_to_monitors true 2024-09-16T01:37:06.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:06 smithi164 ceph-mon[35121]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:37:06.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:06 smithi164 ceph-mon[35121]: from='osd.11 [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T01:37:06.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:06 smithi164 ceph-mon[35121]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T01:37:06.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:06 smithi139 ceph-mon[31175]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:37:06.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:06 smithi139 ceph-mon[31175]: from='osd.11 [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T01:37:06.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:06 smithi139 ceph-mon[31175]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T01:37:06.983 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T01:37:07.239 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T01:37:07.239 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: osdmap e69: 12 total, 11 up, 12 in 2024-09-16T01:37:07.239 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:07.239 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: from='osd.11 [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:37:07.239 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:37:07.239 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:07.239 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:07.240 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:37:07.240 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:37:07.240 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:37:07.240 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:37:07.240 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:07.240 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:07.240 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:07.240 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:07.249 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T01:37:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: osdmap e69: 12 total, 11 up, 12 in 2024-09-16T01:37:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: from='osd.11 [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:37:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:37:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:37:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:37:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:37:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:37:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:07.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:07.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:07.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T01:37:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: osdmap e69: 12 total, 11 up, 12 in 2024-09-16T01:37:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: from='osd.11 [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:37:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-09-16T01:37:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:07.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:37:07.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:37:07.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:37:07.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:37:07.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:07.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:07.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:07.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:07.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.734+0000 7f41a08cd640 1 -- 172.21.15.139:0/2420624830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c1078c0 msgr2=0x7f419c109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:07.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.734+0000 7f41a08cd640 1 --2- 172.21.15.139:0/2420624830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c1078c0 0x7f419c109d10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4194009ca0 tx=0x7f419402a8c0 comp rx=0 tx=0).stop 2024-09-16T01:37:07.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.734+0000 7f41a08cd640 1 -- 172.21.15.139:0/2420624830 shutdown_connections 2024-09-16T01:37:07.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.734+0000 7f41a08cd640 1 --2- 172.21.15.139:0/2420624830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c1078c0 0x7f419c109d10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:07.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.734+0000 7f41a08cd640 1 --2- 172.21.15.139:0/2420624830 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f419c104f90 0x7f419c107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:07.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.734+0000 7f41a08cd640 1 --2- 172.21.15.139:0/2420624830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f419c102660 0x7f419c104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:07.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.734+0000 7f41a08cd640 1 -- 172.21.15.139:0/2420624830 >> 172.21.15.139:0/2420624830 conn(0x7f419c0fc280 msgr2=0x7f419c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:07.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.734+0000 7f41a08cd640 1 -- 172.21.15.139:0/2420624830 shutdown_connections 2024-09-16T01:37:07.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.734+0000 7f41a08cd640 1 -- 172.21.15.139:0/2420624830 wait complete. 2024-09-16T01:37:07.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f41a08cd640 1 Processor -- start 2024-09-16T01:37:07.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f41a08cd640 1 -- start start 2024-09-16T01:37:07.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f41a08cd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c102660 0x7f419c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:07.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f41a08cd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f419c104f90 0x7f419c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:07.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f41a08cd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f419c1078c0 0x7f419c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:07.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f41a08cd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f419c10d8d0 con 0x7f419c102660 2024-09-16T01:37:07.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f41a08cd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f419c10d750 con 0x7f419c1078c0 2024-09-16T01:37:07.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f41a08cd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f419c10da50 con 0x7f419c104f90 2024-09-16T01:37:07.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f419a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c102660 0x7f419c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:07.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f419a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c102660 0x7f419c197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51554/0 (socket says 172.21.15.139:51554) 2024-09-16T01:37:07.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f419a59c640 1 -- 172.21.15.139:0/14974459 learned_addr learned my addr 172.21.15.139:0/14974459 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:07.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f4199d9b640 1 --2- 172.21.15.139:0/14974459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f419c104f90 0x7f419c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:07.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f419ad9d640 1 --2- 172.21.15.139:0/14974459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f419c1078c0 0x7f419c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:07.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f419a59c640 1 -- 172.21.15.139:0/14974459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f419c104f90 msgr2=0x7f419c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:07.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f419a59c640 1 --2- 172.21.15.139:0/14974459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f419c104f90 0x7f419c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:07.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f419a59c640 1 -- 172.21.15.139:0/14974459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f419c1078c0 msgr2=0x7f419c19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:07.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f419a59c640 1 --2- 172.21.15.139:0/14974459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f419c1078c0 0x7f419c19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:07.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f419a59c640 1 -- 172.21.15.139:0/14974459 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419c19faf0 con 0x7f419c102660 2024-09-16T01:37:07.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f419a59c640 1 --2- 172.21.15.139:0/14974459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c102660 0x7f419c197ec0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f418c00d950 tx=0x7f418c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:07.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.735+0000 7f418b7fe640 1 -- 172.21.15.139:0/14974459 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c017030 con 0x7f419c102660 2024-09-16T01:37:07.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.736+0000 7f41a08cd640 1 -- 172.21.15.139:0/14974459 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419c19fde0 con 0x7f419c102660 2024-09-16T01:37:07.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.736+0000 7f418b7fe640 1 -- 172.21.15.139:0/14974459 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f418c004020 con 0x7f419c102660 2024-09-16T01:37:07.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.736+0000 7f418b7fe640 1 -- 172.21.15.139:0/14974459 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c0047a0 con 0x7f419c102660 2024-09-16T01:37:07.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.736+0000 7f41a08cd640 1 -- 172.21.15.139:0/14974459 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f419c1a0230 con 0x7f419c102660 2024-09-16T01:37:07.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.737+0000 7f41a08cd640 1 -- 172.21.15.139:0/14974459 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f419c0fdd70 con 0x7f419c102660 2024-09-16T01:37:07.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.738+0000 7f418b7fe640 1 -- 172.21.15.139:0/14974459 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f418c0041d0 con 0x7f419c102660 2024-09-16T01:37:07.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.738+0000 7f418b7fe640 1 --2- 172.21.15.139:0/14974459 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41840a3270 0x7f41840a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:07.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.738+0000 7f4199d9b640 1 --2- 172.21.15.139:0/14974459 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41840a3270 0x7f41840a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:07.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.738+0000 7f418b7fe640 1 -- 172.21.15.139:0/14974459 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6847+0+0 (secure 0 0 0) 0x7f418c0ced90 con 0x7f419c102660 2024-09-16T01:37:07.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.739+0000 7f4199d9b640 1 --2- 172.21.15.139:0/14974459 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41840a3270 0x7f41840a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f419c1993e0 tx=0x7f4190005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:07.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.742+0000 7f418b7fe640 1 -- 172.21.15.139:0/14974459 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f418c09c070 con 0x7f419c102660 2024-09-16T01:37:07.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.909+0000 7f41a08cd640 1 -- 172.21.15.139:0/14974459 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f419c02da10 con 0x7f419c102660 2024-09-16T01:37:07.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.909+0000 7f418b7fe640 1 -- 172.21.15.139:0/14974459 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f418c09b970 con 0x7f419c102660 2024-09-16T01:37:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:07.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.911+0000 7f41a08cd640 1 -- 172.21.15.139:0/14974459 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41840a3270 msgr2=0x7f41840a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:07.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.911+0000 7f41a08cd640 1 --2- 172.21.15.139:0/14974459 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41840a3270 0x7f41840a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f419c1993e0 tx=0x7f4190005f90 comp rx=0 tx=0).stop 2024-09-16T01:37:07.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.911+0000 7f41a08cd640 1 -- 172.21.15.139:0/14974459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c102660 msgr2=0x7f419c197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:07.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.911+0000 7f41a08cd640 1 --2- 172.21.15.139:0/14974459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c102660 0x7f419c197ec0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f418c00d950 tx=0x7f418c00de10 comp rx=0 tx=0).stop 2024-09-16T01:37:07.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.911+0000 7f41a08cd640 1 -- 172.21.15.139:0/14974459 shutdown_connections 2024-09-16T01:37:07.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.911+0000 7f41a08cd640 1 --2- 172.21.15.139:0/14974459 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41840a3270 0x7f41840a5730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:07.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.911+0000 7f41a08cd640 1 --2- 172.21.15.139:0/14974459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f419c1078c0 0x7f419c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:07.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.911+0000 7f41a08cd640 1 --2- 172.21.15.139:0/14974459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f419c104f90 0x7f419c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:07.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.911+0000 7f41a08cd640 1 --2- 172.21.15.139:0/14974459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c102660 0x7f419c197ec0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:07.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.911+0000 7f41a08cd640 1 -- 172.21.15.139:0/14974459 >> 172.21.15.139:0/14974459 conn(0x7f419c0fc280 msgr2=0x7f419c102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:07.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.911+0000 7f41a08cd640 1 -- 172.21.15.139:0/14974459 shutdown_connections 2024-09-16T01:37:07.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:07.911+0000 7f41a08cd640 1 -- 172.21.15.139:0/14974459 wait complete. 2024-09-16T01:37:07.937 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T01:37:07.609+0000 7fefd3dd0640 -1 osd.11 0 waiting for initial osdmap 2024-09-16T01:37:07.937 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 01:37:07 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T01:37:07.621+0000 7fefcbbf0640 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T01:37:08.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:08 smithi164 ceph-mon[35121]: Detected new or changed devices on smithi176 2024-09-16T01:37:08.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:08 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi176 to 4272M 2024-09-16T01:37:08.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:08 smithi164 ceph-mon[35121]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-16T01:37:08.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:08 smithi164 ceph-mon[35121]: osdmap e70: 12 total, 11 up, 12 in 2024-09-16T01:37:08.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:08 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:08.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:08 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:08.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:08 smithi164 ceph-mon[35121]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:37:08.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:08 smithi164 ceph-mon[35121]: from='osd.11 ' entity='osd.11' 2024-09-16T01:37:08.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:08 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/14974459' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:37:08.604 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:08 smithi139 ceph-mon[31175]: Detected new or changed devices on smithi176 2024-09-16T01:37:08.604 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:08 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi176 to 4272M 2024-09-16T01:37:08.604 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:08 smithi139 ceph-mon[31175]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-16T01:37:08.604 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:08 smithi139 ceph-mon[31175]: osdmap e70: 12 total, 11 up, 12 in 2024-09-16T01:37:08.604 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:08 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:08.604 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:08 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:08.604 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:08 smithi139 ceph-mon[31175]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:37:08.605 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:08 smithi139 ceph-mon[31175]: from='osd.11 ' entity='osd.11' 2024-09-16T01:37:08.605 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:08 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/14974459' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:37:08.635 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1726450603,"num_in_osds":12,"osd_in_since":1726450605,"num_remapped_pgs":0} 2024-09-16T01:37:08.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:08 smithi176 ceph-mon[35324]: Detected new or changed devices on smithi176 2024-09-16T01:37:08.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:08 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi176 to 4272M 2024-09-16T01:37:08.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:08 smithi176 ceph-mon[35324]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-16T01:37:08.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:08 smithi176 ceph-mon[35324]: osdmap e70: 12 total, 11 up, 12 in 2024-09-16T01:37:08.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:08 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:08.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:08 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:08.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:08 smithi176 ceph-mon[35324]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-16T01:37:08.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:08 smithi176 ceph-mon[35324]: from='osd.11 ' entity='osd.11' 2024-09-16T01:37:08.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:08 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/14974459' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:37:09.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:09 smithi164 ceph-mon[35121]: purged_snaps scrub starts 2024-09-16T01:37:09.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:09 smithi164 ceph-mon[35121]: purged_snaps scrub ok 2024-09-16T01:37:09.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:09.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:09 smithi164 ceph-mon[35121]: osd.11 [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565] boot 2024-09-16T01:37:09.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:09 smithi164 ceph-mon[35121]: osdmap e71: 12 total, 12 up, 12 in 2024-09-16T01:37:09.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:09.636 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T01:37:09.664 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:09 smithi139 ceph-mon[31175]: purged_snaps scrub starts 2024-09-16T01:37:09.664 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:09 smithi139 ceph-mon[31175]: purged_snaps scrub ok 2024-09-16T01:37:09.664 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:09.664 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:09 smithi139 ceph-mon[31175]: osd.11 [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565] boot 2024-09-16T01:37:09.664 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:09 smithi139 ceph-mon[31175]: osdmap e71: 12 total, 12 up, 12 in 2024-09-16T01:37:09.664 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:09.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:09 smithi176 ceph-mon[35324]: purged_snaps scrub starts 2024-09-16T01:37:09.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:09 smithi176 ceph-mon[35324]: purged_snaps scrub ok 2024-09-16T01:37:09.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:09.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:09 smithi176 ceph-mon[35324]: osd.11 [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565] boot 2024-09-16T01:37:09.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:09 smithi176 ceph-mon[35324]: osdmap e71: 12 total, 12 up, 12 in 2024-09-16T01:37:09.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:37:09.894 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:10.299 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:10 smithi139 ceph-mon[31175]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:10.299 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:10 smithi139 ceph-mon[31175]: osdmap e72: 12 total, 12 up, 12 in 2024-09-16T01:37:10.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.375+0000 7effc5a4f640 1 -- 172.21.15.139:0/1020820978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc00ffed0 msgr2=0x7effc01002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:10.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.375+0000 7effc5a4f640 1 --2- 172.21.15.139:0/1020820978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc00ffed0 0x7effc01002d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7effb8009cd0 tx=0x7effb802a8e0 comp rx=0 tx=0).stop 2024-09-16T01:37:10.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.375+0000 7effc5a4f640 1 -- 172.21.15.139:0/1020820978 shutdown_connections 2024-09-16T01:37:10.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.375+0000 7effc5a4f640 1 --2- 172.21.15.139:0/1020820978 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7effc0103870 0x7effc010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:10.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.375+0000 7effc5a4f640 1 --2- 172.21.15.139:0/1020820978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc0102eb0 0x7effc0103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:10.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.375+0000 7effc5a4f640 1 --2- 172.21.15.139:0/1020820978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc00ffed0 0x7effc01002d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:10.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.375+0000 7effc5a4f640 1 -- 172.21.15.139:0/1020820978 >> 172.21.15.139:0/1020820978 conn(0x7effc00fba40 msgr2=0x7effc00fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:10.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.375+0000 7effc5a4f640 1 -- 172.21.15.139:0/1020820978 shutdown_connections 2024-09-16T01:37:10.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.375+0000 7effc5a4f640 1 -- 172.21.15.139:0/1020820978 wait complete. 2024-09-16T01:37:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effc5a4f640 1 Processor -- start 2024-09-16T01:37:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effc5a4f640 1 -- start start 2024-09-16T01:37:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effc5a4f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7effc00ffed0 0x7effc0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effc5a4f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc0102eb0 0x7effc0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effc5a4f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc0103870 0x7effc019f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effc5a4f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7effc010d890 con 0x7effc0103870 2024-09-16T01:37:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effc5a4f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7effc010d710 con 0x7effc0102eb0 2024-09-16T01:37:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effc5a4f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7effc010da10 con 0x7effc00ffed0 2024-09-16T01:37:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effbf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc0103870 0x7effc019f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effbf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc0103870 0x7effc019f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51580/0 (socket says 172.21.15.139:51580) 2024-09-16T01:37:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effbf7fe640 1 -- 172.21.15.139:0/444770322 learned_addr learned my addr 172.21.15.139:0/444770322 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effbeffd640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7effc00ffed0 0x7effc0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effbe7fc640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc0102eb0 0x7effc0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effbf7fe640 1 -- 172.21.15.139:0/444770322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7effc00ffed0 msgr2=0x7effc0197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effbf7fe640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7effc00ffed0 0x7effc0197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effbf7fe640 1 -- 172.21.15.139:0/444770322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc0102eb0 msgr2=0x7effc0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effbf7fe640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc0102eb0 0x7effc0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effbf7fe640 1 -- 172.21.15.139:0/444770322 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc019fb80 con 0x7effc0103870 2024-09-16T01:37:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effbe7fc640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc0102eb0 0x7effc0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:37:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.376+0000 7effbf7fe640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc0103870 0x7effc019f510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7effb000c970 tx=0x7effb000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.377+0000 7effc4a4d640 1 -- 172.21.15.139:0/444770322 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0007bb0 con 0x7effc0103870 2024-09-16T01:37:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.377+0000 7effc4a4d640 1 -- 172.21.15.139:0/444770322 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb0007d50 con 0x7effc0103870 2024-09-16T01:37:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.377+0000 7effc5a4f640 1 -- 172.21.15.139:0/444770322 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc019fe70 con 0x7effc0103870 2024-09-16T01:37:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.377+0000 7effc4a4d640 1 -- 172.21.15.139:0/444770322 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0018800 con 0x7effc0103870 2024-09-16T01:37:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.377+0000 7effc5a4f640 1 -- 172.21.15.139:0/444770322 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc01a02d0 con 0x7effc0103870 2024-09-16T01:37:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.378+0000 7effc5a4f640 1 -- 172.21.15.139:0/444770322 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc01091a0 con 0x7effc0103870 2024-09-16T01:37:10.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.379+0000 7effc4a4d640 1 -- 172.21.15.139:0/444770322 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7effb000fa50 con 0x7effc0103870 2024-09-16T01:37:10.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.380+0000 7effc4a4d640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7effac0a3270 0x7effac0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:10.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.380+0000 7effbeffd640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7effac0a3270 0x7effac0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:10.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.380+0000 7effc4a4d640 1 -- 172.21.15.139:0/444770322 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7effb00cf8a0 con 0x7effc0103870 2024-09-16T01:37:10.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.380+0000 7effbeffd640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7effac0a3270 0x7effac0a5730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7effb8006010 tx=0x7effb8033000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:10.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.383+0000 7effc4a4d640 1 -- 172.21.15.139:0/444770322 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7effb0014030 con 0x7effc0103870 2024-09-16T01:37:10.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:10 smithi164 ceph-mon[35121]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:10.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:10 smithi164 ceph-mon[35121]: osdmap e72: 12 total, 12 up, 12 in 2024-09-16T01:37:10.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.551+0000 7effc5a4f640 1 -- 172.21.15.139:0/444770322 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7effc0062df0 con 0x7effc0103870 2024-09-16T01:37:10.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.551+0000 7effc4a4d640 1 -- 172.21.15.139:0/444770322 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7effb009ca40 con 0x7effc0103870 2024-09-16T01:37:10.553 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:10.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.553+0000 7effc5a4f640 1 -- 172.21.15.139:0/444770322 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7effac0a3270 msgr2=0x7effac0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:10.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.553+0000 7effc5a4f640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7effac0a3270 0x7effac0a5730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7effb8006010 tx=0x7effb8033000 comp rx=0 tx=0).stop 2024-09-16T01:37:10.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.553+0000 7effc5a4f640 1 -- 172.21.15.139:0/444770322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc0103870 msgr2=0x7effc019f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:10.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.553+0000 7effc5a4f640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc0103870 0x7effc019f510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7effb000c970 tx=0x7effb000ce30 comp rx=0 tx=0).stop 2024-09-16T01:37:10.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.554+0000 7effc5a4f640 1 -- 172.21.15.139:0/444770322 shutdown_connections 2024-09-16T01:37:10.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.554+0000 7effc5a4f640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7effac0a3270 0x7effac0a5730 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:10.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.554+0000 7effc5a4f640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc0103870 0x7effc019f510 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:10.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.554+0000 7effc5a4f640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc0102eb0 0x7effc0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:10.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.554+0000 7effc5a4f640 1 --2- 172.21.15.139:0/444770322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7effc00ffed0 0x7effc0197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:10.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.554+0000 7effc5a4f640 1 -- 172.21.15.139:0/444770322 >> 172.21.15.139:0/444770322 conn(0x7effc00fba40 msgr2=0x7effc0106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:10.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.554+0000 7effc5a4f640 1 -- 172.21.15.139:0/444770322 shutdown_connections 2024-09-16T01:37:10.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:10.554+0000 7effc5a4f640 1 -- 172.21.15.139:0/444770322 wait complete. 2024-09-16T01:37:10.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:10 smithi176 ceph-mon[35324]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:10.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:10 smithi176 ceph-mon[35324]: osdmap e72: 12 total, 12 up, 12 in 2024-09-16T01:37:11.338 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:11 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/444770322' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:37:11.369 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1726450628,"num_in_osds":12,"osd_in_since":1726450605,"num_remapped_pgs":0} 2024-09-16T01:37:11.369 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T01:37:11.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:11 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/444770322' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:37:11.612 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:11.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:11 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/444770322' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T01:37:12.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.136+0000 7f63a3563640 1 -- 172.21.15.139:0/2533183885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c101df0 msgr2=0x7f639c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:12.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.136+0000 7f63a3563640 1 --2- 172.21.15.139:0/2533183885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c101df0 0x7f639c1021d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6398009900 tx=0x7f639802a410 comp rx=0 tx=0).stop 2024-09-16T01:37:12.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.136+0000 7f63a3563640 1 -- 172.21.15.139:0/2533183885 shutdown_connections 2024-09-16T01:37:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.136+0000 7f63a3563640 1 --2- 172.21.15.139:0/2533183885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f639c106f20 0x7f639c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.136+0000 7f63a3563640 1 --2- 172.21.15.139:0/2533183885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f639c1027a0 0x7f639c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.136+0000 7f63a3563640 1 --2- 172.21.15.139:0/2533183885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c101df0 0x7f639c1021d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.136+0000 7f63a3563640 1 -- 172.21.15.139:0/2533183885 >> 172.21.15.139:0/2533183885 conn(0x7f639c0fd8f0 msgr2=0x7f639c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.136+0000 7f63a3563640 1 -- 172.21.15.139:0/2533183885 shutdown_connections 2024-09-16T01:37:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.136+0000 7f63a3563640 1 -- 172.21.15.139:0/2533183885 wait complete. 2024-09-16T01:37:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.136+0000 7f63a3563640 1 Processor -- start 2024-09-16T01:37:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.136+0000 7f63a3563640 1 -- start start 2024-09-16T01:37:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a3563640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f639c101df0 0x7f639c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a3563640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f639c1027a0 0x7f639c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a3563640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c106f20 0x7f639c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a3563640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f639c111f80 con 0x7f639c106f20 2024-09-16T01:37:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a3563640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f639c111e00 con 0x7f639c101df0 2024-09-16T01:37:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a3563640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f639c112100 con 0x7f639c1027a0 2024-09-16T01:37:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a1b00640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c106f20 0x7f639c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a1b00640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c106f20 0x7f639c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51600/0 (socket says 172.21.15.139:51600) 2024-09-16T01:37:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a1b00640 1 -- 172.21.15.139:0/2020360787 learned_addr learned my addr 172.21.15.139:0/2020360787 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a12ff640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f639c101df0 0x7f639c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a0afe640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f639c1027a0 0x7f639c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a1b00640 1 -- 172.21.15.139:0/2020360787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f639c1027a0 msgr2=0x7f639c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a1b00640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f639c1027a0 0x7f639c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a1b00640 1 -- 172.21.15.139:0/2020360787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f639c101df0 msgr2=0x7f639c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a1b00640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f639c101df0 0x7f639c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a1b00640 1 -- 172.21.15.139:0/2020360787 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639c1a6980 con 0x7f639c106f20 2024-09-16T01:37:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a0afe640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f639c1027a0 0x7f639c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:37:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a12ff640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f639c101df0 0x7f639c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:37:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a1b00640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c106f20 0x7f639c074f70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f639400cad0 tx=0x7f639400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63927fc640 1 -- 172.21.15.139:0/2020360787 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639400ae20 con 0x7f639c106f20 2024-09-16T01:37:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a3563640 1 -- 172.21.15.139:0/2020360787 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639c1a6c70 con 0x7f639c106f20 2024-09-16T01:37:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63927fc640 1 -- 172.21.15.139:0/2020360787 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6394005020 con 0x7f639c106f20 2024-09-16T01:37:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63927fc640 1 -- 172.21.15.139:0/2020360787 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639400f6f0 con 0x7f639c106f20 2024-09-16T01:37:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.137+0000 7f63a3563640 1 -- 172.21.15.139:0/2020360787 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639c1a7160 con 0x7f639c106f20 2024-09-16T01:37:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.138+0000 7f63a3563640 1 -- 172.21.15.139:0/2020360787 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639c1021d0 con 0x7f639c106f20 2024-09-16T01:37:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.140+0000 7f63927fc640 1 -- 172.21.15.139:0/2020360787 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f63940051d0 con 0x7f639c106f20 2024-09-16T01:37:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.141+0000 7f63927fc640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f63880a3240 0x7f63880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.141+0000 7f63a12ff640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f63880a3240 0x7f63880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.141+0000 7f63927fc640 1 -- 172.21.15.139:0/2020360787 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f63940cf8d0 con 0x7f639c106f20 2024-09-16T01:37:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.141+0000 7f63a12ff640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f63880a3240 0x7f63880a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6398005ab0 tx=0x7f6398005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.144+0000 7f63927fc640 1 -- 172.21.15.139:0/2020360787 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6394014030 con 0x7f639c106f20 2024-09-16T01:37:12.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.313+0000 7f63a3563640 1 -- 172.21.15.139:0/2020360787 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f639c075a20 con 0x7f639c106f20 2024-09-16T01:37:12.316 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:12 smithi139 ceph-mon[31175]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:12.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.314+0000 7f63927fc640 1 -- 172.21.15.139:0/2020360787 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18527 (secure 0 0 0) 0x7f639c075a20 con 0x7f639c106f20 2024-09-16T01:37:12.317 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:12.317 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":72,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","created":"2024-09-16T01:30:17.770313+0000","modified":"2024-09-16T01:37:09.613417+0000","last_up_change":"2024-09-16T01:37:08.611901+0000","last_in_change":"2024-09-16T01:36:45.223926+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:33:19.639647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b66696da-ef66-4f25-9527-bbcc9ded2cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6803","nonce":3190810133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6805","nonce":3190810133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6809","nonce":3190810133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6807","nonce":3190810133}]},"public_addr":"172.21.15.139:6803/3190810133","cluster_addr":"172.21.15.139:6805/3190810133","heartbeat_back_addr":"172.21.15.139:6809/3190810133","heartbeat_front_addr":"172.21.15.139:6807/3190810133","state":["exists","up"]},{"osd":1,"uuid":"ccad17f2-0750-425e-9810-49a13a929ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6801","nonce":2699270330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6803","nonce":2699270330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6807","nonce":2699270330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6805","nonce":2699270330}]},"public_addr":"172.21.15.164:6801/2699270330","cluster_addr":"172.21.15.164:6803/2699270330","heartbeat_back_addr":"172.21.15.164:6807/2699270330","heartbeat_front_addr":"172.21.15.164:6805/2699270330","state":["exists","up"]},{"osd":2,"uuid":"baf0ba73-8faf-4e51-a171-c47898800638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6801","nonce":2034471646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6803","nonce":2034471646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6807","nonce":2034471646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6805","nonce":2034471646}]},"public_addr":"172.21.15.176:6801/2034471646","cluster_addr":"172.21.15.176:6803/2034471646","heartbeat_back_addr":"172.21.15.176:6807/2034471646","heartbeat_front_addr":"172.21.15.176:6805/2034471646","state":["exists","up"]},{"osd":3,"uuid":"98957bf1-67bb-4092-9ea4-b263a5706f8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6811","nonce":797147071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6813","nonce":797147071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6817","nonce":797147071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6815","nonce":797147071}]},"public_addr":"172.21.15.139:6811/797147071","cluster_addr":"172.21.15.139:6813/797147071","heartbeat_back_addr":"172.21.15.139:6817/797147071","heartbeat_front_addr":"172.21.15.139:6815/797147071","state":["exists","up"]},{"osd":4,"uuid":"9cd76696-8161-4e82-a571-29e98c11d5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6809","nonce":1421304792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6811","nonce":1421304792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6815","nonce":1421304792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6813","nonce":1421304792}]},"public_addr":"172.21.15.164:6809/1421304792","cluster_addr":"172.21.15.164:6811/1421304792","heartbeat_back_addr":"172.21.15.164:6815/1421304792","heartbeat_front_addr":"172.21.15.164:6813/1421304792","state":["exists","up"]},{"osd":5,"uuid":"ccc17a29-9a58-40ec-b453-5c38bee86f25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6809","nonce":10600602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6811","nonce":10600602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6815","nonce":10600602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6813","nonce":10600602}]},"public_addr":"172.21.15.176:6809/10600602","cluster_addr":"172.21.15.176:6811/10600602","heartbeat_back_addr":"172.21.15.176:6815/10600602","heartbeat_front_addr":"172.21.15.176:6813/10600602","state":["exists","up"]},{"osd":6,"uuid":"e5ee7212-10c6-4a05-992e-622a14a5a6b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6819","nonce":2601111164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6821","nonce":2601111164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6825","nonce":2601111164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6823","nonce":2601111164}]},"public_addr":"172.21.15.139:6819/2601111164","cluster_addr":"172.21.15.139:6821/2601111164","heartbeat_back_addr":"172.21.15.139:6825/2601111164","heartbeat_front_addr":"172.21.15.139:6823/2601111164","state":["exists","up"]},{"osd":7,"uuid":"23143c15-0839-4fd5-9d0a-19ae891000f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6817","nonce":2074080693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6819","nonce":2074080693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6823","nonce":2074080693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6821","nonce":2074080693}]},"public_addr":"172.21.15.164:6817/2074080693","cluster_addr":"172.21.15.164:6819/2074080693","heartbeat_back_addr":"172.21.15.164:6823/2074080693","heartbeat_front_addr":"172.21.15.164:6821/2074080693","state":["exists","up"]},{"osd":8,"uuid":"1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6817","nonce":593133361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6819","nonce":593133361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6823","nonce":593133361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6821","nonce":593133361}]},"public_addr":"172.21.15.176:6817/593133361","cluster_addr":"172.21.15.176:6819/593133361","heartbeat_back_addr":"172.21.15.176:6823/593133361","heartbeat_front_addr":"172.21.15.176:6821/593133361","state":["exists","up"]},{"osd":9,"uuid":"09d25a3b-1e66-480e-9433-ceef7f0b9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6827","nonce":3047942108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6829","nonce":3047942108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6833","nonce":3047942108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6831","nonce":3047942108}]},"public_addr":"172.21.15.139:6827/3047942108","cluster_addr":"172.21.15.139:6829/3047942108","heartbeat_back_addr":"172.21.15.139:6833/3047942108","heartbeat_front_addr":"172.21.15.139:6831/3047942108","state":["exists","up"]},{"osd":10,"uuid":"32b4d8ce-b644-42fa-80cc-a15a1f5f3a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6825","nonce":4172533828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6827","nonce":4172533828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6831","nonce":4172533828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6829","nonce":4172533828}]},"public_addr":"172.21.15.164:6825/4172533828","cluster_addr":"172.21.15.164:6827/4172533828","heartbeat_back_addr":"172.21.15.164:6831/4172533828","heartbeat_front_addr":"172.21.15.164:6829/4172533828","state":["exists","up"]},{"osd":11,"uuid":"49098bdc-58a9-4a74-8045-d408af7711ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6825","nonce":1854547565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6827","nonce":1854547565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6831","nonce":1854547565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6829","nonce":1854547565}]},"public_addr":"172.21.15.176:6825/1854547565","cluster_addr":"172.21.15.176:6827/1854547565","heartbeat_back_addr":"172.21.15.176:6831/1854547565","heartbeat_front_addr":"172.21.15.176:6829/1854547565","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:27.374845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:52.611853+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:17.229567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:43.330503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:07.988876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:33.787864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:58.864910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:23.815006+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:49.551954+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:15.496773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:41.647557+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:37:07.089355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3772232169":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/3997627612":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/1608930262":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6801/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/677216867":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6801/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/1830135963":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/2608207648":"2024-09-17T01:30:44.751168+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:37:12.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.317+0000 7f63a3563640 1 -- 172.21.15.139:0/2020360787 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f63880a3240 msgr2=0x7f63880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:12.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.317+0000 7f63a3563640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f63880a3240 0x7f63880a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6398005ab0 tx=0x7f6398005a20 comp rx=0 tx=0).stop 2024-09-16T01:37:12.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.317+0000 7f63a3563640 1 -- 172.21.15.139:0/2020360787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c106f20 msgr2=0x7f639c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:12.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.317+0000 7f63a3563640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c106f20 0x7f639c074f70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f639400cad0 tx=0x7f639400cf90 comp rx=0 tx=0).stop 2024-09-16T01:37:12.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.317+0000 7f63a3563640 1 -- 172.21.15.139:0/2020360787 shutdown_connections 2024-09-16T01:37:12.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.317+0000 7f63a3563640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f63880a3240 0x7f63880a5700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:12.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.317+0000 7f63a3563640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c106f20 0x7f639c074f70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:12.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.317+0000 7f63a3563640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f639c1027a0 0x7f639c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:12.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.317+0000 7f63a3563640 1 --2- 172.21.15.139:0/2020360787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f639c101df0 0x7f639c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:12.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.317+0000 7f63a3563640 1 -- 172.21.15.139:0/2020360787 >> 172.21.15.139:0/2020360787 conn(0x7f639c0fd8f0 msgr2=0x7f639c0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:12.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.317+0000 7f63a3563640 1 -- 172.21.15.139:0/2020360787 shutdown_connections 2024-09-16T01:37:12.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:12.317+0000 7f63a3563640 1 -- 172.21.15.139:0/2020360787 wait complete. 2024-09-16T01:37:12.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:12 smithi164 ceph-mon[35121]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:12.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:12 smithi176 ceph-mon[35324]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:13.173 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T01:33:19.639647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T01:37:13.173 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T01:37:13.422 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:13.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:13 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2020360787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:37:13.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:13 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2020360787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:37:13.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:13 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2020360787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:37:13.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.883+0000 7fa389cad640 1 -- 172.21.15.139:0/2589127308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa38410c170 msgr2=0x7fa38410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:13.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.883+0000 7fa389cad640 1 --2- 172.21.15.139:0/2589127308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa38410c170 0x7fa38410e610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009900 tx=0x7fa37c02a480 comp rx=0 tx=0).stop 2024-09-16T01:37:13.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.883+0000 7fa389cad640 1 -- 172.21.15.139:0/2589127308 shutdown_connections 2024-09-16T01:37:13.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.883+0000 7fa389cad640 1 --2- 172.21.15.139:0/2589127308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa38410c170 0x7fa38410e610 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:13.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.883+0000 7fa389cad640 1 --2- 172.21.15.139:0/2589127308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa384103660 0x7fa38410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:13.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.883+0000 7fa389cad640 1 --2- 172.21.15.139:0/2589127308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa384102d40 0x7fa384103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:13.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.883+0000 7fa389cad640 1 -- 172.21.15.139:0/2589127308 >> 172.21.15.139:0/2589127308 conn(0x7fa3840fc7a0 msgr2=0x7fa3840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:13.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.883+0000 7fa389cad640 1 -- 172.21.15.139:0/2589127308 shutdown_connections 2024-09-16T01:37:13.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.883+0000 7fa389cad640 1 -- 172.21.15.139:0/2589127308 wait complete. 2024-09-16T01:37:13.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.884+0000 7fa389cad640 1 Processor -- start 2024-09-16T01:37:13.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.884+0000 7fa389cad640 1 -- start start 2024-09-16T01:37:13.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.884+0000 7fa389cad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa384102d40 0x7fa384198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:13.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.884+0000 7fa389cad640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa384103660 0x7fa3841989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:13.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.884+0000 7fa389cad640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa38410c170 0x7fa38419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:13.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.884+0000 7fa389cad640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa384111fd0 con 0x7fa384102d40 2024-09-16T01:37:13.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.884+0000 7fa389cad640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa384111e50 con 0x7fa38410c170 2024-09-16T01:37:13.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.884+0000 7fa389cad640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa384112150 con 0x7fa384103660 2024-09-16T01:37:13.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa3837fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa384102d40 0x7fa384198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:13.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa3837fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa384102d40 0x7fa384198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36854/0 (socket says 172.21.15.139:36854) 2024-09-16T01:37:13.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa3837fe640 1 -- 172.21.15.139:0/4030141742 learned_addr learned my addr 172.21.15.139:0/4030141742 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:13.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa382ffd640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa384103660 0x7fa3841989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:13.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa3837fe640 1 -- 172.21.15.139:0/4030141742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa384103660 msgr2=0x7fa3841989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:13.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa383fff640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa38410c170 0x7fa38419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:13.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa3837fe640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa384103660 0x7fa3841989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:13.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa3837fe640 1 -- 172.21.15.139:0/4030141742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa38410c170 msgr2=0x7fa38419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:13.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa3837fe640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa38410c170 0x7fa38419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:13.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa3837fe640 1 -- 172.21.15.139:0/4030141742 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38419d440 con 0x7fa384102d40 2024-09-16T01:37:13.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa382ffd640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa384103660 0x7fa3841989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:37:13.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa3837fe640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa384102d40 0x7fa384198490 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa37400cce0 tx=0x7fa37400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:13.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa380ff9640 1 -- 172.21.15.139:0/4030141742 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374004b20 con 0x7fa384102d40 2024-09-16T01:37:13.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa380ff9640 1 -- 172.21.15.139:0/4030141742 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa374004cc0 con 0x7fa384102d40 2024-09-16T01:37:13.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa389cad640 1 -- 172.21.15.139:0/4030141742 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38419d730 con 0x7fa384102d40 2024-09-16T01:37:13.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa380ff9640 1 -- 172.21.15.139:0/4030141742 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37400f8c0 con 0x7fa384102d40 2024-09-16T01:37:13.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.885+0000 7fa389cad640 1 -- 172.21.15.139:0/4030141742 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3841a5010 con 0x7fa384102d40 2024-09-16T01:37:13.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.886+0000 7fa389cad640 1 -- 172.21.15.139:0/4030141742 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa384103120 con 0x7fa384102d40 2024-09-16T01:37:13.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.887+0000 7fa380ff9640 1 -- 172.21.15.139:0/4030141742 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa37400fa60 con 0x7fa384102d40 2024-09-16T01:37:13.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.888+0000 7fa380ff9640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa36c0a32c0 0x7fa36c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:13.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.888+0000 7fa380ff9640 1 -- 172.21.15.139:0/4030141742 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fa3740cf560 con 0x7fa384102d40 2024-09-16T01:37:13.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.888+0000 7fa382ffd640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa36c0a32c0 0x7fa36c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:13.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.888+0000 7fa382ffd640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa36c0a32c0 0x7fa36c0a5780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa378006fd0 tx=0x7fa378008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:13.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:13.892+0000 7fa380ff9640 1 -- 172.21.15.139:0/4030141742 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa374014030 con 0x7fa384102d40 2024-09-16T01:37:14.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:14.056+0000 7fa389cad640 1 -- 172.21.15.139:0/4030141742 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa38402d7c0 con 0x7fa384102d40 2024-09-16T01:37:14.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:14.056+0000 7fa380ff9640 1 -- 172.21.15.139:0/4030141742 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7fa37409c720 con 0x7fa384102d40 2024-09-16T01:37:14.058 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-09-16T01:37:14.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:14.058+0000 7fa389cad640 1 -- 172.21.15.139:0/4030141742 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa36c0a32c0 msgr2=0x7fa36c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:14.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:14.058+0000 7fa389cad640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa36c0a32c0 0x7fa36c0a5780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa378006fd0 tx=0x7fa378008040 comp rx=0 tx=0).stop 2024-09-16T01:37:14.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:14.058+0000 7fa389cad640 1 -- 172.21.15.139:0/4030141742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa384102d40 msgr2=0x7fa384198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:14.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:14.058+0000 7fa389cad640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa384102d40 0x7fa384198490 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa37400cce0 tx=0x7fa37400a620 comp rx=0 tx=0).stop 2024-09-16T01:37:14.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:14.058+0000 7fa389cad640 1 -- 172.21.15.139:0/4030141742 shutdown_connections 2024-09-16T01:37:14.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:14.058+0000 7fa389cad640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa36c0a32c0 0x7fa36c0a5780 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:14.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:14.058+0000 7fa389cad640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa38410c170 0x7fa38419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:14.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:14.058+0000 7fa389cad640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa384103660 0x7fa3841989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:14.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:14.058+0000 7fa389cad640 1 --2- 172.21.15.139:0/4030141742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa384102d40 0x7fa384198490 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:14.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:14.058+0000 7fa389cad640 1 -- 172.21.15.139:0/4030141742 >> 172.21.15.139:0/4030141742 conn(0x7fa3840fc7a0 msgr2=0x7fa3840fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:14.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:14.059+0000 7fa389cad640 1 -- 172.21.15.139:0/4030141742 shutdown_connections 2024-09-16T01:37:14.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:14.059+0000 7fa389cad640 1 -- 172.21.15.139:0/4030141742 wait complete. 2024-09-16T01:37:14.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:14 smithi139 ceph-mon[31175]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:14.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:14 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/4030141742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T01:37:14.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:14 smithi164 ceph-mon[35121]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:14.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:14 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/4030141742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T01:37:14.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:14 smithi176 ceph-mon[35324]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:14.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:14 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/4030141742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T01:37:14.896 INFO:tasks.cephadm:Adding mds.a on smithi139 2024-09-16T01:37:14.897 INFO:tasks.cephadm:Adding mds.d on smithi139 2024-09-16T01:37:14.897 INFO:tasks.cephadm:Adding mds.g on smithi139 2024-09-16T01:37:14.897 INFO:tasks.cephadm:Adding mds.j on smithi139 2024-09-16T01:37:14.897 INFO:tasks.cephadm:Adding mds.b on smithi164 2024-09-16T01:37:14.897 INFO:tasks.cephadm:Adding mds.e on smithi164 2024-09-16T01:37:14.897 INFO:tasks.cephadm:Adding mds.h on smithi164 2024-09-16T01:37:14.897 INFO:tasks.cephadm:Adding mds.k on smithi164 2024-09-16T01:37:14.897 INFO:tasks.cephadm:Adding mds.c on smithi176 2024-09-16T01:37:14.897 INFO:tasks.cephadm:Adding mds.f on smithi176 2024-09-16T01:37:14.897 INFO:tasks.cephadm:Adding mds.i on smithi176 2024-09-16T01:37:14.897 INFO:tasks.cephadm:Adding mds.l on smithi176 2024-09-16T01:37:14.897 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi139=a;smithi139=d;smithi139=g;smithi139=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l' 2024-09-16T01:37:15.157 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:37:15.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.627+0000 7f3d76f83640 1 -- 172.21.15.176:0/1204575041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d70102660 msgr2=0x7f3d70104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:15.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.627+0000 7f3d76f83640 1 --2- 172.21.15.176:0/1204575041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d70102660 0x7f3d70104a50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009900 tx=0x7f3d6c02a410 comp rx=0 tx=0).stop 2024-09-16T01:37:15.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 -- 172.21.15.176:0/1204575041 shutdown_connections 2024-09-16T01:37:15.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 --2- 172.21.15.176:0/1204575041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d701078c0 0x7f3d70109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:15.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 --2- 172.21.15.176:0/1204575041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d70104f90 0x7f3d70107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:15.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 --2- 172.21.15.176:0/1204575041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d70102660 0x7f3d70104a50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:15.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 -- 172.21.15.176:0/1204575041 >> 172.21.15.176:0/1204575041 conn(0x7f3d700fc280 msgr2=0x7f3d700fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:15.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 -- 172.21.15.176:0/1204575041 shutdown_connections 2024-09-16T01:37:15.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 -- 172.21.15.176:0/1204575041 wait complete. 2024-09-16T01:37:15.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 Processor -- start 2024-09-16T01:37:15.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 -- start start 2024-09-16T01:37:15.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d70102660 0x7f3d70197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:15.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d70104f90 0x7f3d70198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:15.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d701078c0 0x7f3d7019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:15.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7010d8e0 con 0x7f3d701078c0 2024-09-16T01:37:15.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7010d760 con 0x7f3d70102660 2024-09-16T01:37:15.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.628+0000 7f3d76f83640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7010da60 con 0x7f3d70104f90 2024-09-16T01:37:15.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d67fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d70104f90 0x7f3d70198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:15.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d67fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d70104f90 0x7f3d70198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54838/0 (socket says 172.21.15.176:54838) 2024-09-16T01:37:15.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d67fff640 1 -- 172.21.15.176:0/2545593376 learned_addr learned my addr 172.21.15.176:0/2545593376 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-16T01:37:15.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d75520640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d701078c0 0x7f3d7019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:15.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d74d1f640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d70102660 0x7f3d70197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:15.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d67fff640 1 -- 172.21.15.176:0/2545593376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d70102660 msgr2=0x7f3d70197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:15.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d67fff640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d70102660 0x7f3d70197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:15.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d67fff640 1 -- 172.21.15.176:0/2545593376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d701078c0 msgr2=0x7f3d7019f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:15.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d67fff640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d701078c0 0x7f3d7019f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:15.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d67fff640 1 -- 172.21.15.176:0/2545593376 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7019fb10 con 0x7f3d70104f90 2024-09-16T01:37:15.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d75520640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d701078c0 0x7f3d7019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:37:15.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d67fff640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d70104f90 0x7f3d70198420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000c970 tx=0x7f3d6000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:15.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d65ffb640 1 -- 172.21.15.176:0/2545593376 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60014070 con 0x7f3d70104f90 2024-09-16T01:37:15.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d76f83640 1 -- 172.21.15.176:0/2545593376 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7019fe00 con 0x7f3d70104f90 2024-09-16T01:37:15.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d65ffb640 1 -- 172.21.15.176:0/2545593376 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d60004440 con 0x7f3d70104f90 2024-09-16T01:37:15.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d65ffb640 1 -- 172.21.15.176:0/2545593376 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000f450 con 0x7f3d70104f90 2024-09-16T01:37:15.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.629+0000 7f3d76f83640 1 -- 172.21.15.176:0/2545593376 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d701a0330 con 0x7f3d70104f90 2024-09-16T01:37:15.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.630+0000 7f3d76f83640 1 -- 172.21.15.176:0/2545593376 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d700fdd70 con 0x7f3d70104f90 2024-09-16T01:37:15.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.631+0000 7f3d65ffb640 1 -- 172.21.15.176:0/2545593376 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3d60017070 con 0x7f3d70104f90 2024-09-16T01:37:15.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.632+0000 7f3d65ffb640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3d580a3270 0x7f3d580a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:15.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.632+0000 7f3d65ffb640 1 -- 172.21.15.176:0/2545593376 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f3d600cf2f0 con 0x7f3d70104f90 2024-09-16T01:37:15.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.632+0000 7f3d74d1f640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3d580a3270 0x7f3d580a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:15.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.633+0000 7f3d74d1f640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3d580a3270 0x7f3d580a5730 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c031610 tx=0x7f3d6c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:15.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.636+0000 7f3d65ffb640 1 -- 172.21.15.176:0/2545593376 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d6001e020 con 0x7f3d70104f90 2024-09-16T01:37:15.812 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.811+0000 7f3d76f83640 1 -- 172.21.15.176:0/2545593376 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi139=a;smithi139=d;smithi139=g;smithi139=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l", "target": ["mon-mgr", ""]}) -- 0x7f3d70062730 con 0x7f3d580a3270 2024-09-16T01:37:15.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.821+0000 7f3d65ffb640 1 -- 172.21.15.176:0/2545593376 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f3d70062730 con 0x7f3d580a3270 2024-09-16T01:37:15.822 INFO:teuthology.orchestra.run.smithi176.stdout:Scheduled mds.all update... 2024-09-16T01:37:15.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.823+0000 7f3d76f83640 1 -- 172.21.15.176:0/2545593376 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3d580a3270 msgr2=0x7f3d580a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:15.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.823+0000 7f3d76f83640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3d580a3270 0x7f3d580a5730 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c031610 tx=0x7f3d6c000bc0 comp rx=0 tx=0).stop 2024-09-16T01:37:15.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.823+0000 7f3d76f83640 1 -- 172.21.15.176:0/2545593376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d70104f90 msgr2=0x7f3d70198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:15.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.823+0000 7f3d76f83640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d70104f90 0x7f3d70198420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000c970 tx=0x7f3d6000ce30 comp rx=0 tx=0).stop 2024-09-16T01:37:15.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.823+0000 7f3d76f83640 1 -- 172.21.15.176:0/2545593376 shutdown_connections 2024-09-16T01:37:15.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.823+0000 7f3d76f83640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3d580a3270 0x7f3d580a5730 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:15.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.823+0000 7f3d76f83640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d701078c0 0x7f3d7019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:15.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.823+0000 7f3d76f83640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d70104f90 0x7f3d70198420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:15.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.823+0000 7f3d76f83640 1 --2- 172.21.15.176:0/2545593376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d70102660 0x7f3d70197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:15.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.823+0000 7f3d76f83640 1 -- 172.21.15.176:0/2545593376 >> 172.21.15.176:0/2545593376 conn(0x7f3d700fc280 msgr2=0x7f3d70102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:15.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.823+0000 7f3d76f83640 1 -- 172.21.15.176:0/2545593376 shutdown_connections 2024-09-16T01:37:15.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-16T01:37:15.823+0000 7f3d76f83640 1 -- 172.21.15.176:0/2545593376 wait complete. 2024-09-16T01:37:16.522 DEBUG:teuthology.orchestra.run.smithi139:mds.a> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.a.service 2024-09-16T01:37:16.524 DEBUG:teuthology.orchestra.run.smithi139:mds.d> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.d.service 2024-09-16T01:37:16.567 DEBUG:teuthology.orchestra.run.smithi139:mds.g> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.g.service 2024-09-16T01:37:16.610 DEBUG:teuthology.orchestra.run.smithi139:mds.j> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.j.service 2024-09-16T01:37:16.653 DEBUG:teuthology.orchestra.run.smithi164:mds.b> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.b.service 2024-09-16T01:37:16.656 DEBUG:teuthology.orchestra.run.smithi164:mds.e> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.e.service 2024-09-16T01:37:16.701 DEBUG:teuthology.orchestra.run.smithi164:mds.h> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.h.service 2024-09-16T01:37:16.744 DEBUG:teuthology.orchestra.run.smithi164:mds.k> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.k.service 2024-09-16T01:37:16.763 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:16 smithi164 ceph-mon[35121]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:16.763 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:16.763 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:16.763 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:16.763 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:16.763 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:16.763 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:16.763 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:16.764 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:16 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:16.764 DEBUG:teuthology.orchestra.run.smithi176:mds.c> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.c.service 2024-09-16T01:37:16.767 DEBUG:teuthology.orchestra.run.smithi176:mds.f> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.f.service 2024-09-16T01:37:16.787 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:16 smithi176 ceph-mon[35324]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:16.787 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:16.787 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:16.787 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:16.787 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:16.787 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:16.787 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:16.788 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:16.788 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:16 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:16.788 DEBUG:teuthology.orchestra.run.smithi176:mds.i> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.i.service 2024-09-16T01:37:16.833 DEBUG:teuthology.orchestra.run.smithi176:mds.l> sudo journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.l.service 2024-09-16T01:37:16.877 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-16T01:37:16.877 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:37:16.898 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:16 smithi139 ceph-mon[31175]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:16.899 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:16.899 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:16.899 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:16.899 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:16.899 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:16.899 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:16.899 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:16.899 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:16 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:17.116 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:17.515 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:17 smithi139 ceph-mon[31175]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi139=a;smithi139=d;smithi139=g;smithi139=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:37:17.515 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:17 smithi139 ceph-mon[31175]: Saving service mds.all spec with placement smithi139=a;smithi139=d;smithi139=g;smithi139=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l;count:12 2024-09-16T01:37:17.515 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:17 smithi139 ceph-mon[31175]: Deploying daemon mds.l on smithi176 2024-09-16T01:37:17.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:17 smithi176 ceph-mon[35324]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi139=a;smithi139=d;smithi139=g;smithi139=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:37:17.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:17 smithi176 ceph-mon[35324]: Saving service mds.all spec with placement smithi139=a;smithi139=d;smithi139=g;smithi139=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l;count:12 2024-09-16T01:37:17.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:17 smithi176 ceph-mon[35324]: Deploying daemon mds.l on smithi176 2024-09-16T01:37:17.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.723+0000 7f41627fd640 1 -- 172.21.15.139:0/2681413917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f415c102ef0 msgr2=0x7f415c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:17.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.723+0000 7f41627fd640 1 --2- 172.21.15.139:0/2681413917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f415c102ef0 0x7f415c10da40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4154009ca0 tx=0x7f415402a950 comp rx=0 tx=0).stop 2024-09-16T01:37:17.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.723+0000 7f41627fd640 1 -- 172.21.15.139:0/2681413917 shutdown_connections 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.723+0000 7f41627fd640 1 --2- 172.21.15.139:0/2681413917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f415c102ef0 0x7f415c10da40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.723+0000 7f41627fd640 1 --2- 172.21.15.139:0/2681413917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f415c102570 0x7f415c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.723+0000 7f41627fd640 1 --2- 172.21.15.139:0/2681413917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f415c100e20 0x7f415c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.723+0000 7f41627fd640 1 -- 172.21.15.139:0/2681413917 >> 172.21.15.139:0/2681413917 conn(0x7f415c0fc7d0 msgr2=0x7f415c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.723+0000 7f41627fd640 1 -- 172.21.15.139:0/2681413917 shutdown_connections 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.723+0000 7f41627fd640 1 -- 172.21.15.139:0/2681413917 wait complete. 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.723+0000 7f41627fd640 1 Processor -- start 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f41627fd640 1 -- start start 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f41627fd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f415c100e20 0x7f415c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f41627fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f415c102570 0x7f415c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f41627fd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f415c102ef0 0x7f415c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f41627fd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f415c110fa0 con 0x7f415c100e20 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f41627fd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f415c110e20 con 0x7f415c102570 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f41627fd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f415c111120 con 0x7f415c102ef0 2024-09-16T01:37:17.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f415bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f415c100e20 0x7f415c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:17.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f415bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f415c100e20 0x7f415c197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36878/0 (socket says 172.21.15.139:36878) 2024-09-16T01:37:17.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f415bfff640 1 -- 172.21.15.139:0/390456422 learned_addr learned my addr 172.21.15.139:0/390456422 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:17.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f415b7fe640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f415c102570 0x7f415c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:17.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f4160d9a640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f415c102ef0 0x7f415c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:17.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f415bfff640 1 -- 172.21.15.139:0/390456422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f415c102ef0 msgr2=0x7f415c19bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:17.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f415bfff640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f415c102ef0 0x7f415c19bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:17.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f415bfff640 1 -- 172.21.15.139:0/390456422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f415c102570 msgr2=0x7f415c197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:17.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f415bfff640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f415c102570 0x7f415c197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:17.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f415bfff640 1 -- 172.21.15.139:0/390456422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415c19c2e0 con 0x7f415c100e20 2024-09-16T01:37:17.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f4160d9a640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f415c102ef0 0x7f415c19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:37:17.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f415b7fe640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f415c102570 0x7f415c197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:37:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.724+0000 7f415bfff640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f415c100e20 0x7f415c197330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f414c00c920 tx=0x7f414c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.725+0000 7f41597fa640 1 -- 172.21.15.139:0/390456422 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c01c070 con 0x7f415c100e20 2024-09-16T01:37:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.725+0000 7f41597fa640 1 -- 172.21.15.139:0/390456422 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f414c007c70 con 0x7f415c100e20 2024-09-16T01:37:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.725+0000 7f41597fa640 1 -- 172.21.15.139:0/390456422 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c017840 con 0x7f415c100e20 2024-09-16T01:37:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.725+0000 7f41627fd640 1 -- 172.21.15.139:0/390456422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415c19c5d0 con 0x7f415c100e20 2024-09-16T01:37:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.725+0000 7f41627fd640 1 -- 172.21.15.139:0/390456422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415c069f70 con 0x7f415c100e20 2024-09-16T01:37:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.726+0000 7f41627fd640 1 -- 172.21.15.139:0/390456422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f415c101200 con 0x7f415c100e20 2024-09-16T01:37:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.727+0000 7f41597fa640 1 -- 172.21.15.139:0/390456422 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f414c017a00 con 0x7f415c100e20 2024-09-16T01:37:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.727+0000 7f41597fa640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41480a3240 0x7f41480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.727+0000 7f415b7fe640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41480a3240 0x7f41480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.727+0000 7f41597fa640 1 -- 172.21.15.139:0/390456422 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f414c0d1260 con 0x7f415c100e20 2024-09-16T01:37:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.728+0000 7f415b7fe640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41480a3240 0x7f41480a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f415c0feca0 tx=0x7f4144008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.731+0000 7f41597fa640 1 -- 172.21.15.139:0/390456422 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f414c025080 con 0x7f415c100e20 2024-09-16T01:37:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:17 smithi164 ceph-mon[35121]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi139=a;smithi139=d;smithi139=g;smithi139=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:37:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:17 smithi164 ceph-mon[35121]: Saving service mds.all spec with placement smithi139=a;smithi139=d;smithi139=g;smithi139=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l;count:12 2024-09-16T01:37:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:17 smithi164 ceph-mon[35121]: Deploying daemon mds.l on smithi176 2024-09-16T01:37:17.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.930+0000 7f41627fd640 1 -- 172.21.15.139:0/390456422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f415c06a230 con 0x7f415c100e20 2024-09-16T01:37:17.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.930+0000 7f41597fa640 1 -- 172.21.15.139:0/390456422 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f414c09e380 con 0x7f415c100e20 2024-09-16T01:37:17.932 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:17.932 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":1,"btime":"2024-09-16T01:30:17:770097+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-16T01:37:17.932 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 1 2024-09-16T01:37:17.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.932+0000 7f41627fd640 1 -- 172.21.15.139:0/390456422 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41480a3240 msgr2=0x7f41480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:17.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.932+0000 7f41627fd640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41480a3240 0x7f41480a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f415c0feca0 tx=0x7f4144008040 comp rx=0 tx=0).stop 2024-09-16T01:37:17.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.932+0000 7f41627fd640 1 -- 172.21.15.139:0/390456422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f415c100e20 msgr2=0x7f415c197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:17.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.932+0000 7f41627fd640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f415c100e20 0x7f415c197330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f414c00c920 tx=0x7f414c00cde0 comp rx=0 tx=0).stop 2024-09-16T01:37:17.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.932+0000 7f41627fd640 1 -- 172.21.15.139:0/390456422 shutdown_connections 2024-09-16T01:37:17.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.932+0000 7f41627fd640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41480a3240 0x7f41480a5700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:17.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.932+0000 7f41627fd640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f415c102ef0 0x7f415c19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:17.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.932+0000 7f41627fd640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f415c102570 0x7f415c197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:17.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.932+0000 7f41627fd640 1 --2- 172.21.15.139:0/390456422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f415c100e20 0x7f415c197330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:17.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.932+0000 7f41627fd640 1 -- 172.21.15.139:0/390456422 >> 172.21.15.139:0/390456422 conn(0x7f415c0fc7d0 msgr2=0x7f415c106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:17.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.932+0000 7f41627fd640 1 -- 172.21.15.139:0/390456422 shutdown_connections 2024-09-16T01:37:17.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:17.932+0000 7f41627fd640 1 -- 172.21.15.139:0/390456422 wait complete. 2024-09-16T01:37:18.115 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 01:37:18 smithi176 systemd[1]: Starting Ceph mds.l for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:37:18.436 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 01:37:18 smithi176 podman[64636]: 2024-09-16 01:37:18.229470401 +0000 UTC m=+0.015999433 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:37:18.437 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 01:37:18 smithi176 podman[64636]: 2024-09-16 01:37:18.359876495 +0000 UTC m=+0.146405507 container create 7225b47e96d71bae99fe5193ab0e18048824905586422417b152993c247836b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-l, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD) 2024-09-16T01:37:18.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:18 smithi139 ceph-mon[31175]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:18.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:18 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/390456422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:18.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:18 smithi164 ceph-mon[35121]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:18.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:18 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/390456422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:18.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:18 smithi176 ceph-mon[35324]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:18.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:18 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/390456422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:18.937 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 01:37:18 smithi176 podman[64636]: 2024-09-16 01:37:18.595928997 +0000 UTC m=+0.382458019 container init 7225b47e96d71bae99fe5193ab0e18048824905586422417b152993c247836b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T01:37:18.937 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 01:37:18 smithi176 podman[64636]: 2024-09-16 01:37:18.598872941 +0000 UTC m=+0.385401963 container start 7225b47e96d71bae99fe5193ab0e18048824905586422417b152993c247836b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T01:37:18.937 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 01:37:18 smithi176 ceph-mds[64656]: -- 172.21.15.176:0/140175360 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564fe7303860 con 0x564fe72e1000 2024-09-16T01:37:18.937 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 01:37:18 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-l[64652]: starting mds.l at 2024-09-16T01:37:18.937 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 01:37:18 smithi176 bash[64636]: 7225b47e96d71bae99fe5193ab0e18048824905586422417b152993c247836b6 2024-09-16T01:37:18.937 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 01:37:18 smithi176 systemd[1]: Started Ceph mds.l for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:37:19.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:19 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.176:6832/1026776933,v1:172.21.15.176:6833/1026776933] up:boot 2024-09-16T01:37:19.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:19 smithi164 ceph-mon[35121]: fsmap 1 up:standby 2024-09-16T01:37:19.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:19 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T01:37:19.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:19 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:19.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:19 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:19.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:19 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:19.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:19 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:19.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:19 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:19.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:19 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:19.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:19 smithi164 ceph-mon[35121]: Deploying daemon mds.k on smithi164 2024-09-16T01:37:19.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:19 smithi164 ceph-mon[35121]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:19 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.176:6832/1026776933,v1:172.21.15.176:6833/1026776933] up:boot 2024-09-16T01:37:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:19 smithi139 ceph-mon[31175]: fsmap 1 up:standby 2024-09-16T01:37:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T01:37:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:19 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:19 smithi139 ceph-mon[31175]: Deploying daemon mds.k on smithi164 2024-09-16T01:37:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:19 smithi139 ceph-mon[31175]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:19 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.176:6832/1026776933,v1:172.21.15.176:6833/1026776933] up:boot 2024-09-16T01:37:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:19 smithi176 ceph-mon[35324]: fsmap 1 up:standby 2024-09-16T01:37:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:19 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T01:37:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:19 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:19 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:19 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:19 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:19 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:19 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:19 smithi176 ceph-mon[35324]: Deploying daemon mds.k on smithi164 2024-09-16T01:37:19.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:19 smithi176 ceph-mon[35324]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:20.775 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:37:20.795 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 01:37:20 smithi164 systemd[1]: Starting Ceph mds.k for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:37:21.032 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:21.211 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 01:37:20 smithi164 podman[63705]: 2024-09-16 01:37:20.823200819 +0000 UTC m=+0.015705206 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:37:21.211 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 01:37:20 smithi164 podman[63705]: 2024-09-16 01:37:20.949582306 +0000 UTC m=+0.142086678 container create e666239c623e88cb12b436f83b2732eb19512aac7ae5a29032849026c5d25c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-k, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T01:37:21.211 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 01:37:21 smithi164 podman[63705]: 2024-09-16 01:37:21.210567994 +0000 UTC m=+0.403072369 container init e666239c623e88cb12b436f83b2732eb19512aac7ae5a29032849026c5d25c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-k, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-16T01:37:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.502+0000 7f407f664640 1 -- 172.21.15.139:0/3962611006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078102570 msgr2=0x7f40781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.502+0000 7f407f664640 1 --2- 172.21.15.139:0/3962611006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078102570 0x7f40781029b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4074009900 tx=0x7f407402a480 comp rx=0 tx=0).stop 2024-09-16T01:37:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.502+0000 7f407f664640 1 -- 172.21.15.139:0/3962611006 shutdown_connections 2024-09-16T01:37:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.502+0000 7f407f664640 1 --2- 172.21.15.139:0/3962611006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4078102ef0 0x7f407810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.502+0000 7f407f664640 1 --2- 172.21.15.139:0/3962611006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078102570 0x7f40781029b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.502+0000 7f407f664640 1 --2- 172.21.15.139:0/3962611006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4078100e20 0x7f4078101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.502+0000 7f407f664640 1 -- 172.21.15.139:0/3962611006 >> 172.21.15.139:0/3962611006 conn(0x7f40780fc7d0 msgr2=0x7f40780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.502+0000 7f407f664640 1 -- 172.21.15.139:0/3962611006 shutdown_connections 2024-09-16T01:37:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.502+0000 7f407f664640 1 -- 172.21.15.139:0/3962611006 wait complete. 2024-09-16T01:37:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407f664640 1 Processor -- start 2024-09-16T01:37:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407f664640 1 -- start start 2024-09-16T01:37:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407f664640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4078100e20 0x7f40781984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407f664640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078102570 0x7f40781989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407f664640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4078102ef0 0x7f407819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407f664640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4078110f80 con 0x7f4078102570 2024-09-16T01:37:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407f664640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4078110e00 con 0x7f4078102ef0 2024-09-16T01:37:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407f664640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4078111100 con 0x7f4078100e20 2024-09-16T01:37:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407cbff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078102570 0x7f40781989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407cbff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078102570 0x7f40781989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36896/0 (socket says 172.21.15.139:36896) 2024-09-16T01:37:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407cbff640 1 -- 172.21.15.139:0/1263010114 learned_addr learned my addr 172.21.15.139:0/1263010114 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:21.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407d400640 1 --2- 172.21.15.139:0/1263010114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4078100e20 0x7f40781984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:21.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407dc01640 1 --2- 172.21.15.139:0/1263010114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4078102ef0 0x7f407819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:21.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407cbff640 1 -- 172.21.15.139:0/1263010114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4078100e20 msgr2=0x7f40781984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:21.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407cbff640 1 --2- 172.21.15.139:0/1263010114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4078100e20 0x7f40781984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:21.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407cbff640 1 -- 172.21.15.139:0/1263010114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4078102ef0 msgr2=0x7f407819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:21.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407cbff640 1 --2- 172.21.15.139:0/1263010114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4078102ef0 0x7f407819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:21.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.503+0000 7f407cbff640 1 -- 172.21.15.139:0/1263010114 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407819d460 con 0x7f4078102570 2024-09-16T01:37:21.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.504+0000 7f407cbff640 1 --2- 172.21.15.139:0/1263010114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078102570 0x7f40781989f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4074006010 tx=0x7f4074003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:21.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.504+0000 7f406e7fc640 1 -- 172.21.15.139:0/1263010114 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407403f070 con 0x7f4078102570 2024-09-16T01:37:21.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.504+0000 7f406e7fc640 1 -- 172.21.15.139:0/1263010114 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4074031c30 con 0x7f4078102570 2024-09-16T01:37:21.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.504+0000 7f407f664640 1 -- 172.21.15.139:0/1263010114 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407819d6f0 con 0x7f4078102570 2024-09-16T01:37:21.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.504+0000 7f406e7fc640 1 -- 172.21.15.139:0/1263010114 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4074035030 con 0x7f4078102570 2024-09-16T01:37:21.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.504+0000 7f407f664640 1 -- 172.21.15.139:0/1263010114 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40781a5030 con 0x7f4078102570 2024-09-16T01:37:21.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.505+0000 7f407f664640 1 -- 172.21.15.139:0/1263010114 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4078101200 con 0x7f4078102570 2024-09-16T01:37:21.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.506+0000 7f406e7fc640 1 -- 172.21.15.139:0/1263010114 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4074042070 con 0x7f4078102570 2024-09-16T01:37:21.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.507+0000 7f406e7fc640 1 --2- 172.21.15.139:0/1263010114 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f40640a3240 0x7f40640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:21.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.507+0000 7f407d400640 1 --2- 172.21.15.139:0/1263010114 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f40640a3240 0x7f40640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:21.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.507+0000 7f406e7fc640 1 -- 172.21.15.139:0/1263010114 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f40740efad0 con 0x7f4078102570 2024-09-16T01:37:21.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.508+0000 7f407d400640 1 --2- 172.21.15.139:0/1263010114 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f40640a3240 0x7f40640a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4068009bd0 tx=0x7f4068005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:21.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.512+0000 7f406e7fc640 1 -- 172.21.15.139:0/1263010114 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40740bc640 con 0x7f4078102570 2024-09-16T01:37:21.545 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 01:37:21 smithi164 podman[63705]: 2024-09-16 01:37:21.213691303 +0000 UTC m=+0.406195686 container start e666239c623e88cb12b436f83b2732eb19512aac7ae5a29032849026c5d25c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e) 2024-09-16T01:37:21.545 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 01:37:21 smithi164 ceph-mds[63724]: -- 172.21.15.164:0/2669087611 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4ce229860 con 0x55f4ce207000 2024-09-16T01:37:21.545 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 01:37:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-k[63720]: starting mds.k at 2024-09-16T01:37:21.545 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 01:37:21 smithi164 bash[63705]: e666239c623e88cb12b436f83b2732eb19512aac7ae5a29032849026c5d25c95 2024-09-16T01:37:21.545 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 01:37:21 smithi164 systemd[1]: Started Ceph mds.k for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:37:21.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.733+0000 7f407f664640 1 -- 172.21.15.139:0/1263010114 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f407810a0a0 con 0x7f4078102570 2024-09-16T01:37:21.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.733+0000 7f406e7fc640 1 -- 172.21.15.139:0/1263010114 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7f40740b8070 con 0x7f4078102570 2024-09-16T01:37:21.736 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:21.736 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":3,"btime":"2024-09-16T01:37:21:292577+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-16T01:37:21.736 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 3 2024-09-16T01:37:21.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.736+0000 7f407f664640 1 -- 172.21.15.139:0/1263010114 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f40640a3240 msgr2=0x7f40640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:21.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.736+0000 7f407f664640 1 --2- 172.21.15.139:0/1263010114 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f40640a3240 0x7f40640a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4068009bd0 tx=0x7f4068005f90 comp rx=0 tx=0).stop 2024-09-16T01:37:21.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.736+0000 7f407f664640 1 -- 172.21.15.139:0/1263010114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078102570 msgr2=0x7f40781989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:21.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.736+0000 7f407f664640 1 --2- 172.21.15.139:0/1263010114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078102570 0x7f40781989f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4074006010 tx=0x7f4074003c10 comp rx=0 tx=0).stop 2024-09-16T01:37:21.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.736+0000 7f407f664640 1 -- 172.21.15.139:0/1263010114 shutdown_connections 2024-09-16T01:37:21.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.736+0000 7f407f664640 1 --2- 172.21.15.139:0/1263010114 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f40640a3240 0x7f40640a5700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:21.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.736+0000 7f407f664640 1 --2- 172.21.15.139:0/1263010114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4078102ef0 0x7f407819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:21.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.736+0000 7f407f664640 1 --2- 172.21.15.139:0/1263010114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078102570 0x7f40781989f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:21.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.736+0000 7f407f664640 1 --2- 172.21.15.139:0/1263010114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4078100e20 0x7f40781984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:21.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.736+0000 7f407f664640 1 -- 172.21.15.139:0/1263010114 >> 172.21.15.139:0/1263010114 conn(0x7f40780fc7d0 msgr2=0x7f407806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:21.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.736+0000 7f407f664640 1 -- 172.21.15.139:0/1263010114 shutdown_connections 2024-09-16T01:37:21.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:21.736+0000 7f407f664640 1 -- 172.21.15.139:0/1263010114 wait complete. 2024-09-16T01:37:22.625 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:22 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.164:6832/416925170,v1:172.21.15.164:6833/416925170] up:boot 2024-09-16T01:37:22.625 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:22 smithi164 ceph-mon[35121]: fsmap 2 up:standby 2024-09-16T01:37:22.625 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:22 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T01:37:22.625 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:22 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:22.625 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:22 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:22.625 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:22 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:22.626 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:22 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:22.626 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:22 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:22.626 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:22 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:22.626 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:22 smithi164 ceph-mon[35121]: Deploying daemon mds.j on smithi139 2024-09-16T01:37:22.626 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:22 smithi164 ceph-mon[35121]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:22.626 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:22 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1263010114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:22.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:22 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.164:6832/416925170,v1:172.21.15.164:6833/416925170] up:boot 2024-09-16T01:37:22.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:22 smithi139 ceph-mon[31175]: fsmap 2 up:standby 2024-09-16T01:37:22.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:22 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T01:37:22.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:22 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:22.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:22 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:22.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:22 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:22.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:22 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:22.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:22 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:22.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:22 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:22.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:22 smithi139 ceph-mon[31175]: Deploying daemon mds.j on smithi139 2024-09-16T01:37:22.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:22 smithi139 ceph-mon[31175]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:22.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:22 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1263010114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:22.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:22 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.164:6832/416925170,v1:172.21.15.164:6833/416925170] up:boot 2024-09-16T01:37:22.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:22 smithi176 ceph-mon[35324]: fsmap 2 up:standby 2024-09-16T01:37:22.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:22 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T01:37:22.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:22 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:22.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:22 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:22.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:22 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:22.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:22 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:22.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:22 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:22.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:22 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:22.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:22 smithi176 ceph-mon[35324]: Deploying daemon mds.j on smithi139 2024-09-16T01:37:22.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:22 smithi176 ceph-mon[35324]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:22.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:22 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1263010114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:23.862 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 01:37:23 smithi139 systemd[1]: Starting Ceph mds.j for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:37:24.113 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 01:37:24 smithi139 podman[67226]: 2024-09-16 01:37:23.972878799 +0000 UTC m=+0.015776092 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:37:24.114 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 01:37:24 smithi139 podman[67226]: 2024-09-16 01:37:24.113142367 +0000 UTC m=+0.156039653 container create 2f7c0438797e951ada286950164ef9252360d602ae1342bf6566eebcb0406d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-j, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True) 2024-09-16T01:37:24.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:24 smithi139 ceph-mon[31175]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:24.673 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 01:37:24 smithi139 podman[67226]: 2024-09-16 01:37:24.38304377 +0000 UTC m=+0.425941062 container init 2f7c0438797e951ada286950164ef9252360d602ae1342bf6566eebcb0406d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T01:37:24.673 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 01:37:24 smithi139 podman[67226]: 2024-09-16 01:37:24.385868323 +0000 UTC m=+0.428765610 container start 2f7c0438797e951ada286950164ef9252360d602ae1342bf6566eebcb0406d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-j, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T01:37:24.673 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 01:37:24 smithi139 ceph-mds[67245]: -- 172.21.15.139:0/135172986 <== mon.0 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa76893860 con 0x55fa76871000 2024-09-16T01:37:24.673 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 01:37:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-j[67241]: starting mds.j at 2024-09-16T01:37:24.673 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 01:37:24 smithi139 bash[67226]: 2f7c0438797e951ada286950164ef9252360d602ae1342bf6566eebcb0406d76 2024-09-16T01:37:24.673 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 01:37:24 smithi139 systemd[1]: Started Ceph mds.j for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:37:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:24 smithi164 ceph-mon[35121]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:24.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:24 smithi176 ceph-mon[35324]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:25 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.139:6834/129107170,v1:172.21.15.139:6835/129107170] up:boot 2024-09-16T01:37:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:25 smithi164 ceph-mon[35121]: fsmap 3 up:standby 2024-09-16T01:37:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T01:37:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:25.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:25 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:25.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:25 smithi164 ceph-mon[35121]: Deploying daemon mds.d on smithi139 2024-09-16T01:37:25.891 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:37:25.917 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:25 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.139:6834/129107170,v1:172.21.15.139:6835/129107170] up:boot 2024-09-16T01:37:25.917 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:25 smithi139 ceph-mon[31175]: fsmap 3 up:standby 2024-09-16T01:37:25.917 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T01:37:25.917 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:25.917 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:25.917 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:25.917 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:25.917 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:25.917 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:25 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:25.917 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:25 smithi139 ceph-mon[31175]: Deploying daemon mds.d on smithi139 2024-09-16T01:37:25.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:25 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.139:6834/129107170,v1:172.21.15.139:6835/129107170] up:boot 2024-09-16T01:37:25.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:25 smithi176 ceph-mon[35324]: fsmap 3 up:standby 2024-09-16T01:37:25.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T01:37:25.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:25.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:25.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:25.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:25.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:25.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:25 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:25.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:25 smithi176 ceph-mon[35324]: Deploying daemon mds.d on smithi139 2024-09-16T01:37:26.175 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:26 smithi164 ceph-mon[35121]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:26.804 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:26 smithi139 ceph-mon[31175]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:26.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.802+0000 7f79a3872640 1 -- 172.21.15.139:0/2978231475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f799c101280 msgr2=0x7f799c10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:26.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.802+0000 7f79a3872640 1 --2- 172.21.15.139:0/2978231475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f799c101280 0x7f799c10df40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7990009900 tx=0x7f799002a410 comp rx=0 tx=0).stop 2024-09-16T01:37:26.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.802+0000 7f79a3872640 1 -- 172.21.15.139:0/2978231475 shutdown_connections 2024-09-16T01:37:26.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.802+0000 7f79a3872640 1 --2- 172.21.15.139:0/2978231475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c10e480 0x7f799c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:26.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.802+0000 7f79a3872640 1 --2- 172.21.15.139:0/2978231475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f799c101280 0x7f799c10df40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:26.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.802+0000 7f79a3872640 1 --2- 172.21.15.139:0/2978231475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c1008d0 0x7f799c100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:26.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.802+0000 7f79a3872640 1 -- 172.21.15.139:0/2978231475 >> 172.21.15.139:0/2978231475 conn(0x7f799c0fc7a0 msgr2=0x7f799c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:26.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.802+0000 7f79a3872640 1 -- 172.21.15.139:0/2978231475 shutdown_connections 2024-09-16T01:37:26.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.802+0000 7f79a3872640 1 -- 172.21.15.139:0/2978231475 wait complete. 2024-09-16T01:37:26.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.803+0000 7f79a3872640 1 Processor -- start 2024-09-16T01:37:26.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.803+0000 7f79a3872640 1 -- start start 2024-09-16T01:37:26.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.803+0000 7f79a3872640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c1008d0 0x7f799c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:26.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.803+0000 7f79a3872640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c101280 0x7f799c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:26.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.803+0000 7f79a3872640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f799c10e480 0x7f799c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:26.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.803+0000 7f79a3872640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f799c1141e0 con 0x7f799c10e480 2024-09-16T01:37:26.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.803+0000 7f79a3872640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f799c114060 con 0x7f799c101280 2024-09-16T01:37:26.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.803+0000 7f79a3872640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f799c114360 con 0x7f799c1008d0 2024-09-16T01:37:26.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f79a1e0f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f799c10e480 0x7f799c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:26.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f79a1e0f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f799c10e480 0x7f799c19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57060/0 (socket says 172.21.15.139:57060) 2024-09-16T01:37:26.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f79a1e0f640 1 -- 172.21.15.139:0/2039093257 learned_addr learned my addr 172.21.15.139:0/2039093257 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:26.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f79a0e0d640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c101280 0x7f799c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:26.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f79a160e640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c1008d0 0x7f799c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:26.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f79a1e0f640 1 -- 172.21.15.139:0/2039093257 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c1008d0 msgr2=0x7f799c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:26.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f79a1e0f640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c1008d0 0x7f799c19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:26.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f79a1e0f640 1 -- 172.21.15.139:0/2039093257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c101280 msgr2=0x7f799c19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:26.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f79a1e0f640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c101280 0x7f799c19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:26.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f79a1e0f640 1 -- 172.21.15.139:0/2039093257 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c19f6b0 con 0x7f799c10e480 2024-09-16T01:37:26.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f79a0e0d640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c101280 0x7f799c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:37:26.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f79a1e0f640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f799c10e480 0x7f799c19efd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f798c00b9e0 tx=0x7f798c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:26.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f799a7fc640 1 -- 172.21.15.139:0/2039093257 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c019070 con 0x7f799c10e480 2024-09-16T01:37:26.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f79a3872640 1 -- 172.21.15.139:0/2039093257 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c19f9a0 con 0x7f799c10e480 2024-09-16T01:37:26.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f799a7fc640 1 -- 172.21.15.139:0/2039093257 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f798c014440 con 0x7f799c10e480 2024-09-16T01:37:26.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f799a7fc640 1 -- 172.21.15.139:0/2039093257 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c01d5b0 con 0x7f799c10e480 2024-09-16T01:37:26.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.804+0000 7f79a3872640 1 -- 172.21.15.139:0/2039093257 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c1a7280 con 0x7f799c10e480 2024-09-16T01:37:26.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.805+0000 7f79a3872640 1 -- 172.21.15.139:0/2039093257 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799c100cb0 con 0x7f799c10e480 2024-09-16T01:37:26.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.806+0000 7f799a7fc640 1 -- 172.21.15.139:0/2039093257 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f798c01c070 con 0x7f799c10e480 2024-09-16T01:37:26.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.807+0000 7f799a7fc640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f79840a3240 0x7f79840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:26.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.807+0000 7f799a7fc640 1 -- 172.21.15.139:0/2039093257 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f798c0ceee0 con 0x7f799c10e480 2024-09-16T01:37:26.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.807+0000 7f79a160e640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f79840a3240 0x7f79840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:26.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.808+0000 7f79a160e640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f79840a3240 0x7f79840a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7994005fd0 tx=0x7f7994009470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:26.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:26.810+0000 7f799a7fc640 1 -- 172.21.15.139:0/2039093257 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f798c09c0a0 con 0x7f799c10e480 2024-09-16T01:37:26.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:26 smithi176 ceph-mon[35324]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:27.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:27.035+0000 7f79a3872640 1 -- 172.21.15.139:0/2039093257 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f799c02d7c0 con 0x7f799c10e480 2024-09-16T01:37:27.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:27.036+0000 7f799a7fc640 1 -- 172.21.15.139:0/2039093257 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7f798c09b9a0 con 0x7f799c10e480 2024-09-16T01:37:27.038 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:27.038 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":4,"btime":"2024-09-16T01:37:24:468746+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-16T01:37:27.038 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 4 2024-09-16T01:37:27.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:27.038+0000 7f79a3872640 1 -- 172.21.15.139:0/2039093257 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f79840a3240 msgr2=0x7f79840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:27.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:27.038+0000 7f79a3872640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f79840a3240 0x7f79840a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7994005fd0 tx=0x7f7994009470 comp rx=0 tx=0).stop 2024-09-16T01:37:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:27.038+0000 7f79a3872640 1 -- 172.21.15.139:0/2039093257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f799c10e480 msgr2=0x7f799c19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:27.038+0000 7f79a3872640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f799c10e480 0x7f799c19efd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f798c00b9e0 tx=0x7f798c00bea0 comp rx=0 tx=0).stop 2024-09-16T01:37:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:27.038+0000 7f79a3872640 1 -- 172.21.15.139:0/2039093257 shutdown_connections 2024-09-16T01:37:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:27.038+0000 7f79a3872640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f79840a3240 0x7f79840a5700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:27.038+0000 7f79a3872640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f799c10e480 0x7f799c19efd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:27.038+0000 7f79a3872640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c101280 0x7f799c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:27.038+0000 7f79a3872640 1 --2- 172.21.15.139:0/2039093257 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c1008d0 0x7f799c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:27.038+0000 7f79a3872640 1 -- 172.21.15.139:0/2039093257 >> 172.21.15.139:0/2039093257 conn(0x7f799c0fc7a0 msgr2=0x7f799c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:27.039+0000 7f79a3872640 1 -- 172.21.15.139:0/2039093257 shutdown_connections 2024-09-16T01:37:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:27.039+0000 7f79a3872640 1 -- 172.21.15.139:0/2039093257 wait complete. 2024-09-16T01:37:27.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:27 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2039093257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:27.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:27 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2039093257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:27.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:27 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2039093257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:28.272 INFO:journalctl@ceph.mds.d.smithi139.stdout:Sep 16 01:37:28 smithi139 podman[67626]: 2024-09-16 01:37:28.244846035 +0000 UTC m=+1.234153651 container start ef25e7c09c08911f31dff7f4aac7b28a0c4c52625768b74212ac38b14af188cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-16T01:37:28.539 INFO:journalctl@ceph.mds.d.smithi139.stdout:Sep 16 01:37:28 smithi139 ceph-mds[67663]: -- 172.21.15.139:0/862170124 <== mon.0 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dddf5f9860 con 0x55dddf5d7000 2024-09-16T01:37:28.540 INFO:journalctl@ceph.mds.d.smithi139.stdout:Sep 16 01:37:28 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-d[67655]: starting mds.d at 2024-09-16T01:37:28.540 INFO:journalctl@ceph.mds.d.smithi139.stdout:Sep 16 01:37:28 smithi139 bash[67626]: ef25e7c09c08911f31dff7f4aac7b28a0c4c52625768b74212ac38b14af188cf 2024-09-16T01:37:28.540 INFO:journalctl@ceph.mds.d.smithi139.stdout:Sep 16 01:37:28 smithi139 systemd[1]: Started Ceph mds.d for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:37:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:28 smithi164 ceph-mon[35121]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:28 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:28 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:28 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:28 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:28 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:28 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:28 smithi139 ceph-mon[31175]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:28.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:28 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:28.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:28 smithi176 ceph-mon[35324]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:28.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:28 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:28.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:28 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:28.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:28 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:28.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:28 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:28.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:28 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:28.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:28 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:29.865 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:29 smithi164 ceph-mon[35121]: Deploying daemon mds.e on smithi164 2024-09-16T01:37:29.865 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:29 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.139:6836/465602995,v1:172.21.15.139:6837/465602995] up:boot 2024-09-16T01:37:29.865 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:29 smithi164 ceph-mon[35121]: fsmap 4 up:standby 2024-09-16T01:37:29.865 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:29 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T01:37:29.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:29 smithi139 ceph-mon[31175]: Deploying daemon mds.e on smithi164 2024-09-16T01:37:29.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:29 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.139:6836/465602995,v1:172.21.15.139:6837/465602995] up:boot 2024-09-16T01:37:29.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:29 smithi139 ceph-mon[31175]: fsmap 4 up:standby 2024-09-16T01:37:29.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:29 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T01:37:29.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:29 smithi176 ceph-mon[35324]: Deploying daemon mds.e on smithi164 2024-09-16T01:37:29.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:29 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.139:6836/465602995,v1:172.21.15.139:6837/465602995] up:boot 2024-09-16T01:37:29.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:29 smithi176 ceph-mon[35324]: fsmap 4 up:standby 2024-09-16T01:37:29.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:29 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T01:37:30.048 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:37:30.308 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:30.545 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:37:30 smithi164 systemd[1]: Starting Ceph mds.e for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:37:30.770 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:30 smithi139 ceph-mon[31175]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:30.816 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:30 smithi164 ceph-mon[35121]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:30.816 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:37:30 smithi164 podman[64019]: 2024-09-16 01:37:30.448564676 +0000 UTC m=+0.016050743 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:37:30.816 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:37:30 smithi164 podman[64019]: 2024-09-16 01:37:30.570655833 +0000 UTC m=+0.138141900 container create d9cc4c9246da86a26f00a2258ce9ee4a44803380e559083b7c8ca84cf643c881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-16T01:37:30.816 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:37:30 smithi164 podman[64019]: 2024-09-16 01:37:30.815418416 +0000 UTC m=+0.382904492 container init d9cc4c9246da86a26f00a2258ce9ee4a44803380e559083b7c8ca84cf643c881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T01:37:30.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.849+0000 7f968d099640 1 -- 172.21.15.139:0/909837760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9688103700 msgr2=0x7f968810bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:30.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.849+0000 7f968d099640 1 --2- 172.21.15.139:0/909837760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9688103700 0x7f968810bbf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9670009900 tx=0x7f967002a480 comp rx=0 tx=0).stop 2024-09-16T01:37:30.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.849+0000 7f968d099640 1 -- 172.21.15.139:0/909837760 shutdown_connections 2024-09-16T01:37:30.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.849+0000 7f968d099640 1 --2- 172.21.15.139:0/909837760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f968810c180 0x7f968810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:30.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.849+0000 7f968d099640 1 --2- 172.21.15.139:0/909837760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9688103700 0x7f968810bbf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:30.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.849+0000 7f968d099640 1 --2- 172.21.15.139:0/909837760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9688102de0 0x7f96881031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:30.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.849+0000 7f968d099640 1 -- 172.21.15.139:0/909837760 >> 172.21.15.139:0/909837760 conn(0x7f96880fc7a0 msgr2=0x7f96880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:30.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.849+0000 7f968d099640 1 -- 172.21.15.139:0/909837760 shutdown_connections 2024-09-16T01:37:30.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.849+0000 7f968d099640 1 -- 172.21.15.139:0/909837760 wait complete. 2024-09-16T01:37:30.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968d099640 1 Processor -- start 2024-09-16T01:37:30.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968d099640 1 -- start start 2024-09-16T01:37:30.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968d099640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9688102de0 0x7f9688100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:30.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968d099640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9688103700 0x7f9688100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:30.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968d099640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f968810c180 0x7f96881014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:30.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968d099640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9688111f70 con 0x7f9688103700 2024-09-16T01:37:30.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968d099640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9688111df0 con 0x7f9688102de0 2024-09-16T01:37:30.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968d099640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f96881120f0 con 0x7f968810c180 2024-09-16T01:37:30.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9688103700 0x7f9688100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:30.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f9686d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9688102de0 0x7f9688100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:30.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9688103700 0x7f9688100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57090/0 (socket says 172.21.15.139:57090) 2024-09-16T01:37:30.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968659c640 1 -- 172.21.15.139:0/502412602 learned_addr learned my addr 172.21.15.139:0/502412602 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:30.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968759e640 1 --2- 172.21.15.139:0/502412602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f968810c180 0x7f96881014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:30.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968659c640 1 -- 172.21.15.139:0/502412602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f968810c180 msgr2=0x7f96881014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:30.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968659c640 1 --2- 172.21.15.139:0/502412602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f968810c180 0x7f96881014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:30.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968659c640 1 -- 172.21.15.139:0/502412602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9688102de0 msgr2=0x7f9688100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:30.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.850+0000 7f968659c640 1 --2- 172.21.15.139:0/502412602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9688102de0 0x7f9688100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:30.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.851+0000 7f968659c640 1 -- 172.21.15.139:0/502412602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96881a6900 con 0x7f9688103700 2024-09-16T01:37:30.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.851+0000 7f9686d9d640 1 --2- 172.21.15.139:0/502412602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9688102de0 0x7f9688100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:37:30.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.851+0000 7f968659c640 1 --2- 172.21.15.139:0/502412602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9688103700 0x7f9688100f80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9670006010 tx=0x7f96700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:30.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.851+0000 7f967ffff640 1 -- 172.21.15.139:0/502412602 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967003f030 con 0x7f9688103700 2024-09-16T01:37:30.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.851+0000 7f968d099640 1 -- 172.21.15.139:0/502412602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96881a6b90 con 0x7f9688103700 2024-09-16T01:37:30.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.851+0000 7f967ffff640 1 -- 172.21.15.139:0/502412602 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9670003ce0 con 0x7f9688103700 2024-09-16T01:37:30.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.851+0000 7f967ffff640 1 -- 172.21.15.139:0/502412602 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670035070 con 0x7f9688103700 2024-09-16T01:37:30.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.851+0000 7f968d099640 1 -- 172.21.15.139:0/502412602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96881a7070 con 0x7f9688103700 2024-09-16T01:37:30.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.852+0000 7f968d099640 1 -- 172.21.15.139:0/502412602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96881031c0 con 0x7f9688103700 2024-09-16T01:37:30.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.853+0000 7f967ffff640 1 -- 172.21.15.139:0/502412602 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9670042030 con 0x7f9688103700 2024-09-16T01:37:30.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.854+0000 7f967ffff640 1 --2- 172.21.15.139:0/502412602 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f96740a3240 0x7f96740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:30.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.854+0000 7f9686d9d640 1 --2- 172.21.15.139:0/502412602 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f96740a3240 0x7f96740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:30.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.854+0000 7f967ffff640 1 -- 172.21.15.139:0/502412602 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f96700efa50 con 0x7f9688103700 2024-09-16T01:37:30.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.854+0000 7f9686d9d640 1 --2- 172.21.15.139:0/502412602 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f96740a3240 0x7f96740a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9678005fd0 tx=0x7f967800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:30.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:30.858+0000 7f967ffff640 1 -- 172.21.15.139:0/502412602 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96700bccd0 con 0x7f9688103700 2024-09-16T01:37:30.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:30 smithi176 ceph-mon[35324]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:31.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:31.077+0000 7f968d099640 1 -- 172.21.15.139:0/502412602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f968802d7c0 con 0x7f9688103700 2024-09-16T01:37:31.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:31.077+0000 7f967ffff640 1 -- 172.21.15.139:0/502412602 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7f96700bc5d0 con 0x7f9688103700 2024-09-16T01:37:31.078 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:31.078 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":5,"btime":"2024-09-16T01:37:28:537976+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-16T01:37:31.078 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 5 2024-09-16T01:37:31.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:31.079+0000 7f968d099640 1 -- 172.21.15.139:0/502412602 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f96740a3240 msgr2=0x7f96740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:31.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:31.079+0000 7f968d099640 1 --2- 172.21.15.139:0/502412602 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f96740a3240 0x7f96740a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9678005fd0 tx=0x7f967800b390 comp rx=0 tx=0).stop 2024-09-16T01:37:31.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:31.079+0000 7f968d099640 1 -- 172.21.15.139:0/502412602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9688103700 msgr2=0x7f9688100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:31.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:31.079+0000 7f968d099640 1 --2- 172.21.15.139:0/502412602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9688103700 0x7f9688100f80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9670006010 tx=0x7f96700042f0 comp rx=0 tx=0).stop 2024-09-16T01:37:31.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:31.079+0000 7f968d099640 1 -- 172.21.15.139:0/502412602 shutdown_connections 2024-09-16T01:37:31.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:31.079+0000 7f968d099640 1 --2- 172.21.15.139:0/502412602 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f96740a3240 0x7f96740a5700 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:31.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:31.079+0000 7f968d099640 1 --2- 172.21.15.139:0/502412602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f968810c180 0x7f96881014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:31.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:31.079+0000 7f968d099640 1 --2- 172.21.15.139:0/502412602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9688103700 0x7f9688100f80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:31.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:31.079+0000 7f968d099640 1 --2- 172.21.15.139:0/502412602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9688102de0 0x7f9688100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:31.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:31.080+0000 7f968d099640 1 -- 172.21.15.139:0/502412602 >> 172.21.15.139:0/502412602 conn(0x7f96880fc7a0 msgr2=0x7f96880fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:31.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:31.080+0000 7f968d099640 1 -- 172.21.15.139:0/502412602 shutdown_connections 2024-09-16T01:37:31.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:31.080+0000 7f968d099640 1 -- 172.21.15.139:0/502412602 wait complete. 2024-09-16T01:37:31.295 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:37:30 smithi164 podman[64019]: 2024-09-16 01:37:30.818396749 +0000 UTC m=+0.385882826 container start d9cc4c9246da86a26f00a2258ce9ee4a44803380e559083b7c8ca84cf643c881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-16T01:37:31.295 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:37:30 smithi164 ceph-mds[64038]: -- 172.21.15.164:0/1781615903 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565261bcb860 con 0x565261ba8800 2024-09-16T01:37:31.295 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:37:30 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e[64034]: starting mds.e at 2024-09-16T01:37:31.295 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:37:30 smithi164 bash[64019]: d9cc4c9246da86a26f00a2258ce9ee4a44803380e559083b7c8ca84cf643c881 2024-09-16T01:37:31.295 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:37:30 smithi164 systemd[1]: Started Ceph mds.e for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:37:32.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:31 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:32.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:31 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:32.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:31 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:32.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:31 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:32.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:31 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:32.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:31 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:32.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:31 smithi139 ceph-mon[31175]: Deploying daemon mds.f on smithi176 2024-09-16T01:37:32.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:31 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/502412602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:32.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:31 smithi139 ceph-mon[31175]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:32.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:31 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.164:6834/3662238682,v1:172.21.15.164:6835/3662238682] up:boot 2024-09-16T01:37:32.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:31 smithi139 ceph-mon[31175]: fsmap 5 up:standby 2024-09-16T01:37:32.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:31 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T01:37:32.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:31 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:32.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:31 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:32.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:31 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:32.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:31 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:32.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:31 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:32.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:31 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:32.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:31 smithi164 ceph-mon[35121]: Deploying daemon mds.f on smithi176 2024-09-16T01:37:32.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:31 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/502412602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:32.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:31 smithi164 ceph-mon[35121]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:32.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:31 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.164:6834/3662238682,v1:172.21.15.164:6835/3662238682] up:boot 2024-09-16T01:37:32.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:31 smithi164 ceph-mon[35121]: fsmap 5 up:standby 2024-09-16T01:37:32.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:31 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T01:37:32.395 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:31 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:32.395 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:31 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:32.395 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:31 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:32.395 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:31 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:32.395 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:31 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:32.395 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:31 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:32.395 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:31 smithi176 ceph-mon[35324]: Deploying daemon mds.f on smithi176 2024-09-16T01:37:32.395 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:31 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/502412602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:32.396 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:31 smithi176 ceph-mon[35324]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:32.396 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:31 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.164:6834/3662238682,v1:172.21.15.164:6835/3662238682] up:boot 2024-09-16T01:37:32.396 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:31 smithi176 ceph-mon[35324]: fsmap 5 up:standby 2024-09-16T01:37:32.396 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:31 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T01:37:32.937 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 01:37:32 smithi176 systemd[1]: Starting Ceph mds.f for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:37:33.332 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 01:37:33 smithi176 podman[64940]: 2024-09-16 01:37:33.077659956 +0000 UTC m=+0.109431192 container create 5b773f3b098191d535648a85cec01a270bef4dd38595ce8bf884223090a3be09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-16T01:37:33.332 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 01:37:33 smithi176 podman[64940]: 2024-09-16 01:37:32.984175682 +0000 UTC m=+0.015946928 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:37:33.332 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 01:37:33 smithi176 podman[64940]: 2024-09-16 01:37:33.305216049 +0000 UTC m=+0.336987295 container init 5b773f3b098191d535648a85cec01a270bef4dd38595ce8bf884223090a3be09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-f, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD) 2024-09-16T01:37:33.332 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 01:37:33 smithi176 podman[64940]: 2024-09-16 01:37:33.308021511 +0000 UTC m=+0.339792754 container start 5b773f3b098191d535648a85cec01a270bef4dd38595ce8bf884223090a3be09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T01:37:33.686 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 01:37:33 smithi176 ceph-mds[64959]: -- 172.21.15.176:0/3301598662 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559321d15860 con 0x559321cf3400 2024-09-16T01:37:33.686 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 01:37:33 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-f[64955]: starting mds.f at 2024-09-16T01:37:33.686 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 01:37:33 smithi176 bash[64940]: 5b773f3b098191d535648a85cec01a270bef4dd38595ce8bf884223090a3be09 2024-09-16T01:37:33.687 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 01:37:33 smithi176 systemd[1]: Started Ceph mds.f for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:37:33.798 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:37:34.053 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:34.471 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:34 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] up:boot 2024-09-16T01:37:34.471 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:34 smithi139 ceph-mon[31175]: fsmap 6 up:standby 2024-09-16T01:37:34.471 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T01:37:34.471 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:34 smithi139 ceph-mon[31175]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:34.471 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:34.471 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:34.471 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:34.471 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:34.471 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:34.471 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:34 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:34.471 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:34 smithi139 ceph-mon[31175]: Deploying daemon mds.c on smithi176 2024-09-16T01:37:34.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.552+0000 7f914547c640 1 -- 172.21.15.139:0/3967704962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91401027a0 msgr2=0x7f91401067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:34.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.552+0000 7f914547c640 1 --2- 172.21.15.139:0/3967704962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91401027a0 0x7f91401067f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9128009900 tx=0x7f912802a410 comp rx=0 tx=0).stop 2024-09-16T01:37:34.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.552+0000 7f914547c640 1 -- 172.21.15.139:0/3967704962 shutdown_connections 2024-09-16T01:37:34.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.552+0000 7f914547c640 1 --2- 172.21.15.139:0/3967704962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9140106f20 0x7f914010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:34.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.552+0000 7f914547c640 1 --2- 172.21.15.139:0/3967704962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91401027a0 0x7f91401067f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:34.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.552+0000 7f914547c640 1 --2- 172.21.15.139:0/3967704962 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9140101df0 0x7f91401021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:34.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.552+0000 7f914547c640 1 -- 172.21.15.139:0/3967704962 >> 172.21.15.139:0/3967704962 conn(0x7f91400fd8f0 msgr2=0x7f91400ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:34.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.552+0000 7f914547c640 1 -- 172.21.15.139:0/3967704962 shutdown_connections 2024-09-16T01:37:34.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.552+0000 7f914547c640 1 -- 172.21.15.139:0/3967704962 wait complete. 2024-09-16T01:37:34.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.553+0000 7f914547c640 1 Processor -- start 2024-09-16T01:37:34.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.553+0000 7f914547c640 1 -- start start 2024-09-16T01:37:34.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.553+0000 7f914547c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9140101df0 0x7f9140198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:34.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.553+0000 7f914547c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91401027a0 0x7f91401989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:34.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.553+0000 7f914547c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9140106f20 0x7f914019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:34.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.553+0000 7f914547c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9140111f80 con 0x7f91401027a0 2024-09-16T01:37:34.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.553+0000 7f914547c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9140111e00 con 0x7f9140101df0 2024-09-16T01:37:34.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.553+0000 7f914547c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9140112100 con 0x7f9140106f20 2024-09-16T01:37:34.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.553+0000 7f913e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91401027a0 0x7f91401989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:34.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.553+0000 7f913e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91401027a0 0x7f91401989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42432/0 (socket says 172.21.15.139:42432) 2024-09-16T01:37:34.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.553+0000 7f913e7fc640 1 -- 172.21.15.139:0/1879442306 learned_addr learned my addr 172.21.15.139:0/1879442306 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:34.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.553+0000 7f913effd640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9140101df0 0x7f9140198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:34.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.554+0000 7f913f7fe640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9140106f20 0x7f914019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:34.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.554+0000 7f913e7fc640 1 -- 172.21.15.139:0/1879442306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9140106f20 msgr2=0x7f914019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:34.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.554+0000 7f913e7fc640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9140106f20 0x7f914019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:34.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.554+0000 7f913e7fc640 1 -- 172.21.15.139:0/1879442306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9140101df0 msgr2=0x7f9140198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:34.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.554+0000 7f913e7fc640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9140101df0 0x7f9140198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:34.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.554+0000 7f913e7fc640 1 -- 172.21.15.139:0/1879442306 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914019d410 con 0x7f91401027a0 2024-09-16T01:37:34.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.554+0000 7f913f7fe640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9140106f20 0x7f914019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:37:34.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.554+0000 7f913e7fc640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91401027a0 0x7f91401989a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9128006010 tx=0x7f91280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:34.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.554+0000 7f9137fff640 1 -- 172.21.15.139:0/1879442306 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912803f030 con 0x7f91401027a0 2024-09-16T01:37:34.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.554+0000 7f914547c640 1 -- 172.21.15.139:0/1879442306 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914019d6a0 con 0x7f91401027a0 2024-09-16T01:37:34.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.554+0000 7f9137fff640 1 -- 172.21.15.139:0/1879442306 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9128003ce0 con 0x7f91401027a0 2024-09-16T01:37:34.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.554+0000 7f9137fff640 1 -- 172.21.15.139:0/1879442306 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128035070 con 0x7f91401027a0 2024-09-16T01:37:34.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.554+0000 7f914547c640 1 -- 172.21.15.139:0/1879442306 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91401a4fe0 con 0x7f91401027a0 2024-09-16T01:37:34.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.555+0000 7f914547c640 1 -- 172.21.15.139:0/1879442306 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91401021d0 con 0x7f91401027a0 2024-09-16T01:37:34.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.556+0000 7f9137fff640 1 -- 172.21.15.139:0/1879442306 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9128042030 con 0x7f91401027a0 2024-09-16T01:37:34.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.557+0000 7f9137fff640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f912c0a3240 0x7f912c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:34.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.557+0000 7f913effd640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f912c0a3240 0x7f912c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:34.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.557+0000 7f9137fff640 1 -- 172.21.15.139:0/1879442306 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f91280efc20 con 0x7f91401027a0 2024-09-16T01:37:34.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.557+0000 7f913effd640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f912c0a3240 0x7f912c0a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9138005fd0 tx=0x7f9138009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:34.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.564+0000 7f9137fff640 1 -- 172.21.15.139:0/1879442306 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f91280bcea0 con 0x7f91401027a0 2024-09-16T01:37:34.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:34 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] up:boot 2024-09-16T01:37:34.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:34 smithi176 ceph-mon[35324]: fsmap 6 up:standby 2024-09-16T01:37:34.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:34 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T01:37:34.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:34 smithi176 ceph-mon[35324]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:34.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:34 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:34.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:34 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:34.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:34 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:34.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:34 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:34.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:34 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:34.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:34 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:34.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:34 smithi176 ceph-mon[35324]: Deploying daemon mds.c on smithi176 2024-09-16T01:37:34.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.768+0000 7f914547c640 1 -- 172.21.15.139:0/1879442306 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f914010a160 con 0x7f91401027a0 2024-09-16T01:37:34.770 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:34.770 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":7,"btime":"2024-09-16T01:37:33:386597+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-16T01:37:34.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.768+0000 7f9137fff640 1 -- 172.21.15.139:0/1879442306 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5569 (secure 0 0 0) 0x7f91280bc7a0 con 0x7f91401027a0 2024-09-16T01:37:34.770 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 7 2024-09-16T01:37:34.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.771+0000 7f914547c640 1 -- 172.21.15.139:0/1879442306 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f912c0a3240 msgr2=0x7f912c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:34.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.771+0000 7f914547c640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f912c0a3240 0x7f912c0a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9138005fd0 tx=0x7f9138009500 comp rx=0 tx=0).stop 2024-09-16T01:37:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.771+0000 7f914547c640 1 -- 172.21.15.139:0/1879442306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91401027a0 msgr2=0x7f91401989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.771+0000 7f914547c640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91401027a0 0x7f91401989a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9128006010 tx=0x7f91280042f0 comp rx=0 tx=0).stop 2024-09-16T01:37:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.771+0000 7f914547c640 1 -- 172.21.15.139:0/1879442306 shutdown_connections 2024-09-16T01:37:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.771+0000 7f914547c640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f912c0a3240 0x7f912c0a5700 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.771+0000 7f914547c640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9140106f20 0x7f914019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:34.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.771+0000 7f914547c640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91401027a0 0x7f91401989a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:34.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.771+0000 7f914547c640 1 --2- 172.21.15.139:0/1879442306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9140101df0 0x7f9140198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:34.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.771+0000 7f914547c640 1 -- 172.21.15.139:0/1879442306 >> 172.21.15.139:0/1879442306 conn(0x7f91400fd8f0 msgr2=0x7f91400fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:34.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.771+0000 7f914547c640 1 -- 172.21.15.139:0/1879442306 shutdown_connections 2024-09-16T01:37:34.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:34.771+0000 7f914547c640 1 -- 172.21.15.139:0/1879442306 wait complete. 2024-09-16T01:37:34.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:34 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] up:boot 2024-09-16T01:37:34.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:34 smithi164 ceph-mon[35121]: fsmap 6 up:standby 2024-09-16T01:37:34.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:34 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T01:37:34.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:34 smithi164 ceph-mon[35121]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:34.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:34 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:34.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:34 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:34.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:34 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:34.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:34 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:34.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:34 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:34.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:34 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:34.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:34 smithi164 ceph-mon[35121]: Deploying daemon mds.c on smithi176 2024-09-16T01:37:35.405 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 01:37:35 smithi176 systemd[1]: Starting Ceph mds.c for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:37:35.666 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:35 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1879442306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:35.667 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 01:37:35 smithi176 podman[65239]: 2024-09-16 01:37:35.411759438 +0000 UTC m=+0.015955633 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:37:35.667 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 01:37:35 smithi176 podman[65239]: 2024-09-16 01:37:35.524445741 +0000 UTC m=+0.128641936 container create e845a934580cc5cbb4c758fff469ca2359f4ba077ef2ff76d2d3de49e76b7f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T01:37:35.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:35 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1879442306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:35.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:35 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1879442306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:36.187 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 01:37:35 smithi176 podman[65239]: 2024-09-16 01:37:35.735346262 +0000 UTC m=+0.339542456 container init e845a934580cc5cbb4c758fff469ca2359f4ba077ef2ff76d2d3de49e76b7f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T01:37:36.187 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 01:37:35 smithi176 podman[65239]: 2024-09-16 01:37:35.738080843 +0000 UTC m=+0.342277048 container start e845a934580cc5cbb4c758fff469ca2359f4ba077ef2ff76d2d3de49e76b7f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T01:37:36.187 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 01:37:35 smithi176 ceph-mds[65258]: -- 172.21.15.176:0/1292747185 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb855cf860 con 0x55cb855ac800 2024-09-16T01:37:36.187 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 01:37:35 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-c[65254]: starting mds.c at 2024-09-16T01:37:36.187 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 01:37:35 smithi176 bash[65239]: e845a934580cc5cbb4c758fff469ca2359f4ba077ef2ff76d2d3de49e76b7f91 2024-09-16T01:37:36.188 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 01:37:35 smithi176 systemd[1]: Started Ceph mds.c for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:37:36.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:36 smithi164 ceph-mon[35121]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:36.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:36 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:36.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:36 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:36.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:36 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:36.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:36 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:36.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:36 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:36.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:36 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:36.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:36 smithi139 ceph-mon[31175]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:36.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:36 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:36.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:36 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:36.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:36 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:36.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:36 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:36.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:36 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:36.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:36 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:36.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:36 smithi176 ceph-mon[35324]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:36.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:36 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:36.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:36 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:36.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:36 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:36.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:36 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:36.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:36 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:36.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:36 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:37.488 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:37:37.609 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:37 smithi164 ceph-mon[35121]: Deploying daemon mds.h on smithi164 2024-09-16T01:37:37.609 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:37 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] up:boot 2024-09-16T01:37:37.609 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:37 smithi164 ceph-mon[35121]: fsmap 7 up:standby 2024-09-16T01:37:37.609 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:37 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T01:37:37.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:37 smithi139 ceph-mon[31175]: Deploying daemon mds.h on smithi164 2024-09-16T01:37:37.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:37 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] up:boot 2024-09-16T01:37:37.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:37 smithi139 ceph-mon[31175]: fsmap 7 up:standby 2024-09-16T01:37:37.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:37 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T01:37:37.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:37 smithi176 ceph-mon[35324]: Deploying daemon mds.h on smithi164 2024-09-16T01:37:37.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:37 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] up:boot 2024-09-16T01:37:37.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:37 smithi176 ceph-mon[35324]: fsmap 7 up:standby 2024-09-16T01:37:37.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:37 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T01:37:37.748 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:38.008 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 01:37:37 smithi164 systemd[1]: Starting Ceph mds.h for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:37:38.262 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 01:37:38 smithi164 podman[64322]: 2024-09-16 01:37:37.9100983 +0000 UTC m=+0.015955749 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:37:38.263 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 01:37:38 smithi164 podman[64322]: 2024-09-16 01:37:38.025798118 +0000 UTC m=+0.131655556 container create c0b5c2be9e172a10d157f6ffa3e039d6b5d0558327215ff86390f4d037763b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-h, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-16T01:37:38.263 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 01:37:38 smithi164 podman[64322]: 2024-09-16 01:37:38.261861525 +0000 UTC m=+0.367718964 container init c0b5c2be9e172a10d157f6ffa3e039d6b5d0558327215ff86390f4d037763b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-h, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T01:37:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.249+0000 7ff369a57640 1 -- 172.21.15.139:0/1789625766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3641082a0 msgr2=0x7ff36410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.249+0000 7ff369a57640 1 --2- 172.21.15.139:0/1789625766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3641082a0 0x7ff36410ec40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009cd0 tx=0x7ff35c02a8e0 comp rx=0 tx=0).stop 2024-09-16T01:37:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.249+0000 7ff369a57640 1 -- 172.21.15.139:0/1789625766 shutdown_connections 2024-09-16T01:37:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.249+0000 7ff369a57640 1 --2- 172.21.15.139:0/1789625766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3641082a0 0x7ff36410ec40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.249+0000 7ff369a57640 1 --2- 172.21.15.139:0/1789625766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3641036e0 0x7ff364107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.249+0000 7ff369a57640 1 --2- 172.21.15.139:0/1789625766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff364102d30 0x7ff364103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.249+0000 7ff369a57640 1 -- 172.21.15.139:0/1789625766 >> 172.21.15.139:0/1789625766 conn(0x7ff3640fc7d0 msgr2=0x7ff3640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.249+0000 7ff369a57640 1 -- 172.21.15.139:0/1789625766 shutdown_connections 2024-09-16T01:37:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.249+0000 7ff369a57640 1 -- 172.21.15.139:0/1789625766 wait complete. 2024-09-16T01:37:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.250+0000 7ff369a57640 1 Processor -- start 2024-09-16T01:37:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.250+0000 7ff369a57640 1 -- start start 2024-09-16T01:37:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.250+0000 7ff369a57640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff364102d30 0x7ff3641984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.250+0000 7ff369a57640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3641036e0 0x7ff364198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.250+0000 7ff369a57640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3641082a0 0x7ff36419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.250+0000 7ff369a57640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641120a0 con 0x7ff3641082a0 2024-09-16T01:37:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.250+0000 7ff369a57640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff364111f20 con 0x7ff3641036e0 2024-09-16T01:37:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.250+0000 7ff369a57640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff364112220 con 0x7ff364102d30 2024-09-16T01:37:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.250+0000 7ff3637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3641082a0 0x7ff36419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.250+0000 7ff3637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3641082a0 0x7ff36419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42442/0 (socket says 172.21.15.139:42442) 2024-09-16T01:37:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.250+0000 7ff3637fe640 1 -- 172.21.15.139:0/414213758 learned_addr learned my addr 172.21.15.139:0/414213758 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff362ffd640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff364102d30 0x7ff3641984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff35bfff640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3641036e0 0x7ff364198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff3637fe640 1 -- 172.21.15.139:0/414213758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff364102d30 msgr2=0x7ff3641984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff3637fe640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff364102d30 0x7ff3641984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff3637fe640 1 -- 172.21.15.139:0/414213758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3641036e0 msgr2=0x7ff364198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff3637fe640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3641036e0 0x7ff364198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff3637fe640 1 -- 172.21.15.139:0/414213758 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff36419d480 con 0x7ff3641082a0 2024-09-16T01:37:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff362ffd640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff364102d30 0x7ff3641984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:37:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff35bfff640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3641036e0 0x7ff364198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:37:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff3637fe640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3641082a0 0x7ff36419cda0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff35c005cc0 tx=0x7ff35c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff368a55640 1 -- 172.21.15.139:0/414213758 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c004040 con 0x7ff3641082a0 2024-09-16T01:37:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff368a55640 1 -- 172.21.15.139:0/414213758 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff35c0046c0 con 0x7ff3641082a0 2024-09-16T01:37:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff369a57640 1 -- 172.21.15.139:0/414213758 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff36419d710 con 0x7ff3641082a0 2024-09-16T01:37:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff368a55640 1 -- 172.21.15.139:0/414213758 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c030e00 con 0x7ff3641082a0 2024-09-16T01:37:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.251+0000 7ff369a57640 1 -- 172.21.15.139:0/414213758 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3641a5050 con 0x7ff3641082a0 2024-09-16T01:37:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.252+0000 7ff369a57640 1 -- 172.21.15.139:0/414213758 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff364103110 con 0x7ff3641082a0 2024-09-16T01:37:38.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.253+0000 7ff368a55640 1 -- 172.21.15.139:0/414213758 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff35c0305a0 con 0x7ff3641082a0 2024-09-16T01:37:38.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.254+0000 7ff368a55640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff3500a3240 0x7ff3500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:38.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.254+0000 7ff362ffd640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff3500a3240 0x7ff3500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:38.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.254+0000 7ff368a55640 1 -- 172.21.15.139:0/414213758 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7ff35c02e030 con 0x7ff3641082a0 2024-09-16T01:37:38.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.255+0000 7ff362ffd640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff3500a3240 0x7ff3500a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff35400baa0 tx=0x7ff354005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:38.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.257+0000 7ff368a55640 1 -- 172.21.15.139:0/414213758 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff35c04b060 con 0x7ff3641082a0 2024-09-16T01:37:38.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.464+0000 7ff369a57640 1 -- 172.21.15.139:0/414213758 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff36410a2b0 con 0x7ff3641082a0 2024-09-16T01:37:38.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.465+0000 7ff368a55640 1 -- 172.21.15.139:0/414213758 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7225 (secure 0 0 0) 0x7ff35c0be040 con 0x7ff3641082a0 2024-09-16T01:37:38.466 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:38.466 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":9,"btime":"2024-09-16T01:37:38:414094+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-16T01:37:38.466 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 9 2024-09-16T01:37:38.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.467+0000 7ff369a57640 1 -- 172.21.15.139:0/414213758 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff3500a3240 msgr2=0x7ff3500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:38.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.467+0000 7ff369a57640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff3500a3240 0x7ff3500a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff35400baa0 tx=0x7ff354005d90 comp rx=0 tx=0).stop 2024-09-16T01:37:38.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.467+0000 7ff369a57640 1 -- 172.21.15.139:0/414213758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3641082a0 msgr2=0x7ff36419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:38.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.467+0000 7ff369a57640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3641082a0 0x7ff36419cda0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff35c005cc0 tx=0x7ff35c032f40 comp rx=0 tx=0).stop 2024-09-16T01:37:38.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.467+0000 7ff369a57640 1 -- 172.21.15.139:0/414213758 shutdown_connections 2024-09-16T01:37:38.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.467+0000 7ff369a57640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff3500a3240 0x7ff3500a5700 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:38.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.467+0000 7ff369a57640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3641082a0 0x7ff36419cda0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:38.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.467+0000 7ff369a57640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3641036e0 0x7ff364198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:38.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.467+0000 7ff369a57640 1 --2- 172.21.15.139:0/414213758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff364102d30 0x7ff3641984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:38.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.467+0000 7ff369a57640 1 -- 172.21.15.139:0/414213758 >> 172.21.15.139:0/414213758 conn(0x7ff3640fc7d0 msgr2=0x7ff364106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:38.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.467+0000 7ff369a57640 1 -- 172.21.15.139:0/414213758 shutdown_connections 2024-09-16T01:37:38.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:38.467+0000 7ff369a57640 1 -- 172.21.15.139:0/414213758 wait complete. 2024-09-16T01:37:38.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:38 smithi164 ceph-mon[35121]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:38.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:38 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:38.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:38 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:38.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:38 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:38.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:38 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:38.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:38 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:38.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:38 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:38.546 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 01:37:38 smithi164 podman[64322]: 2024-09-16 01:37:38.26467073 +0000 UTC m=+0.370528169 container start c0b5c2be9e172a10d157f6ffa3e039d6b5d0558327215ff86390f4d037763b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-09-16T01:37:38.546 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 01:37:38 smithi164 ceph-mds[64341]: -- 172.21.15.164:0/2246194405 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556ad0b31860 con 0x556ad0b0e800 2024-09-16T01:37:38.546 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 01:37:38 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-h[64337]: starting mds.h at 2024-09-16T01:37:38.546 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 01:37:38 smithi164 bash[64322]: c0b5c2be9e172a10d157f6ffa3e039d6b5d0558327215ff86390f4d037763b49 2024-09-16T01:37:38.546 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 01:37:38 smithi164 systemd[1]: Started Ceph mds.h for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:37:38.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:38 smithi139 ceph-mon[31175]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:38.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:38 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:38.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:38 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:38.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:38 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:38.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:38 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:38.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:38 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:38.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:38 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:38.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:38 smithi176 ceph-mon[35324]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:38.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:38 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:38.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:38 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:38.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:38 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:38.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:38 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:38.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:38 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:38.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:38 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:39 smithi139 ceph-mon[31175]: Deploying daemon mds.i on smithi176 2024-09-16T01:37:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:39 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] up:boot 2024-09-16T01:37:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:39 smithi139 ceph-mon[31175]: fsmap 8 up:standby 2024-09-16T01:37:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:39 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T01:37:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:39 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/414213758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:39.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:39 smithi176 ceph-mon[35324]: Deploying daemon mds.i on smithi176 2024-09-16T01:37:39.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:39 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] up:boot 2024-09-16T01:37:39.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:39 smithi176 ceph-mon[35324]: fsmap 8 up:standby 2024-09-16T01:37:39.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:39 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T01:37:39.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:39 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/414213758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:39 smithi164 ceph-mon[35121]: Deploying daemon mds.i on smithi176 2024-09-16T01:37:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:39 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] up:boot 2024-09-16T01:37:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:39 smithi164 ceph-mon[35121]: fsmap 8 up:standby 2024-09-16T01:37:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:39 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T01:37:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:39 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/414213758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:40.437 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 01:37:40 smithi176 systemd[1]: Starting Ceph mds.i for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:37:40.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:40 smithi164 ceph-mon[35121]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:40.814 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:40 smithi176 ceph-mon[35324]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:40.815 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 01:37:40 smithi176 podman[65539]: 2024-09-16 01:37:40.456426434 +0000 UTC m=+0.015955835 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:37:40.815 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 01:37:40 smithi176 podman[65539]: 2024-09-16 01:37:40.569805274 +0000 UTC m=+0.129334665 container create 5acde05ee65539e973ac7a634164c21d86aa14cd05d219bd07d5d9144ca9b923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T01:37:40.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:40 smithi139 ceph-mon[31175]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:41.187 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 01:37:40 smithi176 podman[65539]: 2024-09-16 01:37:40.814161137 +0000 UTC m=+0.373690529 container init 5acde05ee65539e973ac7a634164c21d86aa14cd05d219bd07d5d9144ca9b923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T01:37:41.187 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 01:37:40 smithi176 podman[65539]: 2024-09-16 01:37:40.817244322 +0000 UTC m=+0.376773730 container start 5acde05ee65539e973ac7a634164c21d86aa14cd05d219bd07d5d9144ca9b923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-i, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T01:37:41.187 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 01:37:40 smithi176 ceph-mds[65559]: -- 172.21.15.176:0/4065598559 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5612c967f860 con 0x5612c965d400 2024-09-16T01:37:41.187 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 01:37:40 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-i[65555]: starting mds.i at 2024-09-16T01:37:41.187 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 01:37:40 smithi176 bash[65539]: 5acde05ee65539e973ac7a634164c21d86aa14cd05d219bd07d5d9144ca9b923 2024-09-16T01:37:41.187 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 01:37:40 smithi176 systemd[1]: Started Ceph mds.i for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:37:41.300 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:37:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:42.289 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:41 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:42.289 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:41 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:42.289 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:41 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:42.289 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:41 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:42.289 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:41 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:42.289 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:41 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:42.289 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:41 smithi139 ceph-mon[31175]: Deploying daemon mds.g on smithi139 2024-09-16T01:37:42.289 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:41 smithi139 ceph-mon[31175]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:42.289 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:41 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] up:boot 2024-09-16T01:37:42.290 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:41 smithi139 ceph-mon[31175]: fsmap 9 up:standby 2024-09-16T01:37:42.290 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:41 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T01:37:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:41 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:41 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:41 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:41 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:41 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:41 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:41 smithi164 ceph-mon[35121]: Deploying daemon mds.g on smithi139 2024-09-16T01:37:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:41 smithi164 ceph-mon[35121]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:41 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] up:boot 2024-09-16T01:37:42.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:41 smithi164 ceph-mon[35121]: fsmap 9 up:standby 2024-09-16T01:37:42.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:41 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T01:37:42.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:41 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:42.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:41 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:42.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:41 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:42.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:41 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:42.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:41 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:42.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:41 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:42.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:41 smithi176 ceph-mon[35324]: Deploying daemon mds.g on smithi139 2024-09-16T01:37:42.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:41 smithi176 ceph-mon[35324]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:42.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:41 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] up:boot 2024-09-16T01:37:42.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:41 smithi176 ceph-mon[35324]: fsmap 9 up:standby 2024-09-16T01:37:42.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:41 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T01:37:42.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.505+0000 7f9a48d1b640 1 -- 172.21.15.139:0/1447264301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a44101420 msgr2=0x7f9a4410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:42.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.505+0000 7f9a48d1b640 1 --2- 172.21.15.139:0/1447264301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a44101420 0x7f9a4410d950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009930 tx=0x7f9a3402a420 comp rx=0 tx=0).stop 2024-09-16T01:37:42.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.505+0000 7f9a48d1b640 1 -- 172.21.15.139:0/1447264301 shutdown_connections 2024-09-16T01:37:42.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.505+0000 7f9a48d1b640 1 --2- 172.21.15.139:0/1447264301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a44101420 0x7f9a4410d950 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:42.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.505+0000 7f9a48d1b640 1 --2- 172.21.15.139:0/1447264301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a44100a80 0x7f9a44100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:42.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.505+0000 7f9a48d1b640 1 --2- 172.21.15.139:0/1447264301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a44106a80 0x7f9a44106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:42.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.505+0000 7f9a48d1b640 1 -- 172.21.15.139:0/1447264301 >> 172.21.15.139:0/1447264301 conn(0x7f9a440fc7a0 msgr2=0x7f9a440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:42.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.505+0000 7f9a48d1b640 1 -- 172.21.15.139:0/1447264301 shutdown_connections 2024-09-16T01:37:42.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.505+0000 7f9a48d1b640 1 -- 172.21.15.139:0/1447264301 wait complete. 2024-09-16T01:37:42.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.505+0000 7f9a48d1b640 1 Processor -- start 2024-09-16T01:37:42.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a48d1b640 1 -- start start 2024-09-16T01:37:42.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a48d1b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a44100a80 0x7f9a440fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:42.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a48d1b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a44101420 0x7f9a440fd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:42.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a48d1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a44106a80 0x7f9a440fd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:42.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a48d1b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44110ea0 con 0x7f9a44106a80 2024-09-16T01:37:42.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a48d1b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44110d20 con 0x7f9a44101420 2024-09-16T01:37:42.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a48d1b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44111020 con 0x7f9a44100a80 2024-09-16T01:37:42.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a42d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a44106a80 0x7f9a440fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:42.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a42d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a44106a80 0x7f9a440fd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59742/0 (socket says 172.21.15.139:59742) 2024-09-16T01:37:42.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a42d9d640 1 -- 172.21.15.139:0/3974396143 learned_addr learned my addr 172.21.15.139:0/3974396143 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:42.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a42d9d640 1 -- 172.21.15.139:0/3974396143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a44100a80 msgr2=0x7f9a440fccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:42.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a42d9d640 1 --2- 172.21.15.139:0/3974396143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a44100a80 0x7f9a440fccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:42.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a42d9d640 1 -- 172.21.15.139:0/3974396143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a44101420 msgr2=0x7f9a440fd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:42.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a42d9d640 1 --2- 172.21.15.139:0/3974396143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a44101420 0x7f9a440fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:42.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a42d9d640 1 -- 172.21.15.139:0/3974396143 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a441a68b0 con 0x7f9a44106a80 2024-09-16T01:37:42.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a42d9d640 1 --2- 172.21.15.139:0/3974396143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a44106a80 0x7f9a440fd770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a34005ab0 tx=0x7f9a340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:42.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a3b7fe640 1 -- 172.21.15.139:0/3974396143 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3403f030 con 0x7f9a44106a80 2024-09-16T01:37:42.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a48d1b640 1 -- 172.21.15.139:0/3974396143 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a441a6b40 con 0x7f9a44106a80 2024-09-16T01:37:42.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a3b7fe640 1 -- 172.21.15.139:0/3974396143 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a34003ce0 con 0x7f9a44106a80 2024-09-16T01:37:42.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.506+0000 7f9a3b7fe640 1 -- 172.21.15.139:0/3974396143 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34035070 con 0x7f9a44106a80 2024-09-16T01:37:42.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.507+0000 7f9a48d1b640 1 -- 172.21.15.139:0/3974396143 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a441a7020 con 0x7f9a44106a80 2024-09-16T01:37:42.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.507+0000 7f9a48d1b640 1 -- 172.21.15.139:0/3974396143 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a44100ee0 con 0x7f9a44106a80 2024-09-16T01:37:42.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.509+0000 7f9a3b7fe640 1 -- 172.21.15.139:0/3974396143 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9a34042030 con 0x7f9a44106a80 2024-09-16T01:37:42.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.509+0000 7f9a3b7fe640 1 --2- 172.21.15.139:0/3974396143 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9a300a3240 0x7f9a300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:42.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.509+0000 7f9a3b7fe640 1 -- 172.21.15.139:0/3974396143 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f9a340eff60 con 0x7f9a44106a80 2024-09-16T01:37:42.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.509+0000 7f9a4259c640 1 --2- 172.21.15.139:0/3974396143 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9a300a3240 0x7f9a300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:42.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.510+0000 7f9a4259c640 1 --2- 172.21.15.139:0/3974396143 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9a300a3240 0x7f9a300a5700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00ab90 tx=0x7f9a3c009250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:42.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.512+0000 7f9a3b7fe640 1 -- 172.21.15.139:0/3974396143 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a340bd080 con 0x7f9a44106a80 2024-09-16T01:37:42.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.719+0000 7f9a48d1b640 1 -- 172.21.15.139:0/3974396143 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a440fe220 con 0x7f9a44106a80 2024-09-16T01:37:42.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.719+0000 7f9a3b7fe640 1 -- 172.21.15.139:0/3974396143 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8055 (secure 0 0 0) 0x7f9a340bc980 con 0x7f9a44106a80 2024-09-16T01:37:42.720 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:42.720 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":10,"btime":"2024-09-16T01:37:41:474195+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-16T01:37:42.721 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 10 2024-09-16T01:37:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.721+0000 7f9a48d1b640 1 -- 172.21.15.139:0/3974396143 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9a300a3240 msgr2=0x7f9a300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.721+0000 7f9a48d1b640 1 --2- 172.21.15.139:0/3974396143 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9a300a3240 0x7f9a300a5700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00ab90 tx=0x7f9a3c009250 comp rx=0 tx=0).stop 2024-09-16T01:37:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.721+0000 7f9a48d1b640 1 -- 172.21.15.139:0/3974396143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a44106a80 msgr2=0x7f9a440fd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.721+0000 7f9a48d1b640 1 --2- 172.21.15.139:0/3974396143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a44106a80 0x7f9a440fd770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a34005ab0 tx=0x7f9a340042f0 comp rx=0 tx=0).stop 2024-09-16T01:37:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.722+0000 7f9a48d1b640 1 -- 172.21.15.139:0/3974396143 shutdown_connections 2024-09-16T01:37:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.722+0000 7f9a48d1b640 1 --2- 172.21.15.139:0/3974396143 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9a300a3240 0x7f9a300a5700 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.722+0000 7f9a48d1b640 1 --2- 172.21.15.139:0/3974396143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a44106a80 0x7f9a440fd770 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.722+0000 7f9a48d1b640 1 --2- 172.21.15.139:0/3974396143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a44101420 0x7f9a440fd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.722+0000 7f9a48d1b640 1 --2- 172.21.15.139:0/3974396143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a44100a80 0x7f9a440fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:42.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.722+0000 7f9a48d1b640 1 -- 172.21.15.139:0/3974396143 >> 172.21.15.139:0/3974396143 conn(0x7f9a440fc7a0 msgr2=0x7f9a4406a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:42.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.722+0000 7f9a48d1b640 1 -- 172.21.15.139:0/3974396143 shutdown_connections 2024-09-16T01:37:42.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:42.722+0000 7f9a48d1b640 1 -- 172.21.15.139:0/3974396143 wait complete. 2024-09-16T01:37:43.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:42 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3974396143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:43.295 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:42 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3974396143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:43.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:42 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3974396143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:43.673 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 01:37:43 smithi139 systemd[1]: Starting Ceph mds.g for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:37:43.989 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 01:37:43 smithi139 podman[68415]: 2024-09-16 01:37:43.836126542 +0000 UTC m=+0.016154107 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:37:43.990 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 01:37:43 smithi139 podman[68415]: 2024-09-16 01:37:43.961309162 +0000 UTC m=+0.141336715 container create 3e1b0cfb7c3c78852427252697e5ea1e507eff53fe5a29b645b9aff6b7c601c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-g, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T01:37:44.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:43 smithi164 ceph-mon[35121]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:44.298 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:43 smithi139 ceph-mon[31175]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:44.298 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 01:37:44 smithi139 podman[68415]: 2024-09-16 01:37:44.205873215 +0000 UTC m=+0.385900767 container init 3e1b0cfb7c3c78852427252697e5ea1e507eff53fe5a29b645b9aff6b7c601c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-g, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T01:37:44.298 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 01:37:44 smithi139 podman[68415]: 2024-09-16 01:37:44.20864114 +0000 UTC m=+0.388668701 container start 3e1b0cfb7c3c78852427252697e5ea1e507eff53fe5a29b645b9aff6b7c601c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-g, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T01:37:44.298 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 01:37:44 smithi139 ceph-mds[68434]: -- 172.21.15.139:0/2188268471 <== mon.0 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5613a15ed860 con 0x5613a15ca800 2024-09-16T01:37:44.298 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 01:37:44 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-g[68430]: starting mds.g at 2024-09-16T01:37:44.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:43 smithi176 ceph-mon[35324]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:44.672 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 01:37:44 smithi139 bash[68415]: 3e1b0cfb7c3c78852427252697e5ea1e507eff53fe5a29b645b9aff6b7c601c4 2024-09-16T01:37:44.673 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 01:37:44 smithi139 systemd[1]: Started Ceph mds.g for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:37:45.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:45 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:45.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:45 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:45.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:45 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:45.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:45 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:45.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:45 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:45.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:45 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:45.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:45 smithi139 ceph-mon[31175]: Deploying daemon mds.a on smithi139 2024-09-16T01:37:45.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:45 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.139:6838/1075273450,v1:172.21.15.139:6839/1075273450] up:boot 2024-09-16T01:37:45.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:45 smithi139 ceph-mon[31175]: fsmap 10 up:standby 2024-09-16T01:37:45.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:45 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T01:37:45.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:45 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:45.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:45 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:45.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:45 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:45.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:45 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:45.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:45 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:45.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:45 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:45.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:45 smithi176 ceph-mon[35324]: Deploying daemon mds.a on smithi139 2024-09-16T01:37:45.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:45 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.139:6838/1075273450,v1:172.21.15.139:6839/1075273450] up:boot 2024-09-16T01:37:45.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:45 smithi176 ceph-mon[35324]: fsmap 10 up:standby 2024-09-16T01:37:45.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:45 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T01:37:45.775 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:37:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:45 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:45 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:45 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:45 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:45 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:45 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:45 smithi164 ceph-mon[35121]: Deploying daemon mds.a on smithi139 2024-09-16T01:37:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:45 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.139:6838/1075273450,v1:172.21.15.139:6839/1075273450] up:boot 2024-09-16T01:37:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:45 smithi164 ceph-mon[35121]: fsmap 10 up:standby 2024-09-16T01:37:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:45 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T01:37:46.035 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:46.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.524+0000 7fed6090b640 1 -- 172.21.15.139:0/1508749723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c102570 msgr2=0x7fed5c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:46.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.524+0000 7fed6090b640 1 --2- 172.21.15.139:0/1508749723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c102570 0x7fed5c1029b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fed44009900 tx=0x7fed4402a480 comp rx=0 tx=0).stop 2024-09-16T01:37:46.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.524+0000 7fed6090b640 1 -- 172.21.15.139:0/1508749723 shutdown_connections 2024-09-16T01:37:46.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.524+0000 7fed6090b640 1 --2- 172.21.15.139:0/1508749723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed5c102ef0 0x7fed5c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:46.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.524+0000 7fed6090b640 1 --2- 172.21.15.139:0/1508749723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c102570 0x7fed5c1029b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:46.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.524+0000 7fed6090b640 1 --2- 172.21.15.139:0/1508749723 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed5c100e20 0x7fed5c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:46.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.524+0000 7fed6090b640 1 -- 172.21.15.139:0/1508749723 >> 172.21.15.139:0/1508749723 conn(0x7fed5c0fc7d0 msgr2=0x7fed5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:46.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.524+0000 7fed6090b640 1 -- 172.21.15.139:0/1508749723 shutdown_connections 2024-09-16T01:37:46.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.524+0000 7fed6090b640 1 -- 172.21.15.139:0/1508749723 wait complete. 2024-09-16T01:37:46.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed6090b640 1 Processor -- start 2024-09-16T01:37:46.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed6090b640 1 -- start start 2024-09-16T01:37:46.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed6090b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c100e20 0x7fed5c10e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:46.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed6090b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed5c102570 0x7fed5c10e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:46.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed6090b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed5c102ef0 0x7fed5c10ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:46.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed6090b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c112090 con 0x7fed5c100e20 2024-09-16T01:37:46.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed6090b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c111f10 con 0x7fed5c102ef0 2024-09-16T01:37:46.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed6090b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c112210 con 0x7fed5c102570 2024-09-16T01:37:46.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed5a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c100e20 0x7fed5c10e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:46.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed5a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c100e20 0x7fed5c10e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59768/0 (socket says 172.21.15.139:59768) 2024-09-16T01:37:46.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed5a59c640 1 -- 172.21.15.139:0/1677303973 learned_addr learned my addr 172.21.15.139:0/1677303973 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:46.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed5ad9d640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed5c102ef0 0x7fed5c10ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:46.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed59d9b640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed5c102570 0x7fed5c10e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:46.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed5a59c640 1 -- 172.21.15.139:0/1677303973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed5c102570 msgr2=0x7fed5c10e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:46.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed5a59c640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed5c102570 0x7fed5c10e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:46.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.525+0000 7fed5a59c640 1 -- 172.21.15.139:0/1677303973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed5c102ef0 msgr2=0x7fed5c10ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:46.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.526+0000 7fed5a59c640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed5c102ef0 0x7fed5c10ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:46.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.526+0000 7fed5a59c640 1 -- 172.21.15.139:0/1677303973 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c1a6930 con 0x7fed5c100e20 2024-09-16T01:37:46.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.526+0000 7fed59d9b640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed5c102570 0x7fed5c10e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:37:46.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.526+0000 7fed5a59c640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c100e20 0x7fed5c10e1d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed5400d950 tx=0x7fed5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:46.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.526+0000 7fed537fe640 1 -- 172.21.15.139:0/1677303973 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54017070 con 0x7fed5c100e20 2024-09-16T01:37:46.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.526+0000 7fed6090b640 1 -- 172.21.15.139:0/1677303973 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5c1a6c20 con 0x7fed5c100e20 2024-09-16T01:37:46.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.526+0000 7fed537fe640 1 -- 172.21.15.139:0/1677303973 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed54009c30 con 0x7fed5c100e20 2024-09-16T01:37:46.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.526+0000 7fed537fe640 1 -- 172.21.15.139:0/1677303973 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5400f6b0 con 0x7fed5c100e20 2024-09-16T01:37:46.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.526+0000 7fed6090b640 1 -- 172.21.15.139:0/1677303973 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5c1a71b0 con 0x7fed5c100e20 2024-09-16T01:37:46.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.527+0000 7fed6090b640 1 -- 172.21.15.139:0/1677303973 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed5c101200 con 0x7fed5c100e20 2024-09-16T01:37:46.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.528+0000 7fed537fe640 1 -- 172.21.15.139:0/1677303973 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fed540109d0 con 0x7fed5c100e20 2024-09-16T01:37:46.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.529+0000 7fed537fe640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fed480a3240 0x7fed480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:46.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.529+0000 7fed59d9b640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fed480a3240 0x7fed480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:46.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.529+0000 7fed537fe640 1 -- 172.21.15.139:0/1677303973 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fed540cf080 con 0x7fed5c100e20 2024-09-16T01:37:46.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.529+0000 7fed59d9b640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fed480a3240 0x7fed480a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fed5c10fc60 tx=0x7fed44005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:46.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.533+0000 7fed537fe640 1 -- 172.21.15.139:0/1677303973 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed54009de0 con 0x7fed5c100e20 2024-09-16T01:37:46.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:46 smithi139 ceph-mon[31175]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:46.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:46 smithi176 ceph-mon[35324]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:46.697 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:46 smithi164 ceph-mon[35121]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:46.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.752+0000 7fed6090b640 1 -- 172.21.15.139:0/1677303973 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed5c109090 con 0x7fed5c100e20 2024-09-16T01:37:46.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.752+0000 7fed537fe640 1 -- 172.21.15.139:0/1677303973 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8884 (secure 0 0 0) 0x7fed5409c240 con 0x7fed5c100e20 2024-09-16T01:37:46.754 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:46.754 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":11,"btime":"2024-09-16T01:37:44:988894+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-16T01:37:46.755 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 11 2024-09-16T01:37:46.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.754+0000 7fed6090b640 1 -- 172.21.15.139:0/1677303973 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fed480a3240 msgr2=0x7fed480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:46.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.754+0000 7fed6090b640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fed480a3240 0x7fed480a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fed5c10fc60 tx=0x7fed44005a20 comp rx=0 tx=0).stop 2024-09-16T01:37:46.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.754+0000 7fed6090b640 1 -- 172.21.15.139:0/1677303973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c100e20 msgr2=0x7fed5c10e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:46.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.754+0000 7fed6090b640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c100e20 0x7fed5c10e1d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed5400d950 tx=0x7fed5400de10 comp rx=0 tx=0).stop 2024-09-16T01:37:46.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.754+0000 7fed6090b640 1 -- 172.21.15.139:0/1677303973 shutdown_connections 2024-09-16T01:37:46.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.754+0000 7fed6090b640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fed480a3240 0x7fed480a5700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:46.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.754+0000 7fed6090b640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed5c102ef0 0x7fed5c10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:46.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.754+0000 7fed6090b640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed5c102570 0x7fed5c10e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:46.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.754+0000 7fed6090b640 1 --2- 172.21.15.139:0/1677303973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed5c100e20 0x7fed5c10e1d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:46.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.754+0000 7fed6090b640 1 -- 172.21.15.139:0/1677303973 >> 172.21.15.139:0/1677303973 conn(0x7fed5c0fc7d0 msgr2=0x7fed5c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:46.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.755+0000 7fed6090b640 1 -- 172.21.15.139:0/1677303973 shutdown_connections 2024-09-16T01:37:46.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:46.755+0000 7fed6090b640 1 -- 172.21.15.139:0/1677303973 wait complete. 2024-09-16T01:37:47.561 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:47 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1677303973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:47.607 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:47 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1677303973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:47.713 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:47 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1677303973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:47.813 INFO:journalctl@ceph.mds.a.smithi139.stdout:Sep 16 01:37:47 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-a[68838]: starting mds.a at 2024-09-16T01:37:47.813 INFO:journalctl@ceph.mds.a.smithi139.stdout:Sep 16 01:37:47 smithi139 bash[68808]: 73bf12d5392b95f0e54603f40bd6045882e5e5d670f2d5442aeda5d5b6e34d09 2024-09-16T01:37:48.173 INFO:journalctl@ceph.mds.a.smithi139.stdout:Sep 16 01:37:47 smithi139 systemd[1]: Started Ceph mds.a for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:37:48.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:48 smithi164 ceph-mon[35121]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:48.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:48.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:48.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:48.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:48.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:48.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:48 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:48 smithi139 ceph-mon[31175]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:48 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:48 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:48 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:48 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:48 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:48.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:48 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:48.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:48 smithi176 ceph-mon[35324]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:48.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:48 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:48.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:48 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:48.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:48 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:48.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:48 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T01:37:48.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:48 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T01:37:48.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:48 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:49.593 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:37:49.620 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:49 smithi164 ceph-mon[35121]: Deploying daemon mds.b on smithi164 2024-09-16T01:37:49.621 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:49 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.139:6840/1887551794,v1:172.21.15.139:6841/1887551794] up:boot 2024-09-16T01:37:49.621 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:49 smithi164 ceph-mon[35121]: fsmap 11 up:standby 2024-09-16T01:37:49.621 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:49 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T01:37:49.622 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:49 smithi139 ceph-mon[31175]: Deploying daemon mds.b on smithi164 2024-09-16T01:37:49.622 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:49 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.139:6840/1887551794,v1:172.21.15.139:6841/1887551794] up:boot 2024-09-16T01:37:49.622 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:49 smithi139 ceph-mon[31175]: fsmap 11 up:standby 2024-09-16T01:37:49.622 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:49 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T01:37:49.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:49 smithi176 ceph-mon[35324]: Deploying daemon mds.b on smithi164 2024-09-16T01:37:49.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:49 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.139:6840/1887551794,v1:172.21.15.139:6841/1887551794] up:boot 2024-09-16T01:37:49.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:49 smithi176 ceph-mon[35324]: fsmap 11 up:standby 2024-09-16T01:37:49.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:49 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T01:37:49.858 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:50.004 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:37:50 smithi164 systemd[1]: Starting Ceph mds.b for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:37:50.295 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:37:50 smithi164 podman[64632]: 2024-09-16 01:37:50.118339222 +0000 UTC m=+0.015925595 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:37:50.295 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:37:50 smithi164 podman[64632]: 2024-09-16 01:37:50.222629849 +0000 UTC m=+0.120233090 container create f7cd4f522d4f14f2dc04e2b7686d6fd8bd8777bed913e368c895bb91da88c6b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T01:37:50.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.303+0000 7f44af910640 1 -- 172.21.15.139:0/2548822047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a8102570 msgr2=0x7f44a81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:50.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.303+0000 7f44af910640 1 --2- 172.21.15.139:0/2548822047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a8102570 0x7f44a81029b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4498009900 tx=0x7f449802a480 comp rx=0 tx=0).stop 2024-09-16T01:37:50.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.303+0000 7f44af910640 1 -- 172.21.15.139:0/2548822047 shutdown_connections 2024-09-16T01:37:50.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.303+0000 7f44af910640 1 --2- 172.21.15.139:0/2548822047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44a8102ef0 0x7f44a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:50.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.303+0000 7f44af910640 1 --2- 172.21.15.139:0/2548822047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a8102570 0x7f44a81029b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:50.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.303+0000 7f44af910640 1 --2- 172.21.15.139:0/2548822047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a8100e20 0x7f44a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:50.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.303+0000 7f44af910640 1 -- 172.21.15.139:0/2548822047 >> 172.21.15.139:0/2548822047 conn(0x7f44a80fc7d0 msgr2=0x7f44a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:50.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.303+0000 7f44af910640 1 -- 172.21.15.139:0/2548822047 shutdown_connections 2024-09-16T01:37:50.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.304+0000 7f44af910640 1 -- 172.21.15.139:0/2548822047 wait complete. 2024-09-16T01:37:50.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.304+0000 7f44af910640 1 Processor -- start 2024-09-16T01:37:50.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.304+0000 7f44af910640 1 -- start start 2024-09-16T01:37:50.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.304+0000 7f44af910640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44a8100e20 0x7f44a8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:50.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.304+0000 7f44af910640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a8102570 0x7f44a8199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:50.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.304+0000 7f44af910640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a8102ef0 0x7f44a81a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:50.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.304+0000 7f44af910640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8110f80 con 0x7f44a8102ef0 2024-09-16T01:37:50.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.304+0000 7f44af910640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8110e00 con 0x7f44a8102570 2024-09-16T01:37:50.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.304+0000 7f44af910640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8111100 con 0x7f44a8100e20 2024-09-16T01:37:50.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.304+0000 7f44adead640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a8102ef0 0x7f44a81a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:50.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f44aceab640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a8102570 0x7f44a8199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:50.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f44adead640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a8102ef0 0x7f44a81a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59798/0 (socket says 172.21.15.139:59798) 2024-09-16T01:37:50.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f44adead640 1 -- 172.21.15.139:0/3089835364 learned_addr learned my addr 172.21.15.139:0/3089835364 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:50.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f44ad6ac640 1 --2- 172.21.15.139:0/3089835364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44a8100e20 0x7f44a8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:50.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f44adead640 1 -- 172.21.15.139:0/3089835364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44a8100e20 msgr2=0x7f44a8199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:50.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f44adead640 1 --2- 172.21.15.139:0/3089835364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44a8100e20 0x7f44a8199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:50.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f44adead640 1 -- 172.21.15.139:0/3089835364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a8102570 msgr2=0x7f44a8199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:50.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f44adead640 1 --2- 172.21.15.139:0/3089835364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a8102570 0x7f44a8199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:50.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f44adead640 1 -- 172.21.15.139:0/3089835364 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a81a4910 con 0x7f44a8102ef0 2024-09-16T01:37:50.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f44aceab640 1 --2- 172.21.15.139:0/3089835364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a8102570 0x7f44a8199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:37:50.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f44adead640 1 --2- 172.21.15.139:0/3089835364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a8102ef0 0x7f44a81a43d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f44a000eaa0 tx=0x7f44a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:50.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f449e7fc640 1 -- 172.21.15.139:0/3089835364 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a000cdb0 con 0x7f44a8102ef0 2024-09-16T01:37:50.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f449e7fc640 1 -- 172.21.15.139:0/3089835364 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a0005020 con 0x7f44a8102ef0 2024-09-16T01:37:50.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f44af910640 1 -- 172.21.15.139:0/3089835364 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a81a4ae0 con 0x7f44a8102ef0 2024-09-16T01:37:50.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f449e7fc640 1 -- 172.21.15.139:0/3089835364 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a00116b0 con 0x7f44a8102ef0 2024-09-16T01:37:50.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.305+0000 7f44af910640 1 -- 172.21.15.139:0/3089835364 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a81a4fa0 con 0x7f44a8102ef0 2024-09-16T01:37:50.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.306+0000 7f44af910640 1 -- 172.21.15.139:0/3089835364 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a8101200 con 0x7f44a8102ef0 2024-09-16T01:37:50.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.307+0000 7f449e7fc640 1 -- 172.21.15.139:0/3089835364 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f44a0005560 con 0x7f44a8102ef0 2024-09-16T01:37:50.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.308+0000 7f449e7fc640 1 --2- 172.21.15.139:0/3089835364 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f44940a3240 0x7f44940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:50.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.308+0000 7f44ad6ac640 1 --2- 172.21.15.139:0/3089835364 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f44940a3240 0x7f44940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:50.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.308+0000 7f449e7fc640 1 -- 172.21.15.139:0/3089835364 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f44a00cfa60 con 0x7f44a8102ef0 2024-09-16T01:37:50.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.308+0000 7f44ad6ac640 1 --2- 172.21.15.139:0/3089835364 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f44940a3240 0x7f44940a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009b60 tx=0x7f44a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:50.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.311+0000 7f449e7fc640 1 -- 172.21.15.139:0/3089835364 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44a0016030 con 0x7f44a8102ef0 2024-09-16T01:37:50.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.514+0000 7f44af910640 1 -- 172.21.15.139:0/3089835364 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44a8109090 con 0x7f44a8102ef0 2024-09-16T01:37:50.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.515+0000 7f449e7fc640 1 -- 172.21.15.139:0/3089835364 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9713 (secure 0 0 0) 0x7f44a009cb80 con 0x7f44a8102ef0 2024-09-16T01:37:50.517 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:50.517 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":12,"btime":"2024-09-16T01:37:48:355123+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-16T01:37:50.517 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 12 2024-09-16T01:37:50.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.517+0000 7f44af910640 1 -- 172.21.15.139:0/3089835364 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f44940a3240 msgr2=0x7f44940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:50.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.517+0000 7f44af910640 1 --2- 172.21.15.139:0/3089835364 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f44940a3240 0x7f44940a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009b60 tx=0x7f44a4005f90 comp rx=0 tx=0).stop 2024-09-16T01:37:50.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.517+0000 7f44af910640 1 -- 172.21.15.139:0/3089835364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a8102ef0 msgr2=0x7f44a81a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:50.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.517+0000 7f44af910640 1 --2- 172.21.15.139:0/3089835364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a8102ef0 0x7f44a81a43d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f44a000eaa0 tx=0x7f44a000ef60 comp rx=0 tx=0).stop 2024-09-16T01:37:50.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.517+0000 7f44af910640 1 -- 172.21.15.139:0/3089835364 shutdown_connections 2024-09-16T01:37:50.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.517+0000 7f44af910640 1 --2- 172.21.15.139:0/3089835364 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f44940a3240 0x7f44940a5700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:50.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.517+0000 7f44af910640 1 --2- 172.21.15.139:0/3089835364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a8102ef0 0x7f44a81a43d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:50.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.517+0000 7f44af910640 1 --2- 172.21.15.139:0/3089835364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a8102570 0x7f44a8199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:50.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.517+0000 7f44af910640 1 --2- 172.21.15.139:0/3089835364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44a8100e20 0x7f44a8199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:50.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.517+0000 7f44af910640 1 -- 172.21.15.139:0/3089835364 >> 172.21.15.139:0/3089835364 conn(0x7f44a80fc7d0 msgr2=0x7f44a81068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:50.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.517+0000 7f44af910640 1 -- 172.21.15.139:0/3089835364 shutdown_connections 2024-09-16T01:37:50.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:50.517+0000 7f44af910640 1 -- 172.21.15.139:0/3089835364 wait complete. 2024-09-16T01:37:50.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:50 smithi139 ceph-mon[31175]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:50.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:50 smithi176 ceph-mon[35324]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:50.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:50 smithi164 ceph-mon[35121]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:50.795 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:37:50 smithi164 podman[64632]: 2024-09-16 01:37:50.425212519 +0000 UTC m=+0.322798898 container init f7cd4f522d4f14f2dc04e2b7686d6fd8bd8777bed913e368c895bb91da88c6b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T01:37:50.795 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:37:50 smithi164 podman[64632]: 2024-09-16 01:37:50.428435575 +0000 UTC m=+0.326021956 container start f7cd4f522d4f14f2dc04e2b7686d6fd8bd8777bed913e368c895bb91da88c6b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T01:37:50.795 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:37:50 smithi164 ceph-mds[64652]: -- 172.21.15.164:0/2455485287 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564372927860 con 0x564372904800 2024-09-16T01:37:50.795 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:37:50 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b[64648]: starting mds.b at 2024-09-16T01:37:50.795 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:37:50 smithi164 bash[64632]: f7cd4f522d4f14f2dc04e2b7686d6fd8bd8777bed913e368c895bb91da88c6b9 2024-09-16T01:37:50.795 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:37:50 smithi164 systemd[1]: Started Ceph mds.b for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:37:51.570 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:51 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3089835364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:51.570 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:51 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.570 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:51 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.570 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:51 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.570 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:51 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.570 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:51 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.570 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:51 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:51.633 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:51 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3089835364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:51.633 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:51 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.633 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:51 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.633 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:51 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.633 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:51 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.633 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:51 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.633 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:51 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:51.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:51 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3089835364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:51.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:51 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:51 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:51 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:51 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:51 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:51.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:51 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:52.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:52 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] up:boot 2024-09-16T01:37:52.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:52 smithi139 ceph-mon[31175]: fsmap 12 up:standby 2024-09-16T01:37:52.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:52 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T01:37:52.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:52 smithi139 ceph-mon[31175]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:52.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:52 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:52.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:52 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:52.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:52 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:52.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:52 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:52.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:52 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] up:boot 2024-09-16T01:37:52.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:52 smithi176 ceph-mon[35324]: fsmap 12 up:standby 2024-09-16T01:37:52.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:52 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T01:37:52.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:52 smithi176 ceph-mon[35324]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:52.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:52 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:52.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:52 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:52.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:52 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:52.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:52 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:52 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] up:boot 2024-09-16T01:37:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:52 smithi164 ceph-mon[35121]: fsmap 12 up:standby 2024-09-16T01:37:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:52 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T01:37:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:52 smithi164 ceph-mon[35121]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:52 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:52 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:52 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:52 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:53.208 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:37:53.638 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:53.639 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:53.639 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:53.639 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:53.639 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:53.639 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:53.639 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:53 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:53.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:53.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:53.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:53 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:53 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:37:53.873 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:54.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.346+0000 7f2017810640 1 -- 172.21.15.139:0/3648043303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2010103660 msgr2=0x7f201010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:54.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.346+0000 7f2017810640 1 --2- 172.21.15.139:0/3648043303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2010103660 0x7f201010bb50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2004009900 tx=0x7f200402a480 comp rx=0 tx=0).stop 2024-09-16T01:37:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.346+0000 7f2017810640 1 -- 172.21.15.139:0/3648043303 shutdown_connections 2024-09-16T01:37:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.346+0000 7f2017810640 1 --2- 172.21.15.139:0/3648043303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f201010c170 0x7f201010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.346+0000 7f2017810640 1 --2- 172.21.15.139:0/3648043303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2010103660 0x7f201010bb50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.346+0000 7f2017810640 1 --2- 172.21.15.139:0/3648043303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2010102d40 0x7f2010103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.346+0000 7f2017810640 1 -- 172.21.15.139:0/3648043303 >> 172.21.15.139:0/3648043303 conn(0x7f20100fc7a0 msgr2=0x7f20100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.346+0000 7f2017810640 1 -- 172.21.15.139:0/3648043303 shutdown_connections 2024-09-16T01:37:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.346+0000 7f2017810640 1 -- 172.21.15.139:0/3648043303 wait complete. 2024-09-16T01:37:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.347+0000 7f2017810640 1 Processor -- start 2024-09-16T01:37:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.347+0000 7f2017810640 1 -- start start 2024-09-16T01:37:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.347+0000 7f2017810640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2010102d40 0x7f2010198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.347+0000 7f2017810640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2010103660 0x7f2010198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.347+0000 7f2017810640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f201010c170 0x7f201019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.347+0000 7f2017810640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2010111f70 con 0x7f2010103660 2024-09-16T01:37:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.347+0000 7f2017810640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2010111df0 con 0x7f2010102d40 2024-09-16T01:37:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.347+0000 7f2017810640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f20101120f0 con 0x7f201010c170 2024-09-16T01:37:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.347+0000 7f2014dab640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2010103660 0x7f2010198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.347+0000 7f20155ac640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2010102d40 0x7f2010198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.347+0000 7f2014dab640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2010103660 0x7f2010198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56534/0 (socket says 172.21.15.139:56534) 2024-09-16T01:37:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.347+0000 7f2014dab640 1 -- 172.21.15.139:0/3800840578 learned_addr learned my addr 172.21.15.139:0/3800840578 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.348+0000 7f2015dad640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f201010c170 0x7f201019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.348+0000 7f2014dab640 1 -- 172.21.15.139:0/3800840578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f201010c170 msgr2=0x7f201019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.348+0000 7f2014dab640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f201010c170 0x7f201019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.348+0000 7f2014dab640 1 -- 172.21.15.139:0/3800840578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2010102d40 msgr2=0x7f2010198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.348+0000 7f2014dab640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2010102d40 0x7f2010198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.348+0000 7f2014dab640 1 -- 172.21.15.139:0/3800840578 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201019d400 con 0x7f2010103660 2024-09-16T01:37:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.348+0000 7f20155ac640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2010102d40 0x7f2010198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:37:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.348+0000 7f2015dad640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f201010c170 0x7f201019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:37:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.348+0000 7f2014dab640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2010103660 0x7f2010198990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2004006010 tx=0x7f2004031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.348+0000 7f200e7fc640 1 -- 172.21.15.139:0/3800840578 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200403f030 con 0x7f2010103660 2024-09-16T01:37:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.348+0000 7f200e7fc640 1 -- 172.21.15.139:0/3800840578 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20040043d0 con 0x7f2010103660 2024-09-16T01:37:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.348+0000 7f2017810640 1 -- 172.21.15.139:0/3800840578 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201019d690 con 0x7f2010103660 2024-09-16T01:37:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.348+0000 7f200e7fc640 1 -- 172.21.15.139:0/3800840578 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004035070 con 0x7f2010103660 2024-09-16T01:37:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.348+0000 7f2017810640 1 -- 172.21.15.139:0/3800840578 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20101a4fd0 con 0x7f2010103660 2024-09-16T01:37:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.349+0000 7f2017810640 1 -- 172.21.15.139:0/3800840578 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201006b070 con 0x7f2010103660 2024-09-16T01:37:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.350+0000 7f200e7fc640 1 -- 172.21.15.139:0/3800840578 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2004042030 con 0x7f2010103660 2024-09-16T01:37:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.351+0000 7f200e7fc640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1ff80a3240 0x7f1ff80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.351+0000 7f20155ac640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1ff80a3240 0x7f1ff80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.351+0000 7f200e7fc640 1 -- 172.21.15.139:0/3800840578 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f20040efad0 con 0x7f2010103660 2024-09-16T01:37:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.351+0000 7f20155ac640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1ff80a3240 0x7f1ff80a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2008005fd0 tx=0x7f2008009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.354+0000 7f200e7fc640 1 -- 172.21.15.139:0/3800840578 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20040bcd30 con 0x7f2010103660 2024-09-16T01:37:54.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.573+0000 7f2017810640 1 -- 172.21.15.139:0/3800840578 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20101071e0 con 0x7f2010103660 2024-09-16T01:37:54.575 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:54 smithi139 ceph-mon[31175]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:54.575 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:54 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:54.575 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:54 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:54.575 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:54 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:54.575 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:54 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:54.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.573+0000 7f200e7fc640 1 -- 172.21.15.139:0/3800840578 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10542 (secure 0 0 0) 0x7f20040dc020 con 0x7f2010103660 2024-09-16T01:37:54.576 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:54.576 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":13,"btime":"2024-09-16T01:37:51:369848+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-16T01:37:54.577 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 13 2024-09-16T01:37:54.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.576+0000 7f2017810640 1 -- 172.21.15.139:0/3800840578 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1ff80a3240 msgr2=0x7f1ff80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:54.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.576+0000 7f2017810640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1ff80a3240 0x7f1ff80a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2008005fd0 tx=0x7f2008009500 comp rx=0 tx=0).stop 2024-09-16T01:37:54.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.576+0000 7f2017810640 1 -- 172.21.15.139:0/3800840578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2010103660 msgr2=0x7f2010198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:54.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.576+0000 7f2017810640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2010103660 0x7f2010198990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2004006010 tx=0x7f2004031b20 comp rx=0 tx=0).stop 2024-09-16T01:37:54.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.576+0000 7f2017810640 1 -- 172.21.15.139:0/3800840578 shutdown_connections 2024-09-16T01:37:54.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.576+0000 7f2017810640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1ff80a3240 0x7f1ff80a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:54.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.576+0000 7f2017810640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f201010c170 0x7f201019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:54.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.576+0000 7f2017810640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2010103660 0x7f2010198990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:54.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.576+0000 7f2017810640 1 --2- 172.21.15.139:0/3800840578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2010102d40 0x7f2010198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:54.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.576+0000 7f2017810640 1 -- 172.21.15.139:0/3800840578 >> 172.21.15.139:0/3800840578 conn(0x7f20100fc7a0 msgr2=0x7f20100fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:54.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.576+0000 7f2017810640 1 -- 172.21.15.139:0/3800840578 shutdown_connections 2024-09-16T01:37:54.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:54.576+0000 7f2017810640 1 -- 172.21.15.139:0/3800840578 wait complete. 2024-09-16T01:37:54.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:54 smithi176 ceph-mon[35324]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:54.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:54 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:54.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:54 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:54.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:54 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:54.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:54 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:54 smithi164 ceph-mon[35121]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:54 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:54 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:54 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:54 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:55.168 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-16T01:37:55.442 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:55.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:55 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3800840578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:55.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:55 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3800840578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:55.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:55.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:55.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:55.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:55.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:55 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3800840578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:37:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:37:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:37:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:37:55.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.929+0000 7fa1104ef640 1 -- 172.21.15.139:0/3673856710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa108106a90 msgr2=0x7fa108106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:55.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.929+0000 7fa1104ef640 1 --2- 172.21.15.139:0/3673856710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa108106a90 0x7fa108106e70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa104009930 tx=0x7fa10402a420 comp rx=0 tx=0).stop 2024-09-16T01:37:55.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.929+0000 7fa1104ef640 1 -- 172.21.15.139:0/3673856710 shutdown_connections 2024-09-16T01:37:55.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.929+0000 7fa1104ef640 1 --2- 172.21.15.139:0/3673856710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa108101430 0x7fa10810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:55.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.929+0000 7fa1104ef640 1 --2- 172.21.15.139:0/3673856710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa108100a90 0x7fa108100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:55.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.929+0000 7fa1104ef640 1 --2- 172.21.15.139:0/3673856710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa108106a90 0x7fa108106e70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:55.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.929+0000 7fa1104ef640 1 -- 172.21.15.139:0/3673856710 >> 172.21.15.139:0/3673856710 conn(0x7fa1080fc7d0 msgr2=0x7fa1080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:55.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.929+0000 7fa1104ef640 1 -- 172.21.15.139:0/3673856710 shutdown_connections 2024-09-16T01:37:55.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.929+0000 7fa1104ef640 1 -- 172.21.15.139:0/3673856710 wait complete. 2024-09-16T01:37:55.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.929+0000 7fa1104ef640 1 Processor -- start 2024-09-16T01:37:55.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.929+0000 7fa1104ef640 1 -- start start 2024-09-16T01:37:55.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa1104ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa108100a90 0x7fa10810e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:55.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa1104ef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa108101430 0x7fa10810e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:55.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa1104ef640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa108106a90 0x7fa10810ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:55.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa1104ef640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa108112030 con 0x7fa108100a90 2024-09-16T01:37:55.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa1104ef640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa108111eb0 con 0x7fa108101430 2024-09-16T01:37:55.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa1104ef640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081121b0 con 0x7fa108106a90 2024-09-16T01:37:55.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa10e28b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa108100a90 0x7fa10810e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:55.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa10e28b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa108100a90 0x7fa10810e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56546/0 (socket says 172.21.15.139:56546) 2024-09-16T01:37:55.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa10e28b640 1 -- 172.21.15.139:0/76264592 learned_addr learned my addr 172.21.15.139:0/76264592 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:55.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa10da8a640 1 --2- 172.21.15.139:0/76264592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa108101430 0x7fa10810e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:55.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa10ea8c640 1 --2- 172.21.15.139:0/76264592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa108106a90 0x7fa10810ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:55.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa10e28b640 1 -- 172.21.15.139:0/76264592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa108106a90 msgr2=0x7fa10810ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:55.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa10e28b640 1 --2- 172.21.15.139:0/76264592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa108106a90 0x7fa10810ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:55.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa10e28b640 1 -- 172.21.15.139:0/76264592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa108101430 msgr2=0x7fa10810e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:55.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa10e28b640 1 --2- 172.21.15.139:0/76264592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa108101430 0x7fa10810e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:55.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa10e28b640 1 -- 172.21.15.139:0/76264592 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1081a6ab0 con 0x7fa108100a90 2024-09-16T01:37:55.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa10e28b640 1 --2- 172.21.15.139:0/76264592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa108100a90 0x7fa10810e3f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa104009a50 tx=0x7fa104004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:55.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa0ff7fe640 1 -- 172.21.15.139:0/76264592 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10403f030 con 0x7fa108100a90 2024-09-16T01:37:55.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.930+0000 7fa1104ef640 1 -- 172.21.15.139:0/76264592 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1081a6d40 con 0x7fa108100a90 2024-09-16T01:37:55.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.931+0000 7fa0ff7fe640 1 -- 172.21.15.139:0/76264592 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa104003ce0 con 0x7fa108100a90 2024-09-16T01:37:55.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.931+0000 7fa0ff7fe640 1 -- 172.21.15.139:0/76264592 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104035070 con 0x7fa108100a90 2024-09-16T01:37:55.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.931+0000 7fa1104ef640 1 -- 172.21.15.139:0/76264592 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1081a7210 con 0x7fa108100a90 2024-09-16T01:37:55.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.932+0000 7fa1104ef640 1 -- 172.21.15.139:0/76264592 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa108100f70 con 0x7fa108100a90 2024-09-16T01:37:55.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.934+0000 7fa0ff7fe640 1 -- 172.21.15.139:0/76264592 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa104042030 con 0x7fa108100a90 2024-09-16T01:37:55.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.934+0000 7fa0ff7fe640 1 --2- 172.21.15.139:0/76264592 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa0f40a3240 0x7fa0f40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:55.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.934+0000 7fa10da8a640 1 --2- 172.21.15.139:0/76264592 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa0f40a3240 0x7fa0f40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:55.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.934+0000 7fa0ff7fe640 1 -- 172.21.15.139:0/76264592 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fa1040efb10 con 0x7fa108100a90 2024-09-16T01:37:55.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.935+0000 7fa10da8a640 1 --2- 172.21.15.139:0/76264592 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa0f40a3240 0x7fa0f40a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa10810fe80 tx=0x7fa0f8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:55.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:55.937+0000 7fa0ff7fe640 1 -- 172.21.15.139:0/76264592 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1040bccd0 con 0x7fa108100a90 2024-09-16T01:37:56.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:56.104+0000 7fa1104ef640 1 -- 172.21.15.139:0/76264592 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa108109000 con 0x7fa108100a90 2024-09-16T01:37:56.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:56.104+0000 7fa0ff7fe640 1 -- 172.21.15.139:0/76264592 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7fa1040bc5d0 con 0x7fa108100a90 2024-09-16T01:37:56.106 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:37:56.106 INFO:teuthology.orchestra.run.smithi139.stdout:[ 2024-09-16T01:37:56.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:37:56.106 INFO:teuthology.orchestra.run.smithi139.stdout: "poolnum": 1, 2024-09-16T01:37:56.106 INFO:teuthology.orchestra.run.smithi139.stdout: "poolname": ".mgr" 2024-09-16T01:37:56.106 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:37:56.107 INFO:teuthology.orchestra.run.smithi139.stdout:] 2024-09-16T01:37:56.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:56.106+0000 7fa1104ef640 1 -- 172.21.15.139:0/76264592 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa0f40a3240 msgr2=0x7fa0f40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:56.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:56.106+0000 7fa1104ef640 1 --2- 172.21.15.139:0/76264592 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa0f40a3240 0x7fa0f40a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa10810fe80 tx=0x7fa0f8008040 comp rx=0 tx=0).stop 2024-09-16T01:37:56.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:56.106+0000 7fa1104ef640 1 -- 172.21.15.139:0/76264592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa108100a90 msgr2=0x7fa10810e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:56.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:56.106+0000 7fa1104ef640 1 --2- 172.21.15.139:0/76264592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa108100a90 0x7fa10810e3f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa104009a50 tx=0x7fa104004270 comp rx=0 tx=0).stop 2024-09-16T01:37:56.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:56.107+0000 7fa1104ef640 1 -- 172.21.15.139:0/76264592 shutdown_connections 2024-09-16T01:37:56.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:56.107+0000 7fa1104ef640 1 --2- 172.21.15.139:0/76264592 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa0f40a3240 0x7fa0f40a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:56.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:56.107+0000 7fa1104ef640 1 --2- 172.21.15.139:0/76264592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa108106a90 0x7fa10810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:56.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:56.107+0000 7fa1104ef640 1 --2- 172.21.15.139:0/76264592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa108101430 0x7fa10810e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:56.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:56.107+0000 7fa1104ef640 1 --2- 172.21.15.139:0/76264592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa108100a90 0x7fa10810e3f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:56.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:56.107+0000 7fa1104ef640 1 -- 172.21.15.139:0/76264592 >> 172.21.15.139:0/76264592 conn(0x7fa1080fc7d0 msgr2=0x7fa1080fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:56.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:56.107+0000 7fa1104ef640 1 -- 172.21.15.139:0/76264592 shutdown_connections 2024-09-16T01:37:56.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:56.107+0000 7fa1104ef640 1 -- 172.21.15.139:0/76264592 wait complete. 2024-09-16T01:37:56.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:56 smithi176 ceph-mon[35324]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:56.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:56 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/76264592' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T01:37:56.773 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:56 smithi139 ceph-mon[31175]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:56.773 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:56 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/76264592' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T01:37:56.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:56 smithi164 ceph-mon[35121]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:56.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:56 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/76264592' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T01:37:56.801 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T01:37:56.801 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T01:37:57.049 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.534+0000 7fa32bb51640 1 -- 172.21.15.139:0/3086722584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa324106a80 msgr2=0x7fa324106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.534+0000 7fa32bb51640 1 --2- 172.21.15.139:0/3086722584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa324106a80 0x7fa324106e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa320009930 tx=0x7fa32002a420 comp rx=0 tx=0).stop 2024-09-16T01:37:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.534+0000 7fa32bb51640 1 -- 172.21.15.139:0/3086722584 shutdown_connections 2024-09-16T01:37:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.534+0000 7fa32bb51640 1 --2- 172.21.15.139:0/3086722584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa324101420 0x7fa32410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.534+0000 7fa32bb51640 1 --2- 172.21.15.139:0/3086722584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa324100a80 0x7fa324100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.534+0000 7fa32bb51640 1 --2- 172.21.15.139:0/3086722584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa324106a80 0x7fa324106e60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.534+0000 7fa32bb51640 1 -- 172.21.15.139:0/3086722584 >> 172.21.15.139:0/3086722584 conn(0x7fa3240fc7a0 msgr2=0x7fa3240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.534+0000 7fa32bb51640 1 -- 172.21.15.139:0/3086722584 shutdown_connections 2024-09-16T01:37:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.534+0000 7fa32bb51640 1 -- 172.21.15.139:0/3086722584 wait complete. 2024-09-16T01:37:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.534+0000 7fa32bb51640 1 Processor -- start 2024-09-16T01:37:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa32bb51640 1 -- start start 2024-09-16T01:37:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa32bb51640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa324100a80 0x7fa324199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa32bb51640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa324101420 0x7fa324199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa32bb51640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa324106a80 0x7fa3241a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa32bb51640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa324110ea0 con 0x7fa324101420 2024-09-16T01:37:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa32bb51640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa324110d20 con 0x7fa324100a80 2024-09-16T01:37:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa32bb51640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa324111020 con 0x7fa324106a80 2024-09-16T01:37:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa3290ec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa324101420 0x7fa324199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa3290ec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa324101420 0x7fa324199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56566/0 (socket says 172.21.15.139:56566) 2024-09-16T01:37:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa3290ec640 1 -- 172.21.15.139:0/1117379474 learned_addr learned my addr 172.21.15.139:0/1117379474 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa3298ed640 1 --2- 172.21.15.139:0/1117379474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa324100a80 0x7fa324199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa3290ec640 1 -- 172.21.15.139:0/1117379474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa324106a80 msgr2=0x7fa3241a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa3290ec640 1 --2- 172.21.15.139:0/1117379474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa324106a80 0x7fa3241a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa3290ec640 1 -- 172.21.15.139:0/1117379474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa324100a80 msgr2=0x7fa324199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa3290ec640 1 --2- 172.21.15.139:0/1117379474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa324100a80 0x7fa324199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa3290ec640 1 -- 172.21.15.139:0/1117379474 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3241a48f0 con 0x7fa324101420 2024-09-16T01:37:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa3298ed640 1 --2- 172.21.15.139:0/1117379474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa324100a80 0x7fa324199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:37:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa3290ec640 1 --2- 172.21.15.139:0/1117379474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa324101420 0x7fa324199870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa31400d950 tx=0x7fa31400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa31affd640 1 -- 172.21.15.139:0/1117379474 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314017030 con 0x7fa324101420 2024-09-16T01:37:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa32bb51640 1 -- 172.21.15.139:0/1117379474 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3241a4ac0 con 0x7fa324101420 2024-09-16T01:37:57.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa31affd640 1 -- 172.21.15.139:0/1117379474 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa314004020 con 0x7fa324101420 2024-09-16T01:37:57.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa31affd640 1 -- 172.21.15.139:0/1117379474 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3140047a0 con 0x7fa324101420 2024-09-16T01:37:57.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.535+0000 7fa32bb51640 1 -- 172.21.15.139:0/1117379474 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3241a5000 con 0x7fa324101420 2024-09-16T01:37:57.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.537+0000 7fa32bb51640 1 -- 172.21.15.139:0/1117379474 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa324100ee0 con 0x7fa324101420 2024-09-16T01:37:57.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.538+0000 7fa31affd640 1 -- 172.21.15.139:0/1117379474 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa3140041d0 con 0x7fa324101420 2024-09-16T01:37:57.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.538+0000 7fa31affd640 1 --2- 172.21.15.139:0/1117379474 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa3100a3240 0x7fa3100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:57.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.538+0000 7fa3298ed640 1 --2- 172.21.15.139:0/1117379474 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa3100a3240 0x7fa3100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:57.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.538+0000 7fa31affd640 1 -- 172.21.15.139:0/1117379474 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fa3140cef30 con 0x7fa324101420 2024-09-16T01:37:57.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.539+0000 7fa3298ed640 1 --2- 172.21.15.139:0/1117379474 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa3100a3240 0x7fa3100a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa320000c00 tx=0x7fa320032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:57.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.542+0000 7fa31affd640 1 -- 172.21.15.139:0/1117379474 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa31409c0f0 con 0x7fa324101420 2024-09-16T01:37:57.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:57.712+0000 7fa32bb51640 1 -- 172.21.15.139:0/1117379474 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa324108a60 con 0x7fa324101420 2024-09-16T01:37:58.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:58.484+0000 7fa31affd640 1 -- 172.21.15.139:0/1117379474 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7fa31409b9f0 con 0x7fa324101420 2024-09-16T01:37:58.486 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' created 2024-09-16T01:37:58.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:58.486+0000 7fa32bb51640 1 -- 172.21.15.139:0/1117379474 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa3100a3240 msgr2=0x7fa3100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:58.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:58.486+0000 7fa32bb51640 1 --2- 172.21.15.139:0/1117379474 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa3100a3240 0x7fa3100a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa320000c00 tx=0x7fa320032c70 comp rx=0 tx=0).stop 2024-09-16T01:37:58.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:58.486+0000 7fa32bb51640 1 -- 172.21.15.139:0/1117379474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa324101420 msgr2=0x7fa324199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:58.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:58.486+0000 7fa32bb51640 1 --2- 172.21.15.139:0/1117379474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa324101420 0x7fa324199870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa31400d950 tx=0x7fa31400de10 comp rx=0 tx=0).stop 2024-09-16T01:37:58.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:58.486+0000 7fa32bb51640 1 -- 172.21.15.139:0/1117379474 shutdown_connections 2024-09-16T01:37:58.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:58.486+0000 7fa32bb51640 1 --2- 172.21.15.139:0/1117379474 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa3100a3240 0x7fa3100a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:58.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:58.486+0000 7fa32bb51640 1 --2- 172.21.15.139:0/1117379474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa324106a80 0x7fa3241a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:58.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:58.486+0000 7fa32bb51640 1 --2- 172.21.15.139:0/1117379474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa324101420 0x7fa324199870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:58.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:58.486+0000 7fa32bb51640 1 --2- 172.21.15.139:0/1117379474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa324100a80 0x7fa324199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:58.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:58.486+0000 7fa32bb51640 1 -- 172.21.15.139:0/1117379474 >> 172.21.15.139:0/1117379474 conn(0x7fa3240fc7a0 msgr2=0x7fa3240fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:58.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:58.487+0000 7fa32bb51640 1 -- 172.21.15.139:0/1117379474 shutdown_connections 2024-09-16T01:37:58.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:58.487+0000 7fa32bb51640 1 -- 172.21.15.139:0/1117379474 wait complete. 2024-09-16T01:37:58.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:58 smithi176 ceph-mon[35324]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:58.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:58 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1117379474' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T01:37:58.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:58 smithi164 ceph-mon[35121]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:58.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:58 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1117379474' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T01:37:58.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:58 smithi139 ceph-mon[31175]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:37:58.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:58 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1117379474' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T01:37:59.149 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-16T01:37:59.399 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:37:59.768 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:59 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1117379474' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T01:37:59.768 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:37:59 smithi139 ceph-mon[31175]: osdmap e73: 12 total, 12 up, 12 in 2024-09-16T01:37:59.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:59 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1117379474' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T01:37:59.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:37:59 smithi164 ceph-mon[35121]: osdmap e73: 12 total, 12 up, 12 in 2024-09-16T01:37:59.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.856+0000 7f04bff88640 1 -- 172.21.15.139:0/1140764790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b8101de0 msgr2=0x7f04b81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:59.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.856+0000 7f04bff88640 1 --2- 172.21.15.139:0/1140764790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b8101de0 0x7f04b81021c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009900 tx=0x7f04b402a410 comp rx=0 tx=0).stop 2024-09-16T01:37:59.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.856+0000 7f04bff88640 1 -- 172.21.15.139:0/1140764790 shutdown_connections 2024-09-16T01:37:59.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.856+0000 7f04bff88640 1 --2- 172.21.15.139:0/1140764790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04b8106f10 0x7f04b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:59.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.856+0000 7f04bff88640 1 --2- 172.21.15.139:0/1140764790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04b8102790 0x7f04b81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:59.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.856+0000 7f04bff88640 1 --2- 172.21.15.139:0/1140764790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b8101de0 0x7f04b81021c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:59.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.856+0000 7f04bff88640 1 -- 172.21.15.139:0/1140764790 >> 172.21.15.139:0/1140764790 conn(0x7f04b80fd8c0 msgr2=0x7f04b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:37:59.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.856+0000 7f04bff88640 1 -- 172.21.15.139:0/1140764790 shutdown_connections 2024-09-16T01:37:59.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.856+0000 7f04bff88640 1 -- 172.21.15.139:0/1140764790 wait complete. 2024-09-16T01:37:59.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.856+0000 7f04bff88640 1 Processor -- start 2024-09-16T01:37:59.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.856+0000 7f04bff88640 1 -- start start 2024-09-16T01:37:59.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bff88640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b8101de0 0x7f04b8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:59.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bff88640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04b8102790 0x7f04b8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:59.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bff88640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04b8106f10 0x7f04b819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:59.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bff88640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8111f90 con 0x7f04b8101de0 2024-09-16T01:37:59.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bff88640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8111e10 con 0x7f04b8106f10 2024-09-16T01:37:59.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bff88640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8112110 con 0x7f04b8102790 2024-09-16T01:37:59.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bdd24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b8101de0 0x7f04b8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:59.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bdd24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b8101de0 0x7f04b8198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56576/0 (socket says 172.21.15.139:56576) 2024-09-16T01:37:59.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bdd24640 1 -- 172.21.15.139:0/811106711 learned_addr learned my addr 172.21.15.139:0/811106711 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:37:59.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bdd24640 1 -- 172.21.15.139:0/811106711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04b8102790 msgr2=0x7f04b8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:59.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bdd24640 1 --2- 172.21.15.139:0/811106711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04b8102790 0x7f04b8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:59.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bdd24640 1 -- 172.21.15.139:0/811106711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04b8106f10 msgr2=0x7f04b819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:37:59.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bdd24640 1 --2- 172.21.15.139:0/811106711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04b8106f10 0x7f04b819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:37:59.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bdd24640 1 -- 172.21.15.139:0/811106711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b819d3c0 con 0x7f04b8101de0 2024-09-16T01:37:59.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bdd24640 1 --2- 172.21.15.139:0/811106711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b8101de0 0x7f04b8198410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f04b4005ab0 tx=0x7f04b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:59.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04aeffd640 1 -- 172.21.15.139:0/811106711 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b403f030 con 0x7f04b8101de0 2024-09-16T01:37:59.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bff88640 1 -- 172.21.15.139:0/811106711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b819d650 con 0x7f04b8101de0 2024-09-16T01:37:59.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04aeffd640 1 -- 172.21.15.139:0/811106711 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b4003ce0 con 0x7f04b8101de0 2024-09-16T01:37:59.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04aeffd640 1 -- 172.21.15.139:0/811106711 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4035070 con 0x7f04b8101de0 2024-09-16T01:37:59.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.857+0000 7f04bff88640 1 -- 172.21.15.139:0/811106711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b81a4f90 con 0x7f04b8101de0 2024-09-16T01:37:59.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.858+0000 7f04bff88640 1 -- 172.21.15.139:0/811106711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b81021c0 con 0x7f04b8101de0 2024-09-16T01:37:59.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.860+0000 7f04aeffd640 1 -- 172.21.15.139:0/811106711 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f04b4042030 con 0x7f04b8101de0 2024-09-16T01:37:59.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.860+0000 7f04aeffd640 1 --2- 172.21.15.139:0/811106711 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f04a40a3240 0x7f04a40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:37:59.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.860+0000 7f04bd523640 1 --2- 172.21.15.139:0/811106711 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f04a40a3240 0x7f04a40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:37:59.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.860+0000 7f04aeffd640 1 -- 172.21.15.139:0/811106711 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7f04b40f01d0 con 0x7f04b8101de0 2024-09-16T01:37:59.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.861+0000 7f04bd523640 1 --2- 172.21.15.139:0/811106711 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f04a40a3240 0x7f04a40a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f04b80fe130 tx=0x7f04a8005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:37:59.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:37:59.864+0000 7f04aeffd640 1 -- 172.21.15.139:0/811106711 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f04b40bd220 con 0x7f04b8101de0 2024-09-16T01:37:59.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:59 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1117379474' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T01:37:59.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:37:59 smithi176 ceph-mon[35324]: osdmap e73: 12 total, 12 up, 12 in 2024-09-16T01:38:00.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:00.029+0000 7f04bff88640 1 -- 172.21.15.139:0/811106711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f04b810a150 con 0x7f04b8101de0 2024-09-16T01:38:00.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:00.497+0000 7f04aeffd640 1 -- 172.21.15.139:0/811106711 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v75) ==== 171+0+0 (secure 0 0 0) 0x7f04b40bcb20 con 0x7f04b8101de0 2024-09-16T01:38:00.499 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' created 2024-09-16T01:38:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:00.499+0000 7f04bff88640 1 -- 172.21.15.139:0/811106711 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f04a40a3240 msgr2=0x7f04a40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:00.499+0000 7f04bff88640 1 --2- 172.21.15.139:0/811106711 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f04a40a3240 0x7f04a40a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f04b80fe130 tx=0x7f04a8005c70 comp rx=0 tx=0).stop 2024-09-16T01:38:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:00.499+0000 7f04bff88640 1 -- 172.21.15.139:0/811106711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b8101de0 msgr2=0x7f04b8198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:00.499+0000 7f04bff88640 1 --2- 172.21.15.139:0/811106711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b8101de0 0x7f04b8198410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f04b4005ab0 tx=0x7f04b40042f0 comp rx=0 tx=0).stop 2024-09-16T01:38:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:00.499+0000 7f04bff88640 1 -- 172.21.15.139:0/811106711 shutdown_connections 2024-09-16T01:38:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:00.499+0000 7f04bff88640 1 --2- 172.21.15.139:0/811106711 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f04a40a3240 0x7f04a40a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:00.499+0000 7f04bff88640 1 --2- 172.21.15.139:0/811106711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04b8106f10 0x7f04b819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:00.499+0000 7f04bff88640 1 --2- 172.21.15.139:0/811106711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04b8102790 0x7f04b8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:00.499+0000 7f04bff88640 1 --2- 172.21.15.139:0/811106711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b8101de0 0x7f04b8198410 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:00.499+0000 7f04bff88640 1 -- 172.21.15.139:0/811106711 >> 172.21.15.139:0/811106711 conn(0x7f04b80fd8c0 msgr2=0x7f04b80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:00.500+0000 7f04bff88640 1 -- 172.21.15.139:0/811106711 shutdown_connections 2024-09-16T01:38:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:00.500+0000 7f04bff88640 1 -- 172.21.15.139:0/811106711 wait complete. 2024-09-16T01:38:00.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:00 smithi164 ceph-mon[35121]: pgmap v269: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:00.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:00 smithi164 ceph-mon[35121]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T01:38:00.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:00 smithi164 ceph-mon[35121]: osdmap e74: 12 total, 12 up, 12 in 2024-09-16T01:38:00.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:00 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/811106711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T01:38:00.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:00 smithi139 ceph-mon[31175]: pgmap v269: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:00.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:00 smithi139 ceph-mon[31175]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T01:38:00.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:00 smithi139 ceph-mon[31175]: osdmap e74: 12 total, 12 up, 12 in 2024-09-16T01:38:00.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:00 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/811106711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T01:38:00.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:00 smithi176 ceph-mon[35324]: pgmap v269: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:00.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:00 smithi176 ceph-mon[35324]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T01:38:00.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:00 smithi176 ceph-mon[35324]: osdmap e74: 12 total, 12 up, 12 in 2024-09-16T01:38:00.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:00 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/811106711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T01:38:01.201 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T01:38:01.448 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:01.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:01 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/811106711' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T01:38:01.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:01 smithi164 ceph-mon[35121]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T01:38:01.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:01 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/811106711' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T01:38:01.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:01 smithi139 ceph-mon[31175]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T01:38:01.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:01 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/811106711' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T01:38:01.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:01 smithi176 ceph-mon[35324]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T01:38:01.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.916+0000 7f4a26712640 1 -- 172.21.15.139:0/3297778238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a20102ef0 msgr2=0x7f4a2010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:01.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.916+0000 7f4a26712640 1 --2- 172.21.15.139:0/3297778238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a20102ef0 0x7f4a2010da40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009ca0 tx=0x7f4a1802a950 comp rx=0 tx=0).stop 2024-09-16T01:38:01.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.916+0000 7f4a26712640 1 -- 172.21.15.139:0/3297778238 shutdown_connections 2024-09-16T01:38:01.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.916+0000 7f4a26712640 1 --2- 172.21.15.139:0/3297778238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a20102ef0 0x7f4a2010da40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:01.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.916+0000 7f4a26712640 1 --2- 172.21.15.139:0/3297778238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a20102570 0x7f4a201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:01.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.916+0000 7f4a26712640 1 --2- 172.21.15.139:0/3297778238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a20100e20 0x7f4a20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:01.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.916+0000 7f4a26712640 1 -- 172.21.15.139:0/3297778238 >> 172.21.15.139:0/3297778238 conn(0x7f4a200fc7d0 msgr2=0x7f4a200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.916+0000 7f4a26712640 1 -- 172.21.15.139:0/3297778238 shutdown_connections 2024-09-16T01:38:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.916+0000 7f4a26712640 1 -- 172.21.15.139:0/3297778238 wait complete. 2024-09-16T01:38:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a26712640 1 Processor -- start 2024-09-16T01:38:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a26712640 1 -- start start 2024-09-16T01:38:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a26712640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a20100e20 0x7f4a20197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a26712640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a20102570 0x7f4a20197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a26712640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a20102ef0 0x7f4a2019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a26712640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4a20110f80 con 0x7f4a20102ef0 2024-09-16T01:38:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a26712640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4a20110e00 con 0x7f4a20100e20 2024-09-16T01:38:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a26712640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4a20111100 con 0x7f4a20102570 2024-09-16T01:38:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a24caf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a20102ef0 0x7f4a2019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a24caf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a20102ef0 0x7f4a2019bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56582/0 (socket says 172.21.15.139:56582) 2024-09-16T01:38:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a24caf640 1 -- 172.21.15.139:0/1637502916 learned_addr learned my addr 172.21.15.139:0/1637502916 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:01.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a1ffff640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a20100e20 0x7f4a20197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:01.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a1f7fe640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a20102570 0x7f4a20197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:01.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a24caf640 1 -- 172.21.15.139:0/1637502916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a20102570 msgr2=0x7f4a20197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:01.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a24caf640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a20102570 0x7f4a20197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:01.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a24caf640 1 -- 172.21.15.139:0/1637502916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a20100e20 msgr2=0x7f4a20197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:01.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a24caf640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a20100e20 0x7f4a20197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:01.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.917+0000 7f4a24caf640 1 -- 172.21.15.139:0/1637502916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2019c3b0 con 0x7f4a20102ef0 2024-09-16T01:38:01.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.918+0000 7f4a1f7fe640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a20102570 0x7f4a20197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:38:01.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.918+0000 7f4a24caf640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a20102ef0 0x7f4a2019bcd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4a1802ae70 tx=0x7f4a1802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:01.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.918+0000 7f4a1d7fa640 1 -- 172.21.15.139:0/1637502916 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18034040 con 0x7f4a20102ef0 2024-09-16T01:38:01.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.918+0000 7f4a1d7fa640 1 -- 172.21.15.139:0/1637502916 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a18030920 con 0x7f4a20102ef0 2024-09-16T01:38:01.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.918+0000 7f4a26712640 1 -- 172.21.15.139:0/1637502916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a2019c640 con 0x7f4a20102ef0 2024-09-16T01:38:01.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.918+0000 7f4a1d7fa640 1 -- 172.21.15.139:0/1637502916 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18045480 con 0x7f4a20102ef0 2024-09-16T01:38:01.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.918+0000 7f4a26712640 1 -- 172.21.15.139:0/1637502916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a201a3f80 con 0x7f4a20102ef0 2024-09-16T01:38:01.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.919+0000 7f4a26712640 1 -- 172.21.15.139:0/1637502916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a20101200 con 0x7f4a20102ef0 2024-09-16T01:38:01.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.921+0000 7f4a1d7fa640 1 -- 172.21.15.139:0/1637502916 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4a18037020 con 0x7f4a20102ef0 2024-09-16T01:38:01.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.921+0000 7f4a1d7fa640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4a080a3240 0x7f4a080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:01.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.921+0000 7f4a1ffff640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4a080a3240 0x7f4a080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:01.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.921+0000 7f4a1d7fa640 1 -- 172.21.15.139:0/1637502916 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7879+0+0 (secure 0 0 0) 0x7f4a180f0860 con 0x7f4a20102ef0 2024-09-16T01:38:01.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.922+0000 7f4a1ffff640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4a080a3240 0x7f4a080a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4a10005fd0 tx=0x7f4a10005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:01.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:01.925+0000 7f4a1d7fa640 1 -- 172.21.15.139:0/1637502916 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a180811b0 con 0x7f4a20102ef0 2024-09-16T01:38:02.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:02.126+0000 7f4a26712640 1 -- 172.21.15.139:0/1637502916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4a201a4150 con 0x7f4a20102ef0 2024-09-16T01:38:02.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:02.511+0000 7f4a1d7fa640 1 -- 172.21.15.139:0/1637502916 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f4a180bd730 con 0x7f4a20102ef0 2024-09-16T01:38:02.513 INFO:teuthology.orchestra.run.smithi139.stderr:new fs with metadata pool 2 and data pool 3 2024-09-16T01:38:02.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:02.513+0000 7f4a26712640 1 -- 172.21.15.139:0/1637502916 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4a080a3240 msgr2=0x7f4a080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:02.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:02.513+0000 7f4a26712640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4a080a3240 0x7f4a080a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4a10005fd0 tx=0x7f4a10005850 comp rx=0 tx=0).stop 2024-09-16T01:38:02.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:02.513+0000 7f4a26712640 1 -- 172.21.15.139:0/1637502916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a20102ef0 msgr2=0x7f4a2019bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:02.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:02.513+0000 7f4a26712640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a20102ef0 0x7f4a2019bcd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4a1802ae70 tx=0x7f4a1802f5d0 comp rx=0 tx=0).stop 2024-09-16T01:38:02.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:02.513+0000 7f4a26712640 1 -- 172.21.15.139:0/1637502916 shutdown_connections 2024-09-16T01:38:02.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:02.513+0000 7f4a26712640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4a080a3240 0x7f4a080a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:02.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:02.513+0000 7f4a26712640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a20102ef0 0x7f4a2019bcd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:02.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:02.513+0000 7f4a26712640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a20102570 0x7f4a20197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:02.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:02.513+0000 7f4a26712640 1 --2- 172.21.15.139:0/1637502916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a20100e20 0x7f4a20197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:02.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:02.513+0000 7f4a26712640 1 -- 172.21.15.139:0/1637502916 >> 172.21.15.139:0/1637502916 conn(0x7f4a200fc7d0 msgr2=0x7f4a20106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:02.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:02.514+0000 7f4a26712640 1 -- 172.21.15.139:0/1637502916 shutdown_connections 2024-09-16T01:38:02.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:02.514+0000 7f4a26712640 1 -- 172.21.15.139:0/1637502916 wait complete. 2024-09-16T01:38:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:02 smithi164 ceph-mon[35121]: pgmap v272: 97 pgs: 24 creating+peering, 55 unknown, 18 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:02 smithi164 ceph-mon[35121]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T01:38:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:02 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1637502916' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T01:38:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:02 smithi164 ceph-mon[35121]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T01:38:02.897 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:02 smithi139 ceph-mon[31175]: pgmap v272: 97 pgs: 24 creating+peering, 55 unknown, 18 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:02.897 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:02 smithi139 ceph-mon[31175]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T01:38:02.897 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:02 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1637502916' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T01:38:02.897 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:02 smithi139 ceph-mon[31175]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T01:38:02.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:02 smithi176 ceph-mon[35324]: pgmap v272: 97 pgs: 24 creating+peering, 55 unknown, 18 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:02.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:02 smithi176 ceph-mon[35324]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T01:38:02.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:02 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1637502916' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T01:38:02.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:02 smithi176 ceph-mon[35324]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T01:38:03.246 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-16T01:38:03.246 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-16T01:38:03.496 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:03.760 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:03 smithi139 ceph-mon[31175]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T01:38:03.760 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:03 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1637502916' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T01:38:03.760 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:03 smithi139 ceph-mon[31175]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T01:38:03.760 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:03 smithi139 ceph-mon[31175]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T01:38:03.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:03 smithi164 ceph-mon[35121]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T01:38:03.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:03 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1637502916' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T01:38:03.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:03 smithi164 ceph-mon[35121]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T01:38:03.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:03 smithi164 ceph-mon[35121]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T01:38:03.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:03 smithi176 ceph-mon[35324]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T01:38:03.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:03 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1637502916' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T01:38:03.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:03 smithi176 ceph-mon[35324]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T01:38:03.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:03 smithi176 ceph-mon[35324]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T01:38:03.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.984+0000 7f9940be4640 1 -- 172.21.15.139:0/4194492164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993c101de0 msgr2=0x7f993c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:03.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.984+0000 7f9940be4640 1 --2- 172.21.15.139:0/4194492164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993c101de0 0x7f993c1021c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f992c009900 tx=0x7f992c02a410 comp rx=0 tx=0).stop 2024-09-16T01:38:03.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.984+0000 7f9940be4640 1 -- 172.21.15.139:0/4194492164 shutdown_connections 2024-09-16T01:38:03.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.984+0000 7f9940be4640 1 --2- 172.21.15.139:0/4194492164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993c106f10 0x7f993c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:03.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.984+0000 7f9940be4640 1 --2- 172.21.15.139:0/4194492164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f993c102790 0x7f993c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:03.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.984+0000 7f9940be4640 1 --2- 172.21.15.139:0/4194492164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993c101de0 0x7f993c1021c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:03.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.984+0000 7f9940be4640 1 -- 172.21.15.139:0/4194492164 >> 172.21.15.139:0/4194492164 conn(0x7f993c0fd8c0 msgr2=0x7f993c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:03.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.984+0000 7f9940be4640 1 -- 172.21.15.139:0/4194492164 shutdown_connections 2024-09-16T01:38:03.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.984+0000 7f9940be4640 1 -- 172.21.15.139:0/4194492164 wait complete. 2024-09-16T01:38:03.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.985+0000 7f9940be4640 1 Processor -- start 2024-09-16T01:38:03.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.985+0000 7f9940be4640 1 -- start start 2024-09-16T01:38:03.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.985+0000 7f9940be4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993c101de0 0x7f993c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:03.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.985+0000 7f9940be4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993c102790 0x7f993c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:03.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.985+0000 7f9940be4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f993c106f10 0x7f993c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:03.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.985+0000 7f9940be4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f993c111f70 con 0x7f993c101de0 2024-09-16T01:38:03.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.985+0000 7f9940be4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f993c111df0 con 0x7f993c102790 2024-09-16T01:38:03.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.985+0000 7f9940be4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f993c1120f0 con 0x7f993c106f10 2024-09-16T01:38:03.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.985+0000 7f993a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993c101de0 0x7f993c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:03.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.985+0000 7f993a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993c101de0 0x7f993c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54844/0 (socket says 172.21.15.139:54844) 2024-09-16T01:38:03.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.985+0000 7f993a59c640 1 -- 172.21.15.139:0/2102213790 learned_addr learned my addr 172.21.15.139:0/2102213790 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.985+0000 7f9939d9b640 1 --2- 172.21.15.139:0/2102213790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993c102790 0x7f993c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.985+0000 7f993ad9d640 1 --2- 172.21.15.139:0/2102213790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f993c106f10 0x7f993c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.986+0000 7f993a59c640 1 -- 172.21.15.139:0/2102213790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f993c106f10 msgr2=0x7f993c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.986+0000 7f993a59c640 1 --2- 172.21.15.139:0/2102213790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f993c106f10 0x7f993c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.986+0000 7f993a59c640 1 -- 172.21.15.139:0/2102213790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993c102790 msgr2=0x7f993c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.986+0000 7f993a59c640 1 --2- 172.21.15.139:0/2102213790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993c102790 0x7f993c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.986+0000 7f993a59c640 1 -- 172.21.15.139:0/2102213790 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c19d380 con 0x7f993c101de0 2024-09-16T01:38:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.986+0000 7f993a59c640 1 --2- 172.21.15.139:0/2102213790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993c101de0 0x7f993c1983d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f992c006010 tx=0x7f992c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.986+0000 7f992b7fe640 1 -- 172.21.15.139:0/2102213790 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c03f030 con 0x7f993c101de0 2024-09-16T01:38:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.986+0000 7f992b7fe640 1 -- 172.21.15.139:0/2102213790 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f992c003ce0 con 0x7f993c101de0 2024-09-16T01:38:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.986+0000 7f9940be4640 1 -- 172.21.15.139:0/2102213790 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993c19d610 con 0x7f993c101de0 2024-09-16T01:38:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.986+0000 7f992b7fe640 1 -- 172.21.15.139:0/2102213790 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c035070 con 0x7f993c101de0 2024-09-16T01:38:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.986+0000 7f9940be4640 1 -- 172.21.15.139:0/2102213790 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993c1a4f50 con 0x7f993c101de0 2024-09-16T01:38:03.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.987+0000 7f9940be4640 1 -- 172.21.15.139:0/2102213790 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f993c1032c0 con 0x7f993c101de0 2024-09-16T01:38:03.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.988+0000 7f992b7fe640 1 -- 172.21.15.139:0/2102213790 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f992c042030 con 0x7f993c101de0 2024-09-16T01:38:03.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.989+0000 7f992b7fe640 1 --2- 172.21.15.139:0/2102213790 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f99240a3240 0x7f99240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:03.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.989+0000 7f9939d9b640 1 --2- 172.21.15.139:0/2102213790 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f99240a3240 0x7f99240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:03.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.989+0000 7f992b7fe640 1 -- 172.21.15.139:0/2102213790 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f992c0efdd0 con 0x7f993c101de0 2024-09-16T01:38:03.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.989+0000 7f9939d9b640 1 --2- 172.21.15.139:0/2102213790 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f99240a3240 0x7f99240a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f993c0fdce0 tx=0x7f993000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:03.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:03.993+0000 7f992b7fe640 1 -- 172.21.15.139:0/2102213790 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f992c0bc6b0 con 0x7f993c101de0 2024-09-16T01:38:04.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:04.169+0000 7f9940be4640 1 -- 172.21.15.139:0/2102213790 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f993c10a0f0 con 0x7f993c101de0 2024-09-16T01:38:04.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:04.528+0000 7f992b7fe640 1 -- 172.21.15.139:0/2102213790 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v78) ==== 157+0+0 (secure 0 0 0) 0x7f992c0c09b0 con 0x7f993c101de0 2024-09-16T01:38:04.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:04.531+0000 7f9940be4640 1 -- 172.21.15.139:0/2102213790 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f99240a3240 msgr2=0x7f99240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:04.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:04.531+0000 7f9940be4640 1 --2- 172.21.15.139:0/2102213790 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f99240a3240 0x7f99240a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f993c0fdce0 tx=0x7f993000b390 comp rx=0 tx=0).stop 2024-09-16T01:38:04.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:04.531+0000 7f9940be4640 1 -- 172.21.15.139:0/2102213790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993c101de0 msgr2=0x7f993c1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:04.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:04.531+0000 7f9940be4640 1 --2- 172.21.15.139:0/2102213790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993c101de0 0x7f993c1983d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f992c006010 tx=0x7f992c0042f0 comp rx=0 tx=0).stop 2024-09-16T01:38:04.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:04.531+0000 7f9940be4640 1 -- 172.21.15.139:0/2102213790 shutdown_connections 2024-09-16T01:38:04.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:04.531+0000 7f9940be4640 1 --2- 172.21.15.139:0/2102213790 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f99240a3240 0x7f99240a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:04.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:04.531+0000 7f9940be4640 1 --2- 172.21.15.139:0/2102213790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f993c106f10 0x7f993c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:04.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:04.531+0000 7f9940be4640 1 --2- 172.21.15.139:0/2102213790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f993c102790 0x7f993c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:04.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:04.531+0000 7f9940be4640 1 --2- 172.21.15.139:0/2102213790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993c101de0 0x7f993c1983d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:04.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:04.531+0000 7f9940be4640 1 -- 172.21.15.139:0/2102213790 >> 172.21.15.139:0/2102213790 conn(0x7f993c0fd8c0 msgr2=0x7f993c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:04.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:04.531+0000 7f9940be4640 1 -- 172.21.15.139:0/2102213790 shutdown_connections 2024-09-16T01:38:04.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:04.531+0000 7f9940be4640 1 -- 172.21.15.139:0/2102213790 wait complete. 2024-09-16T01:38:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:04 smithi164 ceph-mon[35121]: pgmap v275: 97 pgs: 28 creating+peering, 31 unknown, 38 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-16T01:38:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:04 smithi164 ceph-mon[35121]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T01:38:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:04 smithi164 ceph-mon[35121]: Cluster is now healthy 2024-09-16T01:38:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:04 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] up:active 2024-09-16T01:38:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:04 smithi164 ceph-mon[35121]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:04 smithi164 ceph-mon[35121]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:04 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2102213790' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T01:38:04.915 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:04 smithi139 ceph-mon[31175]: pgmap v275: 97 pgs: 28 creating+peering, 31 unknown, 38 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-16T01:38:04.916 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:04 smithi139 ceph-mon[31175]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T01:38:04.916 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:04 smithi139 ceph-mon[31175]: Cluster is now healthy 2024-09-16T01:38:04.916 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:04 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] up:active 2024-09-16T01:38:04.916 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:04 smithi139 ceph-mon[31175]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:04.916 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:04 smithi139 ceph-mon[31175]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:04.916 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:04 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2102213790' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T01:38:04.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:04 smithi176 ceph-mon[35324]: pgmap v275: 97 pgs: 28 creating+peering, 31 unknown, 38 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-16T01:38:04.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:04 smithi176 ceph-mon[35324]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T01:38:04.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:04 smithi176 ceph-mon[35324]: Cluster is now healthy 2024-09-16T01:38:04.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:04 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] up:active 2024-09-16T01:38:04.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:04 smithi176 ceph-mon[35324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:04.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:04 smithi176 ceph-mon[35324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:04.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:04 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2102213790' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T01:38:05.276 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T01:38:05.529 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:05.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:05 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2102213790' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T01:38:05.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:05 smithi164 ceph-mon[35121]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T01:38:05.800 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:05 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2102213790' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T01:38:05.800 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:05 smithi139 ceph-mon[31175]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T01:38:05.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:05 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2102213790' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T01:38:05.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:05 smithi176 ceph-mon[35324]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T01:38:06.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.005+0000 7f9926172640 1 -- 172.21.15.139:0/179500643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9920101420 msgr2=0x7f992010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:06.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.005+0000 7f9926172640 1 --2- 172.21.15.139:0/179500643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9920101420 0x7f992010d950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9918009cd0 tx=0x7f991802a8e0 comp rx=0 tx=0).stop 2024-09-16T01:38:06.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.005+0000 7f9926172640 1 -- 172.21.15.139:0/179500643 shutdown_connections 2024-09-16T01:38:06.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.005+0000 7f9926172640 1 --2- 172.21.15.139:0/179500643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9920101420 0x7f992010d950 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:06.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.005+0000 7f9926172640 1 --2- 172.21.15.139:0/179500643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9920100a80 0x7f9920100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:06.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.005+0000 7f9926172640 1 --2- 172.21.15.139:0/179500643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9920106a80 0x7f9920106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:06.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.005+0000 7f9926172640 1 -- 172.21.15.139:0/179500643 >> 172.21.15.139:0/179500643 conn(0x7f99200fc7a0 msgr2=0x7f99200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:06.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.005+0000 7f9926172640 1 -- 172.21.15.139:0/179500643 shutdown_connections 2024-09-16T01:38:06.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.005+0000 7f9926172640 1 -- 172.21.15.139:0/179500643 wait complete. 2024-09-16T01:38:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f9926172640 1 Processor -- start 2024-09-16T01:38:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f9926172640 1 -- start start 2024-09-16T01:38:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f9926172640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9920100a80 0x7f9920198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f9926172640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9920101420 0x7f99201988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f9926172640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9920106a80 0x7f992019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f9926172640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9920111fa0 con 0x7f9920106a80 2024-09-16T01:38:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f9926172640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9920111e20 con 0x7f9920101420 2024-09-16T01:38:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f9926172640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9920112120 con 0x7f9920100a80 2024-09-16T01:38:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f991ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9920106a80 0x7f992019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f991ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9920106a80 0x7f992019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54862/0 (socket says 172.21.15.139:54862) 2024-09-16T01:38:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f991ffff640 1 -- 172.21.15.139:0/1357590668 learned_addr learned my addr 172.21.15.139:0/1357590668 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f9917fff640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9920101420 0x7f99201988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f991f7fe640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9920100a80 0x7f9920198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f991ffff640 1 -- 172.21.15.139:0/1357590668 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9920100a80 msgr2=0x7f9920198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f991ffff640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9920100a80 0x7f9920198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f991ffff640 1 -- 172.21.15.139:0/1357590668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9920101420 msgr2=0x7f99201988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f991ffff640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9920101420 0x7f99201988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f991ffff640 1 -- 172.21.15.139:0/1357590668 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992019d340 con 0x7f9920106a80 2024-09-16T01:38:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f9917fff640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9920101420 0x7f99201988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:38:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.006+0000 7f991ffff640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9920106a80 0x7f992019cc60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9918006010 tx=0x7f9918003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.007+0000 7f991d7fa640 1 -- 172.21.15.139:0/1357590668 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918004450 con 0x7f9920106a80 2024-09-16T01:38:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.007+0000 7f9926172640 1 -- 172.21.15.139:0/1357590668 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992019d5d0 con 0x7f9920106a80 2024-09-16T01:38:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.007+0000 7f991d7fa640 1 -- 172.21.15.139:0/1357590668 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9918003e10 con 0x7f9920106a80 2024-09-16T01:38:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.007+0000 7f991d7fa640 1 -- 172.21.15.139:0/1357590668 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99180466e0 con 0x7f9920106a80 2024-09-16T01:38:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.007+0000 7f9926172640 1 -- 172.21.15.139:0/1357590668 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99201a4f10 con 0x7f9920106a80 2024-09-16T01:38:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.008+0000 7f9926172640 1 -- 172.21.15.139:0/1357590668 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9920100ee0 con 0x7f9920106a80 2024-09-16T01:38:06.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.009+0000 7f991d7fa640 1 -- 172.21.15.139:0/1357590668 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f99180305a0 con 0x7f9920106a80 2024-09-16T01:38:06.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.009+0000 7f991d7fa640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f990c0a3240 0x7f990c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:06.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.010+0000 7f991f7fe640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f990c0a3240 0x7f990c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:06.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.010+0000 7f991d7fa640 1 -- 172.21.15.139:0/1357590668 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8277+0+0 (secure 0 0 0) 0x7f991802e030 con 0x7f9920106a80 2024-09-16T01:38:06.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.010+0000 7f991f7fe640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f990c0a3240 0x7f990c0a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9910005d70 tx=0x7f9910005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:06.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.013+0000 7f991d7fa640 1 -- 172.21.15.139:0/1357590668 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f991804b060 con 0x7f9920106a80 2024-09-16T01:38:06.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:06.179+0000 7f9926172640 1 -- 172.21.15.139:0/1357590668 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f99201a50e0 con 0x7f9920106a80 2024-09-16T01:38:06.776 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:38:06 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b[64648]: 2024-09-16T01:38:06.380+0000 7f7d40e8c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T01:38:07.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:06 smithi164 ceph-mon[35121]: pgmap v277: 97 pgs: 25 creating+peering, 8 unknown, 64 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-09-16T01:38:07.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:06 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1357590668' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T01:38:07.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:06 smithi139 ceph-mon[31175]: pgmap v277: 97 pgs: 25 creating+peering, 8 unknown, 64 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-09-16T01:38:07.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:06 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1357590668' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T01:38:07.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:06 smithi176 ceph-mon[35324]: pgmap v277: 97 pgs: 25 creating+peering, 8 unknown, 64 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-09-16T01:38:07.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:06 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1357590668' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T01:38:07.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:07.538+0000 7f991d7fa640 1 -- 172.21.15.139:0/1357590668 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v80) ==== 228+0+0 (secure 0 0 0) 0x7f9918044380 con 0x7f9920106a80 2024-09-16T01:38:07.540 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data_ec' created 2024-09-16T01:38:07.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:07.541+0000 7f9926172640 1 -- 172.21.15.139:0/1357590668 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f990c0a3240 msgr2=0x7f990c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:07.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:07.541+0000 7f9926172640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f990c0a3240 0x7f990c0a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9910005d70 tx=0x7f9910005d00 comp rx=0 tx=0).stop 2024-09-16T01:38:07.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:07.541+0000 7f9926172640 1 -- 172.21.15.139:0/1357590668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9920106a80 msgr2=0x7f992019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:07.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:07.541+0000 7f9926172640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9920106a80 0x7f992019cc60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9918006010 tx=0x7f9918003c20 comp rx=0 tx=0).stop 2024-09-16T01:38:07.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:07.542+0000 7f9926172640 1 -- 172.21.15.139:0/1357590668 shutdown_connections 2024-09-16T01:38:07.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:07.542+0000 7f9926172640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f990c0a3240 0x7f990c0a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:07.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:07.542+0000 7f9926172640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9920106a80 0x7f992019cc60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:07.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:07.542+0000 7f9926172640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9920101420 0x7f99201988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:07.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:07.542+0000 7f9926172640 1 --2- 172.21.15.139:0/1357590668 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9920100a80 0x7f9920198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:07.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:07.542+0000 7f9926172640 1 -- 172.21.15.139:0/1357590668 >> 172.21.15.139:0/1357590668 conn(0x7f99200fc7a0 msgr2=0x7f99200fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:07.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:07.542+0000 7f9926172640 1 -- 172.21.15.139:0/1357590668 shutdown_connections 2024-09-16T01:38:07.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:07.542+0000 7f9926172640 1 -- 172.21.15.139:0/1357590668 wait complete. 2024-09-16T01:38:07.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T01:38:07.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-16T01:38:07.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-16T01:38:07.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-16T01:38:07.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-16T01:38:07.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-16T01:38:07.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-16T01:38:07.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-16T01:38:07.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-16T01:38:07.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-09-16T01:38:07.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-16T01:38:07.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-16T01:38:07.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1357590668' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T01:38:07.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-16T01:38:07.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-16T01:38:07.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-16T01:38:07.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-16T01:38:07.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-16T01:38:07.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-16T01:38:07.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-16T01:38:07.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-16T01:38:07.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-09-16T01:38:07.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-16T01:38:07.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:07 smithi139 ceph-mon[31175]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T01:38:08.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T01:38:08.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-16T01:38:08.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-16T01:38:08.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-16T01:38:08.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-16T01:38:08.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-16T01:38:08.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-16T01:38:08.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-16T01:38:08.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-16T01:38:08.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-09-16T01:38:08.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-16T01:38:08.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-16T01:38:08.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1357590668' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T01:38:08.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-16T01:38:08.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-16T01:38:08.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-16T01:38:08.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-16T01:38:08.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-16T01:38:08.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-16T01:38:08.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-16T01:38:08.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-16T01:38:08.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-09-16T01:38:08.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-16T01:38:08.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:07 smithi164 ceph-mon[35121]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T01:38:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T01:38:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-16T01:38:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-16T01:38:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-16T01:38:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-16T01:38:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-16T01:38:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-16T01:38:08.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-16T01:38:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-16T01:38:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-09-16T01:38:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-16T01:38:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-16T01:38:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1357590668' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T01:38:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-16T01:38:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-16T01:38:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-16T01:38:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-16T01:38:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-16T01:38:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-16T01:38:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-16T01:38:08.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-16T01:38:08.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-09-16T01:38:08.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-16T01:38:08.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:07 smithi176 ceph-mon[35324]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T01:38:08.251 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-16T01:38:08.497 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:09.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.011+0000 7f73fe374640 1 -- 172.21.15.139:0/3677136368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73f8102d40 msgr2=0x7f73f8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:09.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.011+0000 7f73fe374640 1 --2- 172.21.15.139:0/3677136368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73f8102d40 0x7f73f8103120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009900 tx=0x7f73e802a480 comp rx=0 tx=0).stop 2024-09-16T01:38:09.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.011+0000 7f73fe374640 1 -- 172.21.15.139:0/3677136368 shutdown_connections 2024-09-16T01:38:09.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.011+0000 7f73fe374640 1 --2- 172.21.15.139:0/3677136368 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73f810c170 0x7f73f810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:09.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.011+0000 7f73fe374640 1 --2- 172.21.15.139:0/3677136368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73f8103660 0x7f73f810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:09.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.011+0000 7f73fe374640 1 --2- 172.21.15.139:0/3677136368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73f8102d40 0x7f73f8103120 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:09.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.011+0000 7f73fe374640 1 -- 172.21.15.139:0/3677136368 >> 172.21.15.139:0/3677136368 conn(0x7f73f80fc7a0 msgr2=0x7f73f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:09.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.011+0000 7f73fe374640 1 -- 172.21.15.139:0/3677136368 shutdown_connections 2024-09-16T01:38:09.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.011+0000 7f73fe374640 1 -- 172.21.15.139:0/3677136368 wait complete. 2024-09-16T01:38:09.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.012+0000 7f73fe374640 1 Processor -- start 2024-09-16T01:38:09.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.012+0000 7f73fe374640 1 -- start start 2024-09-16T01:38:09.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.012+0000 7f73fe374640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73f8102d40 0x7f73f81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:09.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.012+0000 7f73fe374640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73f8103660 0x7f73f8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:09.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.012+0000 7f73fe374640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73f810c170 0x7f73f8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:09.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.012+0000 7f73fe374640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73f8111fb0 con 0x7f73f8102d40 2024-09-16T01:38:09.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.012+0000 7f73fe374640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73f8111e30 con 0x7f73f810c170 2024-09-16T01:38:09.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.012+0000 7f73fe374640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f73f8112130 con 0x7f73f8103660 2024-09-16T01:38:09.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.012+0000 7f73f7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73f8102d40 0x7f73f81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:09.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.012+0000 7f73f7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73f8102d40 0x7f73f81009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54880/0 (socket says 172.21.15.139:54880) 2024-09-16T01:38:09.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.012+0000 7f73f7fff640 1 -- 172.21.15.139:0/1369680071 learned_addr learned my addr 172.21.15.139:0/1369680071 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:09.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.013+0000 7f73fc911640 1 --2- 172.21.15.139:0/1369680071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73f810c170 0x7f73f8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:09.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.013+0000 7f73f77fe640 1 --2- 172.21.15.139:0/1369680071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73f8103660 0x7f73f8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:09.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.013+0000 7f73f7fff640 1 -- 172.21.15.139:0/1369680071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73f8103660 msgr2=0x7f73f8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:09.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.013+0000 7f73f7fff640 1 --2- 172.21.15.139:0/1369680071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73f8103660 0x7f73f8100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.013+0000 7f73f7fff640 1 -- 172.21.15.139:0/1369680071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73f810c170 msgr2=0x7f73f8101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.013+0000 7f73f7fff640 1 --2- 172.21.15.139:0/1369680071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73f810c170 0x7f73f8101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.013+0000 7f73f7fff640 1 -- 172.21.15.139:0/1369680071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f81a6930 con 0x7f73f8102d40 2024-09-16T01:38:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.013+0000 7f73f7fff640 1 --2- 172.21.15.139:0/1369680071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73f8102d40 0x7f73f81009a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f73e8006010 tx=0x7f73e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.013+0000 7f73f57fa640 1 -- 172.21.15.139:0/1369680071 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e803f030 con 0x7f73f8102d40 2024-09-16T01:38:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.013+0000 7f73f57fa640 1 -- 172.21.15.139:0/1369680071 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e8003ce0 con 0x7f73f8102d40 2024-09-16T01:38:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.013+0000 7f73fe374640 1 -- 172.21.15.139:0/1369680071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f81a6bc0 con 0x7f73f8102d40 2024-09-16T01:38:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.013+0000 7f73f57fa640 1 -- 172.21.15.139:0/1369680071 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8035070 con 0x7f73f8102d40 2024-09-16T01:38:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.013+0000 7f73fe374640 1 -- 172.21.15.139:0/1369680071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f81a70a0 con 0x7f73f8102d40 2024-09-16T01:38:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.014+0000 7f73fe374640 1 -- 172.21.15.139:0/1369680071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f8103120 con 0x7f73f8102d40 2024-09-16T01:38:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.015+0000 7f73f57fa640 1 -- 172.21.15.139:0/1369680071 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f73e8042030 con 0x7f73f8102d40 2024-09-16T01:38:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.016+0000 7f73f57fa640 1 --2- 172.21.15.139:0/1369680071 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f73e00a3240 0x7f73e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.016+0000 7f73f77fe640 1 --2- 172.21.15.139:0/1369680071 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f73e00a3240 0x7f73e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.016+0000 7f73f57fa640 1 -- 172.21.15.139:0/1369680071 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 9060+0+0 (secure 0 0 0) 0x7f73e80f0560 con 0x7f73f8102d40 2024-09-16T01:38:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.017+0000 7f73f77fe640 1 --2- 172.21.15.139:0/1369680071 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f73e00a3240 0x7f73e00a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f73f8102430 tx=0x7f73ec005eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.020+0000 7f73f57fa640 1 -- 172.21.15.139:0/1369680071 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73e80bd000 con 0x7f73f8102d40 2024-09-16T01:38:09.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.183+0000 7f73fe374640 1 -- 172.21.15.139:0/1369680071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f73f8106c80 con 0x7f73f8102d40 2024-09-16T01:38:09.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.545+0000 7f73f57fa640 1 -- 172.21.15.139:0/1369680071 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v82) ==== 169+0+0 (secure 0 0 0) 0x7f73e80bc900 con 0x7f73f8102d40 2024-09-16T01:38:09.548 INFO:teuthology.orchestra.run.smithi139.stderr:set pool 4 allow_ec_overwrites to true 2024-09-16T01:38:09.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.548+0000 7f73fe374640 1 -- 172.21.15.139:0/1369680071 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f73e00a3240 msgr2=0x7f73e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:09.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.548+0000 7f73fe374640 1 --2- 172.21.15.139:0/1369680071 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f73e00a3240 0x7f73e00a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f73f8102430 tx=0x7f73ec005eb0 comp rx=0 tx=0).stop 2024-09-16T01:38:09.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.548+0000 7f73fe374640 1 -- 172.21.15.139:0/1369680071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73f8102d40 msgr2=0x7f73f81009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:09.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.548+0000 7f73fe374640 1 --2- 172.21.15.139:0/1369680071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73f8102d40 0x7f73f81009a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f73e8006010 tx=0x7f73e80042f0 comp rx=0 tx=0).stop 2024-09-16T01:38:09.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.548+0000 7f73fe374640 1 -- 172.21.15.139:0/1369680071 shutdown_connections 2024-09-16T01:38:09.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.548+0000 7f73fe374640 1 --2- 172.21.15.139:0/1369680071 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f73e00a3240 0x7f73e00a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:09.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.548+0000 7f73fe374640 1 --2- 172.21.15.139:0/1369680071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73f810c170 0x7f73f8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:09.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.548+0000 7f73fe374640 1 --2- 172.21.15.139:0/1369680071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73f8103660 0x7f73f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:09.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.548+0000 7f73fe374640 1 --2- 172.21.15.139:0/1369680071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73f8102d40 0x7f73f81009a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:09.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.548+0000 7f73fe374640 1 -- 172.21.15.139:0/1369680071 >> 172.21.15.139:0/1369680071 conn(0x7f73f80fc7a0 msgr2=0x7f73f80fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:09.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.548+0000 7f73fe374640 1 -- 172.21.15.139:0/1369680071 shutdown_connections 2024-09-16T01:38:09.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:09.548+0000 7f73fe374640 1 -- 172.21.15.139:0/1369680071 wait complete. 2024-09-16T01:38:09.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:09 smithi139 ceph-mon[31175]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T01:38:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:09 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1369680071' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T01:38:09.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:09 smithi176 ceph-mon[35324]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T01:38:09.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:09 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1369680071' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T01:38:10.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:09 smithi164 ceph-mon[35121]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T01:38:10.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:09 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1369680071' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T01:38:10.280 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-16T01:38:10.533 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:10.812 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:10 smithi139 ceph-mon[31175]: pgmap v282: 129 pgs: 4 creating+peering, 2 peering, 25 unknown, 98 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 627 B/s wr, 6 op/s 2024-09-16T01:38:10.812 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:10 smithi139 ceph-mon[31175]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T01:38:10.812 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:10 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1369680071' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T01:38:10.812 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:10 smithi139 ceph-mon[31175]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T01:38:10.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:10 smithi176 ceph-mon[35324]: pgmap v282: 129 pgs: 4 creating+peering, 2 peering, 25 unknown, 98 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 627 B/s wr, 6 op/s 2024-09-16T01:38:10.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:10 smithi176 ceph-mon[35324]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T01:38:10.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:10 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1369680071' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T01:38:10.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:10 smithi176 ceph-mon[35324]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T01:38:11.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.998+0000 7f120e1e7640 1 -- 172.21.15.139:0/3145596533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1208101de0 msgr2=0x7f12081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:11.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.998+0000 7f120e1e7640 1 --2- 172.21.15.139:0/3145596533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1208101de0 0x7f12081021c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009900 tx=0x7f11fc02a410 comp rx=0 tx=0).stop 2024-09-16T01:38:11.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.998+0000 7f120e1e7640 1 -- 172.21.15.139:0/3145596533 shutdown_connections 2024-09-16T01:38:11.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.998+0000 7f120e1e7640 1 --2- 172.21.15.139:0/3145596533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1208106f10 0x7f120810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.998+0000 7f120e1e7640 1 --2- 172.21.15.139:0/3145596533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1208102790 0x7f12081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.998+0000 7f120e1e7640 1 --2- 172.21.15.139:0/3145596533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1208101de0 0x7f12081021c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.998+0000 7f120e1e7640 1 -- 172.21.15.139:0/3145596533 >> 172.21.15.139:0/3145596533 conn(0x7f12080fd8c0 msgr2=0x7f12080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.998+0000 7f120e1e7640 1 -- 172.21.15.139:0/3145596533 shutdown_connections 2024-09-16T01:38:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.998+0000 7f120e1e7640 1 -- 172.21.15.139:0/3145596533 wait complete. 2024-09-16T01:38:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.999+0000 7f120e1e7640 1 Processor -- start 2024-09-16T01:38:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.999+0000 7f120e1e7640 1 -- start start 2024-09-16T01:38:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.999+0000 7f120e1e7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1208101de0 0x7f12080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.999+0000 7f120e1e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1208102790 0x7f1208074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.999+0000 7f120e1e7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1208106f10 0x7f1208074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.999+0000 7f120e1e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1208111f80 con 0x7f1208102790 2024-09-16T01:38:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.999+0000 7f120e1e7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1208111e00 con 0x7f1208101de0 2024-09-16T01:38:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.999+0000 7f120e1e7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1208112100 con 0x7f1208106f10 2024-09-16T01:38:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.999+0000 7f1206ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1208102790 0x7f1208074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:11.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.999+0000 7f1206ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1208102790 0x7f1208074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54896/0 (socket says 172.21.15.139:54896) 2024-09-16T01:38:11.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:10.999+0000 7f1206ffd640 1 -- 172.21.15.139:0/551884507 learned_addr learned my addr 172.21.15.139:0/551884507 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:11.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.000+0000 7f12077fe640 1 --2- 172.21.15.139:0/551884507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1208101de0 0x7f12080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:11.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.000+0000 7f1207fff640 1 --2- 172.21.15.139:0/551884507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1208106f10 0x7f1208074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:11.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.000+0000 7f1206ffd640 1 -- 172.21.15.139:0/551884507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1208106f10 msgr2=0x7f1208074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:11.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.000+0000 7f1206ffd640 1 --2- 172.21.15.139:0/551884507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1208106f10 0x7f1208074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:11.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.000+0000 7f1206ffd640 1 -- 172.21.15.139:0/551884507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1208101de0 msgr2=0x7f12080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:11.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.000+0000 7f1206ffd640 1 --2- 172.21.15.139:0/551884507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1208101de0 0x7f12080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:11.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.000+0000 7f1206ffd640 1 -- 172.21.15.139:0/551884507 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12081a68d0 con 0x7f1208102790 2024-09-16T01:38:11.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.000+0000 7f1206ffd640 1 --2- 172.21.15.139:0/551884507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1208102790 0x7f1208074a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f11f800c920 tx=0x7f11f800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:11.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.000+0000 7f1204ff9640 1 -- 172.21.15.139:0/551884507 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8007c70 con 0x7f1208102790 2024-09-16T01:38:11.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.000+0000 7f1204ff9640 1 -- 172.21.15.139:0/551884507 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f8007e10 con 0x7f1208102790 2024-09-16T01:38:11.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.000+0000 7f120e1e7640 1 -- 172.21.15.139:0/551884507 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12081a6bc0 con 0x7f1208102790 2024-09-16T01:38:11.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.000+0000 7f1204ff9640 1 -- 172.21.15.139:0/551884507 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8018890 con 0x7f1208102790 2024-09-16T01:38:11.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.000+0000 7f120e1e7640 1 -- 172.21.15.139:0/551884507 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12081a7100 con 0x7f1208102790 2024-09-16T01:38:11.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.001+0000 7f120e1e7640 1 -- 172.21.15.139:0/551884507 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12081021c0 con 0x7f1208102790 2024-09-16T01:38:11.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.002+0000 7f1204ff9640 1 -- 172.21.15.139:0/551884507 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f11f8018a70 con 0x7f1208102790 2024-09-16T01:38:11.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.003+0000 7f1204ff9640 1 --2- 172.21.15.139:0/551884507 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f11f00a3240 0x7f11f00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:11.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.003+0000 7f12077fe640 1 --2- 172.21.15.139:0/551884507 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f11f00a3240 0x7f11f00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:11.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.003+0000 7f1204ff9640 1 -- 172.21.15.139:0/551884507 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9060+0+0 (secure 0 0 0) 0x7f11f80cfe50 con 0x7f1208102790 2024-09-16T01:38:11.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.003+0000 7f12077fe640 1 --2- 172.21.15.139:0/551884507 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f11f00a3240 0x7f11f00a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f11fc000c00 tx=0x7f11fc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:11.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.007+0000 7f1204ff9640 1 -- 172.21.15.139:0/551884507 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11f8014030 con 0x7f1208102790 2024-09-16T01:38:11.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:10 smithi164 ceph-mon[35121]: pgmap v282: 129 pgs: 4 creating+peering, 2 peering, 25 unknown, 98 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 627 B/s wr, 6 op/s 2024-09-16T01:38:11.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:10 smithi164 ceph-mon[35121]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T01:38:11.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:10 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1369680071' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T01:38:11.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:10 smithi164 ceph-mon[35121]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T01:38:11.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.209+0000 7f120e1e7640 1 -- 172.21.15.139:0/551884507 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f120802d7c0 con 0x7f1208102790 2024-09-16T01:38:11.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.553+0000 7f1204ff9640 1 -- 172.21.15.139:0/551884507 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f11f800de20 con 0x7f1208102790 2024-09-16T01:38:11.556 INFO:teuthology.orchestra.run.smithi139.stderr:added data pool 4 to fsmap 2024-09-16T01:38:11.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.556+0000 7f120e1e7640 1 -- 172.21.15.139:0/551884507 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f11f00a3240 msgr2=0x7f11f00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:11.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.556+0000 7f120e1e7640 1 --2- 172.21.15.139:0/551884507 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f11f00a3240 0x7f11f00a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f11fc000c00 tx=0x7f11fc005a20 comp rx=0 tx=0).stop 2024-09-16T01:38:11.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.556+0000 7f120e1e7640 1 -- 172.21.15.139:0/551884507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1208102790 msgr2=0x7f1208074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:11.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.556+0000 7f120e1e7640 1 --2- 172.21.15.139:0/551884507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1208102790 0x7f1208074a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f11f800c920 tx=0x7f11f800cde0 comp rx=0 tx=0).stop 2024-09-16T01:38:11.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.556+0000 7f120e1e7640 1 -- 172.21.15.139:0/551884507 shutdown_connections 2024-09-16T01:38:11.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.556+0000 7f120e1e7640 1 --2- 172.21.15.139:0/551884507 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f11f00a3240 0x7f11f00a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:11.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.556+0000 7f120e1e7640 1 --2- 172.21.15.139:0/551884507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1208106f10 0x7f1208074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:11.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.556+0000 7f120e1e7640 1 --2- 172.21.15.139:0/551884507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1208102790 0x7f1208074a30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:11.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.556+0000 7f120e1e7640 1 --2- 172.21.15.139:0/551884507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1208101de0 0x7f12080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:11.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.556+0000 7f120e1e7640 1 -- 172.21.15.139:0/551884507 >> 172.21.15.139:0/551884507 conn(0x7f12080fd8c0 msgr2=0x7f12080feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:11.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.556+0000 7f120e1e7640 1 -- 172.21.15.139:0/551884507 shutdown_connections 2024-09-16T01:38:11.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:11.556+0000 7f120e1e7640 1 -- 172.21.15.139:0/551884507 wait complete. 2024-09-16T01:38:11.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:11 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/551884507' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T01:38:11.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:11 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/551884507' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T01:38:12.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:11 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/551884507' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T01:38:12.242 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:38:12.487 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:12.910 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:12 smithi139 ceph-mon[31175]: pgmap v284: 129 pgs: 4 creating+peering, 2 peering, 4 unknown, 119 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-16T01:38:12.910 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:12 smithi139 ceph-mon[31175]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T01:38:12.911 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:12 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/551884507' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T01:38:12.911 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:12 smithi139 ceph-mon[31175]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:12.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:12 smithi176 ceph-mon[35324]: pgmap v284: 129 pgs: 4 creating+peering, 2 peering, 4 unknown, 119 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-16T01:38:12.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:12 smithi176 ceph-mon[35324]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T01:38:12.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:12 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/551884507' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T01:38:12.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:12 smithi176 ceph-mon[35324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:13.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.986+0000 7f193c2ec640 1 -- 172.21.15.139:0/2009116261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1934106f10 msgr2=0x7f193410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:13.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.986+0000 7f193c2ec640 1 --2- 172.21.15.139:0/2009116261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1934106f10 0x7f193410ea90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f192c009ca0 tx=0x7f192c02a8c0 comp rx=0 tx=0).stop 2024-09-16T01:38:13.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.986+0000 7f193c2ec640 1 -- 172.21.15.139:0/2009116261 shutdown_connections 2024-09-16T01:38:13.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.986+0000 7f193c2ec640 1 --2- 172.21.15.139:0/2009116261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1934106f10 0x7f193410ea90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:13.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.986+0000 7f193c2ec640 1 --2- 172.21.15.139:0/2009116261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1934102790 0x7f19341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:13.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.986+0000 7f193c2ec640 1 --2- 172.21.15.139:0/2009116261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1934101de0 0x7f19341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:13.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.986+0000 7f193c2ec640 1 -- 172.21.15.139:0/2009116261 >> 172.21.15.139:0/2009116261 conn(0x7f19340fd8c0 msgr2=0x7f19340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:13.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.986+0000 7f193c2ec640 1 -- 172.21.15.139:0/2009116261 shutdown_connections 2024-09-16T01:38:13.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.986+0000 7f193c2ec640 1 -- 172.21.15.139:0/2009116261 wait complete. 2024-09-16T01:38:13.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.987+0000 7f193c2ec640 1 Processor -- start 2024-09-16T01:38:13.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.987+0000 7f193c2ec640 1 -- start start 2024-09-16T01:38:13.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.987+0000 7f193c2ec640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1934101de0 0x7f19341984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:13.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.987+0000 7f193c2ec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1934102790 0x7f19341989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:13.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.987+0000 7f193c2ec640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1934106f10 0x7f193419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:13.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.987+0000 7f193c2ec640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1934111fb0 con 0x7f1934102790 2024-09-16T01:38:13.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.987+0000 7f193c2ec640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1934111e30 con 0x7f1934101de0 2024-09-16T01:38:13.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.987+0000 7f193c2ec640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1934112130 con 0x7f1934106f10 2024-09-16T01:38:13.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.987+0000 7f1939887640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1934102790 0x7f19341989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.987+0000 7f1939887640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1934102790 0x7f19341989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57428/0 (socket says 172.21.15.139:57428) 2024-09-16T01:38:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.987+0000 7f1939887640 1 -- 172.21.15.139:0/3072937906 learned_addr learned my addr 172.21.15.139:0/3072937906 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.987+0000 7f193a088640 1 --2- 172.21.15.139:0/3072937906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1934101de0 0x7f19341984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.987+0000 7f193a889640 1 --2- 172.21.15.139:0/3072937906 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1934106f10 0x7f193419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.988+0000 7f1939887640 1 -- 172.21.15.139:0/3072937906 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1934106f10 msgr2=0x7f193419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.988+0000 7f1939887640 1 --2- 172.21.15.139:0/3072937906 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1934106f10 0x7f193419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.988+0000 7f1939887640 1 -- 172.21.15.139:0/3072937906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1934101de0 msgr2=0x7f19341984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.988+0000 7f1939887640 1 --2- 172.21.15.139:0/3072937906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1934101de0 0x7f19341984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.988+0000 7f1939887640 1 -- 172.21.15.139:0/3072937906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193419d460 con 0x7f1934102790 2024-09-16T01:38:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.988+0000 7f1939887640 1 --2- 172.21.15.139:0/3072937906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1934102790 0x7f19341989f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f192400b4a0 tx=0x7f192400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.988+0000 7f192b7fe640 1 -- 172.21.15.139:0/3072937906 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924014070 con 0x7f1934102790 2024-09-16T01:38:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.988+0000 7f193c2ec640 1 -- 172.21.15.139:0/3072937906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193419d750 con 0x7f1934102790 2024-09-16T01:38:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.988+0000 7f192b7fe640 1 -- 172.21.15.139:0/3072937906 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1924010af0 con 0x7f1934102790 2024-09-16T01:38:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.988+0000 7f192b7fe640 1 -- 172.21.15.139:0/3072937906 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924017d10 con 0x7f1934102790 2024-09-16T01:38:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.988+0000 7f193c2ec640 1 -- 172.21.15.139:0/3072937906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19341a5030 con 0x7f1934102790 2024-09-16T01:38:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.989+0000 7f193c2ec640 1 -- 172.21.15.139:0/3072937906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19341021c0 con 0x7f1934102790 2024-09-16T01:38:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.990+0000 7f192b7fe640 1 -- 172.21.15.139:0/3072937906 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f1924017070 con 0x7f1934102790 2024-09-16T01:38:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.991+0000 7f192b7fe640 1 --2- 172.21.15.139:0/3072937906 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f19200a3240 0x7f19200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.991+0000 7f193a088640 1 --2- 172.21.15.139:0/3072937906 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f19200a3240 0x7f19200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.991+0000 7f192b7fe640 1 -- 172.21.15.139:0/3072937906 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f19240cf040 con 0x7f1934102790 2024-09-16T01:38:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.991+0000 7f193a088640 1 --2- 172.21.15.139:0/3072937906 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f19200a3240 0x7f19200a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f19300058c0 tx=0x7f193000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:12.995+0000 7f192b7fe640 1 -- 172.21.15.139:0/3072937906 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f192409ba00 con 0x7f1934102790 2024-09-16T01:38:13.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:12 smithi164 ceph-mon[35121]: pgmap v284: 129 pgs: 4 creating+peering, 2 peering, 4 unknown, 119 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-16T01:38:13.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:12 smithi164 ceph-mon[35121]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T01:38:13.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:12 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/551884507' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T01:38:13.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:12 smithi164 ceph-mon[35121]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:13.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:13.196+0000 7f193c2ec640 1 -- 172.21.15.139:0/3072937906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f193410a1b0 con 0x7f1934102790 2024-09-16T01:38:13.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:13.197+0000 7f192b7fe640 1 -- 172.21.15.139:0/3072937906 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11860 (secure 0 0 0) 0x7f192409b300 con 0x7f1934102790 2024-09-16T01:38:13.199 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:38:13.199 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":17,"btime":"2024-09-16T01:38:11:548681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:38:11.211378+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24470,"qdb_cluster":[24470]},"id":1}]} 2024-09-16T01:38:13.199 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 17 2024-09-16T01:38:13.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:13.199+0000 7f193c2ec640 1 -- 172.21.15.139:0/3072937906 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f19200a3240 msgr2=0x7f19200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:13.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:13.199+0000 7f193c2ec640 1 --2- 172.21.15.139:0/3072937906 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f19200a3240 0x7f19200a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f19300058c0 tx=0x7f193000a250 comp rx=0 tx=0).stop 2024-09-16T01:38:13.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:13.199+0000 7f193c2ec640 1 -- 172.21.15.139:0/3072937906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1934102790 msgr2=0x7f19341989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:13.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:13.199+0000 7f193c2ec640 1 --2- 172.21.15.139:0/3072937906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1934102790 0x7f19341989f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f192400b4a0 tx=0x7f192400b960 comp rx=0 tx=0).stop 2024-09-16T01:38:13.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:13.199+0000 7f193c2ec640 1 -- 172.21.15.139:0/3072937906 shutdown_connections 2024-09-16T01:38:13.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:13.199+0000 7f193c2ec640 1 --2- 172.21.15.139:0/3072937906 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f19200a3240 0x7f19200a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:13.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:13.199+0000 7f193c2ec640 1 --2- 172.21.15.139:0/3072937906 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1934106f10 0x7f193419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:13.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:13.199+0000 7f193c2ec640 1 --2- 172.21.15.139:0/3072937906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1934102790 0x7f19341989f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:13.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:13.199+0000 7f193c2ec640 1 --2- 172.21.15.139:0/3072937906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1934101de0 0x7f19341984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:13.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:13.199+0000 7f193c2ec640 1 -- 172.21.15.139:0/3072937906 >> 172.21.15.139:0/3072937906 conn(0x7f19340fd8c0 msgr2=0x7f19340fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:13.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:13.199+0000 7f193c2ec640 1 -- 172.21.15.139:0/3072937906 shutdown_connections 2024-09-16T01:38:13.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:13.199+0000 7f193c2ec640 1 -- 172.21.15.139:0/3072937906 wait complete. 2024-09-16T01:38:13.874 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:13 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3072937906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:38:13.904 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T01:38:13.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:13 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3072937906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:38:14.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:13 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3072937906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:38:14.149 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:14.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.645+0000 7f48f226d640 1 -- 172.21.15.139:0/2841066688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48ec100a80 msgr2=0x7f48ec100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:14.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.645+0000 7f48f226d640 1 --2- 172.21.15.139:0/2841066688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48ec100a80 0x7f48ec100ee0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009930 tx=0x7f48d402a420 comp rx=0 tx=0).stop 2024-09-16T01:38:14.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.645+0000 7f48f226d640 1 -- 172.21.15.139:0/2841066688 shutdown_connections 2024-09-16T01:38:14.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.645+0000 7f48f226d640 1 --2- 172.21.15.139:0/2841066688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ec101420 0x7f48ec10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:14.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.645+0000 7f48f226d640 1 --2- 172.21.15.139:0/2841066688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48ec100a80 0x7f48ec100ee0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:14.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.645+0000 7f48f226d640 1 --2- 172.21.15.139:0/2841066688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48ec106a80 0x7f48ec106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:14.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.645+0000 7f48f226d640 1 -- 172.21.15.139:0/2841066688 >> 172.21.15.139:0/2841066688 conn(0x7f48ec0fc7a0 msgr2=0x7f48ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:14.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.645+0000 7f48f226d640 1 -- 172.21.15.139:0/2841066688 shutdown_connections 2024-09-16T01:38:14.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.645+0000 7f48f226d640 1 -- 172.21.15.139:0/2841066688 wait complete. 2024-09-16T01:38:14.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.646+0000 7f48f226d640 1 Processor -- start 2024-09-16T01:38:14.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.646+0000 7f48f226d640 1 -- start start 2024-09-16T01:38:14.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.646+0000 7f48f226d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ec100a80 0x7f48ec199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:14.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.646+0000 7f48f226d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48ec101420 0x7f48ec199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:14.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.646+0000 7f48f226d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48ec106a80 0x7f48ec1a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:14.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.646+0000 7f48f226d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec110ed0 con 0x7f48ec106a80 2024-09-16T01:38:14.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.646+0000 7f48f226d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec110d50 con 0x7f48ec100a80 2024-09-16T01:38:14.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.646+0000 7f48f226d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec111050 con 0x7f48ec101420 2024-09-16T01:38:14.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.646+0000 7f48f080a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48ec106a80 0x7f48ec1a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:14.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.646+0000 7f48ebfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ec100a80 0x7f48ec199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:14.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.646+0000 7f48f080a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48ec106a80 0x7f48ec1a4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57452/0 (socket says 172.21.15.139:57452) 2024-09-16T01:38:14.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.646+0000 7f48f080a640 1 -- 172.21.15.139:0/1257011155 learned_addr learned my addr 172.21.15.139:0/1257011155 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:14.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.646+0000 7f48e3fff640 1 --2- 172.21.15.139:0/1257011155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48ec101420 0x7f48ec199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:14.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.647+0000 7f48f080a640 1 -- 172.21.15.139:0/1257011155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48ec101420 msgr2=0x7f48ec199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:14.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.647+0000 7f48f080a640 1 --2- 172.21.15.139:0/1257011155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48ec101420 0x7f48ec199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:14.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.647+0000 7f48f080a640 1 -- 172.21.15.139:0/1257011155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ec100a80 msgr2=0x7f48ec199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:14.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.647+0000 7f48f080a640 1 --2- 172.21.15.139:0/1257011155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ec100a80 0x7f48ec199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:14.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.647+0000 7f48f080a640 1 -- 172.21.15.139:0/1257011155 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec1a48c0 con 0x7f48ec106a80 2024-09-16T01:38:14.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.647+0000 7f48f080a640 1 --2- 172.21.15.139:0/1257011155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48ec106a80 0x7f48ec1a4380 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009fd0 tx=0x7f48e400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:14.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.647+0000 7f48e9ffb640 1 -- 172.21.15.139:0/1257011155 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4017070 con 0x7f48ec106a80 2024-09-16T01:38:14.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.647+0000 7f48e9ffb640 1 -- 172.21.15.139:0/1257011155 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e4003e50 con 0x7f48ec106a80 2024-09-16T01:38:14.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.647+0000 7f48f226d640 1 -- 172.21.15.139:0/1257011155 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ec1a4a90 con 0x7f48ec106a80 2024-09-16T01:38:14.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.647+0000 7f48e9ffb640 1 -- 172.21.15.139:0/1257011155 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e40106c0 con 0x7f48ec106a80 2024-09-16T01:38:14.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.647+0000 7f48f226d640 1 -- 172.21.15.139:0/1257011155 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ec1a4f50 con 0x7f48ec106a80 2024-09-16T01:38:14.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.648+0000 7f48f226d640 1 -- 172.21.15.139:0/1257011155 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ec100ee0 con 0x7f48ec106a80 2024-09-16T01:38:14.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.649+0000 7f48e9ffb640 1 -- 172.21.15.139:0/1257011155 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f48e4003970 con 0x7f48ec106a80 2024-09-16T01:38:14.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.650+0000 7f48e9ffb640 1 --2- 172.21.15.139:0/1257011155 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f48d80a3240 0x7f48d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:14.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.650+0000 7f48ebfff640 1 --2- 172.21.15.139:0/1257011155 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f48d80a3240 0x7f48d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:14.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.650+0000 7f48e9ffb640 1 -- 172.21.15.139:0/1257011155 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f48e40cf960 con 0x7f48ec106a80 2024-09-16T01:38:14.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.650+0000 7f48ebfff640 1 --2- 172.21.15.139:0/1257011155 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f48d80a3240 0x7f48d80a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00bb60 tx=0x7f48dc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:14.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.654+0000 7f48e9ffb640 1 -- 172.21.15.139:0/1257011155 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48e409bcd0 con 0x7f48ec106a80 2024-09-16T01:38:14.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.821+0000 7f48f226d640 1 -- 172.21.15.139:0/1257011155 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48ec108ff0 con 0x7f48ec106a80 2024-09-16T01:38:14.823 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:14 smithi139 ceph-mon[31175]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s; 98 B/s, 0 objects/s recovering 2024-09-16T01:38:14.823 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:14 smithi139 ceph-mon[31175]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T01:38:14.823 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:14 smithi139 ceph-mon[31175]: Cluster is now healthy 2024-09-16T01:38:14.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.822+0000 7f48e9ffb640 1 -- 172.21.15.139:0/1257011155 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24797 (secure 0 0 0) 0x7f48e409ffd0 con 0x7f48ec106a80 2024-09-16T01:38:14.824 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:38:14.825 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":83,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","created":"2024-09-16T01:30:17.770313+0000","modified":"2024-09-16T01:38:11.211197+0000","last_up_change":"2024-09-16T01:37:08.611901+0000","last_in_change":"2024-09-16T01:36:45.223926+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:33:19.639647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T01:37:57.948214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T01:38:00.263797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T01:38:06.773799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b66696da-ef66-4f25-9527-bbcc9ded2cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6803","nonce":3190810133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6805","nonce":3190810133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6809","nonce":3190810133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6807","nonce":3190810133}]},"public_addr":"172.21.15.139:6803/3190810133","cluster_addr":"172.21.15.139:6805/3190810133","heartbeat_back_addr":"172.21.15.139:6809/3190810133","heartbeat_front_addr":"172.21.15.139:6807/3190810133","state":["exists","up"]},{"osd":1,"uuid":"ccad17f2-0750-425e-9810-49a13a929ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6801","nonce":2699270330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6803","nonce":2699270330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6807","nonce":2699270330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6805","nonce":2699270330}]},"public_addr":"172.21.15.164:6801/2699270330","cluster_addr":"172.21.15.164:6803/2699270330","heartbeat_back_addr":"172.21.15.164:6807/2699270330","heartbeat_front_addr":"172.21.15.164:6805/2699270330","state":["exists","up"]},{"osd":2,"uuid":"baf0ba73-8faf-4e51-a171-c47898800638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6801","nonce":2034471646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6803","nonce":2034471646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6807","nonce":2034471646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6805","nonce":2034471646}]},"public_addr":"172.21.15.176:6801/2034471646","cluster_addr":"172.21.15.176:6803/2034471646","heartbeat_back_addr":"172.21.15.176:6807/2034471646","heartbeat_front_addr":"172.21.15.176:6805/2034471646","state":["exists","up"]},{"osd":3,"uuid":"98957bf1-67bb-4092-9ea4-b263a5706f8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6811","nonce":797147071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6813","nonce":797147071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6817","nonce":797147071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6815","nonce":797147071}]},"public_addr":"172.21.15.139:6811/797147071","cluster_addr":"172.21.15.139:6813/797147071","heartbeat_back_addr":"172.21.15.139:6817/797147071","heartbeat_front_addr":"172.21.15.139:6815/797147071","state":["exists","up"]},{"osd":4,"uuid":"9cd76696-8161-4e82-a571-29e98c11d5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6809","nonce":1421304792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6811","nonce":1421304792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6815","nonce":1421304792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6813","nonce":1421304792}]},"public_addr":"172.21.15.164:6809/1421304792","cluster_addr":"172.21.15.164:6811/1421304792","heartbeat_back_addr":"172.21.15.164:6815/1421304792","heartbeat_front_addr":"172.21.15.164:6813/1421304792","state":["exists","up"]},{"osd":5,"uuid":"ccc17a29-9a58-40ec-b453-5c38bee86f25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6809","nonce":10600602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6811","nonce":10600602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6815","nonce":10600602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6813","nonce":10600602}]},"public_addr":"172.21.15.176:6809/10600602","cluster_addr":"172.21.15.176:6811/10600602","heartbeat_back_addr":"172.21.15.176:6815/10600602","heartbeat_front_addr":"172.21.15.176:6813/10600602","state":["exists","up"]},{"osd":6,"uuid":"e5ee7212-10c6-4a05-992e-622a14a5a6b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6819","nonce":2601111164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6821","nonce":2601111164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6825","nonce":2601111164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6823","nonce":2601111164}]},"public_addr":"172.21.15.139:6819/2601111164","cluster_addr":"172.21.15.139:6821/2601111164","heartbeat_back_addr":"172.21.15.139:6825/2601111164","heartbeat_front_addr":"172.21.15.139:6823/2601111164","state":["exists","up"]},{"osd":7,"uuid":"23143c15-0839-4fd5-9d0a-19ae891000f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6817","nonce":2074080693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6819","nonce":2074080693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6823","nonce":2074080693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6821","nonce":2074080693}]},"public_addr":"172.21.15.164:6817/2074080693","cluster_addr":"172.21.15.164:6819/2074080693","heartbeat_back_addr":"172.21.15.164:6823/2074080693","heartbeat_front_addr":"172.21.15.164:6821/2074080693","state":["exists","up"]},{"osd":8,"uuid":"1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6817","nonce":593133361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6819","nonce":593133361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6823","nonce":593133361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6821","nonce":593133361}]},"public_addr":"172.21.15.176:6817/593133361","cluster_addr":"172.21.15.176:6819/593133361","heartbeat_back_addr":"172.21.15.176:6823/593133361","heartbeat_front_addr":"172.21.15.176:6821/593133361","state":["exists","up"]},{"osd":9,"uuid":"09d25a3b-1e66-480e-9433-ceef7f0b9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6827","nonce":3047942108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6829","nonce":3047942108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6833","nonce":3047942108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6831","nonce":3047942108}]},"public_addr":"172.21.15.139:6827/3047942108","cluster_addr":"172.21.15.139:6829/3047942108","heartbeat_back_addr":"172.21.15.139:6833/3047942108","heartbeat_front_addr":"172.21.15.139:6831/3047942108","state":["exists","up"]},{"osd":10,"uuid":"32b4d8ce-b644-42fa-80cc-a15a1f5f3a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6825","nonce":4172533828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6827","nonce":4172533828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6831","nonce":4172533828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6829","nonce":4172533828}]},"public_addr":"172.21.15.164:6825/4172533828","cluster_addr":"172.21.15.164:6827/4172533828","heartbeat_back_addr":"172.21.15.164:6831/4172533828","heartbeat_front_addr":"172.21.15.164:6829/4172533828","state":["exists","up"]},{"osd":11,"uuid":"49098bdc-58a9-4a74-8045-d408af7711ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6825","nonce":1854547565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6827","nonce":1854547565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6831","nonce":1854547565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6829","nonce":1854547565}]},"public_addr":"172.21.15.176:6825/1854547565","cluster_addr":"172.21.15.176:6827/1854547565","heartbeat_back_addr":"172.21.15.176:6831/1854547565","heartbeat_front_addr":"172.21.15.176:6829/1854547565","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:27.374845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:52.611853+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:17.229567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:43.330503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:07.988876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:33.787864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:58.864910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:23.815006+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:49.551954+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:15.496773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:41.647557+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:37:07.089355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3772232169":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/3997627612":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/1608930262":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6801/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/677216867":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6801/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/1830135963":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/2608207648":"2024-09-17T01:30:44.751168+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:38:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.825+0000 7f48f226d640 1 -- 172.21.15.139:0/1257011155 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f48d80a3240 msgr2=0x7f48d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.825+0000 7f48f226d640 1 --2- 172.21.15.139:0/1257011155 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f48d80a3240 0x7f48d80a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00bb60 tx=0x7f48dc00b390 comp rx=0 tx=0).stop 2024-09-16T01:38:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.825+0000 7f48f226d640 1 -- 172.21.15.139:0/1257011155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48ec106a80 msgr2=0x7f48ec1a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.825+0000 7f48f226d640 1 --2- 172.21.15.139:0/1257011155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48ec106a80 0x7f48ec1a4380 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009fd0 tx=0x7f48e400b260 comp rx=0 tx=0).stop 2024-09-16T01:38:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.825+0000 7f48f226d640 1 -- 172.21.15.139:0/1257011155 shutdown_connections 2024-09-16T01:38:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.825+0000 7f48f226d640 1 --2- 172.21.15.139:0/1257011155 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f48d80a3240 0x7f48d80a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.825+0000 7f48f226d640 1 --2- 172.21.15.139:0/1257011155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48ec106a80 0x7f48ec1a4380 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.825+0000 7f48f226d640 1 --2- 172.21.15.139:0/1257011155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48ec101420 0x7f48ec199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.825+0000 7f48f226d640 1 --2- 172.21.15.139:0/1257011155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ec100a80 0x7f48ec199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.825+0000 7f48f226d640 1 -- 172.21.15.139:0/1257011155 >> 172.21.15.139:0/1257011155 conn(0x7f48ec0fc7a0 msgr2=0x7f48ec0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.825+0000 7f48f226d640 1 -- 172.21.15.139:0/1257011155 shutdown_connections 2024-09-16T01:38:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:14.825+0000 7f48f226d640 1 -- 172.21.15.139:0/1257011155 wait complete. 2024-09-16T01:38:14.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:14 smithi176 ceph-mon[35324]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s; 98 B/s, 0 objects/s recovering 2024-09-16T01:38:14.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:14 smithi176 ceph-mon[35324]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T01:38:14.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:14 smithi176 ceph-mon[35324]: Cluster is now healthy 2024-09-16T01:38:15.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:14 smithi164 ceph-mon[35121]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s; 98 B/s, 0 objects/s recovering 2024-09-16T01:38:15.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:14 smithi164 ceph-mon[35121]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T01:38:15.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:14 smithi164 ceph-mon[35121]: Cluster is now healthy 2024-09-16T01:38:15.627 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T01:38:15.873 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:15.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:15 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1257011155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:15.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:15 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1257011155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:16.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:15 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1257011155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:16.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.366+0000 7fa8e9116640 1 -- 172.21.15.139:0/758131108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e4101de0 msgr2=0x7fa8e41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:16.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.366+0000 7fa8e9116640 1 --2- 172.21.15.139:0/758131108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e4101de0 0x7fa8e41021c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009900 tx=0x7fa8cc02a410 comp rx=0 tx=0).stop 2024-09-16T01:38:16.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.366+0000 7fa8e9116640 1 -- 172.21.15.139:0/758131108 shutdown_connections 2024-09-16T01:38:16.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.366+0000 7fa8e9116640 1 --2- 172.21.15.139:0/758131108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8e4106f10 0x7fa8e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:16.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.366+0000 7fa8e9116640 1 --2- 172.21.15.139:0/758131108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8e4102790 0x7fa8e41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:16.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.366+0000 7fa8e9116640 1 --2- 172.21.15.139:0/758131108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e4101de0 0x7fa8e41021c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:16.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.366+0000 7fa8e9116640 1 -- 172.21.15.139:0/758131108 >> 172.21.15.139:0/758131108 conn(0x7fa8e40fd8c0 msgr2=0x7fa8e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:16.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.366+0000 7fa8e9116640 1 -- 172.21.15.139:0/758131108 shutdown_connections 2024-09-16T01:38:16.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.366+0000 7fa8e9116640 1 -- 172.21.15.139:0/758131108 wait complete. 2024-09-16T01:38:16.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.366+0000 7fa8e9116640 1 Processor -- start 2024-09-16T01:38:16.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e9116640 1 -- start start 2024-09-16T01:38:16.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e9116640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8e4101de0 0x7fa8e41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:16.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e9116640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e4102790 0x7fa8e4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:16.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e9116640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8e4106f10 0x7fa8e419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:16.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e9116640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e4112010 con 0x7fa8e4102790 2024-09-16T01:38:16.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e9116640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e4111e90 con 0x7fa8e4106f10 2024-09-16T01:38:16.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e9116640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e4112190 con 0x7fa8e4101de0 2024-09-16T01:38:16.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e4102790 0x7fa8e4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:16.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e4102790 0x7fa8e4198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57472/0 (socket says 172.21.15.139:57472) 2024-09-16T01:38:16.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e259c640 1 -- 172.21.15.139:0/3372331689 learned_addr learned my addr 172.21.15.139:0/3372331689 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:16.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e2d9d640 1 --2- 172.21.15.139:0/3372331689 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8e4101de0 0x7fa8e41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:16.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e359e640 1 --2- 172.21.15.139:0/3372331689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8e4106f10 0x7fa8e419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:16.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e259c640 1 -- 172.21.15.139:0/3372331689 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8e4101de0 msgr2=0x7fa8e41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:16.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e259c640 1 --2- 172.21.15.139:0/3372331689 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8e4101de0 0x7fa8e41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:16.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e259c640 1 -- 172.21.15.139:0/3372331689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8e4106f10 msgr2=0x7fa8e419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:16.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e259c640 1 --2- 172.21.15.139:0/3372331689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8e4106f10 0x7fa8e419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:16.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e259c640 1 -- 172.21.15.139:0/3372331689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e419d560 con 0x7fa8e4102790 2024-09-16T01:38:16.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.367+0000 7fa8e259c640 1 --2- 172.21.15.139:0/3372331689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e4102790 0x7fa8e4198af0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400b4a0 tx=0x7fa8d400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:16.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.368+0000 7fa8dbfff640 1 -- 172.21.15.139:0/3372331689 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4014070 con 0x7fa8e4102790 2024-09-16T01:38:16.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.368+0000 7fa8e9116640 1 -- 172.21.15.139:0/3372331689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e419d850 con 0x7fa8e4102790 2024-09-16T01:38:16.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.368+0000 7fa8dbfff640 1 -- 172.21.15.139:0/3372331689 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d4010af0 con 0x7fa8e4102790 2024-09-16T01:38:16.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.368+0000 7fa8dbfff640 1 -- 172.21.15.139:0/3372331689 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4017d10 con 0x7fa8e4102790 2024-09-16T01:38:16.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.368+0000 7fa8e9116640 1 -- 172.21.15.139:0/3372331689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e41a5130 con 0x7fa8e4102790 2024-09-16T01:38:16.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.369+0000 7fa8e9116640 1 -- 172.21.15.139:0/3372331689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e4102240 con 0x7fa8e4102790 2024-09-16T01:38:16.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.370+0000 7fa8dbfff640 1 -- 172.21.15.139:0/3372331689 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa8d4017070 con 0x7fa8e4102790 2024-09-16T01:38:16.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.371+0000 7fa8dbfff640 1 --2- 172.21.15.139:0/3372331689 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa8d00a3240 0x7fa8d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:16.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.371+0000 7fa8e2d9d640 1 --2- 172.21.15.139:0/3372331689 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa8d00a3240 0x7fa8d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:16.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.371+0000 7fa8dbfff640 1 -- 172.21.15.139:0/3372331689 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fa8d40cef90 con 0x7fa8e4102790 2024-09-16T01:38:16.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.371+0000 7fa8e2d9d640 1 --2- 172.21.15.139:0/3372331689 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa8d00a3240 0x7fa8d00a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc000c00 tx=0x7fa8cc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:16.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.375+0000 7fa8dbfff640 1 -- 172.21.15.139:0/3372331689 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8d409b300 con 0x7fa8e4102790 2024-09-16T01:38:16.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.542+0000 7fa8e9116640 1 -- 172.21.15.139:0/3372331689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8e4109b60 con 0x7fa8e4102790 2024-09-16T01:38:16.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.543+0000 7fa8dbfff640 1 -- 172.21.15.139:0/3372331689 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24797 (secure 0 0 0) 0x7fa8d409f600 con 0x7fa8e4102790 2024-09-16T01:38:16.544 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:38:16.544 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":83,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","created":"2024-09-16T01:30:17.770313+0000","modified":"2024-09-16T01:38:11.211197+0000","last_up_change":"2024-09-16T01:37:08.611901+0000","last_in_change":"2024-09-16T01:36:45.223926+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:33:19.639647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T01:37:57.948214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T01:38:00.263797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T01:38:06.773799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b66696da-ef66-4f25-9527-bbcc9ded2cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6803","nonce":3190810133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6805","nonce":3190810133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6809","nonce":3190810133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6807","nonce":3190810133}]},"public_addr":"172.21.15.139:6803/3190810133","cluster_addr":"172.21.15.139:6805/3190810133","heartbeat_back_addr":"172.21.15.139:6809/3190810133","heartbeat_front_addr":"172.21.15.139:6807/3190810133","state":["exists","up"]},{"osd":1,"uuid":"ccad17f2-0750-425e-9810-49a13a929ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6801","nonce":2699270330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6803","nonce":2699270330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6807","nonce":2699270330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6805","nonce":2699270330}]},"public_addr":"172.21.15.164:6801/2699270330","cluster_addr":"172.21.15.164:6803/2699270330","heartbeat_back_addr":"172.21.15.164:6807/2699270330","heartbeat_front_addr":"172.21.15.164:6805/2699270330","state":["exists","up"]},{"osd":2,"uuid":"baf0ba73-8faf-4e51-a171-c47898800638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6801","nonce":2034471646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6803","nonce":2034471646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6807","nonce":2034471646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6805","nonce":2034471646}]},"public_addr":"172.21.15.176:6801/2034471646","cluster_addr":"172.21.15.176:6803/2034471646","heartbeat_back_addr":"172.21.15.176:6807/2034471646","heartbeat_front_addr":"172.21.15.176:6805/2034471646","state":["exists","up"]},{"osd":3,"uuid":"98957bf1-67bb-4092-9ea4-b263a5706f8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6811","nonce":797147071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6813","nonce":797147071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6817","nonce":797147071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6815","nonce":797147071}]},"public_addr":"172.21.15.139:6811/797147071","cluster_addr":"172.21.15.139:6813/797147071","heartbeat_back_addr":"172.21.15.139:6817/797147071","heartbeat_front_addr":"172.21.15.139:6815/797147071","state":["exists","up"]},{"osd":4,"uuid":"9cd76696-8161-4e82-a571-29e98c11d5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6809","nonce":1421304792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6811","nonce":1421304792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6815","nonce":1421304792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6813","nonce":1421304792}]},"public_addr":"172.21.15.164:6809/1421304792","cluster_addr":"172.21.15.164:6811/1421304792","heartbeat_back_addr":"172.21.15.164:6815/1421304792","heartbeat_front_addr":"172.21.15.164:6813/1421304792","state":["exists","up"]},{"osd":5,"uuid":"ccc17a29-9a58-40ec-b453-5c38bee86f25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6809","nonce":10600602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6811","nonce":10600602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6815","nonce":10600602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6813","nonce":10600602}]},"public_addr":"172.21.15.176:6809/10600602","cluster_addr":"172.21.15.176:6811/10600602","heartbeat_back_addr":"172.21.15.176:6815/10600602","heartbeat_front_addr":"172.21.15.176:6813/10600602","state":["exists","up"]},{"osd":6,"uuid":"e5ee7212-10c6-4a05-992e-622a14a5a6b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6819","nonce":2601111164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6821","nonce":2601111164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6825","nonce":2601111164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6823","nonce":2601111164}]},"public_addr":"172.21.15.139:6819/2601111164","cluster_addr":"172.21.15.139:6821/2601111164","heartbeat_back_addr":"172.21.15.139:6825/2601111164","heartbeat_front_addr":"172.21.15.139:6823/2601111164","state":["exists","up"]},{"osd":7,"uuid":"23143c15-0839-4fd5-9d0a-19ae891000f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6817","nonce":2074080693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6819","nonce":2074080693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6823","nonce":2074080693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6821","nonce":2074080693}]},"public_addr":"172.21.15.164:6817/2074080693","cluster_addr":"172.21.15.164:6819/2074080693","heartbeat_back_addr":"172.21.15.164:6823/2074080693","heartbeat_front_addr":"172.21.15.164:6821/2074080693","state":["exists","up"]},{"osd":8,"uuid":"1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6817","nonce":593133361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6819","nonce":593133361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6823","nonce":593133361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6821","nonce":593133361}]},"public_addr":"172.21.15.176:6817/593133361","cluster_addr":"172.21.15.176:6819/593133361","heartbeat_back_addr":"172.21.15.176:6823/593133361","heartbeat_front_addr":"172.21.15.176:6821/593133361","state":["exists","up"]},{"osd":9,"uuid":"09d25a3b-1e66-480e-9433-ceef7f0b9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6827","nonce":3047942108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6829","nonce":3047942108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6833","nonce":3047942108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6831","nonce":3047942108}]},"public_addr":"172.21.15.139:6827/3047942108","cluster_addr":"172.21.15.139:6829/3047942108","heartbeat_back_addr":"172.21.15.139:6833/3047942108","heartbeat_front_addr":"172.21.15.139:6831/3047942108","state":["exists","up"]},{"osd":10,"uuid":"32b4d8ce-b644-42fa-80cc-a15a1f5f3a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6825","nonce":4172533828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6827","nonce":4172533828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6831","nonce":4172533828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6829","nonce":4172533828}]},"public_addr":"172.21.15.164:6825/4172533828","cluster_addr":"172.21.15.164:6827/4172533828","heartbeat_back_addr":"172.21.15.164:6831/4172533828","heartbeat_front_addr":"172.21.15.164:6829/4172533828","state":["exists","up"]},{"osd":11,"uuid":"49098bdc-58a9-4a74-8045-d408af7711ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6825","nonce":1854547565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6827","nonce":1854547565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6831","nonce":1854547565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6829","nonce":1854547565}]},"public_addr":"172.21.15.176:6825/1854547565","cluster_addr":"172.21.15.176:6827/1854547565","heartbeat_back_addr":"172.21.15.176:6831/1854547565","heartbeat_front_addr":"172.21.15.176:6829/1854547565","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:27.374845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:52.611853+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:17.229567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:43.330503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:07.988876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:33.787864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:58.864910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:23.815006+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:49.551954+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:15.496773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:41.647557+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:37:07.089355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3772232169":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/3997627612":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/1608930262":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6801/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/677216867":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6801/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/1830135963":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/2608207648":"2024-09-17T01:30:44.751168+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:38:16.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.546+0000 7fa8e9116640 1 -- 172.21.15.139:0/3372331689 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa8d00a3240 msgr2=0x7fa8d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:16.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.546+0000 7fa8e9116640 1 --2- 172.21.15.139:0/3372331689 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa8d00a3240 0x7fa8d00a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc000c00 tx=0x7fa8cc0058d0 comp rx=0 tx=0).stop 2024-09-16T01:38:16.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.546+0000 7fa8e9116640 1 -- 172.21.15.139:0/3372331689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e4102790 msgr2=0x7fa8e4198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:16.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.546+0000 7fa8e9116640 1 --2- 172.21.15.139:0/3372331689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e4102790 0x7fa8e4198af0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400b4a0 tx=0x7fa8d400b960 comp rx=0 tx=0).stop 2024-09-16T01:38:16.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.546+0000 7fa8e9116640 1 -- 172.21.15.139:0/3372331689 shutdown_connections 2024-09-16T01:38:16.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.546+0000 7fa8e9116640 1 --2- 172.21.15.139:0/3372331689 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa8d00a3240 0x7fa8d00a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:16.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.546+0000 7fa8e9116640 1 --2- 172.21.15.139:0/3372331689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8e4106f10 0x7fa8e419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:16.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.546+0000 7fa8e9116640 1 --2- 172.21.15.139:0/3372331689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e4102790 0x7fa8e4198af0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:16.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.546+0000 7fa8e9116640 1 --2- 172.21.15.139:0/3372331689 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8e4101de0 0x7fa8e41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:16.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.546+0000 7fa8e9116640 1 -- 172.21.15.139:0/3372331689 >> 172.21.15.139:0/3372331689 conn(0x7fa8e40fd8c0 msgr2=0x7fa8e40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:16.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.546+0000 7fa8e9116640 1 -- 172.21.15.139:0/3372331689 shutdown_connections 2024-09-16T01:38:16.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:16.546+0000 7fa8e9116640 1 -- 172.21.15.139:0/3372331689 wait complete. 2024-09-16T01:38:16.838 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:16 smithi176 ceph-mon[35324]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 84 B/s, 0 objects/s recovering 2024-09-16T01:38:16.838 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:16 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3372331689' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:16.916 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:16 smithi139 ceph-mon[31175]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 84 B/s, 0 objects/s recovering 2024-09-16T01:38:16.916 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:16 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3372331689' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:16.946 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:16 smithi164 ceph-mon[35121]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 84 B/s, 0 objects/s recovering 2024-09-16T01:38:16.946 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:16 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3372331689' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:17.363 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:38:17.616 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:18.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.072+0000 7fd43e96d640 1 -- 172.21.15.139:0/2409903193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd438101de0 msgr2=0x7fd4381021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:18.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.072+0000 7fd43e96d640 1 --2- 172.21.15.139:0/2409903193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd438101de0 0x7fd4381021c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd428009900 tx=0x7fd42802a410 comp rx=0 tx=0).stop 2024-09-16T01:38:18.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.072+0000 7fd43e96d640 1 -- 172.21.15.139:0/2409903193 shutdown_connections 2024-09-16T01:38:18.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.072+0000 7fd43e96d640 1 --2- 172.21.15.139:0/2409903193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd438106f10 0x7fd43810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:18.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.072+0000 7fd43e96d640 1 --2- 172.21.15.139:0/2409903193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd438102790 0x7fd4381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:18.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.072+0000 7fd43e96d640 1 --2- 172.21.15.139:0/2409903193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd438101de0 0x7fd4381021c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:18.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.072+0000 7fd43e96d640 1 -- 172.21.15.139:0/2409903193 >> 172.21.15.139:0/2409903193 conn(0x7fd4380fd8c0 msgr2=0x7fd4380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:18.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.072+0000 7fd43e96d640 1 -- 172.21.15.139:0/2409903193 shutdown_connections 2024-09-16T01:38:18.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.072+0000 7fd43e96d640 1 -- 172.21.15.139:0/2409903193 wait complete. 2024-09-16T01:38:18.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.073+0000 7fd43e96d640 1 Processor -- start 2024-09-16T01:38:18.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.073+0000 7fd43e96d640 1 -- start start 2024-09-16T01:38:18.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.073+0000 7fd43e96d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd438101de0 0x7fd4381984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:18.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.073+0000 7fd43e96d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd438102790 0x7fd4381989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:18.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.073+0000 7fd43e96d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd438106f10 0x7fd43819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:18.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.073+0000 7fd43e96d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd438111f70 con 0x7fd438106f10 2024-09-16T01:38:18.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.073+0000 7fd43e96d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd438111df0 con 0x7fd438101de0 2024-09-16T01:38:18.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.073+0000 7fd43e96d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd4381120f0 con 0x7fd438102790 2024-09-16T01:38:18.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.073+0000 7fd43cf0a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd438106f10 0x7fd43819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:18.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd43cf0a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd438106f10 0x7fd43819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57492/0 (socket says 172.21.15.139:57492) 2024-09-16T01:38:18.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd43cf0a640 1 -- 172.21.15.139:0/2253893872 learned_addr learned my addr 172.21.15.139:0/2253893872 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:18.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd437fff640 1 --2- 172.21.15.139:0/2253893872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd438101de0 0x7fd4381984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:18.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd4377fe640 1 --2- 172.21.15.139:0/2253893872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd438102790 0x7fd4381989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:18.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd43cf0a640 1 -- 172.21.15.139:0/2253893872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd438102790 msgr2=0x7fd4381989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:18.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd43cf0a640 1 --2- 172.21.15.139:0/2253893872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd438102790 0x7fd4381989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:18.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd43cf0a640 1 -- 172.21.15.139:0/2253893872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd438101de0 msgr2=0x7fd4381984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:18.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd43cf0a640 1 --2- 172.21.15.139:0/2253893872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd438101de0 0x7fd4381984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:18.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd43cf0a640 1 -- 172.21.15.139:0/2253893872 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43819d460 con 0x7fd438106f10 2024-09-16T01:38:18.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd43cf0a640 1 --2- 172.21.15.139:0/2253893872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd438106f10 0x7fd43819cd80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd43000eb10 tx=0x7fd43000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:18.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd4357fa640 1 -- 172.21.15.139:0/2253893872 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430004b20 con 0x7fd438106f10 2024-09-16T01:38:18.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd43e96d640 1 -- 172.21.15.139:0/2253893872 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43819d750 con 0x7fd438106f10 2024-09-16T01:38:18.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd4357fa640 1 -- 172.21.15.139:0/2253893872 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd430004cc0 con 0x7fd438106f10 2024-09-16T01:38:18.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd4357fa640 1 -- 172.21.15.139:0/2253893872 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430010760 con 0x7fd438106f10 2024-09-16T01:38:18.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.074+0000 7fd43e96d640 1 -- 172.21.15.139:0/2253893872 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4381a5030 con 0x7fd438106f10 2024-09-16T01:38:18.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.075+0000 7fd43e96d640 1 -- 172.21.15.139:0/2253893872 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4381021c0 con 0x7fd438106f10 2024-09-16T01:38:18.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.076+0000 7fd4357fa640 1 -- 172.21.15.139:0/2253893872 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fd430005020 con 0x7fd438106f10 2024-09-16T01:38:18.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.077+0000 7fd4357fa640 1 --2- 172.21.15.139:0/2253893872 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd4200a3240 0x7fd4200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:18.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.077+0000 7fd4357fa640 1 -- 172.21.15.139:0/2253893872 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fd4300d0160 con 0x7fd438106f10 2024-09-16T01:38:18.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.077+0000 7fd437fff640 1 --2- 172.21.15.139:0/2253893872 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd4200a3240 0x7fd4200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:18.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.077+0000 7fd437fff640 1 --2- 172.21.15.139:0/2253893872 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd4200a3240 0x7fd4200a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd428009230 tx=0x7fd4280058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:18.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.081+0000 7fd4357fa640 1 -- 172.21.15.139:0/2253893872 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd43009cb20 con 0x7fd438106f10 2024-09-16T01:38:18.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.280+0000 7fd43e96d640 1 -- 172.21.15.139:0/2253893872 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd43810a150 con 0x7fd438106f10 2024-09-16T01:38:18.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.281+0000 7fd4357fa640 1 -- 172.21.15.139:0/2253893872 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11860 (secure 0 0 0) 0x7fd43009c420 con 0x7fd438106f10 2024-09-16T01:38:18.282 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:38:18.282 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":17,"btime":"2024-09-16T01:38:11:548681+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:38:11.211378+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24470,"qdb_cluster":[24470]},"id":1}]} 2024-09-16T01:38:18.282 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 17 2024-09-16T01:38:18.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.284+0000 7fd43e96d640 1 -- 172.21.15.139:0/2253893872 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd4200a3240 msgr2=0x7fd4200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:18.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.284+0000 7fd43e96d640 1 --2- 172.21.15.139:0/2253893872 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd4200a3240 0x7fd4200a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd428009230 tx=0x7fd4280058d0 comp rx=0 tx=0).stop 2024-09-16T01:38:18.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.284+0000 7fd43e96d640 1 -- 172.21.15.139:0/2253893872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd438106f10 msgr2=0x7fd43819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:18.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.284+0000 7fd43e96d640 1 --2- 172.21.15.139:0/2253893872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd438106f10 0x7fd43819cd80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd43000eb10 tx=0x7fd43000c2d0 comp rx=0 tx=0).stop 2024-09-16T01:38:18.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.284+0000 7fd43e96d640 1 -- 172.21.15.139:0/2253893872 shutdown_connections 2024-09-16T01:38:18.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.284+0000 7fd43e96d640 1 --2- 172.21.15.139:0/2253893872 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd4200a3240 0x7fd4200a5700 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:18.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.284+0000 7fd43e96d640 1 --2- 172.21.15.139:0/2253893872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd438106f10 0x7fd43819cd80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:18.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.284+0000 7fd43e96d640 1 --2- 172.21.15.139:0/2253893872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd438102790 0x7fd4381989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:18.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.284+0000 7fd43e96d640 1 --2- 172.21.15.139:0/2253893872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd438101de0 0x7fd4381984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:18.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.284+0000 7fd43e96d640 1 -- 172.21.15.139:0/2253893872 >> 172.21.15.139:0/2253893872 conn(0x7fd4380fd8c0 msgr2=0x7fd4380fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:18.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.284+0000 7fd43e96d640 1 -- 172.21.15.139:0/2253893872 shutdown_connections 2024-09-16T01:38:18.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:18.284+0000 7fd43e96d640 1 -- 172.21.15.139:0/2253893872 wait complete. 2024-09-16T01:38:18.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:18 smithi139 ceph-mon[31175]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-09-16T01:38:18.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:18 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2253893872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:38:18.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:18 smithi176 ceph-mon[35324]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-09-16T01:38:18.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:18 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2253893872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:38:19.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:18 smithi164 ceph-mon[35121]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-09-16T01:38:19.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:18 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2253893872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:38:19.055 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T01:38:19.299 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:19.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.787+0000 7f96310f1640 1 -- 172.21.15.139:0/3669322470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f962c101de0 msgr2=0x7f962c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:19.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.787+0000 7f96310f1640 1 --2- 172.21.15.139:0/3669322470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f962c101de0 0x7f962c1021c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f961c009900 tx=0x7f961c02a410 comp rx=0 tx=0).stop 2024-09-16T01:38:19.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.787+0000 7f96310f1640 1 -- 172.21.15.139:0/3669322470 shutdown_connections 2024-09-16T01:38:19.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.787+0000 7f96310f1640 1 --2- 172.21.15.139:0/3669322470 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f962c106f10 0x7f962c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:19.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.787+0000 7f96310f1640 1 --2- 172.21.15.139:0/3669322470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f962c102790 0x7f962c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:19.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.787+0000 7f96310f1640 1 --2- 172.21.15.139:0/3669322470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f962c101de0 0x7f962c1021c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:19.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.787+0000 7f96310f1640 1 -- 172.21.15.139:0/3669322470 >> 172.21.15.139:0/3669322470 conn(0x7f962c0fd8c0 msgr2=0x7f962c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:19.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.787+0000 7f96310f1640 1 -- 172.21.15.139:0/3669322470 shutdown_connections 2024-09-16T01:38:19.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.787+0000 7f96310f1640 1 -- 172.21.15.139:0/3669322470 wait complete. 2024-09-16T01:38:19.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.788+0000 7f96310f1640 1 Processor -- start 2024-09-16T01:38:19.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.788+0000 7f96310f1640 1 -- start start 2024-09-16T01:38:19.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.788+0000 7f96310f1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f962c101de0 0x7f962c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:19.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.788+0000 7f96310f1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f962c102790 0x7f962c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:19.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.788+0000 7f96310f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f962c106f10 0x7f962c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:19.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.788+0000 7f96310f1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f962c112010 con 0x7f962c101de0 2024-09-16T01:38:19.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.788+0000 7f96310f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f962c111e90 con 0x7f962c106f10 2024-09-16T01:38:19.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.788+0000 7f96310f1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f962c112190 con 0x7f962c102790 2024-09-16T01:38:19.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.788+0000 7f962ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f962c101de0 0x7f962c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:19.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f962ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f962c101de0 0x7f962c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57510/0 (socket says 172.21.15.139:57510) 2024-09-16T01:38:19.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f962ad9d640 1 -- 172.21.15.139:0/1408443393 learned_addr learned my addr 172.21.15.139:0/1408443393 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:19.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f962a59c640 1 --2- 172.21.15.139:0/1408443393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f962c102790 0x7f962c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:19.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f962ad9d640 1 -- 172.21.15.139:0/1408443393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f962c102790 msgr2=0x7f962c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:19.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f962ad9d640 1 --2- 172.21.15.139:0/1408443393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f962c102790 0x7f962c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:19.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f962b59e640 1 --2- 172.21.15.139:0/1408443393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f962c106f10 0x7f962c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:19.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f962ad9d640 1 -- 172.21.15.139:0/1408443393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f962c106f10 msgr2=0x7f962c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:19.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f962ad9d640 1 --2- 172.21.15.139:0/1408443393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f962c106f10 0x7f962c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:19.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f962ad9d640 1 -- 172.21.15.139:0/1408443393 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962c1a6a70 con 0x7f962c101de0 2024-09-16T01:38:19.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f962ad9d640 1 --2- 172.21.15.139:0/1408443393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f962c101de0 0x7f962c0744f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f961c005ab0 tx=0x7f961c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:19.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f9613fff640 1 -- 172.21.15.139:0/1408443393 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c03f030 con 0x7f962c101de0 2024-09-16T01:38:19.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f96310f1640 1 -- 172.21.15.139:0/1408443393 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962c1a6d00 con 0x7f962c101de0 2024-09-16T01:38:19.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f9613fff640 1 -- 172.21.15.139:0/1408443393 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f961c003ce0 con 0x7f962c101de0 2024-09-16T01:38:19.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f9613fff640 1 -- 172.21.15.139:0/1408443393 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c035070 con 0x7f962c101de0 2024-09-16T01:38:19.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.789+0000 7f96310f1640 1 -- 172.21.15.139:0/1408443393 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962c1a7180 con 0x7f962c101de0 2024-09-16T01:38:19.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.790+0000 7f96310f1640 1 -- 172.21.15.139:0/1408443393 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f962c102240 con 0x7f962c101de0 2024-09-16T01:38:19.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.791+0000 7f9613fff640 1 -- 172.21.15.139:0/1408443393 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f961c042030 con 0x7f962c101de0 2024-09-16T01:38:19.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.792+0000 7f9613fff640 1 --2- 172.21.15.139:0/1408443393 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f96140a3240 0x7f96140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:19.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.792+0000 7f962a59c640 1 --2- 172.21.15.139:0/1408443393 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f96140a3240 0x7f96140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:19.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.792+0000 7f9613fff640 1 -- 172.21.15.139:0/1408443393 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f961c0f0630 con 0x7f962c101de0 2024-09-16T01:38:19.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.792+0000 7f962a59c640 1 --2- 172.21.15.139:0/1408443393 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f96140a3240 0x7f96140a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f962c075f80 tx=0x7f9620008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:19.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.796+0000 7f9613fff640 1 -- 172.21.15.139:0/1408443393 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f961c0bcff0 con 0x7f962c101de0 2024-09-16T01:38:19.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.962+0000 7f96310f1640 1 -- 172.21.15.139:0/1408443393 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f962c109b60 con 0x7f962c101de0 2024-09-16T01:38:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.963+0000 7f9613fff640 1 -- 172.21.15.139:0/1408443393 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24797 (secure 0 0 0) 0x7f961c0bc8f0 con 0x7f962c101de0 2024-09-16T01:38:19.965 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:38:19.965 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":83,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","created":"2024-09-16T01:30:17.770313+0000","modified":"2024-09-16T01:38:11.211197+0000","last_up_change":"2024-09-16T01:37:08.611901+0000","last_in_change":"2024-09-16T01:36:45.223926+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:33:19.639647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T01:37:57.948214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T01:38:00.263797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T01:38:06.773799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b66696da-ef66-4f25-9527-bbcc9ded2cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6803","nonce":3190810133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6805","nonce":3190810133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6809","nonce":3190810133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6807","nonce":3190810133}]},"public_addr":"172.21.15.139:6803/3190810133","cluster_addr":"172.21.15.139:6805/3190810133","heartbeat_back_addr":"172.21.15.139:6809/3190810133","heartbeat_front_addr":"172.21.15.139:6807/3190810133","state":["exists","up"]},{"osd":1,"uuid":"ccad17f2-0750-425e-9810-49a13a929ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6801","nonce":2699270330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6803","nonce":2699270330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6807","nonce":2699270330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6805","nonce":2699270330}]},"public_addr":"172.21.15.164:6801/2699270330","cluster_addr":"172.21.15.164:6803/2699270330","heartbeat_back_addr":"172.21.15.164:6807/2699270330","heartbeat_front_addr":"172.21.15.164:6805/2699270330","state":["exists","up"]},{"osd":2,"uuid":"baf0ba73-8faf-4e51-a171-c47898800638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6801","nonce":2034471646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6803","nonce":2034471646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6807","nonce":2034471646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6805","nonce":2034471646}]},"public_addr":"172.21.15.176:6801/2034471646","cluster_addr":"172.21.15.176:6803/2034471646","heartbeat_back_addr":"172.21.15.176:6807/2034471646","heartbeat_front_addr":"172.21.15.176:6805/2034471646","state":["exists","up"]},{"osd":3,"uuid":"98957bf1-67bb-4092-9ea4-b263a5706f8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6811","nonce":797147071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6813","nonce":797147071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6817","nonce":797147071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6815","nonce":797147071}]},"public_addr":"172.21.15.139:6811/797147071","cluster_addr":"172.21.15.139:6813/797147071","heartbeat_back_addr":"172.21.15.139:6817/797147071","heartbeat_front_addr":"172.21.15.139:6815/797147071","state":["exists","up"]},{"osd":4,"uuid":"9cd76696-8161-4e82-a571-29e98c11d5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6809","nonce":1421304792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6811","nonce":1421304792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6815","nonce":1421304792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6813","nonce":1421304792}]},"public_addr":"172.21.15.164:6809/1421304792","cluster_addr":"172.21.15.164:6811/1421304792","heartbeat_back_addr":"172.21.15.164:6815/1421304792","heartbeat_front_addr":"172.21.15.164:6813/1421304792","state":["exists","up"]},{"osd":5,"uuid":"ccc17a29-9a58-40ec-b453-5c38bee86f25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6809","nonce":10600602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6811","nonce":10600602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6815","nonce":10600602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6813","nonce":10600602}]},"public_addr":"172.21.15.176:6809/10600602","cluster_addr":"172.21.15.176:6811/10600602","heartbeat_back_addr":"172.21.15.176:6815/10600602","heartbeat_front_addr":"172.21.15.176:6813/10600602","state":["exists","up"]},{"osd":6,"uuid":"e5ee7212-10c6-4a05-992e-622a14a5a6b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6819","nonce":2601111164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6821","nonce":2601111164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6825","nonce":2601111164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6823","nonce":2601111164}]},"public_addr":"172.21.15.139:6819/2601111164","cluster_addr":"172.21.15.139:6821/2601111164","heartbeat_back_addr":"172.21.15.139:6825/2601111164","heartbeat_front_addr":"172.21.15.139:6823/2601111164","state":["exists","up"]},{"osd":7,"uuid":"23143c15-0839-4fd5-9d0a-19ae891000f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6817","nonce":2074080693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6819","nonce":2074080693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6823","nonce":2074080693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6821","nonce":2074080693}]},"public_addr":"172.21.15.164:6817/2074080693","cluster_addr":"172.21.15.164:6819/2074080693","heartbeat_back_addr":"172.21.15.164:6823/2074080693","heartbeat_front_addr":"172.21.15.164:6821/2074080693","state":["exists","up"]},{"osd":8,"uuid":"1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6817","nonce":593133361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6819","nonce":593133361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6823","nonce":593133361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6821","nonce":593133361}]},"public_addr":"172.21.15.176:6817/593133361","cluster_addr":"172.21.15.176:6819/593133361","heartbeat_back_addr":"172.21.15.176:6823/593133361","heartbeat_front_addr":"172.21.15.176:6821/593133361","state":["exists","up"]},{"osd":9,"uuid":"09d25a3b-1e66-480e-9433-ceef7f0b9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6827","nonce":3047942108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6829","nonce":3047942108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6833","nonce":3047942108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6831","nonce":3047942108}]},"public_addr":"172.21.15.139:6827/3047942108","cluster_addr":"172.21.15.139:6829/3047942108","heartbeat_back_addr":"172.21.15.139:6833/3047942108","heartbeat_front_addr":"172.21.15.139:6831/3047942108","state":["exists","up"]},{"osd":10,"uuid":"32b4d8ce-b644-42fa-80cc-a15a1f5f3a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6825","nonce":4172533828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6827","nonce":4172533828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6831","nonce":4172533828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6829","nonce":4172533828}]},"public_addr":"172.21.15.164:6825/4172533828","cluster_addr":"172.21.15.164:6827/4172533828","heartbeat_back_addr":"172.21.15.164:6831/4172533828","heartbeat_front_addr":"172.21.15.164:6829/4172533828","state":["exists","up"]},{"osd":11,"uuid":"49098bdc-58a9-4a74-8045-d408af7711ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6825","nonce":1854547565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6827","nonce":1854547565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6831","nonce":1854547565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6829","nonce":1854547565}]},"public_addr":"172.21.15.176:6825/1854547565","cluster_addr":"172.21.15.176:6827/1854547565","heartbeat_back_addr":"172.21.15.176:6831/1854547565","heartbeat_front_addr":"172.21.15.176:6829/1854547565","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:27.374845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:52.611853+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:17.229567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:43.330503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:07.988876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:33.787864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:58.864910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:23.815006+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:49.551954+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:15.496773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:41.647557+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:37:07.089355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3772232169":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/3997627612":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/1608930262":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6801/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/677216867":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6801/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/1830135963":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/2608207648":"2024-09-17T01:30:44.751168+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:38:19.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.966+0000 7f96310f1640 1 -- 172.21.15.139:0/1408443393 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f96140a3240 msgr2=0x7f96140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:19.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.966+0000 7f96310f1640 1 --2- 172.21.15.139:0/1408443393 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f96140a3240 0x7f96140a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f962c075f80 tx=0x7f9620008040 comp rx=0 tx=0).stop 2024-09-16T01:38:19.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.966+0000 7f96310f1640 1 -- 172.21.15.139:0/1408443393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f962c101de0 msgr2=0x7f962c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:19.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.966+0000 7f96310f1640 1 --2- 172.21.15.139:0/1408443393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f962c101de0 0x7f962c0744f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f961c005ab0 tx=0x7f961c0042f0 comp rx=0 tx=0).stop 2024-09-16T01:38:19.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.966+0000 7f96310f1640 1 -- 172.21.15.139:0/1408443393 shutdown_connections 2024-09-16T01:38:19.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.966+0000 7f96310f1640 1 --2- 172.21.15.139:0/1408443393 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f96140a3240 0x7f96140a5700 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:19.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.966+0000 7f96310f1640 1 --2- 172.21.15.139:0/1408443393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f962c106f10 0x7f962c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:19.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.966+0000 7f96310f1640 1 --2- 172.21.15.139:0/1408443393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f962c102790 0x7f962c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:19.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.966+0000 7f96310f1640 1 --2- 172.21.15.139:0/1408443393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f962c101de0 0x7f962c0744f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:19.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.966+0000 7f96310f1640 1 -- 172.21.15.139:0/1408443393 >> 172.21.15.139:0/1408443393 conn(0x7f962c0fd8c0 msgr2=0x7f962c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:19.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.966+0000 7f96310f1640 1 -- 172.21.15.139:0/1408443393 shutdown_connections 2024-09-16T01:38:19.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:19.966+0000 7f96310f1640 1 -- 172.21.15.139:0/1408443393 wait complete. 2024-09-16T01:38:20.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T01:38:02.129081+0000', 'modified': '2024-09-16T01:38:11.211378+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24470}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24470': {'gid': 24470, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.164:6839/2447189436', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 2447189436}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 2447189436}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24470, 'qdb_cluster': [24470]} 2024-09-16T01:38:20.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T01:38:20.756 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T01:38:20.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:20 smithi139 ceph-mon[31175]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 58 B/s, 0 objects/s recovering 2024-09-16T01:38:20.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:20 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1408443393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:20.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:20 smithi176 ceph-mon[35324]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 58 B/s, 0 objects/s recovering 2024-09-16T01:38:20.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:20 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1408443393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:21.003 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:21.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:20 smithi164 ceph-mon[35121]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 58 B/s, 0 objects/s recovering 2024-09-16T01:38:21.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:20 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1408443393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:21.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.487+0000 7f81078ea640 1 -- 172.21.15.139:0/4271106146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8100102570 msgr2=0x7f81001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:21.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.487+0000 7f81078ea640 1 --2- 172.21.15.139:0/4271106146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8100102570 0x7f81001029b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009900 tx=0x7f80f002a480 comp rx=0 tx=0).stop 2024-09-16T01:38:21.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.487+0000 7f81078ea640 1 -- 172.21.15.139:0/4271106146 shutdown_connections 2024-09-16T01:38:21.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.487+0000 7f81078ea640 1 --2- 172.21.15.139:0/4271106146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8100102ef0 0x7f810010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.487+0000 7f81078ea640 1 --2- 172.21.15.139:0/4271106146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8100102570 0x7f81001029b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.487+0000 7f81078ea640 1 --2- 172.21.15.139:0/4271106146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8100100e20 0x7f8100101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.487+0000 7f81078ea640 1 -- 172.21.15.139:0/4271106146 >> 172.21.15.139:0/4271106146 conn(0x7f81000fc7d0 msgr2=0x7f81000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:21.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.487+0000 7f81078ea640 1 -- 172.21.15.139:0/4271106146 shutdown_connections 2024-09-16T01:38:21.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.487+0000 7f81078ea640 1 -- 172.21.15.139:0/4271106146 wait complete. 2024-09-16T01:38:21.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.487+0000 7f81078ea640 1 Processor -- start 2024-09-16T01:38:21.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f81078ea640 1 -- start start 2024-09-16T01:38:21.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f81078ea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8100100e20 0x7f810010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f81078ea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8100102570 0x7f810010ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f81078ea640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8100102ef0 0x7f810010ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f81078ea640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8100112120 con 0x7f8100100e20 2024-09-16T01:38:21.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f81078ea640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8100111fa0 con 0x7f8100102570 2024-09-16T01:38:21.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f81078ea640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f81001122a0 con 0x7f8100102ef0 2024-09-16T01:38:21.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f8105686640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8100100e20 0x7f810010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f8105686640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8100100e20 0x7f810010e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57526/0 (socket says 172.21.15.139:57526) 2024-09-16T01:38:21.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f8105686640 1 -- 172.21.15.139:0/3339541816 learned_addr learned my addr 172.21.15.139:0/3339541816 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:21.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f8105686640 1 -- 172.21.15.139:0/3339541816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8100102ef0 msgr2=0x7f810010ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:21.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f8105686640 1 --2- 172.21.15.139:0/3339541816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8100102ef0 0x7f810010ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f8104e85640 1 --2- 172.21.15.139:0/3339541816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8100102570 0x7f810010ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f8105686640 1 -- 172.21.15.139:0/3339541816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8100102570 msgr2=0x7f810010ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:21.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f8105686640 1 --2- 172.21.15.139:0/3339541816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8100102570 0x7f810010ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f8105686640 1 -- 172.21.15.139:0/3339541816 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81001a6b00 con 0x7f8100100e20 2024-09-16T01:38:21.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f8104e85640 1 --2- 172.21.15.139:0/3339541816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8100102570 0x7f810010ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:38:21.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f8105686640 1 --2- 172.21.15.139:0/3339541816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8100100e20 0x7f810010e4d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00b4a0 tx=0x7f80fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:21.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f80f67fc640 1 -- 172.21.15.139:0/3339541816 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc014070 con 0x7f8100100e20 2024-09-16T01:38:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f80f67fc640 1 -- 172.21.15.139:0/3339541816 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80fc010af0 con 0x7f8100100e20 2024-09-16T01:38:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f80f67fc640 1 -- 172.21.15.139:0/3339541816 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc017d10 con 0x7f8100100e20 2024-09-16T01:38:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.488+0000 7f81078ea640 1 -- 172.21.15.139:0/3339541816 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81001a6df0 con 0x7f8100100e20 2024-09-16T01:38:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.489+0000 7f81078ea640 1 -- 172.21.15.139:0/3339541816 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81001a7250 con 0x7f8100100e20 2024-09-16T01:38:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.491+0000 7f80f67fc640 1 -- 172.21.15.139:0/3339541816 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f80fc017070 con 0x7f8100100e20 2024-09-16T01:38:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.491+0000 7f8109cc3740 1 Processor -- start 2024-09-16T01:38:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.491+0000 7f8109cc3740 1 -- start start 2024-09-16T01:38:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8109cc3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398fefddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8109cc3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398fefe2f0 0x56398ff08670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8109cc3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398ff08bb0 0x56398ff0afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8109cc3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56398fe64090 con 0x56398fefd9d0 2024-09-16T01:38:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8109cc3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56398fe0ba70 con 0x56398ff08bb0 2024-09-16T01:38:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8109cc3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56398fe60050 con 0x56398fefe2f0 2024-09-16T01:38:21.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f80f67fc640 1 --2- 172.21.15.139:0/3339541816 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f80ec0a3240 0x7f80ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8105e87640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398fefddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8105e87640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398fefddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57534/0 (socket says 172.21.15.139:57534) 2024-09-16T01:38:21.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8105e87640 1 -- 172.21.15.139:0/4212772029 learned_addr learned my addr 172.21.15.139:0/4212772029 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:21.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8105e87640 1 --2- 172.21.15.139:0/3339541816 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f80ec0a3240 0x7f80ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8104e85640 1 --2- 172.21.15.139:0/4212772029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398ff08bb0 0x56398ff0afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f80f67fc640 1 -- 172.21.15.139:0/3339541816 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f80fc0cf0f0 con 0x7f8100100e20 2024-09-16T01:38:21.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8105686640 1 --2- 172.21.15.139:0/4212772029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398fefe2f0 0x56398ff08670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8105e87640 1 -- 172.21.15.139:0/4212772029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398fefe2f0 msgr2=0x56398ff08670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:38:21.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8105e87640 1 --2- 172.21.15.139:0/4212772029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398fefe2f0 0x56398ff08670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8105e87640 1 -- 172.21.15.139:0/4212772029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398ff08bb0 msgr2=0x56398ff0afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:38:21.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8105e87640 1 --2- 172.21.15.139:0/4212772029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398ff08bb0 0x56398ff0afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8105e87640 1 -- 172.21.15.139:0/4212772029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56398fe18130 con 0x56398fefd9d0 2024-09-16T01:38:21.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.492+0000 7f8105e87640 1 --2- 172.21.15.139:0/4212772029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398fefddb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f80f80127a0 tx=0x7f80f8012c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc406e3be30b5fa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:21.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.493+0000 7f8105e87640 1 --2- 172.21.15.139:0/3339541816 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f80ec0a3240 0x7f80ec0a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f80f80145b0 tx=0x7f80f8019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:21.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.493+0000 7f80eb7fe640 1 -- 172.21.15.139:0/4212772029 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8014c50 con 0x56398fefd9d0 2024-09-16T01:38:21.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.493+0000 7f80eb7fe640 1 -- 172.21.15.139:0/4212772029 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f8014df0 con 0x56398fefd9d0 2024-09-16T01:38:21.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.493+0000 7f80eb7fe640 1 -- 172.21.15.139:0/4212772029 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f80199c0 con 0x56398fefd9d0 2024-09-16T01:38:21.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.493+0000 7f8109cc3740 1 -- 172.21.15.139:0/4212772029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 msgr2=0x56398fefddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:21.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.493+0000 7f8109cc3740 1 --2- 172.21.15.139:0/4212772029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398fefddb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f80f80127a0 tx=0x7f80f8012c60 comp rx=0 tx=0).stop 2024-09-16T01:38:21.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.493+0000 7f8109cc3740 1 -- 172.21.15.139:0/4212772029 shutdown_connections 2024-09-16T01:38:21.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.493+0000 7f8109cc3740 1 --2- 172.21.15.139:0/4212772029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398ff08bb0 0x56398ff0afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.493+0000 7f8109cc3740 1 --2- 172.21.15.139:0/4212772029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398fefe2f0 0x56398ff08670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.493+0000 7f8109cc3740 1 --2- 172.21.15.139:0/4212772029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398fefddb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.493+0000 7f8109cc3740 1 -- 172.21.15.139:0/4212772029 >> 172.21.15.139:0/4212772029 conn(0x56398fef9f60 msgr2=0x56398fefa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:21.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.493+0000 7f8109cc3740 1 -- 172.21.15.139:0/4212772029 shutdown_connections 2024-09-16T01:38:21.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.493+0000 7f8109cc3740 1 -- 172.21.15.139:0/4212772029 wait complete. 2024-09-16T01:38:21.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.494+0000 7f8109cc3740 1 -- start start 2024-09-16T01:38:21.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.494+0000 7f8109cc3740 1 Processor -- start 2024-09-16T01:38:21.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.494+0000 7f8109cc3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398fefd9d0 0x56398ff08490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.494+0000 7f8109cc3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefe2f0 0x56398ff057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.494+0000 7f8109cc3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 0x56398ff05d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.494+0000 7f8109cc3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56398ff0b780 con 0x56398fefe2f0 2024-09-16T01:38:21.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.494+0000 7f8109cc3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56398fe1ef20 con 0x56398fefd9d0 2024-09-16T01:38:21.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.494+0000 7f8109cc3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56398ff0b900 con 0x56398ff08bb0 2024-09-16T01:38:21.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.494+0000 7f8109cc3740 10 client.0 _ll_register_callbacks cb 0x56398fde8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:38:21.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.494+0000 7f8104e85640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefe2f0 0x56398ff057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.494+0000 7f8104e85640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefe2f0 0x56398ff057e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57542/0 (socket says 172.21.15.139:57542) 2024-09-16T01:38:21.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.494+0000 7f8104e85640 1 -- 172.21.15.139:0/2709966071 learned_addr learned my addr 172.21.15.139:0/2709966071 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:21.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f8105e87640 1 --2- 172.21.15.139:0/2709966071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 0x56398ff05d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f8105686640 1 --2- 172.21.15.139:0/2709966071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398fefd9d0 0x56398ff08490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f8104e85640 1 -- 172.21.15.139:0/2709966071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 msgr2=0x56398ff05d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:38:21.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f8104e85640 1 --2- 172.21.15.139:0/2709966071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 0x56398ff05d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f8104e85640 1 -- 172.21.15.139:0/2709966071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398fefd9d0 msgr2=0x56398ff08490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:38:21.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f8104e85640 1 --2- 172.21.15.139:0/2709966071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398fefd9d0 0x56398ff08490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f8104e85640 1 -- 172.21.15.139:0/2709966071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56398fbd52b0 con 0x56398fefe2f0 2024-09-16T01:38:21.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f8105e87640 1 --2- 172.21.15.139:0/2709966071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 0x56398ff05d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:38:21.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f8104e85640 1 --2- 172.21.15.139:0/2709966071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefe2f0 0x56398ff057e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009230 tx=0x7f80f0004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206f4d7ed4d4c6a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:21.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f80e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:38:21.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f80e8ff9640 1 -- 172.21.15.139:0/2709966071 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0035040 con 0x56398fefe2f0 2024-09-16T01:38:21.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f8109cc3740 1 -- 172.21.15.139:0/2709966071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f8014df0 con 0x56398fefe2f0 2024-09-16T01:38:21.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f80e8ff9640 1 -- 172.21.15.139:0/2709966071 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f80f0005590 con 0x56398fefe2f0 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f80e8ff9640 1 -- 172.21.15.139:0/2709966071 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f80f004d050 con 0x56398fefe2f0 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f8109cc3740 10 client.14724 fetch_fsmap learned FSMap version 17 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f8109cc3740 1 -- 172.21.15.139:0/2709966071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x56398ff170f0 con 0x56398fefe2f0 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.495+0000 7f80e8ff9640 1 -- 172.21.15.139:0/2709966071 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f8014df0 con 0x56398fefe2f0 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.496+0000 7f80e8ff9640 1 -- 172.21.15.139:0/2709966071 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f80f0003a40 con 0x56398fefe2f0 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.496+0000 7f8109cc3740 10 client.14724 fetch_fsmap finished waiting for FSMap version 17 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.496+0000 7f8109cc3740 10 client.14724 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.496+0000 7f8109cc3740 20 client.14724 populate_metadata read hostname 'smithi139' 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.496+0000 7f8109cc3740 1 --2- 172.21.15.139:0/2709966071 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56398ff19780 0x56398ff1abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.496+0000 7f8109cc3740 4 client.14724 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.496+0000 7f8109cc3740 1 -- 172.21.15.139:0/2709966071 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56398fe71600 con 0x56398ff19780 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.496+0000 7f8105686640 1 --2- 172.21.15.139:0/2709966071 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56398ff19780 0x56398ff1abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.496+0000 7f80e8ff9640 10 client.14724 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.496+0000 7f8105686640 1 --2- 172.21.15.139:0/2709966071 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56398ff19780 0x56398ff1abf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=933667e964e8e88b server_cookie=81303af2861d71a8 in_seq=0 out_seq=0 2024-09-16T01:38:21.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.497+0000 7f80e8ff9640 1 -- 172.21.15.139:0/2709966071 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56398fe71600 con 0x56398ff19780 2024-09-16T01:38:21.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.497+0000 7f80e8ff9640 10 client.14724 handle_command_reply: tid=0 2024-09-16T01:38:21.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.497+0000 7f8109cc3740 1 client.14724 shutdown 2024-09-16T01:38:21.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.497+0000 7f8109cc3740 1 -- 172.21.15.139:0/2709966071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefe2f0 msgr2=0x56398ff057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:21.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.497+0000 7f8109cc3740 1 --2- 172.21.15.139:0/2709966071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefe2f0 0x56398ff057e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009230 tx=0x7f80f0004580 comp rx=0 tx=0).stop 2024-09-16T01:38:21.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.498+0000 7f8109cc3740 1 -- 172.21.15.139:0/2709966071 shutdown_connections 2024-09-16T01:38:21.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.498+0000 7f8109cc3740 1 --2- 172.21.15.139:0/2709966071 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56398ff19780 0x56398ff1abf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.498+0000 7f8109cc3740 1 --2- 172.21.15.139:0/2709966071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 0x56398ff05d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.498+0000 7f8109cc3740 1 --2- 172.21.15.139:0/2709966071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefe2f0 0x56398ff057e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.498+0000 7f80e8ff9640 0 client.14724 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:38:21.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.498+0000 7f8109cc3740 1 --2- 172.21.15.139:0/2709966071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398fefd9d0 0x56398ff08490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.498+0000 7f8109cc3740 1 -- 172.21.15.139:0/2709966071 >> 172.21.15.139:0/2709966071 conn(0x56398fef9f60 msgr2=0x56398feffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:21.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.498+0000 7f8109cc3740 1 -- 172.21.15.139:0/2709966071 shutdown_connections 2024-09-16T01:38:21.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.498+0000 7f8109cc3740 1 -- 172.21.15.139:0/2709966071 wait complete. 2024-09-16T01:38:21.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.498+0000 7f8109cc3740 20 client.14724 trim_cache size 0 max 16384 2024-09-16T01:38:21.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.515+0000 7f8109cc3740 1 Processor -- start 2024-09-16T01:38:21.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.515+0000 7f8109cc3740 1 -- start start 2024-09-16T01:38:21.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8109cc3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398ff02a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8109cc3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398fefe2f0 0x56398ff02fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8109cc3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 0x56398ff22ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8109cc3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56398ff18f30 con 0x56398fefd9d0 2024-09-16T01:38:21.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8109cc3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56398ff19600 con 0x56398fefe2f0 2024-09-16T01:38:21.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8109cc3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56398ff19cd0 con 0x56398ff08bb0 2024-09-16T01:38:21.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8105686640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398ff02a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8105686640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398ff02a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57544/0 (socket says 172.21.15.139:57544) 2024-09-16T01:38:21.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8105686640 1 -- 172.21.15.139:0/1941120667 learned_addr learned my addr 172.21.15.139:0/1941120667 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:21.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8105686640 1 -- 172.21.15.139:0/1941120667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 msgr2=0x56398ff22ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:38:21.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8105686640 1 --2- 172.21.15.139:0/1941120667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 0x56398ff22ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8105686640 1 -- 172.21.15.139:0/1941120667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398fefe2f0 msgr2=0x56398ff02fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:38:21.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8104e85640 1 --2- 172.21.15.139:0/1941120667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398fefe2f0 0x56398ff02fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8105686640 1 --2- 172.21.15.139:0/1941120667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398fefe2f0 0x56398ff02fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8105686640 1 -- 172.21.15.139:0/1941120667 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56398ff0b890 con 0x56398fefd9d0 2024-09-16T01:38:21.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f8105686640 1 --2- 172.21.15.139:0/1941120667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398ff02a80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f80fc097d80 tx=0x7f80fc012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cacc0eb50afe44 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:21.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f80eaffd640 1 -- 172.21.15.139:0/1941120667 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc01e040 con 0x56398fefd9d0 2024-09-16T01:38:21.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f80eaffd640 1 -- 172.21.15.139:0/1941120667 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80fc0996b0 con 0x56398fefd9d0 2024-09-16T01:38:21.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.516+0000 7f80eaffd640 1 -- 172.21.15.139:0/1941120667 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc09a800 con 0x56398fefd9d0 2024-09-16T01:38:21.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.517+0000 7f8109cc3740 1 -- 172.21.15.139:0/1941120667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 msgr2=0x56398ff02a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:21.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.517+0000 7f8109cc3740 1 --2- 172.21.15.139:0/1941120667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398ff02a80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f80fc097d80 tx=0x7f80fc012040 comp rx=0 tx=0).stop 2024-09-16T01:38:21.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.517+0000 7f8109cc3740 1 -- 172.21.15.139:0/1941120667 shutdown_connections 2024-09-16T01:38:21.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.517+0000 7f8109cc3740 1 --2- 172.21.15.139:0/1941120667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 0x56398ff22ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.517+0000 7f8109cc3740 1 --2- 172.21.15.139:0/1941120667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398fefe2f0 0x56398ff02fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.517+0000 7f8109cc3740 1 --2- 172.21.15.139:0/1941120667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398ff02a80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.517+0000 7f8109cc3740 1 -- 172.21.15.139:0/1941120667 >> 172.21.15.139:0/1941120667 conn(0x56398fef9f60 msgr2=0x56398ff040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:21.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.517+0000 7f8109cc3740 1 -- 172.21.15.139:0/1941120667 shutdown_connections 2024-09-16T01:38:21.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.517+0000 7f8109cc3740 1 -- 172.21.15.139:0/1941120667 wait complete. 2024-09-16T01:38:21.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.517+0000 7f8109cc3740 1 -- start start 2024-09-16T01:38:21.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.517+0000 7f8109cc3740 1 Processor -- start 2024-09-16T01:38:21.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8109cc3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398ff1e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8109cc3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398fefe2f0 0x56398ff1eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8109cc3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 0x56398ff7bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8109cc3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56398ff1aa70 con 0x56398fefd9d0 2024-09-16T01:38:21.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8109cc3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56398ff1a3a0 con 0x56398fefe2f0 2024-09-16T01:38:21.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8109cc3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56398ff1b140 con 0x56398ff08bb0 2024-09-16T01:38:21.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8109cc3740 10 client.0 _ll_register_callbacks cb 0x7f80f0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:38:21.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8105686640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398ff1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8105686640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398ff1e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57556/0 (socket says 172.21.15.139:57556) 2024-09-16T01:38:21.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8105686640 1 -- 172.21.15.139:0/3166122916 learned_addr learned my addr 172.21.15.139:0/3166122916 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:21.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8105e87640 1 --2- 172.21.15.139:0/3166122916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 0x56398ff7bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8105686640 1 -- 172.21.15.139:0/3166122916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 msgr2=0x56398ff7bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:38:21.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8105686640 1 --2- 172.21.15.139:0/3166122916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 0x56398ff7bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8105686640 1 -- 172.21.15.139:0/3166122916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398fefe2f0 msgr2=0x56398ff1eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:38:21.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8105686640 1 --2- 172.21.15.139:0/3166122916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398fefe2f0 0x56398ff1eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8105686640 1 -- 172.21.15.139:0/3166122916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56398ff7c500 con 0x56398fefd9d0 2024-09-16T01:38:21.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8105e87640 1 --2- 172.21.15.139:0/3166122916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 0x56398ff7bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:38:21.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8105686640 1 --2- 172.21.15.139:0/3166122916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398ff1e580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0ccc00 tx=0x7f80fc0a41a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d202643d6dffda2f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:21.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f80e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:38:21.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f80e8ff9640 1 -- 172.21.15.139:0/3166122916 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc0b8070 con 0x56398fefd9d0 2024-09-16T01:38:21.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f80e8ff9640 1 -- 172.21.15.139:0/3166122916 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f80fc0b4270 con 0x56398fefd9d0 2024-09-16T01:38:21.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.518+0000 7f8109cc3740 1 -- 172.21.15.139:0/3166122916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80fc0996b0 con 0x56398fefd9d0 2024-09-16T01:38:21.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.519+0000 7f80e8ff9640 1 -- 172.21.15.139:0/3166122916 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f80fc0b3030 con 0x56398fefd9d0 2024-09-16T01:38:21.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.519+0000 7f8109cc3740 10 client.14730 fetch_fsmap learned FSMap version 17 2024-09-16T01:38:21.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.519+0000 7f8109cc3740 1 -- 172.21.15.139:0/3166122916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x56398ff83870 con 0x56398fefd9d0 2024-09-16T01:38:21.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.519+0000 7f80e8ff9640 1 -- 172.21.15.139:0/3166122916 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80fc0996b0 con 0x56398fefd9d0 2024-09-16T01:38:21.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.519+0000 7f80e8ff9640 1 -- 172.21.15.139:0/3166122916 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f80fc09a800 con 0x56398fefd9d0 2024-09-16T01:38:21.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.519+0000 7f8109cc3740 10 client.14730 fetch_fsmap finished waiting for FSMap version 17 2024-09-16T01:38:21.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.519+0000 7f8109cc3740 10 client.14730 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T01:38:21.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.519+0000 7f8109cc3740 20 client.14730 populate_metadata read hostname 'smithi139' 2024-09-16T01:38:21.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.519+0000 7f8109cc3740 1 --2- 172.21.15.139:0/3166122916 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56398ff87f70 0x56398ff86c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:21.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.519+0000 7f8109cc3740 4 client.14730 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:38:21.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.519+0000 7f8109cc3740 1 -- 172.21.15.139:0/3166122916 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "status"}) -- 0x56398fe6bb10 con 0x56398ff87f70 2024-09-16T01:38:21.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.519+0000 7f8104e85640 1 --2- 172.21.15.139:0/3166122916 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56398ff87f70 0x56398ff86c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:21.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f80e8ff9640 10 client.14730 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:38:21.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f8104e85640 1 --2- 172.21.15.139:0/3166122916 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56398ff87f70 0x56398ff86c80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1510e01fb9918083 server_cookie=30f6d025f3dc5282 in_seq=0 out_seq=0 2024-09-16T01:38:21.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f80e8ff9640 1 -- 172.21.15.139:0/3166122916 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56398fe6bb10 con 0x56398ff87f70 2024-09-16T01:38:21.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f80e8ff9640 10 client.14730 handle_command_reply: tid=0 2024-09-16T01:38:21.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f8109cc3740 1 client.14730 shutdown 2024-09-16T01:38:21.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f8109cc3740 1 -- 172.21.15.139:0/3166122916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 msgr2=0x56398ff1e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:21.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f8109cc3740 1 --2- 172.21.15.139:0/3166122916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398ff1e580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0ccc00 tx=0x7f80fc0a41a0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f8109cc3740 1 -- 172.21.15.139:0/3166122916 shutdown_connections 2024-09-16T01:38:21.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f8109cc3740 1 --2- 172.21.15.139:0/3166122916 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56398ff87f70 0x56398ff86c80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f8109cc3740 1 --2- 172.21.15.139:0/3166122916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56398ff08bb0 0x56398ff7bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f8109cc3740 1 --2- 172.21.15.139:0/3166122916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56398fefe2f0 0x56398ff1eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f8109cc3740 1 --2- 172.21.15.139:0/3166122916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56398fefd9d0 0x56398ff1e580 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f8109cc3740 1 -- 172.21.15.139:0/3166122916 >> 172.21.15.139:0/3166122916 conn(0x56398fef9f60 msgr2=0x56398fef9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:21.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f80e8ff9640 0 client.14730 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:38:21.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f8109cc3740 1 -- 172.21.15.139:0/3166122916 shutdown_connections 2024-09-16T01:38:21.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f8109cc3740 1 -- 172.21.15.139:0/3166122916 wait complete. 2024-09-16T01:38:21.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.520+0000 7f8109cc3740 20 client.14730 trim_cache size 0 max 16384 2024-09-16T01:38:21.529 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:38:21.529 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:38:21.529 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-09-16T01:38:21.529 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24470, 2024-09-16T01:38:21.529 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:38:21.529 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:38:21.529 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:38:21.529 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 19.006775752999999, 2024-09-16T01:38:21.529 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 17, 2024-09-16T01:38:21.530 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 83, 2024-09-16T01:38:21.530 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 83, 2024-09-16T01:38:21.530 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 31.066224363 2024-09-16T01:38:21.530 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:38:21.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.521+0000 7f81078ea640 1 -- 172.21.15.139:0/3339541816 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f80ec0a3240 msgr2=0x7f80ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:21.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.521+0000 7f81078ea640 1 --2- 172.21.15.139:0/3339541816 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f80ec0a3240 0x7f80ec0a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f80f80145b0 tx=0x7f80f8019000 comp rx=0 tx=0).stop 2024-09-16T01:38:21.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.521+0000 7f81078ea640 1 -- 172.21.15.139:0/3339541816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8100100e20 msgr2=0x7f810010e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:21.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.521+0000 7f81078ea640 1 --2- 172.21.15.139:0/3339541816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8100100e20 0x7f810010e4d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00b4a0 tx=0x7f80fc00b960 comp rx=0 tx=0).stop 2024-09-16T01:38:21.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.521+0000 7f81078ea640 1 -- 172.21.15.139:0/3339541816 shutdown_connections 2024-09-16T01:38:21.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.521+0000 7f81078ea640 1 --2- 172.21.15.139:0/3339541816 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f80ec0a3240 0x7f80ec0a5700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.521+0000 7f81078ea640 1 --2- 172.21.15.139:0/3339541816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8100102ef0 0x7f810010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.521+0000 7f81078ea640 1 --2- 172.21.15.139:0/3339541816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8100102570 0x7f810010ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.521+0000 7f81078ea640 1 --2- 172.21.15.139:0/3339541816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8100100e20 0x7f810010e4d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:21.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.521+0000 7f81078ea640 1 -- 172.21.15.139:0/3339541816 >> 172.21.15.139:0/3339541816 conn(0x7f81000fc7d0 msgr2=0x7f81000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:21.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.521+0000 7f81078ea640 1 -- 172.21.15.139:0/3339541816 shutdown_connections 2024-09-16T01:38:21.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:21.521+0000 7f81078ea640 1 -- 172.21.15.139:0/3339541816 wait complete. 2024-09-16T01:38:22.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:21 smithi164 ceph-mon[35121]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 58 B/s, 0 objects/s recovering 2024-09-16T01:38:22.148 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:21 smithi139 ceph-mon[31175]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 58 B/s, 0 objects/s recovering 2024-09-16T01:38:22.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:21 smithi176 ceph-mon[35324]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 58 B/s, 0 objects/s recovering 2024-09-16T01:38:22.230 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T01:38:22.230 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-16T01:38:22.231 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T01:38:22.476 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:22.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.943+0000 7f5f6f78c640 1 -- 172.21.15.139:0/3481631669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f68106f10 msgr2=0x7f5f6810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:22.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.943+0000 7f5f6f78c640 1 --2- 172.21.15.139:0/3481631669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f68106f10 0x7f5f6810ea90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009ca0 tx=0x7f5f6002a8c0 comp rx=0 tx=0).stop 2024-09-16T01:38:22.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.943+0000 7f5f6f78c640 1 -- 172.21.15.139:0/3481631669 shutdown_connections 2024-09-16T01:38:22.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.943+0000 7f5f6f78c640 1 --2- 172.21.15.139:0/3481631669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f68106f10 0x7f5f6810ea90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:22.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.943+0000 7f5f6f78c640 1 --2- 172.21.15.139:0/3481631669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f68102790 0x7f5f681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:22.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.943+0000 7f5f6f78c640 1 --2- 172.21.15.139:0/3481631669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f68101de0 0x7f5f681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:22.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.943+0000 7f5f6f78c640 1 -- 172.21.15.139:0/3481631669 >> 172.21.15.139:0/3481631669 conn(0x7f5f680fd8c0 msgr2=0x7f5f680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:22.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.943+0000 7f5f6f78c640 1 -- 172.21.15.139:0/3481631669 shutdown_connections 2024-09-16T01:38:22.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.943+0000 7f5f6f78c640 1 -- 172.21.15.139:0/3481631669 wait complete. 2024-09-16T01:38:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.944+0000 7f5f6f78c640 1 Processor -- start 2024-09-16T01:38:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.944+0000 7f5f6f78c640 1 -- start start 2024-09-16T01:38:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.944+0000 7f5f6f78c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f68101de0 0x7f5f680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.944+0000 7f5f6f78c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f68102790 0x7f5f68074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.944+0000 7f5f6f78c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f68106f10 0x7f5f68074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.944+0000 7f5f6f78c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68111f70 con 0x7f5f68101de0 2024-09-16T01:38:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.944+0000 7f5f6f78c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68111df0 con 0x7f5f68102790 2024-09-16T01:38:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.944+0000 7f5f6f78c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5f681120f0 con 0x7f5f68106f10 2024-09-16T01:38:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.944+0000 7f5f6d528640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f68101de0 0x7f5f680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.944+0000 7f5f6d528640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f68101de0 0x7f5f680744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47226/0 (socket says 172.21.15.139:47226) 2024-09-16T01:38:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.944+0000 7f5f6d528640 1 -- 172.21.15.139:0/1042698542 learned_addr learned my addr 172.21.15.139:0/1042698542 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.944+0000 7f5f6cd27640 1 --2- 172.21.15.139:0/1042698542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f68102790 0x7f5f68074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.945+0000 7f5f6d528640 1 -- 172.21.15.139:0/1042698542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f68106f10 msgr2=0x7f5f68074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.945+0000 7f5f6d528640 1 --2- 172.21.15.139:0/1042698542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f68106f10 0x7f5f68074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.945+0000 7f5f6d528640 1 -- 172.21.15.139:0/1042698542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f68102790 msgr2=0x7f5f68074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.945+0000 7f5f6d528640 1 --2- 172.21.15.139:0/1042698542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f68102790 0x7f5f68074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.945+0000 7f5f6d528640 1 -- 172.21.15.139:0/1042698542 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f681a67f0 con 0x7f5f68101de0 2024-09-16T01:38:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.945+0000 7f5f6d528640 1 --2- 172.21.15.139:0/1042698542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f68101de0 0x7f5f680744f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400c920 tx=0x7f5f6400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.945+0000 7f5f5e7fc640 1 -- 172.21.15.139:0/1042698542 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64007c70 con 0x7f5f68101de0 2024-09-16T01:38:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.945+0000 7f5f6f78c640 1 -- 172.21.15.139:0/1042698542 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f681a6ae0 con 0x7f5f68101de0 2024-09-16T01:38:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.945+0000 7f5f5e7fc640 1 -- 172.21.15.139:0/1042698542 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f64007e10 con 0x7f5f68101de0 2024-09-16T01:38:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.945+0000 7f5f5e7fc640 1 -- 172.21.15.139:0/1042698542 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64018890 con 0x7f5f68101de0 2024-09-16T01:38:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.945+0000 7f5f6f78c640 1 -- 172.21.15.139:0/1042698542 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f681a7070 con 0x7f5f68101de0 2024-09-16T01:38:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.946+0000 7f5f6f78c640 1 -- 172.21.15.139:0/1042698542 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f681021c0 con 0x7f5f68101de0 2024-09-16T01:38:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.947+0000 7f5f5e7fc640 1 -- 172.21.15.139:0/1042698542 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f5f64018a70 con 0x7f5f68101de0 2024-09-16T01:38:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.948+0000 7f5f5e7fc640 1 --2- 172.21.15.139:0/1042698542 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5f540a3240 0x7f5f540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:22.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.948+0000 7f5f6cd27640 1 --2- 172.21.15.139:0/1042698542 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5f540a3240 0x7f5f540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:22.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.948+0000 7f5f5e7fc640 1 -- 172.21.15.139:0/1042698542 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f5f640cfdc0 con 0x7f5f68101de0 2024-09-16T01:38:22.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.948+0000 7f5f6cd27640 1 --2- 172.21.15.139:0/1042698542 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5f540a3240 0x7f5f540a5700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5f68075f80 tx=0x7f5f5800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:22.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:22.952+0000 7f5f5e7fc640 1 -- 172.21.15.139:0/1042698542 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f64014030 con 0x7f5f68101de0 2024-09-16T01:38:23.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:23.153+0000 7f5f6f78c640 1 -- 172.21.15.139:0/1042698542 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5f6810a0f0 con 0x7f5f68101de0 2024-09-16T01:38:23.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:23.154+0000 7f5f5e7fc640 1 -- 172.21.15.139:0/1042698542 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f5f6409c780 con 0x7f5f68101de0 2024-09-16T01:38:23.155 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-16T01:38:23.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:23.157+0000 7f5f6f78c640 1 -- 172.21.15.139:0/1042698542 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5f540a3240 msgr2=0x7f5f540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:23.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:23.157+0000 7f5f6f78c640 1 --2- 172.21.15.139:0/1042698542 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5f540a3240 0x7f5f540a5700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5f68075f80 tx=0x7f5f5800b390 comp rx=0 tx=0).stop 2024-09-16T01:38:23.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:23.157+0000 7f5f6f78c640 1 -- 172.21.15.139:0/1042698542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f68101de0 msgr2=0x7f5f680744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:23.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:23.157+0000 7f5f6f78c640 1 --2- 172.21.15.139:0/1042698542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f68101de0 0x7f5f680744f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400c920 tx=0x7f5f6400cde0 comp rx=0 tx=0).stop 2024-09-16T01:38:23.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:23.157+0000 7f5f6f78c640 1 -- 172.21.15.139:0/1042698542 shutdown_connections 2024-09-16T01:38:23.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:23.157+0000 7f5f6f78c640 1 --2- 172.21.15.139:0/1042698542 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5f540a3240 0x7f5f540a5700 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:23.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:23.157+0000 7f5f6f78c640 1 --2- 172.21.15.139:0/1042698542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f68106f10 0x7f5f68074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:23.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:23.157+0000 7f5f6f78c640 1 --2- 172.21.15.139:0/1042698542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f68102790 0x7f5f68074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:23.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:23.157+0000 7f5f6f78c640 1 --2- 172.21.15.139:0/1042698542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f68101de0 0x7f5f680744f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:23.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:23.157+0000 7f5f6f78c640 1 -- 172.21.15.139:0/1042698542 >> 172.21.15.139:0/1042698542 conn(0x7f5f680fd8c0 msgr2=0x7f5f680febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:23.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:23.157+0000 7f5f6f78c640 1 -- 172.21.15.139:0/1042698542 shutdown_connections 2024-09-16T01:38:23.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:23.157+0000 7f5f6f78c640 1 -- 172.21.15.139:0/1042698542 wait complete. 2024-09-16T01:38:23.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:23 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1042698542' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T01:38:23.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:23 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1042698542' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T01:38:23.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:23 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1042698542' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T01:38:23.913 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T01:38:23.913 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T01:38:23.913 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-16T01:38:23.914 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-16T01:38:23.914 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-16T01:38:23.914 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T01:38:23.914 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T01:38:23.914 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T01:38:23.914 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-16T01:38:23.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:38:23.914 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-16T01:38:23.930 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T01:38:23.930 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T01:38:23.930 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T01:38:23.930 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-16T01:38:23.930 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-16T01:38:23.930 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-16T01:38:23.931 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T01:38:23.931 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-16T01:38:23.931 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-16T01:38:23.931 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T01:38:23.931 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-16T01:38:23.931 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-16T01:38:23.931 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T01:38:23.931 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-16T01:38:23.931 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T01:38:23.931 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-16T01:38:23.931 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T01:38:23.931 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42315sec preferred_lft 42315sec 2024-09-16T01:38:23.931 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link noprefixroute 2024-09-16T01:38:23.931 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-16T01:38:23.932 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T01:38:23.932 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T01:38:23.932 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T01:38:23.932 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add name ceph-brx type bridge 2024-09-16T01:38:23.932 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr flush dev ceph-brx 2024-09-16T01:38:23.932 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set ceph-brx up 2024-09-16T01:38:23.932 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T01:38:23.932 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T01:38:24.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T01:38:24.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:38:24.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:38:24.131 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T01:38:24.156 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-16T01:38:24.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:38:24.157 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-16T01:38:24.227 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-16T01:38:24.228 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T01:38:24.228 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T01:38:24.228 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T01:38:24.228 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T01:38:24.228 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T01:38:24.228 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T01:38:24.228 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T01:38:24.229 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T01:38:24.229 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T01:38:24.229 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T01:38:24.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T01:38:24.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:24 smithi139 ceph-mon[31175]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 48 B/s, 0 objects/s recovering 2024-09-16T01:38:24.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:24 smithi164 ceph-mon[35121]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 48 B/s, 0 objects/s recovering 2024-09-16T01:38:24.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:38:24.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:38:24.570 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-16T01:38:24.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:38:24.586 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list-id 2024-09-16T01:38:24.641 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T01:38:24.641 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T01:38:24.641 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T01:38:24.642 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-16T01:38:24.642 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T01:38:24.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:24 smithi176 ceph-mon[35324]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 48 B/s, 0 objects/s recovering 2024-09-16T01:38:24.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T01:38:24.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:38:24.760 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-16T01:38:24.760 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T01:38:24.760 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T01:38:24.761 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-16T01:38:24.761 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T01:38:24.761 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-16T01:38:24.761 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-16T01:38:24.761 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-16T01:38:24.761 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T01:38:24.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T01:38:24.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:38:24.886 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T01:38:24.886 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T01:38:24.886 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-16T01:38:24.886 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T01:38:24.886 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T01:38:24.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T01:38:24.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:38:24.962 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-16T01:38:24.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T01:38:24.962 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-16T01:38:24.976 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-16T01:38:24.977 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T01:38:24.977 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-16T01:38:25.028 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-16T01:38:25.093 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T01:38:25.150 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-16T01:38:25.150 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-16T01:38:25.150 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-16T01:38:25.150 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-16T01:38:25.150 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-16T01:38:25.150 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-16T01:38:25.150 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-16T01:38:25.150 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-16T01:38:25.150 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-16T01:38:25.150 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-16T01:38:25.150 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-16T01:38:25.150 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/selinux 2024-09-16T01:38:25.151 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T01:38:25.151 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-16T01:38:25.151 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-16T01:38:25.151 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-16T01:38:25.151 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-16T01:38:25.151 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-16T01:38:25.151 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T01:38:25.151 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T01:38:25.151 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-16T01:38:25.151 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T01:38:25.151 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T01:38:25.151 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-16T01:38:25.151 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-16T01:38:25.151 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay 2024-09-16T01:38:25.152 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/1e9280e3a65823878e27581ed5a0590643df54ad7a940636d4aab9252a313d58/merged 2024-09-16T01:38:25.152 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/e9fd8991c9e5a69c4c7fc1b5b2b67cb0fd5cbd0cbaa50e1e9cf8ced6d0f57a49/merged 2024-09-16T01:38:25.152 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/0 2024-09-16T01:38:25.152 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T01:38:25.152 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/58a79cc941940e8348df3e4d8b88d2ab60b7b5648bbe58f887f583665760d5a7/merged 2024-09-16T01:38:25.152 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/89dfae6c7730e00376fad6076b72a68d19d22b6ad1bedce57cd90844febbfdef/merged 2024-09-16T01:38:25.152 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/8a2f5ea6d665ab4055d1ccdf09d400921adf29d94c8bdf1d14543a8fcaaefc99/merged 2024-09-16T01:38:25.152 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/02afbc4992dd43dc7c179b25f244a0d0521f25227acd85a5611f2cba59982425/merged 2024-09-16T01:38:25.152 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/eb37016d622dddab9f54636a7b364ca825780b1bf0e3c89f8f401d8d69607e06/merged 2024-09-16T01:38:25.153 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/4ece9e41a495eeaab91e330e8a1f0c14aee0602a267212881ed9fc29c312da05/merged 2024-09-16T01:38:25.153 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/4a25460ff8337d21b2c0e0bf631e5f09f80bab243526c7eba5150ebb2edb27b9/merged 2024-09-16T01:38:25.153 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/202f895fab167316532ee4cdd86fc14a2018fa22a5440a501ddd621046cd6080/merged 2024-09-16T01:38:25.153 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-16T01:38:25.153 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T01:38:25.153 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T01:38:25.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:38:25.153 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-16T01:38:25.205 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-16T01:38:25.206 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-16T01:38:25.248 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-16T01:38:25.272 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T01:38:25.328 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-16T01:38:25.328 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-16T01:38:25.328 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-16T01:38:25.329 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-16T01:38:25.329 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-16T01:38:25.329 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-16T01:38:25.329 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-16T01:38:25.329 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-16T01:38:25.329 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-16T01:38:25.329 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-16T01:38:25.329 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-16T01:38:25.329 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/selinux 2024-09-16T01:38:25.329 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T01:38:25.329 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-16T01:38:25.329 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-16T01:38:25.329 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-16T01:38:25.329 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-16T01:38:25.330 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-16T01:38:25.330 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T01:38:25.330 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T01:38:25.330 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-16T01:38:25.330 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T01:38:25.330 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T01:38:25.330 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-16T01:38:25.330 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-16T01:38:25.330 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay 2024-09-16T01:38:25.330 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/1e9280e3a65823878e27581ed5a0590643df54ad7a940636d4aab9252a313d58/merged 2024-09-16T01:38:25.330 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/e9fd8991c9e5a69c4c7fc1b5b2b67cb0fd5cbd0cbaa50e1e9cf8ced6d0f57a49/merged 2024-09-16T01:38:25.330 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/0 2024-09-16T01:38:25.330 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T01:38:25.330 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/58a79cc941940e8348df3e4d8b88d2ab60b7b5648bbe58f887f583665760d5a7/merged 2024-09-16T01:38:25.331 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/89dfae6c7730e00376fad6076b72a68d19d22b6ad1bedce57cd90844febbfdef/merged 2024-09-16T01:38:25.331 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/8a2f5ea6d665ab4055d1ccdf09d400921adf29d94c8bdf1d14543a8fcaaefc99/merged 2024-09-16T01:38:25.331 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/02afbc4992dd43dc7c179b25f244a0d0521f25227acd85a5611f2cba59982425/merged 2024-09-16T01:38:25.331 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/eb37016d622dddab9f54636a7b364ca825780b1bf0e3c89f8f401d8d69607e06/merged 2024-09-16T01:38:25.331 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/4ece9e41a495eeaab91e330e8a1f0c14aee0602a267212881ed9fc29c312da05/merged 2024-09-16T01:38:25.331 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/4a25460ff8337d21b2c0e0bf631e5f09f80bab243526c7eba5150ebb2edb27b9/merged 2024-09-16T01:38:25.331 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/containers/storage/overlay/202f895fab167316532ee4cdd86fc14a2018fa22a5440a501ddd621046cd6080/merged 2024-09-16T01:38:25.331 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-16T01:38:25.331 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T01:38:25.332 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T01:38:25.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:38:25.332 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-16T01:38:25.358 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi139.stderr:2024-09-16T01:38:25.357+0000 7f77719cf480 -1 init, newargv = 0x561eac716a80 newargc=15 2024-09-16T01:38:25.358 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi139.stderr:ceph-fuse[72034]: starting ceph client 2024-09-16T01:38:25.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi139.stderr:ceph-fuse[72034]: starting fuse 2024-09-16T01:38:25.373 INFO:teuthology.orchestra.run.smithi139.stdout:108 2024-09-16T01:38:25.373 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [108] 2024-09-16T01:38:25.373 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-16T01:38:25.373 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-16T01:38:25.373 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-16T01:38:25.373 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-16T01:38:25.373 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-16T01:38:25.374 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-16T01:38:25.375 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-16T01:38:25.375 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-16T01:38:25.375 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-16T01:38:25.375 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-16T01:38:25.375 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-16T01:38:25.375 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.admin")) 2024-09-16T01:38:25.375 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-16T01:38:25.492 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.admin.72034.asok 2024-09-16T01:38:25.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:38:25.503 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.72034.asok 2024-09-16T01:38:25.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:38:25.503 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.72034.asok status 2024-09-16T01:38:25.637 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:38:25.637 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-16T01:38:25.637 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "8921c8ef947807b717db041d85f0761b7914d5fc", 2024-09-16T01:38:25.637 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T01:38:25.637 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "admin", 2024-09-16T01:38:25.637 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-16T01:38:25.637 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-16T01:38:25.637 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "72034", 2024-09-16T01:38:25.637 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-16T01:38:25.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:38:25.637 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-16T01:38:25.638 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-16T01:38:25.638 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 14742, 2024-09-16T01:38:25.638 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-16T01:38:25.638 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-16T01:38:25.638 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-16T01:38:25.638 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 14742 2024-09-16T01:38:25.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:38:25.638 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-16T01:38:25.638 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T01:38:25.638 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-16T01:38:25.638 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1028043851 2024-09-16T01:38:25.638 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:38:25.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:38:25.639 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-16T01:38:25.639 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T01:38:25.639 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-16T01:38:25.639 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1028043851 2024-09-16T01:38:25.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:38:25.639 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.14742 192.168.0.1:0/1028043851", 2024-09-16T01:38:25.639 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/1028043851", 2024-09-16T01:38:25.639 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-16T01:38:25.639 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 17, 2024-09-16T01:38:25.639 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 83, 2024-09-16T01:38:25.639 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-16T01:38:25.639 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-16T01:38:25.639 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-16T01:38:25.640 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:38:25.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:38:25.645 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-16T01:38:25.645 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-16T01:38:25.661 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-16T01:38:25.661 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-16T01:38:25.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:38:25.661 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-16T01:38:25.727 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-16T01:38:25.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T01:38:25.836 INFO:teuthology.orchestra.run.smithi139.stdout:# file: . 2024-09-16T01:38:25.836 INFO:teuthology.orchestra.run.smithi139.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-16T01:38:25.836 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:38:25.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:38:25.842 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:38:25.842 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T01:38:25.859 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T01:38:26.146 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:26.518 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:26 smithi139 ceph-mon[31175]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:26.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.599+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/448244865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c88106f10 msgr2=0x7f6c8810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:26.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.599+0000 7f6c8c9a7640 1 --2- 172.21.15.139:0/448244865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c88106f10 0x7f6c8810ea90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009ca0 tx=0x7f6c8002a8c0 comp rx=0 tx=0).stop 2024-09-16T01:38:26.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.599+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/448244865 shutdown_connections 2024-09-16T01:38:26.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.599+0000 7f6c8c9a7640 1 --2- 172.21.15.139:0/448244865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c88106f10 0x7f6c8810ea90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:26.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.599+0000 7f6c8c9a7640 1 --2- 172.21.15.139:0/448244865 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c88102790 0x7f6c881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:26.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.599+0000 7f6c8c9a7640 1 --2- 172.21.15.139:0/448244865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c88101de0 0x7f6c881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:26.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.599+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/448244865 >> 172.21.15.139:0/448244865 conn(0x7f6c880fd8c0 msgr2=0x7f6c880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:26.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.599+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/448244865 shutdown_connections 2024-09-16T01:38:26.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.599+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/448244865 wait complete. 2024-09-16T01:38:26.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c8c9a7640 1 Processor -- start 2024-09-16T01:38:26.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c8c9a7640 1 -- start start 2024-09-16T01:38:26.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c8c9a7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c88101de0 0x7f6c88198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:26.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c8c9a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c88102790 0x7f6c88198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:26.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c8c9a7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c88106f10 0x7f6c8819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:26.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c8c9a7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88111f70 con 0x7f6c88102790 2024-09-16T01:38:26.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c8c9a7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88111df0 con 0x7f6c88106f10 2024-09-16T01:38:26.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c8c9a7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881120f0 con 0x7f6c88101de0 2024-09-16T01:38:26.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c85d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c88102790 0x7f6c88198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:26.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c85d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c88102790 0x7f6c88198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47252/0 (socket says 172.21.15.139:47252) 2024-09-16T01:38:26.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c85d9b640 1 -- 172.21.15.139:0/3750375237 learned_addr learned my addr 172.21.15.139:0/3750375237 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:26.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c8659c640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c88101de0 0x7f6c88198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:26.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c86d9d640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c88106f10 0x7f6c8819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:26.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c85d9b640 1 -- 172.21.15.139:0/3750375237 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c88101de0 msgr2=0x7f6c88198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:26.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c85d9b640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c88101de0 0x7f6c88198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:26.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c85d9b640 1 -- 172.21.15.139:0/3750375237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c88106f10 msgr2=0x7f6c8819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:26.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c85d9b640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c88106f10 0x7f6c8819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:26.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c85d9b640 1 -- 172.21.15.139:0/3750375237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8819d2c0 con 0x7f6c88102790 2024-09-16T01:38:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.600+0000 7f6c86d9d640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c88106f10 0x7f6c8819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:38:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.601+0000 7f6c85d9b640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c88102790 0x7f6c88198850 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00c920 tx=0x7f6c7c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.601+0000 7f6c777fe640 1 -- 172.21.15.139:0/3750375237 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c007c70 con 0x7f6c88102790 2024-09-16T01:38:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.601+0000 7f6c777fe640 1 -- 172.21.15.139:0/3750375237 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c007e10 con 0x7f6c88102790 2024-09-16T01:38:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.601+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/3750375237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8819d5b0 con 0x7f6c88102790 2024-09-16T01:38:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.601+0000 7f6c777fe640 1 -- 172.21.15.139:0/3750375237 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c018890 con 0x7f6c88102790 2024-09-16T01:38:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.601+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/3750375237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c881a4e90 con 0x7f6c88102790 2024-09-16T01:38:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.602+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/3750375237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c881021c0 con 0x7f6c88102790 2024-09-16T01:38:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.603+0000 7f6c777fe640 1 -- 172.21.15.139:0/3750375237 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6c7c018a50 con 0x7f6c88102790 2024-09-16T01:38:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.604+0000 7f6c777fe640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6c700a3240 0x7f6c700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.604+0000 7f6c8659c640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6c700a3240 0x7f6c700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.604+0000 7f6c777fe640 1 -- 172.21.15.139:0/3750375237 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f6c7c0d0070 con 0x7f6c88102790 2024-09-16T01:38:26.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.604+0000 7f6c8659c640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6c700a3240 0x7f6c700a5700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005e10 tx=0x7f6c78005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:26.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.607+0000 7f6c777fe640 1 -- 172.21.15.139:0/3750375237 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c7c014030 con 0x7f6c88102790 2024-09-16T01:38:26.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.783+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/3750375237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6c88199670 con 0x7f6c88102790 2024-09-16T01:38:26.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.784+0000 7f6c777fe640 1 -- 172.21.15.139:0/3750375237 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) ==== 81+0+601 (secure 0 0 0) 0x7f6c7c01f300 con 0x7f6c88102790 2024-09-16T01:38:26.785 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/3772232169 2024-09-17T01:31:07.344357+0000 2024-09-16T01:38:26.785 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/3997627612 2024-09-17T01:31:07.344357+0000 2024-09-16T01:38:26.785 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/1608930262 2024-09-17T01:31:07.344357+0000 2024-09-16T01:38:26.785 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6801/1485008061 2024-09-17T01:31:07.344357+0000 2024-09-16T01:38:26.785 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6800/1485008061 2024-09-17T01:31:07.344357+0000 2024-09-16T01:38:26.785 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6800/2853057507 2024-09-17T01:30:44.751168+0000 2024-09-16T01:38:26.785 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/677216867 2024-09-17T01:30:44.751168+0000 2024-09-16T01:38:26.785 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6801/2853057507 2024-09-17T01:30:44.751168+0000 2024-09-16T01:38:26.785 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/1830135963 2024-09-17T01:30:44.751168+0000 2024-09-16T01:38:26.786 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/2608207648 2024-09-17T01:30:44.751168+0000 2024-09-16T01:38:26.786 INFO:teuthology.orchestra.run.smithi139.stderr:listed 10 entries 2024-09-16T01:38:26.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.786+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/3750375237 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6c700a3240 msgr2=0x7f6c700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:26.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.786+0000 7f6c8c9a7640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6c700a3240 0x7f6c700a5700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005e10 tx=0x7f6c78005da0 comp rx=0 tx=0).stop 2024-09-16T01:38:26.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.786+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/3750375237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c88102790 msgr2=0x7f6c88198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:26.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.786+0000 7f6c8c9a7640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c88102790 0x7f6c88198850 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00c920 tx=0x7f6c7c00cde0 comp rx=0 tx=0).stop 2024-09-16T01:38:26.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.786+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/3750375237 shutdown_connections 2024-09-16T01:38:26.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.786+0000 7f6c8c9a7640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6c700a3240 0x7f6c700a5700 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:26.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.786+0000 7f6c8c9a7640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c88106f10 0x7f6c8819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:26.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.786+0000 7f6c8c9a7640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c88102790 0x7f6c88198850 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:26.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.786+0000 7f6c8c9a7640 1 --2- 172.21.15.139:0/3750375237 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c88101de0 0x7f6c88198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:26.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.786+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/3750375237 >> 172.21.15.139:0/3750375237 conn(0x7f6c880fd8c0 msgr2=0x7f6c880ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.786+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/3750375237 shutdown_connections 2024-09-16T01:38:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:26.786+0000 7f6c8c9a7640 1 -- 172.21.15.139:0/3750375237 wait complete. 2024-09-16T01:38:26.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:26 smithi164 ceph-mon[35121]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:26.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:26 smithi176 ceph-mon[35324]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:27.498 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:38:27.498 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T01:38:27.516 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T01:38:27.580 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:27 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3750375237' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T01:38:27.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:27 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3750375237' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T01:38:27.805 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:27.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:27 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3750375237' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T01:38:28.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.238+0000 7f5a5fa91640 1 -- 172.21.15.139:0/2707240779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a58101420 msgr2=0x7f5a5810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:28.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.238+0000 7f5a5fa91640 1 --2- 172.21.15.139:0/2707240779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a58101420 0x7f5a5810d950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009cd0 tx=0x7f5a5002a8e0 comp rx=0 tx=0).stop 2024-09-16T01:38:28.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.238+0000 7f5a5fa91640 1 -- 172.21.15.139:0/2707240779 shutdown_connections 2024-09-16T01:38:28.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.238+0000 7f5a5fa91640 1 --2- 172.21.15.139:0/2707240779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a58101420 0x7f5a5810d950 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:28.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.238+0000 7f5a5fa91640 1 --2- 172.21.15.139:0/2707240779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a58100a80 0x7f5a58100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:28.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.238+0000 7f5a5fa91640 1 --2- 172.21.15.139:0/2707240779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a58106a80 0x7f5a58106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:28.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.238+0000 7f5a5fa91640 1 -- 172.21.15.139:0/2707240779 >> 172.21.15.139:0/2707240779 conn(0x7f5a580fc7a0 msgr2=0x7f5a580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:28.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.238+0000 7f5a5fa91640 1 -- 172.21.15.139:0/2707240779 shutdown_connections 2024-09-16T01:38:28.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.238+0000 7f5a5fa91640 1 -- 172.21.15.139:0/2707240779 wait complete. 2024-09-16T01:38:28.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.239+0000 7f5a5fa91640 1 Processor -- start 2024-09-16T01:38:28.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.239+0000 7f5a5fa91640 1 -- start start 2024-09-16T01:38:28.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.239+0000 7f5a5fa91640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a58100a80 0x7f5a5810e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:28.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.239+0000 7f5a5fa91640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a58101420 0x7f5a5810e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:28.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.239+0000 7f5a5fa91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a58106a80 0x7f5a5810ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:28.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.239+0000 7f5a5fa91640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58111fa0 con 0x7f5a58106a80 2024-09-16T01:38:28.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.239+0000 7f5a5fa91640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58111e20 con 0x7f5a58101420 2024-09-16T01:38:28.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.239+0000 7f5a5fa91640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58112120 con 0x7f5a58100a80 2024-09-16T01:38:28.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.239+0000 7f5a5e02e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a58106a80 0x7f5a5810ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:28.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.239+0000 7f5a5e02e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a58106a80 0x7f5a5810ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47256/0 (socket says 172.21.15.139:47256) 2024-09-16T01:38:28.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.239+0000 7f5a5e02e640 1 -- 172.21.15.139:0/3359989646 learned_addr learned my addr 172.21.15.139:0/3359989646 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:28.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.239+0000 7f5a5d02c640 1 --2- 172.21.15.139:0/3359989646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a58101420 0x7f5a5810e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:28.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.239+0000 7f5a5d82d640 1 --2- 172.21.15.139:0/3359989646 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a58100a80 0x7f5a5810e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:28.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.240+0000 7f5a5e02e640 1 -- 172.21.15.139:0/3359989646 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a58100a80 msgr2=0x7f5a5810e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:28.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.240+0000 7f5a5e02e640 1 --2- 172.21.15.139:0/3359989646 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a58100a80 0x7f5a5810e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:28.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.240+0000 7f5a5e02e640 1 -- 172.21.15.139:0/3359989646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a58101420 msgr2=0x7f5a5810e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:28.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.240+0000 7f5a5e02e640 1 --2- 172.21.15.139:0/3359989646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a58101420 0x7f5a5810e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:28.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.240+0000 7f5a5e02e640 1 -- 172.21.15.139:0/3359989646 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a581a6930 con 0x7f5a58106a80 2024-09-16T01:38:28.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.240+0000 7f5a5e02e640 1 --2- 172.21.15.139:0/3359989646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a58106a80 0x7f5a5810ee60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5a50006010 tx=0x7f5a50003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:28.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.240+0000 7f5a4affd640 1 -- 172.21.15.139:0/3359989646 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a50004450 con 0x7f5a58106a80 2024-09-16T01:38:28.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.240+0000 7f5a5fa91640 1 -- 172.21.15.139:0/3359989646 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a581a6bc0 con 0x7f5a58106a80 2024-09-16T01:38:28.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.240+0000 7f5a4affd640 1 -- 172.21.15.139:0/3359989646 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a50003e10 con 0x7f5a58106a80 2024-09-16T01:38:28.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.240+0000 7f5a4affd640 1 -- 172.21.15.139:0/3359989646 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a50046480 con 0x7f5a58106a80 2024-09-16T01:38:28.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.240+0000 7f5a5fa91640 1 -- 172.21.15.139:0/3359989646 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a581a7150 con 0x7f5a58106a80 2024-09-16T01:38:28.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.242+0000 7f5a5fa91640 1 -- 172.21.15.139:0/3359989646 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a58100ee0 con 0x7f5a58106a80 2024-09-16T01:38:28.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.242+0000 7f5a4affd640 1 -- 172.21.15.139:0/3359989646 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f5a500045f0 con 0x7f5a58106a80 2024-09-16T01:38:28.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.243+0000 7f5a4affd640 1 --2- 172.21.15.139:0/3359989646 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5a440a3240 0x7f5a440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:28.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.243+0000 7f5a5d82d640 1 --2- 172.21.15.139:0/3359989646 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5a440a3240 0x7f5a440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:28.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.243+0000 7f5a4affd640 1 -- 172.21.15.139:0/3359989646 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f5a5002e030 con 0x7f5a58106a80 2024-09-16T01:38:28.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.244+0000 7f5a5d82d640 1 --2- 172.21.15.139:0/3359989646 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5a440a3240 0x7f5a440a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0058c0 tx=0x7f5a4c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:28.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.247+0000 7f5a4affd640 1 -- 172.21.15.139:0/3359989646 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a5004b060 con 0x7f5a58106a80 2024-09-16T01:38:28.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.425+0000 7f5a5fa91640 1 -- 172.21.15.139:0/3359989646 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5a58109010 con 0x7f5a58106a80 2024-09-16T01:38:28.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.426+0000 7f5a4affd640 1 -- 172.21.15.139:0/3359989646 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) ==== 81+0+601 (secure 0 0 0) 0x7f5a500bdf00 con 0x7f5a58106a80 2024-09-16T01:38:28.427 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/3772232169 2024-09-17T01:31:07.344357+0000 2024-09-16T01:38:28.427 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/3997627612 2024-09-17T01:31:07.344357+0000 2024-09-16T01:38:28.427 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/1608930262 2024-09-17T01:31:07.344357+0000 2024-09-16T01:38:28.427 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6801/1485008061 2024-09-17T01:31:07.344357+0000 2024-09-16T01:38:28.427 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6800/1485008061 2024-09-17T01:31:07.344357+0000 2024-09-16T01:38:28.427 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6800/2853057507 2024-09-17T01:30:44.751168+0000 2024-09-16T01:38:28.427 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/677216867 2024-09-17T01:30:44.751168+0000 2024-09-16T01:38:28.427 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6801/2853057507 2024-09-17T01:30:44.751168+0000 2024-09-16T01:38:28.428 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/1830135963 2024-09-17T01:30:44.751168+0000 2024-09-16T01:38:28.428 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/2608207648 2024-09-17T01:30:44.751168+0000 2024-09-16T01:38:28.428 INFO:teuthology.orchestra.run.smithi139.stderr:listed 10 entries 2024-09-16T01:38:28.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.428+0000 7f5a5fa91640 1 -- 172.21.15.139:0/3359989646 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5a440a3240 msgr2=0x7f5a440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:28.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.428+0000 7f5a5fa91640 1 --2- 172.21.15.139:0/3359989646 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5a440a3240 0x7f5a440a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0058c0 tx=0x7f5a4c00a2d0 comp rx=0 tx=0).stop 2024-09-16T01:38:28.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.428+0000 7f5a5fa91640 1 -- 172.21.15.139:0/3359989646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a58106a80 msgr2=0x7f5a5810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:28.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.428+0000 7f5a5fa91640 1 --2- 172.21.15.139:0/3359989646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a58106a80 0x7f5a5810ee60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5a50006010 tx=0x7f5a50003c20 comp rx=0 tx=0).stop 2024-09-16T01:38:28.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.428+0000 7f5a5fa91640 1 -- 172.21.15.139:0/3359989646 shutdown_connections 2024-09-16T01:38:28.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.428+0000 7f5a5fa91640 1 --2- 172.21.15.139:0/3359989646 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f5a440a3240 0x7f5a440a5700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:28.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.428+0000 7f5a5fa91640 1 --2- 172.21.15.139:0/3359989646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a58106a80 0x7f5a5810ee60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:28.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.428+0000 7f5a5fa91640 1 --2- 172.21.15.139:0/3359989646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a58101420 0x7f5a5810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:28.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.428+0000 7f5a5fa91640 1 --2- 172.21.15.139:0/3359989646 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a58100a80 0x7f5a5810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:28.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.428+0000 7f5a5fa91640 1 -- 172.21.15.139:0/3359989646 >> 172.21.15.139:0/3359989646 conn(0x7f5a580fc7a0 msgr2=0x7f5a580fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:28.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.428+0000 7f5a5fa91640 1 -- 172.21.15.139:0/3359989646 shutdown_connections 2024-09-16T01:38:28.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:28.428+0000 7f5a5fa91640 1 -- 172.21.15.139:0/3359989646 wait complete. 2024-09-16T01:38:28.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:28 smithi164 ceph-mon[35121]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:28.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:28 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3359989646' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T01:38:28.797 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:28 smithi139 ceph-mon[31175]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:28.797 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:28 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3359989646' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T01:38:28.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:28 smithi176 ceph-mon[35324]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:28.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:28 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3359989646' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T01:38:29.218 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-16T01:38:29.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:38:29.218 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-16T01:38:29.283 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi139.stderr:ceph-fuse[72034]: fuse finished with error 0 and tester_r 0 2024-09-16T01:38:29.285 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T01:38:29.339 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T01:38:29.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:38:29.339 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-16T01:38:29.366 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T01:38:29.655 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.158+0000 7ff060c2d640 1 -- 172.21.15.139:0/1378923232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff05c100a80 msgr2=0x7ff05c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.158+0000 7ff060c2d640 1 --2- 172.21.15.139:0/1378923232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff05c100a80 0x7ff05c100ee0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff044009930 tx=0x7ff04402a420 comp rx=0 tx=0).stop 2024-09-16T01:38:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.158+0000 7ff060c2d640 1 -- 172.21.15.139:0/1378923232 shutdown_connections 2024-09-16T01:38:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.158+0000 7ff060c2d640 1 --2- 172.21.15.139:0/1378923232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff05c101420 0x7ff05c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.158+0000 7ff060c2d640 1 --2- 172.21.15.139:0/1378923232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff05c100a80 0x7ff05c100ee0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.158+0000 7ff060c2d640 1 --2- 172.21.15.139:0/1378923232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff05c106a80 0x7ff05c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.158+0000 7ff060c2d640 1 -- 172.21.15.139:0/1378923232 >> 172.21.15.139:0/1378923232 conn(0x7ff05c0fc7a0 msgr2=0x7ff05c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:30.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.158+0000 7ff060c2d640 1 -- 172.21.15.139:0/1378923232 shutdown_connections 2024-09-16T01:38:30.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.158+0000 7ff060c2d640 1 -- 172.21.15.139:0/1378923232 wait complete. 2024-09-16T01:38:30.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.159+0000 7ff060c2d640 1 Processor -- start 2024-09-16T01:38:30.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.159+0000 7ff060c2d640 1 -- start start 2024-09-16T01:38:30.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.159+0000 7ff060c2d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff05c100a80 0x7ff05c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:30.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.159+0000 7ff060c2d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff05c101420 0x7ff05c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:30.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.159+0000 7ff060c2d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff05c106a80 0x7ff05c1a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:30.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.159+0000 7ff060c2d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c110ea0 con 0x7ff05c101420 2024-09-16T01:38:30.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.159+0000 7ff060c2d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c110d20 con 0x7ff05c106a80 2024-09-16T01:38:30.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.159+0000 7ff060c2d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c111020 con 0x7ff05c100a80 2024-09-16T01:38:30.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.159+0000 7ff053fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff05c101420 0x7ff05c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:30.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.159+0000 7ff053fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff05c101420 0x7ff05c199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47278/0 (socket says 172.21.15.139:47278) 2024-09-16T01:38:30.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.159+0000 7ff053fff640 1 -- 172.21.15.139:0/1263638061 learned_addr learned my addr 172.21.15.139:0/1263638061 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:30.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.160+0000 7ff05a59c640 1 --2- 172.21.15.139:0/1263638061 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff05c100a80 0x7ff05c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:30.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.160+0000 7ff05ad9d640 1 --2- 172.21.15.139:0/1263638061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff05c106a80 0x7ff05c1a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:30.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.160+0000 7ff053fff640 1 -- 172.21.15.139:0/1263638061 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff05c100a80 msgr2=0x7ff05c199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:30.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.160+0000 7ff053fff640 1 --2- 172.21.15.139:0/1263638061 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff05c100a80 0x7ff05c199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:30.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.160+0000 7ff053fff640 1 -- 172.21.15.139:0/1263638061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff05c106a80 msgr2=0x7ff05c1a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:30.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.160+0000 7ff053fff640 1 --2- 172.21.15.139:0/1263638061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff05c106a80 0x7ff05c1a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:30.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.160+0000 7ff053fff640 1 -- 172.21.15.139:0/1263638061 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05c1a4910 con 0x7ff05c101420 2024-09-16T01:38:30.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.160+0000 7ff053fff640 1 --2- 172.21.15.139:0/1263638061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff05c101420 0x7ff05c199890 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff044009230 tx=0x7ff044004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:30.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.160+0000 7ff0537fe640 1 -- 172.21.15.139:0/1263638061 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04403f030 con 0x7ff05c101420 2024-09-16T01:38:30.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.160+0000 7ff0537fe640 1 -- 172.21.15.139:0/1263638061 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff044003ad0 con 0x7ff05c101420 2024-09-16T01:38:30.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.160+0000 7ff060c2d640 1 -- 172.21.15.139:0/1263638061 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05c1a4ae0 con 0x7ff05c101420 2024-09-16T01:38:30.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.160+0000 7ff0537fe640 1 -- 172.21.15.139:0/1263638061 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044035070 con 0x7ff05c101420 2024-09-16T01:38:30.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.160+0000 7ff060c2d640 1 -- 172.21.15.139:0/1263638061 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05c1a4f30 con 0x7ff05c101420 2024-09-16T01:38:30.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.161+0000 7ff060c2d640 1 -- 172.21.15.139:0/1263638061 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff05c100ee0 con 0x7ff05c101420 2024-09-16T01:38:30.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.162+0000 7ff0537fe640 1 -- 172.21.15.139:0/1263638061 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff044042030 con 0x7ff05c101420 2024-09-16T01:38:30.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.163+0000 7ff0537fe640 1 --2- 172.21.15.139:0/1263638061 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff0480a3240 0x7ff0480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:30.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.163+0000 7ff0537fe640 1 -- 172.21.15.139:0/1263638061 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7ff0440efdf0 con 0x7ff05c101420 2024-09-16T01:38:30.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.163+0000 7ff05a59c640 1 --2- 172.21.15.139:0/1263638061 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff0480a3240 0x7ff0480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:30.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.163+0000 7ff05a59c640 1 --2- 172.21.15.139:0/1263638061 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff0480a3240 0x7ff0480a5700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009990 tx=0x7ff04c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:30.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.167+0000 7ff0537fe640 1 -- 172.21.15.139:0/1263638061 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0440bc7b0 con 0x7ff05c101420 2024-09-16T01:38:30.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.337+0000 7ff060c2d640 1 -- 172.21.15.139:0/1263638061 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff05c109010 con 0x7ff05c101420 2024-09-16T01:38:30.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.338+0000 7ff0537fe640 1 -- 172.21.15.139:0/1263638061 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24797 (secure 0 0 0) 0x7ff0440bc0b0 con 0x7ff05c101420 2024-09-16T01:38:30.339 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:38:30.339 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":83,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","created":"2024-09-16T01:30:17.770313+0000","modified":"2024-09-16T01:38:11.211197+0000","last_up_change":"2024-09-16T01:37:08.611901+0000","last_in_change":"2024-09-16T01:36:45.223926+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:33:19.639647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T01:37:57.948214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T01:38:00.263797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T01:38:06.773799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b66696da-ef66-4f25-9527-bbcc9ded2cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6803","nonce":3190810133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6805","nonce":3190810133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6809","nonce":3190810133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6807","nonce":3190810133}]},"public_addr":"172.21.15.139:6803/3190810133","cluster_addr":"172.21.15.139:6805/3190810133","heartbeat_back_addr":"172.21.15.139:6809/3190810133","heartbeat_front_addr":"172.21.15.139:6807/3190810133","state":["exists","up"]},{"osd":1,"uuid":"ccad17f2-0750-425e-9810-49a13a929ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6801","nonce":2699270330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6803","nonce":2699270330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6807","nonce":2699270330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6805","nonce":2699270330}]},"public_addr":"172.21.15.164:6801/2699270330","cluster_addr":"172.21.15.164:6803/2699270330","heartbeat_back_addr":"172.21.15.164:6807/2699270330","heartbeat_front_addr":"172.21.15.164:6805/2699270330","state":["exists","up"]},{"osd":2,"uuid":"baf0ba73-8faf-4e51-a171-c47898800638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6801","nonce":2034471646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6803","nonce":2034471646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6807","nonce":2034471646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6805","nonce":2034471646}]},"public_addr":"172.21.15.176:6801/2034471646","cluster_addr":"172.21.15.176:6803/2034471646","heartbeat_back_addr":"172.21.15.176:6807/2034471646","heartbeat_front_addr":"172.21.15.176:6805/2034471646","state":["exists","up"]},{"osd":3,"uuid":"98957bf1-67bb-4092-9ea4-b263a5706f8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6811","nonce":797147071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6813","nonce":797147071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6817","nonce":797147071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6815","nonce":797147071}]},"public_addr":"172.21.15.139:6811/797147071","cluster_addr":"172.21.15.139:6813/797147071","heartbeat_back_addr":"172.21.15.139:6817/797147071","heartbeat_front_addr":"172.21.15.139:6815/797147071","state":["exists","up"]},{"osd":4,"uuid":"9cd76696-8161-4e82-a571-29e98c11d5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6809","nonce":1421304792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6811","nonce":1421304792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6815","nonce":1421304792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6813","nonce":1421304792}]},"public_addr":"172.21.15.164:6809/1421304792","cluster_addr":"172.21.15.164:6811/1421304792","heartbeat_back_addr":"172.21.15.164:6815/1421304792","heartbeat_front_addr":"172.21.15.164:6813/1421304792","state":["exists","up"]},{"osd":5,"uuid":"ccc17a29-9a58-40ec-b453-5c38bee86f25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6809","nonce":10600602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6811","nonce":10600602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6815","nonce":10600602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6813","nonce":10600602}]},"public_addr":"172.21.15.176:6809/10600602","cluster_addr":"172.21.15.176:6811/10600602","heartbeat_back_addr":"172.21.15.176:6815/10600602","heartbeat_front_addr":"172.21.15.176:6813/10600602","state":["exists","up"]},{"osd":6,"uuid":"e5ee7212-10c6-4a05-992e-622a14a5a6b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6819","nonce":2601111164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6821","nonce":2601111164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6825","nonce":2601111164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6823","nonce":2601111164}]},"public_addr":"172.21.15.139:6819/2601111164","cluster_addr":"172.21.15.139:6821/2601111164","heartbeat_back_addr":"172.21.15.139:6825/2601111164","heartbeat_front_addr":"172.21.15.139:6823/2601111164","state":["exists","up"]},{"osd":7,"uuid":"23143c15-0839-4fd5-9d0a-19ae891000f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6817","nonce":2074080693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6819","nonce":2074080693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6823","nonce":2074080693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6821","nonce":2074080693}]},"public_addr":"172.21.15.164:6817/2074080693","cluster_addr":"172.21.15.164:6819/2074080693","heartbeat_back_addr":"172.21.15.164:6823/2074080693","heartbeat_front_addr":"172.21.15.164:6821/2074080693","state":["exists","up"]},{"osd":8,"uuid":"1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6817","nonce":593133361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6819","nonce":593133361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6823","nonce":593133361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6821","nonce":593133361}]},"public_addr":"172.21.15.176:6817/593133361","cluster_addr":"172.21.15.176:6819/593133361","heartbeat_back_addr":"172.21.15.176:6823/593133361","heartbeat_front_addr":"172.21.15.176:6821/593133361","state":["exists","up"]},{"osd":9,"uuid":"09d25a3b-1e66-480e-9433-ceef7f0b9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6827","nonce":3047942108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6829","nonce":3047942108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6833","nonce":3047942108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6831","nonce":3047942108}]},"public_addr":"172.21.15.139:6827/3047942108","cluster_addr":"172.21.15.139:6829/3047942108","heartbeat_back_addr":"172.21.15.139:6833/3047942108","heartbeat_front_addr":"172.21.15.139:6831/3047942108","state":["exists","up"]},{"osd":10,"uuid":"32b4d8ce-b644-42fa-80cc-a15a1f5f3a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6825","nonce":4172533828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6827","nonce":4172533828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6831","nonce":4172533828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6829","nonce":4172533828}]},"public_addr":"172.21.15.164:6825/4172533828","cluster_addr":"172.21.15.164:6827/4172533828","heartbeat_back_addr":"172.21.15.164:6831/4172533828","heartbeat_front_addr":"172.21.15.164:6829/4172533828","state":["exists","up"]},{"osd":11,"uuid":"49098bdc-58a9-4a74-8045-d408af7711ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6825","nonce":1854547565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6827","nonce":1854547565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6831","nonce":1854547565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6829","nonce":1854547565}]},"public_addr":"172.21.15.176:6825/1854547565","cluster_addr":"172.21.15.176:6827/1854547565","heartbeat_back_addr":"172.21.15.176:6831/1854547565","heartbeat_front_addr":"172.21.15.176:6829/1854547565","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:27.374845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:52.611853+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:17.229567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:43.330503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:07.988876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:33.787864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:58.864910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:23.815006+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:49.551954+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:15.496773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:41.647557+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:37:07.089355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3772232169":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/3997627612":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/1608930262":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6801/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/677216867":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6801/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/1830135963":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/2608207648":"2024-09-17T01:30:44.751168+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:38:30.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.340+0000 7ff060c2d640 1 -- 172.21.15.139:0/1263638061 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff0480a3240 msgr2=0x7ff0480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:30.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.340+0000 7ff060c2d640 1 --2- 172.21.15.139:0/1263638061 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff0480a3240 0x7ff0480a5700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009990 tx=0x7ff04c009110 comp rx=0 tx=0).stop 2024-09-16T01:38:30.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.340+0000 7ff060c2d640 1 -- 172.21.15.139:0/1263638061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff05c101420 msgr2=0x7ff05c199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:30.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.340+0000 7ff060c2d640 1 --2- 172.21.15.139:0/1263638061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff05c101420 0x7ff05c199890 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff044009230 tx=0x7ff044004270 comp rx=0 tx=0).stop 2024-09-16T01:38:30.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.340+0000 7ff060c2d640 1 -- 172.21.15.139:0/1263638061 shutdown_connections 2024-09-16T01:38:30.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.340+0000 7ff060c2d640 1 --2- 172.21.15.139:0/1263638061 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff0480a3240 0x7ff0480a5700 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:30.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.340+0000 7ff060c2d640 1 --2- 172.21.15.139:0/1263638061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff05c106a80 0x7ff05c1a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:30.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.340+0000 7ff060c2d640 1 --2- 172.21.15.139:0/1263638061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff05c101420 0x7ff05c199890 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:30.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.340+0000 7ff060c2d640 1 --2- 172.21.15.139:0/1263638061 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff05c100a80 0x7ff05c199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:30.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.340+0000 7ff060c2d640 1 -- 172.21.15.139:0/1263638061 >> 172.21.15.139:0/1263638061 conn(0x7ff05c0fc7a0 msgr2=0x7ff05c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:30.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.340+0000 7ff060c2d640 1 -- 172.21.15.139:0/1263638061 shutdown_connections 2024-09-16T01:38:30.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:30.340+0000 7ff060c2d640 1 -- 172.21.15.139:0/1263638061 wait complete. 2024-09-16T01:38:30.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:30 smithi139 ceph-mon[31175]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:30.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:30 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1263638061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:30.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:30 smithi164 ceph-mon[35121]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:30.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:30 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1263638061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:30.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:30 smithi176 ceph-mon[35324]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:38:30.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:30 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1263638061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:31.165 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T01:38:31.412 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:31.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.897+0000 7f2daac8c640 1 -- 172.21.15.139:0/3653887972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2da4102ef0 msgr2=0x7f2da410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:31.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.897+0000 7f2daac8c640 1 --2- 172.21.15.139:0/3653887972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2da4102ef0 0x7f2da410da40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009900 tx=0x7f2d9402a480 comp rx=0 tx=0).stop 2024-09-16T01:38:31.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.897+0000 7f2daac8c640 1 -- 172.21.15.139:0/3653887972 shutdown_connections 2024-09-16T01:38:31.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.897+0000 7f2daac8c640 1 --2- 172.21.15.139:0/3653887972 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2da4102ef0 0x7f2da410da40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:31.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.897+0000 7f2daac8c640 1 --2- 172.21.15.139:0/3653887972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2da4102570 0x7f2da41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:31.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.897+0000 7f2daac8c640 1 --2- 172.21.15.139:0/3653887972 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2da4100e20 0x7f2da4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:31.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.897+0000 7f2daac8c640 1 -- 172.21.15.139:0/3653887972 >> 172.21.15.139:0/3653887972 conn(0x7f2da40fc7d0 msgr2=0x7f2da40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:31.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.897+0000 7f2daac8c640 1 -- 172.21.15.139:0/3653887972 shutdown_connections 2024-09-16T01:38:31.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.897+0000 7f2daac8c640 1 -- 172.21.15.139:0/3653887972 wait complete. 2024-09-16T01:38:31.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.897+0000 7f2daac8c640 1 Processor -- start 2024-09-16T01:38:31.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.897+0000 7f2daac8c640 1 -- start start 2024-09-16T01:38:31.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.897+0000 7f2daac8c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2da4100e20 0x7f2da41991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:31.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2daac8c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2da4102570 0x7f2da4199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:31.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2daac8c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2da4102ef0 0x7f2da41a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:31.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2daac8c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2da4110f80 con 0x7f2da4102ef0 2024-09-16T01:38:31.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2daac8c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2da4110e00 con 0x7f2da4102570 2024-09-16T01:38:31.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2daac8c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2da4111100 con 0x7f2da4100e20 2024-09-16T01:38:31.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da9229640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2da4102ef0 0x7f2da41a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:31.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da9229640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2da4102ef0 0x7f2da41a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47294/0 (socket says 172.21.15.139:47294) 2024-09-16T01:38:31.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da9229640 1 -- 172.21.15.139:0/3799203603 learned_addr learned my addr 172.21.15.139:0/3799203603 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:31.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da8a28640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2da4100e20 0x7f2da41991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:31.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da3fff640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2da4102570 0x7f2da4199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da9229640 1 -- 172.21.15.139:0/3799203603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2da4100e20 msgr2=0x7f2da41991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da9229640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2da4100e20 0x7f2da41991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da9229640 1 -- 172.21.15.139:0/3799203603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2da4102570 msgr2=0x7f2da4199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da9229640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2da4102570 0x7f2da4199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da9229640 1 -- 172.21.15.139:0/3799203603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da41a47b0 con 0x7f2da4102ef0 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da3fff640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2da4102570 0x7f2da4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da9229640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2da4102ef0 0x7f2da41a4270 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2d94005ab0 tx=0x7f2d94003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da1ffb640 1 -- 172.21.15.139:0/3799203603 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9403f070 con 0x7f2da4102ef0 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da1ffb640 1 -- 172.21.15.139:0/3799203603 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d94031e30 con 0x7f2da4102ef0 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2daac8c640 1 -- 172.21.15.139:0/3799203603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da41a4980 con 0x7f2da4102ef0 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.898+0000 7f2da1ffb640 1 -- 172.21.15.139:0/3799203603 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94035030 con 0x7f2da4102ef0 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.899+0000 7f2daac8c640 1 -- 172.21.15.139:0/3799203603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da41a4dd0 con 0x7f2da4102ef0 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.900+0000 7f2daac8c640 1 -- 172.21.15.139:0/3799203603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da4101200 con 0x7f2da4102ef0 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.901+0000 7f2da1ffb640 1 -- 172.21.15.139:0/3799203603 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2d94042070 con 0x7f2da4102ef0 2024-09-16T01:38:31.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.901+0000 7f2da1ffb640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d8c0a3240 0x7f2d8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:31.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.901+0000 7f2da8a28640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d8c0a3240 0x7f2d8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:31.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.901+0000 7f2da1ffb640 1 -- 172.21.15.139:0/3799203603 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f2d940f05e0 con 0x7f2da4102ef0 2024-09-16T01:38:31.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.902+0000 7f2da8a28640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d8c0a3240 0x7f2d8c0a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c007350 tx=0x7f2d9c0072e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:31.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:31.905+0000 7f2da1ffb640 1 -- 172.21.15.139:0/3799203603 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d94003e20 con 0x7f2da4102ef0 2024-09-16T01:38:32.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:32.073+0000 7f2daac8c640 1 -- 172.21.15.139:0/3799203603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2da41a5030 con 0x7f2da4102ef0 2024-09-16T01:38:32.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:32.074+0000 7f2da1ffb640 1 -- 172.21.15.139:0/3799203603 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24797 (secure 0 0 0) 0x7f2d940bcfa0 con 0x7f2da4102ef0 2024-09-16T01:38:32.077 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:38:32.077 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":83,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","created":"2024-09-16T01:30:17.770313+0000","modified":"2024-09-16T01:38:11.211197+0000","last_up_change":"2024-09-16T01:37:08.611901+0000","last_in_change":"2024-09-16T01:36:45.223926+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:33:19.639647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T01:37:57.948214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T01:38:00.263797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T01:38:06.773799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b66696da-ef66-4f25-9527-bbcc9ded2cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6803","nonce":3190810133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6805","nonce":3190810133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6809","nonce":3190810133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6807","nonce":3190810133}]},"public_addr":"172.21.15.139:6803/3190810133","cluster_addr":"172.21.15.139:6805/3190810133","heartbeat_back_addr":"172.21.15.139:6809/3190810133","heartbeat_front_addr":"172.21.15.139:6807/3190810133","state":["exists","up"]},{"osd":1,"uuid":"ccad17f2-0750-425e-9810-49a13a929ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6801","nonce":2699270330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6803","nonce":2699270330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6807","nonce":2699270330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6805","nonce":2699270330}]},"public_addr":"172.21.15.164:6801/2699270330","cluster_addr":"172.21.15.164:6803/2699270330","heartbeat_back_addr":"172.21.15.164:6807/2699270330","heartbeat_front_addr":"172.21.15.164:6805/2699270330","state":["exists","up"]},{"osd":2,"uuid":"baf0ba73-8faf-4e51-a171-c47898800638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6801","nonce":2034471646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6803","nonce":2034471646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6807","nonce":2034471646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6805","nonce":2034471646}]},"public_addr":"172.21.15.176:6801/2034471646","cluster_addr":"172.21.15.176:6803/2034471646","heartbeat_back_addr":"172.21.15.176:6807/2034471646","heartbeat_front_addr":"172.21.15.176:6805/2034471646","state":["exists","up"]},{"osd":3,"uuid":"98957bf1-67bb-4092-9ea4-b263a5706f8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6811","nonce":797147071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6813","nonce":797147071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6817","nonce":797147071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6815","nonce":797147071}]},"public_addr":"172.21.15.139:6811/797147071","cluster_addr":"172.21.15.139:6813/797147071","heartbeat_back_addr":"172.21.15.139:6817/797147071","heartbeat_front_addr":"172.21.15.139:6815/797147071","state":["exists","up"]},{"osd":4,"uuid":"9cd76696-8161-4e82-a571-29e98c11d5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6809","nonce":1421304792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6811","nonce":1421304792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6815","nonce":1421304792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6813","nonce":1421304792}]},"public_addr":"172.21.15.164:6809/1421304792","cluster_addr":"172.21.15.164:6811/1421304792","heartbeat_back_addr":"172.21.15.164:6815/1421304792","heartbeat_front_addr":"172.21.15.164:6813/1421304792","state":["exists","up"]},{"osd":5,"uuid":"ccc17a29-9a58-40ec-b453-5c38bee86f25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6809","nonce":10600602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6811","nonce":10600602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6815","nonce":10600602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6813","nonce":10600602}]},"public_addr":"172.21.15.176:6809/10600602","cluster_addr":"172.21.15.176:6811/10600602","heartbeat_back_addr":"172.21.15.176:6815/10600602","heartbeat_front_addr":"172.21.15.176:6813/10600602","state":["exists","up"]},{"osd":6,"uuid":"e5ee7212-10c6-4a05-992e-622a14a5a6b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6819","nonce":2601111164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6821","nonce":2601111164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6825","nonce":2601111164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6823","nonce":2601111164}]},"public_addr":"172.21.15.139:6819/2601111164","cluster_addr":"172.21.15.139:6821/2601111164","heartbeat_back_addr":"172.21.15.139:6825/2601111164","heartbeat_front_addr":"172.21.15.139:6823/2601111164","state":["exists","up"]},{"osd":7,"uuid":"23143c15-0839-4fd5-9d0a-19ae891000f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6817","nonce":2074080693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6819","nonce":2074080693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6823","nonce":2074080693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6821","nonce":2074080693}]},"public_addr":"172.21.15.164:6817/2074080693","cluster_addr":"172.21.15.164:6819/2074080693","heartbeat_back_addr":"172.21.15.164:6823/2074080693","heartbeat_front_addr":"172.21.15.164:6821/2074080693","state":["exists","up"]},{"osd":8,"uuid":"1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6817","nonce":593133361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6819","nonce":593133361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6823","nonce":593133361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6821","nonce":593133361}]},"public_addr":"172.21.15.176:6817/593133361","cluster_addr":"172.21.15.176:6819/593133361","heartbeat_back_addr":"172.21.15.176:6823/593133361","heartbeat_front_addr":"172.21.15.176:6821/593133361","state":["exists","up"]},{"osd":9,"uuid":"09d25a3b-1e66-480e-9433-ceef7f0b9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6827","nonce":3047942108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6829","nonce":3047942108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6833","nonce":3047942108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6831","nonce":3047942108}]},"public_addr":"172.21.15.139:6827/3047942108","cluster_addr":"172.21.15.139:6829/3047942108","heartbeat_back_addr":"172.21.15.139:6833/3047942108","heartbeat_front_addr":"172.21.15.139:6831/3047942108","state":["exists","up"]},{"osd":10,"uuid":"32b4d8ce-b644-42fa-80cc-a15a1f5f3a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6825","nonce":4172533828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6827","nonce":4172533828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6831","nonce":4172533828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6829","nonce":4172533828}]},"public_addr":"172.21.15.164:6825/4172533828","cluster_addr":"172.21.15.164:6827/4172533828","heartbeat_back_addr":"172.21.15.164:6831/4172533828","heartbeat_front_addr":"172.21.15.164:6829/4172533828","state":["exists","up"]},{"osd":11,"uuid":"49098bdc-58a9-4a74-8045-d408af7711ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6825","nonce":1854547565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6827","nonce":1854547565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6831","nonce":1854547565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6829","nonce":1854547565}]},"public_addr":"172.21.15.176:6825/1854547565","cluster_addr":"172.21.15.176:6827/1854547565","heartbeat_back_addr":"172.21.15.176:6831/1854547565","heartbeat_front_addr":"172.21.15.176:6829/1854547565","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:27.374845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:52.611853+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:17.229567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:43.330503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:07.988876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:33.787864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:58.864910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:23.815006+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:49.551954+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:15.496773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:41.647557+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:37:07.089355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3772232169":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/3997627612":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/1608930262":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6801/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/677216867":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6801/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/1830135963":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/2608207648":"2024-09-17T01:30:44.751168+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:38:32.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:32.077+0000 7f2daac8c640 1 -- 172.21.15.139:0/3799203603 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d8c0a3240 msgr2=0x7f2d8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:32.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:32.077+0000 7f2daac8c640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d8c0a3240 0x7f2d8c0a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c007350 tx=0x7f2d9c0072e0 comp rx=0 tx=0).stop 2024-09-16T01:38:32.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:32.077+0000 7f2daac8c640 1 -- 172.21.15.139:0/3799203603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2da4102ef0 msgr2=0x7f2da41a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:32.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:32.077+0000 7f2daac8c640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2da4102ef0 0x7f2da41a4270 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2d94005ab0 tx=0x7f2d94003c30 comp rx=0 tx=0).stop 2024-09-16T01:38:32.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:32.077+0000 7f2daac8c640 1 -- 172.21.15.139:0/3799203603 shutdown_connections 2024-09-16T01:38:32.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:32.077+0000 7f2daac8c640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f2d8c0a3240 0x7f2d8c0a5700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:32.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:32.077+0000 7f2daac8c640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2da4102ef0 0x7f2da41a4270 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:32.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:32.077+0000 7f2daac8c640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2da4102570 0x7f2da4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:32.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:32.077+0000 7f2daac8c640 1 --2- 172.21.15.139:0/3799203603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2da4100e20 0x7f2da41991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:32.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:32.077+0000 7f2daac8c640 1 -- 172.21.15.139:0/3799203603 >> 172.21.15.139:0/3799203603 conn(0x7f2da40fc7d0 msgr2=0x7f2da40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:32.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:32.077+0000 7f2daac8c640 1 -- 172.21.15.139:0/3799203603 shutdown_connections 2024-09-16T01:38:32.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:32.077+0000 7f2daac8c640 1 -- 172.21.15.139:0/3799203603 wait complete. 2024-09-16T01:38:32.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:32 smithi164 ceph-mon[35121]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T01:38:32.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:32 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3799203603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:32.822 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:32 smithi139 ceph-mon[31175]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T01:38:32.822 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:32 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3799203603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:32.851 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-16T01:38:32.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:32 smithi176 ceph-mon[35324]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T01:38:32.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:32 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3799203603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:33.096 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:33.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.624+0000 7f3f98835640 1 -- 172.21.15.139:0/4182610340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f94101420 msgr2=0x7f3f9410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:33.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.624+0000 7f3f98835640 1 --2- 172.21.15.139:0/4182610340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f94101420 0x7f3f9410d950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009930 tx=0x7f3f8402a420 comp rx=0 tx=0).stop 2024-09-16T01:38:33.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.624+0000 7f3f98835640 1 -- 172.21.15.139:0/4182610340 shutdown_connections 2024-09-16T01:38:33.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.624+0000 7f3f98835640 1 --2- 172.21.15.139:0/4182610340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f94101420 0x7f3f9410d950 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:33.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.624+0000 7f3f98835640 1 --2- 172.21.15.139:0/4182610340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f94100a80 0x7f3f94100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:33.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.624+0000 7f3f98835640 1 --2- 172.21.15.139:0/4182610340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f94106a80 0x7f3f94106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:33.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.624+0000 7f3f98835640 1 -- 172.21.15.139:0/4182610340 >> 172.21.15.139:0/4182610340 conn(0x7f3f940fc7a0 msgr2=0x7f3f940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:33.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.624+0000 7f3f98835640 1 -- 172.21.15.139:0/4182610340 shutdown_connections 2024-09-16T01:38:33.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.624+0000 7f3f98835640 1 -- 172.21.15.139:0/4182610340 wait complete. 2024-09-16T01:38:33.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f98835640 1 Processor -- start 2024-09-16T01:38:33.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f98835640 1 -- start start 2024-09-16T01:38:33.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f98835640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f94100a80 0x7f3f941991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:33.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f98835640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f94101420 0x7f3f94199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:33.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f98835640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f94106a80 0x7f3f941a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:33.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f98835640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3f94110e80 con 0x7f3f94100a80 2024-09-16T01:38:33.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f98835640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3f94110d00 con 0x7f3f94101420 2024-09-16T01:38:33.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f98835640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3f94111000 con 0x7f3f94106a80 2024-09-16T01:38:33.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f9259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f94100a80 0x7f3f941991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:33.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f9259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f94100a80 0x7f3f941991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58204/0 (socket says 172.21.15.139:58204) 2024-09-16T01:38:33.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f9259c640 1 -- 172.21.15.139:0/1050330012 learned_addr learned my addr 172.21.15.139:0/1050330012 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:33.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f91d9b640 1 --2- 172.21.15.139:0/1050330012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f94101420 0x7f3f94199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:33.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f92d9d640 1 --2- 172.21.15.139:0/1050330012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f94106a80 0x7f3f941a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:33.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f9259c640 1 -- 172.21.15.139:0/1050330012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f94106a80 msgr2=0x7f3f941a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:33.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f9259c640 1 --2- 172.21.15.139:0/1050330012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f94106a80 0x7f3f941a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:33.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f9259c640 1 -- 172.21.15.139:0/1050330012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f94101420 msgr2=0x7f3f94199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:33.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f9259c640 1 --2- 172.21.15.139:0/1050330012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f94101420 0x7f3f94199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:33.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f9259c640 1 -- 172.21.15.139:0/1050330012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f941a47b0 con 0x7f3f94100a80 2024-09-16T01:38:33.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.625+0000 7f3f9259c640 1 --2- 172.21.15.139:0/1050330012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f94100a80 0x7f3f941991f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00cce0 tx=0x7f3f8c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:33.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.626+0000 7f3f8b7fe640 1 -- 172.21.15.139:0/1050330012 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00f650 con 0x7f3f94100a80 2024-09-16T01:38:33.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.626+0000 7f3f8b7fe640 1 -- 172.21.15.139:0/1050330012 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f8c00fcd0 con 0x7f3f94100a80 2024-09-16T01:38:33.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.626+0000 7f3f98835640 1 -- 172.21.15.139:0/1050330012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f941a4980 con 0x7f3f94100a80 2024-09-16T01:38:33.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.626+0000 7f3f8b7fe640 1 -- 172.21.15.139:0/1050330012 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c018c70 con 0x7f3f94100a80 2024-09-16T01:38:33.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.626+0000 7f3f98835640 1 -- 172.21.15.139:0/1050330012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f941a4e90 con 0x7f3f94100a80 2024-09-16T01:38:33.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.627+0000 7f3f98835640 1 -- 172.21.15.139:0/1050330012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f94100ee0 con 0x7f3f94100a80 2024-09-16T01:38:33.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.628+0000 7f3f8b7fe640 1 -- 172.21.15.139:0/1050330012 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3f8c00f7f0 con 0x7f3f94100a80 2024-09-16T01:38:33.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.629+0000 7f3f8b7fe640 1 --2- 172.21.15.139:0/1050330012 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3f800a3240 0x7f3f800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:33.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.629+0000 7f3f91d9b640 1 --2- 172.21.15.139:0/1050330012 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3f800a3240 0x7f3f800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:33.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.629+0000 7f3f8b7fe640 1 -- 172.21.15.139:0/1050330012 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f3f8c0d02d0 con 0x7f3f94100a80 2024-09-16T01:38:33.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.629+0000 7f3f91d9b640 1 --2- 172.21.15.139:0/1050330012 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3f800a3240 0x7f3f800a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3f9406b2e0 tx=0x7f3f7c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:33.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.632+0000 7f3f8b7fe640 1 -- 172.21.15.139:0/1050330012 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f8c014030 con 0x7f3f94100a80 2024-09-16T01:38:33.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:33.832+0000 7f3f98835640 1 -- 172.21.15.139:0/1050330012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3f941a5150 con 0x7f3f94100a80 2024-09-16T01:38:34.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:34.498+0000 7f3f8b7fe640 1 -- 172.21.15.139:0/1050330012 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f3f8c09cd20 con 0x7f3f94100a80 2024-09-16T01:38:34.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:34.501+0000 7f3f98835640 1 -- 172.21.15.139:0/1050330012 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3f800a3240 msgr2=0x7f3f800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:34.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:34.501+0000 7f3f98835640 1 --2- 172.21.15.139:0/1050330012 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3f800a3240 0x7f3f800a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3f9406b2e0 tx=0x7f3f7c00b390 comp rx=0 tx=0).stop 2024-09-16T01:38:34.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:34.501+0000 7f3f98835640 1 -- 172.21.15.139:0/1050330012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f94100a80 msgr2=0x7f3f941991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:34.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:34.501+0000 7f3f98835640 1 --2- 172.21.15.139:0/1050330012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f94100a80 0x7f3f941991f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00cce0 tx=0x7f3f8c00a620 comp rx=0 tx=0).stop 2024-09-16T01:38:34.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:34.502+0000 7f3f98835640 1 -- 172.21.15.139:0/1050330012 shutdown_connections 2024-09-16T01:38:34.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:34.502+0000 7f3f98835640 1 --2- 172.21.15.139:0/1050330012 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3f800a3240 0x7f3f800a5700 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:34.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:34.502+0000 7f3f98835640 1 --2- 172.21.15.139:0/1050330012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f94106a80 0x7f3f941a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:34.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:34.502+0000 7f3f98835640 1 --2- 172.21.15.139:0/1050330012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f94101420 0x7f3f94199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:34.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:34.502+0000 7f3f98835640 1 --2- 172.21.15.139:0/1050330012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f94100a80 0x7f3f941991f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:34.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:34.502+0000 7f3f98835640 1 -- 172.21.15.139:0/1050330012 >> 172.21.15.139:0/1050330012 conn(0x7f3f940fc7a0 msgr2=0x7f3f940fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:34.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:34.502+0000 7f3f98835640 1 -- 172.21.15.139:0/1050330012 shutdown_connections 2024-09-16T01:38:34.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:34.502+0000 7f3f98835640 1 -- 172.21.15.139:0/1050330012 wait complete. 2024-09-16T01:38:34.691 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:34 smithi139 ceph-mon[31175]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T01:38:34.692 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:34 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1050330012' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T01:38:34.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:34 smithi164 ceph-mon[35121]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T01:38:34.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:34 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1050330012' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T01:38:34.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:34 smithi176 ceph-mon[35324]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T01:38:34.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:34 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1050330012' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T01:38:35.328 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-16T01:38:35.328 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-16T01:38:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:35.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:35 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1050330012' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T01:38:35.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:35 smithi139 ceph-mon[31175]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:35.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:35 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1050330012' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T01:38:35.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:35 smithi164 ceph-mon[35121]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:35.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:35 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1050330012' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T01:38:35.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:35 smithi176 ceph-mon[35324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:36.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.111+0000 7f4e66f61640 1 -- 172.21.15.139:0/1966887957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e60101de0 msgr2=0x7f4e601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:36.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.111+0000 7f4e66f61640 1 --2- 172.21.15.139:0/1966887957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e60101de0 0x7f4e601021c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009900 tx=0x7f4e5402a410 comp rx=0 tx=0).stop 2024-09-16T01:38:36.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.111+0000 7f4e66f61640 1 -- 172.21.15.139:0/1966887957 shutdown_connections 2024-09-16T01:38:36.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.111+0000 7f4e66f61640 1 --2- 172.21.15.139:0/1966887957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60106f10 0x7f4e6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:36.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.111+0000 7f4e66f61640 1 --2- 172.21.15.139:0/1966887957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e60102790 0x7f4e601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:36.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.111+0000 7f4e66f61640 1 --2- 172.21.15.139:0/1966887957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e60101de0 0x7f4e601021c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:36.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.111+0000 7f4e66f61640 1 -- 172.21.15.139:0/1966887957 >> 172.21.15.139:0/1966887957 conn(0x7f4e600fd8c0 msgr2=0x7f4e600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:36.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.111+0000 7f4e66f61640 1 -- 172.21.15.139:0/1966887957 shutdown_connections 2024-09-16T01:38:36.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.111+0000 7f4e66f61640 1 -- 172.21.15.139:0/1966887957 wait complete. 2024-09-16T01:38:36.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.112+0000 7f4e66f61640 1 Processor -- start 2024-09-16T01:38:36.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.112+0000 7f4e66f61640 1 -- start start 2024-09-16T01:38:36.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.112+0000 7f4e66f61640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60101de0 0x7f4e600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:36.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.112+0000 7f4e66f61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e60102790 0x7f4e60074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:36.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.112+0000 7f4e66f61640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e60106f10 0x7f4e60074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:36.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.112+0000 7f4e66f61640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4e60111f70 con 0x7f4e60102790 2024-09-16T01:38:36.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.112+0000 7f4e66f61640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4e60111df0 con 0x7f4e60101de0 2024-09-16T01:38:36.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.112+0000 7f4e66f61640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601120f0 con 0x7f4e60106f10 2024-09-16T01:38:36.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.112+0000 7f4e5ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e60102790 0x7f4e60074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:36.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.112+0000 7f4e5ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e60102790 0x7f4e60074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58222/0 (socket says 172.21.15.139:58222) 2024-09-16T01:38:36.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.112+0000 7f4e5ffff640 1 -- 172.21.15.139:0/2011942488 learned_addr learned my addr 172.21.15.139:0/2011942488 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:36.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.112+0000 7f4e64cfd640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60101de0 0x7f4e600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:36.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.113+0000 7f4e654fe640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e60106f10 0x7f4e60074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:36.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.113+0000 7f4e5ffff640 1 -- 172.21.15.139:0/2011942488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e60106f10 msgr2=0x7f4e60074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:36.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.113+0000 7f4e5ffff640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e60106f10 0x7f4e60074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:36.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.113+0000 7f4e5ffff640 1 -- 172.21.15.139:0/2011942488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60101de0 msgr2=0x7f4e600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:36.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.113+0000 7f4e5ffff640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60101de0 0x7f4e600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:36.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.113+0000 7f4e5ffff640 1 -- 172.21.15.139:0/2011942488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e601a6890 con 0x7f4e60102790 2024-09-16T01:38:36.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.113+0000 7f4e654fe640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e60106f10 0x7f4e60074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:38:36.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.113+0000 7f4e5ffff640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e60102790 0x7f4e60074a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00c920 tx=0x7f4e4c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:36.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.113+0000 7f4e5dffb640 1 -- 172.21.15.139:0/2011942488 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c007c70 con 0x7f4e60102790 2024-09-16T01:38:36.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.113+0000 7f4e5dffb640 1 -- 172.21.15.139:0/2011942488 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e4c007e10 con 0x7f4e60102790 2024-09-16T01:38:36.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.113+0000 7f4e66f61640 1 -- 172.21.15.139:0/2011942488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e601a6b80 con 0x7f4e60102790 2024-09-16T01:38:36.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.113+0000 7f4e5dffb640 1 -- 172.21.15.139:0/2011942488 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c018890 con 0x7f4e60102790 2024-09-16T01:38:36.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.113+0000 7f4e66f61640 1 -- 172.21.15.139:0/2011942488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e601a70c0 con 0x7f4e60102790 2024-09-16T01:38:36.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.114+0000 7f4e66f61640 1 -- 172.21.15.139:0/2011942488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e601021c0 con 0x7f4e60102790 2024-09-16T01:38:36.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.115+0000 7f4e5dffb640 1 -- 172.21.15.139:0/2011942488 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4e4c018a70 con 0x7f4e60102790 2024-09-16T01:38:36.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.116+0000 7f4e5dffb640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4e500a3240 0x7f4e500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:36.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.116+0000 7f4e5dffb640 1 -- 172.21.15.139:0/2011942488 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f4e4c0cfe70 con 0x7f4e60102790 2024-09-16T01:38:36.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.116+0000 7f4e64cfd640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4e500a3240 0x7f4e500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:36.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.116+0000 7f4e64cfd640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4e500a3240 0x7f4e500a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4e54005ab0 tx=0x7f4e54005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:36.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.120+0000 7f4e5dffb640 1 -- 172.21.15.139:0/2011942488 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e4c014030 con 0x7f4e60102790 2024-09-16T01:38:36.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.322+0000 7f4e66f61640 1 -- 172.21.15.139:0/2011942488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f4e60109b60 con 0x7f4e60102790 2024-09-16T01:38:36.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.512+0000 7f4e5dffb640 1 -- 172.21.15.139:0/2011942488 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f4e4c00de20 con 0x7f4e60102790 2024-09-16T01:38:36.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.514+0000 7f4e66f61640 1 -- 172.21.15.139:0/2011942488 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4e500a3240 msgr2=0x7f4e500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:36.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.514+0000 7f4e66f61640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4e500a3240 0x7f4e500a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4e54005ab0 tx=0x7f4e54005a20 comp rx=0 tx=0).stop 2024-09-16T01:38:36.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.514+0000 7f4e66f61640 1 -- 172.21.15.139:0/2011942488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e60102790 msgr2=0x7f4e60074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:36.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.514+0000 7f4e66f61640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e60102790 0x7f4e60074a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00c920 tx=0x7f4e4c00cde0 comp rx=0 tx=0).stop 2024-09-16T01:38:36.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.514+0000 7f4e66f61640 1 -- 172.21.15.139:0/2011942488 shutdown_connections 2024-09-16T01:38:36.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.514+0000 7f4e66f61640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4e500a3240 0x7f4e500a5700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:36.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.514+0000 7f4e66f61640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e60106f10 0x7f4e60074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:36.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.514+0000 7f4e66f61640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e60102790 0x7f4e60074a30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:36.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.514+0000 7f4e66f61640 1 --2- 172.21.15.139:0/2011942488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60101de0 0x7f4e600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:36.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.514+0000 7f4e66f61640 1 -- 172.21.15.139:0/2011942488 >> 172.21.15.139:0/2011942488 conn(0x7f4e600fd8c0 msgr2=0x7f4e600febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:36.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.515+0000 7f4e66f61640 1 -- 172.21.15.139:0/2011942488 shutdown_connections 2024-09-16T01:38:36.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:36.515+0000 7f4e66f61640 1 -- 172.21.15.139:0/2011942488 wait complete. 2024-09-16T01:38:36.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:36 smithi139 ceph-mon[31175]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T01:38:36.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:36 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2011942488' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-16T01:38:36.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:36 smithi164 ceph-mon[35121]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T01:38:36.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:36 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2011942488' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-16T01:38:36.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:36 smithi176 ceph-mon[35324]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T01:38:36.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:36 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2011942488' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-16T01:38:37.305 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-16T01:38:37.559 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:37.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:37 smithi139 ceph-mon[31175]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T01:38:37.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:37 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2011942488' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-16T01:38:37.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:37 smithi139 ceph-mon[31175]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T01:38:37.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:37 smithi139 ceph-mon[31175]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:37.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:37 smithi139 ceph-mon[31175]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-16T01:38:37.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:37 smithi139 ceph-mon[31175]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-16T01:38:37.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:37 smithi164 ceph-mon[35121]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T01:38:37.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:37 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2011942488' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-16T01:38:37.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:37 smithi164 ceph-mon[35121]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T01:38:37.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:37 smithi164 ceph-mon[35121]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:37.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:37 smithi164 ceph-mon[35121]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-16T01:38:37.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:37 smithi164 ceph-mon[35121]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-16T01:38:37.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:37 smithi176 ceph-mon[35324]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T01:38:37.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:37 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2011942488' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-16T01:38:37.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:37 smithi176 ceph-mon[35324]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T01:38:37.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:37 smithi176 ceph-mon[35324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T01:38:37.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:37 smithi176 ceph-mon[35324]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-16T01:38:37.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:37 smithi176 ceph-mon[35324]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-16T01:38:38.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.069+0000 7f0547282640 1 -- 172.21.15.139:0/1144321927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0540106f10 msgr2=0x7f054010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:38.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.069+0000 7f0547282640 1 --2- 172.21.15.139:0/1144321927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0540106f10 0x7f054010ea90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0538009ca0 tx=0x7f053802a8c0 comp rx=0 tx=0).stop 2024-09-16T01:38:38.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.069+0000 7f0547282640 1 -- 172.21.15.139:0/1144321927 shutdown_connections 2024-09-16T01:38:38.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.069+0000 7f0547282640 1 --2- 172.21.15.139:0/1144321927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0540106f10 0x7f054010ea90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:38.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.069+0000 7f0547282640 1 --2- 172.21.15.139:0/1144321927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0540102790 0x7f05401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:38.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.069+0000 7f0547282640 1 --2- 172.21.15.139:0/1144321927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0540101de0 0x7f05401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:38.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.069+0000 7f0547282640 1 -- 172.21.15.139:0/1144321927 >> 172.21.15.139:0/1144321927 conn(0x7f05400fd8c0 msgr2=0x7f05400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:38.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.069+0000 7f0547282640 1 -- 172.21.15.139:0/1144321927 shutdown_connections 2024-09-16T01:38:38.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.069+0000 7f0547282640 1 -- 172.21.15.139:0/1144321927 wait complete. 2024-09-16T01:38:38.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.069+0000 7f0547282640 1 Processor -- start 2024-09-16T01:38:38.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.069+0000 7f0547282640 1 -- start start 2024-09-16T01:38:38.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f0547282640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0540101de0 0x7f05401985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:38.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f0547282640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0540102790 0x7f0540198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:38.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f0547282640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0540106f10 0x7f054019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:38.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f0547282640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0540112010 con 0x7f0540106f10 2024-09-16T01:38:38.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f0547282640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0540111e90 con 0x7f0540102790 2024-09-16T01:38:38.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f0547282640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0540112190 con 0x7f0540101de0 2024-09-16T01:38:38.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f054581f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0540106f10 0x7f054019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:38.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f054581f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0540106f10 0x7f054019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58236/0 (socket says 172.21.15.139:58236) 2024-09-16T01:38:38.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f054581f640 1 -- 172.21.15.139:0/1170869059 learned_addr learned my addr 172.21.15.139:0/1170869059 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:38.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f054501e640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0540101de0 0x7f05401985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:38.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f054481d640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0540102790 0x7f0540198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:38.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f054581f640 1 -- 172.21.15.139:0/1170869059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0540101de0 msgr2=0x7f05401985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:38.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f054581f640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0540101de0 0x7f05401985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:38.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f054581f640 1 -- 172.21.15.139:0/1170869059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0540102790 msgr2=0x7f0540198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:38.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f054581f640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0540102790 0x7f0540198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:38.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f054581f640 1 -- 172.21.15.139:0/1170869059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054019d5a0 con 0x7f0540106f10 2024-09-16T01:38:38.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f054501e640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0540101de0 0x7f05401985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:38:38.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f054481d640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0540102790 0x7f0540198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:38:38.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f054581f640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0540106f10 0x7f054019cec0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0538031a60 tx=0x7f0538031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:38.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f05327fc640 1 -- 172.21.15.139:0/1170869059 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538003c30 con 0x7f0540106f10 2024-09-16T01:38:38.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f05327fc640 1 -- 172.21.15.139:0/1170869059 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0538003dd0 con 0x7f0540106f10 2024-09-16T01:38:38.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f0547282640 1 -- 172.21.15.139:0/1170869059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054019d830 con 0x7f0540106f10 2024-09-16T01:38:38.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f05327fc640 1 -- 172.21.15.139:0/1170869059 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05380047e0 con 0x7f0540106f10 2024-09-16T01:38:38.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.070+0000 7f0547282640 1 -- 172.21.15.139:0/1170869059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05401a5170 con 0x7f0540106f10 2024-09-16T01:38:38.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.071+0000 7f0547282640 1 -- 172.21.15.139:0/1170869059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0540102240 con 0x7f0540106f10 2024-09-16T01:38:38.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.072+0000 7f05327fc640 1 -- 172.21.15.139:0/1170869059 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f053802f790 con 0x7f0540106f10 2024-09-16T01:38:38.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.073+0000 7f05327fc640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f052c0a3240 0x7f052c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:38.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.073+0000 7f05327fc640 1 -- 172.21.15.139:0/1170869059 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f05380f12f0 con 0x7f0540106f10 2024-09-16T01:38:38.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.073+0000 7f054501e640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f052c0a3240 0x7f052c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:38.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.074+0000 7f054501e640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f052c0a3240 0x7f052c0a5700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0534006fd0 tx=0x7f0534008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:38.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.077+0000 7f05327fc640 1 -- 172.21.15.139:0/1170869059 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05380bdcb0 con 0x7f0540106f10 2024-09-16T01:38:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.279+0000 7f0547282640 1 -- 172.21.15.139:0/1170869059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f054010a150 con 0x7f0540106f10 2024-09-16T01:38:38.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.528+0000 7f05327fc640 1 -- 172.21.15.139:0/1170869059 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f05380bd5b0 con 0x7f0540106f10 2024-09-16T01:38:38.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.531+0000 7f0547282640 1 -- 172.21.15.139:0/1170869059 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f052c0a3240 msgr2=0x7f052c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:38.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.531+0000 7f0547282640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f052c0a3240 0x7f052c0a5700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0534006fd0 tx=0x7f0534008040 comp rx=0 tx=0).stop 2024-09-16T01:38:38.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.531+0000 7f0547282640 1 -- 172.21.15.139:0/1170869059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0540106f10 msgr2=0x7f054019cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:38.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.531+0000 7f0547282640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0540106f10 0x7f054019cec0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0538031a60 tx=0x7f0538031a90 comp rx=0 tx=0).stop 2024-09-16T01:38:38.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.532+0000 7f0547282640 1 -- 172.21.15.139:0/1170869059 shutdown_connections 2024-09-16T01:38:38.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.532+0000 7f0547282640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f052c0a3240 0x7f052c0a5700 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:38.534 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:38 smithi139 ceph-mon[31175]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T01:38:38.534 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:38 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] up:active 2024-09-16T01:38:38.534 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:38 smithi139 ceph-mon[31175]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T01:38:38.534 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:38 smithi139 ceph-mon[31175]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-16T01:38:38.534 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:38 smithi139 ceph-mon[31175]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-16T01:38:38.534 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:38 smithi139 ceph-mon[31175]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-16T01:38:38.534 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:38 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1170869059' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T01:38:38.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.532+0000 7f0547282640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0540106f10 0x7f054019cec0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:38.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.532+0000 7f0547282640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0540102790 0x7f0540198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:38.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.532+0000 7f0547282640 1 --2- 172.21.15.139:0/1170869059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0540101de0 0x7f05401985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:38.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.532+0000 7f0547282640 1 -- 172.21.15.139:0/1170869059 >> 172.21.15.139:0/1170869059 conn(0x7f05400fd8c0 msgr2=0x7f05400fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:38.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.532+0000 7f0547282640 1 -- 172.21.15.139:0/1170869059 shutdown_connections 2024-09-16T01:38:38.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:38.532+0000 7f0547282640 1 -- 172.21.15.139:0/1170869059 wait complete. 2024-09-16T01:38:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:38 smithi164 ceph-mon[35121]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T01:38:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:38 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] up:active 2024-09-16T01:38:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:38 smithi164 ceph-mon[35121]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T01:38:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:38 smithi164 ceph-mon[35121]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-16T01:38:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:38 smithi164 ceph-mon[35121]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-16T01:38:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:38 smithi164 ceph-mon[35121]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-16T01:38:38.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:38 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1170869059' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T01:38:38.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:38 smithi176 ceph-mon[35324]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T01:38:38.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:38 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] up:active 2024-09-16T01:38:38.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:38 smithi176 ceph-mon[35324]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T01:38:38.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:38 smithi176 ceph-mon[35324]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-16T01:38:38.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:38 smithi176 ceph-mon[35324]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-16T01:38:38.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:38 smithi176 ceph-mon[35324]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-16T01:38:38.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:38 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1170869059' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T01:38:39.411 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-16T01:38:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:39 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1170869059' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T01:38:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:39 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] up:active 2024-09-16T01:38:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:39 smithi139 ceph-mon[31175]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-16T01:38:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:39 smithi139 ceph-mon[31175]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-16T01:38:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:39 smithi139 ceph-mon[31175]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-16T01:38:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:39 smithi139 ceph-mon[31175]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-16T01:38:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:39 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1170869059' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T01:38:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:39 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] up:active 2024-09-16T01:38:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:39 smithi164 ceph-mon[35121]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-16T01:38:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:39 smithi164 ceph-mon[35121]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-16T01:38:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:39 smithi164 ceph-mon[35121]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-16T01:38:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:39 smithi164 ceph-mon[35121]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-16T01:38:39.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:39 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1170869059' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T01:38:39.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:39 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] up:active 2024-09-16T01:38:39.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:39 smithi176 ceph-mon[35324]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-16T01:38:39.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:39 smithi176 ceph-mon[35324]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-16T01:38:39.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:39 smithi176 ceph-mon[35324]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-16T01:38:39.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:39 smithi176 ceph-mon[35324]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-16T01:38:40.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.154+0000 7f27c3877640 1 -- 172.21.15.139:0/1687706009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27bc100a80 msgr2=0x7f27bc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:40.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.154+0000 7f27c3877640 1 --2- 172.21.15.139:0/1687706009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27bc100a80 0x7f27bc100ee0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009930 tx=0x7f27b002a420 comp rx=0 tx=0).stop 2024-09-16T01:38:40.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.154+0000 7f27c3877640 1 -- 172.21.15.139:0/1687706009 shutdown_connections 2024-09-16T01:38:40.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.154+0000 7f27c3877640 1 --2- 172.21.15.139:0/1687706009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27bc101420 0x7f27bc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:40.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.154+0000 7f27c3877640 1 --2- 172.21.15.139:0/1687706009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27bc100a80 0x7f27bc100ee0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:40.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.154+0000 7f27c3877640 1 --2- 172.21.15.139:0/1687706009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27bc106a80 0x7f27bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:40.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.154+0000 7f27c3877640 1 -- 172.21.15.139:0/1687706009 >> 172.21.15.139:0/1687706009 conn(0x7f27bc0fc7a0 msgr2=0x7f27bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:40.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.154+0000 7f27c3877640 1 -- 172.21.15.139:0/1687706009 shutdown_connections 2024-09-16T01:38:40.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.154+0000 7f27c3877640 1 -- 172.21.15.139:0/1687706009 wait complete. 2024-09-16T01:38:40.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.155+0000 7f27c3877640 1 Processor -- start 2024-09-16T01:38:40.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.155+0000 7f27c3877640 1 -- start start 2024-09-16T01:38:40.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.155+0000 7f27c3877640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27bc100a80 0x7f27bc1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:40.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.155+0000 7f27c3877640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27bc101420 0x7f27bc197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:40.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.155+0000 7f27c3877640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27bc106a80 0x7f27bc19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:40.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.155+0000 7f27c3877640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc110ea0 con 0x7f27bc101420 2024-09-16T01:38:40.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.155+0000 7f27c3877640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc110d20 con 0x7f27bc100a80 2024-09-16T01:38:40.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.155+0000 7f27c3877640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc111020 con 0x7f27bc106a80 2024-09-16T01:38:40.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.155+0000 7f27c0e12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27bc101420 0x7f27bc197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:40.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.155+0000 7f27c0e12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27bc101420 0x7f27bc197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58246/0 (socket says 172.21.15.139:58246) 2024-09-16T01:38:40.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.155+0000 7f27c0e12640 1 -- 172.21.15.139:0/2030087867 learned_addr learned my addr 172.21.15.139:0/2030087867 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:40.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.156+0000 7f27c1613640 1 --2- 172.21.15.139:0/2030087867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27bc100a80 0x7f27bc1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:40.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.156+0000 7f27c1e14640 1 --2- 172.21.15.139:0/2030087867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27bc106a80 0x7f27bc19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:40.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.156+0000 7f27c0e12640 1 -- 172.21.15.139:0/2030087867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27bc106a80 msgr2=0x7f27bc19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:40.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.156+0000 7f27c0e12640 1 --2- 172.21.15.139:0/2030087867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27bc106a80 0x7f27bc19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:40.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.156+0000 7f27c0e12640 1 -- 172.21.15.139:0/2030087867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27bc100a80 msgr2=0x7f27bc1973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:40.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.156+0000 7f27c0e12640 1 --2- 172.21.15.139:0/2030087867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27bc100a80 0x7f27bc1973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:40.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.156+0000 7f27c0e12640 1 -- 172.21.15.139:0/2030087867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc19c390 con 0x7f27bc101420 2024-09-16T01:38:40.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.156+0000 7f27c0e12640 1 --2- 172.21.15.139:0/2030087867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27bc101420 0x7f27bc197920 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009a50 tx=0x7f27b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:40.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.156+0000 7f27ae7fc640 1 -- 172.21.15.139:0/2030087867 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b003f030 con 0x7f27bc101420 2024-09-16T01:38:40.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.156+0000 7f27ae7fc640 1 -- 172.21.15.139:0/2030087867 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b0003ad0 con 0x7f27bc101420 2024-09-16T01:38:40.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.156+0000 7f27c3877640 1 -- 172.21.15.139:0/2030087867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc19c620 con 0x7f27bc101420 2024-09-16T01:38:40.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.156+0000 7f27ae7fc640 1 -- 172.21.15.139:0/2030087867 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0035070 con 0x7f27bc101420 2024-09-16T01:38:40.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.156+0000 7f27c3877640 1 -- 172.21.15.139:0/2030087867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc1a3f60 con 0x7f27bc101420 2024-09-16T01:38:40.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.157+0000 7f27c3877640 1 -- 172.21.15.139:0/2030087867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27bc1021c0 con 0x7f27bc101420 2024-09-16T01:38:40.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.159+0000 7f27ae7fc640 1 -- 172.21.15.139:0/2030087867 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f27b0042030 con 0x7f27bc101420 2024-09-16T01:38:40.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.160+0000 7f27ae7fc640 1 --2- 172.21.15.139:0/2030087867 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f27a80a32c0 0x7f27a80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:40.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.160+0000 7f27c1613640 1 --2- 172.21.15.139:0/2030087867 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f27a80a32c0 0x7f27a80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:40.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.160+0000 7f27ae7fc640 1 -- 172.21.15.139:0/2030087867 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f27b00f07d0 con 0x7f27bc101420 2024-09-16T01:38:40.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.160+0000 7f27c1613640 1 --2- 172.21.15.139:0/2030087867 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f27a80a32c0 0x7f27a80a5780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009990 tx=0x7f27b8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:40.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.163+0000 7f27ae7fc640 1 -- 172.21.15.139:0/2030087867 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27b00bd190 con 0x7f27bc101420 2024-09-16T01:38:40.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.365+0000 7f27c3877640 1 -- 172.21.15.139:0/2030087867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f27bc100ee0 con 0x7f27bc101420 2024-09-16T01:38:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.553+0000 7f27ae7fc640 1 -- 172.21.15.139:0/2030087867 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7f27bc100ee0 con 0x7f27bc101420 2024-09-16T01:38:40.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.556+0000 7f27c3877640 1 -- 172.21.15.139:0/2030087867 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f27a80a32c0 msgr2=0x7f27a80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:40.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.556+0000 7f27c3877640 1 --2- 172.21.15.139:0/2030087867 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f27a80a32c0 0x7f27a80a5780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009990 tx=0x7f27b8009110 comp rx=0 tx=0).stop 2024-09-16T01:38:40.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.556+0000 7f27c3877640 1 -- 172.21.15.139:0/2030087867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27bc101420 msgr2=0x7f27bc197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:40.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.556+0000 7f27c3877640 1 --2- 172.21.15.139:0/2030087867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27bc101420 0x7f27bc197920 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009a50 tx=0x7f27b0004270 comp rx=0 tx=0).stop 2024-09-16T01:38:40.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.556+0000 7f27c3877640 1 -- 172.21.15.139:0/2030087867 shutdown_connections 2024-09-16T01:38:40.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.556+0000 7f27c3877640 1 --2- 172.21.15.139:0/2030087867 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f27a80a32c0 0x7f27a80a5780 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:40.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.556+0000 7f27c3877640 1 --2- 172.21.15.139:0/2030087867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27bc106a80 0x7f27bc19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:40.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.556+0000 7f27c3877640 1 --2- 172.21.15.139:0/2030087867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27bc101420 0x7f27bc197920 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:40.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.556+0000 7f27c3877640 1 --2- 172.21.15.139:0/2030087867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27bc100a80 0x7f27bc1973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:40.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.556+0000 7f27c3877640 1 -- 172.21.15.139:0/2030087867 >> 172.21.15.139:0/2030087867 conn(0x7f27bc0fc7a0 msgr2=0x7f27bc0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:40.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.556+0000 7f27c3877640 1 -- 172.21.15.139:0/2030087867 shutdown_connections 2024-09-16T01:38:40.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:40.556+0000 7f27c3877640 1 -- 172.21.15.139:0/2030087867 wait complete. 2024-09-16T01:38:40.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:40 smithi139 ceph-mon[31175]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-09-16T01:38:40.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:40 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] up:active 2024-09-16T01:38:40.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:40 smithi139 ceph-mon[31175]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-16T01:38:40.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:40 smithi139 ceph-mon[31175]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T01:38:40.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:40 smithi139 ceph-mon[31175]: Cluster is now healthy 2024-09-16T01:38:40.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:40 smithi139 ceph-mon[31175]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-16T01:38:40.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:40 smithi139 ceph-mon[31175]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-16T01:38:40.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:40 smithi139 ceph-mon[31175]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-16T01:38:40.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:40 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2030087867' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T01:38:40.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:40 smithi164 ceph-mon[35121]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-09-16T01:38:40.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:40 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] up:active 2024-09-16T01:38:40.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:40 smithi164 ceph-mon[35121]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-16T01:38:40.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:40 smithi164 ceph-mon[35121]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T01:38:40.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:40 smithi164 ceph-mon[35121]: Cluster is now healthy 2024-09-16T01:38:40.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:40 smithi164 ceph-mon[35121]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-16T01:38:40.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:40 smithi164 ceph-mon[35121]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-16T01:38:40.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:40 smithi164 ceph-mon[35121]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-16T01:38:40.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:40 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2030087867' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T01:38:40.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:40 smithi176 ceph-mon[35324]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-09-16T01:38:40.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:40 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] up:active 2024-09-16T01:38:40.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:40 smithi176 ceph-mon[35324]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-16T01:38:40.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:40 smithi176 ceph-mon[35324]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T01:38:40.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:40 smithi176 ceph-mon[35324]: Cluster is now healthy 2024-09-16T01:38:40.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:40 smithi176 ceph-mon[35324]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-16T01:38:40.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:40 smithi176 ceph-mon[35324]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-16T01:38:40.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:40 smithi176 ceph-mon[35324]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-16T01:38:40.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:40 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2030087867' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T01:38:41.375 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-16T01:38:41.375 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-16T01:38:41.627 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:41.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:41 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2030087867' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T01:38:41.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:41 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] up:active 2024-09-16T01:38:41.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:41 smithi139 ceph-mon[31175]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-16T01:38:41.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:41 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:38:41.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:41 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2030087867' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T01:38:41.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:41 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] up:active 2024-09-16T01:38:41.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:41 smithi164 ceph-mon[35121]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-16T01:38:41.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:41 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:38:41.795 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:38:41 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b[64648]: 2024-09-16T01:38:41.384+0000 7f7d40e8c640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-16T01:38:41.796 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:38:41 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b[64648]: 2024-09-16T01:38:41.384+0000 7f7d40e8c640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-16T01:38:41.796 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:38:41 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b[64648]: 2024-09-16T01:38:41.384+0000 7f7d40e8c640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-16T01:38:41.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:41 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2030087867' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T01:38:41.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:41 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] up:active 2024-09-16T01:38:41.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:41 smithi176 ceph-mon[35324]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-16T01:38:41.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:41 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:38:42.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.162+0000 7f97e4760640 1 -- 172.21.15.139:0/2258501690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc101de0 msgr2=0x7f97dc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:42.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.162+0000 7f97e4760640 1 --2- 172.21.15.139:0/2258501690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc101de0 0x7f97dc1021c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009900 tx=0x7f97d802a410 comp rx=0 tx=0).stop 2024-09-16T01:38:42.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 -- 172.21.15.139:0/2258501690 shutdown_connections 2024-09-16T01:38:42.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 --2- 172.21.15.139:0/2258501690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97dc106f10 0x7f97dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:42.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 --2- 172.21.15.139:0/2258501690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97dc102790 0x7f97dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:42.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 --2- 172.21.15.139:0/2258501690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc101de0 0x7f97dc1021c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:42.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 -- 172.21.15.139:0/2258501690 >> 172.21.15.139:0/2258501690 conn(0x7f97dc0fd8c0 msgr2=0x7f97dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:42.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 -- 172.21.15.139:0/2258501690 shutdown_connections 2024-09-16T01:38:42.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 -- 172.21.15.139:0/2258501690 wait complete. 2024-09-16T01:38:42.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 Processor -- start 2024-09-16T01:38:42.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 -- start start 2024-09-16T01:38:42.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97dc101de0 0x7f97dc1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:42.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc102790 0x7f97dc1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:42.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97dc106f10 0x7f97dc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:42.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc112010 con 0x7f97dc102790 2024-09-16T01:38:42.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc111e90 con 0x7f97dc106f10 2024-09-16T01:38:42.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e4760640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc112190 con 0x7f97dc101de0 2024-09-16T01:38:42.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.163+0000 7f97e1cfb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc102790 0x7f97dc1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:42.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97e1cfb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc102790 0x7f97dc1a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58262/0 (socket says 172.21.15.139:58262) 2024-09-16T01:38:42.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97e1cfb640 1 -- 172.21.15.139:0/13917706 learned_addr learned my addr 172.21.15.139:0/13917706 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:42.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97e24fc640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97dc101de0 0x7f97dc1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:42.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97e1cfb640 1 -- 172.21.15.139:0/13917706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97dc101de0 msgr2=0x7f97dc1a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:42.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97e1cfb640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97dc101de0 0x7f97dc1a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:42.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97e1cfb640 1 -- 172.21.15.139:0/13917706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97dc106f10 msgr2=0x7f97dc19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:42.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97e2cfd640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97dc106f10 0x7f97dc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:42.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97e1cfb640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97dc106f10 0x7f97dc19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:42.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97e1cfb640 1 -- 172.21.15.139:0/13917706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc19ad50 con 0x7f97dc102790 2024-09-16T01:38:42.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97e2cfd640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97dc106f10 0x7f97dc19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:38:42.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97e1cfb640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc102790 0x7f97dc1a09a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00b4a0 tx=0x7f97cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:42.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97d37fe640 1 -- 172.21.15.139:0/13917706 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc014070 con 0x7f97dc102790 2024-09-16T01:38:42.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97d37fe640 1 -- 172.21.15.139:0/13917706 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97cc010af0 con 0x7f97dc102790 2024-09-16T01:38:42.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97d37fe640 1 -- 172.21.15.139:0/13917706 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc017d10 con 0x7f97dc102790 2024-09-16T01:38:42.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97e4760640 1 -- 172.21.15.139:0/13917706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97dc19b040 con 0x7f97dc102790 2024-09-16T01:38:42.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.164+0000 7f97e4760640 1 -- 172.21.15.139:0/13917706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97dc1a7330 con 0x7f97dc102790 2024-09-16T01:38:42.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.165+0000 7f97e4760640 1 -- 172.21.15.139:0/13917706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97dc102240 con 0x7f97dc102790 2024-09-16T01:38:42.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.166+0000 7f97d37fe640 1 -- 172.21.15.139:0/13917706 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f97cc017070 con 0x7f97dc102790 2024-09-16T01:38:42.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.167+0000 7f97d37fe640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97c80a32c0 0x7f97c80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:42.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.167+0000 7f97e24fc640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97c80a32c0 0x7f97c80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:42.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.167+0000 7f97d37fe640 1 -- 172.21.15.139:0/13917706 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f97cc0cef60 con 0x7f97dc102790 2024-09-16T01:38:42.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.168+0000 7f97e24fc640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97c80a32c0 0x7f97c80a5780 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f97d8000c00 tx=0x7f97d80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:42.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.171+0000 7f97d37fe640 1 -- 172.21.15.139:0/13917706 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97cc09b920 con 0x7f97dc102790 2024-09-16T01:38:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.360+0000 7f97e4760640 1 -- 172.21.15.139:0/13917706 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f97dc075430 con 0x7f97c80a32c0 2024-09-16T01:38:42.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.388+0000 7f97d37fe640 1 -- 172.21.15.139:0/13917706 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f97dc075430 con 0x7f97c80a32c0 2024-09-16T01:38:42.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.390+0000 7f97e4760640 1 -- 172.21.15.139:0/13917706 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97c80a32c0 msgr2=0x7f97c80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:42.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.390+0000 7f97e4760640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97c80a32c0 0x7f97c80a5780 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f97d8000c00 tx=0x7f97d80058d0 comp rx=0 tx=0).stop 2024-09-16T01:38:42.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.390+0000 7f97e4760640 1 -- 172.21.15.139:0/13917706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc102790 msgr2=0x7f97dc1a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:42.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.390+0000 7f97e4760640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc102790 0x7f97dc1a09a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00b4a0 tx=0x7f97cc00b960 comp rx=0 tx=0).stop 2024-09-16T01:38:42.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.390+0000 7f97e4760640 1 -- 172.21.15.139:0/13917706 shutdown_connections 2024-09-16T01:38:42.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.390+0000 7f97e4760640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97c80a32c0 0x7f97c80a5780 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:42.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.390+0000 7f97e4760640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97dc106f10 0x7f97dc19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:42.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.390+0000 7f97e4760640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc102790 0x7f97dc1a09a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:42.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.390+0000 7f97e4760640 1 --2- 172.21.15.139:0/13917706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97dc101de0 0x7f97dc1a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:42.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.390+0000 7f97e4760640 1 -- 172.21.15.139:0/13917706 >> 172.21.15.139:0/13917706 conn(0x7f97dc0fd8c0 msgr2=0x7f97dc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:42.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.391+0000 7f97e4760640 1 -- 172.21.15.139:0/13917706 shutdown_connections 2024-09-16T01:38:42.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:42.391+0000 7f97e4760640 1 -- 172.21.15.139:0/13917706 wait complete. 2024-09-16T01:38:42.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:42 smithi139 ceph-mon[31175]: pgmap v300: 129 pgs: 129 active+clean; 586 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.8 KiB/s wr, 9 op/s 2024-09-16T01:38:42.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:38:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:38:42.365+0000 7f173e3f5640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:38:42.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:38:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:38:42.365+0000 7f173e3f5640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:38:42.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:38:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:38:42.365+0000 7f173e3f5640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:38:42.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:38:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:38:42.365+0000 7f173e3f5640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:38:42.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:38:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:38:42.365+0000 7f173e3f5640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:38:42.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:42 smithi176 ceph-mon[35324]: pgmap v300: 129 pgs: 129 active+clean; 586 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.8 KiB/s wr, 9 op/s 2024-09-16T01:38:43.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:42 smithi164 ceph-mon[35121]: pgmap v300: 129 pgs: 129 active+clean; 586 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.8 KiB/s wr, 9 op/s 2024-09-16T01:38:43.084 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-16T01:38:43.329 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:43.709 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:43 smithi139 ceph-mon[31175]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:43.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.792+0000 7f4bb81d5640 1 -- 172.21.15.139:0/3585706547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01082a0 msgr2=0x7f4bb010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:43.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.792+0000 7f4bb81d5640 1 --2- 172.21.15.139:0/3585706547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01082a0 0x7f4bb010ec40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009930 tx=0x7f4ba802a440 comp rx=0 tx=0).stop 2024-09-16T01:38:43.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.792+0000 7f4bb81d5640 1 -- 172.21.15.139:0/3585706547 shutdown_connections 2024-09-16T01:38:43.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.792+0000 7f4bb81d5640 1 --2- 172.21.15.139:0/3585706547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01082a0 0x7f4bb010ec40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:43.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.792+0000 7f4bb81d5640 1 --2- 172.21.15.139:0/3585706547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4bb01036e0 0x7f4bb0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:43.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.792+0000 7f4bb81d5640 1 --2- 172.21.15.139:0/3585706547 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bb0102d30 0x7f4bb0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:43.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.793+0000 7f4bb81d5640 1 -- 172.21.15.139:0/3585706547 >> 172.21.15.139:0/3585706547 conn(0x7f4bb00fc7d0 msgr2=0x7f4bb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:43.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.793+0000 7f4bb81d5640 1 -- 172.21.15.139:0/3585706547 shutdown_connections 2024-09-16T01:38:43.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.793+0000 7f4bb81d5640 1 -- 172.21.15.139:0/3585706547 wait complete. 2024-09-16T01:38:43.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.793+0000 7f4bb81d5640 1 Processor -- start 2024-09-16T01:38:43.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.793+0000 7f4bb81d5640 1 -- start start 2024-09-16T01:38:43.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.793+0000 7f4bb81d5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4bb0102d30 0x7f4bb01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:43.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.793+0000 7f4bb81d5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bb01036e0 0x7f4bb0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:43.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.793+0000 7f4bb81d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01082a0 0x7f4bb019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:43.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.793+0000 7f4bb81d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb0112100 con 0x7f4bb01082a0 2024-09-16T01:38:43.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.793+0000 7f4bb81d5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb0111f80 con 0x7f4bb0102d30 2024-09-16T01:38:43.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.793+0000 7f4bb81d5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb0112280 con 0x7f4bb01036e0 2024-09-16T01:38:43.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4bb6772640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01082a0 0x7f4bb019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:43.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4bb6772640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01082a0 0x7f4bb019ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55912/0 (socket says 172.21.15.139:55912) 2024-09-16T01:38:43.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4bb6772640 1 -- 172.21.15.139:0/2644764439 learned_addr learned my addr 172.21.15.139:0/2644764439 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:43.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4bb5f71640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4bb0102d30 0x7f4bb01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:43.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4bb5770640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bb01036e0 0x7f4bb0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:43.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4bb6772640 1 -- 172.21.15.139:0/2644764439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bb01036e0 msgr2=0x7f4bb0198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:43.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4bb6772640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bb01036e0 0x7f4bb0198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:43.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4bb6772640 1 -- 172.21.15.139:0/2644764439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4bb0102d30 msgr2=0x7f4bb01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:43.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4bb6772640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4bb0102d30 0x7f4bb01985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:43.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4bb6772640 1 -- 172.21.15.139:0/2644764439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb019d550 con 0x7f4bb01082a0 2024-09-16T01:38:43.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4bb5f71640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4bb0102d30 0x7f4bb01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:38:43.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4bb6772640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01082a0 0x7f4bb019ce70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8005ab0 tx=0x7f4ba80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:43.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4baeffd640 1 -- 172.21.15.139:0/2644764439 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba803f030 con 0x7f4bb01082a0 2024-09-16T01:38:43.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4baeffd640 1 -- 172.21.15.139:0/2644764439 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ba8003ce0 con 0x7f4bb01082a0 2024-09-16T01:38:43.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4baeffd640 1 -- 172.21.15.139:0/2644764439 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8035070 con 0x7f4bb01082a0 2024-09-16T01:38:43.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4bb81d5640 1 -- 172.21.15.139:0/2644764439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb019d7e0 con 0x7f4bb01082a0 2024-09-16T01:38:43.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.794+0000 7f4bb81d5640 1 -- 172.21.15.139:0/2644764439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb01a5120 con 0x7f4bb01082a0 2024-09-16T01:38:43.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.795+0000 7f4bb81d5640 1 -- 172.21.15.139:0/2644764439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb0103110 con 0x7f4bb01082a0 2024-09-16T01:38:43.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.796+0000 7f4baeffd640 1 -- 172.21.15.139:0/2644764439 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4ba8042030 con 0x7f4bb01082a0 2024-09-16T01:38:43.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.797+0000 7f4baeffd640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4b980a3350 0x7f4b980a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:43.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.797+0000 7f4bb5f71640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4b980a3350 0x7f4b980a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:43.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.797+0000 7f4baeffd640 1 -- 172.21.15.139:0/2644764439 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f4ba80f0300 con 0x7f4bb01082a0 2024-09-16T01:38:43.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.797+0000 7f4bb5f71640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4b980a3350 0x7f4b980a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0005c40 tx=0x7f4ba000a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:43.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.801+0000 7f4baeffd640 1 -- 172.21.15.139:0/2644764439 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ba80bcc00 con 0x7f4bb01082a0 2024-09-16T01:38:43.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:43 smithi176 ceph-mon[35324]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:43.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:43.996+0000 7f4bb81d5640 1 -- 172.21.15.139:0/2644764439 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4bb0107420 con 0x7f4b980a3350 2024-09-16T01:38:44.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:44.021+0000 7f4baeffd640 1 -- 172.21.15.139:0/2644764439 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4bb0107420 con 0x7f4b980a3350 2024-09-16T01:38:44.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:44.023+0000 7f4bb81d5640 1 -- 172.21.15.139:0/2644764439 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4b980a3350 msgr2=0x7f4b980a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:44.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:44.023+0000 7f4bb81d5640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4b980a3350 0x7f4b980a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0005c40 tx=0x7f4ba000a410 comp rx=0 tx=0).stop 2024-09-16T01:38:44.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:44.023+0000 7f4bb81d5640 1 -- 172.21.15.139:0/2644764439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01082a0 msgr2=0x7f4bb019ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:44.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:44.023+0000 7f4bb81d5640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01082a0 0x7f4bb019ce70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8005ab0 tx=0x7f4ba80042f0 comp rx=0 tx=0).stop 2024-09-16T01:38:44.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:44.024+0000 7f4bb81d5640 1 -- 172.21.15.139:0/2644764439 shutdown_connections 2024-09-16T01:38:44.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:44.024+0000 7f4bb81d5640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4b980a3350 0x7f4b980a5810 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:44.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:44.024+0000 7f4bb81d5640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bb01082a0 0x7f4bb019ce70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:44.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:44.024+0000 7f4bb81d5640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bb01036e0 0x7f4bb0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:44.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:44.024+0000 7f4bb81d5640 1 --2- 172.21.15.139:0/2644764439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4bb0102d30 0x7f4bb01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:44.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:44.024+0000 7f4bb81d5640 1 -- 172.21.15.139:0/2644764439 >> 172.21.15.139:0/2644764439 conn(0x7f4bb00fc7d0 msgr2=0x7f4bb0105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:44.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:44.024+0000 7f4bb81d5640 1 -- 172.21.15.139:0/2644764439 shutdown_connections 2024-09-16T01:38:44.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:44.024+0000 7f4bb81d5640 1 -- 172.21.15.139:0/2644764439 wait complete. 2024-09-16T01:38:44.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:43 smithi164 ceph-mon[35121]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:44.832 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:44 smithi139 ceph-mon[31175]: pgmap v301: 129 pgs: 129 active+clean; 586 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-09-16T01:38:44.832 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:44 smithi139 ceph-mon[31175]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T01:38:44.864 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-16T01:38:44.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:44 smithi176 ceph-mon[35324]: pgmap v301: 129 pgs: 129 active+clean; 586 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-09-16T01:38:44.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:44 smithi176 ceph-mon[35324]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T01:38:45.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:44 smithi164 ceph-mon[35121]: pgmap v301: 129 pgs: 129 active+clean; 586 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-09-16T01:38:45.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:44 smithi164 ceph-mon[35121]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T01:38:45.113 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:45.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.611+0000 7f8b6e10d640 1 -- 172.21.15.139:0/397846697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b68108290 msgr2=0x7f8b6810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:45.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.611+0000 7f8b6e10d640 1 --2- 172.21.15.139:0/397846697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b68108290 0x7f8b6810ec30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8b60009930 tx=0x7f8b6002a420 comp rx=0 tx=0).stop 2024-09-16T01:38:45.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.612+0000 7f8b6e10d640 1 -- 172.21.15.139:0/397846697 shutdown_connections 2024-09-16T01:38:45.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.612+0000 7f8b6e10d640 1 --2- 172.21.15.139:0/397846697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b68108290 0x7f8b6810ec30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:45.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.612+0000 7f8b6e10d640 1 --2- 172.21.15.139:0/397846697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b68103740 0x7f8b68107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:45.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.612+0000 7f8b6e10d640 1 --2- 172.21.15.139:0/397846697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b68102d90 0x7f8b68103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:45.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.612+0000 7f8b6e10d640 1 -- 172.21.15.139:0/397846697 >> 172.21.15.139:0/397846697 conn(0x7f8b680fc7d0 msgr2=0x7f8b680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:45.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.612+0000 7f8b6e10d640 1 -- 172.21.15.139:0/397846697 shutdown_connections 2024-09-16T01:38:45.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.612+0000 7f8b6e10d640 1 -- 172.21.15.139:0/397846697 wait complete. 2024-09-16T01:38:45.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.612+0000 7f8b6e10d640 1 Processor -- start 2024-09-16T01:38:45.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.612+0000 7f8b6e10d640 1 -- start start 2024-09-16T01:38:45.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b6e10d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b68102d90 0x7f8b681986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:45.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b6e10d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b68103740 0x7f8b68198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:45.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b6e10d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b68108290 0x7f8b6819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:45.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b6e10d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68112150 con 0x7f8b68102d90 2024-09-16T01:38:45.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b6e10d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68111fd0 con 0x7f8b68103740 2024-09-16T01:38:45.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b6e10d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8b681122d0 con 0x7f8b68108290 2024-09-16T01:38:45.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b677fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b68102d90 0x7f8b681986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:45.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b677fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b68102d90 0x7f8b681986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55934/0 (socket says 172.21.15.139:55934) 2024-09-16T01:38:45.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b677fe640 1 -- 172.21.15.139:0/1917462699 learned_addr learned my addr 172.21.15.139:0/1917462699 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:45.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b66ffd640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b68103740 0x7f8b68198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:45.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b67fff640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b68108290 0x7f8b6819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:45.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b677fe640 1 -- 172.21.15.139:0/1917462699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b68108290 msgr2=0x7f8b6819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:45.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b677fe640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b68108290 0x7f8b6819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:45.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b677fe640 1 -- 172.21.15.139:0/1917462699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b68103740 msgr2=0x7f8b68198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:45.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b677fe640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b68103740 0x7f8b68198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:45.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b677fe640 1 -- 172.21.15.139:0/1917462699 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6819d660 con 0x7f8b68102d90 2024-09-16T01:38:45.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b67fff640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b68108290 0x7f8b6819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:38:45.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b677fe640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b68102d90 0x7f8b681986b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800ba50 tx=0x7f8b5800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:45.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b64ff9640 1 -- 172.21.15.139:0/1917462699 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58019070 con 0x7f8b68102d90 2024-09-16T01:38:45.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b6e10d640 1 -- 172.21.15.139:0/1917462699 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6819d950 con 0x7f8b68102d90 2024-09-16T01:38:45.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b64ff9640 1 -- 172.21.15.139:0/1917462699 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b58014440 con 0x7f8b68102d90 2024-09-16T01:38:45.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b64ff9640 1 -- 172.21.15.139:0/1917462699 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b580075d0 con 0x7f8b68102d90 2024-09-16T01:38:45.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.613+0000 7f8b6e10d640 1 -- 172.21.15.139:0/1917462699 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b681a5230 con 0x7f8b68102d90 2024-09-16T01:38:45.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.615+0000 7f8b6e10d640 1 -- 172.21.15.139:0/1917462699 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b6806a7f0 con 0x7f8b68102d90 2024-09-16T01:38:45.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.616+0000 7f8b64ff9640 1 -- 172.21.15.139:0/1917462699 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8b5801c070 con 0x7f8b68102d90 2024-09-16T01:38:45.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.616+0000 7f8b64ff9640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8b500a32d0 0x7f8b500a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:45.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.616+0000 7f8b66ffd640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8b500a32d0 0x7f8b500a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:45.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.616+0000 7f8b64ff9640 1 -- 172.21.15.139:0/1917462699 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f8b580cf340 con 0x7f8b68102d90 2024-09-16T01:38:45.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.617+0000 7f8b66ffd640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8b500a32d0 0x7f8b500a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b680fee50 tx=0x7f8b5c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:45.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.620+0000 7f8b64ff9640 1 -- 172.21.15.139:0/1917462699 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b5809bcb0 con 0x7f8b68102d90 2024-09-16T01:38:45.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.811+0000 7f8b6e10d640 1 -- 172.21.15.139:0/1917462699 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f8b68062630 con 0x7f8b500a32d0 2024-09-16T01:38:45.814 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:45 smithi139 ceph-mon[31175]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:45.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.817+0000 7f8b64ff9640 1 -- 172.21.15.139:0/1917462699 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8b68062630 con 0x7f8b500a32d0 2024-09-16T01:38:45.818 INFO:teuthology.orchestra.run.smithi139.stdout:/volumes/qa/sv_0/09819281-eea3-4ac8-b42a-f728251dcad1 2024-09-16T01:38:45.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.819+0000 7f8b6e10d640 1 -- 172.21.15.139:0/1917462699 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8b500a32d0 msgr2=0x7f8b500a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:45.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.819+0000 7f8b6e10d640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8b500a32d0 0x7f8b500a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b680fee50 tx=0x7f8b5c009110 comp rx=0 tx=0).stop 2024-09-16T01:38:45.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.819+0000 7f8b6e10d640 1 -- 172.21.15.139:0/1917462699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b68102d90 msgr2=0x7f8b681986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:45.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.819+0000 7f8b6e10d640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b68102d90 0x7f8b681986b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800ba50 tx=0x7f8b5800be50 comp rx=0 tx=0).stop 2024-09-16T01:38:45.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.819+0000 7f8b6e10d640 1 -- 172.21.15.139:0/1917462699 shutdown_connections 2024-09-16T01:38:45.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.819+0000 7f8b6e10d640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8b500a32d0 0x7f8b500a5790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:45.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.819+0000 7f8b6e10d640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b68108290 0x7f8b6819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:45.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.819+0000 7f8b6e10d640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b68103740 0x7f8b68198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:45.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.819+0000 7f8b6e10d640 1 --2- 172.21.15.139:0/1917462699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b68102d90 0x7f8b681986b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:45.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.819+0000 7f8b6e10d640 1 -- 172.21.15.139:0/1917462699 >> 172.21.15.139:0/1917462699 conn(0x7f8b680fc7d0 msgr2=0x7f8b68105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:45.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.819+0000 7f8b6e10d640 1 -- 172.21.15.139:0/1917462699 shutdown_connections 2024-09-16T01:38:45.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:45.819+0000 7f8b6e10d640 1 -- 172.21.15.139:0/1917462699 wait complete. 2024-09-16T01:38:45.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:45 smithi176 ceph-mon[35324]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:46.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:45 smithi164 ceph-mon[35121]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:46.460 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-16T01:38:46.707 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:46.828 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:46 smithi139 ceph-mon[31175]: pgmap v302: 129 pgs: 129 active+clean; 607 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.2 KiB/s wr, 18 op/s 2024-09-16T01:38:46.838 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:46 smithi176 ceph-mon[35324]: pgmap v302: 129 pgs: 129 active+clean; 607 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.2 KiB/s wr, 18 op/s 2024-09-16T01:38:46.946 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:46 smithi164 ceph-mon[35121]: pgmap v302: 129 pgs: 129 active+clean; 607 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.2 KiB/s wr, 18 op/s 2024-09-16T01:38:47.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.185+0000 7f33a46d9640 1 -- 172.21.15.139:0/2460071991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f339c100e30 msgr2=0x7f339c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:47.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.185+0000 7f33a46d9640 1 --2- 172.21.15.139:0/2460071991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f339c100e30 0x7f339c104e80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3390009900 tx=0x7f339002a410 comp rx=0 tx=0).stop 2024-09-16T01:38:47.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.185+0000 7f33a46d9640 1 -- 172.21.15.139:0/2460071991 shutdown_connections 2024-09-16T01:38:47.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.185+0000 7f33a46d9640 1 --2- 172.21.15.139:0/2460071991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f339c1055b0 0x7f339c10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:47.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.185+0000 7f33a46d9640 1 --2- 172.21.15.139:0/2460071991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f339c100e30 0x7f339c104e80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:47.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.185+0000 7f33a46d9640 1 --2- 172.21.15.139:0/2460071991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f339c100480 0x7f339c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:47.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.185+0000 7f33a46d9640 1 -- 172.21.15.139:0/2460071991 >> 172.21.15.139:0/2460071991 conn(0x7f339c077210 msgr2=0x7f339c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:47.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.185+0000 7f33a46d9640 1 -- 172.21.15.139:0/2460071991 shutdown_connections 2024-09-16T01:38:47.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.185+0000 7f33a46d9640 1 -- 172.21.15.139:0/2460071991 wait complete. 2024-09-16T01:38:47.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.186+0000 7f33a46d9640 1 Processor -- start 2024-09-16T01:38:47.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.186+0000 7f33a46d9640 1 -- start start 2024-09-16T01:38:47.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.186+0000 7f33a46d9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f339c100480 0x7f339c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:47.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.186+0000 7f33a46d9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f339c100e30 0x7f339c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:47.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.186+0000 7f33a46d9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f339c1055b0 0x7f339c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:47.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.186+0000 7f33a46d9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f339c111f90 con 0x7f339c100480 2024-09-16T01:38:47.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.186+0000 7f33a46d9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f339c111e10 con 0x7f339c100e30 2024-09-16T01:38:47.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.186+0000 7f33a46d9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f339c112110 con 0x7f339c1055b0 2024-09-16T01:38:47.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.186+0000 7f33a2475640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f339c100480 0x7f339c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:47.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.186+0000 7f33a2475640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f339c100480 0x7f339c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55948/0 (socket says 172.21.15.139:55948) 2024-09-16T01:38:47.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.186+0000 7f33a2475640 1 -- 172.21.15.139:0/2994309615 learned_addr learned my addr 172.21.15.139:0/2994309615 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:47.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.186+0000 7f33a1c74640 1 --2- 172.21.15.139:0/2994309615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f339c100e30 0x7f339c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:47.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.186+0000 7f33a2c76640 1 --2- 172.21.15.139:0/2994309615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f339c1055b0 0x7f339c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:47.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.187+0000 7f33a2475640 1 -- 172.21.15.139:0/2994309615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f339c1055b0 msgr2=0x7f339c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:47.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.187+0000 7f33a2475640 1 --2- 172.21.15.139:0/2994309615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f339c1055b0 0x7f339c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:47.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.187+0000 7f33a2475640 1 -- 172.21.15.139:0/2994309615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f339c100e30 msgr2=0x7f339c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:47.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.187+0000 7f33a2475640 1 --2- 172.21.15.139:0/2994309615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f339c100e30 0x7f339c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:47.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.187+0000 7f33a2475640 1 -- 172.21.15.139:0/2994309615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339c19d3a0 con 0x7f339c100480 2024-09-16T01:38:47.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.187+0000 7f33a2475640 1 --2- 172.21.15.139:0/2994309615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f339c100480 0x7f339c1983f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f339800d950 tx=0x7f339800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:47.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.187+0000 7f338f7fe640 1 -- 172.21.15.139:0/2994309615 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398017070 con 0x7f339c100480 2024-09-16T01:38:47.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.187+0000 7f33a46d9640 1 -- 172.21.15.139:0/2994309615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339c19d690 con 0x7f339c100480 2024-09-16T01:38:47.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.187+0000 7f338f7fe640 1 -- 172.21.15.139:0/2994309615 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3398009c30 con 0x7f339c100480 2024-09-16T01:38:47.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.187+0000 7f338f7fe640 1 -- 172.21.15.139:0/2994309615 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339800f6b0 con 0x7f339c100480 2024-09-16T01:38:47.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.187+0000 7f33a46d9640 1 -- 172.21.15.139:0/2994309615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f339c1a4f70 con 0x7f339c100480 2024-09-16T01:38:47.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.188+0000 7f33a46d9640 1 -- 172.21.15.139:0/2994309615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339c100860 con 0x7f339c100480 2024-09-16T01:38:47.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.189+0000 7f338f7fe640 1 -- 172.21.15.139:0/2994309615 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f33980109d0 con 0x7f339c100480 2024-09-16T01:38:47.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.190+0000 7f338f7fe640 1 --2- 172.21.15.139:0/2994309615 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f33880a32d0 0x7f33880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:47.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.190+0000 7f33a1c74640 1 --2- 172.21.15.139:0/2994309615 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f33880a32d0 0x7f33880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:47.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.190+0000 7f338f7fe640 1 -- 172.21.15.139:0/2994309615 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f33980cf9e0 con 0x7f339c100480 2024-09-16T01:38:47.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.190+0000 7f33a1c74640 1 --2- 172.21.15.139:0/2994309615 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f33880a32d0 0x7f33880a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3390009230 tx=0x7f33900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:47.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.194+0000 7f338f7fe640 1 -- 172.21.15.139:0/2994309615 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3398009de0 con 0x7f339c100480 2024-09-16T01:38:47.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.385+0000 7f33a46d9640 1 -- 172.21.15.139:0/2994309615 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f339c062630 con 0x7f33880a32d0 2024-09-16T01:38:47.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.412+0000 7f338f7fe640 1 -- 172.21.15.139:0/2994309615 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f339c062630 con 0x7f33880a32d0 2024-09-16T01:38:47.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.414+0000 7f33a46d9640 1 -- 172.21.15.139:0/2994309615 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f33880a32d0 msgr2=0x7f33880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:47.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.414+0000 7f33a46d9640 1 --2- 172.21.15.139:0/2994309615 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f33880a32d0 0x7f33880a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3390009230 tx=0x7f33900058d0 comp rx=0 tx=0).stop 2024-09-16T01:38:47.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.414+0000 7f33a46d9640 1 -- 172.21.15.139:0/2994309615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f339c100480 msgr2=0x7f339c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:47.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.414+0000 7f33a46d9640 1 --2- 172.21.15.139:0/2994309615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f339c100480 0x7f339c1983f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f339800d950 tx=0x7f339800de10 comp rx=0 tx=0).stop 2024-09-16T01:38:47.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.414+0000 7f33a46d9640 1 -- 172.21.15.139:0/2994309615 shutdown_connections 2024-09-16T01:38:47.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.414+0000 7f33a46d9640 1 --2- 172.21.15.139:0/2994309615 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f33880a32d0 0x7f33880a5790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:47.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.414+0000 7f33a46d9640 1 --2- 172.21.15.139:0/2994309615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f339c1055b0 0x7f339c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:47.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.414+0000 7f33a46d9640 1 --2- 172.21.15.139:0/2994309615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f339c100e30 0x7f339c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:47.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.414+0000 7f33a46d9640 1 --2- 172.21.15.139:0/2994309615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f339c100480 0x7f339c1983f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:47.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.414+0000 7f33a46d9640 1 -- 172.21.15.139:0/2994309615 >> 172.21.15.139:0/2994309615 conn(0x7f339c077210 msgr2=0x7f339c103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:47.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.415+0000 7f33a46d9640 1 -- 172.21.15.139:0/2994309615 shutdown_connections 2024-09-16T01:38:47.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:47.415+0000 7f33a46d9640 1 -- 172.21.15.139:0/2994309615 wait complete. 2024-09-16T01:38:47.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:47 smithi139 ceph-mon[31175]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:47.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:47 smithi176 ceph-mon[35324]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:48.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:47 smithi164 ceph-mon[35121]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:48.130 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-16T01:38:48.375 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:48.764 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:48 smithi139 ceph-mon[31175]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:48.764 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:48 smithi139 ceph-mon[31175]: pgmap v303: 129 pgs: 129 active+clean; 607 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-16T01:38:48.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.847+0000 7f787b441640 1 -- 172.21.15.139:0/1261520260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7874101de0 msgr2=0x7f78741021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:48.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.847+0000 7f787b441640 1 --2- 172.21.15.139:0/1261520260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7874101de0 0x7f78741021c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7868009900 tx=0x7f786802a410 comp rx=0 tx=0).stop 2024-09-16T01:38:48.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.847+0000 7f787b441640 1 -- 172.21.15.139:0/1261520260 shutdown_connections 2024-09-16T01:38:48.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.847+0000 7f787b441640 1 --2- 172.21.15.139:0/1261520260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7874106f10 0x7f787410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:48.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.847+0000 7f787b441640 1 --2- 172.21.15.139:0/1261520260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7874102790 0x7f78741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:48.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.847+0000 7f787b441640 1 --2- 172.21.15.139:0/1261520260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7874101de0 0x7f78741021c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:48.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.847+0000 7f787b441640 1 -- 172.21.15.139:0/1261520260 >> 172.21.15.139:0/1261520260 conn(0x7f78740fd8c0 msgr2=0x7f78740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:48.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.847+0000 7f787b441640 1 -- 172.21.15.139:0/1261520260 shutdown_connections 2024-09-16T01:38:48.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.847+0000 7f787b441640 1 -- 172.21.15.139:0/1261520260 wait complete. 2024-09-16T01:38:48.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.847+0000 7f787b441640 1 Processor -- start 2024-09-16T01:38:48.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f787b441640 1 -- start start 2024-09-16T01:38:48.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f787b441640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7874101de0 0x7f78740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:48.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f787b441640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7874102790 0x7f7874074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:48.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f787b441640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7874106f10 0x7f7874074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:48.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f787b441640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7874111f70 con 0x7f7874101de0 2024-09-16T01:38:48.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f787b441640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7874111df0 con 0x7f7874102790 2024-09-16T01:38:48.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f787b441640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f78741120f0 con 0x7f7874106f10 2024-09-16T01:38:48.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f78791dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7874101de0 0x7f78740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:48.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f78791dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7874101de0 0x7f78740744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55962/0 (socket says 172.21.15.139:55962) 2024-09-16T01:38:48.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f78791dd640 1 -- 172.21.15.139:0/2675863671 learned_addr learned my addr 172.21.15.139:0/2675863671 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:48.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f78789dc640 1 --2- 172.21.15.139:0/2675863671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7874102790 0x7f7874074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:48.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f78799de640 1 --2- 172.21.15.139:0/2675863671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7874106f10 0x7f7874074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:48.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f78791dd640 1 -- 172.21.15.139:0/2675863671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7874106f10 msgr2=0x7f7874074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:48.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f78791dd640 1 --2- 172.21.15.139:0/2675863671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7874106f10 0x7f7874074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:48.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f78791dd640 1 -- 172.21.15.139:0/2675863671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7874102790 msgr2=0x7f7874074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:48.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f78791dd640 1 --2- 172.21.15.139:0/2675863671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7874102790 0x7f7874074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:48.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f78791dd640 1 -- 172.21.15.139:0/2675863671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78741a68f0 con 0x7f7874101de0 2024-09-16T01:38:48.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.848+0000 7f78791dd640 1 --2- 172.21.15.139:0/2675863671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7874101de0 0x7f78740744f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7868006010 tx=0x7f78680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:48.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.849+0000 7f78667fc640 1 -- 172.21.15.139:0/2675863671 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786803f030 con 0x7f7874101de0 2024-09-16T01:38:48.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.849+0000 7f78667fc640 1 -- 172.21.15.139:0/2675863671 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7868003ce0 con 0x7f7874101de0 2024-09-16T01:38:48.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.849+0000 7f78667fc640 1 -- 172.21.15.139:0/2675863671 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868035070 con 0x7f7874101de0 2024-09-16T01:38:48.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.849+0000 7f787b441640 1 -- 172.21.15.139:0/2675863671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78741a6b80 con 0x7f7874101de0 2024-09-16T01:38:48.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.849+0000 7f787b441640 1 -- 172.21.15.139:0/2675863671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78741a70b0 con 0x7f7874101de0 2024-09-16T01:38:48.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.850+0000 7f787b441640 1 -- 172.21.15.139:0/2675863671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78741021c0 con 0x7f7874101de0 2024-09-16T01:38:48.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.851+0000 7f78667fc640 1 -- 172.21.15.139:0/2675863671 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7868042030 con 0x7f7874101de0 2024-09-16T01:38:48.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.851+0000 7f78667fc640 1 --2- 172.21.15.139:0/2675863671 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f78600a3350 0x7f78600a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:48.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.851+0000 7f78789dc640 1 --2- 172.21.15.139:0/2675863671 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f78600a3350 0x7f78600a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:48.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.851+0000 7f78667fc640 1 -- 172.21.15.139:0/2675863671 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f78680f0270 con 0x7f7874101de0 2024-09-16T01:38:48.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.852+0000 7f78789dc640 1 --2- 172.21.15.139:0/2675863671 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f78600a3350 0x7f78600a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7874075f80 tx=0x7f787000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:48.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:48.855+0000 7f78667fc640 1 -- 172.21.15.139:0/2675863671 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78680bc4e0 con 0x7f7874101de0 2024-09-16T01:38:48.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:48 smithi176 ceph-mon[35324]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:48.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:48 smithi176 ceph-mon[35324]: pgmap v303: 129 pgs: 129 active+clean; 607 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-16T01:38:49.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:48 smithi164 ceph-mon[35121]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:49.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:48 smithi164 ceph-mon[35121]: pgmap v303: 129 pgs: 129 active+clean; 607 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-16T01:38:49.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:49.047+0000 7f787b441640 1 -- 172.21.15.139:0/2675863671 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f7874062630 con 0x7f78600a3350 2024-09-16T01:38:49.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:49.052+0000 7f78667fc640 1 -- 172.21.15.139:0/2675863671 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7874062630 con 0x7f78600a3350 2024-09-16T01:38:49.054 INFO:teuthology.orchestra.run.smithi139.stdout:/volumes/qa/sv_1/525beaa2-f2b7-4f44-a32e-e6c7dd91c097 2024-09-16T01:38:49.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:49.054+0000 7f787b441640 1 -- 172.21.15.139:0/2675863671 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f78600a3350 msgr2=0x7f78600a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:49.054+0000 7f787b441640 1 --2- 172.21.15.139:0/2675863671 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f78600a3350 0x7f78600a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7874075f80 tx=0x7f787000a2d0 comp rx=0 tx=0).stop 2024-09-16T01:38:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:49.055+0000 7f787b441640 1 -- 172.21.15.139:0/2675863671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7874101de0 msgr2=0x7f78740744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:49.055+0000 7f787b441640 1 --2- 172.21.15.139:0/2675863671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7874101de0 0x7f78740744f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7868006010 tx=0x7f78680042f0 comp rx=0 tx=0).stop 2024-09-16T01:38:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:49.055+0000 7f787b441640 1 -- 172.21.15.139:0/2675863671 shutdown_connections 2024-09-16T01:38:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:49.055+0000 7f787b441640 1 --2- 172.21.15.139:0/2675863671 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f78600a3350 0x7f78600a5810 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:49.055+0000 7f787b441640 1 --2- 172.21.15.139:0/2675863671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7874106f10 0x7f7874074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:49.055+0000 7f787b441640 1 --2- 172.21.15.139:0/2675863671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7874102790 0x7f7874074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:49.055+0000 7f787b441640 1 --2- 172.21.15.139:0/2675863671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7874101de0 0x7f78740744f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:49.055+0000 7f787b441640 1 -- 172.21.15.139:0/2675863671 >> 172.21.15.139:0/2675863671 conn(0x7f78740fd8c0 msgr2=0x7f78740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:49.055+0000 7f787b441640 1 -- 172.21.15.139:0/2675863671 shutdown_connections 2024-09-16T01:38:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:49.055+0000 7f787b441640 1 -- 172.21.15.139:0/2675863671 wait complete. 2024-09-16T01:38:49.706 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:38:49.951 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:50.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.434+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3636319132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c102790 msgr2=0x7f9d9c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:50.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.434+0000 7f9da2ad6640 1 --2- 172.21.15.139:0/3636319132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c102790 0x7f9d9c1067e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009900 tx=0x7f9d9002a410 comp rx=0 tx=0).stop 2024-09-16T01:38:50.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3636319132 shutdown_connections 2024-09-16T01:38:50.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 --2- 172.21.15.139:0/3636319132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d9c106f10 0x7f9d9c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:50.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 --2- 172.21.15.139:0/3636319132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c102790 0x7f9d9c1067e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:50.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 --2- 172.21.15.139:0/3636319132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d9c101de0 0x7f9d9c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:50.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3636319132 >> 172.21.15.139:0/3636319132 conn(0x7f9d9c0fd8c0 msgr2=0x7f9d9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:50.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3636319132 shutdown_connections 2024-09-16T01:38:50.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3636319132 wait complete. 2024-09-16T01:38:50.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 Processor -- start 2024-09-16T01:38:50.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 -- start start 2024-09-16T01:38:50.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c101de0 0x7f9d9c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:50.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d9c102790 0x7f9d9c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:50.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d9c106f10 0x7f9d9c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:50.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c111f70 con 0x7f9d9c101de0 2024-09-16T01:38:50.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c111df0 con 0x7f9d9c106f10 2024-09-16T01:38:50.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da2ad6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1120f0 con 0x7f9d9c102790 2024-09-16T01:38:50.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da0872640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c101de0 0x7f9d9c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:50.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.435+0000 7f9da0872640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c101de0 0x7f9d9c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55972/0 (socket says 172.21.15.139:55972) 2024-09-16T01:38:50.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.436+0000 7f9da0872640 1 -- 172.21.15.139:0/3924768851 learned_addr learned my addr 172.21.15.139:0/3924768851 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:50.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.436+0000 7f9da0872640 1 -- 172.21.15.139:0/3924768851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d9c102790 msgr2=0x7f9d9c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:50.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.436+0000 7f9da0872640 1 --2- 172.21.15.139:0/3924768851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d9c102790 0x7f9d9c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:50.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.436+0000 7f9da0872640 1 -- 172.21.15.139:0/3924768851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d9c106f10 msgr2=0x7f9d9c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:50.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.436+0000 7f9da0872640 1 --2- 172.21.15.139:0/3924768851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d9c106f10 0x7f9d9c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:50.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.436+0000 7f9da0872640 1 -- 172.21.15.139:0/3924768851 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c19d3e0 con 0x7f9d9c101de0 2024-09-16T01:38:50.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.436+0000 7f9da0872640 1 --2- 172.21.15.139:0/3924768851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c101de0 0x7f9d9c198430 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00d950 tx=0x7f9d8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:50.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.436+0000 7f9d99ffb640 1 -- 172.21.15.139:0/3924768851 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c017030 con 0x7f9d9c101de0 2024-09-16T01:38:50.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.436+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3924768851 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9c19d6d0 con 0x7f9d9c101de0 2024-09-16T01:38:50.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.436+0000 7f9d99ffb640 1 -- 172.21.15.139:0/3924768851 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c004020 con 0x7f9d9c101de0 2024-09-16T01:38:50.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.436+0000 7f9d99ffb640 1 -- 172.21.15.139:0/3924768851 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0047a0 con 0x7f9d9c101de0 2024-09-16T01:38:50.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.436+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3924768851 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9c1a4fb0 con 0x7f9d9c101de0 2024-09-16T01:38:50.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.437+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3924768851 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d9c1021c0 con 0x7f9d9c101de0 2024-09-16T01:38:50.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.438+0000 7f9d99ffb640 1 -- 172.21.15.139:0/3924768851 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9d8c0041d0 con 0x7f9d9c101de0 2024-09-16T01:38:50.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.439+0000 7f9d99ffb640 1 --2- 172.21.15.139:0/3924768851 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9d840a32d0 0x7f9d840a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:50.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.439+0000 7f9d9bfff640 1 --2- 172.21.15.139:0/3924768851 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9d840a32d0 0x7f9d840a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:50.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.439+0000 7f9d99ffb640 1 -- 172.21.15.139:0/3924768851 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f9d8c0cf8a0 con 0x7f9d9c101de0 2024-09-16T01:38:50.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.439+0000 7f9d9bfff640 1 --2- 172.21.15.139:0/3924768851 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9d840a32d0 0x7f9d840a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9d9002a930 tx=0x7f9d90000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:50.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.443+0000 7f9d99ffb640 1 -- 172.21.15.139:0/3924768851 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9d8c09c100 con 0x7f9d9c101de0 2024-09-16T01:38:50.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.649+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3924768851 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d9c02d7c0 con 0x7f9d9c101de0 2024-09-16T01:38:50.651 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:50 smithi139 ceph-mon[31175]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:50.652 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:50 smithi139 ceph-mon[31175]: pgmap v304: 129 pgs: 129 active+clean; 607 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-16T01:38:50.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.650+0000 7f9d99ffb640 1 -- 172.21.15.139:0/3924768851 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11995 (secure 0 0 0) 0x7f9d8c09ba00 con 0x7f9d9c101de0 2024-09-16T01:38:50.652 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:38:50.652 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":28,"btime":"2024-09-16T01:38:40:558188+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:38:40.558170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24454,24443,24442,24433]},"id":1}]} 2024-09-16T01:38:50.653 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 28 2024-09-16T01:38:50.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.652+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3924768851 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9d840a32d0 msgr2=0x7f9d840a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:50.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.652+0000 7f9da2ad6640 1 --2- 172.21.15.139:0/3924768851 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9d840a32d0 0x7f9d840a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9d9002a930 tx=0x7f9d90000bc0 comp rx=0 tx=0).stop 2024-09-16T01:38:50.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.652+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3924768851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c101de0 msgr2=0x7f9d9c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:50.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.652+0000 7f9da2ad6640 1 --2- 172.21.15.139:0/3924768851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c101de0 0x7f9d9c198430 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00d950 tx=0x7f9d8c00de10 comp rx=0 tx=0).stop 2024-09-16T01:38:50.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.652+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3924768851 shutdown_connections 2024-09-16T01:38:50.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.653+0000 7f9da2ad6640 1 --2- 172.21.15.139:0/3924768851 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f9d840a32d0 0x7f9d840a5790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:50.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.653+0000 7f9da2ad6640 1 --2- 172.21.15.139:0/3924768851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d9c106f10 0x7f9d9c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:50.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.653+0000 7f9da2ad6640 1 --2- 172.21.15.139:0/3924768851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d9c102790 0x7f9d9c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:50.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.653+0000 7f9da2ad6640 1 --2- 172.21.15.139:0/3924768851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c101de0 0x7f9d9c198430 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:50.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.653+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3924768851 >> 172.21.15.139:0/3924768851 conn(0x7f9d9c0fd8c0 msgr2=0x7f9d9c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:50.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.653+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3924768851 shutdown_connections 2024-09-16T01:38:50.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:50.653+0000 7f9da2ad6640 1 -- 172.21.15.139:0/3924768851 wait complete. 2024-09-16T01:38:50.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:50 smithi176 ceph-mon[35324]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:50.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:50 smithi176 ceph-mon[35324]: pgmap v304: 129 pgs: 129 active+clean; 607 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-16T01:38:51.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:50 smithi164 ceph-mon[35121]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:38:51.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:50 smithi164 ceph-mon[35121]: pgmap v304: 129 pgs: 129 active+clean; 607 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-16T01:38:51.342 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T01:38:51.591 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:51.842 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:51 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3924768851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:38:51.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:51 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3924768851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:38:52.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:51 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3924768851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:38:52.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.052+0000 7fee5791e640 1 -- 172.21.15.139:0/1350794495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee501036e0 msgr2=0x7fee50107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:52.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.052+0000 7fee5791e640 1 --2- 172.21.15.139:0/1350794495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee501036e0 0x7fee50107b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fee44009930 tx=0x7fee4402a420 comp rx=0 tx=0).stop 2024-09-16T01:38:52.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.052+0000 7fee5791e640 1 -- 172.21.15.139:0/1350794495 shutdown_connections 2024-09-16T01:38:52.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.052+0000 7fee5791e640 1 --2- 172.21.15.139:0/1350794495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee501082a0 0x7fee5010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:52.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.052+0000 7fee5791e640 1 --2- 172.21.15.139:0/1350794495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee501036e0 0x7fee50107b90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:52.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.052+0000 7fee5791e640 1 --2- 172.21.15.139:0/1350794495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee50102d30 0x7fee50103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:52.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.052+0000 7fee5791e640 1 -- 172.21.15.139:0/1350794495 >> 172.21.15.139:0/1350794495 conn(0x7fee500fc7d0 msgr2=0x7fee500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:52.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.052+0000 7fee5791e640 1 -- 172.21.15.139:0/1350794495 shutdown_connections 2024-09-16T01:38:52.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.052+0000 7fee5791e640 1 -- 172.21.15.139:0/1350794495 wait complete. 2024-09-16T01:38:52.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.052+0000 7fee5791e640 1 Processor -- start 2024-09-16T01:38:52.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.052+0000 7fee5791e640 1 -- start start 2024-09-16T01:38:52.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee5791e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee50102d30 0x7fee50198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:52.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee5791e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee501036e0 0x7fee50198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:52.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee5791e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee501082a0 0x7fee5019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:52.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee5791e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fee501120e0 con 0x7fee501036e0 2024-09-16T01:38:52.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee5791e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fee50111f60 con 0x7fee501082a0 2024-09-16T01:38:52.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee5791e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fee50112260 con 0x7fee50102d30 2024-09-16T01:38:52.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee54eb9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee501036e0 0x7fee50198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:52.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee54eb9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee501036e0 0x7fee50198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55996/0 (socket says 172.21.15.139:55996) 2024-09-16T01:38:52.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee54eb9640 1 -- 172.21.15.139:0/2892523540 learned_addr learned my addr 172.21.15.139:0/2892523540 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:52.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee55ebb640 1 --2- 172.21.15.139:0/2892523540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee501082a0 0x7fee5019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:52.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee54eb9640 1 -- 172.21.15.139:0/2892523540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee50102d30 msgr2=0x7fee50198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:52.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee54eb9640 1 --2- 172.21.15.139:0/2892523540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee50102d30 0x7fee50198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:52.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee54eb9640 1 -- 172.21.15.139:0/2892523540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee501082a0 msgr2=0x7fee5019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:52.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee54eb9640 1 --2- 172.21.15.139:0/2892523540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee501082a0 0x7fee5019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:52.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee54eb9640 1 -- 172.21.15.139:0/2892523540 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5019d5f0 con 0x7fee501036e0 2024-09-16T01:38:52.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee54eb9640 1 --2- 172.21.15.139:0/2892523540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee501036e0 0x7fee50198b80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fee44005940 tx=0x7fee440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:52.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee4e7fc640 1 -- 172.21.15.139:0/2892523540 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4403f030 con 0x7fee501036e0 2024-09-16T01:38:52.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee5791e640 1 -- 172.21.15.139:0/2892523540 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5019d880 con 0x7fee501036e0 2024-09-16T01:38:52.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee4e7fc640 1 -- 172.21.15.139:0/2892523540 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee44003ce0 con 0x7fee501036e0 2024-09-16T01:38:52.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee4e7fc640 1 -- 172.21.15.139:0/2892523540 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44035070 con 0x7fee501036e0 2024-09-16T01:38:52.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.053+0000 7fee5791e640 1 -- 172.21.15.139:0/2892523540 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee501a51c0 con 0x7fee501036e0 2024-09-16T01:38:52.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.054+0000 7fee5791e640 1 -- 172.21.15.139:0/2892523540 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee50103110 con 0x7fee501036e0 2024-09-16T01:38:52.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.055+0000 7fee4e7fc640 1 -- 172.21.15.139:0/2892523540 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fee44042030 con 0x7fee501036e0 2024-09-16T01:38:52.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.056+0000 7fee4e7fc640 1 --2- 172.21.15.139:0/2892523540 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fee380a32d0 0x7fee380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:52.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.056+0000 7fee556ba640 1 --2- 172.21.15.139:0/2892523540 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fee380a32d0 0x7fee380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:52.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.056+0000 7fee4e7fc640 1 -- 172.21.15.139:0/2892523540 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fee440f0430 con 0x7fee501036e0 2024-09-16T01:38:52.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.056+0000 7fee556ba640 1 --2- 172.21.15.139:0/2892523540 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fee380a32d0 0x7fee380a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fee40007be0 tx=0x7fee40005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:52.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.060+0000 7fee4e7fc640 1 -- 172.21.15.139:0/2892523540 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee440bcc90 con 0x7fee501036e0 2024-09-16T01:38:52.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.228+0000 7fee5791e640 1 -- 172.21.15.139:0/2892523540 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee5010a2b0 con 0x7fee501036e0 2024-09-16T01:38:52.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.229+0000 7fee4e7fc640 1 -- 172.21.15.139:0/2892523540 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24797 (secure 0 0 0) 0x7fee440bc590 con 0x7fee501036e0 2024-09-16T01:38:52.232 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:38:52.232 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":83,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","created":"2024-09-16T01:30:17.770313+0000","modified":"2024-09-16T01:38:11.211197+0000","last_up_change":"2024-09-16T01:37:08.611901+0000","last_in_change":"2024-09-16T01:36:45.223926+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:33:19.639647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T01:37:57.948214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T01:38:00.263797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T01:38:06.773799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b66696da-ef66-4f25-9527-bbcc9ded2cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6803","nonce":3190810133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6805","nonce":3190810133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6809","nonce":3190810133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6807","nonce":3190810133}]},"public_addr":"172.21.15.139:6803/3190810133","cluster_addr":"172.21.15.139:6805/3190810133","heartbeat_back_addr":"172.21.15.139:6809/3190810133","heartbeat_front_addr":"172.21.15.139:6807/3190810133","state":["exists","up"]},{"osd":1,"uuid":"ccad17f2-0750-425e-9810-49a13a929ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6801","nonce":2699270330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6803","nonce":2699270330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6807","nonce":2699270330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6805","nonce":2699270330}]},"public_addr":"172.21.15.164:6801/2699270330","cluster_addr":"172.21.15.164:6803/2699270330","heartbeat_back_addr":"172.21.15.164:6807/2699270330","heartbeat_front_addr":"172.21.15.164:6805/2699270330","state":["exists","up"]},{"osd":2,"uuid":"baf0ba73-8faf-4e51-a171-c47898800638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6801","nonce":2034471646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6803","nonce":2034471646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6807","nonce":2034471646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6805","nonce":2034471646}]},"public_addr":"172.21.15.176:6801/2034471646","cluster_addr":"172.21.15.176:6803/2034471646","heartbeat_back_addr":"172.21.15.176:6807/2034471646","heartbeat_front_addr":"172.21.15.176:6805/2034471646","state":["exists","up"]},{"osd":3,"uuid":"98957bf1-67bb-4092-9ea4-b263a5706f8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6811","nonce":797147071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6813","nonce":797147071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6817","nonce":797147071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6815","nonce":797147071}]},"public_addr":"172.21.15.139:6811/797147071","cluster_addr":"172.21.15.139:6813/797147071","heartbeat_back_addr":"172.21.15.139:6817/797147071","heartbeat_front_addr":"172.21.15.139:6815/797147071","state":["exists","up"]},{"osd":4,"uuid":"9cd76696-8161-4e82-a571-29e98c11d5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6809","nonce":1421304792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6811","nonce":1421304792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6815","nonce":1421304792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6813","nonce":1421304792}]},"public_addr":"172.21.15.164:6809/1421304792","cluster_addr":"172.21.15.164:6811/1421304792","heartbeat_back_addr":"172.21.15.164:6815/1421304792","heartbeat_front_addr":"172.21.15.164:6813/1421304792","state":["exists","up"]},{"osd":5,"uuid":"ccc17a29-9a58-40ec-b453-5c38bee86f25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6809","nonce":10600602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6811","nonce":10600602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6815","nonce":10600602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6813","nonce":10600602}]},"public_addr":"172.21.15.176:6809/10600602","cluster_addr":"172.21.15.176:6811/10600602","heartbeat_back_addr":"172.21.15.176:6815/10600602","heartbeat_front_addr":"172.21.15.176:6813/10600602","state":["exists","up"]},{"osd":6,"uuid":"e5ee7212-10c6-4a05-992e-622a14a5a6b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6819","nonce":2601111164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6821","nonce":2601111164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6825","nonce":2601111164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6823","nonce":2601111164}]},"public_addr":"172.21.15.139:6819/2601111164","cluster_addr":"172.21.15.139:6821/2601111164","heartbeat_back_addr":"172.21.15.139:6825/2601111164","heartbeat_front_addr":"172.21.15.139:6823/2601111164","state":["exists","up"]},{"osd":7,"uuid":"23143c15-0839-4fd5-9d0a-19ae891000f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6817","nonce":2074080693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6819","nonce":2074080693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6823","nonce":2074080693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6821","nonce":2074080693}]},"public_addr":"172.21.15.164:6817/2074080693","cluster_addr":"172.21.15.164:6819/2074080693","heartbeat_back_addr":"172.21.15.164:6823/2074080693","heartbeat_front_addr":"172.21.15.164:6821/2074080693","state":["exists","up"]},{"osd":8,"uuid":"1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6817","nonce":593133361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6819","nonce":593133361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6823","nonce":593133361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6821","nonce":593133361}]},"public_addr":"172.21.15.176:6817/593133361","cluster_addr":"172.21.15.176:6819/593133361","heartbeat_back_addr":"172.21.15.176:6823/593133361","heartbeat_front_addr":"172.21.15.176:6821/593133361","state":["exists","up"]},{"osd":9,"uuid":"09d25a3b-1e66-480e-9433-ceef7f0b9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6827","nonce":3047942108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6829","nonce":3047942108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6833","nonce":3047942108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6831","nonce":3047942108}]},"public_addr":"172.21.15.139:6827/3047942108","cluster_addr":"172.21.15.139:6829/3047942108","heartbeat_back_addr":"172.21.15.139:6833/3047942108","heartbeat_front_addr":"172.21.15.139:6831/3047942108","state":["exists","up"]},{"osd":10,"uuid":"32b4d8ce-b644-42fa-80cc-a15a1f5f3a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6825","nonce":4172533828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6827","nonce":4172533828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6831","nonce":4172533828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6829","nonce":4172533828}]},"public_addr":"172.21.15.164:6825/4172533828","cluster_addr":"172.21.15.164:6827/4172533828","heartbeat_back_addr":"172.21.15.164:6831/4172533828","heartbeat_front_addr":"172.21.15.164:6829/4172533828","state":["exists","up"]},{"osd":11,"uuid":"49098bdc-58a9-4a74-8045-d408af7711ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6825","nonce":1854547565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6827","nonce":1854547565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6831","nonce":1854547565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6829","nonce":1854547565}]},"public_addr":"172.21.15.176:6825/1854547565","cluster_addr":"172.21.15.176:6827/1854547565","heartbeat_back_addr":"172.21.15.176:6831/1854547565","heartbeat_front_addr":"172.21.15.176:6829/1854547565","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:27.374845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:52.611853+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:17.229567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:43.330503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:07.988876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:33.787864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:58.864910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:23.815006+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:49.551954+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:15.496773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:41.647557+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:37:07.089355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3772232169":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/3997627612":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/1608930262":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6801/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/677216867":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6801/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/1830135963":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/2608207648":"2024-09-17T01:30:44.751168+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:38:52.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.231+0000 7fee5791e640 1 -- 172.21.15.139:0/2892523540 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fee380a32d0 msgr2=0x7fee380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:52.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.231+0000 7fee5791e640 1 --2- 172.21.15.139:0/2892523540 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fee380a32d0 0x7fee380a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fee40007be0 tx=0x7fee40005f70 comp rx=0 tx=0).stop 2024-09-16T01:38:52.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.231+0000 7fee5791e640 1 -- 172.21.15.139:0/2892523540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee501036e0 msgr2=0x7fee50198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:52.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.231+0000 7fee5791e640 1 --2- 172.21.15.139:0/2892523540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee501036e0 0x7fee50198b80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fee44005940 tx=0x7fee440042f0 comp rx=0 tx=0).stop 2024-09-16T01:38:52.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.232+0000 7fee5791e640 1 -- 172.21.15.139:0/2892523540 shutdown_connections 2024-09-16T01:38:52.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.232+0000 7fee5791e640 1 --2- 172.21.15.139:0/2892523540 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fee380a32d0 0x7fee380a5790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:52.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.232+0000 7fee5791e640 1 --2- 172.21.15.139:0/2892523540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee501082a0 0x7fee5019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:52.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.232+0000 7fee5791e640 1 --2- 172.21.15.139:0/2892523540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee501036e0 0x7fee50198b80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:52.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.232+0000 7fee5791e640 1 --2- 172.21.15.139:0/2892523540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee50102d30 0x7fee50198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:52.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.232+0000 7fee5791e640 1 -- 172.21.15.139:0/2892523540 >> 172.21.15.139:0/2892523540 conn(0x7fee500fc7d0 msgr2=0x7fee50105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:52.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.232+0000 7fee5791e640 1 -- 172.21.15.139:0/2892523540 shutdown_connections 2024-09-16T01:38:52.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:52.232+0000 7fee5791e640 1 -- 172.21.15.139:0/2892523540 wait complete. 2024-09-16T01:38:52.854 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:52 smithi139 ceph-mon[31175]: pgmap v305: 129 pgs: 129 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.2 KiB/s wr, 24 op/s 2024-09-16T01:38:52.854 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:52 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2892523540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:52.885 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T01:38:52.885 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T01:38:52.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:52 smithi176 ceph-mon[35324]: pgmap v305: 129 pgs: 129 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.2 KiB/s wr, 24 op/s 2024-09-16T01:38:52.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:52 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2892523540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:53.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:52 smithi164 ceph-mon[35121]: pgmap v305: 129 pgs: 129 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.2 KiB/s wr, 24 op/s 2024-09-16T01:38:53.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:52 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2892523540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:38:53.131 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:53.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.637+0000 7f111c867640 1 -- 172.21.15.139:0/1762449819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1118100a80 msgr2=0x7f1118100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:53.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.637+0000 7f111c867640 1 --2- 172.21.15.139:0/1762449819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1118100a80 0x7f1118100ee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1100009930 tx=0x7f110002a420 comp rx=0 tx=0).stop 2024-09-16T01:38:53.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.637+0000 7f111c867640 1 -- 172.21.15.139:0/1762449819 shutdown_connections 2024-09-16T01:38:53.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.637+0000 7f111c867640 1 --2- 172.21.15.139:0/1762449819 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118101420 0x7f111810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:53.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.637+0000 7f111c867640 1 --2- 172.21.15.139:0/1762449819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1118100a80 0x7f1118100ee0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:53.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.637+0000 7f111c867640 1 --2- 172.21.15.139:0/1762449819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1118106a80 0x7f1118106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.637+0000 7f111c867640 1 -- 172.21.15.139:0/1762449819 >> 172.21.15.139:0/1762449819 conn(0x7f11180fc7a0 msgr2=0x7f11180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.637+0000 7f111c867640 1 -- 172.21.15.139:0/1762449819 shutdown_connections 2024-09-16T01:38:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.637+0000 7f111c867640 1 -- 172.21.15.139:0/1762449819 wait complete. 2024-09-16T01:38:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f111c867640 1 Processor -- start 2024-09-16T01:38:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f111c867640 1 -- start start 2024-09-16T01:38:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f111c867640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1118100a80 0x7f1118198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f111c867640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118101420 0x7f1118198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f111c867640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1118106a80 0x7f111819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f111c867640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1118110e80 con 0x7f1118100a80 2024-09-16T01:38:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f111c867640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1118110d00 con 0x7f1118106a80 2024-09-16T01:38:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f111c867640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1118111000 con 0x7f1118101420 2024-09-16T01:38:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f111659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1118100a80 0x7f1118198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f111659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1118100a80 0x7f1118198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40166/0 (socket says 172.21.15.139:40166) 2024-09-16T01:38:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f111659c640 1 -- 172.21.15.139:0/8575253 learned_addr learned my addr 172.21.15.139:0/8575253 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f1116d9d640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1118106a80 0x7f111819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f110ffff640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118101420 0x7f1118198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f111659c640 1 -- 172.21.15.139:0/8575253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118101420 msgr2=0x7f1118198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.638+0000 7f111659c640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118101420 0x7f1118198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.639+0000 7f111659c640 1 -- 172.21.15.139:0/8575253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1118106a80 msgr2=0x7f111819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.639+0000 7f111659c640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1118106a80 0x7f111819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.639+0000 7f111659c640 1 -- 172.21.15.139:0/8575253 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111819d3e0 con 0x7f1118100a80 2024-09-16T01:38:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.639+0000 7f110ffff640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118101420 0x7f1118198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:38:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.639+0000 7f111659c640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1118100a80 0x7f1118198430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f110800d950 tx=0x7f110800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.639+0000 7f110f7fe640 1 -- 172.21.15.139:0/8575253 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108017070 con 0x7f1118100a80 2024-09-16T01:38:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.639+0000 7f110f7fe640 1 -- 172.21.15.139:0/8575253 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1108009c30 con 0x7f1118100a80 2024-09-16T01:38:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.639+0000 7f111c867640 1 -- 172.21.15.139:0/8575253 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111819d6d0 con 0x7f1118100a80 2024-09-16T01:38:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.639+0000 7f110f7fe640 1 -- 172.21.15.139:0/8575253 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110800f6b0 con 0x7f1118100a80 2024-09-16T01:38:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.639+0000 7f111c867640 1 -- 172.21.15.139:0/8575253 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11181a4fb0 con 0x7f1118100a80 2024-09-16T01:38:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.640+0000 7f111c867640 1 -- 172.21.15.139:0/8575253 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1118100ee0 con 0x7f1118100a80 2024-09-16T01:38:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.641+0000 7f110f7fe640 1 -- 172.21.15.139:0/8575253 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f11080109d0 con 0x7f1118100a80 2024-09-16T01:38:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.642+0000 7f110f7fe640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f11040a32d0 0x7f11040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.642+0000 7f110ffff640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f11040a32d0 0x7f11040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.642+0000 7f110f7fe640 1 -- 172.21.15.139:0/8575253 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f11080cf9b0 con 0x7f1118100a80 2024-09-16T01:38:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.642+0000 7f110ffff640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f11040a32d0 0x7f11040a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1100000c00 tx=0x7f1100005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.645+0000 7f110f7fe640 1 -- 172.21.15.139:0/8575253 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1108009de0 con 0x7f1118100a80 2024-09-16T01:38:53.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.810+0000 7f111c867640 1 -- 172.21.15.139:0/8575253 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1118062630 con 0x7f11040a32d0 2024-09-16T01:38:53.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.820+0000 7f110f7fe640 1 -- 172.21.15.139:0/8575253 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340664 (secure 0 0 0) 0x7f1118062630 con 0x7f11040a32d0 2024-09-16T01:38:53.825 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:38:53.828 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T01:38:53.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.823+0000 7f111c867640 1 -- 172.21.15.139:0/8575253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f11040a32d0 msgr2=0x7f11040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:53.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.823+0000 7f111c867640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f11040a32d0 0x7f11040a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1100000c00 tx=0x7f1100005a20 comp rx=0 tx=0).stop 2024-09-16T01:38:53.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.823+0000 7f111c867640 1 -- 172.21.15.139:0/8575253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1118100a80 msgr2=0x7f1118198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:53.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.823+0000 7f111c867640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1118100a80 0x7f1118198430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f110800d950 tx=0x7f110800de10 comp rx=0 tx=0).stop 2024-09-16T01:38:53.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.823+0000 7f111c867640 1 -- 172.21.15.139:0/8575253 shutdown_connections 2024-09-16T01:38:53.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.823+0000 7f111c867640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f11040a32d0 0x7f11040a5790 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:53.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.823+0000 7f111c867640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1118106a80 0x7f111819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:53.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.823+0000 7f111c867640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118101420 0x7f1118198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:53.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.823+0000 7f111c867640 1 --2- 172.21.15.139:0/8575253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1118100a80 0x7f1118198430 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:53.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.823+0000 7f111c867640 1 -- 172.21.15.139:0/8575253 >> 172.21.15.139:0/8575253 conn(0x7f11180fc7a0 msgr2=0x7f111806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:53.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.823+0000 7f111c867640 1 -- 172.21.15.139:0/8575253 shutdown_connections 2024-09-16T01:38:53.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:53.823+0000 7f111c867640 1 -- 172.21.15.139:0/8575253 wait complete. 2024-09-16T01:38:54.566 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-16T01:38:53.445937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640762,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":273,"num_read_kb":246,"num_write":420,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368856,"kb_used_data":11288,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572968,"statfs":{"total":1151940427776,"available":1151562719232,"internally_reserved":0,"allocated":11558912,"data_stored":7520588,"data_compressed":320082,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41114,"num_objects":30,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":129,"num_read_kb":120,"num_write":90,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011703"},"pg_stats":[{"pgid":"2.1f","version":"83'13","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:50.595835+0000","last_change":"2024-09-16T01:38:08.567909+0000","last_active":"2024-09-16T01:38:50.595835+0000","last_peered":"2024-09-16T01:38:50.595835+0000","last_clean":"2024-09-16T01:38:50.595835+0000","last_became_active":"2024-09-16T01:38:08.560733+0000","last_became_peered":"2024-09-16T01:38:08.560733+0000","last_unstale":"2024-09-16T01:38:50.595835+0000","last_undegraded":"2024-09-16T01:38:50.595835+0000","last_fullsized":"2024-09-16T01:38:50.595835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:00.678040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620682+0000","last_change":"2024-09-16T01:38:01.523064+0000","last_active":"2024-09-16T01:38:11.620682+0000","last_peered":"2024-09-16T01:38:11.620682+0000","last_clean":"2024-09-16T01:38:11.620682+0000","last_became_active":"2024-09-16T01:38:01.521695+0000","last_became_peered":"2024-09-16T01:38:01.521695+0000","last_unstale":"2024-09-16T01:38:11.620682+0000","last_undegraded":"2024-09-16T01:38:11.620682+0000","last_fullsized":"2024-09-16T01:38:11.620682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:28.467115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552424+0000","last_change":"2024-09-16T01:38:08.563557+0000","last_active":"2024-09-16T01:38:11.552424+0000","last_peered":"2024-09-16T01:38:11.552424+0000","last_clean":"2024-09-16T01:38:11.552424+0000","last_became_active":"2024-09-16T01:38:08.563065+0000","last_became_peered":"2024-09-16T01:38:08.563065+0000","last_unstale":"2024-09-16T01:38:11.552424+0000","last_undegraded":"2024-09-16T01:38:11.552424+0000","last_fullsized":"2024-09-16T01:38:11.552424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:56.992915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620823+0000","last_change":"2024-09-16T01:37:59.798884+0000","last_active":"2024-09-16T01:38:11.620823+0000","last_peered":"2024-09-16T01:38:11.620823+0000","last_clean":"2024-09-16T01:38:11.620823+0000","last_became_active":"2024-09-16T01:37:59.797989+0000","last_became_peered":"2024-09-16T01:37:59.797989+0000","last_unstale":"2024-09-16T01:38:11.620823+0000","last_undegraded":"2024-09-16T01:38:11.620823+0000","last_fullsized":"2024-09-16T01:38:11.620823+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:13.633078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620354+0000","last_change":"2024-09-16T01:38:01.528391+0000","last_active":"2024-09-16T01:38:11.620354+0000","last_peered":"2024-09-16T01:38:11.620354+0000","last_clean":"2024-09-16T01:38:11.620354+0000","last_became_active":"2024-09-16T01:38:01.528076+0000","last_became_peered":"2024-09-16T01:38:01.528076+0000","last_unstale":"2024-09-16T01:38:11.620354+0000","last_undegraded":"2024-09-16T01:38:11.620354+0000","last_fullsized":"2024-09-16T01:38:11.620354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.105830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619866+0000","last_change":"2024-09-16T01:38:08.566198+0000","last_active":"2024-09-16T01:38:11.619866+0000","last_peered":"2024-09-16T01:38:11.619866+0000","last_clean":"2024-09-16T01:38:11.619866+0000","last_became_active":"2024-09-16T01:38:08.565739+0000","last_became_peered":"2024-09-16T01:38:08.565739+0000","last_unstale":"2024-09-16T01:38:11.619866+0000","last_undegraded":"2024-09-16T01:38:11.619866+0000","last_fullsized":"2024-09-16T01:38:11.619866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:05.172269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:40.567498+0000","last_change":"2024-09-16T01:37:59.507625+0000","last_active":"2024-09-16T01:38:40.567498+0000","last_peered":"2024-09-16T01:38:40.567498+0000","last_clean":"2024-09-16T01:38:40.567498+0000","last_became_active":"2024-09-16T01:37:59.507280+0000","last_became_peered":"2024-09-16T01:37:59.507280+0000","last_unstale":"2024-09-16T01:38:40.567498+0000","last_undegraded":"2024-09-16T01:38:40.567498+0000","last_fullsized":"2024-09-16T01:38:40.567498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:29:37.761203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621582+0000","last_change":"2024-09-16T01:38:01.528406+0000","last_active":"2024-09-16T01:38:11.621582+0000","last_peered":"2024-09-16T01:38:11.621582+0000","last_clean":"2024-09-16T01:38:11.621582+0000","last_became_active":"2024-09-16T01:38:01.528084+0000","last_became_peered":"2024-09-16T01:38:01.528084+0000","last_unstale":"2024-09-16T01:38:11.621582+0000","last_undegraded":"2024-09-16T01:38:11.621582+0000","last_fullsized":"2024-09-16T01:38:11.621582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:40.766305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621257+0000","last_change":"2024-09-16T01:38:08.563645+0000","last_active":"2024-09-16T01:38:11.621257+0000","last_peered":"2024-09-16T01:38:11.621257+0000","last_clean":"2024-09-16T01:38:11.621257+0000","last_became_active":"2024-09-16T01:38:08.563149+0000","last_became_peered":"2024-09-16T01:38:08.563149+0000","last_unstale":"2024-09-16T01:38:11.621257+0000","last_undegraded":"2024-09-16T01:38:11.621257+0000","last_fullsized":"2024-09-16T01:38:11.621257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:09.010905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.561150+0000","last_change":"2024-09-16T01:37:59.509649+0000","last_active":"2024-09-16T01:38:39.561150+0000","last_peered":"2024-09-16T01:38:39.561150+0000","last_clean":"2024-09-16T01:38:39.561150+0000","last_became_active":"2024-09-16T01:37:59.509236+0000","last_became_peered":"2024-09-16T01:37:59.509236+0000","last_unstale":"2024-09-16T01:38:39.561150+0000","last_undegraded":"2024-09-16T01:38:39.561150+0000","last_fullsized":"2024-09-16T01:38:39.561150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.614205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621748+0000","last_change":"2024-09-16T01:38:01.528074+0000","last_active":"2024-09-16T01:38:11.621748+0000","last_peered":"2024-09-16T01:38:11.621748+0000","last_clean":"2024-09-16T01:38:11.621748+0000","last_became_active":"2024-09-16T01:38:01.527737+0000","last_became_peered":"2024-09-16T01:38:01.527737+0000","last_unstale":"2024-09-16T01:38:11.621748+0000","last_undegraded":"2024-09-16T01:38:11.621748+0000","last_fullsized":"2024-09-16T01:38:11.621748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:37.592557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620038+0000","last_change":"2024-09-16T01:38:08.566340+0000","last_active":"2024-09-16T01:38:11.620038+0000","last_peered":"2024-09-16T01:38:11.620038+0000","last_clean":"2024-09-16T01:38:11.620038+0000","last_became_active":"2024-09-16T01:38:08.565791+0000","last_became_peered":"2024-09-16T01:38:08.565791+0000","last_unstale":"2024-09-16T01:38:11.620038+0000","last_undegraded":"2024-09-16T01:38:11.620038+0000","last_fullsized":"2024-09-16T01:38:11.620038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:35:31.845151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620559+0000","last_change":"2024-09-16T01:38:01.527747+0000","last_active":"2024-09-16T01:38:11.620559+0000","last_peered":"2024-09-16T01:38:11.620559+0000","last_clean":"2024-09-16T01:38:11.620559+0000","last_became_active":"2024-09-16T01:38:01.527325+0000","last_became_peered":"2024-09-16T01:38:01.527325+0000","last_unstale":"2024-09-16T01:38:11.620559+0000","last_undegraded":"2024-09-16T01:38:11.620559+0000","last_fullsized":"2024-09-16T01:38:11.620559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:27.203719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:36.529271+0000","last_change":"2024-09-16T01:37:59.510298+0000","last_active":"2024-09-16T01:38:36.529271+0000","last_peered":"2024-09-16T01:38:36.529271+0000","last_clean":"2024-09-16T01:38:36.529271+0000","last_became_active":"2024-09-16T01:37:59.509952+0000","last_became_peered":"2024-09-16T01:37:59.509952+0000","last_unstale":"2024-09-16T01:38:36.529271+0000","last_undegraded":"2024-09-16T01:38:36.529271+0000","last_fullsized":"2024-09-16T01:38:36.529271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:11.616846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621402+0000","last_change":"2024-09-16T01:38:08.571622+0000","last_active":"2024-09-16T01:38:11.621402+0000","last_peered":"2024-09-16T01:38:11.621402+0000","last_clean":"2024-09-16T01:38:11.621402+0000","last_became_active":"2024-09-16T01:38:08.571125+0000","last_became_peered":"2024-09-16T01:38:08.571125+0000","last_unstale":"2024-09-16T01:38:11.621402+0000","last_undegraded":"2024-09-16T01:38:11.621402+0000","last_fullsized":"2024-09-16T01:38:11.621402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:00.154633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620455+0000","last_change":"2024-09-16T01:38:01.519681+0000","last_active":"2024-09-16T01:38:11.620455+0000","last_peered":"2024-09-16T01:38:11.620455+0000","last_clean":"2024-09-16T01:38:11.620455+0000","last_became_active":"2024-09-16T01:38:01.518310+0000","last_became_peered":"2024-09-16T01:38:01.518310+0000","last_unstale":"2024-09-16T01:38:11.620455+0000","last_undegraded":"2024-09-16T01:38:11.620455+0000","last_fullsized":"2024-09-16T01:38:11.620455+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:23.308008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:38.547465+0000","last_change":"2024-09-16T01:38:08.559469+0000","last_active":"2024-09-16T01:38:38.547465+0000","last_peered":"2024-09-16T01:38:38.547465+0000","last_clean":"2024-09-16T01:38:38.547465+0000","last_became_active":"2024-09-16T01:38:08.559111+0000","last_became_peered":"2024-09-16T01:38:08.559111+0000","last_unstale":"2024-09-16T01:38:38.547465+0000","last_undegraded":"2024-09-16T01:38:38.547465+0000","last_fullsized":"2024-09-16T01:38:38.547465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:58.577749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621895+0000","last_change":"2024-09-16T01:38:08.555029+0000","last_active":"2024-09-16T01:38:11.621895+0000","last_peered":"2024-09-16T01:38:11.621895+0000","last_clean":"2024-09-16T01:38:11.621895+0000","last_became_active":"2024-09-16T01:38:08.554151+0000","last_became_peered":"2024-09-16T01:38:08.554151+0000","last_unstale":"2024-09-16T01:38:11.621895+0000","last_undegraded":"2024-09-16T01:38:11.621895+0000","last_fullsized":"2024-09-16T01:38:11.621895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:05.005904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620140+0000","last_change":"2024-09-16T01:38:01.528114+0000","last_active":"2024-09-16T01:38:11.620140+0000","last_peered":"2024-09-16T01:38:11.620140+0000","last_clean":"2024-09-16T01:38:11.620140+0000","last_became_active":"2024-09-16T01:38:01.527805+0000","last_became_peered":"2024-09-16T01:38:01.527805+0000","last_unstale":"2024-09-16T01:38:11.620140+0000","last_undegraded":"2024-09-16T01:38:11.620140+0000","last_fullsized":"2024-09-16T01:38:11.620140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:15.830936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.558334+0000","last_change":"2024-09-16T01:37:59.507776+0000","last_active":"2024-09-16T01:38:39.558334+0000","last_peered":"2024-09-16T01:38:39.558334+0000","last_clean":"2024-09-16T01:38:39.558334+0000","last_became_active":"2024-09-16T01:37:59.507429+0000","last_became_peered":"2024-09-16T01:37:59.507429+0000","last_unstale":"2024-09-16T01:38:39.558334+0000","last_undegraded":"2024-09-16T01:38:39.558334+0000","last_fullsized":"2024-09-16T01:38:39.558334+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:44:41.274986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551917+0000","last_change":"2024-09-16T01:38:08.562191+0000","last_active":"2024-09-16T01:38:11.551917+0000","last_peered":"2024-09-16T01:38:11.551917+0000","last_clean":"2024-09-16T01:38:11.551917+0000","last_became_active":"2024-09-16T01:38:08.561839+0000","last_became_peered":"2024-09-16T01:38:08.561839+0000","last_unstale":"2024-09-16T01:38:11.551917+0000","last_undegraded":"2024-09-16T01:38:11.551917+0000","last_fullsized":"2024-09-16T01:38:11.551917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:40.618612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620169+0000","last_change":"2024-09-16T01:38:01.518855+0000","last_active":"2024-09-16T01:38:11.620169+0000","last_peered":"2024-09-16T01:38:11.620169+0000","last_clean":"2024-09-16T01:38:11.620169+0000","last_became_active":"2024-09-16T01:38:01.518427+0000","last_became_peered":"2024-09-16T01:38:01.518427+0000","last_unstale":"2024-09-16T01:38:11.620169+0000","last_undegraded":"2024-09-16T01:38:11.620169+0000","last_fullsized":"2024-09-16T01:38:11.620169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:23.054856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:36.529446+0000","last_change":"2024-09-16T01:37:59.799341+0000","last_active":"2024-09-16T01:38:36.529446+0000","last_peered":"2024-09-16T01:38:36.529446+0000","last_clean":"2024-09-16T01:38:36.529446+0000","last_became_active":"2024-09-16T01:37:59.798992+0000","last_became_peered":"2024-09-16T01:37:59.798992+0000","last_unstale":"2024-09-16T01:38:36.529446+0000","last_undegraded":"2024-09-16T01:38:36.529446+0000","last_fullsized":"2024-09-16T01:38:36.529446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:01.708565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621703+0000","last_change":"2024-09-16T01:38:08.572556+0000","last_active":"2024-09-16T01:38:11.621703+0000","last_peered":"2024-09-16T01:38:11.621703+0000","last_clean":"2024-09-16T01:38:11.621703+0000","last_became_active":"2024-09-16T01:38:08.571990+0000","last_became_peered":"2024-09-16T01:38:08.571990+0000","last_unstale":"2024-09-16T01:38:11.621703+0000","last_undegraded":"2024-09-16T01:38:11.621703+0000","last_fullsized":"2024-09-16T01:38:11.621703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:41.793143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620089+0000","last_change":"2024-09-16T01:38:01.524545+0000","last_active":"2024-09-16T01:38:11.620089+0000","last_peered":"2024-09-16T01:38:11.620089+0000","last_clean":"2024-09-16T01:38:11.620089+0000","last_became_active":"2024-09-16T01:38:01.524253+0000","last_became_peered":"2024-09-16T01:38:01.524253+0000","last_unstale":"2024-09-16T01:38:11.620089+0000","last_undegraded":"2024-09-16T01:38:11.620089+0000","last_fullsized":"2024-09-16T01:38:11.620089+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:08.697500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.560936+0000","last_change":"2024-09-16T01:37:59.506779+0000","last_active":"2024-09-16T01:38:39.560936+0000","last_peered":"2024-09-16T01:38:39.560936+0000","last_clean":"2024-09-16T01:38:39.560936+0000","last_became_active":"2024-09-16T01:37:59.505459+0000","last_became_peered":"2024-09-16T01:37:59.505459+0000","last_unstale":"2024-09-16T01:38:39.560936+0000","last_undegraded":"2024-09-16T01:38:39.560936+0000","last_fullsized":"2024-09-16T01:38:39.560936+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:12.973330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552762+0000","last_change":"2024-09-16T01:38:08.557250+0000","last_active":"2024-09-16T01:38:11.552762+0000","last_peered":"2024-09-16T01:38:11.552762+0000","last_clean":"2024-09-16T01:38:11.552762+0000","last_became_active":"2024-09-16T01:38:08.555577+0000","last_became_peered":"2024-09-16T01:38:08.555577+0000","last_unstale":"2024-09-16T01:38:11.552762+0000","last_undegraded":"2024-09-16T01:38:11.552762+0000","last_fullsized":"2024-09-16T01:38:11.552762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:54.965341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620877+0000","last_change":"2024-09-16T01:38:01.522910+0000","last_active":"2024-09-16T01:38:11.620877+0000","last_peered":"2024-09-16T01:38:11.620877+0000","last_clean":"2024-09-16T01:38:11.620877+0000","last_became_active":"2024-09-16T01:38:01.521206+0000","last_became_peered":"2024-09-16T01:38:01.521206+0000","last_unstale":"2024-09-16T01:38:11.620877+0000","last_undegraded":"2024-09-16T01:38:11.620877+0000","last_fullsized":"2024-09-16T01:38:11.620877+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:45.670464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'14","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:50.524562+0000","last_change":"2024-09-16T01:37:59.505962+0000","last_active":"2024-09-16T01:38:50.524562+0000","last_peered":"2024-09-16T01:38:50.524562+0000","last_clean":"2024-09-16T01:38:50.524562+0000","last_became_active":"2024-09-16T01:37:59.505636+0000","last_became_peered":"2024-09-16T01:37:59.505636+0000","last_unstale":"2024-09-16T01:38:50.524562+0000","last_undegraded":"2024-09-16T01:38:50.524562+0000","last_fullsized":"2024-09-16T01:38:50.524562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:57.548513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619604+0000","last_change":"2024-09-16T01:38:08.571249+0000","last_active":"2024-09-16T01:38:11.619604+0000","last_peered":"2024-09-16T01:38:11.619604+0000","last_clean":"2024-09-16T01:38:11.619604+0000","last_became_active":"2024-09-16T01:38:08.570388+0000","last_became_peered":"2024-09-16T01:38:08.570388+0000","last_unstale":"2024-09-16T01:38:11.619604+0000","last_undegraded":"2024-09-16T01:38:11.619604+0000","last_fullsized":"2024-09-16T01:38:11.619604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:59.904704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009588+0000","last_change":"2024-09-16T01:38:01.522040+0000","last_active":"2024-09-16T01:38:12.009588+0000","last_peered":"2024-09-16T01:38:12.009588+0000","last_clean":"2024-09-16T01:38:12.009588+0000","last_became_active":"2024-09-16T01:38:01.521720+0000","last_became_peered":"2024-09-16T01:38:01.521720+0000","last_unstale":"2024-09-16T01:38:12.009588+0000","last_undegraded":"2024-09-16T01:38:12.009588+0000","last_fullsized":"2024-09-16T01:38:12.009588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:44:48.635603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.559237+0000","last_change":"2024-09-16T01:38:08.561870+0000","last_active":"2024-09-16T01:38:39.559237+0000","last_peered":"2024-09-16T01:38:39.559237+0000","last_clean":"2024-09-16T01:38:39.559237+0000","last_became_active":"2024-09-16T01:38:08.561440+0000","last_became_peered":"2024-09-16T01:38:08.561440+0000","last_unstale":"2024-09-16T01:38:39.559237+0000","last_undegraded":"2024-09-16T01:38:39.559237+0000","last_fullsized":"2024-09-16T01:38:39.559237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:43:37.594734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620437+0000","last_change":"2024-09-16T01:38:08.555436+0000","last_active":"2024-09-16T01:38:11.620437+0000","last_peered":"2024-09-16T01:38:11.620437+0000","last_clean":"2024-09-16T01:38:11.620437+0000","last_became_active":"2024-09-16T01:38:08.555026+0000","last_became_peered":"2024-09-16T01:38:08.555026+0000","last_unstale":"2024-09-16T01:38:11.620437+0000","last_undegraded":"2024-09-16T01:38:11.620437+0000","last_fullsized":"2024-09-16T01:38:11.620437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:28.870214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552802+0000","last_change":"2024-09-16T01:38:01.524482+0000","last_active":"2024-09-16T01:38:11.552802+0000","last_peered":"2024-09-16T01:38:11.552802+0000","last_clean":"2024-09-16T01:38:11.552802+0000","last_became_active":"2024-09-16T01:38:01.524160+0000","last_became_peered":"2024-09-16T01:38:01.524160+0000","last_unstale":"2024-09-16T01:38:11.552802+0000","last_undegraded":"2024-09-16T01:38:11.552802+0000","last_fullsized":"2024-09-16T01:38:11.552802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:39:27.621780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551849+0000","last_change":"2024-09-16T01:37:59.800051+0000","last_active":"2024-09-16T01:38:11.551849+0000","last_peered":"2024-09-16T01:38:11.551849+0000","last_clean":"2024-09-16T01:38:11.551849+0000","last_became_active":"2024-09-16T01:37:59.799606+0000","last_became_peered":"2024-09-16T01:37:59.799606+0000","last_unstale":"2024-09-16T01:38:11.551849+0000","last_undegraded":"2024-09-16T01:38:11.551849+0000","last_fullsized":"2024-09-16T01:38:11.551849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:02.694933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620035+0000","last_change":"2024-09-16T01:38:08.570764+0000","last_active":"2024-09-16T01:38:11.620035+0000","last_peered":"2024-09-16T01:38:11.620035+0000","last_clean":"2024-09-16T01:38:11.620035+0000","last_became_active":"2024-09-16T01:38:08.570385+0000","last_became_peered":"2024-09-16T01:38:08.570385+0000","last_unstale":"2024-09-16T01:38:11.620035+0000","last_undegraded":"2024-09-16T01:38:11.620035+0000","last_fullsized":"2024-09-16T01:38:11.620035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:25.349671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552770+0000","last_change":"2024-09-16T01:38:01.519590+0000","last_active":"2024-09-16T01:38:11.552770+0000","last_peered":"2024-09-16T01:38:11.552770+0000","last_clean":"2024-09-16T01:38:11.552770+0000","last_became_active":"2024-09-16T01:38:01.519210+0000","last_became_peered":"2024-09-16T01:38:01.519210+0000","last_unstale":"2024-09-16T01:38:11.552770+0000","last_undegraded":"2024-09-16T01:38:11.552770+0000","last_fullsized":"2024-09-16T01:38:11.552770+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:04.228131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:48.593997+0000","last_change":"2024-09-16T01:37:59.505473+0000","last_active":"2024-09-16T01:38:48.593997+0000","last_peered":"2024-09-16T01:38:48.593997+0000","last_clean":"2024-09-16T01:38:48.593997+0000","last_became_active":"2024-09-16T01:37:59.505123+0000","last_became_peered":"2024-09-16T01:37:59.505123+0000","last_unstale":"2024-09-16T01:38:48.593997+0000","last_undegraded":"2024-09-16T01:38:48.593997+0000","last_fullsized":"2024-09-16T01:38:48.593997+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:17.739625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009514+0000","last_change":"2024-09-16T01:38:08.558534+0000","last_active":"2024-09-16T01:38:12.009514+0000","last_peered":"2024-09-16T01:38:12.009514+0000","last_clean":"2024-09-16T01:38:12.009514+0000","last_became_active":"2024-09-16T01:38:08.558185+0000","last_became_peered":"2024-09-16T01:38:08.558185+0000","last_unstale":"2024-09-16T01:38:12.009514+0000","last_undegraded":"2024-09-16T01:38:12.009514+0000","last_fullsized":"2024-09-16T01:38:12.009514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:09.359426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620222+0000","last_change":"2024-09-16T01:38:01.521445+0000","last_active":"2024-09-16T01:38:11.620222+0000","last_peered":"2024-09-16T01:38:11.620222+0000","last_clean":"2024-09-16T01:38:11.620222+0000","last_became_active":"2024-09-16T01:38:01.521140+0000","last_became_peered":"2024-09-16T01:38:01.521140+0000","last_unstale":"2024-09-16T01:38:11.620222+0000","last_undegraded":"2024-09-16T01:38:11.620222+0000","last_fullsized":"2024-09-16T01:38:11.620222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:05.247515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:50.584452+0000","last_change":"2024-09-16T01:37:59.798365+0000","last_active":"2024-09-16T01:38:50.584452+0000","last_peered":"2024-09-16T01:38:50.584452+0000","last_clean":"2024-09-16T01:38:50.584452+0000","last_became_active":"2024-09-16T01:37:59.798023+0000","last_became_peered":"2024-09-16T01:37:59.798023+0000","last_unstale":"2024-09-16T01:38:50.584452+0000","last_undegraded":"2024-09-16T01:38:50.584452+0000","last_fullsized":"2024-09-16T01:38:50.584452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:30.867080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009383+0000","last_change":"2024-09-16T01:38:08.571704+0000","last_active":"2024-09-16T01:38:12.009383+0000","last_peered":"2024-09-16T01:38:12.009383+0000","last_clean":"2024-09-16T01:38:12.009383+0000","last_became_active":"2024-09-16T01:38:08.571350+0000","last_became_peered":"2024-09-16T01:38:08.571350+0000","last_unstale":"2024-09-16T01:38:12.009383+0000","last_undegraded":"2024-09-16T01:38:12.009383+0000","last_fullsized":"2024-09-16T01:38:12.009383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:01.698770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009013+0000","last_change":"2024-09-16T01:38:08.559433+0000","last_active":"2024-09-16T01:38:12.009013+0000","last_peered":"2024-09-16T01:38:12.009013+0000","last_clean":"2024-09-16T01:38:12.009013+0000","last_became_active":"2024-09-16T01:38:08.559110+0000","last_became_peered":"2024-09-16T01:38:08.559110+0000","last_unstale":"2024-09-16T01:38:12.009013+0000","last_undegraded":"2024-09-16T01:38:12.009013+0000","last_fullsized":"2024-09-16T01:38:12.009013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:39.001916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:37.538688+0000","last_change":"2024-09-16T01:37:59.508236+0000","last_active":"2024-09-16T01:38:37.538688+0000","last_peered":"2024-09-16T01:38:37.538688+0000","last_clean":"2024-09-16T01:38:37.538688+0000","last_became_active":"2024-09-16T01:37:59.507883+0000","last_became_peered":"2024-09-16T01:37:59.507883+0000","last_unstale":"2024-09-16T01:38:37.538688+0000","last_undegraded":"2024-09-16T01:38:37.538688+0000","last_fullsized":"2024-09-16T01:38:37.538688+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:23.948530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621566+0000","last_change":"2024-09-16T01:38:08.572108+0000","last_active":"2024-09-16T01:38:11.621566+0000","last_peered":"2024-09-16T01:38:11.621566+0000","last_clean":"2024-09-16T01:38:11.621566+0000","last_became_active":"2024-09-16T01:38:08.571643+0000","last_became_peered":"2024-09-16T01:38:08.571643+0000","last_unstale":"2024-09-16T01:38:11.621566+0000","last_undegraded":"2024-09-16T01:38:11.621566+0000","last_fullsized":"2024-09-16T01:38:11.621566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:46.636219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620399+0000","last_change":"2024-09-16T01:38:01.517894+0000","last_active":"2024-09-16T01:38:11.620399+0000","last_peered":"2024-09-16T01:38:11.620399+0000","last_clean":"2024-09-16T01:38:11.620399+0000","last_became_active":"2024-09-16T01:38:01.517602+0000","last_became_peered":"2024-09-16T01:38:01.517602+0000","last_unstale":"2024-09-16T01:38:11.620399+0000","last_undegraded":"2024-09-16T01:38:11.620399+0000","last_fullsized":"2024-09-16T01:38:11.620399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:33.483120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:40.567759+0000","last_change":"2024-09-16T01:37:59.507568+0000","last_active":"2024-09-16T01:38:40.567759+0000","last_peered":"2024-09-16T01:38:40.567759+0000","last_clean":"2024-09-16T01:38:40.567759+0000","last_became_active":"2024-09-16T01:37:59.505798+0000","last_became_peered":"2024-09-16T01:37:59.505798+0000","last_unstale":"2024-09-16T01:38:40.567759+0000","last_undegraded":"2024-09-16T01:38:40.567759+0000","last_fullsized":"2024-09-16T01:38:40.567759+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:19:02.711054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":353,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620796+0000","last_change":"2024-09-16T01:36:46.744332+0000","last_active":"2024-09-16T01:38:11.620796+0000","last_peered":"2024-09-16T01:38:11.620796+0000","last_clean":"2024-09-16T01:38:11.620796+0000","last_became_active":"2024-09-16T01:36:46.743902+0000","last_became_peered":"2024-09-16T01:36:46.743902+0000","last_unstale":"2024-09-16T01:38:11.620796+0000","last_undegraded":"2024-09-16T01:38:11.620796+0000","last_fullsized":"2024-09-16T01:38:11.620796+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:33:19.891854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:33:19.891854+0000","last_clean_scrub_stamp":"2024-09-16T01:33:19.891854+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:46.173685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620994+0000","last_change":"2024-09-16T01:38:08.558344+0000","last_active":"2024-09-16T01:38:11.620994+0000","last_peered":"2024-09-16T01:38:11.620994+0000","last_clean":"2024-09-16T01:38:11.620994+0000","last_became_active":"2024-09-16T01:38:08.557938+0000","last_became_peered":"2024-09-16T01:38:08.557938+0000","last_unstale":"2024-09-16T01:38:11.620994+0000","last_undegraded":"2024-09-16T01:38:11.620994+0000","last_fullsized":"2024-09-16T01:38:11.620994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:56.799794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621097+0000","last_change":"2024-09-16T01:38:01.527690+0000","last_active":"2024-09-16T01:38:11.621097+0000","last_peered":"2024-09-16T01:38:11.621097+0000","last_clean":"2024-09-16T01:38:11.621097+0000","last_became_active":"2024-09-16T01:38:01.527339+0000","last_became_peered":"2024-09-16T01:38:01.527339+0000","last_unstale":"2024-09-16T01:38:11.621097+0000","last_undegraded":"2024-09-16T01:38:11.621097+0000","last_fullsized":"2024-09-16T01:38:11.621097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:58.526270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:47.581214+0000","last_change":"2024-09-16T01:37:59.798874+0000","last_active":"2024-09-16T01:38:47.581214+0000","last_peered":"2024-09-16T01:38:47.581214+0000","last_clean":"2024-09-16T01:38:47.581214+0000","last_became_active":"2024-09-16T01:37:59.798424+0000","last_became_peered":"2024-09-16T01:37:59.798424+0000","last_unstale":"2024-09-16T01:38:47.581214+0000","last_undegraded":"2024-09-16T01:38:47.581214+0000","last_fullsized":"2024-09-16T01:38:47.581214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:36.648963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620451+0000","last_change":"2024-09-16T01:38:08.555524+0000","last_active":"2024-09-16T01:38:11.620451+0000","last_peered":"2024-09-16T01:38:11.620451+0000","last_clean":"2024-09-16T01:38:11.620451+0000","last_became_active":"2024-09-16T01:38:08.555075+0000","last_became_peered":"2024-09-16T01:38:08.555075+0000","last_unstale":"2024-09-16T01:38:11.620451+0000","last_undegraded":"2024-09-16T01:38:11.620451+0000","last_fullsized":"2024-09-16T01:38:11.620451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:44.847916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620635+0000","last_change":"2024-09-16T01:38:01.528869+0000","last_active":"2024-09-16T01:38:11.620635+0000","last_peered":"2024-09-16T01:38:11.620635+0000","last_clean":"2024-09-16T01:38:11.620635+0000","last_became_active":"2024-09-16T01:38:01.528559+0000","last_became_peered":"2024-09-16T01:38:01.528559+0000","last_unstale":"2024-09-16T01:38:11.620635+0000","last_undegraded":"2024-09-16T01:38:11.620635+0000","last_fullsized":"2024-09-16T01:38:11.620635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:21.547808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.559919+0000","last_change":"2024-09-16T01:37:59.510871+0000","last_active":"2024-09-16T01:38:39.559919+0000","last_peered":"2024-09-16T01:38:39.559919+0000","last_clean":"2024-09-16T01:38:39.559919+0000","last_became_active":"2024-09-16T01:37:59.509996+0000","last_became_peered":"2024-09-16T01:37:59.509996+0000","last_unstale":"2024-09-16T01:38:39.559919+0000","last_undegraded":"2024-09-16T01:38:39.559919+0000","last_fullsized":"2024-09-16T01:38:39.559919+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:23.390864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620207+0000","last_change":"2024-09-16T01:38:08.555181+0000","last_active":"2024-09-16T01:38:11.620207+0000","last_peered":"2024-09-16T01:38:11.620207+0000","last_clean":"2024-09-16T01:38:11.620207+0000","last_became_active":"2024-09-16T01:38:08.554778+0000","last_became_peered":"2024-09-16T01:38:08.554778+0000","last_unstale":"2024-09-16T01:38:11.620207+0000","last_undegraded":"2024-09-16T01:38:11.620207+0000","last_fullsized":"2024-09-16T01:38:11.620207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:16:42.369236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620832+0000","last_change":"2024-09-16T01:38:01.526260+0000","last_active":"2024-09-16T01:38:11.620832+0000","last_peered":"2024-09-16T01:38:11.620832+0000","last_clean":"2024-09-16T01:38:11.620832+0000","last_became_active":"2024-09-16T01:38:01.525652+0000","last_became_peered":"2024-09-16T01:38:01.525652+0000","last_unstale":"2024-09-16T01:38:11.620832+0000","last_undegraded":"2024-09-16T01:38:11.620832+0000","last_fullsized":"2024-09-16T01:38:11.620832+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:09.180507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.561305+0000","last_change":"2024-09-16T01:38:08.570408+0000","last_active":"2024-09-16T01:38:39.561305+0000","last_peered":"2024-09-16T01:38:39.561305+0000","last_clean":"2024-09-16T01:38:39.561305+0000","last_became_active":"2024-09-16T01:38:08.566122+0000","last_became_peered":"2024-09-16T01:38:08.566122+0000","last_unstale":"2024-09-16T01:38:39.561305+0000","last_undegraded":"2024-09-16T01:38:39.561305+0000","last_fullsized":"2024-09-16T01:38:39.561305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:55.832320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620226+0000","last_change":"2024-09-16T01:38:08.555063+0000","last_active":"2024-09-16T01:38:11.620226+0000","last_peered":"2024-09-16T01:38:11.620226+0000","last_clean":"2024-09-16T01:38:11.620226+0000","last_became_active":"2024-09-16T01:38:08.554665+0000","last_became_peered":"2024-09-16T01:38:08.554665+0000","last_unstale":"2024-09-16T01:38:11.620226+0000","last_undegraded":"2024-09-16T01:38:11.620226+0000","last_fullsized":"2024-09-16T01:38:11.620226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:50.091017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620667+0000","last_change":"2024-09-16T01:38:01.524247+0000","last_active":"2024-09-16T01:38:11.620667+0000","last_peered":"2024-09-16T01:38:11.620667+0000","last_clean":"2024-09-16T01:38:11.620667+0000","last_became_active":"2024-09-16T01:38:01.523911+0000","last_became_peered":"2024-09-16T01:38:01.523911+0000","last_unstale":"2024-09-16T01:38:11.620667+0000","last_undegraded":"2024-09-16T01:38:11.620667+0000","last_fullsized":"2024-09-16T01:38:11.620667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:32.745882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:38.534097+0000","last_change":"2024-09-16T01:37:59.511272+0000","last_active":"2024-09-16T01:38:38.534097+0000","last_peered":"2024-09-16T01:38:38.534097+0000","last_clean":"2024-09-16T01:38:38.534097+0000","last_became_active":"2024-09-16T01:37:59.510920+0000","last_became_peered":"2024-09-16T01:37:59.510920+0000","last_unstale":"2024-09-16T01:38:38.534097+0000","last_undegraded":"2024-09-16T01:38:38.534097+0000","last_fullsized":"2024-09-16T01:38:38.534097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:59.406175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620991+0000","last_change":"2024-09-16T01:38:08.567173+0000","last_active":"2024-09-16T01:38:11.620991+0000","last_peered":"2024-09-16T01:38:11.620991+0000","last_clean":"2024-09-16T01:38:11.620991+0000","last_became_active":"2024-09-16T01:38:08.566788+0000","last_became_peered":"2024-09-16T01:38:08.566788+0000","last_unstale":"2024-09-16T01:38:11.620991+0000","last_undegraded":"2024-09-16T01:38:11.620991+0000","last_fullsized":"2024-09-16T01:38:11.620991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:35.743356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621167+0000","last_change":"2024-09-16T01:38:01.529529+0000","last_active":"2024-09-16T01:38:11.621167+0000","last_peered":"2024-09-16T01:38:11.621167+0000","last_clean":"2024-09-16T01:38:11.621167+0000","last_became_active":"2024-09-16T01:38:01.529048+0000","last_became_peered":"2024-09-16T01:38:01.529048+0000","last_unstale":"2024-09-16T01:38:11.621167+0000","last_undegraded":"2024-09-16T01:38:11.621167+0000","last_fullsized":"2024-09-16T01:38:11.621167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:35.041662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.560346+0000","last_change":"2024-09-16T01:37:59.799536+0000","last_active":"2024-09-16T01:38:39.560346+0000","last_peered":"2024-09-16T01:38:39.560346+0000","last_clean":"2024-09-16T01:38:39.560346+0000","last_became_active":"2024-09-16T01:37:59.799109+0000","last_became_peered":"2024-09-16T01:37:59.799109+0000","last_unstale":"2024-09-16T01:38:39.560346+0000","last_undegraded":"2024-09-16T01:38:39.560346+0000","last_fullsized":"2024-09-16T01:38:39.560346+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:07:25.007434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009492+0000","last_change":"2024-09-16T01:38:08.567019+0000","last_active":"2024-09-16T01:38:12.009492+0000","last_peered":"2024-09-16T01:38:12.009492+0000","last_clean":"2024-09-16T01:38:12.009492+0000","last_became_active":"2024-09-16T01:38:08.566667+0000","last_became_peered":"2024-09-16T01:38:08.566667+0000","last_unstale":"2024-09-16T01:38:12.009492+0000","last_undegraded":"2024-09-16T01:38:12.009492+0000","last_fullsized":"2024-09-16T01:38:12.009492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:11.375095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620513+0000","last_change":"2024-09-16T01:38:01.527965+0000","last_active":"2024-09-16T01:38:11.620513+0000","last_peered":"2024-09-16T01:38:11.620513+0000","last_clean":"2024-09-16T01:38:11.620513+0000","last_became_active":"2024-09-16T01:38:01.526688+0000","last_became_peered":"2024-09-16T01:38:01.526688+0000","last_unstale":"2024-09-16T01:38:11.620513+0000","last_undegraded":"2024-09-16T01:38:11.620513+0000","last_fullsized":"2024-09-16T01:38:11.620513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:06.618756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:48.582123+0000","last_change":"2024-09-16T01:37:59.510331+0000","last_active":"2024-09-16T01:38:48.582123+0000","last_peered":"2024-09-16T01:38:48.582123+0000","last_clean":"2024-09-16T01:38:48.582123+0000","last_became_active":"2024-09-16T01:37:59.509769+0000","last_became_peered":"2024-09-16T01:37:59.509769+0000","last_unstale":"2024-09-16T01:38:48.582123+0000","last_undegraded":"2024-09-16T01:38:48.582123+0000","last_fullsized":"2024-09-16T01:38:48.582123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:18.843091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551902+0000","last_change":"2024-09-16T01:38:08.564073+0000","last_active":"2024-09-16T01:38:11.551902+0000","last_peered":"2024-09-16T01:38:11.551902+0000","last_clean":"2024-09-16T01:38:11.551902+0000","last_became_active":"2024-09-16T01:38:08.563738+0000","last_became_peered":"2024-09-16T01:38:08.563738+0000","last_unstale":"2024-09-16T01:38:11.551902+0000","last_undegraded":"2024-09-16T01:38:11.551902+0000","last_fullsized":"2024-09-16T01:38:11.551902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:08.418956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620518+0000","last_change":"2024-09-16T01:38:01.521254+0000","last_active":"2024-09-16T01:38:11.620518+0000","last_peered":"2024-09-16T01:38:11.620518+0000","last_clean":"2024-09-16T01:38:11.620518+0000","last_became_active":"2024-09-16T01:38:01.520901+0000","last_became_peered":"2024-09-16T01:38:01.520901+0000","last_unstale":"2024-09-16T01:38:11.620518+0000","last_undegraded":"2024-09-16T01:38:11.620518+0000","last_fullsized":"2024-09-16T01:38:11.620518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:27.806954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.561026+0000","last_change":"2024-09-16T01:37:59.505990+0000","last_active":"2024-09-16T01:38:39.561026+0000","last_peered":"2024-09-16T01:38:39.561026+0000","last_clean":"2024-09-16T01:38:39.561026+0000","last_became_active":"2024-09-16T01:37:59.505564+0000","last_became_peered":"2024-09-16T01:37:59.505564+0000","last_unstale":"2024-09-16T01:38:39.561026+0000","last_undegraded":"2024-09-16T01:38:39.561026+0000","last_fullsized":"2024-09-16T01:38:39.561026+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:01.496609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620939+0000","last_change":"2024-09-16T01:38:08.562838+0000","last_active":"2024-09-16T01:38:11.620939+0000","last_peered":"2024-09-16T01:38:11.620939+0000","last_clean":"2024-09-16T01:38:11.620939+0000","last_became_active":"2024-09-16T01:38:08.562426+0000","last_became_peered":"2024-09-16T01:38:08.562426+0000","last_unstale":"2024-09-16T01:38:11.620939+0000","last_undegraded":"2024-09-16T01:38:11.620939+0000","last_fullsized":"2024-09-16T01:38:11.620939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:20.685626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009218+0000","last_change":"2024-09-16T01:38:01.514983+0000","last_active":"2024-09-16T01:38:12.009218+0000","last_peered":"2024-09-16T01:38:12.009218+0000","last_clean":"2024-09-16T01:38:12.009218+0000","last_became_active":"2024-09-16T01:38:01.514648+0000","last_became_peered":"2024-09-16T01:38:01.514648+0000","last_unstale":"2024-09-16T01:38:12.009218+0000","last_undegraded":"2024-09-16T01:38:12.009218+0000","last_fullsized":"2024-09-16T01:38:12.009218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:57.052564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:40.567002+0000","last_change":"2024-09-16T01:37:59.505141+0000","last_active":"2024-09-16T01:38:40.567002+0000","last_peered":"2024-09-16T01:38:40.567002+0000","last_clean":"2024-09-16T01:38:40.567002+0000","last_became_active":"2024-09-16T01:37:59.504730+0000","last_became_peered":"2024-09-16T01:37:59.504730+0000","last_unstale":"2024-09-16T01:38:40.567002+0000","last_undegraded":"2024-09-16T01:38:40.567002+0000","last_fullsized":"2024-09-16T01:38:40.567002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:54.504661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621780+0000","last_change":"2024-09-16T01:38:08.563970+0000","last_active":"2024-09-16T01:38:11.621780+0000","last_peered":"2024-09-16T01:38:11.621780+0000","last_clean":"2024-09-16T01:38:11.621780+0000","last_became_active":"2024-09-16T01:38:08.563555+0000","last_became_peered":"2024-09-16T01:38:08.563555+0000","last_unstale":"2024-09-16T01:38:11.621780+0000","last_undegraded":"2024-09-16T01:38:11.621780+0000","last_fullsized":"2024-09-16T01:38:11.621780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:27:12.086043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621032+0000","last_change":"2024-09-16T01:38:01.521016+0000","last_active":"2024-09-16T01:38:11.621032+0000","last_peered":"2024-09-16T01:38:11.621032+0000","last_clean":"2024-09-16T01:38:11.621032+0000","last_became_active":"2024-09-16T01:38:01.520656+0000","last_became_peered":"2024-09-16T01:38:01.520656+0000","last_unstale":"2024-09-16T01:38:11.621032+0000","last_undegraded":"2024-09-16T01:38:11.621032+0000","last_fullsized":"2024-09-16T01:38:11.621032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:39.001239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'27","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:48.585168+0000","last_change":"2024-09-16T01:37:59.507235+0000","last_active":"2024-09-16T01:38:48.585168+0000","last_peered":"2024-09-16T01:38:48.585168+0000","last_clean":"2024-09-16T01:38:48.585168+0000","last_became_active":"2024-09-16T01:37:59.506255+0000","last_became_peered":"2024-09-16T01:37:59.506255+0000","last_unstale":"2024-09-16T01:38:48.585168+0000","last_undegraded":"2024-09-16T01:38:48.585168+0000","last_fullsized":"2024-09-16T01:38:48.585168+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:44.515663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":54,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551520+0000","last_change":"2024-09-16T01:38:08.558293+0000","last_active":"2024-09-16T01:38:11.551520+0000","last_peered":"2024-09-16T01:38:11.551520+0000","last_clean":"2024-09-16T01:38:11.551520+0000","last_became_active":"2024-09-16T01:38:08.557958+0000","last_became_peered":"2024-09-16T01:38:08.557958+0000","last_unstale":"2024-09-16T01:38:11.551520+0000","last_undegraded":"2024-09-16T01:38:11.551520+0000","last_fullsized":"2024-09-16T01:38:11.551520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:44:56.783271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621689+0000","last_change":"2024-09-16T01:38:01.526641+0000","last_active":"2024-09-16T01:38:11.621689+0000","last_peered":"2024-09-16T01:38:11.621689+0000","last_clean":"2024-09-16T01:38:11.621689+0000","last_became_active":"2024-09-16T01:38:01.526278+0000","last_became_peered":"2024-09-16T01:38:01.526278+0000","last_unstale":"2024-09-16T01:38:11.621689+0000","last_undegraded":"2024-09-16T01:38:11.621689+0000","last_fullsized":"2024-09-16T01:38:11.621689+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:56.156593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:38.547604+0000","last_change":"2024-09-16T01:37:59.800065+0000","last_active":"2024-09-16T01:38:38.547604+0000","last_peered":"2024-09-16T01:38:38.547604+0000","last_clean":"2024-09-16T01:38:38.547604+0000","last_became_active":"2024-09-16T01:37:59.799705+0000","last_became_peered":"2024-09-16T01:37:59.799705+0000","last_unstale":"2024-09-16T01:38:38.547604+0000","last_undegraded":"2024-09-16T01:38:38.547604+0000","last_fullsized":"2024-09-16T01:38:38.547604+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:13.791875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621382+0000","last_change":"2024-09-16T01:38:08.556360+0000","last_active":"2024-09-16T01:38:11.621382+0000","last_peered":"2024-09-16T01:38:11.621382+0000","last_clean":"2024-09-16T01:38:11.621382+0000","last_became_active":"2024-09-16T01:38:08.555219+0000","last_became_peered":"2024-09-16T01:38:08.555219+0000","last_unstale":"2024-09-16T01:38:11.621382+0000","last_undegraded":"2024-09-16T01:38:11.621382+0000","last_fullsized":"2024-09-16T01:38:11.621382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:36.464760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620417+0000","last_change":"2024-09-16T01:38:01.526758+0000","last_active":"2024-09-16T01:38:11.620417+0000","last_peered":"2024-09-16T01:38:11.620417+0000","last_clean":"2024-09-16T01:38:11.620417+0000","last_became_active":"2024-09-16T01:38:01.526545+0000","last_became_peered":"2024-09-16T01:38:01.526545+0000","last_unstale":"2024-09-16T01:38:11.620417+0000","last_undegraded":"2024-09-16T01:38:11.620417+0000","last_fullsized":"2024-09-16T01:38:11.620417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:47.185489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.561227+0000","last_change":"2024-09-16T01:38:08.555125+0000","last_active":"2024-09-16T01:38:39.561227+0000","last_peered":"2024-09-16T01:38:39.561227+0000","last_clean":"2024-09-16T01:38:39.561227+0000","last_became_active":"2024-09-16T01:38:08.554756+0000","last_became_peered":"2024-09-16T01:38:08.554756+0000","last_unstale":"2024-09-16T01:38:39.561227+0000","last_undegraded":"2024-09-16T01:38:39.561227+0000","last_fullsized":"2024-09-16T01:38:39.561227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:08.888006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551318+0000","last_change":"2024-09-16T01:38:08.557906+0000","last_active":"2024-09-16T01:38:11.551318+0000","last_peered":"2024-09-16T01:38:11.551318+0000","last_clean":"2024-09-16T01:38:11.551318+0000","last_became_active":"2024-09-16T01:38:08.557546+0000","last_became_peered":"2024-09-16T01:38:08.557546+0000","last_unstale":"2024-09-16T01:38:11.551318+0000","last_undegraded":"2024-09-16T01:38:11.551318+0000","last_fullsized":"2024-09-16T01:38:11.551318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:27:04.988778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.622024+0000","last_change":"2024-09-16T01:38:01.528430+0000","last_active":"2024-09-16T01:38:11.622024+0000","last_peered":"2024-09-16T01:38:11.622024+0000","last_clean":"2024-09-16T01:38:11.622024+0000","last_became_active":"2024-09-16T01:38:01.528069+0000","last_became_peered":"2024-09-16T01:38:01.528069+0000","last_unstale":"2024-09-16T01:38:11.622024+0000","last_undegraded":"2024-09-16T01:38:11.622024+0000","last_fullsized":"2024-09-16T01:38:11.622024+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:41.317215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:48.581973+0000","last_change":"2024-09-16T01:37:59.509017+0000","last_active":"2024-09-16T01:38:48.581973+0000","last_peered":"2024-09-16T01:38:48.581973+0000","last_clean":"2024-09-16T01:38:48.581973+0000","last_became_active":"2024-09-16T01:37:59.508252+0000","last_became_peered":"2024-09-16T01:37:59.508252+0000","last_unstale":"2024-09-16T01:38:48.581973+0000","last_undegraded":"2024-09-16T01:38:48.581973+0000","last_fullsized":"2024-09-16T01:38:48.581973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:26.215407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619849+0000","last_change":"2024-09-16T01:38:08.565197+0000","last_active":"2024-09-16T01:38:11.619849+0000","last_peered":"2024-09-16T01:38:11.619849+0000","last_clean":"2024-09-16T01:38:11.619849+0000","last_became_active":"2024-09-16T01:38:08.562773+0000","last_became_peered":"2024-09-16T01:38:08.562773+0000","last_unstale":"2024-09-16T01:38:11.619849+0000","last_undegraded":"2024-09-16T01:38:11.619849+0000","last_fullsized":"2024-09-16T01:38:11.619849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:13.316950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619921+0000","last_change":"2024-09-16T01:38:08.554052+0000","last_active":"2024-09-16T01:38:11.619921+0000","last_peered":"2024-09-16T01:38:11.619921+0000","last_clean":"2024-09-16T01:38:11.619921+0000","last_became_active":"2024-09-16T01:38:08.553660+0000","last_became_peered":"2024-09-16T01:38:08.553660+0000","last_unstale":"2024-09-16T01:38:11.619921+0000","last_undegraded":"2024-09-16T01:38:11.619921+0000","last_fullsized":"2024-09-16T01:38:11.619921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:53:52.270695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551393+0000","last_change":"2024-09-16T01:38:01.522688+0000","last_active":"2024-09-16T01:38:11.551393+0000","last_peered":"2024-09-16T01:38:11.551393+0000","last_clean":"2024-09-16T01:38:11.551393+0000","last_became_active":"2024-09-16T01:38:01.519843+0000","last_became_peered":"2024-09-16T01:38:01.519843+0000","last_unstale":"2024-09-16T01:38:11.551393+0000","last_undegraded":"2024-09-16T01:38:11.551393+0000","last_fullsized":"2024-09-16T01:38:11.551393+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:50.971760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.561005+0000","last_change":"2024-09-16T01:37:59.509044+0000","last_active":"2024-09-16T01:38:39.561005+0000","last_peered":"2024-09-16T01:38:39.561005+0000","last_clean":"2024-09-16T01:38:39.561005+0000","last_became_active":"2024-09-16T01:37:59.507085+0000","last_became_peered":"2024-09-16T01:37:59.507085+0000","last_unstale":"2024-09-16T01:38:39.561005+0000","last_undegraded":"2024-09-16T01:38:39.561005+0000","last_fullsized":"2024-09-16T01:38:39.561005+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:25:09.930945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620278+0000","last_change":"2024-09-16T01:38:08.561250+0000","last_active":"2024-09-16T01:38:11.620278+0000","last_peered":"2024-09-16T01:38:11.620278+0000","last_clean":"2024-09-16T01:38:11.620278+0000","last_became_active":"2024-09-16T01:38:08.560866+0000","last_became_peered":"2024-09-16T01:38:08.560866+0000","last_unstale":"2024-09-16T01:38:11.620278+0000","last_undegraded":"2024-09-16T01:38:11.620278+0000","last_fullsized":"2024-09-16T01:38:11.620278+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:31.253817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619727+0000","last_change":"2024-09-16T01:38:01.517400+0000","last_active":"2024-09-16T01:38:11.619727+0000","last_peered":"2024-09-16T01:38:11.619727+0000","last_clean":"2024-09-16T01:38:11.619727+0000","last_became_active":"2024-09-16T01:38:01.517103+0000","last_became_peered":"2024-09-16T01:38:01.517103+0000","last_unstale":"2024-09-16T01:38:11.619727+0000","last_undegraded":"2024-09-16T01:38:11.619727+0000","last_fullsized":"2024-09-16T01:38:11.619727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:37.886179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:37.539100+0000","last_change":"2024-09-16T01:37:59.503370+0000","last_active":"2024-09-16T01:38:37.539100+0000","last_peered":"2024-09-16T01:38:37.539100+0000","last_clean":"2024-09-16T01:38:37.539100+0000","last_became_active":"2024-09-16T01:37:59.503071+0000","last_became_peered":"2024-09-16T01:37:59.503071+0000","last_unstale":"2024-09-16T01:38:37.539100+0000","last_undegraded":"2024-09-16T01:38:37.539100+0000","last_fullsized":"2024-09-16T01:38:37.539100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:51.207702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:38.547216+0000","last_change":"2024-09-16T01:37:59.506110+0000","last_active":"2024-09-16T01:38:38.547216+0000","last_peered":"2024-09-16T01:38:38.547216+0000","last_clean":"2024-09-16T01:38:38.547216+0000","last_became_active":"2024-09-16T01:37:59.505264+0000","last_became_peered":"2024-09-16T01:37:59.505264+0000","last_unstale":"2024-09-16T01:38:38.547216+0000","last_undegraded":"2024-09-16T01:38:38.547216+0000","last_fullsized":"2024-09-16T01:38:38.547216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:35.986433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620965+0000","last_change":"2024-09-16T01:38:08.563900+0000","last_active":"2024-09-16T01:38:11.620965+0000","last_peered":"2024-09-16T01:38:11.620965+0000","last_clean":"2024-09-16T01:38:11.620965+0000","last_became_active":"2024-09-16T01:38:08.563551+0000","last_became_peered":"2024-09-16T01:38:08.563551+0000","last_unstale":"2024-09-16T01:38:11.620965+0000","last_undegraded":"2024-09-16T01:38:11.620965+0000","last_fullsized":"2024-09-16T01:38:11.620965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:31:03.979580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552533+0000","last_change":"2024-09-16T01:38:08.556011+0000","last_active":"2024-09-16T01:38:11.552533+0000","last_peered":"2024-09-16T01:38:11.552533+0000","last_clean":"2024-09-16T01:38:11.552533+0000","last_became_active":"2024-09-16T01:38:08.555615+0000","last_became_peered":"2024-09-16T01:38:08.555615+0000","last_unstale":"2024-09-16T01:38:11.552533+0000","last_undegraded":"2024-09-16T01:38:11.552533+0000","last_fullsized":"2024-09-16T01:38:11.552533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:34.434974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:49.584444+0000","last_change":"2024-09-16T01:37:59.508411+0000","last_active":"2024-09-16T01:38:49.584444+0000","last_peered":"2024-09-16T01:38:49.584444+0000","last_clean":"2024-09-16T01:38:49.584444+0000","last_became_active":"2024-09-16T01:37:59.508058+0000","last_became_peered":"2024-09-16T01:37:59.508058+0000","last_unstale":"2024-09-16T01:38:49.584444+0000","last_undegraded":"2024-09-16T01:38:49.584444+0000","last_fullsized":"2024-09-16T01:38:49.584444+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:47.441673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621306+0000","last_change":"2024-09-16T01:38:08.560930+0000","last_active":"2024-09-16T01:38:11.621306+0000","last_peered":"2024-09-16T01:38:11.621306+0000","last_clean":"2024-09-16T01:38:11.621306+0000","last_became_active":"2024-09-16T01:38:08.560490+0000","last_became_peered":"2024-09-16T01:38:08.560490+0000","last_unstale":"2024-09-16T01:38:11.621306+0000","last_undegraded":"2024-09-16T01:38:11.621306+0000","last_fullsized":"2024-09-16T01:38:11.621306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:11.556113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552811+0000","last_change":"2024-09-16T01:38:08.557564+0000","last_active":"2024-09-16T01:38:11.552811+0000","last_peered":"2024-09-16T01:38:11.552811+0000","last_clean":"2024-09-16T01:38:11.552811+0000","last_became_active":"2024-09-16T01:38:08.557143+0000","last_became_peered":"2024-09-16T01:38:08.557143+0000","last_unstale":"2024-09-16T01:38:11.552811+0000","last_undegraded":"2024-09-16T01:38:11.552811+0000","last_fullsized":"2024-09-16T01:38:11.552811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:43.148870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620013+0000","last_change":"2024-09-16T01:38:01.527353+0000","last_active":"2024-09-16T01:38:11.620013+0000","last_peered":"2024-09-16T01:38:11.620013+0000","last_clean":"2024-09-16T01:38:11.620013+0000","last_became_active":"2024-09-16T01:38:01.527031+0000","last_became_peered":"2024-09-16T01:38:01.527031+0000","last_unstale":"2024-09-16T01:38:11.620013+0000","last_undegraded":"2024-09-16T01:38:11.620013+0000","last_fullsized":"2024-09-16T01:38:11.620013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:27.311638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.553003+0000","last_change":"2024-09-16T01:38:01.526409+0000","last_active":"2024-09-16T01:38:11.553003+0000","last_peered":"2024-09-16T01:38:11.553003+0000","last_clean":"2024-09-16T01:38:11.553003+0000","last_became_active":"2024-09-16T01:38:01.526064+0000","last_became_peered":"2024-09-16T01:38:01.526064+0000","last_unstale":"2024-09-16T01:38:11.553003+0000","last_undegraded":"2024-09-16T01:38:11.553003+0000","last_fullsized":"2024-09-16T01:38:11.553003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:51.571808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552380+0000","last_change":"2024-09-16T01:38:01.521856+0000","last_active":"2024-09-16T01:38:11.552380+0000","last_peered":"2024-09-16T01:38:11.552380+0000","last_clean":"2024-09-16T01:38:11.552380+0000","last_became_active":"2024-09-16T01:38:01.520312+0000","last_became_peered":"2024-09-16T01:38:01.520312+0000","last_unstale":"2024-09-16T01:38:11.552380+0000","last_undegraded":"2024-09-16T01:38:11.552380+0000","last_fullsized":"2024-09-16T01:38:11.552380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:53.618990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620192+0000","last_change":"2024-09-16T01:38:01.520958+0000","last_active":"2024-09-16T01:38:11.620192+0000","last_peered":"2024-09-16T01:38:11.620192+0000","last_clean":"2024-09-16T01:38:11.620192+0000","last_became_active":"2024-09-16T01:38:01.520658+0000","last_became_peered":"2024-09-16T01:38:01.520658+0000","last_unstale":"2024-09-16T01:38:11.620192+0000","last_undegraded":"2024-09-16T01:38:11.620192+0000","last_fullsized":"2024-09-16T01:38:11.620192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:23.770694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009571+0000","last_change":"2024-09-16T01:38:01.519140+0000","last_active":"2024-09-16T01:38:12.009571+0000","last_peered":"2024-09-16T01:38:12.009571+0000","last_clean":"2024-09-16T01:38:12.009571+0000","last_became_active":"2024-09-16T01:38:01.518765+0000","last_became_peered":"2024-09-16T01:38:01.518765+0000","last_unstale":"2024-09-16T01:38:12.009571+0000","last_undegraded":"2024-09-16T01:38:12.009571+0000","last_fullsized":"2024-09-16T01:38:12.009571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:17.194886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621459+0000","last_change":"2024-09-16T01:38:01.523762+0000","last_active":"2024-09-16T01:38:11.621459+0000","last_peered":"2024-09-16T01:38:11.621459+0000","last_clean":"2024-09-16T01:38:11.621459+0000","last_became_active":"2024-09-16T01:38:01.522035+0000","last_became_peered":"2024-09-16T01:38:01.522035+0000","last_unstale":"2024-09-16T01:38:11.621459+0000","last_undegraded":"2024-09-16T01:38:11.621459+0000","last_fullsized":"2024-09-16T01:38:11.621459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:40.405350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552992+0000","last_change":"2024-09-16T01:38:01.527539+0000","last_active":"2024-09-16T01:38:11.552992+0000","last_peered":"2024-09-16T01:38:11.552992+0000","last_clean":"2024-09-16T01:38:11.552992+0000","last_became_active":"2024-09-16T01:38:01.526974+0000","last_became_peered":"2024-09-16T01:38:01.526974+0000","last_unstale":"2024-09-16T01:38:11.552992+0000","last_undegraded":"2024-09-16T01:38:11.552992+0000","last_fullsized":"2024-09-16T01:38:11.552992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:30:03.587141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620841+0000","last_change":"2024-09-16T01:38:01.529131+0000","last_active":"2024-09-16T01:38:11.620841+0000","last_peered":"2024-09-16T01:38:11.620841+0000","last_clean":"2024-09-16T01:38:11.620841+0000","last_became_active":"2024-09-16T01:38:01.528796+0000","last_became_peered":"2024-09-16T01:38:01.528796+0000","last_unstale":"2024-09-16T01:38:11.620841+0000","last_undegraded":"2024-09-16T01:38:11.620841+0000","last_fullsized":"2024-09-16T01:38:11.620841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:38.903638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620130+0000","last_change":"2024-09-16T01:38:01.514779+0000","last_active":"2024-09-16T01:38:11.620130+0000","last_peered":"2024-09-16T01:38:11.620130+0000","last_clean":"2024-09-16T01:38:11.620130+0000","last_became_active":"2024-09-16T01:38:01.514434+0000","last_became_peered":"2024-09-16T01:38:01.514434+0000","last_unstale":"2024-09-16T01:38:11.620130+0000","last_undegraded":"2024-09-16T01:38:11.620130+0000","last_fullsized":"2024-09-16T01:38:11.620130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:31.382111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619991+0000","last_change":"2024-09-16T01:38:01.521771+0000","last_active":"2024-09-16T01:38:11.619991+0000","last_peered":"2024-09-16T01:38:11.619991+0000","last_clean":"2024-09-16T01:38:11.619991+0000","last_became_active":"2024-09-16T01:38:01.521288+0000","last_became_peered":"2024-09-16T01:38:01.521288+0000","last_unstale":"2024-09-16T01:38:11.619991+0000","last_undegraded":"2024-09-16T01:38:11.619991+0000","last_fullsized":"2024-09-16T01:38:11.619991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:11.825164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621313+0000","last_change":"2024-09-16T01:38:01.529247+0000","last_active":"2024-09-16T01:38:11.621313+0000","last_peered":"2024-09-16T01:38:11.621313+0000","last_clean":"2024-09-16T01:38:11.621313+0000","last_became_active":"2024-09-16T01:38:01.528926+0000","last_became_peered":"2024-09-16T01:38:01.528926+0000","last_unstale":"2024-09-16T01:38:11.621313+0000","last_undegraded":"2024-09-16T01:38:11.621313+0000","last_fullsized":"2024-09-16T01:38:11.621313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:20.704114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009229+0000","last_change":"2024-09-16T01:38:01.520830+0000","last_active":"2024-09-16T01:38:12.009229+0000","last_peered":"2024-09-16T01:38:12.009229+0000","last_clean":"2024-09-16T01:38:12.009229+0000","last_became_active":"2024-09-16T01:38:01.520526+0000","last_became_peered":"2024-09-16T01:38:01.520526+0000","last_unstale":"2024-09-16T01:38:12.009229+0000","last_undegraded":"2024-09-16T01:38:12.009229+0000","last_fullsized":"2024-09-16T01:38:12.009229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:28.062812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621042+0000","last_change":"2024-09-16T01:38:01.521983+0000","last_active":"2024-09-16T01:38:11.621042+0000","last_peered":"2024-09-16T01:38:11.621042+0000","last_clean":"2024-09-16T01:38:11.621042+0000","last_became_active":"2024-09-16T01:38:01.521666+0000","last_became_peered":"2024-09-16T01:38:01.521666+0000","last_unstale":"2024-09-16T01:38:11.621042+0000","last_undegraded":"2024-09-16T01:38:11.621042+0000","last_fullsized":"2024-09-16T01:38:11.621042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:50:55.761313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.553027+0000","last_change":"2024-09-16T01:38:08.554304+0000","last_active":"2024-09-16T01:38:11.553027+0000","last_peered":"2024-09-16T01:38:11.553027+0000","last_clean":"2024-09-16T01:38:11.553027+0000","last_became_active":"2024-09-16T01:38:08.553683+0000","last_became_peered":"2024-09-16T01:38:08.553683+0000","last_unstale":"2024-09-16T01:38:11.553027+0000","last_undegraded":"2024-09-16T01:38:11.553027+0000","last_fullsized":"2024-09-16T01:38:11.553027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:56.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551871+0000","last_change":"2024-09-16T01:38:01.520425+0000","last_active":"2024-09-16T01:38:11.551871+0000","last_peered":"2024-09-16T01:38:11.551871+0000","last_clean":"2024-09-16T01:38:11.551871+0000","last_became_active":"2024-09-16T01:38:01.519858+0000","last_became_peered":"2024-09-16T01:38:01.519858+0000","last_unstale":"2024-09-16T01:38:11.551871+0000","last_undegraded":"2024-09-16T01:38:11.551871+0000","last_fullsized":"2024-09-16T01:38:11.551871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:48.534298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620926+0000","last_change":"2024-09-16T01:38:01.532762+0000","last_active":"2024-09-16T01:38:11.620926+0000","last_peered":"2024-09-16T01:38:11.620926+0000","last_clean":"2024-09-16T01:38:11.620926+0000","last_became_active":"2024-09-16T01:38:01.532404+0000","last_became_peered":"2024-09-16T01:38:01.532404+0000","last_unstale":"2024-09-16T01:38:11.620926+0000","last_undegraded":"2024-09-16T01:38:11.620926+0000","last_fullsized":"2024-09-16T01:38:11.620926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:03.135141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620376+0000","last_change":"2024-09-16T01:38:01.530205+0000","last_active":"2024-09-16T01:38:11.620376+0000","last_peered":"2024-09-16T01:38:11.620376+0000","last_clean":"2024-09-16T01:38:11.620376+0000","last_became_active":"2024-09-16T01:38:01.529898+0000","last_became_peered":"2024-09-16T01:38:01.529898+0000","last_unstale":"2024-09-16T01:38:11.620376+0000","last_undegraded":"2024-09-16T01:38:11.620376+0000","last_fullsized":"2024-09-16T01:38:11.620376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:48.444441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619812+0000","last_change":"2024-09-16T01:38:01.529407+0000","last_active":"2024-09-16T01:38:11.619812+0000","last_peered":"2024-09-16T01:38:11.619812+0000","last_clean":"2024-09-16T01:38:11.619812+0000","last_became_active":"2024-09-16T01:38:01.528330+0000","last_became_peered":"2024-09-16T01:38:01.528330+0000","last_unstale":"2024-09-16T01:38:11.619812+0000","last_undegraded":"2024-09-16T01:38:11.619812+0000","last_fullsized":"2024-09-16T01:38:11.619812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:32.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619804+0000","last_change":"2024-09-16T01:38:01.519579+0000","last_active":"2024-09-16T01:38:11.619804+0000","last_peered":"2024-09-16T01:38:11.619804+0000","last_clean":"2024-09-16T01:38:11.619804+0000","last_became_active":"2024-09-16T01:38:01.519211+0000","last_became_peered":"2024-09-16T01:38:01.519211+0000","last_unstale":"2024-09-16T01:38:11.619804+0000","last_undegraded":"2024-09-16T01:38:11.619804+0000","last_fullsized":"2024-09-16T01:38:11.619804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:18.861169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009215+0000","last_change":"2024-09-16T01:38:01.521136+0000","last_active":"2024-09-16T01:38:12.009215+0000","last_peered":"2024-09-16T01:38:12.009215+0000","last_clean":"2024-09-16T01:38:12.009215+0000","last_became_active":"2024-09-16T01:38:01.520819+0000","last_became_peered":"2024-09-16T01:38:01.520819+0000","last_unstale":"2024-09-16T01:38:12.009215+0000","last_undegraded":"2024-09-16T01:38:12.009215+0000","last_fullsized":"2024-09-16T01:38:12.009215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:28.150822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621953+0000","last_change":"2024-09-16T01:38:01.519382+0000","last_active":"2024-09-16T01:38:11.621953+0000","last_peered":"2024-09-16T01:38:11.621953+0000","last_clean":"2024-09-16T01:38:11.621953+0000","last_became_active":"2024-09-16T01:38:01.519039+0000","last_became_peered":"2024-09-16T01:38:01.519039+0000","last_unstale":"2024-09-16T01:38:11.621953+0000","last_undegraded":"2024-09-16T01:38:11.621953+0000","last_fullsized":"2024-09-16T01:38:11.621953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:46.559357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621100+0000","last_change":"2024-09-16T01:38:01.527513+0000","last_active":"2024-09-16T01:38:11.621100+0000","last_peered":"2024-09-16T01:38:11.621100+0000","last_clean":"2024-09-16T01:38:11.621100+0000","last_became_active":"2024-09-16T01:38:01.527184+0000","last_became_peered":"2024-09-16T01:38:01.527184+0000","last_unstale":"2024-09-16T01:38:11.621100+0000","last_undegraded":"2024-09-16T01:38:11.621100+0000","last_fullsized":"2024-09-16T01:38:11.621100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:41.508318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.553164+0000","last_change":"2024-09-16T01:38:01.524239+0000","last_active":"2024-09-16T01:38:11.553164+0000","last_peered":"2024-09-16T01:38:11.553164+0000","last_clean":"2024-09-16T01:38:11.553164+0000","last_became_active":"2024-09-16T01:38:01.523906+0000","last_became_peered":"2024-09-16T01:38:01.523906+0000","last_unstale":"2024-09-16T01:38:11.553164+0000","last_undegraded":"2024-09-16T01:38:11.553164+0000","last_fullsized":"2024-09-16T01:38:11.553164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:11.718796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621894+0000","last_change":"2024-09-16T01:38:01.531564+0000","last_active":"2024-09-16T01:38:11.621894+0000","last_peered":"2024-09-16T01:38:11.621894+0000","last_clean":"2024-09-16T01:38:11.621894+0000","last_became_active":"2024-09-16T01:38:01.531241+0000","last_became_peered":"2024-09-16T01:38:01.531241+0000","last_unstale":"2024-09-16T01:38:11.621894+0000","last_undegraded":"2024-09-16T01:38:11.621894+0000","last_fullsized":"2024-09-16T01:38:11.621894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:16.244641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619652+0000","last_change":"2024-09-16T01:38:01.519913+0000","last_active":"2024-09-16T01:38:11.619652+0000","last_peered":"2024-09-16T01:38:11.619652+0000","last_clean":"2024-09-16T01:38:11.619652+0000","last_became_active":"2024-09-16T01:38:01.518631+0000","last_became_peered":"2024-09-16T01:38:01.518631+0000","last_unstale":"2024-09-16T01:38:11.619652+0000","last_undegraded":"2024-09-16T01:38:11.619652+0000","last_fullsized":"2024-09-16T01:38:11.619652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:48.747188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620304+0000","last_change":"2024-09-16T01:38:01.530805+0000","last_active":"2024-09-16T01:38:11.620304+0000","last_peered":"2024-09-16T01:38:11.620304+0000","last_clean":"2024-09-16T01:38:11.620304+0000","last_became_active":"2024-09-16T01:38:01.530466+0000","last_became_peered":"2024-09-16T01:38:01.530466+0000","last_unstale":"2024-09-16T01:38:11.620304+0000","last_undegraded":"2024-09-16T01:38:11.620304+0000","last_fullsized":"2024-09-16T01:38:11.620304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:41:30.587398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552664+0000","last_change":"2024-09-16T01:38:01.519227+0000","last_active":"2024-09-16T01:38:11.552664+0000","last_peered":"2024-09-16T01:38:11.552664+0000","last_clean":"2024-09-16T01:38:11.552664+0000","last_became_active":"2024-09-16T01:38:01.518875+0000","last_became_peered":"2024-09-16T01:38:01.518875+0000","last_unstale":"2024-09-16T01:38:11.552664+0000","last_undegraded":"2024-09-16T01:38:11.552664+0000","last_fullsized":"2024-09-16T01:38:11.552664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:42.654812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620696+0000","last_change":"2024-09-16T01:38:01.513327+0000","last_active":"2024-09-16T01:38:11.620696+0000","last_peered":"2024-09-16T01:38:11.620696+0000","last_clean":"2024-09-16T01:38:11.620696+0000","last_became_active":"2024-09-16T01:38:01.513000+0000","last_became_peered":"2024-09-16T01:38:01.513000+0000","last_unstale":"2024-09-16T01:38:11.620696+0000","last_undegraded":"2024-09-16T01:38:11.620696+0000","last_fullsized":"2024-09-16T01:38:11.620696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:43.036805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551739+0000","last_change":"2024-09-16T01:38:08.560560+0000","last_active":"2024-09-16T01:38:11.551739+0000","last_peered":"2024-09-16T01:38:11.551739+0000","last_clean":"2024-09-16T01:38:11.551739+0000","last_became_active":"2024-09-16T01:38:08.560229+0000","last_became_peered":"2024-09-16T01:38:08.560229+0000","last_unstale":"2024-09-16T01:38:11.551739+0000","last_undegraded":"2024-09-16T01:38:11.551739+0000","last_fullsized":"2024-09-16T01:38:11.551739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:10:09.462043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619851+0000","last_change":"2024-09-16T01:38:01.529548+0000","last_active":"2024-09-16T01:38:11.619851+0000","last_peered":"2024-09-16T01:38:11.619851+0000","last_clean":"2024-09-16T01:38:11.619851+0000","last_became_active":"2024-09-16T01:38:01.529011+0000","last_became_peered":"2024-09-16T01:38:01.529011+0000","last_unstale":"2024-09-16T01:38:11.619851+0000","last_undegraded":"2024-09-16T01:38:11.619851+0000","last_fullsized":"2024-09-16T01:38:11.619851+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:21.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620009+0000","last_change":"2024-09-16T01:38:01.519488+0000","last_active":"2024-09-16T01:38:11.620009+0000","last_peered":"2024-09-16T01:38:11.620009+0000","last_clean":"2024-09-16T01:38:11.620009+0000","last_became_active":"2024-09-16T01:38:01.519156+0000","last_became_peered":"2024-09-16T01:38:01.519156+0000","last_unstale":"2024-09-16T01:38:11.620009+0000","last_undegraded":"2024-09-16T01:38:11.620009+0000","last_fullsized":"2024-09-16T01:38:11.620009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:18.149874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620820+0000","last_change":"2024-09-16T01:38:01.527519+0000","last_active":"2024-09-16T01:38:11.620820+0000","last_peered":"2024-09-16T01:38:11.620820+0000","last_clean":"2024-09-16T01:38:11.620820+0000","last_became_active":"2024-09-16T01:38:01.527175+0000","last_became_peered":"2024-09-16T01:38:01.527175+0000","last_unstale":"2024-09-16T01:38:11.620820+0000","last_undegraded":"2024-09-16T01:38:11.620820+0000","last_fullsized":"2024-09-16T01:38:11.620820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:04.958842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50160,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":137,"num_read_kb":127,"num_write":237,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":150480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281435,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678039,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":469646,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906972,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":1099302,"data_compressed":53519,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070498,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":473167,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299430,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":468665,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528363,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":1065555,"data_compressed":53519,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659185,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":464575,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888118,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":1061137,"data_compressed":53519,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051643,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":468649,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280576,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":464283,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476805,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":509563,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607626,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":509622,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":466424,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34155,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34155,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34155,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":45769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":45710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":45081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T01:38:54.572 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T01:38:54.818 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:54.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:54 smithi139 ceph-mon[31175]: pgmap v306: 129 pgs: 129 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 18 op/s 2024-09-16T01:38:54.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:54 smithi176 ceph-mon[35324]: pgmap v306: 129 pgs: 129 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 18 op/s 2024-09-16T01:38:55.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:54 smithi164 ceph-mon[35121]: pgmap v306: 129 pgs: 129 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 18 op/s 2024-09-16T01:38:55.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.302+0000 7f75d4b9a640 1 -- 172.21.15.139:0/466004111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d010c170 msgr2=0x7f75d010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:55.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.302+0000 7f75d4b9a640 1 --2- 172.21.15.139:0/466004111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d010c170 0x7f75d010e610 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009ca0 tx=0x7f75c802a950 comp rx=0 tx=0).stop 2024-09-16T01:38:55.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.303+0000 7f75d4b9a640 1 -- 172.21.15.139:0/466004111 shutdown_connections 2024-09-16T01:38:55.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.303+0000 7f75d4b9a640 1 --2- 172.21.15.139:0/466004111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d010c170 0x7f75d010e610 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:55.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.303+0000 7f75d4b9a640 1 --2- 172.21.15.139:0/466004111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75d0103660 0x7f75d010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:55.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.303+0000 7f75d4b9a640 1 --2- 172.21.15.139:0/466004111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d0102d40 0x7f75d0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:55.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.303+0000 7f75d4b9a640 1 -- 172.21.15.139:0/466004111 >> 172.21.15.139:0/466004111 conn(0x7f75d00fc7a0 msgr2=0x7f75d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:55.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.303+0000 7f75d4b9a640 1 -- 172.21.15.139:0/466004111 shutdown_connections 2024-09-16T01:38:55.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.303+0000 7f75d4b9a640 1 -- 172.21.15.139:0/466004111 wait complete. 2024-09-16T01:38:55.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.303+0000 7f75d4b9a640 1 Processor -- start 2024-09-16T01:38:55.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.303+0000 7f75d4b9a640 1 -- start start 2024-09-16T01:38:55.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.303+0000 7f75d4b9a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d0102d40 0x7f75d01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:55.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.303+0000 7f75d4b9a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d0103660 0x7f75d0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:55.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75d4b9a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75d010c170 0x7f75d019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:55.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75d4b9a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0111fb0 con 0x7f75d0103660 2024-09-16T01:38:55.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75d4b9a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0111e30 con 0x7f75d0102d40 2024-09-16T01:38:55.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75d4b9a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0112130 con 0x7f75d010c170 2024-09-16T01:38:55.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75cdd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d0103660 0x7f75d0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:55.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75cdd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d0103660 0x7f75d0198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40176/0 (socket says 172.21.15.139:40176) 2024-09-16T01:38:55.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75cdd9b640 1 -- 172.21.15.139:0/4134237563 learned_addr learned my addr 172.21.15.139:0/4134237563 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:55.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75ce59c640 1 --2- 172.21.15.139:0/4134237563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d0102d40 0x7f75d01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:55.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75cdd9b640 1 -- 172.21.15.139:0/4134237563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75d010c170 msgr2=0x7f75d019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:55.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75cdd9b640 1 --2- 172.21.15.139:0/4134237563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75d010c170 0x7f75d019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:55.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75cdd9b640 1 -- 172.21.15.139:0/4134237563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d0102d40 msgr2=0x7f75d01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:55.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75cdd9b640 1 --2- 172.21.15.139:0/4134237563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d0102d40 0x7f75d01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:55.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75cdd9b640 1 -- 172.21.15.139:0/4134237563 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d019d380 con 0x7f75d0103660 2024-09-16T01:38:55.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75ce59c640 1 --2- 172.21.15.139:0/4134237563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d0102d40 0x7f75d01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:38:55.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75cdd9b640 1 --2- 172.21.15.139:0/4134237563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d0103660 0x7f75d0198910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f75c400b3f0 tx=0x7f75c400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:55.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75bf7fe640 1 -- 172.21.15.139:0/4134237563 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4015020 con 0x7f75d0103660 2024-09-16T01:38:55.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75d4b9a640 1 -- 172.21.15.139:0/4134237563 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d019d670 con 0x7f75d0103660 2024-09-16T01:38:55.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75bf7fe640 1 -- 172.21.15.139:0/4134237563 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75c4011d10 con 0x7f75d0103660 2024-09-16T01:38:55.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75bf7fe640 1 -- 172.21.15.139:0/4134237563 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c400f2e0 con 0x7f75d0103660 2024-09-16T01:38:55.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.304+0000 7f75d4b9a640 1 -- 172.21.15.139:0/4134237563 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d01a4f50 con 0x7f75d0103660 2024-09-16T01:38:55.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.305+0000 7f75d4b9a640 1 -- 172.21.15.139:0/4134237563 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d0103120 con 0x7f75d0103660 2024-09-16T01:38:55.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.307+0000 7f75bf7fe640 1 -- 172.21.15.139:0/4134237563 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f75c4018020 con 0x7f75d0103660 2024-09-16T01:38:55.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.307+0000 7f75bf7fe640 1 --2- 172.21.15.139:0/4134237563 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f75b80a32d0 0x7f75b80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:55.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.307+0000 7f75ce59c640 1 --2- 172.21.15.139:0/4134237563 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f75b80a32d0 0x7f75b80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:55.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.307+0000 7f75bf7fe640 1 -- 172.21.15.139:0/4134237563 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f75c40cfba0 con 0x7f75d0103660 2024-09-16T01:38:55.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.308+0000 7f75ce59c640 1 --2- 172.21.15.139:0/4134237563 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f75b80a32d0 0x7f75b80a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f75c000baa0 tx=0x7f75c0005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:55.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.310+0000 7f75bf7fe640 1 -- 172.21.15.139:0/4134237563 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75c401f020 con 0x7f75d0103660 2024-09-16T01:38:55.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.474+0000 7f75d4b9a640 1 -- 172.21.15.139:0/4134237563 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f75d00feac0 con 0x7f75b80a32d0 2024-09-16T01:38:55.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.483+0000 7f75bf7fe640 1 -- 172.21.15.139:0/4134237563 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340662 (secure 0 0 0) 0x7f75d00feac0 con 0x7f75b80a32d0 2024-09-16T01:38:55.487 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:38:55.489 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T01:38:55.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.486+0000 7f75d4b9a640 1 -- 172.21.15.139:0/4134237563 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f75b80a32d0 msgr2=0x7f75b80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:55.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.486+0000 7f75d4b9a640 1 --2- 172.21.15.139:0/4134237563 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f75b80a32d0 0x7f75b80a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f75c000baa0 tx=0x7f75c0005d90 comp rx=0 tx=0).stop 2024-09-16T01:38:55.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.486+0000 7f75d4b9a640 1 -- 172.21.15.139:0/4134237563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d0103660 msgr2=0x7f75d0198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:55.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.486+0000 7f75d4b9a640 1 --2- 172.21.15.139:0/4134237563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d0103660 0x7f75d0198910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f75c400b3f0 tx=0x7f75c400b8b0 comp rx=0 tx=0).stop 2024-09-16T01:38:55.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.486+0000 7f75d4b9a640 1 -- 172.21.15.139:0/4134237563 shutdown_connections 2024-09-16T01:38:55.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.486+0000 7f75d4b9a640 1 --2- 172.21.15.139:0/4134237563 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f75b80a32d0 0x7f75b80a5790 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:55.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.486+0000 7f75d4b9a640 1 --2- 172.21.15.139:0/4134237563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75d010c170 0x7f75d019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:55.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.486+0000 7f75d4b9a640 1 --2- 172.21.15.139:0/4134237563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d0103660 0x7f75d0198910 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:55.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.486+0000 7f75d4b9a640 1 --2- 172.21.15.139:0/4134237563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d0102d40 0x7f75d01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:55.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.486+0000 7f75d4b9a640 1 -- 172.21.15.139:0/4134237563 >> 172.21.15.139:0/4134237563 conn(0x7f75d00fc7a0 msgr2=0x7f75d00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:55.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.486+0000 7f75d4b9a640 1 -- 172.21.15.139:0/4134237563 shutdown_connections 2024-09-16T01:38:55.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:55.486+0000 7f75d4b9a640 1 -- 172.21.15.139:0/4134237563 wait complete. 2024-09-16T01:38:55.804 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:55 smithi139 ceph-mon[31175]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:38:55.804 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:55 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:38:55.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:55 smithi176 ceph-mon[35324]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:38:55.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:55 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:38:56.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:55 smithi164 ceph-mon[35121]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:38:56.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:55 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:38:56.175 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-16T01:38:55.446319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649825,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":299,"num_read_kb":270,"num_write":424,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368880,"kb_used_data":11312,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572944,"statfs":{"total":1151940427776,"available":1151562694656,"internally_reserved":0,"allocated":11583488,"data_stored":7547777,"data_compressed":320082,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49281,"num_objects":7,"num_object_clones":0,"num_object_copies":23,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":141,"num_read_kb":130,"num_write":41,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011483"},"pg_stats":[{"pgid":"2.1f","version":"83'13","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:50.595835+0000","last_change":"2024-09-16T01:38:08.567909+0000","last_active":"2024-09-16T01:38:50.595835+0000","last_peered":"2024-09-16T01:38:50.595835+0000","last_clean":"2024-09-16T01:38:50.595835+0000","last_became_active":"2024-09-16T01:38:08.560733+0000","last_became_peered":"2024-09-16T01:38:08.560733+0000","last_unstale":"2024-09-16T01:38:50.595835+0000","last_undegraded":"2024-09-16T01:38:50.595835+0000","last_fullsized":"2024-09-16T01:38:50.595835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:00.678040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620682+0000","last_change":"2024-09-16T01:38:01.523064+0000","last_active":"2024-09-16T01:38:11.620682+0000","last_peered":"2024-09-16T01:38:11.620682+0000","last_clean":"2024-09-16T01:38:11.620682+0000","last_became_active":"2024-09-16T01:38:01.521695+0000","last_became_peered":"2024-09-16T01:38:01.521695+0000","last_unstale":"2024-09-16T01:38:11.620682+0000","last_undegraded":"2024-09-16T01:38:11.620682+0000","last_fullsized":"2024-09-16T01:38:11.620682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:28.467115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552424+0000","last_change":"2024-09-16T01:38:08.563557+0000","last_active":"2024-09-16T01:38:11.552424+0000","last_peered":"2024-09-16T01:38:11.552424+0000","last_clean":"2024-09-16T01:38:11.552424+0000","last_became_active":"2024-09-16T01:38:08.563065+0000","last_became_peered":"2024-09-16T01:38:08.563065+0000","last_unstale":"2024-09-16T01:38:11.552424+0000","last_undegraded":"2024-09-16T01:38:11.552424+0000","last_fullsized":"2024-09-16T01:38:11.552424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:56.992915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620823+0000","last_change":"2024-09-16T01:37:59.798884+0000","last_active":"2024-09-16T01:38:11.620823+0000","last_peered":"2024-09-16T01:38:11.620823+0000","last_clean":"2024-09-16T01:38:11.620823+0000","last_became_active":"2024-09-16T01:37:59.797989+0000","last_became_peered":"2024-09-16T01:37:59.797989+0000","last_unstale":"2024-09-16T01:38:11.620823+0000","last_undegraded":"2024-09-16T01:38:11.620823+0000","last_fullsized":"2024-09-16T01:38:11.620823+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:13.633078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620354+0000","last_change":"2024-09-16T01:38:01.528391+0000","last_active":"2024-09-16T01:38:11.620354+0000","last_peered":"2024-09-16T01:38:11.620354+0000","last_clean":"2024-09-16T01:38:11.620354+0000","last_became_active":"2024-09-16T01:38:01.528076+0000","last_became_peered":"2024-09-16T01:38:01.528076+0000","last_unstale":"2024-09-16T01:38:11.620354+0000","last_undegraded":"2024-09-16T01:38:11.620354+0000","last_fullsized":"2024-09-16T01:38:11.620354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.105830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619866+0000","last_change":"2024-09-16T01:38:08.566198+0000","last_active":"2024-09-16T01:38:11.619866+0000","last_peered":"2024-09-16T01:38:11.619866+0000","last_clean":"2024-09-16T01:38:11.619866+0000","last_became_active":"2024-09-16T01:38:08.565739+0000","last_became_peered":"2024-09-16T01:38:08.565739+0000","last_unstale":"2024-09-16T01:38:11.619866+0000","last_undegraded":"2024-09-16T01:38:11.619866+0000","last_fullsized":"2024-09-16T01:38:11.619866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:05.172269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:40.567498+0000","last_change":"2024-09-16T01:37:59.507625+0000","last_active":"2024-09-16T01:38:40.567498+0000","last_peered":"2024-09-16T01:38:40.567498+0000","last_clean":"2024-09-16T01:38:40.567498+0000","last_became_active":"2024-09-16T01:37:59.507280+0000","last_became_peered":"2024-09-16T01:37:59.507280+0000","last_unstale":"2024-09-16T01:38:40.567498+0000","last_undegraded":"2024-09-16T01:38:40.567498+0000","last_fullsized":"2024-09-16T01:38:40.567498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:29:37.761203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621582+0000","last_change":"2024-09-16T01:38:01.528406+0000","last_active":"2024-09-16T01:38:11.621582+0000","last_peered":"2024-09-16T01:38:11.621582+0000","last_clean":"2024-09-16T01:38:11.621582+0000","last_became_active":"2024-09-16T01:38:01.528084+0000","last_became_peered":"2024-09-16T01:38:01.528084+0000","last_unstale":"2024-09-16T01:38:11.621582+0000","last_undegraded":"2024-09-16T01:38:11.621582+0000","last_fullsized":"2024-09-16T01:38:11.621582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:40.766305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621257+0000","last_change":"2024-09-16T01:38:08.563645+0000","last_active":"2024-09-16T01:38:11.621257+0000","last_peered":"2024-09-16T01:38:11.621257+0000","last_clean":"2024-09-16T01:38:11.621257+0000","last_became_active":"2024-09-16T01:38:08.563149+0000","last_became_peered":"2024-09-16T01:38:08.563149+0000","last_unstale":"2024-09-16T01:38:11.621257+0000","last_undegraded":"2024-09-16T01:38:11.621257+0000","last_fullsized":"2024-09-16T01:38:11.621257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:09.010905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.561150+0000","last_change":"2024-09-16T01:37:59.509649+0000","last_active":"2024-09-16T01:38:39.561150+0000","last_peered":"2024-09-16T01:38:39.561150+0000","last_clean":"2024-09-16T01:38:39.561150+0000","last_became_active":"2024-09-16T01:37:59.509236+0000","last_became_peered":"2024-09-16T01:37:59.509236+0000","last_unstale":"2024-09-16T01:38:39.561150+0000","last_undegraded":"2024-09-16T01:38:39.561150+0000","last_fullsized":"2024-09-16T01:38:39.561150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.614205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621748+0000","last_change":"2024-09-16T01:38:01.528074+0000","last_active":"2024-09-16T01:38:11.621748+0000","last_peered":"2024-09-16T01:38:11.621748+0000","last_clean":"2024-09-16T01:38:11.621748+0000","last_became_active":"2024-09-16T01:38:01.527737+0000","last_became_peered":"2024-09-16T01:38:01.527737+0000","last_unstale":"2024-09-16T01:38:11.621748+0000","last_undegraded":"2024-09-16T01:38:11.621748+0000","last_fullsized":"2024-09-16T01:38:11.621748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:37.592557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620038+0000","last_change":"2024-09-16T01:38:08.566340+0000","last_active":"2024-09-16T01:38:11.620038+0000","last_peered":"2024-09-16T01:38:11.620038+0000","last_clean":"2024-09-16T01:38:11.620038+0000","last_became_active":"2024-09-16T01:38:08.565791+0000","last_became_peered":"2024-09-16T01:38:08.565791+0000","last_unstale":"2024-09-16T01:38:11.620038+0000","last_undegraded":"2024-09-16T01:38:11.620038+0000","last_fullsized":"2024-09-16T01:38:11.620038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:35:31.845151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620559+0000","last_change":"2024-09-16T01:38:01.527747+0000","last_active":"2024-09-16T01:38:11.620559+0000","last_peered":"2024-09-16T01:38:11.620559+0000","last_clean":"2024-09-16T01:38:11.620559+0000","last_became_active":"2024-09-16T01:38:01.527325+0000","last_became_peered":"2024-09-16T01:38:01.527325+0000","last_unstale":"2024-09-16T01:38:11.620559+0000","last_undegraded":"2024-09-16T01:38:11.620559+0000","last_fullsized":"2024-09-16T01:38:11.620559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:27.203719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:36.529271+0000","last_change":"2024-09-16T01:37:59.510298+0000","last_active":"2024-09-16T01:38:36.529271+0000","last_peered":"2024-09-16T01:38:36.529271+0000","last_clean":"2024-09-16T01:38:36.529271+0000","last_became_active":"2024-09-16T01:37:59.509952+0000","last_became_peered":"2024-09-16T01:37:59.509952+0000","last_unstale":"2024-09-16T01:38:36.529271+0000","last_undegraded":"2024-09-16T01:38:36.529271+0000","last_fullsized":"2024-09-16T01:38:36.529271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:11.616846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621402+0000","last_change":"2024-09-16T01:38:08.571622+0000","last_active":"2024-09-16T01:38:11.621402+0000","last_peered":"2024-09-16T01:38:11.621402+0000","last_clean":"2024-09-16T01:38:11.621402+0000","last_became_active":"2024-09-16T01:38:08.571125+0000","last_became_peered":"2024-09-16T01:38:08.571125+0000","last_unstale":"2024-09-16T01:38:11.621402+0000","last_undegraded":"2024-09-16T01:38:11.621402+0000","last_fullsized":"2024-09-16T01:38:11.621402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:00.154633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620455+0000","last_change":"2024-09-16T01:38:01.519681+0000","last_active":"2024-09-16T01:38:11.620455+0000","last_peered":"2024-09-16T01:38:11.620455+0000","last_clean":"2024-09-16T01:38:11.620455+0000","last_became_active":"2024-09-16T01:38:01.518310+0000","last_became_peered":"2024-09-16T01:38:01.518310+0000","last_unstale":"2024-09-16T01:38:11.620455+0000","last_undegraded":"2024-09-16T01:38:11.620455+0000","last_fullsized":"2024-09-16T01:38:11.620455+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:23.308008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:38.547465+0000","last_change":"2024-09-16T01:38:08.559469+0000","last_active":"2024-09-16T01:38:38.547465+0000","last_peered":"2024-09-16T01:38:38.547465+0000","last_clean":"2024-09-16T01:38:38.547465+0000","last_became_active":"2024-09-16T01:38:08.559111+0000","last_became_peered":"2024-09-16T01:38:08.559111+0000","last_unstale":"2024-09-16T01:38:38.547465+0000","last_undegraded":"2024-09-16T01:38:38.547465+0000","last_fullsized":"2024-09-16T01:38:38.547465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:58.577749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621895+0000","last_change":"2024-09-16T01:38:08.555029+0000","last_active":"2024-09-16T01:38:11.621895+0000","last_peered":"2024-09-16T01:38:11.621895+0000","last_clean":"2024-09-16T01:38:11.621895+0000","last_became_active":"2024-09-16T01:38:08.554151+0000","last_became_peered":"2024-09-16T01:38:08.554151+0000","last_unstale":"2024-09-16T01:38:11.621895+0000","last_undegraded":"2024-09-16T01:38:11.621895+0000","last_fullsized":"2024-09-16T01:38:11.621895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:05.005904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620140+0000","last_change":"2024-09-16T01:38:01.528114+0000","last_active":"2024-09-16T01:38:11.620140+0000","last_peered":"2024-09-16T01:38:11.620140+0000","last_clean":"2024-09-16T01:38:11.620140+0000","last_became_active":"2024-09-16T01:38:01.527805+0000","last_became_peered":"2024-09-16T01:38:01.527805+0000","last_unstale":"2024-09-16T01:38:11.620140+0000","last_undegraded":"2024-09-16T01:38:11.620140+0000","last_fullsized":"2024-09-16T01:38:11.620140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:15.830936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.558334+0000","last_change":"2024-09-16T01:37:59.507776+0000","last_active":"2024-09-16T01:38:39.558334+0000","last_peered":"2024-09-16T01:38:39.558334+0000","last_clean":"2024-09-16T01:38:39.558334+0000","last_became_active":"2024-09-16T01:37:59.507429+0000","last_became_peered":"2024-09-16T01:37:59.507429+0000","last_unstale":"2024-09-16T01:38:39.558334+0000","last_undegraded":"2024-09-16T01:38:39.558334+0000","last_fullsized":"2024-09-16T01:38:39.558334+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:44:41.274986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551917+0000","last_change":"2024-09-16T01:38:08.562191+0000","last_active":"2024-09-16T01:38:11.551917+0000","last_peered":"2024-09-16T01:38:11.551917+0000","last_clean":"2024-09-16T01:38:11.551917+0000","last_became_active":"2024-09-16T01:38:08.561839+0000","last_became_peered":"2024-09-16T01:38:08.561839+0000","last_unstale":"2024-09-16T01:38:11.551917+0000","last_undegraded":"2024-09-16T01:38:11.551917+0000","last_fullsized":"2024-09-16T01:38:11.551917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:40.618612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620169+0000","last_change":"2024-09-16T01:38:01.518855+0000","last_active":"2024-09-16T01:38:11.620169+0000","last_peered":"2024-09-16T01:38:11.620169+0000","last_clean":"2024-09-16T01:38:11.620169+0000","last_became_active":"2024-09-16T01:38:01.518427+0000","last_became_peered":"2024-09-16T01:38:01.518427+0000","last_unstale":"2024-09-16T01:38:11.620169+0000","last_undegraded":"2024-09-16T01:38:11.620169+0000","last_fullsized":"2024-09-16T01:38:11.620169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:23.054856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:36.529446+0000","last_change":"2024-09-16T01:37:59.799341+0000","last_active":"2024-09-16T01:38:36.529446+0000","last_peered":"2024-09-16T01:38:36.529446+0000","last_clean":"2024-09-16T01:38:36.529446+0000","last_became_active":"2024-09-16T01:37:59.798992+0000","last_became_peered":"2024-09-16T01:37:59.798992+0000","last_unstale":"2024-09-16T01:38:36.529446+0000","last_undegraded":"2024-09-16T01:38:36.529446+0000","last_fullsized":"2024-09-16T01:38:36.529446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:01.708565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621703+0000","last_change":"2024-09-16T01:38:08.572556+0000","last_active":"2024-09-16T01:38:11.621703+0000","last_peered":"2024-09-16T01:38:11.621703+0000","last_clean":"2024-09-16T01:38:11.621703+0000","last_became_active":"2024-09-16T01:38:08.571990+0000","last_became_peered":"2024-09-16T01:38:08.571990+0000","last_unstale":"2024-09-16T01:38:11.621703+0000","last_undegraded":"2024-09-16T01:38:11.621703+0000","last_fullsized":"2024-09-16T01:38:11.621703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:41.793143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620089+0000","last_change":"2024-09-16T01:38:01.524545+0000","last_active":"2024-09-16T01:38:11.620089+0000","last_peered":"2024-09-16T01:38:11.620089+0000","last_clean":"2024-09-16T01:38:11.620089+0000","last_became_active":"2024-09-16T01:38:01.524253+0000","last_became_peered":"2024-09-16T01:38:01.524253+0000","last_unstale":"2024-09-16T01:38:11.620089+0000","last_undegraded":"2024-09-16T01:38:11.620089+0000","last_fullsized":"2024-09-16T01:38:11.620089+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:08.697500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.560936+0000","last_change":"2024-09-16T01:37:59.506779+0000","last_active":"2024-09-16T01:38:39.560936+0000","last_peered":"2024-09-16T01:38:39.560936+0000","last_clean":"2024-09-16T01:38:39.560936+0000","last_became_active":"2024-09-16T01:37:59.505459+0000","last_became_peered":"2024-09-16T01:37:59.505459+0000","last_unstale":"2024-09-16T01:38:39.560936+0000","last_undegraded":"2024-09-16T01:38:39.560936+0000","last_fullsized":"2024-09-16T01:38:39.560936+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:12.973330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552762+0000","last_change":"2024-09-16T01:38:08.557250+0000","last_active":"2024-09-16T01:38:11.552762+0000","last_peered":"2024-09-16T01:38:11.552762+0000","last_clean":"2024-09-16T01:38:11.552762+0000","last_became_active":"2024-09-16T01:38:08.555577+0000","last_became_peered":"2024-09-16T01:38:08.555577+0000","last_unstale":"2024-09-16T01:38:11.552762+0000","last_undegraded":"2024-09-16T01:38:11.552762+0000","last_fullsized":"2024-09-16T01:38:11.552762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:54.965341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620877+0000","last_change":"2024-09-16T01:38:01.522910+0000","last_active":"2024-09-16T01:38:11.620877+0000","last_peered":"2024-09-16T01:38:11.620877+0000","last_clean":"2024-09-16T01:38:11.620877+0000","last_became_active":"2024-09-16T01:38:01.521206+0000","last_became_peered":"2024-09-16T01:38:01.521206+0000","last_unstale":"2024-09-16T01:38:11.620877+0000","last_undegraded":"2024-09-16T01:38:11.620877+0000","last_fullsized":"2024-09-16T01:38:11.620877+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:45.670464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'14","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:50.524562+0000","last_change":"2024-09-16T01:37:59.505962+0000","last_active":"2024-09-16T01:38:50.524562+0000","last_peered":"2024-09-16T01:38:50.524562+0000","last_clean":"2024-09-16T01:38:50.524562+0000","last_became_active":"2024-09-16T01:37:59.505636+0000","last_became_peered":"2024-09-16T01:37:59.505636+0000","last_unstale":"2024-09-16T01:38:50.524562+0000","last_undegraded":"2024-09-16T01:38:50.524562+0000","last_fullsized":"2024-09-16T01:38:50.524562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:57.548513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619604+0000","last_change":"2024-09-16T01:38:08.571249+0000","last_active":"2024-09-16T01:38:11.619604+0000","last_peered":"2024-09-16T01:38:11.619604+0000","last_clean":"2024-09-16T01:38:11.619604+0000","last_became_active":"2024-09-16T01:38:08.570388+0000","last_became_peered":"2024-09-16T01:38:08.570388+0000","last_unstale":"2024-09-16T01:38:11.619604+0000","last_undegraded":"2024-09-16T01:38:11.619604+0000","last_fullsized":"2024-09-16T01:38:11.619604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:59.904704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009588+0000","last_change":"2024-09-16T01:38:01.522040+0000","last_active":"2024-09-16T01:38:12.009588+0000","last_peered":"2024-09-16T01:38:12.009588+0000","last_clean":"2024-09-16T01:38:12.009588+0000","last_became_active":"2024-09-16T01:38:01.521720+0000","last_became_peered":"2024-09-16T01:38:01.521720+0000","last_unstale":"2024-09-16T01:38:12.009588+0000","last_undegraded":"2024-09-16T01:38:12.009588+0000","last_fullsized":"2024-09-16T01:38:12.009588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:44:48.635603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.559237+0000","last_change":"2024-09-16T01:38:08.561870+0000","last_active":"2024-09-16T01:38:39.559237+0000","last_peered":"2024-09-16T01:38:39.559237+0000","last_clean":"2024-09-16T01:38:39.559237+0000","last_became_active":"2024-09-16T01:38:08.561440+0000","last_became_peered":"2024-09-16T01:38:08.561440+0000","last_unstale":"2024-09-16T01:38:39.559237+0000","last_undegraded":"2024-09-16T01:38:39.559237+0000","last_fullsized":"2024-09-16T01:38:39.559237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:43:37.594734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620437+0000","last_change":"2024-09-16T01:38:08.555436+0000","last_active":"2024-09-16T01:38:11.620437+0000","last_peered":"2024-09-16T01:38:11.620437+0000","last_clean":"2024-09-16T01:38:11.620437+0000","last_became_active":"2024-09-16T01:38:08.555026+0000","last_became_peered":"2024-09-16T01:38:08.555026+0000","last_unstale":"2024-09-16T01:38:11.620437+0000","last_undegraded":"2024-09-16T01:38:11.620437+0000","last_fullsized":"2024-09-16T01:38:11.620437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:28.870214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552802+0000","last_change":"2024-09-16T01:38:01.524482+0000","last_active":"2024-09-16T01:38:11.552802+0000","last_peered":"2024-09-16T01:38:11.552802+0000","last_clean":"2024-09-16T01:38:11.552802+0000","last_became_active":"2024-09-16T01:38:01.524160+0000","last_became_peered":"2024-09-16T01:38:01.524160+0000","last_unstale":"2024-09-16T01:38:11.552802+0000","last_undegraded":"2024-09-16T01:38:11.552802+0000","last_fullsized":"2024-09-16T01:38:11.552802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:39:27.621780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551849+0000","last_change":"2024-09-16T01:37:59.800051+0000","last_active":"2024-09-16T01:38:11.551849+0000","last_peered":"2024-09-16T01:38:11.551849+0000","last_clean":"2024-09-16T01:38:11.551849+0000","last_became_active":"2024-09-16T01:37:59.799606+0000","last_became_peered":"2024-09-16T01:37:59.799606+0000","last_unstale":"2024-09-16T01:38:11.551849+0000","last_undegraded":"2024-09-16T01:38:11.551849+0000","last_fullsized":"2024-09-16T01:38:11.551849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:02.694933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620035+0000","last_change":"2024-09-16T01:38:08.570764+0000","last_active":"2024-09-16T01:38:11.620035+0000","last_peered":"2024-09-16T01:38:11.620035+0000","last_clean":"2024-09-16T01:38:11.620035+0000","last_became_active":"2024-09-16T01:38:08.570385+0000","last_became_peered":"2024-09-16T01:38:08.570385+0000","last_unstale":"2024-09-16T01:38:11.620035+0000","last_undegraded":"2024-09-16T01:38:11.620035+0000","last_fullsized":"2024-09-16T01:38:11.620035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:25.349671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552770+0000","last_change":"2024-09-16T01:38:01.519590+0000","last_active":"2024-09-16T01:38:11.552770+0000","last_peered":"2024-09-16T01:38:11.552770+0000","last_clean":"2024-09-16T01:38:11.552770+0000","last_became_active":"2024-09-16T01:38:01.519210+0000","last_became_peered":"2024-09-16T01:38:01.519210+0000","last_unstale":"2024-09-16T01:38:11.552770+0000","last_undegraded":"2024-09-16T01:38:11.552770+0000","last_fullsized":"2024-09-16T01:38:11.552770+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:04.228131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:48.593997+0000","last_change":"2024-09-16T01:37:59.505473+0000","last_active":"2024-09-16T01:38:48.593997+0000","last_peered":"2024-09-16T01:38:48.593997+0000","last_clean":"2024-09-16T01:38:48.593997+0000","last_became_active":"2024-09-16T01:37:59.505123+0000","last_became_peered":"2024-09-16T01:37:59.505123+0000","last_unstale":"2024-09-16T01:38:48.593997+0000","last_undegraded":"2024-09-16T01:38:48.593997+0000","last_fullsized":"2024-09-16T01:38:48.593997+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:17.739625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009514+0000","last_change":"2024-09-16T01:38:08.558534+0000","last_active":"2024-09-16T01:38:12.009514+0000","last_peered":"2024-09-16T01:38:12.009514+0000","last_clean":"2024-09-16T01:38:12.009514+0000","last_became_active":"2024-09-16T01:38:08.558185+0000","last_became_peered":"2024-09-16T01:38:08.558185+0000","last_unstale":"2024-09-16T01:38:12.009514+0000","last_undegraded":"2024-09-16T01:38:12.009514+0000","last_fullsized":"2024-09-16T01:38:12.009514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:09.359426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620222+0000","last_change":"2024-09-16T01:38:01.521445+0000","last_active":"2024-09-16T01:38:11.620222+0000","last_peered":"2024-09-16T01:38:11.620222+0000","last_clean":"2024-09-16T01:38:11.620222+0000","last_became_active":"2024-09-16T01:38:01.521140+0000","last_became_peered":"2024-09-16T01:38:01.521140+0000","last_unstale":"2024-09-16T01:38:11.620222+0000","last_undegraded":"2024-09-16T01:38:11.620222+0000","last_fullsized":"2024-09-16T01:38:11.620222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:05.247515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:50.584452+0000","last_change":"2024-09-16T01:37:59.798365+0000","last_active":"2024-09-16T01:38:50.584452+0000","last_peered":"2024-09-16T01:38:50.584452+0000","last_clean":"2024-09-16T01:38:50.584452+0000","last_became_active":"2024-09-16T01:37:59.798023+0000","last_became_peered":"2024-09-16T01:37:59.798023+0000","last_unstale":"2024-09-16T01:38:50.584452+0000","last_undegraded":"2024-09-16T01:38:50.584452+0000","last_fullsized":"2024-09-16T01:38:50.584452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:30.867080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009383+0000","last_change":"2024-09-16T01:38:08.571704+0000","last_active":"2024-09-16T01:38:12.009383+0000","last_peered":"2024-09-16T01:38:12.009383+0000","last_clean":"2024-09-16T01:38:12.009383+0000","last_became_active":"2024-09-16T01:38:08.571350+0000","last_became_peered":"2024-09-16T01:38:08.571350+0000","last_unstale":"2024-09-16T01:38:12.009383+0000","last_undegraded":"2024-09-16T01:38:12.009383+0000","last_fullsized":"2024-09-16T01:38:12.009383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:01.698770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009013+0000","last_change":"2024-09-16T01:38:08.559433+0000","last_active":"2024-09-16T01:38:12.009013+0000","last_peered":"2024-09-16T01:38:12.009013+0000","last_clean":"2024-09-16T01:38:12.009013+0000","last_became_active":"2024-09-16T01:38:08.559110+0000","last_became_peered":"2024-09-16T01:38:08.559110+0000","last_unstale":"2024-09-16T01:38:12.009013+0000","last_undegraded":"2024-09-16T01:38:12.009013+0000","last_fullsized":"2024-09-16T01:38:12.009013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:39.001916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:37.538688+0000","last_change":"2024-09-16T01:37:59.508236+0000","last_active":"2024-09-16T01:38:37.538688+0000","last_peered":"2024-09-16T01:38:37.538688+0000","last_clean":"2024-09-16T01:38:37.538688+0000","last_became_active":"2024-09-16T01:37:59.507883+0000","last_became_peered":"2024-09-16T01:37:59.507883+0000","last_unstale":"2024-09-16T01:38:37.538688+0000","last_undegraded":"2024-09-16T01:38:37.538688+0000","last_fullsized":"2024-09-16T01:38:37.538688+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:23.948530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621566+0000","last_change":"2024-09-16T01:38:08.572108+0000","last_active":"2024-09-16T01:38:11.621566+0000","last_peered":"2024-09-16T01:38:11.621566+0000","last_clean":"2024-09-16T01:38:11.621566+0000","last_became_active":"2024-09-16T01:38:08.571643+0000","last_became_peered":"2024-09-16T01:38:08.571643+0000","last_unstale":"2024-09-16T01:38:11.621566+0000","last_undegraded":"2024-09-16T01:38:11.621566+0000","last_fullsized":"2024-09-16T01:38:11.621566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:46.636219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620399+0000","last_change":"2024-09-16T01:38:01.517894+0000","last_active":"2024-09-16T01:38:11.620399+0000","last_peered":"2024-09-16T01:38:11.620399+0000","last_clean":"2024-09-16T01:38:11.620399+0000","last_became_active":"2024-09-16T01:38:01.517602+0000","last_became_peered":"2024-09-16T01:38:01.517602+0000","last_unstale":"2024-09-16T01:38:11.620399+0000","last_undegraded":"2024-09-16T01:38:11.620399+0000","last_fullsized":"2024-09-16T01:38:11.620399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:33.483120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:40.567759+0000","last_change":"2024-09-16T01:37:59.507568+0000","last_active":"2024-09-16T01:38:40.567759+0000","last_peered":"2024-09-16T01:38:40.567759+0000","last_clean":"2024-09-16T01:38:40.567759+0000","last_became_active":"2024-09-16T01:37:59.505798+0000","last_became_peered":"2024-09-16T01:37:59.505798+0000","last_unstale":"2024-09-16T01:38:40.567759+0000","last_undegraded":"2024-09-16T01:38:40.567759+0000","last_fullsized":"2024-09-16T01:38:40.567759+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:19:02.711054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":353,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620796+0000","last_change":"2024-09-16T01:36:46.744332+0000","last_active":"2024-09-16T01:38:11.620796+0000","last_peered":"2024-09-16T01:38:11.620796+0000","last_clean":"2024-09-16T01:38:11.620796+0000","last_became_active":"2024-09-16T01:36:46.743902+0000","last_became_peered":"2024-09-16T01:36:46.743902+0000","last_unstale":"2024-09-16T01:38:11.620796+0000","last_undegraded":"2024-09-16T01:38:11.620796+0000","last_fullsized":"2024-09-16T01:38:11.620796+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:33:19.891854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:33:19.891854+0000","last_clean_scrub_stamp":"2024-09-16T01:33:19.891854+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:46.173685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620994+0000","last_change":"2024-09-16T01:38:08.558344+0000","last_active":"2024-09-16T01:38:11.620994+0000","last_peered":"2024-09-16T01:38:11.620994+0000","last_clean":"2024-09-16T01:38:11.620994+0000","last_became_active":"2024-09-16T01:38:08.557938+0000","last_became_peered":"2024-09-16T01:38:08.557938+0000","last_unstale":"2024-09-16T01:38:11.620994+0000","last_undegraded":"2024-09-16T01:38:11.620994+0000","last_fullsized":"2024-09-16T01:38:11.620994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:56.799794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621097+0000","last_change":"2024-09-16T01:38:01.527690+0000","last_active":"2024-09-16T01:38:11.621097+0000","last_peered":"2024-09-16T01:38:11.621097+0000","last_clean":"2024-09-16T01:38:11.621097+0000","last_became_active":"2024-09-16T01:38:01.527339+0000","last_became_peered":"2024-09-16T01:38:01.527339+0000","last_unstale":"2024-09-16T01:38:11.621097+0000","last_undegraded":"2024-09-16T01:38:11.621097+0000","last_fullsized":"2024-09-16T01:38:11.621097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:58.526270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:52.588360+0000","last_change":"2024-09-16T01:37:59.798874+0000","last_active":"2024-09-16T01:38:52.588360+0000","last_peered":"2024-09-16T01:38:52.588360+0000","last_clean":"2024-09-16T01:38:52.588360+0000","last_became_active":"2024-09-16T01:37:59.798424+0000","last_became_peered":"2024-09-16T01:37:59.798424+0000","last_unstale":"2024-09-16T01:38:52.588360+0000","last_undegraded":"2024-09-16T01:38:52.588360+0000","last_fullsized":"2024-09-16T01:38:52.588360+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:36.648963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620451+0000","last_change":"2024-09-16T01:38:08.555524+0000","last_active":"2024-09-16T01:38:11.620451+0000","last_peered":"2024-09-16T01:38:11.620451+0000","last_clean":"2024-09-16T01:38:11.620451+0000","last_became_active":"2024-09-16T01:38:08.555075+0000","last_became_peered":"2024-09-16T01:38:08.555075+0000","last_unstale":"2024-09-16T01:38:11.620451+0000","last_undegraded":"2024-09-16T01:38:11.620451+0000","last_fullsized":"2024-09-16T01:38:11.620451+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:44.847916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620635+0000","last_change":"2024-09-16T01:38:01.528869+0000","last_active":"2024-09-16T01:38:11.620635+0000","last_peered":"2024-09-16T01:38:11.620635+0000","last_clean":"2024-09-16T01:38:11.620635+0000","last_became_active":"2024-09-16T01:38:01.528559+0000","last_became_peered":"2024-09-16T01:38:01.528559+0000","last_unstale":"2024-09-16T01:38:11.620635+0000","last_undegraded":"2024-09-16T01:38:11.620635+0000","last_fullsized":"2024-09-16T01:38:11.620635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:21.547808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.559919+0000","last_change":"2024-09-16T01:37:59.510871+0000","last_active":"2024-09-16T01:38:39.559919+0000","last_peered":"2024-09-16T01:38:39.559919+0000","last_clean":"2024-09-16T01:38:39.559919+0000","last_became_active":"2024-09-16T01:37:59.509996+0000","last_became_peered":"2024-09-16T01:37:59.509996+0000","last_unstale":"2024-09-16T01:38:39.559919+0000","last_undegraded":"2024-09-16T01:38:39.559919+0000","last_fullsized":"2024-09-16T01:38:39.559919+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:23.390864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620207+0000","last_change":"2024-09-16T01:38:08.555181+0000","last_active":"2024-09-16T01:38:11.620207+0000","last_peered":"2024-09-16T01:38:11.620207+0000","last_clean":"2024-09-16T01:38:11.620207+0000","last_became_active":"2024-09-16T01:38:08.554778+0000","last_became_peered":"2024-09-16T01:38:08.554778+0000","last_unstale":"2024-09-16T01:38:11.620207+0000","last_undegraded":"2024-09-16T01:38:11.620207+0000","last_fullsized":"2024-09-16T01:38:11.620207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:16:42.369236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620832+0000","last_change":"2024-09-16T01:38:01.526260+0000","last_active":"2024-09-16T01:38:11.620832+0000","last_peered":"2024-09-16T01:38:11.620832+0000","last_clean":"2024-09-16T01:38:11.620832+0000","last_became_active":"2024-09-16T01:38:01.525652+0000","last_became_peered":"2024-09-16T01:38:01.525652+0000","last_unstale":"2024-09-16T01:38:11.620832+0000","last_undegraded":"2024-09-16T01:38:11.620832+0000","last_fullsized":"2024-09-16T01:38:11.620832+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:09.180507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.561305+0000","last_change":"2024-09-16T01:38:08.570408+0000","last_active":"2024-09-16T01:38:39.561305+0000","last_peered":"2024-09-16T01:38:39.561305+0000","last_clean":"2024-09-16T01:38:39.561305+0000","last_became_active":"2024-09-16T01:38:08.566122+0000","last_became_peered":"2024-09-16T01:38:08.566122+0000","last_unstale":"2024-09-16T01:38:39.561305+0000","last_undegraded":"2024-09-16T01:38:39.561305+0000","last_fullsized":"2024-09-16T01:38:39.561305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:55.832320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620226+0000","last_change":"2024-09-16T01:38:08.555063+0000","last_active":"2024-09-16T01:38:11.620226+0000","last_peered":"2024-09-16T01:38:11.620226+0000","last_clean":"2024-09-16T01:38:11.620226+0000","last_became_active":"2024-09-16T01:38:08.554665+0000","last_became_peered":"2024-09-16T01:38:08.554665+0000","last_unstale":"2024-09-16T01:38:11.620226+0000","last_undegraded":"2024-09-16T01:38:11.620226+0000","last_fullsized":"2024-09-16T01:38:11.620226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:50.091017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620667+0000","last_change":"2024-09-16T01:38:01.524247+0000","last_active":"2024-09-16T01:38:11.620667+0000","last_peered":"2024-09-16T01:38:11.620667+0000","last_clean":"2024-09-16T01:38:11.620667+0000","last_became_active":"2024-09-16T01:38:01.523911+0000","last_became_peered":"2024-09-16T01:38:01.523911+0000","last_unstale":"2024-09-16T01:38:11.620667+0000","last_undegraded":"2024-09-16T01:38:11.620667+0000","last_fullsized":"2024-09-16T01:38:11.620667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:32.745882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:38.534097+0000","last_change":"2024-09-16T01:37:59.511272+0000","last_active":"2024-09-16T01:38:38.534097+0000","last_peered":"2024-09-16T01:38:38.534097+0000","last_clean":"2024-09-16T01:38:38.534097+0000","last_became_active":"2024-09-16T01:37:59.510920+0000","last_became_peered":"2024-09-16T01:37:59.510920+0000","last_unstale":"2024-09-16T01:38:38.534097+0000","last_undegraded":"2024-09-16T01:38:38.534097+0000","last_fullsized":"2024-09-16T01:38:38.534097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:59.406175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620991+0000","last_change":"2024-09-16T01:38:08.567173+0000","last_active":"2024-09-16T01:38:11.620991+0000","last_peered":"2024-09-16T01:38:11.620991+0000","last_clean":"2024-09-16T01:38:11.620991+0000","last_became_active":"2024-09-16T01:38:08.566788+0000","last_became_peered":"2024-09-16T01:38:08.566788+0000","last_unstale":"2024-09-16T01:38:11.620991+0000","last_undegraded":"2024-09-16T01:38:11.620991+0000","last_fullsized":"2024-09-16T01:38:11.620991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:35.743356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621167+0000","last_change":"2024-09-16T01:38:01.529529+0000","last_active":"2024-09-16T01:38:11.621167+0000","last_peered":"2024-09-16T01:38:11.621167+0000","last_clean":"2024-09-16T01:38:11.621167+0000","last_became_active":"2024-09-16T01:38:01.529048+0000","last_became_peered":"2024-09-16T01:38:01.529048+0000","last_unstale":"2024-09-16T01:38:11.621167+0000","last_undegraded":"2024-09-16T01:38:11.621167+0000","last_fullsized":"2024-09-16T01:38:11.621167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:35.041662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.560346+0000","last_change":"2024-09-16T01:37:59.799536+0000","last_active":"2024-09-16T01:38:39.560346+0000","last_peered":"2024-09-16T01:38:39.560346+0000","last_clean":"2024-09-16T01:38:39.560346+0000","last_became_active":"2024-09-16T01:37:59.799109+0000","last_became_peered":"2024-09-16T01:37:59.799109+0000","last_unstale":"2024-09-16T01:38:39.560346+0000","last_undegraded":"2024-09-16T01:38:39.560346+0000","last_fullsized":"2024-09-16T01:38:39.560346+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:07:25.007434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009492+0000","last_change":"2024-09-16T01:38:08.567019+0000","last_active":"2024-09-16T01:38:12.009492+0000","last_peered":"2024-09-16T01:38:12.009492+0000","last_clean":"2024-09-16T01:38:12.009492+0000","last_became_active":"2024-09-16T01:38:08.566667+0000","last_became_peered":"2024-09-16T01:38:08.566667+0000","last_unstale":"2024-09-16T01:38:12.009492+0000","last_undegraded":"2024-09-16T01:38:12.009492+0000","last_fullsized":"2024-09-16T01:38:12.009492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:11.375095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620513+0000","last_change":"2024-09-16T01:38:01.527965+0000","last_active":"2024-09-16T01:38:11.620513+0000","last_peered":"2024-09-16T01:38:11.620513+0000","last_clean":"2024-09-16T01:38:11.620513+0000","last_became_active":"2024-09-16T01:38:01.526688+0000","last_became_peered":"2024-09-16T01:38:01.526688+0000","last_unstale":"2024-09-16T01:38:11.620513+0000","last_undegraded":"2024-09-16T01:38:11.620513+0000","last_fullsized":"2024-09-16T01:38:11.620513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:06.618756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:48.582123+0000","last_change":"2024-09-16T01:37:59.510331+0000","last_active":"2024-09-16T01:38:48.582123+0000","last_peered":"2024-09-16T01:38:48.582123+0000","last_clean":"2024-09-16T01:38:48.582123+0000","last_became_active":"2024-09-16T01:37:59.509769+0000","last_became_peered":"2024-09-16T01:37:59.509769+0000","last_unstale":"2024-09-16T01:38:48.582123+0000","last_undegraded":"2024-09-16T01:38:48.582123+0000","last_fullsized":"2024-09-16T01:38:48.582123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:18.843091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551902+0000","last_change":"2024-09-16T01:38:08.564073+0000","last_active":"2024-09-16T01:38:11.551902+0000","last_peered":"2024-09-16T01:38:11.551902+0000","last_clean":"2024-09-16T01:38:11.551902+0000","last_became_active":"2024-09-16T01:38:08.563738+0000","last_became_peered":"2024-09-16T01:38:08.563738+0000","last_unstale":"2024-09-16T01:38:11.551902+0000","last_undegraded":"2024-09-16T01:38:11.551902+0000","last_fullsized":"2024-09-16T01:38:11.551902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:08.418956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620518+0000","last_change":"2024-09-16T01:38:01.521254+0000","last_active":"2024-09-16T01:38:11.620518+0000","last_peered":"2024-09-16T01:38:11.620518+0000","last_clean":"2024-09-16T01:38:11.620518+0000","last_became_active":"2024-09-16T01:38:01.520901+0000","last_became_peered":"2024-09-16T01:38:01.520901+0000","last_unstale":"2024-09-16T01:38:11.620518+0000","last_undegraded":"2024-09-16T01:38:11.620518+0000","last_fullsized":"2024-09-16T01:38:11.620518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:27.806954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.561026+0000","last_change":"2024-09-16T01:37:59.505990+0000","last_active":"2024-09-16T01:38:39.561026+0000","last_peered":"2024-09-16T01:38:39.561026+0000","last_clean":"2024-09-16T01:38:39.561026+0000","last_became_active":"2024-09-16T01:37:59.505564+0000","last_became_peered":"2024-09-16T01:37:59.505564+0000","last_unstale":"2024-09-16T01:38:39.561026+0000","last_undegraded":"2024-09-16T01:38:39.561026+0000","last_fullsized":"2024-09-16T01:38:39.561026+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:01.496609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620939+0000","last_change":"2024-09-16T01:38:08.562838+0000","last_active":"2024-09-16T01:38:11.620939+0000","last_peered":"2024-09-16T01:38:11.620939+0000","last_clean":"2024-09-16T01:38:11.620939+0000","last_became_active":"2024-09-16T01:38:08.562426+0000","last_became_peered":"2024-09-16T01:38:08.562426+0000","last_unstale":"2024-09-16T01:38:11.620939+0000","last_undegraded":"2024-09-16T01:38:11.620939+0000","last_fullsized":"2024-09-16T01:38:11.620939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:20.685626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009218+0000","last_change":"2024-09-16T01:38:01.514983+0000","last_active":"2024-09-16T01:38:12.009218+0000","last_peered":"2024-09-16T01:38:12.009218+0000","last_clean":"2024-09-16T01:38:12.009218+0000","last_became_active":"2024-09-16T01:38:01.514648+0000","last_became_peered":"2024-09-16T01:38:01.514648+0000","last_unstale":"2024-09-16T01:38:12.009218+0000","last_undegraded":"2024-09-16T01:38:12.009218+0000","last_fullsized":"2024-09-16T01:38:12.009218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:57.052564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:40.567002+0000","last_change":"2024-09-16T01:37:59.505141+0000","last_active":"2024-09-16T01:38:40.567002+0000","last_peered":"2024-09-16T01:38:40.567002+0000","last_clean":"2024-09-16T01:38:40.567002+0000","last_became_active":"2024-09-16T01:37:59.504730+0000","last_became_peered":"2024-09-16T01:37:59.504730+0000","last_unstale":"2024-09-16T01:38:40.567002+0000","last_undegraded":"2024-09-16T01:38:40.567002+0000","last_fullsized":"2024-09-16T01:38:40.567002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:54.504661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621780+0000","last_change":"2024-09-16T01:38:08.563970+0000","last_active":"2024-09-16T01:38:11.621780+0000","last_peered":"2024-09-16T01:38:11.621780+0000","last_clean":"2024-09-16T01:38:11.621780+0000","last_became_active":"2024-09-16T01:38:08.563555+0000","last_became_peered":"2024-09-16T01:38:08.563555+0000","last_unstale":"2024-09-16T01:38:11.621780+0000","last_undegraded":"2024-09-16T01:38:11.621780+0000","last_fullsized":"2024-09-16T01:38:11.621780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:27:12.086043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621032+0000","last_change":"2024-09-16T01:38:01.521016+0000","last_active":"2024-09-16T01:38:11.621032+0000","last_peered":"2024-09-16T01:38:11.621032+0000","last_clean":"2024-09-16T01:38:11.621032+0000","last_became_active":"2024-09-16T01:38:01.520656+0000","last_became_peered":"2024-09-16T01:38:01.520656+0000","last_unstale":"2024-09-16T01:38:11.621032+0000","last_undegraded":"2024-09-16T01:38:11.621032+0000","last_fullsized":"2024-09-16T01:38:11.621032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:39.001239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'31","reported_seq":96,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:53.591133+0000","last_change":"2024-09-16T01:37:59.507235+0000","last_active":"2024-09-16T01:38:53.591133+0000","last_peered":"2024-09-16T01:38:53.591133+0000","last_clean":"2024-09-16T01:38:53.591133+0000","last_became_active":"2024-09-16T01:37:59.506255+0000","last_became_peered":"2024-09-16T01:37:59.506255+0000","last_unstale":"2024-09-16T01:38:53.591133+0000","last_undegraded":"2024-09-16T01:38:53.591133+0000","last_fullsized":"2024-09-16T01:38:53.591133+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:44.515663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52408,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":68,"num_write":33,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551520+0000","last_change":"2024-09-16T01:38:08.558293+0000","last_active":"2024-09-16T01:38:11.551520+0000","last_peered":"2024-09-16T01:38:11.551520+0000","last_clean":"2024-09-16T01:38:11.551520+0000","last_became_active":"2024-09-16T01:38:08.557958+0000","last_became_peered":"2024-09-16T01:38:08.557958+0000","last_unstale":"2024-09-16T01:38:11.551520+0000","last_undegraded":"2024-09-16T01:38:11.551520+0000","last_fullsized":"2024-09-16T01:38:11.551520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:44:56.783271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621689+0000","last_change":"2024-09-16T01:38:01.526641+0000","last_active":"2024-09-16T01:38:11.621689+0000","last_peered":"2024-09-16T01:38:11.621689+0000","last_clean":"2024-09-16T01:38:11.621689+0000","last_became_active":"2024-09-16T01:38:01.526278+0000","last_became_peered":"2024-09-16T01:38:01.526278+0000","last_unstale":"2024-09-16T01:38:11.621689+0000","last_undegraded":"2024-09-16T01:38:11.621689+0000","last_fullsized":"2024-09-16T01:38:11.621689+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:56.156593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:38.547604+0000","last_change":"2024-09-16T01:37:59.800065+0000","last_active":"2024-09-16T01:38:38.547604+0000","last_peered":"2024-09-16T01:38:38.547604+0000","last_clean":"2024-09-16T01:38:38.547604+0000","last_became_active":"2024-09-16T01:37:59.799705+0000","last_became_peered":"2024-09-16T01:37:59.799705+0000","last_unstale":"2024-09-16T01:38:38.547604+0000","last_undegraded":"2024-09-16T01:38:38.547604+0000","last_fullsized":"2024-09-16T01:38:38.547604+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:13.791875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621382+0000","last_change":"2024-09-16T01:38:08.556360+0000","last_active":"2024-09-16T01:38:11.621382+0000","last_peered":"2024-09-16T01:38:11.621382+0000","last_clean":"2024-09-16T01:38:11.621382+0000","last_became_active":"2024-09-16T01:38:08.555219+0000","last_became_peered":"2024-09-16T01:38:08.555219+0000","last_unstale":"2024-09-16T01:38:11.621382+0000","last_undegraded":"2024-09-16T01:38:11.621382+0000","last_fullsized":"2024-09-16T01:38:11.621382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:36.464760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620417+0000","last_change":"2024-09-16T01:38:01.526758+0000","last_active":"2024-09-16T01:38:11.620417+0000","last_peered":"2024-09-16T01:38:11.620417+0000","last_clean":"2024-09-16T01:38:11.620417+0000","last_became_active":"2024-09-16T01:38:01.526545+0000","last_became_peered":"2024-09-16T01:38:01.526545+0000","last_unstale":"2024-09-16T01:38:11.620417+0000","last_undegraded":"2024-09-16T01:38:11.620417+0000","last_fullsized":"2024-09-16T01:38:11.620417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:47.185489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.561227+0000","last_change":"2024-09-16T01:38:08.555125+0000","last_active":"2024-09-16T01:38:39.561227+0000","last_peered":"2024-09-16T01:38:39.561227+0000","last_clean":"2024-09-16T01:38:39.561227+0000","last_became_active":"2024-09-16T01:38:08.554756+0000","last_became_peered":"2024-09-16T01:38:08.554756+0000","last_unstale":"2024-09-16T01:38:39.561227+0000","last_undegraded":"2024-09-16T01:38:39.561227+0000","last_fullsized":"2024-09-16T01:38:39.561227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:08.888006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551318+0000","last_change":"2024-09-16T01:38:08.557906+0000","last_active":"2024-09-16T01:38:11.551318+0000","last_peered":"2024-09-16T01:38:11.551318+0000","last_clean":"2024-09-16T01:38:11.551318+0000","last_became_active":"2024-09-16T01:38:08.557546+0000","last_became_peered":"2024-09-16T01:38:08.557546+0000","last_unstale":"2024-09-16T01:38:11.551318+0000","last_undegraded":"2024-09-16T01:38:11.551318+0000","last_fullsized":"2024-09-16T01:38:11.551318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:27:04.988778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.622024+0000","last_change":"2024-09-16T01:38:01.528430+0000","last_active":"2024-09-16T01:38:11.622024+0000","last_peered":"2024-09-16T01:38:11.622024+0000","last_clean":"2024-09-16T01:38:11.622024+0000","last_became_active":"2024-09-16T01:38:01.528069+0000","last_became_peered":"2024-09-16T01:38:01.528069+0000","last_unstale":"2024-09-16T01:38:11.622024+0000","last_undegraded":"2024-09-16T01:38:11.622024+0000","last_fullsized":"2024-09-16T01:38:11.622024+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:41.317215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":62,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:53.589100+0000","last_change":"2024-09-16T01:37:59.509017+0000","last_active":"2024-09-16T01:38:53.589100+0000","last_peered":"2024-09-16T01:38:53.589100+0000","last_clean":"2024-09-16T01:38:53.589100+0000","last_became_active":"2024-09-16T01:37:59.508252+0000","last_became_peered":"2024-09-16T01:37:59.508252+0000","last_unstale":"2024-09-16T01:38:53.589100+0000","last_undegraded":"2024-09-16T01:38:53.589100+0000","last_fullsized":"2024-09-16T01:38:53.589100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:26.215407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619849+0000","last_change":"2024-09-16T01:38:08.565197+0000","last_active":"2024-09-16T01:38:11.619849+0000","last_peered":"2024-09-16T01:38:11.619849+0000","last_clean":"2024-09-16T01:38:11.619849+0000","last_became_active":"2024-09-16T01:38:08.562773+0000","last_became_peered":"2024-09-16T01:38:08.562773+0000","last_unstale":"2024-09-16T01:38:11.619849+0000","last_undegraded":"2024-09-16T01:38:11.619849+0000","last_fullsized":"2024-09-16T01:38:11.619849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:13.316950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619921+0000","last_change":"2024-09-16T01:38:08.554052+0000","last_active":"2024-09-16T01:38:11.619921+0000","last_peered":"2024-09-16T01:38:11.619921+0000","last_clean":"2024-09-16T01:38:11.619921+0000","last_became_active":"2024-09-16T01:38:08.553660+0000","last_became_peered":"2024-09-16T01:38:08.553660+0000","last_unstale":"2024-09-16T01:38:11.619921+0000","last_undegraded":"2024-09-16T01:38:11.619921+0000","last_fullsized":"2024-09-16T01:38:11.619921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:53:52.270695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551393+0000","last_change":"2024-09-16T01:38:01.522688+0000","last_active":"2024-09-16T01:38:11.551393+0000","last_peered":"2024-09-16T01:38:11.551393+0000","last_clean":"2024-09-16T01:38:11.551393+0000","last_became_active":"2024-09-16T01:38:01.519843+0000","last_became_peered":"2024-09-16T01:38:01.519843+0000","last_unstale":"2024-09-16T01:38:11.551393+0000","last_undegraded":"2024-09-16T01:38:11.551393+0000","last_fullsized":"2024-09-16T01:38:11.551393+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:50.971760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:39.561005+0000","last_change":"2024-09-16T01:37:59.509044+0000","last_active":"2024-09-16T01:38:39.561005+0000","last_peered":"2024-09-16T01:38:39.561005+0000","last_clean":"2024-09-16T01:38:39.561005+0000","last_became_active":"2024-09-16T01:37:59.507085+0000","last_became_peered":"2024-09-16T01:37:59.507085+0000","last_unstale":"2024-09-16T01:38:39.561005+0000","last_undegraded":"2024-09-16T01:38:39.561005+0000","last_fullsized":"2024-09-16T01:38:39.561005+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:25:09.930945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620278+0000","last_change":"2024-09-16T01:38:08.561250+0000","last_active":"2024-09-16T01:38:11.620278+0000","last_peered":"2024-09-16T01:38:11.620278+0000","last_clean":"2024-09-16T01:38:11.620278+0000","last_became_active":"2024-09-16T01:38:08.560866+0000","last_became_peered":"2024-09-16T01:38:08.560866+0000","last_unstale":"2024-09-16T01:38:11.620278+0000","last_undegraded":"2024-09-16T01:38:11.620278+0000","last_fullsized":"2024-09-16T01:38:11.620278+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:31.253817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619727+0000","last_change":"2024-09-16T01:38:01.517400+0000","last_active":"2024-09-16T01:38:11.619727+0000","last_peered":"2024-09-16T01:38:11.619727+0000","last_clean":"2024-09-16T01:38:11.619727+0000","last_became_active":"2024-09-16T01:38:01.517103+0000","last_became_peered":"2024-09-16T01:38:01.517103+0000","last_unstale":"2024-09-16T01:38:11.619727+0000","last_undegraded":"2024-09-16T01:38:11.619727+0000","last_fullsized":"2024-09-16T01:38:11.619727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:37.886179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:37.539100+0000","last_change":"2024-09-16T01:37:59.503370+0000","last_active":"2024-09-16T01:38:37.539100+0000","last_peered":"2024-09-16T01:38:37.539100+0000","last_clean":"2024-09-16T01:38:37.539100+0000","last_became_active":"2024-09-16T01:37:59.503071+0000","last_became_peered":"2024-09-16T01:37:59.503071+0000","last_unstale":"2024-09-16T01:38:37.539100+0000","last_undegraded":"2024-09-16T01:38:37.539100+0000","last_fullsized":"2024-09-16T01:38:37.539100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:51.207702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:38.547216+0000","last_change":"2024-09-16T01:37:59.506110+0000","last_active":"2024-09-16T01:38:38.547216+0000","last_peered":"2024-09-16T01:38:38.547216+0000","last_clean":"2024-09-16T01:38:38.547216+0000","last_became_active":"2024-09-16T01:37:59.505264+0000","last_became_peered":"2024-09-16T01:37:59.505264+0000","last_unstale":"2024-09-16T01:38:38.547216+0000","last_undegraded":"2024-09-16T01:38:38.547216+0000","last_fullsized":"2024-09-16T01:38:38.547216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:35.986433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620965+0000","last_change":"2024-09-16T01:38:08.563900+0000","last_active":"2024-09-16T01:38:11.620965+0000","last_peered":"2024-09-16T01:38:11.620965+0000","last_clean":"2024-09-16T01:38:11.620965+0000","last_became_active":"2024-09-16T01:38:08.563551+0000","last_became_peered":"2024-09-16T01:38:08.563551+0000","last_unstale":"2024-09-16T01:38:11.620965+0000","last_undegraded":"2024-09-16T01:38:11.620965+0000","last_fullsized":"2024-09-16T01:38:11.620965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:31:03.979580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552533+0000","last_change":"2024-09-16T01:38:08.556011+0000","last_active":"2024-09-16T01:38:11.552533+0000","last_peered":"2024-09-16T01:38:11.552533+0000","last_clean":"2024-09-16T01:38:11.552533+0000","last_became_active":"2024-09-16T01:38:08.555615+0000","last_became_peered":"2024-09-16T01:38:08.555615+0000","last_unstale":"2024-09-16T01:38:11.552533+0000","last_undegraded":"2024-09-16T01:38:11.552533+0000","last_fullsized":"2024-09-16T01:38:11.552533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:34.434974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:49.584444+0000","last_change":"2024-09-16T01:37:59.508411+0000","last_active":"2024-09-16T01:38:49.584444+0000","last_peered":"2024-09-16T01:38:49.584444+0000","last_clean":"2024-09-16T01:38:49.584444+0000","last_became_active":"2024-09-16T01:37:59.508058+0000","last_became_peered":"2024-09-16T01:37:59.508058+0000","last_unstale":"2024-09-16T01:38:49.584444+0000","last_undegraded":"2024-09-16T01:38:49.584444+0000","last_fullsized":"2024-09-16T01:38:49.584444+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:47.441673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621306+0000","last_change":"2024-09-16T01:38:08.560930+0000","last_active":"2024-09-16T01:38:11.621306+0000","last_peered":"2024-09-16T01:38:11.621306+0000","last_clean":"2024-09-16T01:38:11.621306+0000","last_became_active":"2024-09-16T01:38:08.560490+0000","last_became_peered":"2024-09-16T01:38:08.560490+0000","last_unstale":"2024-09-16T01:38:11.621306+0000","last_undegraded":"2024-09-16T01:38:11.621306+0000","last_fullsized":"2024-09-16T01:38:11.621306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:11.556113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552811+0000","last_change":"2024-09-16T01:38:08.557564+0000","last_active":"2024-09-16T01:38:11.552811+0000","last_peered":"2024-09-16T01:38:11.552811+0000","last_clean":"2024-09-16T01:38:11.552811+0000","last_became_active":"2024-09-16T01:38:08.557143+0000","last_became_peered":"2024-09-16T01:38:08.557143+0000","last_unstale":"2024-09-16T01:38:11.552811+0000","last_undegraded":"2024-09-16T01:38:11.552811+0000","last_fullsized":"2024-09-16T01:38:11.552811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:43.148870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620013+0000","last_change":"2024-09-16T01:38:01.527353+0000","last_active":"2024-09-16T01:38:11.620013+0000","last_peered":"2024-09-16T01:38:11.620013+0000","last_clean":"2024-09-16T01:38:11.620013+0000","last_became_active":"2024-09-16T01:38:01.527031+0000","last_became_peered":"2024-09-16T01:38:01.527031+0000","last_unstale":"2024-09-16T01:38:11.620013+0000","last_undegraded":"2024-09-16T01:38:11.620013+0000","last_fullsized":"2024-09-16T01:38:11.620013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:27.311638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.553003+0000","last_change":"2024-09-16T01:38:01.526409+0000","last_active":"2024-09-16T01:38:11.553003+0000","last_peered":"2024-09-16T01:38:11.553003+0000","last_clean":"2024-09-16T01:38:11.553003+0000","last_became_active":"2024-09-16T01:38:01.526064+0000","last_became_peered":"2024-09-16T01:38:01.526064+0000","last_unstale":"2024-09-16T01:38:11.553003+0000","last_undegraded":"2024-09-16T01:38:11.553003+0000","last_fullsized":"2024-09-16T01:38:11.553003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:51.571808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552380+0000","last_change":"2024-09-16T01:38:01.521856+0000","last_active":"2024-09-16T01:38:11.552380+0000","last_peered":"2024-09-16T01:38:11.552380+0000","last_clean":"2024-09-16T01:38:11.552380+0000","last_became_active":"2024-09-16T01:38:01.520312+0000","last_became_peered":"2024-09-16T01:38:01.520312+0000","last_unstale":"2024-09-16T01:38:11.552380+0000","last_undegraded":"2024-09-16T01:38:11.552380+0000","last_fullsized":"2024-09-16T01:38:11.552380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:53.618990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620192+0000","last_change":"2024-09-16T01:38:01.520958+0000","last_active":"2024-09-16T01:38:11.620192+0000","last_peered":"2024-09-16T01:38:11.620192+0000","last_clean":"2024-09-16T01:38:11.620192+0000","last_became_active":"2024-09-16T01:38:01.520658+0000","last_became_peered":"2024-09-16T01:38:01.520658+0000","last_unstale":"2024-09-16T01:38:11.620192+0000","last_undegraded":"2024-09-16T01:38:11.620192+0000","last_fullsized":"2024-09-16T01:38:11.620192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:23.770694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009571+0000","last_change":"2024-09-16T01:38:01.519140+0000","last_active":"2024-09-16T01:38:12.009571+0000","last_peered":"2024-09-16T01:38:12.009571+0000","last_clean":"2024-09-16T01:38:12.009571+0000","last_became_active":"2024-09-16T01:38:01.518765+0000","last_became_peered":"2024-09-16T01:38:01.518765+0000","last_unstale":"2024-09-16T01:38:12.009571+0000","last_undegraded":"2024-09-16T01:38:12.009571+0000","last_fullsized":"2024-09-16T01:38:12.009571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:17.194886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621459+0000","last_change":"2024-09-16T01:38:01.523762+0000","last_active":"2024-09-16T01:38:11.621459+0000","last_peered":"2024-09-16T01:38:11.621459+0000","last_clean":"2024-09-16T01:38:11.621459+0000","last_became_active":"2024-09-16T01:38:01.522035+0000","last_became_peered":"2024-09-16T01:38:01.522035+0000","last_unstale":"2024-09-16T01:38:11.621459+0000","last_undegraded":"2024-09-16T01:38:11.621459+0000","last_fullsized":"2024-09-16T01:38:11.621459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:40.405350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552992+0000","last_change":"2024-09-16T01:38:01.527539+0000","last_active":"2024-09-16T01:38:11.552992+0000","last_peered":"2024-09-16T01:38:11.552992+0000","last_clean":"2024-09-16T01:38:11.552992+0000","last_became_active":"2024-09-16T01:38:01.526974+0000","last_became_peered":"2024-09-16T01:38:01.526974+0000","last_unstale":"2024-09-16T01:38:11.552992+0000","last_undegraded":"2024-09-16T01:38:11.552992+0000","last_fullsized":"2024-09-16T01:38:11.552992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:30:03.587141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620841+0000","last_change":"2024-09-16T01:38:01.529131+0000","last_active":"2024-09-16T01:38:11.620841+0000","last_peered":"2024-09-16T01:38:11.620841+0000","last_clean":"2024-09-16T01:38:11.620841+0000","last_became_active":"2024-09-16T01:38:01.528796+0000","last_became_peered":"2024-09-16T01:38:01.528796+0000","last_unstale":"2024-09-16T01:38:11.620841+0000","last_undegraded":"2024-09-16T01:38:11.620841+0000","last_fullsized":"2024-09-16T01:38:11.620841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:38.903638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620130+0000","last_change":"2024-09-16T01:38:01.514779+0000","last_active":"2024-09-16T01:38:11.620130+0000","last_peered":"2024-09-16T01:38:11.620130+0000","last_clean":"2024-09-16T01:38:11.620130+0000","last_became_active":"2024-09-16T01:38:01.514434+0000","last_became_peered":"2024-09-16T01:38:01.514434+0000","last_unstale":"2024-09-16T01:38:11.620130+0000","last_undegraded":"2024-09-16T01:38:11.620130+0000","last_fullsized":"2024-09-16T01:38:11.620130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:31.382111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619991+0000","last_change":"2024-09-16T01:38:01.521771+0000","last_active":"2024-09-16T01:38:11.619991+0000","last_peered":"2024-09-16T01:38:11.619991+0000","last_clean":"2024-09-16T01:38:11.619991+0000","last_became_active":"2024-09-16T01:38:01.521288+0000","last_became_peered":"2024-09-16T01:38:01.521288+0000","last_unstale":"2024-09-16T01:38:11.619991+0000","last_undegraded":"2024-09-16T01:38:11.619991+0000","last_fullsized":"2024-09-16T01:38:11.619991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:11.825164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621313+0000","last_change":"2024-09-16T01:38:01.529247+0000","last_active":"2024-09-16T01:38:11.621313+0000","last_peered":"2024-09-16T01:38:11.621313+0000","last_clean":"2024-09-16T01:38:11.621313+0000","last_became_active":"2024-09-16T01:38:01.528926+0000","last_became_peered":"2024-09-16T01:38:01.528926+0000","last_unstale":"2024-09-16T01:38:11.621313+0000","last_undegraded":"2024-09-16T01:38:11.621313+0000","last_fullsized":"2024-09-16T01:38:11.621313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:20.704114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009229+0000","last_change":"2024-09-16T01:38:01.520830+0000","last_active":"2024-09-16T01:38:12.009229+0000","last_peered":"2024-09-16T01:38:12.009229+0000","last_clean":"2024-09-16T01:38:12.009229+0000","last_became_active":"2024-09-16T01:38:01.520526+0000","last_became_peered":"2024-09-16T01:38:01.520526+0000","last_unstale":"2024-09-16T01:38:12.009229+0000","last_undegraded":"2024-09-16T01:38:12.009229+0000","last_fullsized":"2024-09-16T01:38:12.009229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:28.062812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621042+0000","last_change":"2024-09-16T01:38:01.521983+0000","last_active":"2024-09-16T01:38:11.621042+0000","last_peered":"2024-09-16T01:38:11.621042+0000","last_clean":"2024-09-16T01:38:11.621042+0000","last_became_active":"2024-09-16T01:38:01.521666+0000","last_became_peered":"2024-09-16T01:38:01.521666+0000","last_unstale":"2024-09-16T01:38:11.621042+0000","last_undegraded":"2024-09-16T01:38:11.621042+0000","last_fullsized":"2024-09-16T01:38:11.621042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:50:55.761313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.553027+0000","last_change":"2024-09-16T01:38:08.554304+0000","last_active":"2024-09-16T01:38:11.553027+0000","last_peered":"2024-09-16T01:38:11.553027+0000","last_clean":"2024-09-16T01:38:11.553027+0000","last_became_active":"2024-09-16T01:38:08.553683+0000","last_became_peered":"2024-09-16T01:38:08.553683+0000","last_unstale":"2024-09-16T01:38:11.553027+0000","last_undegraded":"2024-09-16T01:38:11.553027+0000","last_fullsized":"2024-09-16T01:38:11.553027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:56.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551871+0000","last_change":"2024-09-16T01:38:01.520425+0000","last_active":"2024-09-16T01:38:11.551871+0000","last_peered":"2024-09-16T01:38:11.551871+0000","last_clean":"2024-09-16T01:38:11.551871+0000","last_became_active":"2024-09-16T01:38:01.519858+0000","last_became_peered":"2024-09-16T01:38:01.519858+0000","last_unstale":"2024-09-16T01:38:11.551871+0000","last_undegraded":"2024-09-16T01:38:11.551871+0000","last_fullsized":"2024-09-16T01:38:11.551871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:48.534298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620926+0000","last_change":"2024-09-16T01:38:01.532762+0000","last_active":"2024-09-16T01:38:11.620926+0000","last_peered":"2024-09-16T01:38:11.620926+0000","last_clean":"2024-09-16T01:38:11.620926+0000","last_became_active":"2024-09-16T01:38:01.532404+0000","last_became_peered":"2024-09-16T01:38:01.532404+0000","last_unstale":"2024-09-16T01:38:11.620926+0000","last_undegraded":"2024-09-16T01:38:11.620926+0000","last_fullsized":"2024-09-16T01:38:11.620926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:03.135141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620376+0000","last_change":"2024-09-16T01:38:01.530205+0000","last_active":"2024-09-16T01:38:11.620376+0000","last_peered":"2024-09-16T01:38:11.620376+0000","last_clean":"2024-09-16T01:38:11.620376+0000","last_became_active":"2024-09-16T01:38:01.529898+0000","last_became_peered":"2024-09-16T01:38:01.529898+0000","last_unstale":"2024-09-16T01:38:11.620376+0000","last_undegraded":"2024-09-16T01:38:11.620376+0000","last_fullsized":"2024-09-16T01:38:11.620376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:48.444441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619812+0000","last_change":"2024-09-16T01:38:01.529407+0000","last_active":"2024-09-16T01:38:11.619812+0000","last_peered":"2024-09-16T01:38:11.619812+0000","last_clean":"2024-09-16T01:38:11.619812+0000","last_became_active":"2024-09-16T01:38:01.528330+0000","last_became_peered":"2024-09-16T01:38:01.528330+0000","last_unstale":"2024-09-16T01:38:11.619812+0000","last_undegraded":"2024-09-16T01:38:11.619812+0000","last_fullsized":"2024-09-16T01:38:11.619812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:32.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619804+0000","last_change":"2024-09-16T01:38:01.519579+0000","last_active":"2024-09-16T01:38:11.619804+0000","last_peered":"2024-09-16T01:38:11.619804+0000","last_clean":"2024-09-16T01:38:11.619804+0000","last_became_active":"2024-09-16T01:38:01.519211+0000","last_became_peered":"2024-09-16T01:38:01.519211+0000","last_unstale":"2024-09-16T01:38:11.619804+0000","last_undegraded":"2024-09-16T01:38:11.619804+0000","last_fullsized":"2024-09-16T01:38:11.619804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:18.861169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:12.009215+0000","last_change":"2024-09-16T01:38:01.521136+0000","last_active":"2024-09-16T01:38:12.009215+0000","last_peered":"2024-09-16T01:38:12.009215+0000","last_clean":"2024-09-16T01:38:12.009215+0000","last_became_active":"2024-09-16T01:38:01.520819+0000","last_became_peered":"2024-09-16T01:38:01.520819+0000","last_unstale":"2024-09-16T01:38:12.009215+0000","last_undegraded":"2024-09-16T01:38:12.009215+0000","last_fullsized":"2024-09-16T01:38:12.009215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:28.150822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621953+0000","last_change":"2024-09-16T01:38:01.519382+0000","last_active":"2024-09-16T01:38:11.621953+0000","last_peered":"2024-09-16T01:38:11.621953+0000","last_clean":"2024-09-16T01:38:11.621953+0000","last_became_active":"2024-09-16T01:38:01.519039+0000","last_became_peered":"2024-09-16T01:38:01.519039+0000","last_unstale":"2024-09-16T01:38:11.621953+0000","last_undegraded":"2024-09-16T01:38:11.621953+0000","last_fullsized":"2024-09-16T01:38:11.621953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:46.559357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621100+0000","last_change":"2024-09-16T01:38:01.527513+0000","last_active":"2024-09-16T01:38:11.621100+0000","last_peered":"2024-09-16T01:38:11.621100+0000","last_clean":"2024-09-16T01:38:11.621100+0000","last_became_active":"2024-09-16T01:38:01.527184+0000","last_became_peered":"2024-09-16T01:38:01.527184+0000","last_unstale":"2024-09-16T01:38:11.621100+0000","last_undegraded":"2024-09-16T01:38:11.621100+0000","last_fullsized":"2024-09-16T01:38:11.621100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:41.508318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.553164+0000","last_change":"2024-09-16T01:38:01.524239+0000","last_active":"2024-09-16T01:38:11.553164+0000","last_peered":"2024-09-16T01:38:11.553164+0000","last_clean":"2024-09-16T01:38:11.553164+0000","last_became_active":"2024-09-16T01:38:01.523906+0000","last_became_peered":"2024-09-16T01:38:01.523906+0000","last_unstale":"2024-09-16T01:38:11.553164+0000","last_undegraded":"2024-09-16T01:38:11.553164+0000","last_fullsized":"2024-09-16T01:38:11.553164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:11.718796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.621894+0000","last_change":"2024-09-16T01:38:01.531564+0000","last_active":"2024-09-16T01:38:11.621894+0000","last_peered":"2024-09-16T01:38:11.621894+0000","last_clean":"2024-09-16T01:38:11.621894+0000","last_became_active":"2024-09-16T01:38:01.531241+0000","last_became_peered":"2024-09-16T01:38:01.531241+0000","last_unstale":"2024-09-16T01:38:11.621894+0000","last_undegraded":"2024-09-16T01:38:11.621894+0000","last_fullsized":"2024-09-16T01:38:11.621894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:16.244641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619652+0000","last_change":"2024-09-16T01:38:01.519913+0000","last_active":"2024-09-16T01:38:11.619652+0000","last_peered":"2024-09-16T01:38:11.619652+0000","last_clean":"2024-09-16T01:38:11.619652+0000","last_became_active":"2024-09-16T01:38:01.518631+0000","last_became_peered":"2024-09-16T01:38:01.518631+0000","last_unstale":"2024-09-16T01:38:11.619652+0000","last_undegraded":"2024-09-16T01:38:11.619652+0000","last_fullsized":"2024-09-16T01:38:11.619652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:48.747188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620304+0000","last_change":"2024-09-16T01:38:01.530805+0000","last_active":"2024-09-16T01:38:11.620304+0000","last_peered":"2024-09-16T01:38:11.620304+0000","last_clean":"2024-09-16T01:38:11.620304+0000","last_became_active":"2024-09-16T01:38:01.530466+0000","last_became_peered":"2024-09-16T01:38:01.530466+0000","last_unstale":"2024-09-16T01:38:11.620304+0000","last_undegraded":"2024-09-16T01:38:11.620304+0000","last_fullsized":"2024-09-16T01:38:11.620304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:41:30.587398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.552664+0000","last_change":"2024-09-16T01:38:01.519227+0000","last_active":"2024-09-16T01:38:11.552664+0000","last_peered":"2024-09-16T01:38:11.552664+0000","last_clean":"2024-09-16T01:38:11.552664+0000","last_became_active":"2024-09-16T01:38:01.518875+0000","last_became_peered":"2024-09-16T01:38:01.518875+0000","last_unstale":"2024-09-16T01:38:11.552664+0000","last_undegraded":"2024-09-16T01:38:11.552664+0000","last_fullsized":"2024-09-16T01:38:11.552664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:42.654812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620696+0000","last_change":"2024-09-16T01:38:01.513327+0000","last_active":"2024-09-16T01:38:11.620696+0000","last_peered":"2024-09-16T01:38:11.620696+0000","last_clean":"2024-09-16T01:38:11.620696+0000","last_became_active":"2024-09-16T01:38:01.513000+0000","last_became_peered":"2024-09-16T01:38:01.513000+0000","last_unstale":"2024-09-16T01:38:11.620696+0000","last_undegraded":"2024-09-16T01:38:11.620696+0000","last_fullsized":"2024-09-16T01:38:11.620696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:43.036805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.551739+0000","last_change":"2024-09-16T01:38:08.560560+0000","last_active":"2024-09-16T01:38:11.551739+0000","last_peered":"2024-09-16T01:38:11.551739+0000","last_clean":"2024-09-16T01:38:11.551739+0000","last_became_active":"2024-09-16T01:38:08.560229+0000","last_became_peered":"2024-09-16T01:38:08.560229+0000","last_unstale":"2024-09-16T01:38:11.551739+0000","last_undegraded":"2024-09-16T01:38:11.551739+0000","last_fullsized":"2024-09-16T01:38:11.551739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:10:09.462043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.619851+0000","last_change":"2024-09-16T01:38:01.529548+0000","last_active":"2024-09-16T01:38:11.619851+0000","last_peered":"2024-09-16T01:38:11.619851+0000","last_clean":"2024-09-16T01:38:11.619851+0000","last_became_active":"2024-09-16T01:38:01.529011+0000","last_became_peered":"2024-09-16T01:38:01.529011+0000","last_unstale":"2024-09-16T01:38:11.619851+0000","last_undegraded":"2024-09-16T01:38:11.619851+0000","last_fullsized":"2024-09-16T01:38:11.619851+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:21.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620009+0000","last_change":"2024-09-16T01:38:01.519488+0000","last_active":"2024-09-16T01:38:11.620009+0000","last_peered":"2024-09-16T01:38:11.620009+0000","last_clean":"2024-09-16T01:38:11.620009+0000","last_became_active":"2024-09-16T01:38:01.519156+0000","last_became_peered":"2024-09-16T01:38:01.519156+0000","last_unstale":"2024-09-16T01:38:11.620009+0000","last_undegraded":"2024-09-16T01:38:11.620009+0000","last_fullsized":"2024-09-16T01:38:11.620009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:18.149874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T01:38:11.620820+0000","last_change":"2024-09-16T01:38:01.527519+0000","last_active":"2024-09-16T01:38:11.620820+0000","last_peered":"2024-09-16T01:38:11.620820+0000","last_clean":"2024-09-16T01:38:11.620820+0000","last_became_active":"2024-09-16T01:38:01.527175+0000","last_became_peered":"2024-09-16T01:38:01.527175+0000","last_unstale":"2024-09-16T01:38:11.620820+0000","last_undegraded":"2024-09-16T01:38:11.620820+0000","last_fullsized":"2024-09-16T01:38:11.620820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:04.958842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59223,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":163,"num_read_kb":151,"num_write":241,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":177669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281435,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678040,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":469646,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906973,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1108365,"data_compressed":53519,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070498,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":473167,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299430,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":468665,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528363,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":1065555,"data_compressed":53519,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659185,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":464575,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888118,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":1061137,"data_compressed":53519,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051643,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":468649,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280576,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":464283,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476806,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":518626,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607627,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":518685,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":466424,"data_compressed":17725,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34155,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34155,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34155,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":54832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T01:38:56.181 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T01:38:56.181 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T01:38:56.182 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T01:38:56.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:56 smithi139 ceph-mon[31175]: pgmap v307: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 15 op/s 2024-09-16T01:38:56.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:56 smithi139 ceph-mon[31175]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:38:56.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:56 smithi176 ceph-mon[35324]: pgmap v307: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 15 op/s 2024-09-16T01:38:56.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:56 smithi176 ceph-mon[35324]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:38:57.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:56 smithi164 ceph-mon[35121]: pgmap v307: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 15 op/s 2024-09-16T01:38:57.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:56 smithi164 ceph-mon[35121]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:38:57.086 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:57.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.588+0000 7f5698ea6640 1 -- 172.21.15.139:0/3931360237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56941026a0 msgr2=0x7f5694104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:57.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.588+0000 7f5698ea6640 1 --2- 172.21.15.139:0/3931360237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56941026a0 0x7f5694104a90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5684009900 tx=0x7f568402a410 comp rx=0 tx=0).stop 2024-09-16T01:38:57.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.588+0000 7f5698ea6640 1 -- 172.21.15.139:0/3931360237 shutdown_connections 2024-09-16T01:38:57.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.588+0000 7f5698ea6640 1 --2- 172.21.15.139:0/3931360237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5694107900 0x7f5694109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:57.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.588+0000 7f5698ea6640 1 --2- 172.21.15.139:0/3931360237 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5694104fd0 0x7f56941073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:57.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.588+0000 7f5698ea6640 1 --2- 172.21.15.139:0/3931360237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56941026a0 0x7f5694104a90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:57.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.588+0000 7f5698ea6640 1 -- 172.21.15.139:0/3931360237 >> 172.21.15.139:0/3931360237 conn(0x7f56940fc2a0 msgr2=0x7f56940fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:57.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.588+0000 7f5698ea6640 1 -- 172.21.15.139:0/3931360237 shutdown_connections 2024-09-16T01:38:57.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.588+0000 7f5698ea6640 1 -- 172.21.15.139:0/3931360237 wait complete. 2024-09-16T01:38:57.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5698ea6640 1 Processor -- start 2024-09-16T01:38:57.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5698ea6640 1 -- start start 2024-09-16T01:38:57.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5698ea6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56941026a0 0x7f5694197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:57.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5698ea6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5694104fd0 0x7f5694198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5698ea6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5694107900 0x7f569419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5698ea6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f569410d8b0 con 0x7f5694104fd0 2024-09-16T01:38:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5698ea6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f569410d730 con 0x7f5694107900 2024-09-16T01:38:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5698ea6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f569410da30 con 0x7f56941026a0 2024-09-16T01:38:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5691d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5694104fd0 0x7f5694198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5691d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5694104fd0 0x7f5694198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40194/0 (socket says 172.21.15.139:40194) 2024-09-16T01:38:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5691d9b640 1 -- 172.21.15.139:0/1070058059 learned_addr learned my addr 172.21.15.139:0/1070058059 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f569259c640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56941026a0 0x7f5694197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5692d9d640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5694107900 0x7f569419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5691d9b640 1 -- 172.21.15.139:0/1070058059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56941026a0 msgr2=0x7f5694197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5691d9b640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56941026a0 0x7f5694197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5691d9b640 1 -- 172.21.15.139:0/1070058059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5694107900 msgr2=0x7f569419f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5691d9b640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5694107900 0x7f569419f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5691d9b640 1 -- 172.21.15.139:0/1070058059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569419fb10 con 0x7f5694104fd0 2024-09-16T01:38:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5692d9d640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5694107900 0x7f569419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:38:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.589+0000 7f5691d9b640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5694104fd0 0x7f5694198420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f568800c970 tx=0x7f568800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.590+0000 7f56837fe640 1 -- 172.21.15.139:0/1070058059 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688014070 con 0x7f5694104fd0 2024-09-16T01:38:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.590+0000 7f56837fe640 1 -- 172.21.15.139:0/1070058059 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5688004440 con 0x7f5694104fd0 2024-09-16T01:38:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.590+0000 7f5698ea6640 1 -- 172.21.15.139:0/1070058059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569419fe00 con 0x7f5694104fd0 2024-09-16T01:38:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.590+0000 7f56837fe640 1 -- 172.21.15.139:0/1070058059 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568800f450 con 0x7f5694104fd0 2024-09-16T01:38:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.590+0000 7f5698ea6640 1 -- 172.21.15.139:0/1070058059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56941a0300 con 0x7f5694104fd0 2024-09-16T01:38:57.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.591+0000 7f5698ea6640 1 -- 172.21.15.139:0/1070058059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56940fdd90 con 0x7f5694104fd0 2024-09-16T01:38:57.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.592+0000 7f56837fe640 1 -- 172.21.15.139:0/1070058059 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5688017070 con 0x7f5694104fd0 2024-09-16T01:38:57.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.593+0000 7f56837fe640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f567c0a3380 0x7f567c0a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:57.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.593+0000 7f569259c640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f567c0a3380 0x7f567c0a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:57.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.593+0000 7f56837fe640 1 -- 172.21.15.139:0/1070058059 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f56880cf860 con 0x7f5694104fd0 2024-09-16T01:38:57.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.593+0000 7f569259c640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f567c0a3380 0x7f567c0a5840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f56840314e0 tx=0x7f5684000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:57.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.597+0000 7f56837fe640 1 -- 172.21.15.139:0/1070058059 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f568801e030 con 0x7f5694104fd0 2024-09-16T01:38:57.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.859+0000 7f5698ea6640 1 -- 172.21.15.139:0/1070058059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f569402da10 con 0x7f5694104fd0 2024-09-16T01:38:57.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.861+0000 7f56837fe640 1 -- 172.21.15.139:0/1070058059 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f568809c0c0 con 0x7f5694104fd0 2024-09-16T01:38:57.863 INFO:teuthology.orchestra.run.smithi139.stdout:[client.0] 2024-09-16T01:38:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQAxjOdmkwVRMxAAh+lreDcB9/kCOKLRToFGjQ== 2024-09-16T01:38:57.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.865+0000 7f5698ea6640 1 -- 172.21.15.139:0/1070058059 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f567c0a3380 msgr2=0x7f567c0a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:57.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.865+0000 7f5698ea6640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f567c0a3380 0x7f567c0a5840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f56840314e0 tx=0x7f5684000bc0 comp rx=0 tx=0).stop 2024-09-16T01:38:57.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.865+0000 7f5698ea6640 1 -- 172.21.15.139:0/1070058059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5694104fd0 msgr2=0x7f5694198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:57.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.865+0000 7f5698ea6640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5694104fd0 0x7f5694198420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f568800c970 tx=0x7f568800ce30 comp rx=0 tx=0).stop 2024-09-16T01:38:57.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.865+0000 7f5698ea6640 1 -- 172.21.15.139:0/1070058059 shutdown_connections 2024-09-16T01:38:57.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.865+0000 7f5698ea6640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f567c0a3380 0x7f567c0a5840 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:57.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.865+0000 7f5698ea6640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5694107900 0x7f569419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:57.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.865+0000 7f5698ea6640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5694104fd0 0x7f5694198420 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:57.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.865+0000 7f5698ea6640 1 --2- 172.21.15.139:0/1070058059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56941026a0 0x7f5694197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:57.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.865+0000 7f5698ea6640 1 -- 172.21.15.139:0/1070058059 >> 172.21.15.139:0/1070058059 conn(0x7f56940fc2a0 msgr2=0x7f5694102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:57.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.865+0000 7f5698ea6640 1 -- 172.21.15.139:0/1070058059 shutdown_connections 2024-09-16T01:38:57.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:57.865+0000 7f5698ea6640 1 -- 172.21.15.139:0/1070058059 wait complete. 2024-09-16T01:38:58.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:57 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:57 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:57 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:57 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:57 smithi139 ceph-mon[31175]: pgmap v308: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-09-16T01:38:58.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:57 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1070058059' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T01:38:58.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:57 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1070058059' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T01:38:58.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:57 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:57 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:57 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:57 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:57 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:57 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:57 smithi164 ceph-mon[35121]: pgmap v308: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-09-16T01:38:58.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:57 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1070058059' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T01:38:58.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:57 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1070058059' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T01:38:58.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:57 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:57 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:57 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:57 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:57 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:57 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:57 smithi176 ceph-mon[35324]: pgmap v308: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-09-16T01:38:58.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:57 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1070058059' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T01:38:58.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:57 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1070058059' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T01:38:58.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:57 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:57 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:58.575 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:38:58.576 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T01:38:58.576 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T01:38:58.616 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T01:38:58.616 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T01:38:58.617 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T01:38:58.909 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:38:59.347 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:38:59.347 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:38:59.347 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:38:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.345+0000 7f31e369c640 1 -- 172.21.15.139:0/1967771550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc102790 msgr2=0x7f31dc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.345+0000 7f31e369c640 1 --2- 172.21.15.139:0/1967771550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc102790 0x7f31dc1067e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009900 tx=0x7f31d002a410 comp rx=0 tx=0).stop 2024-09-16T01:38:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.345+0000 7f31e369c640 1 -- 172.21.15.139:0/1967771550 shutdown_connections 2024-09-16T01:38:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.345+0000 7f31e369c640 1 --2- 172.21.15.139:0/1967771550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31dc106f10 0x7f31dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.345+0000 7f31e369c640 1 --2- 172.21.15.139:0/1967771550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc102790 0x7f31dc1067e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.345+0000 7f31e369c640 1 --2- 172.21.15.139:0/1967771550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f31dc101de0 0x7f31dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.345+0000 7f31e369c640 1 -- 172.21.15.139:0/1967771550 >> 172.21.15.139:0/1967771550 conn(0x7f31dc0fd8c0 msgr2=0x7f31dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.345+0000 7f31e369c640 1 -- 172.21.15.139:0/1967771550 shutdown_connections 2024-09-16T01:38:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.345+0000 7f31e369c640 1 -- 172.21.15.139:0/1967771550 wait complete. 2024-09-16T01:38:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e369c640 1 Processor -- start 2024-09-16T01:38:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e369c640 1 -- start start 2024-09-16T01:38:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e369c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31dc101de0 0x7f31dc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e369c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f31dc102790 0x7f31dc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e369c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc106f10 0x7f31dc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e369c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc111f30 con 0x7f31dc106f10 2024-09-16T01:38:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e369c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc111db0 con 0x7f31dc101de0 2024-09-16T01:38:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e369c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc1120b0 con 0x7f31dc102790 2024-09-16T01:38:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e1c39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc106f10 0x7f31dc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e1c39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc106f10 0x7f31dc19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40214/0 (socket says 172.21.15.139:40214) 2024-09-16T01:38:59.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e1c39640 1 -- 172.21.15.139:0/3291575523 learned_addr learned my addr 172.21.15.139:0/3291575523 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:38:59.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e1438640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31dc101de0 0x7f31dc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:59.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e0c37640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f31dc102790 0x7f31dc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:59.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e1c39640 1 -- 172.21.15.139:0/3291575523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f31dc102790 msgr2=0x7f31dc198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:59.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e1c39640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f31dc102790 0x7f31dc198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:59.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e1c39640 1 -- 172.21.15.139:0/3291575523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31dc101de0 msgr2=0x7f31dc198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:59.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e1c39640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31dc101de0 0x7f31dc198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:59.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e1c39640 1 -- 172.21.15.139:0/3291575523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31dc19d3e0 con 0x7f31dc106f10 2024-09-16T01:38:59.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e0c37640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f31dc102790 0x7f31dc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:38:59.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.346+0000 7f31e1438640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31dc101de0 0x7f31dc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:38:59.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.347+0000 7f31e1c39640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc106f10 0x7f31dc19cd00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00eaa0 tx=0x7f31cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:59.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.347+0000 7f31da7fc640 1 -- 172.21.15.139:0/3291575523 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc00cdb0 con 0x7f31dc106f10 2024-09-16T01:38:59.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.347+0000 7f31da7fc640 1 -- 172.21.15.139:0/3291575523 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31cc005020 con 0x7f31dc106f10 2024-09-16T01:38:59.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.347+0000 7f31e369c640 1 -- 172.21.15.139:0/3291575523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31dc19d6d0 con 0x7f31dc106f10 2024-09-16T01:38:59.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.347+0000 7f31da7fc640 1 -- 172.21.15.139:0/3291575523 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc0116b0 con 0x7f31dc106f10 2024-09-16T01:38:59.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.347+0000 7f31e369c640 1 -- 172.21.15.139:0/3291575523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31dc1a4fb0 con 0x7f31dc106f10 2024-09-16T01:38:59.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.348+0000 7f31e369c640 1 -- 172.21.15.139:0/3291575523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31dc1021c0 con 0x7f31dc106f10 2024-09-16T01:38:59.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.349+0000 7f31da7fc640 1 -- 172.21.15.139:0/3291575523 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f31cc005560 con 0x7f31dc106f10 2024-09-16T01:38:59.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.349+0000 7f31da7fc640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f31c40a32d0 0x7f31c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:38:59.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.350+0000 7f31e1438640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f31c40a32d0 0x7f31c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:38:59.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.350+0000 7f31da7fc640 1 -- 172.21.15.139:0/3291575523 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f31cc0d00c0 con 0x7f31dc106f10 2024-09-16T01:38:59.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.350+0000 7f31e1438640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f31c40a32d0 0x7f31c40a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f31d40058c0 tx=0x7f31d4005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:38:59.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.354+0000 7f31da7fc640 1 -- 172.21.15.139:0/3291575523 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f31cc016030 con 0x7f31dc106f10 2024-09-16T01:38:59.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:38:59.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:38:59.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:38:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:59.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:38:59.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:38:59.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:38:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:38:59.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.567+0000 7f31e369c640 1 -- 172.21.15.139:0/3291575523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f31dc109b60 con 0x7f31dc106f10 2024-09-16T01:38:59.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.572+0000 7f31da7fc640 1 -- 172.21.15.139:0/3291575523 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264056 (secure 0 0 0) 0x7f31cc09c920 con 0x7f31dc106f10 2024-09-16T01:38:59.576 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:38:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.574+0000 7f31e369c640 1 -- 172.21.15.139:0/3291575523 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f31c40a32d0 msgr2=0x7f31c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.574+0000 7f31e369c640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f31c40a32d0 0x7f31c40a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f31d40058c0 tx=0x7f31d4005ec0 comp rx=0 tx=0).stop 2024-09-16T01:38:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.574+0000 7f31e369c640 1 -- 172.21.15.139:0/3291575523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc106f10 msgr2=0x7f31dc19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:38:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.574+0000 7f31e369c640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc106f10 0x7f31dc19cd00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00eaa0 tx=0x7f31cc00ef60 comp rx=0 tx=0).stop 2024-09-16T01:38:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.574+0000 7f31e369c640 1 -- 172.21.15.139:0/3291575523 shutdown_connections 2024-09-16T01:38:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.574+0000 7f31e369c640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f31c40a32d0 0x7f31c40a5790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.574+0000 7f31e369c640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc106f10 0x7f31dc19cd00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.574+0000 7f31e369c640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f31dc102790 0x7f31dc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.574+0000 7f31e369c640 1 --2- 172.21.15.139:0/3291575523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31dc101de0 0x7f31dc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:38:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.575+0000 7f31e369c640 1 -- 172.21.15.139:0/3291575523 >> 172.21.15.139:0/3291575523 conn(0x7f31dc0fd8c0 msgr2=0x7f31dc0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:38:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.575+0000 7f31e369c640 1 -- 172.21.15.139:0/3291575523 shutdown_connections 2024-09-16T01:38:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:38:59.575+0000 7f31e369c640 1 -- 172.21.15.139:0/3291575523 wait complete. 2024-09-16T01:39:00.283 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":331514141},{"type":"v1","addr":"172.21.15.139:6801","nonce":331514141}]},"active_addr":"172.21.15.139:6801/331514141","active_change":"2024-09-16T01:31:07.344462+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.139:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":2362566775}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":3167151595}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":2901696849}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":1329877785}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":1989300904}]}]} 2024-09-16T01:39:00.288 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T01:39:00.288 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T01:39:00.288 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T01:39:00.310 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:00 smithi139 ceph-mon[31175]: pgmap v309: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-16T01:39:00.310 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:00 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3291575523' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T01:39:00.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:00 smithi176 ceph-mon[35324]: pgmap v309: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-16T01:39:00.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:00 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3291575523' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T01:39:00.534 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:00.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:00 smithi164 ceph-mon[35121]: pgmap v309: 129 pgs: 129 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-16T01:39:00.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:00 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3291575523' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T01:39:01.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.027+0000 7f73cc1ed640 1 -- 172.21.15.139:0/298569110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73c4106a80 msgr2=0x7f73c4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:01.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.027+0000 7f73cc1ed640 1 --2- 172.21.15.139:0/298569110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73c4106a80 0x7f73c4106e60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009930 tx=0x7f73c002a420 comp rx=0 tx=0).stop 2024-09-16T01:39:01.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.027+0000 7f73cc1ed640 1 -- 172.21.15.139:0/298569110 shutdown_connections 2024-09-16T01:39:01.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.027+0000 7f73cc1ed640 1 --2- 172.21.15.139:0/298569110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73c4101420 0x7f73c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:01.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.027+0000 7f73cc1ed640 1 --2- 172.21.15.139:0/298569110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73c4100a80 0x7f73c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:01.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.027+0000 7f73cc1ed640 1 --2- 172.21.15.139:0/298569110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73c4106a80 0x7f73c4106e60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:01.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.027+0000 7f73cc1ed640 1 -- 172.21.15.139:0/298569110 >> 172.21.15.139:0/298569110 conn(0x7f73c40fc7a0 msgr2=0x7f73c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:01.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.027+0000 7f73cc1ed640 1 -- 172.21.15.139:0/298569110 shutdown_connections 2024-09-16T01:39:01.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.027+0000 7f73cc1ed640 1 -- 172.21.15.139:0/298569110 wait complete. 2024-09-16T01:39:01.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.028+0000 7f73cc1ed640 1 Processor -- start 2024-09-16T01:39:01.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.028+0000 7f73cc1ed640 1 -- start start 2024-09-16T01:39:01.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.028+0000 7f73cc1ed640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73c4100a80 0x7f73c41973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:01.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.028+0000 7f73cc1ed640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73c4101420 0x7f73c41978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:01.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.028+0000 7f73cc1ed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73c4106a80 0x7f73c419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:01.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.028+0000 7f73cc1ed640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73c4110e80 con 0x7f73c4100a80 2024-09-16T01:39:01.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.028+0000 7f73cc1ed640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73c4110d00 con 0x7f73c4106a80 2024-09-16T01:39:01.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.028+0000 7f73cc1ed640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f73c4111000 con 0x7f73c4101420 2024-09-16T01:39:01.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.028+0000 7f73c9f89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73c4100a80 0x7f73c41973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:01.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.028+0000 7f73c9f89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73c4100a80 0x7f73c41973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40240/0 (socket says 172.21.15.139:40240) 2024-09-16T01:39:01.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.028+0000 7f73c9f89640 1 -- 172.21.15.139:0/3824684931 learned_addr learned my addr 172.21.15.139:0/3824684931 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:01.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.028+0000 7f73ca78a640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73c4106a80 0x7f73c419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:01.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.029+0000 7f73c9788640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73c4101420 0x7f73c41978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:01.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.029+0000 7f73c9f89640 1 -- 172.21.15.139:0/3824684931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73c4101420 msgr2=0x7f73c41978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:01.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.029+0000 7f73c9f89640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73c4101420 0x7f73c41978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:01.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.029+0000 7f73c9f89640 1 -- 172.21.15.139:0/3824684931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73c4106a80 msgr2=0x7f73c419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:01.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.029+0000 7f73c9f89640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73c4106a80 0x7f73c419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:01.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.029+0000 7f73c9f89640 1 -- 172.21.15.139:0/3824684931 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c419c350 con 0x7f73c4100a80 2024-09-16T01:39:01.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.029+0000 7f73c9788640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73c4101420 0x7f73c41978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:01.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.029+0000 7f73c9f89640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73c4100a80 0x7f73c41973a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009230 tx=0x7f73c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:01.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.029+0000 7f73baffd640 1 -- 172.21.15.139:0/3824684931 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c003f030 con 0x7f73c4100a80 2024-09-16T01:39:01.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.029+0000 7f73cc1ed640 1 -- 172.21.15.139:0/3824684931 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c419c5e0 con 0x7f73c4100a80 2024-09-16T01:39:01.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.029+0000 7f73baffd640 1 -- 172.21.15.139:0/3824684931 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c0003ad0 con 0x7f73c4100a80 2024-09-16T01:39:01.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.029+0000 7f73baffd640 1 -- 172.21.15.139:0/3824684931 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0035070 con 0x7f73c4100a80 2024-09-16T01:39:01.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.029+0000 7f73cc1ed640 1 -- 172.21.15.139:0/3824684931 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c41a3f20 con 0x7f73c4100a80 2024-09-16T01:39:01.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.030+0000 7f73cc1ed640 1 -- 172.21.15.139:0/3824684931 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c41021c0 con 0x7f73c4100a80 2024-09-16T01:39:01.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.031+0000 7f73baffd640 1 -- 172.21.15.139:0/3824684931 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f73c0042030 con 0x7f73c4100a80 2024-09-16T01:39:01.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.032+0000 7f73baffd640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f73b00a32d0 0x7f73b00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:01.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.032+0000 7f73c9788640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f73b00a32d0 0x7f73b00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:01.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.032+0000 7f73baffd640 1 -- 172.21.15.139:0/3824684931 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f73c00f0730 con 0x7f73c4100a80 2024-09-16T01:39:01.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.032+0000 7f73c9788640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f73b00a32d0 0x7f73b00a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f73c40fee50 tx=0x7f73b4005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:01.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.035+0000 7f73baffd640 1 -- 172.21.15.139:0/3824684931 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73c00bcf90 con 0x7f73c4100a80 2024-09-16T01:39:01.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.203+0000 7f73cc1ed640 1 -- 172.21.15.139:0/3824684931 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73c4108ff0 con 0x7f73c4100a80 2024-09-16T01:39:01.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.205+0000 7f73baffd640 1 -- 172.21.15.139:0/3824684931 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24797 (secure 0 0 0) 0x7f73c00bc890 con 0x7f73c4100a80 2024-09-16T01:39:01.207 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:39:01.207 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":83,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","created":"2024-09-16T01:30:17.770313+0000","modified":"2024-09-16T01:38:11.211197+0000","last_up_change":"2024-09-16T01:37:08.611901+0000","last_in_change":"2024-09-16T01:36:45.223926+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:33:19.639647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T01:37:57.948214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T01:38:00.263797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T01:38:06.773799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b66696da-ef66-4f25-9527-bbcc9ded2cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6803","nonce":3190810133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6805","nonce":3190810133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6809","nonce":3190810133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6807","nonce":3190810133}]},"public_addr":"172.21.15.139:6803/3190810133","cluster_addr":"172.21.15.139:6805/3190810133","heartbeat_back_addr":"172.21.15.139:6809/3190810133","heartbeat_front_addr":"172.21.15.139:6807/3190810133","state":["exists","up"]},{"osd":1,"uuid":"ccad17f2-0750-425e-9810-49a13a929ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6801","nonce":2699270330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6803","nonce":2699270330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6807","nonce":2699270330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6805","nonce":2699270330}]},"public_addr":"172.21.15.164:6801/2699270330","cluster_addr":"172.21.15.164:6803/2699270330","heartbeat_back_addr":"172.21.15.164:6807/2699270330","heartbeat_front_addr":"172.21.15.164:6805/2699270330","state":["exists","up"]},{"osd":2,"uuid":"baf0ba73-8faf-4e51-a171-c47898800638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6801","nonce":2034471646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6803","nonce":2034471646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6807","nonce":2034471646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6805","nonce":2034471646}]},"public_addr":"172.21.15.176:6801/2034471646","cluster_addr":"172.21.15.176:6803/2034471646","heartbeat_back_addr":"172.21.15.176:6807/2034471646","heartbeat_front_addr":"172.21.15.176:6805/2034471646","state":["exists","up"]},{"osd":3,"uuid":"98957bf1-67bb-4092-9ea4-b263a5706f8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6811","nonce":797147071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6813","nonce":797147071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6817","nonce":797147071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6815","nonce":797147071}]},"public_addr":"172.21.15.139:6811/797147071","cluster_addr":"172.21.15.139:6813/797147071","heartbeat_back_addr":"172.21.15.139:6817/797147071","heartbeat_front_addr":"172.21.15.139:6815/797147071","state":["exists","up"]},{"osd":4,"uuid":"9cd76696-8161-4e82-a571-29e98c11d5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6809","nonce":1421304792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6811","nonce":1421304792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6815","nonce":1421304792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6813","nonce":1421304792}]},"public_addr":"172.21.15.164:6809/1421304792","cluster_addr":"172.21.15.164:6811/1421304792","heartbeat_back_addr":"172.21.15.164:6815/1421304792","heartbeat_front_addr":"172.21.15.164:6813/1421304792","state":["exists","up"]},{"osd":5,"uuid":"ccc17a29-9a58-40ec-b453-5c38bee86f25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6809","nonce":10600602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6811","nonce":10600602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6815","nonce":10600602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6813","nonce":10600602}]},"public_addr":"172.21.15.176:6809/10600602","cluster_addr":"172.21.15.176:6811/10600602","heartbeat_back_addr":"172.21.15.176:6815/10600602","heartbeat_front_addr":"172.21.15.176:6813/10600602","state":["exists","up"]},{"osd":6,"uuid":"e5ee7212-10c6-4a05-992e-622a14a5a6b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6819","nonce":2601111164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6821","nonce":2601111164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6825","nonce":2601111164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6823","nonce":2601111164}]},"public_addr":"172.21.15.139:6819/2601111164","cluster_addr":"172.21.15.139:6821/2601111164","heartbeat_back_addr":"172.21.15.139:6825/2601111164","heartbeat_front_addr":"172.21.15.139:6823/2601111164","state":["exists","up"]},{"osd":7,"uuid":"23143c15-0839-4fd5-9d0a-19ae891000f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6817","nonce":2074080693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6819","nonce":2074080693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6823","nonce":2074080693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6821","nonce":2074080693}]},"public_addr":"172.21.15.164:6817/2074080693","cluster_addr":"172.21.15.164:6819/2074080693","heartbeat_back_addr":"172.21.15.164:6823/2074080693","heartbeat_front_addr":"172.21.15.164:6821/2074080693","state":["exists","up"]},{"osd":8,"uuid":"1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6817","nonce":593133361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6819","nonce":593133361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6823","nonce":593133361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6821","nonce":593133361}]},"public_addr":"172.21.15.176:6817/593133361","cluster_addr":"172.21.15.176:6819/593133361","heartbeat_back_addr":"172.21.15.176:6823/593133361","heartbeat_front_addr":"172.21.15.176:6821/593133361","state":["exists","up"]},{"osd":9,"uuid":"09d25a3b-1e66-480e-9433-ceef7f0b9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6827","nonce":3047942108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6829","nonce":3047942108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6833","nonce":3047942108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6831","nonce":3047942108}]},"public_addr":"172.21.15.139:6827/3047942108","cluster_addr":"172.21.15.139:6829/3047942108","heartbeat_back_addr":"172.21.15.139:6833/3047942108","heartbeat_front_addr":"172.21.15.139:6831/3047942108","state":["exists","up"]},{"osd":10,"uuid":"32b4d8ce-b644-42fa-80cc-a15a1f5f3a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6825","nonce":4172533828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6827","nonce":4172533828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6831","nonce":4172533828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6829","nonce":4172533828}]},"public_addr":"172.21.15.164:6825/4172533828","cluster_addr":"172.21.15.164:6827/4172533828","heartbeat_back_addr":"172.21.15.164:6831/4172533828","heartbeat_front_addr":"172.21.15.164:6829/4172533828","state":["exists","up"]},{"osd":11,"uuid":"49098bdc-58a9-4a74-8045-d408af7711ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6825","nonce":1854547565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6827","nonce":1854547565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6831","nonce":1854547565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6829","nonce":1854547565}]},"public_addr":"172.21.15.176:6825/1854547565","cluster_addr":"172.21.15.176:6827/1854547565","heartbeat_back_addr":"172.21.15.176:6831/1854547565","heartbeat_front_addr":"172.21.15.176:6829/1854547565","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:27.374845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:52.611853+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:17.229567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:43.330503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:07.988876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:33.787864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:58.864910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:23.815006+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:49.551954+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:15.496773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:41.647557+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:37:07.089355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3772232169":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/3997627612":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/1608930262":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6801/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/677216867":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6801/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/1830135963":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/2608207648":"2024-09-17T01:30:44.751168+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:39:01.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.207+0000 7f73cc1ed640 1 -- 172.21.15.139:0/3824684931 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f73b00a32d0 msgr2=0x7f73b00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:01.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.207+0000 7f73cc1ed640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f73b00a32d0 0x7f73b00a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f73c40fee50 tx=0x7f73b4005d30 comp rx=0 tx=0).stop 2024-09-16T01:39:01.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.207+0000 7f73cc1ed640 1 -- 172.21.15.139:0/3824684931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73c4100a80 msgr2=0x7f73c41973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:01.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.207+0000 7f73cc1ed640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73c4100a80 0x7f73c41973a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009230 tx=0x7f73c0004270 comp rx=0 tx=0).stop 2024-09-16T01:39:01.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.207+0000 7f73cc1ed640 1 -- 172.21.15.139:0/3824684931 shutdown_connections 2024-09-16T01:39:01.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.207+0000 7f73cc1ed640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f73b00a32d0 0x7f73b00a5790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:01.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.207+0000 7f73cc1ed640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73c4106a80 0x7f73c419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:01.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.207+0000 7f73cc1ed640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73c4101420 0x7f73c41978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:01.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.207+0000 7f73cc1ed640 1 --2- 172.21.15.139:0/3824684931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73c4100a80 0x7f73c41973a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:01.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.207+0000 7f73cc1ed640 1 -- 172.21.15.139:0/3824684931 >> 172.21.15.139:0/3824684931 conn(0x7f73c40fc7a0 msgr2=0x7f73c40fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:01.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.207+0000 7f73cc1ed640 1 -- 172.21.15.139:0/3824684931 shutdown_connections 2024-09-16T01:39:01.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:01.207+0000 7f73cc1ed640 1 -- 172.21.15.139:0/3824684931 wait complete. 2024-09-16T01:39:01.528 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:01 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3824684931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:39:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:01 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3824684931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:39:01.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:01 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3824684931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:39:01.849 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T01:39:01.850 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T01:39:02.133 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:02.526 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:02 smithi139 ceph-mon[31175]: pgmap v310: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-16T01:39:02.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:02 smithi164 ceph-mon[35121]: pgmap v310: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-16T01:39:02.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.604+0000 7f34f8282640 1 -- 172.21.15.139:0/2529355275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f0100a80 msgr2=0x7f34f0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:02.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.604+0000 7f34f8282640 1 --2- 172.21.15.139:0/2529355275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f0100a80 0x7f34f0100ee0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009930 tx=0x7f34e002a420 comp rx=0 tx=0).stop 2024-09-16T01:39:02.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.604+0000 7f34f8282640 1 -- 172.21.15.139:0/2529355275 shutdown_connections 2024-09-16T01:39:02.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.604+0000 7f34f8282640 1 --2- 172.21.15.139:0/2529355275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34f0101420 0x7f34f010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:02.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.604+0000 7f34f8282640 1 --2- 172.21.15.139:0/2529355275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f0100a80 0x7f34f0100ee0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.604+0000 7f34f8282640 1 --2- 172.21.15.139:0/2529355275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34f0106a80 0x7f34f0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.604+0000 7f34f8282640 1 -- 172.21.15.139:0/2529355275 >> 172.21.15.139:0/2529355275 conn(0x7f34f00fc7a0 msgr2=0x7f34f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.604+0000 7f34f8282640 1 -- 172.21.15.139:0/2529355275 shutdown_connections 2024-09-16T01:39:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.604+0000 7f34f8282640 1 -- 172.21.15.139:0/2529355275 wait complete. 2024-09-16T01:39:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.605+0000 7f34f8282640 1 Processor -- start 2024-09-16T01:39:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.605+0000 7f34f8282640 1 -- start start 2024-09-16T01:39:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.605+0000 7f34f8282640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34f0100a80 0x7f34f019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.605+0000 7f34f8282640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f0101420 0x7f34f019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.605+0000 7f34f8282640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34f0106a80 0x7f34f019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.605+0000 7f34f8282640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f34f0110e80 con 0x7f34f0101420 2024-09-16T01:39:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.605+0000 7f34f8282640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f34f0110d00 con 0x7f34f0106a80 2024-09-16T01:39:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.605+0000 7f34f8282640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f34f0111000 con 0x7f34f0100a80 2024-09-16T01:39:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.605+0000 7f34f581d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f0101420 0x7f34f019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.606+0000 7f34f581d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f0101420 0x7f34f019abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43094/0 (socket says 172.21.15.139:43094) 2024-09-16T01:39:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.606+0000 7f34f581d640 1 -- 172.21.15.139:0/1731095346 learned_addr learned my addr 172.21.15.139:0/1731095346 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.606+0000 7f34f601e640 1 --2- 172.21.15.139:0/1731095346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34f0100a80 0x7f34f019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.606+0000 7f34f581d640 1 -- 172.21.15.139:0/1731095346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34f0100a80 msgr2=0x7f34f019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.606+0000 7f34f581d640 1 --2- 172.21.15.139:0/1731095346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34f0100a80 0x7f34f019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.606+0000 7f34f581d640 1 -- 172.21.15.139:0/1731095346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34f0106a80 msgr2=0x7f34f019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.606+0000 7f34f581d640 1 --2- 172.21.15.139:0/1731095346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34f0106a80 0x7f34f019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.606+0000 7f34f581d640 1 -- 172.21.15.139:0/1731095346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f019f630 con 0x7f34f0101420 2024-09-16T01:39:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.606+0000 7f34f581d640 1 --2- 172.21.15.139:0/1731095346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f0101420 0x7f34f019abc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009230 tx=0x7f34e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.606+0000 7f34e77fe640 1 -- 172.21.15.139:0/1731095346 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e003f030 con 0x7f34f0101420 2024-09-16T01:39:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.606+0000 7f34f8282640 1 -- 172.21.15.139:0/1731095346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f019f8c0 con 0x7f34f0101420 2024-09-16T01:39:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.606+0000 7f34e77fe640 1 -- 172.21.15.139:0/1731095346 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34e0003ad0 con 0x7f34f0101420 2024-09-16T01:39:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.606+0000 7f34e77fe640 1 -- 172.21.15.139:0/1731095346 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0035070 con 0x7f34f0101420 2024-09-16T01:39:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.606+0000 7f34f8282640 1 -- 172.21.15.139:0/1731095346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f01a7200 con 0x7f34f0101420 2024-09-16T01:39:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.607+0000 7f34f8282640 1 -- 172.21.15.139:0/1731095346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f01021c0 con 0x7f34f0101420 2024-09-16T01:39:02.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.608+0000 7f34e77fe640 1 -- 172.21.15.139:0/1731095346 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f34e0042030 con 0x7f34f0101420 2024-09-16T01:39:02.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.609+0000 7f34e77fe640 1 --2- 172.21.15.139:0/1731095346 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f34dc0a32d0 0x7f34dc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:02.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.609+0000 7f34f601e640 1 --2- 172.21.15.139:0/1731095346 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f34dc0a32d0 0x7f34dc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:02.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.609+0000 7f34e77fe640 1 -- 172.21.15.139:0/1731095346 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f34e00f0530 con 0x7f34f0101420 2024-09-16T01:39:02.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.609+0000 7f34f601e640 1 --2- 172.21.15.139:0/1731095346 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f34dc0a32d0 0x7f34dc0a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009ae0 tx=0x7f34ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:02.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.613+0000 7f34e77fe640 1 -- 172.21.15.139:0/1731095346 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34e00bcd90 con 0x7f34f0101420 2024-09-16T01:39:02.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:02 smithi176 ceph-mon[35324]: pgmap v310: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-16T01:39:02.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.781+0000 7f34f8282640 1 -- 172.21.15.139:0/1731095346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34f01a7460 con 0x7f34f0101420 2024-09-16T01:39:02.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.782+0000 7f34e77fe640 1 -- 172.21.15.139:0/1731095346 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24797 (secure 0 0 0) 0x7f34e00bc690 con 0x7f34f0101420 2024-09-16T01:39:02.783 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:39:02.784 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":83,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","created":"2024-09-16T01:30:17.770313+0000","modified":"2024-09-16T01:38:11.211197+0000","last_up_change":"2024-09-16T01:37:08.611901+0000","last_in_change":"2024-09-16T01:36:45.223926+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:33:19.639647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T01:37:57.948214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T01:38:00.263797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T01:38:06.773799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b66696da-ef66-4f25-9527-bbcc9ded2cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6803","nonce":3190810133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6805","nonce":3190810133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6809","nonce":3190810133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6807","nonce":3190810133}]},"public_addr":"172.21.15.139:6803/3190810133","cluster_addr":"172.21.15.139:6805/3190810133","heartbeat_back_addr":"172.21.15.139:6809/3190810133","heartbeat_front_addr":"172.21.15.139:6807/3190810133","state":["exists","up"]},{"osd":1,"uuid":"ccad17f2-0750-425e-9810-49a13a929ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6801","nonce":2699270330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6803","nonce":2699270330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6807","nonce":2699270330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6805","nonce":2699270330}]},"public_addr":"172.21.15.164:6801/2699270330","cluster_addr":"172.21.15.164:6803/2699270330","heartbeat_back_addr":"172.21.15.164:6807/2699270330","heartbeat_front_addr":"172.21.15.164:6805/2699270330","state":["exists","up"]},{"osd":2,"uuid":"baf0ba73-8faf-4e51-a171-c47898800638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6801","nonce":2034471646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6803","nonce":2034471646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6807","nonce":2034471646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6805","nonce":2034471646}]},"public_addr":"172.21.15.176:6801/2034471646","cluster_addr":"172.21.15.176:6803/2034471646","heartbeat_back_addr":"172.21.15.176:6807/2034471646","heartbeat_front_addr":"172.21.15.176:6805/2034471646","state":["exists","up"]},{"osd":3,"uuid":"98957bf1-67bb-4092-9ea4-b263a5706f8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6811","nonce":797147071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6813","nonce":797147071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6817","nonce":797147071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6815","nonce":797147071}]},"public_addr":"172.21.15.139:6811/797147071","cluster_addr":"172.21.15.139:6813/797147071","heartbeat_back_addr":"172.21.15.139:6817/797147071","heartbeat_front_addr":"172.21.15.139:6815/797147071","state":["exists","up"]},{"osd":4,"uuid":"9cd76696-8161-4e82-a571-29e98c11d5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6809","nonce":1421304792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6811","nonce":1421304792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6815","nonce":1421304792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6813","nonce":1421304792}]},"public_addr":"172.21.15.164:6809/1421304792","cluster_addr":"172.21.15.164:6811/1421304792","heartbeat_back_addr":"172.21.15.164:6815/1421304792","heartbeat_front_addr":"172.21.15.164:6813/1421304792","state":["exists","up"]},{"osd":5,"uuid":"ccc17a29-9a58-40ec-b453-5c38bee86f25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6809","nonce":10600602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6811","nonce":10600602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6815","nonce":10600602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6813","nonce":10600602}]},"public_addr":"172.21.15.176:6809/10600602","cluster_addr":"172.21.15.176:6811/10600602","heartbeat_back_addr":"172.21.15.176:6815/10600602","heartbeat_front_addr":"172.21.15.176:6813/10600602","state":["exists","up"]},{"osd":6,"uuid":"e5ee7212-10c6-4a05-992e-622a14a5a6b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6819","nonce":2601111164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6821","nonce":2601111164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6825","nonce":2601111164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6823","nonce":2601111164}]},"public_addr":"172.21.15.139:6819/2601111164","cluster_addr":"172.21.15.139:6821/2601111164","heartbeat_back_addr":"172.21.15.139:6825/2601111164","heartbeat_front_addr":"172.21.15.139:6823/2601111164","state":["exists","up"]},{"osd":7,"uuid":"23143c15-0839-4fd5-9d0a-19ae891000f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6817","nonce":2074080693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6819","nonce":2074080693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6823","nonce":2074080693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6821","nonce":2074080693}]},"public_addr":"172.21.15.164:6817/2074080693","cluster_addr":"172.21.15.164:6819/2074080693","heartbeat_back_addr":"172.21.15.164:6823/2074080693","heartbeat_front_addr":"172.21.15.164:6821/2074080693","state":["exists","up"]},{"osd":8,"uuid":"1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6817","nonce":593133361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6819","nonce":593133361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6823","nonce":593133361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6821","nonce":593133361}]},"public_addr":"172.21.15.176:6817/593133361","cluster_addr":"172.21.15.176:6819/593133361","heartbeat_back_addr":"172.21.15.176:6823/593133361","heartbeat_front_addr":"172.21.15.176:6821/593133361","state":["exists","up"]},{"osd":9,"uuid":"09d25a3b-1e66-480e-9433-ceef7f0b9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6827","nonce":3047942108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6829","nonce":3047942108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6833","nonce":3047942108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6831","nonce":3047942108}]},"public_addr":"172.21.15.139:6827/3047942108","cluster_addr":"172.21.15.139:6829/3047942108","heartbeat_back_addr":"172.21.15.139:6833/3047942108","heartbeat_front_addr":"172.21.15.139:6831/3047942108","state":["exists","up"]},{"osd":10,"uuid":"32b4d8ce-b644-42fa-80cc-a15a1f5f3a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6825","nonce":4172533828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6827","nonce":4172533828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6831","nonce":4172533828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6829","nonce":4172533828}]},"public_addr":"172.21.15.164:6825/4172533828","cluster_addr":"172.21.15.164:6827/4172533828","heartbeat_back_addr":"172.21.15.164:6831/4172533828","heartbeat_front_addr":"172.21.15.164:6829/4172533828","state":["exists","up"]},{"osd":11,"uuid":"49098bdc-58a9-4a74-8045-d408af7711ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6825","nonce":1854547565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6827","nonce":1854547565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6831","nonce":1854547565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6829","nonce":1854547565}]},"public_addr":"172.21.15.176:6825/1854547565","cluster_addr":"172.21.15.176:6827/1854547565","heartbeat_back_addr":"172.21.15.176:6831/1854547565","heartbeat_front_addr":"172.21.15.176:6829/1854547565","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:27.374845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:52.611853+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:17.229567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:43.330503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:07.988876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:33.787864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:58.864910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:23.815006+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:49.551954+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:15.496773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:41.647557+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:37:07.089355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3772232169":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/3997627612":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/1608930262":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6801/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/677216867":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6801/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/1830135963":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/2608207648":"2024-09-17T01:30:44.751168+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:39:02.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.785+0000 7f34f8282640 1 -- 172.21.15.139:0/1731095346 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f34dc0a32d0 msgr2=0x7f34dc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:02.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.785+0000 7f34f8282640 1 --2- 172.21.15.139:0/1731095346 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f34dc0a32d0 0x7f34dc0a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009ae0 tx=0x7f34ec005f90 comp rx=0 tx=0).stop 2024-09-16T01:39:02.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.785+0000 7f34f8282640 1 -- 172.21.15.139:0/1731095346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f0101420 msgr2=0x7f34f019abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:02.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.785+0000 7f34f8282640 1 --2- 172.21.15.139:0/1731095346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f0101420 0x7f34f019abc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009230 tx=0x7f34e0004270 comp rx=0 tx=0).stop 2024-09-16T01:39:02.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.785+0000 7f34f8282640 1 -- 172.21.15.139:0/1731095346 shutdown_connections 2024-09-16T01:39:02.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.785+0000 7f34f8282640 1 --2- 172.21.15.139:0/1731095346 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f34dc0a32d0 0x7f34dc0a5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:02.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.785+0000 7f34f8282640 1 --2- 172.21.15.139:0/1731095346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34f0106a80 0x7f34f019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:02.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.785+0000 7f34f8282640 1 --2- 172.21.15.139:0/1731095346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f0101420 0x7f34f019abc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:02.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.785+0000 7f34f8282640 1 --2- 172.21.15.139:0/1731095346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34f0100a80 0x7f34f019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:02.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.785+0000 7f34f8282640 1 -- 172.21.15.139:0/1731095346 >> 172.21.15.139:0/1731095346 conn(0x7f34f00fc7a0 msgr2=0x7f34f006a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:02.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.785+0000 7f34f8282640 1 -- 172.21.15.139:0/1731095346 shutdown_connections 2024-09-16T01:39:02.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:02.785+0000 7f34f8282640 1 -- 172.21.15.139:0/1731095346 wait complete. 2024-09-16T01:39:03.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:03 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1731095346' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:39:03.450 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T01:39:03.451 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T01:39:03.451 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T01:39:03.452 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T01:39:03.452 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T01:39:03.452 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T01:39:03.453 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-16T01:39:03.453 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-16T01:39:03.454 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-16T01:39:03.455 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-16T01:39:03.455 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-16T01:39:03.455 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-16T01:39:03.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:03 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1731095346' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:39:03.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:03 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1731095346' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:39:04.171 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:04.185 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:04.241 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:04.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:04 smithi164 ceph-mon[35121]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-16T01:39:04.574 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:04 smithi139 ceph-mon[31175]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-16T01:39:04.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:04 smithi176 ceph-mon[35324]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-16T01:39:04.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.720+0000 7f0055d23640 1 -- 172.21.15.139:0/1382385150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0050102ef0 msgr2=0x7f005010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:04.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.720+0000 7f0055d23640 1 --2- 172.21.15.139:0/1382385150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0050102ef0 0x7f005010da40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0048009ca0 tx=0x7f004802a950 comp rx=0 tx=0).stop 2024-09-16T01:39:04.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.720+0000 7f0055d23640 1 -- 172.21.15.139:0/1382385150 shutdown_connections 2024-09-16T01:39:04.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.720+0000 7f0055d23640 1 --2- 172.21.15.139:0/1382385150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0050102ef0 0x7f005010da40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:04.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.720+0000 7f0055d23640 1 --2- 172.21.15.139:0/1382385150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0050102570 0x7f00501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:04.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.720+0000 7f0055d23640 1 --2- 172.21.15.139:0/1382385150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0050100e20 0x7f0050101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:04.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.720+0000 7f0055d23640 1 -- 172.21.15.139:0/1382385150 >> 172.21.15.139:0/1382385150 conn(0x7f00500fc7d0 msgr2=0x7f00500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:04.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.720+0000 7f0055d23640 1 -- 172.21.15.139:0/1382385150 shutdown_connections 2024-09-16T01:39:04.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.720+0000 7f0055d23640 1 -- 172.21.15.139:0/1382385150 wait complete. 2024-09-16T01:39:04.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.720+0000 7f0055d23640 1 Processor -- start 2024-09-16T01:39:04.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f0055d23640 1 -- start start 2024-09-16T01:39:04.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f0055d23640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0050100e20 0x7f00501984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:04.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f0055d23640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0050102570 0x7f0050198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:04.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f0055d23640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0050102ef0 0x7f005019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:04.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f0055d23640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0050110f80 con 0x7f0050102ef0 2024-09-16T01:39:04.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f0055d23640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0050110e00 con 0x7f0050102570 2024-09-16T01:39:04.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f0055d23640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0050111100 con 0x7f0050100e20 2024-09-16T01:39:04.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f004ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0050102ef0 0x7f005019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:04.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f004ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0050102ef0 0x7f005019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43112/0 (socket says 172.21.15.139:43112) 2024-09-16T01:39:04.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f004ffff640 1 -- 172.21.15.139:0/278630437 learned_addr learned my addr 172.21.15.139:0/278630437 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:04.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f004f7fe640 1 --2- 172.21.15.139:0/278630437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0050100e20 0x7f00501984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:04.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f004effd640 1 --2- 172.21.15.139:0/278630437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0050102570 0x7f0050198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:04.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f004ffff640 1 -- 172.21.15.139:0/278630437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0050100e20 msgr2=0x7f00501984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:04.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f004ffff640 1 --2- 172.21.15.139:0/278630437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0050100e20 0x7f00501984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:04.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f004ffff640 1 -- 172.21.15.139:0/278630437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0050102570 msgr2=0x7f0050198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:04.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f004ffff640 1 --2- 172.21.15.139:0/278630437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0050102570 0x7f0050198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:04.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f004ffff640 1 -- 172.21.15.139:0/278630437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005019d480 con 0x7f0050102ef0 2024-09-16T01:39:04.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.721+0000 7f004ffff640 1 --2- 172.21.15.139:0/278630437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0050102ef0 0x7f005019cda0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f004802ae70 tx=0x7f004802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:04.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.722+0000 7f004cff9640 1 -- 172.21.15.139:0/278630437 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048034040 con 0x7f0050102ef0 2024-09-16T01:39:04.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.722+0000 7f0055d23640 1 -- 172.21.15.139:0/278630437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005019d710 con 0x7f0050102ef0 2024-09-16T01:39:04.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.722+0000 7f004cff9640 1 -- 172.21.15.139:0/278630437 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0048030920 con 0x7f0050102ef0 2024-09-16T01:39:04.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.722+0000 7f004cff9640 1 -- 172.21.15.139:0/278630437 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048045480 con 0x7f0050102ef0 2024-09-16T01:39:04.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.722+0000 7f0055d23640 1 -- 172.21.15.139:0/278630437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00501a5050 con 0x7f0050102ef0 2024-09-16T01:39:04.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.723+0000 7f0055d23640 1 -- 172.21.15.139:0/278630437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0050062980 con 0x7f0050102ef0 2024-09-16T01:39:04.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.724+0000 7f004cff9640 1 -- 172.21.15.139:0/278630437 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0048037020 con 0x7f0050102ef0 2024-09-16T01:39:04.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.725+0000 7f004cff9640 1 --2- 172.21.15.139:0/278630437 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f003c0a32d0 0x7f003c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:04.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.725+0000 7f004f7fe640 1 --2- 172.21.15.139:0/278630437 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f003c0a32d0 0x7f003c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:04.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.725+0000 7f004cff9640 1 -- 172.21.15.139:0/278630437 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f00480f0f20 con 0x7f0050102ef0 2024-09-16T01:39:04.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.725+0000 7f004cff9640 1 --2- 172.21.15.139:0/278630437 >> [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693] conn(0x7f003c0ae4a0 0x7f003c0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:04.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.725+0000 7f004cff9640 1 -- 172.21.15.139:0/278630437 --> [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f004802c030 con 0x7f003c0ae4a0 2024-09-16T01:39:04.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.725+0000 7f004cff9640 1 -- 172.21.15.139:0/278630437 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f00480f1370 con 0x7f0050102ef0 2024-09-16T01:39:04.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.725+0000 7f004f7fe640 1 --2- 172.21.15.139:0/278630437 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f003c0a32d0 0x7f003c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0038005fd0 tx=0x7f003800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:04.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.725+0000 7f004effd640 1 --2- 172.21.15.139:0/278630437 >> [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693] conn(0x7f003c0ae4a0 0x7f003c0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:04.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.726+0000 7f004effd640 1 --2- 172.21.15.139:0/278630437 >> [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693] conn(0x7f003c0ae4a0 0x7f003c0b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:04.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.727+0000 7f004cff9640 1 -- 172.21.15.139:0/278630437 <== osd.7 v2:172.21.15.164:6816/2074080693 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f004802c030 con 0x7f003c0ae4a0 2024-09-16T01:39:04.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.740+0000 7f0055d23640 1 -- 172.21.15.139:0/278630437 --> [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0050062e00 con 0x7f003c0ae4a0 2024-09-16T01:39:04.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.741+0000 7f004cff9640 1 -- 172.21.15.139:0/278630437 <== osd.7 v2:172.21.15.164:6816/2074080693 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0050062e00 con 0x7f003c0ae4a0 2024-09-16T01:39:04.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.741+0000 7f0055d23640 1 -- 172.21.15.139:0/278630437 >> [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693] conn(0x7f003c0ae4a0 msgr2=0x7f003c0b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:04.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.741+0000 7f0055d23640 1 --2- 172.21.15.139:0/278630437 >> [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693] conn(0x7f003c0ae4a0 0x7f003c0b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:04.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.741+0000 7f0055d23640 1 -- 172.21.15.139:0/278630437 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f003c0a32d0 msgr2=0x7f003c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:04.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.741+0000 7f0055d23640 1 --2- 172.21.15.139:0/278630437 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f003c0a32d0 0x7f003c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0038005fd0 tx=0x7f003800b390 comp rx=0 tx=0).stop 2024-09-16T01:39:04.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.741+0000 7f0055d23640 1 -- 172.21.15.139:0/278630437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0050102ef0 msgr2=0x7f005019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:04.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.741+0000 7f0055d23640 1 --2- 172.21.15.139:0/278630437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0050102ef0 0x7f005019cda0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f004802ae70 tx=0x7f004802f5d0 comp rx=0 tx=0).stop 2024-09-16T01:39:04.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.741+0000 7f004ffff640 1 -- 172.21.15.139:0/278630437 reap_dead start 2024-09-16T01:39:04.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.741+0000 7f0055d23640 1 -- 172.21.15.139:0/278630437 shutdown_connections 2024-09-16T01:39:04.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.741+0000 7f0055d23640 1 -- 172.21.15.139:0/278630437 >> 172.21.15.139:0/278630437 conn(0x7f00500fc7d0 msgr2=0x7f005006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:04.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.741+0000 7f0055d23640 1 -- 172.21.15.139:0/278630437 shutdown_connections 2024-09-16T01:39:04.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:04.741+0000 7f0055d23640 1 -- 172.21.15.139:0/278630437 wait complete. 2024-09-16T01:39:05.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.716+0000 7f65ab0a3640 1 -- 172.21.15.139:0/1747667829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65a4102570 msgr2=0x7f65a41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:05.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.716+0000 7f65ab0a3640 1 --2- 172.21.15.139:0/1747667829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65a4102570 0x7f65a41029b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6590009900 tx=0x7f659002a480 comp rx=0 tx=0).stop 2024-09-16T01:39:05.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.716+0000 7f65ab0a3640 1 -- 172.21.15.139:0/1747667829 shutdown_connections 2024-09-16T01:39:05.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.716+0000 7f65ab0a3640 1 --2- 172.21.15.139:0/1747667829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a4102ef0 0x7f65a410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:05.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.716+0000 7f65ab0a3640 1 --2- 172.21.15.139:0/1747667829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65a4102570 0x7f65a41029b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:05.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.716+0000 7f65ab0a3640 1 --2- 172.21.15.139:0/1747667829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65a4100e20 0x7f65a4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:05.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.716+0000 7f65ab0a3640 1 -- 172.21.15.139:0/1747667829 >> 172.21.15.139:0/1747667829 conn(0x7f65a40fc7d0 msgr2=0x7f65a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:05.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.716+0000 7f65ab0a3640 1 -- 172.21.15.139:0/1747667829 shutdown_connections 2024-09-16T01:39:05.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.716+0000 7f65ab0a3640 1 -- 172.21.15.139:0/1747667829 wait complete. 2024-09-16T01:39:05.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65ab0a3640 1 Processor -- start 2024-09-16T01:39:05.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65ab0a3640 1 -- start start 2024-09-16T01:39:05.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65ab0a3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a4100e20 0x7f65a4199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:05.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65ab0a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65a4102570 0x7f65a4199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:05.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65ab0a3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65a4102ef0 0x7f65a41a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:05.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65ab0a3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4110fa0 con 0x7f65a4102570 2024-09-16T01:39:05.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65ab0a3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4110e20 con 0x7f65a4100e20 2024-09-16T01:39:05.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65ab0a3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4111120 con 0x7f65a4102ef0 2024-09-16T01:39:05.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65a3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65a4102570 0x7f65a4199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:05.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65a3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65a4102570 0x7f65a4199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43138/0 (socket says 172.21.15.139:43138) 2024-09-16T01:39:05.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65a3fff640 1 -- 172.21.15.139:0/3957342044 learned_addr learned my addr 172.21.15.139:0/3957342044 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:05.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65a9640640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65a4102ef0 0x7f65a41a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:05.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65a8e3f640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a4100e20 0x7f65a4199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:05.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65a3fff640 1 -- 172.21.15.139:0/3957342044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65a4102ef0 msgr2=0x7f65a41a44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:05.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65a3fff640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65a4102ef0 0x7f65a41a44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:05.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65a3fff640 1 -- 172.21.15.139:0/3957342044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a4100e20 msgr2=0x7f65a4199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:05.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.717+0000 7f65a3fff640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a4100e20 0x7f65a4199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:05.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.718+0000 7f65a3fff640 1 -- 172.21.15.139:0/3957342044 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a41a49f0 con 0x7f65a4102570 2024-09-16T01:39:05.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.718+0000 7f65a8e3f640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65a4100e20 0x7f65a4199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:05.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.718+0000 7f65a3fff640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65a4102570 0x7f65a4199970 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6590006010 tx=0x7f6590003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:05.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.718+0000 7f65a1ffb640 1 -- 172.21.15.139:0/3957342044 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659003f070 con 0x7f65a4102570 2024-09-16T01:39:05.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.718+0000 7f65a1ffb640 1 -- 172.21.15.139:0/3957342044 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6590031c30 con 0x7f65a4102570 2024-09-16T01:39:05.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.718+0000 7f65ab0a3640 1 -- 172.21.15.139:0/3957342044 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a4197220 con 0x7f65a4102570 2024-09-16T01:39:05.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.718+0000 7f65a1ffb640 1 -- 172.21.15.139:0/3957342044 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590035030 con 0x7f65a4102570 2024-09-16T01:39:05.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.718+0000 7f65ab0a3640 1 -- 172.21.15.139:0/3957342044 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a4197700 con 0x7f65a4102570 2024-09-16T01:39:05.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.719+0000 7f65ab0a3640 1 -- 172.21.15.139:0/3957342044 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65a4062900 con 0x7f65a4102570 2024-09-16T01:39:05.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.720+0000 7f65a1ffb640 1 -- 172.21.15.139:0/3957342044 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6590042070 con 0x7f65a4102570 2024-09-16T01:39:05.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.721+0000 7f65a1ffb640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f65940a32d0 0x7f65940a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:05.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.721+0000 7f65a8e3f640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f65940a32d0 0x7f65940a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:05.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.721+0000 7f65a1ffb640 1 -- 172.21.15.139:0/3957342044 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f65900f0e90 con 0x7f65a4102570 2024-09-16T01:39:05.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.721+0000 7f65a1ffb640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646] conn(0x7f65940ae4a0 0x7f65940b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:05.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.721+0000 7f65a1ffb640 1 -- 172.21.15.139:0/3957342044 --> [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6590037ec0 con 0x7f65940ae4a0 2024-09-16T01:39:05.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.721+0000 7f65a1ffb640 1 -- 172.21.15.139:0/3957342044 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f65900f12e0 con 0x7f65a4102570 2024-09-16T01:39:05.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.722+0000 7f65a8e3f640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f65940a32d0 0x7f65940a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6598009930 tx=0x7f6598008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:05.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.722+0000 7f65a9640640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646] conn(0x7f65940ae4a0 0x7f65940b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:05.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.722+0000 7f65a9640640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646] conn(0x7f65940ae4a0 0x7f65940b0940 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:05.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.723+0000 7f65a1ffb640 1 -- 172.21.15.139:0/3957342044 <== osd.2 v2:172.21.15.176:6800/2034471646 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6590037ec0 con 0x7f65940ae4a0 2024-09-16T01:39:05.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.736+0000 7f65ab0a3640 1 -- 172.21.15.139:0/3957342044 --> [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f65a4062d80 con 0x7f65940ae4a0 2024-09-16T01:39:05.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.737+0000 7f65a1ffb640 1 -- 172.21.15.139:0/3957342044 <== osd.2 v2:172.21.15.176:6800/2034471646 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f65a4062d80 con 0x7f65940ae4a0 2024-09-16T01:39:05.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.737+0000 7f65ab0a3640 1 -- 172.21.15.139:0/3957342044 >> [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646] conn(0x7f65940ae4a0 msgr2=0x7f65940b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:05.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.737+0000 7f65ab0a3640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646] conn(0x7f65940ae4a0 0x7f65940b0940 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:05.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.737+0000 7f65ab0a3640 1 -- 172.21.15.139:0/3957342044 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f65940a32d0 msgr2=0x7f65940a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:05.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.737+0000 7f65ab0a3640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f65940a32d0 0x7f65940a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6598009930 tx=0x7f6598008040 comp rx=0 tx=0).stop 2024-09-16T01:39:05.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.737+0000 7f65ab0a3640 1 -- 172.21.15.139:0/3957342044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65a4102570 msgr2=0x7f65a4199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:05.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.737+0000 7f65ab0a3640 1 --2- 172.21.15.139:0/3957342044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65a4102570 0x7f65a4199970 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6590006010 tx=0x7f6590003c10 comp rx=0 tx=0).stop 2024-09-16T01:39:05.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.737+0000 7f65a9640640 1 -- 172.21.15.139:0/3957342044 reap_dead start 2024-09-16T01:39:05.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.737+0000 7f65ab0a3640 1 -- 172.21.15.139:0/3957342044 shutdown_connections 2024-09-16T01:39:05.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.737+0000 7f65ab0a3640 1 -- 172.21.15.139:0/3957342044 >> 172.21.15.139:0/3957342044 conn(0x7f65a40fc7d0 msgr2=0x7f65a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:05.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.737+0000 7f65ab0a3640 1 -- 172.21.15.139:0/3957342044 shutdown_connections 2024-09-16T01:39:05.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:05.737+0000 7f65ab0a3640 1 -- 172.21.15.139:0/3957342044 wait complete. 2024-09-16T01:39:06.183 INFO:teuthology.orchestra.run.smithi139.stdout:193273528366 2024-09-16T01:39:06.184 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-16T01:39:06.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/2971820272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d18102570 msgr2=0x7f8d181029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:06.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 --2- 172.21.15.139:0/2971820272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d18102570 0x7f8d181029b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009900 tx=0x7f8d0802a480 comp rx=0 tx=0).stop 2024-09-16T01:39:06.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/2971820272 shutdown_connections 2024-09-16T01:39:06.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 --2- 172.21.15.139:0/2971820272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d18102ef0 0x7f8d1810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:06.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 --2- 172.21.15.139:0/2971820272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d18102570 0x7f8d181029b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:06.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 --2- 172.21.15.139:0/2971820272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d18100e20 0x7f8d18101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:06.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/2971820272 >> 172.21.15.139:0/2971820272 conn(0x7f8d180fc7d0 msgr2=0x7f8d180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:06.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/2971820272 shutdown_connections 2024-09-16T01:39:06.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/2971820272 wait complete. 2024-09-16T01:39:06.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 Processor -- start 2024-09-16T01:39:06.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 -- start start 2024-09-16T01:39:06.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d18100e20 0x7f8d180fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:06.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d18102570 0x7f8d180fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:06.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d18102ef0 0x7f8d180fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:06.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8d18110f80 con 0x7f8d18102570 2024-09-16T01:39:06.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8d18110e00 con 0x7f8d18102ef0 2024-09-16T01:39:06.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.463+0000 7f8d1ef5f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8d18111100 con 0x7f8d18100e20 2024-09-16T01:39:06.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d0ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d18102570 0x7f8d180fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:06.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d0ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d18102570 0x7f8d180fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43166/0 (socket says 172.21.15.139:43166) 2024-09-16T01:39:06.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d0ffff640 1 -- 172.21.15.139:0/1758000432 learned_addr learned my addr 172.21.15.139:0/1758000432 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:06.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d1d4fc640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d18102ef0 0x7f8d180fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:06.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d0ffff640 1 -- 172.21.15.139:0/1758000432 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d18100e20 msgr2=0x7f8d180fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:06.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d1ccfb640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d18100e20 0x7f8d180fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:06.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d0ffff640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d18100e20 0x7f8d180fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:06.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d0ffff640 1 -- 172.21.15.139:0/1758000432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d18102ef0 msgr2=0x7f8d180fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:06.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d0ffff640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d18102ef0 0x7f8d180fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:06.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d0ffff640 1 -- 172.21.15.139:0/1758000432 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d181a6a50 con 0x7f8d18102570 2024-09-16T01:39:06.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d1ccfb640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d18100e20 0x7f8d180fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:06.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d0ffff640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d18102570 0x7f8d180fd1c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8d08006010 tx=0x7f8d08003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:06.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d0dffb640 1 -- 172.21.15.139:0/1758000432 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0803f070 con 0x7f8d18102570 2024-09-16T01:39:06.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/1758000432 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d181a6ce0 con 0x7f8d18102570 2024-09-16T01:39:06.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d0dffb640 1 -- 172.21.15.139:0/1758000432 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d08031c30 con 0x7f8d18102570 2024-09-16T01:39:06.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.464+0000 7f8d0dffb640 1 -- 172.21.15.139:0/1758000432 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08035030 con 0x7f8d18102570 2024-09-16T01:39:06.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.465+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/1758000432 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d181a71c0 con 0x7f8d18102570 2024-09-16T01:39:06.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.465+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/1758000432 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8d180628c0 con 0x7f8d18102570 2024-09-16T01:39:06.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.467+0000 7f8d0dffb640 1 -- 172.21.15.139:0/1758000432 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8d08042070 con 0x7f8d18102570 2024-09-16T01:39:06.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.467+0000 7f8d0dffb640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8d000a32d0 0x7f8d000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:06.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.467+0000 7f8d1ccfb640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8d000a32d0 0x7f8d000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:06.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.467+0000 7f8d0dffb640 1 -- 172.21.15.139:0/1758000432 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f8d080f0e20 con 0x7f8d18102570 2024-09-16T01:39:06.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.468+0000 7f8d0dffb640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792] conn(0x7f8d000ae4a0 0x7f8d000b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:06.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.468+0000 7f8d0dffb640 1 -- 172.21.15.139:0/1758000432 --> [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d000b1010 con 0x7f8d000ae4a0 2024-09-16T01:39:06.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.468+0000 7f8d0dffb640 1 -- 172.21.15.139:0/1758000432 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f8d080f1270 con 0x7f8d18102570 2024-09-16T01:39:06.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.468+0000 7f8d1d4fc640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792] conn(0x7f8d000ae4a0 0x7f8d000b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:06.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.468+0000 7f8d1ccfb640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8d000a32d0 0x7f8d000a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009b50 tx=0x7f8d14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:06.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.468+0000 7f8d1d4fc640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792] conn(0x7f8d000ae4a0 0x7f8d000b0940 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:06.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.469+0000 7f8d0dffb640 1 -- 172.21.15.139:0/1758000432 <== osd.4 v2:172.21.15.164:6808/1421304792 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d000b1010 con 0x7f8d000ae4a0 2024-09-16T01:39:06.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.483+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/1758000432 --> [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8d18062cc0 con 0x7f8d000ae4a0 2024-09-16T01:39:06.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.483+0000 7f8d0dffb640 1 -- 172.21.15.139:0/1758000432 <== osd.4 v2:172.21.15.164:6808/1421304792 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8d18062cc0 con 0x7f8d000ae4a0 2024-09-16T01:39:06.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.484+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/1758000432 >> [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792] conn(0x7f8d000ae4a0 msgr2=0x7f8d000b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:06.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.484+0000 7f8d1ef5f640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792] conn(0x7f8d000ae4a0 0x7f8d000b0940 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:06.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.484+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/1758000432 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8d000a32d0 msgr2=0x7f8d000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:06.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.484+0000 7f8d1ef5f640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8d000a32d0 0x7f8d000a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009b50 tx=0x7f8d14005f90 comp rx=0 tx=0).stop 2024-09-16T01:39:06.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.484+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/1758000432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d18102570 msgr2=0x7f8d180fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:06.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.484+0000 7f8d1ef5f640 1 --2- 172.21.15.139:0/1758000432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d18102570 0x7f8d180fd1c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8d08006010 tx=0x7f8d08003c10 comp rx=0 tx=0).stop 2024-09-16T01:39:06.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.484+0000 7f8d1d4fc640 1 -- 172.21.15.139:0/1758000432 reap_dead start 2024-09-16T01:39:06.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.484+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/1758000432 shutdown_connections 2024-09-16T01:39:06.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.484+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/1758000432 >> 172.21.15.139:0/1758000432 conn(0x7f8d180fc7d0 msgr2=0x7f8d1806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:06.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.484+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/1758000432 shutdown_connections 2024-09-16T01:39:06.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:06.484+0000 7f8d1ef5f640 1 -- 172.21.15.139:0/1758000432 wait complete. 2024-09-16T01:39:06.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:06 smithi164 ceph-mon[35121]: pgmap v312: 129 pgs: 129 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-09-16T01:39:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:06.915 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:06.916 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:06.916 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:06.917 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:06.918 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:06.921 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:06.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:06 smithi139 ceph-mon[31175]: pgmap v312: 129 pgs: 129 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-09-16T01:39:06.924 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:06.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:06 smithi176 ceph-mon[35324]: pgmap v312: 129 pgs: 129 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-09-16T01:39:07.363 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:07.367 INFO:teuthology.orchestra.run.smithi139.stdout:68719476809 2024-09-16T01:39:07.367 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T01:39:07.819 INFO:teuthology.orchestra.run.smithi139.stdout:124554051647 2024-09-16T01:39:07.819 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T01:39:08.652 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:08 smithi139 ceph-mon[31175]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-16T01:39:08.652 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:08 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-09-16T01:39:08.652 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:08 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-16T01:39:08.652 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:08 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-16T01:39:08.652 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:08 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-16T01:39:08.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:08 smithi164 ceph-mon[35121]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-16T01:39:08.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:08 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-09-16T01:39:08.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:08 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-16T01:39:08.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:08 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-16T01:39:08.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:08 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-16T01:39:08.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:08 smithi176 ceph-mon[35324]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-16T01:39:08.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:08 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-09-16T01:39:08.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:08 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-16T01:39:08.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:08 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-16T01:39:08.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:08 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-16T01:39:09.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.330+0000 7f6b5feca640 1 -- 172.21.15.139:0/1326368525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b58106a80 msgr2=0x7f6b58106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:09.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.330+0000 7f6b5feca640 1 --2- 172.21.15.139:0/1326368525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b58106a80 0x7f6b58106e60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009930 tx=0x7f6b5402a420 comp rx=0 tx=0).stop 2024-09-16T01:39:09.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.330+0000 7f6b5feca640 1 -- 172.21.15.139:0/1326368525 shutdown_connections 2024-09-16T01:39:09.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.330+0000 7f6b5feca640 1 --2- 172.21.15.139:0/1326368525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b58101420 0x7f6b5810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:09.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.330+0000 7f6b5feca640 1 --2- 172.21.15.139:0/1326368525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b58100a80 0x7f6b58100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:09.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.330+0000 7f6b5feca640 1 --2- 172.21.15.139:0/1326368525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b58106a80 0x7f6b58106e60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:09.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.330+0000 7f6b5feca640 1 -- 172.21.15.139:0/1326368525 >> 172.21.15.139:0/1326368525 conn(0x7f6b580fc7a0 msgr2=0x7f6b580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:09.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.330+0000 7f6b5feca640 1 -- 172.21.15.139:0/1326368525 shutdown_connections 2024-09-16T01:39:09.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.330+0000 7f6b5feca640 1 -- 172.21.15.139:0/1326368525 wait complete. 2024-09-16T01:39:09.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5feca640 1 Processor -- start 2024-09-16T01:39:09.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5feca640 1 -- start start 2024-09-16T01:39:09.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5feca640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b58100a80 0x7f6b5819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:09.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5feca640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b58101420 0x7f6b5819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:09.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5feca640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b58106a80 0x7f6b5819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:09.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5feca640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6b58110e80 con 0x7f6b58100a80 2024-09-16T01:39:09.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5feca640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6b58110d00 con 0x7f6b58101420 2024-09-16T01:39:09.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5feca640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6b58111000 con 0x7f6b58106a80 2024-09-16T01:39:09.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5dc66640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b58100a80 0x7f6b5819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:09.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5dc66640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b58100a80 0x7f6b5819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43232/0 (socket says 172.21.15.139:43232) 2024-09-16T01:39:09.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5dc66640 1 -- 172.21.15.139:0/318566425 learned_addr learned my addr 172.21.15.139:0/318566425 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:09.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5d465640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b58101420 0x7f6b5819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:09.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5e467640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b58106a80 0x7f6b5819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:09.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5dc66640 1 -- 172.21.15.139:0/318566425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b58106a80 msgr2=0x7f6b5819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:09.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5dc66640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b58106a80 0x7f6b5819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:09.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5dc66640 1 -- 172.21.15.139:0/318566425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b58101420 msgr2=0x7f6b5819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:09.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5dc66640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b58101420 0x7f6b5819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:09.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5dc66640 1 -- 172.21.15.139:0/318566425 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5819f5b0 con 0x7f6b58100a80 2024-09-16T01:39:09.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.331+0000 7f6b5d465640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b58101420 0x7f6b5819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:09.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.332+0000 7f6b5dc66640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b58100a80 0x7f6b5819a600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009230 tx=0x7f6b54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:09.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.332+0000 7f6b4effd640 1 -- 172.21.15.139:0/318566425 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5403f030 con 0x7f6b58100a80 2024-09-16T01:39:09.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.332+0000 7f6b4effd640 1 -- 172.21.15.139:0/318566425 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b54003ad0 con 0x7f6b58100a80 2024-09-16T01:39:09.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.332+0000 7f6b5feca640 1 -- 172.21.15.139:0/318566425 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5819f840 con 0x7f6b58100a80 2024-09-16T01:39:09.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.332+0000 7f6b4effd640 1 -- 172.21.15.139:0/318566425 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54035070 con 0x7f6b58100a80 2024-09-16T01:39:09.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.332+0000 7f6b5feca640 1 -- 172.21.15.139:0/318566425 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b581a7180 con 0x7f6b58100a80 2024-09-16T01:39:09.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.333+0000 7f6b5feca640 1 -- 172.21.15.139:0/318566425 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b5806b1f0 con 0x7f6b58100a80 2024-09-16T01:39:09.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.334+0000 7f6b4effd640 1 -- 172.21.15.139:0/318566425 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6b54042030 con 0x7f6b58100a80 2024-09-16T01:39:09.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.334+0000 7f6b4effd640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6b440a32d0 0x7f6b440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:09.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.335+0000 7f6b5d465640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6b440a32d0 0x7f6b440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:09.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.335+0000 7f6b4effd640 1 -- 172.21.15.139:0/318566425 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9208+0+0 (secure 0 0 0) 0x7f6b540f0840 con 0x7f6b58100a80 2024-09-16T01:39:09.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.335+0000 7f6b4effd640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828] conn(0x7f6b440ae6a0 0x7f6b440b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:09.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.335+0000 7f6b4effd640 1 -- 172.21.15.139:0/318566425 --> [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b440b1210 con 0x7f6b440ae6a0 2024-09-16T01:39:09.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.335+0000 7f6b4effd640 1 -- 172.21.15.139:0/318566425 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f6b540f0c90 con 0x7f6b58100a80 2024-09-16T01:39:09.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.335+0000 7f6b5d465640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6b440a32d0 0x7f6b440a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6b5810bfc0 tx=0x7f6b4800b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:09.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.335+0000 7f6b5e467640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828] conn(0x7f6b440ae6a0 0x7f6b440b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:09.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.335+0000 7f6b5e467640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828] conn(0x7f6b440ae6a0 0x7f6b440b0b40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:09.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.336+0000 7f6b4effd640 1 -- 172.21.15.139:0/318566425 <== osd.10 v2:172.21.15.164:6824/4172533828 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b440b1210 con 0x7f6b440ae6a0 2024-09-16T01:39:09.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.350+0000 7f6b5feca640 1 -- 172.21.15.139:0/318566425 --> [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6b581021c0 con 0x7f6b440ae6a0 2024-09-16T01:39:09.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.350+0000 7f6b4effd640 1 -- 172.21.15.139:0/318566425 <== osd.10 v2:172.21.15.164:6824/4172533828 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6b581021c0 con 0x7f6b440ae6a0 2024-09-16T01:39:09.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.351+0000 7f6b5feca640 1 -- 172.21.15.139:0/318566425 >> [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828] conn(0x7f6b440ae6a0 msgr2=0x7f6b440b0b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:09.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.351+0000 7f6b5feca640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828] conn(0x7f6b440ae6a0 0x7f6b440b0b40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:09.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.351+0000 7f6b5feca640 1 -- 172.21.15.139:0/318566425 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6b440a32d0 msgr2=0x7f6b440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:09.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.351+0000 7f6b5feca640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6b440a32d0 0x7f6b440a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6b5810bfc0 tx=0x7f6b4800b5f0 comp rx=0 tx=0).stop 2024-09-16T01:39:09.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.351+0000 7f6b5feca640 1 -- 172.21.15.139:0/318566425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b58100a80 msgr2=0x7f6b5819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:09.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.351+0000 7f6b5feca640 1 --2- 172.21.15.139:0/318566425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b58100a80 0x7f6b5819a600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009230 tx=0x7f6b54004270 comp rx=0 tx=0).stop 2024-09-16T01:39:09.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.351+0000 7f6b5e467640 1 -- 172.21.15.139:0/318566425 reap_dead start 2024-09-16T01:39:09.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.351+0000 7f6b5feca640 1 -- 172.21.15.139:0/318566425 shutdown_connections 2024-09-16T01:39:09.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.351+0000 7f6b5feca640 1 -- 172.21.15.139:0/318566425 >> 172.21.15.139:0/318566425 conn(0x7f6b580fc7a0 msgr2=0x7f6b5806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:09.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.351+0000 7f6b5feca640 1 -- 172.21.15.139:0/318566425 shutdown_connections 2024-09-16T01:39:09.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.351+0000 7f6b5feca640 1 -- 172.21.15.139:0/318566425 wait complete. 2024-09-16T01:39:09.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-09-16T01:39:09.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-16T01:39:09.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-16T01:39:09.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:09 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-16T01:39:09.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:09 smithi139 ceph-mon[31175]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T01:39:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-09-16T01:39:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-16T01:39:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-16T01:39:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:09 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-16T01:39:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:09 smithi164 ceph-mon[35121]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T01:39:09.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-09-16T01:39:09.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-16T01:39:09.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-16T01:39:09.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:09 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-16T01:39:09.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:09 smithi176 ceph-mon[35324]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T01:39:10.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.235+0000 7f421037c640 1 -- 172.21.15.139:0/2128273053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4208101de0 msgr2=0x7f42081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:10.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.235+0000 7f421037c640 1 --2- 172.21.15.139:0/2128273053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4208101de0 0x7f42081021c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4204009900 tx=0x7f420402a410 comp rx=0 tx=0).stop 2024-09-16T01:39:10.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.235+0000 7f421037c640 1 -- 172.21.15.139:0/2128273053 shutdown_connections 2024-09-16T01:39:10.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.235+0000 7f421037c640 1 --2- 172.21.15.139:0/2128273053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4208106f10 0x7f420810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:10.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.235+0000 7f421037c640 1 --2- 172.21.15.139:0/2128273053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4208102790 0x7f42081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:10.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.235+0000 7f421037c640 1 --2- 172.21.15.139:0/2128273053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4208101de0 0x7f42081021c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:10.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.235+0000 7f421037c640 1 -- 172.21.15.139:0/2128273053 >> 172.21.15.139:0/2128273053 conn(0x7f42080fd8c0 msgr2=0x7f42080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:10.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.235+0000 7f421037c640 1 -- 172.21.15.139:0/2128273053 shutdown_connections 2024-09-16T01:39:10.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.235+0000 7f421037c640 1 -- 172.21.15.139:0/2128273053 wait complete. 2024-09-16T01:39:10.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f421037c640 1 Processor -- start 2024-09-16T01:39:10.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f421037c640 1 -- start start 2024-09-16T01:39:10.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f421037c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4208101de0 0x7f42081984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:10.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f421037c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4208102790 0x7f42081989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:10.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f421037c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4208106f10 0x7f420819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:10.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f421037c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4208111f90 con 0x7f4208102790 2024-09-16T01:39:10.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f421037c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4208111e10 con 0x7f4208101de0 2024-09-16T01:39:10.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f421037c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4208112110 con 0x7f4208106f10 2024-09-16T01:39:10.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f420d917640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4208102790 0x7f42081989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:10.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f420d917640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4208102790 0x7f42081989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43246/0 (socket says 172.21.15.139:43246) 2024-09-16T01:39:10.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f420d917640 1 -- 172.21.15.139:0/3429514635 learned_addr learned my addr 172.21.15.139:0/3429514635 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:10.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f420e118640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4208101de0 0x7f42081984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:10.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f420e919640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4208106f10 0x7f420819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:10.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f420d917640 1 -- 172.21.15.139:0/3429514635 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4208106f10 msgr2=0x7f420819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:10.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f420d917640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4208106f10 0x7f420819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:10.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f420d917640 1 -- 172.21.15.139:0/3429514635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4208101de0 msgr2=0x7f42081984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:10.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f420d917640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4208101de0 0x7f42081984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:10.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f420d917640 1 -- 172.21.15.139:0/3429514635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420819d460 con 0x7f4208102790 2024-09-16T01:39:10.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.236+0000 7f420e919640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4208106f10 0x7f420819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:10.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.237+0000 7f420d917640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4208102790 0x7f42081989f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f41f800d950 tx=0x7f41f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:10.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.237+0000 7f41ff7fe640 1 -- 172.21.15.139:0/3429514635 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8017070 con 0x7f4208102790 2024-09-16T01:39:10.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.237+0000 7f41ff7fe640 1 -- 172.21.15.139:0/3429514635 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f8009c30 con 0x7f4208102790 2024-09-16T01:39:10.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.237+0000 7f421037c640 1 -- 172.21.15.139:0/3429514635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420819d750 con 0x7f4208102790 2024-09-16T01:39:10.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.237+0000 7f41ff7fe640 1 -- 172.21.15.139:0/3429514635 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f800f6b0 con 0x7f4208102790 2024-09-16T01:39:10.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.237+0000 7f421037c640 1 -- 172.21.15.139:0/3429514635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42081a5030 con 0x7f4208102790 2024-09-16T01:39:10.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.238+0000 7f421037c640 1 -- 172.21.15.139:0/3429514635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4208062980 con 0x7f4208102790 2024-09-16T01:39:10.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.239+0000 7f41ff7fe640 1 -- 172.21.15.139:0/3429514635 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f41f80109d0 con 0x7f4208102790 2024-09-16T01:39:10.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.239+0000 7f41ff7fe640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41f40a3350 0x7f41f40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:10.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.240+0000 7f420e118640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41f40a3350 0x7f41f40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:10.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.240+0000 7f41ff7fe640 1 -- 172.21.15.139:0/3429514635 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f41f80cfef0 con 0x7f4208102790 2024-09-16T01:39:10.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.240+0000 7f41ff7fe640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133] conn(0x7f41f40ae720 0x7f41f40b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:10.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.240+0000 7f41ff7fe640 1 -- 172.21.15.139:0/3429514635 --> [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41f801de50 con 0x7f41f40ae720 2024-09-16T01:39:10.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.240+0000 7f41ff7fe640 1 -- 172.21.15.139:0/3429514635 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f41f80d0340 con 0x7f4208102790 2024-09-16T01:39:10.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.240+0000 7f420e919640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133] conn(0x7f41f40ae720 0x7f41f40b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:10.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.240+0000 7f420e118640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41f40a3350 0x7f41f40a5810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4204005ab0 tx=0x7f4204005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:10.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.240+0000 7f420e919640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133] conn(0x7f41f40ae720 0x7f41f40b0bc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:10.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.241+0000 7f41ff7fe640 1 -- 172.21.15.139:0/3429514635 <== osd.0 v2:172.21.15.139:6802/3190810133 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f41f801de50 con 0x7f41f40ae720 2024-09-16T01:39:10.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.255+0000 7f421037c640 1 -- 172.21.15.139:0/3429514635 --> [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4208062d80 con 0x7f41f40ae720 2024-09-16T01:39:10.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.255+0000 7f41ff7fe640 1 -- 172.21.15.139:0/3429514635 <== osd.0 v2:172.21.15.139:6802/3190810133 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4208062d80 con 0x7f41f40ae720 2024-09-16T01:39:10.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.256+0000 7f421037c640 1 -- 172.21.15.139:0/3429514635 >> [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133] conn(0x7f41f40ae720 msgr2=0x7f41f40b0bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:10.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.256+0000 7f421037c640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133] conn(0x7f41f40ae720 0x7f41f40b0bc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:10.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.256+0000 7f421037c640 1 -- 172.21.15.139:0/3429514635 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41f40a3350 msgr2=0x7f41f40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:10.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.256+0000 7f421037c640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41f40a3350 0x7f41f40a5810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4204005ab0 tx=0x7f4204005a20 comp rx=0 tx=0).stop 2024-09-16T01:39:10.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.256+0000 7f421037c640 1 -- 172.21.15.139:0/3429514635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4208102790 msgr2=0x7f42081989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:10.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.256+0000 7f421037c640 1 --2- 172.21.15.139:0/3429514635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4208102790 0x7f42081989f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f41f800d950 tx=0x7f41f800de10 comp rx=0 tx=0).stop 2024-09-16T01:39:10.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.256+0000 7f420e919640 1 -- 172.21.15.139:0/3429514635 reap_dead start 2024-09-16T01:39:10.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.256+0000 7f421037c640 1 -- 172.21.15.139:0/3429514635 shutdown_connections 2024-09-16T01:39:10.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.256+0000 7f421037c640 1 -- 172.21.15.139:0/3429514635 >> 172.21.15.139:0/3429514635 conn(0x7f42080fd8c0 msgr2=0x7f42080fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:10.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.256+0000 7f421037c640 1 -- 172.21.15.139:0/3429514635 shutdown_connections 2024-09-16T01:39:10.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:10.256+0000 7f421037c640 1 -- 172.21.15.139:0/3429514635 wait complete. 2024-09-16T01:39:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:10 smithi164 ceph-mon[35121]: pgmap v315: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-16T01:39:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:10 smithi164 ceph-mon[35121]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T01:39:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:10 smithi164 ceph-mon[35121]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T01:39:10.899 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:10 smithi139 ceph-mon[31175]: pgmap v315: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-16T01:39:10.899 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:10 smithi139 ceph-mon[31175]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T01:39:10.899 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:10 smithi139 ceph-mon[31175]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T01:39:10.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:10 smithi176 ceph-mon[35324]: pgmap v315: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-16T01:39:10.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:10 smithi176 ceph-mon[35324]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T01:39:10.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:10 smithi176 ceph-mon[35324]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T01:39:11.461 INFO:teuthology.orchestra.run.smithi139.stdout:34359738450 2024-09-16T01:39:11.461 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T01:39:12.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.207+0000 7fb10c7af640 1 -- 172.21.15.139:0/154780482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb104105a90 msgr2=0x7fb104105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:12.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.207+0000 7fb10c7af640 1 --2- 172.21.15.139:0/154780482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb104105a90 0x7fb104105e70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009900 tx=0x7fb0fc02a410 comp rx=0 tx=0).stop 2024-09-16T01:39:12.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.207+0000 7fb10c7af640 1 -- 172.21.15.139:0/154780482 shutdown_connections 2024-09-16T01:39:12.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.207+0000 7fb10c7af640 1 --2- 172.21.15.139:0/154780482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb104101bc0 0x7fb10410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:12.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.207+0000 7fb10c7af640 1 --2- 172.21.15.139:0/154780482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb104101240 0x7fb104101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:12.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.207+0000 7fb10c7af640 1 --2- 172.21.15.139:0/154780482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb104105a90 0x7fb104105e70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:12.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.207+0000 7fb10c7af640 1 -- 172.21.15.139:0/154780482 >> 172.21.15.139:0/154780482 conn(0x7fb104077210 msgr2=0x7fb1040fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:12.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.207+0000 7fb10c7af640 1 -- 172.21.15.139:0/154780482 shutdown_connections 2024-09-16T01:39:12.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.207+0000 7fb10c7af640 1 -- 172.21.15.139:0/154780482 wait complete. 2024-09-16T01:39:12.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb10c7af640 1 Processor -- start 2024-09-16T01:39:12.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb10c7af640 1 -- start start 2024-09-16T01:39:12.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb10c7af640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb104101240 0x7fb104198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:12.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb10c7af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb104101bc0 0x7fb104198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:12.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb10c7af640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb104105a90 0x7fb10419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:12.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb10c7af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041120b0 con 0x7fb104101bc0 2024-09-16T01:39:12.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb10c7af640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb104111f30 con 0x7fb104101240 2024-09-16T01:39:12.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb10c7af640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb104112230 con 0x7fb104105a90 2024-09-16T01:39:12.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb109d4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb104101bc0 0x7fb104198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:12.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb109d4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb104101bc0 0x7fb104198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43262/0 (socket says 172.21.15.139:43262) 2024-09-16T01:39:12.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb109d4a640 1 -- 172.21.15.139:0/2562132538 learned_addr learned my addr 172.21.15.139:0/2562132538 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:12.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb10a54b640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb104101240 0x7fb104198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:12.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb10ad4c640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb104105a90 0x7fb10419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:12.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb109d4a640 1 -- 172.21.15.139:0/2562132538 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb104105a90 msgr2=0x7fb10419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:12.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb109d4a640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb104105a90 0x7fb10419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:12.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb109d4a640 1 -- 172.21.15.139:0/2562132538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb104101240 msgr2=0x7fb104198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:12.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb109d4a640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb104101240 0x7fb104198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:12.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb109d4a640 1 -- 172.21.15.139:0/2562132538 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10419d540 con 0x7fb104101bc0 2024-09-16T01:39:12.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb10a54b640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb104101240 0x7fb104198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:12.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.208+0000 7fb109d4a640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb104101bc0 0x7fb104198ad0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800cce0 tx=0x7fb0f800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:12.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.209+0000 7fb1037fe640 1 -- 172.21.15.139:0/2562132538 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8004b20 con 0x7fb104101bc0 2024-09-16T01:39:12.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.209+0000 7fb10c7af640 1 -- 172.21.15.139:0/2562132538 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10419d830 con 0x7fb104101bc0 2024-09-16T01:39:12.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.209+0000 7fb1037fe640 1 -- 172.21.15.139:0/2562132538 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f8004cc0 con 0x7fb104101bc0 2024-09-16T01:39:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.209+0000 7fb1037fe640 1 -- 172.21.15.139:0/2562132538 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800f7d0 con 0x7fb104101bc0 2024-09-16T01:39:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.209+0000 7fb10c7af640 1 -- 172.21.15.139:0/2562132538 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1041a5110 con 0x7fb104101bc0 2024-09-16T01:39:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.209+0000 7fb10c7af640 1 -- 172.21.15.139:0/2562132538 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb104062900 con 0x7fb104101bc0 2024-09-16T01:39:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.211+0000 7fb1037fe640 1 -- 172.21.15.139:0/2562132538 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb0f800f970 con 0x7fb104101bc0 2024-09-16T01:39:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.211+0000 7fb1037fe640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fb0ec0a32d0 0x7fb0ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.211+0000 7fb10a54b640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fb0ec0a32d0 0x7fb0ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.211+0000 7fb1037fe640 1 -- 172.21.15.139:0/2562132538 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fb0f80d03b0 con 0x7fb104101bc0 2024-09-16T01:39:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.212+0000 7fb1037fe640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108] conn(0x7fb0ec0ae6a0 0x7fb0ec0b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.212+0000 7fb1037fe640 1 -- 172.21.15.139:0/2562132538 --> [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0ec0b1210 con 0x7fb0ec0ae6a0 2024-09-16T01:39:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.212+0000 7fb1037fe640 1 -- 172.21.15.139:0/2562132538 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb0f80d0800 con 0x7fb104101bc0 2024-09-16T01:39:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.212+0000 7fb10ad4c640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108] conn(0x7fb0ec0ae6a0 0x7fb0ec0b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.212+0000 7fb10a54b640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fb0ec0a32d0 0x7fb0ec0a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc005ab0 tx=0x7fb0fc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.212+0000 7fb10ad4c640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108] conn(0x7fb0ec0ae6a0 0x7fb0ec0b0b40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:12.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.213+0000 7fb1037fe640 1 -- 172.21.15.139:0/2562132538 <== osd.9 v2:172.21.15.139:6826/3047942108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0ec0b1210 con 0x7fb0ec0ae6a0 2024-09-16T01:39:12.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.226+0000 7fb10c7af640 1 -- 172.21.15.139:0/2562132538 --> [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb104062d00 con 0x7fb0ec0ae6a0 2024-09-16T01:39:12.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.226+0000 7fb1037fe640 1 -- 172.21.15.139:0/2562132538 <== osd.9 v2:172.21.15.139:6826/3047942108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb104062d00 con 0x7fb0ec0ae6a0 2024-09-16T01:39:12.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.226+0000 7fb10c7af640 1 -- 172.21.15.139:0/2562132538 >> [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108] conn(0x7fb0ec0ae6a0 msgr2=0x7fb0ec0b0b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:12.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.226+0000 7fb10c7af640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108] conn(0x7fb0ec0ae6a0 0x7fb0ec0b0b40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:12.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.227+0000 7fb10c7af640 1 -- 172.21.15.139:0/2562132538 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fb0ec0a32d0 msgr2=0x7fb0ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:12.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.227+0000 7fb10c7af640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fb0ec0a32d0 0x7fb0ec0a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc005ab0 tx=0x7fb0fc005a20 comp rx=0 tx=0).stop 2024-09-16T01:39:12.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.227+0000 7fb10c7af640 1 -- 172.21.15.139:0/2562132538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb104101bc0 msgr2=0x7fb104198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:12.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.227+0000 7fb10c7af640 1 --2- 172.21.15.139:0/2562132538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb104101bc0 0x7fb104198ad0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800cce0 tx=0x7fb0f800a620 comp rx=0 tx=0).stop 2024-09-16T01:39:12.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.227+0000 7fb10ad4c640 1 -- 172.21.15.139:0/2562132538 reap_dead start 2024-09-16T01:39:12.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.227+0000 7fb10c7af640 1 -- 172.21.15.139:0/2562132538 shutdown_connections 2024-09-16T01:39:12.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.227+0000 7fb10c7af640 1 -- 172.21.15.139:0/2562132538 >> 172.21.15.139:0/2562132538 conn(0x7fb104077210 msgr2=0x7fb104103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:12.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.227+0000 7fb10c7af640 1 -- 172.21.15.139:0/2562132538 shutdown_connections 2024-09-16T01:39:12.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.227+0000 7fb10c7af640 1 -- 172.21.15.139:0/2562132538 wait complete. 2024-09-16T01:39:12.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:12 smithi139 ceph-mon[31175]: pgmap v317: 129 pgs: 1 activating, 128 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 383 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T01:39:12.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.732+0000 7fd7cb142640 1 -- 172.21.15.139:0/1338300529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7c4102790 msgr2=0x7fd7c41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:12.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.732+0000 7fd7cb142640 1 --2- 172.21.15.139:0/1338300529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7c4102790 0x7fd7c41067e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009900 tx=0x7fd7b002a410 comp rx=0 tx=0).stop 2024-09-16T01:39:12.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.732+0000 7fd7cb142640 1 -- 172.21.15.139:0/1338300529 shutdown_connections 2024-09-16T01:39:12.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.732+0000 7fd7cb142640 1 --2- 172.21.15.139:0/1338300529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7c4106f10 0x7fd7c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:12.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.732+0000 7fd7cb142640 1 --2- 172.21.15.139:0/1338300529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7c4102790 0x7fd7c41067e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:12.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.732+0000 7fd7cb142640 1 --2- 172.21.15.139:0/1338300529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd7c4101de0 0x7fd7c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:12.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.732+0000 7fd7cb142640 1 -- 172.21.15.139:0/1338300529 >> 172.21.15.139:0/1338300529 conn(0x7fd7c40fd8c0 msgr2=0x7fd7c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:12.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.732+0000 7fd7cb142640 1 -- 172.21.15.139:0/1338300529 shutdown_connections 2024-09-16T01:39:12.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.732+0000 7fd7cb142640 1 -- 172.21.15.139:0/1338300529 wait complete. 2024-09-16T01:39:12.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.733+0000 7fd7cb142640 1 Processor -- start 2024-09-16T01:39:12.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.733+0000 7fd7cb142640 1 -- start start 2024-09-16T01:39:12.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.733+0000 7fd7cb142640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7c4101de0 0x7fd7c4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:12.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.733+0000 7fd7cb142640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd7c4102790 0x7fd7c4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:12.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.733+0000 7fd7cb142640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7c4106f10 0x7fd7c419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:12.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.733+0000 7fd7cb142640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c4111f90 con 0x7fd7c4106f10 2024-09-16T01:39:12.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.733+0000 7fd7cb142640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c4111e10 con 0x7fd7c4101de0 2024-09-16T01:39:12.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.733+0000 7fd7cb142640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c4112110 con 0x7fd7c4102790 2024-09-16T01:39:12.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.733+0000 7fd7c96df640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7c4106f10 0x7fd7c419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:12.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.733+0000 7fd7c96df640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7c4106f10 0x7fd7c419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43188/0 (socket says 172.21.15.139:43188) 2024-09-16T01:39:12.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.733+0000 7fd7c96df640 1 -- 172.21.15.139:0/3930655072 learned_addr learned my addr 172.21.15.139:0/3930655072 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:12.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.734+0000 7fd7c8ede640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7c4101de0 0x7fd7c4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:12.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.734+0000 7fd7c96df640 1 -- 172.21.15.139:0/3930655072 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd7c4102790 msgr2=0x7fd7c4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:12.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.734+0000 7fd7c3fff640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd7c4102790 0x7fd7c4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:12.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.734+0000 7fd7c96df640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd7c4102790 0x7fd7c4198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:12.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.734+0000 7fd7c96df640 1 -- 172.21.15.139:0/3930655072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7c4101de0 msgr2=0x7fd7c4198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:12.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.734+0000 7fd7c96df640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7c4101de0 0x7fd7c4198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:12.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.734+0000 7fd7c96df640 1 -- 172.21.15.139:0/3930655072 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c419d500 con 0x7fd7c4106f10 2024-09-16T01:39:12.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.734+0000 7fd7c8ede640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7c4101de0 0x7fd7c4198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:12.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.734+0000 7fd7c96df640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7c4106f10 0x7fd7c419ce20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00cad0 tx=0x7fd7bc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:12.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.734+0000 7fd7c1ffb640 1 -- 172.21.15.139:0/3930655072 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc00ae20 con 0x7fd7c4106f10 2024-09-16T01:39:12.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.734+0000 7fd7cb142640 1 -- 172.21.15.139:0/3930655072 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c419d7f0 con 0x7fd7c4106f10 2024-09-16T01:39:12.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.734+0000 7fd7c1ffb640 1 -- 172.21.15.139:0/3930655072 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7bc005020 con 0x7fd7c4106f10 2024-09-16T01:39:12.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.734+0000 7fd7c1ffb640 1 -- 172.21.15.139:0/3930655072 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc00f6f0 con 0x7fd7c4106f10 2024-09-16T01:39:12.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.734+0000 7fd7cb142640 1 -- 172.21.15.139:0/3930655072 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c41a50d0 con 0x7fd7c4106f10 2024-09-16T01:39:12.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.735+0000 7fd7cb142640 1 -- 172.21.15.139:0/3930655072 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd7c4062900 con 0x7fd7c4106f10 2024-09-16T01:39:12.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.736+0000 7fd7c1ffb640 1 -- 172.21.15.139:0/3930655072 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd7bc0051d0 con 0x7fd7c4106f10 2024-09-16T01:39:12.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.737+0000 7fd7c1ffb640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd7b40a32d0 0x7fd7b40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:12.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.737+0000 7fd7c8ede640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd7b40a32d0 0x7fd7b40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:12.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.737+0000 7fd7c1ffb640 1 -- 172.21.15.139:0/3930655072 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9208+0+0 (secure 0 0 0) 0x7fd7bc0d0340 con 0x7fd7c4106f10 2024-09-16T01:39:12.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.737+0000 7fd7c1ffb640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071] conn(0x7fd7b40ae6a0 0x7fd7b40b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:12.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.737+0000 7fd7c1ffb640 1 -- 172.21.15.139:0/3930655072 --> [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7b40b1210 con 0x7fd7b40ae6a0 2024-09-16T01:39:12.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.737+0000 7fd7c1ffb640 1 -- 172.21.15.139:0/3930655072 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fd7bc0d0790 con 0x7fd7c4106f10 2024-09-16T01:39:12.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.737+0000 7fd7c3fff640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071] conn(0x7fd7b40ae6a0 0x7fd7b40b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:12.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.738+0000 7fd7c8ede640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd7b40a32d0 0x7fd7b40a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8006fd0 tx=0x7fd7b8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:12.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.738+0000 7fd7c3fff640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071] conn(0x7fd7b40ae6a0 0x7fd7b40b0b40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:12.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.738+0000 7fd7c1ffb640 1 -- 172.21.15.139:0/3930655072 <== osd.3 v2:172.21.15.139:6810/797147071 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd7b40b1210 con 0x7fd7b40ae6a0 2024-09-16T01:39:12.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.752+0000 7fd7cb142640 1 -- 172.21.15.139:0/3930655072 --> [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd7c4062d80 con 0x7fd7b40ae6a0 2024-09-16T01:39:12.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.752+0000 7fd7c1ffb640 1 -- 172.21.15.139:0/3930655072 <== osd.3 v2:172.21.15.139:6810/797147071 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd7c4062d80 con 0x7fd7b40ae6a0 2024-09-16T01:39:12.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.752+0000 7fd7cb142640 1 -- 172.21.15.139:0/3930655072 >> [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071] conn(0x7fd7b40ae6a0 msgr2=0x7fd7b40b0b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:12.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.752+0000 7fd7cb142640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071] conn(0x7fd7b40ae6a0 0x7fd7b40b0b40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:12.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.752+0000 7fd7cb142640 1 -- 172.21.15.139:0/3930655072 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd7b40a32d0 msgr2=0x7fd7b40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:12.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.752+0000 7fd7cb142640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd7b40a32d0 0x7fd7b40a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8006fd0 tx=0x7fd7b8008040 comp rx=0 tx=0).stop 2024-09-16T01:39:12.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.752+0000 7fd7cb142640 1 -- 172.21.15.139:0/3930655072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7c4106f10 msgr2=0x7fd7c419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:12.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.752+0000 7fd7cb142640 1 --2- 172.21.15.139:0/3930655072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7c4106f10 0x7fd7c419ce20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00cad0 tx=0x7fd7bc00cf90 comp rx=0 tx=0).stop 2024-09-16T01:39:12.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.752+0000 7fd7c96df640 1 -- 172.21.15.139:0/3930655072 reap_dead start 2024-09-16T01:39:12.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.753+0000 7fd7cb142640 1 -- 172.21.15.139:0/3930655072 shutdown_connections 2024-09-16T01:39:12.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.753+0000 7fd7cb142640 1 -- 172.21.15.139:0/3930655072 >> 172.21.15.139:0/3930655072 conn(0x7fd7c40fd8c0 msgr2=0x7fd7c40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:12.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.753+0000 7fd7cb142640 1 -- 172.21.15.139:0/3930655072 shutdown_connections 2024-09-16T01:39:12.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:08.753+0000 7fd7cb142640 1 -- 172.21.15.139:0/3930655072 wait complete. 2024-09-16T01:39:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:12 smithi164 ceph-mon[35121]: pgmap v317: 129 pgs: 1 activating, 128 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 383 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T01:39:12.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:12 smithi176 ceph-mon[35324]: pgmap v317: 129 pgs: 1 activating, 128 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 383 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T01:39:13.390 INFO:teuthology.orchestra.run.smithi139.stdout:94489280580 2024-09-16T01:39:13.391 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T01:39:13.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.312+0000 7fbda3ff6640 1 -- 172.21.15.139:0/1010231779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd9c106f10 msgr2=0x7fbd9c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:13.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.312+0000 7fbda3ff6640 1 --2- 172.21.15.139:0/1010231779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd9c106f10 0x7fbd9c10ea90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009ca0 tx=0x7fbd9402a8c0 comp rx=0 tx=0).stop 2024-09-16T01:39:13.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.313+0000 7fbda3ff6640 1 -- 172.21.15.139:0/1010231779 shutdown_connections 2024-09-16T01:39:13.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.313+0000 7fbda3ff6640 1 --2- 172.21.15.139:0/1010231779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd9c106f10 0x7fbd9c10ea90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:13.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.313+0000 7fbda3ff6640 1 --2- 172.21.15.139:0/1010231779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd9c102790 0x7fbd9c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:13.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.313+0000 7fbda3ff6640 1 --2- 172.21.15.139:0/1010231779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd9c101de0 0x7fbd9c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:13.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.313+0000 7fbda3ff6640 1 -- 172.21.15.139:0/1010231779 >> 172.21.15.139:0/1010231779 conn(0x7fbd9c0fd8c0 msgr2=0x7fbd9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:13.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.313+0000 7fbda3ff6640 1 -- 172.21.15.139:0/1010231779 shutdown_connections 2024-09-16T01:39:13.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.313+0000 7fbda3ff6640 1 -- 172.21.15.139:0/1010231779 wait complete. 2024-09-16T01:39:13.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.313+0000 7fbda3ff6640 1 Processor -- start 2024-09-16T01:39:13.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.313+0000 7fbda3ff6640 1 -- start start 2024-09-16T01:39:13.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda3ff6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd9c101de0 0x7fbd9c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:13.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda3ff6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd9c102790 0x7fbd9c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:13.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda3ff6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd9c106f10 0x7fbd9c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:13.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda3ff6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c111f70 con 0x7fbd9c102790 2024-09-16T01:39:13.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda3ff6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c111df0 con 0x7fbd9c101de0 2024-09-16T01:39:13.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda3ff6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1120f0 con 0x7fbd9c106f10 2024-09-16T01:39:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda1591640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd9c102790 0x7fbd9c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda1591640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd9c102790 0x7fbd9c1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46050/0 (socket says 172.21.15.139:46050) 2024-09-16T01:39:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda1591640 1 -- 172.21.15.139:0/2233570714 learned_addr learned my addr 172.21.15.139:0/2233570714 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda2593640 1 --2- 172.21.15.139:0/2233570714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd9c106f10 0x7fbd9c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda1d92640 1 --2- 172.21.15.139:0/2233570714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd9c101de0 0x7fbd9c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda1591640 1 -- 172.21.15.139:0/2233570714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd9c106f10 msgr2=0x7fbd9c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda1591640 1 --2- 172.21.15.139:0/2233570714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd9c106f10 0x7fbd9c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda1591640 1 -- 172.21.15.139:0/2233570714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd9c101de0 msgr2=0x7fbd9c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda1591640 1 --2- 172.21.15.139:0/2233570714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd9c101de0 0x7fbd9c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda1591640 1 -- 172.21.15.139:0/2233570714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c19d350 con 0x7fbd9c102790 2024-09-16T01:39:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda1591640 1 --2- 172.21.15.139:0/2233570714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd9c102790 0x7fbd9c1988e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00d950 tx=0x7fbd8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbd92ffd640 1 -- 172.21.15.139:0/2233570714 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c017030 con 0x7fbd9c102790 2024-09-16T01:39:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbd92ffd640 1 -- 172.21.15.139:0/2233570714 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd8c004020 con 0x7fbd9c102790 2024-09-16T01:39:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda3ff6640 1 -- 172.21.15.139:0/2233570714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c19d640 con 0x7fbd9c102790 2024-09-16T01:39:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbd92ffd640 1 -- 172.21.15.139:0/2233570714 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c0047a0 con 0x7fbd9c102790 2024-09-16T01:39:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.314+0000 7fbda3ff6640 1 -- 172.21.15.139:0/2233570714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c1a4f20 con 0x7fbd9c102790 2024-09-16T01:39:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.315+0000 7fbda3ff6640 1 -- 172.21.15.139:0/2233570714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbd9c062900 con 0x7fbd9c102790 2024-09-16T01:39:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.317+0000 7fbd92ffd640 1 -- 172.21.15.139:0/2233570714 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbd8c0041d0 con 0x7fbd9c102790 2024-09-16T01:39:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.317+0000 7fbd92ffd640 1 --2- 172.21.15.139:0/2233570714 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbd880a32d0 0x7fbd880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.318+0000 7fbda1d92640 1 --2- 172.21.15.139:0/2233570714 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbd880a32d0 0x7fbd880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.318+0000 7fbd92ffd640 1 -- 172.21.15.139:0/2233570714 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fbd8c0cfe00 con 0x7fbd9c102790 2024-09-16T01:39:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.318+0000 7fbd92ffd640 1 --2- 172.21.15.139:0/2233570714 >> [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602] conn(0x7fbd880ae6a0 0x7fbd880b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.318+0000 7fbd92ffd640 1 -- 172.21.15.139:0/2233570714 --> [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd880b1210 con 0x7fbd880ae6a0 2024-09-16T01:39:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.318+0000 7fbd92ffd640 1 -- 172.21.15.139:0/2233570714 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fbd8c0d0250 con 0x7fbd9c102790 2024-09-16T01:39:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.318+0000 7fbda1d92640 1 --2- 172.21.15.139:0/2233570714 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbd880a32d0 0x7fbd880a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd98005fd0 tx=0x7fbd9800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.318+0000 7fbda2593640 1 --2- 172.21.15.139:0/2233570714 >> [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602] conn(0x7fbd880ae6a0 0x7fbd880b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.318+0000 7fbda2593640 1 --2- 172.21.15.139:0/2233570714 >> [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602] conn(0x7fbd880ae6a0 0x7fbd880b0b40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.319+0000 7fbd92ffd640 1 -- 172.21.15.139:0/2233570714 <== osd.5 v2:172.21.15.176:6808/10600602 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd880b1210 con 0x7fbd880ae6a0 2024-09-16T01:39:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.333+0000 7fbda3ff6640 1 -- 172.21.15.139:0/2233570714 --> [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbd9c062d80 con 0x7fbd880ae6a0 2024-09-16T01:39:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.333+0000 7fbd92ffd640 1 -- 172.21.15.139:0/2233570714 <== osd.5 v2:172.21.15.176:6808/10600602 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbd9c062d80 con 0x7fbd880ae6a0 2024-09-16T01:39:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.334+0000 7fbda3ff6640 1 -- 172.21.15.139:0/2233570714 >> [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602] conn(0x7fbd880ae6a0 msgr2=0x7fbd880b0b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:13.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.334+0000 7fbda3ff6640 1 --2- 172.21.15.139:0/2233570714 >> [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602] conn(0x7fbd880ae6a0 0x7fbd880b0b40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:13.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.334+0000 7fbda3ff6640 1 -- 172.21.15.139:0/2233570714 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbd880a32d0 msgr2=0x7fbd880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:13.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.334+0000 7fbda3ff6640 1 --2- 172.21.15.139:0/2233570714 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbd880a32d0 0x7fbd880a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd98005fd0 tx=0x7fbd9800b390 comp rx=0 tx=0).stop 2024-09-16T01:39:13.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.334+0000 7fbda3ff6640 1 -- 172.21.15.139:0/2233570714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd9c102790 msgr2=0x7fbd9c1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:13.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.334+0000 7fbda3ff6640 1 --2- 172.21.15.139:0/2233570714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd9c102790 0x7fbd9c1988e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00d950 tx=0x7fbd8c00de10 comp rx=0 tx=0).stop 2024-09-16T01:39:13.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.334+0000 7fbda2593640 1 -- 172.21.15.139:0/2233570714 reap_dead start 2024-09-16T01:39:13.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.334+0000 7fbda3ff6640 1 -- 172.21.15.139:0/2233570714 shutdown_connections 2024-09-16T01:39:13.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.334+0000 7fbda3ff6640 1 -- 172.21.15.139:0/2233570714 >> 172.21.15.139:0/2233570714 conn(0x7fbd9c0fd8c0 msgr2=0x7fbd9c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:13.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.334+0000 7fbda3ff6640 1 -- 172.21.15.139:0/2233570714 shutdown_connections 2024-09-16T01:39:13.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:12.334+0000 7fbda3ff6640 1 -- 172.21.15.139:0/2233570714 wait complete. 2024-09-16T01:39:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.981+0000 7fcb7921d640 1 -- 172.21.15.139:0/3671495572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb74106a80 msgr2=0x7fcb74106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.981+0000 7fcb7921d640 1 --2- 172.21.15.139:0/3671495572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb74106a80 0x7fcb74106e60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009930 tx=0x7fcb6402a420 comp rx=0 tx=0).stop 2024-09-16T01:39:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.981+0000 7fcb7921d640 1 -- 172.21.15.139:0/3671495572 shutdown_connections 2024-09-16T01:39:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.981+0000 7fcb7921d640 1 --2- 172.21.15.139:0/3671495572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb74101420 0x7fcb7410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.981+0000 7fcb7921d640 1 --2- 172.21.15.139:0/3671495572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb74100a80 0x7fcb74100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.981+0000 7fcb7921d640 1 --2- 172.21.15.139:0/3671495572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb74106a80 0x7fcb74106e60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.981+0000 7fcb7921d640 1 -- 172.21.15.139:0/3671495572 >> 172.21.15.139:0/3671495572 conn(0x7fcb740fc7a0 msgr2=0x7fcb740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.981+0000 7fcb7921d640 1 -- 172.21.15.139:0/3671495572 shutdown_connections 2024-09-16T01:39:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.981+0000 7fcb7921d640 1 -- 172.21.15.139:0/3671495572 wait complete. 2024-09-16T01:39:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.982+0000 7fcb7921d640 1 Processor -- start 2024-09-16T01:39:14.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.982+0000 7fcb7921d640 1 -- start start 2024-09-16T01:39:14.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.982+0000 7fcb7921d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb74100a80 0x7fcb74198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.982+0000 7fcb7921d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb74101420 0x7fcb741989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.982+0000 7fcb7921d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb74106a80 0x7fcb7419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.982+0000 7fcb7921d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcb74110e80 con 0x7fcb74100a80 2024-09-16T01:39:14.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.982+0000 7fcb7921d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb74110d00 con 0x7fcb74101420 2024-09-16T01:39:14.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.982+0000 7fcb7921d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb74111000 con 0x7fcb74106a80 2024-09-16T01:39:14.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb72d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb74100a80 0x7fcb74198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb72d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb74100a80 0x7fcb74198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46056/0 (socket says 172.21.15.139:46056) 2024-09-16T01:39:14.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb72d9d640 1 -- 172.21.15.139:0/3283529695 learned_addr learned my addr 172.21.15.139:0/3283529695 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:14.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb6bfff640 1 --2- 172.21.15.139:0/3283529695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb74101420 0x7fcb741989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb7359e640 1 --2- 172.21.15.139:0/3283529695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb74106a80 0x7fcb7419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb72d9d640 1 -- 172.21.15.139:0/3283529695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb74106a80 msgr2=0x7fcb7419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb72d9d640 1 --2- 172.21.15.139:0/3283529695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb74106a80 0x7fcb7419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb72d9d640 1 -- 172.21.15.139:0/3283529695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb74101420 msgr2=0x7fcb741989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb72d9d640 1 --2- 172.21.15.139:0/3283529695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb74101420 0x7fcb741989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb72d9d640 1 -- 172.21.15.139:0/3283529695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7419d410 con 0x7fcb74100a80 2024-09-16T01:39:14.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb72d9d640 1 --2- 172.21.15.139:0/3283529695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb74100a80 0x7fcb74198460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009230 tx=0x7fcb64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:14.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb70d99640 1 -- 172.21.15.139:0/3283529695 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6403f030 con 0x7fcb74100a80 2024-09-16T01:39:14.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb70d99640 1 -- 172.21.15.139:0/3283529695 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb64003ad0 con 0x7fcb74100a80 2024-09-16T01:39:14.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb7921d640 1 -- 172.21.15.139:0/3283529695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7419d6a0 con 0x7fcb74100a80 2024-09-16T01:39:14.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb70d99640 1 -- 172.21.15.139:0/3283529695 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64035070 con 0x7fcb74100a80 2024-09-16T01:39:14.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.983+0000 7fcb7921d640 1 -- 172.21.15.139:0/3283529695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb741a4fe0 con 0x7fcb74100a80 2024-09-16T01:39:14.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.984+0000 7fcb7921d640 1 -- 172.21.15.139:0/3283529695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcb74062980 con 0x7fcb74100a80 2024-09-16T01:39:14.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.985+0000 7fcb70d99640 1 -- 172.21.15.139:0/3283529695 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcb64042030 con 0x7fcb74100a80 2024-09-16T01:39:14.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.986+0000 7fcb70d99640 1 --2- 172.21.15.139:0/3283529695 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcb600a32d0 0x7fcb600a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.986+0000 7fcb6bfff640 1 --2- 172.21.15.139:0/3283529695 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcb600a32d0 0x7fcb600a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.986+0000 7fcb70d99640 1 -- 172.21.15.139:0/3283529695 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fcb640f0840 con 0x7fcb74100a80 2024-09-16T01:39:14.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.986+0000 7fcb70d99640 1 --2- 172.21.15.139:0/3283529695 >> [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361] conn(0x7fcb600ae6a0 0x7fcb600b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.986+0000 7fcb70d99640 1 -- 172.21.15.139:0/3283529695 --> [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcb600b1210 con 0x7fcb600ae6a0 2024-09-16T01:39:14.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.986+0000 7fcb70d99640 1 -- 172.21.15.139:0/3283529695 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fcb640f0c90 con 0x7fcb74100a80 2024-09-16T01:39:14.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.986+0000 7fcb6bfff640 1 --2- 172.21.15.139:0/3283529695 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcb600a32d0 0x7fcb600a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c005fd0 tx=0x7fcb5c005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:14.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.987+0000 7fcb7359e640 1 --2- 172.21.15.139:0/3283529695 >> [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361] conn(0x7fcb600ae6a0 0x7fcb600b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.987+0000 7fcb7359e640 1 --2- 172.21.15.139:0/3283529695 >> [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361] conn(0x7fcb600ae6a0 0x7fcb600b0b40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:14.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:13.988+0000 7fcb70d99640 1 -- 172.21.15.139:0/3283529695 <== osd.8 v2:172.21.15.176:6816/593133361 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcb600b1210 con 0x7fcb600ae6a0 2024-09-16T01:39:14.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.001+0000 7fcb7921d640 1 -- 172.21.15.139:0/3283529695 --> [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcb74062e00 con 0x7fcb600ae6a0 2024-09-16T01:39:14.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.002+0000 7fcb70d99640 1 -- 172.21.15.139:0/3283529695 <== osd.8 v2:172.21.15.176:6816/593133361 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcb74062e00 con 0x7fcb600ae6a0 2024-09-16T01:39:14.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.002+0000 7fcb7921d640 1 -- 172.21.15.139:0/3283529695 >> [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361] conn(0x7fcb600ae6a0 msgr2=0x7fcb600b0b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.002+0000 7fcb7921d640 1 --2- 172.21.15.139:0/3283529695 >> [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361] conn(0x7fcb600ae6a0 0x7fcb600b0b40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.002+0000 7fcb7921d640 1 -- 172.21.15.139:0/3283529695 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcb600a32d0 msgr2=0x7fcb600a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.002+0000 7fcb7921d640 1 --2- 172.21.15.139:0/3283529695 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcb600a32d0 0x7fcb600a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c005fd0 tx=0x7fcb5c005e20 comp rx=0 tx=0).stop 2024-09-16T01:39:14.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.002+0000 7fcb7921d640 1 -- 172.21.15.139:0/3283529695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb74100a80 msgr2=0x7fcb74198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.002+0000 7fcb7921d640 1 --2- 172.21.15.139:0/3283529695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb74100a80 0x7fcb74198460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009230 tx=0x7fcb64004270 comp rx=0 tx=0).stop 2024-09-16T01:39:14.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.002+0000 7fcb7359e640 1 -- 172.21.15.139:0/3283529695 reap_dead start 2024-09-16T01:39:14.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.002+0000 7fcb7921d640 1 -- 172.21.15.139:0/3283529695 shutdown_connections 2024-09-16T01:39:14.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.002+0000 7fcb7921d640 1 -- 172.21.15.139:0/3283529695 >> 172.21.15.139:0/3283529695 conn(0x7fcb740fc7a0 msgr2=0x7fcb7406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:14.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.002+0000 7fcb7921d640 1 -- 172.21.15.139:0/3283529695 shutdown_connections 2024-09-16T01:39:14.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.002+0000 7fcb7921d640 1 -- 172.21.15.139:0/3283529695 wait complete. 2024-09-16T01:39:14.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:14 smithi139 ceph-mon[31175]: pgmap v318: 129 pgs: 1 activating, 128 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T01:39:14.716 INFO:teuthology.orchestra.run.smithi139.stdout:223338299435 2024-09-16T01:39:14.717 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-16T01:39:14.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:14 smithi164 ceph-mon[35121]: pgmap v318: 129 pgs: 1 activating, 128 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T01:39:14.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.865+0000 7fa873df6640 1 -- 172.21.15.139:0/3596290506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa86c100e20 msgr2=0x7fa86c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.865+0000 7fa873df6640 1 --2- 172.21.15.139:0/3596290506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa86c100e20 0x7fa86c101200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009900 tx=0x7fa85c02a480 comp rx=0 tx=0).stop 2024-09-16T01:39:14.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.865+0000 7fa873df6640 1 -- 172.21.15.139:0/3596290506 shutdown_connections 2024-09-16T01:39:14.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.865+0000 7fa873df6640 1 --2- 172.21.15.139:0/3596290506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa86c102ef0 0x7fa86c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.865+0000 7fa873df6640 1 --2- 172.21.15.139:0/3596290506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa86c102570 0x7fa86c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.865+0000 7fa873df6640 1 --2- 172.21.15.139:0/3596290506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa86c100e20 0x7fa86c101200 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.865+0000 7fa873df6640 1 -- 172.21.15.139:0/3596290506 >> 172.21.15.139:0/3596290506 conn(0x7fa86c0fc7d0 msgr2=0x7fa86c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:14.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.865+0000 7fa873df6640 1 -- 172.21.15.139:0/3596290506 shutdown_connections 2024-09-16T01:39:14.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.865+0000 7fa873df6640 1 -- 172.21.15.139:0/3596290506 wait complete. 2024-09-16T01:39:14.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.865+0000 7fa873df6640 1 Processor -- start 2024-09-16T01:39:14.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.865+0000 7fa873df6640 1 -- start start 2024-09-16T01:39:14.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa873df6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa86c100e20 0x7fa86c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa873df6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa86c102570 0x7fa86c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa873df6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa86c102ef0 0x7fa86c19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa873df6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c110f80 con 0x7fa86c102ef0 2024-09-16T01:39:14.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa873df6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c110e00 con 0x7fa86c100e20 2024-09-16T01:39:14.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa873df6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c111100 con 0x7fa86c102570 2024-09-16T01:39:14.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa872393640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa86c102ef0 0x7fa86c19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa871391640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa86c102570 0x7fa86c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa872393640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa86c102ef0 0x7fa86c19ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46068/0 (socket says 172.21.15.139:46068) 2024-09-16T01:39:14.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa872393640 1 -- 172.21.15.139:0/3158192087 learned_addr learned my addr 172.21.15.139:0/3158192087 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:14.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa871b92640 1 --2- 172.21.15.139:0/3158192087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa86c100e20 0x7fa86c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa872393640 1 -- 172.21.15.139:0/3158192087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa86c102570 msgr2=0x7fa86c198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa872393640 1 --2- 172.21.15.139:0/3158192087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa86c102570 0x7fa86c198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa872393640 1 -- 172.21.15.139:0/3158192087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa86c100e20 msgr2=0x7fa86c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa872393640 1 --2- 172.21.15.139:0/3158192087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa86c100e20 0x7fa86c198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa872393640 1 -- 172.21.15.139:0/3158192087 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86c19d530 con 0x7fa86c102ef0 2024-09-16T01:39:14.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa872393640 1 --2- 172.21.15.139:0/3158192087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa86c102ef0 0x7fa86c19ce50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa86400cad0 tx=0x7fa86400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:14.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa862ffd640 1 -- 172.21.15.139:0/3158192087 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86400ae20 con 0x7fa86c102ef0 2024-09-16T01:39:14.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa873df6640 1 -- 172.21.15.139:0/3158192087 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86c19d820 con 0x7fa86c102ef0 2024-09-16T01:39:14.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa862ffd640 1 -- 172.21.15.139:0/3158192087 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa864005020 con 0x7fa86c102ef0 2024-09-16T01:39:14.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa862ffd640 1 -- 172.21.15.139:0/3158192087 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86400f6f0 con 0x7fa86c102ef0 2024-09-16T01:39:14.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.866+0000 7fa873df6640 1 -- 172.21.15.139:0/3158192087 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa86c1a5100 con 0x7fa86c102ef0 2024-09-16T01:39:14.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.868+0000 7fa873df6640 1 -- 172.21.15.139:0/3158192087 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa86c106bf0 con 0x7fa86c102ef0 2024-09-16T01:39:14.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.869+0000 7fa862ffd640 1 -- 172.21.15.139:0/3158192087 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fa8640051d0 con 0x7fa86c102ef0 2024-09-16T01:39:14.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.869+0000 7fa862ffd640 1 --2- 172.21.15.139:0/3158192087 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa8580a32d0 0x7fa8580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.869+0000 7fa871b92640 1 --2- 172.21.15.139:0/3158192087 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa8580a32d0 0x7fa8580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.869+0000 7fa862ffd640 1 -- 172.21.15.139:0/3158192087 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fa8640d0260 con 0x7fa86c102ef0 2024-09-16T01:39:14.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.870+0000 7fa862ffd640 1 --2- 172.21.15.139:0/3158192087 >> [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330] conn(0x7fa8580ae6a0 0x7fa8580b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.870+0000 7fa862ffd640 1 -- 172.21.15.139:0/3158192087 --> [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8580b1210 con 0x7fa8580ae6a0 2024-09-16T01:39:14.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.870+0000 7fa862ffd640 1 -- 172.21.15.139:0/3158192087 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fa8640d06b0 con 0x7fa86c102ef0 2024-09-16T01:39:14.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.870+0000 7fa871b92640 1 --2- 172.21.15.139:0/3158192087 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa8580a32d0 0x7fa8580a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0096c0 tx=0x7fa85c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:14.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.870+0000 7fa871391640 1 --2- 172.21.15.139:0/3158192087 >> [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330] conn(0x7fa8580ae6a0 0x7fa8580b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.871+0000 7fa871391640 1 --2- 172.21.15.139:0/3158192087 >> [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330] conn(0x7fa8580ae6a0 0x7fa8580b0b40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:14.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.871+0000 7fa862ffd640 1 -- 172.21.15.139:0/3158192087 <== osd.1 v2:172.21.15.164:6800/2699270330 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa8580b1210 con 0x7fa8580ae6a0 2024-09-16T01:39:14.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.899+0000 7fa873df6640 1 -- 172.21.15.139:0/3158192087 --> [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa86c1021b0 con 0x7fa8580ae6a0 2024-09-16T01:39:14.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.899+0000 7fa862ffd640 1 -- 172.21.15.139:0/3158192087 <== osd.1 v2:172.21.15.164:6800/2699270330 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa86c1021b0 con 0x7fa8580ae6a0 2024-09-16T01:39:14.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.900+0000 7fa873df6640 1 -- 172.21.15.139:0/3158192087 >> [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330] conn(0x7fa8580ae6a0 msgr2=0x7fa8580b0b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.900+0000 7fa873df6640 1 --2- 172.21.15.139:0/3158192087 >> [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330] conn(0x7fa8580ae6a0 0x7fa8580b0b40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.900+0000 7fa873df6640 1 -- 172.21.15.139:0/3158192087 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa8580a32d0 msgr2=0x7fa8580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.900+0000 7fa873df6640 1 --2- 172.21.15.139:0/3158192087 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fa8580a32d0 0x7fa8580a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0096c0 tx=0x7fa85c000bc0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.900+0000 7fa873df6640 1 -- 172.21.15.139:0/3158192087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa86c102ef0 msgr2=0x7fa86c19ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.900+0000 7fa873df6640 1 --2- 172.21.15.139:0/3158192087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa86c102ef0 0x7fa86c19ce50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa86400cad0 tx=0x7fa86400cf90 comp rx=0 tx=0).stop 2024-09-16T01:39:14.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.900+0000 7fa872393640 1 -- 172.21.15.139:0/3158192087 reap_dead start 2024-09-16T01:39:14.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.900+0000 7fa873df6640 1 -- 172.21.15.139:0/3158192087 shutdown_connections 2024-09-16T01:39:14.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.900+0000 7fa873df6640 1 -- 172.21.15.139:0/3158192087 >> 172.21.15.139:0/3158192087 conn(0x7fa86c0fc7d0 msgr2=0x7fa86c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:14.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.900+0000 7fa873df6640 1 -- 172.21.15.139:0/3158192087 shutdown_connections 2024-09-16T01:39:14.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:14.900+0000 7fa873df6640 1 -- 172.21.15.139:0/3158192087 wait complete. 2024-09-16T01:39:14.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:14 smithi176 ceph-mon[35324]: pgmap v318: 129 pgs: 1 activating, 128 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T01:39:14.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.236+0000 7f146f853640 1 -- 172.21.15.139:0/2224982977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1468103720 msgr2=0x7f146810bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.236+0000 7f146f853640 1 --2- 172.21.15.139:0/2224982977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1468103720 0x7f146810bc10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f145c009900 tx=0x7f145c02a480 comp rx=0 tx=0).stop 2024-09-16T01:39:14.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.236+0000 7f146f853640 1 -- 172.21.15.139:0/2224982977 shutdown_connections 2024-09-16T01:39:14.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.236+0000 7f146f853640 1 --2- 172.21.15.139:0/2224982977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f146810c1a0 0x7f146810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.236+0000 7f146f853640 1 --2- 172.21.15.139:0/2224982977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1468103720 0x7f146810bc10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.236+0000 7f146f853640 1 --2- 172.21.15.139:0/2224982977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1468102e00 0x7f14681031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.236+0000 7f146f853640 1 -- 172.21.15.139:0/2224982977 >> 172.21.15.139:0/2224982977 conn(0x7f14680fc7a0 msgr2=0x7f14680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:14.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.236+0000 7f146f853640 1 -- 172.21.15.139:0/2224982977 shutdown_connections 2024-09-16T01:39:14.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.236+0000 7f146f853640 1 -- 172.21.15.139:0/2224982977 wait complete. 2024-09-16T01:39:14.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146f853640 1 Processor -- start 2024-09-16T01:39:14.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146f853640 1 -- start start 2024-09-16T01:39:14.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146f853640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1468102e00 0x7f1468100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146f853640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1468103720 0x7f1468100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146f853640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f146810c1a0 0x7f14681014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146f853640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1468111fd0 con 0x7f1468102e00 2024-09-16T01:39:14.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146f853640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1468111e50 con 0x7f1468103720 2024-09-16T01:39:14.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146f853640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1468112150 con 0x7f146810c1a0 2024-09-16T01:39:14.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146d5ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1468102e00 0x7f1468100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146d5ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1468102e00 0x7f1468100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43210/0 (socket says 172.21.15.139:43210) 2024-09-16T01:39:14.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146d5ef640 1 -- 172.21.15.139:0/1414109365 learned_addr learned my addr 172.21.15.139:0/1414109365 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:14.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146ddf0640 1 --2- 172.21.15.139:0/1414109365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f146810c1a0 0x7f14681014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146cdee640 1 --2- 172.21.15.139:0/1414109365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1468103720 0x7f1468100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146d5ef640 1 -- 172.21.15.139:0/1414109365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f146810c1a0 msgr2=0x7f14681014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146d5ef640 1 --2- 172.21.15.139:0/1414109365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f146810c1a0 0x7f14681014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146d5ef640 1 -- 172.21.15.139:0/1414109365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1468103720 msgr2=0x7f1468100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146d5ef640 1 --2- 172.21.15.139:0/1414109365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1468103720 0x7f1468100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.237+0000 7f146d5ef640 1 -- 172.21.15.139:0/1414109365 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14681a6a20 con 0x7f1468102e00 2024-09-16T01:39:14.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.238+0000 7f146d5ef640 1 --2- 172.21.15.139:0/1414109365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1468102e00 0x7f1468100a60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f146400b4a0 tx=0x7f146400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:14.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.238+0000 7f145a7fc640 1 -- 172.21.15.139:0/1414109365 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464014070 con 0x7f1468102e00 2024-09-16T01:39:14.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.238+0000 7f145a7fc640 1 -- 172.21.15.139:0/1414109365 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1464010af0 con 0x7f1468102e00 2024-09-16T01:39:14.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.238+0000 7f146f853640 1 -- 172.21.15.139:0/1414109365 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14681a6d10 con 0x7f1468102e00 2024-09-16T01:39:14.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.238+0000 7f145a7fc640 1 -- 172.21.15.139:0/1414109365 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464017d10 con 0x7f1468102e00 2024-09-16T01:39:14.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.238+0000 7f146f853640 1 -- 172.21.15.139:0/1414109365 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14681a71f0 con 0x7f1468102e00 2024-09-16T01:39:14.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.239+0000 7f146f853640 1 -- 172.21.15.139:0/1414109365 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1468062880 con 0x7f1468102e00 2024-09-16T01:39:14.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.240+0000 7f145a7fc640 1 -- 172.21.15.139:0/1414109365 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1464017070 con 0x7f1468102e00 2024-09-16T01:39:14.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.241+0000 7f145a7fc640 1 --2- 172.21.15.139:0/1414109365 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f14540a32d0 0x7f14540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.241+0000 7f145a7fc640 1 -- 172.21.15.139:0/1414109365 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9208+0+0 (secure 0 0 0) 0x7f14640cf4c0 con 0x7f1468102e00 2024-09-16T01:39:14.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.241+0000 7f146cdee640 1 --2- 172.21.15.139:0/1414109365 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f14540a32d0 0x7f14540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.241+0000 7f145a7fc640 1 --2- 172.21.15.139:0/1414109365 >> [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565] conn(0x7f14540ae6a0 0x7f14540b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:14.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.241+0000 7f145a7fc640 1 -- 172.21.15.139:0/1414109365 --> [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f146400dec0 con 0x7f14540ae6a0 2024-09-16T01:39:14.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.241+0000 7f145a7fc640 1 -- 172.21.15.139:0/1414109365 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f14640cf910 con 0x7f1468102e00 2024-09-16T01:39:14.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.242+0000 7f146cdee640 1 --2- 172.21.15.139:0/1414109365 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f14540a32d0 0x7f14540a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f14680fee70 tx=0x7f145c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:14.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.242+0000 7f146ddf0640 1 --2- 172.21.15.139:0/1414109365 >> [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565] conn(0x7f14540ae6a0 0x7f14540b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:14.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.242+0000 7f146ddf0640 1 --2- 172.21.15.139:0/1414109365 >> [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565] conn(0x7f14540ae6a0 0x7f14540b0b40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:14.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.243+0000 7f145a7fc640 1 -- 172.21.15.139:0/1414109365 <== osd.11 v2:172.21.15.176:6824/1854547565 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f146400dec0 con 0x7f14540ae6a0 2024-09-16T01:39:14.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.258+0000 7f146f853640 1 -- 172.21.15.139:0/1414109365 --> [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1468062c80 con 0x7f14540ae6a0 2024-09-16T01:39:14.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.258+0000 7f145a7fc640 1 -- 172.21.15.139:0/1414109365 <== osd.11 v2:172.21.15.176:6824/1854547565 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1468062c80 con 0x7f14540ae6a0 2024-09-16T01:39:14.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.259+0000 7f146f853640 1 -- 172.21.15.139:0/1414109365 >> [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565] conn(0x7f14540ae6a0 msgr2=0x7f14540b0b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.259+0000 7f146f853640 1 --2- 172.21.15.139:0/1414109365 >> [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565] conn(0x7f14540ae6a0 0x7f14540b0b40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:14.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.259+0000 7f146f853640 1 -- 172.21.15.139:0/1414109365 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f14540a32d0 msgr2=0x7f14540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.259+0000 7f146f853640 1 --2- 172.21.15.139:0/1414109365 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f14540a32d0 0x7f14540a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f14680fee70 tx=0x7f145c005a20 comp rx=0 tx=0).stop 2024-09-16T01:39:14.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.259+0000 7f146f853640 1 -- 172.21.15.139:0/1414109365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1468102e00 msgr2=0x7f1468100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:14.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.259+0000 7f146f853640 1 --2- 172.21.15.139:0/1414109365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1468102e00 0x7f1468100a60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f146400b4a0 tx=0x7f146400b960 comp rx=0 tx=0).stop 2024-09-16T01:39:14.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.259+0000 7f146ddf0640 1 -- 172.21.15.139:0/1414109365 reap_dead start 2024-09-16T01:39:14.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.259+0000 7f146f853640 1 -- 172.21.15.139:0/1414109365 shutdown_connections 2024-09-16T01:39:14.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.259+0000 7f146f853640 1 -- 172.21.15.139:0/1414109365 >> 172.21.15.139:0/1414109365 conn(0x7f14680fc7a0 msgr2=0x7f14680fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:14.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.259+0000 7f146f853640 1 -- 172.21.15.139:0/1414109365 shutdown_connections 2024-09-16T01:39:14.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:09.259+0000 7f146f853640 1 -- 172.21.15.139:0/1414109365 wait complete. 2024-09-16T01:39:15.568 INFO:teuthology.orchestra.run.smithi139.stdout:304942678044 2024-09-16T01:39:15.568 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-16T01:39:15.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:15 smithi139 ceph-mon[31175]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-16T01:39:15.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:15 smithi139 ceph-mon[31175]: Cluster is now healthy 2024-09-16T01:39:15.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:15 smithi176 ceph-mon[35324]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-16T01:39:15.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:15 smithi176 ceph-mon[35324]: Cluster is now healthy 2024-09-16T01:39:16.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:15 smithi164 ceph-mon[35121]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-16T01:39:16.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:15 smithi164 ceph-mon[35121]: Cluster is now healthy 2024-09-16T01:39:16.315 INFO:teuthology.orchestra.run.smithi139.stdout:51539607631 2024-09-16T01:39:16.316 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T01:39:16.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.493+0000 7f7d53952640 1 -- 172.21.15.139:0/3176408131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d4c100310 msgr2=0x7f7d4c1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:16.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.493+0000 7f7d53952640 1 --2- 172.21.15.139:0/3176408131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d4c100310 0x7f7d4c1006f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009900 tx=0x7f7d4802a410 comp rx=0 tx=0).stop 2024-09-16T01:39:16.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.493+0000 7f7d53952640 1 -- 172.21.15.139:0/3176408131 shutdown_connections 2024-09-16T01:39:16.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.493+0000 7f7d53952640 1 --2- 172.21.15.139:0/3176408131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d4c106f30 0x7f7d4c10eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:16.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.493+0000 7f7d53952640 1 --2- 172.21.15.139:0/3176408131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d4c100cc0 0x7f7d4c106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:16.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.493+0000 7f7d53952640 1 --2- 172.21.15.139:0/3176408131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d4c100310 0x7f7d4c1006f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:16.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.493+0000 7f7d53952640 1 -- 172.21.15.139:0/3176408131 >> 172.21.15.139:0/3176408131 conn(0x7f7d4c077210 msgr2=0x7f7d4c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:16.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.494+0000 7f7d53952640 1 -- 172.21.15.139:0/3176408131 shutdown_connections 2024-09-16T01:39:16.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.494+0000 7f7d53952640 1 -- 172.21.15.139:0/3176408131 wait complete. 2024-09-16T01:39:16.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.494+0000 7f7d53952640 1 Processor -- start 2024-09-16T01:39:16.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d53952640 1 -- start start 2024-09-16T01:39:16.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d53952640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d4c100310 0x7f7d4c101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:16.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d53952640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d4c100cc0 0x7f7d4c1017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:16.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d53952640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d4c106f30 0x7f7d4c101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:16.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d53952640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4c111fe0 con 0x7f7d4c100310 2024-09-16T01:39:16.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d53952640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4c111e60 con 0x7f7d4c100cc0 2024-09-16T01:39:16.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d53952640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4c112160 con 0x7f7d4c106f30 2024-09-16T01:39:16.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d516ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d4c100310 0x7f7d4c101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:16.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d516ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d4c100310 0x7f7d4c101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46076/0 (socket says 172.21.15.139:46076) 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d516ee640 1 -- 172.21.15.139:0/4252795752 learned_addr learned my addr 172.21.15.139:0/4252795752 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d51eef640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d4c106f30 0x7f7d4c101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d50eed640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d4c100cc0 0x7f7d4c1017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d516ee640 1 -- 172.21.15.139:0/4252795752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d4c106f30 msgr2=0x7f7d4c101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d516ee640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d4c106f30 0x7f7d4c101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d516ee640 1 -- 172.21.15.139:0/4252795752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d4c100cc0 msgr2=0x7f7d4c1017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d516ee640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d4c100cc0 0x7f7d4c1017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d516ee640 1 -- 172.21.15.139:0/4252795752 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4c1a6900 con 0x7f7d4c100310 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.495+0000 7f7d51eef640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d4c106f30 0x7f7d4c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.496+0000 7f7d516ee640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d4c100310 0x7f7d4c101290 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7d48004770 tx=0x7f7d48003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.496+0000 7f7d427fc640 1 -- 172.21.15.139:0/4252795752 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4803f070 con 0x7f7d4c100310 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.496+0000 7f7d53952640 1 -- 172.21.15.139:0/4252795752 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4c1a6b90 con 0x7f7d4c100310 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.496+0000 7f7d427fc640 1 -- 172.21.15.139:0/4252795752 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d48031ca0 con 0x7f7d4c100310 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.496+0000 7f7d427fc640 1 -- 172.21.15.139:0/4252795752 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48035030 con 0x7f7d4c100310 2024-09-16T01:39:16.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.496+0000 7f7d53952640 1 -- 172.21.15.139:0/4252795752 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d4c1a70c0 con 0x7f7d4c100310 2024-09-16T01:39:16.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.497+0000 7f7d53952640 1 -- 172.21.15.139:0/4252795752 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d4c0628c0 con 0x7f7d4c100310 2024-09-16T01:39:16.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.498+0000 7f7d427fc640 1 -- 172.21.15.139:0/4252795752 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7d48042070 con 0x7f7d4c100310 2024-09-16T01:39:16.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.499+0000 7f7d427fc640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7d380a3350 0x7f7d380a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:16.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.499+0000 7f7d427fc640 1 -- 172.21.15.139:0/4252795752 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f7d480f08f0 con 0x7f7d4c100310 2024-09-16T01:39:16.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.499+0000 7f7d50eed640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7d380a3350 0x7f7d380a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:16.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.499+0000 7f7d427fc640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164] conn(0x7f7d380ae720 0x7f7d380b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:16.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.499+0000 7f7d427fc640 1 -- 172.21.15.139:0/4252795752 --> [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d48032e50 con 0x7f7d380ae720 2024-09-16T01:39:16.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.499+0000 7f7d427fc640 1 -- 172.21.15.139:0/4252795752 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f7d480f0d40 con 0x7f7d4c100310 2024-09-16T01:39:16.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.499+0000 7f7d51eef640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164] conn(0x7f7d380ae720 0x7f7d380b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:16.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.500+0000 7f7d50eed640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7d380a3350 0x7f7d380a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c102d20 tx=0x7f7d3c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:16.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.500+0000 7f7d51eef640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164] conn(0x7f7d380ae720 0x7f7d380b0bc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:16.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.500+0000 7f7d427fc640 1 -- 172.21.15.139:0/4252795752 <== osd.6 v2:172.21.15.139:6818/2601111164 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d48032e50 con 0x7f7d380ae720 2024-09-16T01:39:16.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.514+0000 7f7d53952640 1 -- 172.21.15.139:0/4252795752 --> [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7d4c062cc0 con 0x7f7d380ae720 2024-09-16T01:39:16.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.515+0000 7f7d427fc640 1 -- 172.21.15.139:0/4252795752 <== osd.6 v2:172.21.15.139:6818/2601111164 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7d4c062cc0 con 0x7f7d380ae720 2024-09-16T01:39:16.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.515+0000 7f7d53952640 1 -- 172.21.15.139:0/4252795752 >> [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164] conn(0x7f7d380ae720 msgr2=0x7f7d380b0bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:16.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.515+0000 7f7d53952640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164] conn(0x7f7d380ae720 0x7f7d380b0bc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:16.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.515+0000 7f7d53952640 1 -- 172.21.15.139:0/4252795752 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7d380a3350 msgr2=0x7f7d380a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:16.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.515+0000 7f7d53952640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7d380a3350 0x7f7d380a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c102d20 tx=0x7f7d3c00b390 comp rx=0 tx=0).stop 2024-09-16T01:39:16.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.515+0000 7f7d53952640 1 -- 172.21.15.139:0/4252795752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d4c100310 msgr2=0x7f7d4c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:16.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.515+0000 7f7d53952640 1 --2- 172.21.15.139:0/4252795752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d4c100310 0x7f7d4c101290 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7d48004770 tx=0x7f7d48003c10 comp rx=0 tx=0).stop 2024-09-16T01:39:16.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.515+0000 7f7d51eef640 1 -- 172.21.15.139:0/4252795752 reap_dead start 2024-09-16T01:39:16.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.515+0000 7f7d53952640 1 -- 172.21.15.139:0/4252795752 shutdown_connections 2024-09-16T01:39:16.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.515+0000 7f7d53952640 1 -- 172.21.15.139:0/4252795752 >> 172.21.15.139:0/4252795752 conn(0x7f7d4c077210 msgr2=0x7f7d4c1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:16.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.515+0000 7f7d53952640 1 -- 172.21.15.139:0/4252795752 shutdown_connections 2024-09-16T01:39:16.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:16.515+0000 7f7d53952640 1 -- 172.21.15.139:0/4252795752 wait complete. 2024-09-16T01:39:16.838 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:16 smithi176 ceph-mon[35324]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T01:39:16.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:16 smithi139 ceph-mon[31175]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T01:39:16.946 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:16 smithi164 ceph-mon[35121]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s; 14 B/s, 0 objects/s recovering 2024-09-16T01:39:17.182 INFO:teuthology.orchestra.run.smithi139.stdout:274877906976 2024-09-16T01:39:17.182 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-16T01:39:17.625 INFO:teuthology.orchestra.run.smithi139.stdout:176093659190 2024-09-16T01:39:17.626 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-16T01:39:18.133 INFO:teuthology.orchestra.run.smithi139.stdout:253403070502 2024-09-16T01:39:18.134 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-16T01:39:18.648 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:18 smithi139 ceph-mon[31175]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-16T01:39:18.649 INFO:teuthology.orchestra.run.smithi139.stdout:146028888123 2024-09-16T01:39:18.649 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T01:39:18.663 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:18.663 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:18.663 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:18.667 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:18.669 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:18.673 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:18.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:18 smithi176 ceph-mon[35324]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-16T01:39:19.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:18 smithi164 ceph-mon[35121]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 13 B/s, 0 objects/s recovering 2024-09-16T01:39:19.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.602+0000 7fbb81d1f640 1 -- 172.21.15.139:0/1563815118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c101420 msgr2=0x7fbb7c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:19.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.602+0000 7fbb81d1f640 1 --2- 172.21.15.139:0/1563815118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c101420 0x7fbb7c10d950 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009930 tx=0x7fbb6c02a420 comp rx=0 tx=0).stop 2024-09-16T01:39:19.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.602+0000 7fbb81d1f640 1 -- 172.21.15.139:0/1563815118 shutdown_connections 2024-09-16T01:39:19.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.602+0000 7fbb81d1f640 1 --2- 172.21.15.139:0/1563815118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c101420 0x7fbb7c10d950 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.602+0000 7fbb81d1f640 1 --2- 172.21.15.139:0/1563815118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb7c100a80 0x7fbb7c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.602+0000 7fbb81d1f640 1 --2- 172.21.15.139:0/1563815118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c106a80 0x7fbb7c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.602+0000 7fbb81d1f640 1 -- 172.21.15.139:0/1563815118 >> 172.21.15.139:0/1563815118 conn(0x7fbb7c0fc7a0 msgr2=0x7fbb7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:19.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.602+0000 7fbb81d1f640 1 -- 172.21.15.139:0/1563815118 shutdown_connections 2024-09-16T01:39:19.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.602+0000 7fbb81d1f640 1 -- 172.21.15.139:0/1563815118 wait complete. 2024-09-16T01:39:19.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.602+0000 7fbb81d1f640 1 Processor -- start 2024-09-16T01:39:19.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.603+0000 7fbb81d1f640 1 -- start start 2024-09-16T01:39:19.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.603+0000 7fbb81d1f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c100a80 0x7fbb7c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:19.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.603+0000 7fbb81d1f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb7c101420 0x7fbb7c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:19.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.603+0000 7fbb81d1f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c106a80 0x7fbb7c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:19.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.603+0000 7fbb81d1f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c110e80 con 0x7fbb7c106a80 2024-09-16T01:39:19.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.603+0000 7fbb81d1f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c110d00 con 0x7fbb7c100a80 2024-09-16T01:39:19.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.603+0000 7fbb81d1f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c111000 con 0x7fbb7c101420 2024-09-16T01:39:19.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.603+0000 7fbb7bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c106a80 0x7fbb7c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:19.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.603+0000 7fbb7bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c106a80 0x7fbb7c19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46100/0 (socket says 172.21.15.139:46100) 2024-09-16T01:39:19.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.603+0000 7fbb7bfff640 1 -- 172.21.15.139:0/2241204131 learned_addr learned my addr 172.21.15.139:0/2241204131 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:19.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.603+0000 7fbb7b7fe640 1 --2- 172.21.15.139:0/2241204131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c100a80 0x7fbb7c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:19.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.603+0000 7fbb7affd640 1 --2- 172.21.15.139:0/2241204131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb7c101420 0x7fbb7c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:19.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.604+0000 7fbb7bfff640 1 -- 172.21.15.139:0/2241204131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb7c101420 msgr2=0x7fbb7c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:19.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.604+0000 7fbb7bfff640 1 --2- 172.21.15.139:0/2241204131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb7c101420 0x7fbb7c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.604+0000 7fbb7bfff640 1 -- 172.21.15.139:0/2241204131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c100a80 msgr2=0x7fbb7c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:19.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.604+0000 7fbb7bfff640 1 --2- 172.21.15.139:0/2241204131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c100a80 0x7fbb7c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.604+0000 7fbb7bfff640 1 -- 172.21.15.139:0/2241204131 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c19d320 con 0x7fbb7c106a80 2024-09-16T01:39:19.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.604+0000 7fbb7bfff640 1 --2- 172.21.15.139:0/2241204131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c106a80 0x7fbb7c19cc40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c005ab0 tx=0x7fbb6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:19.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.604+0000 7fbb78ff9640 1 -- 172.21.15.139:0/2241204131 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c03f030 con 0x7fbb7c106a80 2024-09-16T01:39:19.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.604+0000 7fbb81d1f640 1 -- 172.21.15.139:0/2241204131 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7c19d5b0 con 0x7fbb7c106a80 2024-09-16T01:39:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.604+0000 7fbb78ff9640 1 -- 172.21.15.139:0/2241204131 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb6c003ce0 con 0x7fbb7c106a80 2024-09-16T01:39:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.604+0000 7fbb78ff9640 1 -- 172.21.15.139:0/2241204131 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c035070 con 0x7fbb7c106a80 2024-09-16T01:39:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.604+0000 7fbb81d1f640 1 -- 172.21.15.139:0/2241204131 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb7c1a4ef0 con 0x7fbb7c106a80 2024-09-16T01:39:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.605+0000 7fbb81d1f640 1 -- 172.21.15.139:0/2241204131 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb7c1021c0 con 0x7fbb7c106a80 2024-09-16T01:39:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.606+0000 7fbb78ff9640 1 -- 172.21.15.139:0/2241204131 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbb6c042030 con 0x7fbb7c106a80 2024-09-16T01:39:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.607+0000 7fbb78ff9640 1 --2- 172.21.15.139:0/2241204131 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbb640a32d0 0x7fbb640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.607+0000 7fbb7b7fe640 1 --2- 172.21.15.139:0/2241204131 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbb640a32d0 0x7fbb640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.607+0000 7fbb78ff9640 1 -- 172.21.15.139:0/2241204131 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fbb6c0f0220 con 0x7fbb7c106a80 2024-09-16T01:39:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.607+0000 7fbb7b7fe640 1 --2- 172.21.15.139:0/2241204131 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbb640a32d0 0x7fbb640a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbb74007e40 tx=0x7fbb740075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.611+0000 7fbb78ff9640 1 -- 172.21.15.139:0/2241204131 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb6c0bc3c0 con 0x7fbb7c106a80 2024-09-16T01:39:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.786+0000 7fbb81d1f640 1 -- 172.21.15.139:0/2241204131 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbb7c1a50c0 con 0x7fbb7c106a80 2024-09-16T01:39:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.786+0000 7fbb78ff9640 1 -- 172.21.15.139:0/2241204131 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbb6c0c06c0 con 0x7fbb7c106a80 2024-09-16T01:39:19.788 INFO:teuthology.orchestra.run.smithi139.stdout:68719476811 2024-09-16T01:39:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.789+0000 7fbb81d1f640 1 -- 172.21.15.139:0/2241204131 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbb640a32d0 msgr2=0x7fbb640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.789+0000 7fbb81d1f640 1 --2- 172.21.15.139:0/2241204131 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbb640a32d0 0x7fbb640a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbb74007e40 tx=0x7fbb740075b0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.789+0000 7fbb81d1f640 1 -- 172.21.15.139:0/2241204131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c106a80 msgr2=0x7fbb7c19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.789+0000 7fbb81d1f640 1 --2- 172.21.15.139:0/2241204131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c106a80 0x7fbb7c19cc40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c005ab0 tx=0x7fbb6c0042f0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.790+0000 7fbb81d1f640 1 -- 172.21.15.139:0/2241204131 shutdown_connections 2024-09-16T01:39:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.790+0000 7fbb81d1f640 1 --2- 172.21.15.139:0/2241204131 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbb640a32d0 0x7fbb640a5790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.790+0000 7fbb81d1f640 1 --2- 172.21.15.139:0/2241204131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c106a80 0x7fbb7c19cc40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.790+0000 7fbb81d1f640 1 --2- 172.21.15.139:0/2241204131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb7c101420 0x7fbb7c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.790+0000 7fbb81d1f640 1 --2- 172.21.15.139:0/2241204131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c100a80 0x7fbb7c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.790+0000 7fbb81d1f640 1 -- 172.21.15.139:0/2241204131 >> 172.21.15.139:0/2241204131 conn(0x7fbb7c0fc7a0 msgr2=0x7fbb7c06a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.790+0000 7fbb81d1f640 1 -- 172.21.15.139:0/2241204131 shutdown_connections 2024-09-16T01:39:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.790+0000 7fbb81d1f640 1 -- 172.21.15.139:0/2241204131 wait complete. 2024-09-16T01:39:19.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.857+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/1781795833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b41008d0 msgr2=0x7ff1b4100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:19.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.857+0000 7ff1ba9a2640 1 --2- 172.21.15.139:0/1781795833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b41008d0 0x7ff1b4100d30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009930 tx=0x7ff1a402a420 comp rx=0 tx=0).stop 2024-09-16T01:39:19.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.858+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/1781795833 shutdown_connections 2024-09-16T01:39:19.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.858+0000 7ff1ba9a2640 1 --2- 172.21.15.139:0/1781795833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1b4101270 0x7ff1b410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.858+0000 7ff1ba9a2640 1 --2- 172.21.15.139:0/1781795833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b41008d0 0x7ff1b4100d30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.858+0000 7ff1ba9a2640 1 --2- 172.21.15.139:0/1781795833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1b41068d0 0x7ff1b4106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.858+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/1781795833 >> 172.21.15.139:0/1781795833 conn(0x7ff1b40fc610 msgr2=0x7ff1b40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:19.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.858+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/1781795833 shutdown_connections 2024-09-16T01:39:19.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.858+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/1781795833 wait complete. 2024-09-16T01:39:19.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.858+0000 7ff1ba9a2640 1 Processor -- start 2024-09-16T01:39:19.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.858+0000 7ff1ba9a2640 1 -- start start 2024-09-16T01:39:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba9a2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1b41008d0 0x7ff1b41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba9a2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1b4101270 0x7ff1b4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba9a2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b41068d0 0x7ff1b419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba9a2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4110cd0 con 0x7ff1b41068d0 2024-09-16T01:39:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba9a2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4110b50 con 0x7ff1b41008d0 2024-09-16T01:39:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba9a2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4110e50 con 0x7ff1b4101270 2024-09-16T01:39:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba1a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b41068d0 0x7ff1b419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1b919f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1b4101270 0x7ff1b4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1b99a0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1b41008d0 0x7ff1b41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba1a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b41068d0 0x7ff1b419cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46106/0 (socket says 172.21.15.139:46106) 2024-09-16T01:39:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1b919f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1b4101270 0x7ff1b4198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:49982/0 (socket says 172.21.15.139:49982) 2024-09-16T01:39:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba1a1640 1 -- 172.21.15.139:0/2556569324 learned_addr learned my addr 172.21.15.139:0/2556569324 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba1a1640 1 -- 172.21.15.139:0/2556569324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1b4101270 msgr2=0x7ff1b4198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:19.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba1a1640 1 --2- 172.21.15.139:0/2556569324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1b4101270 0x7ff1b4198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba1a1640 1 -- 172.21.15.139:0/2556569324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1b41008d0 msgr2=0x7ff1b41982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:19.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba1a1640 1 --2- 172.21.15.139:0/2556569324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1b41008d0 0x7ff1b41982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:19.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba1a1640 1 -- 172.21.15.139:0/2556569324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b419d270 con 0x7ff1b41068d0 2024-09-16T01:39:19.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1b99a0640 1 --2- 172.21.15.139:0/2556569324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1b41008d0 0x7ff1b41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:39:19.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1ba1a1640 1 --2- 172.21.15.139:0/2556569324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b41068d0 0x7ff1b419cb90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000d950 tx=0x7ff1b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:19.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.859+0000 7ff1aaffd640 1 -- 172.21.15.139:0/2556569324 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0017030 con 0x7ff1b41068d0 2024-09-16T01:39:19.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.860+0000 7ff1aaffd640 1 -- 172.21.15.139:0/2556569324 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1b0004020 con 0x7ff1b41068d0 2024-09-16T01:39:19.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.860+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/2556569324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b419d560 con 0x7ff1b41068d0 2024-09-16T01:39:19.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.860+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/2556569324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b41a4e40 con 0x7ff1b41068d0 2024-09-16T01:39:19.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.860+0000 7ff1aaffd640 1 -- 172.21.15.139:0/2556569324 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0003c00 con 0x7ff1b41068d0 2024-09-16T01:39:19.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.861+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/2556569324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b4102010 con 0x7ff1b41068d0 2024-09-16T01:39:19.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.862+0000 7ff1aaffd640 1 -- 172.21.15.139:0/2556569324 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff1b001a070 con 0x7ff1b41068d0 2024-09-16T01:39:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.862+0000 7ff1aaffd640 1 --2- 172.21.15.139:0/2556569324 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff1a00a31b0 0x7ff1a00a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.862+0000 7ff1b99a0640 1 --2- 172.21.15.139:0/2556569324 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff1a00a31b0 0x7ff1a00a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.862+0000 7ff1aaffd640 1 -- 172.21.15.139:0/2556569324 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7ff1b00cf610 con 0x7ff1b41068d0 2024-09-16T01:39:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.863+0000 7ff1b99a0640 1 --2- 172.21.15.139:0/2556569324 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff1a00a31b0 0x7ff1a00a5670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac005fd0 tx=0x7ff1ac009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:19.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:19.866+0000 7ff1aaffd640 1 -- 172.21.15.139:0/2556569324 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff1b0021070 con 0x7ff1b41068d0 2024-09-16T01:39:20.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.036+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/2556569324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff1b4199650 con 0x7ff1b41068d0 2024-09-16T01:39:20.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.037+0000 7ff1aaffd640 1 -- 172.21.15.139:0/2556569324 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff1b4199650 con 0x7ff1b41068d0 2024-09-16T01:39:20.039 INFO:teuthology.orchestra.run.smithi139.stdout:124554051649 2024-09-16T01:39:20.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.039+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/2556569324 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff1a00a31b0 msgr2=0x7ff1a00a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:20.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.039+0000 7ff1ba9a2640 1 --2- 172.21.15.139:0/2556569324 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff1a00a31b0 0x7ff1a00a5670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac005fd0 tx=0x7ff1ac009500 comp rx=0 tx=0).stop 2024-09-16T01:39:20.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.039+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/2556569324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b41068d0 msgr2=0x7ff1b419cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:20.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.039+0000 7ff1ba9a2640 1 --2- 172.21.15.139:0/2556569324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b41068d0 0x7ff1b419cb90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000d950 tx=0x7ff1b000de10 comp rx=0 tx=0).stop 2024-09-16T01:39:20.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.039+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/2556569324 shutdown_connections 2024-09-16T01:39:20.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.039+0000 7ff1ba9a2640 1 --2- 172.21.15.139:0/2556569324 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff1a00a31b0 0x7ff1a00a5670 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:20.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.039+0000 7ff1ba9a2640 1 --2- 172.21.15.139:0/2556569324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b41068d0 0x7ff1b419cb90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:20.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.039+0000 7ff1ba9a2640 1 --2- 172.21.15.139:0/2556569324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1b4101270 0x7ff1b4198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:20.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.039+0000 7ff1ba9a2640 1 --2- 172.21.15.139:0/2556569324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1b41008d0 0x7ff1b41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:20.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.039+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/2556569324 >> 172.21.15.139:0/2556569324 conn(0x7ff1b40fc610 msgr2=0x7ff1b406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:20.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.039+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/2556569324 shutdown_connections 2024-09-16T01:39:20.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.039+0000 7ff1ba9a2640 1 -- 172.21.15.139:0/2556569324 wait complete. 2024-09-16T01:39:20.366 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:20.724 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:20 smithi139 ceph-mon[31175]: pgmap v321: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 11 B/s, 0 objects/s recovering 2024-09-16T01:39:20.724 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:20 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2241204131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T01:39:20.724 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:20 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2556569324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T01:39:20.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.803+0000 7f21e0a9d640 1 -- 172.21.15.139:0/2525737167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21dc102570 msgr2=0x7f21dc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:20.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.803+0000 7f21e0a9d640 1 --2- 172.21.15.139:0/2525737167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21dc102570 0x7f21dc1029b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009900 tx=0x7f21c402a480 comp rx=0 tx=0).stop 2024-09-16T01:39:20.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.803+0000 7f21e0a9d640 1 -- 172.21.15.139:0/2525737167 shutdown_connections 2024-09-16T01:39:20.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.803+0000 7f21e0a9d640 1 --2- 172.21.15.139:0/2525737167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21dc102ef0 0x7f21dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:20.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.803+0000 7f21e0a9d640 1 --2- 172.21.15.139:0/2525737167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21dc102570 0x7f21dc1029b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:20.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.803+0000 7f21e0a9d640 1 --2- 172.21.15.139:0/2525737167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21dc100e20 0x7f21dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:20.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.803+0000 7f21e0a9d640 1 -- 172.21.15.139:0/2525737167 >> 172.21.15.139:0/2525737167 conn(0x7f21dc0fc7d0 msgr2=0x7f21dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:20.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.803+0000 7f21e0a9d640 1 -- 172.21.15.139:0/2525737167 shutdown_connections 2024-09-16T01:39:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.803+0000 7f21e0a9d640 1 -- 172.21.15.139:0/2525737167 wait complete. 2024-09-16T01:39:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.803+0000 7f21e0a9d640 1 Processor -- start 2024-09-16T01:39:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.803+0000 7f21e0a9d640 1 -- start start 2024-09-16T01:39:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21e0a9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21dc100e20 0x7f21dc10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21e0a9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21dc102570 0x7f21dc10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21e0a9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21dc102ef0 0x7f21dc10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21e0a9d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc1120a0 con 0x7f21dc102570 2024-09-16T01:39:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21e0a9d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc111f20 con 0x7f21dc102ef0 2024-09-16T01:39:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21e0a9d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc112220 con 0x7f21dc100e20 2024-09-16T01:39:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21d3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21dc102570 0x7f21dc10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21d3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21dc102570 0x7f21dc10ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46126/0 (socket says 172.21.15.139:46126) 2024-09-16T01:39:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21d3fff640 1 -- 172.21.15.139:0/729703791 learned_addr learned my addr 172.21.15.139:0/729703791 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21da59c640 1 --2- 172.21.15.139:0/729703791 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21dc100e20 0x7f21dc10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21dad9d640 1 --2- 172.21.15.139:0/729703791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21dc102ef0 0x7f21dc10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21d3fff640 1 -- 172.21.15.139:0/729703791 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21dc100e20 msgr2=0x7f21dc10e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21d3fff640 1 --2- 172.21.15.139:0/729703791 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21dc100e20 0x7f21dc10e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21d3fff640 1 -- 172.21.15.139:0/729703791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21dc102ef0 msgr2=0x7f21dc10ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21d3fff640 1 --2- 172.21.15.139:0/729703791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21dc102ef0 0x7f21dc10ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21d3fff640 1 -- 172.21.15.139:0/729703791 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc1a69f0 con 0x7f21dc102570 2024-09-16T01:39:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21d3fff640 1 --2- 172.21.15.139:0/729703791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21dc102570 0x7f21dc10ea10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f21c4006010 tx=0x7f21c4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.804+0000 7f21d37fe640 1 -- 172.21.15.139:0/729703791 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c403f070 con 0x7f21dc102570 2024-09-16T01:39:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.805+0000 7f21d37fe640 1 -- 172.21.15.139:0/729703791 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c4031e30 con 0x7f21dc102570 2024-09-16T01:39:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.805+0000 7f21e0a9d640 1 -- 172.21.15.139:0/729703791 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21dc1a6c80 con 0x7f21dc102570 2024-09-16T01:39:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.805+0000 7f21d37fe640 1 -- 172.21.15.139:0/729703791 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4035030 con 0x7f21dc102570 2024-09-16T01:39:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.805+0000 7f21e0a9d640 1 -- 172.21.15.139:0/729703791 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21dc1a7160 con 0x7f21dc102570 2024-09-16T01:39:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.806+0000 7f21e0a9d640 1 -- 172.21.15.139:0/729703791 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21dc101200 con 0x7f21dc102570 2024-09-16T01:39:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.807+0000 7f21d37fe640 1 -- 172.21.15.139:0/729703791 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f21c4042070 con 0x7f21dc102570 2024-09-16T01:39:20.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.807+0000 7f21d37fe640 1 --2- 172.21.15.139:0/729703791 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f21c80a32d0 0x7f21c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:20.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.807+0000 7f21da59c640 1 --2- 172.21.15.139:0/729703791 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f21c80a32d0 0x7f21c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:20.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.807+0000 7f21d37fe640 1 -- 172.21.15.139:0/729703791 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f21c40f0870 con 0x7f21dc102570 2024-09-16T01:39:20.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.808+0000 7f21da59c640 1 --2- 172.21.15.139:0/729703791 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f21c80a32d0 0x7f21c80a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f21cc005fd0 tx=0x7f21cc005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:20.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.811+0000 7f21d37fe640 1 -- 172.21.15.139:0/729703791 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f21c40bd060 con 0x7f21dc102570 2024-09-16T01:39:20.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:20 smithi176 ceph-mon[35324]: pgmap v321: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 11 B/s, 0 objects/s recovering 2024-09-16T01:39:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:20 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2241204131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T01:39:20.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:20 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2556569324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T01:39:20.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.982+0000 7f21e0a9d640 1 -- 172.21.15.139:0/729703791 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f21dc1a7330 con 0x7f21dc102570 2024-09-16T01:39:20.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.982+0000 7f21d37fe640 1 -- 172.21.15.139:0/729703791 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f21c40bc960 con 0x7f21dc102570 2024-09-16T01:39:20.984 INFO:teuthology.orchestra.run.smithi139.stdout:274877906978 2024-09-16T01:39:20.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.985+0000 7f21e0a9d640 1 -- 172.21.15.139:0/729703791 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f21c80a32d0 msgr2=0x7f21c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:20.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.985+0000 7f21e0a9d640 1 --2- 172.21.15.139:0/729703791 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f21c80a32d0 0x7f21c80a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f21cc005fd0 tx=0x7f21cc005e20 comp rx=0 tx=0).stop 2024-09-16T01:39:20.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.985+0000 7f21e0a9d640 1 -- 172.21.15.139:0/729703791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21dc102570 msgr2=0x7f21dc10ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:20.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.985+0000 7f21e0a9d640 1 --2- 172.21.15.139:0/729703791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21dc102570 0x7f21dc10ea10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f21c4006010 tx=0x7f21c4003c80 comp rx=0 tx=0).stop 2024-09-16T01:39:20.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.985+0000 7f21e0a9d640 1 -- 172.21.15.139:0/729703791 shutdown_connections 2024-09-16T01:39:20.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.985+0000 7f21e0a9d640 1 --2- 172.21.15.139:0/729703791 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f21c80a32d0 0x7f21c80a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:20.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.985+0000 7f21e0a9d640 1 --2- 172.21.15.139:0/729703791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21dc102ef0 0x7f21dc10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:20.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.985+0000 7f21e0a9d640 1 --2- 172.21.15.139:0/729703791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21dc102570 0x7f21dc10ea10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:20.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.985+0000 7f21e0a9d640 1 --2- 172.21.15.139:0/729703791 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21dc100e20 0x7f21dc10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:20.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.985+0000 7f21e0a9d640 1 -- 172.21.15.139:0/729703791 >> 172.21.15.139:0/729703791 conn(0x7f21dc0fc7d0 msgr2=0x7f21dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:20.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.985+0000 7f21e0a9d640 1 -- 172.21.15.139:0/729703791 shutdown_connections 2024-09-16T01:39:20.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:20.985+0000 7f21e0a9d640 1 -- 172.21.15.139:0/729703791 wait complete. 2024-09-16T01:39:21.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:20 smithi164 ceph-mon[35121]: pgmap v321: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 11 B/s, 0 objects/s recovering 2024-09-16T01:39:21.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:20 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2241204131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T01:39:21.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:20 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2556569324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T01:39:21.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.024+0000 7fb9c2980640 1 -- 172.21.15.139:0/800210368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc103740 msgr2=0x7fb9bc107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:21.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.024+0000 7fb9c2980640 1 --2- 172.21.15.139:0/800210368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc103740 0x7fb9bc107b80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009930 tx=0x7fb9b002a420 comp rx=0 tx=0).stop 2024-09-16T01:39:21.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.024+0000 7fb9c2980640 1 -- 172.21.15.139:0/800210368 shutdown_connections 2024-09-16T01:39:21.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.024+0000 7fb9c2980640 1 --2- 172.21.15.139:0/800210368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc108290 0x7fb9bc10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:21.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.024+0000 7fb9c2980640 1 --2- 172.21.15.139:0/800210368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc103740 0x7fb9bc107b80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:21.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.024+0000 7fb9c2980640 1 --2- 172.21.15.139:0/800210368 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9bc102d90 0x7fb9bc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:21.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.024+0000 7fb9c2980640 1 -- 172.21.15.139:0/800210368 >> 172.21.15.139:0/800210368 conn(0x7fb9bc0fc7d0 msgr2=0x7fb9bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:21.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.024+0000 7fb9c2980640 1 -- 172.21.15.139:0/800210368 shutdown_connections 2024-09-16T01:39:21.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.024+0000 7fb9c2980640 1 -- 172.21.15.139:0/800210368 wait complete. 2024-09-16T01:39:21.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9c2980640 1 Processor -- start 2024-09-16T01:39:21.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9c2980640 1 -- start start 2024-09-16T01:39:21.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9c2980640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc102d90 0x7fb9bc1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:21.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9c2980640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9bc103740 0x7fb9bc198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:21.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9c2980640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc108290 0x7fb9bc19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:21.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9c2980640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc112090 con 0x7fb9bc102d90 2024-09-16T01:39:21.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9c2980640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc111f10 con 0x7fb9bc108290 2024-09-16T01:39:21.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9c2980640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc112210 con 0x7fb9bc103740 2024-09-16T01:39:21.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9bbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc102d90 0x7fb9bc1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:21.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9bbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc102d90 0x7fb9bc1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46152/0 (socket says 172.21.15.139:46152) 2024-09-16T01:39:21.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9bbfff640 1 -- 172.21.15.139:0/3101179236 learned_addr learned my addr 172.21.15.139:0/3101179236 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:21.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9bb7fe640 1 --2- 172.21.15.139:0/3101179236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9bc103740 0x7fb9bc198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:21.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9c0f1d640 1 --2- 172.21.15.139:0/3101179236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc108290 0x7fb9bc19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:21.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9bbfff640 1 -- 172.21.15.139:0/3101179236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9bc103740 msgr2=0x7fb9bc198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:21.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9bbfff640 1 --2- 172.21.15.139:0/3101179236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9bc103740 0x7fb9bc198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:21.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9bbfff640 1 -- 172.21.15.139:0/3101179236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc108290 msgr2=0x7fb9bc19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:21.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.025+0000 7fb9bbfff640 1 --2- 172.21.15.139:0/3101179236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc108290 0x7fb9bc19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:21.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.026+0000 7fb9bbfff640 1 -- 172.21.15.139:0/3101179236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc19d5a0 con 0x7fb9bc102d90 2024-09-16T01:39:21.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.026+0000 7fb9bbfff640 1 --2- 172.21.15.139:0/3101179236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc102d90 0x7fb9bc1985f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00e9f0 tx=0x7fb9ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:21.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.026+0000 7fb9b97fa640 1 -- 172.21.15.139:0/3101179236 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac00cd80 con 0x7fb9bc102d90 2024-09-16T01:39:21.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.026+0000 7fb9b97fa640 1 -- 172.21.15.139:0/3101179236 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ac005500 con 0x7fb9bc102d90 2024-09-16T01:39:21.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.026+0000 7fb9c2980640 1 -- 172.21.15.139:0/3101179236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9bc19d890 con 0x7fb9bc102d90 2024-09-16T01:39:21.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.026+0000 7fb9b97fa640 1 -- 172.21.15.139:0/3101179236 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac010690 con 0x7fb9bc102d90 2024-09-16T01:39:21.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.026+0000 7fb9c2980640 1 -- 172.21.15.139:0/3101179236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9bc1a5170 con 0x7fb9bc102d90 2024-09-16T01:39:21.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.027+0000 7fb9c2980640 1 -- 172.21.15.139:0/3101179236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9bc06a7f0 con 0x7fb9bc102d90 2024-09-16T01:39:21.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.028+0000 7fb9b97fa640 1 -- 172.21.15.139:0/3101179236 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb9ac0046c0 con 0x7fb9bc102d90 2024-09-16T01:39:21.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.029+0000 7fb9b97fa640 1 --2- 172.21.15.139:0/3101179236 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fb9a40a32d0 0x7fb9a40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:21.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.029+0000 7fb9bb7fe640 1 --2- 172.21.15.139:0/3101179236 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fb9a40a32d0 0x7fb9a40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:21.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.029+0000 7fb9b97fa640 1 -- 172.21.15.139:0/3101179236 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fb9ac0d0540 con 0x7fb9bc102d90 2024-09-16T01:39:21.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.030+0000 7fb9bb7fe640 1 --2- 172.21.15.139:0/3101179236 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fb9a40a32d0 0x7fb9a40a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0004770 tx=0x7fb9b00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:21.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.032+0000 7fb9b97fa640 1 -- 172.21.15.139:0/3101179236 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9ac0541b0 con 0x7fb9bc102d90 2024-09-16T01:39:21.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.203+0000 7fb9c2980640 1 -- 172.21.15.139:0/3101179236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb9bc103170 con 0x7fb9bc102d90 2024-09-16T01:39:21.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.204+0000 7fb9b97fa640 1 -- 172.21.15.139:0/3101179236 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb9bc103170 con 0x7fb9bc102d90 2024-09-16T01:39:21.206 INFO:teuthology.orchestra.run.smithi139.stdout:94489280582 2024-09-16T01:39:21.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.206+0000 7fb9c2980640 1 -- 172.21.15.139:0/3101179236 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fb9a40a32d0 msgr2=0x7fb9a40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:21.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.206+0000 7fb9c2980640 1 --2- 172.21.15.139:0/3101179236 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fb9a40a32d0 0x7fb9a40a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0004770 tx=0x7fb9b00058d0 comp rx=0 tx=0).stop 2024-09-16T01:39:21.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.206+0000 7fb9c2980640 1 -- 172.21.15.139:0/3101179236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc102d90 msgr2=0x7fb9bc1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:21.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.206+0000 7fb9c2980640 1 --2- 172.21.15.139:0/3101179236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc102d90 0x7fb9bc1985f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00e9f0 tx=0x7fb9ac00edf0 comp rx=0 tx=0).stop 2024-09-16T01:39:21.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.206+0000 7fb9c2980640 1 -- 172.21.15.139:0/3101179236 shutdown_connections 2024-09-16T01:39:21.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.206+0000 7fb9c2980640 1 --2- 172.21.15.139:0/3101179236 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fb9a40a32d0 0x7fb9a40a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:21.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.206+0000 7fb9c2980640 1 --2- 172.21.15.139:0/3101179236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9bc108290 0x7fb9bc19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:21.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.206+0000 7fb9c2980640 1 --2- 172.21.15.139:0/3101179236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9bc103740 0x7fb9bc198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:21.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.206+0000 7fb9c2980640 1 --2- 172.21.15.139:0/3101179236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9bc102d90 0x7fb9bc1985f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:21.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.206+0000 7fb9c2980640 1 -- 172.21.15.139:0/3101179236 >> 172.21.15.139:0/3101179236 conn(0x7fb9bc0fc7d0 msgr2=0x7fb9bc105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:21.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.206+0000 7fb9c2980640 1 -- 172.21.15.139:0/3101179236 shutdown_connections 2024-09-16T01:39:21.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:21.206+0000 7fb9c2980640 1 -- 172.21.15.139:0/3101179236 wait complete. 2024-09-16T01:39:21.733 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:21 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/729703791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T01:39:21.733 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:21 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3101179236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T01:39:21.761 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906976 got 274877906978 for osd.10 2024-09-16T01:39:21.762 DEBUG:teuthology.parallel:result is None 2024-09-16T01:39:21.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:21 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/729703791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T01:39:21.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:21 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3101179236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T01:39:22.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:21 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/729703791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T01:39:22.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:21 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3101179236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T01:39:22.370 INFO:tasks.cephadm.ceph_manager.ceph:need seq 94489280580 got 94489280582 for osd.3 2024-09-16T01:39:22.371 DEBUG:teuthology.parallel:result is None 2024-09-16T01:39:22.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:22 smithi139 ceph-mon[31175]: pgmap v322: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T01:39:22.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:22 smithi176 ceph-mon[35324]: pgmap v322: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T01:39:23.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:22 smithi164 ceph-mon[35121]: pgmap v322: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T01:39:23.375 INFO:tasks.cephadm.ceph_manager.ceph:need seq 68719476809 got 68719476811 for osd.2 2024-09-16T01:39:23.376 DEBUG:teuthology.parallel:result is None 2024-09-16T01:39:23.485 INFO:tasks.cephadm.ceph_manager.ceph:need seq 124554051647 got 124554051649 for osd.4 2024-09-16T01:39:23.485 DEBUG:teuthology.parallel:result is None 2024-09-16T01:39:23.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.915+0000 7f39d7beb640 1 -- 172.21.15.139:0/3496235929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d0101420 msgr2=0x7f39d010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:23.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.915+0000 7f39d7beb640 1 --2- 172.21.15.139:0/3496235929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d0101420 0x7f39d010d950 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009cd0 tx=0x7f39c802a8e0 comp rx=0 tx=0).stop 2024-09-16T01:39:23.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.915+0000 7f39d7beb640 1 -- 172.21.15.139:0/3496235929 shutdown_connections 2024-09-16T01:39:23.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.915+0000 7f39d7beb640 1 --2- 172.21.15.139:0/3496235929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d0101420 0x7f39d010d950 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:23.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.915+0000 7f39d7beb640 1 --2- 172.21.15.139:0/3496235929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39d0100a80 0x7f39d0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:23.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.915+0000 7f39d7beb640 1 --2- 172.21.15.139:0/3496235929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39d0106a80 0x7f39d0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:23.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.915+0000 7f39d7beb640 1 -- 172.21.15.139:0/3496235929 >> 172.21.15.139:0/3496235929 conn(0x7f39d00fc7a0 msgr2=0x7f39d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:23.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.915+0000 7f39d7beb640 1 -- 172.21.15.139:0/3496235929 shutdown_connections 2024-09-16T01:39:23.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.915+0000 7f39d7beb640 1 -- 172.21.15.139:0/3496235929 wait complete. 2024-09-16T01:39:23.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.915+0000 7f39d7beb640 1 Processor -- start 2024-09-16T01:39:23.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.915+0000 7f39d7beb640 1 -- start start 2024-09-16T01:39:23.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d7beb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d0100a80 0x7f39d019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:23.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d7beb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39d0101420 0x7f39d019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:23.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d7beb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39d0106a80 0x7f39d019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:23.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d7beb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f39d0110f20 con 0x7f39d0100a80 2024-09-16T01:39:23.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d7beb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f39d0110da0 con 0x7f39d0101420 2024-09-16T01:39:23.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d7beb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01110a0 con 0x7f39d0106a80 2024-09-16T01:39:23.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d5987640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d0100a80 0x7f39d019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:23.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d5987640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d0100a80 0x7f39d019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47956/0 (socket says 172.21.15.139:47956) 2024-09-16T01:39:23.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d5987640 1 -- 172.21.15.139:0/2818913875 learned_addr learned my addr 172.21.15.139:0/2818913875 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:23.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d6188640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39d0106a80 0x7f39d019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:23.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d5186640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39d0101420 0x7f39d019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:23.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d5987640 1 -- 172.21.15.139:0/2818913875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39d0106a80 msgr2=0x7f39d019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:23.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d5987640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39d0106a80 0x7f39d019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:23.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d5987640 1 -- 172.21.15.139:0/2818913875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39d0101420 msgr2=0x7f39d019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:23.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d5987640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39d0101420 0x7f39d019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:23.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d5987640 1 -- 172.21.15.139:0/2818913875 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d019f6f0 con 0x7f39d0100a80 2024-09-16T01:39:23.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d6188640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39d0106a80 0x7f39d019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:23.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d5186640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39d0101420 0x7f39d019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:23.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d5987640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d0100a80 0x7f39d019a740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f39c400b4a0 tx=0x7f39c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:23.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39c2ffd640 1 -- 172.21.15.139:0/2818913875 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4014070 con 0x7f39d0100a80 2024-09-16T01:39:23.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39c2ffd640 1 -- 172.21.15.139:0/2818913875 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c4010af0 con 0x7f39d0100a80 2024-09-16T01:39:23.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d7beb640 1 -- 172.21.15.139:0/2818913875 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d019f9e0 con 0x7f39d0100a80 2024-09-16T01:39:23.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39c2ffd640 1 -- 172.21.15.139:0/2818913875 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4017d10 con 0x7f39d0100a80 2024-09-16T01:39:23.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.916+0000 7f39d7beb640 1 -- 172.21.15.139:0/2818913875 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d01a72c0 con 0x7f39d0100a80 2024-09-16T01:39:23.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.918+0000 7f39d7beb640 1 -- 172.21.15.139:0/2818913875 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d0100f60 con 0x7f39d0100a80 2024-09-16T01:39:23.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.919+0000 7f39c2ffd640 1 -- 172.21.15.139:0/2818913875 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f39c4017070 con 0x7f39d0100a80 2024-09-16T01:39:23.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.919+0000 7f39c2ffd640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f39bc0a32d0 0x7f39bc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:23.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.920+0000 7f39d5186640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f39bc0a32d0 0x7f39bc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:23.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.920+0000 7f39c2ffd640 1 -- 172.21.15.139:0/2818913875 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f39c40cf1e0 con 0x7f39d0100a80 2024-09-16T01:39:23.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.920+0000 7f39d5186640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f39bc0a32d0 0x7f39bc0a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f39cc006fd0 tx=0x7f39cc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:23.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:23.924+0000 7f39c2ffd640 1 -- 172.21.15.139:0/2818913875 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39c409bae0 con 0x7f39d0100a80 2024-09-16T01:39:23.945 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:23.945 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:23.945 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:24.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.112+0000 7f39d7beb640 1 -- 172.21.15.139:0/2818913875 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f39d01a74c0 con 0x7f39d0100a80 2024-09-16T01:39:24.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.112+0000 7f39c2ffd640 1 -- 172.21.15.139:0/2818913875 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f39c409b3e0 con 0x7f39d0100a80 2024-09-16T01:39:24.114 INFO:teuthology.orchestra.run.smithi139.stdout:304942678046 2024-09-16T01:39:24.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.114+0000 7f39d7beb640 1 -- 172.21.15.139:0/2818913875 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f39bc0a32d0 msgr2=0x7f39bc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:24.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.114+0000 7f39d7beb640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f39bc0a32d0 0x7f39bc0a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f39cc006fd0 tx=0x7f39cc008040 comp rx=0 tx=0).stop 2024-09-16T01:39:24.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.114+0000 7f39d7beb640 1 -- 172.21.15.139:0/2818913875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d0100a80 msgr2=0x7f39d019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:24.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.114+0000 7f39d7beb640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d0100a80 0x7f39d019a740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f39c400b4a0 tx=0x7f39c400b960 comp rx=0 tx=0).stop 2024-09-16T01:39:24.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.114+0000 7f39d7beb640 1 -- 172.21.15.139:0/2818913875 shutdown_connections 2024-09-16T01:39:24.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.114+0000 7f39d7beb640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f39bc0a32d0 0x7f39bc0a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.114+0000 7f39d7beb640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39d0106a80 0x7f39d019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.114+0000 7f39d7beb640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39d0101420 0x7f39d019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.114+0000 7f39d7beb640 1 --2- 172.21.15.139:0/2818913875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d0100a80 0x7f39d019a740 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.114+0000 7f39d7beb640 1 -- 172.21.15.139:0/2818913875 >> 172.21.15.139:0/2818913875 conn(0x7f39d00fc7a0 msgr2=0x7f39d006a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:24.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.115+0000 7f39d7beb640 1 -- 172.21.15.139:0/2818913875 shutdown_connections 2024-09-16T01:39:24.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.115+0000 7f39d7beb640 1 -- 172.21.15.139:0/2818913875 wait complete. 2024-09-16T01:39:24.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.136+0000 7f407fc7a640 1 -- 172.21.15.139:0/741098290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078100a60 msgr2=0x7f4078100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:24.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.136+0000 7f407fc7a640 1 --2- 172.21.15.139:0/741098290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078100a60 0x7f4078100ec0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f406c009cd0 tx=0x7f406c02a8e0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.136+0000 7f407fc7a640 1 -- 172.21.15.139:0/741098290 shutdown_connections 2024-09-16T01:39:24.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.136+0000 7f407fc7a640 1 --2- 172.21.15.139:0/741098290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4078101400 0x7f407810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.136+0000 7f407fc7a640 1 --2- 172.21.15.139:0/741098290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078100a60 0x7f4078100ec0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.136+0000 7f407fc7a640 1 --2- 172.21.15.139:0/741098290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4078106a60 0x7f4078106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.136+0000 7f407fc7a640 1 -- 172.21.15.139:0/741098290 >> 172.21.15.139:0/741098290 conn(0x7f40780fc780 msgr2=0x7f40780feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:24.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.136+0000 7f407fc7a640 1 -- 172.21.15.139:0/741098290 shutdown_connections 2024-09-16T01:39:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.136+0000 7f407fc7a640 1 -- 172.21.15.139:0/741098290 wait complete. 2024-09-16T01:39:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.136+0000 7f407fc7a640 1 Processor -- start 2024-09-16T01:39:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.137+0000 7f407fc7a640 1 -- start start 2024-09-16T01:39:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.137+0000 7f407fc7a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4078100a60 0x7f4078197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.137+0000 7f407fc7a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078101400 0x7f4078197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.137+0000 7f407fc7a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4078106a60 0x7f4078197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.137+0000 7f407fc7a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4078110e20 con 0x7f4078101400 2024-09-16T01:39:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.137+0000 7f407fc7a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4078110ca0 con 0x7f4078100a60 2024-09-16T01:39:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.137+0000 7f407fc7a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4078110fa0 con 0x7f4078106a60 2024-09-16T01:39:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.137+0000 7f407d215640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078101400 0x7f4078197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.137+0000 7f407e217640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4078106a60 0x7f4078197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.137+0000 7f407da16640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4078100a60 0x7f4078197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.137+0000 7f407d215640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078101400 0x7f4078197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47978/0 (socket says 172.21.15.139:47978) 2024-09-16T01:39:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.137+0000 7f407d215640 1 -- 172.21.15.139:0/844369691 learned_addr learned my addr 172.21.15.139:0/844369691 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.138+0000 7f407da16640 1 -- 172.21.15.139:0/844369691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4078106a60 msgr2=0x7f4078197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.138+0000 7f407da16640 1 --2- 172.21.15.139:0/844369691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4078106a60 0x7f4078197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.138+0000 7f407da16640 1 -- 172.21.15.139:0/844369691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078101400 msgr2=0x7f4078197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.138+0000 7f407da16640 1 --2- 172.21.15.139:0/844369691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078101400 0x7f4078197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.138+0000 7f407da16640 1 -- 172.21.15.139:0/844369691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40781a68b0 con 0x7f4078100a60 2024-09-16T01:39:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.138+0000 7f407d215640 1 --2- 172.21.15.139:0/844369691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078101400 0x7f4078197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:39:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.138+0000 7f407da16640 1 --2- 172.21.15.139:0/844369691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4078100a60 0x7f4078197040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f406800c970 tx=0x7f406800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.138+0000 7f4076ffd640 1 -- 172.21.15.139:0/844369691 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068007bb0 con 0x7f4078100a60 2024-09-16T01:39:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.138+0000 7f4076ffd640 1 -- 172.21.15.139:0/844369691 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4068007d50 con 0x7f4078100a60 2024-09-16T01:39:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.138+0000 7f407fc7a640 1 -- 172.21.15.139:0/844369691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40781a6ba0 con 0x7f4078100a60 2024-09-16T01:39:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.138+0000 7f4076ffd640 1 -- 172.21.15.139:0/844369691 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068018820 con 0x7f4078100a60 2024-09-16T01:39:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.138+0000 7f407fc7a640 1 -- 172.21.15.139:0/844369691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40781a7130 con 0x7f4078100a60 2024-09-16T01:39:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.140+0000 7f407fc7a640 1 -- 172.21.15.139:0/844369691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4078100ec0 con 0x7f4078100a60 2024-09-16T01:39:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.141+0000 7f4076ffd640 1 -- 172.21.15.139:0/844369691 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4068018a80 con 0x7f4078100a60 2024-09-16T01:39:24.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.142+0000 7f4076ffd640 1 --2- 172.21.15.139:0/844369691 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f40600a3350 0x7f40600a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:24.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.142+0000 7f4076ffd640 1 -- 172.21.15.139:0/844369691 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f40680cfd50 con 0x7f4078100a60 2024-09-16T01:39:24.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.142+0000 7f407d215640 1 --2- 172.21.15.139:0/844369691 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f40600a3350 0x7f40600a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:24.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.143+0000 7f407d215640 1 --2- 172.21.15.139:0/844369691 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f40600a3350 0x7f40600a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f40781991a0 tx=0x7f406c0317e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:24.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.147+0000 7f4076ffd640 1 -- 172.21.15.139:0/844369691 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4068014030 con 0x7f4078100a60 2024-09-16T01:39:24.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.323+0000 7f407fc7a640 1 -- 172.21.15.139:0/844369691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f40781a7520 con 0x7f4078100a60 2024-09-16T01:39:24.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.323+0000 7f4076ffd640 1 -- 172.21.15.139:0/844369691 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f406809c540 con 0x7f4078100a60 2024-09-16T01:39:24.325 INFO:teuthology.orchestra.run.smithi139.stdout:223338299437 2024-09-16T01:39:24.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.326+0000 7f407fc7a640 1 -- 172.21.15.139:0/844369691 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f40600a3350 msgr2=0x7f40600a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:24.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.326+0000 7f407fc7a640 1 --2- 172.21.15.139:0/844369691 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f40600a3350 0x7f40600a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f40781991a0 tx=0x7f406c0317e0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.326+0000 7f407fc7a640 1 -- 172.21.15.139:0/844369691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4078100a60 msgr2=0x7f4078197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:24.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.326+0000 7f407fc7a640 1 --2- 172.21.15.139:0/844369691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4078100a60 0x7f4078197040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f406800c970 tx=0x7f406800ce30 comp rx=0 tx=0).stop 2024-09-16T01:39:24.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.326+0000 7f407fc7a640 1 -- 172.21.15.139:0/844369691 shutdown_connections 2024-09-16T01:39:24.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.326+0000 7f407fc7a640 1 --2- 172.21.15.139:0/844369691 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f40600a3350 0x7f40600a5810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.326+0000 7f407fc7a640 1 --2- 172.21.15.139:0/844369691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4078106a60 0x7f4078197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.326+0000 7f407fc7a640 1 --2- 172.21.15.139:0/844369691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4078101400 0x7f4078197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.326+0000 7f407fc7a640 1 --2- 172.21.15.139:0/844369691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4078100a60 0x7f4078197040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:24.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.326+0000 7f407fc7a640 1 -- 172.21.15.139:0/844369691 >> 172.21.15.139:0/844369691 conn(0x7f40780fc780 msgr2=0x7f40780fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:24.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.326+0000 7f407fc7a640 1 -- 172.21.15.139:0/844369691 shutdown_connections 2024-09-16T01:39:24.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:24.326+0000 7f407fc7a640 1 -- 172.21.15.139:0/844369691 wait complete. 2024-09-16T01:39:24.830 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:24 smithi139 ceph-mon[31175]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:39:24.830 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:24 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2818913875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T01:39:24.830 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:24 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/844369691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T01:39:24.853 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678044 got 304942678046 for osd.11 2024-09-16T01:39:24.854 DEBUG:teuthology.parallel:result is None 2024-09-16T01:39:24.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:24 smithi176 ceph-mon[35324]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:39:24.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:24 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2818913875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T01:39:24.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:24 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/844369691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T01:39:25.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:24 smithi164 ceph-mon[35121]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:39:25.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:24 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2818913875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T01:39:25.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:24 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/844369691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T01:39:25.090 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:25.569 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299435 got 223338299437 for osd.8 2024-09-16T01:39:25.570 DEBUG:teuthology.parallel:result is None 2024-09-16T01:39:26.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.376+0000 7f0f3d93a640 1 -- 172.21.15.139:0/4100376213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f38106a70 msgr2=0x7f0f38106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:26.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.376+0000 7f0f3d93a640 1 --2- 172.21.15.139:0/4100376213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f38106a70 0x7f0f38106e50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009930 tx=0x7f0f2802a420 comp rx=0 tx=0).stop 2024-09-16T01:39:26.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.376+0000 7f0f3d93a640 1 -- 172.21.15.139:0/4100376213 shutdown_connections 2024-09-16T01:39:26.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.376+0000 7f0f3d93a640 1 --2- 172.21.15.139:0/4100376213 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f38101410 0x7f0f3810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.376+0000 7f0f3d93a640 1 --2- 172.21.15.139:0/4100376213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f38100a70 0x7f0f38100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.376+0000 7f0f3d93a640 1 --2- 172.21.15.139:0/4100376213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f38106a70 0x7f0f38106e50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.376+0000 7f0f3d93a640 1 -- 172.21.15.139:0/4100376213 >> 172.21.15.139:0/4100376213 conn(0x7f0f380fc790 msgr2=0x7f0f380febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:26.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.376+0000 7f0f3d93a640 1 -- 172.21.15.139:0/4100376213 shutdown_connections 2024-09-16T01:39:26.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.376+0000 7f0f3d93a640 1 -- 172.21.15.139:0/4100376213 wait complete. 2024-09-16T01:39:26.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.377+0000 7f0f3d93a640 1 Processor -- start 2024-09-16T01:39:26.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.377+0000 7f0f3d93a640 1 -- start start 2024-09-16T01:39:26.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.377+0000 7f0f3d93a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f38100a70 0x7f0f381984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:26.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.377+0000 7f0f3d93a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f38101410 0x7f0f381989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:26.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.377+0000 7f0f3d93a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f38106a70 0x7f0f3819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:26.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.377+0000 7f0f3d93a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0f38110e70 con 0x7f0f38100a70 2024-09-16T01:39:26.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.377+0000 7f0f3d93a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0f38110cf0 con 0x7f0f38106a70 2024-09-16T01:39:26.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.377+0000 7f0f3d93a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0f38110ff0 con 0x7f0f38101410 2024-09-16T01:39:26.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.377+0000 7f0f36ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f38100a70 0x7f0f381984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:26.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.377+0000 7f0f36ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f38100a70 0x7f0f381984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47990/0 (socket says 172.21.15.139:47990) 2024-09-16T01:39:26.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.377+0000 7f0f36ffd640 1 -- 172.21.15.139:0/1073468271 learned_addr learned my addr 172.21.15.139:0/1073468271 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:26.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.378+0000 7f0f377fe640 1 --2- 172.21.15.139:0/1073468271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f38106a70 0x7f0f3819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:26.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.378+0000 7f0f367fc640 1 --2- 172.21.15.139:0/1073468271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f38101410 0x7f0f381989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:26.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.378+0000 7f0f36ffd640 1 -- 172.21.15.139:0/1073468271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f38101410 msgr2=0x7f0f381989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:26.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.378+0000 7f0f36ffd640 1 --2- 172.21.15.139:0/1073468271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f38101410 0x7f0f381989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.378+0000 7f0f36ffd640 1 -- 172.21.15.139:0/1073468271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f38106a70 msgr2=0x7f0f3819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:26.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.378+0000 7f0f36ffd640 1 --2- 172.21.15.139:0/1073468271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f38106a70 0x7f0f3819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.378+0000 7f0f36ffd640 1 -- 172.21.15.139:0/1073468271 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3819d460 con 0x7f0f38100a70 2024-09-16T01:39:26.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.378+0000 7f0f36ffd640 1 --2- 172.21.15.139:0/1073468271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f38100a70 0x7f0f381984b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009230 tx=0x7f0f28004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:26.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.378+0000 7f0f3c938640 1 -- 172.21.15.139:0/1073468271 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2803f030 con 0x7f0f38100a70 2024-09-16T01:39:26.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.378+0000 7f0f3d93a640 1 -- 172.21.15.139:0/1073468271 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3819d6f0 con 0x7f0f38100a70 2024-09-16T01:39:26.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.378+0000 7f0f3c938640 1 -- 172.21.15.139:0/1073468271 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f28003ad0 con 0x7f0f38100a70 2024-09-16T01:39:26.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.378+0000 7f0f3c938640 1 -- 172.21.15.139:0/1073468271 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28035070 con 0x7f0f38100a70 2024-09-16T01:39:26.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.378+0000 7f0f3d93a640 1 -- 172.21.15.139:0/1073468271 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f381a5030 con 0x7f0f38100a70 2024-09-16T01:39:26.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.379+0000 7f0f3d93a640 1 -- 172.21.15.139:0/1073468271 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f38100ed0 con 0x7f0f38100a70 2024-09-16T01:39:26.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.380+0000 7f0f3c938640 1 -- 172.21.15.139:0/1073468271 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0f28042030 con 0x7f0f38100a70 2024-09-16T01:39:26.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.381+0000 7f0f3c938640 1 --2- 172.21.15.139:0/1073468271 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0f200a32d0 0x7f0f200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:26.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.381+0000 7f0f367fc640 1 --2- 172.21.15.139:0/1073468271 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0f200a32d0 0x7f0f200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:26.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.381+0000 7f0f3c938640 1 -- 172.21.15.139:0/1073468271 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f0f280f08f0 con 0x7f0f38100a70 2024-09-16T01:39:26.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.381+0000 7f0f367fc640 1 --2- 172.21.15.139:0/1073468271 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0f200a32d0 0x7f0f200a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0f380fed10 tx=0x7f0f2c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:26.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.384+0000 7f0f3c938640 1 -- 172.21.15.139:0/1073468271 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f280bd0e0 con 0x7f0f38100a70 2024-09-16T01:39:26.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.555+0000 7f0f3d93a640 1 -- 172.21.15.139:0/1073468271 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0f381a5200 con 0x7f0f38100a70 2024-09-16T01:39:26.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.555+0000 7f0f3c938640 1 -- 172.21.15.139:0/1073468271 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0f280bc9e0 con 0x7f0f38100a70 2024-09-16T01:39:26.557 INFO:teuthology.orchestra.run.smithi139.stdout:176093659192 2024-09-16T01:39:26.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.557+0000 7f0f3d93a640 1 -- 172.21.15.139:0/1073468271 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0f200a32d0 msgr2=0x7f0f200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:26.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.557+0000 7f0f3d93a640 1 --2- 172.21.15.139:0/1073468271 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0f200a32d0 0x7f0f200a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0f380fed10 tx=0x7f0f2c005f90 comp rx=0 tx=0).stop 2024-09-16T01:39:26.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.557+0000 7f0f3d93a640 1 -- 172.21.15.139:0/1073468271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f38100a70 msgr2=0x7f0f381984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:26.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.557+0000 7f0f3d93a640 1 --2- 172.21.15.139:0/1073468271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f38100a70 0x7f0f381984b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009230 tx=0x7f0f28004270 comp rx=0 tx=0).stop 2024-09-16T01:39:26.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.557+0000 7f0f3d93a640 1 -- 172.21.15.139:0/1073468271 shutdown_connections 2024-09-16T01:39:26.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.557+0000 7f0f3d93a640 1 --2- 172.21.15.139:0/1073468271 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0f200a32d0 0x7f0f200a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.557+0000 7f0f3d93a640 1 --2- 172.21.15.139:0/1073468271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f38106a70 0x7f0f3819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.557+0000 7f0f3d93a640 1 --2- 172.21.15.139:0/1073468271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f38101410 0x7f0f381989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.557+0000 7f0f3d93a640 1 --2- 172.21.15.139:0/1073468271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f38100a70 0x7f0f381984b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.557+0000 7f0f3d93a640 1 -- 172.21.15.139:0/1073468271 >> 172.21.15.139:0/1073468271 conn(0x7f0f380fc790 msgr2=0x7f0f3806a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:26.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.557+0000 7f0f3d93a640 1 -- 172.21.15.139:0/1073468271 shutdown_connections 2024-09-16T01:39:26.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.557+0000 7f0f3d93a640 1 -- 172.21.15.139:0/1073468271 wait complete. 2024-09-16T01:39:26.810 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:26 smithi139 ceph-mon[31175]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:39:26.810 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:26 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1073468271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T01:39:26.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.888+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2243602100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bcc101420 msgr2=0x7f3bcc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:26.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.888+0000 7f3bd45b8640 1 --2- 172.21.15.139:0/2243602100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bcc101420 0x7f3bcc10d950 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009cd0 tx=0x7f3bc402a8e0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.888+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2243602100 shutdown_connections 2024-09-16T01:39:26.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.888+0000 7f3bd45b8640 1 --2- 172.21.15.139:0/2243602100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bcc101420 0x7f3bcc10d950 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.888+0000 7f3bd45b8640 1 --2- 172.21.15.139:0/2243602100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bcc100a80 0x7f3bcc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.888+0000 7f3bd45b8640 1 --2- 172.21.15.139:0/2243602100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bcc106a80 0x7f3bcc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.888+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2243602100 >> 172.21.15.139:0/2243602100 conn(0x7f3bcc0fc7a0 msgr2=0x7f3bcc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:26.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.888+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2243602100 shutdown_connections 2024-09-16T01:39:26.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.888+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2243602100 wait complete. 2024-09-16T01:39:26.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd45b8640 1 Processor -- start 2024-09-16T01:39:26.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd45b8640 1 -- start start 2024-09-16T01:39:26.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd45b8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bcc100a80 0x7f3bcc197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:26.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd45b8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bcc101420 0x7f3bcc197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:26.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd45b8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bcc106a80 0x7f3bcc197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:26.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd45b8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc110e80 con 0x7f3bcc101420 2024-09-16T01:39:26.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd45b8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc110d00 con 0x7f3bcc106a80 2024-09-16T01:39:26.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd45b8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc111000 con 0x7f3bcc100a80 2024-09-16T01:39:26.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd1b53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bcc101420 0x7f3bcc197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:26.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd1b53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bcc101420 0x7f3bcc197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48010/0 (socket says 172.21.15.139:48010) 2024-09-16T01:39:26.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd1b53640 1 -- 172.21.15.139:0/2880782869 learned_addr learned my addr 172.21.15.139:0/2880782869 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:26.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd2354640 1 --2- 172.21.15.139:0/2880782869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bcc100a80 0x7f3bcc197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:26.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd2b55640 1 --2- 172.21.15.139:0/2880782869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bcc106a80 0x7f3bcc197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:26.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd1b53640 1 -- 172.21.15.139:0/2880782869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bcc100a80 msgr2=0x7f3bcc197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:26.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd1b53640 1 --2- 172.21.15.139:0/2880782869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bcc100a80 0x7f3bcc197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd1b53640 1 -- 172.21.15.139:0/2880782869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bcc106a80 msgr2=0x7f3bcc197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:26.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd1b53640 1 --2- 172.21.15.139:0/2880782869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bcc106a80 0x7f3bcc197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:26.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd1b53640 1 -- 172.21.15.139:0/2880782869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc1a6870 con 0x7f3bcc101420 2024-09-16T01:39:26.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.889+0000 7f3bd1b53640 1 --2- 172.21.15.139:0/2880782869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bcc101420 0x7f3bcc197540 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00d950 tx=0x7f3bbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:26.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.890+0000 7f3bc37fe640 1 -- 172.21.15.139:0/2880782869 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc017070 con 0x7f3bcc101420 2024-09-16T01:39:26.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.890+0000 7f3bc37fe640 1 -- 172.21.15.139:0/2880782869 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bbc009c30 con 0x7f3bcc101420 2024-09-16T01:39:26.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.890+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2880782869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bcc1a6b60 con 0x7f3bcc101420 2024-09-16T01:39:26.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.890+0000 7f3bc37fe640 1 -- 172.21.15.139:0/2880782869 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00f6b0 con 0x7f3bcc101420 2024-09-16T01:39:26.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.890+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2880782869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bcc1a70a0 con 0x7f3bcc101420 2024-09-16T01:39:26.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.891+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2880782869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bcc100ee0 con 0x7f3bcc101420 2024-09-16T01:39:26.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.892+0000 7f3bc37fe640 1 -- 172.21.15.139:0/2880782869 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3bbc0109d0 con 0x7f3bcc101420 2024-09-16T01:39:26.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.892+0000 7f3bc37fe640 1 --2- 172.21.15.139:0/2880782869 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3bb80a32d0 0x7f3bb80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:26.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.892+0000 7f3bd2354640 1 --2- 172.21.15.139:0/2880782869 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3bb80a32d0 0x7f3bb80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:26.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.892+0000 7f3bc37fe640 1 -- 172.21.15.139:0/2880782869 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f3bbc0cfac0 con 0x7f3bcc101420 2024-09-16T01:39:26.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.893+0000 7f3bd2354640 1 --2- 172.21.15.139:0/2880782869 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3bb80a32d0 0x7f3bb80a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800bb60 tx=0x7f3bc800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:26.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:26.897+0000 7f3bc37fe640 1 -- 172.21.15.139:0/2880782869 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3bbc009de0 con 0x7f3bcc101420 2024-09-16T01:39:26.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:26 smithi176 ceph-mon[35324]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:39:26.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:26 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1073468271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T01:39:27.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:26 smithi164 ceph-mon[35121]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:39:27.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:26 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1073468271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T01:39:27.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.066+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2880782869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3bcc1a7410 con 0x7f3bcc101420 2024-09-16T01:39:27.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.067+0000 7f3bc37fe640 1 -- 172.21.15.139:0/2880782869 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3bbc09c2b0 con 0x7f3bcc101420 2024-09-16T01:39:27.069 INFO:teuthology.orchestra.run.smithi139.stdout:34359738454 2024-09-16T01:39:27.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.070+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2880782869 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3bb80a32d0 msgr2=0x7f3bb80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:27.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.070+0000 7f3bd45b8640 1 --2- 172.21.15.139:0/2880782869 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3bb80a32d0 0x7f3bb80a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800bb60 tx=0x7f3bc800b390 comp rx=0 tx=0).stop 2024-09-16T01:39:27.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.070+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2880782869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bcc101420 msgr2=0x7f3bcc197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:27.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.070+0000 7f3bd45b8640 1 --2- 172.21.15.139:0/2880782869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bcc101420 0x7f3bcc197540 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00d950 tx=0x7f3bbc00de10 comp rx=0 tx=0).stop 2024-09-16T01:39:27.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.070+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2880782869 shutdown_connections 2024-09-16T01:39:27.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.070+0000 7f3bd45b8640 1 --2- 172.21.15.139:0/2880782869 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3bb80a32d0 0x7f3bb80a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:27.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.070+0000 7f3bd45b8640 1 --2- 172.21.15.139:0/2880782869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bcc106a80 0x7f3bcc197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:27.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.070+0000 7f3bd45b8640 1 --2- 172.21.15.139:0/2880782869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bcc101420 0x7f3bcc197540 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:27.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.070+0000 7f3bd45b8640 1 --2- 172.21.15.139:0/2880782869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bcc100a80 0x7f3bcc197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:27.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.070+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2880782869 >> 172.21.15.139:0/2880782869 conn(0x7f3bcc0fc7a0 msgr2=0x7f3bcc0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:27.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.071+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2880782869 shutdown_connections 2024-09-16T01:39:27.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.071+0000 7f3bd45b8640 1 -- 172.21.15.139:0/2880782869 wait complete. 2024-09-16T01:39:27.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.097+0000 7f46b34f4640 1 -- 172.21.15.139:0/2203406614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac102ef0 msgr2=0x7f46ac10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:27.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.097+0000 7f46b34f4640 1 --2- 172.21.15.139:0/2203406614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac102ef0 0x7f46ac10da40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009ca0 tx=0x7f46a402a950 comp rx=0 tx=0).stop 2024-09-16T01:39:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.097+0000 7f46b34f4640 1 -- 172.21.15.139:0/2203406614 shutdown_connections 2024-09-16T01:39:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.097+0000 7f46b34f4640 1 --2- 172.21.15.139:0/2203406614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac102ef0 0x7f46ac10da40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.097+0000 7f46b34f4640 1 --2- 172.21.15.139:0/2203406614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46ac102570 0x7f46ac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.097+0000 7f46b34f4640 1 --2- 172.21.15.139:0/2203406614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46ac100e20 0x7f46ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.097+0000 7f46b34f4640 1 -- 172.21.15.139:0/2203406614 >> 172.21.15.139:0/2203406614 conn(0x7f46ac0fc7d0 msgr2=0x7f46ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.098+0000 7f46b34f4640 1 -- 172.21.15.139:0/2203406614 shutdown_connections 2024-09-16T01:39:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.098+0000 7f46b34f4640 1 -- 172.21.15.139:0/2203406614 wait complete. 2024-09-16T01:39:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.098+0000 7f46b34f4640 1 Processor -- start 2024-09-16T01:39:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.098+0000 7f46b34f4640 1 -- start start 2024-09-16T01:39:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.098+0000 7f46b34f4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46ac100e20 0x7f46ac199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.098+0000 7f46b34f4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46ac102570 0x7f46ac1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.098+0000 7f46b34f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac102ef0 0x7f46ac1a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46b34f4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac110f80 con 0x7f46ac102ef0 2024-09-16T01:39:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46b34f4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac110e00 con 0x7f46ac102570 2024-09-16T01:39:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46b34f4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac111100 con 0x7f46ac100e20 2024-09-16T01:39:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46b1a91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac102ef0 0x7f46ac1a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46b1a91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac102ef0 0x7f46ac1a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48032/0 (socket says 172.21.15.139:48032) 2024-09-16T01:39:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46b1a91640 1 -- 172.21.15.139:0/496758603 learned_addr learned my addr 172.21.15.139:0/496758603 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46b0a8f640 1 --2- 172.21.15.139:0/496758603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46ac102570 0x7f46ac1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46b1290640 1 --2- 172.21.15.139:0/496758603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46ac100e20 0x7f46ac199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46b1a91640 1 -- 172.21.15.139:0/496758603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46ac100e20 msgr2=0x7f46ac199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46b1a91640 1 --2- 172.21.15.139:0/496758603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46ac100e20 0x7f46ac199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46b1a91640 1 -- 172.21.15.139:0/496758603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46ac102570 msgr2=0x7f46ac1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46b1a91640 1 --2- 172.21.15.139:0/496758603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46ac102570 0x7f46ac1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46b1a91640 1 -- 172.21.15.139:0/496758603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac1a4930 con 0x7f46ac102ef0 2024-09-16T01:39:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46b1a91640 1 --2- 172.21.15.139:0/496758603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac102ef0 0x7f46ac1a43f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f46a402ae70 tx=0x7f46a402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46a27fc640 1 -- 172.21.15.139:0/496758603 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4034040 con 0x7f46ac102ef0 2024-09-16T01:39:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.099+0000 7f46a27fc640 1 -- 172.21.15.139:0/496758603 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a4030920 con 0x7f46ac102ef0 2024-09-16T01:39:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.100+0000 7f46a27fc640 1 -- 172.21.15.139:0/496758603 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4045480 con 0x7f46ac102ef0 2024-09-16T01:39:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.100+0000 7f46b34f4640 1 -- 172.21.15.139:0/496758603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ac1a4b00 con 0x7f46ac102ef0 2024-09-16T01:39:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.100+0000 7f46b34f4640 1 -- 172.21.15.139:0/496758603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ac1a4f00 con 0x7f46ac102ef0 2024-09-16T01:39:27.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.101+0000 7f46b34f4640 1 -- 172.21.15.139:0/496758603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ac101200 con 0x7f46ac102ef0 2024-09-16T01:39:27.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.103+0000 7f46a27fc640 1 -- 172.21.15.139:0/496758603 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f46a4037020 con 0x7f46ac102ef0 2024-09-16T01:39:27.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.103+0000 7f46a27fc640 1 --2- 172.21.15.139:0/496758603 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f46980a32d0 0x7f46980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:27.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.104+0000 7f46a27fc640 1 -- 172.21.15.139:0/496758603 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f46a40f0d20 con 0x7f46ac102ef0 2024-09-16T01:39:27.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.104+0000 7f46b1290640 1 --2- 172.21.15.139:0/496758603 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f46980a32d0 0x7f46980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:27.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.104+0000 7f46b1290640 1 --2- 172.21.15.139:0/496758603 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f46980a32d0 0x7f46980a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f46a8005fd0 tx=0x7f46a8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:27.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.106+0000 7f46a27fc640 1 -- 172.21.15.139:0/496758603 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46a40bd6b0 con 0x7f46ac102ef0 2024-09-16T01:39:27.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.275+0000 7f46b34f4640 1 -- 172.21.15.139:0/496758603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f46ac109090 con 0x7f46ac102ef0 2024-09-16T01:39:27.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.275+0000 7f46a27fc640 1 -- 172.21.15.139:0/496758603 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f46a40bcfb0 con 0x7f46ac102ef0 2024-09-16T01:39:27.277 INFO:teuthology.orchestra.run.smithi139.stdout:51539607634 2024-09-16T01:39:27.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.278+0000 7f46b34f4640 1 -- 172.21.15.139:0/496758603 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f46980a32d0 msgr2=0x7f46980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:27.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.278+0000 7f46b34f4640 1 --2- 172.21.15.139:0/496758603 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f46980a32d0 0x7f46980a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f46a8005fd0 tx=0x7f46a8005850 comp rx=0 tx=0).stop 2024-09-16T01:39:27.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.278+0000 7f46b34f4640 1 -- 172.21.15.139:0/496758603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac102ef0 msgr2=0x7f46ac1a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:27.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.278+0000 7f46b34f4640 1 --2- 172.21.15.139:0/496758603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac102ef0 0x7f46ac1a43f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f46a402ae70 tx=0x7f46a402f5d0 comp rx=0 tx=0).stop 2024-09-16T01:39:27.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.278+0000 7f46b34f4640 1 -- 172.21.15.139:0/496758603 shutdown_connections 2024-09-16T01:39:27.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.278+0000 7f46b34f4640 1 --2- 172.21.15.139:0/496758603 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f46980a32d0 0x7f46980a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:27.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.278+0000 7f46b34f4640 1 --2- 172.21.15.139:0/496758603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac102ef0 0x7f46ac1a43f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:27.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.278+0000 7f46b34f4640 1 --2- 172.21.15.139:0/496758603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46ac102570 0x7f46ac1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:27.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.278+0000 7f46b34f4640 1 --2- 172.21.15.139:0/496758603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46ac100e20 0x7f46ac199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:27.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.278+0000 7f46b34f4640 1 -- 172.21.15.139:0/496758603 >> 172.21.15.139:0/496758603 conn(0x7f46ac0fc7d0 msgr2=0x7f46ac1068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:27.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.278+0000 7f46b34f4640 1 -- 172.21.15.139:0/496758603 shutdown_connections 2024-09-16T01:39:27.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:27.278+0000 7f46b34f4640 1 -- 172.21.15.139:0/496758603 wait complete. 2024-09-16T01:39:27.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:27 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2880782869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T01:39:27.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:27 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/496758603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T01:39:27.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:27 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2880782869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T01:39:27.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:27 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/496758603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T01:39:28.023 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659190 got 176093659192 for osd.6 2024-09-16T01:39:28.023 DEBUG:teuthology.parallel:result is None 2024-09-16T01:39:28.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:27 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2880782869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T01:39:28.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:27 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/496758603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T01:39:28.479 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:28.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.654+0000 7f4ef642a640 1 -- 172.21.15.139:0/3133860094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef0101de0 msgr2=0x7f4ef01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.654+0000 7f4ef642a640 1 --2- 172.21.15.139:0/3133860094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef0101de0 0x7f4ef01021c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009900 tx=0x7f4ee002a410 comp rx=0 tx=0).stop 2024-09-16T01:39:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.654+0000 7f4ef642a640 1 -- 172.21.15.139:0/3133860094 shutdown_connections 2024-09-16T01:39:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.654+0000 7f4ef642a640 1 --2- 172.21.15.139:0/3133860094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef0106f10 0x7f4ef010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.654+0000 7f4ef642a640 1 --2- 172.21.15.139:0/3133860094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef0102790 0x7f4ef01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.654+0000 7f4ef642a640 1 --2- 172.21.15.139:0/3133860094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef0101de0 0x7f4ef01021c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.654+0000 7f4ef642a640 1 -- 172.21.15.139:0/3133860094 >> 172.21.15.139:0/3133860094 conn(0x7f4ef00fd8c0 msgr2=0x7f4ef00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.654+0000 7f4ef642a640 1 -- 172.21.15.139:0/3133860094 shutdown_connections 2024-09-16T01:39:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.654+0000 7f4ef642a640 1 -- 172.21.15.139:0/3133860094 wait complete. 2024-09-16T01:39:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.655+0000 7f4ef642a640 1 Processor -- start 2024-09-16T01:39:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.655+0000 7f4ef642a640 1 -- start start 2024-09-16T01:39:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.655+0000 7f4ef642a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef0101de0 0x7f4ef0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.655+0000 7f4ef642a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef0102790 0x7f4ef01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.655+0000 7f4ef642a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef0106f10 0x7f4ef019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.655+0000 7f4ef642a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef0111f30 con 0x7f4ef0102790 2024-09-16T01:39:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.655+0000 7f4ef642a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef0111db0 con 0x7f4ef0101de0 2024-09-16T01:39:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.655+0000 7f4ef642a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef01120b0 con 0x7f4ef0106f10 2024-09-16T01:39:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.655+0000 7f4eef7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef0102790 0x7f4ef01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.655+0000 7f4eef7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef0102790 0x7f4ef01989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48042/0 (socket says 172.21.15.139:48042) 2024-09-16T01:39:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.655+0000 7f4eef7fe640 1 -- 172.21.15.139:0/1518157731 learned_addr learned my addr 172.21.15.139:0/1518157731 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.656+0000 7f4ef49c7640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef0106f10 0x7f4ef019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.656+0000 7f4eeffff640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef0101de0 0x7f4ef0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.656+0000 7f4eef7fe640 1 -- 172.21.15.139:0/1518157731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef0106f10 msgr2=0x7f4ef019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.656+0000 7f4eef7fe640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef0106f10 0x7f4ef019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.656+0000 7f4eef7fe640 1 -- 172.21.15.139:0/1518157731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef0101de0 msgr2=0x7f4ef0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.656+0000 7f4eef7fe640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef0101de0 0x7f4ef0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.656+0000 7f4eef7fe640 1 -- 172.21.15.139:0/1518157731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef019d440 con 0x7f4ef0102790 2024-09-16T01:39:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.656+0000 7f4eeffff640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef0101de0 0x7f4ef0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.656+0000 7f4eef7fe640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef0102790 0x7f4ef01989d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400c890 tx=0x7f4ee400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.656+0000 7f4eed7fa640 1 -- 172.21.15.139:0/1518157731 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4004220 con 0x7f4ef0102790 2024-09-16T01:39:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.656+0000 7f4ef642a640 1 -- 172.21.15.139:0/1518157731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef019d730 con 0x7f4ef0102790 2024-09-16T01:39:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.656+0000 7f4eed7fa640 1 -- 172.21.15.139:0/1518157731 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ee40043c0 con 0x7f4ef0102790 2024-09-16T01:39:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.656+0000 7f4eed7fa640 1 -- 172.21.15.139:0/1518157731 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40049e0 con 0x7f4ef0102790 2024-09-16T01:39:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.656+0000 7f4ef642a640 1 -- 172.21.15.139:0/1518157731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef01a5010 con 0x7f4ef0102790 2024-09-16T01:39:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.657+0000 7f4ef642a640 1 -- 172.21.15.139:0/1518157731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef01021c0 con 0x7f4ef0102790 2024-09-16T01:39:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.658+0000 7f4eed7fa640 1 -- 172.21.15.139:0/1518157731 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4ee40079b0 con 0x7f4ef0102790 2024-09-16T01:39:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.659+0000 7f4eed7fa640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4ed80a32d0 0x7f4ed80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:28.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.659+0000 7f4eeffff640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4ed80a32d0 0x7f4ed80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:28.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.659+0000 7f4eed7fa640 1 -- 172.21.15.139:0/1518157731 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f4ee40cfed0 con 0x7f4ef0102790 2024-09-16T01:39:28.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.659+0000 7f4eeffff640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4ed80a32d0 0x7f4ed80a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0005ab0 tx=0x7f4ee0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:28.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.663+0000 7f4eed7fa640 1 -- 172.21.15.139:0/1518157731 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ee4014030 con 0x7f4ef0102790 2024-09-16T01:39:28.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.837+0000 7f4ef642a640 1 -- 172.21.15.139:0/1518157731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4ef002d7c0 con 0x7f4ef0102790 2024-09-16T01:39:28.839 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:28 smithi139 ceph-mon[31175]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:39:28.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.837+0000 7f4eed7fa640 1 -- 172.21.15.139:0/1518157731 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4ee409c6c0 con 0x7f4ef0102790 2024-09-16T01:39:28.840 INFO:teuthology.orchestra.run.smithi139.stdout:146028888126 2024-09-16T01:39:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.839+0000 7f4ef642a640 1 -- 172.21.15.139:0/1518157731 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4ed80a32d0 msgr2=0x7f4ed80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.839+0000 7f4ef642a640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4ed80a32d0 0x7f4ed80a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0005ab0 tx=0x7f4ee0005a20 comp rx=0 tx=0).stop 2024-09-16T01:39:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.839+0000 7f4ef642a640 1 -- 172.21.15.139:0/1518157731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef0102790 msgr2=0x7f4ef01989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.839+0000 7f4ef642a640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef0102790 0x7f4ef01989d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400c890 tx=0x7f4ee400cd50 comp rx=0 tx=0).stop 2024-09-16T01:39:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.839+0000 7f4ef642a640 1 -- 172.21.15.139:0/1518157731 shutdown_connections 2024-09-16T01:39:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.839+0000 7f4ef642a640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4ed80a32d0 0x7f4ed80a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.839+0000 7f4ef642a640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef0106f10 0x7f4ef019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.839+0000 7f4ef642a640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef0102790 0x7f4ef01989d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.839+0000 7f4ef642a640 1 --2- 172.21.15.139:0/1518157731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef0101de0 0x7f4ef0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.839+0000 7f4ef642a640 1 -- 172.21.15.139:0/1518157731 >> 172.21.15.139:0/1518157731 conn(0x7f4ef00fd8c0 msgr2=0x7f4ef00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.840+0000 7f4ef642a640 1 -- 172.21.15.139:0/1518157731 shutdown_connections 2024-09-16T01:39:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:28.840+0000 7f4ef642a640 1 -- 172.21.15.139:0/1518157731 wait complete. 2024-09-16T01:39:28.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:28 smithi176 ceph-mon[35324]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:39:29.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:28 smithi164 ceph-mon[35121]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:39:29.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:29 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1518157731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T01:39:29.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:29 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1518157731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T01:39:30.024 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888123 got 146028888126 for osd.5 2024-09-16T01:39:30.024 DEBUG:teuthology.parallel:result is None 2024-09-16T01:39:30.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:29 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1518157731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T01:39:30.463 INFO:tasks.cephadm.ceph_manager.ceph:need seq 51539607631 got 51539607634 for osd.1 2024-09-16T01:39:30.464 DEBUG:teuthology.parallel:result is None 2024-09-16T01:39:30.680 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:30 smithi139 ceph-mon[31175]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T01:39:30.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.787+0000 7f0d31514640 1 -- 172.21.15.139:0/3197979374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d2c1036e0 msgr2=0x7f0d2c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:30.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.787+0000 7f0d31514640 1 --2- 172.21.15.139:0/3197979374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d2c1036e0 0x7f0d2c107b90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009930 tx=0x7f0d2002a420 comp rx=0 tx=0).stop 2024-09-16T01:39:30.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.787+0000 7f0d31514640 1 -- 172.21.15.139:0/3197979374 shutdown_connections 2024-09-16T01:39:30.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.787+0000 7f0d31514640 1 --2- 172.21.15.139:0/3197979374 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d2c1082a0 0x7f0d2c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.787+0000 7f0d31514640 1 --2- 172.21.15.139:0/3197979374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d2c1036e0 0x7f0d2c107b90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.787+0000 7f0d31514640 1 --2- 172.21.15.139:0/3197979374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d2c102d30 0x7f0d2c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.787+0000 7f0d31514640 1 -- 172.21.15.139:0/3197979374 >> 172.21.15.139:0/3197979374 conn(0x7f0d2c0fc7d0 msgr2=0x7f0d2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:30.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.787+0000 7f0d31514640 1 -- 172.21.15.139:0/3197979374 shutdown_connections 2024-09-16T01:39:30.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.787+0000 7f0d31514640 1 -- 172.21.15.139:0/3197979374 wait complete. 2024-09-16T01:39:30.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.788+0000 7f0d31514640 1 Processor -- start 2024-09-16T01:39:30.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.788+0000 7f0d31514640 1 -- start start 2024-09-16T01:39:30.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.788+0000 7f0d31514640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d2c102d30 0x7f0d2c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:30.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.788+0000 7f0d31514640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d2c1036e0 0x7f0d2c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:30.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d31514640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d2c1082a0 0x7f0d2c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:30.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d31514640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c1120a0 con 0x7f0d2c102d30 2024-09-16T01:39:30.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d31514640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c111f20 con 0x7f0d2c1082a0 2024-09-16T01:39:30.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d31514640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c112220 con 0x7f0d2c1036e0 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d2affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d2c102d30 0x7f0d2c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d2affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d2c102d30 0x7f0d2c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48084/0 (socket says 172.21.15.139:48084) 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d2affd640 1 -- 172.21.15.139:0/3278431634 learned_addr learned my addr 172.21.15.139:0/3278431634 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d2a7fc640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d2c1036e0 0x7f0d2c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d2b7fe640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d2c1082a0 0x7f0d2c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d2affd640 1 -- 172.21.15.139:0/3278431634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d2c1036e0 msgr2=0x7f0d2c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d2affd640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d2c1036e0 0x7f0d2c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d2affd640 1 -- 172.21.15.139:0/3278431634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d2c1082a0 msgr2=0x7f0d2c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d2affd640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d2c1082a0 0x7f0d2c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d2affd640 1 -- 172.21.15.139:0/3278431634 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2c1a6b00 con 0x7f0d2c102d30 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d2b7fe640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d2c1082a0 0x7f0d2c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d2a7fc640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d2c1036e0 0x7f0d2c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d2affd640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d2c102d30 0x7f0d2c100990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00d950 tx=0x7f0d1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d13fff640 1 -- 172.21.15.139:0/3278431634 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c017070 con 0x7f0d2c102d30 2024-09-16T01:39:30.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d31514640 1 -- 172.21.15.139:0/3278431634 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2c1a6df0 con 0x7f0d2c102d30 2024-09-16T01:39:30.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d13fff640 1 -- 172.21.15.139:0/3278431634 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1c009c30 con 0x7f0d2c102d30 2024-09-16T01:39:30.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.789+0000 7f0d13fff640 1 -- 172.21.15.139:0/3278431634 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c00f6b0 con 0x7f0d2c102d30 2024-09-16T01:39:30.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.790+0000 7f0d31514640 1 -- 172.21.15.139:0/3278431634 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d2c1a7250 con 0x7f0d2c102d30 2024-09-16T01:39:30.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.790+0000 7f0d31514640 1 -- 172.21.15.139:0/3278431634 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d2c103110 con 0x7f0d2c102d30 2024-09-16T01:39:30.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.792+0000 7f0d13fff640 1 -- 172.21.15.139:0/3278431634 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0d1c0109d0 con 0x7f0d2c102d30 2024-09-16T01:39:30.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.792+0000 7f0d13fff640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0d140a32d0 0x7f0d140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:30.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.792+0000 7f0d2a7fc640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0d140a32d0 0x7f0d140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:30.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.793+0000 7f0d13fff640 1 -- 172.21.15.139:0/3278431634 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f0d1c0cfb10 con 0x7f0d2c102d30 2024-09-16T01:39:30.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.793+0000 7f0d2a7fc640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0d140a32d0 0x7f0d140a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009900 tx=0x7f0d20005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:30.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.800+0000 7f0d13fff640 1 -- 172.21.15.139:0/3278431634 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d1c009de0 con 0x7f0d2c102d30 2024-09-16T01:39:30.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.676+0000 7f1d3d332640 1 -- 172.21.15.139:0/3784172656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d38103660 msgr2=0x7f1d3810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:30.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.676+0000 7f1d3d332640 1 --2- 172.21.15.139:0/3784172656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d38103660 0x7f1d3810bb50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009900 tx=0x7f1d2c02a480 comp rx=0 tx=0).stop 2024-09-16T01:39:30.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.676+0000 7f1d3d332640 1 -- 172.21.15.139:0/3784172656 shutdown_connections 2024-09-16T01:39:30.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.676+0000 7f1d3d332640 1 --2- 172.21.15.139:0/3784172656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d3810c170 0x7f1d3810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.676+0000 7f1d3d332640 1 --2- 172.21.15.139:0/3784172656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d38103660 0x7f1d3810bb50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.676+0000 7f1d3d332640 1 --2- 172.21.15.139:0/3784172656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d38102d40 0x7f1d38103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.676+0000 7f1d3d332640 1 -- 172.21.15.139:0/3784172656 >> 172.21.15.139:0/3784172656 conn(0x7f1d380fc7a0 msgr2=0x7f1d380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:30.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.676+0000 7f1d3d332640 1 -- 172.21.15.139:0/3784172656 shutdown_connections 2024-09-16T01:39:30.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.676+0000 7f1d3d332640 1 -- 172.21.15.139:0/3784172656 wait complete. 2024-09-16T01:39:30.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.677+0000 7f1d3d332640 1 Processor -- start 2024-09-16T01:39:30.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.677+0000 7f1d3d332640 1 -- start start 2024-09-16T01:39:30.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.677+0000 7f1d3d332640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d38102d40 0x7f1d38198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:30.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.677+0000 7f1d3d332640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d38103660 0x7f1d38198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:30.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.677+0000 7f1d3d332640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d3810c170 0x7f1d3819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:30.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.677+0000 7f1d3d332640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1d38111f70 con 0x7f1d38102d40 2024-09-16T01:39:30.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.677+0000 7f1d3d332640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1d38111df0 con 0x7f1d38103660 2024-09-16T01:39:30.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.677+0000 7f1d3d332640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1d381120f0 con 0x7f1d3810c170 2024-09-16T01:39:30.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.677+0000 7f1d36ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d38102d40 0x7f1d38198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:30.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.677+0000 7f1d36ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d38102d40 0x7f1d38198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48060/0 (socket says 172.21.15.139:48060) 2024-09-16T01:39:30.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.677+0000 7f1d36ffd640 1 -- 172.21.15.139:0/1528361263 learned_addr learned my addr 172.21.15.139:0/1528361263 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:30.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.678+0000 7f1d367fc640 1 --2- 172.21.15.139:0/1528361263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d38103660 0x7f1d38198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:30.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.678+0000 7f1d377fe640 1 --2- 172.21.15.139:0/1528361263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d3810c170 0x7f1d3819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:30.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.678+0000 7f1d36ffd640 1 -- 172.21.15.139:0/1528361263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d3810c170 msgr2=0x7f1d3819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:30.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.678+0000 7f1d36ffd640 1 --2- 172.21.15.139:0/1528361263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d3810c170 0x7f1d3819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.678+0000 7f1d36ffd640 1 -- 172.21.15.139:0/1528361263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d38103660 msgr2=0x7f1d38198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:30.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.678+0000 7f1d36ffd640 1 --2- 172.21.15.139:0/1528361263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d38103660 0x7f1d38198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.678+0000 7f1d36ffd640 1 -- 172.21.15.139:0/1528361263 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3819d400 con 0x7f1d38102d40 2024-09-16T01:39:30.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.678+0000 7f1d36ffd640 1 --2- 172.21.15.139:0/1528361263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d38102d40 0x7f1d38198450 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1d2800c890 tx=0x7f1d2800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:30.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.678+0000 7f1d1ffff640 1 -- 172.21.15.139:0/1528361263 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28004220 con 0x7f1d38102d40 2024-09-16T01:39:30.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.678+0000 7f1d3d332640 1 -- 172.21.15.139:0/1528361263 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3819d6f0 con 0x7f1d38102d40 2024-09-16T01:39:30.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.678+0000 7f1d1ffff640 1 -- 172.21.15.139:0/1528361263 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d280043c0 con 0x7f1d38102d40 2024-09-16T01:39:30.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.678+0000 7f1d3d332640 1 -- 172.21.15.139:0/1528361263 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d381a4fd0 con 0x7f1d38102d40 2024-09-16T01:39:30.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.678+0000 7f1d1ffff640 1 -- 172.21.15.139:0/1528361263 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2801d060 con 0x7f1d38102d40 2024-09-16T01:39:30.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.679+0000 7f1d3d332640 1 -- 172.21.15.139:0/1528361263 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d38103120 con 0x7f1d38102d40 2024-09-16T01:39:30.927 INFO:teuthology.orchestra.run.smithi139.stdout:253403070506 2024-09-16T01:39:30.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.681+0000 7f1d1ffff640 1 -- 172.21.15.139:0/1528361263 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1d280079b0 con 0x7f1d38102d40 2024-09-16T01:39:30.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.681+0000 7f1d1ffff640 1 --2- 172.21.15.139:0/1528361263 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1d200a76e0 0x7f1d200a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:30.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.681+0000 7f1d367fc640 1 --2- 172.21.15.139:0/1528361263 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1d200a76e0 0x7f1d200a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:30.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.681+0000 7f1d1ffff640 1 -- 172.21.15.139:0/1528361263 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f1d280cffc0 con 0x7f1d38102d40 2024-09-16T01:39:30.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.682+0000 7f1d367fc640 1 --2- 172.21.15.139:0/1528361263 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1d200a76e0 0x7f1d200a9ba0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009230 tx=0x7f1d2c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:30.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.684+0000 7f1d1ffff640 1 -- 172.21.15.139:0/1528361263 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d28014030 con 0x7f1d38102d40 2024-09-16T01:39:30.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.863+0000 7f1d3d332640 1 -- 172.21.15.139:0/1528361263 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f1d38106c50 con 0x7f1d38102d40 2024-09-16T01:39:30.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.863+0000 7f1d1ffff640 1 -- 172.21.15.139:0/1528361263 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1d2809c7b0 con 0x7f1d38102d40 2024-09-16T01:39:30.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.865+0000 7f1d3d332640 1 -- 172.21.15.139:0/1528361263 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1d200a76e0 msgr2=0x7f1d200a9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:30.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.865+0000 7f1d3d332640 1 --2- 172.21.15.139:0/1528361263 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1d200a76e0 0x7f1d200a9ba0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009230 tx=0x7f1d2c0058d0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.865+0000 7f1d3d332640 1 -- 172.21.15.139:0/1528361263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d38102d40 msgr2=0x7f1d38198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:30.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.865+0000 7f1d3d332640 1 --2- 172.21.15.139:0/1528361263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d38102d40 0x7f1d38198450 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1d2800c890 tx=0x7f1d2800cd50 comp rx=0 tx=0).stop 2024-09-16T01:39:30.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.865+0000 7f1d3d332640 1 -- 172.21.15.139:0/1528361263 shutdown_connections 2024-09-16T01:39:30.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.865+0000 7f1d3d332640 1 --2- 172.21.15.139:0/1528361263 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f1d200a76e0 0x7f1d200a9ba0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.865+0000 7f1d3d332640 1 --2- 172.21.15.139:0/1528361263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d3810c170 0x7f1d3819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.865+0000 7f1d3d332640 1 --2- 172.21.15.139:0/1528361263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d38103660 0x7f1d38198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.865+0000 7f1d3d332640 1 --2- 172.21.15.139:0/1528361263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d38102d40 0x7f1d38198450 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.865+0000 7f1d3d332640 1 -- 172.21.15.139:0/1528361263 >> 172.21.15.139:0/1528361263 conn(0x7f1d380fc7a0 msgr2=0x7f1d380fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:30.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.865+0000 7f1d3d332640 1 -- 172.21.15.139:0/1528361263 shutdown_connections 2024-09-16T01:39:30.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.865+0000 7f1d3d332640 1 -- 172.21.15.139:0/1528361263 wait complete. 2024-09-16T01:39:30.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:30 smithi176 ceph-mon[35324]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T01:39:30.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.974+0000 7f0d31514640 1 -- 172.21.15.139:0/3278431634 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0d2c101ec0 con 0x7f0d2c102d30 2024-09-16T01:39:30.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.974+0000 7f0d13fff640 1 -- 172.21.15.139:0/3278431634 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0d1c09c300 con 0x7f0d2c102d30 2024-09-16T01:39:30.976 INFO:teuthology.orchestra.run.smithi139.stdout:193273528371 2024-09-16T01:39:30.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.976+0000 7f0d31514640 1 -- 172.21.15.139:0/3278431634 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0d140a32d0 msgr2=0x7f0d140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:30.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.976+0000 7f0d31514640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0d140a32d0 0x7f0d140a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009900 tx=0x7f0d20005a20 comp rx=0 tx=0).stop 2024-09-16T01:39:30.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.976+0000 7f0d31514640 1 -- 172.21.15.139:0/3278431634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d2c102d30 msgr2=0x7f0d2c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:30.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.976+0000 7f0d31514640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d2c102d30 0x7f0d2c100990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00d950 tx=0x7f0d1c00de10 comp rx=0 tx=0).stop 2024-09-16T01:39:30.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.976+0000 7f0d31514640 1 -- 172.21.15.139:0/3278431634 shutdown_connections 2024-09-16T01:39:30.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.976+0000 7f0d31514640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f0d140a32d0 0x7f0d140a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.976+0000 7f0d31514640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d2c1082a0 0x7f0d2c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.976+0000 7f0d31514640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d2c1036e0 0x7f0d2c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.976+0000 7f0d31514640 1 --2- 172.21.15.139:0/3278431634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d2c102d30 0x7f0d2c100990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:30.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.976+0000 7f0d31514640 1 -- 172.21.15.139:0/3278431634 >> 172.21.15.139:0/3278431634 conn(0x7f0d2c0fc7d0 msgr2=0x7f0d2c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:30.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.976+0000 7f0d31514640 1 -- 172.21.15.139:0/3278431634 shutdown_connections 2024-09-16T01:39:30.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:30.976+0000 7f0d31514640 1 -- 172.21.15.139:0/3278431634 wait complete. 2024-09-16T01:39:31.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:30 smithi164 ceph-mon[35121]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T01:39:31.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:31 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1528361263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T01:39:31.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:31 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3278431634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T01:39:31.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:31 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1528361263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T01:39:31.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:31 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3278431634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T01:39:31.986 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738450 got 34359738454 for osd.0 2024-09-16T01:39:31.986 DEBUG:teuthology.parallel:result is None 2024-09-16T01:39:32.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:31 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1528361263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T01:39:32.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:31 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3278431634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T01:39:32.502 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070502 got 253403070506 for osd.9 2024-09-16T01:39:32.502 DEBUG:teuthology.parallel:result is None 2024-09-16T01:39:32.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:32 smithi139 ceph-mon[31175]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-16T01:39:32.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:32 smithi176 ceph-mon[35324]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-16T01:39:33.018 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528366 got 193273528371 for osd.7 2024-09-16T01:39:33.018 DEBUG:teuthology.parallel:result is None 2024-09-16T01:39:33.018 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T01:39:33.018 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T01:39:33.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:32 smithi164 ceph-mon[35121]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-16T01:39:33.272 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:33.661 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:33 smithi139 ceph-mon[31175]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:39:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.738+0000 7fdc7155a640 1 -- 172.21.15.139:0/4153765265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c10c2f0 msgr2=0x7fdc6c10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.738+0000 7fdc7155a640 1 --2- 172.21.15.139:0/4153765265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c10c2f0 0x7fdc6c10e790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009900 tx=0x7fdc5c02a410 comp rx=0 tx=0).stop 2024-09-16T01:39:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.738+0000 7fdc7155a640 1 -- 172.21.15.139:0/4153765265 shutdown_connections 2024-09-16T01:39:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.738+0000 7fdc7155a640 1 --2- 172.21.15.139:0/4153765265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c10c2f0 0x7fdc6c10e790 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.738+0000 7fdc7155a640 1 --2- 172.21.15.139:0/4153765265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc6c1037a0 0x7fdc6c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.738+0000 7fdc7155a640 1 --2- 172.21.15.139:0/4153765265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc6c102df0 0x7fdc6c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.738+0000 7fdc7155a640 1 -- 172.21.15.139:0/4153765265 >> 172.21.15.139:0/4153765265 conn(0x7fdc6c0fc7d0 msgr2=0x7fdc6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.738+0000 7fdc7155a640 1 -- 172.21.15.139:0/4153765265 shutdown_connections 2024-09-16T01:39:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.738+0000 7fdc7155a640 1 -- 172.21.15.139:0/4153765265 wait complete. 2024-09-16T01:39:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.738+0000 7fdc7155a640 1 Processor -- start 2024-09-16T01:39:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.738+0000 7fdc7155a640 1 -- start start 2024-09-16T01:39:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc7155a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c102df0 0x7fdc6c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc7155a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc6c1037a0 0x7fdc6c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc7155a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc6c10c2f0 0x7fdc6c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc7155a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c112150 con 0x7fdc6c102df0 2024-09-16T01:39:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc7155a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c111fd0 con 0x7fdc6c10c2f0 2024-09-16T01:39:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc7155a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c1122d0 con 0x7fdc6c1037a0 2024-09-16T01:39:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc6affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c102df0 0x7fdc6c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc6affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c102df0 0x7fdc6c198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39612/0 (socket says 172.21.15.139:39612) 2024-09-16T01:39:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc6affd640 1 -- 172.21.15.139:0/249208207 learned_addr learned my addr 172.21.15.139:0/249208207 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc6a7fc640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc6c1037a0 0x7fdc6c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc6b7fe640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc6c10c2f0 0x7fdc6c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc6affd640 1 -- 172.21.15.139:0/249208207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc6c1037a0 msgr2=0x7fdc6c198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc6affd640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc6c1037a0 0x7fdc6c198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc6affd640 1 -- 172.21.15.139:0/249208207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc6c10c2f0 msgr2=0x7fdc6c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:33.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc6affd640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc6c10c2f0 0x7fdc6c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:33.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc6affd640 1 -- 172.21.15.139:0/249208207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6c19d5c0 con 0x7fdc6c102df0 2024-09-16T01:39:33.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc6b7fe640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc6c10c2f0 0x7fdc6c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:33.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc6affd640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c102df0 0x7fdc6c198610 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400cce0 tx=0x7fdc6400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:33.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.739+0000 7fdc63fff640 1 -- 172.21.15.139:0/249208207 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6400f450 con 0x7fdc6c102df0 2024-09-16T01:39:33.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.740+0000 7fdc7155a640 1 -- 172.21.15.139:0/249208207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6c19d8b0 con 0x7fdc6c102df0 2024-09-16T01:39:33.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.740+0000 7fdc63fff640 1 -- 172.21.15.139:0/249208207 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc6400fad0 con 0x7fdc6c102df0 2024-09-16T01:39:33.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.740+0000 7fdc63fff640 1 -- 172.21.15.139:0/249208207 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64018b80 con 0x7fdc6c102df0 2024-09-16T01:39:33.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.740+0000 7fdc7155a640 1 -- 172.21.15.139:0/249208207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc6c1a5190 con 0x7fdc6c102df0 2024-09-16T01:39:33.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.741+0000 7fdc7155a640 1 -- 172.21.15.139:0/249208207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc6c1031d0 con 0x7fdc6c102df0 2024-09-16T01:39:33.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.742+0000 7fdc63fff640 1 -- 172.21.15.139:0/249208207 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fdc6400f5f0 con 0x7fdc6c102df0 2024-09-16T01:39:33.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.742+0000 7fdc63fff640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fdc4c0a32d0 0x7fdc4c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:33.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.742+0000 7fdc6a7fc640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fdc4c0a32d0 0x7fdc4c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:33.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.742+0000 7fdc63fff640 1 -- 172.21.15.139:0/249208207 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdc640d04c0 con 0x7fdc6c102df0 2024-09-16T01:39:33.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.743+0000 7fdc6a7fc640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fdc4c0a32d0 0x7fdc4c0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c10e120 tx=0x7fdc54009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:33.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.747+0000 7fdc63fff640 1 -- 172.21.15.139:0/249208207 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc64014030 con 0x7fdc6c102df0 2024-09-16T01:39:33.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.914+0000 7fdc7155a640 1 -- 172.21.15.139:0/249208207 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdc6c062630 con 0x7fdc4c0a32d0 2024-09-16T01:39:33.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.922+0000 7fdc63fff640 1 -- 172.21.15.139:0/249208207 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340683 (secure 0 0 0) 0x7fdc6c062630 con 0x7fdc4c0a32d0 2024-09-16T01:39:33.927 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:39:33.929 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T01:39:33.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.925+0000 7fdc7155a640 1 -- 172.21.15.139:0/249208207 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fdc4c0a32d0 msgr2=0x7fdc4c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:33.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.925+0000 7fdc7155a640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fdc4c0a32d0 0x7fdc4c0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c10e120 tx=0x7fdc54009110 comp rx=0 tx=0).stop 2024-09-16T01:39:33.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.925+0000 7fdc7155a640 1 -- 172.21.15.139:0/249208207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c102df0 msgr2=0x7fdc6c198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:33.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.925+0000 7fdc7155a640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c102df0 0x7fdc6c198610 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400cce0 tx=0x7fdc6400a620 comp rx=0 tx=0).stop 2024-09-16T01:39:33.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.925+0000 7fdc7155a640 1 -- 172.21.15.139:0/249208207 shutdown_connections 2024-09-16T01:39:33.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.925+0000 7fdc7155a640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fdc4c0a32d0 0x7fdc4c0a5790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:33.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.925+0000 7fdc7155a640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc6c10c2f0 0x7fdc6c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:33.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.925+0000 7fdc7155a640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc6c1037a0 0x7fdc6c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:33.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.925+0000 7fdc7155a640 1 --2- 172.21.15.139:0/249208207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c102df0 0x7fdc6c198610 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:33.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.925+0000 7fdc7155a640 1 -- 172.21.15.139:0/249208207 >> 172.21.15.139:0/249208207 conn(0x7fdc6c0fc7d0 msgr2=0x7fdc6c0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:33.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.926+0000 7fdc7155a640 1 -- 172.21.15.139:0/249208207 shutdown_connections 2024-09-16T01:39:33.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:33.926+0000 7fdc7155a640 1 -- 172.21.15.139:0/249208207 wait complete. 2024-09-16T01:39:33.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:33 smithi176 ceph-mon[35324]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:39:34.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:33 smithi164 ceph-mon[35121]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:39:34.647 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-16T01:39:33.476077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659493,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":676,"num_read_kb":466,"num_write":431,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":352,"ondisk_log_size":352,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369280,"kb_used_data":11624,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572544,"statfs":{"total":1151940427776,"available":1151562285056,"internally_reserved":0,"allocated":11902976,"data_stored":7804349,"data_compressed":391458,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003882"},"pg_stats":[{"pgid":"2.1f","version":"83'13","reported_seq":111,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:30.642243+0000","last_change":"2024-09-16T01:38:08.567909+0000","last_active":"2024-09-16T01:39:30.642243+0000","last_peered":"2024-09-16T01:39:30.642243+0000","last_clean":"2024-09-16T01:39:30.642243+0000","last_became_active":"2024-09-16T01:38:08.560733+0000","last_became_peered":"2024-09-16T01:38:08.560733+0000","last_unstale":"2024-09-16T01:39:30.642243+0000","last_undegraded":"2024-09-16T01:39:30.642243+0000","last_fullsized":"2024-09-16T01:39:30.642243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:00.678040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518015+0000","last_change":"2024-09-16T01:38:01.523064+0000","last_active":"2024-09-16T01:39:09.518015+0000","last_peered":"2024-09-16T01:39:09.518015+0000","last_clean":"2024-09-16T01:39:09.518015+0000","last_became_active":"2024-09-16T01:38:01.521695+0000","last_became_peered":"2024-09-16T01:38:01.521695+0000","last_unstale":"2024-09-16T01:39:09.518015+0000","last_undegraded":"2024-09-16T01:39:09.518015+0000","last_fullsized":"2024-09-16T01:39:09.518015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:28.467115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.613727+0000","last_change":"2024-09-16T01:38:08.563557+0000","last_active":"2024-09-16T01:39:09.613727+0000","last_peered":"2024-09-16T01:39:09.613727+0000","last_clean":"2024-09-16T01:39:09.613727+0000","last_became_active":"2024-09-16T01:38:08.563065+0000","last_became_peered":"2024-09-16T01:38:08.563065+0000","last_unstale":"2024-09-16T01:39:09.613727+0000","last_undegraded":"2024-09-16T01:39:09.613727+0000","last_fullsized":"2024-09-16T01:39:09.613727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:56.992915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518208+0000","last_change":"2024-09-16T01:37:59.798884+0000","last_active":"2024-09-16T01:39:09.518208+0000","last_peered":"2024-09-16T01:39:09.518208+0000","last_clean":"2024-09-16T01:39:09.518208+0000","last_became_active":"2024-09-16T01:37:59.797989+0000","last_became_peered":"2024-09-16T01:37:59.797989+0000","last_unstale":"2024-09-16T01:39:09.518208+0000","last_undegraded":"2024-09-16T01:39:09.518208+0000","last_fullsized":"2024-09-16T01:39:09.518208+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:13.633078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.048261+0000","last_change":"2024-09-16T01:38:01.528391+0000","last_active":"2024-09-16T01:39:10.048261+0000","last_peered":"2024-09-16T01:39:10.048261+0000","last_clean":"2024-09-16T01:39:10.048261+0000","last_became_active":"2024-09-16T01:38:01.528076+0000","last_became_peered":"2024-09-16T01:38:01.528076+0000","last_unstale":"2024-09-16T01:39:10.048261+0000","last_undegraded":"2024-09-16T01:39:10.048261+0000","last_fullsized":"2024-09-16T01:39:10.048261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.105830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516404+0000","last_change":"2024-09-16T01:38:08.566198+0000","last_active":"2024-09-16T01:39:09.516404+0000","last_peered":"2024-09-16T01:39:09.516404+0000","last_clean":"2024-09-16T01:39:09.516404+0000","last_became_active":"2024-09-16T01:38:08.565739+0000","last_became_peered":"2024-09-16T01:38:08.565739+0000","last_unstale":"2024-09-16T01:39:09.516404+0000","last_undegraded":"2024-09-16T01:39:09.516404+0000","last_fullsized":"2024-09-16T01:39:09.516404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:05.172269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516713+0000","last_change":"2024-09-16T01:37:59.507625+0000","last_active":"2024-09-16T01:39:09.516713+0000","last_peered":"2024-09-16T01:39:09.516713+0000","last_clean":"2024-09-16T01:39:09.516713+0000","last_became_active":"2024-09-16T01:37:59.507280+0000","last_became_peered":"2024-09-16T01:37:59.507280+0000","last_unstale":"2024-09-16T01:39:09.516713+0000","last_undegraded":"2024-09-16T01:39:09.516713+0000","last_fullsized":"2024-09-16T01:39:09.516713+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:29:37.761203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639517+0000","last_change":"2024-09-16T01:38:01.528406+0000","last_active":"2024-09-16T01:39:09.639517+0000","last_peered":"2024-09-16T01:39:09.639517+0000","last_clean":"2024-09-16T01:39:09.639517+0000","last_became_active":"2024-09-16T01:38:01.528084+0000","last_became_peered":"2024-09-16T01:38:01.528084+0000","last_unstale":"2024-09-16T01:39:09.639517+0000","last_undegraded":"2024-09-16T01:39:09.639517+0000","last_fullsized":"2024-09-16T01:39:09.639517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:40.766305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516856+0000","last_change":"2024-09-16T01:38:08.563645+0000","last_active":"2024-09-16T01:39:09.516856+0000","last_peered":"2024-09-16T01:39:09.516856+0000","last_clean":"2024-09-16T01:39:09.516856+0000","last_became_active":"2024-09-16T01:38:08.563149+0000","last_became_peered":"2024-09-16T01:38:08.563149+0000","last_unstale":"2024-09-16T01:39:09.516856+0000","last_undegraded":"2024-09-16T01:39:09.516856+0000","last_fullsized":"2024-09-16T01:39:09.516856+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:09.010905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516923+0000","last_change":"2024-09-16T01:37:59.509649+0000","last_active":"2024-09-16T01:39:09.516923+0000","last_peered":"2024-09-16T01:39:09.516923+0000","last_clean":"2024-09-16T01:39:09.516923+0000","last_became_active":"2024-09-16T01:37:59.509236+0000","last_became_peered":"2024-09-16T01:37:59.509236+0000","last_unstale":"2024-09-16T01:39:09.516923+0000","last_undegraded":"2024-09-16T01:39:09.516923+0000","last_fullsized":"2024-09-16T01:39:09.516923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.614205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518130+0000","last_change":"2024-09-16T01:38:01.528074+0000","last_active":"2024-09-16T01:39:09.518130+0000","last_peered":"2024-09-16T01:39:09.518130+0000","last_clean":"2024-09-16T01:39:09.518130+0000","last_became_active":"2024-09-16T01:38:01.527737+0000","last_became_peered":"2024-09-16T01:38:01.527737+0000","last_unstale":"2024-09-16T01:39:09.518130+0000","last_undegraded":"2024-09-16T01:39:09.518130+0000","last_fullsized":"2024-09-16T01:39:09.518130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:37.592557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516662+0000","last_change":"2024-09-16T01:38:08.566340+0000","last_active":"2024-09-16T01:39:09.516662+0000","last_peered":"2024-09-16T01:39:09.516662+0000","last_clean":"2024-09-16T01:39:09.516662+0000","last_became_active":"2024-09-16T01:38:08.565791+0000","last_became_peered":"2024-09-16T01:38:08.565791+0000","last_unstale":"2024-09-16T01:39:09.516662+0000","last_undegraded":"2024-09-16T01:39:09.516662+0000","last_fullsized":"2024-09-16T01:39:09.516662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:35:31.845151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.521323+0000","last_change":"2024-09-16T01:38:01.527747+0000","last_active":"2024-09-16T01:39:09.521323+0000","last_peered":"2024-09-16T01:39:09.521323+0000","last_clean":"2024-09-16T01:39:09.521323+0000","last_became_active":"2024-09-16T01:38:01.527325+0000","last_became_peered":"2024-09-16T01:38:01.527325+0000","last_unstale":"2024-09-16T01:39:09.521323+0000","last_undegraded":"2024-09-16T01:39:09.521323+0000","last_fullsized":"2024-09-16T01:39:09.521323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:27.203719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516207+0000","last_change":"2024-09-16T01:37:59.510298+0000","last_active":"2024-09-16T01:39:09.516207+0000","last_peered":"2024-09-16T01:39:09.516207+0000","last_clean":"2024-09-16T01:39:09.516207+0000","last_became_active":"2024-09-16T01:37:59.509952+0000","last_became_peered":"2024-09-16T01:37:59.509952+0000","last_unstale":"2024-09-16T01:39:09.516207+0000","last_undegraded":"2024-09-16T01:39:09.516207+0000","last_fullsized":"2024-09-16T01:39:09.516207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:11.616846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516880+0000","last_change":"2024-09-16T01:38:08.571622+0000","last_active":"2024-09-16T01:39:09.516880+0000","last_peered":"2024-09-16T01:39:09.516880+0000","last_clean":"2024-09-16T01:39:09.516880+0000","last_became_active":"2024-09-16T01:38:08.571125+0000","last_became_peered":"2024-09-16T01:38:08.571125+0000","last_unstale":"2024-09-16T01:39:09.516880+0000","last_undegraded":"2024-09-16T01:39:09.516880+0000","last_fullsized":"2024-09-16T01:39:09.516880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:00.154633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517891+0000","last_change":"2024-09-16T01:38:01.519681+0000","last_active":"2024-09-16T01:39:09.517891+0000","last_peered":"2024-09-16T01:39:09.517891+0000","last_clean":"2024-09-16T01:39:09.517891+0000","last_became_active":"2024-09-16T01:38:01.518310+0000","last_became_peered":"2024-09-16T01:38:01.518310+0000","last_unstale":"2024-09-16T01:39:09.517891+0000","last_undegraded":"2024-09-16T01:39:09.517891+0000","last_fullsized":"2024-09-16T01:39:09.517891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:23.308008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519362+0000","last_change":"2024-09-16T01:38:08.559469+0000","last_active":"2024-09-16T01:39:09.519362+0000","last_peered":"2024-09-16T01:39:09.519362+0000","last_clean":"2024-09-16T01:39:09.519362+0000","last_became_active":"2024-09-16T01:38:08.559111+0000","last_became_peered":"2024-09-16T01:38:08.559111+0000","last_unstale":"2024-09-16T01:39:09.519362+0000","last_undegraded":"2024-09-16T01:39:09.519362+0000","last_fullsized":"2024-09-16T01:39:09.519362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:58.577749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516898+0000","last_change":"2024-09-16T01:38:08.555029+0000","last_active":"2024-09-16T01:39:09.516898+0000","last_peered":"2024-09-16T01:39:09.516898+0000","last_clean":"2024-09-16T01:39:09.516898+0000","last_became_active":"2024-09-16T01:38:08.554151+0000","last_became_peered":"2024-09-16T01:38:08.554151+0000","last_unstale":"2024-09-16T01:39:09.516898+0000","last_undegraded":"2024-09-16T01:39:09.516898+0000","last_fullsized":"2024-09-16T01:39:09.516898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:05.005904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519228+0000","last_change":"2024-09-16T01:38:01.528114+0000","last_active":"2024-09-16T01:39:09.519228+0000","last_peered":"2024-09-16T01:39:09.519228+0000","last_clean":"2024-09-16T01:39:09.519228+0000","last_became_active":"2024-09-16T01:38:01.527805+0000","last_became_peered":"2024-09-16T01:38:01.527805+0000","last_unstale":"2024-09-16T01:39:09.519228+0000","last_undegraded":"2024-09-16T01:39:09.519228+0000","last_fullsized":"2024-09-16T01:39:09.519228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:15.830936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518204+0000","last_change":"2024-09-16T01:37:59.507776+0000","last_active":"2024-09-16T01:39:09.518204+0000","last_peered":"2024-09-16T01:39:09.518204+0000","last_clean":"2024-09-16T01:39:09.518204+0000","last_became_active":"2024-09-16T01:37:59.507429+0000","last_became_peered":"2024-09-16T01:37:59.507429+0000","last_unstale":"2024-09-16T01:39:09.518204+0000","last_undegraded":"2024-09-16T01:39:09.518204+0000","last_fullsized":"2024-09-16T01:39:09.518204+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:44:41.274986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522573+0000","last_change":"2024-09-16T01:38:08.562191+0000","last_active":"2024-09-16T01:39:09.522573+0000","last_peered":"2024-09-16T01:39:09.522573+0000","last_clean":"2024-09-16T01:39:09.522573+0000","last_became_active":"2024-09-16T01:38:08.561839+0000","last_became_peered":"2024-09-16T01:38:08.561839+0000","last_unstale":"2024-09-16T01:39:09.522573+0000","last_undegraded":"2024-09-16T01:39:09.522573+0000","last_fullsized":"2024-09-16T01:39:09.522573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:40.618612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.515939+0000","last_change":"2024-09-16T01:38:01.518855+0000","last_active":"2024-09-16T01:39:09.515939+0000","last_peered":"2024-09-16T01:39:09.515939+0000","last_clean":"2024-09-16T01:39:09.515939+0000","last_became_active":"2024-09-16T01:38:01.518427+0000","last_became_peered":"2024-09-16T01:38:01.518427+0000","last_unstale":"2024-09-16T01:39:09.515939+0000","last_undegraded":"2024-09-16T01:39:09.515939+0000","last_fullsized":"2024-09-16T01:39:09.515939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:23.054856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639280+0000","last_change":"2024-09-16T01:37:59.799341+0000","last_active":"2024-09-16T01:39:09.639280+0000","last_peered":"2024-09-16T01:39:09.639280+0000","last_clean":"2024-09-16T01:39:09.639280+0000","last_became_active":"2024-09-16T01:37:59.798992+0000","last_became_peered":"2024-09-16T01:37:59.798992+0000","last_unstale":"2024-09-16T01:39:09.639280+0000","last_undegraded":"2024-09-16T01:39:09.639280+0000","last_fullsized":"2024-09-16T01:39:09.639280+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:01.708565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.524319+0000","last_change":"2024-09-16T01:39:09.524319+0000","last_active":"2024-09-16T01:39:09.524319+0000","last_peered":"2024-09-16T01:39:09.524319+0000","last_clean":"2024-09-16T01:39:09.524319+0000","last_became_active":"2024-09-16T01:39:09.523717+0000","last_became_peered":"2024-09-16T01:39:09.523717+0000","last_unstale":"2024-09-16T01:39:09.524319+0000","last_undegraded":"2024-09-16T01:39:09.524319+0000","last_fullsized":"2024-09-16T01:39:09.524319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:35.488097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.048153+0000","last_change":"2024-09-16T01:38:01.524545+0000","last_active":"2024-09-16T01:39:10.048153+0000","last_peered":"2024-09-16T01:39:10.048153+0000","last_clean":"2024-09-16T01:39:10.048153+0000","last_became_active":"2024-09-16T01:38:01.524253+0000","last_became_peered":"2024-09-16T01:38:01.524253+0000","last_unstale":"2024-09-16T01:39:10.048153+0000","last_undegraded":"2024-09-16T01:39:10.048153+0000","last_fullsized":"2024-09-16T01:39:10.048153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:08.697500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517439+0000","last_change":"2024-09-16T01:37:59.506779+0000","last_active":"2024-09-16T01:39:09.517439+0000","last_peered":"2024-09-16T01:39:09.517439+0000","last_clean":"2024-09-16T01:39:09.517439+0000","last_became_active":"2024-09-16T01:37:59.505459+0000","last_became_peered":"2024-09-16T01:37:59.505459+0000","last_unstale":"2024-09-16T01:39:09.517439+0000","last_undegraded":"2024-09-16T01:39:09.517439+0000","last_fullsized":"2024-09-16T01:39:09.517439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:12.973330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.613873+0000","last_change":"2024-09-16T01:38:08.557250+0000","last_active":"2024-09-16T01:39:09.613873+0000","last_peered":"2024-09-16T01:39:09.613873+0000","last_clean":"2024-09-16T01:39:09.613873+0000","last_became_active":"2024-09-16T01:38:08.555577+0000","last_became_peered":"2024-09-16T01:38:08.555577+0000","last_unstale":"2024-09-16T01:39:09.613873+0000","last_undegraded":"2024-09-16T01:39:09.613873+0000","last_fullsized":"2024-09-16T01:39:09.613873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:54.965341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518312+0000","last_change":"2024-09-16T01:38:01.522910+0000","last_active":"2024-09-16T01:39:09.518312+0000","last_peered":"2024-09-16T01:39:09.518312+0000","last_clean":"2024-09-16T01:39:09.518312+0000","last_became_active":"2024-09-16T01:38:01.521206+0000","last_became_peered":"2024-09-16T01:38:01.521206+0000","last_unstale":"2024-09-16T01:39:09.518312+0000","last_undegraded":"2024-09-16T01:39:09.518312+0000","last_fullsized":"2024-09-16T01:39:09.518312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:45.670464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'16","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522912+0000","last_change":"2024-09-16T01:37:59.505962+0000","last_active":"2024-09-16T01:39:09.522912+0000","last_peered":"2024-09-16T01:39:09.522912+0000","last_clean":"2024-09-16T01:39:09.522912+0000","last_became_active":"2024-09-16T01:37:59.505636+0000","last_became_peered":"2024-09-16T01:37:59.505636+0000","last_unstale":"2024-09-16T01:39:09.522912+0000","last_undegraded":"2024-09-16T01:39:09.522912+0000","last_fullsized":"2024-09-16T01:39:09.522912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:57.548513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517568+0000","last_change":"2024-09-16T01:38:08.571249+0000","last_active":"2024-09-16T01:39:09.517568+0000","last_peered":"2024-09-16T01:39:09.517568+0000","last_clean":"2024-09-16T01:39:09.517568+0000","last_became_active":"2024-09-16T01:38:08.570388+0000","last_became_peered":"2024-09-16T01:38:08.570388+0000","last_unstale":"2024-09-16T01:39:09.517568+0000","last_undegraded":"2024-09-16T01:39:09.517568+0000","last_fullsized":"2024-09-16T01:39:09.517568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:59.904704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611378+0000","last_change":"2024-09-16T01:38:01.522040+0000","last_active":"2024-09-16T01:39:09.611378+0000","last_peered":"2024-09-16T01:39:09.611378+0000","last_clean":"2024-09-16T01:39:09.611378+0000","last_became_active":"2024-09-16T01:38:01.521720+0000","last_became_peered":"2024-09-16T01:38:01.521720+0000","last_unstale":"2024-09-16T01:39:09.611378+0000","last_undegraded":"2024-09-16T01:39:09.611378+0000","last_fullsized":"2024-09-16T01:39:09.611378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:44:48.635603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519678+0000","last_change":"2024-09-16T01:38:08.561870+0000","last_active":"2024-09-16T01:39:09.519678+0000","last_peered":"2024-09-16T01:39:09.519678+0000","last_clean":"2024-09-16T01:39:09.519678+0000","last_became_active":"2024-09-16T01:38:08.561440+0000","last_became_peered":"2024-09-16T01:38:08.561440+0000","last_unstale":"2024-09-16T01:39:09.519678+0000","last_undegraded":"2024-09-16T01:39:09.519678+0000","last_fullsized":"2024-09-16T01:39:09.519678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:43:37.594734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519427+0000","last_change":"2024-09-16T01:38:08.555436+0000","last_active":"2024-09-16T01:39:09.519427+0000","last_peered":"2024-09-16T01:39:09.519427+0000","last_clean":"2024-09-16T01:39:09.519427+0000","last_became_active":"2024-09-16T01:38:08.555026+0000","last_became_peered":"2024-09-16T01:38:08.555026+0000","last_unstale":"2024-09-16T01:39:09.519427+0000","last_undegraded":"2024-09-16T01:39:09.519427+0000","last_fullsized":"2024-09-16T01:39:09.519427+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:28.870214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518712+0000","last_change":"2024-09-16T01:38:01.524482+0000","last_active":"2024-09-16T01:39:09.518712+0000","last_peered":"2024-09-16T01:39:09.518712+0000","last_clean":"2024-09-16T01:39:09.518712+0000","last_became_active":"2024-09-16T01:38:01.524160+0000","last_became_peered":"2024-09-16T01:38:01.524160+0000","last_unstale":"2024-09-16T01:39:09.518712+0000","last_undegraded":"2024-09-16T01:39:09.518712+0000","last_fullsized":"2024-09-16T01:39:09.518712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:39:27.621780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522927+0000","last_change":"2024-09-16T01:37:59.800051+0000","last_active":"2024-09-16T01:39:09.522927+0000","last_peered":"2024-09-16T01:39:09.522927+0000","last_clean":"2024-09-16T01:39:09.522927+0000","last_became_active":"2024-09-16T01:37:59.799606+0000","last_became_peered":"2024-09-16T01:37:59.799606+0000","last_unstale":"2024-09-16T01:39:09.522927+0000","last_undegraded":"2024-09-16T01:39:09.522927+0000","last_fullsized":"2024-09-16T01:39:09.522927+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:02.694933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.515939+0000","last_change":"2024-09-16T01:38:08.570764+0000","last_active":"2024-09-16T01:39:09.515939+0000","last_peered":"2024-09-16T01:39:09.515939+0000","last_clean":"2024-09-16T01:39:09.515939+0000","last_became_active":"2024-09-16T01:38:08.570385+0000","last_became_peered":"2024-09-16T01:38:08.570385+0000","last_unstale":"2024-09-16T01:39:09.515939+0000","last_undegraded":"2024-09-16T01:39:09.515939+0000","last_fullsized":"2024-09-16T01:39:09.515939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:25.349671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519067+0000","last_change":"2024-09-16T01:38:01.519590+0000","last_active":"2024-09-16T01:39:09.519067+0000","last_peered":"2024-09-16T01:39:09.519067+0000","last_clean":"2024-09-16T01:39:09.519067+0000","last_became_active":"2024-09-16T01:38:01.519210+0000","last_became_peered":"2024-09-16T01:38:01.519210+0000","last_unstale":"2024-09-16T01:39:09.519067+0000","last_undegraded":"2024-09-16T01:39:09.519067+0000","last_fullsized":"2024-09-16T01:39:09.519067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:04.228131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":90,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:28.640295+0000","last_change":"2024-09-16T01:37:59.505473+0000","last_active":"2024-09-16T01:39:28.640295+0000","last_peered":"2024-09-16T01:39:28.640295+0000","last_clean":"2024-09-16T01:39:28.640295+0000","last_became_active":"2024-09-16T01:37:59.505123+0000","last_became_peered":"2024-09-16T01:37:59.505123+0000","last_unstale":"2024-09-16T01:39:28.640295+0000","last_undegraded":"2024-09-16T01:39:28.640295+0000","last_fullsized":"2024-09-16T01:39:28.640295+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:17.739625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611176+0000","last_change":"2024-09-16T01:38:08.558534+0000","last_active":"2024-09-16T01:39:09.611176+0000","last_peered":"2024-09-16T01:39:09.611176+0000","last_clean":"2024-09-16T01:39:09.611176+0000","last_became_active":"2024-09-16T01:38:08.558185+0000","last_became_peered":"2024-09-16T01:38:08.558185+0000","last_unstale":"2024-09-16T01:39:09.611176+0000","last_undegraded":"2024-09-16T01:39:09.611176+0000","last_fullsized":"2024-09-16T01:39:09.611176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:09.359426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516572+0000","last_change":"2024-09-16T01:38:01.521445+0000","last_active":"2024-09-16T01:39:09.516572+0000","last_peered":"2024-09-16T01:39:09.516572+0000","last_clean":"2024-09-16T01:39:09.516572+0000","last_became_active":"2024-09-16T01:38:01.521140+0000","last_became_peered":"2024-09-16T01:38:01.521140+0000","last_unstale":"2024-09-16T01:39:09.516572+0000","last_undegraded":"2024-09-16T01:39:09.516572+0000","last_fullsized":"2024-09-16T01:39:09.516572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:05.247515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":91,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:30.638331+0000","last_change":"2024-09-16T01:37:59.798365+0000","last_active":"2024-09-16T01:39:30.638331+0000","last_peered":"2024-09-16T01:39:30.638331+0000","last_clean":"2024-09-16T01:39:30.638331+0000","last_became_active":"2024-09-16T01:37:59.798023+0000","last_became_peered":"2024-09-16T01:37:59.798023+0000","last_unstale":"2024-09-16T01:39:30.638331+0000","last_undegraded":"2024-09-16T01:39:30.638331+0000","last_fullsized":"2024-09-16T01:39:30.638331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:30.867080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611152+0000","last_change":"2024-09-16T01:38:08.571704+0000","last_active":"2024-09-16T01:39:09.611152+0000","last_peered":"2024-09-16T01:39:09.611152+0000","last_clean":"2024-09-16T01:39:09.611152+0000","last_became_active":"2024-09-16T01:38:08.571350+0000","last_became_peered":"2024-09-16T01:38:08.571350+0000","last_unstale":"2024-09-16T01:39:09.611152+0000","last_undegraded":"2024-09-16T01:39:09.611152+0000","last_fullsized":"2024-09-16T01:39:09.611152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:01.698770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.610862+0000","last_change":"2024-09-16T01:38:08.559433+0000","last_active":"2024-09-16T01:39:09.610862+0000","last_peered":"2024-09-16T01:39:09.610862+0000","last_clean":"2024-09-16T01:39:09.610862+0000","last_became_active":"2024-09-16T01:38:08.559110+0000","last_became_peered":"2024-09-16T01:38:08.559110+0000","last_unstale":"2024-09-16T01:39:09.610862+0000","last_undegraded":"2024-09-16T01:39:09.610862+0000","last_fullsized":"2024-09-16T01:39:09.610862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:39.001916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.520680+0000","last_change":"2024-09-16T01:37:59.508236+0000","last_active":"2024-09-16T01:39:09.520680+0000","last_peered":"2024-09-16T01:39:09.520680+0000","last_clean":"2024-09-16T01:39:09.520680+0000","last_became_active":"2024-09-16T01:37:59.507883+0000","last_became_peered":"2024-09-16T01:37:59.507883+0000","last_unstale":"2024-09-16T01:39:09.520680+0000","last_undegraded":"2024-09-16T01:39:09.520680+0000","last_fullsized":"2024-09-16T01:39:09.520680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:23.948530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518000+0000","last_change":"2024-09-16T01:38:08.572108+0000","last_active":"2024-09-16T01:39:09.518000+0000","last_peered":"2024-09-16T01:39:09.518000+0000","last_clean":"2024-09-16T01:39:09.518000+0000","last_became_active":"2024-09-16T01:38:08.571643+0000","last_became_peered":"2024-09-16T01:38:08.571643+0000","last_unstale":"2024-09-16T01:39:09.518000+0000","last_undegraded":"2024-09-16T01:39:09.518000+0000","last_fullsized":"2024-09-16T01:39:09.518000+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:46.636219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517895+0000","last_change":"2024-09-16T01:38:01.517894+0000","last_active":"2024-09-16T01:39:09.517895+0000","last_peered":"2024-09-16T01:39:09.517895+0000","last_clean":"2024-09-16T01:39:09.517895+0000","last_became_active":"2024-09-16T01:38:01.517602+0000","last_became_peered":"2024-09-16T01:38:01.517602+0000","last_unstale":"2024-09-16T01:39:09.517895+0000","last_undegraded":"2024-09-16T01:39:09.517895+0000","last_fullsized":"2024-09-16T01:39:09.517895+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:33.483120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517111+0000","last_change":"2024-09-16T01:37:59.507568+0000","last_active":"2024-09-16T01:39:09.517111+0000","last_peered":"2024-09-16T01:39:09.517111+0000","last_clean":"2024-09-16T01:39:09.517111+0000","last_became_active":"2024-09-16T01:37:59.505798+0000","last_became_peered":"2024-09-16T01:37:59.505798+0000","last_unstale":"2024-09-16T01:39:09.517111+0000","last_undegraded":"2024-09-16T01:39:09.517111+0000","last_fullsized":"2024-09-16T01:39:09.517111+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:19:02.711054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":358,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.520927+0000","last_change":"2024-09-16T01:36:46.744332+0000","last_active":"2024-09-16T01:39:09.520927+0000","last_peered":"2024-09-16T01:39:09.520927+0000","last_clean":"2024-09-16T01:39:09.520927+0000","last_became_active":"2024-09-16T01:36:46.743902+0000","last_became_peered":"2024-09-16T01:36:46.743902+0000","last_unstale":"2024-09-16T01:39:09.520927+0000","last_undegraded":"2024-09-16T01:39:09.520927+0000","last_fullsized":"2024-09-16T01:39:09.520927+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:33:19.891854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:33:19.891854+0000","last_clean_scrub_stamp":"2024-09-16T01:33:19.891854+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:46.173685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.528216+0000","last_change":"2024-09-16T01:39:09.528216+0000","last_active":"2024-09-16T01:39:09.528216+0000","last_peered":"2024-09-16T01:39:09.528216+0000","last_clean":"2024-09-16T01:39:09.528216+0000","last_became_active":"2024-09-16T01:39:09.523634+0000","last_became_peered":"2024-09-16T01:39:09.523634+0000","last_unstale":"2024-09-16T01:39:09.528216+0000","last_undegraded":"2024-09-16T01:39:09.528216+0000","last_fullsized":"2024-09-16T01:39:09.528216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:27.989901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522071+0000","last_change":"2024-09-16T01:38:01.527690+0000","last_active":"2024-09-16T01:39:09.522071+0000","last_peered":"2024-09-16T01:39:09.522071+0000","last_clean":"2024-09-16T01:39:09.522071+0000","last_became_active":"2024-09-16T01:38:01.527339+0000","last_became_peered":"2024-09-16T01:38:01.527339+0000","last_unstale":"2024-09-16T01:39:09.522071+0000","last_undegraded":"2024-09-16T01:39:09.522071+0000","last_fullsized":"2024-09-16T01:39:09.522071+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:58.526270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":92,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:27.639346+0000","last_change":"2024-09-16T01:37:59.798874+0000","last_active":"2024-09-16T01:39:27.639346+0000","last_peered":"2024-09-16T01:39:27.639346+0000","last_clean":"2024-09-16T01:39:27.639346+0000","last_became_active":"2024-09-16T01:37:59.798424+0000","last_became_peered":"2024-09-16T01:37:59.798424+0000","last_unstale":"2024-09-16T01:39:27.639346+0000","last_undegraded":"2024-09-16T01:39:27.639346+0000","last_fullsized":"2024-09-16T01:39:27.639346+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:36.648963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639242+0000","last_change":"2024-09-16T01:38:08.555524+0000","last_active":"2024-09-16T01:39:09.639242+0000","last_peered":"2024-09-16T01:39:09.639242+0000","last_clean":"2024-09-16T01:39:09.639242+0000","last_became_active":"2024-09-16T01:38:08.555075+0000","last_became_peered":"2024-09-16T01:38:08.555075+0000","last_unstale":"2024-09-16T01:39:09.639242+0000","last_undegraded":"2024-09-16T01:39:09.639242+0000","last_fullsized":"2024-09-16T01:39:09.639242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:44.847916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.521685+0000","last_change":"2024-09-16T01:38:01.528869+0000","last_active":"2024-09-16T01:39:09.521685+0000","last_peered":"2024-09-16T01:39:09.521685+0000","last_clean":"2024-09-16T01:39:09.521685+0000","last_became_active":"2024-09-16T01:38:01.528559+0000","last_became_peered":"2024-09-16T01:38:01.528559+0000","last_unstale":"2024-09-16T01:39:09.521685+0000","last_undegraded":"2024-09-16T01:39:09.521685+0000","last_fullsized":"2024-09-16T01:39:09.521685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:21.547808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518564+0000","last_change":"2024-09-16T01:37:59.510871+0000","last_active":"2024-09-16T01:39:09.518564+0000","last_peered":"2024-09-16T01:39:09.518564+0000","last_clean":"2024-09-16T01:39:09.518564+0000","last_became_active":"2024-09-16T01:37:59.509996+0000","last_became_peered":"2024-09-16T01:37:59.509996+0000","last_unstale":"2024-09-16T01:39:09.518564+0000","last_undegraded":"2024-09-16T01:39:09.518564+0000","last_fullsized":"2024-09-16T01:39:09.518564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:23.390864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.047868+0000","last_change":"2024-09-16T01:38:08.555181+0000","last_active":"2024-09-16T01:39:10.047868+0000","last_peered":"2024-09-16T01:39:10.047868+0000","last_clean":"2024-09-16T01:39:10.047868+0000","last_became_active":"2024-09-16T01:38:08.554778+0000","last_became_peered":"2024-09-16T01:38:08.554778+0000","last_unstale":"2024-09-16T01:39:10.047868+0000","last_undegraded":"2024-09-16T01:39:10.047868+0000","last_fullsized":"2024-09-16T01:39:10.047868+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:16:42.369236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522624+0000","last_change":"2024-09-16T01:38:01.526260+0000","last_active":"2024-09-16T01:39:09.522624+0000","last_peered":"2024-09-16T01:39:09.522624+0000","last_clean":"2024-09-16T01:39:09.522624+0000","last_became_active":"2024-09-16T01:38:01.525652+0000","last_became_peered":"2024-09-16T01:38:01.525652+0000","last_unstale":"2024-09-16T01:39:09.522624+0000","last_undegraded":"2024-09-16T01:39:09.522624+0000","last_fullsized":"2024-09-16T01:39:09.522624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:09.180507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611389+0000","last_change":"2024-09-16T01:38:08.570408+0000","last_active":"2024-09-16T01:39:09.611389+0000","last_peered":"2024-09-16T01:39:09.611389+0000","last_clean":"2024-09-16T01:39:09.611389+0000","last_became_active":"2024-09-16T01:38:08.566122+0000","last_became_peered":"2024-09-16T01:38:08.566122+0000","last_unstale":"2024-09-16T01:39:09.611389+0000","last_undegraded":"2024-09-16T01:39:09.611389+0000","last_fullsized":"2024-09-16T01:39:09.611389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:55.832320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516212+0000","last_change":"2024-09-16T01:38:08.555063+0000","last_active":"2024-09-16T01:39:09.516212+0000","last_peered":"2024-09-16T01:39:09.516212+0000","last_clean":"2024-09-16T01:39:09.516212+0000","last_became_active":"2024-09-16T01:38:08.554665+0000","last_became_peered":"2024-09-16T01:38:08.554665+0000","last_unstale":"2024-09-16T01:39:09.516212+0000","last_undegraded":"2024-09-16T01:39:09.516212+0000","last_fullsized":"2024-09-16T01:39:09.516212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:50.091017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519687+0000","last_change":"2024-09-16T01:38:01.524247+0000","last_active":"2024-09-16T01:39:09.519687+0000","last_peered":"2024-09-16T01:39:09.519687+0000","last_clean":"2024-09-16T01:39:09.519687+0000","last_became_active":"2024-09-16T01:38:01.523911+0000","last_became_peered":"2024-09-16T01:38:01.523911+0000","last_unstale":"2024-09-16T01:39:09.519687+0000","last_undegraded":"2024-09-16T01:39:09.519687+0000","last_fullsized":"2024-09-16T01:39:09.519687+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:32.745882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516885+0000","last_change":"2024-09-16T01:37:59.511272+0000","last_active":"2024-09-16T01:39:09.516885+0000","last_peered":"2024-09-16T01:39:09.516885+0000","last_clean":"2024-09-16T01:39:09.516885+0000","last_became_active":"2024-09-16T01:37:59.510920+0000","last_became_peered":"2024-09-16T01:37:59.510920+0000","last_unstale":"2024-09-16T01:39:09.516885+0000","last_undegraded":"2024-09-16T01:39:09.516885+0000","last_fullsized":"2024-09-16T01:39:09.516885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:59.406175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517305+0000","last_change":"2024-09-16T01:38:08.567173+0000","last_active":"2024-09-16T01:39:09.517305+0000","last_peered":"2024-09-16T01:39:09.517305+0000","last_clean":"2024-09-16T01:39:09.517305+0000","last_became_active":"2024-09-16T01:38:08.566788+0000","last_became_peered":"2024-09-16T01:38:08.566788+0000","last_unstale":"2024-09-16T01:39:09.517305+0000","last_undegraded":"2024-09-16T01:39:09.517305+0000","last_fullsized":"2024-09-16T01:39:09.517305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:35.743356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517105+0000","last_change":"2024-09-16T01:38:01.529529+0000","last_active":"2024-09-16T01:39:09.517105+0000","last_peered":"2024-09-16T01:39:09.517105+0000","last_clean":"2024-09-16T01:39:09.517105+0000","last_became_active":"2024-09-16T01:38:01.529048+0000","last_became_peered":"2024-09-16T01:38:01.529048+0000","last_unstale":"2024-09-16T01:39:09.517105+0000","last_undegraded":"2024-09-16T01:39:09.517105+0000","last_fullsized":"2024-09-16T01:39:09.517105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:35.041662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516028+0000","last_change":"2024-09-16T01:37:59.799536+0000","last_active":"2024-09-16T01:39:09.516028+0000","last_peered":"2024-09-16T01:39:09.516028+0000","last_clean":"2024-09-16T01:39:09.516028+0000","last_became_active":"2024-09-16T01:37:59.799109+0000","last_became_peered":"2024-09-16T01:37:59.799109+0000","last_unstale":"2024-09-16T01:39:09.516028+0000","last_undegraded":"2024-09-16T01:39:09.516028+0000","last_fullsized":"2024-09-16T01:39:09.516028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:07:25.007434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611392+0000","last_change":"2024-09-16T01:38:08.567019+0000","last_active":"2024-09-16T01:39:09.611392+0000","last_peered":"2024-09-16T01:39:09.611392+0000","last_clean":"2024-09-16T01:39:09.611392+0000","last_became_active":"2024-09-16T01:38:08.566667+0000","last_became_peered":"2024-09-16T01:38:08.566667+0000","last_unstale":"2024-09-16T01:39:09.611392+0000","last_undegraded":"2024-09-16T01:39:09.611392+0000","last_fullsized":"2024-09-16T01:39:09.611392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:11.375095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.520500+0000","last_change":"2024-09-16T01:38:01.527965+0000","last_active":"2024-09-16T01:39:09.520500+0000","last_peered":"2024-09-16T01:39:09.520500+0000","last_clean":"2024-09-16T01:39:09.520500+0000","last_became_active":"2024-09-16T01:38:01.526688+0000","last_became_peered":"2024-09-16T01:38:01.526688+0000","last_unstale":"2024-09-16T01:39:09.520500+0000","last_undegraded":"2024-09-16T01:39:09.520500+0000","last_fullsized":"2024-09-16T01:39:09.520500+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:06.618756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":91,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:28.636124+0000","last_change":"2024-09-16T01:37:59.510331+0000","last_active":"2024-09-16T01:39:28.636124+0000","last_peered":"2024-09-16T01:39:28.636124+0000","last_clean":"2024-09-16T01:39:28.636124+0000","last_became_active":"2024-09-16T01:37:59.509769+0000","last_became_peered":"2024-09-16T01:37:59.509769+0000","last_unstale":"2024-09-16T01:39:28.636124+0000","last_undegraded":"2024-09-16T01:39:28.636124+0000","last_fullsized":"2024-09-16T01:39:28.636124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:18.843091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.049893+0000","last_change":"2024-09-16T01:39:10.049460+0000","last_active":"2024-09-16T01:39:10.049893+0000","last_peered":"2024-09-16T01:39:10.049893+0000","last_clean":"2024-09-16T01:39:10.049893+0000","last_became_active":"2024-09-16T01:39:10.048862+0000","last_became_peered":"2024-09-16T01:39:10.048862+0000","last_unstale":"2024-09-16T01:39:10.049893+0000","last_undegraded":"2024-09-16T01:39:10.049893+0000","last_fullsized":"2024-09-16T01:39:10.049893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:37.329806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.521463+0000","last_change":"2024-09-16T01:38:01.521254+0000","last_active":"2024-09-16T01:39:09.521463+0000","last_peered":"2024-09-16T01:39:09.521463+0000","last_clean":"2024-09-16T01:39:09.521463+0000","last_became_active":"2024-09-16T01:38:01.520901+0000","last_became_peered":"2024-09-16T01:38:01.520901+0000","last_unstale":"2024-09-16T01:39:09.521463+0000","last_undegraded":"2024-09-16T01:39:09.521463+0000","last_fullsized":"2024-09-16T01:39:09.521463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:27.806954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517118+0000","last_change":"2024-09-16T01:37:59.505990+0000","last_active":"2024-09-16T01:39:09.517118+0000","last_peered":"2024-09-16T01:39:09.517118+0000","last_clean":"2024-09-16T01:39:09.517118+0000","last_became_active":"2024-09-16T01:37:59.505564+0000","last_became_peered":"2024-09-16T01:37:59.505564+0000","last_unstale":"2024-09-16T01:39:09.517118+0000","last_undegraded":"2024-09-16T01:39:09.517118+0000","last_fullsized":"2024-09-16T01:39:09.517118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:01.496609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519430+0000","last_change":"2024-09-16T01:38:08.562838+0000","last_active":"2024-09-16T01:39:09.519430+0000","last_peered":"2024-09-16T01:39:09.519430+0000","last_clean":"2024-09-16T01:39:09.519430+0000","last_became_active":"2024-09-16T01:38:08.562426+0000","last_became_peered":"2024-09-16T01:38:08.562426+0000","last_unstale":"2024-09-16T01:39:09.519430+0000","last_undegraded":"2024-09-16T01:39:09.519430+0000","last_fullsized":"2024-09-16T01:39:09.519430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:20.685626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611059+0000","last_change":"2024-09-16T01:38:01.514983+0000","last_active":"2024-09-16T01:39:09.611059+0000","last_peered":"2024-09-16T01:39:09.611059+0000","last_clean":"2024-09-16T01:39:09.611059+0000","last_became_active":"2024-09-16T01:38:01.514648+0000","last_became_peered":"2024-09-16T01:38:01.514648+0000","last_unstale":"2024-09-16T01:39:09.611059+0000","last_undegraded":"2024-09-16T01:39:09.611059+0000","last_fullsized":"2024-09-16T01:39:09.611059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:57.052564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.614178+0000","last_change":"2024-09-16T01:37:59.505141+0000","last_active":"2024-09-16T01:39:09.614178+0000","last_peered":"2024-09-16T01:39:09.614178+0000","last_clean":"2024-09-16T01:39:09.614178+0000","last_became_active":"2024-09-16T01:37:59.504730+0000","last_became_peered":"2024-09-16T01:37:59.504730+0000","last_unstale":"2024-09-16T01:39:09.614178+0000","last_undegraded":"2024-09-16T01:39:09.614178+0000","last_fullsized":"2024-09-16T01:39:09.614178+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:54.504661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517789+0000","last_change":"2024-09-16T01:38:08.563970+0000","last_active":"2024-09-16T01:39:09.517789+0000","last_peered":"2024-09-16T01:39:09.517789+0000","last_clean":"2024-09-16T01:39:09.517789+0000","last_became_active":"2024-09-16T01:38:08.563555+0000","last_became_peered":"2024-09-16T01:38:08.563555+0000","last_unstale":"2024-09-16T01:39:09.517789+0000","last_undegraded":"2024-09-16T01:39:09.517789+0000","last_fullsized":"2024-09-16T01:39:09.517789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:27:12.086043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516577+0000","last_change":"2024-09-16T01:38:01.521016+0000","last_active":"2024-09-16T01:39:09.516577+0000","last_peered":"2024-09-16T01:39:09.516577+0000","last_clean":"2024-09-16T01:39:09.516577+0000","last_became_active":"2024-09-16T01:38:01.520656+0000","last_became_peered":"2024-09-16T01:38:01.520656+0000","last_unstale":"2024-09-16T01:39:09.516577+0000","last_undegraded":"2024-09-16T01:39:09.516577+0000","last_fullsized":"2024-09-16T01:39:09.516577+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:39.001239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'34","reported_seq":175,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:28.635623+0000","last_change":"2024-09-16T01:37:59.507235+0000","last_active":"2024-09-16T01:39:28.635623+0000","last_peered":"2024-09-16T01:39:28.635623+0000","last_clean":"2024-09-16T01:39:28.635623+0000","last_became_active":"2024-09-16T01:37:59.506255+0000","last_became_peered":"2024-09-16T01:37:59.506255+0000","last_unstale":"2024-09-16T01:39:28.635623+0000","last_undegraded":"2024-09-16T01:39:28.635623+0000","last_fullsized":"2024-09-16T01:39:28.635623+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:44.515663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":114,"num_write":36,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522688+0000","last_change":"2024-09-16T01:39:09.522688+0000","last_active":"2024-09-16T01:39:09.522688+0000","last_peered":"2024-09-16T01:39:09.522688+0000","last_clean":"2024-09-16T01:39:09.522688+0000","last_became_active":"2024-09-16T01:39:09.522137+0000","last_became_peered":"2024-09-16T01:39:09.522137+0000","last_unstale":"2024-09-16T01:39:09.522688+0000","last_undegraded":"2024-09-16T01:39:09.522688+0000","last_fullsized":"2024-09-16T01:39:09.522688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.004315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639485+0000","last_change":"2024-09-16T01:38:01.526641+0000","last_active":"2024-09-16T01:39:09.639485+0000","last_peered":"2024-09-16T01:39:09.639485+0000","last_clean":"2024-09-16T01:39:09.639485+0000","last_became_active":"2024-09-16T01:38:01.526278+0000","last_became_peered":"2024-09-16T01:38:01.526278+0000","last_unstale":"2024-09-16T01:39:09.639485+0000","last_undegraded":"2024-09-16T01:39:09.639485+0000","last_fullsized":"2024-09-16T01:39:09.639485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:56.156593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518309+0000","last_change":"2024-09-16T01:37:59.800065+0000","last_active":"2024-09-16T01:39:09.518309+0000","last_peered":"2024-09-16T01:39:09.518309+0000","last_clean":"2024-09-16T01:39:09.518309+0000","last_became_active":"2024-09-16T01:37:59.799705+0000","last_became_peered":"2024-09-16T01:37:59.799705+0000","last_unstale":"2024-09-16T01:39:09.518309+0000","last_undegraded":"2024-09-16T01:39:09.518309+0000","last_fullsized":"2024-09-16T01:39:09.518309+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:13.791875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517087+0000","last_change":"2024-09-16T01:38:08.556360+0000","last_active":"2024-09-16T01:39:09.517087+0000","last_peered":"2024-09-16T01:39:09.517087+0000","last_clean":"2024-09-16T01:39:09.517087+0000","last_became_active":"2024-09-16T01:38:08.555219+0000","last_became_peered":"2024-09-16T01:38:08.555219+0000","last_unstale":"2024-09-16T01:39:09.517087+0000","last_undegraded":"2024-09-16T01:39:09.517087+0000","last_fullsized":"2024-09-16T01:39:09.517087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:36.464760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.048288+0000","last_change":"2024-09-16T01:38:01.526758+0000","last_active":"2024-09-16T01:39:10.048288+0000","last_peered":"2024-09-16T01:39:10.048288+0000","last_clean":"2024-09-16T01:39:10.048288+0000","last_became_active":"2024-09-16T01:38:01.526545+0000","last_became_peered":"2024-09-16T01:38:01.526545+0000","last_unstale":"2024-09-16T01:39:10.048288+0000","last_undegraded":"2024-09-16T01:39:10.048288+0000","last_fullsized":"2024-09-16T01:39:10.048288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:47.185489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522440+0000","last_change":"2024-09-16T01:38:08.555125+0000","last_active":"2024-09-16T01:39:09.522440+0000","last_peered":"2024-09-16T01:39:09.522440+0000","last_clean":"2024-09-16T01:39:09.522440+0000","last_became_active":"2024-09-16T01:38:08.554756+0000","last_became_peered":"2024-09-16T01:38:08.554756+0000","last_unstale":"2024-09-16T01:39:09.522440+0000","last_undegraded":"2024-09-16T01:39:09.522440+0000","last_fullsized":"2024-09-16T01:39:09.522440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:08.888006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522217+0000","last_change":"2024-09-16T01:38:08.557906+0000","last_active":"2024-09-16T01:39:09.522217+0000","last_peered":"2024-09-16T01:39:09.522217+0000","last_clean":"2024-09-16T01:39:09.522217+0000","last_became_active":"2024-09-16T01:38:08.557546+0000","last_became_peered":"2024-09-16T01:38:08.557546+0000","last_unstale":"2024-09-16T01:39:09.522217+0000","last_undegraded":"2024-09-16T01:39:09.522217+0000","last_fullsized":"2024-09-16T01:39:09.522217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:27:04.988778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639642+0000","last_change":"2024-09-16T01:38:01.528430+0000","last_active":"2024-09-16T01:39:09.639642+0000","last_peered":"2024-09-16T01:39:09.639642+0000","last_clean":"2024-09-16T01:39:09.639642+0000","last_became_active":"2024-09-16T01:38:01.528069+0000","last_became_peered":"2024-09-16T01:38:01.528069+0000","last_unstale":"2024-09-16T01:39:09.639642+0000","last_undegraded":"2024-09-16T01:39:09.639642+0000","last_fullsized":"2024-09-16T01:39:09.639642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:41.317215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":136,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:28.635700+0000","last_change":"2024-09-16T01:37:59.509017+0000","last_active":"2024-09-16T01:39:28.635700+0000","last_peered":"2024-09-16T01:39:28.635700+0000","last_clean":"2024-09-16T01:39:28.635700+0000","last_became_active":"2024-09-16T01:37:59.508252+0000","last_became_peered":"2024-09-16T01:37:59.508252+0000","last_unstale":"2024-09-16T01:39:28.635700+0000","last_undegraded":"2024-09-16T01:39:28.635700+0000","last_fullsized":"2024-09-16T01:39:28.635700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:26.215407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518040+0000","last_change":"2024-09-16T01:38:08.565197+0000","last_active":"2024-09-16T01:39:09.518040+0000","last_peered":"2024-09-16T01:39:09.518040+0000","last_clean":"2024-09-16T01:39:09.518040+0000","last_became_active":"2024-09-16T01:38:08.562773+0000","last_became_peered":"2024-09-16T01:38:08.562773+0000","last_unstale":"2024-09-16T01:39:09.518040+0000","last_undegraded":"2024-09-16T01:39:09.518040+0000","last_fullsized":"2024-09-16T01:39:09.518040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:13.316950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.048048+0000","last_change":"2024-09-16T01:38:08.554052+0000","last_active":"2024-09-16T01:39:10.048048+0000","last_peered":"2024-09-16T01:39:10.048048+0000","last_clean":"2024-09-16T01:39:10.048048+0000","last_became_active":"2024-09-16T01:38:08.553660+0000","last_became_peered":"2024-09-16T01:38:08.553660+0000","last_unstale":"2024-09-16T01:39:10.048048+0000","last_undegraded":"2024-09-16T01:39:10.048048+0000","last_fullsized":"2024-09-16T01:39:10.048048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:53:52.270695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522411+0000","last_change":"2024-09-16T01:38:01.522688+0000","last_active":"2024-09-16T01:39:09.522411+0000","last_peered":"2024-09-16T01:39:09.522411+0000","last_clean":"2024-09-16T01:39:09.522411+0000","last_became_active":"2024-09-16T01:38:01.519843+0000","last_became_peered":"2024-09-16T01:38:01.519843+0000","last_unstale":"2024-09-16T01:39:09.522411+0000","last_undegraded":"2024-09-16T01:39:09.522411+0000","last_fullsized":"2024-09-16T01:39:09.522411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:50.971760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516780+0000","last_change":"2024-09-16T01:37:59.509044+0000","last_active":"2024-09-16T01:39:09.516780+0000","last_peered":"2024-09-16T01:39:09.516780+0000","last_clean":"2024-09-16T01:39:09.516780+0000","last_became_active":"2024-09-16T01:37:59.507085+0000","last_became_peered":"2024-09-16T01:37:59.507085+0000","last_unstale":"2024-09-16T01:39:09.516780+0000","last_undegraded":"2024-09-16T01:39:09.516780+0000","last_fullsized":"2024-09-16T01:39:09.516780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:25:09.930945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519812+0000","last_change":"2024-09-16T01:38:08.561250+0000","last_active":"2024-09-16T01:39:09.519812+0000","last_peered":"2024-09-16T01:39:09.519812+0000","last_clean":"2024-09-16T01:39:09.519812+0000","last_became_active":"2024-09-16T01:38:08.560866+0000","last_became_peered":"2024-09-16T01:38:08.560866+0000","last_unstale":"2024-09-16T01:39:09.519812+0000","last_undegraded":"2024-09-16T01:39:09.519812+0000","last_fullsized":"2024-09-16T01:39:09.519812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:31.253817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517715+0000","last_change":"2024-09-16T01:38:01.517400+0000","last_active":"2024-09-16T01:39:09.517715+0000","last_peered":"2024-09-16T01:39:09.517715+0000","last_clean":"2024-09-16T01:39:09.517715+0000","last_became_active":"2024-09-16T01:38:01.517103+0000","last_became_peered":"2024-09-16T01:38:01.517103+0000","last_unstale":"2024-09-16T01:39:09.517715+0000","last_undegraded":"2024-09-16T01:39:09.517715+0000","last_fullsized":"2024-09-16T01:39:09.517715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:37.886179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.610877+0000","last_change":"2024-09-16T01:37:59.503370+0000","last_active":"2024-09-16T01:39:09.610877+0000","last_peered":"2024-09-16T01:39:09.610877+0000","last_clean":"2024-09-16T01:39:09.610877+0000","last_became_active":"2024-09-16T01:37:59.503071+0000","last_became_peered":"2024-09-16T01:37:59.503071+0000","last_unstale":"2024-09-16T01:39:09.610877+0000","last_undegraded":"2024-09-16T01:39:09.610877+0000","last_fullsized":"2024-09-16T01:39:09.610877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:51.207702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519530+0000","last_change":"2024-09-16T01:37:59.506110+0000","last_active":"2024-09-16T01:39:09.519530+0000","last_peered":"2024-09-16T01:39:09.519530+0000","last_clean":"2024-09-16T01:39:09.519530+0000","last_became_active":"2024-09-16T01:37:59.505264+0000","last_became_peered":"2024-09-16T01:37:59.505264+0000","last_unstale":"2024-09-16T01:39:09.519530+0000","last_undegraded":"2024-09-16T01:39:09.519530+0000","last_fullsized":"2024-09-16T01:39:09.519530+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:35.986433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517383+0000","last_change":"2024-09-16T01:38:08.563900+0000","last_active":"2024-09-16T01:39:09.517383+0000","last_peered":"2024-09-16T01:39:09.517383+0000","last_clean":"2024-09-16T01:39:09.517383+0000","last_became_active":"2024-09-16T01:38:08.563551+0000","last_became_peered":"2024-09-16T01:38:08.563551+0000","last_unstale":"2024-09-16T01:39:09.517383+0000","last_undegraded":"2024-09-16T01:39:09.517383+0000","last_fullsized":"2024-09-16T01:39:09.517383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:31:03.979580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.614148+0000","last_change":"2024-09-16T01:38:08.556011+0000","last_active":"2024-09-16T01:39:09.614148+0000","last_peered":"2024-09-16T01:39:09.614148+0000","last_clean":"2024-09-16T01:39:09.614148+0000","last_became_active":"2024-09-16T01:38:08.555615+0000","last_became_peered":"2024-09-16T01:38:08.555615+0000","last_unstale":"2024-09-16T01:39:09.614148+0000","last_undegraded":"2024-09-16T01:39:09.614148+0000","last_fullsized":"2024-09-16T01:39:09.614148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:34.434974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":92,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:29.637814+0000","last_change":"2024-09-16T01:37:59.508411+0000","last_active":"2024-09-16T01:39:29.637814+0000","last_peered":"2024-09-16T01:39:29.637814+0000","last_clean":"2024-09-16T01:39:29.637814+0000","last_became_active":"2024-09-16T01:37:59.508058+0000","last_became_peered":"2024-09-16T01:37:59.508058+0000","last_unstale":"2024-09-16T01:39:29.637814+0000","last_undegraded":"2024-09-16T01:39:29.637814+0000","last_fullsized":"2024-09-16T01:39:29.637814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:47.441673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516073+0000","last_change":"2024-09-16T01:38:08.560930+0000","last_active":"2024-09-16T01:39:09.516073+0000","last_peered":"2024-09-16T01:39:09.516073+0000","last_clean":"2024-09-16T01:39:09.516073+0000","last_became_active":"2024-09-16T01:38:08.560490+0000","last_became_peered":"2024-09-16T01:38:08.560490+0000","last_unstale":"2024-09-16T01:39:09.516073+0000","last_undegraded":"2024-09-16T01:39:09.516073+0000","last_fullsized":"2024-09-16T01:39:09.516073+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:11.556113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.613929+0000","last_change":"2024-09-16T01:38:08.557564+0000","last_active":"2024-09-16T01:39:09.613929+0000","last_peered":"2024-09-16T01:39:09.613929+0000","last_clean":"2024-09-16T01:39:09.613929+0000","last_became_active":"2024-09-16T01:38:08.557143+0000","last_became_peered":"2024-09-16T01:38:08.557143+0000","last_unstale":"2024-09-16T01:39:09.613929+0000","last_undegraded":"2024-09-16T01:39:09.613929+0000","last_fullsized":"2024-09-16T01:39:09.613929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:43.148870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519117+0000","last_change":"2024-09-16T01:38:01.527353+0000","last_active":"2024-09-16T01:39:09.519117+0000","last_peered":"2024-09-16T01:39:09.519117+0000","last_clean":"2024-09-16T01:39:09.519117+0000","last_became_active":"2024-09-16T01:38:01.527031+0000","last_became_peered":"2024-09-16T01:38:01.527031+0000","last_unstale":"2024-09-16T01:39:09.519117+0000","last_undegraded":"2024-09-16T01:39:09.519117+0000","last_fullsized":"2024-09-16T01:39:09.519117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:27.311638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.614155+0000","last_change":"2024-09-16T01:38:01.526409+0000","last_active":"2024-09-16T01:39:09.614155+0000","last_peered":"2024-09-16T01:39:09.614155+0000","last_clean":"2024-09-16T01:39:09.614155+0000","last_became_active":"2024-09-16T01:38:01.526064+0000","last_became_peered":"2024-09-16T01:38:01.526064+0000","last_unstale":"2024-09-16T01:39:09.614155+0000","last_undegraded":"2024-09-16T01:39:09.614155+0000","last_fullsized":"2024-09-16T01:39:09.614155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:51.571808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.613906+0000","last_change":"2024-09-16T01:38:01.521856+0000","last_active":"2024-09-16T01:39:09.613906+0000","last_peered":"2024-09-16T01:39:09.613906+0000","last_clean":"2024-09-16T01:39:09.613906+0000","last_became_active":"2024-09-16T01:38:01.520312+0000","last_became_peered":"2024-09-16T01:38:01.520312+0000","last_unstale":"2024-09-16T01:39:09.613906+0000","last_undegraded":"2024-09-16T01:39:09.613906+0000","last_fullsized":"2024-09-16T01:39:09.613906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:53.618990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.047889+0000","last_change":"2024-09-16T01:38:01.520958+0000","last_active":"2024-09-16T01:39:10.047889+0000","last_peered":"2024-09-16T01:39:10.047889+0000","last_clean":"2024-09-16T01:39:10.047889+0000","last_became_active":"2024-09-16T01:38:01.520658+0000","last_became_peered":"2024-09-16T01:38:01.520658+0000","last_unstale":"2024-09-16T01:39:10.047889+0000","last_undegraded":"2024-09-16T01:39:10.047889+0000","last_fullsized":"2024-09-16T01:39:10.047889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:23.770694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611348+0000","last_change":"2024-09-16T01:38:01.519140+0000","last_active":"2024-09-16T01:39:09.611348+0000","last_peered":"2024-09-16T01:39:09.611348+0000","last_clean":"2024-09-16T01:39:09.611348+0000","last_became_active":"2024-09-16T01:38:01.518765+0000","last_became_peered":"2024-09-16T01:38:01.518765+0000","last_unstale":"2024-09-16T01:39:09.611348+0000","last_undegraded":"2024-09-16T01:39:09.611348+0000","last_fullsized":"2024-09-16T01:39:09.611348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:17.194886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519792+0000","last_change":"2024-09-16T01:38:01.523762+0000","last_active":"2024-09-16T01:39:09.519792+0000","last_peered":"2024-09-16T01:39:09.519792+0000","last_clean":"2024-09-16T01:39:09.519792+0000","last_became_active":"2024-09-16T01:38:01.522035+0000","last_became_peered":"2024-09-16T01:38:01.522035+0000","last_unstale":"2024-09-16T01:39:09.519792+0000","last_undegraded":"2024-09-16T01:39:09.519792+0000","last_fullsized":"2024-09-16T01:39:09.519792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:40.405350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.614239+0000","last_change":"2024-09-16T01:38:01.527539+0000","last_active":"2024-09-16T01:39:09.614239+0000","last_peered":"2024-09-16T01:39:09.614239+0000","last_clean":"2024-09-16T01:39:09.614239+0000","last_became_active":"2024-09-16T01:38:01.526974+0000","last_became_peered":"2024-09-16T01:38:01.526974+0000","last_unstale":"2024-09-16T01:39:09.614239+0000","last_undegraded":"2024-09-16T01:39:09.614239+0000","last_fullsized":"2024-09-16T01:39:09.614239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:30:03.587141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518254+0000","last_change":"2024-09-16T01:38:01.529131+0000","last_active":"2024-09-16T01:39:09.518254+0000","last_peered":"2024-09-16T01:39:09.518254+0000","last_clean":"2024-09-16T01:39:09.518254+0000","last_became_active":"2024-09-16T01:38:01.528796+0000","last_became_peered":"2024-09-16T01:38:01.528796+0000","last_unstale":"2024-09-16T01:39:09.518254+0000","last_undegraded":"2024-09-16T01:39:09.518254+0000","last_fullsized":"2024-09-16T01:39:09.518254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:38.903638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516476+0000","last_change":"2024-09-16T01:38:01.514779+0000","last_active":"2024-09-16T01:39:09.516476+0000","last_peered":"2024-09-16T01:39:09.516476+0000","last_clean":"2024-09-16T01:39:09.516476+0000","last_became_active":"2024-09-16T01:38:01.514434+0000","last_became_peered":"2024-09-16T01:38:01.514434+0000","last_unstale":"2024-09-16T01:39:09.516476+0000","last_undegraded":"2024-09-16T01:39:09.516476+0000","last_fullsized":"2024-09-16T01:39:09.516476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:31.382111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517771+0000","last_change":"2024-09-16T01:38:01.521771+0000","last_active":"2024-09-16T01:39:09.517771+0000","last_peered":"2024-09-16T01:39:09.517771+0000","last_clean":"2024-09-16T01:39:09.517771+0000","last_became_active":"2024-09-16T01:38:01.521288+0000","last_became_peered":"2024-09-16T01:38:01.521288+0000","last_unstale":"2024-09-16T01:39:09.517771+0000","last_undegraded":"2024-09-16T01:39:09.517771+0000","last_fullsized":"2024-09-16T01:39:09.517771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:11.825164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639394+0000","last_change":"2024-09-16T01:38:01.529247+0000","last_active":"2024-09-16T01:39:09.639394+0000","last_peered":"2024-09-16T01:39:09.639394+0000","last_clean":"2024-09-16T01:39:09.639394+0000","last_became_active":"2024-09-16T01:38:01.528926+0000","last_became_peered":"2024-09-16T01:38:01.528926+0000","last_unstale":"2024-09-16T01:39:09.639394+0000","last_undegraded":"2024-09-16T01:39:09.639394+0000","last_fullsized":"2024-09-16T01:39:09.639394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:20.704114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611066+0000","last_change":"2024-09-16T01:38:01.520830+0000","last_active":"2024-09-16T01:39:09.611066+0000","last_peered":"2024-09-16T01:39:09.611066+0000","last_clean":"2024-09-16T01:39:09.611066+0000","last_became_active":"2024-09-16T01:38:01.520526+0000","last_became_peered":"2024-09-16T01:38:01.520526+0000","last_unstale":"2024-09-16T01:39:09.611066+0000","last_undegraded":"2024-09-16T01:39:09.611066+0000","last_fullsized":"2024-09-16T01:39:09.611066+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:28.062812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519439+0000","last_change":"2024-09-16T01:38:01.521983+0000","last_active":"2024-09-16T01:39:09.519439+0000","last_peered":"2024-09-16T01:39:09.519439+0000","last_clean":"2024-09-16T01:39:09.519439+0000","last_became_active":"2024-09-16T01:38:01.521666+0000","last_became_peered":"2024-09-16T01:38:01.521666+0000","last_unstale":"2024-09-16T01:39:09.519439+0000","last_undegraded":"2024-09-16T01:39:09.519439+0000","last_fullsized":"2024-09-16T01:39:09.519439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:50:55.761313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.614164+0000","last_change":"2024-09-16T01:38:08.554304+0000","last_active":"2024-09-16T01:39:09.614164+0000","last_peered":"2024-09-16T01:39:09.614164+0000","last_clean":"2024-09-16T01:39:09.614164+0000","last_became_active":"2024-09-16T01:38:08.553683+0000","last_became_peered":"2024-09-16T01:38:08.553683+0000","last_unstale":"2024-09-16T01:39:09.614164+0000","last_undegraded":"2024-09-16T01:39:09.614164+0000","last_fullsized":"2024-09-16T01:39:09.614164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:56.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522904+0000","last_change":"2024-09-16T01:38:01.520425+0000","last_active":"2024-09-16T01:39:09.522904+0000","last_peered":"2024-09-16T01:39:09.522904+0000","last_clean":"2024-09-16T01:39:09.522904+0000","last_became_active":"2024-09-16T01:38:01.519858+0000","last_became_peered":"2024-09-16T01:38:01.519858+0000","last_unstale":"2024-09-16T01:39:09.522904+0000","last_undegraded":"2024-09-16T01:39:09.522904+0000","last_fullsized":"2024-09-16T01:39:09.522904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:48.534298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.520100+0000","last_change":"2024-09-16T01:38:01.532762+0000","last_active":"2024-09-16T01:39:09.520100+0000","last_peered":"2024-09-16T01:39:09.520100+0000","last_clean":"2024-09-16T01:39:09.520100+0000","last_became_active":"2024-09-16T01:38:01.532404+0000","last_became_peered":"2024-09-16T01:38:01.532404+0000","last_unstale":"2024-09-16T01:39:09.520100+0000","last_undegraded":"2024-09-16T01:39:09.520100+0000","last_fullsized":"2024-09-16T01:39:09.520100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:03.135141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639153+0000","last_change":"2024-09-16T01:38:01.530205+0000","last_active":"2024-09-16T01:39:09.639153+0000","last_peered":"2024-09-16T01:39:09.639153+0000","last_clean":"2024-09-16T01:39:09.639153+0000","last_became_active":"2024-09-16T01:38:01.529898+0000","last_became_peered":"2024-09-16T01:38:01.529898+0000","last_unstale":"2024-09-16T01:39:09.639153+0000","last_undegraded":"2024-09-16T01:39:09.639153+0000","last_fullsized":"2024-09-16T01:39:09.639153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:48.444441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.047689+0000","last_change":"2024-09-16T01:38:01.529407+0000","last_active":"2024-09-16T01:39:10.047689+0000","last_peered":"2024-09-16T01:39:10.047689+0000","last_clean":"2024-09-16T01:39:10.047689+0000","last_became_active":"2024-09-16T01:38:01.528330+0000","last_became_peered":"2024-09-16T01:38:01.528330+0000","last_unstale":"2024-09-16T01:39:10.047689+0000","last_undegraded":"2024-09-16T01:39:10.047689+0000","last_fullsized":"2024-09-16T01:39:10.047689+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:32.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.047759+0000","last_change":"2024-09-16T01:38:01.519579+0000","last_active":"2024-09-16T01:39:10.047759+0000","last_peered":"2024-09-16T01:39:10.047759+0000","last_clean":"2024-09-16T01:39:10.047759+0000","last_became_active":"2024-09-16T01:38:01.519211+0000","last_became_peered":"2024-09-16T01:38:01.519211+0000","last_unstale":"2024-09-16T01:39:10.047759+0000","last_undegraded":"2024-09-16T01:39:10.047759+0000","last_fullsized":"2024-09-16T01:39:10.047759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:18.861169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611059+0000","last_change":"2024-09-16T01:38:01.521136+0000","last_active":"2024-09-16T01:39:09.611059+0000","last_peered":"2024-09-16T01:39:09.611059+0000","last_clean":"2024-09-16T01:39:09.611059+0000","last_became_active":"2024-09-16T01:38:01.520819+0000","last_became_peered":"2024-09-16T01:38:01.520819+0000","last_unstale":"2024-09-16T01:39:09.611059+0000","last_undegraded":"2024-09-16T01:39:09.611059+0000","last_fullsized":"2024-09-16T01:39:09.611059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:28.150822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516967+0000","last_change":"2024-09-16T01:38:01.519382+0000","last_active":"2024-09-16T01:39:09.516967+0000","last_peered":"2024-09-16T01:39:09.516967+0000","last_clean":"2024-09-16T01:39:09.516967+0000","last_became_active":"2024-09-16T01:38:01.519039+0000","last_became_peered":"2024-09-16T01:38:01.519039+0000","last_unstale":"2024-09-16T01:39:09.516967+0000","last_undegraded":"2024-09-16T01:39:09.516967+0000","last_fullsized":"2024-09-16T01:39:09.516967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:46.559357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.521938+0000","last_change":"2024-09-16T01:38:01.527513+0000","last_active":"2024-09-16T01:39:09.521938+0000","last_peered":"2024-09-16T01:39:09.521938+0000","last_clean":"2024-09-16T01:39:09.521938+0000","last_became_active":"2024-09-16T01:38:01.527184+0000","last_became_peered":"2024-09-16T01:38:01.527184+0000","last_unstale":"2024-09-16T01:39:09.521938+0000","last_undegraded":"2024-09-16T01:39:09.521938+0000","last_fullsized":"2024-09-16T01:39:09.521938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:41.508318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519580+0000","last_change":"2024-09-16T01:38:01.524239+0000","last_active":"2024-09-16T01:39:09.519580+0000","last_peered":"2024-09-16T01:39:09.519580+0000","last_clean":"2024-09-16T01:39:09.519580+0000","last_became_active":"2024-09-16T01:38:01.523906+0000","last_became_peered":"2024-09-16T01:38:01.523906+0000","last_unstale":"2024-09-16T01:39:09.519580+0000","last_undegraded":"2024-09-16T01:39:09.519580+0000","last_fullsized":"2024-09-16T01:39:09.519580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:11.718796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639776+0000","last_change":"2024-09-16T01:38:01.531564+0000","last_active":"2024-09-16T01:39:09.639776+0000","last_peered":"2024-09-16T01:39:09.639776+0000","last_clean":"2024-09-16T01:39:09.639776+0000","last_became_active":"2024-09-16T01:38:01.531241+0000","last_became_peered":"2024-09-16T01:38:01.531241+0000","last_unstale":"2024-09-16T01:39:09.639776+0000","last_undegraded":"2024-09-16T01:39:09.639776+0000","last_fullsized":"2024-09-16T01:39:09.639776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:16.244641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517794+0000","last_change":"2024-09-16T01:38:01.519913+0000","last_active":"2024-09-16T01:39:09.517794+0000","last_peered":"2024-09-16T01:39:09.517794+0000","last_clean":"2024-09-16T01:39:09.517794+0000","last_became_active":"2024-09-16T01:38:01.518631+0000","last_became_peered":"2024-09-16T01:38:01.518631+0000","last_unstale":"2024-09-16T01:39:09.517794+0000","last_undegraded":"2024-09-16T01:39:09.517794+0000","last_fullsized":"2024-09-16T01:39:09.517794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:48.747188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.520386+0000","last_change":"2024-09-16T01:38:01.530805+0000","last_active":"2024-09-16T01:39:09.520386+0000","last_peered":"2024-09-16T01:39:09.520386+0000","last_clean":"2024-09-16T01:39:09.520386+0000","last_became_active":"2024-09-16T01:38:01.530466+0000","last_became_peered":"2024-09-16T01:38:01.530466+0000","last_unstale":"2024-09-16T01:39:09.520386+0000","last_undegraded":"2024-09-16T01:39:09.520386+0000","last_fullsized":"2024-09-16T01:39:09.520386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:41:30.587398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519485+0000","last_change":"2024-09-16T01:38:01.519227+0000","last_active":"2024-09-16T01:39:09.519485+0000","last_peered":"2024-09-16T01:39:09.519485+0000","last_clean":"2024-09-16T01:39:09.519485+0000","last_became_active":"2024-09-16T01:38:01.518875+0000","last_became_peered":"2024-09-16T01:38:01.518875+0000","last_unstale":"2024-09-16T01:39:09.519485+0000","last_undegraded":"2024-09-16T01:39:09.519485+0000","last_fullsized":"2024-09-16T01:39:09.519485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:42.654812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516085+0000","last_change":"2024-09-16T01:38:01.513327+0000","last_active":"2024-09-16T01:39:09.516085+0000","last_peered":"2024-09-16T01:39:09.516085+0000","last_clean":"2024-09-16T01:39:09.516085+0000","last_became_active":"2024-09-16T01:38:01.513000+0000","last_became_peered":"2024-09-16T01:38:01.513000+0000","last_unstale":"2024-09-16T01:39:09.516085+0000","last_undegraded":"2024-09-16T01:39:09.516085+0000","last_fullsized":"2024-09-16T01:39:09.516085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:43.036805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522826+0000","last_change":"2024-09-16T01:38:08.560560+0000","last_active":"2024-09-16T01:39:09.522826+0000","last_peered":"2024-09-16T01:39:09.522826+0000","last_clean":"2024-09-16T01:39:09.522826+0000","last_became_active":"2024-09-16T01:38:08.560229+0000","last_became_peered":"2024-09-16T01:38:08.560229+0000","last_unstale":"2024-09-16T01:39:09.522826+0000","last_undegraded":"2024-09-16T01:39:09.522826+0000","last_fullsized":"2024-09-16T01:39:09.522826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:10:09.462043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.048060+0000","last_change":"2024-09-16T01:38:01.529548+0000","last_active":"2024-09-16T01:39:10.048060+0000","last_peered":"2024-09-16T01:39:10.048060+0000","last_clean":"2024-09-16T01:39:10.048060+0000","last_became_active":"2024-09-16T01:38:01.529011+0000","last_became_peered":"2024-09-16T01:38:01.529011+0000","last_unstale":"2024-09-16T01:39:10.048060+0000","last_undegraded":"2024-09-16T01:39:10.048060+0000","last_fullsized":"2024-09-16T01:39:10.048060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:21.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.048107+0000","last_change":"2024-09-16T01:38:01.519488+0000","last_active":"2024-09-16T01:39:10.048107+0000","last_peered":"2024-09-16T01:39:10.048107+0000","last_clean":"2024-09-16T01:39:10.048107+0000","last_became_active":"2024-09-16T01:38:01.519156+0000","last_became_peered":"2024-09-16T01:38:01.519156+0000","last_unstale":"2024-09-16T01:39:10.048107+0000","last_undegraded":"2024-09-16T01:39:10.048107+0000","last_fullsized":"2024-09-16T01:39:10.048107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:18.149874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.520009+0000","last_change":"2024-09-16T01:38:01.527519+0000","last_active":"2024-09-16T01:39:09.520009+0000","last_peered":"2024-09-16T01:39:09.520009+0000","last_clean":"2024-09-16T01:39:09.520009+0000","last_became_active":"2024-09-16T01:38:01.527175+0000","last_became_peered":"2024-09-16T01:38:01.527175+0000","last_unstale":"2024-09-16T01:39:09.520009+0000","last_undegraded":"2024-09-16T01:39:09.520009+0000","last_fullsized":"2024-09-16T01:39:09.520009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:04.958842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68891,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":540,"num_read_kb":347,"num_write":248,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":206673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281435,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678048,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":954368,"data_stored":488610,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906981,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1097728,"data_stored":1141093,"data_compressed":59467,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070507,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":488035,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299439,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":950272,"data_stored":487629,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528372,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1019904,"data_stored":1084519,"data_compressed":59467,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":946176,"data_stored":483539,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888127,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1028096,"data_stored":1080101,"data_compressed":59467,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051652,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":954368,"data_stored":487613,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280585,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":958464,"data_stored":483247,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476814,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1032192,"data_stored":547258,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607635,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1019904,"data_stored":547317,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":970752,"data_stored":485388,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34155,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34155,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34155,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T01:39:34.656 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T01:39:34.901 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:34.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:34 smithi139 ceph-mon[31175]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:39:34.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:34 smithi176 ceph-mon[35324]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:39:35.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:34 smithi164 ceph-mon[35121]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:39:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.417+0000 7f02904d3640 1 -- 172.21.15.139:0/3817190718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0288100e20 msgr2=0x7f0288101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.417+0000 7f02904d3640 1 --2- 172.21.15.139:0/3817190718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0288100e20 0x7f0288101200 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0284009900 tx=0x7f028402a480 comp rx=0 tx=0).stop 2024-09-16T01:39:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.417+0000 7f02904d3640 1 -- 172.21.15.139:0/3817190718 shutdown_connections 2024-09-16T01:39:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.417+0000 7f02904d3640 1 --2- 172.21.15.139:0/3817190718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0288102ef0 0x7f028810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.417+0000 7f02904d3640 1 --2- 172.21.15.139:0/3817190718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0288102570 0x7f02881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.417+0000 7f02904d3640 1 --2- 172.21.15.139:0/3817190718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0288100e20 0x7f0288101200 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.417+0000 7f02904d3640 1 -- 172.21.15.139:0/3817190718 >> 172.21.15.139:0/3817190718 conn(0x7f02880fc7d0 msgr2=0x7f02880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.417+0000 7f02904d3640 1 -- 172.21.15.139:0/3817190718 shutdown_connections 2024-09-16T01:39:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.417+0000 7f02904d3640 1 -- 172.21.15.139:0/3817190718 wait complete. 2024-09-16T01:39:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.418+0000 7f02904d3640 1 Processor -- start 2024-09-16T01:39:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.418+0000 7f02904d3640 1 -- start start 2024-09-16T01:39:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.418+0000 7f02904d3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0288100e20 0x7f0288198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.418+0000 7f02904d3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0288102570 0x7f0288198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.418+0000 7f02904d3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0288102ef0 0x7f028819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.418+0000 7f02904d3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0288111020 con 0x7f0288102570 2024-09-16T01:39:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.418+0000 7f02904d3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0288110ea0 con 0x7f0288100e20 2024-09-16T01:39:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.418+0000 7f02904d3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f02881111a0 con 0x7f0288102ef0 2024-09-16T01:39:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.418+0000 7f028da6e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0288102570 0x7f0288198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.418+0000 7f028da6e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0288102570 0x7f0288198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39618/0 (socket says 172.21.15.139:39618) 2024-09-16T01:39:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.418+0000 7f028da6e640 1 -- 172.21.15.139:0/2239013003 learned_addr learned my addr 172.21.15.139:0/2239013003 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.418+0000 7f028e26f640 1 --2- 172.21.15.139:0/2239013003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0288100e20 0x7f0288198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.418+0000 7f028ea70640 1 --2- 172.21.15.139:0/2239013003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0288102ef0 0x7f028819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.419+0000 7f028da6e640 1 -- 172.21.15.139:0/2239013003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0288102ef0 msgr2=0x7f028819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.419+0000 7f028da6e640 1 --2- 172.21.15.139:0/2239013003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0288102ef0 0x7f028819cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:35.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.419+0000 7f028da6e640 1 -- 172.21.15.139:0/2239013003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0288100e20 msgr2=0x7f0288198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:35.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.419+0000 7f028da6e640 1 --2- 172.21.15.139:0/2239013003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0288100e20 0x7f0288198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:35.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.419+0000 7f028da6e640 1 -- 172.21.15.139:0/2239013003 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028819d620 con 0x7f0288102570 2024-09-16T01:39:35.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.419+0000 7f028da6e640 1 --2- 172.21.15.139:0/2239013003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0288102570 0x7f0288198bb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f027800b4a0 tx=0x7f027800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:35.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.419+0000 7f027f7fe640 1 -- 172.21.15.139:0/2239013003 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278014070 con 0x7f0288102570 2024-09-16T01:39:35.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.419+0000 7f02904d3640 1 -- 172.21.15.139:0/2239013003 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028819d910 con 0x7f0288102570 2024-09-16T01:39:35.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.419+0000 7f027f7fe640 1 -- 172.21.15.139:0/2239013003 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0278010af0 con 0x7f0288102570 2024-09-16T01:39:35.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.419+0000 7f027f7fe640 1 -- 172.21.15.139:0/2239013003 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278017d10 con 0x7f0288102570 2024-09-16T01:39:35.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.419+0000 7f02904d3640 1 -- 172.21.15.139:0/2239013003 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02881a51f0 con 0x7f0288102570 2024-09-16T01:39:35.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.420+0000 7f02904d3640 1 -- 172.21.15.139:0/2239013003 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0288101280 con 0x7f0288102570 2024-09-16T01:39:35.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.421+0000 7f027f7fe640 1 -- 172.21.15.139:0/2239013003 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0278017070 con 0x7f0288102570 2024-09-16T01:39:35.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.422+0000 7f027f7fe640 1 --2- 172.21.15.139:0/2239013003 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f02740a32d0 0x7f02740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:35.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.422+0000 7f028e26f640 1 --2- 172.21.15.139:0/2239013003 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f02740a32d0 0x7f02740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:35.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.422+0000 7f027f7fe640 1 -- 172.21.15.139:0/2239013003 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f02780cf080 con 0x7f0288102570 2024-09-16T01:39:35.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.422+0000 7f028e26f640 1 --2- 172.21.15.139:0/2239013003 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f02740a32d0 0x7f02740a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0284000c00 tx=0x7f02840058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:35.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.426+0000 7f027f7fe640 1 -- 172.21.15.139:0/2239013003 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f027809b330 con 0x7f0288102570 2024-09-16T01:39:35.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.598+0000 7f02904d3640 1 -- 172.21.15.139:0/2239013003 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0288062630 con 0x7f02740a32d0 2024-09-16T01:39:35.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.614+0000 7f027f7fe640 1 -- 172.21.15.139:0/2239013003 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340683 (secure 0 0 0) 0x7f0288062630 con 0x7f02740a32d0 2024-09-16T01:39:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:39:35.620 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T01:39:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.619+0000 7f02904d3640 1 -- 172.21.15.139:0/2239013003 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f02740a32d0 msgr2=0x7f02740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.619+0000 7f02904d3640 1 --2- 172.21.15.139:0/2239013003 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f02740a32d0 0x7f02740a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0284000c00 tx=0x7f02840058d0 comp rx=0 tx=0).stop 2024-09-16T01:39:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.619+0000 7f02904d3640 1 -- 172.21.15.139:0/2239013003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0288102570 msgr2=0x7f0288198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.619+0000 7f02904d3640 1 --2- 172.21.15.139:0/2239013003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0288102570 0x7f0288198bb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f027800b4a0 tx=0x7f027800b960 comp rx=0 tx=0).stop 2024-09-16T01:39:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.619+0000 7f02904d3640 1 -- 172.21.15.139:0/2239013003 shutdown_connections 2024-09-16T01:39:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.619+0000 7f02904d3640 1 --2- 172.21.15.139:0/2239013003 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f02740a32d0 0x7f02740a5790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.619+0000 7f02904d3640 1 --2- 172.21.15.139:0/2239013003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0288102ef0 0x7f028819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.619+0000 7f02904d3640 1 --2- 172.21.15.139:0/2239013003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0288102570 0x7f0288198bb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.619+0000 7f02904d3640 1 --2- 172.21.15.139:0/2239013003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0288100e20 0x7f0288198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.619+0000 7f02904d3640 1 -- 172.21.15.139:0/2239013003 >> 172.21.15.139:0/2239013003 conn(0x7f02880fc7d0 msgr2=0x7f028806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.619+0000 7f02904d3640 1 -- 172.21.15.139:0/2239013003 shutdown_connections 2024-09-16T01:39:35.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:35.619+0000 7f02904d3640 1 -- 172.21.15.139:0/2239013003 wait complete. 2024-09-16T01:39:35.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:35 smithi139 ceph-mon[31175]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:39:35.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:35 smithi139 ceph-mon[31175]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:39:35.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:35 smithi176 ceph-mon[35324]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:39:35.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:35 smithi176 ceph-mon[35324]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:39:36.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:35 smithi164 ceph-mon[35121]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:39:36.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:35 smithi164 ceph-mon[35121]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:39:36.510 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-16T01:39:35.476494+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659493,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":701,"num_read_kb":481,"num_write":431,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":352,"ondisk_log_size":352,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369280,"kb_used_data":11624,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572544,"statfs":{"total":1151940427776,"available":1151562285056,"internally_reserved":0,"allocated":11902976,"data_stored":7804349,"data_compressed":391458,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003716"},"pg_stats":[{"pgid":"2.1f","version":"83'13","reported_seq":111,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:30.642243+0000","last_change":"2024-09-16T01:38:08.567909+0000","last_active":"2024-09-16T01:39:30.642243+0000","last_peered":"2024-09-16T01:39:30.642243+0000","last_clean":"2024-09-16T01:39:30.642243+0000","last_became_active":"2024-09-16T01:38:08.560733+0000","last_became_peered":"2024-09-16T01:38:08.560733+0000","last_unstale":"2024-09-16T01:39:30.642243+0000","last_undegraded":"2024-09-16T01:39:30.642243+0000","last_fullsized":"2024-09-16T01:39:30.642243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:00.678040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518015+0000","last_change":"2024-09-16T01:38:01.523064+0000","last_active":"2024-09-16T01:39:09.518015+0000","last_peered":"2024-09-16T01:39:09.518015+0000","last_clean":"2024-09-16T01:39:09.518015+0000","last_became_active":"2024-09-16T01:38:01.521695+0000","last_became_peered":"2024-09-16T01:38:01.521695+0000","last_unstale":"2024-09-16T01:39:09.518015+0000","last_undegraded":"2024-09-16T01:39:09.518015+0000","last_fullsized":"2024-09-16T01:39:09.518015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:28.467115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.613727+0000","last_change":"2024-09-16T01:38:08.563557+0000","last_active":"2024-09-16T01:39:09.613727+0000","last_peered":"2024-09-16T01:39:09.613727+0000","last_clean":"2024-09-16T01:39:09.613727+0000","last_became_active":"2024-09-16T01:38:08.563065+0000","last_became_peered":"2024-09-16T01:38:08.563065+0000","last_unstale":"2024-09-16T01:39:09.613727+0000","last_undegraded":"2024-09-16T01:39:09.613727+0000","last_fullsized":"2024-09-16T01:39:09.613727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:56.992915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518208+0000","last_change":"2024-09-16T01:37:59.798884+0000","last_active":"2024-09-16T01:39:09.518208+0000","last_peered":"2024-09-16T01:39:09.518208+0000","last_clean":"2024-09-16T01:39:09.518208+0000","last_became_active":"2024-09-16T01:37:59.797989+0000","last_became_peered":"2024-09-16T01:37:59.797989+0000","last_unstale":"2024-09-16T01:39:09.518208+0000","last_undegraded":"2024-09-16T01:39:09.518208+0000","last_fullsized":"2024-09-16T01:39:09.518208+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:13.633078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.048261+0000","last_change":"2024-09-16T01:38:01.528391+0000","last_active":"2024-09-16T01:39:10.048261+0000","last_peered":"2024-09-16T01:39:10.048261+0000","last_clean":"2024-09-16T01:39:10.048261+0000","last_became_active":"2024-09-16T01:38:01.528076+0000","last_became_peered":"2024-09-16T01:38:01.528076+0000","last_unstale":"2024-09-16T01:39:10.048261+0000","last_undegraded":"2024-09-16T01:39:10.048261+0000","last_fullsized":"2024-09-16T01:39:10.048261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.105830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516404+0000","last_change":"2024-09-16T01:38:08.566198+0000","last_active":"2024-09-16T01:39:09.516404+0000","last_peered":"2024-09-16T01:39:09.516404+0000","last_clean":"2024-09-16T01:39:09.516404+0000","last_became_active":"2024-09-16T01:38:08.565739+0000","last_became_peered":"2024-09-16T01:38:08.565739+0000","last_unstale":"2024-09-16T01:39:09.516404+0000","last_undegraded":"2024-09-16T01:39:09.516404+0000","last_fullsized":"2024-09-16T01:39:09.516404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:05.172269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516713+0000","last_change":"2024-09-16T01:37:59.507625+0000","last_active":"2024-09-16T01:39:09.516713+0000","last_peered":"2024-09-16T01:39:09.516713+0000","last_clean":"2024-09-16T01:39:09.516713+0000","last_became_active":"2024-09-16T01:37:59.507280+0000","last_became_peered":"2024-09-16T01:37:59.507280+0000","last_unstale":"2024-09-16T01:39:09.516713+0000","last_undegraded":"2024-09-16T01:39:09.516713+0000","last_fullsized":"2024-09-16T01:39:09.516713+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:29:37.761203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639517+0000","last_change":"2024-09-16T01:38:01.528406+0000","last_active":"2024-09-16T01:39:09.639517+0000","last_peered":"2024-09-16T01:39:09.639517+0000","last_clean":"2024-09-16T01:39:09.639517+0000","last_became_active":"2024-09-16T01:38:01.528084+0000","last_became_peered":"2024-09-16T01:38:01.528084+0000","last_unstale":"2024-09-16T01:39:09.639517+0000","last_undegraded":"2024-09-16T01:39:09.639517+0000","last_fullsized":"2024-09-16T01:39:09.639517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:40.766305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516856+0000","last_change":"2024-09-16T01:38:08.563645+0000","last_active":"2024-09-16T01:39:09.516856+0000","last_peered":"2024-09-16T01:39:09.516856+0000","last_clean":"2024-09-16T01:39:09.516856+0000","last_became_active":"2024-09-16T01:38:08.563149+0000","last_became_peered":"2024-09-16T01:38:08.563149+0000","last_unstale":"2024-09-16T01:39:09.516856+0000","last_undegraded":"2024-09-16T01:39:09.516856+0000","last_fullsized":"2024-09-16T01:39:09.516856+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:09.010905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516923+0000","last_change":"2024-09-16T01:37:59.509649+0000","last_active":"2024-09-16T01:39:09.516923+0000","last_peered":"2024-09-16T01:39:09.516923+0000","last_clean":"2024-09-16T01:39:09.516923+0000","last_became_active":"2024-09-16T01:37:59.509236+0000","last_became_peered":"2024-09-16T01:37:59.509236+0000","last_unstale":"2024-09-16T01:39:09.516923+0000","last_undegraded":"2024-09-16T01:39:09.516923+0000","last_fullsized":"2024-09-16T01:39:09.516923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.614205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518130+0000","last_change":"2024-09-16T01:38:01.528074+0000","last_active":"2024-09-16T01:39:09.518130+0000","last_peered":"2024-09-16T01:39:09.518130+0000","last_clean":"2024-09-16T01:39:09.518130+0000","last_became_active":"2024-09-16T01:38:01.527737+0000","last_became_peered":"2024-09-16T01:38:01.527737+0000","last_unstale":"2024-09-16T01:39:09.518130+0000","last_undegraded":"2024-09-16T01:39:09.518130+0000","last_fullsized":"2024-09-16T01:39:09.518130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:37.592557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516662+0000","last_change":"2024-09-16T01:38:08.566340+0000","last_active":"2024-09-16T01:39:09.516662+0000","last_peered":"2024-09-16T01:39:09.516662+0000","last_clean":"2024-09-16T01:39:09.516662+0000","last_became_active":"2024-09-16T01:38:08.565791+0000","last_became_peered":"2024-09-16T01:38:08.565791+0000","last_unstale":"2024-09-16T01:39:09.516662+0000","last_undegraded":"2024-09-16T01:39:09.516662+0000","last_fullsized":"2024-09-16T01:39:09.516662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:35:31.845151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.521323+0000","last_change":"2024-09-16T01:38:01.527747+0000","last_active":"2024-09-16T01:39:09.521323+0000","last_peered":"2024-09-16T01:39:09.521323+0000","last_clean":"2024-09-16T01:39:09.521323+0000","last_became_active":"2024-09-16T01:38:01.527325+0000","last_became_peered":"2024-09-16T01:38:01.527325+0000","last_unstale":"2024-09-16T01:39:09.521323+0000","last_undegraded":"2024-09-16T01:39:09.521323+0000","last_fullsized":"2024-09-16T01:39:09.521323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:27.203719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516207+0000","last_change":"2024-09-16T01:37:59.510298+0000","last_active":"2024-09-16T01:39:09.516207+0000","last_peered":"2024-09-16T01:39:09.516207+0000","last_clean":"2024-09-16T01:39:09.516207+0000","last_became_active":"2024-09-16T01:37:59.509952+0000","last_became_peered":"2024-09-16T01:37:59.509952+0000","last_unstale":"2024-09-16T01:39:09.516207+0000","last_undegraded":"2024-09-16T01:39:09.516207+0000","last_fullsized":"2024-09-16T01:39:09.516207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:11.616846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516880+0000","last_change":"2024-09-16T01:38:08.571622+0000","last_active":"2024-09-16T01:39:09.516880+0000","last_peered":"2024-09-16T01:39:09.516880+0000","last_clean":"2024-09-16T01:39:09.516880+0000","last_became_active":"2024-09-16T01:38:08.571125+0000","last_became_peered":"2024-09-16T01:38:08.571125+0000","last_unstale":"2024-09-16T01:39:09.516880+0000","last_undegraded":"2024-09-16T01:39:09.516880+0000","last_fullsized":"2024-09-16T01:39:09.516880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:00.154633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517891+0000","last_change":"2024-09-16T01:38:01.519681+0000","last_active":"2024-09-16T01:39:09.517891+0000","last_peered":"2024-09-16T01:39:09.517891+0000","last_clean":"2024-09-16T01:39:09.517891+0000","last_became_active":"2024-09-16T01:38:01.518310+0000","last_became_peered":"2024-09-16T01:38:01.518310+0000","last_unstale":"2024-09-16T01:39:09.517891+0000","last_undegraded":"2024-09-16T01:39:09.517891+0000","last_fullsized":"2024-09-16T01:39:09.517891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:23.308008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519362+0000","last_change":"2024-09-16T01:38:08.559469+0000","last_active":"2024-09-16T01:39:09.519362+0000","last_peered":"2024-09-16T01:39:09.519362+0000","last_clean":"2024-09-16T01:39:09.519362+0000","last_became_active":"2024-09-16T01:38:08.559111+0000","last_became_peered":"2024-09-16T01:38:08.559111+0000","last_unstale":"2024-09-16T01:39:09.519362+0000","last_undegraded":"2024-09-16T01:39:09.519362+0000","last_fullsized":"2024-09-16T01:39:09.519362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:58.577749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516898+0000","last_change":"2024-09-16T01:38:08.555029+0000","last_active":"2024-09-16T01:39:09.516898+0000","last_peered":"2024-09-16T01:39:09.516898+0000","last_clean":"2024-09-16T01:39:09.516898+0000","last_became_active":"2024-09-16T01:38:08.554151+0000","last_became_peered":"2024-09-16T01:38:08.554151+0000","last_unstale":"2024-09-16T01:39:09.516898+0000","last_undegraded":"2024-09-16T01:39:09.516898+0000","last_fullsized":"2024-09-16T01:39:09.516898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:05.005904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519228+0000","last_change":"2024-09-16T01:38:01.528114+0000","last_active":"2024-09-16T01:39:09.519228+0000","last_peered":"2024-09-16T01:39:09.519228+0000","last_clean":"2024-09-16T01:39:09.519228+0000","last_became_active":"2024-09-16T01:38:01.527805+0000","last_became_peered":"2024-09-16T01:38:01.527805+0000","last_unstale":"2024-09-16T01:39:09.519228+0000","last_undegraded":"2024-09-16T01:39:09.519228+0000","last_fullsized":"2024-09-16T01:39:09.519228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:15.830936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518204+0000","last_change":"2024-09-16T01:37:59.507776+0000","last_active":"2024-09-16T01:39:09.518204+0000","last_peered":"2024-09-16T01:39:09.518204+0000","last_clean":"2024-09-16T01:39:09.518204+0000","last_became_active":"2024-09-16T01:37:59.507429+0000","last_became_peered":"2024-09-16T01:37:59.507429+0000","last_unstale":"2024-09-16T01:39:09.518204+0000","last_undegraded":"2024-09-16T01:39:09.518204+0000","last_fullsized":"2024-09-16T01:39:09.518204+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:44:41.274986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522573+0000","last_change":"2024-09-16T01:38:08.562191+0000","last_active":"2024-09-16T01:39:09.522573+0000","last_peered":"2024-09-16T01:39:09.522573+0000","last_clean":"2024-09-16T01:39:09.522573+0000","last_became_active":"2024-09-16T01:38:08.561839+0000","last_became_peered":"2024-09-16T01:38:08.561839+0000","last_unstale":"2024-09-16T01:39:09.522573+0000","last_undegraded":"2024-09-16T01:39:09.522573+0000","last_fullsized":"2024-09-16T01:39:09.522573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:40.618612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.515939+0000","last_change":"2024-09-16T01:38:01.518855+0000","last_active":"2024-09-16T01:39:09.515939+0000","last_peered":"2024-09-16T01:39:09.515939+0000","last_clean":"2024-09-16T01:39:09.515939+0000","last_became_active":"2024-09-16T01:38:01.518427+0000","last_became_peered":"2024-09-16T01:38:01.518427+0000","last_unstale":"2024-09-16T01:39:09.515939+0000","last_undegraded":"2024-09-16T01:39:09.515939+0000","last_fullsized":"2024-09-16T01:39:09.515939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:23.054856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639280+0000","last_change":"2024-09-16T01:37:59.799341+0000","last_active":"2024-09-16T01:39:09.639280+0000","last_peered":"2024-09-16T01:39:09.639280+0000","last_clean":"2024-09-16T01:39:09.639280+0000","last_became_active":"2024-09-16T01:37:59.798992+0000","last_became_peered":"2024-09-16T01:37:59.798992+0000","last_unstale":"2024-09-16T01:39:09.639280+0000","last_undegraded":"2024-09-16T01:39:09.639280+0000","last_fullsized":"2024-09-16T01:39:09.639280+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:01.708565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.524319+0000","last_change":"2024-09-16T01:39:09.524319+0000","last_active":"2024-09-16T01:39:09.524319+0000","last_peered":"2024-09-16T01:39:09.524319+0000","last_clean":"2024-09-16T01:39:09.524319+0000","last_became_active":"2024-09-16T01:39:09.523717+0000","last_became_peered":"2024-09-16T01:39:09.523717+0000","last_unstale":"2024-09-16T01:39:09.524319+0000","last_undegraded":"2024-09-16T01:39:09.524319+0000","last_fullsized":"2024-09-16T01:39:09.524319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:35.488097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.048153+0000","last_change":"2024-09-16T01:38:01.524545+0000","last_active":"2024-09-16T01:39:10.048153+0000","last_peered":"2024-09-16T01:39:10.048153+0000","last_clean":"2024-09-16T01:39:10.048153+0000","last_became_active":"2024-09-16T01:38:01.524253+0000","last_became_peered":"2024-09-16T01:38:01.524253+0000","last_unstale":"2024-09-16T01:39:10.048153+0000","last_undegraded":"2024-09-16T01:39:10.048153+0000","last_fullsized":"2024-09-16T01:39:10.048153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:08.697500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517439+0000","last_change":"2024-09-16T01:37:59.506779+0000","last_active":"2024-09-16T01:39:09.517439+0000","last_peered":"2024-09-16T01:39:09.517439+0000","last_clean":"2024-09-16T01:39:09.517439+0000","last_became_active":"2024-09-16T01:37:59.505459+0000","last_became_peered":"2024-09-16T01:37:59.505459+0000","last_unstale":"2024-09-16T01:39:09.517439+0000","last_undegraded":"2024-09-16T01:39:09.517439+0000","last_fullsized":"2024-09-16T01:39:09.517439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:12.973330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.613873+0000","last_change":"2024-09-16T01:38:08.557250+0000","last_active":"2024-09-16T01:39:09.613873+0000","last_peered":"2024-09-16T01:39:09.613873+0000","last_clean":"2024-09-16T01:39:09.613873+0000","last_became_active":"2024-09-16T01:38:08.555577+0000","last_became_peered":"2024-09-16T01:38:08.555577+0000","last_unstale":"2024-09-16T01:39:09.613873+0000","last_undegraded":"2024-09-16T01:39:09.613873+0000","last_fullsized":"2024-09-16T01:39:09.613873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:54.965341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518312+0000","last_change":"2024-09-16T01:38:01.522910+0000","last_active":"2024-09-16T01:39:09.518312+0000","last_peered":"2024-09-16T01:39:09.518312+0000","last_clean":"2024-09-16T01:39:09.518312+0000","last_became_active":"2024-09-16T01:38:01.521206+0000","last_became_peered":"2024-09-16T01:38:01.521206+0000","last_unstale":"2024-09-16T01:39:09.518312+0000","last_undegraded":"2024-09-16T01:39:09.518312+0000","last_fullsized":"2024-09-16T01:39:09.518312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:45.670464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'16","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522912+0000","last_change":"2024-09-16T01:37:59.505962+0000","last_active":"2024-09-16T01:39:09.522912+0000","last_peered":"2024-09-16T01:39:09.522912+0000","last_clean":"2024-09-16T01:39:09.522912+0000","last_became_active":"2024-09-16T01:37:59.505636+0000","last_became_peered":"2024-09-16T01:37:59.505636+0000","last_unstale":"2024-09-16T01:39:09.522912+0000","last_undegraded":"2024-09-16T01:39:09.522912+0000","last_fullsized":"2024-09-16T01:39:09.522912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:57.548513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517568+0000","last_change":"2024-09-16T01:38:08.571249+0000","last_active":"2024-09-16T01:39:09.517568+0000","last_peered":"2024-09-16T01:39:09.517568+0000","last_clean":"2024-09-16T01:39:09.517568+0000","last_became_active":"2024-09-16T01:38:08.570388+0000","last_became_peered":"2024-09-16T01:38:08.570388+0000","last_unstale":"2024-09-16T01:39:09.517568+0000","last_undegraded":"2024-09-16T01:39:09.517568+0000","last_fullsized":"2024-09-16T01:39:09.517568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:59.904704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611378+0000","last_change":"2024-09-16T01:38:01.522040+0000","last_active":"2024-09-16T01:39:09.611378+0000","last_peered":"2024-09-16T01:39:09.611378+0000","last_clean":"2024-09-16T01:39:09.611378+0000","last_became_active":"2024-09-16T01:38:01.521720+0000","last_became_peered":"2024-09-16T01:38:01.521720+0000","last_unstale":"2024-09-16T01:39:09.611378+0000","last_undegraded":"2024-09-16T01:39:09.611378+0000","last_fullsized":"2024-09-16T01:39:09.611378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:44:48.635603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519678+0000","last_change":"2024-09-16T01:38:08.561870+0000","last_active":"2024-09-16T01:39:09.519678+0000","last_peered":"2024-09-16T01:39:09.519678+0000","last_clean":"2024-09-16T01:39:09.519678+0000","last_became_active":"2024-09-16T01:38:08.561440+0000","last_became_peered":"2024-09-16T01:38:08.561440+0000","last_unstale":"2024-09-16T01:39:09.519678+0000","last_undegraded":"2024-09-16T01:39:09.519678+0000","last_fullsized":"2024-09-16T01:39:09.519678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:43:37.594734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519427+0000","last_change":"2024-09-16T01:38:08.555436+0000","last_active":"2024-09-16T01:39:09.519427+0000","last_peered":"2024-09-16T01:39:09.519427+0000","last_clean":"2024-09-16T01:39:09.519427+0000","last_became_active":"2024-09-16T01:38:08.555026+0000","last_became_peered":"2024-09-16T01:38:08.555026+0000","last_unstale":"2024-09-16T01:39:09.519427+0000","last_undegraded":"2024-09-16T01:39:09.519427+0000","last_fullsized":"2024-09-16T01:39:09.519427+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:28.870214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518712+0000","last_change":"2024-09-16T01:38:01.524482+0000","last_active":"2024-09-16T01:39:09.518712+0000","last_peered":"2024-09-16T01:39:09.518712+0000","last_clean":"2024-09-16T01:39:09.518712+0000","last_became_active":"2024-09-16T01:38:01.524160+0000","last_became_peered":"2024-09-16T01:38:01.524160+0000","last_unstale":"2024-09-16T01:39:09.518712+0000","last_undegraded":"2024-09-16T01:39:09.518712+0000","last_fullsized":"2024-09-16T01:39:09.518712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:39:27.621780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522927+0000","last_change":"2024-09-16T01:37:59.800051+0000","last_active":"2024-09-16T01:39:09.522927+0000","last_peered":"2024-09-16T01:39:09.522927+0000","last_clean":"2024-09-16T01:39:09.522927+0000","last_became_active":"2024-09-16T01:37:59.799606+0000","last_became_peered":"2024-09-16T01:37:59.799606+0000","last_unstale":"2024-09-16T01:39:09.522927+0000","last_undegraded":"2024-09-16T01:39:09.522927+0000","last_fullsized":"2024-09-16T01:39:09.522927+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:02.694933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.515939+0000","last_change":"2024-09-16T01:38:08.570764+0000","last_active":"2024-09-16T01:39:09.515939+0000","last_peered":"2024-09-16T01:39:09.515939+0000","last_clean":"2024-09-16T01:39:09.515939+0000","last_became_active":"2024-09-16T01:38:08.570385+0000","last_became_peered":"2024-09-16T01:38:08.570385+0000","last_unstale":"2024-09-16T01:39:09.515939+0000","last_undegraded":"2024-09-16T01:39:09.515939+0000","last_fullsized":"2024-09-16T01:39:09.515939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:25.349671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519067+0000","last_change":"2024-09-16T01:38:01.519590+0000","last_active":"2024-09-16T01:39:09.519067+0000","last_peered":"2024-09-16T01:39:09.519067+0000","last_clean":"2024-09-16T01:39:09.519067+0000","last_became_active":"2024-09-16T01:38:01.519210+0000","last_became_peered":"2024-09-16T01:38:01.519210+0000","last_unstale":"2024-09-16T01:39:09.519067+0000","last_undegraded":"2024-09-16T01:39:09.519067+0000","last_fullsized":"2024-09-16T01:39:09.519067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:04.228131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":90,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:28.640295+0000","last_change":"2024-09-16T01:37:59.505473+0000","last_active":"2024-09-16T01:39:28.640295+0000","last_peered":"2024-09-16T01:39:28.640295+0000","last_clean":"2024-09-16T01:39:28.640295+0000","last_became_active":"2024-09-16T01:37:59.505123+0000","last_became_peered":"2024-09-16T01:37:59.505123+0000","last_unstale":"2024-09-16T01:39:28.640295+0000","last_undegraded":"2024-09-16T01:39:28.640295+0000","last_fullsized":"2024-09-16T01:39:28.640295+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:17.739625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611176+0000","last_change":"2024-09-16T01:38:08.558534+0000","last_active":"2024-09-16T01:39:09.611176+0000","last_peered":"2024-09-16T01:39:09.611176+0000","last_clean":"2024-09-16T01:39:09.611176+0000","last_became_active":"2024-09-16T01:38:08.558185+0000","last_became_peered":"2024-09-16T01:38:08.558185+0000","last_unstale":"2024-09-16T01:39:09.611176+0000","last_undegraded":"2024-09-16T01:39:09.611176+0000","last_fullsized":"2024-09-16T01:39:09.611176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:09.359426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516572+0000","last_change":"2024-09-16T01:38:01.521445+0000","last_active":"2024-09-16T01:39:09.516572+0000","last_peered":"2024-09-16T01:39:09.516572+0000","last_clean":"2024-09-16T01:39:09.516572+0000","last_became_active":"2024-09-16T01:38:01.521140+0000","last_became_peered":"2024-09-16T01:38:01.521140+0000","last_unstale":"2024-09-16T01:39:09.516572+0000","last_undegraded":"2024-09-16T01:39:09.516572+0000","last_fullsized":"2024-09-16T01:39:09.516572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:05.247515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":91,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:30.638331+0000","last_change":"2024-09-16T01:37:59.798365+0000","last_active":"2024-09-16T01:39:30.638331+0000","last_peered":"2024-09-16T01:39:30.638331+0000","last_clean":"2024-09-16T01:39:30.638331+0000","last_became_active":"2024-09-16T01:37:59.798023+0000","last_became_peered":"2024-09-16T01:37:59.798023+0000","last_unstale":"2024-09-16T01:39:30.638331+0000","last_undegraded":"2024-09-16T01:39:30.638331+0000","last_fullsized":"2024-09-16T01:39:30.638331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:30.867080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611152+0000","last_change":"2024-09-16T01:38:08.571704+0000","last_active":"2024-09-16T01:39:09.611152+0000","last_peered":"2024-09-16T01:39:09.611152+0000","last_clean":"2024-09-16T01:39:09.611152+0000","last_became_active":"2024-09-16T01:38:08.571350+0000","last_became_peered":"2024-09-16T01:38:08.571350+0000","last_unstale":"2024-09-16T01:39:09.611152+0000","last_undegraded":"2024-09-16T01:39:09.611152+0000","last_fullsized":"2024-09-16T01:39:09.611152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:01.698770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.610862+0000","last_change":"2024-09-16T01:38:08.559433+0000","last_active":"2024-09-16T01:39:09.610862+0000","last_peered":"2024-09-16T01:39:09.610862+0000","last_clean":"2024-09-16T01:39:09.610862+0000","last_became_active":"2024-09-16T01:38:08.559110+0000","last_became_peered":"2024-09-16T01:38:08.559110+0000","last_unstale":"2024-09-16T01:39:09.610862+0000","last_undegraded":"2024-09-16T01:39:09.610862+0000","last_fullsized":"2024-09-16T01:39:09.610862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:39.001916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.520680+0000","last_change":"2024-09-16T01:37:59.508236+0000","last_active":"2024-09-16T01:39:09.520680+0000","last_peered":"2024-09-16T01:39:09.520680+0000","last_clean":"2024-09-16T01:39:09.520680+0000","last_became_active":"2024-09-16T01:37:59.507883+0000","last_became_peered":"2024-09-16T01:37:59.507883+0000","last_unstale":"2024-09-16T01:39:09.520680+0000","last_undegraded":"2024-09-16T01:39:09.520680+0000","last_fullsized":"2024-09-16T01:39:09.520680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:23.948530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518000+0000","last_change":"2024-09-16T01:38:08.572108+0000","last_active":"2024-09-16T01:39:09.518000+0000","last_peered":"2024-09-16T01:39:09.518000+0000","last_clean":"2024-09-16T01:39:09.518000+0000","last_became_active":"2024-09-16T01:38:08.571643+0000","last_became_peered":"2024-09-16T01:38:08.571643+0000","last_unstale":"2024-09-16T01:39:09.518000+0000","last_undegraded":"2024-09-16T01:39:09.518000+0000","last_fullsized":"2024-09-16T01:39:09.518000+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:46.636219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517895+0000","last_change":"2024-09-16T01:38:01.517894+0000","last_active":"2024-09-16T01:39:09.517895+0000","last_peered":"2024-09-16T01:39:09.517895+0000","last_clean":"2024-09-16T01:39:09.517895+0000","last_became_active":"2024-09-16T01:38:01.517602+0000","last_became_peered":"2024-09-16T01:38:01.517602+0000","last_unstale":"2024-09-16T01:39:09.517895+0000","last_undegraded":"2024-09-16T01:39:09.517895+0000","last_fullsized":"2024-09-16T01:39:09.517895+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:33.483120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517111+0000","last_change":"2024-09-16T01:37:59.507568+0000","last_active":"2024-09-16T01:39:09.517111+0000","last_peered":"2024-09-16T01:39:09.517111+0000","last_clean":"2024-09-16T01:39:09.517111+0000","last_became_active":"2024-09-16T01:37:59.505798+0000","last_became_peered":"2024-09-16T01:37:59.505798+0000","last_unstale":"2024-09-16T01:39:09.517111+0000","last_undegraded":"2024-09-16T01:39:09.517111+0000","last_fullsized":"2024-09-16T01:39:09.517111+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:19:02.711054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":358,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.520927+0000","last_change":"2024-09-16T01:36:46.744332+0000","last_active":"2024-09-16T01:39:09.520927+0000","last_peered":"2024-09-16T01:39:09.520927+0000","last_clean":"2024-09-16T01:39:09.520927+0000","last_became_active":"2024-09-16T01:36:46.743902+0000","last_became_peered":"2024-09-16T01:36:46.743902+0000","last_unstale":"2024-09-16T01:39:09.520927+0000","last_undegraded":"2024-09-16T01:39:09.520927+0000","last_fullsized":"2024-09-16T01:39:09.520927+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:33:19.891854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:33:19.891854+0000","last_clean_scrub_stamp":"2024-09-16T01:33:19.891854+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:46.173685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.528216+0000","last_change":"2024-09-16T01:39:09.528216+0000","last_active":"2024-09-16T01:39:09.528216+0000","last_peered":"2024-09-16T01:39:09.528216+0000","last_clean":"2024-09-16T01:39:09.528216+0000","last_became_active":"2024-09-16T01:39:09.523634+0000","last_became_peered":"2024-09-16T01:39:09.523634+0000","last_unstale":"2024-09-16T01:39:09.528216+0000","last_undegraded":"2024-09-16T01:39:09.528216+0000","last_fullsized":"2024-09-16T01:39:09.528216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:27.989901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522071+0000","last_change":"2024-09-16T01:38:01.527690+0000","last_active":"2024-09-16T01:39:09.522071+0000","last_peered":"2024-09-16T01:39:09.522071+0000","last_clean":"2024-09-16T01:39:09.522071+0000","last_became_active":"2024-09-16T01:38:01.527339+0000","last_became_peered":"2024-09-16T01:38:01.527339+0000","last_unstale":"2024-09-16T01:39:09.522071+0000","last_undegraded":"2024-09-16T01:39:09.522071+0000","last_fullsized":"2024-09-16T01:39:09.522071+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:58.526270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":97,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:32.645841+0000","last_change":"2024-09-16T01:37:59.798874+0000","last_active":"2024-09-16T01:39:32.645841+0000","last_peered":"2024-09-16T01:39:32.645841+0000","last_clean":"2024-09-16T01:39:32.645841+0000","last_became_active":"2024-09-16T01:37:59.798424+0000","last_became_peered":"2024-09-16T01:37:59.798424+0000","last_unstale":"2024-09-16T01:39:32.645841+0000","last_undegraded":"2024-09-16T01:39:32.645841+0000","last_fullsized":"2024-09-16T01:39:32.645841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:36.648963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639242+0000","last_change":"2024-09-16T01:38:08.555524+0000","last_active":"2024-09-16T01:39:09.639242+0000","last_peered":"2024-09-16T01:39:09.639242+0000","last_clean":"2024-09-16T01:39:09.639242+0000","last_became_active":"2024-09-16T01:38:08.555075+0000","last_became_peered":"2024-09-16T01:38:08.555075+0000","last_unstale":"2024-09-16T01:39:09.639242+0000","last_undegraded":"2024-09-16T01:39:09.639242+0000","last_fullsized":"2024-09-16T01:39:09.639242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:44.847916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.521685+0000","last_change":"2024-09-16T01:38:01.528869+0000","last_active":"2024-09-16T01:39:09.521685+0000","last_peered":"2024-09-16T01:39:09.521685+0000","last_clean":"2024-09-16T01:39:09.521685+0000","last_became_active":"2024-09-16T01:38:01.528559+0000","last_became_peered":"2024-09-16T01:38:01.528559+0000","last_unstale":"2024-09-16T01:39:09.521685+0000","last_undegraded":"2024-09-16T01:39:09.521685+0000","last_fullsized":"2024-09-16T01:39:09.521685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:21.547808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518564+0000","last_change":"2024-09-16T01:37:59.510871+0000","last_active":"2024-09-16T01:39:09.518564+0000","last_peered":"2024-09-16T01:39:09.518564+0000","last_clean":"2024-09-16T01:39:09.518564+0000","last_became_active":"2024-09-16T01:37:59.509996+0000","last_became_peered":"2024-09-16T01:37:59.509996+0000","last_unstale":"2024-09-16T01:39:09.518564+0000","last_undegraded":"2024-09-16T01:39:09.518564+0000","last_fullsized":"2024-09-16T01:39:09.518564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:23.390864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.047868+0000","last_change":"2024-09-16T01:38:08.555181+0000","last_active":"2024-09-16T01:39:10.047868+0000","last_peered":"2024-09-16T01:39:10.047868+0000","last_clean":"2024-09-16T01:39:10.047868+0000","last_became_active":"2024-09-16T01:38:08.554778+0000","last_became_peered":"2024-09-16T01:38:08.554778+0000","last_unstale":"2024-09-16T01:39:10.047868+0000","last_undegraded":"2024-09-16T01:39:10.047868+0000","last_fullsized":"2024-09-16T01:39:10.047868+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:16:42.369236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522624+0000","last_change":"2024-09-16T01:38:01.526260+0000","last_active":"2024-09-16T01:39:09.522624+0000","last_peered":"2024-09-16T01:39:09.522624+0000","last_clean":"2024-09-16T01:39:09.522624+0000","last_became_active":"2024-09-16T01:38:01.525652+0000","last_became_peered":"2024-09-16T01:38:01.525652+0000","last_unstale":"2024-09-16T01:39:09.522624+0000","last_undegraded":"2024-09-16T01:39:09.522624+0000","last_fullsized":"2024-09-16T01:39:09.522624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:09.180507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611389+0000","last_change":"2024-09-16T01:38:08.570408+0000","last_active":"2024-09-16T01:39:09.611389+0000","last_peered":"2024-09-16T01:39:09.611389+0000","last_clean":"2024-09-16T01:39:09.611389+0000","last_became_active":"2024-09-16T01:38:08.566122+0000","last_became_peered":"2024-09-16T01:38:08.566122+0000","last_unstale":"2024-09-16T01:39:09.611389+0000","last_undegraded":"2024-09-16T01:39:09.611389+0000","last_fullsized":"2024-09-16T01:39:09.611389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:55.832320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516212+0000","last_change":"2024-09-16T01:38:08.555063+0000","last_active":"2024-09-16T01:39:09.516212+0000","last_peered":"2024-09-16T01:39:09.516212+0000","last_clean":"2024-09-16T01:39:09.516212+0000","last_became_active":"2024-09-16T01:38:08.554665+0000","last_became_peered":"2024-09-16T01:38:08.554665+0000","last_unstale":"2024-09-16T01:39:09.516212+0000","last_undegraded":"2024-09-16T01:39:09.516212+0000","last_fullsized":"2024-09-16T01:39:09.516212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:50.091017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519687+0000","last_change":"2024-09-16T01:38:01.524247+0000","last_active":"2024-09-16T01:39:09.519687+0000","last_peered":"2024-09-16T01:39:09.519687+0000","last_clean":"2024-09-16T01:39:09.519687+0000","last_became_active":"2024-09-16T01:38:01.523911+0000","last_became_peered":"2024-09-16T01:38:01.523911+0000","last_unstale":"2024-09-16T01:39:09.519687+0000","last_undegraded":"2024-09-16T01:39:09.519687+0000","last_fullsized":"2024-09-16T01:39:09.519687+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:32.745882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516885+0000","last_change":"2024-09-16T01:37:59.511272+0000","last_active":"2024-09-16T01:39:09.516885+0000","last_peered":"2024-09-16T01:39:09.516885+0000","last_clean":"2024-09-16T01:39:09.516885+0000","last_became_active":"2024-09-16T01:37:59.510920+0000","last_became_peered":"2024-09-16T01:37:59.510920+0000","last_unstale":"2024-09-16T01:39:09.516885+0000","last_undegraded":"2024-09-16T01:39:09.516885+0000","last_fullsized":"2024-09-16T01:39:09.516885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:59.406175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517305+0000","last_change":"2024-09-16T01:38:08.567173+0000","last_active":"2024-09-16T01:39:09.517305+0000","last_peered":"2024-09-16T01:39:09.517305+0000","last_clean":"2024-09-16T01:39:09.517305+0000","last_became_active":"2024-09-16T01:38:08.566788+0000","last_became_peered":"2024-09-16T01:38:08.566788+0000","last_unstale":"2024-09-16T01:39:09.517305+0000","last_undegraded":"2024-09-16T01:39:09.517305+0000","last_fullsized":"2024-09-16T01:39:09.517305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:35.743356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517105+0000","last_change":"2024-09-16T01:38:01.529529+0000","last_active":"2024-09-16T01:39:09.517105+0000","last_peered":"2024-09-16T01:39:09.517105+0000","last_clean":"2024-09-16T01:39:09.517105+0000","last_became_active":"2024-09-16T01:38:01.529048+0000","last_became_peered":"2024-09-16T01:38:01.529048+0000","last_unstale":"2024-09-16T01:39:09.517105+0000","last_undegraded":"2024-09-16T01:39:09.517105+0000","last_fullsized":"2024-09-16T01:39:09.517105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:35.041662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516028+0000","last_change":"2024-09-16T01:37:59.799536+0000","last_active":"2024-09-16T01:39:09.516028+0000","last_peered":"2024-09-16T01:39:09.516028+0000","last_clean":"2024-09-16T01:39:09.516028+0000","last_became_active":"2024-09-16T01:37:59.799109+0000","last_became_peered":"2024-09-16T01:37:59.799109+0000","last_unstale":"2024-09-16T01:39:09.516028+0000","last_undegraded":"2024-09-16T01:39:09.516028+0000","last_fullsized":"2024-09-16T01:39:09.516028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:07:25.007434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611392+0000","last_change":"2024-09-16T01:38:08.567019+0000","last_active":"2024-09-16T01:39:09.611392+0000","last_peered":"2024-09-16T01:39:09.611392+0000","last_clean":"2024-09-16T01:39:09.611392+0000","last_became_active":"2024-09-16T01:38:08.566667+0000","last_became_peered":"2024-09-16T01:38:08.566667+0000","last_unstale":"2024-09-16T01:39:09.611392+0000","last_undegraded":"2024-09-16T01:39:09.611392+0000","last_fullsized":"2024-09-16T01:39:09.611392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:11.375095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.520500+0000","last_change":"2024-09-16T01:38:01.527965+0000","last_active":"2024-09-16T01:39:09.520500+0000","last_peered":"2024-09-16T01:39:09.520500+0000","last_clean":"2024-09-16T01:39:09.520500+0000","last_became_active":"2024-09-16T01:38:01.526688+0000","last_became_peered":"2024-09-16T01:38:01.526688+0000","last_unstale":"2024-09-16T01:39:09.520500+0000","last_undegraded":"2024-09-16T01:39:09.520500+0000","last_fullsized":"2024-09-16T01:39:09.520500+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:06.618756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":91,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:28.636124+0000","last_change":"2024-09-16T01:37:59.510331+0000","last_active":"2024-09-16T01:39:28.636124+0000","last_peered":"2024-09-16T01:39:28.636124+0000","last_clean":"2024-09-16T01:39:28.636124+0000","last_became_active":"2024-09-16T01:37:59.509769+0000","last_became_peered":"2024-09-16T01:37:59.509769+0000","last_unstale":"2024-09-16T01:39:28.636124+0000","last_undegraded":"2024-09-16T01:39:28.636124+0000","last_fullsized":"2024-09-16T01:39:28.636124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:18.843091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.049893+0000","last_change":"2024-09-16T01:39:10.049460+0000","last_active":"2024-09-16T01:39:10.049893+0000","last_peered":"2024-09-16T01:39:10.049893+0000","last_clean":"2024-09-16T01:39:10.049893+0000","last_became_active":"2024-09-16T01:39:10.048862+0000","last_became_peered":"2024-09-16T01:39:10.048862+0000","last_unstale":"2024-09-16T01:39:10.049893+0000","last_undegraded":"2024-09-16T01:39:10.049893+0000","last_fullsized":"2024-09-16T01:39:10.049893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:37.329806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.521463+0000","last_change":"2024-09-16T01:38:01.521254+0000","last_active":"2024-09-16T01:39:09.521463+0000","last_peered":"2024-09-16T01:39:09.521463+0000","last_clean":"2024-09-16T01:39:09.521463+0000","last_became_active":"2024-09-16T01:38:01.520901+0000","last_became_peered":"2024-09-16T01:38:01.520901+0000","last_unstale":"2024-09-16T01:39:09.521463+0000","last_undegraded":"2024-09-16T01:39:09.521463+0000","last_fullsized":"2024-09-16T01:39:09.521463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:27.806954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517118+0000","last_change":"2024-09-16T01:37:59.505990+0000","last_active":"2024-09-16T01:39:09.517118+0000","last_peered":"2024-09-16T01:39:09.517118+0000","last_clean":"2024-09-16T01:39:09.517118+0000","last_became_active":"2024-09-16T01:37:59.505564+0000","last_became_peered":"2024-09-16T01:37:59.505564+0000","last_unstale":"2024-09-16T01:39:09.517118+0000","last_undegraded":"2024-09-16T01:39:09.517118+0000","last_fullsized":"2024-09-16T01:39:09.517118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:01.496609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519430+0000","last_change":"2024-09-16T01:38:08.562838+0000","last_active":"2024-09-16T01:39:09.519430+0000","last_peered":"2024-09-16T01:39:09.519430+0000","last_clean":"2024-09-16T01:39:09.519430+0000","last_became_active":"2024-09-16T01:38:08.562426+0000","last_became_peered":"2024-09-16T01:38:08.562426+0000","last_unstale":"2024-09-16T01:39:09.519430+0000","last_undegraded":"2024-09-16T01:39:09.519430+0000","last_fullsized":"2024-09-16T01:39:09.519430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:20.685626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611059+0000","last_change":"2024-09-16T01:38:01.514983+0000","last_active":"2024-09-16T01:39:09.611059+0000","last_peered":"2024-09-16T01:39:09.611059+0000","last_clean":"2024-09-16T01:39:09.611059+0000","last_became_active":"2024-09-16T01:38:01.514648+0000","last_became_peered":"2024-09-16T01:38:01.514648+0000","last_unstale":"2024-09-16T01:39:09.611059+0000","last_undegraded":"2024-09-16T01:39:09.611059+0000","last_fullsized":"2024-09-16T01:39:09.611059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:57.052564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.614178+0000","last_change":"2024-09-16T01:37:59.505141+0000","last_active":"2024-09-16T01:39:09.614178+0000","last_peered":"2024-09-16T01:39:09.614178+0000","last_clean":"2024-09-16T01:39:09.614178+0000","last_became_active":"2024-09-16T01:37:59.504730+0000","last_became_peered":"2024-09-16T01:37:59.504730+0000","last_unstale":"2024-09-16T01:39:09.614178+0000","last_undegraded":"2024-09-16T01:39:09.614178+0000","last_fullsized":"2024-09-16T01:39:09.614178+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:54.504661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517789+0000","last_change":"2024-09-16T01:38:08.563970+0000","last_active":"2024-09-16T01:39:09.517789+0000","last_peered":"2024-09-16T01:39:09.517789+0000","last_clean":"2024-09-16T01:39:09.517789+0000","last_became_active":"2024-09-16T01:38:08.563555+0000","last_became_peered":"2024-09-16T01:38:08.563555+0000","last_unstale":"2024-09-16T01:39:09.517789+0000","last_undegraded":"2024-09-16T01:39:09.517789+0000","last_fullsized":"2024-09-16T01:39:09.517789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:27:12.086043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516577+0000","last_change":"2024-09-16T01:38:01.521016+0000","last_active":"2024-09-16T01:39:09.516577+0000","last_peered":"2024-09-16T01:39:09.516577+0000","last_clean":"2024-09-16T01:39:09.516577+0000","last_became_active":"2024-09-16T01:38:01.520656+0000","last_became_peered":"2024-09-16T01:38:01.520656+0000","last_unstale":"2024-09-16T01:39:09.516577+0000","last_undegraded":"2024-09-16T01:39:09.516577+0000","last_fullsized":"2024-09-16T01:39:09.516577+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:39.001239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'34","reported_seq":185,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:33.642311+0000","last_change":"2024-09-16T01:37:59.507235+0000","last_active":"2024-09-16T01:39:33.642311+0000","last_peered":"2024-09-16T01:39:33.642311+0000","last_clean":"2024-09-16T01:39:33.642311+0000","last_became_active":"2024-09-16T01:37:59.506255+0000","last_became_peered":"2024-09-16T01:37:59.506255+0000","last_unstale":"2024-09-16T01:39:33.642311+0000","last_undegraded":"2024-09-16T01:39:33.642311+0000","last_fullsized":"2024-09-16T01:39:33.642311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:44.515663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":117,"num_read_kb":119,"num_write":36,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522688+0000","last_change":"2024-09-16T01:39:09.522688+0000","last_active":"2024-09-16T01:39:09.522688+0000","last_peered":"2024-09-16T01:39:09.522688+0000","last_clean":"2024-09-16T01:39:09.522688+0000","last_became_active":"2024-09-16T01:39:09.522137+0000","last_became_peered":"2024-09-16T01:39:09.522137+0000","last_unstale":"2024-09-16T01:39:09.522688+0000","last_undegraded":"2024-09-16T01:39:09.522688+0000","last_fullsized":"2024-09-16T01:39:09.522688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.004315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639485+0000","last_change":"2024-09-16T01:38:01.526641+0000","last_active":"2024-09-16T01:39:09.639485+0000","last_peered":"2024-09-16T01:39:09.639485+0000","last_clean":"2024-09-16T01:39:09.639485+0000","last_became_active":"2024-09-16T01:38:01.526278+0000","last_became_peered":"2024-09-16T01:38:01.526278+0000","last_unstale":"2024-09-16T01:39:09.639485+0000","last_undegraded":"2024-09-16T01:39:09.639485+0000","last_fullsized":"2024-09-16T01:39:09.639485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:56.156593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518309+0000","last_change":"2024-09-16T01:37:59.800065+0000","last_active":"2024-09-16T01:39:09.518309+0000","last_peered":"2024-09-16T01:39:09.518309+0000","last_clean":"2024-09-16T01:39:09.518309+0000","last_became_active":"2024-09-16T01:37:59.799705+0000","last_became_peered":"2024-09-16T01:37:59.799705+0000","last_unstale":"2024-09-16T01:39:09.518309+0000","last_undegraded":"2024-09-16T01:39:09.518309+0000","last_fullsized":"2024-09-16T01:39:09.518309+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:13.791875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517087+0000","last_change":"2024-09-16T01:38:08.556360+0000","last_active":"2024-09-16T01:39:09.517087+0000","last_peered":"2024-09-16T01:39:09.517087+0000","last_clean":"2024-09-16T01:39:09.517087+0000","last_became_active":"2024-09-16T01:38:08.555219+0000","last_became_peered":"2024-09-16T01:38:08.555219+0000","last_unstale":"2024-09-16T01:39:09.517087+0000","last_undegraded":"2024-09-16T01:39:09.517087+0000","last_fullsized":"2024-09-16T01:39:09.517087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:36.464760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.048288+0000","last_change":"2024-09-16T01:38:01.526758+0000","last_active":"2024-09-16T01:39:10.048288+0000","last_peered":"2024-09-16T01:39:10.048288+0000","last_clean":"2024-09-16T01:39:10.048288+0000","last_became_active":"2024-09-16T01:38:01.526545+0000","last_became_peered":"2024-09-16T01:38:01.526545+0000","last_unstale":"2024-09-16T01:39:10.048288+0000","last_undegraded":"2024-09-16T01:39:10.048288+0000","last_fullsized":"2024-09-16T01:39:10.048288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:47.185489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522440+0000","last_change":"2024-09-16T01:38:08.555125+0000","last_active":"2024-09-16T01:39:09.522440+0000","last_peered":"2024-09-16T01:39:09.522440+0000","last_clean":"2024-09-16T01:39:09.522440+0000","last_became_active":"2024-09-16T01:38:08.554756+0000","last_became_peered":"2024-09-16T01:38:08.554756+0000","last_unstale":"2024-09-16T01:39:09.522440+0000","last_undegraded":"2024-09-16T01:39:09.522440+0000","last_fullsized":"2024-09-16T01:39:09.522440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:08.888006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522217+0000","last_change":"2024-09-16T01:38:08.557906+0000","last_active":"2024-09-16T01:39:09.522217+0000","last_peered":"2024-09-16T01:39:09.522217+0000","last_clean":"2024-09-16T01:39:09.522217+0000","last_became_active":"2024-09-16T01:38:08.557546+0000","last_became_peered":"2024-09-16T01:38:08.557546+0000","last_unstale":"2024-09-16T01:39:09.522217+0000","last_undegraded":"2024-09-16T01:39:09.522217+0000","last_fullsized":"2024-09-16T01:39:09.522217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:27:04.988778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639642+0000","last_change":"2024-09-16T01:38:01.528430+0000","last_active":"2024-09-16T01:39:09.639642+0000","last_peered":"2024-09-16T01:39:09.639642+0000","last_clean":"2024-09-16T01:39:09.639642+0000","last_became_active":"2024-09-16T01:38:01.528069+0000","last_became_peered":"2024-09-16T01:38:01.528069+0000","last_unstale":"2024-09-16T01:39:09.639642+0000","last_undegraded":"2024-09-16T01:39:09.639642+0000","last_fullsized":"2024-09-16T01:39:09.639642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:41.317215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":146,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:33.642455+0000","last_change":"2024-09-16T01:37:59.509017+0000","last_active":"2024-09-16T01:39:33.642455+0000","last_peered":"2024-09-16T01:39:33.642455+0000","last_clean":"2024-09-16T01:39:33.642455+0000","last_became_active":"2024-09-16T01:37:59.508252+0000","last_became_peered":"2024-09-16T01:37:59.508252+0000","last_unstale":"2024-09-16T01:39:33.642455+0000","last_undegraded":"2024-09-16T01:39:33.642455+0000","last_fullsized":"2024-09-16T01:39:33.642455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:26.215407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518040+0000","last_change":"2024-09-16T01:38:08.565197+0000","last_active":"2024-09-16T01:39:09.518040+0000","last_peered":"2024-09-16T01:39:09.518040+0000","last_clean":"2024-09-16T01:39:09.518040+0000","last_became_active":"2024-09-16T01:38:08.562773+0000","last_became_peered":"2024-09-16T01:38:08.562773+0000","last_unstale":"2024-09-16T01:39:09.518040+0000","last_undegraded":"2024-09-16T01:39:09.518040+0000","last_fullsized":"2024-09-16T01:39:09.518040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:13.316950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.048048+0000","last_change":"2024-09-16T01:38:08.554052+0000","last_active":"2024-09-16T01:39:10.048048+0000","last_peered":"2024-09-16T01:39:10.048048+0000","last_clean":"2024-09-16T01:39:10.048048+0000","last_became_active":"2024-09-16T01:38:08.553660+0000","last_became_peered":"2024-09-16T01:38:08.553660+0000","last_unstale":"2024-09-16T01:39:10.048048+0000","last_undegraded":"2024-09-16T01:39:10.048048+0000","last_fullsized":"2024-09-16T01:39:10.048048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:53:52.270695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522411+0000","last_change":"2024-09-16T01:38:01.522688+0000","last_active":"2024-09-16T01:39:09.522411+0000","last_peered":"2024-09-16T01:39:09.522411+0000","last_clean":"2024-09-16T01:39:09.522411+0000","last_became_active":"2024-09-16T01:38:01.519843+0000","last_became_peered":"2024-09-16T01:38:01.519843+0000","last_unstale":"2024-09-16T01:39:09.522411+0000","last_undegraded":"2024-09-16T01:39:09.522411+0000","last_fullsized":"2024-09-16T01:39:09.522411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:50.971760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516780+0000","last_change":"2024-09-16T01:37:59.509044+0000","last_active":"2024-09-16T01:39:09.516780+0000","last_peered":"2024-09-16T01:39:09.516780+0000","last_clean":"2024-09-16T01:39:09.516780+0000","last_became_active":"2024-09-16T01:37:59.507085+0000","last_became_peered":"2024-09-16T01:37:59.507085+0000","last_unstale":"2024-09-16T01:39:09.516780+0000","last_undegraded":"2024-09-16T01:39:09.516780+0000","last_fullsized":"2024-09-16T01:39:09.516780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:25:09.930945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519812+0000","last_change":"2024-09-16T01:38:08.561250+0000","last_active":"2024-09-16T01:39:09.519812+0000","last_peered":"2024-09-16T01:39:09.519812+0000","last_clean":"2024-09-16T01:39:09.519812+0000","last_became_active":"2024-09-16T01:38:08.560866+0000","last_became_peered":"2024-09-16T01:38:08.560866+0000","last_unstale":"2024-09-16T01:39:09.519812+0000","last_undegraded":"2024-09-16T01:39:09.519812+0000","last_fullsized":"2024-09-16T01:39:09.519812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:31.253817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517715+0000","last_change":"2024-09-16T01:38:01.517400+0000","last_active":"2024-09-16T01:39:09.517715+0000","last_peered":"2024-09-16T01:39:09.517715+0000","last_clean":"2024-09-16T01:39:09.517715+0000","last_became_active":"2024-09-16T01:38:01.517103+0000","last_became_peered":"2024-09-16T01:38:01.517103+0000","last_unstale":"2024-09-16T01:39:09.517715+0000","last_undegraded":"2024-09-16T01:39:09.517715+0000","last_fullsized":"2024-09-16T01:39:09.517715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:37.886179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.610877+0000","last_change":"2024-09-16T01:37:59.503370+0000","last_active":"2024-09-16T01:39:09.610877+0000","last_peered":"2024-09-16T01:39:09.610877+0000","last_clean":"2024-09-16T01:39:09.610877+0000","last_became_active":"2024-09-16T01:37:59.503071+0000","last_became_peered":"2024-09-16T01:37:59.503071+0000","last_unstale":"2024-09-16T01:39:09.610877+0000","last_undegraded":"2024-09-16T01:39:09.610877+0000","last_fullsized":"2024-09-16T01:39:09.610877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:51.207702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519530+0000","last_change":"2024-09-16T01:37:59.506110+0000","last_active":"2024-09-16T01:39:09.519530+0000","last_peered":"2024-09-16T01:39:09.519530+0000","last_clean":"2024-09-16T01:39:09.519530+0000","last_became_active":"2024-09-16T01:37:59.505264+0000","last_became_peered":"2024-09-16T01:37:59.505264+0000","last_unstale":"2024-09-16T01:39:09.519530+0000","last_undegraded":"2024-09-16T01:39:09.519530+0000","last_fullsized":"2024-09-16T01:39:09.519530+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:35.986433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517383+0000","last_change":"2024-09-16T01:38:08.563900+0000","last_active":"2024-09-16T01:39:09.517383+0000","last_peered":"2024-09-16T01:39:09.517383+0000","last_clean":"2024-09-16T01:39:09.517383+0000","last_became_active":"2024-09-16T01:38:08.563551+0000","last_became_peered":"2024-09-16T01:38:08.563551+0000","last_unstale":"2024-09-16T01:39:09.517383+0000","last_undegraded":"2024-09-16T01:39:09.517383+0000","last_fullsized":"2024-09-16T01:39:09.517383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:31:03.979580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.614148+0000","last_change":"2024-09-16T01:38:08.556011+0000","last_active":"2024-09-16T01:39:09.614148+0000","last_peered":"2024-09-16T01:39:09.614148+0000","last_clean":"2024-09-16T01:39:09.614148+0000","last_became_active":"2024-09-16T01:38:08.555615+0000","last_became_peered":"2024-09-16T01:38:08.555615+0000","last_unstale":"2024-09-16T01:39:09.614148+0000","last_undegraded":"2024-09-16T01:39:09.614148+0000","last_fullsized":"2024-09-16T01:39:09.614148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:34.434974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":92,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:29.637814+0000","last_change":"2024-09-16T01:37:59.508411+0000","last_active":"2024-09-16T01:39:29.637814+0000","last_peered":"2024-09-16T01:39:29.637814+0000","last_clean":"2024-09-16T01:39:29.637814+0000","last_became_active":"2024-09-16T01:37:59.508058+0000","last_became_peered":"2024-09-16T01:37:59.508058+0000","last_unstale":"2024-09-16T01:39:29.637814+0000","last_undegraded":"2024-09-16T01:39:29.637814+0000","last_fullsized":"2024-09-16T01:39:29.637814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:47.441673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516073+0000","last_change":"2024-09-16T01:38:08.560930+0000","last_active":"2024-09-16T01:39:09.516073+0000","last_peered":"2024-09-16T01:39:09.516073+0000","last_clean":"2024-09-16T01:39:09.516073+0000","last_became_active":"2024-09-16T01:38:08.560490+0000","last_became_peered":"2024-09-16T01:38:08.560490+0000","last_unstale":"2024-09-16T01:39:09.516073+0000","last_undegraded":"2024-09-16T01:39:09.516073+0000","last_fullsized":"2024-09-16T01:39:09.516073+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:11.556113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.613929+0000","last_change":"2024-09-16T01:38:08.557564+0000","last_active":"2024-09-16T01:39:09.613929+0000","last_peered":"2024-09-16T01:39:09.613929+0000","last_clean":"2024-09-16T01:39:09.613929+0000","last_became_active":"2024-09-16T01:38:08.557143+0000","last_became_peered":"2024-09-16T01:38:08.557143+0000","last_unstale":"2024-09-16T01:39:09.613929+0000","last_undegraded":"2024-09-16T01:39:09.613929+0000","last_fullsized":"2024-09-16T01:39:09.613929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:43.148870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519117+0000","last_change":"2024-09-16T01:38:01.527353+0000","last_active":"2024-09-16T01:39:09.519117+0000","last_peered":"2024-09-16T01:39:09.519117+0000","last_clean":"2024-09-16T01:39:09.519117+0000","last_became_active":"2024-09-16T01:38:01.527031+0000","last_became_peered":"2024-09-16T01:38:01.527031+0000","last_unstale":"2024-09-16T01:39:09.519117+0000","last_undegraded":"2024-09-16T01:39:09.519117+0000","last_fullsized":"2024-09-16T01:39:09.519117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:27.311638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.614155+0000","last_change":"2024-09-16T01:38:01.526409+0000","last_active":"2024-09-16T01:39:09.614155+0000","last_peered":"2024-09-16T01:39:09.614155+0000","last_clean":"2024-09-16T01:39:09.614155+0000","last_became_active":"2024-09-16T01:38:01.526064+0000","last_became_peered":"2024-09-16T01:38:01.526064+0000","last_unstale":"2024-09-16T01:39:09.614155+0000","last_undegraded":"2024-09-16T01:39:09.614155+0000","last_fullsized":"2024-09-16T01:39:09.614155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:51.571808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.613906+0000","last_change":"2024-09-16T01:38:01.521856+0000","last_active":"2024-09-16T01:39:09.613906+0000","last_peered":"2024-09-16T01:39:09.613906+0000","last_clean":"2024-09-16T01:39:09.613906+0000","last_became_active":"2024-09-16T01:38:01.520312+0000","last_became_peered":"2024-09-16T01:38:01.520312+0000","last_unstale":"2024-09-16T01:39:09.613906+0000","last_undegraded":"2024-09-16T01:39:09.613906+0000","last_fullsized":"2024-09-16T01:39:09.613906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:53.618990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.047889+0000","last_change":"2024-09-16T01:38:01.520958+0000","last_active":"2024-09-16T01:39:10.047889+0000","last_peered":"2024-09-16T01:39:10.047889+0000","last_clean":"2024-09-16T01:39:10.047889+0000","last_became_active":"2024-09-16T01:38:01.520658+0000","last_became_peered":"2024-09-16T01:38:01.520658+0000","last_unstale":"2024-09-16T01:39:10.047889+0000","last_undegraded":"2024-09-16T01:39:10.047889+0000","last_fullsized":"2024-09-16T01:39:10.047889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:23.770694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611348+0000","last_change":"2024-09-16T01:38:01.519140+0000","last_active":"2024-09-16T01:39:09.611348+0000","last_peered":"2024-09-16T01:39:09.611348+0000","last_clean":"2024-09-16T01:39:09.611348+0000","last_became_active":"2024-09-16T01:38:01.518765+0000","last_became_peered":"2024-09-16T01:38:01.518765+0000","last_unstale":"2024-09-16T01:39:09.611348+0000","last_undegraded":"2024-09-16T01:39:09.611348+0000","last_fullsized":"2024-09-16T01:39:09.611348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:17.194886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519792+0000","last_change":"2024-09-16T01:38:01.523762+0000","last_active":"2024-09-16T01:39:09.519792+0000","last_peered":"2024-09-16T01:39:09.519792+0000","last_clean":"2024-09-16T01:39:09.519792+0000","last_became_active":"2024-09-16T01:38:01.522035+0000","last_became_peered":"2024-09-16T01:38:01.522035+0000","last_unstale":"2024-09-16T01:39:09.519792+0000","last_undegraded":"2024-09-16T01:39:09.519792+0000","last_fullsized":"2024-09-16T01:39:09.519792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:40.405350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.614239+0000","last_change":"2024-09-16T01:38:01.527539+0000","last_active":"2024-09-16T01:39:09.614239+0000","last_peered":"2024-09-16T01:39:09.614239+0000","last_clean":"2024-09-16T01:39:09.614239+0000","last_became_active":"2024-09-16T01:38:01.526974+0000","last_became_peered":"2024-09-16T01:38:01.526974+0000","last_unstale":"2024-09-16T01:39:09.614239+0000","last_undegraded":"2024-09-16T01:39:09.614239+0000","last_fullsized":"2024-09-16T01:39:09.614239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:30:03.587141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.518254+0000","last_change":"2024-09-16T01:38:01.529131+0000","last_active":"2024-09-16T01:39:09.518254+0000","last_peered":"2024-09-16T01:39:09.518254+0000","last_clean":"2024-09-16T01:39:09.518254+0000","last_became_active":"2024-09-16T01:38:01.528796+0000","last_became_peered":"2024-09-16T01:38:01.528796+0000","last_unstale":"2024-09-16T01:39:09.518254+0000","last_undegraded":"2024-09-16T01:39:09.518254+0000","last_fullsized":"2024-09-16T01:39:09.518254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:38.903638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516476+0000","last_change":"2024-09-16T01:38:01.514779+0000","last_active":"2024-09-16T01:39:09.516476+0000","last_peered":"2024-09-16T01:39:09.516476+0000","last_clean":"2024-09-16T01:39:09.516476+0000","last_became_active":"2024-09-16T01:38:01.514434+0000","last_became_peered":"2024-09-16T01:38:01.514434+0000","last_unstale":"2024-09-16T01:39:09.516476+0000","last_undegraded":"2024-09-16T01:39:09.516476+0000","last_fullsized":"2024-09-16T01:39:09.516476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:31.382111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517771+0000","last_change":"2024-09-16T01:38:01.521771+0000","last_active":"2024-09-16T01:39:09.517771+0000","last_peered":"2024-09-16T01:39:09.517771+0000","last_clean":"2024-09-16T01:39:09.517771+0000","last_became_active":"2024-09-16T01:38:01.521288+0000","last_became_peered":"2024-09-16T01:38:01.521288+0000","last_unstale":"2024-09-16T01:39:09.517771+0000","last_undegraded":"2024-09-16T01:39:09.517771+0000","last_fullsized":"2024-09-16T01:39:09.517771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:11.825164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639394+0000","last_change":"2024-09-16T01:38:01.529247+0000","last_active":"2024-09-16T01:39:09.639394+0000","last_peered":"2024-09-16T01:39:09.639394+0000","last_clean":"2024-09-16T01:39:09.639394+0000","last_became_active":"2024-09-16T01:38:01.528926+0000","last_became_peered":"2024-09-16T01:38:01.528926+0000","last_unstale":"2024-09-16T01:39:09.639394+0000","last_undegraded":"2024-09-16T01:39:09.639394+0000","last_fullsized":"2024-09-16T01:39:09.639394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:20.704114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611066+0000","last_change":"2024-09-16T01:38:01.520830+0000","last_active":"2024-09-16T01:39:09.611066+0000","last_peered":"2024-09-16T01:39:09.611066+0000","last_clean":"2024-09-16T01:39:09.611066+0000","last_became_active":"2024-09-16T01:38:01.520526+0000","last_became_peered":"2024-09-16T01:38:01.520526+0000","last_unstale":"2024-09-16T01:39:09.611066+0000","last_undegraded":"2024-09-16T01:39:09.611066+0000","last_fullsized":"2024-09-16T01:39:09.611066+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:28.062812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519439+0000","last_change":"2024-09-16T01:38:01.521983+0000","last_active":"2024-09-16T01:39:09.519439+0000","last_peered":"2024-09-16T01:39:09.519439+0000","last_clean":"2024-09-16T01:39:09.519439+0000","last_became_active":"2024-09-16T01:38:01.521666+0000","last_became_peered":"2024-09-16T01:38:01.521666+0000","last_unstale":"2024-09-16T01:39:09.519439+0000","last_undegraded":"2024-09-16T01:39:09.519439+0000","last_fullsized":"2024-09-16T01:39:09.519439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:50:55.761313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.614164+0000","last_change":"2024-09-16T01:38:08.554304+0000","last_active":"2024-09-16T01:39:09.614164+0000","last_peered":"2024-09-16T01:39:09.614164+0000","last_clean":"2024-09-16T01:39:09.614164+0000","last_became_active":"2024-09-16T01:38:08.553683+0000","last_became_peered":"2024-09-16T01:38:08.553683+0000","last_unstale":"2024-09-16T01:39:09.614164+0000","last_undegraded":"2024-09-16T01:39:09.614164+0000","last_fullsized":"2024-09-16T01:39:09.614164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:56.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522904+0000","last_change":"2024-09-16T01:38:01.520425+0000","last_active":"2024-09-16T01:39:09.522904+0000","last_peered":"2024-09-16T01:39:09.522904+0000","last_clean":"2024-09-16T01:39:09.522904+0000","last_became_active":"2024-09-16T01:38:01.519858+0000","last_became_peered":"2024-09-16T01:38:01.519858+0000","last_unstale":"2024-09-16T01:39:09.522904+0000","last_undegraded":"2024-09-16T01:39:09.522904+0000","last_fullsized":"2024-09-16T01:39:09.522904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:48.534298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.520100+0000","last_change":"2024-09-16T01:38:01.532762+0000","last_active":"2024-09-16T01:39:09.520100+0000","last_peered":"2024-09-16T01:39:09.520100+0000","last_clean":"2024-09-16T01:39:09.520100+0000","last_became_active":"2024-09-16T01:38:01.532404+0000","last_became_peered":"2024-09-16T01:38:01.532404+0000","last_unstale":"2024-09-16T01:39:09.520100+0000","last_undegraded":"2024-09-16T01:39:09.520100+0000","last_fullsized":"2024-09-16T01:39:09.520100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:03.135141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639153+0000","last_change":"2024-09-16T01:38:01.530205+0000","last_active":"2024-09-16T01:39:09.639153+0000","last_peered":"2024-09-16T01:39:09.639153+0000","last_clean":"2024-09-16T01:39:09.639153+0000","last_became_active":"2024-09-16T01:38:01.529898+0000","last_became_peered":"2024-09-16T01:38:01.529898+0000","last_unstale":"2024-09-16T01:39:09.639153+0000","last_undegraded":"2024-09-16T01:39:09.639153+0000","last_fullsized":"2024-09-16T01:39:09.639153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:48.444441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.047689+0000","last_change":"2024-09-16T01:38:01.529407+0000","last_active":"2024-09-16T01:39:10.047689+0000","last_peered":"2024-09-16T01:39:10.047689+0000","last_clean":"2024-09-16T01:39:10.047689+0000","last_became_active":"2024-09-16T01:38:01.528330+0000","last_became_peered":"2024-09-16T01:38:01.528330+0000","last_unstale":"2024-09-16T01:39:10.047689+0000","last_undegraded":"2024-09-16T01:39:10.047689+0000","last_fullsized":"2024-09-16T01:39:10.047689+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:32.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.047759+0000","last_change":"2024-09-16T01:38:01.519579+0000","last_active":"2024-09-16T01:39:10.047759+0000","last_peered":"2024-09-16T01:39:10.047759+0000","last_clean":"2024-09-16T01:39:10.047759+0000","last_became_active":"2024-09-16T01:38:01.519211+0000","last_became_peered":"2024-09-16T01:38:01.519211+0000","last_unstale":"2024-09-16T01:39:10.047759+0000","last_undegraded":"2024-09-16T01:39:10.047759+0000","last_fullsized":"2024-09-16T01:39:10.047759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:18.861169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.611059+0000","last_change":"2024-09-16T01:38:01.521136+0000","last_active":"2024-09-16T01:39:09.611059+0000","last_peered":"2024-09-16T01:39:09.611059+0000","last_clean":"2024-09-16T01:39:09.611059+0000","last_became_active":"2024-09-16T01:38:01.520819+0000","last_became_peered":"2024-09-16T01:38:01.520819+0000","last_unstale":"2024-09-16T01:39:09.611059+0000","last_undegraded":"2024-09-16T01:39:09.611059+0000","last_fullsized":"2024-09-16T01:39:09.611059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:28.150822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516967+0000","last_change":"2024-09-16T01:38:01.519382+0000","last_active":"2024-09-16T01:39:09.516967+0000","last_peered":"2024-09-16T01:39:09.516967+0000","last_clean":"2024-09-16T01:39:09.516967+0000","last_became_active":"2024-09-16T01:38:01.519039+0000","last_became_peered":"2024-09-16T01:38:01.519039+0000","last_unstale":"2024-09-16T01:39:09.516967+0000","last_undegraded":"2024-09-16T01:39:09.516967+0000","last_fullsized":"2024-09-16T01:39:09.516967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:46.559357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.521938+0000","last_change":"2024-09-16T01:38:01.527513+0000","last_active":"2024-09-16T01:39:09.521938+0000","last_peered":"2024-09-16T01:39:09.521938+0000","last_clean":"2024-09-16T01:39:09.521938+0000","last_became_active":"2024-09-16T01:38:01.527184+0000","last_became_peered":"2024-09-16T01:38:01.527184+0000","last_unstale":"2024-09-16T01:39:09.521938+0000","last_undegraded":"2024-09-16T01:39:09.521938+0000","last_fullsized":"2024-09-16T01:39:09.521938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:41.508318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519580+0000","last_change":"2024-09-16T01:38:01.524239+0000","last_active":"2024-09-16T01:39:09.519580+0000","last_peered":"2024-09-16T01:39:09.519580+0000","last_clean":"2024-09-16T01:39:09.519580+0000","last_became_active":"2024-09-16T01:38:01.523906+0000","last_became_peered":"2024-09-16T01:38:01.523906+0000","last_unstale":"2024-09-16T01:39:09.519580+0000","last_undegraded":"2024-09-16T01:39:09.519580+0000","last_fullsized":"2024-09-16T01:39:09.519580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:11.718796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.639776+0000","last_change":"2024-09-16T01:38:01.531564+0000","last_active":"2024-09-16T01:39:09.639776+0000","last_peered":"2024-09-16T01:39:09.639776+0000","last_clean":"2024-09-16T01:39:09.639776+0000","last_became_active":"2024-09-16T01:38:01.531241+0000","last_became_peered":"2024-09-16T01:38:01.531241+0000","last_unstale":"2024-09-16T01:39:09.639776+0000","last_undegraded":"2024-09-16T01:39:09.639776+0000","last_fullsized":"2024-09-16T01:39:09.639776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:16.244641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.517794+0000","last_change":"2024-09-16T01:38:01.519913+0000","last_active":"2024-09-16T01:39:09.517794+0000","last_peered":"2024-09-16T01:39:09.517794+0000","last_clean":"2024-09-16T01:39:09.517794+0000","last_became_active":"2024-09-16T01:38:01.518631+0000","last_became_peered":"2024-09-16T01:38:01.518631+0000","last_unstale":"2024-09-16T01:39:09.517794+0000","last_undegraded":"2024-09-16T01:39:09.517794+0000","last_fullsized":"2024-09-16T01:39:09.517794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:48.747188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.520386+0000","last_change":"2024-09-16T01:38:01.530805+0000","last_active":"2024-09-16T01:39:09.520386+0000","last_peered":"2024-09-16T01:39:09.520386+0000","last_clean":"2024-09-16T01:39:09.520386+0000","last_became_active":"2024-09-16T01:38:01.530466+0000","last_became_peered":"2024-09-16T01:38:01.530466+0000","last_unstale":"2024-09-16T01:39:09.520386+0000","last_undegraded":"2024-09-16T01:39:09.520386+0000","last_fullsized":"2024-09-16T01:39:09.520386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:41:30.587398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.519485+0000","last_change":"2024-09-16T01:38:01.519227+0000","last_active":"2024-09-16T01:39:09.519485+0000","last_peered":"2024-09-16T01:39:09.519485+0000","last_clean":"2024-09-16T01:39:09.519485+0000","last_became_active":"2024-09-16T01:38:01.518875+0000","last_became_peered":"2024-09-16T01:38:01.518875+0000","last_unstale":"2024-09-16T01:39:09.519485+0000","last_undegraded":"2024-09-16T01:39:09.519485+0000","last_fullsized":"2024-09-16T01:39:09.519485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:42.654812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.516085+0000","last_change":"2024-09-16T01:38:01.513327+0000","last_active":"2024-09-16T01:39:09.516085+0000","last_peered":"2024-09-16T01:39:09.516085+0000","last_clean":"2024-09-16T01:39:09.516085+0000","last_became_active":"2024-09-16T01:38:01.513000+0000","last_became_peered":"2024-09-16T01:38:01.513000+0000","last_unstale":"2024-09-16T01:39:09.516085+0000","last_undegraded":"2024-09-16T01:39:09.516085+0000","last_fullsized":"2024-09-16T01:39:09.516085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:43.036805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.522826+0000","last_change":"2024-09-16T01:38:08.560560+0000","last_active":"2024-09-16T01:39:09.522826+0000","last_peered":"2024-09-16T01:39:09.522826+0000","last_clean":"2024-09-16T01:39:09.522826+0000","last_became_active":"2024-09-16T01:38:08.560229+0000","last_became_peered":"2024-09-16T01:38:08.560229+0000","last_unstale":"2024-09-16T01:39:09.522826+0000","last_undegraded":"2024-09-16T01:39:09.522826+0000","last_fullsized":"2024-09-16T01:39:09.522826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:10:09.462043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.048060+0000","last_change":"2024-09-16T01:38:01.529548+0000","last_active":"2024-09-16T01:39:10.048060+0000","last_peered":"2024-09-16T01:39:10.048060+0000","last_clean":"2024-09-16T01:39:10.048060+0000","last_became_active":"2024-09-16T01:38:01.529011+0000","last_became_peered":"2024-09-16T01:38:01.529011+0000","last_unstale":"2024-09-16T01:39:10.048060+0000","last_undegraded":"2024-09-16T01:39:10.048060+0000","last_fullsized":"2024-09-16T01:39:10.048060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:21.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:10.048107+0000","last_change":"2024-09-16T01:38:01.519488+0000","last_active":"2024-09-16T01:39:10.048107+0000","last_peered":"2024-09-16T01:39:10.048107+0000","last_clean":"2024-09-16T01:39:10.048107+0000","last_became_active":"2024-09-16T01:38:01.519156+0000","last_became_peered":"2024-09-16T01:38:01.519156+0000","last_unstale":"2024-09-16T01:39:10.048107+0000","last_undegraded":"2024-09-16T01:39:10.048107+0000","last_fullsized":"2024-09-16T01:39:10.048107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:18.149874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T01:39:09.520009+0000","last_change":"2024-09-16T01:38:01.527519+0000","last_active":"2024-09-16T01:39:09.520009+0000","last_peered":"2024-09-16T01:39:09.520009+0000","last_clean":"2024-09-16T01:39:09.520009+0000","last_became_active":"2024-09-16T01:38:01.527175+0000","last_became_peered":"2024-09-16T01:38:01.527175+0000","last_unstale":"2024-09-16T01:39:09.520009+0000","last_undegraded":"2024-09-16T01:39:09.520009+0000","last_fullsized":"2024-09-16T01:39:09.520009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:04.958842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68891,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":565,"num_read_kb":362,"num_write":248,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":206673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281435,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678049,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":954368,"data_stored":488610,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906982,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1097728,"data_stored":1141093,"data_compressed":59467,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070507,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":970752,"data_stored":488035,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299439,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":950272,"data_stored":487629,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528372,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1019904,"data_stored":1084519,"data_compressed":59467,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":946176,"data_stored":483539,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888127,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1028096,"data_stored":1080101,"data_compressed":59467,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051652,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":954368,"data_stored":487613,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280585,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":958464,"data_stored":483247,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476815,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1032192,"data_stored":547258,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607636,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1019904,"data_stored":547317,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":970752,"data_stored":485388,"data_compressed":23673,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34155,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34155,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34155,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T01:39:36.515 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T01:39:36.515 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T01:39:36.515 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T01:39:36.515 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T01:39:36.804 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:37.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.283+0000 7f65b8d99640 1 -- 172.21.15.139:0/2427282753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65b4106050 msgr2=0x7f65b410dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:37.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.283+0000 7f65b8d99640 1 --2- 172.21.15.139:0/2427282753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65b4106050 0x7f65b410dbd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009ca0 tx=0x7f65ac02a8c0 comp rx=0 tx=0).stop 2024-09-16T01:39:37.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.283+0000 7f65b8d99640 1 -- 172.21.15.139:0/2427282753 shutdown_connections 2024-09-16T01:39:37.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.283+0000 7f65b8d99640 1 --2- 172.21.15.139:0/2427282753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65b4106050 0x7f65b410dbd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:37.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.283+0000 7f65b8d99640 1 --2- 172.21.15.139:0/2427282753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65b4101df0 0x7f65b4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:37.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.283+0000 7f65b8d99640 1 --2- 172.21.15.139:0/2427282753 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65b4101440 0x7f65b4101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:37.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.283+0000 7f65b8d99640 1 -- 172.21.15.139:0/2427282753 >> 172.21.15.139:0/2427282753 conn(0x7f65b4077210 msgr2=0x7f65b40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:37.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.283+0000 7f65b8d99640 1 -- 172.21.15.139:0/2427282753 shutdown_connections 2024-09-16T01:39:37.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.283+0000 7f65b8d99640 1 -- 172.21.15.139:0/2427282753 wait complete. 2024-09-16T01:39:37.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.284+0000 7f65b8d99640 1 Processor -- start 2024-09-16T01:39:37.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.284+0000 7f65b8d99640 1 -- start start 2024-09-16T01:39:37.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.284+0000 7f65b8d99640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65b4101440 0x7f65b4198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:37.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.284+0000 7f65b8d99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65b4101df0 0x7f65b4198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:37.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.284+0000 7f65b8d99640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65b4106050 0x7f65b419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:37.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.284+0000 7f65b8d99640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f65b41121d0 con 0x7f65b4101df0 2024-09-16T01:39:37.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.284+0000 7f65b8d99640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f65b4112050 con 0x7f65b4106050 2024-09-16T01:39:37.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.284+0000 7f65b8d99640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f65b4112350 con 0x7f65b4101440 2024-09-16T01:39:37.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.284+0000 7f65b1d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65b4101df0 0x7f65b4198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:37.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65b1d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65b4101df0 0x7f65b4198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39642/0 (socket says 172.21.15.139:39642) 2024-09-16T01:39:37.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65b1d9b640 1 -- 172.21.15.139:0/659325652 learned_addr learned my addr 172.21.15.139:0/659325652 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:37.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65b259c640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65b4101440 0x7f65b4198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:37.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65b2d9d640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65b4106050 0x7f65b419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:37.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65b1d9b640 1 -- 172.21.15.139:0/659325652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65b4101440 msgr2=0x7f65b4198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:37.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65b1d9b640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65b4101440 0x7f65b4198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:37.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65b1d9b640 1 -- 172.21.15.139:0/659325652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65b4106050 msgr2=0x7f65b419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:37.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65b1d9b640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65b4106050 0x7f65b419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:37.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65b1d9b640 1 -- 172.21.15.139:0/659325652 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b419d640 con 0x7f65b4101df0 2024-09-16T01:39:37.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65b2d9d640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65b4106050 0x7f65b419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:37.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65b1d9b640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65b4101df0 0x7f65b4198bd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f65a800d950 tx=0x7f65a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:37.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65a37fe640 1 -- 172.21.15.139:0/659325652 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8017030 con 0x7f65b4101df0 2024-09-16T01:39:37.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65a37fe640 1 -- 172.21.15.139:0/659325652 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a8004020 con 0x7f65b4101df0 2024-09-16T01:39:37.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65b8d99640 1 -- 172.21.15.139:0/659325652 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b419d930 con 0x7f65b4101df0 2024-09-16T01:39:37.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65a37fe640 1 -- 172.21.15.139:0/659325652 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a80047a0 con 0x7f65b4101df0 2024-09-16T01:39:37.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.285+0000 7f65b8d99640 1 -- 172.21.15.139:0/659325652 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b41a5210 con 0x7f65b4101df0 2024-09-16T01:39:37.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.286+0000 7f65b8d99640 1 -- 172.21.15.139:0/659325652 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b4101820 con 0x7f65b4101df0 2024-09-16T01:39:37.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.287+0000 7f65a37fe640 1 -- 172.21.15.139:0/659325652 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f65a80041d0 con 0x7f65b4101df0 2024-09-16T01:39:37.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.288+0000 7f65a37fe640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f659c0a32d0 0x7f659c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:37.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.288+0000 7f65b259c640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f659c0a32d0 0x7f659c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:37.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.288+0000 7f65a37fe640 1 -- 172.21.15.139:0/659325652 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f65a80cffb0 con 0x7f65b4101df0 2024-09-16T01:39:37.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.289+0000 7f65b259c640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f659c0a32d0 0x7f659c0a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009890 tx=0x7f65a4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:37.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.291+0000 7f65a37fe640 1 -- 172.21.15.139:0/659325652 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65a809c7a0 con 0x7f65b4101df0 2024-09-16T01:39:37.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.494+0000 7f65b8d99640 1 -- 172.21.15.139:0/659325652 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f65b41999f0 con 0x7f65b4101df0 2024-09-16T01:39:37.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.495+0000 7f65a37fe640 1 -- 172.21.15.139:0/659325652 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f65b41999f0 con 0x7f65b4101df0 2024-09-16T01:39:37.497 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:39:37.497 INFO:teuthology.orchestra.run.smithi139.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T01:39:37.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.497+0000 7f65b8d99640 1 -- 172.21.15.139:0/659325652 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f659c0a32d0 msgr2=0x7f659c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:37.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.497+0000 7f65b8d99640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f659c0a32d0 0x7f659c0a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009890 tx=0x7f65a4008040 comp rx=0 tx=0).stop 2024-09-16T01:39:37.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.497+0000 7f65b8d99640 1 -- 172.21.15.139:0/659325652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65b4101df0 msgr2=0x7f65b4198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:37.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.497+0000 7f65b8d99640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65b4101df0 0x7f65b4198bd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f65a800d950 tx=0x7f65a800de10 comp rx=0 tx=0).stop 2024-09-16T01:39:37.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.497+0000 7f65b8d99640 1 -- 172.21.15.139:0/659325652 shutdown_connections 2024-09-16T01:39:37.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.497+0000 7f65b8d99640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f659c0a32d0 0x7f659c0a5790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:37.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.497+0000 7f65b8d99640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65b4106050 0x7f65b419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:37.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.497+0000 7f65b8d99640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65b4101df0 0x7f65b4198bd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:37.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.497+0000 7f65b8d99640 1 --2- 172.21.15.139:0/659325652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65b4101440 0x7f65b4198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:37.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.497+0000 7f65b8d99640 1 -- 172.21.15.139:0/659325652 >> 172.21.15.139:0/659325652 conn(0x7f65b4077210 msgr2=0x7f65b41047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:37.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.497+0000 7f65b8d99640 1 -- 172.21.15.139:0/659325652 shutdown_connections 2024-09-16T01:39:37.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:37.497+0000 7f65b8d99640 1 -- 172.21.15.139:0/659325652 wait complete. 2024-09-16T01:39:37.929 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:37 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/659325652' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T01:39:37.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:37 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/659325652' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T01:39:38.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:37 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/659325652' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T01:39:38.203 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T01:39:38.204 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:39:38.452 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:38.830 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:38 smithi139 ceph-mon[31175]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:39:38.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:38 smithi176 ceph-mon[35324]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:39:38.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.907+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3684819585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39410c170 msgr2=0x7fc39410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:38.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.907+0000 7fc39a0f2640 1 --2- 172.21.15.139:0/3684819585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39410c170 0x7fc39410e610 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009ca0 tx=0x7fc38c02a950 comp rx=0 tx=0).stop 2024-09-16T01:39:38.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.907+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3684819585 shutdown_connections 2024-09-16T01:39:38.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.907+0000 7fc39a0f2640 1 --2- 172.21.15.139:0/3684819585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39410c170 0x7fc39410e610 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:38.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.907+0000 7fc39a0f2640 1 --2- 172.21.15.139:0/3684819585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc394103660 0x7fc39410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:38.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.907+0000 7fc39a0f2640 1 --2- 172.21.15.139:0/3684819585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc394102d40 0x7fc394103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:38.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.907+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3684819585 >> 172.21.15.139:0/3684819585 conn(0x7fc3940fc7a0 msgr2=0x7fc3940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:38.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.907+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3684819585 shutdown_connections 2024-09-16T01:39:38.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.907+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3684819585 wait complete. 2024-09-16T01:39:38.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.908+0000 7fc39a0f2640 1 Processor -- start 2024-09-16T01:39:38.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.908+0000 7fc39a0f2640 1 -- start start 2024-09-16T01:39:38.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.908+0000 7fc39a0f2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc394102d40 0x7fc394198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:38.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.908+0000 7fc39a0f2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc394103660 0x7fc394198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:38.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.908+0000 7fc39a0f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39410c170 0x7fc39419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:38.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.908+0000 7fc39a0f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc394111f30 con 0x7fc39410c170 2024-09-16T01:39:38.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.908+0000 7fc39a0f2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc394111db0 con 0x7fc394103660 2024-09-16T01:39:38.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.908+0000 7fc39a0f2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc3941120b0 con 0x7fc394102d40 2024-09-16T01:39:38.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.908+0000 7fc393fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39410c170 0x7fc39419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:38.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.908+0000 7fc393fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39410c170 0x7fc39419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39648/0 (socket says 172.21.15.139:39648) 2024-09-16T01:39:38.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.908+0000 7fc393fff640 1 -- 172.21.15.139:0/3194763460 learned_addr learned my addr 172.21.15.139:0/3194763460 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:38.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.908+0000 7fc3937fe640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc394102d40 0x7fc394198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:38.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.909+0000 7fc392ffd640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc394103660 0x7fc394198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:38.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.909+0000 7fc393fff640 1 -- 172.21.15.139:0/3194763460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc394102d40 msgr2=0x7fc394198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:38.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.909+0000 7fc393fff640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc394102d40 0x7fc394198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:38.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.909+0000 7fc393fff640 1 -- 172.21.15.139:0/3194763460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc394103660 msgr2=0x7fc394198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:38.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.909+0000 7fc393fff640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc394103660 0x7fc394198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:38.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.909+0000 7fc393fff640 1 -- 172.21.15.139:0/3194763460 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39419d2e0 con 0x7fc39410c170 2024-09-16T01:39:38.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.909+0000 7fc3937fe640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc394102d40 0x7fc394198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:38.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.909+0000 7fc392ffd640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc394103660 0x7fc394198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:38.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.909+0000 7fc393fff640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39410c170 0x7fc39419cc00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc38c006010 tx=0x7fc38c02f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:38.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.909+0000 7fc390ff9640 1 -- 172.21.15.139:0/3194763460 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c034040 con 0x7fc39410c170 2024-09-16T01:39:38.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.909+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3194763460 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39419d570 con 0x7fc39410c170 2024-09-16T01:39:38.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.909+0000 7fc390ff9640 1 -- 172.21.15.139:0/3194763460 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc38c003e00 con 0x7fc39410c170 2024-09-16T01:39:38.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.909+0000 7fc390ff9640 1 -- 172.21.15.139:0/3194763460 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c0456e0 con 0x7fc39410c170 2024-09-16T01:39:38.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.909+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3194763460 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3941a4eb0 con 0x7fc39410c170 2024-09-16T01:39:38.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.910+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3194763460 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc394103120 con 0x7fc39410c170 2024-09-16T01:39:38.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.911+0000 7fc390ff9640 1 -- 172.21.15.139:0/3194763460 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc38c037020 con 0x7fc39410c170 2024-09-16T01:39:38.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.912+0000 7fc390ff9640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fc37c0a32d0 0x7fc37c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:38.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.912+0000 7fc3937fe640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fc37c0a32d0 0x7fc37c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:38.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.912+0000 7fc390ff9640 1 -- 172.21.15.139:0/3194763460 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fc38c0f0fe0 con 0x7fc39410c170 2024-09-16T01:39:38.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.912+0000 7fc3937fe640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fc37c0a32d0 0x7fc37c0a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc38400abe0 tx=0x7fc38400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:38.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:38.915+0000 7fc390ff9640 1 -- 172.21.15.139:0/3194763460 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc38c0bd8e0 con 0x7fc39410c170 2024-09-16T01:39:39.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:38 smithi164 ceph-mon[35121]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:39:39.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:39.115+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3194763460 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc394199690 con 0x7fc39410c170 2024-09-16T01:39:39.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:39.116+0000 7fc390ff9640 1 -- 172.21.15.139:0/3194763460 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11995 (secure 0 0 0) 0x7fc38c0dbe20 con 0x7fc39410c170 2024-09-16T01:39:39.118 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:39:39.118 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":28,"btime":"2024-09-16T01:38:40:558188+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:38:40.558170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24454,24443,24442,24433]},"id":1}]} 2024-09-16T01:39:39.118 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 28 2024-09-16T01:39:39.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:39.119+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3194763460 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fc37c0a32d0 msgr2=0x7fc37c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:39.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:39.119+0000 7fc39a0f2640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fc37c0a32d0 0x7fc37c0a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc38400abe0 tx=0x7fc38400a5c0 comp rx=0 tx=0).stop 2024-09-16T01:39:39.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:39.119+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3194763460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39410c170 msgr2=0x7fc39419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:39.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:39.119+0000 7fc39a0f2640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39410c170 0x7fc39419cc00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc38c006010 tx=0x7fc38c02f710 comp rx=0 tx=0).stop 2024-09-16T01:39:39.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:39.119+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3194763460 shutdown_connections 2024-09-16T01:39:39.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:39.119+0000 7fc39a0f2640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fc37c0a32d0 0x7fc37c0a5790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:39.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:39.119+0000 7fc39a0f2640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39410c170 0x7fc39419cc00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:39.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:39.119+0000 7fc39a0f2640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc394103660 0x7fc394198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:39.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:39.119+0000 7fc39a0f2640 1 --2- 172.21.15.139:0/3194763460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc394102d40 0x7fc394198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:39.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:39.119+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3194763460 >> 172.21.15.139:0/3194763460 conn(0x7fc3940fc7a0 msgr2=0x7fc3940fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:39.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:39.119+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3194763460 shutdown_connections 2024-09-16T01:39:39.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:39.119+0000 7fc39a0f2640 1 -- 172.21.15.139:0/3194763460 wait complete. 2024-09-16T01:39:39.827 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:39 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3194763460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:39:39.857 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:39:39.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:39 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3194763460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:39:40.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:39 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3194763460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:39:40.109 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:40.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.583+0000 7f150b98f640 1 -- 172.21.15.139:0/46517029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1504101420 msgr2=0x7f150410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:40.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.583+0000 7f150b98f640 1 --2- 172.21.15.139:0/46517029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1504101420 0x7f150410d950 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009cd0 tx=0x7f14f402a8e0 comp rx=0 tx=0).stop 2024-09-16T01:39:40.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.583+0000 7f150b98f640 1 -- 172.21.15.139:0/46517029 shutdown_connections 2024-09-16T01:39:40.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.583+0000 7f150b98f640 1 --2- 172.21.15.139:0/46517029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1504101420 0x7f150410d950 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:40.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.583+0000 7f150b98f640 1 --2- 172.21.15.139:0/46517029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1504100a80 0x7f1504100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:40.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.583+0000 7f150b98f640 1 --2- 172.21.15.139:0/46517029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1504106a80 0x7f1504106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:40.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.583+0000 7f150b98f640 1 -- 172.21.15.139:0/46517029 >> 172.21.15.139:0/46517029 conn(0x7f15040fc7a0 msgr2=0x7f15040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:40.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.583+0000 7f150b98f640 1 -- 172.21.15.139:0/46517029 shutdown_connections 2024-09-16T01:39:40.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.583+0000 7f150b98f640 1 -- 172.21.15.139:0/46517029 wait complete. 2024-09-16T01:39:40.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f150b98f640 1 Processor -- start 2024-09-16T01:39:40.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f150b98f640 1 -- start start 2024-09-16T01:39:40.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f150b98f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1504100a80 0x7f1504102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:40.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f150b98f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1504101420 0x7f1504102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:40.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f150b98f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1504106a80 0x7f15041034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:40.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f150b98f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1504110e80 con 0x7f1504106a80 2024-09-16T01:39:40.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f150b98f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1504110d00 con 0x7f1504100a80 2024-09-16T01:39:40.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f150b98f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1504111000 con 0x7f1504101420 2024-09-16T01:39:40.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f1509f2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1504106a80 0x7f15041034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:40.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f1509f2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1504106a80 0x7f15041034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39676/0 (socket says 172.21.15.139:39676) 2024-09-16T01:39:40.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f1509f2c640 1 -- 172.21.15.139:0/3999111733 learned_addr learned my addr 172.21.15.139:0/3999111733 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:40.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f1508f2a640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1504101420 0x7f1504102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:40.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f150972b640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1504100a80 0x7f1504102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:40.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f1509f2c640 1 -- 172.21.15.139:0/3999111733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1504101420 msgr2=0x7f1504102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:40.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f1509f2c640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1504101420 0x7f1504102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:40.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f1509f2c640 1 -- 172.21.15.139:0/3999111733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1504100a80 msgr2=0x7f1504102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:40.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f1509f2c640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1504100a80 0x7f1504102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:40.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f1509f2c640 1 -- 172.21.15.139:0/3999111733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15041a68b0 con 0x7f1504106a80 2024-09-16T01:39:40.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f150972b640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1504100a80 0x7f1504102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:40.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.584+0000 7f1509f2c640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1504106a80 0x7f15041034b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005cc0 tx=0x7f14f4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:40.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.585+0000 7f15027fc640 1 -- 172.21.15.139:0/3999111733 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4004040 con 0x7f1504106a80 2024-09-16T01:39:40.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.585+0000 7f15027fc640 1 -- 172.21.15.139:0/3999111733 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14f40046c0 con 0x7f1504106a80 2024-09-16T01:39:40.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.585+0000 7f150b98f640 1 -- 172.21.15.139:0/3999111733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15041a6b40 con 0x7f1504106a80 2024-09-16T01:39:40.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.585+0000 7f15027fc640 1 -- 172.21.15.139:0/3999111733 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4046480 con 0x7f1504106a80 2024-09-16T01:39:40.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.585+0000 7f150b98f640 1 -- 172.21.15.139:0/3999111733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15041a7080 con 0x7f1504106a80 2024-09-16T01:39:40.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.586+0000 7f150b98f640 1 -- 172.21.15.139:0/3999111733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1504100ee0 con 0x7f1504106a80 2024-09-16T01:39:40.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.587+0000 7f15027fc640 1 -- 172.21.15.139:0/3999111733 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f14f40305a0 con 0x7f1504106a80 2024-09-16T01:39:40.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.587+0000 7f15027fc640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f14ec0a32d0 0x7f14ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:40.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.587+0000 7f150972b640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f14ec0a32d0 0x7f14ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:40.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.588+0000 7f15027fc640 1 -- 172.21.15.139:0/3999111733 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f14f402e030 con 0x7f1504106a80 2024-09-16T01:39:40.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.588+0000 7f150972b640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f14ec0a32d0 0x7f14ec0a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f14fc005fd0 tx=0x7f14fc0094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:40.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.590+0000 7f15027fc640 1 -- 172.21.15.139:0/3999111733 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f14f404b060 con 0x7f1504106a80 2024-09-16T01:39:40.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.793+0000 7f150b98f640 1 -- 172.21.15.139:0/3999111733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15041a72b0 con 0x7f1504106a80 2024-09-16T01:39:40.795 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:40 smithi139 ceph-mon[31175]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T01:39:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.793+0000 7f15027fc640 1 -- 172.21.15.139:0/3999111733 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11995 (secure 0 0 0) 0x7f14f40bdd90 con 0x7f1504106a80 2024-09-16T01:39:40.796 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:39:40.796 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":28,"btime":"2024-09-16T01:38:40:558188+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:38:40.558170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24454,24443,24442,24433]},"id":1}]} 2024-09-16T01:39:40.796 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 28 2024-09-16T01:39:40.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.796+0000 7f150b98f640 1 -- 172.21.15.139:0/3999111733 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f14ec0a32d0 msgr2=0x7f14ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:40.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.796+0000 7f150b98f640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f14ec0a32d0 0x7f14ec0a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f14fc005fd0 tx=0x7f14fc0094d0 comp rx=0 tx=0).stop 2024-09-16T01:39:40.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.796+0000 7f150b98f640 1 -- 172.21.15.139:0/3999111733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1504106a80 msgr2=0x7f15041034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:40.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.796+0000 7f150b98f640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1504106a80 0x7f15041034b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005cc0 tx=0x7f14f4003970 comp rx=0 tx=0).stop 2024-09-16T01:39:40.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.796+0000 7f150b98f640 1 -- 172.21.15.139:0/3999111733 shutdown_connections 2024-09-16T01:39:40.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.796+0000 7f150b98f640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f14ec0a32d0 0x7f14ec0a5790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:40.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.796+0000 7f150b98f640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1504106a80 0x7f15041034b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:40.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.796+0000 7f150b98f640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1504101420 0x7f1504102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:40.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.796+0000 7f150b98f640 1 --2- 172.21.15.139:0/3999111733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1504100a80 0x7f1504102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:40.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.796+0000 7f150b98f640 1 -- 172.21.15.139:0/3999111733 >> 172.21.15.139:0/3999111733 conn(0x7f15040fc7a0 msgr2=0x7f1504104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:40.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.796+0000 7f150b98f640 1 -- 172.21.15.139:0/3999111733 shutdown_connections 2024-09-16T01:39:40.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:40.796+0000 7f150b98f640 1 -- 172.21.15.139:0/3999111733 wait complete. 2024-09-16T01:39:40.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:40 smithi176 ceph-mon[35324]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T01:39:41.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:40 smithi164 ceph-mon[35121]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T01:39:41.510 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T01:39:41.759 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:41.879 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:41 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3999111733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:39:41.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:41 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3999111733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:39:42.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:41 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3999111733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:39:42.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.250+0000 7f976ec7e640 1 -- 172.21.15.139:0/1832004064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97680ffb60 msgr2=0x7f97680fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:42.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.250+0000 7f976ec7e640 1 --2- 172.21.15.139:0/1832004064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97680ffb60 0x7f97680fffc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9750009900 tx=0x7f975002a410 comp rx=0 tx=0).stop 2024-09-16T01:39:42.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.250+0000 7f976ec7e640 1 -- 172.21.15.139:0/1832004064 shutdown_connections 2024-09-16T01:39:42.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.250+0000 7f976ec7e640 1 --2- 172.21.15.139:0/1832004064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9768100500 0x7f976810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:42.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.250+0000 7f976ec7e640 1 --2- 172.21.15.139:0/1832004064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97680ffb60 0x7f97680fffc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:42.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.250+0000 7f976ec7e640 1 --2- 172.21.15.139:0/1832004064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9768106b80 0x7f9768106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:42.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.250+0000 7f976ec7e640 1 -- 172.21.15.139:0/1832004064 >> 172.21.15.139:0/1832004064 conn(0x7f97680fba90 msgr2=0x7f97680fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:42.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.250+0000 7f976ec7e640 1 -- 172.21.15.139:0/1832004064 shutdown_connections 2024-09-16T01:39:42.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.250+0000 7f976ec7e640 1 -- 172.21.15.139:0/1832004064 wait complete. 2024-09-16T01:39:42.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.251+0000 7f976ec7e640 1 Processor -- start 2024-09-16T01:39:42.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.251+0000 7f976ec7e640 1 -- start start 2024-09-16T01:39:42.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.251+0000 7f976ec7e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97680ffb60 0x7f976819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:42.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.251+0000 7f976ec7e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9768100500 0x7f976819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:42.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.251+0000 7f976ec7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9768106b80 0x7f976819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:42.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.251+0000 7f976ec7e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9768111fa0 con 0x7f9768106b80 2024-09-16T01:39:42.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.251+0000 7f976ec7e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9768111e20 con 0x7f9768100500 2024-09-16T01:39:42.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.251+0000 7f976ec7e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9768112120 con 0x7f97680ffb60 2024-09-16T01:39:42.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.251+0000 7f976d21b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9768106b80 0x7f976819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:42.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.251+0000 7f976d21b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9768106b80 0x7f976819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39690/0 (socket says 172.21.15.139:39690) 2024-09-16T01:39:42.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.251+0000 7f976d21b640 1 -- 172.21.15.139:0/1090273892 learned_addr learned my addr 172.21.15.139:0/1090273892 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:42.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.251+0000 7f9767fff640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9768100500 0x7f976819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:42.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.252+0000 7f976ca1a640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97680ffb60 0x7f976819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:42.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.252+0000 7f976d21b640 1 -- 172.21.15.139:0/1090273892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97680ffb60 msgr2=0x7f976819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:42.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.252+0000 7f976d21b640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97680ffb60 0x7f976819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:42.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.252+0000 7f976d21b640 1 -- 172.21.15.139:0/1090273892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9768100500 msgr2=0x7f976819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:42.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.252+0000 7f976d21b640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9768100500 0x7f976819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:42.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.252+0000 7f976d21b640 1 -- 172.21.15.139:0/1090273892 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976819f650 con 0x7f9768106b80 2024-09-16T01:39:42.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.252+0000 7f976ca1a640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97680ffb60 0x7f976819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:42.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.252+0000 7f976d21b640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9768106b80 0x7f976819ef70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f976000e9d0 tx=0x7f976000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:42.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.252+0000 7f9765ffb640 1 -- 172.21.15.139:0/1090273892 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976000cd80 con 0x7f9768106b80 2024-09-16T01:39:42.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.252+0000 7f976ec7e640 1 -- 172.21.15.139:0/1090273892 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976819f940 con 0x7f9768106b80 2024-09-16T01:39:42.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.252+0000 7f9765ffb640 1 -- 172.21.15.139:0/1090273892 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9760005020 con 0x7f9768106b80 2024-09-16T01:39:42.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.252+0000 7f9765ffb640 1 -- 172.21.15.139:0/1090273892 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97600116d0 con 0x7f9768106b80 2024-09-16T01:39:42.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.252+0000 7f976ec7e640 1 -- 172.21.15.139:0/1090273892 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97681a7220 con 0x7f9768106b80 2024-09-16T01:39:42.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.253+0000 7f976ec7e640 1 -- 172.21.15.139:0/1090273892 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97680fffc0 con 0x7f9768106b80 2024-09-16T01:39:42.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.254+0000 7f9765ffb640 1 -- 172.21.15.139:0/1090273892 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f97600051d0 con 0x7f9768106b80 2024-09-16T01:39:42.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.255+0000 7f9765ffb640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97540a32d0 0x7f97540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:42.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.255+0000 7f976ca1a640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97540a32d0 0x7f97540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:42.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.255+0000 7f9765ffb640 1 -- 172.21.15.139:0/1090273892 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f97600d03c0 con 0x7f9768106b80 2024-09-16T01:39:42.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.255+0000 7f976ca1a640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97540a32d0 0x7f97540a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9758009930 tx=0x7f9758008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:42.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.259+0000 7f9765ffb640 1 -- 172.21.15.139:0/1090273892 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9760016030 con 0x7f9768106b80 2024-09-16T01:39:42.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.429+0000 7f976ec7e640 1 -- 172.21.15.139:0/1090273892 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97681a7520 con 0x7f9768106b80 2024-09-16T01:39:42.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.431+0000 7f9765ffb640 1 -- 172.21.15.139:0/1090273892 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24985 (secure 0 0 0) 0x7f976009cc50 con 0x7f9768106b80 2024-09-16T01:39:42.433 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:39:42.433 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":85,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","created":"2024-09-16T01:30:17.770313+0000","modified":"2024-09-16T01:39:09.511648+0000","last_up_change":"2024-09-16T01:37:08.611901+0000","last_in_change":"2024-09-16T01:36:45.223926+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:33:19.639647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T01:37:57.948214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T01:38:00.263797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T01:38:06.773799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b66696da-ef66-4f25-9527-bbcc9ded2cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6803","nonce":3190810133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6805","nonce":3190810133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6809","nonce":3190810133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6807","nonce":3190810133}]},"public_addr":"172.21.15.139:6803/3190810133","cluster_addr":"172.21.15.139:6805/3190810133","heartbeat_back_addr":"172.21.15.139:6809/3190810133","heartbeat_front_addr":"172.21.15.139:6807/3190810133","state":["exists","up"]},{"osd":1,"uuid":"ccad17f2-0750-425e-9810-49a13a929ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6801","nonce":2699270330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6803","nonce":2699270330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6807","nonce":2699270330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6805","nonce":2699270330}]},"public_addr":"172.21.15.164:6801/2699270330","cluster_addr":"172.21.15.164:6803/2699270330","heartbeat_back_addr":"172.21.15.164:6807/2699270330","heartbeat_front_addr":"172.21.15.164:6805/2699270330","state":["exists","up"]},{"osd":2,"uuid":"baf0ba73-8faf-4e51-a171-c47898800638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6801","nonce":2034471646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6803","nonce":2034471646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6807","nonce":2034471646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6805","nonce":2034471646}]},"public_addr":"172.21.15.176:6801/2034471646","cluster_addr":"172.21.15.176:6803/2034471646","heartbeat_back_addr":"172.21.15.176:6807/2034471646","heartbeat_front_addr":"172.21.15.176:6805/2034471646","state":["exists","up"]},{"osd":3,"uuid":"98957bf1-67bb-4092-9ea4-b263a5706f8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6811","nonce":797147071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6813","nonce":797147071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6817","nonce":797147071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6815","nonce":797147071}]},"public_addr":"172.21.15.139:6811/797147071","cluster_addr":"172.21.15.139:6813/797147071","heartbeat_back_addr":"172.21.15.139:6817/797147071","heartbeat_front_addr":"172.21.15.139:6815/797147071","state":["exists","up"]},{"osd":4,"uuid":"9cd76696-8161-4e82-a571-29e98c11d5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6809","nonce":1421304792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6811","nonce":1421304792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6815","nonce":1421304792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6813","nonce":1421304792}]},"public_addr":"172.21.15.164:6809/1421304792","cluster_addr":"172.21.15.164:6811/1421304792","heartbeat_back_addr":"172.21.15.164:6815/1421304792","heartbeat_front_addr":"172.21.15.164:6813/1421304792","state":["exists","up"]},{"osd":5,"uuid":"ccc17a29-9a58-40ec-b453-5c38bee86f25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6809","nonce":10600602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6811","nonce":10600602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6815","nonce":10600602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6813","nonce":10600602}]},"public_addr":"172.21.15.176:6809/10600602","cluster_addr":"172.21.15.176:6811/10600602","heartbeat_back_addr":"172.21.15.176:6815/10600602","heartbeat_front_addr":"172.21.15.176:6813/10600602","state":["exists","up"]},{"osd":6,"uuid":"e5ee7212-10c6-4a05-992e-622a14a5a6b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6819","nonce":2601111164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6821","nonce":2601111164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6825","nonce":2601111164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6823","nonce":2601111164}]},"public_addr":"172.21.15.139:6819/2601111164","cluster_addr":"172.21.15.139:6821/2601111164","heartbeat_back_addr":"172.21.15.139:6825/2601111164","heartbeat_front_addr":"172.21.15.139:6823/2601111164","state":["exists","up"]},{"osd":7,"uuid":"23143c15-0839-4fd5-9d0a-19ae891000f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6817","nonce":2074080693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6819","nonce":2074080693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6823","nonce":2074080693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6821","nonce":2074080693}]},"public_addr":"172.21.15.164:6817/2074080693","cluster_addr":"172.21.15.164:6819/2074080693","heartbeat_back_addr":"172.21.15.164:6823/2074080693","heartbeat_front_addr":"172.21.15.164:6821/2074080693","state":["exists","up"]},{"osd":8,"uuid":"1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6817","nonce":593133361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6819","nonce":593133361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6823","nonce":593133361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6821","nonce":593133361}]},"public_addr":"172.21.15.176:6817/593133361","cluster_addr":"172.21.15.176:6819/593133361","heartbeat_back_addr":"172.21.15.176:6823/593133361","heartbeat_front_addr":"172.21.15.176:6821/593133361","state":["exists","up"]},{"osd":9,"uuid":"09d25a3b-1e66-480e-9433-ceef7f0b9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6827","nonce":3047942108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6829","nonce":3047942108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6833","nonce":3047942108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6831","nonce":3047942108}]},"public_addr":"172.21.15.139:6827/3047942108","cluster_addr":"172.21.15.139:6829/3047942108","heartbeat_back_addr":"172.21.15.139:6833/3047942108","heartbeat_front_addr":"172.21.15.139:6831/3047942108","state":["exists","up"]},{"osd":10,"uuid":"32b4d8ce-b644-42fa-80cc-a15a1f5f3a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6825","nonce":4172533828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6827","nonce":4172533828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6831","nonce":4172533828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6829","nonce":4172533828}]},"public_addr":"172.21.15.164:6825/4172533828","cluster_addr":"172.21.15.164:6827/4172533828","heartbeat_back_addr":"172.21.15.164:6831/4172533828","heartbeat_front_addr":"172.21.15.164:6829/4172533828","state":["exists","up"]},{"osd":11,"uuid":"49098bdc-58a9-4a74-8045-d408af7711ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6825","nonce":1854547565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6827","nonce":1854547565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6831","nonce":1854547565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6829","nonce":1854547565}]},"public_addr":"172.21.15.176:6825/1854547565","cluster_addr":"172.21.15.176:6827/1854547565","heartbeat_back_addr":"172.21.15.176:6831/1854547565","heartbeat_front_addr":"172.21.15.176:6829/1854547565","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:27.374845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:52.611853+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:17.229567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:43.330503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:07.988876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:33.787864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:58.864910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:23.815006+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:49.551954+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:15.496773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:41.647557+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:37:07.089355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3772232169":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/3997627612":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/1608930262":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6801/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/677216867":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6801/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/1830135963":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/2608207648":"2024-09-17T01:30:44.751168+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:39:42.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.434+0000 7f976ec7e640 1 -- 172.21.15.139:0/1090273892 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97540a32d0 msgr2=0x7f97540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:42.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.434+0000 7f976ec7e640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97540a32d0 0x7f97540a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9758009930 tx=0x7f9758008040 comp rx=0 tx=0).stop 2024-09-16T01:39:42.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.434+0000 7f976ec7e640 1 -- 172.21.15.139:0/1090273892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9768106b80 msgr2=0x7f976819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:42.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.434+0000 7f976ec7e640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9768106b80 0x7f976819ef70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f976000e9d0 tx=0x7f976000ee90 comp rx=0 tx=0).stop 2024-09-16T01:39:42.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.434+0000 7f976ec7e640 1 -- 172.21.15.139:0/1090273892 shutdown_connections 2024-09-16T01:39:42.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.434+0000 7f976ec7e640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97540a32d0 0x7f97540a5790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:42.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.434+0000 7f976ec7e640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9768106b80 0x7f976819ef70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:42.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.434+0000 7f976ec7e640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9768100500 0x7f976819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:42.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.434+0000 7f976ec7e640 1 --2- 172.21.15.139:0/1090273892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97680ffb60 0x7f976819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:42.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.434+0000 7f976ec7e640 1 -- 172.21.15.139:0/1090273892 >> 172.21.15.139:0/1090273892 conn(0x7f97680fba90 msgr2=0x7f976810d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:42.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.434+0000 7f976ec7e640 1 -- 172.21.15.139:0/1090273892 shutdown_connections 2024-09-16T01:39:42.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:42.434+0000 7f976ec7e640 1 -- 172.21.15.139:0/1090273892 wait complete. 2024-09-16T01:39:42.914 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:42 smithi139 ceph-mon[31175]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-16T01:39:42.915 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:42 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1090273892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:39:42.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:42 smithi176 ceph-mon[35324]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-16T01:39:42.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:42 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1090273892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:39:43.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:42 smithi164 ceph-mon[35121]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-16T01:39:43.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:42 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1090273892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:39:43.264 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:39:43.510 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:44.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.984+0000 7fe23adc1640 1 -- 172.21.15.139:0/2913171421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe234102ef0 msgr2=0x7fe23410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:44.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.984+0000 7fe23adc1640 1 --2- 172.21.15.139:0/2913171421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe234102ef0 0x7fe23410da40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009ca0 tx=0x7fe22c02a950 comp rx=0 tx=0).stop 2024-09-16T01:39:44.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.984+0000 7fe23adc1640 1 -- 172.21.15.139:0/2913171421 shutdown_connections 2024-09-16T01:39:44.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.984+0000 7fe23adc1640 1 --2- 172.21.15.139:0/2913171421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe234102ef0 0x7fe23410da40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:44.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.984+0000 7fe23adc1640 1 --2- 172.21.15.139:0/2913171421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe234102570 0x7fe2341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:44.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.984+0000 7fe23adc1640 1 --2- 172.21.15.139:0/2913171421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe234100e20 0x7fe234101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:44.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.984+0000 7fe23adc1640 1 -- 172.21.15.139:0/2913171421 >> 172.21.15.139:0/2913171421 conn(0x7fe2340fc7d0 msgr2=0x7fe2340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:44.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.984+0000 7fe23adc1640 1 -- 172.21.15.139:0/2913171421 shutdown_connections 2024-09-16T01:39:44.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.984+0000 7fe23adc1640 1 -- 172.21.15.139:0/2913171421 wait complete. 2024-09-16T01:39:44.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.985+0000 7fe23adc1640 1 Processor -- start 2024-09-16T01:39:44.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.985+0000 7fe23adc1640 1 -- start start 2024-09-16T01:39:44.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.985+0000 7fe23adc1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe234100e20 0x7fe234199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:44.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.985+0000 7fe23adc1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe234102570 0x7fe2341997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:44.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.985+0000 7fe23adc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe234102ef0 0x7fe2341a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:44.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.985+0000 7fe23adc1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe234110f80 con 0x7fe234102ef0 2024-09-16T01:39:44.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.985+0000 7fe23adc1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe234110e00 con 0x7fe234100e20 2024-09-16T01:39:44.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.985+0000 7fe23adc1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe234111100 con 0x7fe234102570 2024-09-16T01:39:44.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.985+0000 7fe23935e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe234102ef0 0x7fe2341a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:44.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.985+0000 7fe23935e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe234102ef0 0x7fe2341a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43678/0 (socket says 172.21.15.139:43678) 2024-09-16T01:39:44.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.985+0000 7fe23935e640 1 -- 172.21.15.139:0/1900159109 learned_addr learned my addr 172.21.15.139:0/1900159109 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:44.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.985+0000 7fe238b5d640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe234100e20 0x7fe234199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:44.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.985+0000 7fe233fff640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe234102570 0x7fe2341997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:44.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.986+0000 7fe23935e640 1 -- 172.21.15.139:0/1900159109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe234102570 msgr2=0x7fe2341997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:44.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.986+0000 7fe23935e640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe234102570 0x7fe2341997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:44.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.986+0000 7fe23935e640 1 -- 172.21.15.139:0/1900159109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe234100e20 msgr2=0x7fe234199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:44.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.986+0000 7fe23935e640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe234100e20 0x7fe234199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:44.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.986+0000 7fe23935e640 1 -- 172.21.15.139:0/1900159109 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2341a4850 con 0x7fe234102ef0 2024-09-16T01:39:44.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.986+0000 7fe238b5d640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe234100e20 0x7fe234199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:39:44.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.986+0000 7fe23935e640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe234102ef0 0x7fe2341a4310 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe22c02ae70 tx=0x7fe22c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:44.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.986+0000 7fe231ffb640 1 -- 172.21.15.139:0/1900159109 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c034040 con 0x7fe234102ef0 2024-09-16T01:39:44.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.986+0000 7fe23adc1640 1 -- 172.21.15.139:0/1900159109 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2341a4a20 con 0x7fe234102ef0 2024-09-16T01:39:44.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.986+0000 7fe231ffb640 1 -- 172.21.15.139:0/1900159109 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe22c030920 con 0x7fe234102ef0 2024-09-16T01:39:44.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.986+0000 7fe231ffb640 1 -- 172.21.15.139:0/1900159109 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c045480 con 0x7fe234102ef0 2024-09-16T01:39:44.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.986+0000 7fe23adc1640 1 -- 172.21.15.139:0/1900159109 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2341a4ed0 con 0x7fe234102ef0 2024-09-16T01:39:44.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.987+0000 7fe23adc1640 1 -- 172.21.15.139:0/1900159109 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe234101200 con 0x7fe234102ef0 2024-09-16T01:39:44.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.989+0000 7fe231ffb640 1 -- 172.21.15.139:0/1900159109 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe22c037020 con 0x7fe234102ef0 2024-09-16T01:39:44.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.990+0000 7fe231ffb640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fe2240a32d0 0x7fe2240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:44.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.990+0000 7fe231ffb640 1 -- 172.21.15.139:0/1900159109 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fe22c0f0d10 con 0x7fe234102ef0 2024-09-16T01:39:44.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.990+0000 7fe238b5d640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fe2240a32d0 0x7fe2240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:44.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.990+0000 7fe238b5d640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fe2240a32d0 0x7fe2240a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe228005d00 tx=0x7fe22800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:44.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:43.993+0000 7fe231ffb640 1 -- 172.21.15.139:0/1900159109 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe22c0811b0 con 0x7fe234102ef0 2024-09-16T01:39:44.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:44.195+0000 7fe23adc1640 1 -- 172.21.15.139:0/1900159109 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2341a5100 con 0x7fe234102ef0 2024-09-16T01:39:44.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:44.195+0000 7fe231ffb640 1 -- 172.21.15.139:0/1900159109 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11995 (secure 0 0 0) 0x7fe22c0bd610 con 0x7fe234102ef0 2024-09-16T01:39:44.197 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:39:44.198 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":28,"btime":"2024-09-16T01:38:40:558188+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:38:40.558170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24454,24443,24442,24433]},"id":1}]} 2024-09-16T01:39:44.198 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 28 2024-09-16T01:39:44.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:44.198+0000 7fe23adc1640 1 -- 172.21.15.139:0/1900159109 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fe2240a32d0 msgr2=0x7fe2240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:44.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:44.198+0000 7fe23adc1640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fe2240a32d0 0x7fe2240a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe228005d00 tx=0x7fe22800b410 comp rx=0 tx=0).stop 2024-09-16T01:39:44.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:44.198+0000 7fe23adc1640 1 -- 172.21.15.139:0/1900159109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe234102ef0 msgr2=0x7fe2341a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:44.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:44.198+0000 7fe23adc1640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe234102ef0 0x7fe2341a4310 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe22c02ae70 tx=0x7fe22c02f5d0 comp rx=0 tx=0).stop 2024-09-16T01:39:44.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:44.198+0000 7fe23adc1640 1 -- 172.21.15.139:0/1900159109 shutdown_connections 2024-09-16T01:39:44.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:44.198+0000 7fe23adc1640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fe2240a32d0 0x7fe2240a5790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:44.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:44.198+0000 7fe23adc1640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe234102ef0 0x7fe2341a4310 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:44.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:44.198+0000 7fe23adc1640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe234102570 0x7fe2341997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:44.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:44.198+0000 7fe23adc1640 1 --2- 172.21.15.139:0/1900159109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe234100e20 0x7fe234199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:44.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:44.198+0000 7fe23adc1640 1 -- 172.21.15.139:0/1900159109 >> 172.21.15.139:0/1900159109 conn(0x7fe2340fc7d0 msgr2=0x7fe234106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:44.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:44.198+0000 7fe23adc1640 1 -- 172.21.15.139:0/1900159109 shutdown_connections 2024-09-16T01:39:44.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:44.198+0000 7fe23adc1640 1 -- 172.21.15.139:0/1900159109 wait complete. 2024-09-16T01:39:44.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:44 smithi139 ceph-mon[31175]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:39:44.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:44 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1900159109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:39:44.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:44 smithi176 ceph-mon[35324]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:39:44.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:44 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1900159109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:39:44.991 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T01:38:02.129081+0000', 'modified': '2024-09-16T01:38:40.558170+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24470, 'mds_1': 24454, 'mds_2': 24443, 'mds_3': 24442, 'mds_4': 24433}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14604': {'gid': 14604, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6839/1075273450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 1075273450}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 1075273450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14616': {'gid': 14616, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6841/1887551794', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6840', 'nonce': 1887551794}, {'type': 'v1', 'addr': '172.21.15.139:6841', 'nonce': 1887551794}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24395': {'gid': 24395, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6833/416925170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 416925170}, {'type': 'v1', 'addr': '172.21.15.164:6833', 'nonce': 416925170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.176:6833/1026776933', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6832', 'nonce': 1026776933}, {'type': 'v1', 'addr': '172.21.15.176:6833', 'nonce': 1026776933}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6835/3662238682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 3662238682}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 3662238682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.176:6835/3060423566', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6834', 'nonce': 3060423566}, {'type': 'v1', 'addr': '172.21.15.176:6835', 'nonce': 3060423566}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.176:6837/1942543693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6836', 'nonce': 1942543693}, {'type': 'v1', 'addr': '172.21.15.176:6837', 'nonce': 1942543693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.164:6837/1270226509', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 1270226509}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 1270226509}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.176:6839/3912805068', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6838', 'nonce': 3912805068}, {'type': 'v1', 'addr': '172.21.15.176:6839', 'nonce': 3912805068}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.164:6839/2447189436', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 2447189436}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 2447189436}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24470, 'qdb_cluster': [24470, 24454, 24443, 24442, 24433]} 2024-09-16T01:39:44.991 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-16T01:39:44.991 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-16T01:39:45.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:44 smithi164 ceph-mon[35121]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:39:45.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:44 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1900159109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:39:45.237 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:45.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.719+0000 7fd86fa47640 1 -- 172.21.15.139:0/2314868403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd868100b30 msgr2=0x7fd868100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:45.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.719+0000 7fd86fa47640 1 --2- 172.21.15.139:0/2314868403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd868100b30 0x7fd868100f10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd864009900 tx=0x7fd86402a470 comp rx=0 tx=0).stop 2024-09-16T01:39:45.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.719+0000 7fd86fa47640 1 -- 172.21.15.139:0/2314868403 shutdown_connections 2024-09-16T01:39:45.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.719+0000 7fd86fa47640 1 --2- 172.21.15.139:0/2314868403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd868105f40 0x7fd8681063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.719+0000 7fd86fa47640 1 --2- 172.21.15.139:0/2314868403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd868101450 0x7fd868105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.719+0000 7fd86fa47640 1 --2- 172.21.15.139:0/2314868403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd868100b30 0x7fd868100f10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.719+0000 7fd86fa47640 1 -- 172.21.15.139:0/2314868403 >> 172.21.15.139:0/2314868403 conn(0x7fd8680fc7a0 msgr2=0x7fd8680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:45.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.719+0000 7fd86fa47640 1 -- 172.21.15.139:0/2314868403 shutdown_connections 2024-09-16T01:39:45.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.719+0000 7fd86fa47640 1 -- 172.21.15.139:0/2314868403 wait complete. 2024-09-16T01:39:45.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.720+0000 7fd86fa47640 1 Processor -- start 2024-09-16T01:39:45.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.720+0000 7fd86fa47640 1 -- start start 2024-09-16T01:39:45.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.720+0000 7fd86fa47640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd868100b30 0x7fd8681a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.720+0000 7fd86fa47640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd868101450 0x7fd8681a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.720+0000 7fd86fa47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd868105f40 0x7fd86819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.720+0000 7fd86fa47640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd868118570 con 0x7fd868105f40 2024-09-16T01:39:45.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.720+0000 7fd86fa47640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd8681183f0 con 0x7fd868100b30 2024-09-16T01:39:45.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.720+0000 7fd86fa47640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd8681186f0 con 0x7fd868101450 2024-09-16T01:39:45.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.720+0000 7fd86dfe4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd868105f40 0x7fd86819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.720+0000 7fd86dfe4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd868105f40 0x7fd86819e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43696/0 (socket says 172.21.15.139:43696) 2024-09-16T01:39:45.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.720+0000 7fd86dfe4640 1 -- 172.21.15.139:0/4100496570 learned_addr learned my addr 172.21.15.139:0/4100496570 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:45.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.720+0000 7fd86d7e3640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd868100b30 0x7fd8681a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.720+0000 7fd86cfe2640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd868101450 0x7fd8681a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.721+0000 7fd86dfe4640 1 -- 172.21.15.139:0/4100496570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd868101450 msgr2=0x7fd8681a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:45.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.721+0000 7fd86dfe4640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd868101450 0x7fd8681a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.721+0000 7fd86dfe4640 1 -- 172.21.15.139:0/4100496570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd868100b30 msgr2=0x7fd8681a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:45.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.721+0000 7fd86dfe4640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd868100b30 0x7fd8681a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.721+0000 7fd86dfe4640 1 -- 172.21.15.139:0/4100496570 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86819f000 con 0x7fd868105f40 2024-09-16T01:39:45.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.721+0000 7fd86cfe2640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd868101450 0x7fd8681a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:45.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.721+0000 7fd86dfe4640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd868105f40 0x7fd86819e8d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd86000eaa0 tx=0x7fd86000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:45.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.721+0000 7fd85e7fc640 1 -- 172.21.15.139:0/4100496570 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86000cdb0 con 0x7fd868105f40 2024-09-16T01:39:45.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.721+0000 7fd86fa47640 1 -- 172.21.15.139:0/4100496570 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86819f2f0 con 0x7fd868105f40 2024-09-16T01:39:45.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.721+0000 7fd85e7fc640 1 -- 172.21.15.139:0/4100496570 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd860005020 con 0x7fd868105f40 2024-09-16T01:39:45.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.721+0000 7fd85e7fc640 1 -- 172.21.15.139:0/4100496570 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8600116d0 con 0x7fd868105f40 2024-09-16T01:39:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.721+0000 7fd86fa47640 1 -- 172.21.15.139:0/4100496570 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8681ab5e0 con 0x7fd868105f40 2024-09-16T01:39:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.723+0000 7fd85e7fc640 1 -- 172.21.15.139:0/4100496570 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd8600054d0 con 0x7fd868105f40 2024-09-16T01:39:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd871e20740 1 Processor -- start 2024-09-16T01:39:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd871e20740 1 -- start start 2024-09-16T01:39:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd871e20740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5e59d0 0x5557fa5e5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd871e20740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e62f0 0x5557fa5f0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd871e20740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5f0bb0 0x5557fa5f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd871e20740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5557fa54c090 con 0x5557fa5f0bb0 2024-09-16T01:39:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd871e20740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5557fa4f3a70 con 0x5557fa5e59d0 2024-09-16T01:39:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd871e20740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5557fa548050 con 0x5557fa5e62f0 2024-09-16T01:39:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd85e7fc640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd8540a32d0 0x7fd8540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd86cfe2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5f0bb0 0x5557fa5f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd86cfe2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5f0bb0 0x5557fa5f2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43700/0 (socket says 172.21.15.139:43700) 2024-09-16T01:39:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd86d7e3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e62f0 0x5557fa5f0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd86cfe2640 1 -- 172.21.15.139:0/3014850045 learned_addr learned my addr 172.21.15.139:0/3014850045 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd86cfe2640 1 --2- 172.21.15.139:0/3014850045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5e59d0 0x5557fa5e5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd86dfe4640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd8540a32d0 0x7fd8540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd85e7fc640 1 -- 172.21.15.139:0/4100496570 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fd8600d0070 con 0x7fd868105f40 2024-09-16T01:39:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd86d7e3640 1 -- 172.21.15.139:0/3014850045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5e59d0 msgr2=0x5557fa5e5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd86d7e3640 1 --2- 172.21.15.139:0/3014850045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5e59d0 0x5557fa5e5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd86d7e3640 1 -- 172.21.15.139:0/3014850045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5f0bb0 msgr2=0x5557fa5f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd86d7e3640 1 --2- 172.21.15.139:0/3014850045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5f0bb0 0x5557fa5f2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd86d7e3640 1 -- 172.21.15.139:0/3014850045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557fa500130 con 0x5557fa5e62f0 2024-09-16T01:39:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.724+0000 7fd86cfe2640 1 --2- 172.21.15.139:0/3014850045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5f0bb0 0x5557fa5f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:39:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.725+0000 7fd86d7e3640 1 --2- 172.21.15.139:0/3014850045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e62f0 0x5557fa5f0670 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd864005940 tx=0x7fd864005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=624606273d656952 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.725+0000 7fd86dfe4640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd8540a32d0 0x7fd8540a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd86009c0f0 tx=0x7fd860009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.725+0000 7fd8537fe640 1 -- 172.21.15.139:0/3014850045 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864040070 con 0x5557fa5e62f0 2024-09-16T01:39:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.725+0000 7fd8537fe640 1 -- 172.21.15.139:0/3014850045 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8640319f0 con 0x5557fa5e62f0 2024-09-16T01:39:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.725+0000 7fd8537fe640 1 -- 172.21.15.139:0/3014850045 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864035040 con 0x5557fa5e62f0 2024-09-16T01:39:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.726+0000 7fd871e20740 1 -- 172.21.15.139:0/3014850045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e62f0 msgr2=0x5557fa5f0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.726+0000 7fd871e20740 1 --2- 172.21.15.139:0/3014850045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e62f0 0x5557fa5f0670 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd864005940 tx=0x7fd864005e80 comp rx=0 tx=0).stop 2024-09-16T01:39:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.726+0000 7fd871e20740 1 -- 172.21.15.139:0/3014850045 shutdown_connections 2024-09-16T01:39:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.726+0000 7fd871e20740 1 --2- 172.21.15.139:0/3014850045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5f0bb0 0x5557fa5f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.726+0000 7fd871e20740 1 --2- 172.21.15.139:0/3014850045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e62f0 0x5557fa5f0670 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.726+0000 7fd871e20740 1 --2- 172.21.15.139:0/3014850045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5e59d0 0x5557fa5e5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.726+0000 7fd871e20740 1 -- 172.21.15.139:0/3014850045 >> 172.21.15.139:0/3014850045 conn(0x5557fa5e1f60 msgr2=0x5557fa5e2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.726+0000 7fd871e20740 1 -- 172.21.15.139:0/3014850045 shutdown_connections 2024-09-16T01:39:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.726+0000 7fd871e20740 1 -- 172.21.15.139:0/3014850045 wait complete. 2024-09-16T01:39:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.726+0000 7fd871e20740 1 -- start start 2024-09-16T01:39:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.726+0000 7fd871e20740 1 Processor -- start 2024-09-16T01:39:45.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.726+0000 7fd871e20740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e59d0 0x5557fa5f03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd871e20740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e62f0 0x5557fa5ed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd871e20740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5f0bb0 0x5557fa5edc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd871e20740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5557fa5f3780 con 0x5557fa5e62f0 2024-09-16T01:39:45.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd871e20740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5557fa506f20 con 0x5557fa5f0bb0 2024-09-16T01:39:45.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd871e20740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5557fa5f3900 con 0x5557fa5e59d0 2024-09-16T01:39:45.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd871e20740 10 client.0 _ll_register_callbacks cb 0x5557fa4d0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:39:45.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd86d7e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e62f0 0x5557fa5ed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd86d7e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e62f0 0x5557fa5ed6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43710/0 (socket says 172.21.15.139:43710) 2024-09-16T01:39:45.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd86d7e3640 1 -- 172.21.15.139:0/1287446808 learned_addr learned my addr 172.21.15.139:0/1287446808 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:45.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd86cfe2640 1 --2- 172.21.15.139:0/1287446808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e59d0 0x5557fa5f03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd86cfe2640 1 --2- 172.21.15.139:0/1287446808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5f0bb0 0x5557fa5edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd86d7e3640 1 -- 172.21.15.139:0/1287446808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e59d0 msgr2=0x5557fa5f03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd86d7e3640 1 --2- 172.21.15.139:0/1287446808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e59d0 0x5557fa5f03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd86d7e3640 1 -- 172.21.15.139:0/1287446808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5f0bb0 msgr2=0x5557fa5edc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd86d7e3640 1 --2- 172.21.15.139:0/1287446808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5f0bb0 0x5557fa5edc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd86d7e3640 1 -- 172.21.15.139:0/1287446808 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557fa2bd2b0 con 0x5557fa5e62f0 2024-09-16T01:39:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd86cfe2640 1 --2- 172.21.15.139:0/1287446808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5f0bb0 0x5557fa5edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd86d7e3640 1 --2- 172.21.15.139:0/1287446808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e62f0 0x5557fa5ed6f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd864006010 tx=0x7fd864004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac71bc010c17995d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd850ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:39:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd850ff9640 1 -- 172.21.15.139:0/1287446808 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86404c020 con 0x5557fa5e62f0 2024-09-16T01:39:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd850ff9640 1 -- 172.21.15.139:0/1287446808 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fd864004260 con 0x5557fa5e62f0 2024-09-16T01:39:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd871e20740 1 -- 172.21.15.139:0/1287446808 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8640319f0 con 0x5557fa5e62f0 2024-09-16T01:39:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd850ff9640 1 -- 172.21.15.139:0/1287446808 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd864057050 con 0x5557fa5e62f0 2024-09-16T01:39:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd871e20740 10 client.15072 fetch_fsmap learned FSMap version 28 2024-09-16T01:39:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.727+0000 7fd871e20740 1 -- 172.21.15.139:0/1287446808 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557fa5feff0 con 0x5557fa5e62f0 2024-09-16T01:39:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.728+0000 7fd850ff9640 1 -- 172.21.15.139:0/1287446808 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8640319f0 con 0x5557fa5e62f0 2024-09-16T01:39:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.728+0000 7fd850ff9640 1 -- 172.21.15.139:0/1287446808 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fd864004db0 con 0x5557fa5e62f0 2024-09-16T01:39:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.728+0000 7fd871e20740 10 client.15072 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:39:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.728+0000 7fd871e20740 10 client.15072 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T01:39:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.728+0000 7fd871e20740 20 client.15072 populate_metadata read hostname 'smithi139' 2024-09-16T01:39:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.728+0000 7fd871e20740 1 --2- 172.21.15.139:0/1287446808 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x5557fa601210 0x5557fa602680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.728+0000 7fd871e20740 4 client.15072 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:39:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.728+0000 7fd871e20740 1 -- 172.21.15.139:0/1287446808 --> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557fa559600 con 0x5557fa601210 2024-09-16T01:39:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.728+0000 7fd86cfe2640 1 --2- 172.21.15.139:0/1287446808 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x5557fa601210 0x5557fa602680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.729+0000 7fd86cfe2640 1 --2- 172.21.15.139:0/1287446808 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x5557fa601210 0x5557fa602680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=281b139a72dc6699 server_cookie=a2e93807ef18bcf6 in_seq=0 out_seq=0 2024-09-16T01:39:45.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.729+0000 7fd850ff9640 10 client.15072 ms_handle_connect on v2:172.21.15.176:6834/3060423566 2024-09-16T01:39:45.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.729+0000 7fd850ff9640 1 -- 172.21.15.139:0/1287446808 <== mds.4 v2:172.21.15.176:6834/3060423566 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557fa559600 con 0x5557fa601210 2024-09-16T01:39:45.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.729+0000 7fd850ff9640 10 client.15072 handle_command_reply: tid=0 2024-09-16T01:39:45.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.730+0000 7fd871e20740 1 client.15072 shutdown 2024-09-16T01:39:45.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.730+0000 7fd871e20740 1 -- 172.21.15.139:0/1287446808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e62f0 msgr2=0x5557fa5ed6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:45.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.730+0000 7fd871e20740 1 --2- 172.21.15.139:0/1287446808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e62f0 0x5557fa5ed6f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd864006010 tx=0x7fd864004a00 comp rx=0 tx=0).stop 2024-09-16T01:39:45.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.730+0000 7fd871e20740 1 -- 172.21.15.139:0/1287446808 shutdown_connections 2024-09-16T01:39:45.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.730+0000 7fd871e20740 1 --2- 172.21.15.139:0/1287446808 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x5557fa601210 0x5557fa602680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.730+0000 7fd871e20740 1 --2- 172.21.15.139:0/1287446808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5f0bb0 0x5557fa5edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.730+0000 7fd871e20740 1 --2- 172.21.15.139:0/1287446808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e62f0 0x5557fa5ed6f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.730+0000 7fd871e20740 1 --2- 172.21.15.139:0/1287446808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e59d0 0x5557fa5f03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.730+0000 7fd850ff9640 0 client.15072 ms_handle_reset on v2:172.21.15.176:6834/3060423566 2024-09-16T01:39:45.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.730+0000 7fd871e20740 1 -- 172.21.15.139:0/1287446808 >> 172.21.15.139:0/1287446808 conn(0x5557fa5e1f60 msgr2=0x5557fa5e7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.730+0000 7fd871e20740 1 -- 172.21.15.139:0/1287446808 shutdown_connections 2024-09-16T01:39:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.730+0000 7fd871e20740 1 -- 172.21.15.139:0/1287446808 wait complete. 2024-09-16T01:39:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.730+0000 7fd871e20740 20 client.15072 trim_cache size 0 max 16384 2024-09-16T01:39:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.747+0000 7fd871e20740 1 Processor -- start 2024-09-16T01:39:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.747+0000 7fd871e20740 1 -- start start 2024-09-16T01:39:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd871e20740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5e59d0 0x5557fa600b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd871e20740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e62f0 0x5557fa602680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd871e20740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5f0bb0 0x5557fa5ea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd871e20740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5557fa6009c0 con 0x5557fa5e62f0 2024-09-16T01:39:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd871e20740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5557fa601090 con 0x5557fa5e59d0 2024-09-16T01:39:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd871e20740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5557fa601760 con 0x5557fa5f0bb0 2024-09-16T01:39:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd86d7e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e62f0 0x5557fa602680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd86d7e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e62f0 0x5557fa602680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43724/0 (socket says 172.21.15.139:43724) 2024-09-16T01:39:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd86d7e3640 1 -- 172.21.15.139:0/1886714342 learned_addr learned my addr 172.21.15.139:0/1886714342 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd86d7e3640 1 -- 172.21.15.139:0/1886714342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5f0bb0 msgr2=0x5557fa5ea950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd86d7e3640 1 --2- 172.21.15.139:0/1886714342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5f0bb0 0x5557fa5ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd86d7e3640 1 -- 172.21.15.139:0/1886714342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5e59d0 msgr2=0x5557fa600b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd86d7e3640 1 --2- 172.21.15.139:0/1886714342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5e59d0 0x5557fa600b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd86d7e3640 1 -- 172.21.15.139:0/1886714342 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557fa5e6120 con 0x5557fa5e62f0 2024-09-16T01:39:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd86d7e3640 1 --2- 172.21.15.139:0/1886714342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e62f0 0x5557fa602680 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd864005390 tx=0x7fd864005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd7431c1d916b63 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd852ffd640 1 -- 172.21.15.139:0/1886714342 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86404c040 con 0x5557fa5e62f0 2024-09-16T01:39:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd852ffd640 1 -- 172.21.15.139:0/1886714342 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd864005610 con 0x5557fa5e62f0 2024-09-16T01:39:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.748+0000 7fd852ffd640 1 -- 172.21.15.139:0/1886714342 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864040070 con 0x5557fa5e62f0 2024-09-16T01:39:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.749+0000 7fd871e20740 1 -- 172.21.15.139:0/1886714342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e62f0 msgr2=0x5557fa602680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.749+0000 7fd871e20740 1 --2- 172.21.15.139:0/1886714342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e62f0 0x5557fa602680 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd864005390 tx=0x7fd864005900 comp rx=0 tx=0).stop 2024-09-16T01:39:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.749+0000 7fd871e20740 1 -- 172.21.15.139:0/1886714342 shutdown_connections 2024-09-16T01:39:45.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.749+0000 7fd871e20740 1 --2- 172.21.15.139:0/1886714342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5f0bb0 0x5557fa5ea950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.749+0000 7fd871e20740 1 --2- 172.21.15.139:0/1886714342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e62f0 0x5557fa602680 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.749+0000 7fd871e20740 1 --2- 172.21.15.139:0/1886714342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5e59d0 0x5557fa600b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.749+0000 7fd871e20740 1 -- 172.21.15.139:0/1886714342 >> 172.21.15.139:0/1886714342 conn(0x5557fa5e1f60 msgr2=0x5557fa5dcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:45.771 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:39:45.771 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:39:45.771 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 4, 2024-09-16T01:39:45.771 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24433, 2024-09-16T01:39:45.771 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:39:45.772 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:39:45.772 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:39:45.772 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 66.202897626999999, 2024-09-16T01:39:45.772 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 28, 2024-09-16T01:39:45.772 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 85, 2024-09-16T01:39:45.772 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 83, 2024-09-16T01:39:45.772 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 132.41969137800001 2024-09-16T01:39:45.772 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:39:45.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.749+0000 7fd871e20740 1 -- 172.21.15.139:0/1886714342 shutdown_connections 2024-09-16T01:39:45.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.749+0000 7fd871e20740 1 -- 172.21.15.139:0/1886714342 wait complete. 2024-09-16T01:39:45.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.749+0000 7fd871e20740 1 -- start start 2024-09-16T01:39:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.749+0000 7fd871e20740 1 Processor -- start 2024-09-16T01:39:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd871e20740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e59d0 0x5557fa6087d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd871e20740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e62f0 0x5557fa608d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd871e20740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5f0bb0 0x5557fa603e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd871e20740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5557fa602500 con 0x5557fa5e59d0 2024-09-16T01:39:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd871e20740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5557fa601e30 con 0x5557fa5f0bb0 2024-09-16T01:39:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd871e20740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5557fa602bd0 con 0x5557fa5e62f0 2024-09-16T01:39:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd871e20740 10 client.0 _ll_register_callbacks cb 0x7fd86806b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:39:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd86cfe2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e59d0 0x5557fa6087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd86cfe2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e59d0 0x5557fa6087d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43730/0 (socket says 172.21.15.139:43730) 2024-09-16T01:39:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd86cfe2640 1 -- 172.21.15.139:0/4268473493 learned_addr learned my addr 172.21.15.139:0/4268473493 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd86cfe2640 1 -- 172.21.15.139:0/4268473493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e62f0 msgr2=0x5557fa608d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd86d7e3640 1 --2- 172.21.15.139:0/4268473493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e62f0 0x5557fa608d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd86cfe2640 1 --2- 172.21.15.139:0/4268473493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e62f0 0x5557fa608d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd86cfe2640 1 -- 172.21.15.139:0/4268473493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5f0bb0 msgr2=0x5557fa603e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd86cfe2640 1 --2- 172.21.15.139:0/4268473493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5f0bb0 0x5557fa603e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd86cfe2640 1 -- 172.21.15.139:0/4268473493 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557fa604500 con 0x5557fa5e59d0 2024-09-16T01:39:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd86cfe2640 1 --2- 172.21.15.139:0/4268473493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e59d0 0x5557fa6087d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd85800b400 tx=0x7fd858014d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a446ce92b82f638b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd850ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:39:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd850ff9640 1 -- 172.21.15.139:0/4268473493 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858004220 con 0x5557fa5e59d0 2024-09-16T01:39:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd850ff9640 1 -- 172.21.15.139:0/4268473493 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fd8580043c0 con 0x5557fa5e59d0 2024-09-16T01:39:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.750+0000 7fd871e20740 1 -- 172.21.15.139:0/4268473493 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd864005610 con 0x5557fa5e59d0 2024-09-16T01:39:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.751+0000 7fd850ff9640 1 -- 172.21.15.139:0/4268473493 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd858033050 con 0x5557fa5e59d0 2024-09-16T01:39:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.751+0000 7fd871e20740 10 client.15078 fetch_fsmap learned FSMap version 28 2024-09-16T01:39:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.751+0000 7fd871e20740 1 -- 172.21.15.139:0/4268473493 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557fa607f20 con 0x5557fa5e59d0 2024-09-16T01:39:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.751+0000 7fd850ff9640 1 -- 172.21.15.139:0/4268473493 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd864005610 con 0x5557fa5e59d0 2024-09-16T01:39:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.751+0000 7fd850ff9640 1 -- 172.21.15.139:0/4268473493 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fd85800f040 con 0x5557fa5e59d0 2024-09-16T01:39:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.751+0000 7fd871e20740 10 client.15078 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:39:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.751+0000 7fd871e20740 10 client.15078 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T01:39:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.751+0000 7fd871e20740 20 client.15078 populate_metadata read hostname 'smithi139' 2024-09-16T01:39:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.751+0000 7fd871e20740 1 --2- 172.21.15.139:0/4268473493 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x5557fa66ffb0 0x5557fa66ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.751+0000 7fd871e20740 4 client.15078 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:39:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.751+0000 7fd871e20740 1 -- 172.21.15.139:0/4268473493 --> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] -- command(tid 0: {"prefix": "status"}) -- 0x5557fa553b10 con 0x5557fa66ffb0 2024-09-16T01:39:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.751+0000 7fd86d7e3640 1 --2- 172.21.15.139:0/4268473493 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x5557fa66ffb0 0x5557fa66ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd86d7e3640 1 --2- 172.21.15.139:0/4268473493 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x5557fa66ffb0 0x5557fa66ecc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=df2c2a331c6002cc server_cookie=a4a96b2fcad50052 in_seq=0 out_seq=0 2024-09-16T01:39:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd850ff9640 10 client.15078 ms_handle_connect on v2:172.21.15.176:6834/3060423566 2024-09-16T01:39:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd850ff9640 1 -- 172.21.15.139:0/4268473493 <== mds.4 v2:172.21.15.176:6834/3060423566 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5557fa553b10 con 0x5557fa66ffb0 2024-09-16T01:39:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd850ff9640 10 client.15078 handle_command_reply: tid=0 2024-09-16T01:39:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd871e20740 1 client.15078 shutdown 2024-09-16T01:39:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd871e20740 1 -- 172.21.15.139:0/4268473493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e59d0 msgr2=0x5557fa6087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:45.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd871e20740 1 --2- 172.21.15.139:0/4268473493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e59d0 0x5557fa6087d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd85800b400 tx=0x7fd858014d00 comp rx=0 tx=0).stop 2024-09-16T01:39:45.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd871e20740 1 -- 172.21.15.139:0/4268473493 shutdown_connections 2024-09-16T01:39:45.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd871e20740 1 --2- 172.21.15.139:0/4268473493 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x5557fa66ffb0 0x5557fa66ecc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd871e20740 1 --2- 172.21.15.139:0/4268473493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fa5f0bb0 0x5557fa603e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd871e20740 1 --2- 172.21.15.139:0/4268473493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557fa5e62f0 0x5557fa608d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd871e20740 1 --2- 172.21.15.139:0/4268473493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5557fa5e59d0 0x5557fa6087d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd850ff9640 0 client.15078 ms_handle_reset on v2:172.21.15.176:6834/3060423566 2024-09-16T01:39:45.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd871e20740 1 -- 172.21.15.139:0/4268473493 >> 172.21.15.139:0/4268473493 conn(0x5557fa5e1f60 msgr2=0x5557fa5e07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:45.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd871e20740 1 -- 172.21.15.139:0/4268473493 shutdown_connections 2024-09-16T01:39:45.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd871e20740 1 -- 172.21.15.139:0/4268473493 wait complete. 2024-09-16T01:39:45.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.752+0000 7fd871e20740 20 client.15078 trim_cache size 0 max 16384 2024-09-16T01:39:45.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.753+0000 7fd86fa47640 1 -- 172.21.15.139:0/4100496570 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd8540a32d0 msgr2=0x7fd8540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:45.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.753+0000 7fd86fa47640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd8540a32d0 0x7fd8540a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd86009c0f0 tx=0x7fd860009270 comp rx=0 tx=0).stop 2024-09-16T01:39:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.753+0000 7fd86fa47640 1 -- 172.21.15.139:0/4100496570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd868105f40 msgr2=0x7fd86819e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.753+0000 7fd86fa47640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd868105f40 0x7fd86819e8d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd86000eaa0 tx=0x7fd86000ef60 comp rx=0 tx=0).stop 2024-09-16T01:39:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.753+0000 7fd86fa47640 1 -- 172.21.15.139:0/4100496570 shutdown_connections 2024-09-16T01:39:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.753+0000 7fd86fa47640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fd8540a32d0 0x7fd8540a5790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.753+0000 7fd86fa47640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd868105f40 0x7fd86819e8d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.753+0000 7fd86fa47640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd868101450 0x7fd8681a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.753+0000 7fd86fa47640 1 --2- 172.21.15.139:0/4100496570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd868100b30 0x7fd8681a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.753+0000 7fd86fa47640 1 -- 172.21.15.139:0/4100496570 >> 172.21.15.139:0/4100496570 conn(0x7fd8680fc7a0 msgr2=0x7fd86810f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.753+0000 7fd86fa47640 1 -- 172.21.15.139:0/4100496570 shutdown_connections 2024-09-16T01:39:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:45.753+0000 7fd86fa47640 1 -- 172.21.15.139:0/4100496570 wait complete. 2024-09-16T01:39:46.561 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-16T01:39:46.810 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:46.847 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:46 smithi164 ceph-mon[35121]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:39:46.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:46 smithi139 ceph-mon[31175]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:39:46.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:46 smithi176 ceph-mon[35324]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:39:47.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.305+0000 7f91fe373640 1 -- 172.21.15.139:0/3364468920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91f8102ef0 msgr2=0x7f91f810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:47.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.305+0000 7f91fe373640 1 --2- 172.21.15.139:0/3364468920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91f8102ef0 0x7f91f810da40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009ca0 tx=0x7f91f002a950 comp rx=0 tx=0).stop 2024-09-16T01:39:47.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.305+0000 7f91fe373640 1 -- 172.21.15.139:0/3364468920 shutdown_connections 2024-09-16T01:39:47.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.305+0000 7f91fe373640 1 --2- 172.21.15.139:0/3364468920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91f8102ef0 0x7f91f810da40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.305+0000 7f91fe373640 1 --2- 172.21.15.139:0/3364468920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91f8102570 0x7f91f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.305+0000 7f91fe373640 1 --2- 172.21.15.139:0/3364468920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91f8100e20 0x7f91f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.305+0000 7f91fe373640 1 -- 172.21.15.139:0/3364468920 >> 172.21.15.139:0/3364468920 conn(0x7f91f80fc7d0 msgr2=0x7f91f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:47.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.305+0000 7f91fe373640 1 -- 172.21.15.139:0/3364468920 shutdown_connections 2024-09-16T01:39:47.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.305+0000 7f91fe373640 1 -- 172.21.15.139:0/3364468920 wait complete. 2024-09-16T01:39:47.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.305+0000 7f91fe373640 1 Processor -- start 2024-09-16T01:39:47.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.305+0000 7f91fe373640 1 -- start start 2024-09-16T01:39:47.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91fe373640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91f8100e20 0x7f91f819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91fe373640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91f8102570 0x7f91f819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91fe373640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91f8102ef0 0x7f91f819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91fe373640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f91f81120a0 con 0x7f91f8100e20 2024-09-16T01:39:47.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91fe373640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f91f8111f20 con 0x7f91f8102570 2024-09-16T01:39:47.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91fe373640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f91f8112220 con 0x7f91f8102ef0 2024-09-16T01:39:47.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91f7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91f8100e20 0x7f91f819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91f7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91f8100e20 0x7f91f819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43752/0 (socket says 172.21.15.139:43752) 2024-09-16T01:39:47.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91f7fff640 1 -- 172.21.15.139:0/4221593746 learned_addr learned my addr 172.21.15.139:0/4221593746 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:47.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91fc910640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91f8102ef0 0x7f91f819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91f77fe640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91f8102570 0x7f91f819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91f7fff640 1 -- 172.21.15.139:0/4221593746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91f8102ef0 msgr2=0x7f91f819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:47.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91f7fff640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91f8102ef0 0x7f91f819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91f7fff640 1 -- 172.21.15.139:0/4221593746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91f8102570 msgr2=0x7f91f819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:47.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91f7fff640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91f8102570 0x7f91f819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91f7fff640 1 -- 172.21.15.139:0/4221593746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f819f5b0 con 0x7f91f8100e20 2024-09-16T01:39:47.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91f77fe640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91f8102570 0x7f91f819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:47.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91f7fff640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91f8100e20 0x7f91f819a600 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f91e800c920 tx=0x7f91e800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:47.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91f57fa640 1 -- 172.21.15.139:0/4221593746 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8007c70 con 0x7f91f8100e20 2024-09-16T01:39:47.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91f57fa640 1 -- 172.21.15.139:0/4221593746 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91e8007e10 con 0x7f91f8100e20 2024-09-16T01:39:47.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91fe373640 1 -- 172.21.15.139:0/4221593746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f819f8a0 con 0x7f91f8100e20 2024-09-16T01:39:47.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91f57fa640 1 -- 172.21.15.139:0/4221593746 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8018890 con 0x7f91f8100e20 2024-09-16T01:39:47.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.306+0000 7f91fe373640 1 -- 172.21.15.139:0/4221593746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f81a7180 con 0x7f91f8100e20 2024-09-16T01:39:47.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.309+0000 7f91f57fa640 1 -- 172.21.15.139:0/4221593746 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f91e8018a70 con 0x7f91f8100e20 2024-09-16T01:39:47.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.309+0000 7f920074c740 1 Processor -- start 2024-09-16T01:39:47.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.309+0000 7f920074c740 1 -- start start 2024-09-16T01:39:47.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.309+0000 7f91f57fa640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f91e00a32d0 0x7f91e00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.309+0000 7f920074c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694c9d0 0x5614e694cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.309+0000 7f920074c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694d2f0 0x5614e6957670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.309+0000 7f920074c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e6957bb0 0x5614e6959fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.309+0000 7f920074c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5614e68b3090 con 0x5614e6957bb0 2024-09-16T01:39:47.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.309+0000 7f91fc910640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f91e00a32d0 0x7f91e00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.309+0000 7f920074c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5614e685aa70 con 0x5614e694d2f0 2024-09-16T01:39:47.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.309+0000 7f920074c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5614e68af050 con 0x5614e694c9d0 2024-09-16T01:39:47.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.309+0000 7f91f57fa640 1 -- 172.21.15.139:0/4221593746 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f91e80d00d0 con 0x7f91f8100e20 2024-09-16T01:39:47.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91fc910640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e6957bb0 0x5614e6959fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91fc910640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e6957bb0 0x5614e6959fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43762/0 (socket says 172.21.15.139:43762) 2024-09-16T01:39:47.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91fc910640 1 -- 172.21.15.139:0/4116572719 learned_addr learned my addr 172.21.15.139:0/4116572719 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:47.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91f7fff640 1 --2- 172.21.15.139:0/4116572719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694c9d0 0x5614e694cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91f77fe640 1 --2- 172.21.15.139:0/4116572719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694d2f0 0x5614e6957670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91f7fff640 1 -- 172.21.15.139:0/4116572719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694d2f0 msgr2=0x5614e6957670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:47.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91f7fff640 1 --2- 172.21.15.139:0/4116572719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694d2f0 0x5614e6957670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91f7fff640 1 -- 172.21.15.139:0/4116572719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e6957bb0 msgr2=0x5614e6959fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:47.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91f7fff640 1 --2- 172.21.15.139:0/4116572719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e6957bb0 0x5614e6959fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91f7fff640 1 -- 172.21.15.139:0/4116572719 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e6867130 con 0x5614e694c9d0 2024-09-16T01:39:47.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91fc910640 1 --2- 172.21.15.139:0/4116572719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e6957bb0 0x5614e6959fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:39:47.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91fc910640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f91e00a32d0 0x7f91e00a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f91f0030040 tx=0x7f91f0009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:47.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91f7fff640 1 --2- 172.21.15.139:0/4116572719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694c9d0 0x5614e694cdb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f91e80cd650 tx=0x7f91e800de20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebcf5b58c5b27519 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:47.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91de7fc640 1 -- 172.21.15.139:0/4116572719 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e80a0510 con 0x5614e694c9d0 2024-09-16T01:39:47.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91de7fc640 1 -- 172.21.15.139:0/4116572719 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91e80a4dd0 con 0x5614e694c9d0 2024-09-16T01:39:47.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.310+0000 7f91de7fc640 1 -- 172.21.15.139:0/4116572719 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e80cd8a0 con 0x5614e694c9d0 2024-09-16T01:39:47.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.311+0000 7f920074c740 1 -- 172.21.15.139:0/4116572719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694c9d0 msgr2=0x5614e694cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:47.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.311+0000 7f920074c740 1 --2- 172.21.15.139:0/4116572719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694c9d0 0x5614e694cdb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f91e80cd650 tx=0x7f91e800de20 comp rx=0 tx=0).stop 2024-09-16T01:39:47.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.311+0000 7f920074c740 1 -- 172.21.15.139:0/4116572719 shutdown_connections 2024-09-16T01:39:47.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.311+0000 7f920074c740 1 --2- 172.21.15.139:0/4116572719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e6957bb0 0x5614e6959fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.311+0000 7f920074c740 1 --2- 172.21.15.139:0/4116572719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694d2f0 0x5614e6957670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.311+0000 7f920074c740 1 --2- 172.21.15.139:0/4116572719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694c9d0 0x5614e694cdb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.311+0000 7f920074c740 1 -- 172.21.15.139:0/4116572719 >> 172.21.15.139:0/4116572719 conn(0x5614e6948f60 msgr2=0x5614e6949340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:47.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.311+0000 7f920074c740 1 -- 172.21.15.139:0/4116572719 shutdown_connections 2024-09-16T01:39:47.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.311+0000 7f920074c740 1 -- 172.21.15.139:0/4116572719 wait complete. 2024-09-16T01:39:47.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.311+0000 7f920074c740 1 -- start start 2024-09-16T01:39:47.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f920074c740 1 Processor -- start 2024-09-16T01:39:47.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f920074c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e694c9d0 0x5614e69573d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f920074c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694d2f0 0x5614e6954720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f920074c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e6957bb0 0x5614e6954c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f920074c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5614e695a780 con 0x5614e694c9d0 2024-09-16T01:39:47.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f920074c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5614e686df20 con 0x5614e694d2f0 2024-09-16T01:39:47.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f920074c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5614e695a900 con 0x5614e6957bb0 2024-09-16T01:39:47.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f920074c740 10 client.0 _ll_register_callbacks cb 0x5614e6837030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:39:47.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f91f7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e694c9d0 0x5614e69573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f91f7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e694c9d0 0x5614e69573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43774/0 (socket says 172.21.15.139:43774) 2024-09-16T01:39:47.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f91f7fff640 1 -- 172.21.15.139:0/2888793179 learned_addr learned my addr 172.21.15.139:0/2888793179 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:47.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f91fc910640 1 --2- 172.21.15.139:0/2888793179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e6957bb0 0x5614e6954c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f91f77fe640 1 --2- 172.21.15.139:0/2888793179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694d2f0 0x5614e6954720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f91f7fff640 1 -- 172.21.15.139:0/2888793179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e6957bb0 msgr2=0x5614e6954c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:47.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f91f7fff640 1 --2- 172.21.15.139:0/2888793179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e6957bb0 0x5614e6954c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f91f7fff640 1 -- 172.21.15.139:0/2888793179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694d2f0 msgr2=0x5614e6954720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:47.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f91f7fff640 1 --2- 172.21.15.139:0/2888793179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694d2f0 0x5614e6954720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f91f7fff640 1 -- 172.21.15.139:0/2888793179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e66242b0 con 0x5614e694c9d0 2024-09-16T01:39:47.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.312+0000 7f91f77fe640 1 --2- 172.21.15.139:0/2888793179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694d2f0 0x5614e6954720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:47.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f91f7fff640 1 --2- 172.21.15.139:0/2888793179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e694c9d0 0x5614e69573d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f91e800c8f0 tx=0x7f91e800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda36a2b5094b8f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:47.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f91d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:39:47.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f91d77fe640 1 -- 172.21.15.139:0/2888793179 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e80512b0 con 0x5614e694c9d0 2024-09-16T01:39:47.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f91d77fe640 1 -- 172.21.15.139:0/2888793179 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f91e80bc0b0 con 0x5614e694c9d0 2024-09-16T01:39:47.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f920074c740 1 -- 172.21.15.139:0/2888793179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91e80a4dd0 con 0x5614e694c9d0 2024-09-16T01:39:47.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f91d77fe640 1 -- 172.21.15.139:0/2888793179 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91e80b5150 con 0x5614e694c9d0 2024-09-16T01:39:47.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f920074c740 10 client.15090 fetch_fsmap learned FSMap version 28 2024-09-16T01:39:47.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f920074c740 1 -- 172.21.15.139:0/2888793179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5614e6966030 con 0x5614e694c9d0 2024-09-16T01:39:47.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f91d77fe640 1 -- 172.21.15.139:0/2888793179 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91e80a4dd0 con 0x5614e694c9d0 2024-09-16T01:39:47.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f91d77fe640 1 -- 172.21.15.139:0/2888793179 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f91e80565b0 con 0x5614e694c9d0 2024-09-16T01:39:47.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f920074c740 10 client.15090 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:39:47.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f920074c740 10 client.15090 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T01:39:47.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f920074c740 20 client.15090 populate_metadata read hostname 'smithi139' 2024-09-16T01:39:47.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f920074c740 1 --2- 172.21.15.139:0/2888793179 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5614e6968120 0x5614e6967380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f920074c740 4 client.15090 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:39:47.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f920074c740 1 -- 172.21.15.139:0/2888793179 --> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614e68c0600 con 0x5614e6968120 2024-09-16T01:39:47.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.313+0000 7f91f77fe640 1 --2- 172.21.15.139:0/2888793179 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5614e6968120 0x5614e6967380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.314+0000 7f91d77fe640 10 client.15090 ms_handle_connect on v2:172.21.15.176:6836/1942543693 2024-09-16T01:39:47.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.314+0000 7f91f77fe640 1 --2- 172.21.15.139:0/2888793179 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5614e6968120 0x5614e6967380 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d94b708cea6a1f23 server_cookie=f00c3acba5a2e9b0 in_seq=0 out_seq=0 2024-09-16T01:39:47.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f91d77fe640 1 -- 172.21.15.139:0/2888793179 <== mds.3 v2:172.21.15.176:6836/1942543693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614e68c0600 con 0x5614e6968120 2024-09-16T01:39:47.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f91d77fe640 10 client.15090 handle_command_reply: tid=0 2024-09-16T01:39:47.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f920074c740 1 client.15090 shutdown 2024-09-16T01:39:47.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f920074c740 1 -- 172.21.15.139:0/2888793179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e694c9d0 msgr2=0x5614e69573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:47.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f920074c740 1 --2- 172.21.15.139:0/2888793179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e694c9d0 0x5614e69573d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f91e800c8f0 tx=0x7f91e800cf80 comp rx=0 tx=0).stop 2024-09-16T01:39:47.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f920074c740 1 -- 172.21.15.139:0/2888793179 shutdown_connections 2024-09-16T01:39:47.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f920074c740 1 --2- 172.21.15.139:0/2888793179 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5614e6968120 0x5614e6967380 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f920074c740 1 --2- 172.21.15.139:0/2888793179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e6957bb0 0x5614e6954c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f91d77fe640 0 client.15090 ms_handle_reset on v2:172.21.15.176:6836/1942543693 2024-09-16T01:39:47.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f920074c740 1 --2- 172.21.15.139:0/2888793179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694d2f0 0x5614e6954720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f920074c740 1 --2- 172.21.15.139:0/2888793179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e694c9d0 0x5614e69573d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f920074c740 1 -- 172.21.15.139:0/2888793179 >> 172.21.15.139:0/2888793179 conn(0x5614e6948f60 msgr2=0x5614e694ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:47.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f920074c740 1 -- 172.21.15.139:0/2888793179 shutdown_connections 2024-09-16T01:39:47.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f920074c740 1 -- 172.21.15.139:0/2888793179 wait complete. 2024-09-16T01:39:47.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.315+0000 7f920074c740 20 client.15090 trim_cache size 0 max 16384 2024-09-16T01:39:47.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f920074c740 1 Processor -- start 2024-09-16T01:39:47.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f920074c740 1 -- start start 2024-09-16T01:39:47.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f920074c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694c9d0 0x5614e6967a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f920074c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694d2f0 0x5614e6968ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f920074c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e6957bb0 0x5614e6952ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f920074c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5614e69678d0 con 0x5614e6957bb0 2024-09-16T01:39:47.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f920074c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5614e6967fa0 con 0x5614e694c9d0 2024-09-16T01:39:47.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f920074c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5614e6968670 con 0x5614e694d2f0 2024-09-16T01:39:47.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f91fc910640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e6957bb0 0x5614e6952ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f91fc910640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e6957bb0 0x5614e6952ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43790/0 (socket says 172.21.15.139:43790) 2024-09-16T01:39:47.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f91fc910640 1 -- 172.21.15.139:0/3711424519 learned_addr learned my addr 172.21.15.139:0/3711424519 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:47.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f91f7fff640 1 --2- 172.21.15.139:0/3711424519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694c9d0 0x5614e6967a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f91fc910640 1 -- 172.21.15.139:0/3711424519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694d2f0 msgr2=0x5614e6968ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:47.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f91fc910640 1 --2- 172.21.15.139:0/3711424519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694d2f0 0x5614e6968ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f91fc910640 1 -- 172.21.15.139:0/3711424519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694c9d0 msgr2=0x5614e6967a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:47.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f91fc910640 1 --2- 172.21.15.139:0/3711424519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694c9d0 0x5614e6967a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f91fc910640 1 -- 172.21.15.139:0/3711424519 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e695a890 con 0x5614e6957bb0 2024-09-16T01:39:47.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f91f7fff640 1 --2- 172.21.15.139:0/3711424519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694c9d0 0x5614e6967a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:47.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f91fc910640 1 --2- 172.21.15.139:0/3711424519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e6957bb0 0x5614e6952ff0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f91f002ae70 tx=0x7f91f004a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6504e652b7c80b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:47.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.333+0000 7f91ddffb640 1 -- 172.21.15.139:0/3711424519 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0034040 con 0x5614e6957bb0 2024-09-16T01:39:47.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.334+0000 7f91ddffb640 1 -- 172.21.15.139:0/3711424519 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f0003e50 con 0x5614e6957bb0 2024-09-16T01:39:47.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.334+0000 7f91ddffb640 1 -- 172.21.15.139:0/3711424519 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0004860 con 0x5614e6957bb0 2024-09-16T01:39:47.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.334+0000 7f920074c740 1 -- 172.21.15.139:0/3711424519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e6957bb0 msgr2=0x5614e6952ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:47.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.334+0000 7f920074c740 1 --2- 172.21.15.139:0/3711424519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e6957bb0 0x5614e6952ff0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f91f002ae70 tx=0x7f91f004a000 comp rx=0 tx=0).stop 2024-09-16T01:39:47.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.334+0000 7f920074c740 1 -- 172.21.15.139:0/3711424519 shutdown_connections 2024-09-16T01:39:47.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.334+0000 7f920074c740 1 --2- 172.21.15.139:0/3711424519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e6957bb0 0x5614e6952ff0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.334+0000 7f920074c740 1 --2- 172.21.15.139:0/3711424519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694d2f0 0x5614e6968ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.334+0000 7f920074c740 1 --2- 172.21.15.139:0/3711424519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e694c9d0 0x5614e6967a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.334+0000 7f920074c740 1 -- 172.21.15.139:0/3711424519 >> 172.21.15.139:0/3711424519 conn(0x5614e6948f60 msgr2=0x5614e6953fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:47.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.334+0000 7f920074c740 1 -- 172.21.15.139:0/3711424519 shutdown_connections 2024-09-16T01:39:47.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.334+0000 7f920074c740 1 -- 172.21.15.139:0/3711424519 wait complete. 2024-09-16T01:39:47.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.335+0000 7f920074c740 1 -- start start 2024-09-16T01:39:47.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.335+0000 7f920074c740 1 Processor -- start 2024-09-16T01:39:47.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.335+0000 7f920074c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e694c9d0 0x5614e696cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.335+0000 7f920074c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694d2f0 0x5614e696d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.335+0000 7f920074c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e6957bb0 0x5614e69d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.335+0000 7f920074c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5614e6969410 con 0x5614e694c9d0 2024-09-16T01:39:47.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.335+0000 7f920074c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5614e6968d40 con 0x5614e6957bb0 2024-09-16T01:39:47.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.335+0000 7f920074c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5614e6969ae0 con 0x5614e694d2f0 2024-09-16T01:39:47.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.335+0000 7f920074c740 10 client.0 _ll_register_callbacks cb 0x7f91e809bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:39:47.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.335+0000 7f91f7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e694c9d0 0x5614e696cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.335+0000 7f91f7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e694c9d0 0x5614e696cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43802/0 (socket says 172.21.15.139:43802) 2024-09-16T01:39:47.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.335+0000 7f91f7fff640 1 -- 172.21.15.139:0/2560453275 learned_addr learned my addr 172.21.15.139:0/2560453275 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:47.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.335+0000 7f91fc910640 1 --2- 172.21.15.139:0/2560453275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e6957bb0 0x5614e69d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.335+0000 7f91f77fe640 1 --2- 172.21.15.139:0/2560453275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694d2f0 0x5614e696d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f91f7fff640 1 -- 172.21.15.139:0/2560453275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694d2f0 msgr2=0x5614e696d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:47.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f91f7fff640 1 --2- 172.21.15.139:0/2560453275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694d2f0 0x5614e696d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f91f7fff640 1 -- 172.21.15.139:0/2560453275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e6957bb0 msgr2=0x5614e69d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:47.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f91f7fff640 1 --2- 172.21.15.139:0/2560453275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e6957bb0 0x5614e69d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f91f7fff640 1 -- 172.21.15.139:0/2560453275 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e696d9b0 con 0x5614e694c9d0 2024-09-16T01:39:47.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f91f7fff640 1 --2- 172.21.15.139:0/2560453275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e694c9d0 0x5614e696cb80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f91e80a48a0 tx=0x7f91e8098940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d472956e0bade6d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:47.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f91d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:39:47.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f91d77fe640 1 -- 172.21.15.139:0/2560453275 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e80a4940 con 0x5614e694c9d0 2024-09-16T01:39:47.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f91d77fe640 1 -- 172.21.15.139:0/2560453275 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f91e809b7c0 con 0x5614e694c9d0 2024-09-16T01:39:47.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f920074c740 1 -- 172.21.15.139:0/2560453275 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91f0003e50 con 0x5614e694c9d0 2024-09-16T01:39:47.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f91d77fe640 1 -- 172.21.15.139:0/2560453275 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91e8098400 con 0x5614e694c9d0 2024-09-16T01:39:47.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f920074c740 10 client.15096 fetch_fsmap learned FSMap version 28 2024-09-16T01:39:47.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f920074c740 1 -- 172.21.15.139:0/2560453275 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5614e69d1e00 con 0x5614e694c9d0 2024-09-16T01:39:47.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f91d77fe640 1 -- 172.21.15.139:0/2560453275 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f0003e50 con 0x5614e694c9d0 2024-09-16T01:39:47.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f91d77fe640 1 -- 172.21.15.139:0/2560453275 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f91e809b020 con 0x5614e694c9d0 2024-09-16T01:39:47.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f920074c740 10 client.15096 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:39:47.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f920074c740 10 client.15096 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T01:39:47.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f920074c740 20 client.15096 populate_metadata read hostname 'smithi139' 2024-09-16T01:39:47.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.336+0000 7f920074c740 1 --2- 172.21.15.139:0/2560453275 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5614e6968120 0x5614e69d5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:47.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.337+0000 7f920074c740 4 client.15096 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:39:47.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.337+0000 7f920074c740 1 -- 172.21.15.139:0/2560453275 --> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] -- command(tid 0: {"prefix": "status"}) -- 0x5614e68bab10 con 0x5614e6968120 2024-09-16T01:39:47.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.337+0000 7f91f77fe640 1 --2- 172.21.15.139:0/2560453275 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5614e6968120 0x5614e69d5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:47.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.337+0000 7f91d77fe640 10 client.15096 ms_handle_connect on v2:172.21.15.176:6836/1942543693 2024-09-16T01:39:47.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.337+0000 7f91f77fe640 1 --2- 172.21.15.139:0/2560453275 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5614e6968120 0x5614e69d5590 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d015b681c2aead83 server_cookie=92aae8e5da976679 in_seq=0 out_seq=0 2024-09-16T01:39:47.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.337+0000 7f91d77fe640 1 -- 172.21.15.139:0/2560453275 <== mds.3 v2:172.21.15.176:6836/1942543693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5614e68bab10 con 0x5614e6968120 2024-09-16T01:39:47.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.337+0000 7f91d77fe640 10 client.15096 handle_command_reply: tid=0 2024-09-16T01:39:47.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.337+0000 7f920074c740 1 client.15096 shutdown 2024-09-16T01:39:47.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f920074c740 1 -- 172.21.15.139:0/2560453275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e694c9d0 msgr2=0x5614e696cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:47.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f920074c740 1 --2- 172.21.15.139:0/2560453275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e694c9d0 0x5614e696cb80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f91e80a48a0 tx=0x7f91e8098940 comp rx=0 tx=0).stop 2024-09-16T01:39:47.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f920074c740 1 -- 172.21.15.139:0/2560453275 shutdown_connections 2024-09-16T01:39:47.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f920074c740 1 --2- 172.21.15.139:0/2560453275 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5614e6968120 0x5614e69d5590 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f91d77fe640 0 client.15096 ms_handle_reset on v2:172.21.15.176:6836/1942543693 2024-09-16T01:39:47.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f920074c740 1 --2- 172.21.15.139:0/2560453275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614e6957bb0 0x5614e69d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f920074c740 1 --2- 172.21.15.139:0/2560453275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614e694d2f0 0x5614e696d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f920074c740 1 --2- 172.21.15.139:0/2560453275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5614e694c9d0 0x5614e696cb80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f920074c740 1 -- 172.21.15.139:0/2560453275 >> 172.21.15.139:0/2560453275 conn(0x5614e6948f60 msgr2=0x5614e6952bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:47.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f920074c740 1 -- 172.21.15.139:0/2560453275 shutdown_connections 2024-09-16T01:39:47.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f920074c740 1 -- 172.21.15.139:0/2560453275 wait complete. 2024-09-16T01:39:47.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f920074c740 20 client.15096 trim_cache size 0 max 16384 2024-09-16T01:39:47.345 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:39:47.345 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:39:47.345 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 3, 2024-09-16T01:39:47.345 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24442, 2024-09-16T01:39:47.345 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:39:47.345 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:39:47.346 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:39:47.346 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 68.800595767999994, 2024-09-16T01:39:47.346 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 28, 2024-09-16T01:39:47.346 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 85, 2024-09-16T01:39:47.346 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 83, 2024-09-16T01:39:47.346 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 131.57514892699999 2024-09-16T01:39:47.346 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:39:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f91fe373640 1 -- 172.21.15.139:0/4221593746 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f91e00a32d0 msgr2=0x7f91e00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f91fe373640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f91e00a32d0 0x7f91e00a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f91f0030040 tx=0x7f91f0009800 comp rx=0 tx=0).stop 2024-09-16T01:39:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f91fe373640 1 -- 172.21.15.139:0/4221593746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91f8100e20 msgr2=0x7f91f819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f91fe373640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91f8100e20 0x7f91f819a600 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f91e800c920 tx=0x7f91e800cde0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f91fe373640 1 -- 172.21.15.139:0/4221593746 shutdown_connections 2024-09-16T01:39:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f91fe373640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f91e00a32d0 0x7f91e00a5790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f91fe373640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91f8102ef0 0x7f91f819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f91fe373640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91f8102570 0x7f91f819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f91fe373640 1 --2- 172.21.15.139:0/4221593746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91f8100e20 0x7f91f819a600 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f91fe373640 1 -- 172.21.15.139:0/4221593746 >> 172.21.15.139:0/4221593746 conn(0x7f91f80fc7d0 msgr2=0x7f91f8106b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.338+0000 7f91fe373640 1 -- 172.21.15.139:0/4221593746 shutdown_connections 2024-09-16T01:39:47.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:47.339+0000 7f91fe373640 1 -- 172.21.15.139:0/4221593746 wait complete. 2024-09-16T01:39:48.246 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T01:39:48.496 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:48.911 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:48 smithi139 ceph-mon[31175]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:39:48.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:48 smithi176 ceph-mon[35324]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:39:49.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.989+0000 7f578c14f640 1 -- 172.21.15.139:0/898016524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5784100b30 msgr2=0x7f5784100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:49.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.989+0000 7f578c14f640 1 --2- 172.21.15.139:0/898016524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5784100b30 0x7f5784100f10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5780009900 tx=0x7f578002a470 comp rx=0 tx=0).stop 2024-09-16T01:39:49.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.989+0000 7f578c14f640 1 -- 172.21.15.139:0/898016524 shutdown_connections 2024-09-16T01:39:49.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.989+0000 7f578c14f640 1 --2- 172.21.15.139:0/898016524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5784105f40 0x7f57841063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.989+0000 7f578c14f640 1 --2- 172.21.15.139:0/898016524 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5784101450 0x7f5784105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.989+0000 7f578c14f640 1 --2- 172.21.15.139:0/898016524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5784100b30 0x7f5784100f10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.989+0000 7f578c14f640 1 -- 172.21.15.139:0/898016524 >> 172.21.15.139:0/898016524 conn(0x7f57840fc7a0 msgr2=0x7f57840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:49.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.989+0000 7f578c14f640 1 -- 172.21.15.139:0/898016524 shutdown_connections 2024-09-16T01:39:49.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.989+0000 7f578c14f640 1 -- 172.21.15.139:0/898016524 wait complete. 2024-09-16T01:39:49.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f578c14f640 1 Processor -- start 2024-09-16T01:39:49.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f578c14f640 1 -- start start 2024-09-16T01:39:49.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f578c14f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5784100b30 0x7f57841a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f578c14f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5784101450 0x7f57841a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f578c14f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5784105f40 0x7f578419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f578c14f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f57841185c0 con 0x7f5784100b30 2024-09-16T01:39:49.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f578c14f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5784118440 con 0x7f5784105f40 2024-09-16T01:39:49.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f578c14f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5784118740 con 0x7f5784101450 2024-09-16T01:39:49.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f5789eeb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5784100b30 0x7f57841a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f5789eeb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5784100b30 0x7f57841a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43806/0 (socket says 172.21.15.139:43806) 2024-09-16T01:39:49.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f5789eeb640 1 -- 172.21.15.139:0/280492720 learned_addr learned my addr 172.21.15.139:0/280492720 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:49.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f578a6ec640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5784105f40 0x7f578419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f57896ea640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5784101450 0x7f57841a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f5789eeb640 1 -- 172.21.15.139:0/280492720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5784101450 msgr2=0x7f57841a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:49.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f5789eeb640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5784101450 0x7f57841a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f5789eeb640 1 -- 172.21.15.139:0/280492720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5784105f40 msgr2=0x7f578419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:49.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f5789eeb640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5784105f40 0x7f578419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f5789eeb640 1 -- 172.21.15.139:0/280492720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578419efe0 con 0x7f5784100b30 2024-09-16T01:39:49.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.990+0000 7f57896ea640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5784101450 0x7f57841a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:49.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.992+0000 7f5789eeb640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5784100b30 0x7f57841a46f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5780005ab0 tx=0x7f57800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:49.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.992+0000 7f577affd640 1 -- 172.21.15.139:0/280492720 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578003f030 con 0x7f5784100b30 2024-09-16T01:39:49.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.992+0000 7f578c14f640 1 -- 172.21.15.139:0/280492720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578419f270 con 0x7f5784100b30 2024-09-16T01:39:49.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.992+0000 7f577affd640 1 -- 172.21.15.139:0/280492720 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5780003ce0 con 0x7f5784100b30 2024-09-16T01:39:49.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.992+0000 7f577affd640 1 -- 172.21.15.139:0/280492720 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780035070 con 0x7f5784100b30 2024-09-16T01:39:49.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.992+0000 7f578c14f640 1 -- 172.21.15.139:0/280492720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57841ab5c0 con 0x7f5784100b30 2024-09-16T01:39:49.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.994+0000 7f577affd640 1 -- 172.21.15.139:0/280492720 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5780042030 con 0x7f5784100b30 2024-09-16T01:39:49.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.995+0000 7f578e528740 1 Processor -- start 2024-09-16T01:39:49.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.995+0000 7f578e528740 1 -- start start 2024-09-16T01:39:49.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.995+0000 7f578e528740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade32db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.995+0000 7f578e528740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade332f0 0x5621ade3d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.995+0000 7f578e528740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade3dbb0 0x5621ade3ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.995+0000 7f578e528740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5621add99090 con 0x5621ade329d0 2024-09-16T01:39:49.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.995+0000 7f578e528740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5621add40a70 con 0x5621ade3dbb0 2024-09-16T01:39:49.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.995+0000 7f578e528740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5621add95050 con 0x5621ade332f0 2024-09-16T01:39:49.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.995+0000 7f578a6ec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade32db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.995+0000 7f578a6ec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade32db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43822/0 (socket says 172.21.15.139:43822) 2024-09-16T01:39:49.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.995+0000 7f578a6ec640 1 -- 172.21.15.139:0/463590888 learned_addr learned my addr 172.21.15.139:0/463590888 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:49.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.995+0000 7f577affd640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f57700a32d0 0x7f57700a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.995+0000 7f57896ea640 1 --2- 172.21.15.139:0/463590888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade3dbb0 0x5621ade3ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.996+0000 7f5789eeb640 1 --2- 172.21.15.139:0/463590888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade332f0 0x5621ade3d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.996+0000 7f578a6ec640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f57700a32d0 0x7f57700a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.996+0000 7f578a6ec640 1 -- 172.21.15.139:0/463590888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade332f0 msgr2=0x5621ade3d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:49.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.996+0000 7f578a6ec640 1 --2- 172.21.15.139:0/463590888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade332f0 0x5621ade3d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.996+0000 7f578a6ec640 1 -- 172.21.15.139:0/463590888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade3dbb0 msgr2=0x5621ade3ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:49.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.996+0000 7f578a6ec640 1 --2- 172.21.15.139:0/463590888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade3dbb0 0x5621ade3ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.996+0000 7f577affd640 1 -- 172.21.15.139:0/280492720 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f57800f07c0 con 0x7f5784100b30 2024-09-16T01:39:49.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.996+0000 7f578a6ec640 1 -- 172.21.15.139:0/463590888 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621add4d130 con 0x5621ade329d0 2024-09-16T01:39:49.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.996+0000 7f57896ea640 1 --2- 172.21.15.139:0/463590888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade3dbb0 0x5621ade3ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:49.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.996+0000 7f578a6ec640 1 --2- 172.21.15.139:0/463590888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade32db0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f577c013a50 tx=0x7f577c013f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc4d4b187bd041b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:49.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.996+0000 7f576ffff640 1 -- 172.21.15.139:0/463590888 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c013050 con 0x5621ade329d0 2024-09-16T01:39:49.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.996+0000 7f576ffff640 1 -- 172.21.15.139:0/463590888 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f577c022070 con 0x5621ade329d0 2024-09-16T01:39:49.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.996+0000 7f576ffff640 1 -- 172.21.15.139:0/463590888 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c012650 con 0x5621ade329d0 2024-09-16T01:39:49.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.997+0000 7f578e528740 1 -- 172.21.15.139:0/463590888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 msgr2=0x5621ade32db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:49.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.997+0000 7f578e528740 1 --2- 172.21.15.139:0/463590888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade32db0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f577c013a50 tx=0x7f577c013f10 comp rx=0 tx=0).stop 2024-09-16T01:39:49.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.997+0000 7f578a6ec640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f57700a32d0 0x7f57700a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f577c021000 tx=0x7f577c01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:49.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.997+0000 7f578e528740 1 -- 172.21.15.139:0/463590888 shutdown_connections 2024-09-16T01:39:49.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.997+0000 7f578e528740 1 --2- 172.21.15.139:0/463590888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade3dbb0 0x5621ade3ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.997+0000 7f578e528740 1 --2- 172.21.15.139:0/463590888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade332f0 0x5621ade3d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.997+0000 7f578e528740 1 --2- 172.21.15.139:0/463590888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade32db0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.997+0000 7f578e528740 1 -- 172.21.15.139:0/463590888 >> 172.21.15.139:0/463590888 conn(0x5621ade2ef60 msgr2=0x5621ade2f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:49.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.997+0000 7f578e528740 1 -- 172.21.15.139:0/463590888 shutdown_connections 2024-09-16T01:39:49.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.997+0000 7f578e528740 1 -- 172.21.15.139:0/463590888 wait complete. 2024-09-16T01:39:49.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.997+0000 7f578e528740 1 -- start start 2024-09-16T01:39:49.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.997+0000 7f578e528740 1 Processor -- start 2024-09-16T01:39:49.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f578e528740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade3d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f578e528740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade332f0 0x5621ade3a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f578e528740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade3dbb0 0x5621ade3ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f578e528740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5621ade40780 con 0x5621ade329d0 2024-09-16T01:39:49.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f578e528740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5621add53f20 con 0x5621ade3dbb0 2024-09-16T01:39:49.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f578e528740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5621ade40900 con 0x5621ade332f0 2024-09-16T01:39:49.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f578e528740 10 client.0 _ll_register_callbacks cb 0x5621add1d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:39:49.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f5789eeb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade3d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f5789eeb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade3d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43836/0 (socket says 172.21.15.139:43836) 2024-09-16T01:39:49.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f5789eeb640 1 -- 172.21.15.139:0/1639045996 learned_addr learned my addr 172.21.15.139:0/1639045996 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:49.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f57896ea640 1 --2- 172.21.15.139:0/1639045996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade332f0 0x5621ade3a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f578a6ec640 1 --2- 172.21.15.139:0/1639045996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade3dbb0 0x5621ade3ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f5789eeb640 1 -- 172.21.15.139:0/1639045996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade332f0 msgr2=0x5621ade3a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:49.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f5789eeb640 1 --2- 172.21.15.139:0/1639045996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade332f0 0x5621ade3a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f5789eeb640 1 -- 172.21.15.139:0/1639045996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade3dbb0 msgr2=0x5621ade3ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:49.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f5789eeb640 1 --2- 172.21.15.139:0/1639045996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade3dbb0 0x5621ade3ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f5789eeb640 1 -- 172.21.15.139:0/1639045996 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621adb0a2b0 con 0x5621ade329d0 2024-09-16T01:39:49.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f578a6ec640 1 --2- 172.21.15.139:0/1639045996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade3dbb0 0x5621ade3ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:49.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f5789eeb640 1 --2- 172.21.15.139:0/1639045996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade3d490 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f57800edd70 tx=0x7f578002a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cefa5a41f0a8564 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:49.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.998+0000 7f576d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:39:49.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f576d7fa640 1 -- 172.21.15.139:0/1639045996 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780049040 con 0x5621ade329d0 2024-09-16T01:39:49.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f576d7fa640 1 -- 172.21.15.139:0/1639045996 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f57800edf90 con 0x5621ade329d0 2024-09-16T01:39:49.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f578e528740 1 -- 172.21.15.139:0/1639045996 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f577c022070 con 0x5621ade329d0 2024-09-16T01:39:49.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f576d7fa640 1 -- 172.21.15.139:0/1639045996 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f57800bcf50 con 0x5621ade329d0 2024-09-16T01:39:49.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f578e528740 10 client.15108 fetch_fsmap learned FSMap version 28 2024-09-16T01:39:49.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f578e528740 1 -- 172.21.15.139:0/1639045996 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5621ade4c0f0 con 0x5621ade329d0 2024-09-16T01:39:49.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f576d7fa640 1 -- 172.21.15.139:0/1639045996 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f577c022070 con 0x5621ade329d0 2024-09-16T01:39:49.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f576d7fa640 1 -- 172.21.15.139:0/1639045996 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f578003b460 con 0x5621ade329d0 2024-09-16T01:39:49.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f578e528740 10 client.15108 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:39:49.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f578e528740 10 client.15108 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T01:39:49.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f578e528740 20 client.15108 populate_metadata read hostname 'smithi139' 2024-09-16T01:39:49.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f578e528740 1 --2- 172.21.15.139:0/1639045996 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5621ade4e810 0x5621ade4fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f578e528740 4 client.15108 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:39:49.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f578e528740 1 -- 172.21.15.139:0/1639045996 --> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621adda6600 con 0x5621ade4e810 2024-09-16T01:39:49.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:48.999+0000 7f57896ea640 1 --2- 172.21.15.139:0/1639045996 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5621ade4e810 0x5621ade4fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.000+0000 7f57896ea640 1 --2- 172.21.15.139:0/1639045996 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5621ade4e810 0x5621ade4fc80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=623bd713045c9180 server_cookie=99b907c41e783587 in_seq=0 out_seq=0 2024-09-16T01:39:49.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.000+0000 7f576d7fa640 10 client.15108 ms_handle_connect on v2:172.21.15.164:6836/1270226509 2024-09-16T01:39:49.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f576d7fa640 1 -- 172.21.15.139:0/1639045996 <== mds.2 v2:172.21.15.164:6836/1270226509 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621adda6600 con 0x5621ade4e810 2024-09-16T01:39:49.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f576d7fa640 10 client.15108 handle_command_reply: tid=0 2024-09-16T01:39:49.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f578e528740 1 client.15108 shutdown 2024-09-16T01:39:49.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f578e528740 1 -- 172.21.15.139:0/1639045996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 msgr2=0x5621ade3d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:49.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f578e528740 1 --2- 172.21.15.139:0/1639045996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade3d490 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f57800edd70 tx=0x7f578002a470 comp rx=0 tx=0).stop 2024-09-16T01:39:49.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f578e528740 1 -- 172.21.15.139:0/1639045996 shutdown_connections 2024-09-16T01:39:49.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f578e528740 1 --2- 172.21.15.139:0/1639045996 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5621ade4e810 0x5621ade4fc80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f578e528740 1 --2- 172.21.15.139:0/1639045996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade3dbb0 0x5621ade3ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f578e528740 1 --2- 172.21.15.139:0/1639045996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade332f0 0x5621ade3a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f578e528740 1 --2- 172.21.15.139:0/1639045996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade3d490 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f576d7fa640 0 client.15108 ms_handle_reset on v2:172.21.15.164:6836/1270226509 2024-09-16T01:39:49.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f578e528740 1 -- 172.21.15.139:0/1639045996 >> 172.21.15.139:0/1639045996 conn(0x5621ade2ef60 msgr2=0x5621ade34c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:49.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f578e528740 1 -- 172.21.15.139:0/1639045996 shutdown_connections 2024-09-16T01:39:49.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f578e528740 1 -- 172.21.15.139:0/1639045996 wait complete. 2024-09-16T01:39:49.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.001+0000 7f578e528740 20 client.15108 trim_cache size 0 max 16384 2024-09-16T01:39:49.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f578e528740 1 Processor -- start 2024-09-16T01:39:49.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f578e528740 1 -- start start 2024-09-16T01:39:49.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f578e528740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade4eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f578e528740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade332f0 0x5621ade29ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f578e528740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade3dbb0 0x5621ade390d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f578e528740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5621ade4dfc0 con 0x5621ade329d0 2024-09-16T01:39:49.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f578e528740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5621ade4e690 con 0x5621ade332f0 2024-09-16T01:39:49.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f578e528740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5621ade4ed60 con 0x5621ade3dbb0 2024-09-16T01:39:49.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f5789eeb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade4eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f5789eeb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade4eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43848/0 (socket says 172.21.15.139:43848) 2024-09-16T01:39:49.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f5789eeb640 1 -- 172.21.15.139:0/2361397175 learned_addr learned my addr 172.21.15.139:0/2361397175 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:49.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f57896ea640 1 --2- 172.21.15.139:0/2361397175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade332f0 0x5621ade29ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f578a6ec640 1 --2- 172.21.15.139:0/2361397175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade3dbb0 0x5621ade390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f5789eeb640 1 -- 172.21.15.139:0/2361397175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade3dbb0 msgr2=0x5621ade390d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:49.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f5789eeb640 1 --2- 172.21.15.139:0/2361397175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade3dbb0 0x5621ade390d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f5789eeb640 1 -- 172.21.15.139:0/2361397175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade332f0 msgr2=0x5621ade29ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:49.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f5789eeb640 1 --2- 172.21.15.139:0/2361397175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade332f0 0x5621ade29ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f5789eeb640 1 -- 172.21.15.139:0/2361397175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621ade3d0b0 con 0x5621ade329d0 2024-09-16T01:39:49.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f5789eeb640 1 --2- 172.21.15.139:0/2361397175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade4eee0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f57800bab00 tx=0x7f57800f0a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0a74bd88c35964 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:49.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.019+0000 7f576f7fe640 1 -- 172.21.15.139:0/2361397175 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780049070 con 0x5621ade329d0 2024-09-16T01:39:49.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.020+0000 7f576f7fe640 1 -- 172.21.15.139:0/2361397175 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57800bc850 con 0x5621ade329d0 2024-09-16T01:39:49.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.020+0000 7f576f7fe640 1 -- 172.21.15.139:0/2361397175 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57800b9020 con 0x5621ade329d0 2024-09-16T01:39:49.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.020+0000 7f578e528740 1 -- 172.21.15.139:0/2361397175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 msgr2=0x5621ade4eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:49.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.020+0000 7f578e528740 1 --2- 172.21.15.139:0/2361397175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade4eee0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f57800bab00 tx=0x7f57800f0a40 comp rx=0 tx=0).stop 2024-09-16T01:39:49.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.020+0000 7f578e528740 1 -- 172.21.15.139:0/2361397175 shutdown_connections 2024-09-16T01:39:49.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.020+0000 7f578e528740 1 --2- 172.21.15.139:0/2361397175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade3dbb0 0x5621ade390d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.020+0000 7f578e528740 1 --2- 172.21.15.139:0/2361397175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade332f0 0x5621ade29ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.020+0000 7f578e528740 1 --2- 172.21.15.139:0/2361397175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade329d0 0x5621ade4eee0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.020+0000 7f578e528740 1 -- 172.21.15.139:0/2361397175 >> 172.21.15.139:0/2361397175 conn(0x5621ade2ef60 msgr2=0x5621ade4d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:49.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.020+0000 7f578e528740 1 -- 172.21.15.139:0/2361397175 shutdown_connections 2024-09-16T01:39:49.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.020+0000 7f578e528740 1 -- 172.21.15.139:0/2361397175 wait complete. 2024-09-16T01:39:49.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.020+0000 7f578e528740 1 -- start start 2024-09-16T01:39:49.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f578e528740 1 Processor -- start 2024-09-16T01:39:49.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f578e528740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade329d0 0x5621ade58130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f578e528740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade332f0 0x5621ade58670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f578e528740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade3dbb0 0x5621ade37070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f578e528740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5621ade4fb00 con 0x5621ade332f0 2024-09-16T01:39:49.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f578e528740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5621ade4f430 con 0x5621ade329d0 2024-09-16T01:39:49.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f578e528740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5621ade501d0 con 0x5621ade3dbb0 2024-09-16T01:39:49.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f578e528740 10 client.0 _ll_register_callbacks cb 0x7f57800b8af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:39:49.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f57896ea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade332f0 0x5621ade58670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f57896ea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade332f0 0x5621ade58670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43850/0 (socket says 172.21.15.139:43850) 2024-09-16T01:39:49.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f57896ea640 1 -- 172.21.15.139:0/3581662894 learned_addr learned my addr 172.21.15.139:0/3581662894 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:49.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f5789eeb640 1 --2- 172.21.15.139:0/3581662894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade329d0 0x5621ade58130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f578a6ec640 1 --2- 172.21.15.139:0/3581662894 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade3dbb0 0x5621ade37070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f57896ea640 1 -- 172.21.15.139:0/3581662894 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade3dbb0 msgr2=0x5621ade37070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:49.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f57896ea640 1 --2- 172.21.15.139:0/3581662894 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade3dbb0 0x5621ade37070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f57896ea640 1 -- 172.21.15.139:0/3581662894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade329d0 msgr2=0x5621ade58130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:49.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f57896ea640 1 --2- 172.21.15.139:0/3581662894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade329d0 0x5621ade58130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.021+0000 7f57896ea640 1 -- 172.21.15.139:0/3581662894 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621ade378a0 con 0x5621ade332f0 2024-09-16T01:39:49.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f57896ea640 1 --2- 172.21.15.139:0/3581662894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade332f0 0x5621ade58670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f577400ba80 tx=0x7f5774015420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73daa40e5f2896c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:49.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f576d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:39:49.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f576d7fa640 1 -- 172.21.15.139:0/3581662894 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774015d30 con 0x5621ade332f0 2024-09-16T01:39:49.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f576d7fa640 1 -- 172.21.15.139:0/3581662894 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f57740143b0 con 0x5621ade332f0 2024-09-16T01:39:49.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f578e528740 1 -- 172.21.15.139:0/3581662894 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57800bc850 con 0x5621ade332f0 2024-09-16T01:39:49.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f576d7fa640 1 -- 172.21.15.139:0/3581662894 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f57740145b0 con 0x5621ade332f0 2024-09-16T01:39:49.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f578e528740 10 client.15114 fetch_fsmap learned FSMap version 28 2024-09-16T01:39:49.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f578e528740 1 -- 172.21.15.139:0/3581662894 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5621ade57a20 con 0x5621ade332f0 2024-09-16T01:39:49.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f576d7fa640 1 -- 172.21.15.139:0/3581662894 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57800bc850 con 0x5621ade332f0 2024-09-16T01:39:49.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f576d7fa640 1 -- 172.21.15.139:0/3581662894 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f5774025870 con 0x5621ade332f0 2024-09-16T01:39:49.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f578e528740 10 client.15114 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:39:49.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f578e528740 10 client.15114 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T01:39:49.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f578e528740 20 client.15114 populate_metadata read hostname 'smithi139' 2024-09-16T01:39:49.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f578e528740 1 --2- 172.21.15.139:0/3581662894 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5621adebd650 0x5621adebc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:49.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f578e528740 4 client.15114 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:39:49.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.022+0000 7f578e528740 1 -- 172.21.15.139:0/3581662894 --> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] -- command(tid 0: {"prefix": "status"}) -- 0x5621adda0b10 con 0x5621adebd650 2024-09-16T01:39:49.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.023+0000 7f5789eeb640 1 --2- 172.21.15.139:0/3581662894 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5621adebd650 0x5621adebc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:49.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.023+0000 7f576d7fa640 10 client.15114 ms_handle_connect on v2:172.21.15.164:6836/1270226509 2024-09-16T01:39:49.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.023+0000 7f5789eeb640 1 --2- 172.21.15.139:0/3581662894 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5621adebd650 0x5621adebc360 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae47cefb1faf0583 server_cookie=7492d9e0d7f2dddc in_seq=0 out_seq=0 2024-09-16T01:39:49.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.023+0000 7f576d7fa640 1 -- 172.21.15.139:0/3581662894 <== mds.2 v2:172.21.15.164:6836/1270226509 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5621adda0b10 con 0x5621adebd650 2024-09-16T01:39:49.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.023+0000 7f576d7fa640 10 client.15114 handle_command_reply: tid=0 2024-09-16T01:39:49.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.023+0000 7f578e528740 1 client.15114 shutdown 2024-09-16T01:39:49.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578e528740 1 -- 172.21.15.139:0/3581662894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade332f0 msgr2=0x5621ade58670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:49.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578e528740 1 --2- 172.21.15.139:0/3581662894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade332f0 0x5621ade58670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f577400ba80 tx=0x7f5774015420 comp rx=0 tx=0).stop 2024-09-16T01:39:49.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578e528740 1 -- 172.21.15.139:0/3581662894 shutdown_connections 2024-09-16T01:39:49.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578e528740 1 --2- 172.21.15.139:0/3581662894 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5621adebd650 0x5621adebc360 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578e528740 1 --2- 172.21.15.139:0/3581662894 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621ade3dbb0 0x5621ade37070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578e528740 1 --2- 172.21.15.139:0/3581662894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621ade332f0 0x5621ade58670 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f576d7fa640 0 client.15114 ms_handle_reset on v2:172.21.15.164:6836/1270226509 2024-09-16T01:39:49.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578e528740 1 --2- 172.21.15.139:0/3581662894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621ade329d0 0x5621ade58130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578e528740 1 -- 172.21.15.139:0/3581662894 >> 172.21.15.139:0/3581662894 conn(0x5621ade2ef60 msgr2=0x5621ade39090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:49.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578e528740 1 -- 172.21.15.139:0/3581662894 shutdown_connections 2024-09-16T01:39:49.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578e528740 1 -- 172.21.15.139:0/3581662894 wait complete. 2024-09-16T01:39:49.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578e528740 20 client.15114 trim_cache size 0 max 16384 2024-09-16T01:39:49.044 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:39:49.044 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:39:49.044 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 2, 2024-09-16T01:39:49.044 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24443, 2024-09-16T01:39:49.044 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:39:49.044 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:39:49.044 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:39:49.044 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 71.494373519999996, 2024-09-16T01:39:49.044 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 28, 2024-09-16T01:39:49.045 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 85, 2024-09-16T01:39:49.045 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 83, 2024-09-16T01:39:49.045 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 130.73345334699999 2024-09-16T01:39:49.045 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:39:49.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:48 smithi164 ceph-mon[35121]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:39:49.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578c14f640 1 -- 172.21.15.139:0/280492720 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f57700a32d0 msgr2=0x7f57700a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:49.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578c14f640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f57700a32d0 0x7f57700a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f577c021000 tx=0x7f577c01f000 comp rx=0 tx=0).stop 2024-09-16T01:39:49.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578c14f640 1 -- 172.21.15.139:0/280492720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5784100b30 msgr2=0x7f57841a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:49.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578c14f640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5784100b30 0x7f57841a46f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5780005ab0 tx=0x7f57800042f0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.024+0000 7f578c14f640 1 -- 172.21.15.139:0/280492720 shutdown_connections 2024-09-16T01:39:49.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.025+0000 7f578c14f640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f57700a32d0 0x7f57700a5790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.025+0000 7f578c14f640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5784105f40 0x7f578419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.025+0000 7f578c14f640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5784101450 0x7f57841a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.025+0000 7f578c14f640 1 --2- 172.21.15.139:0/280492720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5784100b30 0x7f57841a46f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:49.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.025+0000 7f578c14f640 1 -- 172.21.15.139:0/280492720 >> 172.21.15.139:0/280492720 conn(0x7f57840fc7a0 msgr2=0x7f578410f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:49.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.025+0000 7f578c14f640 1 -- 172.21.15.139:0/280492720 shutdown_connections 2024-09-16T01:39:49.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:49.025+0000 7f578c14f640 1 -- 172.21.15.139:0/280492720 wait complete. 2024-09-16T01:39:49.829 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-16T01:39:50.073 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:50.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.558+0000 7f12a4f6c640 1 -- 172.21.15.139:0/1390813389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12a0102570 msgr2=0x7f12a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:50.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.558+0000 7f12a4f6c640 1 --2- 172.21.15.139:0/1390813389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12a0102570 0x7f12a01029b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1294009ca0 tx=0x7f129402a950 comp rx=0 tx=0).stop 2024-09-16T01:39:50.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.558+0000 7f12a4f6c640 1 -- 172.21.15.139:0/1390813389 shutdown_connections 2024-09-16T01:39:50.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.558+0000 7f12a4f6c640 1 --2- 172.21.15.139:0/1390813389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12a0102ef0 0x7f12a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.558+0000 7f12a4f6c640 1 --2- 172.21.15.139:0/1390813389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12a0102570 0x7f12a01029b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.558+0000 7f12a4f6c640 1 --2- 172.21.15.139:0/1390813389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12a0100e20 0x7f12a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.558+0000 7f12a4f6c640 1 -- 172.21.15.139:0/1390813389 >> 172.21.15.139:0/1390813389 conn(0x7f12a00fc7d0 msgr2=0x7f12a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:50.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.558+0000 7f12a4f6c640 1 -- 172.21.15.139:0/1390813389 shutdown_connections 2024-09-16T01:39:50.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.558+0000 7f12a4f6c640 1 -- 172.21.15.139:0/1390813389 wait complete. 2024-09-16T01:39:50.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.559+0000 7f12a4f6c640 1 Processor -- start 2024-09-16T01:39:50.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.559+0000 7f12a4f6c640 1 -- start start 2024-09-16T01:39:50.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.559+0000 7f12a4f6c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12a0100e20 0x7f12a0197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.559+0000 7f12a4f6c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12a0102570 0x7f12a0197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.559+0000 7f12a4f6c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12a0102ef0 0x7f12a019be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.559+0000 7f12a4f6c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f12a0111020 con 0x7f12a0100e20 2024-09-16T01:39:50.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.559+0000 7f12a4f6c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12a0110ea0 con 0x7f12a0102570 2024-09-16T01:39:50.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.559+0000 7f12a4f6c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f12a01111a0 con 0x7f12a0102ef0 2024-09-16T01:39:50.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.559+0000 7f129e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12a0100e20 0x7f12a0197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f129e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12a0100e20 0x7f12a0197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43866/0 (socket says 172.21.15.139:43866) 2024-09-16T01:39:50.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f129e59c640 1 -- 172.21.15.139:0/46982881 learned_addr learned my addr 172.21.15.139:0/46982881 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:50.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f129dd9b640 1 --2- 172.21.15.139:0/46982881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12a0102570 0x7f12a0197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f129ed9d640 1 --2- 172.21.15.139:0/46982881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12a0102ef0 0x7f12a019be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f129e59c640 1 -- 172.21.15.139:0/46982881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12a0102ef0 msgr2=0x7f12a019be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:50.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f129e59c640 1 --2- 172.21.15.139:0/46982881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12a0102ef0 0x7f12a019be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f129e59c640 1 -- 172.21.15.139:0/46982881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12a0102570 msgr2=0x7f12a0197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:50.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f129e59c640 1 --2- 172.21.15.139:0/46982881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12a0102570 0x7f12a0197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f129e59c640 1 -- 172.21.15.139:0/46982881 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a019c4f0 con 0x7f12a0100e20 2024-09-16T01:39:50.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f129e59c640 1 --2- 172.21.15.139:0/46982881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12a0100e20 0x7f12a0197540 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f129000b4a0 tx=0x7f129000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:50.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f128f7fe640 1 -- 172.21.15.139:0/46982881 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290014070 con 0x7f12a0100e20 2024-09-16T01:39:50.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f12a4f6c640 1 -- 172.21.15.139:0/46982881 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a019c7e0 con 0x7f12a0100e20 2024-09-16T01:39:50.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f128f7fe640 1 -- 172.21.15.139:0/46982881 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1290010af0 con 0x7f12a0100e20 2024-09-16T01:39:50.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f128f7fe640 1 -- 172.21.15.139:0/46982881 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290017d10 con 0x7f12a0100e20 2024-09-16T01:39:50.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.560+0000 7f12a4f6c640 1 -- 172.21.15.139:0/46982881 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a01a40c0 con 0x7f12a0100e20 2024-09-16T01:39:50.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.562+0000 7f128f7fe640 1 -- 172.21.15.139:0/46982881 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1290017070 con 0x7f12a0100e20 2024-09-16T01:39:50.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f12a7345740 1 Processor -- start 2024-09-16T01:39:50.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f12a7345740 1 -- start start 2024-09-16T01:39:50.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f12a7345740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f3230 0x5570561f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f12a7345740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561f2000 0x5570561f2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f12a7345740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561fec70 0x557056201060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f12a7345740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557056156090 con 0x5570561f2000 2024-09-16T01:39:50.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f12a7345740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5570560fda70 con 0x5570561fec70 2024-09-16T01:39:50.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f12a7345740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557056152050 con 0x5570561f3230 2024-09-16T01:39:50.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f129e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561f2000 0x5570561f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f128f7fe640 1 --2- 172.21.15.139:0/46982881 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f12880a32d0 0x7f12880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f129e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561f2000 0x5570561f2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43874/0 (socket says 172.21.15.139:43874) 2024-09-16T01:39:50.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f129e59c640 1 -- 172.21.15.139:0/2477160639 learned_addr learned my addr 172.21.15.139:0/2477160639 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:50.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f129dd9b640 1 --2- 172.21.15.139:0/2477160639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561fec70 0x557056201060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f129ed9d640 1 --2- 172.21.15.139:0/46982881 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f12880a32d0 0x7f12880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.563+0000 7f129ed9d640 1 --2- 172.21.15.139:0/2477160639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f3230 0x5570561f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.564+0000 7f129e59c640 1 -- 172.21.15.139:0/2477160639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f3230 msgr2=0x5570561f1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:50.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.564+0000 7f129e59c640 1 --2- 172.21.15.139:0/2477160639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f3230 0x5570561f1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.564+0000 7f129e59c640 1 -- 172.21.15.139:0/2477160639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561fec70 msgr2=0x557056201060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:50.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.564+0000 7f129e59c640 1 --2- 172.21.15.139:0/2477160639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561fec70 0x557056201060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.564+0000 7f129e59c640 1 -- 172.21.15.139:0/2477160639 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55705610a130 con 0x5570561f2000 2024-09-16T01:39:50.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.564+0000 7f128f7fe640 1 -- 172.21.15.139:0/46982881 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f12900cf130 con 0x7f12a0100e20 2024-09-16T01:39:50.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.564+0000 7f129e59c640 1 --2- 172.21.15.139:0/2477160639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561f2000 0x5570561f2460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f12900cc790 tx=0x7f129009adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5155ae38daece181 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:50.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.564+0000 7f129ed9d640 1 --2- 172.21.15.139:0/46982881 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f12880a32d0 0x7f12880a5790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f129800cd60 tx=0x7f1298006d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:50.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.564+0000 7f128cff9640 1 -- 172.21.15.139:0/2477160639 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129001e070 con 0x5570561f2000 2024-09-16T01:39:50.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.564+0000 7f128cff9640 1 -- 172.21.15.139:0/2477160639 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f129009ba30 con 0x5570561f2000 2024-09-16T01:39:50.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.564+0000 7f128cff9640 1 -- 172.21.15.139:0/2477160639 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12900cc960 con 0x5570561f2000 2024-09-16T01:39:50.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 -- 172.21.15.139:0/2477160639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561f2000 msgr2=0x5570561f2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:50.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 --2- 172.21.15.139:0/2477160639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561f2000 0x5570561f2460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f12900cc790 tx=0x7f129009adc0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 -- 172.21.15.139:0/2477160639 shutdown_connections 2024-09-16T01:39:50.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 --2- 172.21.15.139:0/2477160639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561fec70 0x557056201060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 --2- 172.21.15.139:0/2477160639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561f2000 0x5570561f2460 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 --2- 172.21.15.139:0/2477160639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f3230 0x5570561f1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 -- 172.21.15.139:0/2477160639 >> 172.21.15.139:0/2477160639 conn(0x5570561ee170 msgr2=0x5570561ee550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:50.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 -- 172.21.15.139:0/2477160639 shutdown_connections 2024-09-16T01:39:50.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 -- 172.21.15.139:0/2477160639 wait complete. 2024-09-16T01:39:50.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 -- start start 2024-09-16T01:39:50.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 Processor -- start 2024-09-16T01:39:50.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f2000 0x5570561fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f3230 0x5570561fb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561fec70 0x5570561fbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557056201b80 con 0x5570561fec70 2024-09-16T01:39:50.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557056110f20 con 0x5570561f2000 2024-09-16T01:39:50.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.565+0000 7f12a7345740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557056201d00 con 0x5570561f3230 2024-09-16T01:39:50.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f12a7345740 10 client.0 _ll_register_callbacks cb 0x5570560da030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:39:50.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f129ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561fec70 0x5570561fbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f129ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561fec70 0x5570561fbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43890/0 (socket says 172.21.15.139:43890) 2024-09-16T01:39:50.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f129ed9d640 1 -- 172.21.15.139:0/2634570433 learned_addr learned my addr 172.21.15.139:0/2634570433 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:50.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f129dd9b640 1 --2- 172.21.15.139:0/2634570433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f3230 0x5570561fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f129e59c640 1 --2- 172.21.15.139:0/2634570433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f2000 0x5570561fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f129ed9d640 1 -- 172.21.15.139:0/2634570433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f3230 msgr2=0x5570561fb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:50.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f129ed9d640 1 --2- 172.21.15.139:0/2634570433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f3230 0x5570561fb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f129ed9d640 1 -- 172.21.15.139:0/2634570433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f2000 msgr2=0x5570561fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:50.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f129ed9d640 1 --2- 172.21.15.139:0/2634570433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f2000 0x5570561fe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f129ed9d640 1 -- 172.21.15.139:0/2634570433 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557055ec72b0 con 0x5570561fec70 2024-09-16T01:39:50.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f129e59c640 1 --2- 172.21.15.139:0/2634570433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f2000 0x5570561fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:50.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f129ed9d640 1 --2- 172.21.15.139:0/2634570433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561fec70 0x5570561fbe80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1298014010 tx=0x7f12980047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43f94a1a37367db server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:50.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f1285ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:39:50.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f1285ffb640 1 -- 172.21.15.139:0/2634570433 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298026060 con 0x5570561fec70 2024-09-16T01:39:50.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f1285ffb640 1 -- 172.21.15.139:0/2634570433 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f1298021410 con 0x5570561fec70 2024-09-16T01:39:50.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f12a7345740 1 -- 172.21.15.139:0/2634570433 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f129009ba30 con 0x5570561fec70 2024-09-16T01:39:50.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f1285ffb640 1 -- 172.21.15.139:0/2634570433 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f129802d050 con 0x5570561fec70 2024-09-16T01:39:50.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f12a7345740 10 client.15126 fetch_fsmap learned FSMap version 28 2024-09-16T01:39:50.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.566+0000 7f12a7345740 1 -- 172.21.15.139:0/2634570433 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55705620d5f0 con 0x5570561fec70 2024-09-16T01:39:50.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.567+0000 7f1285ffb640 1 -- 172.21.15.139:0/2634570433 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f129009ba30 con 0x5570561fec70 2024-09-16T01:39:50.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.567+0000 7f1285ffb640 1 -- 172.21.15.139:0/2634570433 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1298015070 con 0x5570561fec70 2024-09-16T01:39:50.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.567+0000 7f12a7345740 10 client.15126 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:39:50.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.567+0000 7f12a7345740 10 client.15126 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T01:39:50.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.567+0000 7f12a7345740 20 client.15126 populate_metadata read hostname 'smithi139' 2024-09-16T01:39:50.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.567+0000 7f12a7345740 1 --2- 172.21.15.139:0/2634570433 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x55705620fc40 0x5570562110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.567+0000 7f12a7345740 4 client.15126 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:39:50.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.567+0000 7f12a7345740 1 -- 172.21.15.139:0/2634570433 --> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557056163600 con 0x55705620fc40 2024-09-16T01:39:50.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.567+0000 7f129e59c640 1 --2- 172.21.15.139:0/2634570433 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x55705620fc40 0x5570562110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.567+0000 7f129e59c640 1 --2- 172.21.15.139:0/2634570433 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x55705620fc40 0x5570562110b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac524cfe6d333b04 server_cookie=b737ce9204d93554 in_seq=0 out_seq=0 2024-09-16T01:39:50.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.567+0000 7f1285ffb640 10 client.15126 ms_handle_connect on v2:172.21.15.176:6838/3912805068 2024-09-16T01:39:50.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.568+0000 7f1285ffb640 1 -- 172.21.15.139:0/2634570433 <== mds.1 v2:172.21.15.176:6838/3912805068 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557056163600 con 0x55705620fc40 2024-09-16T01:39:50.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.568+0000 7f1285ffb640 10 client.15126 handle_command_reply: tid=0 2024-09-16T01:39:50.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.568+0000 7f12a7345740 1 client.15126 shutdown 2024-09-16T01:39:50.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.568+0000 7f12a7345740 1 -- 172.21.15.139:0/2634570433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561fec70 msgr2=0x5570561fbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:50.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.568+0000 7f12a7345740 1 --2- 172.21.15.139:0/2634570433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561fec70 0x5570561fbe80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1298014010 tx=0x7f12980047d0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.568+0000 7f12a7345740 1 -- 172.21.15.139:0/2634570433 shutdown_connections 2024-09-16T01:39:50.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.568+0000 7f12a7345740 1 --2- 172.21.15.139:0/2634570433 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x55705620fc40 0x5570562110b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.568+0000 7f12a7345740 1 --2- 172.21.15.139:0/2634570433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561fec70 0x5570561fbe80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.568+0000 7f12a7345740 1 --2- 172.21.15.139:0/2634570433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f3230 0x5570561fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.568+0000 7f1285ffb640 0 client.15126 ms_handle_reset on v2:172.21.15.176:6838/3912805068 2024-09-16T01:39:50.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.568+0000 7f12a7345740 1 --2- 172.21.15.139:0/2634570433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f2000 0x5570561fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.568+0000 7f12a7345740 1 -- 172.21.15.139:0/2634570433 >> 172.21.15.139:0/2634570433 conn(0x5570561ee170 msgr2=0x5570561f3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:50.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.569+0000 7f12a7345740 1 -- 172.21.15.139:0/2634570433 shutdown_connections 2024-09-16T01:39:50.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.569+0000 7f12a7345740 1 -- 172.21.15.139:0/2634570433 wait complete. 2024-09-16T01:39:50.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.569+0000 7f12a7345740 20 client.15126 trim_cache size 0 max 16384 2024-09-16T01:39:50.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.586+0000 7f12a7345740 1 Processor -- start 2024-09-16T01:39:50.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f12a7345740 1 -- start start 2024-09-16T01:39:50.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f12a7345740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f2000 0x5570562110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f12a7345740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561f3230 0x5570561f9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f12a7345740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561fec70 0x5570561fa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f12a7345740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55705620f3f0 con 0x5570561f3230 2024-09-16T01:39:50.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f12a7345740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55705620fac0 con 0x5570561f2000 2024-09-16T01:39:50.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f12a7345740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557056210190 con 0x5570561fec70 2024-09-16T01:39:50.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f129dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561f3230 0x5570561f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f129dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561f3230 0x5570561f9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43892/0 (socket says 172.21.15.139:43892) 2024-09-16T01:39:50.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f129dd9b640 1 -- 172.21.15.139:0/1652238888 learned_addr learned my addr 172.21.15.139:0/1652238888 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:50.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f129ed9d640 1 --2- 172.21.15.139:0/1652238888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561fec70 0x5570561fa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f129dd9b640 1 -- 172.21.15.139:0/1652238888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561fec70 msgr2=0x5570561fa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:50.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f129dd9b640 1 --2- 172.21.15.139:0/1652238888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561fec70 0x5570561fa7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f129dd9b640 1 -- 172.21.15.139:0/1652238888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f2000 msgr2=0x5570562110b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:50.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f129dd9b640 1 --2- 172.21.15.139:0/1652238888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f2000 0x5570562110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f129dd9b640 1 -- 172.21.15.139:0/1652238888 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570561ed8f0 con 0x5570561f3230 2024-09-16T01:39:50.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f129dd9b640 1 --2- 172.21.15.139:0/1652238888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561f3230 0x5570561f9450 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1294035000 tx=0x7f129402fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef1c9ff8c45871e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:50.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.587+0000 7f1287fff640 1 -- 172.21.15.139:0/1652238888 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129404a070 con 0x5570561f3230 2024-09-16T01:39:50.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.588+0000 7f1287fff640 1 -- 172.21.15.139:0/1652238888 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1294003e50 con 0x5570561f3230 2024-09-16T01:39:50.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.588+0000 7f1287fff640 1 -- 172.21.15.139:0/1652238888 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294034040 con 0x5570561f3230 2024-09-16T01:39:50.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.588+0000 7f12a7345740 1 -- 172.21.15.139:0/1652238888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561f3230 msgr2=0x5570561f9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:50.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.588+0000 7f12a7345740 1 --2- 172.21.15.139:0/1652238888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561f3230 0x5570561f9450 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1294035000 tx=0x7f129402fc00 comp rx=0 tx=0).stop 2024-09-16T01:39:50.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.588+0000 7f12a7345740 1 -- 172.21.15.139:0/1652238888 shutdown_connections 2024-09-16T01:39:50.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.588+0000 7f12a7345740 1 --2- 172.21.15.139:0/1652238888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561fec70 0x5570561fa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.588+0000 7f12a7345740 1 --2- 172.21.15.139:0/1652238888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561f3230 0x5570561f9450 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.588+0000 7f12a7345740 1 --2- 172.21.15.139:0/1652238888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f2000 0x5570562110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.588+0000 7f12a7345740 1 -- 172.21.15.139:0/1652238888 >> 172.21.15.139:0/1652238888 conn(0x5570561ee170 msgr2=0x5570561fa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:50.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f12a7345740 1 -- 172.21.15.139:0/1652238888 shutdown_connections 2024-09-16T01:39:50.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f12a7345740 1 -- 172.21.15.139:0/1652238888 wait complete. 2024-09-16T01:39:50.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f12a7345740 1 -- start start 2024-09-16T01:39:50.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f12a7345740 1 Processor -- start 2024-09-16T01:39:50.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f12a7345740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f2000 0x557056219ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f12a7345740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f3230 0x5570561eb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f12a7345740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561fec70 0x5570561eb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f12a7345740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557056210f30 con 0x5570561fec70 2024-09-16T01:39:50.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f12a7345740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557056210860 con 0x5570561f3230 2024-09-16T01:39:50.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f12a7345740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557056211600 con 0x5570561f2000 2024-09-16T01:39:50.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f12a7345740 10 client.0 _ll_register_callbacks cb 0x7f129800c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:39:50.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f129ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561fec70 0x5570561eb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f129ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561fec70 0x5570561eb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43900/0 (socket says 172.21.15.139:43900) 2024-09-16T01:39:50.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f129ed9d640 1 -- 172.21.15.139:0/1668104071 learned_addr learned my addr 172.21.15.139:0/1668104071 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:50.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.589+0000 7f129dd9b640 1 --2- 172.21.15.139:0/1668104071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f3230 0x5570561eb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f129ed9d640 1 -- 172.21.15.139:0/1668104071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f2000 msgr2=0x557056219ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:50.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f129e59c640 1 --2- 172.21.15.139:0/1668104071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f2000 0x557056219ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f129ed9d640 1 --2- 172.21.15.139:0/1668104071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f2000 0x557056219ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f129ed9d640 1 -- 172.21.15.139:0/1668104071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f3230 msgr2=0x5570561eb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:50.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f129ed9d640 1 --2- 172.21.15.139:0/1668104071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f3230 0x5570561eb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f129ed9d640 1 -- 172.21.15.139:0/1668104071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570561fc1f0 con 0x5570561fec70 2024-09-16T01:39:50.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f129ed9d640 1 --2- 172.21.15.139:0/1668104071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561fec70 0x5570561eb780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f12980155d0 tx=0x7f1298018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba7556ebf17f41c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:50.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f1285ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:39:50.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f1285ffb640 1 -- 172.21.15.139:0/1668104071 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129800f040 con 0x5570561fec70 2024-09-16T01:39:50.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f1285ffb640 1 -- 172.21.15.139:0/1668104071 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f129801e040 con 0x5570561fec70 2024-09-16T01:39:50.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f12a7345740 1 -- 172.21.15.139:0/1668104071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1294003e50 con 0x5570561fec70 2024-09-16T01:39:50.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f1285ffb640 1 -- 172.21.15.139:0/1668104071 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f129803d050 con 0x5570561fec70 2024-09-16T01:39:50.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f12a7345740 10 client.15132 fetch_fsmap learned FSMap version 28 2024-09-16T01:39:50.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f12a7345740 1 -- 172.21.15.139:0/1668104071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55705627a350 con 0x5570561fec70 2024-09-16T01:39:50.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.590+0000 7f1285ffb640 1 -- 172.21.15.139:0/1668104071 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1294003e50 con 0x5570561fec70 2024-09-16T01:39:50.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.591+0000 7f1285ffb640 1 -- 172.21.15.139:0/1668104071 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1298015040 con 0x5570561fec70 2024-09-16T01:39:50.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.591+0000 7f12a7345740 10 client.15132 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:39:50.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.591+0000 7f12a7345740 10 client.15132 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T01:39:50.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.591+0000 7f12a7345740 20 client.15132 populate_metadata read hostname 'smithi139' 2024-09-16T01:39:50.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.591+0000 7f12a7345740 1 --2- 172.21.15.139:0/1668104071 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x55705627ecc0 0x55705627d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:50.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.591+0000 7f12a7345740 4 client.15132 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:39:50.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.591+0000 7f12a7345740 1 -- 172.21.15.139:0/1668104071 --> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] -- command(tid 0: {"prefix": "status"}) -- 0x55705615db10 con 0x55705627ecc0 2024-09-16T01:39:50.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.591+0000 7f129e59c640 1 --2- 172.21.15.139:0/1668104071 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x55705627ecc0 0x55705627d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:50.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.591+0000 7f129e59c640 1 --2- 172.21.15.139:0/1668104071 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x55705627ecc0 0x55705627d9d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e450343609ea75f0 server_cookie=4c5a94a06c413f0 in_seq=0 out_seq=0 2024-09-16T01:39:50.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.591+0000 7f1285ffb640 10 client.15132 ms_handle_connect on v2:172.21.15.176:6838/3912805068 2024-09-16T01:39:50.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f1285ffb640 1 -- 172.21.15.139:0/1668104071 <== mds.1 v2:172.21.15.176:6838/3912805068 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55705615db10 con 0x55705627ecc0 2024-09-16T01:39:50.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f1285ffb640 10 client.15132 handle_command_reply: tid=0 2024-09-16T01:39:50.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a7345740 1 client.15132 shutdown 2024-09-16T01:39:50.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a7345740 1 -- 172.21.15.139:0/1668104071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561fec70 msgr2=0x5570561eb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:50.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a7345740 1 --2- 172.21.15.139:0/1668104071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561fec70 0x5570561eb780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f12980155d0 tx=0x7f1298018f90 comp rx=0 tx=0).stop 2024-09-16T01:39:50.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a7345740 1 -- 172.21.15.139:0/1668104071 shutdown_connections 2024-09-16T01:39:50.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a7345740 1 --2- 172.21.15.139:0/1668104071 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x55705627ecc0 0x55705627d9d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a7345740 1 --2- 172.21.15.139:0/1668104071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5570561fec70 0x5570561eb780 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a7345740 1 --2- 172.21.15.139:0/1668104071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570561f3230 0x5570561eb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a7345740 1 --2- 172.21.15.139:0/1668104071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570561f2000 0x557056219ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f1285ffb640 0 client.15132 ms_handle_reset on v2:172.21.15.176:6838/3912805068 2024-09-16T01:39:50.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a7345740 1 -- 172.21.15.139:0/1668104071 >> 172.21.15.139:0/1668104071 conn(0x5570561ee170 msgr2=0x5570561ed100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:50.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a7345740 1 -- 172.21.15.139:0/1668104071 shutdown_connections 2024-09-16T01:39:50.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a7345740 1 -- 172.21.15.139:0/1668104071 wait complete. 2024-09-16T01:39:50.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a7345740 20 client.15132 trim_cache size 0 max 16384 2024-09-16T01:39:50.608 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:39:50.608 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:39:50.608 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 1, 2024-09-16T01:39:50.608 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24454, 2024-09-16T01:39:50.609 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:39:50.609 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:39:50.609 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:39:50.609 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 74.072807135000005, 2024-09-16T01:39:50.609 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 28, 2024-09-16T01:39:50.609 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 85, 2024-09-16T01:39:50.609 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 83, 2024-09-16T01:39:50.609 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 129.74868878800001 2024-09-16T01:39:50.609 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:39:50.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a4f6c640 1 -- 172.21.15.139:0/46982881 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f12880a32d0 msgr2=0x7f12880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:50.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a4f6c640 1 --2- 172.21.15.139:0/46982881 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f12880a32d0 0x7f12880a5790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f129800cd60 tx=0x7f1298006d30 comp rx=0 tx=0).stop 2024-09-16T01:39:50.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a4f6c640 1 -- 172.21.15.139:0/46982881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12a0100e20 msgr2=0x7f12a0197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:50.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a4f6c640 1 --2- 172.21.15.139:0/46982881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12a0100e20 0x7f12a0197540 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f129000b4a0 tx=0x7f129000b960 comp rx=0 tx=0).stop 2024-09-16T01:39:50.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a4f6c640 1 -- 172.21.15.139:0/46982881 shutdown_connections 2024-09-16T01:39:50.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a4f6c640 1 --2- 172.21.15.139:0/46982881 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f12880a32d0 0x7f12880a5790 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a4f6c640 1 --2- 172.21.15.139:0/46982881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12a0102ef0 0x7f12a019be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a4f6c640 1 --2- 172.21.15.139:0/46982881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12a0102570 0x7f12a0197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a4f6c640 1 --2- 172.21.15.139:0/46982881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12a0100e20 0x7f12a0197540 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:50.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.592+0000 7f12a4f6c640 1 -- 172.21.15.139:0/46982881 >> 172.21.15.139:0/46982881 conn(0x7f12a00fc7d0 msgr2=0x7f12a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:50.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.593+0000 7f12a4f6c640 1 -- 172.21.15.139:0/46982881 shutdown_connections 2024-09-16T01:39:50.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:50.593+0000 7f12a4f6c640 1 -- 172.21.15.139:0/46982881 wait complete. 2024-09-16T01:39:50.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:50 smithi139 ceph-mon[31175]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T01:39:50.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:50 smithi176 ceph-mon[35324]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T01:39:51.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:50 smithi164 ceph-mon[35121]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T01:39:51.463 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T01:39:51.714 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:52.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.176+0000 7f3a9fce2640 1 -- 172.21.15.139:0/3215040072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a98101de0 msgr2=0x7f3a981021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:52.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.176+0000 7f3a9fce2640 1 --2- 172.21.15.139:0/3215040072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a98101de0 0x7f3a981021c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009900 tx=0x7f3a9002a410 comp rx=0 tx=0).stop 2024-09-16T01:39:52.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.176+0000 7f3a9fce2640 1 -- 172.21.15.139:0/3215040072 shutdown_connections 2024-09-16T01:39:52.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.176+0000 7f3a9fce2640 1 --2- 172.21.15.139:0/3215040072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a98106f10 0x7f3a9810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.176+0000 7f3a9fce2640 1 --2- 172.21.15.139:0/3215040072 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a98102790 0x7f3a981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.176+0000 7f3a9fce2640 1 --2- 172.21.15.139:0/3215040072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a98101de0 0x7f3a981021c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.176+0000 7f3a9fce2640 1 -- 172.21.15.139:0/3215040072 >> 172.21.15.139:0/3215040072 conn(0x7f3a980fd8c0 msgr2=0x7f3a980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:52.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.176+0000 7f3a9fce2640 1 -- 172.21.15.139:0/3215040072 shutdown_connections 2024-09-16T01:39:52.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.176+0000 7f3a9fce2640 1 -- 172.21.15.139:0/3215040072 wait complete. 2024-09-16T01:39:52.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9fce2640 1 Processor -- start 2024-09-16T01:39:52.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9fce2640 1 -- start start 2024-09-16T01:39:52.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9fce2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a98101de0 0x7f3a981985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9fce2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a98102790 0x7f3a98198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9fce2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a98106f10 0x7f3a9819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9fce2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3a98112010 con 0x7f3a98101de0 2024-09-16T01:39:52.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9fce2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3a98111e90 con 0x7f3a98106f10 2024-09-16T01:39:52.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9fce2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3a98112190 con 0x7f3a98102790 2024-09-16T01:39:52.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9da7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a98101de0 0x7f3a981985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9da7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a98101de0 0x7f3a981985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43912/0 (socket says 172.21.15.139:43912) 2024-09-16T01:39:52.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9da7e640 1 -- 172.21.15.139:0/2066506724 learned_addr learned my addr 172.21.15.139:0/2066506724 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:52.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9d27d640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a98102790 0x7f3a98198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9da7e640 1 -- 172.21.15.139:0/2066506724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a98102790 msgr2=0x7f3a98198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:52.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a98102790 0x7f3a98198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9e27f640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a98106f10 0x7f3a9819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9da7e640 1 -- 172.21.15.139:0/2066506724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a98106f10 msgr2=0x7f3a9819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:52.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a98106f10 0x7f3a9819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9da7e640 1 -- 172.21.15.139:0/2066506724 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9819d560 con 0x7f3a98101de0 2024-09-16T01:39:52.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9d27d640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a98102790 0x7f3a98198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:52.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9e27f640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a98106f10 0x7f3a9819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:39:52.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.177+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a98101de0 0x7f3a981985b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3a90005ab0 tx=0x7f3a900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:52.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.178+0000 7f3a96ffd640 1 -- 172.21.15.139:0/2066506724 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9003f030 con 0x7f3a98101de0 2024-09-16T01:39:52.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.178+0000 7f3a9fce2640 1 -- 172.21.15.139:0/2066506724 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9819d7f0 con 0x7f3a98101de0 2024-09-16T01:39:52.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.178+0000 7f3a96ffd640 1 -- 172.21.15.139:0/2066506724 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a90003ce0 con 0x7f3a98101de0 2024-09-16T01:39:52.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.178+0000 7f3a96ffd640 1 -- 172.21.15.139:0/2066506724 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90035070 con 0x7f3a98101de0 2024-09-16T01:39:52.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.178+0000 7f3a9fce2640 1 -- 172.21.15.139:0/2066506724 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a981a5130 con 0x7f3a98101de0 2024-09-16T01:39:52.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.180+0000 7f3a96ffd640 1 -- 172.21.15.139:0/2066506724 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3a90042030 con 0x7f3a98101de0 2024-09-16T01:39:52.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.180+0000 7f3aa20bb740 1 Processor -- start 2024-09-16T01:39:52.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.180+0000 7f3aa20bb740 1 -- start start 2024-09-16T01:39:52.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.180+0000 7f3aa20bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a276c9d0 0x5606a276cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.180+0000 7f3aa20bb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276d2f0 0x5606a2777670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.180+0000 7f3aa20bb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a2777bb0 0x5606a2779fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.180+0000 7f3aa20bb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5606a26d3090 con 0x5606a276d2f0 2024-09-16T01:39:52.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.180+0000 7f3aa20bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5606a267aa70 con 0x5606a276c9d0 2024-09-16T01:39:52.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.180+0000 7f3aa20bb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5606a26cf050 con 0x5606a2777bb0 2024-09-16T01:39:52.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a9da7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276d2f0 0x5606a2777670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a9da7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276d2f0 0x5606a2777670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43928/0 (socket says 172.21.15.139:43928) 2024-09-16T01:39:52.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a9da7e640 1 -- 172.21.15.139:0/902114978 learned_addr learned my addr 172.21.15.139:0/902114978 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:52.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a96ffd640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3a800a32d0 0x7f3a800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a9e27f640 1 --2- 172.21.15.139:0/902114978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a276c9d0 0x5606a276cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a9d27d640 1 --2- 172.21.15.139:0/902114978 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a2777bb0 0x5606a2779fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a9e27f640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3a800a32d0 0x7f3a800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a96ffd640 1 -- 172.21.15.139:0/2066506724 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f3a900f0490 con 0x7f3a98101de0 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a9da7e640 1 -- 172.21.15.139:0/902114978 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a2777bb0 msgr2=0x5606a2779fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/902114978 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a2777bb0 0x5606a2779fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a9da7e640 1 -- 172.21.15.139:0/902114978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a276c9d0 msgr2=0x5606a276cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/902114978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a276c9d0 0x5606a276cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a9da7e640 1 -- 172.21.15.139:0/902114978 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606a2687130 con 0x5606a276d2f0 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/902114978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276d2f0 0x5606a2777670 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3a900bc280 tx=0x7f3a9002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f1af2c0c81792a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a7ffff640 1 -- 172.21.15.139:0/902114978 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90049030 con 0x5606a276d2f0 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a7ffff640 1 -- 172.21.15.139:0/902114978 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a900bcc80 con 0x5606a276d2f0 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a7ffff640 1 -- 172.21.15.139:0/902114978 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a900c51f0 con 0x5606a276d2f0 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.181+0000 7f3a9e27f640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3a800a32d0 0x7f3a800a5790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800e820 tx=0x7f3a880096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.182+0000 7f3aa20bb740 1 -- 172.21.15.139:0/902114978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276d2f0 msgr2=0x5606a2777670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.182+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/902114978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276d2f0 0x5606a2777670 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3a900bc280 tx=0x7f3a9002a410 comp rx=0 tx=0).stop 2024-09-16T01:39:52.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.182+0000 7f3aa20bb740 1 -- 172.21.15.139:0/902114978 shutdown_connections 2024-09-16T01:39:52.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.182+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/902114978 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a2777bb0 0x5606a2779fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.182+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/902114978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276d2f0 0x5606a2777670 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.182+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/902114978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a276c9d0 0x5606a276cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.182+0000 7f3aa20bb740 1 -- 172.21.15.139:0/902114978 >> 172.21.15.139:0/902114978 conn(0x5606a2768f60 msgr2=0x5606a2769340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:52.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.182+0000 7f3aa20bb740 1 -- 172.21.15.139:0/902114978 shutdown_connections 2024-09-16T01:39:52.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.182+0000 7f3aa20bb740 1 -- 172.21.15.139:0/902114978 wait complete. 2024-09-16T01:39:52.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3aa20bb740 1 -- start start 2024-09-16T01:39:52.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3aa20bb740 1 Processor -- start 2024-09-16T01:39:52.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3aa20bb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276c9d0 0x5606a2777460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3aa20bb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276d2f0 0x5606a27747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3aa20bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 0x5606a2774cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3aa20bb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5606a277a540 con 0x5606a276d2f0 2024-09-16T01:39:52.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3aa20bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5606a268df20 con 0x5606a2777bb0 2024-09-16T01:39:52.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3aa20bb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5606a277a6c0 con 0x5606a276c9d0 2024-09-16T01:39:52.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3aa20bb740 10 client.0 _ll_register_callbacks cb 0x5606a2657030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:39:52.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3a9d27d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276d2f0 0x5606a27747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3a9d27d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276d2f0 0x5606a27747b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43936/0 (socket says 172.21.15.139:43936) 2024-09-16T01:39:52.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3a9d27d640 1 -- 172.21.15.139:0/209260838 learned_addr learned my addr 172.21.15.139:0/209260838 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:52.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3a9e27f640 1 --2- 172.21.15.139:0/209260838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 0x5606a2774cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/209260838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276c9d0 0x5606a2777460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3a9d27d640 1 -- 172.21.15.139:0/209260838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276c9d0 msgr2=0x5606a2777460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:52.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.183+0000 7f3a9d27d640 1 --2- 172.21.15.139:0/209260838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276c9d0 0x5606a2777460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3a9d27d640 1 -- 172.21.15.139:0/209260838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 msgr2=0x5606a2774cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:52.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3a9d27d640 1 --2- 172.21.15.139:0/209260838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 0x5606a2774cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3a9d27d640 1 -- 172.21.15.139:0/209260838 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606a24442b0 con 0x5606a276d2f0 2024-09-16T01:39:52.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3a9d27d640 1 --2- 172.21.15.139:0/209260838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276d2f0 0x5606a27747b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00e170 tx=0x7f3a8c00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1e2257f64bb434 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:52.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3a7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:39:52.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3a7d7fa640 1 -- 172.21.15.139:0/209260838 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c017020 con 0x5606a276d2f0 2024-09-16T01:39:52.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3a7d7fa640 1 -- 172.21.15.139:0/209260838 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f3a8c013410 con 0x5606a276d2f0 2024-09-16T01:39:52.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3aa20bb740 1 -- 172.21.15.139:0/209260838 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a900bcc80 con 0x5606a276d2f0 2024-09-16T01:39:52.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3a7d7fa640 1 -- 172.21.15.139:0/209260838 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3a8c022050 con 0x5606a276d2f0 2024-09-16T01:39:52.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3aa20bb740 10 client.15144 fetch_fsmap learned FSMap version 28 2024-09-16T01:39:52.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3aa20bb740 1 -- 172.21.15.139:0/209260838 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5606a2785e30 con 0x5606a276d2f0 2024-09-16T01:39:52.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3a7d7fa640 1 -- 172.21.15.139:0/209260838 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a900bcc80 con 0x5606a276d2f0 2024-09-16T01:39:52.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3a7d7fa640 1 -- 172.21.15.139:0/209260838 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f3a8c012050 con 0x5606a276d2f0 2024-09-16T01:39:52.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3aa20bb740 10 client.15144 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:39:52.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.184+0000 7f3aa20bb740 10 client.15144 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T01:39:52.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.185+0000 7f3aa20bb740 20 client.15144 populate_metadata read hostname 'smithi139' 2024-09-16T01:39:52.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.185+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/209260838 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606a27884f0 0x5606a2789960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.185+0000 7f3aa20bb740 4 client.15144 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:39:52.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.185+0000 7f3aa20bb740 1 -- 172.21.15.139:0/209260838 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606a26e0600 con 0x5606a27884f0 2024-09-16T01:39:52.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.185+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/209260838 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606a27884f0 0x5606a2789960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.185+0000 7f3a7d7fa640 10 client.15144 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:39:52.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.185+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/209260838 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606a27884f0 0x5606a2789960 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e503d9d0ab9664d1 server_cookie=1b0fb5195c36c348 in_seq=0 out_seq=0 2024-09-16T01:39:52.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3a7d7fa640 1 -- 172.21.15.139:0/209260838 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606a26e0600 con 0x5606a27884f0 2024-09-16T01:39:52.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3a7d7fa640 10 client.15144 handle_command_reply: tid=0 2024-09-16T01:39:52.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3aa20bb740 1 client.15144 shutdown 2024-09-16T01:39:52.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3aa20bb740 1 -- 172.21.15.139:0/209260838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276d2f0 msgr2=0x5606a27747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:52.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/209260838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276d2f0 0x5606a27747b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00e170 tx=0x7f3a8c00e570 comp rx=0 tx=0).stop 2024-09-16T01:39:52.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3aa20bb740 1 -- 172.21.15.139:0/209260838 shutdown_connections 2024-09-16T01:39:52.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/209260838 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606a27884f0 0x5606a2789960 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/209260838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 0x5606a2774cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3a7d7fa640 0 client.15144 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:39:52.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/209260838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276d2f0 0x5606a27747b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/209260838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276c9d0 0x5606a2777460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3aa20bb740 1 -- 172.21.15.139:0/209260838 >> 172.21.15.139:0/209260838 conn(0x5606a2768f60 msgr2=0x5606a276ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:52.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3aa20bb740 1 -- 172.21.15.139:0/209260838 shutdown_connections 2024-09-16T01:39:52.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3aa20bb740 1 -- 172.21.15.139:0/209260838 wait complete. 2024-09-16T01:39:52.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.186+0000 7f3aa20bb740 20 client.15144 trim_cache size 0 max 16384 2024-09-16T01:39:52.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3aa20bb740 1 Processor -- start 2024-09-16T01:39:52.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3aa20bb740 1 -- start start 2024-09-16T01:39:52.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3aa20bb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276c9d0 0x5606a2783590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3aa20bb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276d2f0 0x5606a27872f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3aa20bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 0x5606a2791c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3aa20bb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5606a2787ca0 con 0x5606a276c9d0 2024-09-16T01:39:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3aa20bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5606a2788370 con 0x5606a2777bb0 2024-09-16T01:39:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3aa20bb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5606a2788a40 con 0x5606a276d2f0 2024-09-16T01:39:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3a9da7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276c9d0 0x5606a2783590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3a9da7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276c9d0 0x5606a2783590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43944/0 (socket says 172.21.15.139:43944) 2024-09-16T01:39:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3a9da7e640 1 -- 172.21.15.139:0/256290919 learned_addr learned my addr 172.21.15.139:0/256290919 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3a9e27f640 1 --2- 172.21.15.139:0/256290919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 0x5606a2791c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3a9d27d640 1 --2- 172.21.15.139:0/256290919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276d2f0 0x5606a27872f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3a9da7e640 1 -- 172.21.15.139:0/256290919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276d2f0 msgr2=0x5606a27872f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/256290919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276d2f0 0x5606a27872f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3a9da7e640 1 -- 172.21.15.139:0/256290919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 msgr2=0x5606a2791c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/256290919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 0x5606a2791c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3a9da7e640 1 -- 172.21.15.139:0/256290919 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606a2769c50 con 0x5606a276c9d0 2024-09-16T01:39:52.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.204+0000 7f3a9e27f640 1 --2- 172.21.15.139:0/256290919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 0x5606a2791c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:52.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.205+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/256290919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276c9d0 0x5606a2783590 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3a900bc280 tx=0x7f3a9003b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ab8672bd63272e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:52.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.205+0000 7f3a7f7fe640 1 -- 172.21.15.139:0/256290919 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90049040 con 0x5606a276c9d0 2024-09-16T01:39:52.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.205+0000 7f3a7f7fe640 1 -- 172.21.15.139:0/256290919 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a900bcc80 con 0x5606a276c9d0 2024-09-16T01:39:52.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.205+0000 7f3a7f7fe640 1 -- 172.21.15.139:0/256290919 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a900edcc0 con 0x5606a276c9d0 2024-09-16T01:39:52.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.205+0000 7f3aa20bb740 1 -- 172.21.15.139:0/256290919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276c9d0 msgr2=0x5606a2783590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:52.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.205+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/256290919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276c9d0 0x5606a2783590 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3a900bc280 tx=0x7f3a9003b800 comp rx=0 tx=0).stop 2024-09-16T01:39:52.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.205+0000 7f3aa20bb740 1 -- 172.21.15.139:0/256290919 shutdown_connections 2024-09-16T01:39:52.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.205+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/256290919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 0x5606a2791c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.205+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/256290919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276d2f0 0x5606a27872f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.205+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/256290919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276c9d0 0x5606a2783590 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.205+0000 7f3aa20bb740 1 -- 172.21.15.139:0/256290919 >> 172.21.15.139:0/256290919 conn(0x5606a2768f60 msgr2=0x5606a2773770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:52.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.205+0000 7f3aa20bb740 1 -- 172.21.15.139:0/256290919 shutdown_connections 2024-09-16T01:39:52.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3aa20bb740 1 -- 172.21.15.139:0/256290919 wait complete. 2024-09-16T01:39:52.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3aa20bb740 1 -- start start 2024-09-16T01:39:52.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3aa20bb740 1 Processor -- start 2024-09-16T01:39:52.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3aa20bb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276c9d0 0x5606a278cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3aa20bb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276d2f0 0x5606a278d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3aa20bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 0x5606a27f0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3aa20bb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5606a27897e0 con 0x5606a276c9d0 2024-09-16T01:39:52.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3aa20bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5606a2789110 con 0x5606a2777bb0 2024-09-16T01:39:52.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3aa20bb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5606a2789eb0 con 0x5606a276d2f0 2024-09-16T01:39:52.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3a9da7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276c9d0 0x5606a278cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3aa20bb740 10 client.0 _ll_register_callbacks cb 0x7f3a8c009180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:39:52.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3a9da7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276c9d0 0x5606a278cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43950/0 (socket says 172.21.15.139:43950) 2024-09-16T01:39:52.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3a9da7e640 1 -- 172.21.15.139:0/3968939890 learned_addr learned my addr 172.21.15.139:0/3968939890 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:52.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3a9d27d640 1 --2- 172.21.15.139:0/3968939890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276d2f0 0x5606a278d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.206+0000 7f3a9e27f640 1 --2- 172.21.15.139:0/3968939890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 0x5606a27f0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3a9da7e640 1 -- 172.21.15.139:0/3968939890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276d2f0 msgr2=0x5606a278d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:52.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/3968939890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276d2f0 0x5606a278d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3a9da7e640 1 -- 172.21.15.139:0/3968939890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 msgr2=0x5606a27f0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:39:52.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/3968939890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 0x5606a27f0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3a9da7e640 1 -- 172.21.15.139:0/3968939890 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606a27f1130 con 0x5606a276c9d0 2024-09-16T01:39:52.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3a9da7e640 1 --2- 172.21.15.139:0/3968939890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276c9d0 0x5606a278cfc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3a90000c00 tx=0x7f3a90003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef34461ae50ae33 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:52.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3a7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:39:52.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3a7d7fa640 1 -- 172.21.15.139:0/3968939890 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90049040 con 0x5606a276c9d0 2024-09-16T01:39:52.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3a7d7fa640 1 -- 172.21.15.139:0/3968939890 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f3a900edcc0 con 0x5606a276c9d0 2024-09-16T01:39:52.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3aa20bb740 1 -- 172.21.15.139:0/3968939890 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a900bcc80 con 0x5606a276c9d0 2024-09-16T01:39:52.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3a7d7fa640 1 -- 172.21.15.139:0/3968939890 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3a900c51f0 con 0x5606a276c9d0 2024-09-16T01:39:52.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3aa20bb740 10 client.15150 fetch_fsmap learned FSMap version 28 2024-09-16T01:39:52.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3aa20bb740 1 -- 172.21.15.139:0/3968939890 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5606a27f23b0 con 0x5606a276c9d0 2024-09-16T01:39:52.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3a7d7fa640 1 -- 172.21.15.139:0/3968939890 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a900bcc80 con 0x5606a276c9d0 2024-09-16T01:39:52.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.207+0000 7f3a7d7fa640 1 -- 172.21.15.139:0/3968939890 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f3a900bc7e0 con 0x5606a276c9d0 2024-09-16T01:39:52.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.208+0000 7f3aa20bb740 10 client.15150 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:39:52.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.208+0000 7f3aa20bb740 10 client.15150 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T01:39:52.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.208+0000 7f3aa20bb740 20 client.15150 populate_metadata read hostname 'smithi139' 2024-09-16T01:39:52.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.208+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/3968939890 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606a27f6b40 0x5606a27f5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:52.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.208+0000 7f3aa20bb740 4 client.15150 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:39:52.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.208+0000 7f3aa20bb740 1 -- 172.21.15.139:0/3968939890 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "status"}) -- 0x5606a26dab10 con 0x5606a27f6b40 2024-09-16T01:39:52.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.208+0000 7f3a9d27d640 1 --2- 172.21.15.139:0/3968939890 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606a27f6b40 0x5606a27f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:52.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.208+0000 7f3a9d27d640 1 --2- 172.21.15.139:0/3968939890 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606a27f6b40 0x5606a27f5850 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cdd0cb09565b46e server_cookie=480697db319e46c8 in_seq=0 out_seq=0 2024-09-16T01:39:52.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.208+0000 7f3a7d7fa640 10 client.15150 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:39:52.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.208+0000 7f3a7d7fa640 1 -- 172.21.15.139:0/3968939890 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5606a26dab10 con 0x5606a27f6b40 2024-09-16T01:39:52.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.208+0000 7f3a7d7fa640 10 client.15150 handle_command_reply: tid=0 2024-09-16T01:39:52.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.208+0000 7f3aa20bb740 1 client.15150 shutdown 2024-09-16T01:39:52.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3aa20bb740 1 -- 172.21.15.139:0/3968939890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276c9d0 msgr2=0x5606a278cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:52.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/3968939890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276c9d0 0x5606a278cfc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3a90000c00 tx=0x7f3a90003f10 comp rx=0 tx=0).stop 2024-09-16T01:39:52.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3aa20bb740 1 -- 172.21.15.139:0/3968939890 shutdown_connections 2024-09-16T01:39:52.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/3968939890 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606a27f6b40 0x5606a27f5850 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/3968939890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606a2777bb0 0x5606a27f0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/3968939890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606a276d2f0 0x5606a278d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3a7d7fa640 0 client.15150 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:39:52.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3aa20bb740 1 --2- 172.21.15.139:0/3968939890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606a276c9d0 0x5606a278cfc0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3aa20bb740 1 -- 172.21.15.139:0/3968939890 >> 172.21.15.139:0/3968939890 conn(0x5606a2768f60 msgr2=0x5606a2772f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:52.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3aa20bb740 1 -- 172.21.15.139:0/3968939890 shutdown_connections 2024-09-16T01:39:52.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3aa20bb740 1 -- 172.21.15.139:0/3968939890 wait complete. 2024-09-16T01:39:52.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3aa20bb740 20 client.15150 trim_cache size 0 max 16384 2024-09-16T01:39:52.217 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:39:52.217 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:39:52.217 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-09-16T01:39:52.217 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24470, 2024-09-16T01:39:52.217 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:39:52.217 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:39:52.217 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:39:52.217 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 109.69541626, 2024-09-16T01:39:52.217 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 28, 2024-09-16T01:39:52.217 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 85, 2024-09-16T01:39:52.218 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 83, 2024-09-16T01:39:52.218 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 121.754864514 2024-09-16T01:39:52.218 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:39:52.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3a9fce2640 1 -- 172.21.15.139:0/2066506724 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3a800a32d0 msgr2=0x7f3a800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:52.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3a9fce2640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3a800a32d0 0x7f3a800a5790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800e820 tx=0x7f3a880096c0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3a9fce2640 1 -- 172.21.15.139:0/2066506724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a98101de0 msgr2=0x7f3a981985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:52.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3a9fce2640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a98101de0 0x7f3a981985b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3a90005ab0 tx=0x7f3a900042f0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3a9fce2640 1 -- 172.21.15.139:0/2066506724 shutdown_connections 2024-09-16T01:39:52.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3a9fce2640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f3a800a32d0 0x7f3a800a5790 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3a9fce2640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a98106f10 0x7f3a9819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3a9fce2640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a98102790 0x7f3a98198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3a9fce2640 1 --2- 172.21.15.139:0/2066506724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a98101de0 0x7f3a981985b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:52.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.209+0000 7f3a9fce2640 1 -- 172.21.15.139:0/2066506724 >> 172.21.15.139:0/2066506724 conn(0x7f3a980fd8c0 msgr2=0x7f3a980fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:52.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.210+0000 7f3a9fce2640 1 -- 172.21.15.139:0/2066506724 shutdown_connections 2024-09-16T01:39:52.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:52.210+0000 7f3a9fce2640 1 -- 172.21.15.139:0/2066506724 wait complete. 2024-09-16T01:39:52.856 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:52 smithi139 ceph-mon[31175]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-16T01:39:52.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:52 smithi176 ceph-mon[35324]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-16T01:39:52.963 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T01:39:52.963 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T01:39:52.973 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T01:39:52.973 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-16T01:39:53.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:52 smithi164 ceph-mon[35121]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-16T01:39:53.219 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:53.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.752+0000 7f7fe1980640 1 -- 172.21.15.139:0/1234288441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fdc101d10 msgr2=0x7f7fdc102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:53.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.752+0000 7f7fe1980640 1 --2- 172.21.15.139:0/1234288441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fdc101d10 0x7f7fdc102190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009900 tx=0x7f7fc402a410 comp rx=0 tx=0).stop 2024-09-16T01:39:53.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.752+0000 7f7fe1980640 1 -- 172.21.15.139:0/1234288441 shutdown_connections 2024-09-16T01:39:53.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.752+0000 7f7fe1980640 1 --2- 172.21.15.139:0/1234288441 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fdc1026d0 0x7f7fdc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:53.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.752+0000 7f7fe1980640 1 --2- 172.21.15.139:0/1234288441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fdc101d10 0x7f7fdc102190 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:53.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.752+0000 7f7fe1980640 1 --2- 172.21.15.139:0/1234288441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fdc100b10 0x7f7fdc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:53.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.752+0000 7f7fe1980640 1 -- 172.21.15.139:0/1234288441 >> 172.21.15.139:0/1234288441 conn(0x7f7fdc0fc2a0 msgr2=0x7f7fdc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:53.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.752+0000 7f7fe1980640 1 -- 172.21.15.139:0/1234288441 shutdown_connections 2024-09-16T01:39:53.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.752+0000 7f7fe1980640 1 -- 172.21.15.139:0/1234288441 wait complete. 2024-09-16T01:39:53.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fe1980640 1 Processor -- start 2024-09-16T01:39:53.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fe1980640 1 -- start start 2024-09-16T01:39:53.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fe1980640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fdc100b10 0x7f7fdc197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:53.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fe1980640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fdc101d10 0x7f7fdc198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:53.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fe1980640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fdc1026d0 0x7f7fdc19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:53.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fe1980640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10c790 con 0x7f7fdc100b10 2024-09-16T01:39:53.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fe1980640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10c610 con 0x7f7fdc1026d0 2024-09-16T01:39:53.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fe1980640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10c910 con 0x7f7fdc101d10 2024-09-16T01:39:53.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fdaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fdc100b10 0x7f7fdc197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:53.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fdaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fdc100b10 0x7f7fdc197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37024/0 (socket says 172.21.15.139:37024) 2024-09-16T01:39:53.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fdaffd640 1 -- 172.21.15.139:0/1882471924 learned_addr learned my addr 172.21.15.139:0/1882471924 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:53.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fd3fff640 1 --2- 172.21.15.139:0/1882471924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fdc101d10 0x7f7fdc198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:53.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fdb7fe640 1 --2- 172.21.15.139:0/1882471924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fdc1026d0 0x7f7fdc19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:53.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fdaffd640 1 -- 172.21.15.139:0/1882471924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fdc101d10 msgr2=0x7f7fdc198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:53.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fdaffd640 1 --2- 172.21.15.139:0/1882471924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fdc101d10 0x7f7fdc198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:53.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fdaffd640 1 -- 172.21.15.139:0/1882471924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fdc1026d0 msgr2=0x7f7fdc19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:53.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fdaffd640 1 --2- 172.21.15.139:0/1882471924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fdc1026d0 0x7f7fdc19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:53.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fdaffd640 1 -- 172.21.15.139:0/1882471924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fdc19fb10 con 0x7f7fdc100b10 2024-09-16T01:39:53.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.753+0000 7f7fdaffd640 1 --2- 172.21.15.139:0/1882471924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fdc100b10 0x7f7fdc197ee0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00d950 tx=0x7f7fcc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:53.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.754+0000 7f7fe097e640 1 -- 172.21.15.139:0/1882471924 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc017020 con 0x7f7fdc100b10 2024-09-16T01:39:53.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.754+0000 7f7fe097e640 1 -- 172.21.15.139:0/1882471924 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc009c30 con 0x7f7fdc100b10 2024-09-16T01:39:53.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.754+0000 7f7fe097e640 1 -- 172.21.15.139:0/1882471924 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc00f740 con 0x7f7fdc100b10 2024-09-16T01:39:53.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.754+0000 7f7fe1980640 1 -- 172.21.15.139:0/1882471924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fdc19fe00 con 0x7f7fdc100b10 2024-09-16T01:39:53.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.754+0000 7f7fe1980640 1 -- 172.21.15.139:0/1882471924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fdc1a0260 con 0x7f7fdc100b10 2024-09-16T01:39:53.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.755+0000 7f7fe1980640 1 -- 172.21.15.139:0/1882471924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fdc1a04f0 con 0x7f7fdc100b10 2024-09-16T01:39:53.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.756+0000 7f7fe097e640 1 -- 172.21.15.139:0/1882471924 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7fcc01a020 con 0x7f7fdc100b10 2024-09-16T01:39:53.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.757+0000 7f7fe097e640 1 --2- 172.21.15.139:0/1882471924 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7fc80a3380 0x7f7fc80a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:53.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.757+0000 7f7fd3fff640 1 --2- 172.21.15.139:0/1882471924 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7fc80a3380 0x7f7fc80a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:53.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.757+0000 7f7fe097e640 1 -- 172.21.15.139:0/1882471924 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f7fcc0cfbe0 con 0x7f7fdc100b10 2024-09-16T01:39:53.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.758+0000 7f7fd3fff640 1 --2- 172.21.15.139:0/1882471924 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7fc80a3380 0x7f7fc80a5840 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4000c00 tx=0x7f7fc40315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:53.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.760+0000 7f7fe097e640 1 -- 172.21.15.139:0/1882471924 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7fcc021070 con 0x7f7fdc100b10 2024-09-16T01:39:53.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.945+0000 7f7fe1980640 1 -- 172.21.15.139:0/1882471924 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f7fdc1a06c0 con 0x7f7fc80a3380 2024-09-16T01:39:53.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.946+0000 7f7fe097e640 1 -- 172.21.15.139:0/1882471924 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f7fdc1a06c0 con 0x7f7fc80a3380 2024-09-16T01:39:53.947 INFO:teuthology.orchestra.run.smithi139.stdout:Backend: cephadm 2024-09-16T01:39:53.947 INFO:teuthology.orchestra.run.smithi139.stdout:Available: Yes 2024-09-16T01:39:53.947 INFO:teuthology.orchestra.run.smithi139.stdout:Paused: No 2024-09-16T01:39:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.948+0000 7f7fe1980640 1 -- 172.21.15.139:0/1882471924 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7fc80a3380 msgr2=0x7f7fc80a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.948+0000 7f7fe1980640 1 --2- 172.21.15.139:0/1882471924 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7fc80a3380 0x7f7fc80a5840 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4000c00 tx=0x7f7fc40315d0 comp rx=0 tx=0).stop 2024-09-16T01:39:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.948+0000 7f7fe1980640 1 -- 172.21.15.139:0/1882471924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fdc100b10 msgr2=0x7f7fdc197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.948+0000 7f7fe1980640 1 --2- 172.21.15.139:0/1882471924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fdc100b10 0x7f7fdc197ee0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00d950 tx=0x7f7fcc00de10 comp rx=0 tx=0).stop 2024-09-16T01:39:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.948+0000 7f7fe1980640 1 -- 172.21.15.139:0/1882471924 shutdown_connections 2024-09-16T01:39:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.948+0000 7f7fe1980640 1 --2- 172.21.15.139:0/1882471924 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7fc80a3380 0x7f7fc80a5840 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.948+0000 7f7fe1980640 1 --2- 172.21.15.139:0/1882471924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fdc1026d0 0x7f7fdc19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:53.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.948+0000 7f7fe1980640 1 --2- 172.21.15.139:0/1882471924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fdc101d10 0x7f7fdc198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:53.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.948+0000 7f7fe1980640 1 --2- 172.21.15.139:0/1882471924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fdc100b10 0x7f7fdc197ee0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:53.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.948+0000 7f7fe1980640 1 -- 172.21.15.139:0/1882471924 >> 172.21.15.139:0/1882471924 conn(0x7f7fdc0fc2a0 msgr2=0x7f7fdc06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:53.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.948+0000 7f7fe1980640 1 -- 172.21.15.139:0/1882471924 shutdown_connections 2024-09-16T01:39:53.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:53.948+0000 7f7fe1980640 1 -- 172.21.15.139:0/1882471924 wait complete. 2024-09-16T01:39:54.631 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-16T01:39:54.879 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:54.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:54 smithi139 ceph-mon[31175]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:39:54.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:54 smithi176 ceph-mon[35324]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:39:55.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:54 smithi164 ceph-mon[35121]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:39:55.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.335+0000 7ff1620a7640 1 -- 172.21.15.139:0/358378165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15c076220 msgr2=0x7ff15c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:55.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.335+0000 7ff1620a7640 1 --2- 172.21.15.139:0/358378165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15c076220 0x7ff15c074680 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff14c009900 tx=0x7ff14c02a410 comp rx=0 tx=0).stop 2024-09-16T01:39:55.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.335+0000 7ff1620a7640 1 -- 172.21.15.139:0/358378165 shutdown_connections 2024-09-16T01:39:55.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.335+0000 7ff1620a7640 1 --2- 172.21.15.139:0/358378165 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff15c10e1e0 0x7ff15c1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:55.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.335+0000 7ff1620a7640 1 --2- 172.21.15.139:0/358378165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff15c074c50 0x7ff15c0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:55.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.335+0000 7ff1620a7640 1 --2- 172.21.15.139:0/358378165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15c076220 0x7ff15c074680 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:55.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.335+0000 7ff1620a7640 1 -- 172.21.15.139:0/358378165 >> 172.21.15.139:0/358378165 conn(0x7ff15c0fd390 msgr2=0x7ff15c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:55.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.335+0000 7ff1620a7640 1 -- 172.21.15.139:0/358378165 shutdown_connections 2024-09-16T01:39:55.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.335+0000 7ff1620a7640 1 -- 172.21.15.139:0/358378165 wait complete. 2024-09-16T01:39:55.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff1620a7640 1 Processor -- start 2024-09-16T01:39:55.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff1620a7640 1 -- start start 2024-09-16T01:39:55.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff1620a7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff15c074c50 0x7ff15c19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:55.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff1620a7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff15c076220 0x7ff15c19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:55.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff1620a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15c10e1e0 0x7ff15c1a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:55.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff1620a7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c113ee0 con 0x7ff15c10e1e0 2024-09-16T01:39:55.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff1620a7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c113d60 con 0x7ff15c076220 2024-09-16T01:39:55.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff1620a7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c114060 con 0x7ff15c074c50 2024-09-16T01:39:55.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff15bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15c10e1e0 0x7ff15c1a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:55.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff15bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15c10e1e0 0x7ff15c1a5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37048/0 (socket says 172.21.15.139:37048) 2024-09-16T01:39:55.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff15b7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff15c074c50 0x7ff15c19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:55.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff15bfff640 1 -- 172.21.15.139:0/981724793 learned_addr learned my addr 172.21.15.139:0/981724793 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:55.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff15affd640 1 --2- 172.21.15.139:0/981724793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff15c076220 0x7ff15c19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:55.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff15bfff640 1 -- 172.21.15.139:0/981724793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff15c074c50 msgr2=0x7ff15c19e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:55.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff15bfff640 1 --2- 172.21.15.139:0/981724793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff15c074c50 0x7ff15c19e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:55.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff15bfff640 1 -- 172.21.15.139:0/981724793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff15c076220 msgr2=0x7ff15c19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:55.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff15bfff640 1 --2- 172.21.15.139:0/981724793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff15c076220 0x7ff15c19eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:55.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.336+0000 7ff15bfff640 1 -- 172.21.15.139:0/981724793 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15c1a61a0 con 0x7ff15c10e1e0 2024-09-16T01:39:55.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.337+0000 7ff15bfff640 1 --2- 172.21.15.139:0/981724793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15c10e1e0 0x7ff15c1a5b30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff15400b970 tx=0x7ff15400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:55.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.337+0000 7ff158ff9640 1 -- 172.21.15.139:0/981724793 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154019070 con 0x7ff15c10e1e0 2024-09-16T01:39:55.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.337+0000 7ff1620a7640 1 -- 172.21.15.139:0/981724793 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15c1a6490 con 0x7ff15c10e1e0 2024-09-16T01:39:55.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.337+0000 7ff158ff9640 1 -- 172.21.15.139:0/981724793 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff15400cb40 con 0x7ff15c10e1e0 2024-09-16T01:39:55.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.337+0000 7ff158ff9640 1 -- 172.21.15.139:0/981724793 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154007650 con 0x7ff15c10e1e0 2024-09-16T01:39:55.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.337+0000 7ff1620a7640 1 -- 172.21.15.139:0/981724793 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff15c1a6a50 con 0x7ff15c10e1e0 2024-09-16T01:39:55.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.338+0000 7ff1620a7640 1 -- 172.21.15.139:0/981724793 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff15c104290 con 0x7ff15c10e1e0 2024-09-16T01:39:55.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.339+0000 7ff158ff9640 1 -- 172.21.15.139:0/981724793 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff15401c070 con 0x7ff15c10e1e0 2024-09-16T01:39:55.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.339+0000 7ff158ff9640 1 --2- 172.21.15.139:0/981724793 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff1440a3300 0x7ff1440a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:55.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.340+0000 7ff15b7fe640 1 --2- 172.21.15.139:0/981724793 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff1440a3300 0x7ff1440a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:55.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.340+0000 7ff158ff9640 1 -- 172.21.15.139:0/981724793 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7ff1540cf280 con 0x7ff15c10e1e0 2024-09-16T01:39:55.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.340+0000 7ff15b7fe640 1 --2- 172.21.15.139:0/981724793 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff1440a3300 0x7ff1440a57c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff14c005ab0 tx=0x7ff14c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:55.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.343+0000 7ff158ff9640 1 -- 172.21.15.139:0/981724793 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff15409b530 con 0x7ff15c10e1e0 2024-09-16T01:39:55.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.519+0000 7ff1620a7640 1 -- 172.21.15.139:0/981724793 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff15c062630 con 0x7ff1440a3300 2024-09-16T01:39:55.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.532+0000 7ff158ff9640 1 -- 172.21.15.139:0/981724793 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7ff15c062630 con 0x7ff1440a3300 2024-09-16T01:39:55.533 INFO:teuthology.orchestra.run.smithi139.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T01:39:55.533 INFO:teuthology.orchestra.run.smithi139.stdout:mds.a smithi139 running (2m) 57s ago 2m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 73bf12d5392b 2024-09-16T01:39:55.533 INFO:teuthology.orchestra.run.smithi139.stdout:mds.b smithi164 running (2m) 58s ago 2m 26.7M - 19.1.1-326-g8921c8ef f1455bd0219c f7cd4f522d4f 2024-09-16T01:39:55.533 INFO:teuthology.orchestra.run.smithi139.stdout:mds.c smithi176 running (2m) 58s ago 2m 19.6M - 19.1.1-326-g8921c8ef f1455bd0219c e845a934580c 2024-09-16T01:39:55.533 INFO:teuthology.orchestra.run.smithi139.stdout:mds.d smithi139 running (2m) 57s ago 2m 16.6M - 19.1.1-326-g8921c8ef f1455bd0219c ef25e7c09c08 2024-09-16T01:39:55.533 INFO:teuthology.orchestra.run.smithi139.stdout:mds.e smithi164 running (2m) 58s ago 2m 18.9M - 19.1.1-326-g8921c8ef f1455bd0219c d9cc4c9246da 2024-09-16T01:39:55.533 INFO:teuthology.orchestra.run.smithi139.stdout:mds.f smithi176 running (2m) 58s ago 2m 16.4M - 19.1.1-326-g8921c8ef f1455bd0219c 5b773f3b0981 2024-09-16T01:39:55.533 INFO:teuthology.orchestra.run.smithi139.stdout:mds.g smithi139 running (2m) 57s ago 2m 18.0M - 19.1.1-326-g8921c8ef f1455bd0219c 3e1b0cfb7c3c 2024-09-16T01:39:55.533 INFO:teuthology.orchestra.run.smithi139.stdout:mds.h smithi164 running (2m) 58s ago 2m 17.5M - 19.1.1-326-g8921c8ef f1455bd0219c c0b5c2be9e17 2024-09-16T01:39:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:mds.i smithi176 running (2m) 58s ago 2m 18.9M - 19.1.1-326-g8921c8ef f1455bd0219c 5acde05ee655 2024-09-16T01:39:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:mds.j smithi139 running (2m) 57s ago 2m 15.9M - 19.1.1-326-g8921c8ef f1455bd0219c 2f7c0438797e 2024-09-16T01:39:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:mds.k smithi164 running (2m) 58s ago 2m 15.8M - 19.1.1-326-g8921c8ef f1455bd0219c e666239c623e 2024-09-16T01:39:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:mds.l smithi176 running (2m) 58s ago 2m 15.0M - 19.1.1-326-g8921c8ef f1455bd0219c 7225b47e96d7 2024-09-16T01:39:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:mgr.x smithi139 *:9283,8765 running (9m) 57s ago 9m 510M - 19.1.1-326-g8921c8ef f1455bd0219c 6c955bee72d8 2024-09-16T01:39:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:mgr.y smithi164 *:8443,8765 running (7m) 58s ago 7m 443M - 19.1.1-326-g8921c8ef f1455bd0219c bc3233dc2909 2024-09-16T01:39:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:mgr.z smithi176 *:8443,8765 running (7m) 58s ago 7m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 48120aef7454 2024-09-16T01:39:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:mon.a smithi139 running (9m) 57s ago 9m 50.1M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 9a9750ef87b6 2024-09-16T01:39:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:mon.b smithi164 running (8m) 58s ago 8m 41.1M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 05e60772d0f7 2024-09-16T01:39:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:mon.c smithi176 running (8m) 58s ago 8m 41.6M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 33fe964f1df3 2024-09-16T01:39:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0 smithi139 running (7m) 57s ago 7m 57.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 1773d18d1250 2024-09-16T01:39:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1 smithi164 running (7m) 58s ago 7m 60.9M 4272M 19.1.1-326-g8921c8ef f1455bd0219c a8ebe4ceeb63 2024-09-16T01:39:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2 smithi176 running (6m) 58s ago 6m 60.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 4dc399cdb899 2024-09-16T01:39:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3 smithi139 running (6m) 57s ago 6m 47.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 0a9d58b5157c 2024-09-16T01:39:55.535 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4 smithi164 running (5m) 58s ago 5m 55.7M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 5211890e6a69 2024-09-16T01:39:55.535 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5 smithi176 running (5m) 58s ago 5m 44.7M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 23589ee43667 2024-09-16T01:39:55.535 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6 smithi139 running (5m) 57s ago 5m 53.4M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 87c1d82250ac 2024-09-16T01:39:55.535 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7 smithi164 running (4m) 58s ago 4m 55.1M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 87604f5cf10e 2024-09-16T01:39:55.535 INFO:teuthology.orchestra.run.smithi139.stdout:osd.8 smithi176 running (4m) 58s ago 4m 43.8M 4272M 19.1.1-326-g8921c8ef f1455bd0219c a26a3d4316f9 2024-09-16T01:39:55.535 INFO:teuthology.orchestra.run.smithi139.stdout:osd.9 smithi139 running (3m) 57s ago 3m 42.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 3879fd37f965 2024-09-16T01:39:55.535 INFO:teuthology.orchestra.run.smithi139.stdout:osd.10 smithi164 running (3m) 58s ago 3m 55.1M 4272M 19.1.1-326-g8921c8ef f1455bd0219c ef6ac774dcea 2024-09-16T01:39:55.535 INFO:teuthology.orchestra.run.smithi139.stdout:osd.11 smithi176 running (2m) 58s ago 2m 53.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 1f9d6dc4cc16 2024-09-16T01:39:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.534+0000 7ff1620a7640 1 -- 172.21.15.139:0/981724793 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff1440a3300 msgr2=0x7ff1440a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.534+0000 7ff1620a7640 1 --2- 172.21.15.139:0/981724793 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff1440a3300 0x7ff1440a57c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff14c005ab0 tx=0x7ff14c000bc0 comp rx=0 tx=0).stop 2024-09-16T01:39:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.534+0000 7ff1620a7640 1 -- 172.21.15.139:0/981724793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15c10e1e0 msgr2=0x7ff15c1a5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.534+0000 7ff1620a7640 1 --2- 172.21.15.139:0/981724793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15c10e1e0 0x7ff15c1a5b30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff15400b970 tx=0x7ff15400be30 comp rx=0 tx=0).stop 2024-09-16T01:39:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.534+0000 7ff1620a7640 1 -- 172.21.15.139:0/981724793 shutdown_connections 2024-09-16T01:39:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.534+0000 7ff1620a7640 1 --2- 172.21.15.139:0/981724793 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff1440a3300 0x7ff1440a57c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.534+0000 7ff1620a7640 1 --2- 172.21.15.139:0/981724793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15c10e1e0 0x7ff15c1a5b30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.534+0000 7ff1620a7640 1 --2- 172.21.15.139:0/981724793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff15c076220 0x7ff15c19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.534+0000 7ff1620a7640 1 --2- 172.21.15.139:0/981724793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff15c074c50 0x7ff15c19e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.534+0000 7ff1620a7640 1 -- 172.21.15.139:0/981724793 >> 172.21.15.139:0/981724793 conn(0x7ff15c0fd390 msgr2=0x7ff15c101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:55.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.535+0000 7ff1620a7640 1 -- 172.21.15.139:0/981724793 shutdown_connections 2024-09-16T01:39:55.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:55.535+0000 7ff1620a7640 1 -- 172.21.15.139:0/981724793 wait complete. 2024-09-16T01:39:55.877 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:55 smithi139 ceph-mon[31175]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:39:55.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:55 smithi176 ceph-mon[35324]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:39:56.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:55 smithi164 ceph-mon[35121]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:39:56.375 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-16T01:39:56.623 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:56.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:56 smithi139 ceph-mon[31175]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:39:56.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:56 smithi139 ceph-mon[31175]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:39:56.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:56 smithi176 ceph-mon[35324]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:39:56.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:56 smithi176 ceph-mon[35324]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:39:57.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:56 smithi164 ceph-mon[35121]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:39:57.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:56 smithi164 ceph-mon[35121]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:39:57.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.103+0000 7f19fba63640 1 -- 172.21.15.139:0/4192151230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f41026d0 msgr2=0x7f19f4108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:57.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.103+0000 7f19fba63640 1 --2- 172.21.15.139:0/4192151230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f41026d0 0x7f19f4108f60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009ca0 tx=0x7f19ec02a8c0 comp rx=0 tx=0).stop 2024-09-16T01:39:57.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.103+0000 7f19fba63640 1 -- 172.21.15.139:0/4192151230 shutdown_connections 2024-09-16T01:39:57.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.103+0000 7f19fba63640 1 --2- 172.21.15.139:0/4192151230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f41026d0 0x7f19f4108f60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.103+0000 7f19fba63640 1 --2- 172.21.15.139:0/4192151230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19f4101d10 0x7f19f4102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.103+0000 7f19fba63640 1 --2- 172.21.15.139:0/4192151230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19f4100b10 0x7f19f4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.103+0000 7f19fba63640 1 -- 172.21.15.139:0/4192151230 >> 172.21.15.139:0/4192151230 conn(0x7f19f40fc2a0 msgr2=0x7f19f40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.103+0000 7f19fba63640 1 -- 172.21.15.139:0/4192151230 shutdown_connections 2024-09-16T01:39:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.103+0000 7f19fba63640 1 -- 172.21.15.139:0/4192151230 wait complete. 2024-09-16T01:39:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.103+0000 7f19fba63640 1 Processor -- start 2024-09-16T01:39:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fba63640 1 -- start start 2024-09-16T01:39:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fba63640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19f4100b10 0x7f19f419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fba63640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19f4101d10 0x7f19f419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fba63640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f41026d0 0x7f19f41a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fba63640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f19f410b700 con 0x7f19f41026d0 2024-09-16T01:39:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fba63640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f19f410b580 con 0x7f19f4100b10 2024-09-16T01:39:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fba63640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f19f410b880 con 0x7f19f4101d10 2024-09-16T01:39:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fa000640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f41026d0 0x7f19f41a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:57.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fa000640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f41026d0 0x7f19f41a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37068/0 (socket says 172.21.15.139:37068) 2024-09-16T01:39:57.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fa000640 1 -- 172.21.15.139:0/1647530091 learned_addr learned my addr 172.21.15.139:0/1647530091 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:57.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19f8ffe640 1 --2- 172.21.15.139:0/1647530091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19f4101d10 0x7f19f419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:57.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19f97ff640 1 --2- 172.21.15.139:0/1647530091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19f4100b10 0x7f19f419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:57.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fa000640 1 -- 172.21.15.139:0/1647530091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19f4101d10 msgr2=0x7f19f419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:57.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fa000640 1 --2- 172.21.15.139:0/1647530091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19f4101d10 0x7f19f419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:57.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fa000640 1 -- 172.21.15.139:0/1647530091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19f4100b10 msgr2=0x7f19f419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:57.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fa000640 1 --2- 172.21.15.139:0/1647530091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19f4100b10 0x7f19f419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:57.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fa000640 1 -- 172.21.15.139:0/1647530091 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f41a1cb0 con 0x7f19f41026d0 2024-09-16T01:39:57.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.104+0000 7f19fa000640 1 --2- 172.21.15.139:0/1647530091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f41026d0 0x7f19f41a1640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0098d0 tx=0x7f19ec02f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:57.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.105+0000 7f19ea7fc640 1 -- 172.21.15.139:0/1647530091 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec02fc30 con 0x7f19f41026d0 2024-09-16T01:39:57.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.105+0000 7f19fba63640 1 -- 172.21.15.139:0/1647530091 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f41a1f40 con 0x7f19f41026d0 2024-09-16T01:39:57.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.105+0000 7f19ea7fc640 1 -- 172.21.15.139:0/1647530091 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19ec003e50 con 0x7f19f41026d0 2024-09-16T01:39:57.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.105+0000 7f19ea7fc640 1 -- 172.21.15.139:0/1647530091 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec004810 con 0x7f19f41026d0 2024-09-16T01:39:57.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.105+0000 7f19fba63640 1 -- 172.21.15.139:0/1647530091 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f41a2470 con 0x7f19f41026d0 2024-09-16T01:39:57.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.106+0000 7f19fba63640 1 -- 172.21.15.139:0/1647530091 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f4107ec0 con 0x7f19f41026d0 2024-09-16T01:39:57.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.107+0000 7f19ea7fc640 1 -- 172.21.15.139:0/1647530091 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f19ec004b40 con 0x7f19f41026d0 2024-09-16T01:39:57.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.108+0000 7f19ea7fc640 1 --2- 172.21.15.139:0/1647530091 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f19e00a32f0 0x7f19e00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:57.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.108+0000 7f19f97ff640 1 --2- 172.21.15.139:0/1647530091 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f19e00a32f0 0x7f19e00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:57.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.108+0000 7f19ea7fc640 1 -- 172.21.15.139:0/1647530091 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f19ec0f12b0 con 0x7f19f41026d0 2024-09-16T01:39:57.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.108+0000 7f19f97ff640 1 --2- 172.21.15.139:0/1647530091 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f19e00a32f0 0x7f19e00a57b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f19f4101b70 tx=0x7f19f0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:57.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.111+0000 7f19ea7fc640 1 -- 172.21.15.139:0/1647530091 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19ec0bdb50 con 0x7f19f41026d0 2024-09-16T01:39:57.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.285+0000 7f19fba63640 1 -- 172.21.15.139:0/1647530091 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f19f4062630 con 0x7f19e00a32f0 2024-09-16T01:39:57.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.291+0000 7f19ea7fc640 1 -- 172.21.15.139:0/1647530091 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f19f4062630 con 0x7f19e00a32f0 2024-09-16T01:39:57.292 INFO:teuthology.orchestra.run.smithi139.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-16T01:39:57.292 INFO:teuthology.orchestra.run.smithi139.stdout:mds.all 12/12 60s ago 2m smithi139=a;smithi139=d;smithi139=g;smithi139=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l;count:12 2024-09-16T01:39:57.292 INFO:teuthology.orchestra.run.smithi139.stdout:mgr 3/3 60s ago 8m smithi139=x;smithi164=y;smithi176=z;count:3 2024-09-16T01:39:57.292 INFO:teuthology.orchestra.run.smithi139.stdout:mon 3/3 60s ago 8m smithi139:172.21.15.139=a;smithi164:172.21.15.164=b;smithi176:172.21.15.176=c;count:3 2024-09-16T01:39:57.292 INFO:teuthology.orchestra.run.smithi139.stdout:osd 12 60s ago - 2024-09-16T01:39:57.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.293+0000 7f19fba63640 1 -- 172.21.15.139:0/1647530091 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f19e00a32f0 msgr2=0x7f19e00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:57.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.293+0000 7f19fba63640 1 --2- 172.21.15.139:0/1647530091 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f19e00a32f0 0x7f19e00a57b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f19f4101b70 tx=0x7f19f0008040 comp rx=0 tx=0).stop 2024-09-16T01:39:57.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.293+0000 7f19fba63640 1 -- 172.21.15.139:0/1647530091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f41026d0 msgr2=0x7f19f41a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:57.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.293+0000 7f19fba63640 1 --2- 172.21.15.139:0/1647530091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f41026d0 0x7f19f41a1640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0098d0 tx=0x7f19ec02f840 comp rx=0 tx=0).stop 2024-09-16T01:39:57.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.293+0000 7f19fba63640 1 -- 172.21.15.139:0/1647530091 shutdown_connections 2024-09-16T01:39:57.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.293+0000 7f19fba63640 1 --2- 172.21.15.139:0/1647530091 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f19e00a32f0 0x7f19e00a57b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:57.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.293+0000 7f19fba63640 1 --2- 172.21.15.139:0/1647530091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f41026d0 0x7f19f41a1640 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:57.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.293+0000 7f19fba63640 1 --2- 172.21.15.139:0/1647530091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19f4101d10 0x7f19f419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:57.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.293+0000 7f19fba63640 1 --2- 172.21.15.139:0/1647530091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19f4100b10 0x7f19f419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:57.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.293+0000 7f19fba63640 1 -- 172.21.15.139:0/1647530091 >> 172.21.15.139:0/1647530091 conn(0x7f19f40fc2a0 msgr2=0x7f19f4105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:57.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.293+0000 7f19fba63640 1 -- 172.21.15.139:0/1647530091 shutdown_connections 2024-09-16T01:39:57.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:57.293+0000 7f19fba63640 1 -- 172.21.15.139:0/1647530091 wait complete. 2024-09-16T01:39:58.011 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-16T01:39:58.258 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:39:58.658 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:58 smithi139 ceph-mon[31175]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:39:58.658 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:58 smithi139 ceph-mon[31175]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:39:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.739+0000 7feda923b640 1 -- 172.21.15.139:0/2170612913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda4102e00 msgr2=0x7feda4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.739+0000 7feda923b640 1 --2- 172.21.15.139:0/2170612913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda4102e00 0x7feda4103280 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fed98009900 tx=0x7fed9802a410 comp rx=0 tx=0).stop 2024-09-16T01:39:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.739+0000 7feda923b640 1 -- 172.21.15.139:0/2170612913 shutdown_connections 2024-09-16T01:39:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.739+0000 7feda923b640 1 --2- 172.21.15.139:0/2170612913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feda41037c0 0x7feda410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.739+0000 7feda923b640 1 --2- 172.21.15.139:0/2170612913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda4102e00 0x7feda4103280 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.739+0000 7feda923b640 1 --2- 172.21.15.139:0/2170612913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feda4101c00 0x7feda4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.739+0000 7feda923b640 1 -- 172.21.15.139:0/2170612913 >> 172.21.15.139:0/2170612913 conn(0x7feda40fd390 msgr2=0x7feda40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.739+0000 7feda923b640 1 -- 172.21.15.139:0/2170612913 shutdown_connections 2024-09-16T01:39:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.739+0000 7feda923b640 1 -- 172.21.15.139:0/2170612913 wait complete. 2024-09-16T01:39:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.740+0000 7feda923b640 1 Processor -- start 2024-09-16T01:39:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.740+0000 7feda923b640 1 -- start start 2024-09-16T01:39:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.740+0000 7feda923b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feda4101c00 0x7feda4075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.740+0000 7feda923b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda4102e00 0x7feda40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.740+0000 7feda923b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feda41037c0 0x7feda4074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.740+0000 7feda923b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feda410d860 con 0x7feda4102e00 2024-09-16T01:39:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.740+0000 7feda923b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feda410d6e0 con 0x7feda41037c0 2024-09-16T01:39:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.740+0000 7feda923b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feda410d9e0 con 0x7feda4101c00 2024-09-16T01:39:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.740+0000 7feda259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda4102e00 0x7feda40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.740+0000 7feda259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda4102e00 0x7feda40745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37084/0 (socket says 172.21.15.139:37084) 2024-09-16T01:39:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.740+0000 7feda259c640 1 -- 172.21.15.139:0/2097777969 learned_addr learned my addr 172.21.15.139:0/2097777969 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:39:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.740+0000 7feda359e640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feda41037c0 0x7feda4074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.740+0000 7feda2d9d640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feda4101c00 0x7feda4075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.740+0000 7feda259c640 1 -- 172.21.15.139:0/2097777969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feda4101c00 msgr2=0x7feda4075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.741+0000 7feda259c640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feda4101c00 0x7feda4075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:58.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.741+0000 7feda259c640 1 -- 172.21.15.139:0/2097777969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feda41037c0 msgr2=0x7feda4074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:58.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.741+0000 7feda259c640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feda41037c0 0x7feda4074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:58.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.741+0000 7feda259c640 1 -- 172.21.15.139:0/2097777969 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda40753e0 con 0x7feda4102e00 2024-09-16T01:39:58.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.741+0000 7feda2d9d640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feda4101c00 0x7feda4075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:39:58.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.741+0000 7feda259c640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda4102e00 0x7feda40745e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fed98009230 tx=0x7fed98004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:58.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.741+0000 7fed8bfff640 1 -- 172.21.15.139:0/2097777969 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9803f030 con 0x7feda4102e00 2024-09-16T01:39:58.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.741+0000 7feda923b640 1 -- 172.21.15.139:0/2097777969 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda41a2070 con 0x7feda4102e00 2024-09-16T01:39:58.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.741+0000 7fed8bfff640 1 -- 172.21.15.139:0/2097777969 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed98003ce0 con 0x7feda4102e00 2024-09-16T01:39:58.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.742+0000 7fed8bfff640 1 -- 172.21.15.139:0/2097777969 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98035070 con 0x7feda4102e00 2024-09-16T01:39:58.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.742+0000 7feda923b640 1 -- 172.21.15.139:0/2097777969 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda41a2570 con 0x7feda4102e00 2024-09-16T01:39:58.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.743+0000 7feda923b640 1 -- 172.21.15.139:0/2097777969 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda4102000 con 0x7feda4102e00 2024-09-16T01:39:58.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.743+0000 7fed8bfff640 1 -- 172.21.15.139:0/2097777969 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fed98042030 con 0x7feda4102e00 2024-09-16T01:39:58.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.744+0000 7fed8bfff640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fed8c0a32d0 0x7fed8c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:39:58.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.744+0000 7feda2d9d640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fed8c0a32d0 0x7fed8c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:39:58.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.744+0000 7fed8bfff640 1 -- 172.21.15.139:0/2097777969 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fed980f0860 con 0x7feda4102e00 2024-09-16T01:39:58.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.744+0000 7feda2d9d640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fed8c0a32d0 0x7fed8c0a5790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fed940058c0 tx=0x7fed94005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:39:58.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.748+0000 7fed8bfff640 1 -- 172.21.15.139:0/2097777969 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed980bd050 con 0x7feda4102e00 2024-09-16T01:39:58.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.924+0000 7feda923b640 1 -- 172.21.15.139:0/2097777969 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7feda4062630 con 0x7fed8c0a32d0 2024-09-16T01:39:58.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.926+0000 7fed8bfff640 1 -- 172.21.15.139:0/2097777969 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7feda4062630 con 0x7fed8c0a32d0 2024-09-16T01:39:58.926 INFO:teuthology.orchestra.run.smithi139.stdout:HOST ADDR LABELS STATUS 2024-09-16T01:39:58.926 INFO:teuthology.orchestra.run.smithi139.stdout:smithi139 172.21.15.139 2024-09-16T01:39:58.927 INFO:teuthology.orchestra.run.smithi139.stdout:smithi164 172.21.15.164 2024-09-16T01:39:58.927 INFO:teuthology.orchestra.run.smithi139.stdout:smithi176 172.21.15.176 2024-09-16T01:39:58.927 INFO:teuthology.orchestra.run.smithi139.stdout:3 hosts in cluster 2024-09-16T01:39:58.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.928+0000 7feda923b640 1 -- 172.21.15.139:0/2097777969 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fed8c0a32d0 msgr2=0x7fed8c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:58.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.928+0000 7feda923b640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fed8c0a32d0 0x7fed8c0a5790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fed940058c0 tx=0x7fed94005850 comp rx=0 tx=0).stop 2024-09-16T01:39:58.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.928+0000 7feda923b640 1 -- 172.21.15.139:0/2097777969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda4102e00 msgr2=0x7feda40745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:39:58.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.928+0000 7feda923b640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda4102e00 0x7feda40745e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fed98009230 tx=0x7fed98004360 comp rx=0 tx=0).stop 2024-09-16T01:39:58.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.928+0000 7feda923b640 1 -- 172.21.15.139:0/2097777969 shutdown_connections 2024-09-16T01:39:58.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.928+0000 7feda923b640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fed8c0a32d0 0x7fed8c0a5790 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:58.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.928+0000 7feda923b640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feda41037c0 0x7feda4074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:58.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.928+0000 7feda923b640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda4102e00 0x7feda40745e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:58.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.928+0000 7feda923b640 1 --2- 172.21.15.139:0/2097777969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feda4101c00 0x7feda4075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:39:58.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.928+0000 7feda923b640 1 -- 172.21.15.139:0/2097777969 >> 172.21.15.139:0/2097777969 conn(0x7feda40fd390 msgr2=0x7feda41061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:39:58.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.928+0000 7feda923b640 1 -- 172.21.15.139:0/2097777969 shutdown_connections 2024-09-16T01:39:58.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:39:58.928+0000 7feda923b640 1 -- 172.21.15.139:0/2097777969 wait complete. 2024-09-16T01:39:58.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:58 smithi176 ceph-mon[35324]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:39:58.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:58 smithi176 ceph-mon[35324]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:39:59.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:58 smithi164 ceph-mon[35121]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:39:59.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:58 smithi164 ceph-mon[35121]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:39:59.693 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-16T01:39:59.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:39:59 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:39:59.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:39:59 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:39:59.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:39:59 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:40:00.692 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:00.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:00 smithi139 ceph-mon[31175]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:00.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:00 smithi139 ceph-mon[31175]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T01:40:00.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:00 smithi139 ceph-mon[31175]: overall HEALTH_OK 2024-09-16T01:40:00.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:00.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:00.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:00.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:00 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:00.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:00 smithi176 ceph-mon[35324]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:00.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:00 smithi176 ceph-mon[35324]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T01:40:00.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:00 smithi176 ceph-mon[35324]: overall HEALTH_OK 2024-09-16T01:40:00.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:00.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:00.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:00.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:00 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:01.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:00 smithi164 ceph-mon[35121]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:01.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:00 smithi164 ceph-mon[35121]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-16T01:40:01.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:00 smithi164 ceph-mon[35121]: overall HEALTH_OK 2024-09-16T01:40:01.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:01.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:01.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:01.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:00 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:01.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.218+0000 7fcee85dd640 1 -- 172.21.15.139:0/173776390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee0102e00 msgr2=0x7fcee0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:01.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.218+0000 7fcee85dd640 1 --2- 172.21.15.139:0/173776390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee0102e00 0x7fcee0103280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fced0009900 tx=0x7fced002a410 comp rx=0 tx=0).stop 2024-09-16T01:40:01.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.218+0000 7fcee85dd640 1 -- 172.21.15.139:0/173776390 shutdown_connections 2024-09-16T01:40:01.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.218+0000 7fcee85dd640 1 --2- 172.21.15.139:0/173776390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcee01037c0 0x7fcee010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:01.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.218+0000 7fcee85dd640 1 --2- 172.21.15.139:0/173776390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee0102e00 0x7fcee0103280 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:01.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.218+0000 7fcee85dd640 1 --2- 172.21.15.139:0/173776390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee0101c00 0x7fcee0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:01.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.218+0000 7fcee85dd640 1 -- 172.21.15.139:0/173776390 >> 172.21.15.139:0/173776390 conn(0x7fcee00fd390 msgr2=0x7fcee00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:01.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.219+0000 7fcee85dd640 1 -- 172.21.15.139:0/173776390 shutdown_connections 2024-09-16T01:40:01.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.219+0000 7fcee85dd640 1 -- 172.21.15.139:0/173776390 wait complete. 2024-09-16T01:40:01.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.219+0000 7fcee85dd640 1 Processor -- start 2024-09-16T01:40:01.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.219+0000 7fcee85dd640 1 -- start start 2024-09-16T01:40:01.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.219+0000 7fcee85dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee0101c00 0x7fcee0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:01.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.219+0000 7fcee85dd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcee0102e00 0x7fcee01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:01.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.219+0000 7fcee85dd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee01037c0 0x7fcee019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:01.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.219+0000 7fcee85dd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcee010d980 con 0x7fcee0101c00 2024-09-16T01:40:01.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.219+0000 7fcee85dd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcee010d800 con 0x7fcee01037c0 2024-09-16T01:40:01.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.219+0000 7fcee85dd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcee010db00 con 0x7fcee0102e00 2024-09-16T01:40:01.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.219+0000 7fcee6379640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee0101c00 0x7fcee0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:01.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.219+0000 7fcee6379640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee0101c00 0x7fcee0198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37096/0 (socket says 172.21.15.139:37096) 2024-09-16T01:40:01.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.219+0000 7fcee6379640 1 -- 172.21.15.139:0/3387366065 learned_addr learned my addr 172.21.15.139:0/3387366065 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:01.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.220+0000 7fcee6b7a640 1 --2- 172.21.15.139:0/3387366065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee01037c0 0x7fcee019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:01.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.220+0000 7fcee6379640 1 -- 172.21.15.139:0/3387366065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcee0102e00 msgr2=0x7fcee01985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:01.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.220+0000 7fcee6379640 1 --2- 172.21.15.139:0/3387366065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcee0102e00 0x7fcee01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:01.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.220+0000 7fcee6379640 1 -- 172.21.15.139:0/3387366065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee01037c0 msgr2=0x7fcee019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:01.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.220+0000 7fcee6379640 1 --2- 172.21.15.139:0/3387366065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee01037c0 0x7fcee019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:01.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.220+0000 7fcee6379640 1 -- 172.21.15.139:0/3387366065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee019fcc0 con 0x7fcee0101c00 2024-09-16T01:40:01.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.220+0000 7fcee6b7a640 1 --2- 172.21.15.139:0/3387366065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee01037c0 0x7fcee019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:01.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.220+0000 7fcee6379640 1 --2- 172.21.15.139:0/3387366065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee0101c00 0x7fcee0198090 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00b450 tx=0x7fcedc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:01.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.220+0000 7fced77fe640 1 -- 172.21.15.139:0/3387366065 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc015020 con 0x7fcee0101c00 2024-09-16T01:40:01.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.220+0000 7fced77fe640 1 -- 172.21.15.139:0/3387366065 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcedc011d70 con 0x7fcee0101c00 2024-09-16T01:40:01.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.220+0000 7fcee85dd640 1 -- 172.21.15.139:0/3387366065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee019ffb0 con 0x7fcee0101c00 2024-09-16T01:40:01.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.220+0000 7fced77fe640 1 -- 172.21.15.139:0/3387366065 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc00f340 con 0x7fcee0101c00 2024-09-16T01:40:01.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.220+0000 7fcee85dd640 1 -- 172.21.15.139:0/3387366065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee01a04e0 con 0x7fcee0101c00 2024-09-16T01:40:01.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.221+0000 7fcee85dd640 1 -- 172.21.15.139:0/3387366065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee0109070 con 0x7fcee0101c00 2024-09-16T01:40:01.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.222+0000 7fced77fe640 1 -- 172.21.15.139:0/3387366065 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcedc018020 con 0x7fcee0101c00 2024-09-16T01:40:01.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.223+0000 7fced77fe640 1 --2- 172.21.15.139:0/3387366065 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcecc0a3300 0x7fcecc0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:01.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.223+0000 7fcee5b78640 1 --2- 172.21.15.139:0/3387366065 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcecc0a3300 0x7fcecc0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:01.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.223+0000 7fced77fe640 1 -- 172.21.15.139:0/3387366065 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fcedc0cfd30 con 0x7fcee0101c00 2024-09-16T01:40:01.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.223+0000 7fcee5b78640 1 --2- 172.21.15.139:0/3387366065 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcecc0a3300 0x7fcecc0a57c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fced00096c0 tx=0x7fced0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:01.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.226+0000 7fced77fe640 1 -- 172.21.15.139:0/3387366065 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcedc01f020 con 0x7fcee0101c00 2024-09-16T01:40:01.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.411+0000 7fcee85dd640 1 -- 172.21.15.139:0/3387366065 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fcee0062630 con 0x7fcecc0a3300 2024-09-16T01:40:01.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.417+0000 7fced77fe640 1 -- 172.21.15.139:0/3387366065 <== mgr.14150 v2:172.21.15.139:6800/331514141 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fcee0062630 con 0x7fcecc0a3300 2024-09-16T01:40:01.418 INFO:teuthology.orchestra.run.smithi139.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-16T01:40:01.418 INFO:teuthology.orchestra.run.smithi139.stdout:smithi139 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500N6400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-16T01:40:01.418 INFO:teuthology.orchestra.run.smithi139.stdout:smithi164 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300H0400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-16T01:40:01.418 INFO:teuthology.orchestra.run.smithi139.stdout:smithi176 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330099400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-16T01:40:01.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.419+0000 7fcee85dd640 1 -- 172.21.15.139:0/3387366065 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcecc0a3300 msgr2=0x7fcecc0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:01.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.419+0000 7fcee85dd640 1 --2- 172.21.15.139:0/3387366065 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcecc0a3300 0x7fcecc0a57c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fced00096c0 tx=0x7fced0031400 comp rx=0 tx=0).stop 2024-09-16T01:40:01.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.419+0000 7fcee85dd640 1 -- 172.21.15.139:0/3387366065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee0101c00 msgr2=0x7fcee0198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:01.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.419+0000 7fcee85dd640 1 --2- 172.21.15.139:0/3387366065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee0101c00 0x7fcee0198090 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00b450 tx=0x7fcedc00b910 comp rx=0 tx=0).stop 2024-09-16T01:40:01.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.419+0000 7fcee85dd640 1 -- 172.21.15.139:0/3387366065 shutdown_connections 2024-09-16T01:40:01.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.419+0000 7fcee85dd640 1 --2- 172.21.15.139:0/3387366065 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fcecc0a3300 0x7fcecc0a57c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:01.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.419+0000 7fcee85dd640 1 --2- 172.21.15.139:0/3387366065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee01037c0 0x7fcee019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:01.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.419+0000 7fcee85dd640 1 --2- 172.21.15.139:0/3387366065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcee0102e00 0x7fcee01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:01.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.419+0000 7fcee85dd640 1 --2- 172.21.15.139:0/3387366065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee0101c00 0x7fcee0198090 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:01.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.419+0000 7fcee85dd640 1 -- 172.21.15.139:0/3387366065 >> 172.21.15.139:0/3387366065 conn(0x7fcee00fd390 msgr2=0x7fcee0106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:01.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.419+0000 7fcee85dd640 1 -- 172.21.15.139:0/3387366065 shutdown_connections 2024-09-16T01:40:01.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:01.419+0000 7fcee85dd640 1 -- 172.21.15.139:0/3387366065 wait complete. 2024-09-16T01:40:02.095 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T01:40:02.106 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T01:40:02.106 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T01:40:02.352 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:02 smithi164 ceph-mon[35121]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:02 smithi164 ceph-mon[35121]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-16T01:40:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:40:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:40:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:02 smithi164 ceph-mon[35121]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:02.805 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:02 smithi139 ceph-mon[31175]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:02.805 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:02.805 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:02.805 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:02 smithi139 ceph-mon[31175]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-16T01:40:02.805 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:40:02.805 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:40:02.806 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:02 smithi139 ceph-mon[31175]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:02.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.886+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/1256879902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6d01078c0 msgr2=0x7ff6d0109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:02.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.886+0000 7ff6d7fe0640 1 --2- 172.21.15.139:0/1256879902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6d01078c0 0x7ff6d0109d10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009ca0 tx=0x7ff6c802a8c0 comp rx=0 tx=0).stop 2024-09-16T01:40:02.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.886+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/1256879902 shutdown_connections 2024-09-16T01:40:02.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.886+0000 7ff6d7fe0640 1 --2- 172.21.15.139:0/1256879902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6d01078c0 0x7ff6d0109d10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:02.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.886+0000 7ff6d7fe0640 1 --2- 172.21.15.139:0/1256879902 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6d0104f90 0x7ff6d0107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:02.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.886+0000 7ff6d7fe0640 1 --2- 172.21.15.139:0/1256879902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d0102660 0x7ff6d0104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:02.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.886+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/1256879902 >> 172.21.15.139:0/1256879902 conn(0x7ff6d00fc280 msgr2=0x7ff6d00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:02.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.886+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/1256879902 shutdown_connections 2024-09-16T01:40:02.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.886+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/1256879902 wait complete. 2024-09-16T01:40:02.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.886+0000 7ff6d7fe0640 1 Processor -- start 2024-09-16T01:40:02.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d7fe0640 1 -- start start 2024-09-16T01:40:02.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d7fe0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6d0102660 0x7ff6d0101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:02.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d7fe0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6d0104f90 0x7ff6d0100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:02.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d7fe0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d01078c0 0x7ff6d0100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:02.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d7fe0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d010d880 con 0x7ff6d0102660 2024-09-16T01:40:02.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d7fe0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d010d700 con 0x7ff6d01078c0 2024-09-16T01:40:02.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d7fe0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d010da00 con 0x7ff6d0104f90 2024-09-16T01:40:02.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d5d7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6d0102660 0x7ff6d0101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d5d7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6d0102660 0x7ff6d0101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59764/0 (socket says 172.21.15.139:59764) 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d5d7c640 1 -- 172.21.15.139:0/4189342372 learned_addr learned my addr 172.21.15.139:0/4189342372 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d657d640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d01078c0 0x7ff6d0100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d557b640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6d0104f90 0x7ff6d0100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d5d7c640 1 -- 172.21.15.139:0/4189342372 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6d0104f90 msgr2=0x7ff6d0100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d5d7c640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6d0104f90 0x7ff6d0100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d5d7c640 1 -- 172.21.15.139:0/4189342372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d01078c0 msgr2=0x7ff6d0100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d5d7c640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d01078c0 0x7ff6d0100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d5d7c640 1 -- 172.21.15.139:0/4189342372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d0101410 con 0x7ff6d0102660 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d657d640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d01078c0 0x7ff6d0100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6d5d7c640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6d0102660 0x7ff6d0101fc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00c920 tx=0x7ff6cc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.887+0000 7ff6c6ffd640 1 -- 172.21.15.139:0/4189342372 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc007c70 con 0x7ff6d0102660 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.888+0000 7ff6c6ffd640 1 -- 172.21.15.139:0/4189342372 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6cc007e10 con 0x7ff6d0102660 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.888+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/4189342372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d01a1fe0 con 0x7ff6d0102660 2024-09-16T01:40:02.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.888+0000 7ff6c6ffd640 1 -- 172.21.15.139:0/4189342372 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc018890 con 0x7ff6d0102660 2024-09-16T01:40:02.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.888+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/4189342372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d01a23d0 con 0x7ff6d0102660 2024-09-16T01:40:02.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.889+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/4189342372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d00fdd70 con 0x7ff6d0102660 2024-09-16T01:40:02.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.890+0000 7ff6c6ffd640 1 -- 172.21.15.139:0/4189342372 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff6cc018a50 con 0x7ff6d0102660 2024-09-16T01:40:02.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.890+0000 7ff6c6ffd640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff6bc0a32d0 0x7ff6bc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:02.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.891+0000 7ff6d557b640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff6bc0a32d0 0x7ff6bc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:02.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.891+0000 7ff6c6ffd640 1 -- 172.21.15.139:0/4189342372 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7ff6cc0d02a0 con 0x7ff6d0102660 2024-09-16T01:40:02.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.891+0000 7ff6d557b640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff6bc0a32d0 0x7ff6bc0a5790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0005fd0 tx=0x7ff6c0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:02.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:02.894+0000 7ff6c6ffd640 1 -- 172.21.15.139:0/4189342372 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff6cc014030 con 0x7ff6d0102660 2024-09-16T01:40:02.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:02 smithi176 ceph-mon[35324]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:02.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:02.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:02.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:02 smithi176 ceph-mon[35324]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-16T01:40:02.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:40:02.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:40:02.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:02 smithi176 ceph-mon[35324]: from='mgr.14150 172.21.15.139:0/2758549114' entity='mgr.x' 2024-09-16T01:40:03.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:03.094+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/4189342372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff6d002d7c0 con 0x7ff6d0102660 2024-09-16T01:40:03.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:03.095+0000 7ff6c6ffd640 1 -- 172.21.15.139:0/4189342372 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3144 (secure 0 0 0) 0x7ff6cc09cb20 con 0x7ff6d0102660 2024-09-16T01:40:03.096 INFO:teuthology.orchestra.run.smithi139.stdout:e28 2024-09-16T01:40:03.096 INFO:teuthology.orchestra.run.smithi139.stdout:btime 2024-09-16T01:38:40:558188+0000 2024-09-16T01:40:03.096 INFO:teuthology.orchestra.run.smithi139.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T01:40:03.096 INFO:teuthology.orchestra.run.smithi139.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T01:40:03.096 INFO:teuthology.orchestra.run.smithi139.stdout:legacy client fscid: 1 2024-09-16T01:40:03.097 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:40:03.097 INFO:teuthology.orchestra.run.smithi139.stdout:Filesystem 'cephfs' (1) 2024-09-16T01:40:03.097 INFO:teuthology.orchestra.run.smithi139.stdout:fs_name cephfs 2024-09-16T01:40:03.097 INFO:teuthology.orchestra.run.smithi139.stdout:epoch 28 2024-09-16T01:40:03.097 INFO:teuthology.orchestra.run.smithi139.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T01:40:03.097 INFO:teuthology.orchestra.run.smithi139.stdout:created 2024-09-16T01:38:02.129081+0000 2024-09-16T01:40:03.097 INFO:teuthology.orchestra.run.smithi139.stdout:modified 2024-09-16T01:38:40.558170+0000 2024-09-16T01:40:03.097 INFO:teuthology.orchestra.run.smithi139.stdout:tableserver 0 2024-09-16T01:40:03.097 INFO:teuthology.orchestra.run.smithi139.stdout:root 0 2024-09-16T01:40:03.097 INFO:teuthology.orchestra.run.smithi139.stdout:session_timeout 300 2024-09-16T01:40:03.097 INFO:teuthology.orchestra.run.smithi139.stdout:session_autoclose 300 2024-09-16T01:40:03.097 INFO:teuthology.orchestra.run.smithi139.stdout:max_file_size 1099511627776 2024-09-16T01:40:03.097 INFO:teuthology.orchestra.run.smithi139.stdout:max_xattr_size 65536 2024-09-16T01:40:03.098 INFO:teuthology.orchestra.run.smithi139.stdout:required_client_features {} 2024-09-16T01:40:03.098 INFO:teuthology.orchestra.run.smithi139.stdout:last_failure 0 2024-09-16T01:40:03.098 INFO:teuthology.orchestra.run.smithi139.stdout:last_failure_osd_epoch 0 2024-09-16T01:40:03.098 INFO:teuthology.orchestra.run.smithi139.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T01:40:03.098 INFO:teuthology.orchestra.run.smithi139.stdout:max_mds 5 2024-09-16T01:40:03.098 INFO:teuthology.orchestra.run.smithi139.stdout:in 0,1,2,3,4 2024-09-16T01:40:03.098 INFO:teuthology.orchestra.run.smithi139.stdout:up {0=24470,1=24454,2=24443,3=24442,4=24433} 2024-09-16T01:40:03.098 INFO:teuthology.orchestra.run.smithi139.stdout:failed 2024-09-16T01:40:03.098 INFO:teuthology.orchestra.run.smithi139.stdout:damaged 2024-09-16T01:40:03.098 INFO:teuthology.orchestra.run.smithi139.stdout:stopped 2024-09-16T01:40:03.098 INFO:teuthology.orchestra.run.smithi139.stdout:data_pools [3,4] 2024-09-16T01:40:03.098 INFO:teuthology.orchestra.run.smithi139.stdout:metadata_pool 2 2024-09-16T01:40:03.098 INFO:teuthology.orchestra.run.smithi139.stdout:inline_data disabled 2024-09-16T01:40:03.099 INFO:teuthology.orchestra.run.smithi139.stdout:balancer 2024-09-16T01:40:03.099 INFO:teuthology.orchestra.run.smithi139.stdout:bal_rank_mask -1 2024-09-16T01:40:03.099 INFO:teuthology.orchestra.run.smithi139.stdout:standby_count_wanted 0 2024-09-16T01:40:03.099 INFO:teuthology.orchestra.run.smithi139.stdout:qdb_cluster leader: 24470 members: 24470,24454,24443,24442,24433 2024-09-16T01:40:03.099 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b{0:24470} state up:active seq 5 addr [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T01:40:03.099 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.e{0:24416} state up:standby-replay seq 1 addr [v2:172.21.15.164:6834/3662238682,v1:172.21.15.164:6835/3662238682] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T01:40:03.099 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.i{1:24454} state up:active seq 15 addr [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T01:40:03.099 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.l{1:24400} state up:standby-replay seq 1 addr [v2:172.21.15.176:6832/1026776933,v1:172.21.15.176:6833/1026776933] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T01:40:03.099 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.h{2:24443} state up:active seq 16 addr [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T01:40:03.099 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.k{2:24395} state up:standby-replay seq 1 addr [v2:172.21.15.164:6832/416925170,v1:172.21.15.164:6833/416925170] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T01:40:03.099 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c{3:24442} state up:active seq 17 addr [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T01:40:03.099 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a{3:14616} state up:standby-replay seq 1 addr [v2:172.21.15.139:6840/1887551794,v1:172.21.15.139:6841/1887551794] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T01:40:03.099 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.f{4:24433} state up:active seq 18 addr [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T01:40:03.099 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.g{4:14604} state up:standby-replay seq 1 addr [v2:172.21.15.139:6838/1075273450,v1:172.21.15.139:6839/1075273450] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T01:40:03.100 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:40:03.100 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:40:03.100 INFO:teuthology.orchestra.run.smithi139.stdout:Standby daemons: 2024-09-16T01:40:03.100 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:40:03.100 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.139:6834/129107170,v1:172.21.15.139:6835/129107170] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T01:40:03.100 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d{-1:14556} state up:standby seq 1 addr [v2:172.21.15.139:6836/465602995,v1:172.21.15.139:6837/465602995] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T01:40:03.101 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 28 2024-09-16T01:40:03.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:03.097+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/4189342372 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff6bc0a32d0 msgr2=0x7ff6bc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:03.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:03.097+0000 7ff6d7fe0640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff6bc0a32d0 0x7ff6bc0a5790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0005fd0 tx=0x7ff6c0005850 comp rx=0 tx=0).stop 2024-09-16T01:40:03.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:03.097+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/4189342372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6d0102660 msgr2=0x7ff6d0101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:03.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:03.097+0000 7ff6d7fe0640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6d0102660 0x7ff6d0101fc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00c920 tx=0x7ff6cc00cde0 comp rx=0 tx=0).stop 2024-09-16T01:40:03.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:03.097+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/4189342372 shutdown_connections 2024-09-16T01:40:03.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:03.097+0000 7ff6d7fe0640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff6bc0a32d0 0x7ff6bc0a5790 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:03.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:03.097+0000 7ff6d7fe0640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d01078c0 0x7ff6d0100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:03.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:03.097+0000 7ff6d7fe0640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6d0104f90 0x7ff6d0100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:03.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:03.097+0000 7ff6d7fe0640 1 --2- 172.21.15.139:0/4189342372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6d0102660 0x7ff6d0101fc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:03.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:03.097+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/4189342372 >> 172.21.15.139:0/4189342372 conn(0x7ff6d00fc280 msgr2=0x7ff6d0102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:03.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:03.097+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/4189342372 shutdown_connections 2024-09-16T01:40:03.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:03.097+0000 7ff6d7fe0640 1 -- 172.21.15.139:0/4189342372 wait complete. 2024-09-16T01:40:03.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:03 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/4189342372' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T01:40:03.732 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-16T01:40:03.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:03 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/4189342372' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T01:40:03.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:03 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/4189342372' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T01:40:03.980 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:04.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.477+0000 7f504aa25640 1 -- 172.21.15.139:0/3723865107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5044101c00 msgr2=0x7f5044102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:04.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.477+0000 7f504aa25640 1 --2- 172.21.15.139:0/3723865107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5044101c00 0x7f5044102000 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5034009900 tx=0x7f503402a410 comp rx=0 tx=0).stop 2024-09-16T01:40:04.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.477+0000 7f504aa25640 1 -- 172.21.15.139:0/3723865107 shutdown_connections 2024-09-16T01:40:04.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.477+0000 7f504aa25640 1 --2- 172.21.15.139:0/3723865107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50441037c0 0x7f504410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:04.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.477+0000 7f504aa25640 1 --2- 172.21.15.139:0/3723865107 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5044102e00 0x7f5044103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:04.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.477+0000 7f504aa25640 1 --2- 172.21.15.139:0/3723865107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5044101c00 0x7f5044102000 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:04.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.478+0000 7f504aa25640 1 -- 172.21.15.139:0/3723865107 >> 172.21.15.139:0/3723865107 conn(0x7f50440fd390 msgr2=0x7f50440ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:04.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.478+0000 7f504aa25640 1 -- 172.21.15.139:0/3723865107 shutdown_connections 2024-09-16T01:40:04.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.478+0000 7f504aa25640 1 -- 172.21.15.139:0/3723865107 wait complete. 2024-09-16T01:40:04.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.478+0000 7f504aa25640 1 Processor -- start 2024-09-16T01:40:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.478+0000 7f504aa25640 1 -- start start 2024-09-16T01:40:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.478+0000 7f504aa25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5044101c00 0x7f5044197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.478+0000 7f504aa25640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5044102e00 0x7f50441983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.478+0000 7f504aa25640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50441037c0 0x7f504419f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.478+0000 7f504aa25640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f504410d860 con 0x7f5044101c00 2024-09-16T01:40:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.478+0000 7f504aa25640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f504410d6e0 con 0x7f50441037c0 2024-09-16T01:40:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.478+0000 7f504aa25640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f504410d9e0 con 0x7f5044102e00 2024-09-16T01:40:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f5043fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5044101c00 0x7f5044197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f5043fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5044101c00 0x7f5044197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59780/0 (socket says 172.21.15.139:59780) 2024-09-16T01:40:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f5043fff640 1 -- 172.21.15.139:0/4167695201 learned_addr learned my addr 172.21.15.139:0/4167695201 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f50437fe640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5044102e00 0x7f50441983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f5048fc2640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50441037c0 0x7f504419f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f5043fff640 1 -- 172.21.15.139:0/4167695201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5044102e00 msgr2=0x7f50441983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f5043fff640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5044102e00 0x7f50441983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f5043fff640 1 -- 172.21.15.139:0/4167695201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50441037c0 msgr2=0x7f504419f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f5043fff640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50441037c0 0x7f504419f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f5043fff640 1 -- 172.21.15.139:0/4167695201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504419fab0 con 0x7f5044101c00 2024-09-16T01:40:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f5048fc2640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50441037c0 0x7f504419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f5043fff640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5044101c00 0x7f5044197e80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5034031610 tx=0x7f50340043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f50417fa640 1 -- 172.21.15.139:0/4167695201 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503403f030 con 0x7f5044101c00 2024-09-16T01:40:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f50417fa640 1 -- 172.21.15.139:0/4167695201 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5034003ce0 con 0x7f5044101c00 2024-09-16T01:40:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f504aa25640 1 -- 172.21.15.139:0/4167695201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504419fd40 con 0x7f5044101c00 2024-09-16T01:40:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f50417fa640 1 -- 172.21.15.139:0/4167695201 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034035070 con 0x7f5044101c00 2024-09-16T01:40:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.479+0000 7f504aa25640 1 -- 172.21.15.139:0/4167695201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50441a01c0 con 0x7f5044101c00 2024-09-16T01:40:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.480+0000 7f504aa25640 1 -- 172.21.15.139:0/4167695201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5044109000 con 0x7f5044101c00 2024-09-16T01:40:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.481+0000 7f50417fa640 1 -- 172.21.15.139:0/4167695201 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5034042030 con 0x7f5044101c00 2024-09-16T01:40:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.482+0000 7f50417fa640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f502c0a3380 0x7f502c0a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.482+0000 7f50437fe640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f502c0a3380 0x7f502c0a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.482+0000 7f50417fa640 1 -- 172.21.15.139:0/4167695201 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f50340f03c0 con 0x7f5044101c00 2024-09-16T01:40:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.482+0000 7f50437fe640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f502c0a3380 0x7f502c0a5840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5038005e10 tx=0x7f5038005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.486+0000 7f50417fa640 1 -- 172.21.15.139:0/4167695201 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50340bcc50 con 0x7f5044101c00 2024-09-16T01:40:04.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.654+0000 7f504aa25640 1 -- 172.21.15.139:0/4167695201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f5044062e90 con 0x7f5044101c00 2024-09-16T01:40:04.656 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:04 smithi139 ceph-mon[31175]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:40:04.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.655+0000 7f50417fa640 1 -- 172.21.15.139:0/4167695201 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) ==== 56+0+5728 (secure 0 0 0) 0x7f50340bc550 con 0x7f5044101c00 2024-09-16T01:40:04.657 INFO:teuthology.orchestra.run.smithi139.stdout:epoch 85 2024-09-16T01:40:04.657 INFO:teuthology.orchestra.run.smithi139.stdout:fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T01:40:04.657 INFO:teuthology.orchestra.run.smithi139.stdout:created 2024-09-16T01:30:17.770313+0000 2024-09-16T01:40:04.657 INFO:teuthology.orchestra.run.smithi139.stdout:modified 2024-09-16T01:39:09.511648+0000 2024-09-16T01:40:04.657 INFO:teuthology.orchestra.run.smithi139.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T01:40:04.657 INFO:teuthology.orchestra.run.smithi139.stdout:crush_version 27 2024-09-16T01:40:04.657 INFO:teuthology.orchestra.run.smithi139.stdout:full_ratio 0.95 2024-09-16T01:40:04.657 INFO:teuthology.orchestra.run.smithi139.stdout:backfillfull_ratio 0.9 2024-09-16T01:40:04.657 INFO:teuthology.orchestra.run.smithi139.stdout:nearfull_ratio 0.85 2024-09-16T01:40:04.658 INFO:teuthology.orchestra.run.smithi139.stdout:require_min_compat_client luminous 2024-09-16T01:40:04.658 INFO:teuthology.orchestra.run.smithi139.stdout:min_compat_client luminous 2024-09-16T01:40:04.658 INFO:teuthology.orchestra.run.smithi139.stdout:require_osd_release squid 2024-09-16T01:40:04.658 INFO:teuthology.orchestra.run.smithi139.stdout:stretch_mode_enabled false 2024-09-16T01:40:04.658 INFO:teuthology.orchestra.run.smithi139.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-16T01:40:04.658 INFO:teuthology.orchestra.run.smithi139.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-16T01:40:04.658 INFO:teuthology.orchestra.run.smithi139.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-16T01:40:04.658 INFO:teuthology.orchestra.run.smithi139.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-16T01:40:04.658 INFO:teuthology.orchestra.run.smithi139.stdout:max_osd 12 2024-09-16T01:40:04.658 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6802/3190810133,v1:172.21.15.139:6803/3190810133] [v2:172.21.15.139:6804/3190810133,v1:172.21.15.139:6805/3190810133] exists,up b66696da-ef66-4f25-9527-bbcc9ded2cee 2024-09-16T01:40:04.658 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1 up in weight 1 up_from 12 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6800/2699270330,v1:172.21.15.164:6801/2699270330] [v2:172.21.15.164:6802/2699270330,v1:172.21.15.164:6803/2699270330] exists,up ccad17f2-0750-425e-9810-49a13a929ac3 2024-09-16T01:40:04.658 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2 up in weight 1 up_from 16 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6800/2034471646,v1:172.21.15.176:6801/2034471646] [v2:172.21.15.176:6802/2034471646,v1:172.21.15.176:6803/2034471646] exists,up baf0ba73-8faf-4e51-a171-c47898800638 2024-09-16T01:40:04.658 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3 up in weight 1 up_from 22 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6810/797147071,v1:172.21.15.139:6811/797147071] [v2:172.21.15.139:6812/797147071,v1:172.21.15.139:6813/797147071] exists,up 98957bf1-67bb-4092-9ea4-b263a5706f8e 2024-09-16T01:40:04.659 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4 up in weight 1 up_from 29 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6808/1421304792,v1:172.21.15.164:6809/1421304792] [v2:172.21.15.164:6810/1421304792,v1:172.21.15.164:6811/1421304792] exists,up 9cd76696-8161-4e82-a571-29e98c11d5a0 2024-09-16T01:40:04.659 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5 up in weight 1 up_from 34 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6808/10600602,v1:172.21.15.176:6809/10600602] [v2:172.21.15.176:6810/10600602,v1:172.21.15.176:6811/10600602] exists,up ccc17a29-9a58-40ec-b453-5c38bee86f25 2024-09-16T01:40:04.659 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6 up in weight 1 up_from 41 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6818/2601111164,v1:172.21.15.139:6819/2601111164] [v2:172.21.15.139:6820/2601111164,v1:172.21.15.139:6821/2601111164] exists,up e5ee7212-10c6-4a05-992e-622a14a5a6b5 2024-09-16T01:40:04.659 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7 up in weight 1 up_from 45 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6816/2074080693,v1:172.21.15.164:6817/2074080693] [v2:172.21.15.164:6818/2074080693,v1:172.21.15.164:6819/2074080693] exists,up 23143c15-0839-4fd5-9d0a-19ae891000f0 2024-09-16T01:40:04.659 INFO:teuthology.orchestra.run.smithi139.stdout:osd.8 up in weight 1 up_from 52 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6816/593133361,v1:172.21.15.176:6817/593133361] [v2:172.21.15.176:6818/593133361,v1:172.21.15.176:6819/593133361] exists,up 1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5 2024-09-16T01:40:04.659 INFO:teuthology.orchestra.run.smithi139.stdout:osd.9 up in weight 1 up_from 59 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6826/3047942108,v1:172.21.15.139:6827/3047942108] [v2:172.21.15.139:6828/3047942108,v1:172.21.15.139:6829/3047942108] exists,up 09d25a3b-1e66-480e-9433-ceef7f0b9279 2024-09-16T01:40:04.659 INFO:teuthology.orchestra.run.smithi139.stdout:osd.10 up in weight 1 up_from 64 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6824/4172533828,v1:172.21.15.164:6825/4172533828] [v2:172.21.15.164:6826/4172533828,v1:172.21.15.164:6827/4172533828] exists,up 32b4d8ce-b644-42fa-80cc-a15a1f5f3a42 2024-09-16T01:40:04.659 INFO:teuthology.orchestra.run.smithi139.stdout:osd.11 up in weight 1 up_from 71 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6824/1854547565,v1:172.21.15.176:6825/1854547565] [v2:172.21.15.176:6826/1854547565,v1:172.21.15.176:6827/1854547565] exists,up 49098bdc-58a9-4a74-8045-d408af7711ef 2024-09-16T01:40:04.659 INFO:teuthology.orchestra.run.smithi139.stdout:pg_upmap_items 2.5 [8,11] 2024-09-16T01:40:04.659 INFO:teuthology.orchestra.run.smithi139.stdout:pg_upmap_items 2.a [2,11] 2024-09-16T01:40:04.659 INFO:teuthology.orchestra.run.smithi139.stdout:pg_upmap_items 2.e [2,4] 2024-09-16T01:40:04.659 INFO:teuthology.orchestra.run.smithi139.stdout:pg_upmap_items 2.16 [2,4] 2024-09-16T01:40:04.660 INFO:teuthology.orchestra.run.smithi139.stdout:pg_upmap_items 2.1f [8,1] 2024-09-16T01:40:04.660 INFO:teuthology.orchestra.run.smithi139.stdout:pg_upmap_items 3.1 [4,9] 2024-09-16T01:40:04.660 INFO:teuthology.orchestra.run.smithi139.stdout:pg_upmap_items 3.1a [5,6] 2024-09-16T01:40:04.660 INFO:teuthology.orchestra.run.smithi139.stdout:pg_upmap_items 3.1b [4,8] 2024-09-16T01:40:04.660 INFO:teuthology.orchestra.run.smithi139.stdout:pg_upmap_items 3.2d [3,9] 2024-09-16T01:40:04.660 INFO:teuthology.orchestra.run.smithi139.stdout:pg_upmap_items 3.3c [4,9] 2024-09-16T01:40:04.660 INFO:teuthology.orchestra.run.smithi139.stdout:pg_upmap_items 4.5 [9,10] 2024-09-16T01:40:04.660 INFO:teuthology.orchestra.run.smithi139.stdout:pg_upmap_items 4.e [5,8] 2024-09-16T01:40:04.660 INFO:teuthology.orchestra.run.smithi139.stdout:pg_upmap_items 4.12 [9,10] 2024-09-16T01:40:04.660 INFO:teuthology.orchestra.run.smithi139.stdout:pg_upmap_items 4.17 [9,8] 2024-09-16T01:40:04.660 INFO:teuthology.orchestra.run.smithi139.stdout:blocklist 172.21.15.139:0/3772232169 expires 2024-09-17T01:31:07.344357+0000 2024-09-16T01:40:04.660 INFO:teuthology.orchestra.run.smithi139.stdout:blocklist 172.21.15.139:0/3997627612 expires 2024-09-17T01:31:07.344357+0000 2024-09-16T01:40:04.660 INFO:teuthology.orchestra.run.smithi139.stdout:blocklist 172.21.15.139:0/1608930262 expires 2024-09-17T01:31:07.344357+0000 2024-09-16T01:40:04.661 INFO:teuthology.orchestra.run.smithi139.stdout:blocklist 172.21.15.139:6801/1485008061 expires 2024-09-17T01:31:07.344357+0000 2024-09-16T01:40:04.661 INFO:teuthology.orchestra.run.smithi139.stdout:blocklist 172.21.15.139:6800/1485008061 expires 2024-09-17T01:31:07.344357+0000 2024-09-16T01:40:04.661 INFO:teuthology.orchestra.run.smithi139.stdout:blocklist 172.21.15.139:6800/2853057507 expires 2024-09-17T01:30:44.751168+0000 2024-09-16T01:40:04.661 INFO:teuthology.orchestra.run.smithi139.stdout:blocklist 172.21.15.139:0/677216867 expires 2024-09-17T01:30:44.751168+0000 2024-09-16T01:40:04.661 INFO:teuthology.orchestra.run.smithi139.stdout:blocklist 172.21.15.139:6801/2853057507 expires 2024-09-17T01:30:44.751168+0000 2024-09-16T01:40:04.661 INFO:teuthology.orchestra.run.smithi139.stdout:blocklist 172.21.15.139:0/1830135963 expires 2024-09-17T01:30:44.751168+0000 2024-09-16T01:40:04.661 INFO:teuthology.orchestra.run.smithi139.stdout:blocklist 172.21.15.139:0/2608207648 expires 2024-09-17T01:30:44.751168+0000 2024-09-16T01:40:04.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.658+0000 7f504aa25640 1 -- 172.21.15.139:0/4167695201 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f502c0a3380 msgr2=0x7f502c0a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:04.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.658+0000 7f504aa25640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f502c0a3380 0x7f502c0a5840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5038005e10 tx=0x7f5038005da0 comp rx=0 tx=0).stop 2024-09-16T01:40:04.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.658+0000 7f504aa25640 1 -- 172.21.15.139:0/4167695201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5044101c00 msgr2=0x7f5044197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:04.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.658+0000 7f504aa25640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5044101c00 0x7f5044197e80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5034031610 tx=0x7f50340043b0 comp rx=0 tx=0).stop 2024-09-16T01:40:04.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.658+0000 7f504aa25640 1 -- 172.21.15.139:0/4167695201 shutdown_connections 2024-09-16T01:40:04.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.658+0000 7f504aa25640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f502c0a3380 0x7f502c0a5840 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:04.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.658+0000 7f504aa25640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50441037c0 0x7f504419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:04.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.658+0000 7f504aa25640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5044102e00 0x7f50441983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:04.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.658+0000 7f504aa25640 1 --2- 172.21.15.139:0/4167695201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5044101c00 0x7f5044197e80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:04.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.658+0000 7f504aa25640 1 -- 172.21.15.139:0/4167695201 >> 172.21.15.139:0/4167695201 conn(0x7f50440fd390 msgr2=0x7f50441061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:04.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.658+0000 7f504aa25640 1 -- 172.21.15.139:0/4167695201 shutdown_connections 2024-09-16T01:40:04.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:04.658+0000 7f504aa25640 1 -- 172.21.15.139:0/4167695201 wait complete. 2024-09-16T01:40:04.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:04 smithi164 ceph-mon[35121]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:40:04.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:04 smithi176 ceph-mon[35324]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T01:40:05.340 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-16T01:40:05.355 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:40:05.615 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:05.623 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:05 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/4167695201' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T01:40:05.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:05 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/4167695201' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T01:40:05.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:05 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/4167695201' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T01:40:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.102+0000 7f4eba2d4640 1 -- 172.21.15.139:0/1400476009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eb41036e0 msgr2=0x7f4eb4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.102+0000 7f4eba2d4640 1 --2- 172.21.15.139:0/1400476009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eb41036e0 0x7f4eb4107b90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009930 tx=0x7f4ea802a420 comp rx=0 tx=0).stop 2024-09-16T01:40:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.102+0000 7f4eba2d4640 1 -- 172.21.15.139:0/1400476009 shutdown_connections 2024-09-16T01:40:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.102+0000 7f4eba2d4640 1 --2- 172.21.15.139:0/1400476009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb41082a0 0x7f4eb410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.102+0000 7f4eba2d4640 1 --2- 172.21.15.139:0/1400476009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eb41036e0 0x7f4eb4107b90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.102+0000 7f4eba2d4640 1 --2- 172.21.15.139:0/1400476009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4eb4102d30 0x7f4eb4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.102+0000 7f4eba2d4640 1 -- 172.21.15.139:0/1400476009 >> 172.21.15.139:0/1400476009 conn(0x7f4eb40fc7d0 msgr2=0x7f4eb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.102+0000 7f4eba2d4640 1 -- 172.21.15.139:0/1400476009 shutdown_connections 2024-09-16T01:40:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.103+0000 7f4eba2d4640 1 -- 172.21.15.139:0/1400476009 wait complete. 2024-09-16T01:40:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.103+0000 7f4eba2d4640 1 Processor -- start 2024-09-16T01:40:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.103+0000 7f4eba2d4640 1 -- start start 2024-09-16T01:40:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eba2d4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb4102d30 0x7f4eb4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eba2d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eb41036e0 0x7f4eb4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eba2d4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4eb41082a0 0x7f4eb419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eba2d4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41120a0 con 0x7f4eb41036e0 2024-09-16T01:40:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eba2d4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb4111f20 con 0x7f4eb4102d30 2024-09-16T01:40:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eba2d4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb4112220 con 0x7f4eb41082a0 2024-09-16T01:40:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eb37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eb41036e0 0x7f4eb4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eb37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eb41036e0 0x7f4eb4198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59794/0 (socket says 172.21.15.139:59794) 2024-09-16T01:40:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eb37fe640 1 -- 172.21.15.139:0/919949240 learned_addr learned my addr 172.21.15.139:0/919949240 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eb3fff640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb4102d30 0x7f4eb4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eb8871640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4eb41082a0 0x7f4eb419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eb37fe640 1 -- 172.21.15.139:0/919949240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4eb41082a0 msgr2=0x7f4eb419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eb37fe640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4eb41082a0 0x7f4eb419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eb37fe640 1 -- 172.21.15.139:0/919949240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb4102d30 msgr2=0x7f4eb4198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eb37fe640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb4102d30 0x7f4eb4198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eb37fe640 1 -- 172.21.15.139:0/919949240 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb419d530 con 0x7f4eb41036e0 2024-09-16T01:40:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.104+0000 7f4eb8871640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4eb41082a0 0x7f4eb419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.105+0000 7f4eb37fe640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eb41036e0 0x7f4eb4198ac0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005940 tx=0x7f4ea80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.105+0000 7f4eb17fa640 1 -- 172.21.15.139:0/919949240 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea803f030 con 0x7f4eb41036e0 2024-09-16T01:40:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.105+0000 7f4eba2d4640 1 -- 172.21.15.139:0/919949240 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb419d7c0 con 0x7f4eb41036e0 2024-09-16T01:40:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.105+0000 7f4eb17fa640 1 -- 172.21.15.139:0/919949240 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea8003ce0 con 0x7f4eb41036e0 2024-09-16T01:40:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.105+0000 7f4eb17fa640 1 -- 172.21.15.139:0/919949240 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8035070 con 0x7f4eb41036e0 2024-09-16T01:40:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.105+0000 7f4eba2d4640 1 -- 172.21.15.139:0/919949240 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb41a5100 con 0x7f4eb41036e0 2024-09-16T01:40:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.106+0000 7f4eba2d4640 1 -- 172.21.15.139:0/919949240 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb4103110 con 0x7f4eb41036e0 2024-09-16T01:40:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.107+0000 7f4eb17fa640 1 -- 172.21.15.139:0/919949240 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4ea8042030 con 0x7f4eb41036e0 2024-09-16T01:40:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.107+0000 7f4eb17fa640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4e9c0a32d0 0x7f4e9c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.108+0000 7f4eb17fa640 1 -- 172.21.15.139:0/919949240 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f4ea80f0590 con 0x7f4eb41036e0 2024-09-16T01:40:06.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.108+0000 7f4eb3fff640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4e9c0a32d0 0x7f4e9c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:06.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.108+0000 7f4eb3fff640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4e9c0a32d0 0x7f4e9c0a5790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4005fd0 tx=0x7f4ea400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:06.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.111+0000 7f4eb17fa640 1 -- 172.21.15.139:0/919949240 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ea80bcd80 con 0x7f4eb41036e0 2024-09-16T01:40:06.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.319+0000 7f4eba2d4640 1 -- 172.21.15.139:0/919949240 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4eb41998e0 con 0x7f4eb41036e0 2024-09-16T01:40:06.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.320+0000 7f4eb17fa640 1 -- 172.21.15.139:0/919949240 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11995 (secure 0 0 0) 0x7f4eb41998e0 con 0x7f4eb41036e0 2024-09-16T01:40:06.322 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:40:06.322 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":28,"btime":"2024-09-16T01:38:40:558188+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:38:40.558170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24454,24443,24442,24433]},"id":1}]} 2024-09-16T01:40:06.322 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 28 2024-09-16T01:40:06.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.322+0000 7f4eba2d4640 1 -- 172.21.15.139:0/919949240 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4e9c0a32d0 msgr2=0x7f4e9c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:06.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.322+0000 7f4eba2d4640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4e9c0a32d0 0x7f4e9c0a5790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4005fd0 tx=0x7f4ea400b390 comp rx=0 tx=0).stop 2024-09-16T01:40:06.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.322+0000 7f4eba2d4640 1 -- 172.21.15.139:0/919949240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eb41036e0 msgr2=0x7f4eb4198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:06.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.322+0000 7f4eba2d4640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eb41036e0 0x7f4eb4198ac0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005940 tx=0x7f4ea80042f0 comp rx=0 tx=0).stop 2024-09-16T01:40:06.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.322+0000 7f4eba2d4640 1 -- 172.21.15.139:0/919949240 shutdown_connections 2024-09-16T01:40:06.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.322+0000 7f4eba2d4640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f4e9c0a32d0 0x7f4e9c0a5790 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:06.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.322+0000 7f4eba2d4640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4eb41082a0 0x7f4eb419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:06.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.322+0000 7f4eba2d4640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eb41036e0 0x7f4eb4198ac0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:06.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.322+0000 7f4eba2d4640 1 --2- 172.21.15.139:0/919949240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb4102d30 0x7f4eb4198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:06.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.322+0000 7f4eba2d4640 1 -- 172.21.15.139:0/919949240 >> 172.21.15.139:0/919949240 conn(0x7f4eb40fc7d0 msgr2=0x7f4eb4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:06.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.323+0000 7f4eba2d4640 1 -- 172.21.15.139:0/919949240 shutdown_connections 2024-09-16T01:40:06.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:06.323+0000 7f4eba2d4640 1 -- 172.21.15.139:0/919949240 wait complete. 2024-09-16T01:40:06.632 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:06 smithi139 ceph-mon[31175]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:40:06.632 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:06 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/919949240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:06.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:06 smithi164 ceph-mon[35121]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:40:06.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:06 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/919949240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:06.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:06 smithi176 ceph-mon[35324]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-16T01:40:06.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:06 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/919949240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:07.016 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:40:07.265 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:07.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.770+0000 7f03ee431640 1 -- 172.21.15.139:0/4227565377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03e8103660 msgr2=0x7f03e810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:07.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.770+0000 7f03ee431640 1 --2- 172.21.15.139:0/4227565377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03e8103660 0x7f03e810bb50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009900 tx=0x7f03d002a480 comp rx=0 tx=0).stop 2024-09-16T01:40:07.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.770+0000 7f03ee431640 1 -- 172.21.15.139:0/4227565377 shutdown_connections 2024-09-16T01:40:07.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.770+0000 7f03ee431640 1 --2- 172.21.15.139:0/4227565377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03e810c170 0x7f03e810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:07.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.770+0000 7f03ee431640 1 --2- 172.21.15.139:0/4227565377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03e8103660 0x7f03e810bb50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:07.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.770+0000 7f03ee431640 1 --2- 172.21.15.139:0/4227565377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03e8102d40 0x7f03e8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:07.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.770+0000 7f03ee431640 1 -- 172.21.15.139:0/4227565377 >> 172.21.15.139:0/4227565377 conn(0x7f03e80fc7a0 msgr2=0x7f03e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:07.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.770+0000 7f03ee431640 1 -- 172.21.15.139:0/4227565377 shutdown_connections 2024-09-16T01:40:07.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.770+0000 7f03ee431640 1 -- 172.21.15.139:0/4227565377 wait complete. 2024-09-16T01:40:07.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.770+0000 7f03ee431640 1 Processor -- start 2024-09-16T01:40:07.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03ee431640 1 -- start start 2024-09-16T01:40:07.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03ee431640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03e8102d40 0x7f03e8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:07.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03ee431640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03e8103660 0x7f03e8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:07.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03ee431640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03e810c170 0x7f03e819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:07.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03ee431640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f03e8111fb0 con 0x7f03e8102d40 2024-09-16T01:40:07.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03ee431640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f03e8111e30 con 0x7f03e8103660 2024-09-16T01:40:07.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03ee431640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f03e8112130 con 0x7f03e810c170 2024-09-16T01:40:07.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03e8102d40 0x7f03e8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:07.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03e8102d40 0x7f03e8198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59798/0 (socket says 172.21.15.139:59798) 2024-09-16T01:40:07.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03e7fff640 1 -- 172.21.15.139:0/1616103264 learned_addr learned my addr 172.21.15.139:0/1616103264 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:07.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03e77fe640 1 --2- 172.21.15.139:0/1616103264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03e8103660 0x7f03e8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:07.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03ec9ce640 1 --2- 172.21.15.139:0/1616103264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03e810c170 0x7f03e819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:07.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03e7fff640 1 -- 172.21.15.139:0/1616103264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03e810c170 msgr2=0x7f03e819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:07.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03e7fff640 1 --2- 172.21.15.139:0/1616103264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03e810c170 0x7f03e819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:07.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03e7fff640 1 -- 172.21.15.139:0/1616103264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03e8103660 msgr2=0x7f03e8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:07.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03e7fff640 1 --2- 172.21.15.139:0/1616103264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03e8103660 0x7f03e8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:07.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.771+0000 7f03e7fff640 1 -- 172.21.15.139:0/1616103264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e819d3c0 con 0x7f03e8102d40 2024-09-16T01:40:07.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.772+0000 7f03e7fff640 1 --2- 172.21.15.139:0/1616103264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03e8102d40 0x7f03e8198410 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c920 tx=0x7f03d800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:07.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.772+0000 7f03e57fa640 1 -- 172.21.15.139:0/1616103264 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8007c70 con 0x7f03e8102d40 2024-09-16T01:40:07.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.772+0000 7f03ee431640 1 -- 172.21.15.139:0/1616103264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e819d6b0 con 0x7f03e8102d40 2024-09-16T01:40:07.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.772+0000 7f03e57fa640 1 -- 172.21.15.139:0/1616103264 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d8007e10 con 0x7f03e8102d40 2024-09-16T01:40:07.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.772+0000 7f03ee431640 1 -- 172.21.15.139:0/1616103264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e81a4f90 con 0x7f03e8102d40 2024-09-16T01:40:07.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.772+0000 7f03e57fa640 1 -- 172.21.15.139:0/1616103264 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8005260 con 0x7f03e8102d40 2024-09-16T01:40:07.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.773+0000 7f03ee431640 1 -- 172.21.15.139:0/1616103264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e8103120 con 0x7f03e8102d40 2024-09-16T01:40:07.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.774+0000 7f03e57fa640 1 -- 172.21.15.139:0/1616103264 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f03d8005400 con 0x7f03e8102d40 2024-09-16T01:40:07.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.775+0000 7f03e57fa640 1 --2- 172.21.15.139:0/1616103264 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f03d40a3240 0x7f03d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:07.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.775+0000 7f03e77fe640 1 --2- 172.21.15.139:0/1616103264 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f03d40a3240 0x7f03d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:07.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.775+0000 7f03e57fa640 1 -- 172.21.15.139:0/1616103264 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f03d80cff10 con 0x7f03e8102d40 2024-09-16T01:40:07.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.775+0000 7f03e77fe640 1 --2- 172.21.15.139:0/1616103264 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f03d40a3240 0x7f03d40a5700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f03e80fed30 tx=0x7f03d0031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:07.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.779+0000 7f03e57fa640 1 -- 172.21.15.139:0/1616103264 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03d8014030 con 0x7f03e8102d40 2024-09-16T01:40:07.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.978+0000 7f03ee431640 1 -- 172.21.15.139:0/1616103264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03e802d7c0 con 0x7f03e8102d40 2024-09-16T01:40:07.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.979+0000 7f03e57fa640 1 -- 172.21.15.139:0/1616103264 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11995 (secure 0 0 0) 0x7f03d801f300 con 0x7f03e8102d40 2024-09-16T01:40:07.981 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:40:07.981 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":28,"btime":"2024-09-16T01:38:40:558188+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:38:40.558170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24454,24443,24442,24433]},"id":1}]} 2024-09-16T01:40:07.981 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 28 2024-09-16T01:40:07.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.981+0000 7f03ee431640 1 -- 172.21.15.139:0/1616103264 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f03d40a3240 msgr2=0x7f03d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:07.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.981+0000 7f03ee431640 1 --2- 172.21.15.139:0/1616103264 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f03d40a3240 0x7f03d40a5700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f03e80fed30 tx=0x7f03d0031490 comp rx=0 tx=0).stop 2024-09-16T01:40:07.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.981+0000 7f03ee431640 1 -- 172.21.15.139:0/1616103264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03e8102d40 msgr2=0x7f03e8198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:07.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.981+0000 7f03ee431640 1 --2- 172.21.15.139:0/1616103264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03e8102d40 0x7f03e8198410 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c920 tx=0x7f03d800cde0 comp rx=0 tx=0).stop 2024-09-16T01:40:07.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.981+0000 7f03ee431640 1 -- 172.21.15.139:0/1616103264 shutdown_connections 2024-09-16T01:40:07.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.981+0000 7f03ee431640 1 --2- 172.21.15.139:0/1616103264 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f03d40a3240 0x7f03d40a5700 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:07.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.981+0000 7f03ee431640 1 --2- 172.21.15.139:0/1616103264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03e810c170 0x7f03e819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:07.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.981+0000 7f03ee431640 1 --2- 172.21.15.139:0/1616103264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03e8103660 0x7f03e8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:07.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.981+0000 7f03ee431640 1 --2- 172.21.15.139:0/1616103264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03e8102d40 0x7f03e8198410 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:07.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.981+0000 7f03ee431640 1 -- 172.21.15.139:0/1616103264 >> 172.21.15.139:0/1616103264 conn(0x7f03e80fc7a0 msgr2=0x7f03e80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:07.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.981+0000 7f03ee431640 1 -- 172.21.15.139:0/1616103264 shutdown_connections 2024-09-16T01:40:07.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:07.981+0000 7f03ee431640 1 -- 172.21.15.139:0/1616103264 wait complete. 2024-09-16T01:40:08.648 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:08 smithi139 ceph-mon[31175]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:40:08.648 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:08 smithi139 ceph-mon[31175]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T01:40:08.648 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:08 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1616103264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:08.675 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T01:40:08.921 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:08.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:08 smithi176 ceph-mon[35324]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:40:08.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:08 smithi176 ceph-mon[35324]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T01:40:08.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:08 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1616103264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:09.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:08 smithi164 ceph-mon[35121]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T01:40:09.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:08 smithi164 ceph-mon[35121]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T01:40:09.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:08 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1616103264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:09.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.398+0000 7f97bd3b7640 1 -- 172.21.15.139:0/451392660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97b8101bc0 msgr2=0x7f97b810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:09.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.398+0000 7f97bd3b7640 1 --2- 172.21.15.139:0/451392660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97b8101bc0 0x7f97b810d910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009ca0 tx=0x7f97b002a8c0 comp rx=0 tx=0).stop 2024-09-16T01:40:09.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.398+0000 7f97bd3b7640 1 -- 172.21.15.139:0/451392660 shutdown_connections 2024-09-16T01:40:09.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.398+0000 7f97bd3b7640 1 --2- 172.21.15.139:0/451392660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97b8101bc0 0x7f97b810d910 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:09.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.398+0000 7f97bd3b7640 1 --2- 172.21.15.139:0/451392660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97b8101240 0x7f97b8101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:09.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.398+0000 7f97bd3b7640 1 --2- 172.21.15.139:0/451392660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97b8105a90 0x7f97b8105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:09.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.398+0000 7f97bd3b7640 1 -- 172.21.15.139:0/451392660 >> 172.21.15.139:0/451392660 conn(0x7f97b8077210 msgr2=0x7f97b80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.398+0000 7f97bd3b7640 1 -- 172.21.15.139:0/451392660 shutdown_connections 2024-09-16T01:40:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.399+0000 7f97bd3b7640 1 -- 172.21.15.139:0/451392660 wait complete. 2024-09-16T01:40:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.399+0000 7f97bd3b7640 1 Processor -- start 2024-09-16T01:40:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97bd3b7640 1 -- start start 2024-09-16T01:40:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97bd3b7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97b8101240 0x7f97b80fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97bd3b7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97b8101bc0 0x7f97b80fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97bd3b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97b8105a90 0x7f97b80fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97bd3b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97b8110f70 con 0x7f97b8105a90 2024-09-16T01:40:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97bd3b7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f97b8110df0 con 0x7f97b8101bc0 2024-09-16T01:40:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97bd3b7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81110f0 con 0x7f97b8101240 2024-09-16T01:40:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97b77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97b8105a90 0x7f97b80fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97b77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97b8105a90 0x7f97b80fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59828/0 (socket says 172.21.15.139:59828) 2024-09-16T01:40:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97b77fe640 1 -- 172.21.15.139:0/1684835019 learned_addr learned my addr 172.21.15.139:0/1684835019 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97b6ffd640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97b8101240 0x7f97b80fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97b67fc640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97b8101bc0 0x7f97b80fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97b77fe640 1 -- 172.21.15.139:0/1684835019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97b8101240 msgr2=0x7f97b80fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97b77fe640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97b8101240 0x7f97b80fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97b77fe640 1 -- 172.21.15.139:0/1684835019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97b8101bc0 msgr2=0x7f97b80fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97b77fe640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97b8101bc0 0x7f97b80fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97b77fe640 1 -- 172.21.15.139:0/1684835019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b81a6950 con 0x7f97b8105a90 2024-09-16T01:40:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97b67fc640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97b8101bc0 0x7f97b80fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.400+0000 7f97b77fe640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97b8105a90 0x7f97b80fd110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f97b0006010 tx=0x7f97b0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.401+0000 7f979ffff640 1 -- 172.21.15.139:0/1684835019 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0031650 con 0x7f97b8105a90 2024-09-16T01:40:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.401+0000 7f97bd3b7640 1 -- 172.21.15.139:0/1684835019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b81a6be0 con 0x7f97b8105a90 2024-09-16T01:40:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.401+0000 7f979ffff640 1 -- 172.21.15.139:0/1684835019 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b00317f0 con 0x7f97b8105a90 2024-09-16T01:40:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.401+0000 7f979ffff640 1 -- 172.21.15.139:0/1684835019 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b00309e0 con 0x7f97b8105a90 2024-09-16T01:40:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.401+0000 7f97bd3b7640 1 -- 172.21.15.139:0/1684835019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b81a7120 con 0x7f97b8105a90 2024-09-16T01:40:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.402+0000 7f97bd3b7640 1 -- 172.21.15.139:0/1684835019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b8101680 con 0x7f97b8105a90 2024-09-16T01:40:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.403+0000 7f979ffff640 1 -- 172.21.15.139:0/1684835019 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f97b0030c60 con 0x7f97b8105a90 2024-09-16T01:40:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.404+0000 7f979ffff640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97a00a3240 0x7f97a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.404+0000 7f97b6ffd640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97a00a3240 0x7f97a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.404+0000 7f979ffff640 1 -- 172.21.15.139:0/1684835019 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f97b00f1360 con 0x7f97b8105a90 2024-09-16T01:40:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.404+0000 7f97b6ffd640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97a00a3240 0x7f97a00a5700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f97a8005d00 tx=0x7f97a800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.407+0000 7f979ffff640 1 -- 172.21.15.139:0/1684835019 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97b00bdcb0 con 0x7f97b8105a90 2024-09-16T01:40:09.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.581+0000 7f97bd3b7640 1 -- 172.21.15.139:0/1684835019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97b81a73b0 con 0x7f97b8105a90 2024-09-16T01:40:09.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.583+0000 7f979ffff640 1 -- 172.21.15.139:0/1684835019 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24985 (secure 0 0 0) 0x7f97b00bd5b0 con 0x7f97b8105a90 2024-09-16T01:40:09.585 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:40:09.585 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":85,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","created":"2024-09-16T01:30:17.770313+0000","modified":"2024-09-16T01:39:09.511648+0000","last_up_change":"2024-09-16T01:37:08.611901+0000","last_in_change":"2024-09-16T01:36:45.223926+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:33:19.639647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T01:37:57.948214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T01:38:00.263797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T01:38:06.773799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b66696da-ef66-4f25-9527-bbcc9ded2cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6803","nonce":3190810133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6805","nonce":3190810133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6809","nonce":3190810133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6807","nonce":3190810133}]},"public_addr":"172.21.15.139:6803/3190810133","cluster_addr":"172.21.15.139:6805/3190810133","heartbeat_back_addr":"172.21.15.139:6809/3190810133","heartbeat_front_addr":"172.21.15.139:6807/3190810133","state":["exists","up"]},{"osd":1,"uuid":"ccad17f2-0750-425e-9810-49a13a929ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6801","nonce":2699270330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6803","nonce":2699270330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6807","nonce":2699270330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6805","nonce":2699270330}]},"public_addr":"172.21.15.164:6801/2699270330","cluster_addr":"172.21.15.164:6803/2699270330","heartbeat_back_addr":"172.21.15.164:6807/2699270330","heartbeat_front_addr":"172.21.15.164:6805/2699270330","state":["exists","up"]},{"osd":2,"uuid":"baf0ba73-8faf-4e51-a171-c47898800638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6801","nonce":2034471646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6803","nonce":2034471646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6807","nonce":2034471646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6805","nonce":2034471646}]},"public_addr":"172.21.15.176:6801/2034471646","cluster_addr":"172.21.15.176:6803/2034471646","heartbeat_back_addr":"172.21.15.176:6807/2034471646","heartbeat_front_addr":"172.21.15.176:6805/2034471646","state":["exists","up"]},{"osd":3,"uuid":"98957bf1-67bb-4092-9ea4-b263a5706f8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6811","nonce":797147071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6813","nonce":797147071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6817","nonce":797147071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6815","nonce":797147071}]},"public_addr":"172.21.15.139:6811/797147071","cluster_addr":"172.21.15.139:6813/797147071","heartbeat_back_addr":"172.21.15.139:6817/797147071","heartbeat_front_addr":"172.21.15.139:6815/797147071","state":["exists","up"]},{"osd":4,"uuid":"9cd76696-8161-4e82-a571-29e98c11d5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6809","nonce":1421304792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6811","nonce":1421304792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6815","nonce":1421304792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6813","nonce":1421304792}]},"public_addr":"172.21.15.164:6809/1421304792","cluster_addr":"172.21.15.164:6811/1421304792","heartbeat_back_addr":"172.21.15.164:6815/1421304792","heartbeat_front_addr":"172.21.15.164:6813/1421304792","state":["exists","up"]},{"osd":5,"uuid":"ccc17a29-9a58-40ec-b453-5c38bee86f25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6809","nonce":10600602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6811","nonce":10600602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6815","nonce":10600602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6813","nonce":10600602}]},"public_addr":"172.21.15.176:6809/10600602","cluster_addr":"172.21.15.176:6811/10600602","heartbeat_back_addr":"172.21.15.176:6815/10600602","heartbeat_front_addr":"172.21.15.176:6813/10600602","state":["exists","up"]},{"osd":6,"uuid":"e5ee7212-10c6-4a05-992e-622a14a5a6b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6819","nonce":2601111164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6821","nonce":2601111164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6825","nonce":2601111164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6823","nonce":2601111164}]},"public_addr":"172.21.15.139:6819/2601111164","cluster_addr":"172.21.15.139:6821/2601111164","heartbeat_back_addr":"172.21.15.139:6825/2601111164","heartbeat_front_addr":"172.21.15.139:6823/2601111164","state":["exists","up"]},{"osd":7,"uuid":"23143c15-0839-4fd5-9d0a-19ae891000f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6817","nonce":2074080693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6819","nonce":2074080693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6823","nonce":2074080693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6821","nonce":2074080693}]},"public_addr":"172.21.15.164:6817/2074080693","cluster_addr":"172.21.15.164:6819/2074080693","heartbeat_back_addr":"172.21.15.164:6823/2074080693","heartbeat_front_addr":"172.21.15.164:6821/2074080693","state":["exists","up"]},{"osd":8,"uuid":"1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6817","nonce":593133361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6819","nonce":593133361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6823","nonce":593133361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6821","nonce":593133361}]},"public_addr":"172.21.15.176:6817/593133361","cluster_addr":"172.21.15.176:6819/593133361","heartbeat_back_addr":"172.21.15.176:6823/593133361","heartbeat_front_addr":"172.21.15.176:6821/593133361","state":["exists","up"]},{"osd":9,"uuid":"09d25a3b-1e66-480e-9433-ceef7f0b9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6827","nonce":3047942108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6829","nonce":3047942108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6833","nonce":3047942108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6831","nonce":3047942108}]},"public_addr":"172.21.15.139:6827/3047942108","cluster_addr":"172.21.15.139:6829/3047942108","heartbeat_back_addr":"172.21.15.139:6833/3047942108","heartbeat_front_addr":"172.21.15.139:6831/3047942108","state":["exists","up"]},{"osd":10,"uuid":"32b4d8ce-b644-42fa-80cc-a15a1f5f3a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6825","nonce":4172533828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6827","nonce":4172533828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6831","nonce":4172533828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6829","nonce":4172533828}]},"public_addr":"172.21.15.164:6825/4172533828","cluster_addr":"172.21.15.164:6827/4172533828","heartbeat_back_addr":"172.21.15.164:6831/4172533828","heartbeat_front_addr":"172.21.15.164:6829/4172533828","state":["exists","up"]},{"osd":11,"uuid":"49098bdc-58a9-4a74-8045-d408af7711ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6825","nonce":1854547565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6827","nonce":1854547565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6831","nonce":1854547565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6829","nonce":1854547565}]},"public_addr":"172.21.15.176:6825/1854547565","cluster_addr":"172.21.15.176:6827/1854547565","heartbeat_back_addr":"172.21.15.176:6831/1854547565","heartbeat_front_addr":"172.21.15.176:6829/1854547565","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:27.374845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:52.611853+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:17.229567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:43.330503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:07.988876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:33.787864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:58.864910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:23.815006+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:49.551954+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:15.496773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:41.647557+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:37:07.089355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3772232169":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/3997627612":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/1608930262":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6801/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6800/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/677216867":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6801/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/1830135963":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/2608207648":"2024-09-17T01:30:44.751168+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:40:09.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.586+0000 7f97bd3b7640 1 -- 172.21.15.139:0/1684835019 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97a00a3240 msgr2=0x7f97a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:09.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.586+0000 7f97bd3b7640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97a00a3240 0x7f97a00a5700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f97a8005d00 tx=0x7f97a800b410 comp rx=0 tx=0).stop 2024-09-16T01:40:09.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.586+0000 7f97bd3b7640 1 -- 172.21.15.139:0/1684835019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97b8105a90 msgr2=0x7f97b80fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:09.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.586+0000 7f97bd3b7640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97b8105a90 0x7f97b80fd110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f97b0006010 tx=0x7f97b0003f10 comp rx=0 tx=0).stop 2024-09-16T01:40:09.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.586+0000 7f97bd3b7640 1 -- 172.21.15.139:0/1684835019 shutdown_connections 2024-09-16T01:40:09.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.586+0000 7f97bd3b7640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f97a00a3240 0x7f97a00a5700 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:09.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.586+0000 7f97bd3b7640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97b8105a90 0x7f97b80fd110 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:09.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.586+0000 7f97bd3b7640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97b8101bc0 0x7f97b80fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:09.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.586+0000 7f97bd3b7640 1 --2- 172.21.15.139:0/1684835019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97b8101240 0x7f97b80fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:09.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.586+0000 7f97bd3b7640 1 -- 172.21.15.139:0/1684835019 >> 172.21.15.139:0/1684835019 conn(0x7f97b8077210 msgr2=0x7f97b81002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:09.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.586+0000 7f97bd3b7640 1 -- 172.21.15.139:0/1684835019 shutdown_connections 2024-09-16T01:40:09.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:09.586+0000 7f97bd3b7640 1 -- 172.21.15.139:0/1684835019 wait complete. 2024-09-16T01:40:10.220 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T01:38:02.129081+0000', 'modified': '2024-09-16T01:38:40.558170+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24470, 'mds_1': 24454, 'mds_2': 24443, 'mds_3': 24442, 'mds_4': 24433}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14604': {'gid': 14604, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6839/1075273450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 1075273450}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 1075273450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14616': {'gid': 14616, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6841/1887551794', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6840', 'nonce': 1887551794}, {'type': 'v1', 'addr': '172.21.15.139:6841', 'nonce': 1887551794}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24395': {'gid': 24395, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6833/416925170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 416925170}, {'type': 'v1', 'addr': '172.21.15.164:6833', 'nonce': 416925170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.176:6833/1026776933', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6832', 'nonce': 1026776933}, {'type': 'v1', 'addr': '172.21.15.176:6833', 'nonce': 1026776933}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6835/3662238682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 3662238682}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 3662238682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.176:6835/3060423566', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6834', 'nonce': 3060423566}, {'type': 'v1', 'addr': '172.21.15.176:6835', 'nonce': 3060423566}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.176:6837/1942543693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6836', 'nonce': 1942543693}, {'type': 'v1', 'addr': '172.21.15.176:6837', 'nonce': 1942543693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.164:6837/1270226509', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 1270226509}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 1270226509}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.176:6839/3912805068', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6838', 'nonce': 3912805068}, {'type': 'v1', 'addr': '172.21.15.176:6839', 'nonce': 3912805068}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.164:6839/2447189436', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 2447189436}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 2447189436}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24470, 'qdb_cluster': [24470, 24454, 24443, 24442, 24433]} 2024-09-16T01:40:10.220 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-16T01:40:10.220 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-16T01:40:10.466 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:10.870 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:10 smithi139 ceph-mon[31175]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T01:40:10.870 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:10 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1684835019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:40:10.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:10 smithi176 ceph-mon[35324]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T01:40:10.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:10 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1684835019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:40:10.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.949+0000 7f7e43dd1640 1 -- 172.21.15.139:0/130794219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3c108290 msgr2=0x7f7e3c10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:10.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.949+0000 7f7e43dd1640 1 --2- 172.21.15.139:0/130794219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3c108290 0x7f7e3c10ec30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009cd0 tx=0x7f7e3402a8e0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.949+0000 7f7e43dd1640 1 -- 172.21.15.139:0/130794219 shutdown_connections 2024-09-16T01:40:10.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.949+0000 7f7e43dd1640 1 --2- 172.21.15.139:0/130794219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3c108290 0x7f7e3c10ec30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.949+0000 7f7e43dd1640 1 --2- 172.21.15.139:0/130794219 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e3c103740 0x7f7e3c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.949+0000 7f7e43dd1640 1 --2- 172.21.15.139:0/130794219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e3c102d90 0x7f7e3c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.949+0000 7f7e43dd1640 1 -- 172.21.15.139:0/130794219 >> 172.21.15.139:0/130794219 conn(0x7f7e3c0fc7d0 msgr2=0x7f7e3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:10.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.949+0000 7f7e43dd1640 1 -- 172.21.15.139:0/130794219 shutdown_connections 2024-09-16T01:40:10.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.949+0000 7f7e43dd1640 1 -- 172.21.15.139:0/130794219 wait complete. 2024-09-16T01:40:10.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.949+0000 7f7e43dd1640 1 Processor -- start 2024-09-16T01:40:10.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.949+0000 7f7e43dd1640 1 -- start start 2024-09-16T01:40:10.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e43dd1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e3c102d90 0x7f7e3c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e43dd1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3c103740 0x7f7e3c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e43dd1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e3c108290 0x7f7e3c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e43dd1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c112090 con 0x7f7e3c103740 2024-09-16T01:40:10.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e43dd1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c111f10 con 0x7f7e3c108290 2024-09-16T01:40:10.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e43dd1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c112210 con 0x7f7e3c102d90 2024-09-16T01:40:10.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e4136c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3c103740 0x7f7e3c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e4136c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3c103740 0x7f7e3c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59850/0 (socket says 172.21.15.139:59850) 2024-09-16T01:40:10.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e4136c640 1 -- 172.21.15.139:0/2281220493 learned_addr learned my addr 172.21.15.139:0/2281220493 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:10.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e4236e640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e3c108290 0x7f7e3c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e41b6d640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e3c102d90 0x7f7e3c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e4136c640 1 -- 172.21.15.139:0/2281220493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e3c102d90 msgr2=0x7f7e3c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:10.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e4136c640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e3c102d90 0x7f7e3c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e4136c640 1 -- 172.21.15.139:0/2281220493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e3c108290 msgr2=0x7f7e3c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:10.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e4136c640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e3c108290 0x7f7e3c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e4136c640 1 -- 172.21.15.139:0/2281220493 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3c19d510 con 0x7f7e3c103740 2024-09-16T01:40:10.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e41b6d640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e3c102d90 0x7f7e3c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:10.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e4236e640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e3c108290 0x7f7e3c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:10.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e4136c640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3c103740 0x7f7e3c198aa0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c005fd0 tx=0x7f7e2c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e32ffd640 1 -- 172.21.15.139:0/2281220493 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c017070 con 0x7f7e3c103740 2024-09-16T01:40:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.950+0000 7f7e32ffd640 1 -- 172.21.15.139:0/2281220493 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e2c009c30 con 0x7f7e3c103740 2024-09-16T01:40:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.951+0000 7f7e32ffd640 1 -- 172.21.15.139:0/2281220493 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c010460 con 0x7f7e3c103740 2024-09-16T01:40:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.951+0000 7f7e43dd1640 1 -- 172.21.15.139:0/2281220493 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3c19d800 con 0x7f7e3c103740 2024-09-16T01:40:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.951+0000 7f7e43dd1640 1 -- 172.21.15.139:0/2281220493 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3c1a50e0 con 0x7f7e3c103740 2024-09-16T01:40:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.953+0000 7f7e32ffd640 1 -- 172.21.15.139:0/2281220493 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f7e2c003e70 con 0x7f7e3c103740 2024-09-16T01:40:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.953+0000 7f7e461aa740 1 Processor -- start 2024-09-16T01:40:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.953+0000 7f7e461aa740 1 -- start start 2024-09-16T01:40:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.953+0000 7f7e32ffd640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7e280a3240 0x7f7e280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e461aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f1f8b0 0x55be96f1fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e461aa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f201d0 0x55be96f2ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e461aa740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f2efc0 0x55be96f313b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e461aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55be96e86090 con 0x55be96f1f8b0 2024-09-16T01:40:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e461aa740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55be96e2da70 con 0x55be96f2efc0 2024-09-16T01:40:10.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e461aa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55be96e82050 con 0x55be96f201d0 2024-09-16T01:40:10.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e4236e640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7e280a3240 0x7f7e280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e41b6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f1f8b0 0x55be96f1fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e32ffd640 1 -- 172.21.15.139:0/2281220493 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f7e2c0cf970 con 0x7f7e3c103740 2024-09-16T01:40:10.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e41b6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f1f8b0 0x55be96f1fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59854/0 (socket says 172.21.15.139:59854) 2024-09-16T01:40:10.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e41b6d640 1 -- 172.21.15.139:0/4285021335 learned_addr learned my addr 172.21.15.139:0/4285021335 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:10.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e4136c640 1 --2- 172.21.15.139:0/4285021335 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f201d0 0x55be96f2ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e41b6d640 1 -- 172.21.15.139:0/4285021335 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f201d0 msgr2=0x55be96f2ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:10.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e41b6d640 1 --2- 172.21.15.139:0/4285021335 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f201d0 0x55be96f2ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e41b6d640 1 -- 172.21.15.139:0/4285021335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f2efc0 msgr2=0x55be96f313b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:10.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e41b6d640 1 --2- 172.21.15.139:0/4285021335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f2efc0 0x55be96f313b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e41b6d640 1 -- 172.21.15.139:0/4285021335 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be96e3a130 con 0x55be96f1f8b0 2024-09-16T01:40:10.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e4136c640 1 --2- 172.21.15.139:0/4285021335 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f201d0 0x55be96f2ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e4236e640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7e280a3240 0x7f7e280a5700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009430 tx=0x7f7e34005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e41b6d640 1 --2- 172.21.15.139:0/4285021335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f1f8b0 0x55be96f1fc90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009d50 tx=0x7f7e380117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa71f7be36bbeb99 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e27fff640 1 -- 172.21.15.139:0/4285021335 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e38012650 con 0x55be96f1f8b0 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e27fff640 1 -- 172.21.15.139:0/4285021335 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e38009e40 con 0x55be96f1f8b0 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.954+0000 7f7e27fff640 1 -- 172.21.15.139:0/4285021335 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800fc50 con 0x55be96f1f8b0 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.955+0000 7f7e461aa740 1 -- 172.21.15.139:0/4285021335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f1f8b0 msgr2=0x55be96f1fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.955+0000 7f7e461aa740 1 --2- 172.21.15.139:0/4285021335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f1f8b0 0x55be96f1fc90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009d50 tx=0x7f7e380117b0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.955+0000 7f7e461aa740 1 -- 172.21.15.139:0/4285021335 shutdown_connections 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.955+0000 7f7e461aa740 1 --2- 172.21.15.139:0/4285021335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f2efc0 0x55be96f313b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.955+0000 7f7e461aa740 1 --2- 172.21.15.139:0/4285021335 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f201d0 0x55be96f2ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.955+0000 7f7e461aa740 1 --2- 172.21.15.139:0/4285021335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f1f8b0 0x55be96f1fc90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.955+0000 7f7e461aa740 1 -- 172.21.15.139:0/4285021335 >> 172.21.15.139:0/4285021335 conn(0x55be96f1bf60 msgr2=0x55be96f1c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.955+0000 7f7e461aa740 1 -- 172.21.15.139:0/4285021335 shutdown_connections 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.955+0000 7f7e461aa740 1 -- 172.21.15.139:0/4285021335 wait complete. 2024-09-16T01:40:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.956+0000 7f7e461aa740 1 -- start start 2024-09-16T01:40:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.956+0000 7f7e461aa740 1 Processor -- start 2024-09-16T01:40:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.956+0000 7f7e461aa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f1f8b0 0x55be96f2e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.956+0000 7f7e461aa740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f201d0 0x55be96f2bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.956+0000 7f7e461aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f2c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.956+0000 7f7e461aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55be96f31b90 con 0x55be96f2efc0 2024-09-16T01:40:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.956+0000 7f7e461aa740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55be96e40f20 con 0x55be96f201d0 2024-09-16T01:40:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.956+0000 7f7e461aa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55be96f31d10 con 0x55be96f1f8b0 2024-09-16T01:40:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.956+0000 7f7e461aa740 10 client.0 _ll_register_callbacks cb 0x55be96e0a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.956+0000 7f7e4236e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f2c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.956+0000 7f7e4236e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f2c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59856/0 (socket says 172.21.15.139:59856) 2024-09-16T01:40:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.956+0000 7f7e4236e640 1 -- 172.21.15.139:0/335070070 learned_addr learned my addr 172.21.15.139:0/335070070 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.956+0000 7f7e4136c640 1 --2- 172.21.15.139:0/335070070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f201d0 0x55be96f2bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.956+0000 7f7e41b6d640 1 --2- 172.21.15.139:0/335070070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f1f8b0 0x55be96f2e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e4236e640 1 -- 172.21.15.139:0/335070070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f1f8b0 msgr2=0x55be96f2e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e4236e640 1 --2- 172.21.15.139:0/335070070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f1f8b0 0x55be96f2e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e4236e640 1 -- 172.21.15.139:0/335070070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f201d0 msgr2=0x55be96f2bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e4236e640 1 --2- 172.21.15.139:0/335070070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f201d0 0x55be96f2bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e4236e640 1 -- 172.21.15.139:0/335070070 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be96bf72b0 con 0x55be96f2efc0 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e4236e640 1 --2- 172.21.15.139:0/335070070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f2c0b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7e34035000 tx=0x7f7e3402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747f6d2de1b5d90a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e257fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e257fa640 1 -- 172.21.15.139:0/335070070 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34043070 con 0x55be96f2efc0 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e257fa640 1 -- 172.21.15.139:0/335070070 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f7e340047c0 con 0x55be96f2efc0 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e461aa740 1 -- 172.21.15.139:0/335070070 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be96e93600 con 0x55be96f2efc0 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e257fa640 1 -- 172.21.15.139:0/335070070 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e34057050 con 0x55be96f2efc0 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e461aa740 10 client.15222 fetch_fsmap learned FSMap version 28 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e461aa740 1 -- 172.21.15.139:0/335070070 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be96f3d450 con 0x55be96f2efc0 2024-09-16T01:40:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e257fa640 1 -- 172.21.15.139:0/335070070 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55be96e93600 con 0x55be96f2efc0 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e257fa640 1 -- 172.21.15.139:0/335070070 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7e34030070 con 0x55be96f2efc0 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.957+0000 7f7e461aa740 10 client.15222 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.958+0000 7f7e461aa740 10 client.15222 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.958+0000 7f7e461aa740 20 client.15222 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.958+0000 7f7e461aa740 1 --2- 172.21.15.139:0/335070070 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55be96f3fae0 0x55be96f40f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.958+0000 7f7e461aa740 4 client.15222 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.958+0000 7f7e461aa740 1 -- 172.21.15.139:0/335070070 --> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be96e8db10 con 0x55be96f3fae0 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.958+0000 7f7e41b6d640 1 --2- 172.21.15.139:0/335070070 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55be96f3fae0 0x55be96f40f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.958+0000 7f7e257fa640 10 client.15222 ms_handle_connect on v2:172.21.15.176:6834/3060423566 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.958+0000 7f7e41b6d640 1 --2- 172.21.15.139:0/335070070 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55be96f3fae0 0x55be96f40f50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=776b90170fd52e63 server_cookie=55de519e960911fe in_seq=0 out_seq=0 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.959+0000 7f7e257fa640 1 -- 172.21.15.139:0/335070070 <== mds.4 v2:172.21.15.176:6834/3060423566 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be96e8db10 con 0x55be96f3fae0 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.959+0000 7f7e257fa640 10 client.15222 handle_command_reply: tid=0 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.959+0000 7f7e461aa740 1 client.15222 shutdown 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.959+0000 7f7e461aa740 1 -- 172.21.15.139:0/335070070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 msgr2=0x55be96f2c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:10.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.959+0000 7f7e461aa740 1 --2- 172.21.15.139:0/335070070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f2c0b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7e34035000 tx=0x7f7e3402a8e0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.959+0000 7f7e461aa740 1 -- 172.21.15.139:0/335070070 shutdown_connections 2024-09-16T01:40:10.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.959+0000 7f7e461aa740 1 --2- 172.21.15.139:0/335070070 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55be96f3fae0 0x55be96f40f50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.959+0000 7f7e461aa740 1 --2- 172.21.15.139:0/335070070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f2c0b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.959+0000 7f7e461aa740 1 --2- 172.21.15.139:0/335070070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f201d0 0x55be96f2bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.959+0000 7f7e257fa640 0 client.15222 ms_handle_reset on v2:172.21.15.176:6834/3060423566 2024-09-16T01:40:10.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.959+0000 7f7e461aa740 1 --2- 172.21.15.139:0/335070070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f1f8b0 0x55be96f2e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.959+0000 7f7e461aa740 1 -- 172.21.15.139:0/335070070 >> 172.21.15.139:0/335070070 conn(0x55be96f1bf60 msgr2=0x55be96f21cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:10.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.960+0000 7f7e461aa740 1 -- 172.21.15.139:0/335070070 shutdown_connections 2024-09-16T01:40:10.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.960+0000 7f7e461aa740 1 -- 172.21.15.139:0/335070070 wait complete. 2024-09-16T01:40:10.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.960+0000 7f7e461aa740 20 client.15222 trim_cache size 0 max 16384 2024-09-16T01:40:10.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.977+0000 7f7e461aa740 1 Processor -- start 2024-09-16T01:40:10.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.977+0000 7f7e461aa740 1 -- start start 2024-09-16T01:40:10.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.977+0000 7f7e461aa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f1f8b0 0x55be96f40f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.977+0000 7f7e461aa740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f201d0 0x55be96f2a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.977+0000 7f7e461aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f2a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.977+0000 7f7e461aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55be96f3f290 con 0x55be96f2efc0 2024-09-16T01:40:10.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.977+0000 7f7e461aa740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55be96f3f960 con 0x55be96f201d0 2024-09-16T01:40:10.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.977+0000 7f7e461aa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55be96f40030 con 0x55be96f1f8b0 2024-09-16T01:40:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.977+0000 7f7e4236e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f2a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.977+0000 7f7e4236e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f2a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59870/0 (socket says 172.21.15.139:59870) 2024-09-16T01:40:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.977+0000 7f7e4236e640 1 -- 172.21.15.139:0/1304441860 learned_addr learned my addr 172.21.15.139:0/1304441860 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.978+0000 7f7e41b6d640 1 --2- 172.21.15.139:0/1304441860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f1f8b0 0x55be96f40f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.978+0000 7f7e4136c640 1 --2- 172.21.15.139:0/1304441860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f201d0 0x55be96f2a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.978+0000 7f7e4236e640 1 -- 172.21.15.139:0/1304441860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f1f8b0 msgr2=0x55be96f40f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.978+0000 7f7e4236e640 1 --2- 172.21.15.139:0/1304441860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f1f8b0 0x55be96f40f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.978+0000 7f7e4236e640 1 -- 172.21.15.139:0/1304441860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f201d0 msgr2=0x55be96f2a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.978+0000 7f7e4236e640 1 --2- 172.21.15.139:0/1304441860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f201d0 0x55be96f2a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.978+0000 7f7e4236e640 1 -- 172.21.15.139:0/1304441860 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be96f20000 con 0x55be96f2efc0 2024-09-16T01:40:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.978+0000 7f7e4136c640 1 --2- 172.21.15.139:0/1304441860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f201d0 0x55be96f2a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.978+0000 7f7e4236e640 1 --2- 172.21.15.139:0/1304441860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f2a9e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7e34031450 tx=0x7f7e34048cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2db7dd144e4095 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.978+0000 7f7e277fe640 1 -- 172.21.15.139:0/1304441860 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34004410 con 0x55be96f2efc0 2024-09-16T01:40:10.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.978+0000 7f7e277fe640 1 -- 172.21.15.139:0/1304441860 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e34036040 con 0x55be96f2efc0 2024-09-16T01:40:10.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.978+0000 7f7e277fe640 1 -- 172.21.15.139:0/1304441860 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34030ce0 con 0x55be96f2efc0 2024-09-16T01:40:10.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.979+0000 7f7e461aa740 1 -- 172.21.15.139:0/1304441860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 msgr2=0x55be96f2a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:10.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.979+0000 7f7e461aa740 1 --2- 172.21.15.139:0/1304441860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f2a9e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7e34031450 tx=0x7f7e34048cb0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.979+0000 7f7e461aa740 1 -- 172.21.15.139:0/1304441860 shutdown_connections 2024-09-16T01:40:10.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.979+0000 7f7e461aa740 1 --2- 172.21.15.139:0/1304441860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f2a9e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.979+0000 7f7e461aa740 1 --2- 172.21.15.139:0/1304441860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f201d0 0x55be96f2a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.979+0000 7f7e461aa740 1 --2- 172.21.15.139:0/1304441860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f1f8b0 0x55be96f40f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.979+0000 7f7e461aa740 1 -- 172.21.15.139:0/1304441860 >> 172.21.15.139:0/1304441860 conn(0x55be96f1bf60 msgr2=0x55be96f2bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:10.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.979+0000 7f7e461aa740 1 -- 172.21.15.139:0/1304441860 shutdown_connections 2024-09-16T01:40:10.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.979+0000 7f7e461aa740 1 -- 172.21.15.139:0/1304441860 wait complete. 2024-09-16T01:40:10.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.979+0000 7f7e461aa740 1 -- start start 2024-09-16T01:40:10.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.979+0000 7f7e461aa740 1 Processor -- start 2024-09-16T01:40:10.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e461aa740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f1f8b0 0x55be96f285e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e461aa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f201d0 0x55be96f28b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e461aa740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f481a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e461aa740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55be96f40dd0 con 0x55be96f2efc0 2024-09-16T01:40:10.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e461aa740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55be96f40700 con 0x55be96f1f8b0 2024-09-16T01:40:10.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e461aa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55be96f414a0 con 0x55be96f201d0 2024-09-16T01:40:10.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e461aa740 10 client.0 _ll_register_callbacks cb 0x7f7e38005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:10.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e4236e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e4236e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59876/0 (socket says 172.21.15.139:59876) 2024-09-16T01:40:10.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e4236e640 1 -- 172.21.15.139:0/808219428 learned_addr learned my addr 172.21.15.139:0/808219428 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:10.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e4136c640 1 --2- 172.21.15.139:0/808219428 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f201d0 0x55be96f28b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e41b6d640 1 --2- 172.21.15.139:0/808219428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f1f8b0 0x55be96f285e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e4236e640 1 -- 172.21.15.139:0/808219428 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f201d0 msgr2=0x55be96f28b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:10.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e4236e640 1 --2- 172.21.15.139:0/808219428 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f201d0 0x55be96f28b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e4236e640 1 -- 172.21.15.139:0/808219428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f1f8b0 msgr2=0x55be96f285e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:10.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e4236e640 1 --2- 172.21.15.139:0/808219428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f1f8b0 0x55be96f285e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e4236e640 1 -- 172.21.15.139:0/808219428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be96f487e0 con 0x55be96f2efc0 2024-09-16T01:40:10.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e41b6d640 1 --2- 172.21.15.139:0/808219428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f1f8b0 0x55be96f285e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:10.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e4236e640 1 --2- 172.21.15.139:0/808219428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f481a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7e3402a920 tx=0x7f7e3402afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece765bb63f61fd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:10.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e257fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:10.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e257fa640 1 -- 172.21.15.139:0/808219428 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34068020 con 0x55be96f2efc0 2024-09-16T01:40:10.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e257fa640 1 -- 172.21.15.139:0/808219428 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f7e340305a0 con 0x55be96f2efc0 2024-09-16T01:40:10.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.980+0000 7f7e461aa740 1 -- 172.21.15.139:0/808219428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e34036040 con 0x55be96f2efc0 2024-09-16T01:40:10.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.981+0000 7f7e257fa640 1 -- 172.21.15.139:0/808219428 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e34073050 con 0x55be96f2efc0 2024-09-16T01:40:10.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.981+0000 7f7e461aa740 10 client.15228 fetch_fsmap learned FSMap version 28 2024-09-16T01:40:10.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.981+0000 7f7e461aa740 1 -- 172.21.15.139:0/808219428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be96f49ac0 con 0x55be96f2efc0 2024-09-16T01:40:10.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.981+0000 7f7e257fa640 1 -- 172.21.15.139:0/808219428 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e34036040 con 0x55be96f2efc0 2024-09-16T01:40:10.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.981+0000 7f7e257fa640 1 -- 172.21.15.139:0/808219428 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7e34051690 con 0x55be96f2efc0 2024-09-16T01:40:10.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.981+0000 7f7e461aa740 10 client.15228 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:40:10.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.981+0000 7f7e461aa740 10 client.15228 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T01:40:10.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.981+0000 7f7e461aa740 20 client.15228 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:10.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.981+0000 7f7e461aa740 1 --2- 172.21.15.139:0/808219428 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55be96fae470 0x55be96fad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:10.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.981+0000 7f7e461aa740 4 client.15228 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:40:10.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.981+0000 7f7e461aa740 1 -- 172.21.15.139:0/808219428 --> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] -- command(tid 0: {"prefix": "status"}) -- 0x55be96e0a0e0 con 0x55be96fae470 2024-09-16T01:40:10.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.981+0000 7f7e41b6d640 1 --2- 172.21.15.139:0/808219428 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55be96fae470 0x55be96fad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:10.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e257fa640 10 client.15228 ms_handle_connect on v2:172.21.15.176:6834/3060423566 2024-09-16T01:40:10.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e41b6d640 1 --2- 172.21.15.139:0/808219428 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55be96fae470 0x55be96fad180 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a97d379511a60b1d server_cookie=32e87c112eb1476e in_seq=0 out_seq=0 2024-09-16T01:40:10.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e257fa640 1 -- 172.21.15.139:0/808219428 <== mds.4 v2:172.21.15.176:6834/3060423566 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55be96e0a0e0 con 0x55be96fae470 2024-09-16T01:40:10.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e257fa640 10 client.15228 handle_command_reply: tid=0 2024-09-16T01:40:10.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e461aa740 1 client.15228 shutdown 2024-09-16T01:40:10.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e461aa740 1 -- 172.21.15.139:0/808219428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 msgr2=0x55be96f481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:10.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e461aa740 1 --2- 172.21.15.139:0/808219428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f481a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7e3402a920 tx=0x7f7e3402afa0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e461aa740 1 -- 172.21.15.139:0/808219428 shutdown_connections 2024-09-16T01:40:10.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e461aa740 1 --2- 172.21.15.139:0/808219428 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55be96fae470 0x55be96fad180 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e461aa740 1 --2- 172.21.15.139:0/808219428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55be96f2efc0 0x55be96f481a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e461aa740 1 --2- 172.21.15.139:0/808219428 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55be96f201d0 0x55be96f28b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e257fa640 0 client.15228 ms_handle_reset on v2:172.21.15.176:6834/3060423566 2024-09-16T01:40:10.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e461aa740 1 --2- 172.21.15.139:0/808219428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be96f1f8b0 0x55be96f285e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e461aa740 1 -- 172.21.15.139:0/808219428 >> 172.21.15.139:0/808219428 conn(0x55be96f1bf60 msgr2=0x55be96f26a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:10.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e461aa740 1 -- 172.21.15.139:0/808219428 shutdown_connections 2024-09-16T01:40:10.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e461aa740 1 -- 172.21.15.139:0/808219428 wait complete. 2024-09-16T01:40:10.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.982+0000 7f7e461aa740 20 client.15228 trim_cache size 0 max 16384 2024-09-16T01:40:10.997 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:40:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:40:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 4, 2024-09-16T01:40:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24433, 2024-09-16T01:40:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:40:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:40:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:40:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 91.432831153999999, 2024-09-16T01:40:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 28, 2024-09-16T01:40:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 85, 2024-09-16T01:40:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 83, 2024-09-16T01:40:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 157.64962377200001 2024-09-16T01:40:10.998 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:40:10.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.983+0000 7f7e43dd1640 1 -- 172.21.15.139:0/2281220493 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7e280a3240 msgr2=0x7f7e280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:10.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.983+0000 7f7e43dd1640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7e280a3240 0x7f7e280a5700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009430 tx=0x7f7e34005f60 comp rx=0 tx=0).stop 2024-09-16T01:40:10.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.983+0000 7f7e43dd1640 1 -- 172.21.15.139:0/2281220493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3c103740 msgr2=0x7f7e3c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:10.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.983+0000 7f7e43dd1640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3c103740 0x7f7e3c198aa0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c005fd0 tx=0x7f7e2c00dde0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.983+0000 7f7e43dd1640 1 -- 172.21.15.139:0/2281220493 shutdown_connections 2024-09-16T01:40:10.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.983+0000 7f7e43dd1640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f7e280a3240 0x7f7e280a5700 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.983+0000 7f7e43dd1640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e3c108290 0x7f7e3c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.983+0000 7f7e43dd1640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3c103740 0x7f7e3c198aa0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:10.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.983+0000 7f7e43dd1640 1 --2- 172.21.15.139:0/2281220493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e3c102d90 0x7f7e3c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:11.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.983+0000 7f7e43dd1640 1 -- 172.21.15.139:0/2281220493 >> 172.21.15.139:0/2281220493 conn(0x7f7e3c0fc7d0 msgr2=0x7f7e3c106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:11.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.983+0000 7f7e43dd1640 1 -- 172.21.15.139:0/2281220493 shutdown_connections 2024-09-16T01:40:11.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:10.983+0000 7f7e43dd1640 1 -- 172.21.15.139:0/2281220493 wait complete. 2024-09-16T01:40:11.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:10 smithi164 ceph-mon[35121]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T01:40:11.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:10 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1684835019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:40:11.703 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-16T01:40:11.948 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:12.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.610+0000 7f99f9719640 1 -- 172.21.15.139:0/2114831308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f4102790 msgr2=0x7f99f41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:12.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.610+0000 7f99f9719640 1 --2- 172.21.15.139:0/2114831308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f4102790 0x7f99f41067e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009900 tx=0x7f99e802a410 comp rx=0 tx=0).stop 2024-09-16T01:40:12.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.610+0000 7f99f9719640 1 -- 172.21.15.139:0/2114831308 shutdown_connections 2024-09-16T01:40:12.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.610+0000 7f99f9719640 1 --2- 172.21.15.139:0/2114831308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f4106f10 0x7f99f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.610+0000 7f99f9719640 1 --2- 172.21.15.139:0/2114831308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f4102790 0x7f99f41067e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.610+0000 7f99f9719640 1 --2- 172.21.15.139:0/2114831308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99f4101de0 0x7f99f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.610+0000 7f99f9719640 1 -- 172.21.15.139:0/2114831308 >> 172.21.15.139:0/2114831308 conn(0x7f99f40fd8c0 msgr2=0x7f99f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:12.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.610+0000 7f99f9719640 1 -- 172.21.15.139:0/2114831308 shutdown_connections 2024-09-16T01:40:12.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.610+0000 7f99f9719640 1 -- 172.21.15.139:0/2114831308 wait complete. 2024-09-16T01:40:12.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.611+0000 7f99f9719640 1 Processor -- start 2024-09-16T01:40:12.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.611+0000 7f99f9719640 1 -- start start 2024-09-16T01:40:12.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.611+0000 7f99f9719640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f4101de0 0x7f99f40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.611+0000 7f99f9719640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f4102790 0x7f99f4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.611+0000 7f99f9719640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99f4106f10 0x7f99f4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.611+0000 7f99f9719640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99f4111f70 con 0x7f99f4101de0 2024-09-16T01:40:12.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.611+0000 7f99f9719640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f99f4111df0 con 0x7f99f4102790 2024-09-16T01:40:12.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.611+0000 7f99f9719640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f99f41120f0 con 0x7f99f4106f10 2024-09-16T01:40:12.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99f2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f4101de0 0x7f99f40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99f2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f4101de0 0x7f99f40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55300/0 (socket says 172.21.15.139:55300) 2024-09-16T01:40:12.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99f2ffd640 1 -- 172.21.15.139:0/2916868863 learned_addr learned my addr 172.21.15.139:0/2916868863 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:12.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99f37fe640 1 --2- 172.21.15.139:0/2916868863 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99f4106f10 0x7f99f4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99f27fc640 1 --2- 172.21.15.139:0/2916868863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f4102790 0x7f99f4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99f2ffd640 1 -- 172.21.15.139:0/2916868863 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99f4106f10 msgr2=0x7f99f4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:12.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/2916868863 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99f4106f10 0x7f99f4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99f2ffd640 1 -- 172.21.15.139:0/2916868863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f4102790 msgr2=0x7f99f4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:12.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/2916868863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f4102790 0x7f99f4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99f2ffd640 1 -- 172.21.15.139:0/2916868863 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f41a6910 con 0x7f99f4101de0 2024-09-16T01:40:12.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/2916868863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f4101de0 0x7f99f40744f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f99e400d950 tx=0x7f99e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:12.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99dbfff640 1 -- 172.21.15.139:0/2916868863 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e4017070 con 0x7f99f4101de0 2024-09-16T01:40:12.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99dbfff640 1 -- 172.21.15.139:0/2916868863 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e4009c30 con 0x7f99f4101de0 2024-09-16T01:40:12.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99f9719640 1 -- 172.21.15.139:0/2916868863 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f41a6c00 con 0x7f99f4101de0 2024-09-16T01:40:12.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99dbfff640 1 -- 172.21.15.139:0/2916868863 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e400f6b0 con 0x7f99f4101de0 2024-09-16T01:40:12.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.612+0000 7f99f9719640 1 -- 172.21.15.139:0/2916868863 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f41a7110 con 0x7f99f4101de0 2024-09-16T01:40:12.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.615+0000 7f99dbfff640 1 -- 172.21.15.139:0/2916868863 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f99e40109d0 con 0x7f99f4101de0 2024-09-16T01:40:12.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.615+0000 7f99fbaf2740 1 Processor -- start 2024-09-16T01:40:12.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.615+0000 7f99fbaf2740 1 -- start start 2024-09-16T01:40:12.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.615+0000 7f99fbaf2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b3f9d0 0x5613d1b3fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.615+0000 7f99fbaf2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 0x5613d1b4a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.615+0000 7f99fbaf2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b4cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.615+0000 7f99fbaf2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5613d1aa6090 con 0x5613d1b4abb0 2024-09-16T01:40:12.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.615+0000 7f99fbaf2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5613d1a4da70 con 0x5613d1b402f0 2024-09-16T01:40:12.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.615+0000 7f99fbaf2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5613d1aa2050 con 0x5613d1b3f9d0 2024-09-16T01:40:12.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99f37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b3f9d0 0x5613d1b3fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99f27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b4cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99f37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b3f9d0 0x5613d1b3fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:40806/0 (socket says 172.21.15.139:40806) 2024-09-16T01:40:12.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99f37fe640 1 -- 172.21.15.139:0/281988984 learned_addr learned my addr 172.21.15.139:0/281988984 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:12.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/281988984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 0x5613d1b4a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99dbfff640 1 --2- 172.21.15.139:0/2916868863 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f99dc0a3240 0x7f99dc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99f37fe640 1 -- 172.21.15.139:0/281988984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 msgr2=0x5613d1b4a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:12.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99f37fe640 1 --2- 172.21.15.139:0/281988984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 0x5613d1b4a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99f37fe640 1 -- 172.21.15.139:0/281988984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 msgr2=0x5613d1b4cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:12.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99f37fe640 1 --2- 172.21.15.139:0/281988984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b4cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99f37fe640 1 -- 172.21.15.139:0/281988984 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d1a5a130 con 0x5613d1b3f9d0 2024-09-16T01:40:12.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99f37fe640 1 --2- 172.21.15.139:0/2916868863 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f99dc0a3240 0x7f99dc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99dbfff640 1 -- 172.21.15.139:0/2916868863 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f99e40cf860 con 0x7f99f4101de0 2024-09-16T01:40:12.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99f37fe640 1 --2- 172.21.15.139:0/281988984 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b3f9d0 0x5613d1b3fdb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0104a0 tx=0x7f99ec010960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97fccdb94c5bcb91 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:12.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99d97fa640 1 -- 172.21.15.139:0/281988984 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec022070 con 0x5613d1b3f9d0 2024-09-16T01:40:12.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99d97fa640 1 -- 172.21.15.139:0/281988984 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99ec013c60 con 0x5613d1b3f9d0 2024-09-16T01:40:12.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.616+0000 7f99d97fa640 1 -- 172.21.15.139:0/281988984 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec01b3c0 con 0x5613d1b3f9d0 2024-09-16T01:40:12.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.617+0000 7f99fbaf2740 1 -- 172.21.15.139:0/281988984 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b3f9d0 msgr2=0x5613d1b3fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:12.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.617+0000 7f99f37fe640 1 --2- 172.21.15.139:0/2916868863 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f99dc0a3240 0x7f99dc0a5700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f99ec01f000 tx=0x7f99ec01c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:12.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.617+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/281988984 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b3f9d0 0x5613d1b3fdb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0104a0 tx=0x7f99ec010960 comp rx=0 tx=0).stop 2024-09-16T01:40:12.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.617+0000 7f99fbaf2740 1 -- 172.21.15.139:0/281988984 shutdown_connections 2024-09-16T01:40:12.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.617+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/281988984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b4cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.617+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/281988984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 0x5613d1b4a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.617+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/281988984 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b3f9d0 0x5613d1b3fdb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.617+0000 7f99fbaf2740 1 -- 172.21.15.139:0/281988984 >> 172.21.15.139:0/281988984 conn(0x5613d1b3bf60 msgr2=0x5613d1b3c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:12.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.617+0000 7f99fbaf2740 1 -- 172.21.15.139:0/281988984 shutdown_connections 2024-09-16T01:40:12.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.617+0000 7f99fbaf2740 1 -- 172.21.15.139:0/281988984 wait complete. 2024-09-16T01:40:12.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.617+0000 7f99fbaf2740 1 -- start start 2024-09-16T01:40:12.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99fbaf2740 1 Processor -- start 2024-09-16T01:40:12.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99fbaf2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b3f9d0 0x5613d1b4a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99fbaf2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b402f0 0x5613d1b47720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99fbaf2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b47c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99fbaf2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5613d1b4d780 con 0x5613d1b4abb0 2024-09-16T01:40:12.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99fbaf2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5613d1a60f20 con 0x5613d1b3f9d0 2024-09-16T01:40:12.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99fbaf2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5613d1b4d900 con 0x5613d1b402f0 2024-09-16T01:40:12.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99fbaf2740 10 client.0 _ll_register_callbacks cb 0x5613d1a2a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:12.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99f37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b47c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99f37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b47c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55330/0 (socket says 172.21.15.139:55330) 2024-09-16T01:40:12.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99f37fe640 1 -- 172.21.15.139:0/3338729008 learned_addr learned my addr 172.21.15.139:0/3338729008 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:12.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/3338729008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b3f9d0 0x5613d1b4a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99f27fc640 1 --2- 172.21.15.139:0/3338729008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b402f0 0x5613d1b47720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99f37fe640 1 -- 172.21.15.139:0/3338729008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b402f0 msgr2=0x5613d1b47720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:12.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99f37fe640 1 --2- 172.21.15.139:0/3338729008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b402f0 0x5613d1b47720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99f37fe640 1 -- 172.21.15.139:0/3338729008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b3f9d0 msgr2=0x5613d1b4a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:12.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99f37fe640 1 --2- 172.21.15.139:0/3338729008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b3f9d0 0x5613d1b4a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99f37fe640 1 -- 172.21.15.139:0/3338729008 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d18172b0 con 0x5613d1b4abb0 2024-09-16T01:40:12.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99f37fe640 1 --2- 172.21.15.139:0/3338729008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b47c60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00eb40 tx=0x7f99ec013b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4e96e043eff3f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:12.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.618+0000 7f99d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:12.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.619+0000 7f99d6ff5640 1 -- 172.21.15.139:0/3338729008 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec022040 con 0x5613d1b4abb0 2024-09-16T01:40:12.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.619+0000 7f99d6ff5640 1 -- 172.21.15.139:0/3338729008 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f99ec00e430 con 0x5613d1b4abb0 2024-09-16T01:40:12.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.619+0000 7f99fbaf2740 1 -- 172.21.15.139:0/3338729008 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99ec013c60 con 0x5613d1b4abb0 2024-09-16T01:40:12.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.619+0000 7f99d6ff5640 1 -- 172.21.15.139:0/3338729008 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f99ec013280 con 0x5613d1b4abb0 2024-09-16T01:40:12.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.619+0000 7f99fbaf2740 10 client.15240 fetch_fsmap learned FSMap version 28 2024-09-16T01:40:12.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.619+0000 7f99fbaf2740 1 -- 172.21.15.139:0/3338729008 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5613d1b59030 con 0x5613d1b4abb0 2024-09-16T01:40:12.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.619+0000 7f99d6ff5640 1 -- 172.21.15.139:0/3338729008 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99ec013c60 con 0x5613d1b4abb0 2024-09-16T01:40:12.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.619+0000 7f99d6ff5640 1 -- 172.21.15.139:0/3338729008 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f99ec02e070 con 0x5613d1b4abb0 2024-09-16T01:40:12.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.619+0000 7f99fbaf2740 10 client.15240 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:40:12.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.619+0000 7f99fbaf2740 10 client.15240 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T01:40:12.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.619+0000 7f99fbaf2740 20 client.15240 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:12.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.619+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/3338729008 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5613d1b5b400 0x5613d1b5c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.619+0000 7f99fbaf2740 4 client.15240 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:40:12.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.619+0000 7f99fbaf2740 1 -- 172.21.15.139:0/3338729008 --> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613d1ab3600 con 0x5613d1b5b400 2024-09-16T01:40:12.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.620+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/3338729008 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5613d1b5b400 0x5613d1b5c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.620+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/3338729008 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5613d1b5b400 0x5613d1b5c870 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7351a7501e3a0ba8 server_cookie=95f386f076747860 in_seq=0 out_seq=0 2024-09-16T01:40:12.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.620+0000 7f99d6ff5640 10 client.15240 ms_handle_connect on v2:172.21.15.176:6836/1942543693 2024-09-16T01:40:12.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99d6ff5640 1 -- 172.21.15.139:0/3338729008 <== mds.3 v2:172.21.15.176:6836/1942543693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613d1ab3600 con 0x5613d1b5b400 2024-09-16T01:40:12.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99d6ff5640 10 client.15240 handle_command_reply: tid=0 2024-09-16T01:40:12.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99fbaf2740 1 client.15240 shutdown 2024-09-16T01:40:12.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99fbaf2740 1 -- 172.21.15.139:0/3338729008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 msgr2=0x5613d1b47c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:12.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/3338729008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b47c60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00eb40 tx=0x7f99ec013b70 comp rx=0 tx=0).stop 2024-09-16T01:40:12.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99fbaf2740 1 -- 172.21.15.139:0/3338729008 shutdown_connections 2024-09-16T01:40:12.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/3338729008 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5613d1b5b400 0x5613d1b5c870 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/3338729008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b47c60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99d6ff5640 0 client.15240 ms_handle_reset on v2:172.21.15.176:6836/1942543693 2024-09-16T01:40:12.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/3338729008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b402f0 0x5613d1b47720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/3338729008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b3f9d0 0x5613d1b4a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99fbaf2740 1 -- 172.21.15.139:0/3338729008 >> 172.21.15.139:0/3338729008 conn(0x5613d1b3bf60 msgr2=0x5613d1b41c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:12.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99fbaf2740 1 -- 172.21.15.139:0/3338729008 shutdown_connections 2024-09-16T01:40:12.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99fbaf2740 1 -- 172.21.15.139:0/3338729008 wait complete. 2024-09-16T01:40:12.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.621+0000 7f99fbaf2740 20 client.15240 trim_cache size 0 max 16384 2024-09-16T01:40:12.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.638+0000 7f99fbaf2740 1 Processor -- start 2024-09-16T01:40:12.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.638+0000 7f99fbaf2740 1 -- start start 2024-09-16T01:40:12.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99fbaf2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b3f9d0 0x5613d1b5a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99fbaf2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 0x5613d1b3aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99fbaf2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b4abb0 0x5613d1b44980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99fbaf2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5613d1b5abb0 con 0x5613d1b3f9d0 2024-09-16T01:40:12.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99fbaf2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5613d1b5b280 con 0x5613d1b402f0 2024-09-16T01:40:12.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99fbaf2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5613d1b5b950 con 0x5613d1b4abb0 2024-09-16T01:40:12.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99f2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b3f9d0 0x5613d1b5a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99f2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b3f9d0 0x5613d1b5a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55346/0 (socket says 172.21.15.139:55346) 2024-09-16T01:40:12.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99f2ffd640 1 -- 172.21.15.139:0/1800116930 learned_addr learned my addr 172.21.15.139:0/1800116930 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:12.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99f27fc640 1 --2- 172.21.15.139:0/1800116930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 0x5613d1b3aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99f2ffd640 1 -- 172.21.15.139:0/1800116930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b4abb0 msgr2=0x5613d1b44980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:12.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/1800116930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b4abb0 0x5613d1b44980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99f2ffd640 1 -- 172.21.15.139:0/1800116930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 msgr2=0x5613d1b3aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:12.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/1800116930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 0x5613d1b3aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99f2ffd640 1 -- 172.21.15.139:0/1800116930 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d1b48260 con 0x5613d1b3f9d0 2024-09-16T01:40:12.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99f27fc640 1 --2- 172.21.15.139:0/1800116930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 0x5613d1b3aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:12.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/1800116930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b3f9d0 0x5613d1b5a380 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f99e4097fc0 tx=0x7f99e4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7602f7f3a480c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:12.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99d8ff9640 1 -- 172.21.15.139:0/1800116930 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e4020040 con 0x5613d1b3f9d0 2024-09-16T01:40:12.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99d8ff9640 1 -- 172.21.15.139:0/1800116930 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e40d2e50 con 0x5613d1b3f9d0 2024-09-16T01:40:12.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.639+0000 7f99d8ff9640 1 -- 172.21.15.139:0/1800116930 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e4086310 con 0x5613d1b3f9d0 2024-09-16T01:40:12.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.640+0000 7f99fbaf2740 1 -- 172.21.15.139:0/1800116930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b3f9d0 msgr2=0x5613d1b5a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:12.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.640+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/1800116930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b3f9d0 0x5613d1b5a380 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f99e4097fc0 tx=0x7f99e4000bc0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.640+0000 7f99fbaf2740 1 -- 172.21.15.139:0/1800116930 shutdown_connections 2024-09-16T01:40:12.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.640+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/1800116930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b4abb0 0x5613d1b44980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.640+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/1800116930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 0x5613d1b3aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.640+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/1800116930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b3f9d0 0x5613d1b5a380 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.640+0000 7f99fbaf2740 1 -- 172.21.15.139:0/1800116930 >> 172.21.15.139:0/1800116930 conn(0x5613d1b3bf60 msgr2=0x5613d1b3a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:12.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.640+0000 7f99fbaf2740 1 -- 172.21.15.139:0/1800116930 shutdown_connections 2024-09-16T01:40:12.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.640+0000 7f99fbaf2740 1 -- 172.21.15.139:0/1800116930 wait complete. 2024-09-16T01:40:12.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.640+0000 7f99fbaf2740 1 -- start start 2024-09-16T01:40:12.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99fbaf2740 1 Processor -- start 2024-09-16T01:40:12.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99fbaf2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b3f9d0 0x5613d1b60250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99fbaf2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 0x5613d1b60790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99fbaf2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99fbaf2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5613d1b5c6f0 con 0x5613d1b4abb0 2024-09-16T01:40:12.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99fbaf2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5613d1b5c020 con 0x5613d1b402f0 2024-09-16T01:40:12.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99fbaf2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5613d1b5cdc0 con 0x5613d1b3f9d0 2024-09-16T01:40:12.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99fbaf2740 10 client.0 _ll_register_callbacks cb 0x7f99ec020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:12.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99f37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99f37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b648d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55362/0 (socket says 172.21.15.139:55362) 2024-09-16T01:40:12.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99f37fe640 1 -- 172.21.15.139:0/1281504797 learned_addr learned my addr 172.21.15.139:0/1281504797 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:12.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99f27fc640 1 --2- 172.21.15.139:0/1281504797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 0x5613d1b60790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/1281504797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b3f9d0 0x5613d1b60250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99f37fe640 1 -- 172.21.15.139:0/1281504797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b3f9d0 msgr2=0x5613d1b60250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:12.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99f37fe640 1 --2- 172.21.15.139:0/1281504797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b3f9d0 0x5613d1b60250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99f37fe640 1 -- 172.21.15.139:0/1281504797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 msgr2=0x5613d1b60790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:12.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99f37fe640 1 --2- 172.21.15.139:0/1281504797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 0x5613d1b60790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99f37fe640 1 -- 172.21.15.139:0/1281504797 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d1b64fc0 con 0x5613d1b4abb0 2024-09-16T01:40:12.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/1281504797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b3f9d0 0x5613d1b60250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:12.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99f37fe640 1 --2- 172.21.15.139:0/1281504797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b648d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f99ec013040 tx=0x7f99ec013d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62b78a6e7511d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:12.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.641+0000 7f99d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:12.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.642+0000 7f99d6ff5640 1 -- 172.21.15.139:0/1281504797 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec022040 con 0x5613d1b4abb0 2024-09-16T01:40:12.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.642+0000 7f99d6ff5640 1 -- 172.21.15.139:0/1281504797 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f99ec00e430 con 0x5613d1b4abb0 2024-09-16T01:40:12.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.642+0000 7f99fbaf2740 1 -- 172.21.15.139:0/1281504797 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99e40d2e50 con 0x5613d1b4abb0 2024-09-16T01:40:12.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.642+0000 7f99d6ff5640 1 -- 172.21.15.139:0/1281504797 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f99ec044050 con 0x5613d1b4abb0 2024-09-16T01:40:12.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.642+0000 7f99fbaf2740 10 client.15246 fetch_fsmap learned FSMap version 28 2024-09-16T01:40:12.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.642+0000 7f99fbaf2740 1 -- 172.21.15.139:0/1281504797 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5613d1bc55b0 con 0x5613d1b4abb0 2024-09-16T01:40:12.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.642+0000 7f99d6ff5640 1 -- 172.21.15.139:0/1281504797 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e40d2e50 con 0x5613d1b4abb0 2024-09-16T01:40:12.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.642+0000 7f99d6ff5640 1 -- 172.21.15.139:0/1281504797 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f99ec03d060 con 0x5613d1b4abb0 2024-09-16T01:40:12.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.642+0000 7f99fbaf2740 10 client.15246 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:40:12.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.642+0000 7f99fbaf2740 10 client.15246 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T01:40:12.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.642+0000 7f99fbaf2740 20 client.15246 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:12.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.642+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/1281504797 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5613d1b5b400 0x5613d1bc8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:12.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.642+0000 7f99fbaf2740 4 client.15246 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:40:12.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.642+0000 7f99fbaf2740 1 -- 172.21.15.139:0/1281504797 --> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] -- command(tid 0: {"prefix": "status"}) -- 0x5613d1aadb10 con 0x5613d1b5b400 2024-09-16T01:40:12.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/1281504797 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5613d1b5b400 0x5613d1bc8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:12.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/1281504797 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5613d1b5b400 0x5613d1bc8e80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cba4f28645cc0e1c server_cookie=3c27230c6a40481d in_seq=0 out_seq=0 2024-09-16T01:40:12.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99d6ff5640 10 client.15246 ms_handle_connect on v2:172.21.15.176:6836/1942543693 2024-09-16T01:40:12.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99d6ff5640 1 -- 172.21.15.139:0/1281504797 <== mds.3 v2:172.21.15.176:6836/1942543693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5613d1aadb10 con 0x5613d1b5b400 2024-09-16T01:40:12.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99d6ff5640 10 client.15246 handle_command_reply: tid=0 2024-09-16T01:40:12.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99fbaf2740 1 client.15246 shutdown 2024-09-16T01:40:12.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99fbaf2740 1 -- 172.21.15.139:0/1281504797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 msgr2=0x5613d1b648d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:12.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/1281504797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b648d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f99ec013040 tx=0x7f99ec013d80 comp rx=0 tx=0).stop 2024-09-16T01:40:12.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99fbaf2740 1 -- 172.21.15.139:0/1281504797 shutdown_connections 2024-09-16T01:40:12.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/1281504797 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x5613d1b5b400 0x5613d1bc8e80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/1281504797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5613d1b4abb0 0x5613d1b648d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/1281504797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5613d1b402f0 0x5613d1b60790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99d6ff5640 0 client.15246 ms_handle_reset on v2:172.21.15.176:6836/1942543693 2024-09-16T01:40:12.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99fbaf2740 1 --2- 172.21.15.139:0/1281504797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613d1b3f9d0 0x5613d1b60250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99fbaf2740 1 -- 172.21.15.139:0/1281504797 >> 172.21.15.139:0/1281504797 conn(0x5613d1b3bf60 msgr2=0x5613d1b49570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:12.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99fbaf2740 1 -- 172.21.15.139:0/1281504797 shutdown_connections 2024-09-16T01:40:12.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.643+0000 7f99fbaf2740 1 -- 172.21.15.139:0/1281504797 wait complete. 2024-09-16T01:40:12.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.644+0000 7f99fbaf2740 20 client.15246 trim_cache size 0 max 16384 2024-09-16T01:40:12.651 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:40:12.651 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:40:12.651 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 3, 2024-09-16T01:40:12.651 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24442, 2024-09-16T01:40:12.651 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:40:12.651 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:40:12.651 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:40:12.651 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 94.106362462999996, 2024-09-16T01:40:12.651 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 28, 2024-09-16T01:40:12.651 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 85, 2024-09-16T01:40:12.651 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 83, 2024-09-16T01:40:12.651 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 156.88091570399999 2024-09-16T01:40:12.651 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:40:12.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.644+0000 7f99f9719640 1 -- 172.21.15.139:0/2916868863 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f99dc0a3240 msgr2=0x7f99dc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:12.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.644+0000 7f99f9719640 1 --2- 172.21.15.139:0/2916868863 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f99dc0a3240 0x7f99dc0a5700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f99ec01f000 tx=0x7f99ec01c000 comp rx=0 tx=0).stop 2024-09-16T01:40:12.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.644+0000 7f99f9719640 1 -- 172.21.15.139:0/2916868863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f4101de0 msgr2=0x7f99f40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:12.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.644+0000 7f99f9719640 1 --2- 172.21.15.139:0/2916868863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f4101de0 0x7f99f40744f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f99e400d950 tx=0x7f99e400de10 comp rx=0 tx=0).stop 2024-09-16T01:40:12.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.644+0000 7f99f9719640 1 -- 172.21.15.139:0/2916868863 shutdown_connections 2024-09-16T01:40:12.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.644+0000 7f99f9719640 1 --2- 172.21.15.139:0/2916868863 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f99dc0a3240 0x7f99dc0a5700 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.644+0000 7f99f9719640 1 --2- 172.21.15.139:0/2916868863 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99f4106f10 0x7f99f4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.644+0000 7f99f9719640 1 --2- 172.21.15.139:0/2916868863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f4102790 0x7f99f4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.644+0000 7f99f9719640 1 --2- 172.21.15.139:0/2916868863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f4101de0 0x7f99f40744f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:12.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.644+0000 7f99f9719640 1 -- 172.21.15.139:0/2916868863 >> 172.21.15.139:0/2916868863 conn(0x7f99f40fd8c0 msgr2=0x7f99f40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:12.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.644+0000 7f99f9719640 1 -- 172.21.15.139:0/2916868863 shutdown_connections 2024-09-16T01:40:12.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:12.644+0000 7f99f9719640 1 -- 172.21.15.139:0/2916868863 wait complete. 2024-09-16T01:40:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:12 smithi139 ceph-mon[31175]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-16T01:40:12.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:12 smithi176 ceph-mon[35324]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-16T01:40:13.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:12 smithi164 ceph-mon[35121]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-16T01:40:13.337 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T01:40:13.583 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.066+0000 7fddf053f640 1 -- 172.21.15.139:0/2520146492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde8101de0 msgr2=0x7fdde81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.066+0000 7fddf053f640 1 --2- 172.21.15.139:0/2520146492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde8101de0 0x7fdde81021c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009900 tx=0x7fdde402a410 comp rx=0 tx=0).stop 2024-09-16T01:40:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.066+0000 7fddf053f640 1 -- 172.21.15.139:0/2520146492 shutdown_connections 2024-09-16T01:40:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.066+0000 7fddf053f640 1 --2- 172.21.15.139:0/2520146492 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdde8106f10 0x7fdde810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.066+0000 7fddf053f640 1 --2- 172.21.15.139:0/2520146492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdde8102790 0x7fdde81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.066+0000 7fddf053f640 1 --2- 172.21.15.139:0/2520146492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde8101de0 0x7fdde81021c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.066+0000 7fddf053f640 1 -- 172.21.15.139:0/2520146492 >> 172.21.15.139:0/2520146492 conn(0x7fdde80fd8c0 msgr2=0x7fdde80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.066+0000 7fddf053f640 1 -- 172.21.15.139:0/2520146492 shutdown_connections 2024-09-16T01:40:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.066+0000 7fddf053f640 1 -- 172.21.15.139:0/2520146492 wait complete. 2024-09-16T01:40:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddf053f640 1 Processor -- start 2024-09-16T01:40:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddf053f640 1 -- start start 2024-09-16T01:40:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddf053f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdde8101de0 0x7fdde81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddf053f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde8102790 0x7fdde8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddf053f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdde8106f10 0x7fdde819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddf053f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8111f30 con 0x7fdde8102790 2024-09-16T01:40:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddf053f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8111db0 con 0x7fdde8106f10 2024-09-16T01:40:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddf053f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdde81120b0 con 0x7fdde8101de0 2024-09-16T01:40:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddedada640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde8102790 0x7fdde8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddedada640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde8102790 0x7fdde8198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55392/0 (socket says 172.21.15.139:55392) 2024-09-16T01:40:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddedada640 1 -- 172.21.15.139:0/3768155596 learned_addr learned my addr 172.21.15.139:0/3768155596 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddeeadc640 1 --2- 172.21.15.139:0/3768155596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdde8106f10 0x7fdde819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddee2db640 1 --2- 172.21.15.139:0/3768155596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdde8101de0 0x7fdde81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddedada640 1 -- 172.21.15.139:0/3768155596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdde8101de0 msgr2=0x7fdde81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddedada640 1 --2- 172.21.15.139:0/3768155596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdde8101de0 0x7fdde81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddedada640 1 -- 172.21.15.139:0/3768155596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdde8106f10 msgr2=0x7fdde819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.067+0000 7fddedada640 1 --2- 172.21.15.139:0/3768155596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdde8106f10 0x7fdde819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.068+0000 7fddedada640 1 -- 172.21.15.139:0/3768155596 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde819d380 con 0x7fdde8102790 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.068+0000 7fddedada640 1 --2- 172.21.15.139:0/3768155596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde8102790 0x7fdde8198910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fddd800c920 tx=0x7fddd800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.068+0000 7fdddf7fe640 1 -- 172.21.15.139:0/3768155596 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8007c70 con 0x7fdde8102790 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.068+0000 7fddf053f640 1 -- 172.21.15.139:0/3768155596 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde819d670 con 0x7fdde8102790 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.068+0000 7fdddf7fe640 1 -- 172.21.15.139:0/3768155596 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddd8007e10 con 0x7fdde8102790 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.068+0000 7fdddf7fe640 1 -- 172.21.15.139:0/3768155596 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8018890 con 0x7fdde8102790 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.068+0000 7fddf053f640 1 -- 172.21.15.139:0/3768155596 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde81a4f50 con 0x7fdde8102790 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.070+0000 7fdddf7fe640 1 -- 172.21.15.139:0/3768155596 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fddd8018a70 con 0x7fdde8102790 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.070+0000 7fddf2918740 1 Processor -- start 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddf2918740 1 -- start start 2024-09-16T01:40:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fdddf7fe640 1 --2- 172.21.15.139:0/3768155596 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fddd40a3240 0x7fddd40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddf2918740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cecda9d0 0x5611cecdadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddf2918740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cece5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddf2918740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cece5bb0 0x5611cece7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddf2918740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5611cec41090 con 0x5611cecdb2f0 2024-09-16T01:40:14.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddf2918740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5611cebe8a70 con 0x5611cece5bb0 2024-09-16T01:40:14.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddf2918740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5611cec3d050 con 0x5611cecda9d0 2024-09-16T01:40:14.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddeeadc640 1 --2- 172.21.15.139:0/3768155596 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fddd40a3240 0x7fddd40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fdddf7fe640 1 -- 172.21.15.139:0/3768155596 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fddd80cfd80 con 0x7fdde8102790 2024-09-16T01:40:14.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddedada640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cece5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddedada640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cece5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55404/0 (socket says 172.21.15.139:55404) 2024-09-16T01:40:14.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddedada640 1 -- 172.21.15.139:0/736976396 learned_addr learned my addr 172.21.15.139:0/736976396 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:14.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddeeadc640 1 --2- 172.21.15.139:0/736976396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cece5bb0 0x5611cece7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddee2db640 1 --2- 172.21.15.139:0/736976396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cecda9d0 0x5611cecdadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddedada640 1 -- 172.21.15.139:0/736976396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cecda9d0 msgr2=0x5611cecdadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:14.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddedada640 1 --2- 172.21.15.139:0/736976396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cecda9d0 0x5611cecdadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddedada640 1 -- 172.21.15.139:0/736976396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cece5bb0 msgr2=0x5611cece7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:14.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddedada640 1 --2- 172.21.15.139:0/736976396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cece5bb0 0x5611cece7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.071+0000 7fddedada640 1 -- 172.21.15.139:0/736976396 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611cebf5130 con 0x5611cecdb2f0 2024-09-16T01:40:14.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.072+0000 7fddee2db640 1 --2- 172.21.15.139:0/736976396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cecda9d0 0x5611cecdadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:14.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.072+0000 7fddeeadc640 1 --2- 172.21.15.139:0/736976396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cece5bb0 0x5611cece7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:14.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.072+0000 7fddeeadc640 1 --2- 172.21.15.139:0/3768155596 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fddd40a3240 0x7fddd40a5700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdde0005fd0 tx=0x7fdde0014b70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:14.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.072+0000 7fddedada640 1 --2- 172.21.15.139:0/736976396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cece5670 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fddd80989a0 tx=0x7fddd809ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f052390c8af9abc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:14.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.072+0000 7fdddcff9640 1 -- 172.21.15.139:0/736976396 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd809c6d0 con 0x5611cecdb2f0 2024-09-16T01:40:14.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.072+0000 7fdddcff9640 1 -- 172.21.15.139:0/736976396 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddd809bfd0 con 0x5611cecdb2f0 2024-09-16T01:40:14.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.072+0000 7fdddcff9640 1 -- 172.21.15.139:0/736976396 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd800fb30 con 0x5611cecdb2f0 2024-09-16T01:40:14.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.072+0000 7fddf2918740 1 -- 172.21.15.139:0/736976396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 msgr2=0x5611cece5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:14.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 --2- 172.21.15.139:0/736976396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cece5670 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fddd80989a0 tx=0x7fddd809ba60 comp rx=0 tx=0).stop 2024-09-16T01:40:14.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 -- 172.21.15.139:0/736976396 shutdown_connections 2024-09-16T01:40:14.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 --2- 172.21.15.139:0/736976396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cece5bb0 0x5611cece7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 --2- 172.21.15.139:0/736976396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cece5670 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 --2- 172.21.15.139:0/736976396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cecda9d0 0x5611cecdadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 -- 172.21.15.139:0/736976396 >> 172.21.15.139:0/736976396 conn(0x5611cecd6f60 msgr2=0x5611cecd7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:14.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 -- 172.21.15.139:0/736976396 shutdown_connections 2024-09-16T01:40:14.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 -- 172.21.15.139:0/736976396 wait complete. 2024-09-16T01:40:14.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 -- start start 2024-09-16T01:40:14.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 Processor -- start 2024-09-16T01:40:14.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cecda9d0 0x5611cece5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cece27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cece5bb0 0x5611cece2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5611cece8780 con 0x5611cecdb2f0 2024-09-16T01:40:14.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5611cebfbf20 con 0x5611cecda9d0 2024-09-16T01:40:14.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5611cece8900 con 0x5611cece5bb0 2024-09-16T01:40:14.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddf2918740 10 client.0 _ll_register_callbacks cb 0x5611cebc5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:14.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddedada640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cece27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddedada640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cece27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55416/0 (socket says 172.21.15.139:55416) 2024-09-16T01:40:14.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.073+0000 7fddedada640 1 -- 172.21.15.139:0/3736603736 learned_addr learned my addr 172.21.15.139:0/3736603736 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:14.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddeeadc640 1 --2- 172.21.15.139:0/3736603736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cece5bb0 0x5611cece2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddee2db640 1 --2- 172.21.15.139:0/3736603736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cecda9d0 0x5611cece5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddedada640 1 -- 172.21.15.139:0/3736603736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cece5bb0 msgr2=0x5611cece2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:14.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddedada640 1 --2- 172.21.15.139:0/3736603736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cece5bb0 0x5611cece2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddedada640 1 -- 172.21.15.139:0/3736603736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cecda9d0 msgr2=0x5611cece5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:14.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddedada640 1 --2- 172.21.15.139:0/3736603736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cecda9d0 0x5611cece5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddedada640 1 -- 172.21.15.139:0/3736603736 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611ce9b22b0 con 0x5611cecdb2f0 2024-09-16T01:40:14.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddeeadc640 1 --2- 172.21.15.139:0/3736603736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cece5bb0 0x5611cece2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:14.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddedada640 1 --2- 172.21.15.139:0/3736603736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cece27f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fddd8098430 tx=0x7fddd8018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0c52a4bddc5df server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:14.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:14.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddd1ffb640 1 -- 172.21.15.139:0/3736603736 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd80a4c40 con 0x5611cecdb2f0 2024-09-16T01:40:14.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddd1ffb640 1 -- 172.21.15.139:0/3736603736 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fddd809a070 con 0x5611cecdb2f0 2024-09-16T01:40:14.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddf2918740 1 -- 172.21.15.139:0/3736603736 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddd809bfd0 con 0x5611cecdb2f0 2024-09-16T01:40:14.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddd1ffb640 1 -- 172.21.15.139:0/3736603736 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fddd809c6d0 con 0x5611cecdb2f0 2024-09-16T01:40:14.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddf2918740 10 client.15258 fetch_fsmap learned FSMap version 28 2024-09-16T01:40:14.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddf2918740 1 -- 172.21.15.139:0/3736603736 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5611cecf40f0 con 0x5611cecdb2f0 2024-09-16T01:40:14.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddd1ffb640 1 -- 172.21.15.139:0/3736603736 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddd809bfd0 con 0x5611cecdb2f0 2024-09-16T01:40:14.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddd1ffb640 1 -- 172.21.15.139:0/3736603736 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fddd8096c60 con 0x5611cecdb2f0 2024-09-16T01:40:14.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddf2918740 10 client.15258 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:40:14.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.074+0000 7fddf2918740 10 client.15258 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T01:40:14.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.075+0000 7fddf2918740 20 client.15258 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:14.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.075+0000 7fddf2918740 1 --2- 172.21.15.139:0/3736603736 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5611cecf5440 0x5611cecf7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.075+0000 7fddf2918740 4 client.15258 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:40:14.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.075+0000 7fddf2918740 1 -- 172.21.15.139:0/3736603736 --> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611cec4e600 con 0x5611cecf5440 2024-09-16T01:40:14.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.075+0000 7fddee2db640 1 --2- 172.21.15.139:0/3736603736 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5611cecf5440 0x5611cecf7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.075+0000 7fddee2db640 1 --2- 172.21.15.139:0/3736603736 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5611cecf5440 0x5611cecf7540 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1822edb059ba60ee server_cookie=e495769a0ee8d31f in_seq=0 out_seq=0 2024-09-16T01:40:14.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.075+0000 7fddd1ffb640 10 client.15258 ms_handle_connect on v2:172.21.15.164:6836/1270226509 2024-09-16T01:40:14.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddd1ffb640 1 -- 172.21.15.139:0/3736603736 <== mds.2 v2:172.21.15.164:6836/1270226509 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611cec4e600 con 0x5611cecf5440 2024-09-16T01:40:14.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddd1ffb640 10 client.15258 handle_command_reply: tid=0 2024-09-16T01:40:14.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddf2918740 1 client.15258 shutdown 2024-09-16T01:40:14.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddf2918740 1 -- 172.21.15.139:0/3736603736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 msgr2=0x5611cece27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:14.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddf2918740 1 --2- 172.21.15.139:0/3736603736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cece27f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fddd8098430 tx=0x7fddd8018f90 comp rx=0 tx=0).stop 2024-09-16T01:40:14.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddf2918740 1 -- 172.21.15.139:0/3736603736 shutdown_connections 2024-09-16T01:40:14.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddf2918740 1 --2- 172.21.15.139:0/3736603736 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5611cecf5440 0x5611cecf7540 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddf2918740 1 --2- 172.21.15.139:0/3736603736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cece5bb0 0x5611cece2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddf2918740 1 --2- 172.21.15.139:0/3736603736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cece27f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddd1ffb640 0 client.15258 ms_handle_reset on v2:172.21.15.164:6836/1270226509 2024-09-16T01:40:14.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddf2918740 1 --2- 172.21.15.139:0/3736603736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cecda9d0 0x5611cece5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddf2918740 1 -- 172.21.15.139:0/3736603736 >> 172.21.15.139:0/3736603736 conn(0x5611cecd6f60 msgr2=0x5611cecdcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:14.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddf2918740 1 -- 172.21.15.139:0/3736603736 shutdown_connections 2024-09-16T01:40:14.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddf2918740 1 -- 172.21.15.139:0/3736603736 wait complete. 2024-09-16T01:40:14.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.076+0000 7fddf2918740 20 client.15258 trim_cache size 0 max 16384 2024-09-16T01:40:14.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddf2918740 1 Processor -- start 2024-09-16T01:40:14.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddf2918740 1 -- start start 2024-09-16T01:40:14.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddf2918740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cecda9d0 0x5611cecf6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddf2918740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cecf60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddf2918740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cece5bb0 0x5611cecf5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddf2918740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5611cecf5880 con 0x5611cecdb2f0 2024-09-16T01:40:14.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddf2918740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5611cecf5f50 con 0x5611cecda9d0 2024-09-16T01:40:14.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddf2918740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5611cecf6620 con 0x5611cece5bb0 2024-09-16T01:40:14.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddedada640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cecf60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddedada640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cecf60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55426/0 (socket says 172.21.15.139:55426) 2024-09-16T01:40:14.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddedada640 1 -- 172.21.15.139:0/869812725 learned_addr learned my addr 172.21.15.139:0/869812725 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:14.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddeeadc640 1 --2- 172.21.15.139:0/869812725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cece5bb0 0x5611cecf5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddedada640 1 -- 172.21.15.139:0/869812725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cece5bb0 msgr2=0x5611cecf5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:14.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddedada640 1 --2- 172.21.15.139:0/869812725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cece5bb0 0x5611cecf5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddedada640 1 -- 172.21.15.139:0/869812725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cecda9d0 msgr2=0x5611cecf6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:14.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddee2db640 1 --2- 172.21.15.139:0/869812725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cecda9d0 0x5611cecf6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddedada640 1 --2- 172.21.15.139:0/869812725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cecda9d0 0x5611cecf6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddedada640 1 -- 172.21.15.139:0/869812725 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611cece57c0 con 0x5611cecdb2f0 2024-09-16T01:40:14.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.095+0000 7fddedada640 1 --2- 172.21.15.139:0/869812725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cecf60d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fddd80cd820 tx=0x7fddd809a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34fbbda8230d7f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:14.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.096+0000 7fddd3fff640 1 -- 172.21.15.139:0/869812725 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd80a4c40 con 0x5611cecdb2f0 2024-09-16T01:40:14.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.096+0000 7fddd3fff640 1 -- 172.21.15.139:0/869812725 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddd809bfd0 con 0x5611cecdb2f0 2024-09-16T01:40:14.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.096+0000 7fddd3fff640 1 -- 172.21.15.139:0/869812725 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd80a47b0 con 0x5611cecdb2f0 2024-09-16T01:40:14.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.096+0000 7fddf2918740 1 -- 172.21.15.139:0/869812725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 msgr2=0x5611cecf60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:14.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.096+0000 7fddf2918740 1 --2- 172.21.15.139:0/869812725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cecf60d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fddd80cd820 tx=0x7fddd809a420 comp rx=0 tx=0).stop 2024-09-16T01:40:14.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.096+0000 7fddf2918740 1 -- 172.21.15.139:0/869812725 shutdown_connections 2024-09-16T01:40:14.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.096+0000 7fddf2918740 1 --2- 172.21.15.139:0/869812725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cece5bb0 0x5611cecf5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.096+0000 7fddf2918740 1 --2- 172.21.15.139:0/869812725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecdb2f0 0x5611cecf60d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.096+0000 7fddf2918740 1 --2- 172.21.15.139:0/869812725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cecda9d0 0x5611cecf6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.096+0000 7fddf2918740 1 -- 172.21.15.139:0/869812725 >> 172.21.15.139:0/869812725 conn(0x5611cecd6f60 msgr2=0x5611cece3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:14.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.096+0000 7fddf2918740 1 -- 172.21.15.139:0/869812725 shutdown_connections 2024-09-16T01:40:14.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.096+0000 7fddf2918740 1 -- 172.21.15.139:0/869812725 wait complete. 2024-09-16T01:40:14.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddf2918740 1 -- start start 2024-09-16T01:40:14.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddf2918740 1 Processor -- start 2024-09-16T01:40:14.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddf2918740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecda9d0 0x5611cecfae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddf2918740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cecdb2f0 0x5611cecfb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddf2918740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cece5bb0 0x5611ced5e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddf2918740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5611cecf73c0 con 0x5611cecda9d0 2024-09-16T01:40:14.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddf2918740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5611cecf6cf0 con 0x5611cece5bb0 2024-09-16T01:40:14.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddf2918740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5611cecf7a90 con 0x5611cecdb2f0 2024-09-16T01:40:14.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddf2918740 10 client.0 _ll_register_callbacks cb 0x7fddd809b930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:14.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddee2db640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecda9d0 0x5611cecfae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddee2db640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecda9d0 0x5611cecfae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55432/0 (socket says 172.21.15.139:55432) 2024-09-16T01:40:14.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddee2db640 1 -- 172.21.15.139:0/1370581562 learned_addr learned my addr 172.21.15.139:0/1370581562 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:14.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddedada640 1 --2- 172.21.15.139:0/1370581562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cecdb2f0 0x5611cecfb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddeeadc640 1 --2- 172.21.15.139:0/1370581562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cece5bb0 0x5611ced5e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddee2db640 1 -- 172.21.15.139:0/1370581562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cecdb2f0 msgr2=0x5611cecfb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:14.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddee2db640 1 --2- 172.21.15.139:0/1370581562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cecdb2f0 0x5611cecfb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddee2db640 1 -- 172.21.15.139:0/1370581562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cece5bb0 msgr2=0x5611ced5e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:14.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddee2db640 1 --2- 172.21.15.139:0/1370581562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cece5bb0 0x5611ced5e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddee2db640 1 -- 172.21.15.139:0/1370581562 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611ced5ee10 con 0x5611cecda9d0 2024-09-16T01:40:14.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.097+0000 7fddeeadc640 1 --2- 172.21.15.139:0/1370581562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cece5bb0 0x5611ced5e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:14.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddee2db640 1 --2- 172.21.15.139:0/1370581562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecda9d0 0x5611cecfae60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009900 tx=0x7fdde4031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d215b2a5d4b68e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:14.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:14.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddd1ffb640 1 -- 172.21.15.139:0/1370581562 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4049030 con 0x5611cecda9d0 2024-09-16T01:40:14.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddd1ffb640 1 -- 172.21.15.139:0/1370581562 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdde4004250 con 0x5611cecda9d0 2024-09-16T01:40:14.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddf2918740 1 -- 172.21.15.139:0/1370581562 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddd809bfd0 con 0x5611cecda9d0 2024-09-16T01:40:14.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddd1ffb640 1 -- 172.21.15.139:0/1370581562 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdde4055050 con 0x5611cecda9d0 2024-09-16T01:40:14.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddf2918740 10 client.15264 fetch_fsmap learned FSMap version 28 2024-09-16T01:40:14.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddf2918740 1 -- 172.21.15.139:0/1370581562 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5611ced600d0 con 0x5611cecda9d0 2024-09-16T01:40:14.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddd1ffb640 1 -- 172.21.15.139:0/1370581562 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddd809bfd0 con 0x5611cecda9d0 2024-09-16T01:40:14.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddd1ffb640 1 -- 172.21.15.139:0/1370581562 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fdde40452c0 con 0x5611cecda9d0 2024-09-16T01:40:14.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddf2918740 10 client.15264 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:40:14.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddf2918740 10 client.15264 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T01:40:14.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddf2918740 20 client.15264 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:14.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddf2918740 1 --2- 172.21.15.139:0/1370581562 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5611ced64c70 0x5611ced63980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:14.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddf2918740 4 client.15264 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:40:14.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddf2918740 1 -- 172.21.15.139:0/1370581562 --> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] -- command(tid 0: {"prefix": "status"}) -- 0x5611cec48b10 con 0x5611ced64c70 2024-09-16T01:40:14.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.098+0000 7fddedada640 1 --2- 172.21.15.139:0/1370581562 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5611ced64c70 0x5611ced63980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:14.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddedada640 1 --2- 172.21.15.139:0/1370581562 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5611ced64c70 0x5611ced63980 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6de268d49f18c3e2 server_cookie=77215284fcddabe3 in_seq=0 out_seq=0 2024-09-16T01:40:14.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddd1ffb640 10 client.15264 ms_handle_connect on v2:172.21.15.164:6836/1270226509 2024-09-16T01:40:14.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddd1ffb640 1 -- 172.21.15.139:0/1370581562 <== mds.2 v2:172.21.15.164:6836/1270226509 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5611cec48b10 con 0x5611ced64c70 2024-09-16T01:40:14.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddd1ffb640 10 client.15264 handle_command_reply: tid=0 2024-09-16T01:40:14.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddf2918740 1 client.15264 shutdown 2024-09-16T01:40:14.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddf2918740 1 -- 172.21.15.139:0/1370581562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecda9d0 msgr2=0x5611cecfae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:14.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddf2918740 1 --2- 172.21.15.139:0/1370581562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecda9d0 0x5611cecfae60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009900 tx=0x7fdde4031950 comp rx=0 tx=0).stop 2024-09-16T01:40:14.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddf2918740 1 -- 172.21.15.139:0/1370581562 shutdown_connections 2024-09-16T01:40:14.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddf2918740 1 --2- 172.21.15.139:0/1370581562 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x5611ced64c70 0x5611ced63980 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddf2918740 1 --2- 172.21.15.139:0/1370581562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611cece5bb0 0x5611ced5e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddf2918740 1 --2- 172.21.15.139:0/1370581562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611cecdb2f0 0x5611cecfb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddf2918740 1 --2- 172.21.15.139:0/1370581562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5611cecda9d0 0x5611cecfae60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddf2918740 1 -- 172.21.15.139:0/1370581562 >> 172.21.15.139:0/1370581562 conn(0x5611cecd6f60 msgr2=0x5611cecd6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:14.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddd1ffb640 0 client.15264 ms_handle_reset on v2:172.21.15.164:6836/1270226509 2024-09-16T01:40:14.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddf2918740 1 -- 172.21.15.139:0/1370581562 shutdown_connections 2024-09-16T01:40:14.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddf2918740 1 -- 172.21.15.139:0/1370581562 wait complete. 2024-09-16T01:40:14.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.099+0000 7fddf2918740 20 client.15264 trim_cache size 0 max 16384 2024-09-16T01:40:14.107 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:40:14.107 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:40:14.107 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 2, 2024-09-16T01:40:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24443, 2024-09-16T01:40:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:40:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:40:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:40:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 96.570927897999994, 2024-09-16T01:40:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 28, 2024-09-16T01:40:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 85, 2024-09-16T01:40:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 83, 2024-09-16T01:40:14.108 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 155.810007853 2024-09-16T01:40:14.108 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:40:14.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.100+0000 7fddf053f640 1 -- 172.21.15.139:0/3768155596 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fddd40a3240 msgr2=0x7fddd40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:14.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.100+0000 7fddf053f640 1 --2- 172.21.15.139:0/3768155596 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fddd40a3240 0x7fddd40a5700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdde0005fd0 tx=0x7fdde0014b70 comp rx=0 tx=0).stop 2024-09-16T01:40:14.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.100+0000 7fddf053f640 1 -- 172.21.15.139:0/3768155596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde8102790 msgr2=0x7fdde8198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:14.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.100+0000 7fddf053f640 1 --2- 172.21.15.139:0/3768155596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde8102790 0x7fdde8198910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fddd800c920 tx=0x7fddd800cde0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.100+0000 7fddf053f640 1 -- 172.21.15.139:0/3768155596 shutdown_connections 2024-09-16T01:40:14.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.100+0000 7fddf053f640 1 --2- 172.21.15.139:0/3768155596 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fddd40a3240 0x7fddd40a5700 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.100+0000 7fddf053f640 1 --2- 172.21.15.139:0/3768155596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdde8106f10 0x7fdde819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.100+0000 7fddf053f640 1 --2- 172.21.15.139:0/3768155596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde8102790 0x7fdde8198910 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.100+0000 7fddf053f640 1 --2- 172.21.15.139:0/3768155596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdde8101de0 0x7fdde81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:14.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.100+0000 7fddf053f640 1 -- 172.21.15.139:0/3768155596 >> 172.21.15.139:0/3768155596 conn(0x7fdde80fd8c0 msgr2=0x7fdde80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:14.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.100+0000 7fddf053f640 1 -- 172.21.15.139:0/3768155596 shutdown_connections 2024-09-16T01:40:14.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:14.100+0000 7fddf053f640 1 -- 172.21.15.139:0/3768155596 wait complete. 2024-09-16T01:40:14.835 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:14 smithi139 ceph-mon[31175]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-16T01:40:14.865 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-16T01:40:14.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:14 smithi176 ceph-mon[35324]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-16T01:40:15.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:14 smithi164 ceph-mon[35121]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-16T01:40:15.112 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:15.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.592+0000 7f15690b0640 1 -- 172.21.15.139:0/60755015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564101de0 msgr2=0x7f15641021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:15.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.592+0000 7f15690b0640 1 --2- 172.21.15.139:0/60755015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564101de0 0x7f15641021c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1554009900 tx=0x7f155402a410 comp rx=0 tx=0).stop 2024-09-16T01:40:15.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.592+0000 7f15690b0640 1 -- 172.21.15.139:0/60755015 shutdown_connections 2024-09-16T01:40:15.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.592+0000 7f15690b0640 1 --2- 172.21.15.139:0/60755015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1564106f10 0x7f156410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.592+0000 7f15690b0640 1 --2- 172.21.15.139:0/60755015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1564102790 0x7f15641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.592+0000 7f15690b0640 1 --2- 172.21.15.139:0/60755015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564101de0 0x7f15641021c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.592+0000 7f15690b0640 1 -- 172.21.15.139:0/60755015 >> 172.21.15.139:0/60755015 conn(0x7f15640fd8c0 msgr2=0x7f15640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:15.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.592+0000 7f15690b0640 1 -- 172.21.15.139:0/60755015 shutdown_connections 2024-09-16T01:40:15.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.592+0000 7f15690b0640 1 -- 172.21.15.139:0/60755015 wait complete. 2024-09-16T01:40:15.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f15690b0640 1 Processor -- start 2024-09-16T01:40:15.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f15690b0640 1 -- start start 2024-09-16T01:40:15.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f15690b0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1564101de0 0x7f15641984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f15690b0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1564102790 0x7f15641989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f15690b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564106f10 0x7f156419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f15690b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1564111f90 con 0x7f1564106f10 2024-09-16T01:40:15.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f15690b0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1564111e10 con 0x7f1564101de0 2024-09-16T01:40:15.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f15690b0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1564112110 con 0x7f1564102790 2024-09-16T01:40:15.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f156359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564106f10 0x7f156419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f156359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564106f10 0x7f156419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55446/0 (socket says 172.21.15.139:55446) 2024-09-16T01:40:15.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f156359e640 1 -- 172.21.15.139:0/4044981253 learned_addr learned my addr 172.21.15.139:0/4044981253 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:15.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f1562d9d640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1564101de0 0x7f15641984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f155bfff640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1564102790 0x7f15641989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f156359e640 1 -- 172.21.15.139:0/4044981253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1564102790 msgr2=0x7f15641989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:15.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.593+0000 7f156359e640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1564102790 0x7f15641989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.594+0000 7f156359e640 1 -- 172.21.15.139:0/4044981253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1564101de0 msgr2=0x7f15641984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:15.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.594+0000 7f156359e640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1564101de0 0x7f15641984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.594+0000 7f156359e640 1 -- 172.21.15.139:0/4044981253 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156419d460 con 0x7f1564106f10 2024-09-16T01:40:15.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.594+0000 7f155bfff640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1564102790 0x7f15641989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:15.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.594+0000 7f156359e640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564106f10 0x7f156419cd80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f155c00b970 tx=0x7f155c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:15.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.594+0000 7f1560d99640 1 -- 172.21.15.139:0/4044981253 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c019070 con 0x7f1564106f10 2024-09-16T01:40:15.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.594+0000 7f1560d99640 1 -- 172.21.15.139:0/4044981253 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f155c00cb40 con 0x7f1564106f10 2024-09-16T01:40:15.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.594+0000 7f15690b0640 1 -- 172.21.15.139:0/4044981253 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156419d750 con 0x7f1564106f10 2024-09-16T01:40:15.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.594+0000 7f1560d99640 1 -- 172.21.15.139:0/4044981253 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c007650 con 0x7f1564106f10 2024-09-16T01:40:15.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.594+0000 7f15690b0640 1 -- 172.21.15.139:0/4044981253 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15641a5030 con 0x7f1564106f10 2024-09-16T01:40:15.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.596+0000 7f1560d99640 1 -- 172.21.15.139:0/4044981253 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f155c01c070 con 0x7f1564106f10 2024-09-16T01:40:15.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.597+0000 7f156b489740 1 Processor -- start 2024-09-16T01:40:15.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.597+0000 7f156b489740 1 -- start start 2024-09-16T01:40:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.597+0000 7f156b489740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9c9d0 0x557c80b9cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.597+0000 7f156b489740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9d2f0 0x557c80ba7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.597+0000 7f156b489740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80ba7bb0 0x557c80ba9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.597+0000 7f156b489740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557c80b03090 con 0x557c80b9d2f0 2024-09-16T01:40:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.597+0000 7f156b489740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557c80aaaa70 con 0x557c80ba7bb0 2024-09-16T01:40:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.597+0000 7f156b489740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557c80aff050 con 0x557c80b9c9d0 2024-09-16T01:40:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.597+0000 7f1562d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9d2f0 0x557c80ba7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.597+0000 7f1562d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9d2f0 0x557c80ba7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55458/0 (socket says 172.21.15.139:55458) 2024-09-16T01:40:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.597+0000 7f1562d9d640 1 -- 172.21.15.139:0/3210231592 learned_addr learned my addr 172.21.15.139:0/3210231592 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.597+0000 7f1560d99640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f15500a3240 0x7f15500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.597+0000 7f155bfff640 1 --2- 172.21.15.139:0/3210231592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9c9d0 0x557c80b9cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.597+0000 7f155bfff640 1 --2- 172.21.15.139:0/3210231592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80ba7bb0 0x557c80ba9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.598+0000 7f156359e640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f15500a3240 0x7f15500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.598+0000 7f1562d9d640 1 -- 172.21.15.139:0/3210231592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9c9d0 msgr2=0x557c80b9cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.598+0000 7f1562d9d640 1 --2- 172.21.15.139:0/3210231592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9c9d0 0x557c80b9cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.598+0000 7f1562d9d640 1 -- 172.21.15.139:0/3210231592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80ba7bb0 msgr2=0x557c80ba9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.598+0000 7f1562d9d640 1 --2- 172.21.15.139:0/3210231592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80ba7bb0 0x557c80ba9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.598+0000 7f1562d9d640 1 -- 172.21.15.139:0/3210231592 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c80ab7130 con 0x557c80b9d2f0 2024-09-16T01:40:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.598+0000 7f1560d99640 1 -- 172.21.15.139:0/4044981253 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f155c0cf350 con 0x7f1564106f10 2024-09-16T01:40:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.598+0000 7f1562d9d640 1 --2- 172.21.15.139:0/3210231592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9d2f0 0x557c80ba7670 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1554033000 tx=0x7f1554032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8218525a0f0855 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.598+0000 7f156359e640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f15500a3240 0x7f15500a5700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f155c0cf640 tx=0x7f155c065f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.598+0000 7f15597fa640 1 -- 172.21.15.139:0/3210231592 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155403f070 con 0x557c80b9d2f0 2024-09-16T01:40:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.598+0000 7f15597fa640 1 -- 172.21.15.139:0/3210231592 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15540317b0 con 0x557c80b9d2f0 2024-09-16T01:40:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.598+0000 7f15597fa640 1 -- 172.21.15.139:0/3210231592 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554035070 con 0x557c80b9d2f0 2024-09-16T01:40:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.599+0000 7f156b489740 1 -- 172.21.15.139:0/3210231592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9d2f0 msgr2=0x557c80ba7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.599+0000 7f156b489740 1 --2- 172.21.15.139:0/3210231592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9d2f0 0x557c80ba7670 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1554033000 tx=0x7f1554032ce0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.599+0000 7f156b489740 1 -- 172.21.15.139:0/3210231592 shutdown_connections 2024-09-16T01:40:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.599+0000 7f156b489740 1 --2- 172.21.15.139:0/3210231592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80ba7bb0 0x557c80ba9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.599+0000 7f156b489740 1 --2- 172.21.15.139:0/3210231592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9d2f0 0x557c80ba7670 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.599+0000 7f156b489740 1 --2- 172.21.15.139:0/3210231592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9c9d0 0x557c80b9cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.599+0000 7f156b489740 1 -- 172.21.15.139:0/3210231592 >> 172.21.15.139:0/3210231592 conn(0x557c80b98f60 msgr2=0x557c80b99340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:15.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.599+0000 7f156b489740 1 -- 172.21.15.139:0/3210231592 shutdown_connections 2024-09-16T01:40:15.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.599+0000 7f156b489740 1 -- 172.21.15.139:0/3210231592 wait complete. 2024-09-16T01:40:15.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.599+0000 7f156b489740 1 -- start start 2024-09-16T01:40:15.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f156b489740 1 Processor -- start 2024-09-16T01:40:15.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f156b489740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80ba7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f156b489740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80b9d2f0 0x557c80ba4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f156b489740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80ba7bb0 0x557c80ba4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f156b489740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557c80baa780 con 0x557c80b9c9d0 2024-09-16T01:40:15.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f156b489740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557c80abdf20 con 0x557c80b9d2f0 2024-09-16T01:40:15.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f156b489740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557c80baa900 con 0x557c80ba7bb0 2024-09-16T01:40:15.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f156b489740 10 client.0 _ll_register_callbacks cb 0x557c80a87030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f155bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80ba7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f155bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80ba7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55474/0 (socket says 172.21.15.139:55474) 2024-09-16T01:40:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f155bfff640 1 -- 172.21.15.139:0/272770129 learned_addr learned my addr 172.21.15.139:0/272770129 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f1562d9d640 1 --2- 172.21.15.139:0/272770129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80b9d2f0 0x557c80ba4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f155bfff640 1 -- 172.21.15.139:0/272770129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80ba7bb0 msgr2=0x557c80ba4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f155bfff640 1 --2- 172.21.15.139:0/272770129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80ba7bb0 0x557c80ba4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f155bfff640 1 -- 172.21.15.139:0/272770129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80b9d2f0 msgr2=0x557c80ba4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f155bfff640 1 --2- 172.21.15.139:0/272770129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80b9d2f0 0x557c80ba4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f155bfff640 1 -- 172.21.15.139:0/272770129 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c808742b0 con 0x557c80b9c9d0 2024-09-16T01:40:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f155bfff640 1 --2- 172.21.15.139:0/272770129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80ba7430 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f154c00bd60 tx=0x7f154c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19cbbeca37872226 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f154a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f154a7fc640 1 -- 172.21.15.139:0/272770129 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c011ab0 con 0x557c80b9c9d0 2024-09-16T01:40:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.600+0000 7f154a7fc640 1 -- 172.21.15.139:0/272770129 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f154c011c50 con 0x557c80b9c9d0 2024-09-16T01:40:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.601+0000 7f156b489740 1 -- 172.21.15.139:0/272770129 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15540317b0 con 0x557c80b9c9d0 2024-09-16T01:40:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.601+0000 7f154a7fc640 1 -- 172.21.15.139:0/272770129 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f154c011e50 con 0x557c80b9c9d0 2024-09-16T01:40:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.601+0000 7f156b489740 10 client.15276 fetch_fsmap learned FSMap version 28 2024-09-16T01:40:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.601+0000 7f156b489740 1 -- 172.21.15.139:0/272770129 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c80bb60f0 con 0x557c80b9c9d0 2024-09-16T01:40:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.601+0000 7f154a7fc640 1 -- 172.21.15.139:0/272770129 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15540317b0 con 0x557c80b9c9d0 2024-09-16T01:40:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.601+0000 7f154a7fc640 1 -- 172.21.15.139:0/272770129 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f154c00f040 con 0x557c80b9c9d0 2024-09-16T01:40:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.601+0000 7f156b489740 10 client.15276 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:40:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.601+0000 7f156b489740 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T01:40:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.601+0000 7f156b489740 20 client.15276 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.601+0000 7f156b489740 1 --2- 172.21.15.139:0/272770129 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x557c80bb86d0 0x557c80bb9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.601+0000 7f156b489740 4 client.15276 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:40:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.601+0000 7f156b489740 1 -- 172.21.15.139:0/272770129 --> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c80b10600 con 0x557c80bb86d0 2024-09-16T01:40:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.601+0000 7f1562d9d640 1 --2- 172.21.15.139:0/272770129 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x557c80bb86d0 0x557c80bb9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.602+0000 7f1562d9d640 1 --2- 172.21.15.139:0/272770129 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x557c80bb86d0 0x557c80bb9b40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=412c1e396a3930ec server_cookie=ac3195c1a5fa0c40 in_seq=0 out_seq=0 2024-09-16T01:40:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.602+0000 7f154a7fc640 10 client.15276 ms_handle_connect on v2:172.21.15.176:6838/3912805068 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f154a7fc640 1 -- 172.21.15.139:0/272770129 <== mds.1 v2:172.21.15.176:6838/3912805068 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c80b10600 con 0x557c80bb86d0 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f154a7fc640 10 client.15276 handle_command_reply: tid=0 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f156b489740 1 client.15276 shutdown 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f156b489740 1 -- 172.21.15.139:0/272770129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 msgr2=0x557c80ba7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f156b489740 1 --2- 172.21.15.139:0/272770129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80ba7430 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f154c00bd60 tx=0x7f154c00be50 comp rx=0 tx=0).stop 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f156b489740 1 -- 172.21.15.139:0/272770129 shutdown_connections 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f156b489740 1 --2- 172.21.15.139:0/272770129 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x557c80bb86d0 0x557c80bb9b40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f156b489740 1 --2- 172.21.15.139:0/272770129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80ba7bb0 0x557c80ba4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f156b489740 1 --2- 172.21.15.139:0/272770129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80b9d2f0 0x557c80ba4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f156b489740 1 --2- 172.21.15.139:0/272770129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80ba7430 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f154a7fc640 0 client.15276 ms_handle_reset on v2:172.21.15.176:6838/3912805068 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f156b489740 1 -- 172.21.15.139:0/272770129 >> 172.21.15.139:0/272770129 conn(0x557c80b98f60 msgr2=0x557c80b9ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f156b489740 1 -- 172.21.15.139:0/272770129 shutdown_connections 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f156b489740 1 -- 172.21.15.139:0/272770129 wait complete. 2024-09-16T01:40:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.603+0000 7f156b489740 20 client.15276 trim_cache size 0 max 16384 2024-09-16T01:40:15.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.620+0000 7f156b489740 1 Processor -- start 2024-09-16T01:40:15.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f156b489740 1 -- start start 2024-09-16T01:40:15.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f156b489740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80b97ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f156b489740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9d2f0 0x557c80b98430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f156b489740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80ba7bb0 0x557c80ba19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f156b489740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557c80bb7e80 con 0x557c80b9c9d0 2024-09-16T01:40:15.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f156b489740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557c80bb8550 con 0x557c80ba7bb0 2024-09-16T01:40:15.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f156b489740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557c80bb8c20 con 0x557c80b9d2f0 2024-09-16T01:40:15.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f155bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80b97ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f155bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80b97ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55484/0 (socket says 172.21.15.139:55484) 2024-09-16T01:40:15.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f155bfff640 1 -- 172.21.15.139:0/3754270973 learned_addr learned my addr 172.21.15.139:0/3754270973 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:15.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f155bfff640 1 -- 172.21.15.139:0/3754270973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9d2f0 msgr2=0x557c80b98430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:15.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f155bfff640 1 --2- 172.21.15.139:0/3754270973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9d2f0 0x557c80b98430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f155bfff640 1 -- 172.21.15.139:0/3754270973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80ba7bb0 msgr2=0x557c80ba19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:15.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f155bfff640 1 --2- 172.21.15.139:0/3754270973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80ba7bb0 0x557c80ba19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f155bfff640 1 -- 172.21.15.139:0/3754270973 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c80b9c800 con 0x557c80b9c9d0 2024-09-16T01:40:15.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.621+0000 7f155bfff640 1 --2- 172.21.15.139:0/3754270973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80b97ef0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f154c0057c0 tx=0x7f154c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f67d1fc94840c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:15.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.622+0000 7f1558ff9640 1 -- 172.21.15.139:0/3754270973 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c011c50 con 0x557c80b9c9d0 2024-09-16T01:40:15.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.622+0000 7f1558ff9640 1 -- 172.21.15.139:0/3754270973 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154c011df0 con 0x557c80b9c9d0 2024-09-16T01:40:15.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.622+0000 7f1558ff9640 1 -- 172.21.15.139:0/3754270973 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c01d630 con 0x557c80b9c9d0 2024-09-16T01:40:15.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.622+0000 7f156b489740 1 -- 172.21.15.139:0/3754270973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 msgr2=0x557c80b97ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:15.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.622+0000 7f156b489740 1 --2- 172.21.15.139:0/3754270973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80b97ef0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f154c0057c0 tx=0x7f154c00b880 comp rx=0 tx=0).stop 2024-09-16T01:40:15.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.622+0000 7f156b489740 1 -- 172.21.15.139:0/3754270973 shutdown_connections 2024-09-16T01:40:15.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.622+0000 7f156b489740 1 --2- 172.21.15.139:0/3754270973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80ba7bb0 0x557c80ba19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.622+0000 7f156b489740 1 --2- 172.21.15.139:0/3754270973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9d2f0 0x557c80b98430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.622+0000 7f156b489740 1 --2- 172.21.15.139:0/3754270973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80b97ef0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.622+0000 7f156b489740 1 -- 172.21.15.139:0/3754270973 >> 172.21.15.139:0/3754270973 conn(0x557c80b98f60 msgr2=0x557c80b977e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:15.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.623+0000 7f156b489740 1 -- 172.21.15.139:0/3754270973 shutdown_connections 2024-09-16T01:40:15.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.623+0000 7f156b489740 1 -- 172.21.15.139:0/3754270973 wait complete. 2024-09-16T01:40:15.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.623+0000 7f156b489740 1 -- start start 2024-09-16T01:40:15.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.623+0000 7f156b489740 1 Processor -- start 2024-09-16T01:40:15.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.623+0000 7f156b489740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80bc22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.623+0000 7f156b489740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9d2f0 0x557c80bbd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.623+0000 7f156b489740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80ba7bb0 0x557c80bbdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.623+0000 7f156b489740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557c80bb99c0 con 0x557c80b9c9d0 2024-09-16T01:40:15.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.623+0000 7f156b489740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557c80bb92f0 con 0x557c80ba7bb0 2024-09-16T01:40:15.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.623+0000 7f156b489740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557c80bba090 con 0x557c80b9d2f0 2024-09-16T01:40:15.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.623+0000 7f156b489740 10 client.0 _ll_register_callbacks cb 0x7f154c00bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:15.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.623+0000 7f155bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80bc22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.623+0000 7f155bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80bc22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55490/0 (socket says 172.21.15.139:55490) 2024-09-16T01:40:15.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.623+0000 7f155bfff640 1 -- 172.21.15.139:0/975460452 learned_addr learned my addr 172.21.15.139:0/975460452 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:15.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f1562d9d640 1 --2- 172.21.15.139:0/975460452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9d2f0 0x557c80bbd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f155bfff640 1 --2- 172.21.15.139:0/975460452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80ba7bb0 0x557c80bbdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f155bfff640 1 -- 172.21.15.139:0/975460452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9d2f0 msgr2=0x557c80bbd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:15.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f155bfff640 1 --2- 172.21.15.139:0/975460452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9d2f0 0x557c80bbd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f155bfff640 1 -- 172.21.15.139:0/975460452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80ba7bb0 msgr2=0x557c80bbdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:15.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f155bfff640 1 --2- 172.21.15.139:0/975460452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80ba7bb0 0x557c80bbdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f155bfff640 1 -- 172.21.15.139:0/975460452 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c80ba52a0 con 0x557c80b9c9d0 2024-09-16T01:40:15.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f1562d9d640 1 --2- 172.21.15.139:0/975460452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9d2f0 0x557c80bbd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:15.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f155bfff640 1 --2- 172.21.15.139:0/975460452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80bc22f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f154c012940 tx=0x7f154c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9e665f942f866b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:15.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f154a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:15.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f154a7fc640 1 -- 172.21.15.139:0/975460452 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c03d060 con 0x557c80b9c9d0 2024-09-16T01:40:15.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f154a7fc640 1 -- 172.21.15.139:0/975460452 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f154c004320 con 0x557c80b9c9d0 2024-09-16T01:40:15.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f156b489740 1 -- 172.21.15.139:0/975460452 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f154c011df0 con 0x557c80b9c9d0 2024-09-16T01:40:15.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f154a7fc640 1 -- 172.21.15.139:0/975460452 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f154c005040 con 0x557c80b9c9d0 2024-09-16T01:40:15.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f156b489740 10 client.15282 fetch_fsmap learned FSMap version 28 2024-09-16T01:40:15.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f156b489740 1 -- 172.21.15.139:0/975460452 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c80c229a0 con 0x557c80b9c9d0 2024-09-16T01:40:15.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.624+0000 7f154a7fc640 1 -- 172.21.15.139:0/975460452 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154c011df0 con 0x557c80b9c9d0 2024-09-16T01:40:15.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.625+0000 7f154a7fc640 1 -- 172.21.15.139:0/975460452 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f154c004c40 con 0x557c80b9c9d0 2024-09-16T01:40:15.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.625+0000 7f156b489740 10 client.15282 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:40:15.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.625+0000 7f156b489740 10 client.15282 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T01:40:15.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.625+0000 7f156b489740 20 client.15282 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:15.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.625+0000 7f156b489740 1 --2- 172.21.15.139:0/975460452 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x557c80bb86d0 0x557c80c25f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:15.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.625+0000 7f156b489740 4 client.15282 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:40:15.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.625+0000 7f156b489740 1 -- 172.21.15.139:0/975460452 --> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] -- command(tid 0: {"prefix": "status"}) -- 0x557c80b0ab10 con 0x557c80bb86d0 2024-09-16T01:40:15.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.625+0000 7f1562d9d640 1 --2- 172.21.15.139:0/975460452 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x557c80bb86d0 0x557c80c25f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:15.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.625+0000 7f1562d9d640 1 --2- 172.21.15.139:0/975460452 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x557c80bb86d0 0x557c80c25f50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5020dfc1fc024cde server_cookie=1fc8f6348614ac94 in_seq=0 out_seq=0 2024-09-16T01:40:15.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.625+0000 7f154a7fc640 10 client.15282 ms_handle_connect on v2:172.21.15.176:6838/3912805068 2024-09-16T01:40:15.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.625+0000 7f154a7fc640 1 -- 172.21.15.139:0/975460452 <== mds.1 v2:172.21.15.176:6838/3912805068 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557c80b0ab10 con 0x557c80bb86d0 2024-09-16T01:40:15.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.625+0000 7f154a7fc640 10 client.15282 handle_command_reply: tid=0 2024-09-16T01:40:15.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.625+0000 7f156b489740 1 client.15282 shutdown 2024-09-16T01:40:15.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f156b489740 1 -- 172.21.15.139:0/975460452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 msgr2=0x557c80bc22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:15.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f156b489740 1 --2- 172.21.15.139:0/975460452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80bc22f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f154c012940 tx=0x7f154c0039d0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f156b489740 1 -- 172.21.15.139:0/975460452 shutdown_connections 2024-09-16T01:40:15.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f156b489740 1 --2- 172.21.15.139:0/975460452 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x557c80bb86d0 0x557c80c25f50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f156b489740 1 --2- 172.21.15.139:0/975460452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c80ba7bb0 0x557c80bbdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f156b489740 1 --2- 172.21.15.139:0/975460452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c80b9d2f0 0x557c80bbd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f156b489740 1 --2- 172.21.15.139:0/975460452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c80b9c9d0 0x557c80bc22f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f156b489740 1 -- 172.21.15.139:0/975460452 >> 172.21.15.139:0/975460452 conn(0x557c80b98f60 msgr2=0x557c80b98000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:15.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f154a7fc640 0 client.15282 ms_handle_reset on v2:172.21.15.176:6838/3912805068 2024-09-16T01:40:15.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f156b489740 1 -- 172.21.15.139:0/975460452 shutdown_connections 2024-09-16T01:40:15.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f156b489740 1 -- 172.21.15.139:0/975460452 wait complete. 2024-09-16T01:40:15.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f156b489740 20 client.15282 trim_cache size 0 max 16384 2024-09-16T01:40:15.635 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:40:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:40:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 1, 2024-09-16T01:40:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24454, 2024-09-16T01:40:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:40:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:40:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:40:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 99.107698772000006, 2024-09-16T01:40:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 28, 2024-09-16T01:40:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 85, 2024-09-16T01:40:15.636 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 83, 2024-09-16T01:40:15.636 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 154.783580503 2024-09-16T01:40:15.636 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:40:15.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f15690b0640 1 -- 172.21.15.139:0/4044981253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f15500a3240 msgr2=0x7f15500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:15.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f15690b0640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f15500a3240 0x7f15500a5700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f155c0cf640 tx=0x7f155c065f80 comp rx=0 tx=0).stop 2024-09-16T01:40:15.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f15690b0640 1 -- 172.21.15.139:0/4044981253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564106f10 msgr2=0x7f156419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:15.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f15690b0640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564106f10 0x7f156419cd80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f155c00b970 tx=0x7f155c00be30 comp rx=0 tx=0).stop 2024-09-16T01:40:15.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f15690b0640 1 -- 172.21.15.139:0/4044981253 shutdown_connections 2024-09-16T01:40:15.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f15690b0640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f15500a3240 0x7f15500a5700 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f15690b0640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564106f10 0x7f156419cd80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f15690b0640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1564102790 0x7f15641989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f15690b0640 1 --2- 172.21.15.139:0/4044981253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1564101de0 0x7f15641984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:15.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.626+0000 7f15690b0640 1 -- 172.21.15.139:0/4044981253 >> 172.21.15.139:0/4044981253 conn(0x7f15640fd8c0 msgr2=0x7f15640fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:15.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.627+0000 7f15690b0640 1 -- 172.21.15.139:0/4044981253 shutdown_connections 2024-09-16T01:40:15.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:15.627+0000 7f15690b0640 1 -- 172.21.15.139:0/4044981253 wait complete. 2024-09-16T01:40:16.390 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T01:40:16.637 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:16.847 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:16 smithi164 ceph-mon[35121]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-16T01:40:16.897 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:16 smithi139 ceph-mon[31175]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-16T01:40:16.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:16 smithi176 ceph-mon[35324]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-16T01:40:17.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.127+0000 7fbc5e724640 1 -- 172.21.15.139:0/2864449827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc58102570 msgr2=0x7fbc581029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:17.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.127+0000 7fbc5e724640 1 --2- 172.21.15.139:0/2864449827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc58102570 0x7fbc581029b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009900 tx=0x7fbc4c02a480 comp rx=0 tx=0).stop 2024-09-16T01:40:17.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.127+0000 7fbc5e724640 1 -- 172.21.15.139:0/2864449827 shutdown_connections 2024-09-16T01:40:17.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.127+0000 7fbc5e724640 1 --2- 172.21.15.139:0/2864449827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc58102ef0 0x7fbc5810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.127+0000 7fbc5e724640 1 --2- 172.21.15.139:0/2864449827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc58102570 0x7fbc581029b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.127+0000 7fbc5e724640 1 --2- 172.21.15.139:0/2864449827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc58100e20 0x7fbc58101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.127+0000 7fbc5e724640 1 -- 172.21.15.139:0/2864449827 >> 172.21.15.139:0/2864449827 conn(0x7fbc580fc7d0 msgr2=0x7fbc580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:17.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.127+0000 7fbc5e724640 1 -- 172.21.15.139:0/2864449827 shutdown_connections 2024-09-16T01:40:17.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.127+0000 7fbc5e724640 1 -- 172.21.15.139:0/2864449827 wait complete. 2024-09-16T01:40:17.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.128+0000 7fbc5e724640 1 Processor -- start 2024-09-16T01:40:17.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.128+0000 7fbc5e724640 1 -- start start 2024-09-16T01:40:17.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.128+0000 7fbc5e724640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc58100e20 0x7fbc581993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.128+0000 7fbc5e724640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc58102570 0x7fbc58199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.128+0000 7fbc5e724640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc58102ef0 0x7fbc581a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.128+0000 7fbc5e724640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58110f80 con 0x7fbc58102570 2024-09-16T01:40:17.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.128+0000 7fbc5e724640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58110e00 con 0x7fbc58102ef0 2024-09-16T01:40:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.128+0000 7fbc5e724640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58111100 con 0x7fbc58100e20 2024-09-16T01:40:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.128+0000 7fbc577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc58102570 0x7fbc58199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.128+0000 7fbc57fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc58100e20 0x7fbc581993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.128+0000 7fbc577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc58102570 0x7fbc58199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55506/0 (socket says 172.21.15.139:55506) 2024-09-16T01:40:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.128+0000 7fbc577fe640 1 -- 172.21.15.139:0/852550176 learned_addr learned my addr 172.21.15.139:0/852550176 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.129+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/852550176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc58102ef0 0x7fbc581a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.129+0000 7fbc57fff640 1 -- 172.21.15.139:0/852550176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc58102ef0 msgr2=0x7fbc581a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.129+0000 7fbc57fff640 1 --2- 172.21.15.139:0/852550176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc58102ef0 0x7fbc581a4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.129+0000 7fbc57fff640 1 -- 172.21.15.139:0/852550176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc58102570 msgr2=0x7fbc58199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.129+0000 7fbc57fff640 1 --2- 172.21.15.139:0/852550176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc58102570 0x7fbc58199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.129+0000 7fbc57fff640 1 -- 172.21.15.139:0/852550176 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc581a49a0 con 0x7fbc58100e20 2024-09-16T01:40:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.129+0000 7fbc577fe640 1 --2- 172.21.15.139:0/852550176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc58102570 0x7fbc58199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:40:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.129+0000 7fbc57fff640 1 --2- 172.21.15.139:0/852550176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc58100e20 0x7fbc581993e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800e9f0 tx=0x7fbc4800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.129+0000 7fbc557fa640 1 -- 172.21.15.139:0/852550176 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4800ccc0 con 0x7fbc58100e20 2024-09-16T01:40:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.129+0000 7fbc5e724640 1 -- 172.21.15.139:0/852550176 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc581a4b70 con 0x7fbc58100e20 2024-09-16T01:40:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.129+0000 7fbc557fa640 1 -- 172.21.15.139:0/852550176 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc48005020 con 0x7fbc58100e20 2024-09-16T01:40:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.129+0000 7fbc557fa640 1 -- 172.21.15.139:0/852550176 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc480049e0 con 0x7fbc58100e20 2024-09-16T01:40:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.129+0000 7fbc5e724640 1 -- 172.21.15.139:0/852550176 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc581a5080 con 0x7fbc58100e20 2024-09-16T01:40:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.132+0000 7fbc60afd740 1 Processor -- start 2024-09-16T01:40:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.132+0000 7fbc557fa640 1 -- 172.21.15.139:0/852550176 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fbc48005560 con 0x7fbc58100e20 2024-09-16T01:40:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.132+0000 7fbc60afd740 1 -- start start 2024-09-16T01:40:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.132+0000 7fbc60afd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070cf9d0 0x5566070cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.132+0000 7fbc60afd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070d02f0 0x5566070da670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.132+0000 7fbc60afd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070dabb0 0x5566070dcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.132+0000 7fbc60afd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556607036090 con 0x5566070cf9d0 2024-09-16T01:40:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.132+0000 7fbc60afd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556606fdda70 con 0x5566070dabb0 2024-09-16T01:40:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.132+0000 7fbc60afd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556607032050 con 0x5566070d02f0 2024-09-16T01:40:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc5ccc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070cf9d0 0x5566070cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc5ccc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070cf9d0 0x5566070cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55516/0 (socket says 172.21.15.139:55516) 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc5ccc1640 1 -- 172.21.15.139:0/2248597918 learned_addr learned my addr 172.21.15.139:0/2248597918 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc577fe640 1 --2- 172.21.15.139:0/2248597918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070dabb0 0x5566070dcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc57fff640 1 --2- 172.21.15.139:0/2248597918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070d02f0 0x5566070da670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc5ccc1640 1 -- 172.21.15.139:0/2248597918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070d02f0 msgr2=0x5566070da670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/2248597918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070d02f0 0x5566070da670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc5ccc1640 1 -- 172.21.15.139:0/2248597918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070dabb0 msgr2=0x5566070dcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/2248597918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070dabb0 0x5566070dcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc5ccc1640 1 -- 172.21.15.139:0/2248597918 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556606fea130 con 0x5566070cf9d0 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc57fff640 1 --2- 172.21.15.139:0/2248597918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070d02f0 0x5566070da670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc557fa640 1 --2- 172.21.15.139:0/852550176 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbc400a3240 0x7fbc400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/2248597918 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070cf9d0 0x5566070cfdb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009f90 tx=0x7fbc50010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f190ae5cdd34739 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc57fff640 1 --2- 172.21.15.139:0/852550176 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbc400a3240 0x7fbc400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc3e7fc640 1 -- 172.21.15.139:0/2248597918 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50011700 con 0x5566070cf9d0 2024-09-16T01:40:17.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc3e7fc640 1 -- 172.21.15.139:0/2248597918 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc50011d80 con 0x5566070cf9d0 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc557fa640 1 -- 172.21.15.139:0/852550176 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fbc480cfc30 con 0x7fbc58100e20 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.133+0000 7fbc3e7fc640 1 -- 172.21.15.139:0/2248597918 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50018380 con 0x5566070cf9d0 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.134+0000 7fbc60afd740 1 -- 172.21.15.139:0/2248597918 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070cf9d0 msgr2=0x5566070cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.134+0000 7fbc57fff640 1 --2- 172.21.15.139:0/852550176 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbc400a3240 0x7fbc400a5700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbc480cd6d0 tx=0x7fbc480054d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.134+0000 7fbc60afd740 1 --2- 172.21.15.139:0/2248597918 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070cf9d0 0x5566070cfdb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009f90 tx=0x7fbc50010860 comp rx=0 tx=0).stop 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.134+0000 7fbc60afd740 1 -- 172.21.15.139:0/2248597918 shutdown_connections 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.134+0000 7fbc60afd740 1 --2- 172.21.15.139:0/2248597918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070dabb0 0x5566070dcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.134+0000 7fbc60afd740 1 --2- 172.21.15.139:0/2248597918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070d02f0 0x5566070da670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.134+0000 7fbc60afd740 1 --2- 172.21.15.139:0/2248597918 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070cf9d0 0x5566070cfdb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.134+0000 7fbc60afd740 1 -- 172.21.15.139:0/2248597918 >> 172.21.15.139:0/2248597918 conn(0x5566070cbf60 msgr2=0x5566070cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.134+0000 7fbc60afd740 1 -- 172.21.15.139:0/2248597918 shutdown_connections 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.134+0000 7fbc60afd740 1 -- 172.21.15.139:0/2248597918 wait complete. 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.134+0000 7fbc60afd740 1 -- start start 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc60afd740 1 Processor -- start 2024-09-16T01:40:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc60afd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070cf9d0 0x5566070da3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc60afd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070d02f0 0x5566070d7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc60afd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070dabb0 0x5566070d7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc60afd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5566070dd780 con 0x5566070dabb0 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc60afd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556606ff0f20 con 0x5566070d02f0 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc60afd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5566070dd900 con 0x5566070cf9d0 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc60afd740 10 client.0 _ll_register_callbacks cb 0x556606fba030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc5ccc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070dabb0 0x5566070d7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc577fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070d02f0 0x5566070d7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc5ccc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070dabb0 0x5566070d7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55518/0 (socket says 172.21.15.139:55518) 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc5ccc1640 1 -- 172.21.15.139:0/202389073 learned_addr learned my addr 172.21.15.139:0/202389073 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/202389073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070cf9d0 0x5566070da3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc577fe640 1 -- 172.21.15.139:0/202389073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070cf9d0 msgr2=0x5566070da3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc577fe640 1 --2- 172.21.15.139:0/202389073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070cf9d0 0x5566070da3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc577fe640 1 -- 172.21.15.139:0/202389073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070dabb0 msgr2=0x5566070d7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:17.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc577fe640 1 --2- 172.21.15.139:0/202389073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070dabb0 0x5566070d7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc577fe640 1 -- 172.21.15.139:0/202389073 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556606da72b0 con 0x5566070d02f0 2024-09-16T01:40:17.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.135+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/202389073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070dabb0 0x5566070d7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:40:17.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc577fe640 1 --2- 172.21.15.139:0/202389073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070d02f0 0x5566070d7720 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009230 tx=0x7fbc4c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=651b021e8be5e845 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:17.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc377fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:40:17.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc377fe640 1 -- 172.21.15.139:0/202389073 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c035040 con 0x5566070d02f0 2024-09-16T01:40:17.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc60afd740 1 -- 172.21.15.139:0/202389073 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc50011d80 con 0x5566070d02f0 2024-09-16T01:40:17.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc377fe640 1 -- 172.21.15.139:0/202389073 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fbc4c003e20 con 0x5566070d02f0 2024-09-16T01:40:17.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc377fe640 1 -- 172.21.15.139:0/202389073 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc4c04f050 con 0x5566070d02f0 2024-09-16T01:40:17.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc60afd740 10 client.24902 fetch_fsmap learned FSMap version 28 2024-09-16T01:40:17.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc60afd740 1 -- 172.21.15.139:0/202389073 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5566070e9030 con 0x5566070d02f0 2024-09-16T01:40:17.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc377fe640 1 -- 172.21.15.139:0/202389073 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc50011d80 con 0x5566070d02f0 2024-09-16T01:40:17.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc377fe640 1 -- 172.21.15.139:0/202389073 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbc4c046750 con 0x5566070d02f0 2024-09-16T01:40:17.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc60afd740 10 client.24902 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:40:17.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc60afd740 10 client.24902 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T01:40:17.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc60afd740 20 client.24902 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:17.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc60afd740 1 --2- 172.21.15.139:0/202389073 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5566070eb600 0x5566070eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc60afd740 4 client.24902 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:40:17.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.136+0000 7fbc60afd740 1 -- 172.21.15.139:0/202389073 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556607043600 con 0x5566070eb600 2024-09-16T01:40:17.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.137+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/202389073 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5566070eb600 0x5566070eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.137+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/202389073 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5566070eb600 0x5566070eca70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3edcac4b18231b3 server_cookie=ad4d09ce844c4025 in_seq=0 out_seq=0 2024-09-16T01:40:17.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.137+0000 7fbc377fe640 10 client.24902 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:17.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc377fe640 1 -- 172.21.15.139:0/202389073 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556607043600 con 0x5566070eb600 2024-09-16T01:40:17.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc377fe640 10 client.24902 handle_command_reply: tid=0 2024-09-16T01:40:17.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc60afd740 1 client.24902 shutdown 2024-09-16T01:40:17.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc60afd740 1 -- 172.21.15.139:0/202389073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070d02f0 msgr2=0x5566070d7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:17.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc60afd740 1 --2- 172.21.15.139:0/202389073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070d02f0 0x5566070d7720 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009230 tx=0x7fbc4c003820 comp rx=0 tx=0).stop 2024-09-16T01:40:17.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc60afd740 1 -- 172.21.15.139:0/202389073 shutdown_connections 2024-09-16T01:40:17.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc60afd740 1 --2- 172.21.15.139:0/202389073 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5566070eb600 0x5566070eca70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc60afd740 1 --2- 172.21.15.139:0/202389073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070dabb0 0x5566070d7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc60afd740 1 --2- 172.21.15.139:0/202389073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070d02f0 0x5566070d7720 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc377fe640 0 client.24902 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:17.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc60afd740 1 --2- 172.21.15.139:0/202389073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070cf9d0 0x5566070da3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc60afd740 1 -- 172.21.15.139:0/202389073 >> 172.21.15.139:0/202389073 conn(0x5566070cbf60 msgr2=0x5566070d1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:17.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc60afd740 1 -- 172.21.15.139:0/202389073 shutdown_connections 2024-09-16T01:40:17.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc60afd740 1 -- 172.21.15.139:0/202389073 wait complete. 2024-09-16T01:40:17.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.138+0000 7fbc60afd740 20 client.24902 trim_cache size 0 max 16384 2024-09-16T01:40:17.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc60afd740 1 Processor -- start 2024-09-16T01:40:17.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc60afd740 1 -- start start 2024-09-16T01:40:17.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc60afd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070cf9d0 0x5566070eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc60afd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070d02f0 0x5566070f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc60afd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070dabb0 0x5566070d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc60afd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5566070eadb0 con 0x5566070dabb0 2024-09-16T01:40:17.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc60afd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5566070eb480 con 0x5566070d02f0 2024-09-16T01:40:17.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc60afd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5566070ebb50 con 0x5566070cf9d0 2024-09-16T01:40:17.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc5ccc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070dabb0 0x5566070d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc5ccc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070dabb0 0x5566070d2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55532/0 (socket says 172.21.15.139:55532) 2024-09-16T01:40:17.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc5ccc1640 1 -- 172.21.15.139:0/433165620 learned_addr learned my addr 172.21.15.139:0/433165620 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:17.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc577fe640 1 --2- 172.21.15.139:0/433165620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070d02f0 0x5566070f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/433165620 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070cf9d0 0x5566070eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc5ccc1640 1 -- 172.21.15.139:0/433165620 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070cf9d0 msgr2=0x5566070eca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:17.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/433165620 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070cf9d0 0x5566070eca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc5ccc1640 1 -- 172.21.15.139:0/433165620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070d02f0 msgr2=0x5566070f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:17.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/433165620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070d02f0 0x5566070f5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.158+0000 7fbc5ccc1640 1 -- 172.21.15.139:0/433165620 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566070d0120 con 0x5566070dabb0 2024-09-16T01:40:17.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.159+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/433165620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070dabb0 0x5566070d2610 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbc50011fd0 tx=0x7fbc500096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4913e0c84c0221d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:17.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.159+0000 7fbc3dffb640 1 -- 172.21.15.139:0/433165620 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc500196d0 con 0x5566070dabb0 2024-09-16T01:40:17.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.159+0000 7fbc3dffb640 1 -- 172.21.15.139:0/433165620 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc50020040 con 0x5566070dabb0 2024-09-16T01:40:17.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.159+0000 7fbc3dffb640 1 -- 172.21.15.139:0/433165620 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc500054b0 con 0x5566070dabb0 2024-09-16T01:40:17.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.159+0000 7fbc60afd740 1 -- 172.21.15.139:0/433165620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070dabb0 msgr2=0x5566070d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:17.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.159+0000 7fbc60afd740 1 --2- 172.21.15.139:0/433165620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070dabb0 0x5566070d2610 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbc50011fd0 tx=0x7fbc500096a0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.159+0000 7fbc60afd740 1 -- 172.21.15.139:0/433165620 shutdown_connections 2024-09-16T01:40:17.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.159+0000 7fbc60afd740 1 --2- 172.21.15.139:0/433165620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070dabb0 0x5566070d2610 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.159+0000 7fbc60afd740 1 --2- 172.21.15.139:0/433165620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070d02f0 0x5566070f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.159+0000 7fbc60afd740 1 --2- 172.21.15.139:0/433165620 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070cf9d0 0x5566070eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.159+0000 7fbc60afd740 1 -- 172.21.15.139:0/433165620 >> 172.21.15.139:0/433165620 conn(0x5566070cbf60 msgr2=0x5566070d5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:17.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc60afd740 1 -- 172.21.15.139:0/433165620 shutdown_connections 2024-09-16T01:40:17.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc60afd740 1 -- 172.21.15.139:0/433165620 wait complete. 2024-09-16T01:40:17.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc60afd740 1 -- start start 2024-09-16T01:40:17.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc60afd740 1 Processor -- start 2024-09-16T01:40:17.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc60afd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070cf9d0 0x5566070f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc60afd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070d02f0 0x5566070f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc60afd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070dabb0 0x5566070f5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc60afd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5566070ec8f0 con 0x5566070d02f0 2024-09-16T01:40:17.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc60afd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5566070ec220 con 0x5566070cf9d0 2024-09-16T01:40:17.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc60afd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5566070ecfc0 con 0x5566070dabb0 2024-09-16T01:40:17.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc60afd740 10 client.0 _ll_register_callbacks cb 0x7fbc4c004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:17.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070d02f0 0x5566070f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070d02f0 0x5566070f4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55546/0 (socket says 172.21.15.139:55546) 2024-09-16T01:40:17.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc577fe640 1 -- 172.21.15.139:0/661120259 learned_addr learned my addr 172.21.15.139:0/661120259 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:17.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.160+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/661120259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070cf9d0 0x5566070f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/661120259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070dabb0 0x5566070f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc577fe640 1 -- 172.21.15.139:0/661120259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070dabb0 msgr2=0x5566070f5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:17.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc577fe640 1 --2- 172.21.15.139:0/661120259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070dabb0 0x5566070f5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc577fe640 1 -- 172.21.15.139:0/661120259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070cf9d0 msgr2=0x5566070f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc577fe640 1 --2- 172.21.15.139:0/661120259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070cf9d0 0x5566070f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc577fe640 1 -- 172.21.15.139:0/661120259 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556607158d40 con 0x5566070d02f0 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/661120259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070dabb0 0x5566070f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc577fe640 1 --2- 172.21.15.139:0/661120259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070d02f0 0x5566070f4f10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0096c0 tx=0x7fbc4c009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76d9ac21be3d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc377fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc377fe640 1 -- 172.21.15.139:0/661120259 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c035040 con 0x5566070d02f0 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc377fe640 1 -- 172.21.15.139:0/661120259 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fbc4c04c040 con 0x5566070d02f0 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc60afd740 1 -- 172.21.15.139:0/661120259 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc50020040 con 0x5566070d02f0 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc377fe640 1 -- 172.21.15.139:0/661120259 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc4c05f050 con 0x5566070d02f0 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc60afd740 10 client.15300 fetch_fsmap learned FSMap version 28 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc60afd740 1 -- 172.21.15.139:0/661120259 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556607159f40 con 0x5566070d02f0 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.161+0000 7fbc377fe640 1 -- 172.21.15.139:0/661120259 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc50020040 con 0x5566070d02f0 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.162+0000 7fbc377fe640 1 -- 172.21.15.139:0/661120259 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbc4c0318a0 con 0x5566070d02f0 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.162+0000 7fbc60afd740 10 client.15300 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T01:40:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.162+0000 7fbc60afd740 10 client.15300 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T01:40:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.162+0000 7fbc60afd740 20 client.15300 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.162+0000 7fbc60afd740 1 --2- 172.21.15.139:0/661120259 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55660715ea70 0x55660715d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.162+0000 7fbc60afd740 4 client.15300 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:40:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.162+0000 7fbc60afd740 1 -- 172.21.15.139:0/661120259 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "status"}) -- 0x55660703db10 con 0x55660715ea70 2024-09-16T01:40:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.162+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/661120259 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55660715ea70 0x55660715d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.162+0000 7fbc5ccc1640 1 --2- 172.21.15.139:0/661120259 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55660715ea70 0x55660715d780 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e24cb071a878cf92 server_cookie=6152b6f3f58928f5 in_seq=0 out_seq=0 2024-09-16T01:40:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.162+0000 7fbc377fe640 10 client.15300 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.162+0000 7fbc377fe640 1 -- 172.21.15.139:0/661120259 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55660703db10 con 0x55660715ea70 2024-09-16T01:40:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.162+0000 7fbc377fe640 10 client.15300 handle_command_reply: tid=0 2024-09-16T01:40:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.162+0000 7fbc60afd740 1 client.15300 shutdown 2024-09-16T01:40:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc60afd740 1 -- 172.21.15.139:0/661120259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070d02f0 msgr2=0x5566070f4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc60afd740 1 --2- 172.21.15.139:0/661120259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070d02f0 0x5566070f4f10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0096c0 tx=0x7fbc4c009590 comp rx=0 tx=0).stop 2024-09-16T01:40:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc60afd740 1 -- 172.21.15.139:0/661120259 shutdown_connections 2024-09-16T01:40:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc60afd740 1 --2- 172.21.15.139:0/661120259 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55660715ea70 0x55660715d780 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc60afd740 1 --2- 172.21.15.139:0/661120259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566070dabb0 0x5566070f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc60afd740 1 --2- 172.21.15.139:0/661120259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5566070d02f0 0x5566070f4f10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc60afd740 1 --2- 172.21.15.139:0/661120259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566070cf9d0 0x5566070f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc377fe640 0 client.15300 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:17.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc60afd740 1 -- 172.21.15.139:0/661120259 >> 172.21.15.139:0/661120259 conn(0x5566070cbf60 msgr2=0x5566070d1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:17.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc60afd740 1 -- 172.21.15.139:0/661120259 shutdown_connections 2024-09-16T01:40:17.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc60afd740 1 -- 172.21.15.139:0/661120259 wait complete. 2024-09-16T01:40:17.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc60afd740 20 client.15300 trim_cache size 0 max 16384 2024-09-16T01:40:17.170 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:40:17.170 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:40:17.170 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-09-16T01:40:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24470, 2024-09-16T01:40:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:40:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:40:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:40:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 134.65045755599999, 2024-09-16T01:40:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 28, 2024-09-16T01:40:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 85, 2024-09-16T01:40:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 83, 2024-09-16T01:40:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 146.709906738 2024-09-16T01:40:17.171 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:40:17.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc5e724640 1 -- 172.21.15.139:0/852550176 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbc400a3240 msgr2=0x7fbc400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:17.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc5e724640 1 --2- 172.21.15.139:0/852550176 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbc400a3240 0x7fbc400a5700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbc480cd6d0 tx=0x7fbc480054d0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc5e724640 1 -- 172.21.15.139:0/852550176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc58100e20 msgr2=0x7fbc581993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:17.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc5e724640 1 --2- 172.21.15.139:0/852550176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc58100e20 0x7fbc581993e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800e9f0 tx=0x7fbc4800eeb0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc5e724640 1 -- 172.21.15.139:0/852550176 shutdown_connections 2024-09-16T01:40:17.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc5e724640 1 --2- 172.21.15.139:0/852550176 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fbc400a3240 0x7fbc400a5700 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc5e724640 1 --2- 172.21.15.139:0/852550176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc58102ef0 0x7fbc581a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.163+0000 7fbc5e724640 1 --2- 172.21.15.139:0/852550176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc58102570 0x7fbc58199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.164+0000 7fbc5e724640 1 --2- 172.21.15.139:0/852550176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc58100e20 0x7fbc581993e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.164+0000 7fbc5e724640 1 -- 172.21.15.139:0/852550176 >> 172.21.15.139:0/852550176 conn(0x7fbc580fc7d0 msgr2=0x7fbc580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:17.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.164+0000 7fbc5e724640 1 -- 172.21.15.139:0/852550176 shutdown_connections 2024-09-16T01:40:17.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.164+0000 7fbc5e724640 1 -- 172.21.15.139:0/852550176 wait complete. 2024-09-16T01:40:17.858 INFO:teuthology.run_tasks:Running task sequential... 2024-09-16T01:40:17.871 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-16T01:40:17.871 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-16T01:40:17.871 INFO:teuthology.task.print:Enabling mgr modules 2024-09-16T01:40:17.871 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-16T01:40:17.872 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T01:40:17.872 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T01:40:17.872 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-16T01:40:17.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.970+0000 7f926bfe8640 1 -- 172.21.15.139:0/2058084640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92640ffba0 msgr2=0x7f9264100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:17.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.970+0000 7f926bfe8640 1 --2- 172.21.15.139:0/2058084640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92640ffba0 0x7f9264100020 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9258009930 tx=0x7f925802a420 comp rx=0 tx=0).stop 2024-09-16T01:40:17.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.970+0000 7f926bfe8640 1 -- 172.21.15.139:0/2058084640 shutdown_connections 2024-09-16T01:40:17.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.970+0000 7f926bfe8640 1 --2- 172.21.15.139:0/2058084640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9264100560 0x7f926410a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.970+0000 7f926bfe8640 1 --2- 172.21.15.139:0/2058084640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92640ffba0 0x7f9264100020 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.970+0000 7f926bfe8640 1 --2- 172.21.15.139:0/2058084640 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9264103130 0x7f9264103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.970+0000 7f926bfe8640 1 -- 172.21.15.139:0/2058084640 >> 172.21.15.139:0/2058084640 conn(0x7f92640fb970 msgr2=0x7f92640fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:17.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.970+0000 7f926bfe8640 1 -- 172.21.15.139:0/2058084640 shutdown_connections 2024-09-16T01:40:17.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.970+0000 7f926bfe8640 1 -- 172.21.15.139:0/2058084640 wait complete. 2024-09-16T01:40:17.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f926bfe8640 1 Processor -- start 2024-09-16T01:40:17.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f926bfe8640 1 -- start start 2024-09-16T01:40:17.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f926bfe8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92640ffba0 0x7f9264197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f926bfe8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9264100560 0x7f9264198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f926bfe8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9264103130 0x7f926419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f926bfe8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f926410d7d0 con 0x7f92640ffba0 2024-09-16T01:40:17.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f926bfe8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f926410d650 con 0x7f9264103130 2024-09-16T01:40:17.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f926bfe8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f926410d950 con 0x7f9264100560 2024-09-16T01:40:17.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f9269d84640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92640ffba0 0x7f9264197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f9269d84640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92640ffba0 0x7f9264197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55570/0 (socket says 172.21.15.139:55570) 2024-09-16T01:40:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f9269d84640 1 -- 172.21.15.139:0/478055029 learned_addr learned my addr 172.21.15.139:0/478055029 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f926a585640 1 --2- 172.21.15.139:0/478055029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9264103130 0x7f926419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f9269583640 1 --2- 172.21.15.139:0/478055029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9264100560 0x7f9264198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f9269d84640 1 -- 172.21.15.139:0/478055029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9264100560 msgr2=0x7f9264198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f9269d84640 1 --2- 172.21.15.139:0/478055029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9264100560 0x7f9264198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f9269d84640 1 -- 172.21.15.139:0/478055029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9264103130 msgr2=0x7f926419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f9269d84640 1 --2- 172.21.15.139:0/478055029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9264103130 0x7f926419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.971+0000 7f9269d84640 1 -- 172.21.15.139:0/478055029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926419fa10 con 0x7f92640ffba0 2024-09-16T01:40:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.972+0000 7f9269d84640 1 --2- 172.21.15.139:0/478055029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92640ffba0 0x7f9264197df0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f926000d770 tx=0x7f926000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.972+0000 7f9256ffd640 1 -- 172.21.15.139:0/478055029 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260017070 con 0x7f92640ffba0 2024-09-16T01:40:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.972+0000 7f9256ffd640 1 -- 172.21.15.139:0/478055029 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9260009c30 con 0x7f92640ffba0 2024-09-16T01:40:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.972+0000 7f9256ffd640 1 -- 172.21.15.139:0/478055029 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260003dd0 con 0x7f92640ffba0 2024-09-16T01:40:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.972+0000 7f926bfe8640 1 -- 172.21.15.139:0/478055029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926419fd00 con 0x7f92640ffba0 2024-09-16T01:40:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.972+0000 7f926bfe8640 1 -- 172.21.15.139:0/478055029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92641a01d0 con 0x7f92640ffba0 2024-09-16T01:40:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.973+0000 7f926bfe8640 1 -- 172.21.15.139:0/478055029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9264109070 con 0x7f92640ffba0 2024-09-16T01:40:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.974+0000 7f9256ffd640 1 -- 172.21.15.139:0/478055029 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f9260003fb0 con 0x7f92640ffba0 2024-09-16T01:40:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.975+0000 7f9256ffd640 1 --2- 172.21.15.139:0/478055029 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f92500a32f0 0x7f92500a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.975+0000 7f9269583640 1 --2- 172.21.15.139:0/478055029 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f92500a32f0 0x7f92500a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.975+0000 7f9256ffd640 1 -- 172.21.15.139:0/478055029 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f92600cf990 con 0x7f92640ffba0 2024-09-16T01:40:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.975+0000 7f9269583640 1 --2- 172.21.15.139:0/478055029 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f92500a32f0 0x7f92500a57b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9258005ab0 tx=0x7f9258000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:17.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:17.978+0000 7f9256ffd640 1 -- 172.21.15.139:0/478055029 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9260009de0 con 0x7f92640ffba0 2024-09-16T01:40:18.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:18.190+0000 7f926bfe8640 1 -- 172.21.15.139:0/478055029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f926402d970 con 0x7f92640ffba0 2024-09-16T01:40:18.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:18.592+0000 7f9256ffd640 1 -- 172.21.15.139:0/478055029 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f926009bc90 con 0x7f92640ffba0 2024-09-16T01:40:18.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:18.594+0000 7f926bfe8640 1 -- 172.21.15.139:0/478055029 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f92500a32f0 msgr2=0x7f92500a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:18.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:18.594+0000 7f926bfe8640 1 --2- 172.21.15.139:0/478055029 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f92500a32f0 0x7f92500a57b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9258005ab0 tx=0x7f9258000bc0 comp rx=0 tx=0).stop 2024-09-16T01:40:18.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:18.594+0000 7f926bfe8640 1 -- 172.21.15.139:0/478055029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92640ffba0 msgr2=0x7f9264197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:18.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:18.594+0000 7f926bfe8640 1 --2- 172.21.15.139:0/478055029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92640ffba0 0x7f9264197df0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f926000d770 tx=0x7f926000dc30 comp rx=0 tx=0).stop 2024-09-16T01:40:18.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:18.595+0000 7f926bfe8640 1 -- 172.21.15.139:0/478055029 shutdown_connections 2024-09-16T01:40:18.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:18.595+0000 7f926bfe8640 1 --2- 172.21.15.139:0/478055029 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f92500a32f0 0x7f92500a57b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:18.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:18.595+0000 7f926bfe8640 1 --2- 172.21.15.139:0/478055029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9264103130 0x7f926419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:18.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:18.595+0000 7f926bfe8640 1 --2- 172.21.15.139:0/478055029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9264100560 0x7f9264198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:18.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:18.595+0000 7f926bfe8640 1 --2- 172.21.15.139:0/478055029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92640ffba0 0x7f9264197df0 secure :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f926000d770 tx=0x7f926000dc30 comp rx=0 tx=0).stop 2024-09-16T01:40:18.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:18.595+0000 7f926bfe8640 1 -- 172.21.15.139:0/478055029 >> 172.21.15.139:0/478055029 conn(0x7f92640fb970 msgr2=0x7f9264106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:18.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:18.595+0000 7f926bfe8640 1 -- 172.21.15.139:0/478055029 shutdown_connections 2024-09-16T01:40:18.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:18.597+0000 7f926bfe8640 1 -- 172.21.15.139:0/478055029 wait complete. 2024-09-16T01:40:18.613 INFO:teuthology.run_tasks:Running task kclient... 2024-09-16T01:40:18.629 INFO:tasks.kclient:Mounting kernel clients... 2024-09-16T01:40:18.630 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-16T01:40:18.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:40:18.630 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-16T01:40:18.645 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-16T01:40:18.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:40:18.646 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T01:40:18.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:40:18.712 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link delete ceph-brx 2024-09-16T01:40:18.836 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-16T01:40:18.836 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T01:40:18.836 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T01:40:18.836 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T01:40:18.858 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:18 smithi139 ceph-mon[31175]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T01:40:18.859 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:18 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/478055029' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-16T01:40:18.859 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:18 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: ignoring --setuser ceph since I am not root 2024-09-16T01:40:18.859 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:18 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: ignoring --setgroup ceph since I am not root 2024-09-16T01:40:18.859 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:18 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:18.748+0000 7fe25b1ee040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:40:18.859 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:18 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:18.809+0000 7fe25b1ee040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:40:18.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:18 smithi176 ceph-mon[35324]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T01:40:18.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:18 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/478055029' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-16T01:40:18.938 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:18 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: ignoring --setuser ceph since I am not root 2024-09-16T01:40:18.938 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:18 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: ignoring --setgroup ceph since I am not root 2024-09-16T01:40:18.938 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:18 smithi176 ceph-mgr[37132]: -- 172.21.15.176:0/1114507425 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cf0b610680 con 0x55cf0b5e5400 2024-09-16T01:40:18.938 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:18 smithi176 ceph-mgr[37132]: -- 172.21.15.176:0/1114507425 <== mon.0 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cf0b6070e0 con 0x55cf0b5e5400 2024-09-16T01:40:18.938 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:18 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:18.743+0000 7f08182ec040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:40:18.939 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:18 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:18.799+0000 7f08182ec040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:40:18.949 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:18 smithi164 ceph-mon[35121]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-16T01:40:18.949 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:18 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/478055029' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-16T01:40:18.950 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:18 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: ignoring --setuser ceph since I am not root 2024-09-16T01:40:18.950 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:18 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: ignoring --setgroup ceph since I am not root 2024-09-16T01:40:18.950 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:18 smithi164 ceph-mgr[36488]: -- 172.21.15.164:0/4121317186 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5647faa570e0 con 0x5647faa34800 2024-09-16T01:40:18.950 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:18 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:18.742+0000 7f7cfbbf8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T01:40:18.950 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:18 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:18.799+0000 7f7cfbbf8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T01:40:18.950 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:18 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:18.948+0000 7f7cfbbf8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:40:19.096 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:19.172 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:18 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:18.964+0000 7fe25b1ee040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:40:19.173 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:19 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:19.035+0000 7fe25b1ee040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:40:19.295 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:19 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:19.014+0000 7f7cfbbf8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:40:19.295 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:19 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:19.162+0000 7f7cfbbf8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:40:19.299 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:18 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:18.946+0000 7f08182ec040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T01:40:19.299 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:19 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:19.011+0000 7f08182ec040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T01:40:19.299 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:19 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:19.157+0000 7f08182ec040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:40:19.299 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:19 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:19.296+0000 7f08182ec040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:40:19.526 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:19 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:19.201+0000 7fe25b1ee040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:40:19.526 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:19 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:19.344+0000 7fe25b1ee040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:40:19.597 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:19 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:19.303+0000 7f7cfbbf8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T01:40:19.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.606+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2317762770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e3c102790 msgr2=0x7f6e3c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:19.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.606+0000 7f6e41ec8640 1 --2- 172.21.15.139:0/2317762770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e3c102790 0x7f6e3c1067e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009900 tx=0x7f6e2402a410 comp rx=0 tx=0).stop 2024-09-16T01:40:19.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.606+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2317762770 shutdown_connections 2024-09-16T01:40:19.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.606+0000 7f6e41ec8640 1 --2- 172.21.15.139:0/2317762770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e3c106f10 0x7f6e3c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:19.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.606+0000 7f6e41ec8640 1 --2- 172.21.15.139:0/2317762770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e3c102790 0x7f6e3c1067e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:19.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.606+0000 7f6e41ec8640 1 --2- 172.21.15.139:0/2317762770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e3c101de0 0x7f6e3c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:19.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.606+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2317762770 >> 172.21.15.139:0/2317762770 conn(0x7f6e3c0fd8c0 msgr2=0x7f6e3c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:19.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.606+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2317762770 shutdown_connections 2024-09-16T01:40:19.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.606+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2317762770 wait complete. 2024-09-16T01:40:19.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.607+0000 7f6e41ec8640 1 Processor -- start 2024-09-16T01:40:19.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.607+0000 7f6e41ec8640 1 -- start start 2024-09-16T01:40:19.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.607+0000 7f6e41ec8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e3c101de0 0x7f6e3c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:19.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.607+0000 7f6e41ec8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e3c102790 0x7f6e3c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:19.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.607+0000 7f6e41ec8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e3c106f10 0x7f6e3c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:19.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.607+0000 7f6e41ec8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c111f70 con 0x7f6e3c106f10 2024-09-16T01:40:19.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.607+0000 7f6e41ec8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c111df0 con 0x7f6e3c102790 2024-09-16T01:40:19.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.607+0000 7f6e41ec8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c1120f0 con 0x7f6e3c101de0 2024-09-16T01:40:19.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.607+0000 7f6e3bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e3c106f10 0x7f6e3c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:19.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.607+0000 7f6e3bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e3c106f10 0x7f6e3c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55618/0 (socket says 172.21.15.139:55618) 2024-09-16T01:40:19.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.607+0000 7f6e3bfff640 1 -- 172.21.15.139:0/2284645065 learned_addr learned my addr 172.21.15.139:0/2284645065 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:19.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.607+0000 7f6e3b7fe640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e3c101de0 0x7f6e3c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:19.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.607+0000 7f6e3affd640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e3c102790 0x7f6e3c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:19.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.608+0000 7f6e3b7fe640 1 -- 172.21.15.139:0/2284645065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e3c102790 msgr2=0x7f6e3c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:19.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.608+0000 7f6e3b7fe640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e3c102790 0x7f6e3c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:19.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.608+0000 7f6e3b7fe640 1 -- 172.21.15.139:0/2284645065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e3c106f10 msgr2=0x7f6e3c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:19.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.608+0000 7f6e3b7fe640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e3c106f10 0x7f6e3c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:19.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.608+0000 7f6e3b7fe640 1 -- 172.21.15.139:0/2284645065 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c19d400 con 0x7f6e3c101de0 2024-09-16T01:40:19.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.608+0000 7f6e3bfff640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e3c106f10 0x7f6e3c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:40:19.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.608+0000 7f6e3b7fe640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e3c101de0 0x7f6e3c198450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00e9f0 tx=0x7f6e2c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:19.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.608+0000 7f6e38ff9640 1 -- 172.21.15.139:0/2284645065 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c00ccc0 con 0x7f6e3c101de0 2024-09-16T01:40:19.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.608+0000 7f6e38ff9640 1 -- 172.21.15.139:0/2284645065 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2c005020 con 0x7f6e3c101de0 2024-09-16T01:40:19.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.608+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2284645065 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3c19d6f0 con 0x7f6e3c101de0 2024-09-16T01:40:19.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.608+0000 7f6e38ff9640 1 -- 172.21.15.139:0/2284645065 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0049e0 con 0x7f6e3c101de0 2024-09-16T01:40:19.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.608+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2284645065 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3c1a4fd0 con 0x7f6e3c101de0 2024-09-16T01:40:19.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.609+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2284645065 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e3c1021c0 con 0x7f6e3c101de0 2024-09-16T01:40:19.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.610+0000 7f6e38ff9640 1 -- 172.21.15.139:0/2284645065 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f6e2c005560 con 0x7f6e3c101de0 2024-09-16T01:40:19.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.612+0000 7f6e38ff9640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6e280a3290 0x7f6e280a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:19.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.612+0000 7f6e3affd640 1 -- 172.21.15.139:0/2284645065 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6e280a3290 msgr2=0x7f6e280a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/331514141 2024-09-16T01:40:19.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.612+0000 7f6e38ff9640 1 -- 172.21.15.139:0/2284645065 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f6e2c0cffc0 con 0x7f6e3c101de0 2024-09-16T01:40:19.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.612+0000 7f6e3affd640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6e280a3290 0x7f6e280a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:40:19.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.614+0000 7f6e38ff9640 1 -- 172.21.15.139:0/2284645065 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e2c0051d0 con 0x7f6e3c101de0 2024-09-16T01:40:19.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.813+0000 7f6e3affd640 1 -- 172.21.15.139:0/2284645065 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6e280a3290 msgr2=0x7f6e280a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/331514141 2024-09-16T01:40:19.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.813+0000 7f6e3affd640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6e280a3290 0x7f6e280a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:40:19.815 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:19 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/478055029' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-16T01:40:19.815 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:19 smithi139 ceph-mon[31175]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-16T01:40:19.815 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:19 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:19.671+0000 7fe25b1ee040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:40:19.815 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:19 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:19.734+0000 7fe25b1ee040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:40:19.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.823+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2284645065 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6e3c1997b0 con 0x7f6e3c101de0 2024-09-16T01:40:19.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.823+0000 7f6e38ff9640 1 -- 172.21.15.139:0/2284645065 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7f6e3c1997b0 con 0x7f6e3c101de0 2024-09-16T01:40:19.825 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-16T01:40:19.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.825+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2284645065 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6e280a3290 msgr2=0x7f6e280a5750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:40:19.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.825+0000 7f6e41ec8640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6e280a3290 0x7f6e280a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:19.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.825+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2284645065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e3c101de0 msgr2=0x7f6e3c198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:19.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.825+0000 7f6e41ec8640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e3c101de0 0x7f6e3c198450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00e9f0 tx=0x7f6e2c00eeb0 comp rx=0 tx=0).stop 2024-09-16T01:40:19.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.825+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2284645065 shutdown_connections 2024-09-16T01:40:19.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.825+0000 7f6e41ec8640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f6e280a3290 0x7f6e280a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:19.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.825+0000 7f6e41ec8640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e3c106f10 0x7f6e3c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:19.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.825+0000 7f6e41ec8640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e3c102790 0x7f6e3c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:19.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.825+0000 7f6e41ec8640 1 --2- 172.21.15.139:0/2284645065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e3c101de0 0x7f6e3c198450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:19.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.825+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2284645065 >> 172.21.15.139:0/2284645065 conn(0x7f6e3c0fd8c0 msgr2=0x7f6e3c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:19.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.826+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2284645065 shutdown_connections 2024-09-16T01:40:19.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:19.826+0000 7f6e41ec8640 1 -- 172.21.15.139:0/2284645065 wait complete. 2024-09-16T01:40:19.855 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:19 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/478055029' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-16T01:40:19.855 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:19 smithi176 ceph-mon[35324]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-16T01:40:19.856 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:19 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:19.607+0000 7f08182ec040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:40:19.856 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:19 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:19.668+0000 7f08182ec040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:40:19.869 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:19 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/478055029' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-16T01:40:19.869 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:19 smithi164 ceph-mon[35121]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-16T01:40:19.869 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:19 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:19.620+0000 7f7cfbbf8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T01:40:19.869 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:19 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:19.681+0000 7f7cfbbf8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T01:40:19.869 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:19 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:19.867+0000 7f7cfbbf8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:40:20.172 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:19 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:19.925+0000 7fe25b1ee040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:40:20.186 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:19 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:19.853+0000 7f08182ec040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T01:40:20.537 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T01:40:20.537 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T01:40:20.537 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-16T01:40:20.537 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T01:40:20.537 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T01:40:20.537 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T01:40:20.537 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T01:40:20.537 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T01:40:20.537 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T01:40:20.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:40:20.538 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-16T01:40:20.594 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T01:40:20.594 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T01:40:20.594 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T01:40:20.594 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-16T01:40:20.594 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-16T01:40:20.594 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-16T01:40:20.595 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T01:40:20.595 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-16T01:40:20.595 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-16T01:40:20.595 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T01:40:20.595 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-16T01:40:20.595 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-16T01:40:20.595 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T01:40:20.595 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-16T01:40:20.595 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T01:40:20.595 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-16T01:40:20.595 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T01:40:20.595 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42198sec preferred_lft 42198sec 2024-09-16T01:40:20.596 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link noprefixroute 2024-09-16T01:40:20.596 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-16T01:40:20.596 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T01:40:20.596 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T01:40:20.596 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T01:40:20.597 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add name ceph-brx type bridge 2024-09-16T01:40:20.597 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr flush dev ceph-brx 2024-09-16T01:40:20.597 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set ceph-brx up 2024-09-16T01:40:20.597 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T01:40:20.597 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T01:40:20.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T01:40:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:40:20.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:40:20.738 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T01:40:20.761 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:20 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2284645065' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T01:40:20.764 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-16T01:40:20.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:40:20.766 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-16T01:40:20.822 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-16T01:40:20.822 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T01:40:20.822 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T01:40:20.823 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T01:40:20.823 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T01:40:20.823 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T01:40:20.823 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T01:40:20.823 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T01:40:20.823 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T01:40:20.823 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T01:40:20.823 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T01:40:20.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T01:40:20.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:20 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2284645065' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T01:40:20.936 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:20 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:20.732+0000 7f08182ec040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:40:20.937 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:20 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:20.799+0000 7f08182ec040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:40:20.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:40:20.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:40:20.952 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-16T01:40:20.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:40:20.968 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list-id 2024-09-16T01:40:21.023 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T01:40:21.023 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T01:40:21.023 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T01:40:21.023 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-16T01:40:21.023 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T01:40:21.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:20 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2284645065' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T01:40:21.045 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:20 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:20.751+0000 7f7cfbbf8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:40:21.045 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:20 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:20.817+0000 7f7cfbbf8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:40:21.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T01:40:21.121 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:20 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:20.853+0000 7fe25b1ee040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T01:40:21.121 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:20 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:20.926+0000 7fe25b1ee040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T01:40:21.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:40:21.142 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-16T01:40:21.143 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T01:40:21.143 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T01:40:21.143 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-16T01:40:21.143 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T01:40:21.143 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-16T01:40:21.143 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-16T01:40:21.143 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-16T01:40:21.143 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T01:40:21.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T01:40:21.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:40:21.260 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T01:40:21.260 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T01:40:21.261 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-16T01:40:21.261 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T01:40:21.261 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T01:40:21.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T01:40:21.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:40:21.346 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T01:40:21.346 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T01:40:21.361 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T01:40:21.362 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T01:40:21.362 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T01:40:21.417 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-16T01:40:21.417 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-16T01:40:21.417 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/525beaa2-f2b7-4f44-a32e-e6c7dd91c097 2024-09-16T01:40:21.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:40:21.417 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/525beaa2-f2b7-4f44-a32e-e6c7dd91c097 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-16T01:40:21.502 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-16T01:40:21.502 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-16T01:40:21.502 INFO:teuthology.orchestra.run.smithi139.stdout:invalid new device string format 2024-09-16T01:40:21.645 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-16T01:40:21.645 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-16T01:40:21.646 INFO:teuthology.orchestra.run.smithi139.stdout:invalid new device string format 2024-09-16T01:40:21.646 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.164:3300,172.21.15.176:3300" 2024-09-16T01:40:21.646 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.139:3300,172.21.15.164:3300,172.21.15.176:3300:/volumes/qa/sv_1/525beaa2-f2b7-4f44-a32e-e6c7dd91c097 2024-09-16T01:40:21.646 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=1d90dd3a-73cb-11ef-bceb-c7b262605968" will pass to kernel 2024-09-16T01:40:21.647 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T01:40:21.647 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-09-16T01:40:21.647 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T01:40:21.647 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T01:40:21.647 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T01:40:21.648 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T01:40:21.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:40:21.649 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T01:40:21.674 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T01:40:21.764 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:21 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:21.503+0000 7f08182ec040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:40:21.764 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:21 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:21.619+0000 7f08182ec040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:40:21.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T01:40:21.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:40:21.772 INFO:teuthology.orchestra.run.smithi139.stdout:client24938 2024-09-16T01:40:21.773 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:40:21.773 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T01:40:21.789 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T01:40:21.794 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:21.535+0000 7f7cfbbf8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:40:21.794 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:21.653+0000 7f7cfbbf8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:40:21.810 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:21 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:21.685+0000 7fe25b1ee040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T01:40:21.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T01:40:21.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:40:21.866 INFO:teuthology.orchestra.run.smithi139.stdout:client24938 2024-09-16T01:40:21.866 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T01:40:21.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T01:40:21.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:40:21.923 INFO:teuthology.orchestra.run.smithi139.stdout:client24938 2024-09-16T01:40:21.923 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968.client24938/status') 2024-09-16T01:40:21.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T01:40:21.982 INFO:teuthology.orchestra.run.smithi139.stdout:instance: client.24938 (3)172.21.15.139:0/1792605146 2024-09-16T01:40:21.982 INFO:teuthology.orchestra.run.smithi139.stdout:blocklisted: false 2024-09-16T01:40:21.982 INFO:teuthology.orchestra.run.smithi139.stderr:0+1 records in 2024-09-16T01:40:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:0+1 records out 2024-09-16T01:40:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:72 bytes copied, 2.3694e-05 s, 3.0 MB/s 2024-09-16T01:40:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T01:40:21.990 INFO:teuthology.run_tasks:Running task exec... 2024-09-16T01:40:21.999 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T01:40:21.999 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T01:40:21.999 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-16T01:40:22.040 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:21 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:21.763+0000 7f08182ec040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:40:22.040 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:21 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:21.824+0000 7f08182ec040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:40:22.040 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:21 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:21.971+0000 7f08182ec040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:40:22.040 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:22 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:22.039+0000 7f08182ec040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:40:22.080 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:21.798+0000 7f7cfbbf8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:40:22.080 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:21.860+0000 7f7cfbbf8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:40:22.080 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:22.009+0000 7f7cfbbf8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:40:22.080 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:22.078+0000 7f7cfbbf8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:40:22.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.099+0000 7ff48b8a5640 1 -- 172.21.15.139:0/1968301207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4841025f0 msgr2=0x7ff484108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:22.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.099+0000 7ff48b8a5640 1 --2- 172.21.15.139:0/1968301207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4841025f0 0x7ff484108e80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009ca0 tx=0x7ff47c02a8c0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.099+0000 7ff48b8a5640 1 -- 172.21.15.139:0/1968301207 shutdown_connections 2024-09-16T01:40:22.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.099+0000 7ff48b8a5640 1 --2- 172.21.15.139:0/1968301207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4841025f0 0x7ff484108e80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.099+0000 7ff48b8a5640 1 --2- 172.21.15.139:0/1968301207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff484101c30 0x7ff4841020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.099+0000 7ff48b8a5640 1 --2- 172.21.15.139:0/1968301207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff484100a40 0x7ff484100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.099+0000 7ff48b8a5640 1 -- 172.21.15.139:0/1968301207 >> 172.21.15.139:0/1968301207 conn(0x7ff4840fc1e0 msgr2=0x7ff4840fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.099+0000 7ff48b8a5640 1 -- 172.21.15.139:0/1968301207 shutdown_connections 2024-09-16T01:40:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.099+0000 7ff48b8a5640 1 -- 172.21.15.139:0/1968301207 wait complete. 2024-09-16T01:40:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.100+0000 7ff48b8a5640 1 Processor -- start 2024-09-16T01:40:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.100+0000 7ff48b8a5640 1 -- start start 2024-09-16T01:40:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.100+0000 7ff48b8a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff484100a40 0x7ff48410b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.100+0000 7ff48b8a5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff484101c30 0x7ff484109a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.100+0000 7ff48b8a5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4841025f0 0x7ff484109f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.100+0000 7ff48b8a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff48410d7d0 con 0x7ff484100a40 2024-09-16T01:40:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.100+0000 7ff48b8a5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff48410d650 con 0x7ff484101c30 2024-09-16T01:40:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.100+0000 7ff48b8a5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff48410d950 con 0x7ff4841025f0 2024-09-16T01:40:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.100+0000 7ff489641640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff484100a40 0x7ff48410b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.100+0000 7ff489641640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff484100a40 0x7ff48410b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55636/0 (socket says 172.21.15.139:55636) 2024-09-16T01:40:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.100+0000 7ff489641640 1 -- 172.21.15.139:0/2964492194 learned_addr learned my addr 172.21.15.139:0/2964492194 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.100+0000 7ff488e40640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff484101c30 0x7ff484109a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.100+0000 7ff489e42640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4841025f0 0x7ff484109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.101+0000 7ff489641640 1 -- 172.21.15.139:0/2964492194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4841025f0 msgr2=0x7ff484109f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.101+0000 7ff489641640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4841025f0 0x7ff484109f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.101+0000 7ff489641640 1 -- 172.21.15.139:0/2964492194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff484101c30 msgr2=0x7ff484109a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.101+0000 7ff489641640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff484101c30 0x7ff484109a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.101+0000 7ff489641640 1 -- 172.21.15.139:0/2964492194 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48410a7d0 con 0x7ff484100a40 2024-09-16T01:40:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.101+0000 7ff489e42640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4841025f0 0x7ff484109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.101+0000 7ff489641640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff484100a40 0x7ff48410b3c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff48000e9c0 tx=0x7ff48000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.101+0000 7ff47a7fc640 1 -- 172.21.15.139:0/2964492194 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48000cde0 con 0x7ff484100a40 2024-09-16T01:40:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.101+0000 7ff47a7fc640 1 -- 172.21.15.139:0/2964492194 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff480010910 con 0x7ff484100a40 2024-09-16T01:40:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.101+0000 7ff48b8a5640 1 -- 172.21.15.139:0/2964492194 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4841a1ec0 con 0x7ff484100a40 2024-09-16T01:40:22.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.101+0000 7ff47a7fc640 1 -- 172.21.15.139:0/2964492194 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800189c0 con 0x7ff484100a40 2024-09-16T01:40:22.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.101+0000 7ff48b8a5640 1 -- 172.21.15.139:0/2964492194 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4841a22d0 con 0x7ff484100a40 2024-09-16T01:40:22.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.102+0000 7ff48b8a5640 1 -- 172.21.15.139:0/2964492194 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff484107e30 con 0x7ff484100a40 2024-09-16T01:40:22.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.103+0000 7ff47a7fc640 1 -- 172.21.15.139:0/2964492194 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7ff480010ac0 con 0x7ff484100a40 2024-09-16T01:40:22.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.104+0000 7ff47a7fc640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff4700a3340 0x7ff4700a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:22.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.104+0000 7ff488e40640 1 -- 172.21.15.139:0/2964492194 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff4700a3340 msgr2=0x7ff4700a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/331514141 2024-09-16T01:40:22.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.104+0000 7ff47a7fc640 1 -- 172.21.15.139:0/2964492194 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7ff4800d02a0 con 0x7ff484100a40 2024-09-16T01:40:22.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.104+0000 7ff488e40640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff4700a3340 0x7ff4700a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:40:22.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.108+0000 7ff47a7fc640 1 -- 172.21.15.139:0/2964492194 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4800541b0 con 0x7ff484100a40 2024-09-16T01:40:22.172 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:21 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:21.807+0000 7fe25b1ee040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T01:40:22.173 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:21 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:21.960+0000 7fe25b1ee040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T01:40:22.173 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:22 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:22.026+0000 7fe25b1ee040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T01:40:22.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.305+0000 7ff488e40640 1 -- 172.21.15.139:0/2964492194 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff4700a3340 msgr2=0x7ff4700a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/331514141 2024-09-16T01:40:22.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.305+0000 7ff488e40640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff4700a3340 0x7ff4700a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:40:22.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.315+0000 7ff48b8a5640 1 -- 172.21.15.139:0/2964492194 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7ff484062ef0 con 0x7ff484100a40 2024-09-16T01:40:22.323 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:22 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:22.098+0000 7f08182ec040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:40:22.323 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:22 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:22.257+0000 7f08182ec040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:40:22.362 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:22.139+0000 7f7cfbbf8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:40:22.362 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:22.296+0000 7f7cfbbf8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:40:22.362 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:22.360+0000 7f7cfbbf8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:40:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.367+0000 7ff47a7fc640 1 -- 172.21.15.139:0/2964492194 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7ff48009cbf0 con 0x7ff484100a40 2024-09-16T01:40:22.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.370+0000 7ff48b8a5640 1 -- 172.21.15.139:0/2964492194 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff4700a3340 msgr2=0x7ff4700a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:40:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.370+0000 7ff48b8a5640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff4700a3340 0x7ff4700a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.370+0000 7ff48b8a5640 1 -- 172.21.15.139:0/2964492194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff484100a40 msgr2=0x7ff48410b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.370+0000 7ff48b8a5640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff484100a40 0x7ff48410b3c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff48000e9c0 tx=0x7ff48000ee80 comp rx=0 tx=0).stop 2024-09-16T01:40:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.370+0000 7ff48b8a5640 1 -- 172.21.15.139:0/2964492194 shutdown_connections 2024-09-16T01:40:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.370+0000 7ff48b8a5640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7ff4700a3340 0x7ff4700a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.371+0000 7ff48b8a5640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4841025f0 0x7ff484109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.371+0000 7ff48b8a5640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff484101c30 0x7ff484109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.371+0000 7ff48b8a5640 1 --2- 172.21.15.139:0/2964492194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff484100a40 0x7ff48410b3c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.371+0000 7ff48b8a5640 1 -- 172.21.15.139:0/2964492194 >> 172.21.15.139:0/2964492194 conn(0x7ff4840fc1e0 msgr2=0x7ff484105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.371+0000 7ff48b8a5640 1 -- 172.21.15.139:0/2964492194 shutdown_connections 2024-09-16T01:40:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.371+0000 7ff48b8a5640 1 -- 172.21.15.139:0/2964492194 wait complete. 2024-09-16T01:40:22.385 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-16T01:40:22.397 INFO:teuthology.run_tasks:Running task exec... 2024-09-16T01:40:22.406 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T01:40:22.406 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T01:40:22.406 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-16T01:40:22.482 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:22 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2964492194' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-16T01:40:22.482 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:22 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:22.181+0000 7fe25b1ee040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T01:40:22.482 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:22 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:22.251+0000 7fe25b1ee040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T01:40:22.483 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:22 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:22.312+0000 7fe25b1ee040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T01:40:22.483 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:22 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:22.480+0000 7fe25b1ee040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T01:40:22.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.508+0000 7f41cb711640 1 -- 172.21.15.139:0/426495277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c41037a0 msgr2=0x7f41c410a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.508+0000 7f41cb711640 1 --2- 172.21.15.139:0/426495277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c41037a0 0x7f41c410a030 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009ca0 tx=0x7f41bc02a8c0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.508+0000 7f41cb711640 1 -- 172.21.15.139:0/426495277 shutdown_connections 2024-09-16T01:40:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.508+0000 7f41cb711640 1 --2- 172.21.15.139:0/426495277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c41037a0 0x7f41c410a030 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.508+0000 7f41cb711640 1 --2- 172.21.15.139:0/426495277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41c4102de0 0x7f41c4103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.508+0000 7f41cb711640 1 --2- 172.21.15.139:0/426495277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41c40ffe10 0x7f41c4100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.508+0000 7f41cb711640 1 -- 172.21.15.139:0/426495277 >> 172.21.15.139:0/426495277 conn(0x7f41c40fb930 msgr2=0x7f41c40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.508+0000 7f41cb711640 1 -- 172.21.15.139:0/426495277 shutdown_connections 2024-09-16T01:40:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.508+0000 7f41cb711640 1 -- 172.21.15.139:0/426495277 wait complete. 2024-09-16T01:40:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.508+0000 7f41cb711640 1 Processor -- start 2024-09-16T01:40:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.508+0000 7f41cb711640 1 -- start start 2024-09-16T01:40:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41cb711640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41c40ffe10 0x7f41c4197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41cb711640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41c4102de0 0x7f41c4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41cb711640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c41037a0 0x7f41c419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41cb711640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f41c410d7e0 con 0x7f41c41037a0 2024-09-16T01:40:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41cb711640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f41c410d660 con 0x7f41c40ffe10 2024-09-16T01:40:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41cb711640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f41c410d960 con 0x7f41c4102de0 2024-09-16T01:40:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41c9cae640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c41037a0 0x7f41c419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41c9cae640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c41037a0 0x7f41c419f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50392/0 (socket says 172.21.15.139:50392) 2024-09-16T01:40:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41c9cae640 1 -- 172.21.15.139:0/2988861401 learned_addr learned my addr 172.21.15.139:0/2988861401 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41c94ad640 1 --2- 172.21.15.139:0/2988861401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41c40ffe10 0x7f41c4197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41c8cac640 1 --2- 172.21.15.139:0/2988861401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41c4102de0 0x7f41c4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41c9cae640 1 -- 172.21.15.139:0/2988861401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41c4102de0 msgr2=0x7f41c4198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41c9cae640 1 --2- 172.21.15.139:0/2988861401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41c4102de0 0x7f41c4198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41c9cae640 1 -- 172.21.15.139:0/2988861401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41c40ffe10 msgr2=0x7f41c4197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41c9cae640 1 --2- 172.21.15.139:0/2988861401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41c40ffe10 0x7f41c4197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41c9cae640 1 -- 172.21.15.139:0/2988861401 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c419f9d0 con 0x7f41c41037a0 2024-09-16T01:40:22.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41c94ad640 1 --2- 172.21.15.139:0/2988861401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41c40ffe10 0x7f41c4197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:40:22.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41c9cae640 1 --2- 172.21.15.139:0/2988861401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c41037a0 0x7f41c419f330 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f41bc031ae0 tx=0x7f41bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:22.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41ba7fc640 1 -- 172.21.15.139:0/2988861401 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc003b70 con 0x7f41c41037a0 2024-09-16T01:40:22.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41ba7fc640 1 -- 172.21.15.139:0/2988861401 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41bc003d10 con 0x7f41c41037a0 2024-09-16T01:40:22.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41cb711640 1 -- 172.21.15.139:0/2988861401 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c419fc60 con 0x7f41c41037a0 2024-09-16T01:40:22.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41ba7fc640 1 -- 172.21.15.139:0/2988861401 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc02fc10 con 0x7f41c41037a0 2024-09-16T01:40:22.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.509+0000 7f41cb711640 1 -- 172.21.15.139:0/2988861401 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c41a01b0 con 0x7f41c41037a0 2024-09-16T01:40:22.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.510+0000 7f41cb711640 1 -- 172.21.15.139:0/2988861401 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c4108f10 con 0x7f41c41037a0 2024-09-16T01:40:22.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.512+0000 7f41ba7fc640 1 -- 172.21.15.139:0/2988861401 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f41bc004b40 con 0x7f41c41037a0 2024-09-16T01:40:22.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.513+0000 7f41ba7fc640 1 --2- 172.21.15.139:0/2988861401 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41b00a3340 0x7f41b00a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:22.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.513+0000 7f41c94ad640 1 -- 172.21.15.139:0/2988861401 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41b00a3340 msgr2=0x7f41b00a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/331514141 2024-09-16T01:40:22.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.513+0000 7f41ba7fc640 1 -- 172.21.15.139:0/2988861401 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f41bc0f1060 con 0x7f41c41037a0 2024-09-16T01:40:22.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.513+0000 7f41c94ad640 1 --2- 172.21.15.139:0/2988861401 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41b00a3340 0x7f41b00a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:40:22.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.516+0000 7f41ba7fc640 1 -- 172.21.15.139:0/2988861401 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41bc0bd950 con 0x7f41c41037a0 2024-09-16T01:40:22.595 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:22 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2964492194' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-16T01:40:22.595 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:22 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:22.322+0000 7f08182ec040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:40:22.595 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:22 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:22.390+0000 7f08182ec040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:40:22.595 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:22 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:22.473+0000 7f08182ec040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:40:22.595 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:22 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:22.533+0000 7f08182ec040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:40:22.595 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:22 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:22.593+0000 7f08182ec040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:40:22.633 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:22 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2964492194' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-16T01:40:22.633 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:22.428+0000 7f7cfbbf8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:40:22.633 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:22.512+0000 7f7cfbbf8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:40:22.633 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:22.572+0000 7f7cfbbf8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:40:22.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.688+0000 7f41cb711640 1 -- 172.21.15.139:0/2988861401 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f41c402d7c0 con 0x7f41c41037a0 2024-09-16T01:40:22.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.692+0000 7f41ba7fc640 1 -- 172.21.15.139:0/2988861401 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v31)=0 v31) ==== 129+0+0 (secure 0 0 0) 0x7f41bc0bd250 con 0x7f41c41037a0 2024-09-16T01:40:22.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.695+0000 7f41cb711640 1 -- 172.21.15.139:0/2988861401 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41b00a3340 msgr2=0x7f41b00a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:40:22.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.695+0000 7f41cb711640 1 --2- 172.21.15.139:0/2988861401 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41b00a3340 0x7f41b00a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.695+0000 7f41cb711640 1 -- 172.21.15.139:0/2988861401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c41037a0 msgr2=0x7f41c419f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:22.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.695+0000 7f41cb711640 1 --2- 172.21.15.139:0/2988861401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c41037a0 0x7f41c419f330 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f41bc031ae0 tx=0x7f41bc004ab0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.695+0000 7f41cb711640 1 -- 172.21.15.139:0/2988861401 shutdown_connections 2024-09-16T01:40:22.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.695+0000 7f41cb711640 1 --2- 172.21.15.139:0/2988861401 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f41b00a3340 0x7f41b00a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.695+0000 7f41cb711640 1 --2- 172.21.15.139:0/2988861401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c41037a0 0x7f41c419f330 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.695+0000 7f41cb711640 1 --2- 172.21.15.139:0/2988861401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41c4102de0 0x7f41c4198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.695+0000 7f41cb711640 1 --2- 172.21.15.139:0/2988861401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41c40ffe10 0x7f41c4197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.695+0000 7f41cb711640 1 -- 172.21.15.139:0/2988861401 >> 172.21.15.139:0/2988861401 conn(0x7f41c40fb930 msgr2=0x7f41c41060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:22.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.695+0000 7f41cb711640 1 -- 172.21.15.139:0/2988861401 shutdown_connections 2024-09-16T01:40:22.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.695+0000 7f41cb711640 1 -- 172.21.15.139:0/2988861401 wait complete. 2024-09-16T01:40:22.708 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-16T01:40:22.772 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:22 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:22.551+0000 7fe25b1ee040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T01:40:22.772 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:22 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:22.619+0000 7fe25b1ee040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T01:40:22.772 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:22 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:22.706+0000 7fe25b1ee040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T01:40:22.773 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:22 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:22.770+0000 7fe25b1ee040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T01:40:22.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.809+0000 7f772a354640 1 -- 172.21.15.139:0/1891327743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7724101b60 msgr2=0x7f7724101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:22.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.809+0000 7f772a354640 1 --2- 172.21.15.139:0/1891327743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7724101b60 0x7f7724101f60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7714009900 tx=0x7f771402a410 comp rx=0 tx=0).stop 2024-09-16T01:40:22.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.809+0000 7f772a354640 1 -- 172.21.15.139:0/1891327743 shutdown_connections 2024-09-16T01:40:22.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.809+0000 7f772a354640 1 --2- 172.21.15.139:0/1891327743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7724103710 0x7f7724109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.809+0000 7f772a354640 1 --2- 172.21.15.139:0/1891327743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7724102d50 0x7f77241031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.809+0000 7f772a354640 1 --2- 172.21.15.139:0/1891327743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7724101b60 0x7f7724101f60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.809+0000 7f772a354640 1 -- 172.21.15.139:0/1891327743 >> 172.21.15.139:0/1891327743 conn(0x7f77240fd2c0 msgr2=0x7f77240ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:22.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.809+0000 7f772a354640 1 -- 172.21.15.139:0/1891327743 shutdown_connections 2024-09-16T01:40:22.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.809+0000 7f772a354640 1 -- 172.21.15.139:0/1891327743 wait complete. 2024-09-16T01:40:22.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f772a354640 1 Processor -- start 2024-09-16T01:40:22.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f772a354640 1 -- start start 2024-09-16T01:40:22.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f772a354640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7724101b60 0x7f7724197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:22.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f772a354640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7724102d50 0x7f77241983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:22.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f772a354640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7724103710 0x7f772419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:22.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f772a354640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f772410d7e0 con 0x7f7724101b60 2024-09-16T01:40:22.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f772a354640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f772410d660 con 0x7f7724102d50 2024-09-16T01:40:22.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f772a354640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f772410d960 con 0x7f7724103710 2024-09-16T01:40:22.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f7723fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7724101b60 0x7f7724197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:22.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f7723fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7724101b60 0x7f7724197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50402/0 (socket says 172.21.15.139:50402) 2024-09-16T01:40:22.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f7723fff640 1 -- 172.21.15.139:0/674871345 learned_addr learned my addr 172.21.15.139:0/674871345 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:22.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f77237fe640 1 --2- 172.21.15.139:0/674871345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7724102d50 0x7f77241983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:22.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f77288f1640 1 --2- 172.21.15.139:0/674871345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7724103710 0x7f772419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:22.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f7723fff640 1 -- 172.21.15.139:0/674871345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7724103710 msgr2=0x7f772419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:22.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f7723fff640 1 --2- 172.21.15.139:0/674871345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7724103710 0x7f772419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f7723fff640 1 -- 172.21.15.139:0/674871345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7724102d50 msgr2=0x7f77241983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:22.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f7723fff640 1 --2- 172.21.15.139:0/674871345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7724102d50 0x7f77241983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:22.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f7723fff640 1 -- 172.21.15.139:0/674871345 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772419fa90 con 0x7f7724101b60 2024-09-16T01:40:22.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.810+0000 7f77237fe640 1 --2- 172.21.15.139:0/674871345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7724102d50 0x7f77241983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:40:22.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.811+0000 7f7723fff640 1 --2- 172.21.15.139:0/674871345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7724101b60 0x7f7724197e70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7714006010 tx=0x7f7714004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:22.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.811+0000 7f77217fa640 1 -- 172.21.15.139:0/674871345 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771403f030 con 0x7f7724101b60 2024-09-16T01:40:22.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.811+0000 7f77217fa640 1 -- 172.21.15.139:0/674871345 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7714031a20 con 0x7f7724101b60 2024-09-16T01:40:22.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.811+0000 7f772a354640 1 -- 172.21.15.139:0/674871345 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772419fd20 con 0x7f7724101b60 2024-09-16T01:40:22.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.811+0000 7f77217fa640 1 -- 172.21.15.139:0/674871345 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7714035070 con 0x7f7724101b60 2024-09-16T01:40:22.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.811+0000 7f772a354640 1 -- 172.21.15.139:0/674871345 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77241a01c0 con 0x7f7724101b60 2024-09-16T01:40:22.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.812+0000 7f772a354640 1 -- 172.21.15.139:0/674871345 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7724108f40 con 0x7f7724101b60 2024-09-16T01:40:22.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.813+0000 7f77217fa640 1 -- 172.21.15.139:0/674871345 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f7714042030 con 0x7f7724101b60 2024-09-16T01:40:22.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.814+0000 7f77217fa640 1 --2- 172.21.15.139:0/674871345 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f770c0a3340 0x7f770c0a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:22.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.814+0000 7f77237fe640 1 -- 172.21.15.139:0/674871345 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f770c0a3340 msgr2=0x7f770c0a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/331514141 2024-09-16T01:40:22.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.814+0000 7f77217fa640 1 -- 172.21.15.139:0/674871345 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f77140f0360 con 0x7f7724101b60 2024-09-16T01:40:22.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.814+0000 7f77237fe640 1 --2- 172.21.15.139:0/674871345 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f770c0a3340 0x7f770c0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:40:22.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.821+0000 7f77217fa640 1 -- 172.21.15.139:0/674871345 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77140bccb0 con 0x7f7724101b60 2024-09-16T01:40:22.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.991+0000 7f772a354640 1 -- 172.21.15.139:0/674871345 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f772402d7c0 con 0x7f7724101b60 2024-09-16T01:40:22.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.995+0000 7f77217fa640 1 -- 172.21.15.139:0/674871345 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v32)=0 v32) ==== 138+0+0 (secure 0 0 0) 0x7f77140bc5b0 con 0x7f7724101b60 2024-09-16T01:40:22.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.997+0000 7f772a354640 1 -- 172.21.15.139:0/674871345 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f770c0a3340 msgr2=0x7f770c0a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:40:23.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.997+0000 7f772a354640 1 --2- 172.21.15.139:0/674871345 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f770c0a3340 0x7f770c0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.997+0000 7f772a354640 1 -- 172.21.15.139:0/674871345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7724101b60 msgr2=0x7f7724197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:23.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.997+0000 7f772a354640 1 --2- 172.21.15.139:0/674871345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7724101b60 0x7f7724197e70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7714006010 tx=0x7f7714004270 comp rx=0 tx=0).stop 2024-09-16T01:40:23.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.997+0000 7f772a354640 1 -- 172.21.15.139:0/674871345 shutdown_connections 2024-09-16T01:40:23.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.997+0000 7f772a354640 1 --2- 172.21.15.139:0/674871345 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f770c0a3340 0x7f770c0a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.998+0000 7f772a354640 1 --2- 172.21.15.139:0/674871345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7724103710 0x7f772419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.998+0000 7f772a354640 1 --2- 172.21.15.139:0/674871345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7724102d50 0x7f77241983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.998+0000 7f772a354640 1 --2- 172.21.15.139:0/674871345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7724101b60 0x7f7724197e70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.998+0000 7f772a354640 1 -- 172.21.15.139:0/674871345 >> 172.21.15.139:0/674871345 conn(0x7f77240fd2c0 msgr2=0x7f7724106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:23.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.998+0000 7f772a354640 1 -- 172.21.15.139:0/674871345 shutdown_connections 2024-09-16T01:40:23.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:22.998+0000 7f772a354640 1 -- 172.21.15.139:0/674871345 wait complete. 2024-09-16T01:40:23.010 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-16T01:40:23.032 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:22 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:22.837+0000 7fe25b1ee040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:40:23.045 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:22.631+0000 7f7cfbbf8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T01:40:23.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.112+0000 7f8fa35bd640 1 -- 172.21.15.139:0/4106266118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c100a40 msgr2=0x7f8f9c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:23.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.112+0000 7f8fa35bd640 1 --2- 172.21.15.139:0/4106266118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c100a40 0x7f8f9c100e40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009900 tx=0x7f8f9802a410 comp rx=0 tx=0).stop 2024-09-16T01:40:23.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.112+0000 7f8fa35bd640 1 -- 172.21.15.139:0/4106266118 shutdown_connections 2024-09-16T01:40:23.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.112+0000 7f8fa35bd640 1 --2- 172.21.15.139:0/4106266118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f9c1025f0 0x7f8f9c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.112+0000 7f8fa35bd640 1 --2- 172.21.15.139:0/4106266118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f9c101c30 0x7f8f9c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.112+0000 7f8fa35bd640 1 --2- 172.21.15.139:0/4106266118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c100a40 0x7f8f9c100e40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.112+0000 7f8fa35bd640 1 -- 172.21.15.139:0/4106266118 >> 172.21.15.139:0/4106266118 conn(0x7f8f9c0fc1e0 msgr2=0x7f8f9c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:23.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.112+0000 7f8fa35bd640 1 -- 172.21.15.139:0/4106266118 shutdown_connections 2024-09-16T01:40:23.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.112+0000 7f8fa35bd640 1 -- 172.21.15.139:0/4106266118 wait complete. 2024-09-16T01:40:23.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.112+0000 7f8fa35bd640 1 Processor -- start 2024-09-16T01:40:23.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.112+0000 7f8fa35bd640 1 -- start start 2024-09-16T01:40:23.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa35bd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f9c100a40 0x7f8f9c196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:23.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa35bd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f9c101c30 0x7f8f9c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:23.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa35bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c1025f0 0x7f8f9c19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:23.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa35bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c10c6a0 con 0x7f8f9c1025f0 2024-09-16T01:40:23.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa35bd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c10c520 con 0x7f8f9c101c30 2024-09-16T01:40:23.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa35bd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c10c820 con 0x7f8f9c100a40 2024-09-16T01:40:23.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa1b5a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c1025f0 0x7f8f9c19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:23.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa1b5a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c1025f0 0x7f8f9c19e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50424/0 (socket says 172.21.15.139:50424) 2024-09-16T01:40:23.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa1b5a640 1 -- 172.21.15.139:0/2948163527 learned_addr learned my addr 172.21.15.139:0/2948163527 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:23.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa1359640 1 --2- 172.21.15.139:0/2948163527 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f9c100a40 0x7f8f9c196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:23.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa0b58640 1 --2- 172.21.15.139:0/2948163527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f9c101c30 0x7f8f9c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:23.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa1b5a640 1 -- 172.21.15.139:0/2948163527 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f9c100a40 msgr2=0x7f8f9c196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:23.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa1b5a640 1 --2- 172.21.15.139:0/2948163527 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f9c100a40 0x7f8f9c196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa1b5a640 1 -- 172.21.15.139:0/2948163527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f9c101c30 msgr2=0x7f8f9c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:23.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa1b5a640 1 --2- 172.21.15.139:0/2948163527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f9c101c30 0x7f8f9c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa1b5a640 1 -- 172.21.15.139:0/2948163527 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9c19e920 con 0x7f8f9c1025f0 2024-09-16T01:40:23.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa1b5a640 1 --2- 172.21.15.139:0/2948163527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c1025f0 0x7f8f9c19e280 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400efd0 tx=0x7f8f9400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:23.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8f927fc640 1 -- 172.21.15.139:0/2948163527 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94003b70 con 0x7f8f9c1025f0 2024-09-16T01:40:23.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa35bd640 1 -- 172.21.15.139:0/2948163527 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9c19ec10 con 0x7f8f9c1025f0 2024-09-16T01:40:23.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8f927fc640 1 -- 172.21.15.139:0/2948163527 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f94003d10 con 0x7f8f9c1025f0 2024-09-16T01:40:23.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8fa35bd640 1 -- 172.21.15.139:0/2948163527 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9c19f130 con 0x7f8f9c1025f0 2024-09-16T01:40:23.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.113+0000 7f8f927fc640 1 -- 172.21.15.139:0/2948163527 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94010990 con 0x7f8f9c1025f0 2024-09-16T01:40:23.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.115+0000 7f8fa35bd640 1 -- 172.21.15.139:0/2948163527 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f9c107db0 con 0x7f8f9c1025f0 2024-09-16T01:40:23.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.116+0000 7f8f927fc640 1 -- 172.21.15.139:0/2948163527 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f8f94004000 con 0x7f8f9c1025f0 2024-09-16T01:40:23.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.116+0000 7f8f927fc640 1 --2- 172.21.15.139:0/2948163527 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8f880a3340 0x7f8f880a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:23.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.116+0000 7f8fa1359640 1 -- 172.21.15.139:0/2948163527 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8f880a3340 msgr2=0x7f8f880a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/331514141 2024-09-16T01:40:23.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.116+0000 7f8f927fc640 1 -- 172.21.15.139:0/2948163527 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8f940d05b0 con 0x7f8f9c1025f0 2024-09-16T01:40:23.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.116+0000 7f8fa1359640 1 --2- 172.21.15.139:0/2948163527 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8f880a3340 0x7f8f880a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:40:23.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.120+0000 7f8f927fc640 1 -- 172.21.15.139:0/2948163527 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f9409cf00 con 0x7f8f9c1025f0 2024-09-16T01:40:23.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.294+0000 7f8fa35bd640 1 -- 172.21.15.139:0/2948163527 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f8f9c02d7c0 con 0x7f8f9c1025f0 2024-09-16T01:40:23.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.298+0000 7f8f927fc640 1 -- 172.21.15.139:0/2948163527 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v33)=0 v33) ==== 133+0+0 (secure 0 0 0) 0x7f8f9409c800 con 0x7f8f9c1025f0 2024-09-16T01:40:23.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.300+0000 7f8fa35bd640 1 -- 172.21.15.139:0/2948163527 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8f880a3340 msgr2=0x7f8f880a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:40:23.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.300+0000 7f8fa35bd640 1 --2- 172.21.15.139:0/2948163527 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8f880a3340 0x7f8f880a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.300+0000 7f8fa35bd640 1 -- 172.21.15.139:0/2948163527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c1025f0 msgr2=0x7f8f9c19e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:23.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.300+0000 7f8fa35bd640 1 --2- 172.21.15.139:0/2948163527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c1025f0 0x7f8f9c19e280 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400efd0 tx=0x7f8f9400c6a0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.300+0000 7f8fa35bd640 1 -- 172.21.15.139:0/2948163527 shutdown_connections 2024-09-16T01:40:23.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.300+0000 7f8fa35bd640 1 --2- 172.21.15.139:0/2948163527 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7f8f880a3340 0x7f8f880a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.300+0000 7f8fa35bd640 1 --2- 172.21.15.139:0/2948163527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c1025f0 0x7f8f9c19e280 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.300+0000 7f8fa35bd640 1 --2- 172.21.15.139:0/2948163527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f9c101c30 0x7f8f9c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.300+0000 7f8fa35bd640 1 --2- 172.21.15.139:0/2948163527 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f9c100a40 0x7f8f9c196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.300+0000 7f8fa35bd640 1 -- 172.21.15.139:0/2948163527 >> 172.21.15.139:0/2948163527 conn(0x7f8f9c0fc1e0 msgr2=0x7f8f9c104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:23.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.300+0000 7f8fa35bd640 1 -- 172.21.15.139:0/2948163527 shutdown_connections 2024-09-16T01:40:23.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.300+0000 7f8fa35bd640 1 -- 172.21.15.139:0/2948163527 wait complete. 2024-09-16T01:40:23.313 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-16T01:40:23.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.458+0000 7fe92016d640 1 -- 172.21.15.139:0/4125100182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9181037a0 msgr2=0x7fe91810a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:23.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.458+0000 7fe92016d640 1 --2- 172.21.15.139:0/4125100182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9181037a0 0x7fe91810a030 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe910009ca0 tx=0x7fe91002a8c0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.458+0000 7fe92016d640 1 -- 172.21.15.139:0/4125100182 shutdown_connections 2024-09-16T01:40:23.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.458+0000 7fe92016d640 1 --2- 172.21.15.139:0/4125100182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9181037a0 0x7fe91810a030 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.458+0000 7fe92016d640 1 --2- 172.21.15.139:0/4125100182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe918102de0 0x7fe918103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.458+0000 7fe92016d640 1 --2- 172.21.15.139:0/4125100182 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9180ffe10 0x7fe918100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.458+0000 7fe92016d640 1 -- 172.21.15.139:0/4125100182 >> 172.21.15.139:0/4125100182 conn(0x7fe9180fb930 msgr2=0x7fe9180fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.458+0000 7fe92016d640 1 -- 172.21.15.139:0/4125100182 shutdown_connections 2024-09-16T01:40:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.458+0000 7fe92016d640 1 -- 172.21.15.139:0/4125100182 wait complete. 2024-09-16T01:40:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.459+0000 7fe92016d640 1 Processor -- start 2024-09-16T01:40:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.459+0000 7fe92016d640 1 -- start start 2024-09-16T01:40:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.459+0000 7fe92016d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9180ffe10 0x7fe918197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.459+0000 7fe92016d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe918102de0 0x7fe918198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.459+0000 7fe92016d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9181037a0 0x7fe91819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.459+0000 7fe92016d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe91810d830 con 0x7fe9181037a0 2024-09-16T01:40:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.459+0000 7fe92016d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe91810d6b0 con 0x7fe9180ffe10 2024-09-16T01:40:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.459+0000 7fe92016d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe91810d9b0 con 0x7fe918102de0 2024-09-16T01:40:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.459+0000 7fe91e70a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9181037a0 0x7fe91819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:23.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.459+0000 7fe91df09640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9180ffe10 0x7fe918197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:23.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.459+0000 7fe91e70a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9181037a0 0x7fe91819f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50448/0 (socket says 172.21.15.139:50448) 2024-09-16T01:40:23.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.459+0000 7fe91e70a640 1 -- 172.21.15.139:0/3196101350 learned_addr learned my addr 172.21.15.139:0/3196101350 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:23.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.459+0000 7fe91d708640 1 --2- 172.21.15.139:0/3196101350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe918102de0 0x7fe918198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:23.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.460+0000 7fe91df09640 1 -- 172.21.15.139:0/3196101350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe918102de0 msgr2=0x7fe918198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:23.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.460+0000 7fe91df09640 1 --2- 172.21.15.139:0/3196101350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe918102de0 0x7fe918198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.460+0000 7fe91df09640 1 -- 172.21.15.139:0/3196101350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9181037a0 msgr2=0x7fe91819f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:23.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.460+0000 7fe91df09640 1 --2- 172.21.15.139:0/3196101350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9181037a0 0x7fe91819f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.460+0000 7fe91df09640 1 -- 172.21.15.139:0/3196101350 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91819fa10 con 0x7fe9180ffe10 2024-09-16T01:40:23.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.460+0000 7fe91df09640 1 --2- 172.21.15.139:0/3196101350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9180ffe10 0x7fe918197df0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe90800e9c0 tx=0x7fe90800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:23.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.460+0000 7fe90effd640 1 -- 172.21.15.139:0/3196101350 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90800ccc0 con 0x7fe9180ffe10 2024-09-16T01:40:23.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.460+0000 7fe92016d640 1 -- 172.21.15.139:0/3196101350 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91819fd00 con 0x7fe9180ffe10 2024-09-16T01:40:23.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.460+0000 7fe90effd640 1 -- 172.21.15.139:0/3196101350 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9080035f0 con 0x7fe9180ffe10 2024-09-16T01:40:23.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.460+0000 7fe90effd640 1 -- 172.21.15.139:0/3196101350 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90800ce60 con 0x7fe9180ffe10 2024-09-16T01:40:23.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.460+0000 7fe92016d640 1 -- 172.21.15.139:0/3196101350 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9181a01a0 con 0x7fe9180ffe10 2024-09-16T01:40:23.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.461+0000 7fe92016d640 1 -- 172.21.15.139:0/3196101350 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe918108f10 con 0x7fe9180ffe10 2024-09-16T01:40:23.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.462+0000 7fe90effd640 1 -- 172.21.15.139:0/3196101350 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7fe9080037a0 con 0x7fe9180ffe10 2024-09-16T01:40:23.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.463+0000 7fe90effd640 1 --2- 172.21.15.139:0/3196101350 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fe9040a3340 0x7fe9040a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:23.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.463+0000 7fe91d708640 1 -- 172.21.15.139:0/3196101350 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fe9040a3340 msgr2=0x7fe9040a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/331514141 2024-09-16T01:40:23.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.463+0000 7fe90effd640 1 -- 172.21.15.139:0/3196101350 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fe9080d0240 con 0x7fe9180ffe10 2024-09-16T01:40:23.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.463+0000 7fe91d708640 1 --2- 172.21.15.139:0/3196101350 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fe9040a3340 0x7fe9040a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T01:40:23.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.467+0000 7fe90effd640 1 -- 172.21.15.139:0/3196101350 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe90809cb90 con 0x7fe9180ffe10 2024-09-16T01:40:23.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.663+0000 7fe92016d640 1 -- 172.21.15.139:0/3196101350 --> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fe918062630 con 0x7fe9040a3340 2024-09-16T01:40:23.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.664+0000 7fe91d708640 1 -- 172.21.15.139:0/3196101350 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fe9040a3340 msgr2=0x7fe9040a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6800/331514141 2024-09-16T01:40:23.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.664+0000 7fe91d708640 1 --2- 172.21.15.139:0/3196101350 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fe9040a3340 0x7fe9040a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T01:40:23.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2964492194' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-16T01:40:23.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:40:23.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2988861401' entity='client.admin' 2024-09-16T01:40:23.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/674871345' entity='client.admin' 2024-09-16T01:40:23.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2948163527' entity='client.admin' 2024-09-16T01:40:23.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:23 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2964492194' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-16T01:40:23.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:23 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:40:23.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:23 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2988861401' entity='client.admin' 2024-09-16T01:40:23.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:23 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/674871345' entity='client.admin' 2024-09-16T01:40:23.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:23 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2948163527' entity='client.admin' 2024-09-16T01:40:23.687 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 01:40:23 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z[37128]: 2024-09-16T01:40:23.387+0000 7f08182ec040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:40:23.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.685+0000 7fe90effd640 1 -- 172.21.15.139:0/3196101350 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mgrmap(e 20) ==== 138040+0+0 (secure 0 0 0) 0x7fe908098b00 con 0x7fe9180ffe10 2024-09-16T01:40:23.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.685+0000 7fe90effd640 1 -- 172.21.15.139:0/3196101350 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fe9040a3340 msgr2=0x7fe9040a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:40:23.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:23.685+0000 7fe90effd640 1 --2- 172.21.15.139:0/3196101350 >> [v2:172.21.15.139:6800/331514141,v1:172.21.15.139:6801/331514141] conn(0x7fe9040a3340 0x7fe9040a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:23.795 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 01:40:23 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y[36484]: 2024-09-16T01:40:23.428+0000 7f7cfbbf8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:40:23.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:23 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2964492194' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-16T01:40:23.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:23 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:40:23.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:23 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2988861401' entity='client.admin' 2024-09-16T01:40:23.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:23 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/674871345' entity='client.admin' 2024-09-16T01:40:23.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:23 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2948163527' entity='client.admin' 2024-09-16T01:40:24.011 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:23.672+0000 7fe25b1ee040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T01:40:24.011 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:23.908+0000 7fe1c32fc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:40:24.011 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:23.908+0000 7fe1c32fc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:40:24.011 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:23.908+0000 7fe1c32fc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:40:24.011 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:23.908+0000 7fe1c32fc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:40:24.011 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:23.908+0000 7fe1c32fc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:40:24.011 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:23.911+0000 7fe1c6302640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:40:24.011 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:23.911+0000 7fe1c6302640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:40:24.011 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:23.911+0000 7fe1c6302640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:40:24.011 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:23.911+0000 7fe1c6302640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:40:24.011 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:23.911+0000 7fe1c6302640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:40:24.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: Standby manager daemon z restarted 2024-09-16T01:40:24.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: Standby manager daemon z started 2024-09-16T01:40:24.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.176:0/906774758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T01:40:24.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.176:0/906774758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.176:0/906774758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.176:0/906774758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: Standby manager daemon y restarted 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: Standby manager daemon y started 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.164:0/3336000374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.164:0/3336000374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.164:0/3336000374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.? 172.21.15.164:0/3336000374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: Active manager daemon x restarted 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: Activating manager daemon x 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: mgrmap e20: x(active, starting, since 0.0127955s), standbys: z, y 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:40:24.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T01:40:24.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T01:40:24.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T01:40:24.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T01:40:24.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T01:40:24.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T01:40:24.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T01:40:24.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T01:40:24.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T01:40:24.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T01:40:24.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T01:40:24.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T01:40:24.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T01:40:24.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T01:40:24.676 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T01:40:24.676 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:40:24.676 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:40:24.676 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:40:24.676 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:40:24.676 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:40:24.676 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:40:24.676 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:40:24.676 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:40:24.676 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:40:24.676 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:40:24.676 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:40:24.676 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:40:24.677 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T01:40:24.677 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T01:40:24.677 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T01:40:24.677 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: Manager daemon x is now available 2024-09-16T01:40:24.677 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:40:24.677 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T01:40:24.677 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T01:40:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: Standby manager daemon z restarted 2024-09-16T01:40:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: Standby manager daemon z started 2024-09-16T01:40:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.176:0/906774758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T01:40:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.176:0/906774758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:40:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.176:0/906774758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T01:40:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.176:0/906774758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:40:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: Standby manager daemon y restarted 2024-09-16T01:40:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: Standby manager daemon y started 2024-09-16T01:40:24.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.164:0/3336000374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T01:40:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.164:0/3336000374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:40:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.164:0/3336000374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T01:40:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.? 172.21.15.164:0/3336000374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:40:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: Active manager daemon x restarted 2024-09-16T01:40:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: Activating manager daemon x 2024-09-16T01:40:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T01:40:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: mgrmap e20: x(active, starting, since 0.0127955s), standbys: z, y 2024-09-16T01:40:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:40:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:40:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:40:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T01:40:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T01:40:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T01:40:24.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T01:40:24.689 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T01:40:24.689 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T01:40:24.689 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T01:40:24.689 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T01:40:24.689 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T01:40:24.689 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T01:40:24.689 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T01:40:24.689 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T01:40:24.689 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T01:40:24.690 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T01:40:24.691 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T01:40:24.691 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: Manager daemon x is now available 2024-09-16T01:40:24.691 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:40:24.691 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T01:40:24.691 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:24 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T01:40:24.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.690+0000 7fe90effd640 1 -- 172.21.15.139:0/3196101350 <== mon.2 v2:172.21.15.164:3300/0 8 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7fe9080a3030 con 0x7fe9180ffe10 2024-09-16T01:40:24.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.690+0000 7fe90effd640 1 --2- 172.21.15.139:0/3196101350 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe904051da0 0x7fe904052180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:24.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.690+0000 7fe90effd640 1 -- 172.21.15.139:0/3196101350 --> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fe904034b90 con 0x7fe904051da0 2024-09-16T01:40:24.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.693+0000 7fe91d708640 1 --2- 172.21.15.139:0/3196101350 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe904051da0 0x7fe904052180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:24.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.696+0000 7fe91d708640 1 --2- 172.21.15.139:0/3196101350 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe904051da0 0x7fe904052180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe9140095f0 tx=0x7fe914005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:24.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.758+0000 7fe90effd640 1 -- 172.21.15.139:0/3196101350 <== mgr.15324 v2:172.21.15.139:6800/4031776938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe904034b90 con 0x7fe904051da0 2024-09-16T01:40:24.760 INFO:teuthology.orchestra.run.smithi139.stdout:Schedule set for path / 2024-09-16T01:40:24.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.760+0000 7fe92016d640 1 -- 172.21.15.139:0/3196101350 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe904051da0 msgr2=0x7fe904052180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:24.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.760+0000 7fe92016d640 1 --2- 172.21.15.139:0/3196101350 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe904051da0 0x7fe904052180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe9140095f0 tx=0x7fe914005f90 comp rx=0 tx=0).stop 2024-09-16T01:40:24.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.760+0000 7fe92016d640 1 -- 172.21.15.139:0/3196101350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9180ffe10 msgr2=0x7fe918197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:24.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.760+0000 7fe92016d640 1 --2- 172.21.15.139:0/3196101350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9180ffe10 0x7fe918197df0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe90800e9c0 tx=0x7fe90800ee80 comp rx=0 tx=0).stop 2024-09-16T01:40:24.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.760+0000 7fe91e70a640 1 -- 172.21.15.139:0/3196101350 reap_dead start 2024-09-16T01:40:24.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.760+0000 7fe92016d640 1 -- 172.21.15.139:0/3196101350 shutdown_connections 2024-09-16T01:40:24.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.760+0000 7fe92016d640 1 -- 172.21.15.139:0/3196101350 >> 172.21.15.139:0/3196101350 conn(0x7fe9180fb930 msgr2=0x7fe9181060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:24.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.760+0000 7fe92016d640 1 -- 172.21.15.139:0/3196101350 shutdown_connections 2024-09-16T01:40:24.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.760+0000 7fe92016d640 1 -- 172.21.15.139:0/3196101350 wait complete. 2024-09-16T01:40:24.773 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-16T01:40:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: Standby manager daemon z restarted 2024-09-16T01:40:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: Standby manager daemon z started 2024-09-16T01:40:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.176:0/906774758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T01:40:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.176:0/906774758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:40:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.176:0/906774758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T01:40:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.176:0/906774758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:40:24.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: Standby manager daemon y restarted 2024-09-16T01:40:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: Standby manager daemon y started 2024-09-16T01:40:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.164:0/3336000374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T01:40:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.164:0/3336000374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T01:40:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.164:0/3336000374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T01:40:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.? 172.21.15.164:0/3336000374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T01:40:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: Active manager daemon x restarted 2024-09-16T01:40:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: Activating manager daemon x 2024-09-16T01:40:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T01:40:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: mgrmap e20: x(active, starting, since 0.0127955s), standbys: z, y 2024-09-16T01:40:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T01:40:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T01:40:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T01:40:24.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T01:40:24.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T01:40:24.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T01:40:24.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T01:40:24.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T01:40:24.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T01:40:24.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T01:40:24.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T01:40:24.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T01:40:24.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T01:40:24.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T01:40:24.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T01:40:24.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T01:40:24.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T01:40:24.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T01:40:24.798 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T01:40:24.798 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T01:40:24.798 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T01:40:24.798 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T01:40:24.798 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T01:40:24.798 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T01:40:24.798 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T01:40:24.798 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T01:40:24.798 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T01:40:24.798 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T01:40:24.798 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T01:40:24.798 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T01:40:24.798 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T01:40:24.799 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T01:40:24.799 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T01:40:24.799 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: Manager daemon x is now available 2024-09-16T01:40:24.799 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:40:24.799 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T01:40:24.799 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:24 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T01:40:24.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.879+0000 7f87af39f640 1 -- 172.21.15.139:0/1793989065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87a8100a40 msgr2=0x7f87a8100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:24.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.879+0000 7f87af39f640 1 --2- 172.21.15.139:0/1793989065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87a8100a40 0x7f87a8100e40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009ca0 tx=0x7f87a402a8c0 comp rx=0 tx=0).stop 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.879+0000 7f87af39f640 1 -- 172.21.15.139:0/1793989065 shutdown_connections 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.879+0000 7f87af39f640 1 --2- 172.21.15.139:0/1793989065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87a81025f0 0x7f87a8108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.879+0000 7f87af39f640 1 --2- 172.21.15.139:0/1793989065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a8101c30 0x7f87a81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.879+0000 7f87af39f640 1 --2- 172.21.15.139:0/1793989065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87a8100a40 0x7f87a8100e40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.879+0000 7f87af39f640 1 -- 172.21.15.139:0/1793989065 >> 172.21.15.139:0/1793989065 conn(0x7f87a80fc1e0 msgr2=0x7f87a80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.879+0000 7f87af39f640 1 -- 172.21.15.139:0/1793989065 shutdown_connections 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.879+0000 7f87af39f640 1 -- 172.21.15.139:0/1793989065 wait complete. 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.879+0000 7f87af39f640 1 Processor -- start 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87af39f640 1 -- start start 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87af39f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87a8100a40 0x7f87a806a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87af39f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87a8101c30 0x7f87a819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87af39f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a81025f0 0x7f87a819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87af39f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810c6a0 con 0x7f87a8100a40 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87af39f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810c520 con 0x7f87a81025f0 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87af39f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810c820 con 0x7f87a8101c30 2024-09-16T01:40:24.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87ad13b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87a8100a40 0x7f87a806a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:24.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87ad13b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87a8100a40 0x7f87a806a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50540/0 (socket says 172.21.15.139:50540) 2024-09-16T01:40:24.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87ac93a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87a8101c30 0x7f87a819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:24.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87ad13b640 1 -- 172.21.15.139:0/4166246006 learned_addr learned my addr 172.21.15.139:0/4166246006 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:24.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87ad93c640 1 --2- 172.21.15.139:0/4166246006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a81025f0 0x7f87a819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:24.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87ac93a640 1 -- 172.21.15.139:0/4166246006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a81025f0 msgr2=0x7f87a819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:24.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87ac93a640 1 --2- 172.21.15.139:0/4166246006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a81025f0 0x7f87a819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:24.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87ac93a640 1 -- 172.21.15.139:0/4166246006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87a8100a40 msgr2=0x7f87a806a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:24.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87ac93a640 1 --2- 172.21.15.139:0/4166246006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87a8100a40 0x7f87a806a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:24.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87ac93a640 1 -- 172.21.15.139:0/4166246006 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a819fa10 con 0x7f87a8101c30 2024-09-16T01:40:24.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87ad13b640 1 --2- 172.21.15.139:0/4166246006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87a8100a40 0x7f87a806a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:40:24.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f87ac93a640 1 --2- 172.21.15.139:0/4166246006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87a8101c30 0x7f87a819cb00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f879800e9f0 tx=0x7f879800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:24.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f879e7fc640 1 -- 172.21.15.139:0/4166246006 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879800ccc0 con 0x7f87a8101c30 2024-09-16T01:40:24.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f879e7fc640 1 -- 172.21.15.139:0/4166246006 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8798010910 con 0x7f87a8101c30 2024-09-16T01:40:24.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.880+0000 7f879e7fc640 1 -- 172.21.15.139:0/4166246006 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798018960 con 0x7f87a8101c30 2024-09-16T01:40:24.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.881+0000 7f87af39f640 1 -- 172.21.15.139:0/4166246006 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a819fd00 con 0x7f87a8101c30 2024-09-16T01:40:24.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.881+0000 7f87af39f640 1 -- 172.21.15.139:0/4166246006 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a81a01f0 con 0x7f87a8101c30 2024-09-16T01:40:24.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.881+0000 7f87af39f640 1 -- 172.21.15.139:0/4166246006 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a8107e30 con 0x7f87a8101c30 2024-09-16T01:40:24.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.883+0000 7f879e7fc640 1 -- 172.21.15.139:0/4166246006 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7f8798010ac0 con 0x7f87a8101c30 2024-09-16T01:40:24.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.883+0000 7f879e7fc640 1 --2- 172.21.15.139:0/4166246006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f87940a3100 0x7f87940a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:24.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.883+0000 7f879e7fc640 1 -- 172.21.15.139:0/4166246006 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f87980cffe0 con 0x7f87a8101c30 2024-09-16T01:40:24.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.883+0000 7f87ad13b640 1 --2- 172.21.15.139:0/4166246006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f87940a3100 0x7f87940a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:24.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.884+0000 7f87ad13b640 1 --2- 172.21.15.139:0/4166246006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f87940a3100 0x7f87940a55c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f87a4006010 tx=0x7f87a4000f40 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:24.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:24.887+0000 7f879e7fc640 1 -- 172.21.15.139:0/4166246006 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f879809c490 con 0x7f87a8101c30 2024-09-16T01:40:25.080 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:24.930+0000 7fe1a8a0d640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:01:40:24] ENGINE Error in HTTPServer.serve 2024-09-16T01:40:25.080 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: Traceback (most recent call last): 2024-09-16T01:40:25.080 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:40:25.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self._connections.run(self.expiration_interval) 2024-09-16T01:40:25.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:40:25.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self._run(expiration_interval) 2024-09-16T01:40:25.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:40:25.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:40:25.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:40:25.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:40:25.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:40:25.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: s = self.context.wrap_socket( 2024-09-16T01:40:25.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:40:25.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: return self.sslsocket_class._create( 2024-09-16T01:40:25.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:40:25.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self.do_handshake() 2024-09-16T01:40:25.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:40:25.082 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self._sslobj.do_handshake() 2024-09-16T01:40:25.082 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:40:25.082 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:40:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:40:25.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.082+0000 7f87af39f640 1 -- 172.21.15.139:0/4166246006 --> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f87a8062790 con 0x7f87940a3100 2024-09-16T01:40:25.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.136+0000 7f879e7fc640 1 -- 172.21.15.139:0/4166246006 <== mgr.15324 v2:172.21.15.139:6800/4031776938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f87a8062790 con 0x7f87940a3100 2024-09-16T01:40:25.138 INFO:teuthology.orchestra.run.smithi139.stdout:Retention added to path / 2024-09-16T01:40:25.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.138+0000 7f87af39f640 1 -- 172.21.15.139:0/4166246006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f87940a3100 msgr2=0x7f87940a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:25.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.138+0000 7f87af39f640 1 --2- 172.21.15.139:0/4166246006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f87940a3100 0x7f87940a55c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f87a4006010 tx=0x7f87a4000f40 comp rx=0 tx=0).stop 2024-09-16T01:40:25.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.138+0000 7f87af39f640 1 -- 172.21.15.139:0/4166246006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87a8101c30 msgr2=0x7f87a819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:25.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.138+0000 7f87af39f640 1 --2- 172.21.15.139:0/4166246006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87a8101c30 0x7f87a819cb00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f879800e9f0 tx=0x7f879800eeb0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.138+0000 7f87af39f640 1 -- 172.21.15.139:0/4166246006 shutdown_connections 2024-09-16T01:40:25.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.138+0000 7f87af39f640 1 --2- 172.21.15.139:0/4166246006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f87940a3100 0x7f87940a55c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.138+0000 7f87af39f640 1 --2- 172.21.15.139:0/4166246006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a81025f0 0x7f87a819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.138+0000 7f87af39f640 1 --2- 172.21.15.139:0/4166246006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87a8101c30 0x7f87a819cb00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.138+0000 7f87af39f640 1 --2- 172.21.15.139:0/4166246006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87a8100a40 0x7f87a806a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.138+0000 7f87af39f640 1 -- 172.21.15.139:0/4166246006 >> 172.21.15.139:0/4166246006 conn(0x7f87a80fc1e0 msgr2=0x7f87a8105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:25.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.138+0000 7f87af39f640 1 -- 172.21.15.139:0/4166246006 shutdown_connections 2024-09-16T01:40:25.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.138+0000 7f87af39f640 1 -- 172.21.15.139:0/4166246006 wait complete. 2024-09-16T01:40:25.150 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-16T01:40:25.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.251+0000 7fe3ec378640 1 -- 172.21.15.139:0/3357013084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e4103710 msgr2=0x7fe3e4109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:25.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.251+0000 7fe3ec378640 1 --2- 172.21.15.139:0/3357013084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e4103710 0x7fe3e4109fa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc009ca0 tx=0x7fe3dc02a8c0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.251+0000 7fe3ec378640 1 -- 172.21.15.139:0/3357013084 shutdown_connections 2024-09-16T01:40:25.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.251+0000 7fe3ec378640 1 --2- 172.21.15.139:0/3357013084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e4103710 0x7fe3e4109fa0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.251+0000 7fe3ec378640 1 --2- 172.21.15.139:0/3357013084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e4102d50 0x7fe3e41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.251+0000 7fe3ec378640 1 --2- 172.21.15.139:0/3357013084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3e4101b60 0x7fe3e4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.251+0000 7fe3ec378640 1 -- 172.21.15.139:0/3357013084 >> 172.21.15.139:0/3357013084 conn(0x7fe3e40fd2e0 msgr2=0x7fe3e40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:25.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.251+0000 7fe3ec378640 1 -- 172.21.15.139:0/3357013084 shutdown_connections 2024-09-16T01:40:25.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.251+0000 7fe3ec378640 1 -- 172.21.15.139:0/3357013084 wait complete. 2024-09-16T01:40:25.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.251+0000 7fe3ec378640 1 Processor -- start 2024-09-16T01:40:25.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.251+0000 7fe3ec378640 1 -- start start 2024-09-16T01:40:25.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.251+0000 7fe3ec378640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3e4101b60 0x7fe3e4197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:25.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ec378640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e4102d50 0x7fe3e41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:25.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ec378640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e4103710 0x7fe3e419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:25.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ec378640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e410d7c0 con 0x7fe3e4103710 2024-09-16T01:40:25.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ec378640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e410d640 con 0x7fe3e4102d50 2024-09-16T01:40:25.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ec378640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e410d940 con 0x7fe3e4101b60 2024-09-16T01:40:25.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ea915640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e4103710 0x7fe3e419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:25.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ea915640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e4103710 0x7fe3e419f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50550/0 (socket says 172.21.15.139:50550) 2024-09-16T01:40:25.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ea915640 1 -- 172.21.15.139:0/2990819645 learned_addr learned my addr 172.21.15.139:0/2990819645 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:25.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3e9913640 1 --2- 172.21.15.139:0/2990819645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e4102d50 0x7fe3e41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:25.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ea114640 1 --2- 172.21.15.139:0/2990819645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3e4101b60 0x7fe3e4197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:25.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ea915640 1 -- 172.21.15.139:0/2990819645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3e4101b60 msgr2=0x7fe3e4197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:25.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ea915640 1 --2- 172.21.15.139:0/2990819645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3e4101b60 0x7fe3e4197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ea915640 1 -- 172.21.15.139:0/2990819645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e4102d50 msgr2=0x7fe3e41982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:25.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ea915640 1 --2- 172.21.15.139:0/2990819645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e4102d50 0x7fe3e41982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ea915640 1 -- 172.21.15.139:0/2990819645 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e419f9b0 con 0x7fe3e4103710 2024-09-16T01:40:25.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ea915640 1 --2- 172.21.15.139:0/2990819645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e4103710 0x7fe3e419f310 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc031ae0 tx=0x7fe3dc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3db7fe640 1 -- 172.21.15.139:0/2990819645 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc003c30 con 0x7fe3e4103710 2024-09-16T01:40:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3db7fe640 1 -- 172.21.15.139:0/2990819645 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3dc003dd0 con 0x7fe3e4103710 2024-09-16T01:40:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3db7fe640 1 -- 172.21.15.139:0/2990819645 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc02fcb0 con 0x7fe3e4103710 2024-09-16T01:40:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ec378640 1 -- 172.21.15.139:0/2990819645 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e419fc40 con 0x7fe3e4103710 2024-09-16T01:40:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.252+0000 7fe3ec378640 1 -- 172.21.15.139:0/2990819645 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e41a0190 con 0x7fe3e4103710 2024-09-16T01:40:25.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.253+0000 7fe3ec378640 1 -- 172.21.15.139:0/2990819645 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e4108f00 con 0x7fe3e4103710 2024-09-16T01:40:25.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.254+0000 7fe3db7fe640 1 -- 172.21.15.139:0/2990819645 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7fe3dc004050 con 0x7fe3e4103710 2024-09-16T01:40:25.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.255+0000 7fe3db7fe640 1 --2- 172.21.15.139:0/2990819645 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe3d00a3180 0x7fe3d00a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:25.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.255+0000 7fe3ea114640 1 --2- 172.21.15.139:0/2990819645 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe3d00a3180 0x7fe3d00a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:25.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.255+0000 7fe3db7fe640 1 -- 172.21.15.139:0/2990819645 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fe3dc0f0eb0 con 0x7fe3e4103710 2024-09-16T01:40:25.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.256+0000 7fe3ea114640 1 --2- 172.21.15.139:0/2990819645 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe3d00a3180 0x7fe3d00a5640 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4199270 tx=0x7fe3e000b410 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:25.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.258+0000 7fe3db7fe640 1 -- 172.21.15.139:0/2990819645 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3dc0bd9f0 con 0x7fe3e4103710 2024-09-16T01:40:25.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.473+0000 7fe3ec378640 1 -- 172.21.15.139:0/2990819645 --> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fe3e4062630 con 0x7fe3d00a3180 2024-09-16T01:40:25.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.480+0000 7fe3db7fe640 1 -- 172.21.15.139:0/2990819645 <== mgr.15324 v2:172.21.15.139:6800/4031776938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fe3e4062630 con 0x7fe3d00a3180 2024-09-16T01:40:25.485 INFO:teuthology.orchestra.run.smithi139.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-16T00:00:00", "created": "2024-09-16T01:40:24", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-16T01:40:25.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.482+0000 7fe3ec378640 1 -- 172.21.15.139:0/2990819645 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe3d00a3180 msgr2=0x7fe3d00a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:25.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.482+0000 7fe3ec378640 1 --2- 172.21.15.139:0/2990819645 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe3d00a3180 0x7fe3d00a5640 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4199270 tx=0x7fe3e000b410 comp rx=0 tx=0).stop 2024-09-16T01:40:25.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.482+0000 7fe3ec378640 1 -- 172.21.15.139:0/2990819645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e4103710 msgr2=0x7fe3e419f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:25.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.482+0000 7fe3ec378640 1 --2- 172.21.15.139:0/2990819645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e4103710 0x7fe3e419f310 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc031ae0 tx=0x7fe3dc031ee0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.482+0000 7fe3ec378640 1 -- 172.21.15.139:0/2990819645 shutdown_connections 2024-09-16T01:40:25.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.482+0000 7fe3ec378640 1 --2- 172.21.15.139:0/2990819645 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe3d00a3180 0x7fe3d00a5640 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.482+0000 7fe3ec378640 1 --2- 172.21.15.139:0/2990819645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e4103710 0x7fe3e419f310 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.482+0000 7fe3ec378640 1 --2- 172.21.15.139:0/2990819645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e4102d50 0x7fe3e41982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.482+0000 7fe3ec378640 1 --2- 172.21.15.139:0/2990819645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3e4101b60 0x7fe3e4197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.482+0000 7fe3ec378640 1 -- 172.21.15.139:0/2990819645 >> 172.21.15.139:0/2990819645 conn(0x7fe3e40fd2e0 msgr2=0x7fe3e41060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:25.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.482+0000 7fe3ec378640 1 -- 172.21.15.139:0/2990819645 shutdown_connections 2024-09-16T01:40:25.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.482+0000 7fe3ec378640 1 -- 172.21.15.139:0/2990819645 wait complete. 2024-09-16T01:40:25.495 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-16T01:40:25.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.607+0000 7f536a123640 1 -- 172.21.15.139:0/1113432119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5364102d40 msgr2=0x7f53641031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:25.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.607+0000 7f536a123640 1 --2- 172.21.15.139:0/1113432119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5364102d40 0x7f53641031c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5358009900 tx=0x7f535802a410 comp rx=0 tx=0).stop 2024-09-16T01:40:25.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.607+0000 7f536a123640 1 -- 172.21.15.139:0/1113432119 shutdown_connections 2024-09-16T01:40:25.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.607+0000 7f536a123640 1 --2- 172.21.15.139:0/1113432119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5364103700 0x7f5364109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.607+0000 7f536a123640 1 --2- 172.21.15.139:0/1113432119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5364102d40 0x7f53641031c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.607+0000 7f536a123640 1 --2- 172.21.15.139:0/1113432119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5364101b50 0x7f5364101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.607+0000 7f536a123640 1 -- 172.21.15.139:0/1113432119 >> 172.21.15.139:0/1113432119 conn(0x7f53640fd2b0 msgr2=0x7f53640ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:25.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.607+0000 7f536a123640 1 -- 172.21.15.139:0/1113432119 shutdown_connections 2024-09-16T01:40:25.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.607+0000 7f536a123640 1 -- 172.21.15.139:0/1113432119 wait complete. 2024-09-16T01:40:25.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.607+0000 7f536a123640 1 Processor -- start 2024-09-16T01:40:25.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f536a123640 1 -- start start 2024-09-16T01:40:25.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f536a123640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5364101b50 0x7f5364197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:25.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f536a123640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5364102d40 0x7f5364198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:25.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f536a123640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5364103700 0x7f536419f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:25.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f536a123640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f536410d7b0 con 0x7f5364103700 2024-09-16T01:40:25.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f536a123640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f536410d630 con 0x7f5364101b50 2024-09-16T01:40:25.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f536a123640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f536410d930 con 0x7f5364102d40 2024-09-16T01:40:25.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f53637fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5364101b50 0x7f5364197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:25.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f5362ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5364102d40 0x7f5364198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:25.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f53637fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5364101b50 0x7f5364197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:38072/0 (socket says 172.21.15.139:38072) 2024-09-16T01:40:25.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f53637fe640 1 -- 172.21.15.139:0/994801094 learned_addr learned my addr 172.21.15.139:0/994801094 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:25.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f5363fff640 1 --2- 172.21.15.139:0/994801094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5364103700 0x7f536419f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:25.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f53637fe640 1 -- 172.21.15.139:0/994801094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5364102d40 msgr2=0x7f5364198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:25.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f53637fe640 1 --2- 172.21.15.139:0/994801094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5364102d40 0x7f5364198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f53637fe640 1 -- 172.21.15.139:0/994801094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5364103700 msgr2=0x7f536419f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:25.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f53637fe640 1 --2- 172.21.15.139:0/994801094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5364103700 0x7f536419f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.608+0000 7f53637fe640 1 -- 172.21.15.139:0/994801094 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536419f9e0 con 0x7f5364101b50 2024-09-16T01:40:25.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.609+0000 7f53637fe640 1 --2- 172.21.15.139:0/994801094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5364101b50 0x7f5364197dc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f535400ebc0 tx=0x7f535400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:25.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.609+0000 7f5360ff9640 1 -- 172.21.15.139:0/994801094 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5354003b70 con 0x7f5364101b50 2024-09-16T01:40:25.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.609+0000 7f5360ff9640 1 -- 172.21.15.139:0/994801094 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5354003d10 con 0x7f5364101b50 2024-09-16T01:40:25.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.609+0000 7f5360ff9640 1 -- 172.21.15.139:0/994801094 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5354019980 con 0x7f5364101b50 2024-09-16T01:40:25.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.609+0000 7f536a123640 1 -- 172.21.15.139:0/994801094 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536419fcd0 con 0x7f5364101b50 2024-09-16T01:40:25.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.609+0000 7f536a123640 1 -- 172.21.15.139:0/994801094 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53641a01a0 con 0x7f5364101b50 2024-09-16T01:40:25.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.610+0000 7f536a123640 1 -- 172.21.15.139:0/994801094 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5364108f40 con 0x7f5364101b50 2024-09-16T01:40:25.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.611+0000 7f5360ff9640 1 -- 172.21.15.139:0/994801094 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7f5354004b60 con 0x7f5364101b50 2024-09-16T01:40:25.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.612+0000 7f5360ff9640 1 --2- 172.21.15.139:0/994801094 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f534c0a3100 0x7f534c0a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:25.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.612+0000 7f5362ffd640 1 --2- 172.21.15.139:0/994801094 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f534c0a3100 0x7f534c0a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:25.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.612+0000 7f5360ff9640 1 -- 172.21.15.139:0/994801094 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f53540d05e0 con 0x7f5364101b50 2024-09-16T01:40:25.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.612+0000 7f5362ffd640 1 --2- 172.21.15.139:0/994801094 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f534c0a3100 0x7f534c0a55c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f53580314e0 tx=0x7f5358000bc0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:25.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.615+0000 7f5360ff9640 1 -- 172.21.15.139:0/994801094 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f535409d0e0 con 0x7f5364101b50 2024-09-16T01:40:25.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.831+0000 7f536a123640 1 -- 172.21.15.139:0/994801094 --> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5364062630 con 0x7f534c0a3100 2024-09-16T01:40:25.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.838+0000 7f5360ff9640 1 -- 172.21.15.139:0/994801094 <== mgr.15324 v2:172.21.15.139:6800/4031776938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5364062630 con 0x7f534c0a3100 2024-09-16T01:40:25.840 INFO:teuthology.orchestra.run.smithi139.stdout:/ 1m 6m3h 2024-09-16T01:40:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.840+0000 7f536a123640 1 -- 172.21.15.139:0/994801094 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f534c0a3100 msgr2=0x7f534c0a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.840+0000 7f536a123640 1 --2- 172.21.15.139:0/994801094 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f534c0a3100 0x7f534c0a55c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f53580314e0 tx=0x7f5358000bc0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.840+0000 7f536a123640 1 -- 172.21.15.139:0/994801094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5364101b50 msgr2=0x7f5364197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.840+0000 7f536a123640 1 --2- 172.21.15.139:0/994801094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5364101b50 0x7f5364197dc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f535400ebc0 tx=0x7f535400c6a0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.840+0000 7f536a123640 1 -- 172.21.15.139:0/994801094 shutdown_connections 2024-09-16T01:40:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.840+0000 7f536a123640 1 --2- 172.21.15.139:0/994801094 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f534c0a3100 0x7f534c0a55c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.840+0000 7f536a123640 1 --2- 172.21.15.139:0/994801094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5364103700 0x7f536419f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.840+0000 7f536a123640 1 --2- 172.21.15.139:0/994801094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5364102d40 0x7f5364198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.840+0000 7f536a123640 1 --2- 172.21.15.139:0/994801094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5364101b50 0x7f5364197dc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:25.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.840+0000 7f536a123640 1 -- 172.21.15.139:0/994801094 >> 172.21.15.139:0/994801094 conn(0x7f53640fd2b0 msgr2=0x7f5364106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:25.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.840+0000 7f536a123640 1 -- 172.21.15.139:0/994801094 shutdown_connections 2024-09-16T01:40:25.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:25.840+0000 7f536a123640 1 -- 172.21.15.139:0/994801094 wait complete. 2024-09-16T01:40:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:25 smithi139 ceph-mon[31175]: mgrmap e21: x(active, since 1.01829s), standbys: z, y 2024-09-16T01:40:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:25 smithi139 ceph-mon[31175]: [16/Sep/2024:01:40:24] ENGINE Bus STARTING 2024-09-16T01:40:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:25 smithi139 ceph-mon[31175]: [16/Sep/2024:01:40:24] ENGINE Serving on http://172.21.15.139:8765 2024-09-16T01:40:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:25 smithi139 ceph-mon[31175]: [16/Sep/2024:01:40:24] ENGINE Serving on https://172.21.15.139:7150 2024-09-16T01:40:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:25 smithi139 ceph-mon[31175]: [16/Sep/2024:01:40:24] ENGINE Bus STARTED 2024-09-16T01:40:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:25 smithi139 ceph-mon[31175]: [16/Sep/2024:01:40:24] ENGINE Error in HTTPServer.serve 2024-09-16T01:40:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout: Traceback (most recent call last): 2024-09-16T01:40:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:40:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout: self._connections.run(self.expiration_interval) 2024-09-16T01:40:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:40:25.852 INFO:journalctl@ceph.mon.a.smithi139.stdout: self._run(expiration_interval) 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: s = self.context.wrap_socket( 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: return self.sslsocket_class._create( 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: self.do_handshake() 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: self._sslobj.do_handshake() 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:25 smithi139 ceph-mon[31175]: from='client.24937 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:25.853 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:25 smithi139 ceph-mon[31175]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:25 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:25.854 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:25 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:25.854 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-16T01:40:25.880 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-16T01:40:25.890 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-16T01:40:25.899 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T01:40:25.899 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-16T01:40:25.923 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-16T01:40:25.936 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T01:40:25.936 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T01:40:26.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:25 smithi164 ceph-mon[35121]: mgrmap e21: x(active, since 1.01829s), standbys: z, y 2024-09-16T01:40:26.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:25 smithi164 ceph-mon[35121]: [16/Sep/2024:01:40:24] ENGINE Bus STARTING 2024-09-16T01:40:26.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:25 smithi164 ceph-mon[35121]: [16/Sep/2024:01:40:24] ENGINE Serving on http://172.21.15.139:8765 2024-09-16T01:40:26.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:25 smithi164 ceph-mon[35121]: [16/Sep/2024:01:40:24] ENGINE Serving on https://172.21.15.139:7150 2024-09-16T01:40:26.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:25 smithi164 ceph-mon[35121]: [16/Sep/2024:01:40:24] ENGINE Bus STARTED 2024-09-16T01:40:26.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:25 smithi164 ceph-mon[35121]: [16/Sep/2024:01:40:24] ENGINE Error in HTTPServer.serve 2024-09-16T01:40:26.045 INFO:journalctl@ceph.mon.b.smithi164.stdout: Traceback (most recent call last): 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: self._connections.run(self.expiration_interval) 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: self._run(expiration_interval) 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: s = self.context.wrap_socket( 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: return self.sslsocket_class._create( 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: self.do_handshake() 2024-09-16T01:40:26.046 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:40:26.047 INFO:journalctl@ceph.mon.b.smithi164.stdout: self._sslobj.do_handshake() 2024-09-16T01:40:26.047 INFO:journalctl@ceph.mon.b.smithi164.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:40:26.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:25 smithi164 ceph-mon[35121]: from='client.24937 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:26.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:25 smithi164 ceph-mon[35121]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:26.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:25 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:26.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:25 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:26.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:25 smithi176 ceph-mon[35324]: mgrmap e21: x(active, since 1.01829s), standbys: z, y 2024-09-16T01:40:26.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:25 smithi176 ceph-mon[35324]: [16/Sep/2024:01:40:24] ENGINE Bus STARTING 2024-09-16T01:40:26.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:25 smithi176 ceph-mon[35324]: [16/Sep/2024:01:40:24] ENGINE Serving on http://172.21.15.139:8765 2024-09-16T01:40:26.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:25 smithi176 ceph-mon[35324]: [16/Sep/2024:01:40:24] ENGINE Serving on https://172.21.15.139:7150 2024-09-16T01:40:26.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:25 smithi176 ceph-mon[35324]: [16/Sep/2024:01:40:24] ENGINE Bus STARTED 2024-09-16T01:40:26.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:25 smithi176 ceph-mon[35324]: [16/Sep/2024:01:40:24] ENGINE Error in HTTPServer.serve 2024-09-16T01:40:26.187 INFO:journalctl@ceph.mon.c.smithi176.stdout: Traceback (most recent call last): 2024-09-16T01:40:26.187 INFO:journalctl@ceph.mon.c.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T01:40:26.187 INFO:journalctl@ceph.mon.c.smithi176.stdout: self._connections.run(self.expiration_interval) 2024-09-16T01:40:26.187 INFO:journalctl@ceph.mon.c.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T01:40:26.187 INFO:journalctl@ceph.mon.c.smithi176.stdout: self._run(expiration_interval) 2024-09-16T01:40:26.187 INFO:journalctl@ceph.mon.c.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T01:40:26.187 INFO:journalctl@ceph.mon.c.smithi176.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T01:40:26.188 INFO:journalctl@ceph.mon.c.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T01:40:26.188 INFO:journalctl@ceph.mon.c.smithi176.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T01:40:26.188 INFO:journalctl@ceph.mon.c.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T01:40:26.188 INFO:journalctl@ceph.mon.c.smithi176.stdout: s = self.context.wrap_socket( 2024-09-16T01:40:26.188 INFO:journalctl@ceph.mon.c.smithi176.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T01:40:26.188 INFO:journalctl@ceph.mon.c.smithi176.stdout: return self.sslsocket_class._create( 2024-09-16T01:40:26.188 INFO:journalctl@ceph.mon.c.smithi176.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T01:40:26.188 INFO:journalctl@ceph.mon.c.smithi176.stdout: self.do_handshake() 2024-09-16T01:40:26.188 INFO:journalctl@ceph.mon.c.smithi176.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T01:40:26.188 INFO:journalctl@ceph.mon.c.smithi176.stdout: self._sslobj.do_handshake() 2024-09-16T01:40:26.188 INFO:journalctl@ceph.mon.c.smithi176.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T01:40:26.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:25 smithi176 ceph-mon[35324]: from='client.24937 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:26.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:25 smithi176 ceph-mon[35324]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:26.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:25 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:26.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:25 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:26.192 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:26.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.716+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/666977372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fec1036e0 msgr2=0x7f0fec107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:26.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.716+0000 7f0ff2ff4640 1 --2- 172.21.15.139:0/666977372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fec1036e0 0x7f0fec107b90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009930 tx=0x7f0fd802a420 comp rx=0 tx=0).stop 2024-09-16T01:40:26.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.716+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/666977372 shutdown_connections 2024-09-16T01:40:26.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.716+0000 7f0ff2ff4640 1 --2- 172.21.15.139:0/666977372 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0fec1082a0 0x7f0fec10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:26.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.716+0000 7f0ff2ff4640 1 --2- 172.21.15.139:0/666977372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fec1036e0 0x7f0fec107b90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:26.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.716+0000 7f0ff2ff4640 1 --2- 172.21.15.139:0/666977372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fec102d30 0x7f0fec103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:26.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.716+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/666977372 >> 172.21.15.139:0/666977372 conn(0x7f0fec0fc7d0 msgr2=0x7f0fec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:26.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.716+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/666977372 shutdown_connections 2024-09-16T01:40:26.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.716+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/666977372 wait complete. 2024-09-16T01:40:26.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.716+0000 7f0ff2ff4640 1 Processor -- start 2024-09-16T01:40:26.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.716+0000 7f0ff2ff4640 1 -- start start 2024-09-16T01:40:26.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff2ff4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fec102d30 0x7f0fec1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:26.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff2ff4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0fec1036e0 0x7f0fec1968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:26.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff2ff4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fec1082a0 0x7f0fec19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:26.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff2ff4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec10fe90 con 0x7f0fec102d30 2024-09-16T01:40:26.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff2ff4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec10fd10 con 0x7f0fec1082a0 2024-09-16T01:40:26.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff2ff4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec110010 con 0x7f0fec1036e0 2024-09-16T01:40:26.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff0d90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fec102d30 0x7f0fec1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:26.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff0d90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fec102d30 0x7f0fec1963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50626/0 (socket says 172.21.15.139:50626) 2024-09-16T01:40:26.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff0d90640 1 -- 172.21.15.139:0/1785276681 learned_addr learned my addr 172.21.15.139:0/1785276681 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:26.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0febfff640 1 --2- 172.21.15.139:0/1785276681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0fec1036e0 0x7f0fec1968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:26.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff1591640 1 --2- 172.21.15.139:0/1785276681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fec1082a0 0x7f0fec19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:26.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff0d90640 1 -- 172.21.15.139:0/1785276681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0fec1036e0 msgr2=0x7f0fec1968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:26.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff0d90640 1 --2- 172.21.15.139:0/1785276681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0fec1036e0 0x7f0fec1968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:26.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff0d90640 1 -- 172.21.15.139:0/1785276681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fec1082a0 msgr2=0x7f0fec19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:26.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff0d90640 1 --2- 172.21.15.139:0/1785276681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fec1082a0 0x7f0fec19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:26.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff0d90640 1 -- 172.21.15.139:0/1785276681 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fec19b360 con 0x7f0fec102d30 2024-09-16T01:40:26.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.717+0000 7f0ff0d90640 1 --2- 172.21.15.139:0/1785276681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fec102d30 0x7f0fec1963b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000e9f0 tx=0x7f0fe000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.718+0000 7f0fe9ffb640 1 -- 172.21.15.139:0/1785276681 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe000cd80 con 0x7f0fec102d30 2024-09-16T01:40:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.718+0000 7f0fe9ffb640 1 -- 172.21.15.139:0/1785276681 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe0010910 con 0x7f0fec102d30 2024-09-16T01:40:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.718+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/1785276681 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fec19b650 con 0x7f0fec102d30 2024-09-16T01:40:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.718+0000 7f0fe9ffb640 1 -- 172.21.15.139:0/1785276681 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0018990 con 0x7f0fec102d30 2024-09-16T01:40:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.718+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/1785276681 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fec1a2f30 con 0x7f0fec102d30 2024-09-16T01:40:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.719+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/1785276681 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fec103110 con 0x7f0fec102d30 2024-09-16T01:40:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.720+0000 7f0fe9ffb640 1 -- 172.21.15.139:0/1785276681 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 22) ==== 138476+0+0 (secure 0 0 0) 0x7f0fe0010ac0 con 0x7f0fec102d30 2024-09-16T01:40:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.721+0000 7f0fe9ffb640 1 --2- 172.21.15.139:0/1785276681 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0fdc0a33b0 0x7f0fdc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.721+0000 7f0febfff640 1 --2- 172.21.15.139:0/1785276681 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0fdc0a33b0 0x7f0fdc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.721+0000 7f0fe9ffb640 1 -- 172.21.15.139:0/1785276681 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f0fe00d03b0 con 0x7f0fec102d30 2024-09-16T01:40:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.721+0000 7f0febfff640 1 --2- 172.21.15.139:0/1785276681 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0fdc0a33b0 0x7f0fdc0a5870 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8005940 tx=0x7f0fd80058d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.725+0000 7f0fe9ffb640 1 -- 172.21.15.139:0/1785276681 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0fe009c390 con 0x7f0fec102d30 2024-09-16T01:40:26.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.898+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/1785276681 --> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0fec062630 con 0x7f0fdc0a33b0 2024-09-16T01:40:26.901 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:40:26.901 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:26.901 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:26.901 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:26.901 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:26.901 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='client.25004 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:26.901 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:26.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:26.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:26.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:26.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:26.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:26.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi164 to 176.0M 2024-09-16T01:40:26.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: Unable to set osd_memory_target on smithi164 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-16T01:40:26.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:26.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:26.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:26.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:26.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:26.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:26.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi176 to 176.0M 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: Unable to set osd_memory_target on smithi176 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi139 to 176.0M 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: Unable to set osd_memory_target on smithi139 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: Updating smithi176:/etc/ceph/ceph.conf 2024-09-16T01:40:26.903 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:26 smithi139 ceph-mon[31175]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-16T01:40:26.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.909+0000 7f0fe9ffb640 1 -- 172.21.15.139:0/1785276681 <== mgr.15324 v2:172.21.15.139:6800/4031776938 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339718 (secure 0 0 0) 0x7f0fec062630 con 0x7f0fdc0a33b0 2024-09-16T01:40:26.915 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:40:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T01:40:26.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.912+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/1785276681 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0fdc0a33b0 msgr2=0x7f0fdc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:26.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.912+0000 7f0ff2ff4640 1 --2- 172.21.15.139:0/1785276681 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0fdc0a33b0 0x7f0fdc0a5870 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8005940 tx=0x7f0fd80058d0 comp rx=0 tx=0).stop 2024-09-16T01:40:26.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.912+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/1785276681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fec102d30 msgr2=0x7f0fec1963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:26.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.912+0000 7f0ff2ff4640 1 --2- 172.21.15.139:0/1785276681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fec102d30 0x7f0fec1963b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000e9f0 tx=0x7f0fe000eeb0 comp rx=0 tx=0).stop 2024-09-16T01:40:26.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.912+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/1785276681 shutdown_connections 2024-09-16T01:40:26.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.912+0000 7f0ff2ff4640 1 --2- 172.21.15.139:0/1785276681 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0fdc0a33b0 0x7f0fdc0a5870 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:26.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.912+0000 7f0ff2ff4640 1 --2- 172.21.15.139:0/1785276681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fec1082a0 0x7f0fec19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:26.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.912+0000 7f0ff2ff4640 1 --2- 172.21.15.139:0/1785276681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0fec1036e0 0x7f0fec1968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:26.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.912+0000 7f0ff2ff4640 1 --2- 172.21.15.139:0/1785276681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fec102d30 0x7f0fec1963b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:26.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.912+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/1785276681 >> 172.21.15.139:0/1785276681 conn(0x7f0fec0fc7d0 msgr2=0x7f0fec105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:26.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.912+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/1785276681 shutdown_connections 2024-09-16T01:40:26.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:26.912+0000 7f0ff2ff4640 1 -- 172.21.15.139:0/1785276681 wait complete. 2024-09-16T01:40:27.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:40:27.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='client.25004 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:27.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi164 to 176.0M 2024-09-16T01:40:27.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: Unable to set osd_memory_target on smithi164 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-16T01:40:27.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi176 to 176.0M 2024-09-16T01:40:27.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: Unable to set osd_memory_target on smithi176 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-16T01:40:27.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi139 to 176.0M 2024-09-16T01:40:27.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: Unable to set osd_memory_target on smithi139 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-16T01:40:27.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:40:27.048 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:40:27.048 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-16T01:40:27.048 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-16T01:40:27.048 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: Updating smithi176:/etc/ceph/ceph.conf 2024-09-16T01:40:27.048 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:26 smithi164 ceph-mon[35121]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-16T01:40:27.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:40:27.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='client.25004 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:40:27.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi164 to 176.0M 2024-09-16T01:40:27.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: Unable to set osd_memory_target on smithi164 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-16T01:40:27.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi176 to 176.0M 2024-09-16T01:40:27.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: Unable to set osd_memory_target on smithi176 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-16T01:40:27.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:27.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:40:27.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi139 to 176.0M 2024-09-16T01:40:27.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: Unable to set osd_memory_target on smithi139 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-16T01:40:27.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:40:27.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:40:27.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: Updating smithi139:/etc/ceph/ceph.conf 2024-09-16T01:40:27.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-16T01:40:27.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: Updating smithi176:/etc/ceph/ceph.conf 2024-09-16T01:40:27.189 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:26 smithi176 ceph-mon[35324]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-16T01:40:27.672 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-16T01:40:25.693617+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192345,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1311,"num_read_kb":817,"num_write":562,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":943,"ondisk_log_size":943,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373688,"kb_used_data":11924,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568136,"statfs":{"total":1151940427776,"available":1151557771264,"internally_reserved":0,"allocated":12210176,"data_stored":9521600,"data_compressed":502956,"data_compressed_allocated":675840,"data_compressed_original":4202496,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.989632"},"pg_stats":[{"pgid":"2.1f","version":"86'14","reported_seq":167,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707440+0000","last_change":"2024-09-16T01:38:08.567909+0000","last_active":"2024-09-16T01:40:23.707440+0000","last_peered":"2024-09-16T01:40:23.707440+0000","last_clean":"2024-09-16T01:40:23.707440+0000","last_became_active":"2024-09-16T01:38:08.560733+0000","last_became_peered":"2024-09-16T01:38:08.560733+0000","last_unstale":"2024-09-16T01:40:23.707440+0000","last_undegraded":"2024-09-16T01:40:23.707440+0000","last_fullsized":"2024-09-16T01:40:23.707440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:00.678040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681164+0000","last_change":"2024-09-16T01:38:01.523064+0000","last_active":"2024-09-16T01:40:23.681164+0000","last_peered":"2024-09-16T01:40:23.681164+0000","last_clean":"2024-09-16T01:40:23.681164+0000","last_became_active":"2024-09-16T01:38:01.521695+0000","last_became_peered":"2024-09-16T01:38:01.521695+0000","last_unstale":"2024-09-16T01:40:23.681164+0000","last_undegraded":"2024-09-16T01:40:23.681164+0000","last_fullsized":"2024-09-16T01:40:23.681164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:28.467115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716031+0000","last_change":"2024-09-16T01:38:08.563557+0000","last_active":"2024-09-16T01:40:23.716031+0000","last_peered":"2024-09-16T01:40:23.716031+0000","last_clean":"2024-09-16T01:40:23.716031+0000","last_became_active":"2024-09-16T01:38:08.563065+0000","last_became_peered":"2024-09-16T01:38:08.563065+0000","last_unstale":"2024-09-16T01:40:23.716031+0000","last_undegraded":"2024-09-16T01:40:23.716031+0000","last_fullsized":"2024-09-16T01:40:23.716031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:56.992915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681383+0000","last_change":"2024-09-16T01:37:59.798884+0000","last_active":"2024-09-16T01:40:23.681383+0000","last_peered":"2024-09-16T01:40:23.681383+0000","last_clean":"2024-09-16T01:40:23.681383+0000","last_became_active":"2024-09-16T01:37:59.797989+0000","last_became_peered":"2024-09-16T01:37:59.797989+0000","last_unstale":"2024-09-16T01:40:23.681383+0000","last_undegraded":"2024-09-16T01:40:23.681383+0000","last_fullsized":"2024-09-16T01:40:23.681383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:13.633078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918394+0000","last_change":"2024-09-16T01:38:01.528391+0000","last_active":"2024-09-16T01:40:23.918394+0000","last_peered":"2024-09-16T01:40:23.918394+0000","last_clean":"2024-09-16T01:40:23.918394+0000","last_became_active":"2024-09-16T01:38:01.528076+0000","last_became_peered":"2024-09-16T01:38:01.528076+0000","last_unstale":"2024-09-16T01:40:23.918394+0000","last_undegraded":"2024-09-16T01:40:23.918394+0000","last_fullsized":"2024-09-16T01:40:23.918394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.105830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708846+0000","last_change":"2024-09-16T01:38:08.566198+0000","last_active":"2024-09-16T01:40:23.708846+0000","last_peered":"2024-09-16T01:40:23.708846+0000","last_clean":"2024-09-16T01:40:23.708846+0000","last_became_active":"2024-09-16T01:38:08.565739+0000","last_became_peered":"2024-09-16T01:38:08.565739+0000","last_unstale":"2024-09-16T01:40:23.708846+0000","last_undegraded":"2024-09-16T01:40:23.708846+0000","last_fullsized":"2024-09-16T01:40:23.708846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:05.172269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716538+0000","last_change":"2024-09-16T01:37:59.507625+0000","last_active":"2024-09-16T01:40:23.716538+0000","last_peered":"2024-09-16T01:40:23.716538+0000","last_clean":"2024-09-16T01:40:23.716538+0000","last_became_active":"2024-09-16T01:37:59.507280+0000","last_became_peered":"2024-09-16T01:37:59.507280+0000","last_unstale":"2024-09-16T01:40:23.716538+0000","last_undegraded":"2024-09-16T01:40:23.716538+0000","last_fullsized":"2024-09-16T01:40:23.716538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:29:37.761203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918422+0000","last_change":"2024-09-16T01:38:01.528406+0000","last_active":"2024-09-16T01:40:23.918422+0000","last_peered":"2024-09-16T01:40:23.918422+0000","last_clean":"2024-09-16T01:40:23.918422+0000","last_became_active":"2024-09-16T01:38:01.528084+0000","last_became_peered":"2024-09-16T01:38:01.528084+0000","last_unstale":"2024-09-16T01:40:23.918422+0000","last_undegraded":"2024-09-16T01:40:23.918422+0000","last_fullsized":"2024-09-16T01:40:23.918422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:40.766305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709276+0000","last_change":"2024-09-16T01:38:08.563645+0000","last_active":"2024-09-16T01:40:23.709276+0000","last_peered":"2024-09-16T01:40:23.709276+0000","last_clean":"2024-09-16T01:40:23.709276+0000","last_became_active":"2024-09-16T01:38:08.563149+0000","last_became_peered":"2024-09-16T01:38:08.563149+0000","last_unstale":"2024-09-16T01:40:23.709276+0000","last_undegraded":"2024-09-16T01:40:23.709276+0000","last_fullsized":"2024-09-16T01:40:23.709276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:09.010905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716807+0000","last_change":"2024-09-16T01:37:59.509649+0000","last_active":"2024-09-16T01:40:23.716807+0000","last_peered":"2024-09-16T01:40:23.716807+0000","last_clean":"2024-09-16T01:40:23.716807+0000","last_became_active":"2024-09-16T01:37:59.509236+0000","last_became_peered":"2024-09-16T01:37:59.509236+0000","last_unstale":"2024-09-16T01:40:23.716807+0000","last_undegraded":"2024-09-16T01:40:23.716807+0000","last_fullsized":"2024-09-16T01:40:23.716807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.614205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709245+0000","last_change":"2024-09-16T01:38:01.528074+0000","last_active":"2024-09-16T01:40:23.709245+0000","last_peered":"2024-09-16T01:40:23.709245+0000","last_clean":"2024-09-16T01:40:23.709245+0000","last_became_active":"2024-09-16T01:38:01.527737+0000","last_became_peered":"2024-09-16T01:38:01.527737+0000","last_unstale":"2024-09-16T01:40:23.709245+0000","last_undegraded":"2024-09-16T01:40:23.709245+0000","last_fullsized":"2024-09-16T01:40:23.709245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:37.592557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708957+0000","last_change":"2024-09-16T01:38:08.566340+0000","last_active":"2024-09-16T01:40:23.708957+0000","last_peered":"2024-09-16T01:40:23.708957+0000","last_clean":"2024-09-16T01:40:23.708957+0000","last_became_active":"2024-09-16T01:38:08.565791+0000","last_became_peered":"2024-09-16T01:38:08.565791+0000","last_unstale":"2024-09-16T01:40:23.708957+0000","last_undegraded":"2024-09-16T01:40:23.708957+0000","last_fullsized":"2024-09-16T01:40:23.708957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:35:31.845151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681718+0000","last_change":"2024-09-16T01:38:01.527747+0000","last_active":"2024-09-16T01:40:23.681718+0000","last_peered":"2024-09-16T01:40:23.681718+0000","last_clean":"2024-09-16T01:40:23.681718+0000","last_became_active":"2024-09-16T01:38:01.527325+0000","last_became_peered":"2024-09-16T01:38:01.527325+0000","last_unstale":"2024-09-16T01:40:23.681718+0000","last_undegraded":"2024-09-16T01:40:23.681718+0000","last_fullsized":"2024-09-16T01:40:23.681718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:27.203719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716467+0000","last_change":"2024-09-16T01:37:59.510298+0000","last_active":"2024-09-16T01:40:23.716467+0000","last_peered":"2024-09-16T01:40:23.716467+0000","last_clean":"2024-09-16T01:40:23.716467+0000","last_became_active":"2024-09-16T01:37:59.509952+0000","last_became_peered":"2024-09-16T01:37:59.509952+0000","last_unstale":"2024-09-16T01:40:23.716467+0000","last_undegraded":"2024-09-16T01:40:23.716467+0000","last_fullsized":"2024-09-16T01:40:23.716467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:11.616846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709120+0000","last_change":"2024-09-16T01:38:08.571622+0000","last_active":"2024-09-16T01:40:23.709120+0000","last_peered":"2024-09-16T01:40:23.709120+0000","last_clean":"2024-09-16T01:40:23.709120+0000","last_became_active":"2024-09-16T01:38:08.571125+0000","last_became_peered":"2024-09-16T01:38:08.571125+0000","last_unstale":"2024-09-16T01:40:23.709120+0000","last_undegraded":"2024-09-16T01:40:23.709120+0000","last_fullsized":"2024-09-16T01:40:23.709120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:00.154633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680985+0000","last_change":"2024-09-16T01:38:01.519681+0000","last_active":"2024-09-16T01:40:23.680985+0000","last_peered":"2024-09-16T01:40:23.680985+0000","last_clean":"2024-09-16T01:40:23.680985+0000","last_became_active":"2024-09-16T01:38:01.518310+0000","last_became_peered":"2024-09-16T01:38:01.518310+0000","last_unstale":"2024-09-16T01:40:23.680985+0000","last_undegraded":"2024-09-16T01:40:23.680985+0000","last_fullsized":"2024-09-16T01:40:23.680985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:23.308008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:24.556034+0000","last_change":"2024-09-16T01:38:08.559469+0000","last_active":"2024-09-16T01:40:24.556034+0000","last_peered":"2024-09-16T01:40:24.556034+0000","last_clean":"2024-09-16T01:40:24.556034+0000","last_became_active":"2024-09-16T01:38:08.559111+0000","last_became_peered":"2024-09-16T01:38:08.559111+0000","last_unstale":"2024-09-16T01:40:24.556034+0000","last_undegraded":"2024-09-16T01:40:24.556034+0000","last_fullsized":"2024-09-16T01:40:24.556034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:58.577749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709153+0000","last_change":"2024-09-16T01:38:08.555029+0000","last_active":"2024-09-16T01:40:23.709153+0000","last_peered":"2024-09-16T01:40:23.709153+0000","last_clean":"2024-09-16T01:40:23.709153+0000","last_became_active":"2024-09-16T01:38:08.554151+0000","last_became_peered":"2024-09-16T01:38:08.554151+0000","last_unstale":"2024-09-16T01:40:23.709153+0000","last_undegraded":"2024-09-16T01:40:23.709153+0000","last_fullsized":"2024-09-16T01:40:23.709153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:05.005904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682035+0000","last_change":"2024-09-16T01:38:01.528114+0000","last_active":"2024-09-16T01:40:23.682035+0000","last_peered":"2024-09-16T01:40:23.682035+0000","last_clean":"2024-09-16T01:40:23.682035+0000","last_became_active":"2024-09-16T01:38:01.527805+0000","last_became_peered":"2024-09-16T01:38:01.527805+0000","last_unstale":"2024-09-16T01:40:23.682035+0000","last_undegraded":"2024-09-16T01:40:23.682035+0000","last_fullsized":"2024-09-16T01:40:23.682035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:15.830936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680769+0000","last_change":"2024-09-16T01:37:59.507776+0000","last_active":"2024-09-16T01:40:23.680769+0000","last_peered":"2024-09-16T01:40:23.680769+0000","last_clean":"2024-09-16T01:40:23.680769+0000","last_became_active":"2024-09-16T01:37:59.507429+0000","last_became_peered":"2024-09-16T01:37:59.507429+0000","last_unstale":"2024-09-16T01:40:23.680769+0000","last_undegraded":"2024-09-16T01:40:23.680769+0000","last_fullsized":"2024-09-16T01:40:23.680769+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:44:41.274986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707084+0000","last_change":"2024-09-16T01:38:08.562191+0000","last_active":"2024-09-16T01:40:23.707084+0000","last_peered":"2024-09-16T01:40:23.707084+0000","last_clean":"2024-09-16T01:40:23.707084+0000","last_became_active":"2024-09-16T01:38:08.561839+0000","last_became_peered":"2024-09-16T01:38:08.561839+0000","last_unstale":"2024-09-16T01:40:23.707084+0000","last_undegraded":"2024-09-16T01:40:23.707084+0000","last_fullsized":"2024-09-16T01:40:23.707084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:40.618612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716123+0000","last_change":"2024-09-16T01:38:01.518855+0000","last_active":"2024-09-16T01:40:23.716123+0000","last_peered":"2024-09-16T01:40:23.716123+0000","last_clean":"2024-09-16T01:40:23.716123+0000","last_became_active":"2024-09-16T01:38:01.518427+0000","last_became_peered":"2024-09-16T01:38:01.518427+0000","last_unstale":"2024-09-16T01:40:23.716123+0000","last_undegraded":"2024-09-16T01:40:23.716123+0000","last_fullsized":"2024-09-16T01:40:23.716123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:23.054856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918053+0000","last_change":"2024-09-16T01:37:59.799341+0000","last_active":"2024-09-16T01:40:23.918053+0000","last_peered":"2024-09-16T01:40:23.918053+0000","last_clean":"2024-09-16T01:40:23.918053+0000","last_became_active":"2024-09-16T01:37:59.798992+0000","last_became_peered":"2024-09-16T01:37:59.798992+0000","last_unstale":"2024-09-16T01:40:23.918053+0000","last_undegraded":"2024-09-16T01:40:23.918053+0000","last_fullsized":"2024-09-16T01:40:23.918053+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:01.708565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709518+0000","last_change":"2024-09-16T01:39:09.524319+0000","last_active":"2024-09-16T01:40:23.709518+0000","last_peered":"2024-09-16T01:40:23.709518+0000","last_clean":"2024-09-16T01:40:23.709518+0000","last_became_active":"2024-09-16T01:39:09.523717+0000","last_became_peered":"2024-09-16T01:39:09.523717+0000","last_unstale":"2024-09-16T01:40:23.709518+0000","last_undegraded":"2024-09-16T01:40:23.709518+0000","last_fullsized":"2024-09-16T01:40:23.709518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:35.488097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918444+0000","last_change":"2024-09-16T01:38:01.524545+0000","last_active":"2024-09-16T01:40:23.918444+0000","last_peered":"2024-09-16T01:40:23.918444+0000","last_clean":"2024-09-16T01:40:23.918444+0000","last_became_active":"2024-09-16T01:38:01.524253+0000","last_became_peered":"2024-09-16T01:38:01.524253+0000","last_unstale":"2024-09-16T01:40:23.918444+0000","last_undegraded":"2024-09-16T01:40:23.918444+0000","last_fullsized":"2024-09-16T01:40:23.918444+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:08.697500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709429+0000","last_change":"2024-09-16T01:37:59.506779+0000","last_active":"2024-09-16T01:40:23.709429+0000","last_peered":"2024-09-16T01:40:23.709429+0000","last_clean":"2024-09-16T01:40:23.709429+0000","last_became_active":"2024-09-16T01:37:59.505459+0000","last_became_peered":"2024-09-16T01:37:59.505459+0000","last_unstale":"2024-09-16T01:40:23.709429+0000","last_undegraded":"2024-09-16T01:40:23.709429+0000","last_fullsized":"2024-09-16T01:40:23.709429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:12.973330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716178+0000","last_change":"2024-09-16T01:38:08.557250+0000","last_active":"2024-09-16T01:40:23.716178+0000","last_peered":"2024-09-16T01:40:23.716178+0000","last_clean":"2024-09-16T01:40:23.716178+0000","last_became_active":"2024-09-16T01:38:08.555577+0000","last_became_peered":"2024-09-16T01:38:08.555577+0000","last_unstale":"2024-09-16T01:40:23.716178+0000","last_undegraded":"2024-09-16T01:40:23.716178+0000","last_fullsized":"2024-09-16T01:40:23.716178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:54.965341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681174+0000","last_change":"2024-09-16T01:38:01.522910+0000","last_active":"2024-09-16T01:40:23.681174+0000","last_peered":"2024-09-16T01:40:23.681174+0000","last_clean":"2024-09-16T01:40:23.681174+0000","last_became_active":"2024-09-16T01:38:01.521206+0000","last_became_peered":"2024-09-16T01:38:01.521206+0000","last_unstale":"2024-09-16T01:40:23.681174+0000","last_undegraded":"2024-09-16T01:40:23.681174+0000","last_fullsized":"2024-09-16T01:40:23.681174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:45.670464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'52","reported_seq":111,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.879588+0000","last_change":"2024-09-16T01:37:59.505962+0000","last_active":"2024-09-16T01:40:23.879588+0000","last_peered":"2024-09-16T01:40:23.879588+0000","last_clean":"2024-09-16T01:40:23.879588+0000","last_became_active":"2024-09-16T01:37:59.505636+0000","last_became_peered":"2024-09-16T01:37:59.505636+0000","last_unstale":"2024-09-16T01:40:23.879588+0000","last_undegraded":"2024-09-16T01:40:23.879588+0000","last_fullsized":"2024-09-16T01:40:23.879588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:57.548513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680470+0000","last_change":"2024-09-16T01:38:08.571249+0000","last_active":"2024-09-16T01:40:23.680470+0000","last_peered":"2024-09-16T01:40:23.680470+0000","last_clean":"2024-09-16T01:40:23.680470+0000","last_became_active":"2024-09-16T01:38:08.570388+0000","last_became_peered":"2024-09-16T01:38:08.570388+0000","last_unstale":"2024-09-16T01:40:23.680470+0000","last_undegraded":"2024-09-16T01:40:23.680470+0000","last_fullsized":"2024-09-16T01:40:23.680470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:59.904704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706816+0000","last_change":"2024-09-16T01:38:01.522040+0000","last_active":"2024-09-16T01:40:23.706816+0000","last_peered":"2024-09-16T01:40:23.706816+0000","last_clean":"2024-09-16T01:40:23.706816+0000","last_became_active":"2024-09-16T01:38:01.521720+0000","last_became_peered":"2024-09-16T01:38:01.521720+0000","last_unstale":"2024-09-16T01:40:23.706816+0000","last_undegraded":"2024-09-16T01:40:23.706816+0000","last_fullsized":"2024-09-16T01:40:23.706816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:44:48.635603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682302+0000","last_change":"2024-09-16T01:38:08.561870+0000","last_active":"2024-09-16T01:40:23.682302+0000","last_peered":"2024-09-16T01:40:23.682302+0000","last_clean":"2024-09-16T01:40:23.682302+0000","last_became_active":"2024-09-16T01:38:08.561440+0000","last_became_peered":"2024-09-16T01:38:08.561440+0000","last_unstale":"2024-09-16T01:40:23.682302+0000","last_undegraded":"2024-09-16T01:40:23.682302+0000","last_fullsized":"2024-09-16T01:40:23.682302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:43:37.594734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682173+0000","last_change":"2024-09-16T01:38:08.555436+0000","last_active":"2024-09-16T01:40:23.682173+0000","last_peered":"2024-09-16T01:40:23.682173+0000","last_clean":"2024-09-16T01:40:23.682173+0000","last_became_active":"2024-09-16T01:38:08.555026+0000","last_became_peered":"2024-09-16T01:38:08.555026+0000","last_unstale":"2024-09-16T01:40:23.682173+0000","last_undegraded":"2024-09-16T01:40:23.682173+0000","last_fullsized":"2024-09-16T01:40:23.682173+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:28.870214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"85'4","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706753+0000","last_change":"2024-09-16T01:38:01.524482+0000","last_active":"2024-09-16T01:40:23.706753+0000","last_peered":"2024-09-16T01:40:23.706753+0000","last_clean":"2024-09-16T01:40:23.706753+0000","last_became_active":"2024-09-16T01:38:01.524160+0000","last_became_peered":"2024-09-16T01:38:01.524160+0000","last_unstale":"2024-09-16T01:40:23.706753+0000","last_undegraded":"2024-09-16T01:40:23.706753+0000","last_fullsized":"2024-09-16T01:40:23.706753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:39:27.621780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706766+0000","last_change":"2024-09-16T01:37:59.800051+0000","last_active":"2024-09-16T01:40:23.706766+0000","last_peered":"2024-09-16T01:40:23.706766+0000","last_clean":"2024-09-16T01:40:23.706766+0000","last_became_active":"2024-09-16T01:37:59.799606+0000","last_became_peered":"2024-09-16T01:37:59.799606+0000","last_unstale":"2024-09-16T01:40:23.706766+0000","last_undegraded":"2024-09-16T01:40:23.706766+0000","last_fullsized":"2024-09-16T01:40:23.706766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:02.694933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716183+0000","last_change":"2024-09-16T01:38:08.570764+0000","last_active":"2024-09-16T01:40:23.716183+0000","last_peered":"2024-09-16T01:40:23.716183+0000","last_clean":"2024-09-16T01:40:23.716183+0000","last_became_active":"2024-09-16T01:38:08.570385+0000","last_became_peered":"2024-09-16T01:38:08.570385+0000","last_unstale":"2024-09-16T01:40:23.716183+0000","last_undegraded":"2024-09-16T01:40:23.716183+0000","last_fullsized":"2024-09-16T01:40:23.716183+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:25.349671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706565+0000","last_change":"2024-09-16T01:38:01.519590+0000","last_active":"2024-09-16T01:40:23.706565+0000","last_peered":"2024-09-16T01:40:23.706565+0000","last_clean":"2024-09-16T01:40:23.706565+0000","last_became_active":"2024-09-16T01:38:01.519210+0000","last_became_peered":"2024-09-16T01:38:01.519210+0000","last_unstale":"2024-09-16T01:40:23.706565+0000","last_undegraded":"2024-09-16T01:40:23.706565+0000","last_fullsized":"2024-09-16T01:40:23.706565+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:04.228131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707867+0000","last_change":"2024-09-16T01:37:59.505473+0000","last_active":"2024-09-16T01:40:23.707867+0000","last_peered":"2024-09-16T01:40:23.707867+0000","last_clean":"2024-09-16T01:40:23.707867+0000","last_became_active":"2024-09-16T01:37:59.505123+0000","last_became_peered":"2024-09-16T01:37:59.505123+0000","last_unstale":"2024-09-16T01:40:23.707867+0000","last_undegraded":"2024-09-16T01:40:23.707867+0000","last_fullsized":"2024-09-16T01:40:23.707867+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:17.739625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706690+0000","last_change":"2024-09-16T01:38:08.558534+0000","last_active":"2024-09-16T01:40:23.706690+0000","last_peered":"2024-09-16T01:40:23.706690+0000","last_clean":"2024-09-16T01:40:23.706690+0000","last_became_active":"2024-09-16T01:38:08.558185+0000","last_became_peered":"2024-09-16T01:38:08.558185+0000","last_unstale":"2024-09-16T01:40:23.706690+0000","last_undegraded":"2024-09-16T01:40:23.706690+0000","last_fullsized":"2024-09-16T01:40:23.706690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:09.359426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708932+0000","last_change":"2024-09-16T01:38:01.521445+0000","last_active":"2024-09-16T01:40:23.708932+0000","last_peered":"2024-09-16T01:40:23.708932+0000","last_clean":"2024-09-16T01:40:23.708932+0000","last_became_active":"2024-09-16T01:38:01.521140+0000","last_became_peered":"2024-09-16T01:38:01.521140+0000","last_unstale":"2024-09-16T01:40:23.708932+0000","last_undegraded":"2024-09-16T01:40:23.708932+0000","last_fullsized":"2024-09-16T01:40:23.708932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:05.247515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":147,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707420+0000","last_change":"2024-09-16T01:37:59.798365+0000","last_active":"2024-09-16T01:40:23.707420+0000","last_peered":"2024-09-16T01:40:23.707420+0000","last_clean":"2024-09-16T01:40:23.707420+0000","last_became_active":"2024-09-16T01:37:59.798023+0000","last_became_peered":"2024-09-16T01:37:59.798023+0000","last_unstale":"2024-09-16T01:40:23.707420+0000","last_undegraded":"2024-09-16T01:40:23.707420+0000","last_fullsized":"2024-09-16T01:40:23.707420+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:30.867080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707046+0000","last_change":"2024-09-16T01:38:08.571704+0000","last_active":"2024-09-16T01:40:23.707046+0000","last_peered":"2024-09-16T01:40:23.707046+0000","last_clean":"2024-09-16T01:40:23.707046+0000","last_became_active":"2024-09-16T01:38:08.571350+0000","last_became_peered":"2024-09-16T01:38:08.571350+0000","last_unstale":"2024-09-16T01:40:23.707046+0000","last_undegraded":"2024-09-16T01:40:23.707046+0000","last_fullsized":"2024-09-16T01:40:23.707046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:01.698770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706388+0000","last_change":"2024-09-16T01:38:08.559433+0000","last_active":"2024-09-16T01:40:23.706388+0000","last_peered":"2024-09-16T01:40:23.706388+0000","last_clean":"2024-09-16T01:40:23.706388+0000","last_became_active":"2024-09-16T01:38:08.559110+0000","last_became_peered":"2024-09-16T01:38:08.559110+0000","last_unstale":"2024-09-16T01:40:23.706388+0000","last_undegraded":"2024-09-16T01:40:23.706388+0000","last_fullsized":"2024-09-16T01:40:23.706388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:39.001916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681418+0000","last_change":"2024-09-16T01:37:59.508236+0000","last_active":"2024-09-16T01:40:23.681418+0000","last_peered":"2024-09-16T01:40:23.681418+0000","last_clean":"2024-09-16T01:40:23.681418+0000","last_became_active":"2024-09-16T01:37:59.507883+0000","last_became_peered":"2024-09-16T01:37:59.507883+0000","last_unstale":"2024-09-16T01:40:23.681418+0000","last_undegraded":"2024-09-16T01:40:23.681418+0000","last_fullsized":"2024-09-16T01:40:23.681418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:23.948530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709040+0000","last_change":"2024-09-16T01:38:08.572108+0000","last_active":"2024-09-16T01:40:23.709040+0000","last_peered":"2024-09-16T01:40:23.709040+0000","last_clean":"2024-09-16T01:40:23.709040+0000","last_became_active":"2024-09-16T01:38:08.571643+0000","last_became_peered":"2024-09-16T01:38:08.571643+0000","last_unstale":"2024-09-16T01:40:23.709040+0000","last_undegraded":"2024-09-16T01:40:23.709040+0000","last_fullsized":"2024-09-16T01:40:23.709040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:46.636219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680849+0000","last_change":"2024-09-16T01:38:01.517894+0000","last_active":"2024-09-16T01:40:23.680849+0000","last_peered":"2024-09-16T01:40:23.680849+0000","last_clean":"2024-09-16T01:40:23.680849+0000","last_became_active":"2024-09-16T01:38:01.517602+0000","last_became_peered":"2024-09-16T01:38:01.517602+0000","last_unstale":"2024-09-16T01:40:23.680849+0000","last_undegraded":"2024-09-16T01:40:23.680849+0000","last_fullsized":"2024-09-16T01:40:23.680849+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:33.483120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709294+0000","last_change":"2024-09-16T01:37:59.507568+0000","last_active":"2024-09-16T01:40:23.709294+0000","last_peered":"2024-09-16T01:40:23.709294+0000","last_clean":"2024-09-16T01:40:23.709294+0000","last_became_active":"2024-09-16T01:37:59.505798+0000","last_became_peered":"2024-09-16T01:37:59.505798+0000","last_unstale":"2024-09-16T01:40:23.709294+0000","last_undegraded":"2024-09-16T01:40:23.709294+0000","last_fullsized":"2024-09-16T01:40:23.709294+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:19:02.711054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":360,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681514+0000","last_change":"2024-09-16T01:36:46.744332+0000","last_active":"2024-09-16T01:40:23.681514+0000","last_peered":"2024-09-16T01:40:23.681514+0000","last_clean":"2024-09-16T01:40:23.681514+0000","last_became_active":"2024-09-16T01:36:46.743902+0000","last_became_peered":"2024-09-16T01:36:46.743902+0000","last_unstale":"2024-09-16T01:40:23.681514+0000","last_undegraded":"2024-09-16T01:40:23.681514+0000","last_fullsized":"2024-09-16T01:40:23.681514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:33:19.891854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:33:19.891854+0000","last_clean_scrub_stamp":"2024-09-16T01:33:19.891854+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:46.173685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681871+0000","last_change":"2024-09-16T01:39:09.528216+0000","last_active":"2024-09-16T01:40:23.681871+0000","last_peered":"2024-09-16T01:40:23.681871+0000","last_clean":"2024-09-16T01:40:23.681871+0000","last_became_active":"2024-09-16T01:39:09.523634+0000","last_became_peered":"2024-09-16T01:39:09.523634+0000","last_unstale":"2024-09-16T01:40:23.681871+0000","last_undegraded":"2024-09-16T01:40:23.681871+0000","last_fullsized":"2024-09-16T01:40:23.681871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:27.989901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682064+0000","last_change":"2024-09-16T01:38:01.527690+0000","last_active":"2024-09-16T01:40:23.682064+0000","last_peered":"2024-09-16T01:40:23.682064+0000","last_clean":"2024-09-16T01:40:23.682064+0000","last_became_active":"2024-09-16T01:38:01.527339+0000","last_became_peered":"2024-09-16T01:38:01.527339+0000","last_unstale":"2024-09-16T01:40:23.682064+0000","last_undegraded":"2024-09-16T01:40:23.682064+0000","last_fullsized":"2024-09-16T01:40:23.682064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:58.526270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'8","reported_seq":153,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.717334+0000","last_change":"2024-09-16T01:37:59.798874+0000","last_active":"2024-09-16T01:40:23.717334+0000","last_peered":"2024-09-16T01:40:23.717334+0000","last_clean":"2024-09-16T01:40:23.717334+0000","last_became_active":"2024-09-16T01:37:59.798424+0000","last_became_peered":"2024-09-16T01:37:59.798424+0000","last_unstale":"2024-09-16T01:40:23.717334+0000","last_undegraded":"2024-09-16T01:40:23.717334+0000","last_fullsized":"2024-09-16T01:40:23.717334+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:36.648963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918289+0000","last_change":"2024-09-16T01:38:08.555524+0000","last_active":"2024-09-16T01:40:23.918289+0000","last_peered":"2024-09-16T01:40:23.918289+0000","last_clean":"2024-09-16T01:40:23.918289+0000","last_became_active":"2024-09-16T01:38:08.555075+0000","last_became_peered":"2024-09-16T01:38:08.555075+0000","last_unstale":"2024-09-16T01:40:23.918289+0000","last_undegraded":"2024-09-16T01:40:23.918289+0000","last_fullsized":"2024-09-16T01:40:23.918289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:44.847916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681953+0000","last_change":"2024-09-16T01:38:01.528869+0000","last_active":"2024-09-16T01:40:23.681953+0000","last_peered":"2024-09-16T01:40:23.681953+0000","last_clean":"2024-09-16T01:40:23.681953+0000","last_became_active":"2024-09-16T01:38:01.528559+0000","last_became_peered":"2024-09-16T01:38:01.528559+0000","last_unstale":"2024-09-16T01:40:23.681953+0000","last_undegraded":"2024-09-16T01:40:23.681953+0000","last_fullsized":"2024-09-16T01:40:23.681953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:21.547808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'41","reported_seq":86,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.876871+0000","last_change":"2024-09-16T01:37:59.510871+0000","last_active":"2024-09-16T01:40:23.876871+0000","last_peered":"2024-09-16T01:40:23.876871+0000","last_clean":"2024-09-16T01:40:23.876871+0000","last_became_active":"2024-09-16T01:37:59.509996+0000","last_became_peered":"2024-09-16T01:37:59.509996+0000","last_unstale":"2024-09-16T01:40:23.876871+0000","last_undegraded":"2024-09-16T01:40:23.876871+0000","last_fullsized":"2024-09-16T01:40:23.876871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:23.390864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918027+0000","last_change":"2024-09-16T01:38:08.555181+0000","last_active":"2024-09-16T01:40:23.918027+0000","last_peered":"2024-09-16T01:40:23.918027+0000","last_clean":"2024-09-16T01:40:23.918027+0000","last_became_active":"2024-09-16T01:38:08.554778+0000","last_became_peered":"2024-09-16T01:38:08.554778+0000","last_unstale":"2024-09-16T01:40:23.918027+0000","last_undegraded":"2024-09-16T01:40:23.918027+0000","last_fullsized":"2024-09-16T01:40:23.918027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:16:42.369236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682039+0000","last_change":"2024-09-16T01:38:01.526260+0000","last_active":"2024-09-16T01:40:23.682039+0000","last_peered":"2024-09-16T01:40:23.682039+0000","last_clean":"2024-09-16T01:40:23.682039+0000","last_became_active":"2024-09-16T01:38:01.525652+0000","last_became_peered":"2024-09-16T01:38:01.525652+0000","last_unstale":"2024-09-16T01:40:23.682039+0000","last_undegraded":"2024-09-16T01:40:23.682039+0000","last_fullsized":"2024-09-16T01:40:23.682039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:09.180507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707015+0000","last_change":"2024-09-16T01:38:08.570408+0000","last_active":"2024-09-16T01:40:23.707015+0000","last_peered":"2024-09-16T01:40:23.707015+0000","last_clean":"2024-09-16T01:40:23.707015+0000","last_became_active":"2024-09-16T01:38:08.566122+0000","last_became_peered":"2024-09-16T01:38:08.566122+0000","last_unstale":"2024-09-16T01:40:23.707015+0000","last_undegraded":"2024-09-16T01:40:23.707015+0000","last_fullsized":"2024-09-16T01:40:23.707015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:55.832320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708804+0000","last_change":"2024-09-16T01:38:08.555063+0000","last_active":"2024-09-16T01:40:23.708804+0000","last_peered":"2024-09-16T01:40:23.708804+0000","last_clean":"2024-09-16T01:40:23.708804+0000","last_became_active":"2024-09-16T01:38:08.554665+0000","last_became_peered":"2024-09-16T01:38:08.554665+0000","last_unstale":"2024-09-16T01:40:23.708804+0000","last_undegraded":"2024-09-16T01:40:23.708804+0000","last_fullsized":"2024-09-16T01:40:23.708804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:50.091017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682633+0000","last_change":"2024-09-16T01:38:01.524247+0000","last_active":"2024-09-16T01:40:23.682633+0000","last_peered":"2024-09-16T01:40:23.682633+0000","last_clean":"2024-09-16T01:40:23.682633+0000","last_became_active":"2024-09-16T01:38:01.523911+0000","last_became_peered":"2024-09-16T01:38:01.523911+0000","last_unstale":"2024-09-16T01:40:23.682633+0000","last_undegraded":"2024-09-16T01:40:23.682633+0000","last_fullsized":"2024-09-16T01:40:23.682633+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:32.745882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716800+0000","last_change":"2024-09-16T01:37:59.511272+0000","last_active":"2024-09-16T01:40:23.716800+0000","last_peered":"2024-09-16T01:40:23.716800+0000","last_clean":"2024-09-16T01:40:23.716800+0000","last_became_active":"2024-09-16T01:37:59.510920+0000","last_became_peered":"2024-09-16T01:37:59.510920+0000","last_unstale":"2024-09-16T01:40:23.716800+0000","last_undegraded":"2024-09-16T01:40:23.716800+0000","last_fullsized":"2024-09-16T01:40:23.716800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:59.406175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708902+0000","last_change":"2024-09-16T01:38:08.567173+0000","last_active":"2024-09-16T01:40:23.708902+0000","last_peered":"2024-09-16T01:40:23.708902+0000","last_clean":"2024-09-16T01:40:23.708902+0000","last_became_active":"2024-09-16T01:38:08.566788+0000","last_became_peered":"2024-09-16T01:38:08.566788+0000","last_unstale":"2024-09-16T01:40:23.708902+0000","last_undegraded":"2024-09-16T01:40:23.708902+0000","last_fullsized":"2024-09-16T01:40:23.708902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:35.743356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709105+0000","last_change":"2024-09-16T01:38:01.529529+0000","last_active":"2024-09-16T01:40:23.709105+0000","last_peered":"2024-09-16T01:40:23.709105+0000","last_clean":"2024-09-16T01:40:23.709105+0000","last_became_active":"2024-09-16T01:38:01.529048+0000","last_became_peered":"2024-09-16T01:38:01.529048+0000","last_unstale":"2024-09-16T01:40:23.709105+0000","last_undegraded":"2024-09-16T01:40:23.709105+0000","last_fullsized":"2024-09-16T01:40:23.709105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:35.041662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716379+0000","last_change":"2024-09-16T01:37:59.799536+0000","last_active":"2024-09-16T01:40:23.716379+0000","last_peered":"2024-09-16T01:40:23.716379+0000","last_clean":"2024-09-16T01:40:23.716379+0000","last_became_active":"2024-09-16T01:37:59.799109+0000","last_became_peered":"2024-09-16T01:37:59.799109+0000","last_unstale":"2024-09-16T01:40:23.716379+0000","last_undegraded":"2024-09-16T01:40:23.716379+0000","last_fullsized":"2024-09-16T01:40:23.716379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:07:25.007434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707289+0000","last_change":"2024-09-16T01:38:08.567019+0000","last_active":"2024-09-16T01:40:23.707289+0000","last_peered":"2024-09-16T01:40:23.707289+0000","last_clean":"2024-09-16T01:40:23.707289+0000","last_became_active":"2024-09-16T01:38:08.566667+0000","last_became_peered":"2024-09-16T01:38:08.566667+0000","last_unstale":"2024-09-16T01:40:23.707289+0000","last_undegraded":"2024-09-16T01:40:23.707289+0000","last_fullsized":"2024-09-16T01:40:23.707289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:11.375095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681386+0000","last_change":"2024-09-16T01:38:01.527965+0000","last_active":"2024-09-16T01:40:23.681386+0000","last_peered":"2024-09-16T01:40:23.681386+0000","last_clean":"2024-09-16T01:40:23.681386+0000","last_became_active":"2024-09-16T01:38:01.526688+0000","last_became_peered":"2024-09-16T01:38:01.526688+0000","last_unstale":"2024-09-16T01:40:23.681386+0000","last_undegraded":"2024-09-16T01:40:23.681386+0000","last_fullsized":"2024-09-16T01:40:23.681386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:06.618756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":149,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.923723+0000","last_change":"2024-09-16T01:37:59.510331+0000","last_active":"2024-09-16T01:40:23.923723+0000","last_peered":"2024-09-16T01:40:23.923723+0000","last_clean":"2024-09-16T01:40:23.923723+0000","last_became_active":"2024-09-16T01:37:59.509769+0000","last_became_peered":"2024-09-16T01:37:59.509769+0000","last_unstale":"2024-09-16T01:40:23.923723+0000","last_undegraded":"2024-09-16T01:40:23.923723+0000","last_fullsized":"2024-09-16T01:40:23.923723+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:18.843091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716548+0000","last_change":"2024-09-16T01:39:10.049460+0000","last_active":"2024-09-16T01:40:23.716548+0000","last_peered":"2024-09-16T01:40:23.716548+0000","last_clean":"2024-09-16T01:40:23.716548+0000","last_became_active":"2024-09-16T01:39:10.048862+0000","last_became_peered":"2024-09-16T01:39:10.048862+0000","last_unstale":"2024-09-16T01:40:23.716548+0000","last_undegraded":"2024-09-16T01:40:23.716548+0000","last_fullsized":"2024-09-16T01:40:23.716548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:37.329806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681820+0000","last_change":"2024-09-16T01:38:01.521254+0000","last_active":"2024-09-16T01:40:23.681820+0000","last_peered":"2024-09-16T01:40:23.681820+0000","last_clean":"2024-09-16T01:40:23.681820+0000","last_became_active":"2024-09-16T01:38:01.520901+0000","last_became_peered":"2024-09-16T01:38:01.520901+0000","last_unstale":"2024-09-16T01:40:23.681820+0000","last_undegraded":"2024-09-16T01:40:23.681820+0000","last_fullsized":"2024-09-16T01:40:23.681820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:27.806954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709051+0000","last_change":"2024-09-16T01:37:59.505990+0000","last_active":"2024-09-16T01:40:23.709051+0000","last_peered":"2024-09-16T01:40:23.709051+0000","last_clean":"2024-09-16T01:40:23.709051+0000","last_became_active":"2024-09-16T01:37:59.505564+0000","last_became_peered":"2024-09-16T01:37:59.505564+0000","last_unstale":"2024-09-16T01:40:23.709051+0000","last_undegraded":"2024-09-16T01:40:23.709051+0000","last_fullsized":"2024-09-16T01:40:23.709051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:01.496609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682127+0000","last_change":"2024-09-16T01:38:08.562838+0000","last_active":"2024-09-16T01:40:23.682127+0000","last_peered":"2024-09-16T01:40:23.682127+0000","last_clean":"2024-09-16T01:40:23.682127+0000","last_became_active":"2024-09-16T01:38:08.562426+0000","last_became_peered":"2024-09-16T01:38:08.562426+0000","last_unstale":"2024-09-16T01:40:23.682127+0000","last_undegraded":"2024-09-16T01:40:23.682127+0000","last_fullsized":"2024-09-16T01:40:23.682127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:20.685626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707454+0000","last_change":"2024-09-16T01:38:01.514983+0000","last_active":"2024-09-16T01:40:23.707454+0000","last_peered":"2024-09-16T01:40:23.707454+0000","last_clean":"2024-09-16T01:40:23.707454+0000","last_became_active":"2024-09-16T01:38:01.514648+0000","last_became_peered":"2024-09-16T01:38:01.514648+0000","last_unstale":"2024-09-16T01:40:23.707454+0000","last_undegraded":"2024-09-16T01:40:23.707454+0000","last_fullsized":"2024-09-16T01:40:23.707454+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:57.052564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.717151+0000","last_change":"2024-09-16T01:37:59.505141+0000","last_active":"2024-09-16T01:40:23.717151+0000","last_peered":"2024-09-16T01:40:23.717151+0000","last_clean":"2024-09-16T01:40:23.717151+0000","last_became_active":"2024-09-16T01:37:59.504730+0000","last_became_peered":"2024-09-16T01:37:59.504730+0000","last_unstale":"2024-09-16T01:40:23.717151+0000","last_undegraded":"2024-09-16T01:40:23.717151+0000","last_fullsized":"2024-09-16T01:40:23.717151+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:54.504661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709184+0000","last_change":"2024-09-16T01:38:08.563970+0000","last_active":"2024-09-16T01:40:23.709184+0000","last_peered":"2024-09-16T01:40:23.709184+0000","last_clean":"2024-09-16T01:40:23.709184+0000","last_became_active":"2024-09-16T01:38:08.563555+0000","last_became_peered":"2024-09-16T01:38:08.563555+0000","last_unstale":"2024-09-16T01:40:23.709184+0000","last_undegraded":"2024-09-16T01:40:23.709184+0000","last_fullsized":"2024-09-16T01:40:23.709184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:27:12.086043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716549+0000","last_change":"2024-09-16T01:38:01.521016+0000","last_active":"2024-09-16T01:40:23.716549+0000","last_peered":"2024-09-16T01:40:23.716549+0000","last_clean":"2024-09-16T01:40:23.716549+0000","last_became_active":"2024-09-16T01:38:01.520656+0000","last_became_peered":"2024-09-16T01:38:01.520656+0000","last_unstale":"2024-09-16T01:40:23.716549+0000","last_undegraded":"2024-09-16T01:40:23.716549+0000","last_fullsized":"2024-09-16T01:40:23.716549+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:39.001239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'41","reported_seq":297,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.923336+0000","last_change":"2024-09-16T01:37:59.507235+0000","last_active":"2024-09-16T01:40:23.923336+0000","last_peered":"2024-09-16T01:40:23.923336+0000","last_clean":"2024-09-16T01:40:23.923336+0000","last_became_active":"2024-09-16T01:37:59.506255+0000","last_became_peered":"2024-09-16T01:37:59.506255+0000","last_unstale":"2024-09-16T01:40:23.923336+0000","last_undegraded":"2024-09-16T01:40:23.923336+0000","last_fullsized":"2024-09-16T01:40:23.923336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:44.515663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68792,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":219,"num_read_kb":171,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706438+0000","last_change":"2024-09-16T01:39:09.522688+0000","last_active":"2024-09-16T01:40:23.706438+0000","last_peered":"2024-09-16T01:40:23.706438+0000","last_clean":"2024-09-16T01:40:23.706438+0000","last_became_active":"2024-09-16T01:39:09.522137+0000","last_became_peered":"2024-09-16T01:39:09.522137+0000","last_unstale":"2024-09-16T01:40:23.706438+0000","last_undegraded":"2024-09-16T01:40:23.706438+0000","last_fullsized":"2024-09-16T01:40:23.706438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.004315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918498+0000","last_change":"2024-09-16T01:38:01.526641+0000","last_active":"2024-09-16T01:40:23.918498+0000","last_peered":"2024-09-16T01:40:23.918498+0000","last_clean":"2024-09-16T01:40:23.918498+0000","last_became_active":"2024-09-16T01:38:01.526278+0000","last_became_peered":"2024-09-16T01:38:01.526278+0000","last_unstale":"2024-09-16T01:40:23.918498+0000","last_undegraded":"2024-09-16T01:40:23.918498+0000","last_fullsized":"2024-09-16T01:40:23.918498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:56.156593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709392+0000","last_change":"2024-09-16T01:37:59.800065+0000","last_active":"2024-09-16T01:40:23.709392+0000","last_peered":"2024-09-16T01:40:23.709392+0000","last_clean":"2024-09-16T01:40:23.709392+0000","last_became_active":"2024-09-16T01:37:59.799705+0000","last_became_peered":"2024-09-16T01:37:59.799705+0000","last_unstale":"2024-09-16T01:40:23.709392+0000","last_undegraded":"2024-09-16T01:40:23.709392+0000","last_fullsized":"2024-09-16T01:40:23.709392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:13.791875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709196+0000","last_change":"2024-09-16T01:38:08.556360+0000","last_active":"2024-09-16T01:40:23.709196+0000","last_peered":"2024-09-16T01:40:23.709196+0000","last_clean":"2024-09-16T01:40:23.709196+0000","last_became_active":"2024-09-16T01:38:08.555219+0000","last_became_peered":"2024-09-16T01:38:08.555219+0000","last_unstale":"2024-09-16T01:40:23.709196+0000","last_undegraded":"2024-09-16T01:40:23.709196+0000","last_fullsized":"2024-09-16T01:40:23.709196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:36.464760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918517+0000","last_change":"2024-09-16T01:38:01.526758+0000","last_active":"2024-09-16T01:40:23.918517+0000","last_peered":"2024-09-16T01:40:23.918517+0000","last_clean":"2024-09-16T01:40:23.918517+0000","last_became_active":"2024-09-16T01:38:01.526545+0000","last_became_peered":"2024-09-16T01:38:01.526545+0000","last_unstale":"2024-09-16T01:40:23.918517+0000","last_undegraded":"2024-09-16T01:40:23.918517+0000","last_fullsized":"2024-09-16T01:40:23.918517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:47.185489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682092+0000","last_change":"2024-09-16T01:38:08.555125+0000","last_active":"2024-09-16T01:40:23.682092+0000","last_peered":"2024-09-16T01:40:23.682092+0000","last_clean":"2024-09-16T01:40:23.682092+0000","last_became_active":"2024-09-16T01:38:08.554756+0000","last_became_peered":"2024-09-16T01:38:08.554756+0000","last_unstale":"2024-09-16T01:40:23.682092+0000","last_undegraded":"2024-09-16T01:40:23.682092+0000","last_fullsized":"2024-09-16T01:40:23.682092+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:08.888006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.705898+0000","last_change":"2024-09-16T01:38:08.557906+0000","last_active":"2024-09-16T01:40:23.705898+0000","last_peered":"2024-09-16T01:40:23.705898+0000","last_clean":"2024-09-16T01:40:23.705898+0000","last_became_active":"2024-09-16T01:38:08.557546+0000","last_became_peered":"2024-09-16T01:38:08.557546+0000","last_unstale":"2024-09-16T01:40:23.705898+0000","last_undegraded":"2024-09-16T01:40:23.705898+0000","last_fullsized":"2024-09-16T01:40:23.705898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:27:04.988778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918558+0000","last_change":"2024-09-16T01:38:01.528430+0000","last_active":"2024-09-16T01:40:23.918558+0000","last_peered":"2024-09-16T01:40:23.918558+0000","last_clean":"2024-09-16T01:40:23.918558+0000","last_became_active":"2024-09-16T01:38:01.528069+0000","last_became_peered":"2024-09-16T01:38:01.528069+0000","last_unstale":"2024-09-16T01:40:23.918558+0000","last_undegraded":"2024-09-16T01:40:23.918558+0000","last_fullsized":"2024-09-16T01:40:23.918558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:41.317215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'6","reported_seq":251,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.916402+0000","last_change":"2024-09-16T01:37:59.509017+0000","last_active":"2024-09-16T01:40:23.916402+0000","last_peered":"2024-09-16T01:40:23.916402+0000","last_clean":"2024-09-16T01:40:23.916402+0000","last_became_active":"2024-09-16T01:37:59.508252+0000","last_became_peered":"2024-09-16T01:37:59.508252+0000","last_unstale":"2024-09-16T01:40:23.916402+0000","last_undegraded":"2024-09-16T01:40:23.916402+0000","last_fullsized":"2024-09-16T01:40:23.916402+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:26.215407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":208,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680587+0000","last_change":"2024-09-16T01:38:08.565197+0000","last_active":"2024-09-16T01:40:23.680587+0000","last_peered":"2024-09-16T01:40:23.680587+0000","last_clean":"2024-09-16T01:40:23.680587+0000","last_became_active":"2024-09-16T01:38:08.562773+0000","last_became_peered":"2024-09-16T01:38:08.562773+0000","last_unstale":"2024-09-16T01:40:23.680587+0000","last_undegraded":"2024-09-16T01:40:23.680587+0000","last_fullsized":"2024-09-16T01:40:23.680587+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:13.316950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918230+0000","last_change":"2024-09-16T01:38:08.554052+0000","last_active":"2024-09-16T01:40:23.918230+0000","last_peered":"2024-09-16T01:40:23.918230+0000","last_clean":"2024-09-16T01:40:23.918230+0000","last_became_active":"2024-09-16T01:38:08.553660+0000","last_became_peered":"2024-09-16T01:38:08.553660+0000","last_unstale":"2024-09-16T01:40:23.918230+0000","last_undegraded":"2024-09-16T01:40:23.918230+0000","last_fullsized":"2024-09-16T01:40:23.918230+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:53:52.270695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706069+0000","last_change":"2024-09-16T01:38:01.522688+0000","last_active":"2024-09-16T01:40:23.706069+0000","last_peered":"2024-09-16T01:40:23.706069+0000","last_clean":"2024-09-16T01:40:23.706069+0000","last_became_active":"2024-09-16T01:38:01.519843+0000","last_became_peered":"2024-09-16T01:38:01.519843+0000","last_unstale":"2024-09-16T01:40:23.706069+0000","last_undegraded":"2024-09-16T01:40:23.706069+0000","last_fullsized":"2024-09-16T01:40:23.706069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:50.971760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709088+0000","last_change":"2024-09-16T01:37:59.509044+0000","last_active":"2024-09-16T01:40:23.709088+0000","last_peered":"2024-09-16T01:40:23.709088+0000","last_clean":"2024-09-16T01:40:23.709088+0000","last_became_active":"2024-09-16T01:37:59.507085+0000","last_became_peered":"2024-09-16T01:37:59.507085+0000","last_unstale":"2024-09-16T01:40:23.709088+0000","last_undegraded":"2024-09-16T01:40:23.709088+0000","last_fullsized":"2024-09-16T01:40:23.709088+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:25:09.930945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682550+0000","last_change":"2024-09-16T01:38:08.561250+0000","last_active":"2024-09-16T01:40:23.682550+0000","last_peered":"2024-09-16T01:40:23.682550+0000","last_clean":"2024-09-16T01:40:23.682550+0000","last_became_active":"2024-09-16T01:38:08.560866+0000","last_became_peered":"2024-09-16T01:38:08.560866+0000","last_unstale":"2024-09-16T01:40:23.682550+0000","last_undegraded":"2024-09-16T01:40:23.682550+0000","last_fullsized":"2024-09-16T01:40:23.682550+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:31.253817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680608+0000","last_change":"2024-09-16T01:38:01.517400+0000","last_active":"2024-09-16T01:40:23.680608+0000","last_peered":"2024-09-16T01:40:23.680608+0000","last_clean":"2024-09-16T01:40:23.680608+0000","last_became_active":"2024-09-16T01:38:01.517103+0000","last_became_peered":"2024-09-16T01:38:01.517103+0000","last_unstale":"2024-09-16T01:40:23.680608+0000","last_undegraded":"2024-09-16T01:40:23.680608+0000","last_fullsized":"2024-09-16T01:40:23.680608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:37.886179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706415+0000","last_change":"2024-09-16T01:37:59.503370+0000","last_active":"2024-09-16T01:40:23.706415+0000","last_peered":"2024-09-16T01:40:23.706415+0000","last_clean":"2024-09-16T01:40:23.706415+0000","last_became_active":"2024-09-16T01:37:59.503071+0000","last_became_peered":"2024-09-16T01:37:59.503071+0000","last_unstale":"2024-09-16T01:40:23.706415+0000","last_undegraded":"2024-09-16T01:40:23.706415+0000","last_fullsized":"2024-09-16T01:40:23.706415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:51.207702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706631+0000","last_change":"2024-09-16T01:37:59.506110+0000","last_active":"2024-09-16T01:40:23.706631+0000","last_peered":"2024-09-16T01:40:23.706631+0000","last_clean":"2024-09-16T01:40:23.706631+0000","last_became_active":"2024-09-16T01:37:59.505264+0000","last_became_peered":"2024-09-16T01:37:59.505264+0000","last_unstale":"2024-09-16T01:40:23.706631+0000","last_undegraded":"2024-09-16T01:40:23.706631+0000","last_fullsized":"2024-09-16T01:40:23.706631+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:35.986433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709024+0000","last_change":"2024-09-16T01:38:08.563900+0000","last_active":"2024-09-16T01:40:23.709024+0000","last_peered":"2024-09-16T01:40:23.709024+0000","last_clean":"2024-09-16T01:40:23.709024+0000","last_became_active":"2024-09-16T01:38:08.563551+0000","last_became_peered":"2024-09-16T01:38:08.563551+0000","last_unstale":"2024-09-16T01:40:23.709024+0000","last_undegraded":"2024-09-16T01:40:23.709024+0000","last_fullsized":"2024-09-16T01:40:23.709024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:31:03.979580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716744+0000","last_change":"2024-09-16T01:38:08.556011+0000","last_active":"2024-09-16T01:40:23.716744+0000","last_peered":"2024-09-16T01:40:23.716744+0000","last_clean":"2024-09-16T01:40:23.716744+0000","last_became_active":"2024-09-16T01:38:08.555615+0000","last_became_peered":"2024-09-16T01:38:08.555615+0000","last_unstale":"2024-09-16T01:40:23.716744+0000","last_undegraded":"2024-09-16T01:40:23.716744+0000","last_fullsized":"2024-09-16T01:40:23.716744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:34.434974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":147,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709207+0000","last_change":"2024-09-16T01:37:59.508411+0000","last_active":"2024-09-16T01:40:23.709207+0000","last_peered":"2024-09-16T01:40:23.709207+0000","last_clean":"2024-09-16T01:40:23.709207+0000","last_became_active":"2024-09-16T01:37:59.508058+0000","last_became_peered":"2024-09-16T01:37:59.508058+0000","last_unstale":"2024-09-16T01:40:23.709207+0000","last_undegraded":"2024-09-16T01:40:23.709207+0000","last_fullsized":"2024-09-16T01:40:23.709207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:47.441673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716319+0000","last_change":"2024-09-16T01:38:08.560930+0000","last_active":"2024-09-16T01:40:23.716319+0000","last_peered":"2024-09-16T01:40:23.716319+0000","last_clean":"2024-09-16T01:40:23.716319+0000","last_became_active":"2024-09-16T01:38:08.560490+0000","last_became_peered":"2024-09-16T01:38:08.560490+0000","last_unstale":"2024-09-16T01:40:23.716319+0000","last_undegraded":"2024-09-16T01:40:23.716319+0000","last_fullsized":"2024-09-16T01:40:23.716319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:11.556113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716818+0000","last_change":"2024-09-16T01:38:08.557564+0000","last_active":"2024-09-16T01:40:23.716818+0000","last_peered":"2024-09-16T01:40:23.716818+0000","last_clean":"2024-09-16T01:40:23.716818+0000","last_became_active":"2024-09-16T01:38:08.557143+0000","last_became_peered":"2024-09-16T01:38:08.557143+0000","last_unstale":"2024-09-16T01:40:23.716818+0000","last_undegraded":"2024-09-16T01:40:23.716818+0000","last_fullsized":"2024-09-16T01:40:23.716818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:43.148870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681960+0000","last_change":"2024-09-16T01:38:01.527353+0000","last_active":"2024-09-16T01:40:23.681960+0000","last_peered":"2024-09-16T01:40:23.681960+0000","last_clean":"2024-09-16T01:40:23.681960+0000","last_became_active":"2024-09-16T01:38:01.527031+0000","last_became_peered":"2024-09-16T01:38:01.527031+0000","last_unstale":"2024-09-16T01:40:23.681960+0000","last_undegraded":"2024-09-16T01:40:23.681960+0000","last_fullsized":"2024-09-16T01:40:23.681960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:27.311638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716301+0000","last_change":"2024-09-16T01:38:01.526409+0000","last_active":"2024-09-16T01:40:23.716301+0000","last_peered":"2024-09-16T01:40:23.716301+0000","last_clean":"2024-09-16T01:40:23.716301+0000","last_became_active":"2024-09-16T01:38:01.526064+0000","last_became_peered":"2024-09-16T01:38:01.526064+0000","last_unstale":"2024-09-16T01:40:23.716301+0000","last_undegraded":"2024-09-16T01:40:23.716301+0000","last_fullsized":"2024-09-16T01:40:23.716301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:51.571808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716432+0000","last_change":"2024-09-16T01:38:01.521856+0000","last_active":"2024-09-16T01:40:23.716432+0000","last_peered":"2024-09-16T01:40:23.716432+0000","last_clean":"2024-09-16T01:40:23.716432+0000","last_became_active":"2024-09-16T01:38:01.520312+0000","last_became_peered":"2024-09-16T01:38:01.520312+0000","last_unstale":"2024-09-16T01:40:23.716432+0000","last_undegraded":"2024-09-16T01:40:23.716432+0000","last_fullsized":"2024-09-16T01:40:23.716432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:53.618990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918080+0000","last_change":"2024-09-16T01:38:01.520958+0000","last_active":"2024-09-16T01:40:23.918080+0000","last_peered":"2024-09-16T01:40:23.918080+0000","last_clean":"2024-09-16T01:40:23.918080+0000","last_became_active":"2024-09-16T01:38:01.520658+0000","last_became_peered":"2024-09-16T01:38:01.520658+0000","last_unstale":"2024-09-16T01:40:23.918080+0000","last_undegraded":"2024-09-16T01:40:23.918080+0000","last_fullsized":"2024-09-16T01:40:23.918080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:23.770694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707380+0000","last_change":"2024-09-16T01:38:01.519140+0000","last_active":"2024-09-16T01:40:23.707380+0000","last_peered":"2024-09-16T01:40:23.707380+0000","last_clean":"2024-09-16T01:40:23.707380+0000","last_became_active":"2024-09-16T01:38:01.518765+0000","last_became_peered":"2024-09-16T01:38:01.518765+0000","last_unstale":"2024-09-16T01:40:23.707380+0000","last_undegraded":"2024-09-16T01:40:23.707380+0000","last_fullsized":"2024-09-16T01:40:23.707380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:17.194886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682443+0000","last_change":"2024-09-16T01:38:01.523762+0000","last_active":"2024-09-16T01:40:23.682443+0000","last_peered":"2024-09-16T01:40:23.682443+0000","last_clean":"2024-09-16T01:40:23.682443+0000","last_became_active":"2024-09-16T01:38:01.522035+0000","last_became_peered":"2024-09-16T01:38:01.522035+0000","last_unstale":"2024-09-16T01:40:23.682443+0000","last_undegraded":"2024-09-16T01:40:23.682443+0000","last_fullsized":"2024-09-16T01:40:23.682443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:40.405350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.717020+0000","last_change":"2024-09-16T01:38:01.527539+0000","last_active":"2024-09-16T01:40:23.717020+0000","last_peered":"2024-09-16T01:40:23.717020+0000","last_clean":"2024-09-16T01:40:23.717020+0000","last_became_active":"2024-09-16T01:38:01.526974+0000","last_became_peered":"2024-09-16T01:38:01.526974+0000","last_unstale":"2024-09-16T01:40:23.717020+0000","last_undegraded":"2024-09-16T01:40:23.717020+0000","last_fullsized":"2024-09-16T01:40:23.717020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:30:03.587141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681144+0000","last_change":"2024-09-16T01:38:01.529131+0000","last_active":"2024-09-16T01:40:23.681144+0000","last_peered":"2024-09-16T01:40:23.681144+0000","last_clean":"2024-09-16T01:40:23.681144+0000","last_became_active":"2024-09-16T01:38:01.528796+0000","last_became_peered":"2024-09-16T01:38:01.528796+0000","last_unstale":"2024-09-16T01:40:23.681144+0000","last_undegraded":"2024-09-16T01:40:23.681144+0000","last_fullsized":"2024-09-16T01:40:23.681144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:38.903638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708881+0000","last_change":"2024-09-16T01:38:01.514779+0000","last_active":"2024-09-16T01:40:23.708881+0000","last_peered":"2024-09-16T01:40:23.708881+0000","last_clean":"2024-09-16T01:40:23.708881+0000","last_became_active":"2024-09-16T01:38:01.514434+0000","last_became_peered":"2024-09-16T01:38:01.514434+0000","last_unstale":"2024-09-16T01:40:23.708881+0000","last_undegraded":"2024-09-16T01:40:23.708881+0000","last_fullsized":"2024-09-16T01:40:23.708881+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:31.382111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680678+0000","last_change":"2024-09-16T01:38:01.521771+0000","last_active":"2024-09-16T01:40:23.680678+0000","last_peered":"2024-09-16T01:40:23.680678+0000","last_clean":"2024-09-16T01:40:23.680678+0000","last_became_active":"2024-09-16T01:38:01.521288+0000","last_became_peered":"2024-09-16T01:38:01.521288+0000","last_unstale":"2024-09-16T01:40:23.680678+0000","last_undegraded":"2024-09-16T01:40:23.680678+0000","last_fullsized":"2024-09-16T01:40:23.680678+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:11.825164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918399+0000","last_change":"2024-09-16T01:38:01.529247+0000","last_active":"2024-09-16T01:40:23.918399+0000","last_peered":"2024-09-16T01:40:23.918399+0000","last_clean":"2024-09-16T01:40:23.918399+0000","last_became_active":"2024-09-16T01:38:01.528926+0000","last_became_peered":"2024-09-16T01:38:01.528926+0000","last_unstale":"2024-09-16T01:40:23.918399+0000","last_undegraded":"2024-09-16T01:40:23.918399+0000","last_fullsized":"2024-09-16T01:40:23.918399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:20.704114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707157+0000","last_change":"2024-09-16T01:38:01.520830+0000","last_active":"2024-09-16T01:40:23.707157+0000","last_peered":"2024-09-16T01:40:23.707157+0000","last_clean":"2024-09-16T01:40:23.707157+0000","last_became_active":"2024-09-16T01:38:01.520526+0000","last_became_peered":"2024-09-16T01:38:01.520526+0000","last_unstale":"2024-09-16T01:40:23.707157+0000","last_undegraded":"2024-09-16T01:40:23.707157+0000","last_fullsized":"2024-09-16T01:40:23.707157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:28.062812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682262+0000","last_change":"2024-09-16T01:38:01.521983+0000","last_active":"2024-09-16T01:40:23.682262+0000","last_peered":"2024-09-16T01:40:23.682262+0000","last_clean":"2024-09-16T01:40:23.682262+0000","last_became_active":"2024-09-16T01:38:01.521666+0000","last_became_peered":"2024-09-16T01:38:01.521666+0000","last_unstale":"2024-09-16T01:40:23.682262+0000","last_undegraded":"2024-09-16T01:40:23.682262+0000","last_fullsized":"2024-09-16T01:40:23.682262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:50:55.761313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.717006+0000","last_change":"2024-09-16T01:38:08.554304+0000","last_active":"2024-09-16T01:40:23.717006+0000","last_peered":"2024-09-16T01:40:23.717006+0000","last_clean":"2024-09-16T01:40:23.717006+0000","last_became_active":"2024-09-16T01:38:08.553683+0000","last_became_peered":"2024-09-16T01:38:08.553683+0000","last_unstale":"2024-09-16T01:40:23.717006+0000","last_undegraded":"2024-09-16T01:40:23.717006+0000","last_fullsized":"2024-09-16T01:40:23.717006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:56.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706919+0000","last_change":"2024-09-16T01:38:01.520425+0000","last_active":"2024-09-16T01:40:23.706919+0000","last_peered":"2024-09-16T01:40:23.706919+0000","last_clean":"2024-09-16T01:40:23.706919+0000","last_became_active":"2024-09-16T01:38:01.519858+0000","last_became_peered":"2024-09-16T01:38:01.519858+0000","last_unstale":"2024-09-16T01:40:23.706919+0000","last_undegraded":"2024-09-16T01:40:23.706919+0000","last_fullsized":"2024-09-16T01:40:23.706919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:48.534298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682855+0000","last_change":"2024-09-16T01:38:01.532762+0000","last_active":"2024-09-16T01:40:23.682855+0000","last_peered":"2024-09-16T01:40:23.682855+0000","last_clean":"2024-09-16T01:40:23.682855+0000","last_became_active":"2024-09-16T01:38:01.532404+0000","last_became_peered":"2024-09-16T01:38:01.532404+0000","last_unstale":"2024-09-16T01:40:23.682855+0000","last_undegraded":"2024-09-16T01:40:23.682855+0000","last_fullsized":"2024-09-16T01:40:23.682855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:03.135141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918405+0000","last_change":"2024-09-16T01:38:01.530205+0000","last_active":"2024-09-16T01:40:23.918405+0000","last_peered":"2024-09-16T01:40:23.918405+0000","last_clean":"2024-09-16T01:40:23.918405+0000","last_became_active":"2024-09-16T01:38:01.529898+0000","last_became_peered":"2024-09-16T01:38:01.529898+0000","last_unstale":"2024-09-16T01:40:23.918405+0000","last_undegraded":"2024-09-16T01:40:23.918405+0000","last_fullsized":"2024-09-16T01:40:23.918405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:48.444441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.917747+0000","last_change":"2024-09-16T01:38:01.529407+0000","last_active":"2024-09-16T01:40:23.917747+0000","last_peered":"2024-09-16T01:40:23.917747+0000","last_clean":"2024-09-16T01:40:23.917747+0000","last_became_active":"2024-09-16T01:38:01.528330+0000","last_became_peered":"2024-09-16T01:38:01.528330+0000","last_unstale":"2024-09-16T01:40:23.917747+0000","last_undegraded":"2024-09-16T01:40:23.917747+0000","last_fullsized":"2024-09-16T01:40:23.917747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:32.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.917675+0000","last_change":"2024-09-16T01:38:01.519579+0000","last_active":"2024-09-16T01:40:23.917675+0000","last_peered":"2024-09-16T01:40:23.917675+0000","last_clean":"2024-09-16T01:40:23.917675+0000","last_became_active":"2024-09-16T01:38:01.519211+0000","last_became_peered":"2024-09-16T01:38:01.519211+0000","last_unstale":"2024-09-16T01:40:23.917675+0000","last_undegraded":"2024-09-16T01:40:23.917675+0000","last_fullsized":"2024-09-16T01:40:23.917675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:18.861169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707196+0000","last_change":"2024-09-16T01:38:01.521136+0000","last_active":"2024-09-16T01:40:23.707196+0000","last_peered":"2024-09-16T01:40:23.707196+0000","last_clean":"2024-09-16T01:40:23.707196+0000","last_became_active":"2024-09-16T01:38:01.520819+0000","last_became_peered":"2024-09-16T01:38:01.520819+0000","last_unstale":"2024-09-16T01:40:23.707196+0000","last_undegraded":"2024-09-16T01:40:23.707196+0000","last_fullsized":"2024-09-16T01:40:23.707196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:28.150822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709215+0000","last_change":"2024-09-16T01:38:01.519382+0000","last_active":"2024-09-16T01:40:23.709215+0000","last_peered":"2024-09-16T01:40:23.709215+0000","last_clean":"2024-09-16T01:40:23.709215+0000","last_became_active":"2024-09-16T01:38:01.519039+0000","last_became_peered":"2024-09-16T01:38:01.519039+0000","last_unstale":"2024-09-16T01:40:23.709215+0000","last_undegraded":"2024-09-16T01:40:23.709215+0000","last_fullsized":"2024-09-16T01:40:23.709215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:46.559357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682055+0000","last_change":"2024-09-16T01:38:01.527513+0000","last_active":"2024-09-16T01:40:23.682055+0000","last_peered":"2024-09-16T01:40:23.682055+0000","last_clean":"2024-09-16T01:40:23.682055+0000","last_became_active":"2024-09-16T01:38:01.527184+0000","last_became_peered":"2024-09-16T01:38:01.527184+0000","last_unstale":"2024-09-16T01:40:23.682055+0000","last_undegraded":"2024-09-16T01:40:23.682055+0000","last_fullsized":"2024-09-16T01:40:23.682055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:41.508318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706944+0000","last_change":"2024-09-16T01:38:01.524239+0000","last_active":"2024-09-16T01:40:23.706944+0000","last_peered":"2024-09-16T01:40:23.706944+0000","last_clean":"2024-09-16T01:40:23.706944+0000","last_became_active":"2024-09-16T01:38:01.523906+0000","last_became_peered":"2024-09-16T01:38:01.523906+0000","last_unstale":"2024-09-16T01:40:23.706944+0000","last_undegraded":"2024-09-16T01:40:23.706944+0000","last_fullsized":"2024-09-16T01:40:23.706944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:11.718796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918437+0000","last_change":"2024-09-16T01:38:01.531564+0000","last_active":"2024-09-16T01:40:23.918437+0000","last_peered":"2024-09-16T01:40:23.918437+0000","last_clean":"2024-09-16T01:40:23.918437+0000","last_became_active":"2024-09-16T01:38:01.531241+0000","last_became_peered":"2024-09-16T01:38:01.531241+0000","last_unstale":"2024-09-16T01:40:23.918437+0000","last_undegraded":"2024-09-16T01:40:23.918437+0000","last_fullsized":"2024-09-16T01:40:23.918437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:16.244641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680580+0000","last_change":"2024-09-16T01:38:01.519913+0000","last_active":"2024-09-16T01:40:23.680580+0000","last_peered":"2024-09-16T01:40:23.680580+0000","last_clean":"2024-09-16T01:40:23.680580+0000","last_became_active":"2024-09-16T01:38:01.518631+0000","last_became_peered":"2024-09-16T01:38:01.518631+0000","last_unstale":"2024-09-16T01:40:23.680580+0000","last_undegraded":"2024-09-16T01:40:23.680580+0000","last_fullsized":"2024-09-16T01:40:23.680580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:48.747188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681475+0000","last_change":"2024-09-16T01:38:01.530805+0000","last_active":"2024-09-16T01:40:23.681475+0000","last_peered":"2024-09-16T01:40:23.681475+0000","last_clean":"2024-09-16T01:40:23.681475+0000","last_became_active":"2024-09-16T01:38:01.530466+0000","last_became_peered":"2024-09-16T01:38:01.530466+0000","last_unstale":"2024-09-16T01:40:23.681475+0000","last_undegraded":"2024-09-16T01:40:23.681475+0000","last_fullsized":"2024-09-16T01:40:23.681475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:41:30.587398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706566+0000","last_change":"2024-09-16T01:38:01.519227+0000","last_active":"2024-09-16T01:40:23.706566+0000","last_peered":"2024-09-16T01:40:23.706566+0000","last_clean":"2024-09-16T01:40:23.706566+0000","last_became_active":"2024-09-16T01:38:01.518875+0000","last_became_peered":"2024-09-16T01:38:01.518875+0000","last_unstale":"2024-09-16T01:40:23.706566+0000","last_undegraded":"2024-09-16T01:40:23.706566+0000","last_fullsized":"2024-09-16T01:40:23.706566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:42.654812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716329+0000","last_change":"2024-09-16T01:38:01.513327+0000","last_active":"2024-09-16T01:40:23.716329+0000","last_peered":"2024-09-16T01:40:23.716329+0000","last_clean":"2024-09-16T01:40:23.716329+0000","last_became_active":"2024-09-16T01:38:01.513000+0000","last_became_peered":"2024-09-16T01:38:01.513000+0000","last_unstale":"2024-09-16T01:40:23.716329+0000","last_undegraded":"2024-09-16T01:40:23.716329+0000","last_fullsized":"2024-09-16T01:40:23.716329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:43.036805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706554+0000","last_change":"2024-09-16T01:38:08.560560+0000","last_active":"2024-09-16T01:40:23.706554+0000","last_peered":"2024-09-16T01:40:23.706554+0000","last_clean":"2024-09-16T01:40:23.706554+0000","last_became_active":"2024-09-16T01:38:08.560229+0000","last_became_peered":"2024-09-16T01:38:08.560229+0000","last_unstale":"2024-09-16T01:40:23.706554+0000","last_undegraded":"2024-09-16T01:40:23.706554+0000","last_fullsized":"2024-09-16T01:40:23.706554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:10:09.462043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.917771+0000","last_change":"2024-09-16T01:38:01.529548+0000","last_active":"2024-09-16T01:40:23.917771+0000","last_peered":"2024-09-16T01:40:23.917771+0000","last_clean":"2024-09-16T01:40:23.917771+0000","last_became_active":"2024-09-16T01:38:01.529011+0000","last_became_peered":"2024-09-16T01:38:01.529011+0000","last_unstale":"2024-09-16T01:40:23.917771+0000","last_undegraded":"2024-09-16T01:40:23.917771+0000","last_fullsized":"2024-09-16T01:40:23.917771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:21.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918332+0000","last_change":"2024-09-16T01:38:01.519488+0000","last_active":"2024-09-16T01:40:23.918332+0000","last_peered":"2024-09-16T01:40:23.918332+0000","last_clean":"2024-09-16T01:40:23.918332+0000","last_became_active":"2024-09-16T01:38:01.519156+0000","last_became_peered":"2024-09-16T01:38:01.519156+0000","last_unstale":"2024-09-16T01:40:23.918332+0000","last_undegraded":"2024-09-16T01:40:23.918332+0000","last_fullsized":"2024-09-16T01:40:23.918332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:18.149874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682782+0000","last_change":"2024-09-16T01:38:01.527519+0000","last_active":"2024-09-16T01:40:23.682782+0000","last_peered":"2024-09-16T01:40:23.682782+0000","last_clean":"2024-09-16T01:40:23.682782+0000","last_became_active":"2024-09-16T01:38:01.527175+0000","last_became_peered":"2024-09-16T01:38:01.527175+0000","last_unstale":"2024-09-16T01:40:23.682782+0000","last_undegraded":"2024-09-16T01:40:23.682782+0000","last_fullsized":"2024-09-16T01:40:23.682782+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:04.958842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601743,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1175,"num_read_kb":698,"num_write":379,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1805229,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107382,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678058,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":498675,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906991,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1138688,"data_stored":1289726,"data_compressed":68589,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070517,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":891175,"data_compressed":45785,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":890769,"data_compressed":45785,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528382,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1032192,"data_stored":1095229,"data_compressed":62433,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659204,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":958464,"data_stored":493856,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888137,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1221625,"data_compressed":68589,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051662,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":497930,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280595,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":1017987,"data_compressed":51941,"data_compressed_allocated":69632,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476824,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1052672,"data_stored":564432,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607645,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1040384,"data_stored":564350,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":983040,"data_stored":495846,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134420,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202521,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T01:40:27.677 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T01:40:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:28.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.418+0000 7f0d4f66e640 1 -- 172.21.15.139:0/216605583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d48101420 msgr2=0x7f0d4810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:28.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.418+0000 7f0d4f66e640 1 --2- 172.21.15.139:0/216605583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d48101420 0x7f0d4810d950 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009930 tx=0x7f0d4002a420 comp rx=0 tx=0).stop 2024-09-16T01:40:28.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.418+0000 7f0d4f66e640 1 -- 172.21.15.139:0/216605583 shutdown_connections 2024-09-16T01:40:28.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.418+0000 7f0d4f66e640 1 --2- 172.21.15.139:0/216605583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d48101420 0x7f0d4810d950 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:28.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.418+0000 7f0d4f66e640 1 --2- 172.21.15.139:0/216605583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d48100a80 0x7f0d48100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:28.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.418+0000 7f0d4f66e640 1 --2- 172.21.15.139:0/216605583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d48106a80 0x7f0d48106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:28.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.418+0000 7f0d4f66e640 1 -- 172.21.15.139:0/216605583 >> 172.21.15.139:0/216605583 conn(0x7f0d480fc7a0 msgr2=0x7f0d480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:28.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.418+0000 7f0d4f66e640 1 -- 172.21.15.139:0/216605583 shutdown_connections 2024-09-16T01:40:28.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.418+0000 7f0d4f66e640 1 -- 172.21.15.139:0/216605583 wait complete. 2024-09-16T01:40:28.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.419+0000 7f0d4f66e640 1 Processor -- start 2024-09-16T01:40:28.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.419+0000 7f0d4f66e640 1 -- start start 2024-09-16T01:40:28.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.419+0000 7f0d4f66e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d48100a80 0x7f0d48197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:28.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.419+0000 7f0d4f66e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d48101420 0x7f0d48197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:28.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.419+0000 7f0d4f66e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d48106a80 0x7f0d4819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:28.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.419+0000 7f0d4f66e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48110e80 con 0x7f0d48101420 2024-09-16T01:40:28.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.419+0000 7f0d4f66e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48110d00 con 0x7f0d48100a80 2024-09-16T01:40:28.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.419+0000 7f0d4f66e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48111000 con 0x7f0d48106a80 2024-09-16T01:40:28.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.419+0000 7f0d4cc09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d48101420 0x7f0d48197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:28.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.419+0000 7f0d4cc09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d48101420 0x7f0d48197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50650/0 (socket says 172.21.15.139:50650) 2024-09-16T01:40:28.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.419+0000 7f0d4cc09640 1 -- 172.21.15.139:0/3385191630 learned_addr learned my addr 172.21.15.139:0/3385191630 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:28.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.419+0000 7f0d4d40a640 1 --2- 172.21.15.139:0/3385191630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d48100a80 0x7f0d48197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:28.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.420+0000 7f0d4dc0b640 1 --2- 172.21.15.139:0/3385191630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d48106a80 0x7f0d4819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:28.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.420+0000 7f0d4cc09640 1 -- 172.21.15.139:0/3385191630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d48106a80 msgr2=0x7f0d4819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:28.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.420+0000 7f0d4cc09640 1 --2- 172.21.15.139:0/3385191630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d48106a80 0x7f0d4819bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:28.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.420+0000 7f0d4cc09640 1 -- 172.21.15.139:0/3385191630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d48100a80 msgr2=0x7f0d48197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:28.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.420+0000 7f0d4cc09640 1 --2- 172.21.15.139:0/3385191630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d48100a80 0x7f0d48197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:28.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.420+0000 7f0d4cc09640 1 -- 172.21.15.139:0/3385191630 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4819c2f0 con 0x7f0d48101420 2024-09-16T01:40:28.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.420+0000 7f0d4cc09640 1 --2- 172.21.15.139:0/3385191630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d48101420 0x7f0d48197880 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00d770 tx=0x7f0d3c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:28.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.420+0000 7f0d467fc640 1 -- 172.21.15.139:0/3385191630 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c017070 con 0x7f0d48101420 2024-09-16T01:40:28.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.420+0000 7f0d467fc640 1 -- 172.21.15.139:0/3385191630 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3c005500 con 0x7f0d48101420 2024-09-16T01:40:28.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.420+0000 7f0d4f66e640 1 -- 172.21.15.139:0/3385191630 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4819c5e0 con 0x7f0d48101420 2024-09-16T01:40:28.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.420+0000 7f0d467fc640 1 -- 172.21.15.139:0/3385191630 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c004e00 con 0x7f0d48101420 2024-09-16T01:40:28.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.420+0000 7f0d4f66e640 1 -- 172.21.15.139:0/3385191630 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d481a3ec0 con 0x7f0d48101420 2024-09-16T01:40:28.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.421+0000 7f0d4f66e640 1 -- 172.21.15.139:0/3385191630 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d48100ee0 con 0x7f0d48101420 2024-09-16T01:40:28.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.422+0000 7f0d467fc640 1 -- 172.21.15.139:0/3385191630 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 22) ==== 138476+0+0 (secure 0 0 0) 0x7f0d3c004fa0 con 0x7f0d48101420 2024-09-16T01:40:28.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.423+0000 7f0d467fc640 1 --2- 172.21.15.139:0/3385191630 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0d300a33b0 0x7f0d300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:28.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.423+0000 7f0d467fc640 1 -- 172.21.15.139:0/3385191630 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f0d3c0cfa90 con 0x7f0d48101420 2024-09-16T01:40:28.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.423+0000 7f0d4d40a640 1 --2- 172.21.15.139:0/3385191630 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0d300a33b0 0x7f0d300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:28.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.423+0000 7f0d4d40a640 1 --2- 172.21.15.139:0/3385191630 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0d300a33b0 0x7f0d300a5870 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0d38005fd0 tx=0x7f0d38009680 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:28.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.427+0000 7f0d467fc640 1 -- 172.21.15.139:0/3385191630 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d3c09ba70 con 0x7f0d48101420 2024-09-16T01:40:28.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.568+0000 7f0d467fc640 1 -- 172.21.15.139:0/3385191630 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mgrmap(e 23) ==== 138532+0+0 (secure 0 0 0) 0x7f0d3c0cd0b0 con 0x7f0d48101420 2024-09-16T01:40:28.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.597+0000 7f0d4f66e640 1 -- 172.21.15.139:0/3385191630 --> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0d48062630 con 0x7f0d300a33b0 2024-09-16T01:40:28.599 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: Updating smithi176:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:40:28.599 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: Updating smithi164:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:40:28.599 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: Updating smithi139:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:40:28.599 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:40:28.599 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:40:28.599 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:40:28.599 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:40:28.600 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: Updating smithi176:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:40:28.600 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: Updating smithi164:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:40:28.600 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: Updating smithi139:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:40:28.600 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.600 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.600 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.600 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.600 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.600 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.600 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.600 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:40:28.600 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:40:28.600 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:40:28.600 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.605+0000 7f0d467fc640 1 -- 172.21.15.139:0/3385191630 <== mgr.15324 v2:172.21.15.139:6800/4031776938 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339718 (secure 0 0 0) 0x7f0d48062630 con 0x7f0d300a33b0 2024-09-16T01:40:28.611 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:40:28.612 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T01:40:28.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.608+0000 7f0d4f66e640 1 -- 172.21.15.139:0/3385191630 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0d300a33b0 msgr2=0x7f0d300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:28.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.608+0000 7f0d4f66e640 1 --2- 172.21.15.139:0/3385191630 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0d300a33b0 0x7f0d300a5870 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0d38005fd0 tx=0x7f0d38009680 comp rx=0 tx=0).stop 2024-09-16T01:40:28.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.608+0000 7f0d4f66e640 1 -- 172.21.15.139:0/3385191630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d48101420 msgr2=0x7f0d48197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:28.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.608+0000 7f0d4f66e640 1 --2- 172.21.15.139:0/3385191630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d48101420 0x7f0d48197880 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00d770 tx=0x7f0d3c00dc30 comp rx=0 tx=0).stop 2024-09-16T01:40:28.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.608+0000 7f0d4f66e640 1 -- 172.21.15.139:0/3385191630 shutdown_connections 2024-09-16T01:40:28.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.608+0000 7f0d4f66e640 1 --2- 172.21.15.139:0/3385191630 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0d300a33b0 0x7f0d300a5870 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:28.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.608+0000 7f0d4f66e640 1 --2- 172.21.15.139:0/3385191630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d48106a80 0x7f0d4819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:28.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.608+0000 7f0d4f66e640 1 --2- 172.21.15.139:0/3385191630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d48101420 0x7f0d48197880 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:28.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.608+0000 7f0d4f66e640 1 --2- 172.21.15.139:0/3385191630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d48100a80 0x7f0d48197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:28.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.608+0000 7f0d4f66e640 1 -- 172.21.15.139:0/3385191630 >> 172.21.15.139:0/3385191630 conn(0x7f0d480fc7a0 msgr2=0x7f0d480fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:28.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.609+0000 7f0d4f66e640 1 -- 172.21.15.139:0/3385191630 shutdown_connections 2024-09-16T01:40:28.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:28.609+0000 7f0d4f66e640 1 -- 172.21.15.139:0/3385191630 wait complete. 2024-09-16T01:40:28.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: Updating smithi176:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:40:28.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: Updating smithi164:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:40:28.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: Updating smithi139:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:40:28.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:40:28.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:40:28.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:40:28.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:40:28.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: Updating smithi176:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:40:28.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: Updating smithi164:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:40:28.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: Updating smithi139:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:40:28.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:40:28.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:40:28.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:40:28.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: Updating smithi176:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:40:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: Updating smithi164:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:40:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: Updating smithi139:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T01:40:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:40:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:40:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:40:28.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-09-16T01:40:28.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: Updating smithi176:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:40:28.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: Updating smithi164:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:40:28.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: Updating smithi139:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T01:40:28.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:28.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:40:28.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:40:28.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:40:28.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:40:29.302 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-16T01:40:27.694075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192345,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1311,"num_read_kb":817,"num_write":562,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":943,"ondisk_log_size":943,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373688,"kb_used_data":11924,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568136,"statfs":{"total":1151940427776,"available":1151557771264,"internally_reserved":0,"allocated":12210176,"data_stored":9521600,"data_compressed":502956,"data_compressed_allocated":675840,"data_compressed_original":4202496,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990090"},"pg_stats":[{"pgid":"2.1f","version":"86'14","reported_seq":167,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707440+0000","last_change":"2024-09-16T01:38:08.567909+0000","last_active":"2024-09-16T01:40:23.707440+0000","last_peered":"2024-09-16T01:40:23.707440+0000","last_clean":"2024-09-16T01:40:23.707440+0000","last_became_active":"2024-09-16T01:38:08.560733+0000","last_became_peered":"2024-09-16T01:38:08.560733+0000","last_unstale":"2024-09-16T01:40:23.707440+0000","last_undegraded":"2024-09-16T01:40:23.707440+0000","last_fullsized":"2024-09-16T01:40:23.707440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:00.678040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681164+0000","last_change":"2024-09-16T01:38:01.523064+0000","last_active":"2024-09-16T01:40:23.681164+0000","last_peered":"2024-09-16T01:40:23.681164+0000","last_clean":"2024-09-16T01:40:23.681164+0000","last_became_active":"2024-09-16T01:38:01.521695+0000","last_became_peered":"2024-09-16T01:38:01.521695+0000","last_unstale":"2024-09-16T01:40:23.681164+0000","last_undegraded":"2024-09-16T01:40:23.681164+0000","last_fullsized":"2024-09-16T01:40:23.681164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:28.467115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716031+0000","last_change":"2024-09-16T01:38:08.563557+0000","last_active":"2024-09-16T01:40:23.716031+0000","last_peered":"2024-09-16T01:40:23.716031+0000","last_clean":"2024-09-16T01:40:23.716031+0000","last_became_active":"2024-09-16T01:38:08.563065+0000","last_became_peered":"2024-09-16T01:38:08.563065+0000","last_unstale":"2024-09-16T01:40:23.716031+0000","last_undegraded":"2024-09-16T01:40:23.716031+0000","last_fullsized":"2024-09-16T01:40:23.716031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:56.992915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681383+0000","last_change":"2024-09-16T01:37:59.798884+0000","last_active":"2024-09-16T01:40:23.681383+0000","last_peered":"2024-09-16T01:40:23.681383+0000","last_clean":"2024-09-16T01:40:23.681383+0000","last_became_active":"2024-09-16T01:37:59.797989+0000","last_became_peered":"2024-09-16T01:37:59.797989+0000","last_unstale":"2024-09-16T01:40:23.681383+0000","last_undegraded":"2024-09-16T01:40:23.681383+0000","last_fullsized":"2024-09-16T01:40:23.681383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:13.633078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918394+0000","last_change":"2024-09-16T01:38:01.528391+0000","last_active":"2024-09-16T01:40:23.918394+0000","last_peered":"2024-09-16T01:40:23.918394+0000","last_clean":"2024-09-16T01:40:23.918394+0000","last_became_active":"2024-09-16T01:38:01.528076+0000","last_became_peered":"2024-09-16T01:38:01.528076+0000","last_unstale":"2024-09-16T01:40:23.918394+0000","last_undegraded":"2024-09-16T01:40:23.918394+0000","last_fullsized":"2024-09-16T01:40:23.918394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.105830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708846+0000","last_change":"2024-09-16T01:38:08.566198+0000","last_active":"2024-09-16T01:40:23.708846+0000","last_peered":"2024-09-16T01:40:23.708846+0000","last_clean":"2024-09-16T01:40:23.708846+0000","last_became_active":"2024-09-16T01:38:08.565739+0000","last_became_peered":"2024-09-16T01:38:08.565739+0000","last_unstale":"2024-09-16T01:40:23.708846+0000","last_undegraded":"2024-09-16T01:40:23.708846+0000","last_fullsized":"2024-09-16T01:40:23.708846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:05.172269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716538+0000","last_change":"2024-09-16T01:37:59.507625+0000","last_active":"2024-09-16T01:40:23.716538+0000","last_peered":"2024-09-16T01:40:23.716538+0000","last_clean":"2024-09-16T01:40:23.716538+0000","last_became_active":"2024-09-16T01:37:59.507280+0000","last_became_peered":"2024-09-16T01:37:59.507280+0000","last_unstale":"2024-09-16T01:40:23.716538+0000","last_undegraded":"2024-09-16T01:40:23.716538+0000","last_fullsized":"2024-09-16T01:40:23.716538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:29:37.761203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918422+0000","last_change":"2024-09-16T01:38:01.528406+0000","last_active":"2024-09-16T01:40:23.918422+0000","last_peered":"2024-09-16T01:40:23.918422+0000","last_clean":"2024-09-16T01:40:23.918422+0000","last_became_active":"2024-09-16T01:38:01.528084+0000","last_became_peered":"2024-09-16T01:38:01.528084+0000","last_unstale":"2024-09-16T01:40:23.918422+0000","last_undegraded":"2024-09-16T01:40:23.918422+0000","last_fullsized":"2024-09-16T01:40:23.918422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:40.766305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709276+0000","last_change":"2024-09-16T01:38:08.563645+0000","last_active":"2024-09-16T01:40:23.709276+0000","last_peered":"2024-09-16T01:40:23.709276+0000","last_clean":"2024-09-16T01:40:23.709276+0000","last_became_active":"2024-09-16T01:38:08.563149+0000","last_became_peered":"2024-09-16T01:38:08.563149+0000","last_unstale":"2024-09-16T01:40:23.709276+0000","last_undegraded":"2024-09-16T01:40:23.709276+0000","last_fullsized":"2024-09-16T01:40:23.709276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:09.010905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716807+0000","last_change":"2024-09-16T01:37:59.509649+0000","last_active":"2024-09-16T01:40:23.716807+0000","last_peered":"2024-09-16T01:40:23.716807+0000","last_clean":"2024-09-16T01:40:23.716807+0000","last_became_active":"2024-09-16T01:37:59.509236+0000","last_became_peered":"2024-09-16T01:37:59.509236+0000","last_unstale":"2024-09-16T01:40:23.716807+0000","last_undegraded":"2024-09-16T01:40:23.716807+0000","last_fullsized":"2024-09-16T01:40:23.716807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.614205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709245+0000","last_change":"2024-09-16T01:38:01.528074+0000","last_active":"2024-09-16T01:40:23.709245+0000","last_peered":"2024-09-16T01:40:23.709245+0000","last_clean":"2024-09-16T01:40:23.709245+0000","last_became_active":"2024-09-16T01:38:01.527737+0000","last_became_peered":"2024-09-16T01:38:01.527737+0000","last_unstale":"2024-09-16T01:40:23.709245+0000","last_undegraded":"2024-09-16T01:40:23.709245+0000","last_fullsized":"2024-09-16T01:40:23.709245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:37.592557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708957+0000","last_change":"2024-09-16T01:38:08.566340+0000","last_active":"2024-09-16T01:40:23.708957+0000","last_peered":"2024-09-16T01:40:23.708957+0000","last_clean":"2024-09-16T01:40:23.708957+0000","last_became_active":"2024-09-16T01:38:08.565791+0000","last_became_peered":"2024-09-16T01:38:08.565791+0000","last_unstale":"2024-09-16T01:40:23.708957+0000","last_undegraded":"2024-09-16T01:40:23.708957+0000","last_fullsized":"2024-09-16T01:40:23.708957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:35:31.845151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681718+0000","last_change":"2024-09-16T01:38:01.527747+0000","last_active":"2024-09-16T01:40:23.681718+0000","last_peered":"2024-09-16T01:40:23.681718+0000","last_clean":"2024-09-16T01:40:23.681718+0000","last_became_active":"2024-09-16T01:38:01.527325+0000","last_became_peered":"2024-09-16T01:38:01.527325+0000","last_unstale":"2024-09-16T01:40:23.681718+0000","last_undegraded":"2024-09-16T01:40:23.681718+0000","last_fullsized":"2024-09-16T01:40:23.681718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:27.203719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716467+0000","last_change":"2024-09-16T01:37:59.510298+0000","last_active":"2024-09-16T01:40:23.716467+0000","last_peered":"2024-09-16T01:40:23.716467+0000","last_clean":"2024-09-16T01:40:23.716467+0000","last_became_active":"2024-09-16T01:37:59.509952+0000","last_became_peered":"2024-09-16T01:37:59.509952+0000","last_unstale":"2024-09-16T01:40:23.716467+0000","last_undegraded":"2024-09-16T01:40:23.716467+0000","last_fullsized":"2024-09-16T01:40:23.716467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:11.616846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709120+0000","last_change":"2024-09-16T01:38:08.571622+0000","last_active":"2024-09-16T01:40:23.709120+0000","last_peered":"2024-09-16T01:40:23.709120+0000","last_clean":"2024-09-16T01:40:23.709120+0000","last_became_active":"2024-09-16T01:38:08.571125+0000","last_became_peered":"2024-09-16T01:38:08.571125+0000","last_unstale":"2024-09-16T01:40:23.709120+0000","last_undegraded":"2024-09-16T01:40:23.709120+0000","last_fullsized":"2024-09-16T01:40:23.709120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:00.154633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680985+0000","last_change":"2024-09-16T01:38:01.519681+0000","last_active":"2024-09-16T01:40:23.680985+0000","last_peered":"2024-09-16T01:40:23.680985+0000","last_clean":"2024-09-16T01:40:23.680985+0000","last_became_active":"2024-09-16T01:38:01.518310+0000","last_became_peered":"2024-09-16T01:38:01.518310+0000","last_unstale":"2024-09-16T01:40:23.680985+0000","last_undegraded":"2024-09-16T01:40:23.680985+0000","last_fullsized":"2024-09-16T01:40:23.680985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:23.308008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:24.556034+0000","last_change":"2024-09-16T01:38:08.559469+0000","last_active":"2024-09-16T01:40:24.556034+0000","last_peered":"2024-09-16T01:40:24.556034+0000","last_clean":"2024-09-16T01:40:24.556034+0000","last_became_active":"2024-09-16T01:38:08.559111+0000","last_became_peered":"2024-09-16T01:38:08.559111+0000","last_unstale":"2024-09-16T01:40:24.556034+0000","last_undegraded":"2024-09-16T01:40:24.556034+0000","last_fullsized":"2024-09-16T01:40:24.556034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:58.577749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709153+0000","last_change":"2024-09-16T01:38:08.555029+0000","last_active":"2024-09-16T01:40:23.709153+0000","last_peered":"2024-09-16T01:40:23.709153+0000","last_clean":"2024-09-16T01:40:23.709153+0000","last_became_active":"2024-09-16T01:38:08.554151+0000","last_became_peered":"2024-09-16T01:38:08.554151+0000","last_unstale":"2024-09-16T01:40:23.709153+0000","last_undegraded":"2024-09-16T01:40:23.709153+0000","last_fullsized":"2024-09-16T01:40:23.709153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:05.005904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682035+0000","last_change":"2024-09-16T01:38:01.528114+0000","last_active":"2024-09-16T01:40:23.682035+0000","last_peered":"2024-09-16T01:40:23.682035+0000","last_clean":"2024-09-16T01:40:23.682035+0000","last_became_active":"2024-09-16T01:38:01.527805+0000","last_became_peered":"2024-09-16T01:38:01.527805+0000","last_unstale":"2024-09-16T01:40:23.682035+0000","last_undegraded":"2024-09-16T01:40:23.682035+0000","last_fullsized":"2024-09-16T01:40:23.682035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:15.830936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680769+0000","last_change":"2024-09-16T01:37:59.507776+0000","last_active":"2024-09-16T01:40:23.680769+0000","last_peered":"2024-09-16T01:40:23.680769+0000","last_clean":"2024-09-16T01:40:23.680769+0000","last_became_active":"2024-09-16T01:37:59.507429+0000","last_became_peered":"2024-09-16T01:37:59.507429+0000","last_unstale":"2024-09-16T01:40:23.680769+0000","last_undegraded":"2024-09-16T01:40:23.680769+0000","last_fullsized":"2024-09-16T01:40:23.680769+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:44:41.274986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707084+0000","last_change":"2024-09-16T01:38:08.562191+0000","last_active":"2024-09-16T01:40:23.707084+0000","last_peered":"2024-09-16T01:40:23.707084+0000","last_clean":"2024-09-16T01:40:23.707084+0000","last_became_active":"2024-09-16T01:38:08.561839+0000","last_became_peered":"2024-09-16T01:38:08.561839+0000","last_unstale":"2024-09-16T01:40:23.707084+0000","last_undegraded":"2024-09-16T01:40:23.707084+0000","last_fullsized":"2024-09-16T01:40:23.707084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:40.618612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716123+0000","last_change":"2024-09-16T01:38:01.518855+0000","last_active":"2024-09-16T01:40:23.716123+0000","last_peered":"2024-09-16T01:40:23.716123+0000","last_clean":"2024-09-16T01:40:23.716123+0000","last_became_active":"2024-09-16T01:38:01.518427+0000","last_became_peered":"2024-09-16T01:38:01.518427+0000","last_unstale":"2024-09-16T01:40:23.716123+0000","last_undegraded":"2024-09-16T01:40:23.716123+0000","last_fullsized":"2024-09-16T01:40:23.716123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:23.054856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918053+0000","last_change":"2024-09-16T01:37:59.799341+0000","last_active":"2024-09-16T01:40:23.918053+0000","last_peered":"2024-09-16T01:40:23.918053+0000","last_clean":"2024-09-16T01:40:23.918053+0000","last_became_active":"2024-09-16T01:37:59.798992+0000","last_became_peered":"2024-09-16T01:37:59.798992+0000","last_unstale":"2024-09-16T01:40:23.918053+0000","last_undegraded":"2024-09-16T01:40:23.918053+0000","last_fullsized":"2024-09-16T01:40:23.918053+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:01.708565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709518+0000","last_change":"2024-09-16T01:39:09.524319+0000","last_active":"2024-09-16T01:40:23.709518+0000","last_peered":"2024-09-16T01:40:23.709518+0000","last_clean":"2024-09-16T01:40:23.709518+0000","last_became_active":"2024-09-16T01:39:09.523717+0000","last_became_peered":"2024-09-16T01:39:09.523717+0000","last_unstale":"2024-09-16T01:40:23.709518+0000","last_undegraded":"2024-09-16T01:40:23.709518+0000","last_fullsized":"2024-09-16T01:40:23.709518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:35.488097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918444+0000","last_change":"2024-09-16T01:38:01.524545+0000","last_active":"2024-09-16T01:40:23.918444+0000","last_peered":"2024-09-16T01:40:23.918444+0000","last_clean":"2024-09-16T01:40:23.918444+0000","last_became_active":"2024-09-16T01:38:01.524253+0000","last_became_peered":"2024-09-16T01:38:01.524253+0000","last_unstale":"2024-09-16T01:40:23.918444+0000","last_undegraded":"2024-09-16T01:40:23.918444+0000","last_fullsized":"2024-09-16T01:40:23.918444+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:08.697500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709429+0000","last_change":"2024-09-16T01:37:59.506779+0000","last_active":"2024-09-16T01:40:23.709429+0000","last_peered":"2024-09-16T01:40:23.709429+0000","last_clean":"2024-09-16T01:40:23.709429+0000","last_became_active":"2024-09-16T01:37:59.505459+0000","last_became_peered":"2024-09-16T01:37:59.505459+0000","last_unstale":"2024-09-16T01:40:23.709429+0000","last_undegraded":"2024-09-16T01:40:23.709429+0000","last_fullsized":"2024-09-16T01:40:23.709429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:12.973330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716178+0000","last_change":"2024-09-16T01:38:08.557250+0000","last_active":"2024-09-16T01:40:23.716178+0000","last_peered":"2024-09-16T01:40:23.716178+0000","last_clean":"2024-09-16T01:40:23.716178+0000","last_became_active":"2024-09-16T01:38:08.555577+0000","last_became_peered":"2024-09-16T01:38:08.555577+0000","last_unstale":"2024-09-16T01:40:23.716178+0000","last_undegraded":"2024-09-16T01:40:23.716178+0000","last_fullsized":"2024-09-16T01:40:23.716178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:54.965341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681174+0000","last_change":"2024-09-16T01:38:01.522910+0000","last_active":"2024-09-16T01:40:23.681174+0000","last_peered":"2024-09-16T01:40:23.681174+0000","last_clean":"2024-09-16T01:40:23.681174+0000","last_became_active":"2024-09-16T01:38:01.521206+0000","last_became_peered":"2024-09-16T01:38:01.521206+0000","last_unstale":"2024-09-16T01:40:23.681174+0000","last_undegraded":"2024-09-16T01:40:23.681174+0000","last_fullsized":"2024-09-16T01:40:23.681174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:45.670464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'52","reported_seq":111,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.879588+0000","last_change":"2024-09-16T01:37:59.505962+0000","last_active":"2024-09-16T01:40:23.879588+0000","last_peered":"2024-09-16T01:40:23.879588+0000","last_clean":"2024-09-16T01:40:23.879588+0000","last_became_active":"2024-09-16T01:37:59.505636+0000","last_became_peered":"2024-09-16T01:37:59.505636+0000","last_unstale":"2024-09-16T01:40:23.879588+0000","last_undegraded":"2024-09-16T01:40:23.879588+0000","last_fullsized":"2024-09-16T01:40:23.879588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:57.548513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680470+0000","last_change":"2024-09-16T01:38:08.571249+0000","last_active":"2024-09-16T01:40:23.680470+0000","last_peered":"2024-09-16T01:40:23.680470+0000","last_clean":"2024-09-16T01:40:23.680470+0000","last_became_active":"2024-09-16T01:38:08.570388+0000","last_became_peered":"2024-09-16T01:38:08.570388+0000","last_unstale":"2024-09-16T01:40:23.680470+0000","last_undegraded":"2024-09-16T01:40:23.680470+0000","last_fullsized":"2024-09-16T01:40:23.680470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:59.904704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706816+0000","last_change":"2024-09-16T01:38:01.522040+0000","last_active":"2024-09-16T01:40:23.706816+0000","last_peered":"2024-09-16T01:40:23.706816+0000","last_clean":"2024-09-16T01:40:23.706816+0000","last_became_active":"2024-09-16T01:38:01.521720+0000","last_became_peered":"2024-09-16T01:38:01.521720+0000","last_unstale":"2024-09-16T01:40:23.706816+0000","last_undegraded":"2024-09-16T01:40:23.706816+0000","last_fullsized":"2024-09-16T01:40:23.706816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:44:48.635603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682302+0000","last_change":"2024-09-16T01:38:08.561870+0000","last_active":"2024-09-16T01:40:23.682302+0000","last_peered":"2024-09-16T01:40:23.682302+0000","last_clean":"2024-09-16T01:40:23.682302+0000","last_became_active":"2024-09-16T01:38:08.561440+0000","last_became_peered":"2024-09-16T01:38:08.561440+0000","last_unstale":"2024-09-16T01:40:23.682302+0000","last_undegraded":"2024-09-16T01:40:23.682302+0000","last_fullsized":"2024-09-16T01:40:23.682302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:43:37.594734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682173+0000","last_change":"2024-09-16T01:38:08.555436+0000","last_active":"2024-09-16T01:40:23.682173+0000","last_peered":"2024-09-16T01:40:23.682173+0000","last_clean":"2024-09-16T01:40:23.682173+0000","last_became_active":"2024-09-16T01:38:08.555026+0000","last_became_peered":"2024-09-16T01:38:08.555026+0000","last_unstale":"2024-09-16T01:40:23.682173+0000","last_undegraded":"2024-09-16T01:40:23.682173+0000","last_fullsized":"2024-09-16T01:40:23.682173+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:28.870214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"85'4","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706753+0000","last_change":"2024-09-16T01:38:01.524482+0000","last_active":"2024-09-16T01:40:23.706753+0000","last_peered":"2024-09-16T01:40:23.706753+0000","last_clean":"2024-09-16T01:40:23.706753+0000","last_became_active":"2024-09-16T01:38:01.524160+0000","last_became_peered":"2024-09-16T01:38:01.524160+0000","last_unstale":"2024-09-16T01:40:23.706753+0000","last_undegraded":"2024-09-16T01:40:23.706753+0000","last_fullsized":"2024-09-16T01:40:23.706753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:39:27.621780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706766+0000","last_change":"2024-09-16T01:37:59.800051+0000","last_active":"2024-09-16T01:40:23.706766+0000","last_peered":"2024-09-16T01:40:23.706766+0000","last_clean":"2024-09-16T01:40:23.706766+0000","last_became_active":"2024-09-16T01:37:59.799606+0000","last_became_peered":"2024-09-16T01:37:59.799606+0000","last_unstale":"2024-09-16T01:40:23.706766+0000","last_undegraded":"2024-09-16T01:40:23.706766+0000","last_fullsized":"2024-09-16T01:40:23.706766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:02.694933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716183+0000","last_change":"2024-09-16T01:38:08.570764+0000","last_active":"2024-09-16T01:40:23.716183+0000","last_peered":"2024-09-16T01:40:23.716183+0000","last_clean":"2024-09-16T01:40:23.716183+0000","last_became_active":"2024-09-16T01:38:08.570385+0000","last_became_peered":"2024-09-16T01:38:08.570385+0000","last_unstale":"2024-09-16T01:40:23.716183+0000","last_undegraded":"2024-09-16T01:40:23.716183+0000","last_fullsized":"2024-09-16T01:40:23.716183+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:25.349671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706565+0000","last_change":"2024-09-16T01:38:01.519590+0000","last_active":"2024-09-16T01:40:23.706565+0000","last_peered":"2024-09-16T01:40:23.706565+0000","last_clean":"2024-09-16T01:40:23.706565+0000","last_became_active":"2024-09-16T01:38:01.519210+0000","last_became_peered":"2024-09-16T01:38:01.519210+0000","last_unstale":"2024-09-16T01:40:23.706565+0000","last_undegraded":"2024-09-16T01:40:23.706565+0000","last_fullsized":"2024-09-16T01:40:23.706565+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:04.228131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707867+0000","last_change":"2024-09-16T01:37:59.505473+0000","last_active":"2024-09-16T01:40:23.707867+0000","last_peered":"2024-09-16T01:40:23.707867+0000","last_clean":"2024-09-16T01:40:23.707867+0000","last_became_active":"2024-09-16T01:37:59.505123+0000","last_became_peered":"2024-09-16T01:37:59.505123+0000","last_unstale":"2024-09-16T01:40:23.707867+0000","last_undegraded":"2024-09-16T01:40:23.707867+0000","last_fullsized":"2024-09-16T01:40:23.707867+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:17.739625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706690+0000","last_change":"2024-09-16T01:38:08.558534+0000","last_active":"2024-09-16T01:40:23.706690+0000","last_peered":"2024-09-16T01:40:23.706690+0000","last_clean":"2024-09-16T01:40:23.706690+0000","last_became_active":"2024-09-16T01:38:08.558185+0000","last_became_peered":"2024-09-16T01:38:08.558185+0000","last_unstale":"2024-09-16T01:40:23.706690+0000","last_undegraded":"2024-09-16T01:40:23.706690+0000","last_fullsized":"2024-09-16T01:40:23.706690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:09.359426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708932+0000","last_change":"2024-09-16T01:38:01.521445+0000","last_active":"2024-09-16T01:40:23.708932+0000","last_peered":"2024-09-16T01:40:23.708932+0000","last_clean":"2024-09-16T01:40:23.708932+0000","last_became_active":"2024-09-16T01:38:01.521140+0000","last_became_peered":"2024-09-16T01:38:01.521140+0000","last_unstale":"2024-09-16T01:40:23.708932+0000","last_undegraded":"2024-09-16T01:40:23.708932+0000","last_fullsized":"2024-09-16T01:40:23.708932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:05.247515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":147,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707420+0000","last_change":"2024-09-16T01:37:59.798365+0000","last_active":"2024-09-16T01:40:23.707420+0000","last_peered":"2024-09-16T01:40:23.707420+0000","last_clean":"2024-09-16T01:40:23.707420+0000","last_became_active":"2024-09-16T01:37:59.798023+0000","last_became_peered":"2024-09-16T01:37:59.798023+0000","last_unstale":"2024-09-16T01:40:23.707420+0000","last_undegraded":"2024-09-16T01:40:23.707420+0000","last_fullsized":"2024-09-16T01:40:23.707420+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:30.867080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707046+0000","last_change":"2024-09-16T01:38:08.571704+0000","last_active":"2024-09-16T01:40:23.707046+0000","last_peered":"2024-09-16T01:40:23.707046+0000","last_clean":"2024-09-16T01:40:23.707046+0000","last_became_active":"2024-09-16T01:38:08.571350+0000","last_became_peered":"2024-09-16T01:38:08.571350+0000","last_unstale":"2024-09-16T01:40:23.707046+0000","last_undegraded":"2024-09-16T01:40:23.707046+0000","last_fullsized":"2024-09-16T01:40:23.707046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:01.698770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706388+0000","last_change":"2024-09-16T01:38:08.559433+0000","last_active":"2024-09-16T01:40:23.706388+0000","last_peered":"2024-09-16T01:40:23.706388+0000","last_clean":"2024-09-16T01:40:23.706388+0000","last_became_active":"2024-09-16T01:38:08.559110+0000","last_became_peered":"2024-09-16T01:38:08.559110+0000","last_unstale":"2024-09-16T01:40:23.706388+0000","last_undegraded":"2024-09-16T01:40:23.706388+0000","last_fullsized":"2024-09-16T01:40:23.706388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:39.001916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681418+0000","last_change":"2024-09-16T01:37:59.508236+0000","last_active":"2024-09-16T01:40:23.681418+0000","last_peered":"2024-09-16T01:40:23.681418+0000","last_clean":"2024-09-16T01:40:23.681418+0000","last_became_active":"2024-09-16T01:37:59.507883+0000","last_became_peered":"2024-09-16T01:37:59.507883+0000","last_unstale":"2024-09-16T01:40:23.681418+0000","last_undegraded":"2024-09-16T01:40:23.681418+0000","last_fullsized":"2024-09-16T01:40:23.681418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:23.948530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709040+0000","last_change":"2024-09-16T01:38:08.572108+0000","last_active":"2024-09-16T01:40:23.709040+0000","last_peered":"2024-09-16T01:40:23.709040+0000","last_clean":"2024-09-16T01:40:23.709040+0000","last_became_active":"2024-09-16T01:38:08.571643+0000","last_became_peered":"2024-09-16T01:38:08.571643+0000","last_unstale":"2024-09-16T01:40:23.709040+0000","last_undegraded":"2024-09-16T01:40:23.709040+0000","last_fullsized":"2024-09-16T01:40:23.709040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:46.636219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680849+0000","last_change":"2024-09-16T01:38:01.517894+0000","last_active":"2024-09-16T01:40:23.680849+0000","last_peered":"2024-09-16T01:40:23.680849+0000","last_clean":"2024-09-16T01:40:23.680849+0000","last_became_active":"2024-09-16T01:38:01.517602+0000","last_became_peered":"2024-09-16T01:38:01.517602+0000","last_unstale":"2024-09-16T01:40:23.680849+0000","last_undegraded":"2024-09-16T01:40:23.680849+0000","last_fullsized":"2024-09-16T01:40:23.680849+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:33.483120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709294+0000","last_change":"2024-09-16T01:37:59.507568+0000","last_active":"2024-09-16T01:40:23.709294+0000","last_peered":"2024-09-16T01:40:23.709294+0000","last_clean":"2024-09-16T01:40:23.709294+0000","last_became_active":"2024-09-16T01:37:59.505798+0000","last_became_peered":"2024-09-16T01:37:59.505798+0000","last_unstale":"2024-09-16T01:40:23.709294+0000","last_undegraded":"2024-09-16T01:40:23.709294+0000","last_fullsized":"2024-09-16T01:40:23.709294+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:19:02.711054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":360,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681514+0000","last_change":"2024-09-16T01:36:46.744332+0000","last_active":"2024-09-16T01:40:23.681514+0000","last_peered":"2024-09-16T01:40:23.681514+0000","last_clean":"2024-09-16T01:40:23.681514+0000","last_became_active":"2024-09-16T01:36:46.743902+0000","last_became_peered":"2024-09-16T01:36:46.743902+0000","last_unstale":"2024-09-16T01:40:23.681514+0000","last_undegraded":"2024-09-16T01:40:23.681514+0000","last_fullsized":"2024-09-16T01:40:23.681514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:33:19.891854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:33:19.891854+0000","last_clean_scrub_stamp":"2024-09-16T01:33:19.891854+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:46.173685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681871+0000","last_change":"2024-09-16T01:39:09.528216+0000","last_active":"2024-09-16T01:40:23.681871+0000","last_peered":"2024-09-16T01:40:23.681871+0000","last_clean":"2024-09-16T01:40:23.681871+0000","last_became_active":"2024-09-16T01:39:09.523634+0000","last_became_peered":"2024-09-16T01:39:09.523634+0000","last_unstale":"2024-09-16T01:40:23.681871+0000","last_undegraded":"2024-09-16T01:40:23.681871+0000","last_fullsized":"2024-09-16T01:40:23.681871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:27.989901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682064+0000","last_change":"2024-09-16T01:38:01.527690+0000","last_active":"2024-09-16T01:40:23.682064+0000","last_peered":"2024-09-16T01:40:23.682064+0000","last_clean":"2024-09-16T01:40:23.682064+0000","last_became_active":"2024-09-16T01:38:01.527339+0000","last_became_peered":"2024-09-16T01:38:01.527339+0000","last_unstale":"2024-09-16T01:40:23.682064+0000","last_undegraded":"2024-09-16T01:40:23.682064+0000","last_fullsized":"2024-09-16T01:40:23.682064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:58.526270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'8","reported_seq":153,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.717334+0000","last_change":"2024-09-16T01:37:59.798874+0000","last_active":"2024-09-16T01:40:23.717334+0000","last_peered":"2024-09-16T01:40:23.717334+0000","last_clean":"2024-09-16T01:40:23.717334+0000","last_became_active":"2024-09-16T01:37:59.798424+0000","last_became_peered":"2024-09-16T01:37:59.798424+0000","last_unstale":"2024-09-16T01:40:23.717334+0000","last_undegraded":"2024-09-16T01:40:23.717334+0000","last_fullsized":"2024-09-16T01:40:23.717334+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:36.648963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918289+0000","last_change":"2024-09-16T01:38:08.555524+0000","last_active":"2024-09-16T01:40:23.918289+0000","last_peered":"2024-09-16T01:40:23.918289+0000","last_clean":"2024-09-16T01:40:23.918289+0000","last_became_active":"2024-09-16T01:38:08.555075+0000","last_became_peered":"2024-09-16T01:38:08.555075+0000","last_unstale":"2024-09-16T01:40:23.918289+0000","last_undegraded":"2024-09-16T01:40:23.918289+0000","last_fullsized":"2024-09-16T01:40:23.918289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:44.847916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681953+0000","last_change":"2024-09-16T01:38:01.528869+0000","last_active":"2024-09-16T01:40:23.681953+0000","last_peered":"2024-09-16T01:40:23.681953+0000","last_clean":"2024-09-16T01:40:23.681953+0000","last_became_active":"2024-09-16T01:38:01.528559+0000","last_became_peered":"2024-09-16T01:38:01.528559+0000","last_unstale":"2024-09-16T01:40:23.681953+0000","last_undegraded":"2024-09-16T01:40:23.681953+0000","last_fullsized":"2024-09-16T01:40:23.681953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:21.547808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'41","reported_seq":86,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.876871+0000","last_change":"2024-09-16T01:37:59.510871+0000","last_active":"2024-09-16T01:40:23.876871+0000","last_peered":"2024-09-16T01:40:23.876871+0000","last_clean":"2024-09-16T01:40:23.876871+0000","last_became_active":"2024-09-16T01:37:59.509996+0000","last_became_peered":"2024-09-16T01:37:59.509996+0000","last_unstale":"2024-09-16T01:40:23.876871+0000","last_undegraded":"2024-09-16T01:40:23.876871+0000","last_fullsized":"2024-09-16T01:40:23.876871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:23.390864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918027+0000","last_change":"2024-09-16T01:38:08.555181+0000","last_active":"2024-09-16T01:40:23.918027+0000","last_peered":"2024-09-16T01:40:23.918027+0000","last_clean":"2024-09-16T01:40:23.918027+0000","last_became_active":"2024-09-16T01:38:08.554778+0000","last_became_peered":"2024-09-16T01:38:08.554778+0000","last_unstale":"2024-09-16T01:40:23.918027+0000","last_undegraded":"2024-09-16T01:40:23.918027+0000","last_fullsized":"2024-09-16T01:40:23.918027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:16:42.369236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682039+0000","last_change":"2024-09-16T01:38:01.526260+0000","last_active":"2024-09-16T01:40:23.682039+0000","last_peered":"2024-09-16T01:40:23.682039+0000","last_clean":"2024-09-16T01:40:23.682039+0000","last_became_active":"2024-09-16T01:38:01.525652+0000","last_became_peered":"2024-09-16T01:38:01.525652+0000","last_unstale":"2024-09-16T01:40:23.682039+0000","last_undegraded":"2024-09-16T01:40:23.682039+0000","last_fullsized":"2024-09-16T01:40:23.682039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:09.180507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707015+0000","last_change":"2024-09-16T01:38:08.570408+0000","last_active":"2024-09-16T01:40:23.707015+0000","last_peered":"2024-09-16T01:40:23.707015+0000","last_clean":"2024-09-16T01:40:23.707015+0000","last_became_active":"2024-09-16T01:38:08.566122+0000","last_became_peered":"2024-09-16T01:38:08.566122+0000","last_unstale":"2024-09-16T01:40:23.707015+0000","last_undegraded":"2024-09-16T01:40:23.707015+0000","last_fullsized":"2024-09-16T01:40:23.707015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:55.832320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708804+0000","last_change":"2024-09-16T01:38:08.555063+0000","last_active":"2024-09-16T01:40:23.708804+0000","last_peered":"2024-09-16T01:40:23.708804+0000","last_clean":"2024-09-16T01:40:23.708804+0000","last_became_active":"2024-09-16T01:38:08.554665+0000","last_became_peered":"2024-09-16T01:38:08.554665+0000","last_unstale":"2024-09-16T01:40:23.708804+0000","last_undegraded":"2024-09-16T01:40:23.708804+0000","last_fullsized":"2024-09-16T01:40:23.708804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:50.091017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682633+0000","last_change":"2024-09-16T01:38:01.524247+0000","last_active":"2024-09-16T01:40:23.682633+0000","last_peered":"2024-09-16T01:40:23.682633+0000","last_clean":"2024-09-16T01:40:23.682633+0000","last_became_active":"2024-09-16T01:38:01.523911+0000","last_became_peered":"2024-09-16T01:38:01.523911+0000","last_unstale":"2024-09-16T01:40:23.682633+0000","last_undegraded":"2024-09-16T01:40:23.682633+0000","last_fullsized":"2024-09-16T01:40:23.682633+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:32.745882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716800+0000","last_change":"2024-09-16T01:37:59.511272+0000","last_active":"2024-09-16T01:40:23.716800+0000","last_peered":"2024-09-16T01:40:23.716800+0000","last_clean":"2024-09-16T01:40:23.716800+0000","last_became_active":"2024-09-16T01:37:59.510920+0000","last_became_peered":"2024-09-16T01:37:59.510920+0000","last_unstale":"2024-09-16T01:40:23.716800+0000","last_undegraded":"2024-09-16T01:40:23.716800+0000","last_fullsized":"2024-09-16T01:40:23.716800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:59.406175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708902+0000","last_change":"2024-09-16T01:38:08.567173+0000","last_active":"2024-09-16T01:40:23.708902+0000","last_peered":"2024-09-16T01:40:23.708902+0000","last_clean":"2024-09-16T01:40:23.708902+0000","last_became_active":"2024-09-16T01:38:08.566788+0000","last_became_peered":"2024-09-16T01:38:08.566788+0000","last_unstale":"2024-09-16T01:40:23.708902+0000","last_undegraded":"2024-09-16T01:40:23.708902+0000","last_fullsized":"2024-09-16T01:40:23.708902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:35.743356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709105+0000","last_change":"2024-09-16T01:38:01.529529+0000","last_active":"2024-09-16T01:40:23.709105+0000","last_peered":"2024-09-16T01:40:23.709105+0000","last_clean":"2024-09-16T01:40:23.709105+0000","last_became_active":"2024-09-16T01:38:01.529048+0000","last_became_peered":"2024-09-16T01:38:01.529048+0000","last_unstale":"2024-09-16T01:40:23.709105+0000","last_undegraded":"2024-09-16T01:40:23.709105+0000","last_fullsized":"2024-09-16T01:40:23.709105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:35.041662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716379+0000","last_change":"2024-09-16T01:37:59.799536+0000","last_active":"2024-09-16T01:40:23.716379+0000","last_peered":"2024-09-16T01:40:23.716379+0000","last_clean":"2024-09-16T01:40:23.716379+0000","last_became_active":"2024-09-16T01:37:59.799109+0000","last_became_peered":"2024-09-16T01:37:59.799109+0000","last_unstale":"2024-09-16T01:40:23.716379+0000","last_undegraded":"2024-09-16T01:40:23.716379+0000","last_fullsized":"2024-09-16T01:40:23.716379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:07:25.007434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707289+0000","last_change":"2024-09-16T01:38:08.567019+0000","last_active":"2024-09-16T01:40:23.707289+0000","last_peered":"2024-09-16T01:40:23.707289+0000","last_clean":"2024-09-16T01:40:23.707289+0000","last_became_active":"2024-09-16T01:38:08.566667+0000","last_became_peered":"2024-09-16T01:38:08.566667+0000","last_unstale":"2024-09-16T01:40:23.707289+0000","last_undegraded":"2024-09-16T01:40:23.707289+0000","last_fullsized":"2024-09-16T01:40:23.707289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:11.375095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681386+0000","last_change":"2024-09-16T01:38:01.527965+0000","last_active":"2024-09-16T01:40:23.681386+0000","last_peered":"2024-09-16T01:40:23.681386+0000","last_clean":"2024-09-16T01:40:23.681386+0000","last_became_active":"2024-09-16T01:38:01.526688+0000","last_became_peered":"2024-09-16T01:38:01.526688+0000","last_unstale":"2024-09-16T01:40:23.681386+0000","last_undegraded":"2024-09-16T01:40:23.681386+0000","last_fullsized":"2024-09-16T01:40:23.681386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:06.618756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":149,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.923723+0000","last_change":"2024-09-16T01:37:59.510331+0000","last_active":"2024-09-16T01:40:23.923723+0000","last_peered":"2024-09-16T01:40:23.923723+0000","last_clean":"2024-09-16T01:40:23.923723+0000","last_became_active":"2024-09-16T01:37:59.509769+0000","last_became_peered":"2024-09-16T01:37:59.509769+0000","last_unstale":"2024-09-16T01:40:23.923723+0000","last_undegraded":"2024-09-16T01:40:23.923723+0000","last_fullsized":"2024-09-16T01:40:23.923723+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:18.843091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716548+0000","last_change":"2024-09-16T01:39:10.049460+0000","last_active":"2024-09-16T01:40:23.716548+0000","last_peered":"2024-09-16T01:40:23.716548+0000","last_clean":"2024-09-16T01:40:23.716548+0000","last_became_active":"2024-09-16T01:39:10.048862+0000","last_became_peered":"2024-09-16T01:39:10.048862+0000","last_unstale":"2024-09-16T01:40:23.716548+0000","last_undegraded":"2024-09-16T01:40:23.716548+0000","last_fullsized":"2024-09-16T01:40:23.716548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:37.329806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681820+0000","last_change":"2024-09-16T01:38:01.521254+0000","last_active":"2024-09-16T01:40:23.681820+0000","last_peered":"2024-09-16T01:40:23.681820+0000","last_clean":"2024-09-16T01:40:23.681820+0000","last_became_active":"2024-09-16T01:38:01.520901+0000","last_became_peered":"2024-09-16T01:38:01.520901+0000","last_unstale":"2024-09-16T01:40:23.681820+0000","last_undegraded":"2024-09-16T01:40:23.681820+0000","last_fullsized":"2024-09-16T01:40:23.681820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:27.806954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709051+0000","last_change":"2024-09-16T01:37:59.505990+0000","last_active":"2024-09-16T01:40:23.709051+0000","last_peered":"2024-09-16T01:40:23.709051+0000","last_clean":"2024-09-16T01:40:23.709051+0000","last_became_active":"2024-09-16T01:37:59.505564+0000","last_became_peered":"2024-09-16T01:37:59.505564+0000","last_unstale":"2024-09-16T01:40:23.709051+0000","last_undegraded":"2024-09-16T01:40:23.709051+0000","last_fullsized":"2024-09-16T01:40:23.709051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:01.496609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682127+0000","last_change":"2024-09-16T01:38:08.562838+0000","last_active":"2024-09-16T01:40:23.682127+0000","last_peered":"2024-09-16T01:40:23.682127+0000","last_clean":"2024-09-16T01:40:23.682127+0000","last_became_active":"2024-09-16T01:38:08.562426+0000","last_became_peered":"2024-09-16T01:38:08.562426+0000","last_unstale":"2024-09-16T01:40:23.682127+0000","last_undegraded":"2024-09-16T01:40:23.682127+0000","last_fullsized":"2024-09-16T01:40:23.682127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:20.685626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707454+0000","last_change":"2024-09-16T01:38:01.514983+0000","last_active":"2024-09-16T01:40:23.707454+0000","last_peered":"2024-09-16T01:40:23.707454+0000","last_clean":"2024-09-16T01:40:23.707454+0000","last_became_active":"2024-09-16T01:38:01.514648+0000","last_became_peered":"2024-09-16T01:38:01.514648+0000","last_unstale":"2024-09-16T01:40:23.707454+0000","last_undegraded":"2024-09-16T01:40:23.707454+0000","last_fullsized":"2024-09-16T01:40:23.707454+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:57.052564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.717151+0000","last_change":"2024-09-16T01:37:59.505141+0000","last_active":"2024-09-16T01:40:23.717151+0000","last_peered":"2024-09-16T01:40:23.717151+0000","last_clean":"2024-09-16T01:40:23.717151+0000","last_became_active":"2024-09-16T01:37:59.504730+0000","last_became_peered":"2024-09-16T01:37:59.504730+0000","last_unstale":"2024-09-16T01:40:23.717151+0000","last_undegraded":"2024-09-16T01:40:23.717151+0000","last_fullsized":"2024-09-16T01:40:23.717151+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:54.504661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709184+0000","last_change":"2024-09-16T01:38:08.563970+0000","last_active":"2024-09-16T01:40:23.709184+0000","last_peered":"2024-09-16T01:40:23.709184+0000","last_clean":"2024-09-16T01:40:23.709184+0000","last_became_active":"2024-09-16T01:38:08.563555+0000","last_became_peered":"2024-09-16T01:38:08.563555+0000","last_unstale":"2024-09-16T01:40:23.709184+0000","last_undegraded":"2024-09-16T01:40:23.709184+0000","last_fullsized":"2024-09-16T01:40:23.709184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:27:12.086043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716549+0000","last_change":"2024-09-16T01:38:01.521016+0000","last_active":"2024-09-16T01:40:23.716549+0000","last_peered":"2024-09-16T01:40:23.716549+0000","last_clean":"2024-09-16T01:40:23.716549+0000","last_became_active":"2024-09-16T01:38:01.520656+0000","last_became_peered":"2024-09-16T01:38:01.520656+0000","last_unstale":"2024-09-16T01:40:23.716549+0000","last_undegraded":"2024-09-16T01:40:23.716549+0000","last_fullsized":"2024-09-16T01:40:23.716549+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:39.001239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'41","reported_seq":297,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.923336+0000","last_change":"2024-09-16T01:37:59.507235+0000","last_active":"2024-09-16T01:40:23.923336+0000","last_peered":"2024-09-16T01:40:23.923336+0000","last_clean":"2024-09-16T01:40:23.923336+0000","last_became_active":"2024-09-16T01:37:59.506255+0000","last_became_peered":"2024-09-16T01:37:59.506255+0000","last_unstale":"2024-09-16T01:40:23.923336+0000","last_undegraded":"2024-09-16T01:40:23.923336+0000","last_fullsized":"2024-09-16T01:40:23.923336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:44.515663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68792,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":219,"num_read_kb":171,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706438+0000","last_change":"2024-09-16T01:39:09.522688+0000","last_active":"2024-09-16T01:40:23.706438+0000","last_peered":"2024-09-16T01:40:23.706438+0000","last_clean":"2024-09-16T01:40:23.706438+0000","last_became_active":"2024-09-16T01:39:09.522137+0000","last_became_peered":"2024-09-16T01:39:09.522137+0000","last_unstale":"2024-09-16T01:40:23.706438+0000","last_undegraded":"2024-09-16T01:40:23.706438+0000","last_fullsized":"2024-09-16T01:40:23.706438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.004315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918498+0000","last_change":"2024-09-16T01:38:01.526641+0000","last_active":"2024-09-16T01:40:23.918498+0000","last_peered":"2024-09-16T01:40:23.918498+0000","last_clean":"2024-09-16T01:40:23.918498+0000","last_became_active":"2024-09-16T01:38:01.526278+0000","last_became_peered":"2024-09-16T01:38:01.526278+0000","last_unstale":"2024-09-16T01:40:23.918498+0000","last_undegraded":"2024-09-16T01:40:23.918498+0000","last_fullsized":"2024-09-16T01:40:23.918498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:56.156593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709392+0000","last_change":"2024-09-16T01:37:59.800065+0000","last_active":"2024-09-16T01:40:23.709392+0000","last_peered":"2024-09-16T01:40:23.709392+0000","last_clean":"2024-09-16T01:40:23.709392+0000","last_became_active":"2024-09-16T01:37:59.799705+0000","last_became_peered":"2024-09-16T01:37:59.799705+0000","last_unstale":"2024-09-16T01:40:23.709392+0000","last_undegraded":"2024-09-16T01:40:23.709392+0000","last_fullsized":"2024-09-16T01:40:23.709392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:13.791875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709196+0000","last_change":"2024-09-16T01:38:08.556360+0000","last_active":"2024-09-16T01:40:23.709196+0000","last_peered":"2024-09-16T01:40:23.709196+0000","last_clean":"2024-09-16T01:40:23.709196+0000","last_became_active":"2024-09-16T01:38:08.555219+0000","last_became_peered":"2024-09-16T01:38:08.555219+0000","last_unstale":"2024-09-16T01:40:23.709196+0000","last_undegraded":"2024-09-16T01:40:23.709196+0000","last_fullsized":"2024-09-16T01:40:23.709196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:36.464760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918517+0000","last_change":"2024-09-16T01:38:01.526758+0000","last_active":"2024-09-16T01:40:23.918517+0000","last_peered":"2024-09-16T01:40:23.918517+0000","last_clean":"2024-09-16T01:40:23.918517+0000","last_became_active":"2024-09-16T01:38:01.526545+0000","last_became_peered":"2024-09-16T01:38:01.526545+0000","last_unstale":"2024-09-16T01:40:23.918517+0000","last_undegraded":"2024-09-16T01:40:23.918517+0000","last_fullsized":"2024-09-16T01:40:23.918517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:47.185489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682092+0000","last_change":"2024-09-16T01:38:08.555125+0000","last_active":"2024-09-16T01:40:23.682092+0000","last_peered":"2024-09-16T01:40:23.682092+0000","last_clean":"2024-09-16T01:40:23.682092+0000","last_became_active":"2024-09-16T01:38:08.554756+0000","last_became_peered":"2024-09-16T01:38:08.554756+0000","last_unstale":"2024-09-16T01:40:23.682092+0000","last_undegraded":"2024-09-16T01:40:23.682092+0000","last_fullsized":"2024-09-16T01:40:23.682092+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:08.888006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.705898+0000","last_change":"2024-09-16T01:38:08.557906+0000","last_active":"2024-09-16T01:40:23.705898+0000","last_peered":"2024-09-16T01:40:23.705898+0000","last_clean":"2024-09-16T01:40:23.705898+0000","last_became_active":"2024-09-16T01:38:08.557546+0000","last_became_peered":"2024-09-16T01:38:08.557546+0000","last_unstale":"2024-09-16T01:40:23.705898+0000","last_undegraded":"2024-09-16T01:40:23.705898+0000","last_fullsized":"2024-09-16T01:40:23.705898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:27:04.988778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918558+0000","last_change":"2024-09-16T01:38:01.528430+0000","last_active":"2024-09-16T01:40:23.918558+0000","last_peered":"2024-09-16T01:40:23.918558+0000","last_clean":"2024-09-16T01:40:23.918558+0000","last_became_active":"2024-09-16T01:38:01.528069+0000","last_became_peered":"2024-09-16T01:38:01.528069+0000","last_unstale":"2024-09-16T01:40:23.918558+0000","last_undegraded":"2024-09-16T01:40:23.918558+0000","last_fullsized":"2024-09-16T01:40:23.918558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:41.317215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'6","reported_seq":251,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.916402+0000","last_change":"2024-09-16T01:37:59.509017+0000","last_active":"2024-09-16T01:40:23.916402+0000","last_peered":"2024-09-16T01:40:23.916402+0000","last_clean":"2024-09-16T01:40:23.916402+0000","last_became_active":"2024-09-16T01:37:59.508252+0000","last_became_peered":"2024-09-16T01:37:59.508252+0000","last_unstale":"2024-09-16T01:40:23.916402+0000","last_undegraded":"2024-09-16T01:40:23.916402+0000","last_fullsized":"2024-09-16T01:40:23.916402+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:26.215407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":208,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680587+0000","last_change":"2024-09-16T01:38:08.565197+0000","last_active":"2024-09-16T01:40:23.680587+0000","last_peered":"2024-09-16T01:40:23.680587+0000","last_clean":"2024-09-16T01:40:23.680587+0000","last_became_active":"2024-09-16T01:38:08.562773+0000","last_became_peered":"2024-09-16T01:38:08.562773+0000","last_unstale":"2024-09-16T01:40:23.680587+0000","last_undegraded":"2024-09-16T01:40:23.680587+0000","last_fullsized":"2024-09-16T01:40:23.680587+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:13.316950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918230+0000","last_change":"2024-09-16T01:38:08.554052+0000","last_active":"2024-09-16T01:40:23.918230+0000","last_peered":"2024-09-16T01:40:23.918230+0000","last_clean":"2024-09-16T01:40:23.918230+0000","last_became_active":"2024-09-16T01:38:08.553660+0000","last_became_peered":"2024-09-16T01:38:08.553660+0000","last_unstale":"2024-09-16T01:40:23.918230+0000","last_undegraded":"2024-09-16T01:40:23.918230+0000","last_fullsized":"2024-09-16T01:40:23.918230+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:53:52.270695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706069+0000","last_change":"2024-09-16T01:38:01.522688+0000","last_active":"2024-09-16T01:40:23.706069+0000","last_peered":"2024-09-16T01:40:23.706069+0000","last_clean":"2024-09-16T01:40:23.706069+0000","last_became_active":"2024-09-16T01:38:01.519843+0000","last_became_peered":"2024-09-16T01:38:01.519843+0000","last_unstale":"2024-09-16T01:40:23.706069+0000","last_undegraded":"2024-09-16T01:40:23.706069+0000","last_fullsized":"2024-09-16T01:40:23.706069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:50.971760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709088+0000","last_change":"2024-09-16T01:37:59.509044+0000","last_active":"2024-09-16T01:40:23.709088+0000","last_peered":"2024-09-16T01:40:23.709088+0000","last_clean":"2024-09-16T01:40:23.709088+0000","last_became_active":"2024-09-16T01:37:59.507085+0000","last_became_peered":"2024-09-16T01:37:59.507085+0000","last_unstale":"2024-09-16T01:40:23.709088+0000","last_undegraded":"2024-09-16T01:40:23.709088+0000","last_fullsized":"2024-09-16T01:40:23.709088+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:25:09.930945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682550+0000","last_change":"2024-09-16T01:38:08.561250+0000","last_active":"2024-09-16T01:40:23.682550+0000","last_peered":"2024-09-16T01:40:23.682550+0000","last_clean":"2024-09-16T01:40:23.682550+0000","last_became_active":"2024-09-16T01:38:08.560866+0000","last_became_peered":"2024-09-16T01:38:08.560866+0000","last_unstale":"2024-09-16T01:40:23.682550+0000","last_undegraded":"2024-09-16T01:40:23.682550+0000","last_fullsized":"2024-09-16T01:40:23.682550+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:31.253817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680608+0000","last_change":"2024-09-16T01:38:01.517400+0000","last_active":"2024-09-16T01:40:23.680608+0000","last_peered":"2024-09-16T01:40:23.680608+0000","last_clean":"2024-09-16T01:40:23.680608+0000","last_became_active":"2024-09-16T01:38:01.517103+0000","last_became_peered":"2024-09-16T01:38:01.517103+0000","last_unstale":"2024-09-16T01:40:23.680608+0000","last_undegraded":"2024-09-16T01:40:23.680608+0000","last_fullsized":"2024-09-16T01:40:23.680608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:37.886179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706415+0000","last_change":"2024-09-16T01:37:59.503370+0000","last_active":"2024-09-16T01:40:23.706415+0000","last_peered":"2024-09-16T01:40:23.706415+0000","last_clean":"2024-09-16T01:40:23.706415+0000","last_became_active":"2024-09-16T01:37:59.503071+0000","last_became_peered":"2024-09-16T01:37:59.503071+0000","last_unstale":"2024-09-16T01:40:23.706415+0000","last_undegraded":"2024-09-16T01:40:23.706415+0000","last_fullsized":"2024-09-16T01:40:23.706415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:51.207702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706631+0000","last_change":"2024-09-16T01:37:59.506110+0000","last_active":"2024-09-16T01:40:23.706631+0000","last_peered":"2024-09-16T01:40:23.706631+0000","last_clean":"2024-09-16T01:40:23.706631+0000","last_became_active":"2024-09-16T01:37:59.505264+0000","last_became_peered":"2024-09-16T01:37:59.505264+0000","last_unstale":"2024-09-16T01:40:23.706631+0000","last_undegraded":"2024-09-16T01:40:23.706631+0000","last_fullsized":"2024-09-16T01:40:23.706631+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:35.986433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709024+0000","last_change":"2024-09-16T01:38:08.563900+0000","last_active":"2024-09-16T01:40:23.709024+0000","last_peered":"2024-09-16T01:40:23.709024+0000","last_clean":"2024-09-16T01:40:23.709024+0000","last_became_active":"2024-09-16T01:38:08.563551+0000","last_became_peered":"2024-09-16T01:38:08.563551+0000","last_unstale":"2024-09-16T01:40:23.709024+0000","last_undegraded":"2024-09-16T01:40:23.709024+0000","last_fullsized":"2024-09-16T01:40:23.709024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:31:03.979580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716744+0000","last_change":"2024-09-16T01:38:08.556011+0000","last_active":"2024-09-16T01:40:23.716744+0000","last_peered":"2024-09-16T01:40:23.716744+0000","last_clean":"2024-09-16T01:40:23.716744+0000","last_became_active":"2024-09-16T01:38:08.555615+0000","last_became_peered":"2024-09-16T01:38:08.555615+0000","last_unstale":"2024-09-16T01:40:23.716744+0000","last_undegraded":"2024-09-16T01:40:23.716744+0000","last_fullsized":"2024-09-16T01:40:23.716744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:34.434974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":147,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709207+0000","last_change":"2024-09-16T01:37:59.508411+0000","last_active":"2024-09-16T01:40:23.709207+0000","last_peered":"2024-09-16T01:40:23.709207+0000","last_clean":"2024-09-16T01:40:23.709207+0000","last_became_active":"2024-09-16T01:37:59.508058+0000","last_became_peered":"2024-09-16T01:37:59.508058+0000","last_unstale":"2024-09-16T01:40:23.709207+0000","last_undegraded":"2024-09-16T01:40:23.709207+0000","last_fullsized":"2024-09-16T01:40:23.709207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:47.441673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716319+0000","last_change":"2024-09-16T01:38:08.560930+0000","last_active":"2024-09-16T01:40:23.716319+0000","last_peered":"2024-09-16T01:40:23.716319+0000","last_clean":"2024-09-16T01:40:23.716319+0000","last_became_active":"2024-09-16T01:38:08.560490+0000","last_became_peered":"2024-09-16T01:38:08.560490+0000","last_unstale":"2024-09-16T01:40:23.716319+0000","last_undegraded":"2024-09-16T01:40:23.716319+0000","last_fullsized":"2024-09-16T01:40:23.716319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:11.556113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716818+0000","last_change":"2024-09-16T01:38:08.557564+0000","last_active":"2024-09-16T01:40:23.716818+0000","last_peered":"2024-09-16T01:40:23.716818+0000","last_clean":"2024-09-16T01:40:23.716818+0000","last_became_active":"2024-09-16T01:38:08.557143+0000","last_became_peered":"2024-09-16T01:38:08.557143+0000","last_unstale":"2024-09-16T01:40:23.716818+0000","last_undegraded":"2024-09-16T01:40:23.716818+0000","last_fullsized":"2024-09-16T01:40:23.716818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:43.148870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681960+0000","last_change":"2024-09-16T01:38:01.527353+0000","last_active":"2024-09-16T01:40:23.681960+0000","last_peered":"2024-09-16T01:40:23.681960+0000","last_clean":"2024-09-16T01:40:23.681960+0000","last_became_active":"2024-09-16T01:38:01.527031+0000","last_became_peered":"2024-09-16T01:38:01.527031+0000","last_unstale":"2024-09-16T01:40:23.681960+0000","last_undegraded":"2024-09-16T01:40:23.681960+0000","last_fullsized":"2024-09-16T01:40:23.681960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:27.311638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716301+0000","last_change":"2024-09-16T01:38:01.526409+0000","last_active":"2024-09-16T01:40:23.716301+0000","last_peered":"2024-09-16T01:40:23.716301+0000","last_clean":"2024-09-16T01:40:23.716301+0000","last_became_active":"2024-09-16T01:38:01.526064+0000","last_became_peered":"2024-09-16T01:38:01.526064+0000","last_unstale":"2024-09-16T01:40:23.716301+0000","last_undegraded":"2024-09-16T01:40:23.716301+0000","last_fullsized":"2024-09-16T01:40:23.716301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:51.571808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716432+0000","last_change":"2024-09-16T01:38:01.521856+0000","last_active":"2024-09-16T01:40:23.716432+0000","last_peered":"2024-09-16T01:40:23.716432+0000","last_clean":"2024-09-16T01:40:23.716432+0000","last_became_active":"2024-09-16T01:38:01.520312+0000","last_became_peered":"2024-09-16T01:38:01.520312+0000","last_unstale":"2024-09-16T01:40:23.716432+0000","last_undegraded":"2024-09-16T01:40:23.716432+0000","last_fullsized":"2024-09-16T01:40:23.716432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:53.618990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918080+0000","last_change":"2024-09-16T01:38:01.520958+0000","last_active":"2024-09-16T01:40:23.918080+0000","last_peered":"2024-09-16T01:40:23.918080+0000","last_clean":"2024-09-16T01:40:23.918080+0000","last_became_active":"2024-09-16T01:38:01.520658+0000","last_became_peered":"2024-09-16T01:38:01.520658+0000","last_unstale":"2024-09-16T01:40:23.918080+0000","last_undegraded":"2024-09-16T01:40:23.918080+0000","last_fullsized":"2024-09-16T01:40:23.918080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:23.770694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707380+0000","last_change":"2024-09-16T01:38:01.519140+0000","last_active":"2024-09-16T01:40:23.707380+0000","last_peered":"2024-09-16T01:40:23.707380+0000","last_clean":"2024-09-16T01:40:23.707380+0000","last_became_active":"2024-09-16T01:38:01.518765+0000","last_became_peered":"2024-09-16T01:38:01.518765+0000","last_unstale":"2024-09-16T01:40:23.707380+0000","last_undegraded":"2024-09-16T01:40:23.707380+0000","last_fullsized":"2024-09-16T01:40:23.707380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:17.194886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682443+0000","last_change":"2024-09-16T01:38:01.523762+0000","last_active":"2024-09-16T01:40:23.682443+0000","last_peered":"2024-09-16T01:40:23.682443+0000","last_clean":"2024-09-16T01:40:23.682443+0000","last_became_active":"2024-09-16T01:38:01.522035+0000","last_became_peered":"2024-09-16T01:38:01.522035+0000","last_unstale":"2024-09-16T01:40:23.682443+0000","last_undegraded":"2024-09-16T01:40:23.682443+0000","last_fullsized":"2024-09-16T01:40:23.682443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:40.405350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.717020+0000","last_change":"2024-09-16T01:38:01.527539+0000","last_active":"2024-09-16T01:40:23.717020+0000","last_peered":"2024-09-16T01:40:23.717020+0000","last_clean":"2024-09-16T01:40:23.717020+0000","last_became_active":"2024-09-16T01:38:01.526974+0000","last_became_peered":"2024-09-16T01:38:01.526974+0000","last_unstale":"2024-09-16T01:40:23.717020+0000","last_undegraded":"2024-09-16T01:40:23.717020+0000","last_fullsized":"2024-09-16T01:40:23.717020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:30:03.587141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681144+0000","last_change":"2024-09-16T01:38:01.529131+0000","last_active":"2024-09-16T01:40:23.681144+0000","last_peered":"2024-09-16T01:40:23.681144+0000","last_clean":"2024-09-16T01:40:23.681144+0000","last_became_active":"2024-09-16T01:38:01.528796+0000","last_became_peered":"2024-09-16T01:38:01.528796+0000","last_unstale":"2024-09-16T01:40:23.681144+0000","last_undegraded":"2024-09-16T01:40:23.681144+0000","last_fullsized":"2024-09-16T01:40:23.681144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:38.903638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708881+0000","last_change":"2024-09-16T01:38:01.514779+0000","last_active":"2024-09-16T01:40:23.708881+0000","last_peered":"2024-09-16T01:40:23.708881+0000","last_clean":"2024-09-16T01:40:23.708881+0000","last_became_active":"2024-09-16T01:38:01.514434+0000","last_became_peered":"2024-09-16T01:38:01.514434+0000","last_unstale":"2024-09-16T01:40:23.708881+0000","last_undegraded":"2024-09-16T01:40:23.708881+0000","last_fullsized":"2024-09-16T01:40:23.708881+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:31.382111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680678+0000","last_change":"2024-09-16T01:38:01.521771+0000","last_active":"2024-09-16T01:40:23.680678+0000","last_peered":"2024-09-16T01:40:23.680678+0000","last_clean":"2024-09-16T01:40:23.680678+0000","last_became_active":"2024-09-16T01:38:01.521288+0000","last_became_peered":"2024-09-16T01:38:01.521288+0000","last_unstale":"2024-09-16T01:40:23.680678+0000","last_undegraded":"2024-09-16T01:40:23.680678+0000","last_fullsized":"2024-09-16T01:40:23.680678+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:11.825164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918399+0000","last_change":"2024-09-16T01:38:01.529247+0000","last_active":"2024-09-16T01:40:23.918399+0000","last_peered":"2024-09-16T01:40:23.918399+0000","last_clean":"2024-09-16T01:40:23.918399+0000","last_became_active":"2024-09-16T01:38:01.528926+0000","last_became_peered":"2024-09-16T01:38:01.528926+0000","last_unstale":"2024-09-16T01:40:23.918399+0000","last_undegraded":"2024-09-16T01:40:23.918399+0000","last_fullsized":"2024-09-16T01:40:23.918399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:20.704114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707157+0000","last_change":"2024-09-16T01:38:01.520830+0000","last_active":"2024-09-16T01:40:23.707157+0000","last_peered":"2024-09-16T01:40:23.707157+0000","last_clean":"2024-09-16T01:40:23.707157+0000","last_became_active":"2024-09-16T01:38:01.520526+0000","last_became_peered":"2024-09-16T01:38:01.520526+0000","last_unstale":"2024-09-16T01:40:23.707157+0000","last_undegraded":"2024-09-16T01:40:23.707157+0000","last_fullsized":"2024-09-16T01:40:23.707157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:28.062812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682262+0000","last_change":"2024-09-16T01:38:01.521983+0000","last_active":"2024-09-16T01:40:23.682262+0000","last_peered":"2024-09-16T01:40:23.682262+0000","last_clean":"2024-09-16T01:40:23.682262+0000","last_became_active":"2024-09-16T01:38:01.521666+0000","last_became_peered":"2024-09-16T01:38:01.521666+0000","last_unstale":"2024-09-16T01:40:23.682262+0000","last_undegraded":"2024-09-16T01:40:23.682262+0000","last_fullsized":"2024-09-16T01:40:23.682262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:50:55.761313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.717006+0000","last_change":"2024-09-16T01:38:08.554304+0000","last_active":"2024-09-16T01:40:23.717006+0000","last_peered":"2024-09-16T01:40:23.717006+0000","last_clean":"2024-09-16T01:40:23.717006+0000","last_became_active":"2024-09-16T01:38:08.553683+0000","last_became_peered":"2024-09-16T01:38:08.553683+0000","last_unstale":"2024-09-16T01:40:23.717006+0000","last_undegraded":"2024-09-16T01:40:23.717006+0000","last_fullsized":"2024-09-16T01:40:23.717006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:56.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706919+0000","last_change":"2024-09-16T01:38:01.520425+0000","last_active":"2024-09-16T01:40:23.706919+0000","last_peered":"2024-09-16T01:40:23.706919+0000","last_clean":"2024-09-16T01:40:23.706919+0000","last_became_active":"2024-09-16T01:38:01.519858+0000","last_became_peered":"2024-09-16T01:38:01.519858+0000","last_unstale":"2024-09-16T01:40:23.706919+0000","last_undegraded":"2024-09-16T01:40:23.706919+0000","last_fullsized":"2024-09-16T01:40:23.706919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:48.534298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682855+0000","last_change":"2024-09-16T01:38:01.532762+0000","last_active":"2024-09-16T01:40:23.682855+0000","last_peered":"2024-09-16T01:40:23.682855+0000","last_clean":"2024-09-16T01:40:23.682855+0000","last_became_active":"2024-09-16T01:38:01.532404+0000","last_became_peered":"2024-09-16T01:38:01.532404+0000","last_unstale":"2024-09-16T01:40:23.682855+0000","last_undegraded":"2024-09-16T01:40:23.682855+0000","last_fullsized":"2024-09-16T01:40:23.682855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:03.135141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918405+0000","last_change":"2024-09-16T01:38:01.530205+0000","last_active":"2024-09-16T01:40:23.918405+0000","last_peered":"2024-09-16T01:40:23.918405+0000","last_clean":"2024-09-16T01:40:23.918405+0000","last_became_active":"2024-09-16T01:38:01.529898+0000","last_became_peered":"2024-09-16T01:38:01.529898+0000","last_unstale":"2024-09-16T01:40:23.918405+0000","last_undegraded":"2024-09-16T01:40:23.918405+0000","last_fullsized":"2024-09-16T01:40:23.918405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:48.444441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.917747+0000","last_change":"2024-09-16T01:38:01.529407+0000","last_active":"2024-09-16T01:40:23.917747+0000","last_peered":"2024-09-16T01:40:23.917747+0000","last_clean":"2024-09-16T01:40:23.917747+0000","last_became_active":"2024-09-16T01:38:01.528330+0000","last_became_peered":"2024-09-16T01:38:01.528330+0000","last_unstale":"2024-09-16T01:40:23.917747+0000","last_undegraded":"2024-09-16T01:40:23.917747+0000","last_fullsized":"2024-09-16T01:40:23.917747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:32.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.917675+0000","last_change":"2024-09-16T01:38:01.519579+0000","last_active":"2024-09-16T01:40:23.917675+0000","last_peered":"2024-09-16T01:40:23.917675+0000","last_clean":"2024-09-16T01:40:23.917675+0000","last_became_active":"2024-09-16T01:38:01.519211+0000","last_became_peered":"2024-09-16T01:38:01.519211+0000","last_unstale":"2024-09-16T01:40:23.917675+0000","last_undegraded":"2024-09-16T01:40:23.917675+0000","last_fullsized":"2024-09-16T01:40:23.917675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:18.861169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707196+0000","last_change":"2024-09-16T01:38:01.521136+0000","last_active":"2024-09-16T01:40:23.707196+0000","last_peered":"2024-09-16T01:40:23.707196+0000","last_clean":"2024-09-16T01:40:23.707196+0000","last_became_active":"2024-09-16T01:38:01.520819+0000","last_became_peered":"2024-09-16T01:38:01.520819+0000","last_unstale":"2024-09-16T01:40:23.707196+0000","last_undegraded":"2024-09-16T01:40:23.707196+0000","last_fullsized":"2024-09-16T01:40:23.707196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:28.150822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709215+0000","last_change":"2024-09-16T01:38:01.519382+0000","last_active":"2024-09-16T01:40:23.709215+0000","last_peered":"2024-09-16T01:40:23.709215+0000","last_clean":"2024-09-16T01:40:23.709215+0000","last_became_active":"2024-09-16T01:38:01.519039+0000","last_became_peered":"2024-09-16T01:38:01.519039+0000","last_unstale":"2024-09-16T01:40:23.709215+0000","last_undegraded":"2024-09-16T01:40:23.709215+0000","last_fullsized":"2024-09-16T01:40:23.709215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:46.559357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682055+0000","last_change":"2024-09-16T01:38:01.527513+0000","last_active":"2024-09-16T01:40:23.682055+0000","last_peered":"2024-09-16T01:40:23.682055+0000","last_clean":"2024-09-16T01:40:23.682055+0000","last_became_active":"2024-09-16T01:38:01.527184+0000","last_became_peered":"2024-09-16T01:38:01.527184+0000","last_unstale":"2024-09-16T01:40:23.682055+0000","last_undegraded":"2024-09-16T01:40:23.682055+0000","last_fullsized":"2024-09-16T01:40:23.682055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:41.508318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706944+0000","last_change":"2024-09-16T01:38:01.524239+0000","last_active":"2024-09-16T01:40:23.706944+0000","last_peered":"2024-09-16T01:40:23.706944+0000","last_clean":"2024-09-16T01:40:23.706944+0000","last_became_active":"2024-09-16T01:38:01.523906+0000","last_became_peered":"2024-09-16T01:38:01.523906+0000","last_unstale":"2024-09-16T01:40:23.706944+0000","last_undegraded":"2024-09-16T01:40:23.706944+0000","last_fullsized":"2024-09-16T01:40:23.706944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:11.718796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918437+0000","last_change":"2024-09-16T01:38:01.531564+0000","last_active":"2024-09-16T01:40:23.918437+0000","last_peered":"2024-09-16T01:40:23.918437+0000","last_clean":"2024-09-16T01:40:23.918437+0000","last_became_active":"2024-09-16T01:38:01.531241+0000","last_became_peered":"2024-09-16T01:38:01.531241+0000","last_unstale":"2024-09-16T01:40:23.918437+0000","last_undegraded":"2024-09-16T01:40:23.918437+0000","last_fullsized":"2024-09-16T01:40:23.918437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:16.244641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680580+0000","last_change":"2024-09-16T01:38:01.519913+0000","last_active":"2024-09-16T01:40:23.680580+0000","last_peered":"2024-09-16T01:40:23.680580+0000","last_clean":"2024-09-16T01:40:23.680580+0000","last_became_active":"2024-09-16T01:38:01.518631+0000","last_became_peered":"2024-09-16T01:38:01.518631+0000","last_unstale":"2024-09-16T01:40:23.680580+0000","last_undegraded":"2024-09-16T01:40:23.680580+0000","last_fullsized":"2024-09-16T01:40:23.680580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:48.747188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681475+0000","last_change":"2024-09-16T01:38:01.530805+0000","last_active":"2024-09-16T01:40:23.681475+0000","last_peered":"2024-09-16T01:40:23.681475+0000","last_clean":"2024-09-16T01:40:23.681475+0000","last_became_active":"2024-09-16T01:38:01.530466+0000","last_became_peered":"2024-09-16T01:38:01.530466+0000","last_unstale":"2024-09-16T01:40:23.681475+0000","last_undegraded":"2024-09-16T01:40:23.681475+0000","last_fullsized":"2024-09-16T01:40:23.681475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:41:30.587398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706566+0000","last_change":"2024-09-16T01:38:01.519227+0000","last_active":"2024-09-16T01:40:23.706566+0000","last_peered":"2024-09-16T01:40:23.706566+0000","last_clean":"2024-09-16T01:40:23.706566+0000","last_became_active":"2024-09-16T01:38:01.518875+0000","last_became_peered":"2024-09-16T01:38:01.518875+0000","last_unstale":"2024-09-16T01:40:23.706566+0000","last_undegraded":"2024-09-16T01:40:23.706566+0000","last_fullsized":"2024-09-16T01:40:23.706566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:42.654812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716329+0000","last_change":"2024-09-16T01:38:01.513327+0000","last_active":"2024-09-16T01:40:23.716329+0000","last_peered":"2024-09-16T01:40:23.716329+0000","last_clean":"2024-09-16T01:40:23.716329+0000","last_became_active":"2024-09-16T01:38:01.513000+0000","last_became_peered":"2024-09-16T01:38:01.513000+0000","last_unstale":"2024-09-16T01:40:23.716329+0000","last_undegraded":"2024-09-16T01:40:23.716329+0000","last_fullsized":"2024-09-16T01:40:23.716329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:43.036805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706554+0000","last_change":"2024-09-16T01:38:08.560560+0000","last_active":"2024-09-16T01:40:23.706554+0000","last_peered":"2024-09-16T01:40:23.706554+0000","last_clean":"2024-09-16T01:40:23.706554+0000","last_became_active":"2024-09-16T01:38:08.560229+0000","last_became_peered":"2024-09-16T01:38:08.560229+0000","last_unstale":"2024-09-16T01:40:23.706554+0000","last_undegraded":"2024-09-16T01:40:23.706554+0000","last_fullsized":"2024-09-16T01:40:23.706554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:10:09.462043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.917771+0000","last_change":"2024-09-16T01:38:01.529548+0000","last_active":"2024-09-16T01:40:23.917771+0000","last_peered":"2024-09-16T01:40:23.917771+0000","last_clean":"2024-09-16T01:40:23.917771+0000","last_became_active":"2024-09-16T01:38:01.529011+0000","last_became_peered":"2024-09-16T01:38:01.529011+0000","last_unstale":"2024-09-16T01:40:23.917771+0000","last_undegraded":"2024-09-16T01:40:23.917771+0000","last_fullsized":"2024-09-16T01:40:23.917771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:21.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918332+0000","last_change":"2024-09-16T01:38:01.519488+0000","last_active":"2024-09-16T01:40:23.918332+0000","last_peered":"2024-09-16T01:40:23.918332+0000","last_clean":"2024-09-16T01:40:23.918332+0000","last_became_active":"2024-09-16T01:38:01.519156+0000","last_became_peered":"2024-09-16T01:38:01.519156+0000","last_unstale":"2024-09-16T01:40:23.918332+0000","last_undegraded":"2024-09-16T01:40:23.918332+0000","last_fullsized":"2024-09-16T01:40:23.918332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:18.149874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682782+0000","last_change":"2024-09-16T01:38:01.527519+0000","last_active":"2024-09-16T01:40:23.682782+0000","last_peered":"2024-09-16T01:40:23.682782+0000","last_clean":"2024-09-16T01:40:23.682782+0000","last_became_active":"2024-09-16T01:38:01.527175+0000","last_became_peered":"2024-09-16T01:38:01.527175+0000","last_unstale":"2024-09-16T01:40:23.682782+0000","last_undegraded":"2024-09-16T01:40:23.682782+0000","last_fullsized":"2024-09-16T01:40:23.682782+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:04.958842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601743,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1175,"num_read_kb":698,"num_write":379,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1805229,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107382,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678058,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":498675,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906991,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1138688,"data_stored":1289726,"data_compressed":68589,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070517,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":891175,"data_compressed":45785,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":890769,"data_compressed":45785,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528382,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1032192,"data_stored":1095229,"data_compressed":62433,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659204,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":958464,"data_stored":493856,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888137,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1221625,"data_compressed":68589,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051662,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":497930,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280595,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":1017987,"data_compressed":51941,"data_compressed_allocated":69632,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476824,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1052672,"data_stored":564432,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607645,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1040384,"data_stored":564350,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":983040,"data_stored":495846,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134420,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202521,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T01:40:29.307 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T01:40:29.307 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T01:40:29.561 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:29.837 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:29 smithi139 ceph-mon[31175]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:40:29.838 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:29 smithi139 ceph-mon[31175]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-16T01:40:29.838 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:29 smithi139 ceph-mon[31175]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:40:29.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:29 smithi176 ceph-mon[35324]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:40:29.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:29 smithi176 ceph-mon[35324]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-16T01:40:29.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:29 smithi176 ceph-mon[35324]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:40:30.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:29 smithi164 ceph-mon[35121]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:40:30.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:29 smithi164 ceph-mon[35121]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-16T01:40:30.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:29 smithi164 ceph-mon[35121]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:40:30.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.068+0000 7fbe58333640 1 -- 172.21.15.139:0/1425789950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe50106f10 msgr2=0x7fbe5010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:30.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.068+0000 7fbe58333640 1 --2- 172.21.15.139:0/1425789950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe50106f10 0x7fbe5010ea90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009900 tx=0x7fbe4802a410 comp rx=0 tx=0).stop 2024-09-16T01:40:30.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.068+0000 7fbe58333640 1 -- 172.21.15.139:0/1425789950 shutdown_connections 2024-09-16T01:40:30.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.068+0000 7fbe58333640 1 --2- 172.21.15.139:0/1425789950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe50106f10 0x7fbe5010ea90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:30.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.068+0000 7fbe58333640 1 --2- 172.21.15.139:0/1425789950 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe50102790 0x7fbe501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:30.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.068+0000 7fbe58333640 1 --2- 172.21.15.139:0/1425789950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe50101de0 0x7fbe501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:30.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.068+0000 7fbe58333640 1 -- 172.21.15.139:0/1425789950 >> 172.21.15.139:0/1425789950 conn(0x7fbe500fd8c0 msgr2=0x7fbe500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:30.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.068+0000 7fbe58333640 1 -- 172.21.15.139:0/1425789950 shutdown_connections 2024-09-16T01:40:30.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.068+0000 7fbe58333640 1 -- 172.21.15.139:0/1425789950 wait complete. 2024-09-16T01:40:30.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.069+0000 7fbe58333640 1 Processor -- start 2024-09-16T01:40:30.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.069+0000 7fbe58333640 1 -- start start 2024-09-16T01:40:30.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.069+0000 7fbe58333640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe50101de0 0x7fbe50198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:30.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.069+0000 7fbe58333640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe50102790 0x7fbe50198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:30.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.069+0000 7fbe58333640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe50106f10 0x7fbe5019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:30.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.069+0000 7fbe58333640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbe50111f70 con 0x7fbe50106f10 2024-09-16T01:40:30.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.069+0000 7fbe58333640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbe50111df0 con 0x7fbe50102790 2024-09-16T01:40:30.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.069+0000 7fbe58333640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501120f0 con 0x7fbe50101de0 2024-09-16T01:40:30.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.069+0000 7fbe568d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe50106f10 0x7fbe5019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:30.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.069+0000 7fbe568d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe50106f10 0x7fbe5019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50656/0 (socket says 172.21.15.139:50656) 2024-09-16T01:40:30.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.069+0000 7fbe568d0640 1 -- 172.21.15.139:0/3425089239 learned_addr learned my addr 172.21.15.139:0/3425089239 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.069+0000 7fbe558ce640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe50102790 0x7fbe50198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.069+0000 7fbe560cf640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe50101de0 0x7fbe50198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.070+0000 7fbe568d0640 1 -- 172.21.15.139:0/3425089239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe50101de0 msgr2=0x7fbe50198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.070+0000 7fbe568d0640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe50101de0 0x7fbe50198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.070+0000 7fbe568d0640 1 -- 172.21.15.139:0/3425089239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe50102790 msgr2=0x7fbe50198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.070+0000 7fbe568d0640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe50102790 0x7fbe50198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.070+0000 7fbe568d0640 1 -- 172.21.15.139:0/3425089239 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5019d3c0 con 0x7fbe50106f10 2024-09-16T01:40:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.070+0000 7fbe560cf640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe50101de0 0x7fbe50198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.070+0000 7fbe568d0640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe50106f10 0x7fbe5019cce0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbe48005ab0 tx=0x7fbe48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.070+0000 7fbe4f7fe640 1 -- 172.21.15.139:0/3425089239 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4803f030 con 0x7fbe50106f10 2024-09-16T01:40:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.070+0000 7fbe58333640 1 -- 172.21.15.139:0/3425089239 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5019d650 con 0x7fbe50106f10 2024-09-16T01:40:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.070+0000 7fbe4f7fe640 1 -- 172.21.15.139:0/3425089239 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe48003ad0 con 0x7fbe50106f10 2024-09-16T01:40:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.070+0000 7fbe4f7fe640 1 -- 172.21.15.139:0/3425089239 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe48035070 con 0x7fbe50106f10 2024-09-16T01:40:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.070+0000 7fbe58333640 1 -- 172.21.15.139:0/3425089239 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe501a4f90 con 0x7fbe50106f10 2024-09-16T01:40:30.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.071+0000 7fbe58333640 1 -- 172.21.15.139:0/3425089239 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe501021c0 con 0x7fbe50106f10 2024-09-16T01:40:30.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.072+0000 7fbe4f7fe640 1 -- 172.21.15.139:0/3425089239 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 23) ==== 138532+0+0 (secure 0 0 0) 0x7fbe48042030 con 0x7fbe50106f10 2024-09-16T01:40:30.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.073+0000 7fbe4f7fe640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbe380a3440 0x7fbe380a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:30.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.073+0000 7fbe560cf640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbe380a3440 0x7fbe380a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:30.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.073+0000 7fbe4f7fe640 1 -- 172.21.15.139:0/3425089239 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fbe480f0c70 con 0x7fbe50106f10 2024-09-16T01:40:30.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.073+0000 7fbe560cf640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbe380a3440 0x7fbe380a5900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbe40005fd0 tx=0x7fbe40009500 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:30.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.077+0000 7fbe4f7fe640 1 -- 172.21.15.139:0/3425089239 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe480bd130 con 0x7fbe50106f10 2024-09-16T01:40:30.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.241+0000 7fbe58333640 1 -- 172.21.15.139:0/3425089239 --> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe5010a1c0 con 0x7fbe380a3440 2024-09-16T01:40:30.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.249+0000 7fbe4f7fe640 1 -- 172.21.15.139:0/3425089239 <== mgr.15324 v2:172.21.15.139:6800/4031776938 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339720 (secure 0 0 0) 0x7fbe40011440 con 0x7fbe380a3440 2024-09-16T01:40:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:40:30.256 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T01:40:30.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.253+0000 7fbe58333640 1 -- 172.21.15.139:0/3425089239 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbe380a3440 msgr2=0x7fbe380a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:30.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.253+0000 7fbe58333640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbe380a3440 0x7fbe380a5900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbe40005fd0 tx=0x7fbe40009500 comp rx=0 tx=0).stop 2024-09-16T01:40:30.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.253+0000 7fbe58333640 1 -- 172.21.15.139:0/3425089239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe50106f10 msgr2=0x7fbe5019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:30.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.253+0000 7fbe58333640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe50106f10 0x7fbe5019cce0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbe48005ab0 tx=0x7fbe48004270 comp rx=0 tx=0).stop 2024-09-16T01:40:30.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.253+0000 7fbe58333640 1 -- 172.21.15.139:0/3425089239 shutdown_connections 2024-09-16T01:40:30.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.253+0000 7fbe58333640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbe380a3440 0x7fbe380a5900 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:30.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.253+0000 7fbe58333640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe50106f10 0x7fbe5019cce0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:30.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.253+0000 7fbe58333640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe50102790 0x7fbe50198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:30.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.253+0000 7fbe58333640 1 --2- 172.21.15.139:0/3425089239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe50101de0 0x7fbe50198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:30.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.253+0000 7fbe58333640 1 -- 172.21.15.139:0/3425089239 >> 172.21.15.139:0/3425089239 conn(0x7fbe500fd8c0 msgr2=0x7fbe500feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:30.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.253+0000 7fbe58333640 1 -- 172.21.15.139:0/3425089239 shutdown_connections 2024-09-16T01:40:30.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:30.253+0000 7fbe58333640 1 -- 172.21.15.139:0/3425089239 wait complete. 2024-09-16T01:40:30.959 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-09-16T01:40:29.694406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192345,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1311,"num_read_kb":817,"num_write":562,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":943,"ondisk_log_size":943,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373720,"kb_used_data":11948,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568104,"statfs":{"total":1151940427776,"available":1151557738496,"internally_reserved":0,"allocated":12234752,"data_stored":9718232,"data_compressed":512145,"data_compressed_allocated":688128,"data_compressed_original":4386816,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.990421"},"pg_stats":[{"pgid":"2.1f","version":"86'14","reported_seq":167,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707440+0000","last_change":"2024-09-16T01:38:08.567909+0000","last_active":"2024-09-16T01:40:23.707440+0000","last_peered":"2024-09-16T01:40:23.707440+0000","last_clean":"2024-09-16T01:40:23.707440+0000","last_became_active":"2024-09-16T01:38:08.560733+0000","last_became_peered":"2024-09-16T01:38:08.560733+0000","last_unstale":"2024-09-16T01:40:23.707440+0000","last_undegraded":"2024-09-16T01:40:23.707440+0000","last_fullsized":"2024-09-16T01:40:23.707440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:00.678040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681164+0000","last_change":"2024-09-16T01:38:01.523064+0000","last_active":"2024-09-16T01:40:23.681164+0000","last_peered":"2024-09-16T01:40:23.681164+0000","last_clean":"2024-09-16T01:40:23.681164+0000","last_became_active":"2024-09-16T01:38:01.521695+0000","last_became_peered":"2024-09-16T01:38:01.521695+0000","last_unstale":"2024-09-16T01:40:23.681164+0000","last_undegraded":"2024-09-16T01:40:23.681164+0000","last_fullsized":"2024-09-16T01:40:23.681164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:28.467115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716031+0000","last_change":"2024-09-16T01:38:08.563557+0000","last_active":"2024-09-16T01:40:23.716031+0000","last_peered":"2024-09-16T01:40:23.716031+0000","last_clean":"2024-09-16T01:40:23.716031+0000","last_became_active":"2024-09-16T01:38:08.563065+0000","last_became_peered":"2024-09-16T01:38:08.563065+0000","last_unstale":"2024-09-16T01:40:23.716031+0000","last_undegraded":"2024-09-16T01:40:23.716031+0000","last_fullsized":"2024-09-16T01:40:23.716031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:56.992915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681383+0000","last_change":"2024-09-16T01:37:59.798884+0000","last_active":"2024-09-16T01:40:23.681383+0000","last_peered":"2024-09-16T01:40:23.681383+0000","last_clean":"2024-09-16T01:40:23.681383+0000","last_became_active":"2024-09-16T01:37:59.797989+0000","last_became_peered":"2024-09-16T01:37:59.797989+0000","last_unstale":"2024-09-16T01:40:23.681383+0000","last_undegraded":"2024-09-16T01:40:23.681383+0000","last_fullsized":"2024-09-16T01:40:23.681383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:13.633078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918394+0000","last_change":"2024-09-16T01:38:01.528391+0000","last_active":"2024-09-16T01:40:23.918394+0000","last_peered":"2024-09-16T01:40:23.918394+0000","last_clean":"2024-09-16T01:40:23.918394+0000","last_became_active":"2024-09-16T01:38:01.528076+0000","last_became_peered":"2024-09-16T01:38:01.528076+0000","last_unstale":"2024-09-16T01:40:23.918394+0000","last_undegraded":"2024-09-16T01:40:23.918394+0000","last_fullsized":"2024-09-16T01:40:23.918394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:44.105830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708846+0000","last_change":"2024-09-16T01:38:08.566198+0000","last_active":"2024-09-16T01:40:23.708846+0000","last_peered":"2024-09-16T01:40:23.708846+0000","last_clean":"2024-09-16T01:40:23.708846+0000","last_became_active":"2024-09-16T01:38:08.565739+0000","last_became_peered":"2024-09-16T01:38:08.565739+0000","last_unstale":"2024-09-16T01:40:23.708846+0000","last_undegraded":"2024-09-16T01:40:23.708846+0000","last_fullsized":"2024-09-16T01:40:23.708846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:05.172269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716538+0000","last_change":"2024-09-16T01:37:59.507625+0000","last_active":"2024-09-16T01:40:23.716538+0000","last_peered":"2024-09-16T01:40:23.716538+0000","last_clean":"2024-09-16T01:40:23.716538+0000","last_became_active":"2024-09-16T01:37:59.507280+0000","last_became_peered":"2024-09-16T01:37:59.507280+0000","last_unstale":"2024-09-16T01:40:23.716538+0000","last_undegraded":"2024-09-16T01:40:23.716538+0000","last_fullsized":"2024-09-16T01:40:23.716538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:29:37.761203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918422+0000","last_change":"2024-09-16T01:38:01.528406+0000","last_active":"2024-09-16T01:40:23.918422+0000","last_peered":"2024-09-16T01:40:23.918422+0000","last_clean":"2024-09-16T01:40:23.918422+0000","last_became_active":"2024-09-16T01:38:01.528084+0000","last_became_peered":"2024-09-16T01:38:01.528084+0000","last_unstale":"2024-09-16T01:40:23.918422+0000","last_undegraded":"2024-09-16T01:40:23.918422+0000","last_fullsized":"2024-09-16T01:40:23.918422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:40.766305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709276+0000","last_change":"2024-09-16T01:38:08.563645+0000","last_active":"2024-09-16T01:40:23.709276+0000","last_peered":"2024-09-16T01:40:23.709276+0000","last_clean":"2024-09-16T01:40:23.709276+0000","last_became_active":"2024-09-16T01:38:08.563149+0000","last_became_peered":"2024-09-16T01:38:08.563149+0000","last_unstale":"2024-09-16T01:40:23.709276+0000","last_undegraded":"2024-09-16T01:40:23.709276+0000","last_fullsized":"2024-09-16T01:40:23.709276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:09.010905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716807+0000","last_change":"2024-09-16T01:37:59.509649+0000","last_active":"2024-09-16T01:40:23.716807+0000","last_peered":"2024-09-16T01:40:23.716807+0000","last_clean":"2024-09-16T01:40:23.716807+0000","last_became_active":"2024-09-16T01:37:59.509236+0000","last_became_peered":"2024-09-16T01:37:59.509236+0000","last_unstale":"2024-09-16T01:40:23.716807+0000","last_undegraded":"2024-09-16T01:40:23.716807+0000","last_fullsized":"2024-09-16T01:40:23.716807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.614205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709245+0000","last_change":"2024-09-16T01:38:01.528074+0000","last_active":"2024-09-16T01:40:23.709245+0000","last_peered":"2024-09-16T01:40:23.709245+0000","last_clean":"2024-09-16T01:40:23.709245+0000","last_became_active":"2024-09-16T01:38:01.527737+0000","last_became_peered":"2024-09-16T01:38:01.527737+0000","last_unstale":"2024-09-16T01:40:23.709245+0000","last_undegraded":"2024-09-16T01:40:23.709245+0000","last_fullsized":"2024-09-16T01:40:23.709245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:37.592557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708957+0000","last_change":"2024-09-16T01:38:08.566340+0000","last_active":"2024-09-16T01:40:23.708957+0000","last_peered":"2024-09-16T01:40:23.708957+0000","last_clean":"2024-09-16T01:40:23.708957+0000","last_became_active":"2024-09-16T01:38:08.565791+0000","last_became_peered":"2024-09-16T01:38:08.565791+0000","last_unstale":"2024-09-16T01:40:23.708957+0000","last_undegraded":"2024-09-16T01:40:23.708957+0000","last_fullsized":"2024-09-16T01:40:23.708957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:35:31.845151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681718+0000","last_change":"2024-09-16T01:38:01.527747+0000","last_active":"2024-09-16T01:40:23.681718+0000","last_peered":"2024-09-16T01:40:23.681718+0000","last_clean":"2024-09-16T01:40:23.681718+0000","last_became_active":"2024-09-16T01:38:01.527325+0000","last_became_peered":"2024-09-16T01:38:01.527325+0000","last_unstale":"2024-09-16T01:40:23.681718+0000","last_undegraded":"2024-09-16T01:40:23.681718+0000","last_fullsized":"2024-09-16T01:40:23.681718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:27.203719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716467+0000","last_change":"2024-09-16T01:37:59.510298+0000","last_active":"2024-09-16T01:40:23.716467+0000","last_peered":"2024-09-16T01:40:23.716467+0000","last_clean":"2024-09-16T01:40:23.716467+0000","last_became_active":"2024-09-16T01:37:59.509952+0000","last_became_peered":"2024-09-16T01:37:59.509952+0000","last_unstale":"2024-09-16T01:40:23.716467+0000","last_undegraded":"2024-09-16T01:40:23.716467+0000","last_fullsized":"2024-09-16T01:40:23.716467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:11.616846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709120+0000","last_change":"2024-09-16T01:38:08.571622+0000","last_active":"2024-09-16T01:40:23.709120+0000","last_peered":"2024-09-16T01:40:23.709120+0000","last_clean":"2024-09-16T01:40:23.709120+0000","last_became_active":"2024-09-16T01:38:08.571125+0000","last_became_peered":"2024-09-16T01:38:08.571125+0000","last_unstale":"2024-09-16T01:40:23.709120+0000","last_undegraded":"2024-09-16T01:40:23.709120+0000","last_fullsized":"2024-09-16T01:40:23.709120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:00.154633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680985+0000","last_change":"2024-09-16T01:38:01.519681+0000","last_active":"2024-09-16T01:40:23.680985+0000","last_peered":"2024-09-16T01:40:23.680985+0000","last_clean":"2024-09-16T01:40:23.680985+0000","last_became_active":"2024-09-16T01:38:01.518310+0000","last_became_peered":"2024-09-16T01:38:01.518310+0000","last_unstale":"2024-09-16T01:40:23.680985+0000","last_undegraded":"2024-09-16T01:40:23.680985+0000","last_fullsized":"2024-09-16T01:40:23.680985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:23.308008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:24.556034+0000","last_change":"2024-09-16T01:38:08.559469+0000","last_active":"2024-09-16T01:40:24.556034+0000","last_peered":"2024-09-16T01:40:24.556034+0000","last_clean":"2024-09-16T01:40:24.556034+0000","last_became_active":"2024-09-16T01:38:08.559111+0000","last_became_peered":"2024-09-16T01:38:08.559111+0000","last_unstale":"2024-09-16T01:40:24.556034+0000","last_undegraded":"2024-09-16T01:40:24.556034+0000","last_fullsized":"2024-09-16T01:40:24.556034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:58.577749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709153+0000","last_change":"2024-09-16T01:38:08.555029+0000","last_active":"2024-09-16T01:40:23.709153+0000","last_peered":"2024-09-16T01:40:23.709153+0000","last_clean":"2024-09-16T01:40:23.709153+0000","last_became_active":"2024-09-16T01:38:08.554151+0000","last_became_peered":"2024-09-16T01:38:08.554151+0000","last_unstale":"2024-09-16T01:40:23.709153+0000","last_undegraded":"2024-09-16T01:40:23.709153+0000","last_fullsized":"2024-09-16T01:40:23.709153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:05.005904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682035+0000","last_change":"2024-09-16T01:38:01.528114+0000","last_active":"2024-09-16T01:40:23.682035+0000","last_peered":"2024-09-16T01:40:23.682035+0000","last_clean":"2024-09-16T01:40:23.682035+0000","last_became_active":"2024-09-16T01:38:01.527805+0000","last_became_peered":"2024-09-16T01:38:01.527805+0000","last_unstale":"2024-09-16T01:40:23.682035+0000","last_undegraded":"2024-09-16T01:40:23.682035+0000","last_fullsized":"2024-09-16T01:40:23.682035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:15.830936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680769+0000","last_change":"2024-09-16T01:37:59.507776+0000","last_active":"2024-09-16T01:40:23.680769+0000","last_peered":"2024-09-16T01:40:23.680769+0000","last_clean":"2024-09-16T01:40:23.680769+0000","last_became_active":"2024-09-16T01:37:59.507429+0000","last_became_peered":"2024-09-16T01:37:59.507429+0000","last_unstale":"2024-09-16T01:40:23.680769+0000","last_undegraded":"2024-09-16T01:40:23.680769+0000","last_fullsized":"2024-09-16T01:40:23.680769+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:44:41.274986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707084+0000","last_change":"2024-09-16T01:38:08.562191+0000","last_active":"2024-09-16T01:40:23.707084+0000","last_peered":"2024-09-16T01:40:23.707084+0000","last_clean":"2024-09-16T01:40:23.707084+0000","last_became_active":"2024-09-16T01:38:08.561839+0000","last_became_peered":"2024-09-16T01:38:08.561839+0000","last_unstale":"2024-09-16T01:40:23.707084+0000","last_undegraded":"2024-09-16T01:40:23.707084+0000","last_fullsized":"2024-09-16T01:40:23.707084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:40.618612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716123+0000","last_change":"2024-09-16T01:38:01.518855+0000","last_active":"2024-09-16T01:40:23.716123+0000","last_peered":"2024-09-16T01:40:23.716123+0000","last_clean":"2024-09-16T01:40:23.716123+0000","last_became_active":"2024-09-16T01:38:01.518427+0000","last_became_peered":"2024-09-16T01:38:01.518427+0000","last_unstale":"2024-09-16T01:40:23.716123+0000","last_undegraded":"2024-09-16T01:40:23.716123+0000","last_fullsized":"2024-09-16T01:40:23.716123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:23.054856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918053+0000","last_change":"2024-09-16T01:37:59.799341+0000","last_active":"2024-09-16T01:40:23.918053+0000","last_peered":"2024-09-16T01:40:23.918053+0000","last_clean":"2024-09-16T01:40:23.918053+0000","last_became_active":"2024-09-16T01:37:59.798992+0000","last_became_peered":"2024-09-16T01:37:59.798992+0000","last_unstale":"2024-09-16T01:40:23.918053+0000","last_undegraded":"2024-09-16T01:40:23.918053+0000","last_fullsized":"2024-09-16T01:40:23.918053+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:01.708565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709518+0000","last_change":"2024-09-16T01:39:09.524319+0000","last_active":"2024-09-16T01:40:23.709518+0000","last_peered":"2024-09-16T01:40:23.709518+0000","last_clean":"2024-09-16T01:40:23.709518+0000","last_became_active":"2024-09-16T01:39:09.523717+0000","last_became_peered":"2024-09-16T01:39:09.523717+0000","last_unstale":"2024-09-16T01:40:23.709518+0000","last_undegraded":"2024-09-16T01:40:23.709518+0000","last_fullsized":"2024-09-16T01:40:23.709518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:35.488097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918444+0000","last_change":"2024-09-16T01:38:01.524545+0000","last_active":"2024-09-16T01:40:23.918444+0000","last_peered":"2024-09-16T01:40:23.918444+0000","last_clean":"2024-09-16T01:40:23.918444+0000","last_became_active":"2024-09-16T01:38:01.524253+0000","last_became_peered":"2024-09-16T01:38:01.524253+0000","last_unstale":"2024-09-16T01:40:23.918444+0000","last_undegraded":"2024-09-16T01:40:23.918444+0000","last_fullsized":"2024-09-16T01:40:23.918444+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:08.697500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709429+0000","last_change":"2024-09-16T01:37:59.506779+0000","last_active":"2024-09-16T01:40:23.709429+0000","last_peered":"2024-09-16T01:40:23.709429+0000","last_clean":"2024-09-16T01:40:23.709429+0000","last_became_active":"2024-09-16T01:37:59.505459+0000","last_became_peered":"2024-09-16T01:37:59.505459+0000","last_unstale":"2024-09-16T01:40:23.709429+0000","last_undegraded":"2024-09-16T01:40:23.709429+0000","last_fullsized":"2024-09-16T01:40:23.709429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:12.973330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716178+0000","last_change":"2024-09-16T01:38:08.557250+0000","last_active":"2024-09-16T01:40:23.716178+0000","last_peered":"2024-09-16T01:40:23.716178+0000","last_clean":"2024-09-16T01:40:23.716178+0000","last_became_active":"2024-09-16T01:38:08.555577+0000","last_became_peered":"2024-09-16T01:38:08.555577+0000","last_unstale":"2024-09-16T01:40:23.716178+0000","last_undegraded":"2024-09-16T01:40:23.716178+0000","last_fullsized":"2024-09-16T01:40:23.716178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:54.965341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681174+0000","last_change":"2024-09-16T01:38:01.522910+0000","last_active":"2024-09-16T01:40:23.681174+0000","last_peered":"2024-09-16T01:40:23.681174+0000","last_clean":"2024-09-16T01:40:23.681174+0000","last_became_active":"2024-09-16T01:38:01.521206+0000","last_became_peered":"2024-09-16T01:38:01.521206+0000","last_unstale":"2024-09-16T01:40:23.681174+0000","last_undegraded":"2024-09-16T01:40:23.681174+0000","last_fullsized":"2024-09-16T01:40:23.681174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:45.670464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'52","reported_seq":111,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.879588+0000","last_change":"2024-09-16T01:37:59.505962+0000","last_active":"2024-09-16T01:40:23.879588+0000","last_peered":"2024-09-16T01:40:23.879588+0000","last_clean":"2024-09-16T01:40:23.879588+0000","last_became_active":"2024-09-16T01:37:59.505636+0000","last_became_peered":"2024-09-16T01:37:59.505636+0000","last_unstale":"2024-09-16T01:40:23.879588+0000","last_undegraded":"2024-09-16T01:40:23.879588+0000","last_fullsized":"2024-09-16T01:40:23.879588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:57.548513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680470+0000","last_change":"2024-09-16T01:38:08.571249+0000","last_active":"2024-09-16T01:40:23.680470+0000","last_peered":"2024-09-16T01:40:23.680470+0000","last_clean":"2024-09-16T01:40:23.680470+0000","last_became_active":"2024-09-16T01:38:08.570388+0000","last_became_peered":"2024-09-16T01:38:08.570388+0000","last_unstale":"2024-09-16T01:40:23.680470+0000","last_undegraded":"2024-09-16T01:40:23.680470+0000","last_fullsized":"2024-09-16T01:40:23.680470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:59.904704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706816+0000","last_change":"2024-09-16T01:38:01.522040+0000","last_active":"2024-09-16T01:40:23.706816+0000","last_peered":"2024-09-16T01:40:23.706816+0000","last_clean":"2024-09-16T01:40:23.706816+0000","last_became_active":"2024-09-16T01:38:01.521720+0000","last_became_peered":"2024-09-16T01:38:01.521720+0000","last_unstale":"2024-09-16T01:40:23.706816+0000","last_undegraded":"2024-09-16T01:40:23.706816+0000","last_fullsized":"2024-09-16T01:40:23.706816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:44:48.635603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682302+0000","last_change":"2024-09-16T01:38:08.561870+0000","last_active":"2024-09-16T01:40:23.682302+0000","last_peered":"2024-09-16T01:40:23.682302+0000","last_clean":"2024-09-16T01:40:23.682302+0000","last_became_active":"2024-09-16T01:38:08.561440+0000","last_became_peered":"2024-09-16T01:38:08.561440+0000","last_unstale":"2024-09-16T01:40:23.682302+0000","last_undegraded":"2024-09-16T01:40:23.682302+0000","last_fullsized":"2024-09-16T01:40:23.682302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:43:37.594734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682173+0000","last_change":"2024-09-16T01:38:08.555436+0000","last_active":"2024-09-16T01:40:23.682173+0000","last_peered":"2024-09-16T01:40:23.682173+0000","last_clean":"2024-09-16T01:40:23.682173+0000","last_became_active":"2024-09-16T01:38:08.555026+0000","last_became_peered":"2024-09-16T01:38:08.555026+0000","last_unstale":"2024-09-16T01:40:23.682173+0000","last_undegraded":"2024-09-16T01:40:23.682173+0000","last_fullsized":"2024-09-16T01:40:23.682173+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:28.870214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"85'4","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706753+0000","last_change":"2024-09-16T01:38:01.524482+0000","last_active":"2024-09-16T01:40:23.706753+0000","last_peered":"2024-09-16T01:40:23.706753+0000","last_clean":"2024-09-16T01:40:23.706753+0000","last_became_active":"2024-09-16T01:38:01.524160+0000","last_became_peered":"2024-09-16T01:38:01.524160+0000","last_unstale":"2024-09-16T01:40:23.706753+0000","last_undegraded":"2024-09-16T01:40:23.706753+0000","last_fullsized":"2024-09-16T01:40:23.706753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:39:27.621780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706766+0000","last_change":"2024-09-16T01:37:59.800051+0000","last_active":"2024-09-16T01:40:23.706766+0000","last_peered":"2024-09-16T01:40:23.706766+0000","last_clean":"2024-09-16T01:40:23.706766+0000","last_became_active":"2024-09-16T01:37:59.799606+0000","last_became_peered":"2024-09-16T01:37:59.799606+0000","last_unstale":"2024-09-16T01:40:23.706766+0000","last_undegraded":"2024-09-16T01:40:23.706766+0000","last_fullsized":"2024-09-16T01:40:23.706766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:02.694933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716183+0000","last_change":"2024-09-16T01:38:08.570764+0000","last_active":"2024-09-16T01:40:23.716183+0000","last_peered":"2024-09-16T01:40:23.716183+0000","last_clean":"2024-09-16T01:40:23.716183+0000","last_became_active":"2024-09-16T01:38:08.570385+0000","last_became_peered":"2024-09-16T01:38:08.570385+0000","last_unstale":"2024-09-16T01:40:23.716183+0000","last_undegraded":"2024-09-16T01:40:23.716183+0000","last_fullsized":"2024-09-16T01:40:23.716183+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:25.349671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706565+0000","last_change":"2024-09-16T01:38:01.519590+0000","last_active":"2024-09-16T01:40:23.706565+0000","last_peered":"2024-09-16T01:40:23.706565+0000","last_clean":"2024-09-16T01:40:23.706565+0000","last_became_active":"2024-09-16T01:38:01.519210+0000","last_became_peered":"2024-09-16T01:38:01.519210+0000","last_unstale":"2024-09-16T01:40:23.706565+0000","last_undegraded":"2024-09-16T01:40:23.706565+0000","last_fullsized":"2024-09-16T01:40:23.706565+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:04.228131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707867+0000","last_change":"2024-09-16T01:37:59.505473+0000","last_active":"2024-09-16T01:40:23.707867+0000","last_peered":"2024-09-16T01:40:23.707867+0000","last_clean":"2024-09-16T01:40:23.707867+0000","last_became_active":"2024-09-16T01:37:59.505123+0000","last_became_peered":"2024-09-16T01:37:59.505123+0000","last_unstale":"2024-09-16T01:40:23.707867+0000","last_undegraded":"2024-09-16T01:40:23.707867+0000","last_fullsized":"2024-09-16T01:40:23.707867+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:17.739625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706690+0000","last_change":"2024-09-16T01:38:08.558534+0000","last_active":"2024-09-16T01:40:23.706690+0000","last_peered":"2024-09-16T01:40:23.706690+0000","last_clean":"2024-09-16T01:40:23.706690+0000","last_became_active":"2024-09-16T01:38:08.558185+0000","last_became_peered":"2024-09-16T01:38:08.558185+0000","last_unstale":"2024-09-16T01:40:23.706690+0000","last_undegraded":"2024-09-16T01:40:23.706690+0000","last_fullsized":"2024-09-16T01:40:23.706690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:09.359426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708932+0000","last_change":"2024-09-16T01:38:01.521445+0000","last_active":"2024-09-16T01:40:23.708932+0000","last_peered":"2024-09-16T01:40:23.708932+0000","last_clean":"2024-09-16T01:40:23.708932+0000","last_became_active":"2024-09-16T01:38:01.521140+0000","last_became_peered":"2024-09-16T01:38:01.521140+0000","last_unstale":"2024-09-16T01:40:23.708932+0000","last_undegraded":"2024-09-16T01:40:23.708932+0000","last_fullsized":"2024-09-16T01:40:23.708932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:05.247515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":147,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707420+0000","last_change":"2024-09-16T01:37:59.798365+0000","last_active":"2024-09-16T01:40:23.707420+0000","last_peered":"2024-09-16T01:40:23.707420+0000","last_clean":"2024-09-16T01:40:23.707420+0000","last_became_active":"2024-09-16T01:37:59.798023+0000","last_became_peered":"2024-09-16T01:37:59.798023+0000","last_unstale":"2024-09-16T01:40:23.707420+0000","last_undegraded":"2024-09-16T01:40:23.707420+0000","last_fullsized":"2024-09-16T01:40:23.707420+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:30.867080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707046+0000","last_change":"2024-09-16T01:38:08.571704+0000","last_active":"2024-09-16T01:40:23.707046+0000","last_peered":"2024-09-16T01:40:23.707046+0000","last_clean":"2024-09-16T01:40:23.707046+0000","last_became_active":"2024-09-16T01:38:08.571350+0000","last_became_peered":"2024-09-16T01:38:08.571350+0000","last_unstale":"2024-09-16T01:40:23.707046+0000","last_undegraded":"2024-09-16T01:40:23.707046+0000","last_fullsized":"2024-09-16T01:40:23.707046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:01.698770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706388+0000","last_change":"2024-09-16T01:38:08.559433+0000","last_active":"2024-09-16T01:40:23.706388+0000","last_peered":"2024-09-16T01:40:23.706388+0000","last_clean":"2024-09-16T01:40:23.706388+0000","last_became_active":"2024-09-16T01:38:08.559110+0000","last_became_peered":"2024-09-16T01:38:08.559110+0000","last_unstale":"2024-09-16T01:40:23.706388+0000","last_undegraded":"2024-09-16T01:40:23.706388+0000","last_fullsized":"2024-09-16T01:40:23.706388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:39.001916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681418+0000","last_change":"2024-09-16T01:37:59.508236+0000","last_active":"2024-09-16T01:40:23.681418+0000","last_peered":"2024-09-16T01:40:23.681418+0000","last_clean":"2024-09-16T01:40:23.681418+0000","last_became_active":"2024-09-16T01:37:59.507883+0000","last_became_peered":"2024-09-16T01:37:59.507883+0000","last_unstale":"2024-09-16T01:40:23.681418+0000","last_undegraded":"2024-09-16T01:40:23.681418+0000","last_fullsized":"2024-09-16T01:40:23.681418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:23.948530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709040+0000","last_change":"2024-09-16T01:38:08.572108+0000","last_active":"2024-09-16T01:40:23.709040+0000","last_peered":"2024-09-16T01:40:23.709040+0000","last_clean":"2024-09-16T01:40:23.709040+0000","last_became_active":"2024-09-16T01:38:08.571643+0000","last_became_peered":"2024-09-16T01:38:08.571643+0000","last_unstale":"2024-09-16T01:40:23.709040+0000","last_undegraded":"2024-09-16T01:40:23.709040+0000","last_fullsized":"2024-09-16T01:40:23.709040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:46.636219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680849+0000","last_change":"2024-09-16T01:38:01.517894+0000","last_active":"2024-09-16T01:40:23.680849+0000","last_peered":"2024-09-16T01:40:23.680849+0000","last_clean":"2024-09-16T01:40:23.680849+0000","last_became_active":"2024-09-16T01:38:01.517602+0000","last_became_peered":"2024-09-16T01:38:01.517602+0000","last_unstale":"2024-09-16T01:40:23.680849+0000","last_undegraded":"2024-09-16T01:40:23.680849+0000","last_fullsized":"2024-09-16T01:40:23.680849+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:33.483120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709294+0000","last_change":"2024-09-16T01:37:59.507568+0000","last_active":"2024-09-16T01:40:23.709294+0000","last_peered":"2024-09-16T01:40:23.709294+0000","last_clean":"2024-09-16T01:40:23.709294+0000","last_became_active":"2024-09-16T01:37:59.505798+0000","last_became_peered":"2024-09-16T01:37:59.505798+0000","last_unstale":"2024-09-16T01:40:23.709294+0000","last_undegraded":"2024-09-16T01:40:23.709294+0000","last_fullsized":"2024-09-16T01:40:23.709294+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:19:02.711054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":360,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681514+0000","last_change":"2024-09-16T01:36:46.744332+0000","last_active":"2024-09-16T01:40:23.681514+0000","last_peered":"2024-09-16T01:40:23.681514+0000","last_clean":"2024-09-16T01:40:23.681514+0000","last_became_active":"2024-09-16T01:36:46.743902+0000","last_became_peered":"2024-09-16T01:36:46.743902+0000","last_unstale":"2024-09-16T01:40:23.681514+0000","last_undegraded":"2024-09-16T01:40:23.681514+0000","last_fullsized":"2024-09-16T01:40:23.681514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:33:19.891854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:33:19.891854+0000","last_clean_scrub_stamp":"2024-09-16T01:33:19.891854+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:46.173685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681871+0000","last_change":"2024-09-16T01:39:09.528216+0000","last_active":"2024-09-16T01:40:23.681871+0000","last_peered":"2024-09-16T01:40:23.681871+0000","last_clean":"2024-09-16T01:40:23.681871+0000","last_became_active":"2024-09-16T01:39:09.523634+0000","last_became_peered":"2024-09-16T01:39:09.523634+0000","last_unstale":"2024-09-16T01:40:23.681871+0000","last_undegraded":"2024-09-16T01:40:23.681871+0000","last_fullsized":"2024-09-16T01:40:23.681871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:27.989901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682064+0000","last_change":"2024-09-16T01:38:01.527690+0000","last_active":"2024-09-16T01:40:23.682064+0000","last_peered":"2024-09-16T01:40:23.682064+0000","last_clean":"2024-09-16T01:40:23.682064+0000","last_became_active":"2024-09-16T01:38:01.527339+0000","last_became_peered":"2024-09-16T01:38:01.527339+0000","last_unstale":"2024-09-16T01:40:23.682064+0000","last_undegraded":"2024-09-16T01:40:23.682064+0000","last_fullsized":"2024-09-16T01:40:23.682064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:58.526270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'8","reported_seq":153,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.717334+0000","last_change":"2024-09-16T01:37:59.798874+0000","last_active":"2024-09-16T01:40:23.717334+0000","last_peered":"2024-09-16T01:40:23.717334+0000","last_clean":"2024-09-16T01:40:23.717334+0000","last_became_active":"2024-09-16T01:37:59.798424+0000","last_became_peered":"2024-09-16T01:37:59.798424+0000","last_unstale":"2024-09-16T01:40:23.717334+0000","last_undegraded":"2024-09-16T01:40:23.717334+0000","last_fullsized":"2024-09-16T01:40:23.717334+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:36.648963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918289+0000","last_change":"2024-09-16T01:38:08.555524+0000","last_active":"2024-09-16T01:40:23.918289+0000","last_peered":"2024-09-16T01:40:23.918289+0000","last_clean":"2024-09-16T01:40:23.918289+0000","last_became_active":"2024-09-16T01:38:08.555075+0000","last_became_peered":"2024-09-16T01:38:08.555075+0000","last_unstale":"2024-09-16T01:40:23.918289+0000","last_undegraded":"2024-09-16T01:40:23.918289+0000","last_fullsized":"2024-09-16T01:40:23.918289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:28:44.847916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681953+0000","last_change":"2024-09-16T01:38:01.528869+0000","last_active":"2024-09-16T01:40:23.681953+0000","last_peered":"2024-09-16T01:40:23.681953+0000","last_clean":"2024-09-16T01:40:23.681953+0000","last_became_active":"2024-09-16T01:38:01.528559+0000","last_became_peered":"2024-09-16T01:38:01.528559+0000","last_unstale":"2024-09-16T01:40:23.681953+0000","last_undegraded":"2024-09-16T01:40:23.681953+0000","last_fullsized":"2024-09-16T01:40:23.681953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:21.547808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'41","reported_seq":86,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.876871+0000","last_change":"2024-09-16T01:37:59.510871+0000","last_active":"2024-09-16T01:40:23.876871+0000","last_peered":"2024-09-16T01:40:23.876871+0000","last_clean":"2024-09-16T01:40:23.876871+0000","last_became_active":"2024-09-16T01:37:59.509996+0000","last_became_peered":"2024-09-16T01:37:59.509996+0000","last_unstale":"2024-09-16T01:40:23.876871+0000","last_undegraded":"2024-09-16T01:40:23.876871+0000","last_fullsized":"2024-09-16T01:40:23.876871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:23.390864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918027+0000","last_change":"2024-09-16T01:38:08.555181+0000","last_active":"2024-09-16T01:40:23.918027+0000","last_peered":"2024-09-16T01:40:23.918027+0000","last_clean":"2024-09-16T01:40:23.918027+0000","last_became_active":"2024-09-16T01:38:08.554778+0000","last_became_peered":"2024-09-16T01:38:08.554778+0000","last_unstale":"2024-09-16T01:40:23.918027+0000","last_undegraded":"2024-09-16T01:40:23.918027+0000","last_fullsized":"2024-09-16T01:40:23.918027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:16:42.369236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682039+0000","last_change":"2024-09-16T01:38:01.526260+0000","last_active":"2024-09-16T01:40:23.682039+0000","last_peered":"2024-09-16T01:40:23.682039+0000","last_clean":"2024-09-16T01:40:23.682039+0000","last_became_active":"2024-09-16T01:38:01.525652+0000","last_became_peered":"2024-09-16T01:38:01.525652+0000","last_unstale":"2024-09-16T01:40:23.682039+0000","last_undegraded":"2024-09-16T01:40:23.682039+0000","last_fullsized":"2024-09-16T01:40:23.682039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:09.180507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707015+0000","last_change":"2024-09-16T01:38:08.570408+0000","last_active":"2024-09-16T01:40:23.707015+0000","last_peered":"2024-09-16T01:40:23.707015+0000","last_clean":"2024-09-16T01:40:23.707015+0000","last_became_active":"2024-09-16T01:38:08.566122+0000","last_became_peered":"2024-09-16T01:38:08.566122+0000","last_unstale":"2024-09-16T01:40:23.707015+0000","last_undegraded":"2024-09-16T01:40:23.707015+0000","last_fullsized":"2024-09-16T01:40:23.707015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:55.832320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708804+0000","last_change":"2024-09-16T01:38:08.555063+0000","last_active":"2024-09-16T01:40:23.708804+0000","last_peered":"2024-09-16T01:40:23.708804+0000","last_clean":"2024-09-16T01:40:23.708804+0000","last_became_active":"2024-09-16T01:38:08.554665+0000","last_became_peered":"2024-09-16T01:38:08.554665+0000","last_unstale":"2024-09-16T01:40:23.708804+0000","last_undegraded":"2024-09-16T01:40:23.708804+0000","last_fullsized":"2024-09-16T01:40:23.708804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:50.091017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682633+0000","last_change":"2024-09-16T01:38:01.524247+0000","last_active":"2024-09-16T01:40:23.682633+0000","last_peered":"2024-09-16T01:40:23.682633+0000","last_clean":"2024-09-16T01:40:23.682633+0000","last_became_active":"2024-09-16T01:38:01.523911+0000","last_became_peered":"2024-09-16T01:38:01.523911+0000","last_unstale":"2024-09-16T01:40:23.682633+0000","last_undegraded":"2024-09-16T01:40:23.682633+0000","last_fullsized":"2024-09-16T01:40:23.682633+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:32.745882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716800+0000","last_change":"2024-09-16T01:37:59.511272+0000","last_active":"2024-09-16T01:40:23.716800+0000","last_peered":"2024-09-16T01:40:23.716800+0000","last_clean":"2024-09-16T01:40:23.716800+0000","last_became_active":"2024-09-16T01:37:59.510920+0000","last_became_peered":"2024-09-16T01:37:59.510920+0000","last_unstale":"2024-09-16T01:40:23.716800+0000","last_undegraded":"2024-09-16T01:40:23.716800+0000","last_fullsized":"2024-09-16T01:40:23.716800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:59.406175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708902+0000","last_change":"2024-09-16T01:38:08.567173+0000","last_active":"2024-09-16T01:40:23.708902+0000","last_peered":"2024-09-16T01:40:23.708902+0000","last_clean":"2024-09-16T01:40:23.708902+0000","last_became_active":"2024-09-16T01:38:08.566788+0000","last_became_peered":"2024-09-16T01:38:08.566788+0000","last_unstale":"2024-09-16T01:40:23.708902+0000","last_undegraded":"2024-09-16T01:40:23.708902+0000","last_fullsized":"2024-09-16T01:40:23.708902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:35.743356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709105+0000","last_change":"2024-09-16T01:38:01.529529+0000","last_active":"2024-09-16T01:40:23.709105+0000","last_peered":"2024-09-16T01:40:23.709105+0000","last_clean":"2024-09-16T01:40:23.709105+0000","last_became_active":"2024-09-16T01:38:01.529048+0000","last_became_peered":"2024-09-16T01:38:01.529048+0000","last_unstale":"2024-09-16T01:40:23.709105+0000","last_undegraded":"2024-09-16T01:40:23.709105+0000","last_fullsized":"2024-09-16T01:40:23.709105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:35.041662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716379+0000","last_change":"2024-09-16T01:37:59.799536+0000","last_active":"2024-09-16T01:40:23.716379+0000","last_peered":"2024-09-16T01:40:23.716379+0000","last_clean":"2024-09-16T01:40:23.716379+0000","last_became_active":"2024-09-16T01:37:59.799109+0000","last_became_peered":"2024-09-16T01:37:59.799109+0000","last_unstale":"2024-09-16T01:40:23.716379+0000","last_undegraded":"2024-09-16T01:40:23.716379+0000","last_fullsized":"2024-09-16T01:40:23.716379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:07:25.007434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707289+0000","last_change":"2024-09-16T01:38:08.567019+0000","last_active":"2024-09-16T01:40:23.707289+0000","last_peered":"2024-09-16T01:40:23.707289+0000","last_clean":"2024-09-16T01:40:23.707289+0000","last_became_active":"2024-09-16T01:38:08.566667+0000","last_became_peered":"2024-09-16T01:38:08.566667+0000","last_unstale":"2024-09-16T01:40:23.707289+0000","last_undegraded":"2024-09-16T01:40:23.707289+0000","last_fullsized":"2024-09-16T01:40:23.707289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:11.375095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681386+0000","last_change":"2024-09-16T01:38:01.527965+0000","last_active":"2024-09-16T01:40:23.681386+0000","last_peered":"2024-09-16T01:40:23.681386+0000","last_clean":"2024-09-16T01:40:23.681386+0000","last_became_active":"2024-09-16T01:38:01.526688+0000","last_became_peered":"2024-09-16T01:38:01.526688+0000","last_unstale":"2024-09-16T01:40:23.681386+0000","last_undegraded":"2024-09-16T01:40:23.681386+0000","last_fullsized":"2024-09-16T01:40:23.681386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:06.618756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":149,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.923723+0000","last_change":"2024-09-16T01:37:59.510331+0000","last_active":"2024-09-16T01:40:23.923723+0000","last_peered":"2024-09-16T01:40:23.923723+0000","last_clean":"2024-09-16T01:40:23.923723+0000","last_became_active":"2024-09-16T01:37:59.509769+0000","last_became_peered":"2024-09-16T01:37:59.509769+0000","last_unstale":"2024-09-16T01:40:23.923723+0000","last_undegraded":"2024-09-16T01:40:23.923723+0000","last_fullsized":"2024-09-16T01:40:23.923723+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:18.843091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716548+0000","last_change":"2024-09-16T01:39:10.049460+0000","last_active":"2024-09-16T01:40:23.716548+0000","last_peered":"2024-09-16T01:40:23.716548+0000","last_clean":"2024-09-16T01:40:23.716548+0000","last_became_active":"2024-09-16T01:39:10.048862+0000","last_became_peered":"2024-09-16T01:39:10.048862+0000","last_unstale":"2024-09-16T01:40:23.716548+0000","last_undegraded":"2024-09-16T01:40:23.716548+0000","last_fullsized":"2024-09-16T01:40:23.716548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:37.329806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681820+0000","last_change":"2024-09-16T01:38:01.521254+0000","last_active":"2024-09-16T01:40:23.681820+0000","last_peered":"2024-09-16T01:40:23.681820+0000","last_clean":"2024-09-16T01:40:23.681820+0000","last_became_active":"2024-09-16T01:38:01.520901+0000","last_became_peered":"2024-09-16T01:38:01.520901+0000","last_unstale":"2024-09-16T01:40:23.681820+0000","last_undegraded":"2024-09-16T01:40:23.681820+0000","last_fullsized":"2024-09-16T01:40:23.681820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:27.806954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709051+0000","last_change":"2024-09-16T01:37:59.505990+0000","last_active":"2024-09-16T01:40:23.709051+0000","last_peered":"2024-09-16T01:40:23.709051+0000","last_clean":"2024-09-16T01:40:23.709051+0000","last_became_active":"2024-09-16T01:37:59.505564+0000","last_became_peered":"2024-09-16T01:37:59.505564+0000","last_unstale":"2024-09-16T01:40:23.709051+0000","last_undegraded":"2024-09-16T01:40:23.709051+0000","last_fullsized":"2024-09-16T01:40:23.709051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:01.496609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682127+0000","last_change":"2024-09-16T01:38:08.562838+0000","last_active":"2024-09-16T01:40:23.682127+0000","last_peered":"2024-09-16T01:40:23.682127+0000","last_clean":"2024-09-16T01:40:23.682127+0000","last_became_active":"2024-09-16T01:38:08.562426+0000","last_became_peered":"2024-09-16T01:38:08.562426+0000","last_unstale":"2024-09-16T01:40:23.682127+0000","last_undegraded":"2024-09-16T01:40:23.682127+0000","last_fullsized":"2024-09-16T01:40:23.682127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:20.685626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707454+0000","last_change":"2024-09-16T01:38:01.514983+0000","last_active":"2024-09-16T01:40:23.707454+0000","last_peered":"2024-09-16T01:40:23.707454+0000","last_clean":"2024-09-16T01:40:23.707454+0000","last_became_active":"2024-09-16T01:38:01.514648+0000","last_became_peered":"2024-09-16T01:38:01.514648+0000","last_unstale":"2024-09-16T01:40:23.707454+0000","last_undegraded":"2024-09-16T01:40:23.707454+0000","last_fullsized":"2024-09-16T01:40:23.707454+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:57.052564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.717151+0000","last_change":"2024-09-16T01:37:59.505141+0000","last_active":"2024-09-16T01:40:23.717151+0000","last_peered":"2024-09-16T01:40:23.717151+0000","last_clean":"2024-09-16T01:40:23.717151+0000","last_became_active":"2024-09-16T01:37:59.504730+0000","last_became_peered":"2024-09-16T01:37:59.504730+0000","last_unstale":"2024-09-16T01:40:23.717151+0000","last_undegraded":"2024-09-16T01:40:23.717151+0000","last_fullsized":"2024-09-16T01:40:23.717151+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:54.504661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709184+0000","last_change":"2024-09-16T01:38:08.563970+0000","last_active":"2024-09-16T01:40:23.709184+0000","last_peered":"2024-09-16T01:40:23.709184+0000","last_clean":"2024-09-16T01:40:23.709184+0000","last_became_active":"2024-09-16T01:38:08.563555+0000","last_became_peered":"2024-09-16T01:38:08.563555+0000","last_unstale":"2024-09-16T01:40:23.709184+0000","last_undegraded":"2024-09-16T01:40:23.709184+0000","last_fullsized":"2024-09-16T01:40:23.709184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:27:12.086043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716549+0000","last_change":"2024-09-16T01:38:01.521016+0000","last_active":"2024-09-16T01:40:23.716549+0000","last_peered":"2024-09-16T01:40:23.716549+0000","last_clean":"2024-09-16T01:40:23.716549+0000","last_became_active":"2024-09-16T01:38:01.520656+0000","last_became_peered":"2024-09-16T01:38:01.520656+0000","last_unstale":"2024-09-16T01:40:23.716549+0000","last_undegraded":"2024-09-16T01:40:23.716549+0000","last_fullsized":"2024-09-16T01:40:23.716549+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:39.001239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'41","reported_seq":297,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.923336+0000","last_change":"2024-09-16T01:37:59.507235+0000","last_active":"2024-09-16T01:40:23.923336+0000","last_peered":"2024-09-16T01:40:23.923336+0000","last_clean":"2024-09-16T01:40:23.923336+0000","last_became_active":"2024-09-16T01:37:59.506255+0000","last_became_peered":"2024-09-16T01:37:59.506255+0000","last_unstale":"2024-09-16T01:40:23.923336+0000","last_undegraded":"2024-09-16T01:40:23.923336+0000","last_fullsized":"2024-09-16T01:40:23.923336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:44.515663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68792,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":219,"num_read_kb":171,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706438+0000","last_change":"2024-09-16T01:39:09.522688+0000","last_active":"2024-09-16T01:40:23.706438+0000","last_peered":"2024-09-16T01:40:23.706438+0000","last_clean":"2024-09-16T01:40:23.706438+0000","last_became_active":"2024-09-16T01:39:09.522137+0000","last_became_peered":"2024-09-16T01:39:09.522137+0000","last_unstale":"2024-09-16T01:40:23.706438+0000","last_undegraded":"2024-09-16T01:40:23.706438+0000","last_fullsized":"2024-09-16T01:40:23.706438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:43.004315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918498+0000","last_change":"2024-09-16T01:38:01.526641+0000","last_active":"2024-09-16T01:40:23.918498+0000","last_peered":"2024-09-16T01:40:23.918498+0000","last_clean":"2024-09-16T01:40:23.918498+0000","last_became_active":"2024-09-16T01:38:01.526278+0000","last_became_peered":"2024-09-16T01:38:01.526278+0000","last_unstale":"2024-09-16T01:40:23.918498+0000","last_undegraded":"2024-09-16T01:40:23.918498+0000","last_fullsized":"2024-09-16T01:40:23.918498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:56.156593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709392+0000","last_change":"2024-09-16T01:37:59.800065+0000","last_active":"2024-09-16T01:40:23.709392+0000","last_peered":"2024-09-16T01:40:23.709392+0000","last_clean":"2024-09-16T01:40:23.709392+0000","last_became_active":"2024-09-16T01:37:59.799705+0000","last_became_peered":"2024-09-16T01:37:59.799705+0000","last_unstale":"2024-09-16T01:40:23.709392+0000","last_undegraded":"2024-09-16T01:40:23.709392+0000","last_fullsized":"2024-09-16T01:40:23.709392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:13.791875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709196+0000","last_change":"2024-09-16T01:38:08.556360+0000","last_active":"2024-09-16T01:40:23.709196+0000","last_peered":"2024-09-16T01:40:23.709196+0000","last_clean":"2024-09-16T01:40:23.709196+0000","last_became_active":"2024-09-16T01:38:08.555219+0000","last_became_peered":"2024-09-16T01:38:08.555219+0000","last_unstale":"2024-09-16T01:40:23.709196+0000","last_undegraded":"2024-09-16T01:40:23.709196+0000","last_fullsized":"2024-09-16T01:40:23.709196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:36.464760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918560+0000","last_change":"2024-09-16T01:38:01.526758+0000","last_active":"2024-09-16T01:40:23.918560+0000","last_peered":"2024-09-16T01:40:23.918560+0000","last_clean":"2024-09-16T01:40:23.918560+0000","last_became_active":"2024-09-16T01:38:01.526545+0000","last_became_peered":"2024-09-16T01:38:01.526545+0000","last_unstale":"2024-09-16T01:40:23.918560+0000","last_undegraded":"2024-09-16T01:40:23.918560+0000","last_fullsized":"2024-09-16T01:40:23.918560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:47.185489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682092+0000","last_change":"2024-09-16T01:38:08.555125+0000","last_active":"2024-09-16T01:40:23.682092+0000","last_peered":"2024-09-16T01:40:23.682092+0000","last_clean":"2024-09-16T01:40:23.682092+0000","last_became_active":"2024-09-16T01:38:08.554756+0000","last_became_peered":"2024-09-16T01:38:08.554756+0000","last_unstale":"2024-09-16T01:40:23.682092+0000","last_undegraded":"2024-09-16T01:40:23.682092+0000","last_fullsized":"2024-09-16T01:40:23.682092+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:08.888006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.705898+0000","last_change":"2024-09-16T01:38:08.557906+0000","last_active":"2024-09-16T01:40:23.705898+0000","last_peered":"2024-09-16T01:40:23.705898+0000","last_clean":"2024-09-16T01:40:23.705898+0000","last_became_active":"2024-09-16T01:38:08.557546+0000","last_became_peered":"2024-09-16T01:38:08.557546+0000","last_unstale":"2024-09-16T01:40:23.705898+0000","last_undegraded":"2024-09-16T01:40:23.705898+0000","last_fullsized":"2024-09-16T01:40:23.705898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:27:04.988778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918558+0000","last_change":"2024-09-16T01:38:01.528430+0000","last_active":"2024-09-16T01:40:23.918558+0000","last_peered":"2024-09-16T01:40:23.918558+0000","last_clean":"2024-09-16T01:40:23.918558+0000","last_became_active":"2024-09-16T01:38:01.528069+0000","last_became_peered":"2024-09-16T01:38:01.528069+0000","last_unstale":"2024-09-16T01:40:23.918558+0000","last_undegraded":"2024-09-16T01:40:23.918558+0000","last_fullsized":"2024-09-16T01:40:23.918558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:41.317215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'6","reported_seq":251,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.916402+0000","last_change":"2024-09-16T01:37:59.509017+0000","last_active":"2024-09-16T01:40:23.916402+0000","last_peered":"2024-09-16T01:40:23.916402+0000","last_clean":"2024-09-16T01:40:23.916402+0000","last_became_active":"2024-09-16T01:37:59.508252+0000","last_became_peered":"2024-09-16T01:37:59.508252+0000","last_unstale":"2024-09-16T01:40:23.916402+0000","last_undegraded":"2024-09-16T01:40:23.916402+0000","last_fullsized":"2024-09-16T01:40:23.916402+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:26.215407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":208,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680647+0000","last_change":"2024-09-16T01:38:08.565197+0000","last_active":"2024-09-16T01:40:23.680647+0000","last_peered":"2024-09-16T01:40:23.680647+0000","last_clean":"2024-09-16T01:40:23.680647+0000","last_became_active":"2024-09-16T01:38:08.562773+0000","last_became_peered":"2024-09-16T01:38:08.562773+0000","last_unstale":"2024-09-16T01:40:23.680647+0000","last_undegraded":"2024-09-16T01:40:23.680647+0000","last_fullsized":"2024-09-16T01:40:23.680647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:13.316950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918230+0000","last_change":"2024-09-16T01:38:08.554052+0000","last_active":"2024-09-16T01:40:23.918230+0000","last_peered":"2024-09-16T01:40:23.918230+0000","last_clean":"2024-09-16T01:40:23.918230+0000","last_became_active":"2024-09-16T01:38:08.553660+0000","last_became_peered":"2024-09-16T01:38:08.553660+0000","last_unstale":"2024-09-16T01:40:23.918230+0000","last_undegraded":"2024-09-16T01:40:23.918230+0000","last_fullsized":"2024-09-16T01:40:23.918230+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:53:52.270695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706069+0000","last_change":"2024-09-16T01:38:01.522688+0000","last_active":"2024-09-16T01:40:23.706069+0000","last_peered":"2024-09-16T01:40:23.706069+0000","last_clean":"2024-09-16T01:40:23.706069+0000","last_became_active":"2024-09-16T01:38:01.519843+0000","last_became_peered":"2024-09-16T01:38:01.519843+0000","last_unstale":"2024-09-16T01:40:23.706069+0000","last_undegraded":"2024-09-16T01:40:23.706069+0000","last_fullsized":"2024-09-16T01:40:23.706069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:45:50.971760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709088+0000","last_change":"2024-09-16T01:37:59.509044+0000","last_active":"2024-09-16T01:40:23.709088+0000","last_peered":"2024-09-16T01:40:23.709088+0000","last_clean":"2024-09-16T01:40:23.709088+0000","last_became_active":"2024-09-16T01:37:59.507085+0000","last_became_peered":"2024-09-16T01:37:59.507085+0000","last_unstale":"2024-09-16T01:40:23.709088+0000","last_undegraded":"2024-09-16T01:40:23.709088+0000","last_fullsized":"2024-09-16T01:40:23.709088+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:25:09.930945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682550+0000","last_change":"2024-09-16T01:38:08.561250+0000","last_active":"2024-09-16T01:40:23.682550+0000","last_peered":"2024-09-16T01:40:23.682550+0000","last_clean":"2024-09-16T01:40:23.682550+0000","last_became_active":"2024-09-16T01:38:08.560866+0000","last_became_peered":"2024-09-16T01:38:08.560866+0000","last_unstale":"2024-09-16T01:40:23.682550+0000","last_undegraded":"2024-09-16T01:40:23.682550+0000","last_fullsized":"2024-09-16T01:40:23.682550+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:31.253817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680655+0000","last_change":"2024-09-16T01:38:01.517400+0000","last_active":"2024-09-16T01:40:23.680655+0000","last_peered":"2024-09-16T01:40:23.680655+0000","last_clean":"2024-09-16T01:40:23.680655+0000","last_became_active":"2024-09-16T01:38:01.517103+0000","last_became_peered":"2024-09-16T01:38:01.517103+0000","last_unstale":"2024-09-16T01:40:23.680655+0000","last_undegraded":"2024-09-16T01:40:23.680655+0000","last_fullsized":"2024-09-16T01:40:23.680655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:37.886179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706415+0000","last_change":"2024-09-16T01:37:59.503370+0000","last_active":"2024-09-16T01:40:23.706415+0000","last_peered":"2024-09-16T01:40:23.706415+0000","last_clean":"2024-09-16T01:40:23.706415+0000","last_became_active":"2024-09-16T01:37:59.503071+0000","last_became_peered":"2024-09-16T01:37:59.503071+0000","last_unstale":"2024-09-16T01:40:23.706415+0000","last_undegraded":"2024-09-16T01:40:23.706415+0000","last_fullsized":"2024-09-16T01:40:23.706415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:51.207702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706631+0000","last_change":"2024-09-16T01:37:59.506110+0000","last_active":"2024-09-16T01:40:23.706631+0000","last_peered":"2024-09-16T01:40:23.706631+0000","last_clean":"2024-09-16T01:40:23.706631+0000","last_became_active":"2024-09-16T01:37:59.505264+0000","last_became_peered":"2024-09-16T01:37:59.505264+0000","last_unstale":"2024-09-16T01:40:23.706631+0000","last_undegraded":"2024-09-16T01:40:23.706631+0000","last_fullsized":"2024-09-16T01:40:23.706631+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:35.986433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709024+0000","last_change":"2024-09-16T01:38:08.563900+0000","last_active":"2024-09-16T01:40:23.709024+0000","last_peered":"2024-09-16T01:40:23.709024+0000","last_clean":"2024-09-16T01:40:23.709024+0000","last_became_active":"2024-09-16T01:38:08.563551+0000","last_became_peered":"2024-09-16T01:38:08.563551+0000","last_unstale":"2024-09-16T01:40:23.709024+0000","last_undegraded":"2024-09-16T01:40:23.709024+0000","last_fullsized":"2024-09-16T01:40:23.709024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:31:03.979580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716744+0000","last_change":"2024-09-16T01:38:08.556011+0000","last_active":"2024-09-16T01:40:23.716744+0000","last_peered":"2024-09-16T01:40:23.716744+0000","last_clean":"2024-09-16T01:40:23.716744+0000","last_became_active":"2024-09-16T01:38:08.555615+0000","last_became_peered":"2024-09-16T01:38:08.555615+0000","last_unstale":"2024-09-16T01:40:23.716744+0000","last_undegraded":"2024-09-16T01:40:23.716744+0000","last_fullsized":"2024-09-16T01:40:23.716744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:34.434974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":147,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709207+0000","last_change":"2024-09-16T01:37:59.508411+0000","last_active":"2024-09-16T01:40:23.709207+0000","last_peered":"2024-09-16T01:40:23.709207+0000","last_clean":"2024-09-16T01:40:23.709207+0000","last_became_active":"2024-09-16T01:37:59.508058+0000","last_became_peered":"2024-09-16T01:37:59.508058+0000","last_unstale":"2024-09-16T01:40:23.709207+0000","last_undegraded":"2024-09-16T01:40:23.709207+0000","last_fullsized":"2024-09-16T01:40:23.709207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:37:58.475730+0000","last_clean_scrub_stamp":"2024-09-16T01:37:58.475730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:47.441673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716319+0000","last_change":"2024-09-16T01:38:08.560930+0000","last_active":"2024-09-16T01:40:23.716319+0000","last_peered":"2024-09-16T01:40:23.716319+0000","last_clean":"2024-09-16T01:40:23.716319+0000","last_became_active":"2024-09-16T01:38:08.560490+0000","last_became_peered":"2024-09-16T01:38:08.560490+0000","last_unstale":"2024-09-16T01:40:23.716319+0000","last_undegraded":"2024-09-16T01:40:23.716319+0000","last_fullsized":"2024-09-16T01:40:23.716319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:11.556113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716818+0000","last_change":"2024-09-16T01:38:08.557564+0000","last_active":"2024-09-16T01:40:23.716818+0000","last_peered":"2024-09-16T01:40:23.716818+0000","last_clean":"2024-09-16T01:40:23.716818+0000","last_became_active":"2024-09-16T01:38:08.557143+0000","last_became_peered":"2024-09-16T01:38:08.557143+0000","last_unstale":"2024-09-16T01:40:23.716818+0000","last_undegraded":"2024-09-16T01:40:23.716818+0000","last_fullsized":"2024-09-16T01:40:23.716818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:07.538211+0000","last_clean_scrub_stamp":"2024-09-16T01:38:07.538211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:43.148870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681960+0000","last_change":"2024-09-16T01:38:01.527353+0000","last_active":"2024-09-16T01:40:23.681960+0000","last_peered":"2024-09-16T01:40:23.681960+0000","last_clean":"2024-09-16T01:40:23.681960+0000","last_became_active":"2024-09-16T01:38:01.527031+0000","last_became_peered":"2024-09-16T01:38:01.527031+0000","last_unstale":"2024-09-16T01:40:23.681960+0000","last_undegraded":"2024-09-16T01:40:23.681960+0000","last_fullsized":"2024-09-16T01:40:23.681960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:23:27.311638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716301+0000","last_change":"2024-09-16T01:38:01.526409+0000","last_active":"2024-09-16T01:40:23.716301+0000","last_peered":"2024-09-16T01:40:23.716301+0000","last_clean":"2024-09-16T01:40:23.716301+0000","last_became_active":"2024-09-16T01:38:01.526064+0000","last_became_peered":"2024-09-16T01:38:01.526064+0000","last_unstale":"2024-09-16T01:40:23.716301+0000","last_undegraded":"2024-09-16T01:40:23.716301+0000","last_fullsized":"2024-09-16T01:40:23.716301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:51.571808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716432+0000","last_change":"2024-09-16T01:38:01.521856+0000","last_active":"2024-09-16T01:40:23.716432+0000","last_peered":"2024-09-16T01:40:23.716432+0000","last_clean":"2024-09-16T01:40:23.716432+0000","last_became_active":"2024-09-16T01:38:01.520312+0000","last_became_peered":"2024-09-16T01:38:01.520312+0000","last_unstale":"2024-09-16T01:40:23.716432+0000","last_undegraded":"2024-09-16T01:40:23.716432+0000","last_fullsized":"2024-09-16T01:40:23.716432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:53.618990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918080+0000","last_change":"2024-09-16T01:38:01.520958+0000","last_active":"2024-09-16T01:40:23.918080+0000","last_peered":"2024-09-16T01:40:23.918080+0000","last_clean":"2024-09-16T01:40:23.918080+0000","last_became_active":"2024-09-16T01:38:01.520658+0000","last_became_peered":"2024-09-16T01:38:01.520658+0000","last_unstale":"2024-09-16T01:40:23.918080+0000","last_undegraded":"2024-09-16T01:40:23.918080+0000","last_fullsized":"2024-09-16T01:40:23.918080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:23.770694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707380+0000","last_change":"2024-09-16T01:38:01.519140+0000","last_active":"2024-09-16T01:40:23.707380+0000","last_peered":"2024-09-16T01:40:23.707380+0000","last_clean":"2024-09-16T01:40:23.707380+0000","last_became_active":"2024-09-16T01:38:01.518765+0000","last_became_peered":"2024-09-16T01:38:01.518765+0000","last_unstale":"2024-09-16T01:40:23.707380+0000","last_undegraded":"2024-09-16T01:40:23.707380+0000","last_fullsized":"2024-09-16T01:40:23.707380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:17.194886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682443+0000","last_change":"2024-09-16T01:38:01.523762+0000","last_active":"2024-09-16T01:40:23.682443+0000","last_peered":"2024-09-16T01:40:23.682443+0000","last_clean":"2024-09-16T01:40:23.682443+0000","last_became_active":"2024-09-16T01:38:01.522035+0000","last_became_peered":"2024-09-16T01:38:01.522035+0000","last_unstale":"2024-09-16T01:40:23.682443+0000","last_undegraded":"2024-09-16T01:40:23.682443+0000","last_fullsized":"2024-09-16T01:40:23.682443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:40.405350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.717020+0000","last_change":"2024-09-16T01:38:01.527539+0000","last_active":"2024-09-16T01:40:23.717020+0000","last_peered":"2024-09-16T01:40:23.717020+0000","last_clean":"2024-09-16T01:40:23.717020+0000","last_became_active":"2024-09-16T01:38:01.526974+0000","last_became_peered":"2024-09-16T01:38:01.526974+0000","last_unstale":"2024-09-16T01:40:23.717020+0000","last_undegraded":"2024-09-16T01:40:23.717020+0000","last_fullsized":"2024-09-16T01:40:23.717020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:30:03.587141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681144+0000","last_change":"2024-09-16T01:38:01.529131+0000","last_active":"2024-09-16T01:40:23.681144+0000","last_peered":"2024-09-16T01:40:23.681144+0000","last_clean":"2024-09-16T01:40:23.681144+0000","last_became_active":"2024-09-16T01:38:01.528796+0000","last_became_peered":"2024-09-16T01:38:01.528796+0000","last_unstale":"2024-09-16T01:40:23.681144+0000","last_undegraded":"2024-09-16T01:40:23.681144+0000","last_fullsized":"2024-09-16T01:40:23.681144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:38.903638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.708881+0000","last_change":"2024-09-16T01:38:01.514779+0000","last_active":"2024-09-16T01:40:23.708881+0000","last_peered":"2024-09-16T01:40:23.708881+0000","last_clean":"2024-09-16T01:40:23.708881+0000","last_became_active":"2024-09-16T01:38:01.514434+0000","last_became_peered":"2024-09-16T01:38:01.514434+0000","last_unstale":"2024-09-16T01:40:23.708881+0000","last_undegraded":"2024-09-16T01:40:23.708881+0000","last_fullsized":"2024-09-16T01:40:23.708881+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:31.382111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680678+0000","last_change":"2024-09-16T01:38:01.521771+0000","last_active":"2024-09-16T01:40:23.680678+0000","last_peered":"2024-09-16T01:40:23.680678+0000","last_clean":"2024-09-16T01:40:23.680678+0000","last_became_active":"2024-09-16T01:38:01.521288+0000","last_became_peered":"2024-09-16T01:38:01.521288+0000","last_unstale":"2024-09-16T01:40:23.680678+0000","last_undegraded":"2024-09-16T01:40:23.680678+0000","last_fullsized":"2024-09-16T01:40:23.680678+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:11.825164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918399+0000","last_change":"2024-09-16T01:38:01.529247+0000","last_active":"2024-09-16T01:40:23.918399+0000","last_peered":"2024-09-16T01:40:23.918399+0000","last_clean":"2024-09-16T01:40:23.918399+0000","last_became_active":"2024-09-16T01:38:01.528926+0000","last_became_peered":"2024-09-16T01:38:01.528926+0000","last_unstale":"2024-09-16T01:40:23.918399+0000","last_undegraded":"2024-09-16T01:40:23.918399+0000","last_fullsized":"2024-09-16T01:40:23.918399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:20.704114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707157+0000","last_change":"2024-09-16T01:38:01.520830+0000","last_active":"2024-09-16T01:40:23.707157+0000","last_peered":"2024-09-16T01:40:23.707157+0000","last_clean":"2024-09-16T01:40:23.707157+0000","last_became_active":"2024-09-16T01:38:01.520526+0000","last_became_peered":"2024-09-16T01:38:01.520526+0000","last_unstale":"2024-09-16T01:40:23.707157+0000","last_undegraded":"2024-09-16T01:40:23.707157+0000","last_fullsized":"2024-09-16T01:40:23.707157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:28.062812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682262+0000","last_change":"2024-09-16T01:38:01.521983+0000","last_active":"2024-09-16T01:40:23.682262+0000","last_peered":"2024-09-16T01:40:23.682262+0000","last_clean":"2024-09-16T01:40:23.682262+0000","last_became_active":"2024-09-16T01:38:01.521666+0000","last_became_peered":"2024-09-16T01:38:01.521666+0000","last_unstale":"2024-09-16T01:40:23.682262+0000","last_undegraded":"2024-09-16T01:40:23.682262+0000","last_fullsized":"2024-09-16T01:40:23.682262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:50:55.761313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.717006+0000","last_change":"2024-09-16T01:38:08.554304+0000","last_active":"2024-09-16T01:40:23.717006+0000","last_peered":"2024-09-16T01:40:23.717006+0000","last_clean":"2024-09-16T01:40:23.717006+0000","last_became_active":"2024-09-16T01:38:08.553683+0000","last_became_peered":"2024-09-16T01:38:08.553683+0000","last_unstale":"2024-09-16T01:40:23.717006+0000","last_undegraded":"2024-09-16T01:40:23.717006+0000","last_fullsized":"2024-09-16T01:40:23.717006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:56.881140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706919+0000","last_change":"2024-09-16T01:38:01.520425+0000","last_active":"2024-09-16T01:40:23.706919+0000","last_peered":"2024-09-16T01:40:23.706919+0000","last_clean":"2024-09-16T01:40:23.706919+0000","last_became_active":"2024-09-16T01:38:01.519858+0000","last_became_peered":"2024-09-16T01:38:01.519858+0000","last_unstale":"2024-09-16T01:40:23.706919+0000","last_undegraded":"2024-09-16T01:40:23.706919+0000","last_fullsized":"2024-09-16T01:40:23.706919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:48.534298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682855+0000","last_change":"2024-09-16T01:38:01.532762+0000","last_active":"2024-09-16T01:40:23.682855+0000","last_peered":"2024-09-16T01:40:23.682855+0000","last_clean":"2024-09-16T01:40:23.682855+0000","last_became_active":"2024-09-16T01:38:01.532404+0000","last_became_peered":"2024-09-16T01:38:01.532404+0000","last_unstale":"2024-09-16T01:40:23.682855+0000","last_undegraded":"2024-09-16T01:40:23.682855+0000","last_fullsized":"2024-09-16T01:40:23.682855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:03.135141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918405+0000","last_change":"2024-09-16T01:38:01.530205+0000","last_active":"2024-09-16T01:40:23.918405+0000","last_peered":"2024-09-16T01:40:23.918405+0000","last_clean":"2024-09-16T01:40:23.918405+0000","last_became_active":"2024-09-16T01:38:01.529898+0000","last_became_peered":"2024-09-16T01:38:01.529898+0000","last_unstale":"2024-09-16T01:40:23.918405+0000","last_undegraded":"2024-09-16T01:40:23.918405+0000","last_fullsized":"2024-09-16T01:40:23.918405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:48.444441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.917747+0000","last_change":"2024-09-16T01:38:01.529407+0000","last_active":"2024-09-16T01:40:23.917747+0000","last_peered":"2024-09-16T01:40:23.917747+0000","last_clean":"2024-09-16T01:40:23.917747+0000","last_became_active":"2024-09-16T01:38:01.528330+0000","last_became_peered":"2024-09-16T01:38:01.528330+0000","last_unstale":"2024-09-16T01:40:23.917747+0000","last_undegraded":"2024-09-16T01:40:23.917747+0000","last_fullsized":"2024-09-16T01:40:23.917747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:32.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.917730+0000","last_change":"2024-09-16T01:38:01.519579+0000","last_active":"2024-09-16T01:40:23.917730+0000","last_peered":"2024-09-16T01:40:23.917730+0000","last_clean":"2024-09-16T01:40:23.917730+0000","last_became_active":"2024-09-16T01:38:01.519211+0000","last_became_peered":"2024-09-16T01:38:01.519211+0000","last_unstale":"2024-09-16T01:40:23.917730+0000","last_undegraded":"2024-09-16T01:40:23.917730+0000","last_fullsized":"2024-09-16T01:40:23.917730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:18.861169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.707196+0000","last_change":"2024-09-16T01:38:01.521136+0000","last_active":"2024-09-16T01:40:23.707196+0000","last_peered":"2024-09-16T01:40:23.707196+0000","last_clean":"2024-09-16T01:40:23.707196+0000","last_became_active":"2024-09-16T01:38:01.520819+0000","last_became_peered":"2024-09-16T01:38:01.520819+0000","last_unstale":"2024-09-16T01:40:23.707196+0000","last_undegraded":"2024-09-16T01:40:23.707196+0000","last_fullsized":"2024-09-16T01:40:23.707196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:28.150822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.709215+0000","last_change":"2024-09-16T01:38:01.519382+0000","last_active":"2024-09-16T01:40:23.709215+0000","last_peered":"2024-09-16T01:40:23.709215+0000","last_clean":"2024-09-16T01:40:23.709215+0000","last_became_active":"2024-09-16T01:38:01.519039+0000","last_became_peered":"2024-09-16T01:38:01.519039+0000","last_unstale":"2024-09-16T01:40:23.709215+0000","last_undegraded":"2024-09-16T01:40:23.709215+0000","last_fullsized":"2024-09-16T01:40:23.709215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:46.559357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682055+0000","last_change":"2024-09-16T01:38:01.527513+0000","last_active":"2024-09-16T01:40:23.682055+0000","last_peered":"2024-09-16T01:40:23.682055+0000","last_clean":"2024-09-16T01:40:23.682055+0000","last_became_active":"2024-09-16T01:38:01.527184+0000","last_became_peered":"2024-09-16T01:38:01.527184+0000","last_unstale":"2024-09-16T01:40:23.682055+0000","last_undegraded":"2024-09-16T01:40:23.682055+0000","last_fullsized":"2024-09-16T01:40:23.682055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:41.508318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706944+0000","last_change":"2024-09-16T01:38:01.524239+0000","last_active":"2024-09-16T01:40:23.706944+0000","last_peered":"2024-09-16T01:40:23.706944+0000","last_clean":"2024-09-16T01:40:23.706944+0000","last_became_active":"2024-09-16T01:38:01.523906+0000","last_became_peered":"2024-09-16T01:38:01.523906+0000","last_unstale":"2024-09-16T01:40:23.706944+0000","last_undegraded":"2024-09-16T01:40:23.706944+0000","last_fullsized":"2024-09-16T01:40:23.706944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:17:11.718796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918437+0000","last_change":"2024-09-16T01:38:01.531564+0000","last_active":"2024-09-16T01:40:23.918437+0000","last_peered":"2024-09-16T01:40:23.918437+0000","last_clean":"2024-09-16T01:40:23.918437+0000","last_became_active":"2024-09-16T01:38:01.531241+0000","last_became_peered":"2024-09-16T01:38:01.531241+0000","last_unstale":"2024-09-16T01:40:23.918437+0000","last_undegraded":"2024-09-16T01:40:23.918437+0000","last_fullsized":"2024-09-16T01:40:23.918437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:16.244641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.680627+0000","last_change":"2024-09-16T01:38:01.519913+0000","last_active":"2024-09-16T01:40:23.680627+0000","last_peered":"2024-09-16T01:40:23.680627+0000","last_clean":"2024-09-16T01:40:23.680627+0000","last_became_active":"2024-09-16T01:38:01.518631+0000","last_became_peered":"2024-09-16T01:38:01.518631+0000","last_unstale":"2024-09-16T01:40:23.680627+0000","last_undegraded":"2024-09-16T01:40:23.680627+0000","last_fullsized":"2024-09-16T01:40:23.680627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:48.747188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.681475+0000","last_change":"2024-09-16T01:38:01.530805+0000","last_active":"2024-09-16T01:40:23.681475+0000","last_peered":"2024-09-16T01:40:23.681475+0000","last_clean":"2024-09-16T01:40:23.681475+0000","last_became_active":"2024-09-16T01:38:01.530466+0000","last_became_peered":"2024-09-16T01:38:01.530466+0000","last_unstale":"2024-09-16T01:40:23.681475+0000","last_undegraded":"2024-09-16T01:40:23.681475+0000","last_fullsized":"2024-09-16T01:40:23.681475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:41:30.587398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706566+0000","last_change":"2024-09-16T01:38:01.519227+0000","last_active":"2024-09-16T01:40:23.706566+0000","last_peered":"2024-09-16T01:40:23.706566+0000","last_clean":"2024-09-16T01:40:23.706566+0000","last_became_active":"2024-09-16T01:38:01.518875+0000","last_became_peered":"2024-09-16T01:38:01.518875+0000","last_unstale":"2024-09-16T01:40:23.706566+0000","last_undegraded":"2024-09-16T01:40:23.706566+0000","last_fullsized":"2024-09-16T01:40:23.706566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:42.654812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.716329+0000","last_change":"2024-09-16T01:38:01.513327+0000","last_active":"2024-09-16T01:40:23.716329+0000","last_peered":"2024-09-16T01:40:23.716329+0000","last_clean":"2024-09-16T01:40:23.716329+0000","last_became_active":"2024-09-16T01:38:01.513000+0000","last_became_peered":"2024-09-16T01:38:01.513000+0000","last_unstale":"2024-09-16T01:40:23.716329+0000","last_undegraded":"2024-09-16T01:40:23.716329+0000","last_fullsized":"2024-09-16T01:40:23.716329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:43.036805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.706554+0000","last_change":"2024-09-16T01:38:08.560560+0000","last_active":"2024-09-16T01:40:23.706554+0000","last_peered":"2024-09-16T01:40:23.706554+0000","last_clean":"2024-09-16T01:40:23.706554+0000","last_became_active":"2024-09-16T01:38:08.560229+0000","last_became_peered":"2024-09-16T01:38:08.560229+0000","last_unstale":"2024-09-16T01:40:23.706554+0000","last_undegraded":"2024-09-16T01:40:23.706554+0000","last_fullsized":"2024-09-16T01:40:23.706554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:10:09.462043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.917771+0000","last_change":"2024-09-16T01:38:01.529548+0000","last_active":"2024-09-16T01:40:23.917771+0000","last_peered":"2024-09-16T01:40:23.917771+0000","last_clean":"2024-09-16T01:40:23.917771+0000","last_became_active":"2024-09-16T01:38:01.529011+0000","last_became_peered":"2024-09-16T01:38:01.529011+0000","last_unstale":"2024-09-16T01:40:23.917771+0000","last_undegraded":"2024-09-16T01:40:23.917771+0000","last_fullsized":"2024-09-16T01:40:23.917771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:21.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.918332+0000","last_change":"2024-09-16T01:38:01.519488+0000","last_active":"2024-09-16T01:40:23.918332+0000","last_peered":"2024-09-16T01:40:23.918332+0000","last_clean":"2024-09-16T01:40:23.918332+0000","last_became_active":"2024-09-16T01:38:01.519156+0000","last_became_peered":"2024-09-16T01:38:01.519156+0000","last_unstale":"2024-09-16T01:40:23.918332+0000","last_undegraded":"2024-09-16T01:40:23.918332+0000","last_fullsized":"2024-09-16T01:40:23.918332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:18.149874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"85'8","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T01:40:23.682782+0000","last_change":"2024-09-16T01:38:01.527519+0000","last_active":"2024-09-16T01:40:23.682782+0000","last_peered":"2024-09-16T01:40:23.682782+0000","last_clean":"2024-09-16T01:40:23.682782+0000","last_became_active":"2024-09-16T01:38:01.527175+0000","last_became_peered":"2024-09-16T01:38:01.527175+0000","last_unstale":"2024-09-16T01:40:23.682782+0000","last_undegraded":"2024-09-16T01:40:23.682782+0000","last_fullsized":"2024-09-16T01:40:23.682782+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T01:38:00.492529+0000","last_clean_scrub_stamp":"2024-09-16T01:38:00.492529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:04.958842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601743,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1175,"num_read_kb":698,"num_write":379,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":749568,"data_stored":2001861,"data_compressed":85095,"data_compressed_allocated":110592,"data_compressed_original":1732608,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107382,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678058,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":498675,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906991,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1138688,"data_stored":1289726,"data_compressed":68589,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070517,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":891175,"data_compressed":45785,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":890769,"data_compressed":45785,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528382,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1032192,"data_stored":1095229,"data_compressed":62433,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659204,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":958464,"data_stored":493856,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888137,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1221625,"data_compressed":68589,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051662,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":497930,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280596,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":1214619,"data_compressed":61130,"data_compressed_allocated":81920,"data_compressed_original":774144,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476824,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1052672,"data_stored":564432,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607645,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1040384,"data_stored":564350,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":983040,"data_stored":495846,"data_compressed":26639,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35794,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134420,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202521,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T01:40:31.394 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:40:31.648 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:31 smithi139 ceph-mon[31175]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:40:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:31 smithi139 ceph-mon[31175]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:40:31.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:31 smithi176 ceph-mon[35324]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:40:31.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:31 smithi176 ceph-mon[35324]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:40:32.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:31 smithi164 ceph-mon[35121]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T01:40:32.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:31 smithi164 ceph-mon[35121]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T01:40:32.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.161+0000 7f746c4b5640 1 -- 172.21.15.139:0/674747558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7464102dd0 msgr2=0x7f74641031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:32.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.161+0000 7f746c4b5640 1 --2- 172.21.15.139:0/674747558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7464102dd0 0x7f74641031b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7460009930 tx=0x7f746002a420 comp rx=0 tx=0).stop 2024-09-16T01:40:32.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.161+0000 7f746c4b5640 1 -- 172.21.15.139:0/674747558 shutdown_connections 2024-09-16T01:40:32.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.161+0000 7f746c4b5640 1 --2- 172.21.15.139:0/674747558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7464108300 0x7f746410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:32.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.161+0000 7f746c4b5640 1 --2- 172.21.15.139:0/674747558 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464103780 0x7f7464107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.161+0000 7f746c4b5640 1 --2- 172.21.15.139:0/674747558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7464102dd0 0x7f74641031b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.161+0000 7f746c4b5640 1 -- 172.21.15.139:0/674747558 >> 172.21.15.139:0/674747558 conn(0x7f74640fc7d0 msgr2=0x7f74640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.161+0000 7f746c4b5640 1 -- 172.21.15.139:0/674747558 shutdown_connections 2024-09-16T01:40:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.161+0000 7f746c4b5640 1 -- 172.21.15.139:0/674747558 wait complete. 2024-09-16T01:40:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.161+0000 7f746c4b5640 1 Processor -- start 2024-09-16T01:40:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.161+0000 7f746c4b5640 1 -- start start 2024-09-16T01:40:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746c4b5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7464102dd0 0x7f7464198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746c4b5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464103780 0x7f7464198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746c4b5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7464108300 0x7f746419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746c4b5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7464112090 con 0x7f7464102dd0 2024-09-16T01:40:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746c4b5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7464111f10 con 0x7f7464108300 2024-09-16T01:40:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746c4b5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7464112210 con 0x7f7464103780 2024-09-16T01:40:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746a251640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7464102dd0 0x7f7464198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746a251640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7464102dd0 0x7f7464198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50668/0 (socket says 172.21.15.139:50668) 2024-09-16T01:40:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746a251640 1 -- 172.21.15.139:0/2448201551 learned_addr learned my addr 172.21.15.139:0/2448201551 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f7469a50640 1 --2- 172.21.15.139:0/2448201551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464103780 0x7f7464198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746aa52640 1 --2- 172.21.15.139:0/2448201551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7464108300 0x7f746419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746a251640 1 -- 172.21.15.139:0/2448201551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464103780 msgr2=0x7f7464198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746a251640 1 --2- 172.21.15.139:0/2448201551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464103780 0x7f7464198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746a251640 1 -- 172.21.15.139:0/2448201551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7464108300 msgr2=0x7f746419cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746a251640 1 --2- 172.21.15.139:0/2448201551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7464108300 0x7f746419cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746a251640 1 -- 172.21.15.139:0/2448201551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746419d600 con 0x7f7464102dd0 2024-09-16T01:40:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f746a251640 1 --2- 172.21.15.139:0/2448201551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7464102dd0 0x7f7464198650 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7460009230 tx=0x7f7460004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.162+0000 7f745b7fe640 1 -- 172.21.15.139:0/2448201551 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746003f030 con 0x7f7464102dd0 2024-09-16T01:40:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.163+0000 7f745b7fe640 1 -- 172.21.15.139:0/2448201551 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7460003ad0 con 0x7f7464102dd0 2024-09-16T01:40:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.163+0000 7f746c4b5640 1 -- 172.21.15.139:0/2448201551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746419d890 con 0x7f7464102dd0 2024-09-16T01:40:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.163+0000 7f745b7fe640 1 -- 172.21.15.139:0/2448201551 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460035070 con 0x7f7464102dd0 2024-09-16T01:40:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.163+0000 7f746c4b5640 1 -- 172.21.15.139:0/2448201551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74641a51d0 con 0x7f7464102dd0 2024-09-16T01:40:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.163+0000 7f746c4b5640 1 -- 172.21.15.139:0/2448201551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f746406a7b0 con 0x7f7464102dd0 2024-09-16T01:40:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.165+0000 7f745b7fe640 1 -- 172.21.15.139:0/2448201551 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 23) ==== 138532+0+0 (secure 0 0 0) 0x7f7460042030 con 0x7f7464102dd0 2024-09-16T01:40:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.166+0000 7f745b7fe640 1 --2- 172.21.15.139:0/2448201551 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f74500a3440 0x7f74500a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.166+0000 7f7469a50640 1 --2- 172.21.15.139:0/2448201551 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f74500a3440 0x7f74500a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.166+0000 7f745b7fe640 1 -- 172.21.15.139:0/2448201551 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f74600f0ba0 con 0x7f7464102dd0 2024-09-16T01:40:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.166+0000 7f7469a50640 1 --2- 172.21.15.139:0/2448201551 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f74500a3440 0x7f74500a5900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f74640fefa0 tx=0x7f745400b410 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.169+0000 7f745b7fe640 1 -- 172.21.15.139:0/2448201551 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74600bd060 con 0x7f7464102dd0 2024-09-16T01:40:32.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.370+0000 7f746c4b5640 1 -- 172.21.15.139:0/2448201551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74641031b0 con 0x7f7464102dd0 2024-09-16T01:40:32.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.371+0000 7f745b7fe640 1 -- 172.21.15.139:0/2448201551 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11995 (secure 0 0 0) 0x7f74600bc960 con 0x7f7464102dd0 2024-09-16T01:40:32.373 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:40:32.373 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":29,"btime":"2024-09-16T01:40:22:367807+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:40:22.317639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:40:32.374 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 29 2024-09-16T01:40:32.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.374+0000 7f746c4b5640 1 -- 172.21.15.139:0/2448201551 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f74500a3440 msgr2=0x7f74500a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:32.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.374+0000 7f746c4b5640 1 --2- 172.21.15.139:0/2448201551 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f74500a3440 0x7f74500a5900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f74640fefa0 tx=0x7f745400b410 comp rx=0 tx=0).stop 2024-09-16T01:40:32.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.374+0000 7f746c4b5640 1 -- 172.21.15.139:0/2448201551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7464102dd0 msgr2=0x7f7464198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:32.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.374+0000 7f746c4b5640 1 --2- 172.21.15.139:0/2448201551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7464102dd0 0x7f7464198650 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7460009230 tx=0x7f7460004270 comp rx=0 tx=0).stop 2024-09-16T01:40:32.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.374+0000 7f746c4b5640 1 -- 172.21.15.139:0/2448201551 shutdown_connections 2024-09-16T01:40:32.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.374+0000 7f746c4b5640 1 --2- 172.21.15.139:0/2448201551 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f74500a3440 0x7f74500a5900 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:32.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.374+0000 7f746c4b5640 1 --2- 172.21.15.139:0/2448201551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7464108300 0x7f746419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:32.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.374+0000 7f746c4b5640 1 --2- 172.21.15.139:0/2448201551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464103780 0x7f7464198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:32.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.374+0000 7f746c4b5640 1 --2- 172.21.15.139:0/2448201551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7464102dd0 0x7f7464198650 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:32.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.374+0000 7f746c4b5640 1 -- 172.21.15.139:0/2448201551 >> 172.21.15.139:0/2448201551 conn(0x7f74640fc7d0 msgr2=0x7f7464105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:32.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.374+0000 7f746c4b5640 1 -- 172.21.15.139:0/2448201551 shutdown_connections 2024-09-16T01:40:32.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:32.374+0000 7f746c4b5640 1 -- 172.21.15.139:0/2448201551 wait complete. 2024-09-16T01:40:32.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:32 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2448201551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:32.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:32 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2448201551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:33.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:32 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2448201551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:33.092 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:40:33.341 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:33.751 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:33 smithi139 ceph-mon[31175]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 131 KiB/s wr, 79 op/s 2024-09-16T01:40:33.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.828+0000 7f037b827640 1 -- 172.21.15.139:0/3046680965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0374108160 msgr2=0x7f037410eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:33.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.828+0000 7f037b827640 1 --2- 172.21.15.139:0/3046680965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0374108160 0x7f037410eb00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f036c009cd0 tx=0x7f036c02a910 comp rx=0 tx=0).stop 2024-09-16T01:40:33.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.828+0000 7f037b827640 1 -- 172.21.15.139:0/3046680965 shutdown_connections 2024-09-16T01:40:33.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.828+0000 7f037b827640 1 --2- 172.21.15.139:0/3046680965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0374108160 0x7f037410eb00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:33.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.828+0000 7f037b827640 1 --2- 172.21.15.139:0/3046680965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0374101290 0x7f0374107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:33.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.828+0000 7f037b827640 1 --2- 172.21.15.139:0/3046680965 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03741008e0 0x7f0374100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:33.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.828+0000 7f037b827640 1 -- 172.21.15.139:0/3046680965 >> 172.21.15.139:0/3046680965 conn(0x7f0374077210 msgr2=0x7f0374077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:33.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.828+0000 7f037b827640 1 -- 172.21.15.139:0/3046680965 shutdown_connections 2024-09-16T01:40:33.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.828+0000 7f037b827640 1 -- 172.21.15.139:0/3046680965 wait complete. 2024-09-16T01:40:33.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.828+0000 7f037b827640 1 Processor -- start 2024-09-16T01:40:33.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f037b827640 1 -- start start 2024-09-16T01:40:33.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f037b827640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03741008e0 0x7f03741984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:33.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f037b827640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0374101290 0x7f0374198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:33.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f037b827640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0374108160 0x7f037419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:33.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f037b827640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0374112030 con 0x7f03741008e0 2024-09-16T01:40:33.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f037b827640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0374111eb0 con 0x7f0374108160 2024-09-16T01:40:33.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f037b827640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f03741121b0 con 0x7f0374101290 2024-09-16T01:40:33.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f03795c3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03741008e0 0x7f03741984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:33.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f03795c3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03741008e0 0x7f03741984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46452/0 (socket says 172.21.15.139:46452) 2024-09-16T01:40:33.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f03795c3640 1 -- 172.21.15.139:0/2495115003 learned_addr learned my addr 172.21.15.139:0/2495115003 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:33.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f0378dc2640 1 --2- 172.21.15.139:0/2495115003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0374101290 0x7f0374198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:33.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f03795c3640 1 -- 172.21.15.139:0/2495115003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0374101290 msgr2=0x7f0374198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:33.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f03795c3640 1 --2- 172.21.15.139:0/2495115003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0374101290 0x7f0374198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:33.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f03795c3640 1 -- 172.21.15.139:0/2495115003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0374108160 msgr2=0x7f037419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:33.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f03795c3640 1 --2- 172.21.15.139:0/2495115003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0374108160 0x7f037419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:33.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f03795c3640 1 -- 172.21.15.139:0/2495115003 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037419d480 con 0x7f03741008e0 2024-09-16T01:40:33.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f03795c3640 1 --2- 172.21.15.139:0/2495115003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03741008e0 0x7f03741984d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f037000d770 tx=0x7f037000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:33.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f036a7fc640 1 -- 172.21.15.139:0/2495115003 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0370017070 con 0x7f03741008e0 2024-09-16T01:40:33.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f037b827640 1 -- 172.21.15.139:0/2495115003 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f037419d770 con 0x7f03741008e0 2024-09-16T01:40:33.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f036a7fc640 1 -- 172.21.15.139:0/2495115003 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0370005500 con 0x7f03741008e0 2024-09-16T01:40:33.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.829+0000 7f036a7fc640 1 -- 172.21.15.139:0/2495115003 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0370004e00 con 0x7f03741008e0 2024-09-16T01:40:33.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.830+0000 7f037b827640 1 -- 172.21.15.139:0/2495115003 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03741a5050 con 0x7f03741008e0 2024-09-16T01:40:33.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.831+0000 7f037b827640 1 -- 172.21.15.139:0/2495115003 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0374100cc0 con 0x7f03741008e0 2024-09-16T01:40:33.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.832+0000 7f036a7fc640 1 -- 172.21.15.139:0/2495115003 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 23) ==== 138532+0+0 (secure 0 0 0) 0x7f0370004fa0 con 0x7f03741008e0 2024-09-16T01:40:33.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.832+0000 7f036a7fc640 1 --2- 172.21.15.139:0/2495115003 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f03600a3440 0x7f03600a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:33.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.832+0000 7f0378dc2640 1 --2- 172.21.15.139:0/2495115003 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f03600a3440 0x7f03600a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:33.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.832+0000 7f036a7fc640 1 -- 172.21.15.139:0/2495115003 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f03700d0180 con 0x7f03741008e0 2024-09-16T01:40:33.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.833+0000 7f0378dc2640 1 --2- 172.21.15.139:0/2495115003 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f03600a3440 0x7f03600a5900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0374107a30 tx=0x7f0364008040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:33.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:33.835+0000 7f036a7fc640 1 -- 172.21.15.139:0/2495115003 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f037009c640 con 0x7f03741008e0 2024-09-16T01:40:33.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:33 smithi176 ceph-mon[35324]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 131 KiB/s wr, 79 op/s 2024-09-16T01:40:34.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:34.037+0000 7f037b827640 1 -- 172.21.15.139:0/2495115003 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f037402d7c0 con 0x7f03741008e0 2024-09-16T01:40:34.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:34.038+0000 7f036a7fc640 1 -- 172.21.15.139:0/2495115003 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11995 (secure 0 0 0) 0x7f037009bf40 con 0x7f03741008e0 2024-09-16T01:40:34.040 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:40:34.040 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":29,"btime":"2024-09-16T01:40:22:367807+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:40:22.317639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:40:34.041 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 29 2024-09-16T01:40:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:34.041+0000 7f037b827640 1 -- 172.21.15.139:0/2495115003 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f03600a3440 msgr2=0x7f03600a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:34.041+0000 7f037b827640 1 --2- 172.21.15.139:0/2495115003 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f03600a3440 0x7f03600a5900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0374107a30 tx=0x7f0364008040 comp rx=0 tx=0).stop 2024-09-16T01:40:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:34.041+0000 7f037b827640 1 -- 172.21.15.139:0/2495115003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03741008e0 msgr2=0x7f03741984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:34.041+0000 7f037b827640 1 --2- 172.21.15.139:0/2495115003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03741008e0 0x7f03741984d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f037000d770 tx=0x7f037000dc30 comp rx=0 tx=0).stop 2024-09-16T01:40:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:34.041+0000 7f037b827640 1 -- 172.21.15.139:0/2495115003 shutdown_connections 2024-09-16T01:40:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:34.041+0000 7f037b827640 1 --2- 172.21.15.139:0/2495115003 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f03600a3440 0x7f03600a5900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:34.041+0000 7f037b827640 1 --2- 172.21.15.139:0/2495115003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0374108160 0x7f037419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:34.041+0000 7f037b827640 1 --2- 172.21.15.139:0/2495115003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0374101290 0x7f0374198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:34.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:34.041+0000 7f037b827640 1 --2- 172.21.15.139:0/2495115003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03741008e0 0x7f03741984d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:34.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:34.041+0000 7f037b827640 1 -- 172.21.15.139:0/2495115003 >> 172.21.15.139:0/2495115003 conn(0x7f0374077210 msgr2=0x7f03740ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:34.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:34.041+0000 7f037b827640 1 -- 172.21.15.139:0/2495115003 shutdown_connections 2024-09-16T01:40:34.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:34.041+0000 7f037b827640 1 -- 172.21.15.139:0/2495115003 wait complete. 2024-09-16T01:40:34.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:33 smithi164 ceph-mon[35121]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 131 KiB/s wr, 79 op/s 2024-09-16T01:40:34.762 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T01:40:34.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:34 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2495115003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:34.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:34 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2495115003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:35.014 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:35.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:34 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2495115003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:35.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.524+0000 7fab114f2640 1 -- 172.21.15.139:0/3717763133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab0c101df0 msgr2=0x7fab0c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:35.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.524+0000 7fab114f2640 1 --2- 172.21.15.139:0/3717763133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab0c101df0 0x7fab0c1021d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7faafc009900 tx=0x7faafc02a410 comp rx=0 tx=0).stop 2024-09-16T01:40:35.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.524+0000 7fab114f2640 1 -- 172.21.15.139:0/3717763133 shutdown_connections 2024-09-16T01:40:35.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.524+0000 7fab114f2640 1 --2- 172.21.15.139:0/3717763133 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab0c106f20 0x7fab0c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:35.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.524+0000 7fab114f2640 1 --2- 172.21.15.139:0/3717763133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c1027a0 0x7fab0c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:35.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.524+0000 7fab114f2640 1 --2- 172.21.15.139:0/3717763133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab0c101df0 0x7fab0c1021d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:35.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.524+0000 7fab114f2640 1 -- 172.21.15.139:0/3717763133 >> 172.21.15.139:0/3717763133 conn(0x7fab0c0fd8b0 msgr2=0x7fab0c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:35.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.524+0000 7fab114f2640 1 -- 172.21.15.139:0/3717763133 shutdown_connections 2024-09-16T01:40:35.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.524+0000 7fab114f2640 1 -- 172.21.15.139:0/3717763133 wait complete. 2024-09-16T01:40:35.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.524+0000 7fab114f2640 1 Processor -- start 2024-09-16T01:40:35.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab114f2640 1 -- start start 2024-09-16T01:40:35.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab114f2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab0c101df0 0x7fab0c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:35.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab114f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab0c1027a0 0x7fab0c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:35.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab114f2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c106f20 0x7fab0c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:35.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab114f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c111f40 con 0x7fab0c1027a0 2024-09-16T01:40:35.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab114f2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c111dc0 con 0x7fab0c106f20 2024-09-16T01:40:35.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab114f2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c1120c0 con 0x7fab0c101df0 2024-09-16T01:40:35.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab0a7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab0c1027a0 0x7fab0c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:35.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab0a7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab0c1027a0 0x7fab0c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46484/0 (socket says 172.21.15.139:46484) 2024-09-16T01:40:35.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab0a7fc640 1 -- 172.21.15.139:0/857335496 learned_addr learned my addr 172.21.15.139:0/857335496 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:35.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab0affd640 1 --2- 172.21.15.139:0/857335496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab0c101df0 0x7fab0c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:35.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab0b7fe640 1 --2- 172.21.15.139:0/857335496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c106f20 0x7fab0c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:35.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab0a7fc640 1 -- 172.21.15.139:0/857335496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab0c101df0 msgr2=0x7fab0c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:35.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab0a7fc640 1 --2- 172.21.15.139:0/857335496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab0c101df0 0x7fab0c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:35.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab0a7fc640 1 -- 172.21.15.139:0/857335496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c106f20 msgr2=0x7fab0c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:35.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab0a7fc640 1 --2- 172.21.15.139:0/857335496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c106f20 0x7fab0c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:35.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.525+0000 7fab0a7fc640 1 -- 172.21.15.139:0/857335496 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c19d3d0 con 0x7fab0c1027a0 2024-09-16T01:40:35.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.526+0000 7fab0a7fc640 1 --2- 172.21.15.139:0/857335496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab0c1027a0 0x7fab0c198960 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fab0000d950 tx=0x7fab0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:35.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.526+0000 7faaf3fff640 1 -- 172.21.15.139:0/857335496 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00017030 con 0x7fab0c1027a0 2024-09-16T01:40:35.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.526+0000 7faaf3fff640 1 -- 172.21.15.139:0/857335496 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab00004020 con 0x7fab0c1027a0 2024-09-16T01:40:35.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.526+0000 7fab114f2640 1 -- 172.21.15.139:0/857335496 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c19d6c0 con 0x7fab0c1027a0 2024-09-16T01:40:35.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.526+0000 7faaf3fff640 1 -- 172.21.15.139:0/857335496 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab000047a0 con 0x7fab0c1027a0 2024-09-16T01:40:35.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.526+0000 7fab114f2640 1 -- 172.21.15.139:0/857335496 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c1a4fa0 con 0x7fab0c1027a0 2024-09-16T01:40:35.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.527+0000 7fab114f2640 1 -- 172.21.15.139:0/857335496 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c1021d0 con 0x7fab0c1027a0 2024-09-16T01:40:35.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.528+0000 7faaf3fff640 1 -- 172.21.15.139:0/857335496 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 23) ==== 138532+0+0 (secure 0 0 0) 0x7fab000041d0 con 0x7fab0c1027a0 2024-09-16T01:40:35.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.528+0000 7faaf3fff640 1 --2- 172.21.15.139:0/857335496 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faaf40a3440 0x7faaf40a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:35.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.528+0000 7faaf3fff640 1 -- 172.21.15.139:0/857335496 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fab000cfba0 con 0x7fab0c1027a0 2024-09-16T01:40:35.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.529+0000 7fab0affd640 1 --2- 172.21.15.139:0/857335496 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faaf40a3440 0x7faaf40a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:35.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.529+0000 7fab0affd640 1 --2- 172.21.15.139:0/857335496 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faaf40a3440 0x7faaf40a5900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faafc009230 tx=0x7faafc0058d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:35.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.533+0000 7faaf3fff640 1 -- 172.21.15.139:0/857335496 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab0009c060 con 0x7fab0c1027a0 2024-09-16T01:40:35.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.703+0000 7fab114f2640 1 -- 172.21.15.139:0/857335496 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab0c199780 con 0x7fab0c1027a0 2024-09-16T01:40:35.704 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:35 smithi139 ceph-mon[31175]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-09-16T01:40:35.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.705+0000 7faaf3fff640 1 -- 172.21.15.139:0/857335496 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+25367 (secure 0 0 0) 0x7fab0c199780 con 0x7fab0c1027a0 2024-09-16T01:40:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:40:35.706 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":86,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","created":"2024-09-16T01:30:17.770313+0000","modified":"2024-09-16T01:40:23.675005+0000","last_up_change":"2024-09-16T01:37:08.611901+0000","last_in_change":"2024-09-16T01:36:45.223926+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:33:19.639647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T01:37:57.948214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T01:38:00.263797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T01:38:06.773799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b66696da-ef66-4f25-9527-bbcc9ded2cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6803","nonce":3190810133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6805","nonce":3190810133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6809","nonce":3190810133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6807","nonce":3190810133}]},"public_addr":"172.21.15.139:6803/3190810133","cluster_addr":"172.21.15.139:6805/3190810133","heartbeat_back_addr":"172.21.15.139:6809/3190810133","heartbeat_front_addr":"172.21.15.139:6807/3190810133","state":["exists","up"]},{"osd":1,"uuid":"ccad17f2-0750-425e-9810-49a13a929ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6801","nonce":2699270330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6803","nonce":2699270330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6807","nonce":2699270330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6805","nonce":2699270330}]},"public_addr":"172.21.15.164:6801/2699270330","cluster_addr":"172.21.15.164:6803/2699270330","heartbeat_back_addr":"172.21.15.164:6807/2699270330","heartbeat_front_addr":"172.21.15.164:6805/2699270330","state":["exists","up"]},{"osd":2,"uuid":"baf0ba73-8faf-4e51-a171-c47898800638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6801","nonce":2034471646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6803","nonce":2034471646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6807","nonce":2034471646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6805","nonce":2034471646}]},"public_addr":"172.21.15.176:6801/2034471646","cluster_addr":"172.21.15.176:6803/2034471646","heartbeat_back_addr":"172.21.15.176:6807/2034471646","heartbeat_front_addr":"172.21.15.176:6805/2034471646","state":["exists","up"]},{"osd":3,"uuid":"98957bf1-67bb-4092-9ea4-b263a5706f8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6811","nonce":797147071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6813","nonce":797147071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6817","nonce":797147071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6815","nonce":797147071}]},"public_addr":"172.21.15.139:6811/797147071","cluster_addr":"172.21.15.139:6813/797147071","heartbeat_back_addr":"172.21.15.139:6817/797147071","heartbeat_front_addr":"172.21.15.139:6815/797147071","state":["exists","up"]},{"osd":4,"uuid":"9cd76696-8161-4e82-a571-29e98c11d5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6809","nonce":1421304792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6811","nonce":1421304792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6815","nonce":1421304792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6813","nonce":1421304792}]},"public_addr":"172.21.15.164:6809/1421304792","cluster_addr":"172.21.15.164:6811/1421304792","heartbeat_back_addr":"172.21.15.164:6815/1421304792","heartbeat_front_addr":"172.21.15.164:6813/1421304792","state":["exists","up"]},{"osd":5,"uuid":"ccc17a29-9a58-40ec-b453-5c38bee86f25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6809","nonce":10600602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6811","nonce":10600602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6815","nonce":10600602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6813","nonce":10600602}]},"public_addr":"172.21.15.176:6809/10600602","cluster_addr":"172.21.15.176:6811/10600602","heartbeat_back_addr":"172.21.15.176:6815/10600602","heartbeat_front_addr":"172.21.15.176:6813/10600602","state":["exists","up"]},{"osd":6,"uuid":"e5ee7212-10c6-4a05-992e-622a14a5a6b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6819","nonce":2601111164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6821","nonce":2601111164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6825","nonce":2601111164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6823","nonce":2601111164}]},"public_addr":"172.21.15.139:6819/2601111164","cluster_addr":"172.21.15.139:6821/2601111164","heartbeat_back_addr":"172.21.15.139:6825/2601111164","heartbeat_front_addr":"172.21.15.139:6823/2601111164","state":["exists","up"]},{"osd":7,"uuid":"23143c15-0839-4fd5-9d0a-19ae891000f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6817","nonce":2074080693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6819","nonce":2074080693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6823","nonce":2074080693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6821","nonce":2074080693}]},"public_addr":"172.21.15.164:6817/2074080693","cluster_addr":"172.21.15.164:6819/2074080693","heartbeat_back_addr":"172.21.15.164:6823/2074080693","heartbeat_front_addr":"172.21.15.164:6821/2074080693","state":["exists","up"]},{"osd":8,"uuid":"1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6817","nonce":593133361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6819","nonce":593133361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6823","nonce":593133361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6821","nonce":593133361}]},"public_addr":"172.21.15.176:6817/593133361","cluster_addr":"172.21.15.176:6819/593133361","heartbeat_back_addr":"172.21.15.176:6823/593133361","heartbeat_front_addr":"172.21.15.176:6821/593133361","state":["exists","up"]},{"osd":9,"uuid":"09d25a3b-1e66-480e-9433-ceef7f0b9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6827","nonce":3047942108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6829","nonce":3047942108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6833","nonce":3047942108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6831","nonce":3047942108}]},"public_addr":"172.21.15.139:6827/3047942108","cluster_addr":"172.21.15.139:6829/3047942108","heartbeat_back_addr":"172.21.15.139:6833/3047942108","heartbeat_front_addr":"172.21.15.139:6831/3047942108","state":["exists","up"]},{"osd":10,"uuid":"32b4d8ce-b644-42fa-80cc-a15a1f5f3a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6825","nonce":4172533828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6827","nonce":4172533828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6831","nonce":4172533828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6829","nonce":4172533828}]},"public_addr":"172.21.15.164:6825/4172533828","cluster_addr":"172.21.15.164:6827/4172533828","heartbeat_back_addr":"172.21.15.164:6831/4172533828","heartbeat_front_addr":"172.21.15.164:6829/4172533828","state":["exists","up"]},{"osd":11,"uuid":"49098bdc-58a9-4a74-8045-d408af7711ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6825","nonce":1854547565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6827","nonce":1854547565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6831","nonce":1854547565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6829","nonce":1854547565}]},"public_addr":"172.21.15.176:6825/1854547565","cluster_addr":"172.21.15.176:6827/1854547565","heartbeat_back_addr":"172.21.15.176:6831/1854547565","heartbeat_front_addr":"172.21.15.176:6829/1854547565","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:27.374845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:52.611853+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:17.229567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:43.330503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:07.988876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:33.787864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:58.864910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:23.815006+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:49.551954+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:15.496773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:41.647557+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:37:07.089355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2901696849":"2024-09-17T01:40:23.674956+0000","172.21.15.139:0/2608207648":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/1830135963":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/677216867":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6800/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/3167151595":"2024-09-17T01:40:23.674956+0000","172.21.15.139:0/1329877785":"2024-09-17T01:40:23.674956+0000","172.21.15.139:6800/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6801/331514141":"2024-09-17T01:40:23.674956+0000","172.21.15.139:6801/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/1608930262":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/2362566775":"2024-09-17T01:40:23.674956+0000","172.21.15.139:6801/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6800/331514141":"2024-09-17T01:40:23.674956+0000","172.21.15.139:0/3997627612":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/3772232169":"2024-09-17T01:31:07.344357+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:40:35.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.707+0000 7fab114f2640 1 -- 172.21.15.139:0/857335496 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faaf40a3440 msgr2=0x7faaf40a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:35.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.707+0000 7fab114f2640 1 --2- 172.21.15.139:0/857335496 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faaf40a3440 0x7faaf40a5900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faafc009230 tx=0x7faafc0058d0 comp rx=0 tx=0).stop 2024-09-16T01:40:35.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.707+0000 7fab114f2640 1 -- 172.21.15.139:0/857335496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab0c1027a0 msgr2=0x7fab0c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:35.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.707+0000 7fab114f2640 1 --2- 172.21.15.139:0/857335496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab0c1027a0 0x7fab0c198960 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fab0000d950 tx=0x7fab0000de10 comp rx=0 tx=0).stop 2024-09-16T01:40:35.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.707+0000 7fab114f2640 1 -- 172.21.15.139:0/857335496 shutdown_connections 2024-09-16T01:40:35.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.707+0000 7fab114f2640 1 --2- 172.21.15.139:0/857335496 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faaf40a3440 0x7faaf40a5900 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:35.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.707+0000 7fab114f2640 1 --2- 172.21.15.139:0/857335496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c106f20 0x7fab0c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:35.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.707+0000 7fab114f2640 1 --2- 172.21.15.139:0/857335496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab0c1027a0 0x7fab0c198960 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:35.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.707+0000 7fab114f2640 1 --2- 172.21.15.139:0/857335496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab0c101df0 0x7fab0c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:35.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.707+0000 7fab114f2640 1 -- 172.21.15.139:0/857335496 >> 172.21.15.139:0/857335496 conn(0x7fab0c0fd8b0 msgr2=0x7fab0c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:35.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.707+0000 7fab114f2640 1 -- 172.21.15.139:0/857335496 shutdown_connections 2024-09-16T01:40:35.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:35.707+0000 7fab114f2640 1 -- 172.21.15.139:0/857335496 wait complete. 2024-09-16T01:40:35.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:35 smithi176 ceph-mon[35324]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-09-16T01:40:36.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:35 smithi164 ceph-mon[35121]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-09-16T01:40:36.471 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-16T01:40:36.720 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:36.841 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:36 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/857335496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:40:36.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:36 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/857335496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:40:37.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:36 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/857335496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:40:37.870 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:37 smithi139 ceph-mon[31175]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-16T01:40:37.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:37 smithi176 ceph-mon[35324]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-16T01:40:37.954 INFO:teuthology.orchestra.run.smithi139.stdout:{"run_dir":"/var/run/ceph"} 2024-09-16T01:40:37.955 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-16T01:40:37.955 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 3393 2024-09-16T01:40:37.955 DEBUG:tasks.quiescer:Yielding 2024-09-16T01:40:37.955 INFO:teuthology.run_tasks:Running task workunit... 2024-09-16T01:40:37.971 INFO:tasks.workunit:Pulling workunits from ref ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T01:40:37.971 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-16T01:40:37.971 INFO:tasks.workunit:timeout=3h 2024-09-16T01:40:37.971 INFO:tasks.workunit:cleanup=True 2024-09-16T01:40:37.972 DEBUG:teuthology.orchestra.run.smithi139:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-16T01:40:37.972 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:40:37.988 INFO:teuthology.orchestra.run.smithi139.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-16T01:40:37.989 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-16T01:40:37.989 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 6ch/108d Inode: 1099511628283 Links: 2 2024-09-16T01:40:37.989 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T01:40:37.989 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-16T01:40:37.989 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-16 01:38:47.391693589 +0000 2024-09-16T01:40:37.989 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-16 01:38:47.391693589 +0000 2024-09-16T01:40:37.989 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-16 01:40:21.670621091 +0000 2024-09-16T01:40:37.989 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-16 01:38:47.391693589 +0000 2024-09-16T01:40:37.989 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-16T01:40:37.989 DEBUG:teuthology.orchestra.run.smithi139:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-16T01:40:38.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:37 smithi164 ceph-mon[35121]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-16T01:40:38.064 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T01:40:38.120 INFO:tasks.workunit.client.0.smithi139.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-16T01:40:38.226 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:38.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.735+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1333300756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f54106f50 msgr2=0x7f5f5410eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:38.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.735+0000 7f5f5bd9b640 1 --2- 172.21.15.139:0/1333300756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f54106f50 0x7f5f5410eaf0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009930 tx=0x7f5f4402a420 comp rx=0 tx=0).stop 2024-09-16T01:40:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.736+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1333300756 shutdown_connections 2024-09-16T01:40:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.736+0000 7f5f5bd9b640 1 --2- 172.21.15.139:0/1333300756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f54106f50 0x7f5f5410eaf0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.736+0000 7f5f5bd9b640 1 --2- 172.21.15.139:0/1333300756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f54100ca0 0x7f5f54106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.736+0000 7f5f5bd9b640 1 --2- 172.21.15.139:0/1333300756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f541002f0 0x7f5f541006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.736+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1333300756 >> 172.21.15.139:0/1333300756 conn(0x7f5f54077210 msgr2=0x7f5f540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.736+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1333300756 shutdown_connections 2024-09-16T01:40:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.736+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1333300756 wait complete. 2024-09-16T01:40:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.736+0000 7f5f5bd9b640 1 Processor -- start 2024-09-16T01:40:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.736+0000 7f5f5bd9b640 1 -- start start 2024-09-16T01:40:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.736+0000 7f5f5bd9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f541002f0 0x7f5f54101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f5bd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f54100ca0 0x7f5f541017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f5bd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f54106f50 0x7f5f54101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f5bd9b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5f54111fa0 con 0x7f5f54106f50 2024-09-16T01:40:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f5bd9b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5f54111e20 con 0x7f5f541002f0 2024-09-16T01:40:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f5bd9b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5f54112120 con 0x7f5f54100ca0 2024-09-16T01:40:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f5a338640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f54106f50 0x7f5f54101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f5a338640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f54106f50 0x7f5f54101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46502/0 (socket says 172.21.15.139:46502) 2024-09-16T01:40:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f5a338640 1 -- 172.21.15.139:0/1737423236 learned_addr learned my addr 172.21.15.139:0/1737423236 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f59b37640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f541002f0 0x7f5f54101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f59336640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f54100ca0 0x7f5f541017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f5a338640 1 -- 172.21.15.139:0/1737423236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f54100ca0 msgr2=0x7f5f541017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f5a338640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f54100ca0 0x7f5f541017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f5a338640 1 -- 172.21.15.139:0/1737423236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f541002f0 msgr2=0x7f5f54101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f5a338640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f541002f0 0x7f5f54101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f5a338640 1 -- 172.21.15.139:0/1737423236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f541a68e0 con 0x7f5f54106f50 2024-09-16T01:40:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f59336640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f54100ca0 0x7f5f541017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:38.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f5a338640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f54106f50 0x7f5f54101cf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5f44005ab0 tx=0x7f5f440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:38.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.737+0000 7f5f52ffd640 1 -- 172.21.15.139:0/1737423236 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4403f030 con 0x7f5f54106f50 2024-09-16T01:40:38.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.738+0000 7f5f52ffd640 1 -- 172.21.15.139:0/1737423236 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f44003ce0 con 0x7f5f54106f50 2024-09-16T01:40:38.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.738+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1737423236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f541a6b70 con 0x7f5f54106f50 2024-09-16T01:40:38.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.738+0000 7f5f52ffd640 1 -- 172.21.15.139:0/1737423236 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f44035070 con 0x7f5f54106f50 2024-09-16T01:40:38.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.738+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1737423236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f541a7050 con 0x7f5f54106f50 2024-09-16T01:40:38.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.739+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1737423236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f541006d0 con 0x7f5f54106f50 2024-09-16T01:40:38.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.740+0000 7f5f52ffd640 1 -- 172.21.15.139:0/1737423236 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 23) ==== 138532+0+0 (secure 0 0 0) 0x7f5f44042030 con 0x7f5f54106f50 2024-09-16T01:40:38.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.741+0000 7f5f52ffd640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5f3c0a3440 0x7f5f3c0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:38.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.741+0000 7f5f59b37640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5f3c0a3440 0x7f5f3c0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:38.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.741+0000 7f5f52ffd640 1 -- 172.21.15.139:0/1737423236 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f5f440f04b0 con 0x7f5f54106f50 2024-09-16T01:40:38.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.741+0000 7f5f59b37640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5f3c0a3440 0x7f5f3c0a5900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0058c0 tx=0x7f5f4c00a2d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:38.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.745+0000 7f5f52ffd640 1 -- 172.21.15.139:0/1737423236 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f440bc970 con 0x7f5f54106f50 2024-09-16T01:40:38.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.953+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1737423236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f5410a120 con 0x7f5f54106f50 2024-09-16T01:40:38.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.954+0000 7f5f52ffd640 1 -- 172.21.15.139:0/1737423236 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11995 (secure 0 0 0) 0x7f5f440bc270 con 0x7f5f54106f50 2024-09-16T01:40:38.955 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:40:38.955 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":29,"btime":"2024-09-16T01:40:22:367807+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:40:22.317639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:40:38.955 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 29 2024-09-16T01:40:38.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.956+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1737423236 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5f3c0a3440 msgr2=0x7f5f3c0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:38.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.956+0000 7f5f5bd9b640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5f3c0a3440 0x7f5f3c0a5900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0058c0 tx=0x7f5f4c00a2d0 comp rx=0 tx=0).stop 2024-09-16T01:40:38.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.956+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1737423236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f54106f50 msgr2=0x7f5f54101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:38.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.956+0000 7f5f5bd9b640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f54106f50 0x7f5f54101cf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5f44005ab0 tx=0x7f5f440042f0 comp rx=0 tx=0).stop 2024-09-16T01:40:38.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.956+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1737423236 shutdown_connections 2024-09-16T01:40:38.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.956+0000 7f5f5bd9b640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5f3c0a3440 0x7f5f3c0a5900 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:38.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.956+0000 7f5f5bd9b640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f54106f50 0x7f5f54101cf0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:38.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.956+0000 7f5f5bd9b640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f54100ca0 0x7f5f541017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:38.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.956+0000 7f5f5bd9b640 1 --2- 172.21.15.139:0/1737423236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f541002f0 0x7f5f54101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:38.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.956+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1737423236 >> 172.21.15.139:0/1737423236 conn(0x7f5f54077210 msgr2=0x7f5f54105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:38.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.957+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1737423236 shutdown_connections 2024-09-16T01:40:38.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:38.957+0000 7f5f5bd9b640 1 -- 172.21.15.139:0/1737423236 wait complete. 2024-09-16T01:40:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 -- 172.21.15.139:0/3116459168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f01009b0 msgr2=0x7f85f0100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 --2- 172.21.15.139:0/3116459168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f01009b0 0x7f85f0100e10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009930 tx=0x7f85e002a420 comp rx=0 tx=0).stop 2024-09-16T01:40:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 -- 172.21.15.139:0/3116459168 shutdown_connections 2024-09-16T01:40:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 --2- 172.21.15.139:0/3116459168 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85f0101350 0x7f85f010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 --2- 172.21.15.139:0/3116459168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f01009b0 0x7f85f0100e10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 --2- 172.21.15.139:0/3116459168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85f01069c0 0x7f85f0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 -- 172.21.15.139:0/3116459168 >> 172.21.15.139:0/3116459168 conn(0x7f85f0077990 msgr2=0x7f85f00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 -- 172.21.15.139:0/3116459168 shutdown_connections 2024-09-16T01:40:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 -- 172.21.15.139:0/3116459168 wait complete. 2024-09-16T01:40:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 Processor -- start 2024-09-16T01:40:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 -- start start 2024-09-16T01:40:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85f01009b0 0x7f85f0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f0101350 0x7f85f0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85f01069c0 0x7f85f019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f85f0111ef0 con 0x7f85f0101350 2024-09-16T01:40:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f85f0111d70 con 0x7f85f01009b0 2024-09-16T01:40:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.054+0000 7f85f77cf640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f85f0112070 con 0x7f85f01069c0 2024-09-16T01:40:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85f4d6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f0101350 0x7f85f0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85f4d6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f0101350 0x7f85f0198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46522/0 (socket says 172.21.15.139:46522) 2024-09-16T01:40:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85f4d6a640 1 -- 172.21.15.139:0/1025400303 learned_addr learned my addr 172.21.15.139:0/1025400303 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85f556b640 1 --2- 172.21.15.139:0/1025400303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85f01009b0 0x7f85f0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85f5d6c640 1 --2- 172.21.15.139:0/1025400303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85f01069c0 0x7f85f019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85f4d6a640 1 -- 172.21.15.139:0/1025400303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85f01069c0 msgr2=0x7f85f019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85f4d6a640 1 --2- 172.21.15.139:0/1025400303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85f01069c0 0x7f85f019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85f4d6a640 1 -- 172.21.15.139:0/1025400303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85f01009b0 msgr2=0x7f85f0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85f4d6a640 1 --2- 172.21.15.139:0/1025400303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85f01009b0 0x7f85f0198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85f4d6a640 1 -- 172.21.15.139:0/1025400303 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f019d330 con 0x7f85f0101350 2024-09-16T01:40:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85f4d6a640 1 --2- 172.21.15.139:0/1025400303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f0101350 0x7f85f0198880 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009230 tx=0x7f85e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85e67fc640 1 -- 172.21.15.139:0/1025400303 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e003f030 con 0x7f85f0101350 2024-09-16T01:40:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85e67fc640 1 -- 172.21.15.139:0/1025400303 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e0003ad0 con 0x7f85f0101350 2024-09-16T01:40:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85f77cf640 1 -- 172.21.15.139:0/1025400303 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f019d5c0 con 0x7f85f0101350 2024-09-16T01:40:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85e67fc640 1 -- 172.21.15.139:0/1025400303 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0035070 con 0x7f85f0101350 2024-09-16T01:40:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.055+0000 7f85f77cf640 1 -- 172.21.15.139:0/1025400303 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f01a4eb0 con 0x7f85f0101350 2024-09-16T01:40:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.057+0000 7f85e67fc640 1 -- 172.21.15.139:0/1025400303 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 23) ==== 138532+0+0 (secure 0 0 0) 0x7f85e0042030 con 0x7f85f0101350 2024-09-16T01:40:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85fa644740 1 Processor -- start 2024-09-16T01:40:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85fa644740 1 -- start start 2024-09-16T01:40:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85fa644740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fcbaf0 0x55c614fcbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85fa644740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcc410 0x55c614fd6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85fa644740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fd6cb0 0x55c614fd90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85fa644740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c614f302f0 con 0x55c614fd6cb0 2024-09-16T01:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85fa644740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c614ed8270 con 0x55c614fcc410 2024-09-16T01:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85fa644740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c614f2c2b0 con 0x55c614fcbaf0 2024-09-16T01:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85e67fc640 1 --2- 172.21.15.139:0/1025400303 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f85dc0a3500 0x7f85dc0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85f4d6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fd6cb0 0x55c614fd90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85f4d6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fd6cb0 0x55c614fd90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46530/0 (socket says 172.21.15.139:46530) 2024-09-16T01:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85f4d6a640 1 -- 172.21.15.139:0/3991413411 learned_addr learned my addr 172.21.15.139:0/3991413411 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85f5d6c640 1 --2- 172.21.15.139:0/3991413411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fcbaf0 0x55c614fcbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85f556b640 1 --2- 172.21.15.139:0/3991413411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcc410 0x55c614fd6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.058+0000 7f85f5d6c640 1 --2- 172.21.15.139:0/1025400303 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f85dc0a3500 0x7f85dc0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.059+0000 7f85e67fc640 1 -- 172.21.15.139:0/1025400303 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f85e00f0710 con 0x7f85f0101350 2024-09-16T01:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.059+0000 7f85f4d6a640 1 -- 172.21.15.139:0/3991413411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fcbaf0 msgr2=0x55c614fcbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.059+0000 7f85f4d6a640 1 --2- 172.21.15.139:0/3991413411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fcbaf0 0x55c614fcbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.059+0000 7f85f4d6a640 1 -- 172.21.15.139:0/3991413411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcc410 msgr2=0x55c614fd6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.059+0000 7f85f4d6a640 1 --2- 172.21.15.139:0/3991413411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcc410 0x55c614fd6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.059+0000 7f85f4d6a640 1 -- 172.21.15.139:0/3991413411 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c614ed7340 con 0x55c614fd6cb0 2024-09-16T01:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.059+0000 7f85f5d6c640 1 --2- 172.21.15.139:0/3991413411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fcbaf0 0x55c614fcbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.059+0000 7f85f4d6a640 1 --2- 172.21.15.139:0/3991413411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fd6cb0 0x55c614fd90a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f85e00c4c10 tx=0x7f85e0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4722459b99880103 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.059+0000 7f85f5d6c640 1 --2- 172.21.15.139:0/1025400303 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f85dc0a3500 0x7f85dc0a59c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009fd0 tx=0x7f85e8009600 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.059+0000 7f85db7fe640 1 -- 172.21.15.139:0/3991413411 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00bc580 con 0x55c614fd6cb0 2024-09-16T01:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.059+0000 7f85db7fe640 1 -- 172.21.15.139:0/3991413411 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e00c0880 con 0x55c614fd6cb0 2024-09-16T01:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.059+0000 7f85db7fe640 1 -- 172.21.15.139:0/3991413411 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00edca0 con 0x55c614fd6cb0 2024-09-16T01:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.060+0000 7f85fa644740 1 -- 172.21.15.139:0/3991413411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fd6cb0 msgr2=0x55c614fd90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.060+0000 7f85fa644740 1 --2- 172.21.15.139:0/3991413411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fd6cb0 0x55c614fd90a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f85e00c4c10 tx=0x7f85e0009260 comp rx=0 tx=0).stop 2024-09-16T01:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.060+0000 7f85fa644740 1 -- 172.21.15.139:0/3991413411 shutdown_connections 2024-09-16T01:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.060+0000 7f85fa644740 1 --2- 172.21.15.139:0/3991413411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fd6cb0 0x55c614fd90a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.060+0000 7f85fa644740 1 --2- 172.21.15.139:0/3991413411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcc410 0x55c614fd6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.060+0000 7f85fa644740 1 --2- 172.21.15.139:0/3991413411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fcbaf0 0x55c614fcbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.060+0000 7f85fa644740 1 -- 172.21.15.139:0/3991413411 >> 172.21.15.139:0/3991413411 conn(0x55c614fc8060 msgr2=0x55c614fc8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.060+0000 7f85fa644740 1 -- 172.21.15.139:0/3991413411 shutdown_connections 2024-09-16T01:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.060+0000 7f85fa644740 1 -- 172.21.15.139:0/3991413411 wait complete. 2024-09-16T01:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.060+0000 7f85fa644740 1 -- start start 2024-09-16T01:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.060+0000 7f85fa644740 1 Processor -- start 2024-09-16T01:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85fa644740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcbaf0 0x55c614fd3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85fa644740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcc410 0x55c614fd4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85fa644740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 0x55c614fd46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85fa644740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c614fd9640 con 0x55c614fcc410 2024-09-16T01:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85fa644740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c614eb9290 con 0x55c614fcbaf0 2024-09-16T01:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85fa644740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c614fd97c0 con 0x55c614fd6cb0 2024-09-16T01:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85fa644740 10 client.0 _ll_register_callbacks cb 0x55c614eea140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85f4d6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcc410 0x55c614fd4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85f4d6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcc410 0x55c614fd4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46536/0 (socket says 172.21.15.139:46536) 2024-09-16T01:40:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85f4d6a640 1 -- 172.21.15.139:0/2358262923 learned_addr learned my addr 172.21.15.139:0/2358262923 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85f556b640 1 --2- 172.21.15.139:0/2358262923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcbaf0 0x55c614fd3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85f5d6c640 1 --2- 172.21.15.139:0/2358262923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 0x55c614fd46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85f4d6a640 1 -- 172.21.15.139:0/2358262923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 msgr2=0x55c614fd46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85f4d6a640 1 --2- 172.21.15.139:0/2358262923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 0x55c614fd46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85f4d6a640 1 -- 172.21.15.139:0/2358262923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcbaf0 msgr2=0x55c614fd3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85f4d6a640 1 --2- 172.21.15.139:0/2358262923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcbaf0 0x55c614fd3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85f4d6a640 1 -- 172.21.15.139:0/2358262923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c614c43210 con 0x55c614fcc410 2024-09-16T01:40:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85f4d6a640 1 --2- 172.21.15.139:0/2358262923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcc410 0x55c614fd4180 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009a50 tx=0x7f85e00f0a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed52d06ffa9d55f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85d8ff9640 1 -- 172.21.15.139:0/2358262923 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0049050 con 0x55c614fcc410 2024-09-16T01:40:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85d8ff9640 1 -- 172.21.15.139:0/2358262923 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f85e00f0300 con 0x55c614fcc410 2024-09-16T01:40:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85fa644740 1 -- 172.21.15.139:0/2358262923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85e00c0880 con 0x55c614fcc410 2024-09-16T01:40:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85d8ff9640 1 -- 172.21.15.139:0/2358262923 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85e00edeb0 con 0x55c614fcc410 2024-09-16T01:40:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85fa644740 10 client.15471 fetch_fsmap learned FSMap version 29 2024-09-16T01:40:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.061+0000 7f85fa644740 1 -- 172.21.15.139:0/2358262923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c614d0f620 con 0x55c614fcc410 2024-09-16T01:40:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.062+0000 7f85d8ff9640 1 -- 172.21.15.139:0/2358262923 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e00c0880 con 0x55c614fcc410 2024-09-16T01:40:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.062+0000 7f85d8ff9640 1 -- 172.21.15.139:0/2358262923 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f85e00c4cb0 con 0x55c614fcc410 2024-09-16T01:40:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.062+0000 7f85fa644740 10 client.15471 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:40:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.062+0000 7f85fa644740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:40:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.062+0000 7f85fa644740 20 client.15471 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.062+0000 7f85fa644740 1 --2- 172.21.15.139:0/2358262923 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c614fe6860 0x55c614fe8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.062+0000 7f85fa644740 4 client.15471 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:40:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.062+0000 7f85fa644740 1 -- 172.21.15.139:0/2358262923 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c614f3da20 con 0x55c614fe6860 2024-09-16T01:40:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.062+0000 7f85f556b640 1 --2- 172.21.15.139:0/2358262923 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c614fe6860 0x55c614fe8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.063+0000 7f85d8ff9640 10 client.15471 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.063+0000 7f85f556b640 1 --2- 172.21.15.139:0/2358262923 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c614fe6860 0x55c614fe8c50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cad9bada0d9695e server_cookie=610682e860e03ad4 in_seq=0 out_seq=0 2024-09-16T01:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.063+0000 7f85d8ff9640 1 -- 172.21.15.139:0/2358262923 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c614f3da20 con 0x55c614fe6860 2024-09-16T01:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.063+0000 7f85d8ff9640 10 client.15471 handle_command_reply: tid=0 2024-09-16T01:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.063+0000 7f85fa644740 1 client.15471 shutdown 2024-09-16T01:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.063+0000 7f85fa644740 1 -- 172.21.15.139:0/2358262923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcc410 msgr2=0x55c614fd4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.063+0000 7f85fa644740 1 --2- 172.21.15.139:0/2358262923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcc410 0x55c614fd4180 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009a50 tx=0x7f85e00f0a80 comp rx=0 tx=0).stop 2024-09-16T01:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.063+0000 7f85fa644740 1 -- 172.21.15.139:0/2358262923 shutdown_connections 2024-09-16T01:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.063+0000 7f85fa644740 1 --2- 172.21.15.139:0/2358262923 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c614fe6860 0x55c614fe8c50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.063+0000 7f85fa644740 1 --2- 172.21.15.139:0/2358262923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 0x55c614fd46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.063+0000 7f85d8ff9640 0 client.15471 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.063+0000 7f85fa644740 1 --2- 172.21.15.139:0/2358262923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcc410 0x55c614fd4180 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.063+0000 7f85fa644740 1 --2- 172.21.15.139:0/2358262923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcbaf0 0x55c614fd3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.063+0000 7f85fa644740 1 -- 172.21.15.139:0/2358262923 >> 172.21.15.139:0/2358262923 conn(0x55c614fc8060 msgr2=0x55c614fcdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.064+0000 7f85fa644740 1 -- 172.21.15.139:0/2358262923 shutdown_connections 2024-09-16T01:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.064+0000 7f85fa644740 1 -- 172.21.15.139:0/2358262923 wait complete. 2024-09-16T01:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.064+0000 7f85fa644740 20 client.15471 trim_cache size 0 max 16384 2024-09-16T01:40:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85fa644740 1 Processor -- start 2024-09-16T01:40:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85fa644740 1 -- start start 2024-09-16T01:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85fa644740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcbaf0 0x55c614fce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85fa644740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcc410 0x55c614fd44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85fa644740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 0x55c614fee450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85fa644740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f85f01af000 con 0x55c614fcbaf0 2024-09-16T01:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85fa644740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f85f01b0b40 con 0x55c614fcc410 2024-09-16T01:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85fa644740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f85f01af6d0 con 0x55c614fd6cb0 2024-09-16T01:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85f556b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcbaf0 0x55c614fce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85f556b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcbaf0 0x55c614fce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46544/0 (socket says 172.21.15.139:46544) 2024-09-16T01:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85f556b640 1 -- 172.21.15.139:0/3193737404 learned_addr learned my addr 172.21.15.139:0/3193737404 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85f556b640 1 -- 172.21.15.139:0/3193737404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 msgr2=0x55c614fee450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85f5d6c640 1 --2- 172.21.15.139:0/3193737404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 0x55c614fee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85f556b640 1 --2- 172.21.15.139:0/3193737404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 0x55c614fee450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85f556b640 1 -- 172.21.15.139:0/3193737404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcc410 msgr2=0x55c614fd44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85f556b640 1 --2- 172.21.15.139:0/3193737404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcc410 0x55c614fd44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85f556b640 1 -- 172.21.15.139:0/3193737404 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c614fd9410 con 0x55c614fcbaf0 2024-09-16T01:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85f5d6c640 1 --2- 172.21.15.139:0/3193737404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 0x55c614fee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85f556b640 1 --2- 172.21.15.139:0/3193737404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcbaf0 0x55c614fce560 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f85ec005de0 tx=0x7f85ec00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8092bc85c0e51e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85daffd640 1 -- 172.21.15.139:0/3193737404 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec004220 con 0x55c614fcbaf0 2024-09-16T01:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85daffd640 1 -- 172.21.15.139:0/3193737404 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ec0043c0 con 0x55c614fcbaf0 2024-09-16T01:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.078+0000 7f85daffd640 1 -- 172.21.15.139:0/3193737404 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0035f0 con 0x55c614fcbaf0 2024-09-16T01:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.079+0000 7f85fa644740 1 -- 172.21.15.139:0/3193737404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcbaf0 msgr2=0x55c614fce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.079+0000 7f85fa644740 1 --2- 172.21.15.139:0/3193737404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcbaf0 0x55c614fce560 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f85ec005de0 tx=0x7f85ec00b410 comp rx=0 tx=0).stop 2024-09-16T01:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.079+0000 7f85fa644740 1 -- 172.21.15.139:0/3193737404 shutdown_connections 2024-09-16T01:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.079+0000 7f85fa644740 1 --2- 172.21.15.139:0/3193737404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 0x55c614fee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.079+0000 7f85fa644740 1 --2- 172.21.15.139:0/3193737404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcc410 0x55c614fd44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.079+0000 7f85fa644740 1 --2- 172.21.15.139:0/3193737404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcbaf0 0x55c614fce560 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.079+0000 7f85fa644740 1 -- 172.21.15.139:0/3193737404 >> 172.21.15.139:0/3193737404 conn(0x55c614fc8060 msgr2=0x55c614fd5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.079+0000 7f85fa644740 1 -- 172.21.15.139:0/3193737404 shutdown_connections 2024-09-16T01:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.079+0000 7f85fa644740 1 -- 172.21.15.139:0/3193737404 wait complete. 2024-09-16T01:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.079+0000 7f85fa644740 1 -- start start 2024-09-16T01:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.079+0000 7f85fa644740 1 Processor -- start 2024-09-16T01:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85fa644740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcbaf0 0x55c614fda6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85fa644740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcc410 0x55c614fdac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85fa644740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 0x55c614fdb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85fa644740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f85f01b0470 con 0x55c614fcc410 2024-09-16T01:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85fa644740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f85f01afda0 con 0x55c614fcbaf0 2024-09-16T01:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85fa644740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f85f01acfe0 con 0x55c614fd6cb0 2024-09-16T01:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85fa644740 10 client.0 _ll_register_callbacks cb 0x7f85ec00b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85f4d6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcc410 0x55c614fdac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85f4d6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcc410 0x55c614fdac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46550/0 (socket says 172.21.15.139:46550) 2024-09-16T01:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85f4d6a640 1 -- 172.21.15.139:0/2385653387 learned_addr learned my addr 172.21.15.139:0/2385653387 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85f5d6c640 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 0x55c614fdb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85f556b640 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcbaf0 0x55c614fda6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85f4d6a640 1 -- 172.21.15.139:0/2385653387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 msgr2=0x55c614fdb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85f4d6a640 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 0x55c614fdb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85f4d6a640 1 -- 172.21.15.139:0/2385653387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcbaf0 msgr2=0x55c614fda6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85f4d6a640 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcbaf0 0x55c614fda6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85f4d6a640 1 -- 172.21.15.139:0/2385653387 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c615050420 con 0x55c614fcc410 2024-09-16T01:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85f5d6c640 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 0x55c614fdb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85f556b640 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcbaf0 0x55c614fda6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85f4d6a640 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcc410 0x55c614fdac30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f85e00edad0 tx=0x7f85e0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eeebde89c328854 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.080+0000 7f85d8ff9640 1 -- 172.21.15.139:0/2385653387 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00d4070 con 0x55c614fcc410 2024-09-16T01:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.081+0000 7f85d8ff9640 1 -- 172.21.15.139:0/2385653387 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f85e0003c80 con 0x55c614fcc410 2024-09-16T01:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.081+0000 7f85fa644740 1 -- 172.21.15.139:0/2385653387 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85ec0043c0 con 0x55c614fcc410 2024-09-16T01:40:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.081+0000 7f85d8ff9640 1 -- 172.21.15.139:0/2385653387 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85e00b8820 con 0x55c614fcc410 2024-09-16T01:40:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.081+0000 7f85fa644740 10 client.15477 fetch_fsmap learned FSMap version 29 2024-09-16T01:40:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.081+0000 7f85fa644740 1 -- 172.21.15.139:0/2385653387 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c615050420 con 0x55c614fcc410 2024-09-16T01:40:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.081+0000 7f85d8ff9640 1 -- 172.21.15.139:0/2385653387 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ec0043c0 con 0x55c614fcc410 2024-09-16T01:40:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.081+0000 7f85d8ff9640 1 -- 172.21.15.139:0/2385653387 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f85e0031bd0 con 0x55c614fcc410 2024-09-16T01:40:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.081+0000 7f85fa644740 10 client.15477 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:40:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.081+0000 7f85fa644740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:40:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.081+0000 7f85fa644740 20 client.15477 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.081+0000 7f85fa644740 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c615051a30 0x55c615053e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.081+0000 7f85fa644740 4 client.15477 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:40:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.081+0000 7f85fa644740 1 -- 172.21.15.139:0/2385653387 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c614f37d70 con 0x55c615051a30 2024-09-16T01:40:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.081+0000 7f85f556b640 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c615051a30 0x55c615053e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.082+0000 7f85f556b640 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c615051a30 0x55c615053e50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a2368f02cefc37c server_cookie=7977c1b664c02718 in_seq=0 out_seq=0 2024-09-16T01:40:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.082+0000 7f85d8ff9640 10 client.15477 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85d8ff9640 1 -- 172.21.15.139:0/2385653387 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c614f37d70 con 0x55c615051a30 2024-09-16T01:40:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85d8ff9640 10 client.15477 handle_command_reply: tid=0 2024-09-16T01:40:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85fa644740 1 client.15477 shutdown 2024-09-16T01:40:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85fa644740 1 -- 172.21.15.139:0/2385653387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcc410 msgr2=0x55c614fdac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85fa644740 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcc410 0x55c614fdac30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f85e00edad0 tx=0x7f85e0005bd0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85fa644740 1 -- 172.21.15.139:0/2385653387 shutdown_connections 2024-09-16T01:40:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85fa644740 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c615051a30 0x55c615053e50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85fa644740 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c614fd6cb0 0x55c614fdb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85fa644740 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c614fcc410 0x55c614fdac30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85fa644740 1 --2- 172.21.15.139:0/2385653387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c614fcbaf0 0x55c614fda6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85d8ff9640 0 client.15477 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85fa644740 1 -- 172.21.15.139:0/2385653387 >> 172.21.15.139:0/2385653387 conn(0x55c614fc8060 msgr2=0x55c614fc69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85fa644740 1 -- 172.21.15.139:0/2385653387 shutdown_connections 2024-09-16T01:40:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85fa644740 1 -- 172.21.15.139:0/2385653387 wait complete. 2024-09-16T01:40:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.096+0000 7f85fa644740 20 client.15477 trim_cache size 0 max 16384 2024-09-16T01:40:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:40:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:40:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:40:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:40:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.097+0000 7f85f77cf640 1 -- 172.21.15.139:0/1025400303 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f85dc0a3500 msgr2=0x7f85dc0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.097+0000 7f85f77cf640 1 --2- 172.21.15.139:0/1025400303 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f85dc0a3500 0x7f85dc0a59c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009fd0 tx=0x7f85e8009600 comp rx=0 tx=0).stop 2024-09-16T01:40:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.097+0000 7f85f77cf640 1 -- 172.21.15.139:0/1025400303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f0101350 msgr2=0x7f85f0198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.097+0000 7f85f77cf640 1 --2- 172.21.15.139:0/1025400303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f0101350 0x7f85f0198880 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009230 tx=0x7f85e0004270 comp rx=0 tx=0).stop 2024-09-16T01:40:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.097+0000 7f85f77cf640 1 -- 172.21.15.139:0/1025400303 shutdown_connections 2024-09-16T01:40:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.097+0000 7f85f77cf640 1 --2- 172.21.15.139:0/1025400303 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f85dc0a3500 0x7f85dc0a59c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.097+0000 7f85f77cf640 1 --2- 172.21.15.139:0/1025400303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85f01069c0 0x7f85f019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.097+0000 7f85f77cf640 1 --2- 172.21.15.139:0/1025400303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f0101350 0x7f85f0198880 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.097+0000 7f85f77cf640 1 --2- 172.21.15.139:0/1025400303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85f01009b0 0x7f85f0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.097+0000 7f85f77cf640 1 -- 172.21.15.139:0/1025400303 >> 172.21.15.139:0/1025400303 conn(0x7f85f0077990 msgr2=0x7f85f0103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.097+0000 7f85f77cf640 1 -- 172.21.15.139:0/1025400303 shutdown_connections 2024-09-16T01:40:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:39.097+0000 7f85f77cf640 1 -- 172.21.15.139:0/1025400303 wait complete. 2024-09-16T01:40:39.688 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T01:38:02.129081+0000', 'modified': '2024-09-16T01:40:22.317639+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24470, 'mds_1': 24454, 'mds_2': 24443, 'mds_3': 24442, 'mds_4': 24433}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14604': {'gid': 14604, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6839/1075273450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 1075273450}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 1075273450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14616': {'gid': 14616, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6841/1887551794', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6840', 'nonce': 1887551794}, {'type': 'v1', 'addr': '172.21.15.139:6841', 'nonce': 1887551794}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24395': {'gid': 24395, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6833/416925170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 416925170}, {'type': 'v1', 'addr': '172.21.15.164:6833', 'nonce': 416925170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.176:6833/1026776933', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6832', 'nonce': 1026776933}, {'type': 'v1', 'addr': '172.21.15.176:6833', 'nonce': 1026776933}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6835/3662238682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 3662238682}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 3662238682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.176:6835/3060423566', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6834', 'nonce': 3060423566}, {'type': 'v1', 'addr': '172.21.15.176:6835', 'nonce': 3060423566}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.176:6837/1942543693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6836', 'nonce': 1942543693}, {'type': 'v1', 'addr': '172.21.15.176:6837', 'nonce': 1942543693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.164:6837/1270226509', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 1270226509}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 1270226509}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.176:6839/3912805068', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6838', 'nonce': 3912805068}, {'type': 'v1', 'addr': '172.21.15.176:6839', 'nonce': 3912805068}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.164:6839/2447189436', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 2447189436}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 2447189436}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24470, 'qdb_cluster': [24433, 24442, 24443, 24454, 24470]} 2024-09-16T01:40:39.688 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-16T01:40:39.688 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-16T01:40:39.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:39 smithi139 ceph-mon[31175]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-16T01:40:39.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:39 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1737423236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:39.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:39 smithi176 ceph-mon[35324]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-16T01:40:39.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:39 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1737423236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:39.975 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:40.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:39 smithi164 ceph-mon[35121]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-16T01:40:40.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:39 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1737423236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:40:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.426+0000 7fbe8b2ca640 1 -- 172.21.15.139:0/466901581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe84101de0 msgr2=0x7fbe841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.426+0000 7fbe8b2ca640 1 --2- 172.21.15.139:0/466901581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe84101de0 0x7fbe841021c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009900 tx=0x7fbe8002a3f0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.426+0000 7fbe8b2ca640 1 -- 172.21.15.139:0/466901581 shutdown_connections 2024-09-16T01:40:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.426+0000 7fbe8b2ca640 1 --2- 172.21.15.139:0/466901581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe84106f10 0x7fbe8410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.426+0000 7fbe8b2ca640 1 --2- 172.21.15.139:0/466901581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe84102790 0x7fbe841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.426+0000 7fbe8b2ca640 1 --2- 172.21.15.139:0/466901581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe84101de0 0x7fbe841021c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.426+0000 7fbe8b2ca640 1 -- 172.21.15.139:0/466901581 >> 172.21.15.139:0/466901581 conn(0x7fbe840fd8c0 msgr2=0x7fbe840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.426+0000 7fbe8b2ca640 1 -- 172.21.15.139:0/466901581 shutdown_connections 2024-09-16T01:40:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.426+0000 7fbe8b2ca640 1 -- 172.21.15.139:0/466901581 wait complete. 2024-09-16T01:40:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe8b2ca640 1 Processor -- start 2024-09-16T01:40:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe8b2ca640 1 -- start start 2024-09-16T01:40:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe8b2ca640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe84101de0 0x7fbe84198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe8b2ca640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe84102790 0x7fbe84198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe8b2ca640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe84106f10 0x7fbe8419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe8b2ca640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbe84111f90 con 0x7fbe84106f10 2024-09-16T01:40:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe8b2ca640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbe84111e10 con 0x7fbe84102790 2024-09-16T01:40:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe8b2ca640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbe84112110 con 0x7fbe84101de0 2024-09-16T01:40:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe89867640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe84106f10 0x7fbe8419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe89867640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe84106f10 0x7fbe8419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46572/0 (socket says 172.21.15.139:46572) 2024-09-16T01:40:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe89867640 1 -- 172.21.15.139:0/3842428244 learned_addr learned my addr 172.21.15.139:0/3842428244 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe89066640 1 --2- 172.21.15.139:0/3842428244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe84101de0 0x7fbe84198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe88865640 1 --2- 172.21.15.139:0/3842428244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe84102790 0x7fbe84198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe89867640 1 -- 172.21.15.139:0/3842428244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe84101de0 msgr2=0x7fbe84198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe89867640 1 --2- 172.21.15.139:0/3842428244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe84101de0 0x7fbe84198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe89867640 1 -- 172.21.15.139:0/3842428244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe84102790 msgr2=0x7fbe84198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe89867640 1 --2- 172.21.15.139:0/3842428244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe84102790 0x7fbe84198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.427+0000 7fbe89867640 1 -- 172.21.15.139:0/3842428244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8419d400 con 0x7fbe84106f10 2024-09-16T01:40:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.428+0000 7fbe89867640 1 --2- 172.21.15.139:0/3842428244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe84106f10 0x7fbe8419cd20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00cce0 tx=0x7fbe7c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.428+0000 7fbe7a7fc640 1 -- 172.21.15.139:0/3842428244 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c00f450 con 0x7fbe84106f10 2024-09-16T01:40:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.428+0000 7fbe7a7fc640 1 -- 172.21.15.139:0/3842428244 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe7c00fad0 con 0x7fbe84106f10 2024-09-16T01:40:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.428+0000 7fbe8b2ca640 1 -- 172.21.15.139:0/3842428244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe8419d6f0 con 0x7fbe84106f10 2024-09-16T01:40:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.428+0000 7fbe7a7fc640 1 -- 172.21.15.139:0/3842428244 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c018b60 con 0x7fbe84106f10 2024-09-16T01:40:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.428+0000 7fbe8b2ca640 1 -- 172.21.15.139:0/3842428244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe841a4fd0 con 0x7fbe84106f10 2024-09-16T01:40:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.430+0000 7fbe7a7fc640 1 -- 172.21.15.139:0/3842428244 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 23) ==== 138532+0+0 (secure 0 0 0) 0x7fbe7c00fc80 con 0x7fbe84106f10 2024-09-16T01:40:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe8d6a3740 1 Processor -- start 2024-09-16T01:40:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe8d6a3740 1 -- start start 2024-09-16T01:40:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe8d6a3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f2d8b0 0x55f4f9f2dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe8d6a3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f3ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe8d6a3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f3f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe8d6a3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f4f9e94090 con 0x55f4f9f2e1d0 2024-09-16T01:40:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe8d6a3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f4f9e3ba70 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe8d6a3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f4f9e90050 con 0x55f4f9f2d8b0 2024-09-16T01:40:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe7a7fc640 1 --2- 172.21.15.139:0/3842428244 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbe700a34c0 0x7fbe700a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe89066640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f3ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe89066640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f3ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46580/0 (socket says 172.21.15.139:46580) 2024-09-16T01:40:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe88865640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f2d8b0 0x55f4f9f2dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe89066640 1 -- 172.21.15.139:0/3372948530 learned_addr learned my addr 172.21.15.139:0/3372948530 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe88865640 1 --2- 172.21.15.139:0/3372948530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f3f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe7a7fc640 1 -- 172.21.15.139:0/3842428244 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fbe7c0d0660 con 0x7fbe84106f10 2024-09-16T01:40:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe89867640 1 --2- 172.21.15.139:0/3842428244 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbe700a34c0 0x7fbe700a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe89066640 1 -- 172.21.15.139:0/3372948530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f2d8b0 msgr2=0x55f4f9f2dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe89066640 1 --2- 172.21.15.139:0/3372948530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f2d8b0 0x55f4f9f2dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe89066640 1 -- 172.21.15.139:0/3372948530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f3cfc0 msgr2=0x55f4f9f3f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe89066640 1 --2- 172.21.15.139:0/3372948530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f3f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.431+0000 7fbe89066640 1 -- 172.21.15.139:0/3372948530 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f9e48130 con 0x55f4f9f2e1d0 2024-09-16T01:40:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.432+0000 7fbe89066640 1 --2- 172.21.15.139:0/3372948530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f3ca80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009900 tx=0x7fbe800316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc84f05d25748e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.432+0000 7fbe89867640 1 --2- 172.21.15.139:0/3842428244 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbe700a34c0 0x7fbe700a5980 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c0cd9a0 tx=0x7fbe7c018400 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.432+0000 7fbe6f7fe640 1 -- 172.21.15.139:0/3372948530 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80040070 con 0x55f4f9f2e1d0 2024-09-16T01:40:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.432+0000 7fbe6f7fe640 1 -- 172.21.15.139:0/3372948530 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe80005020 con 0x55f4f9f2e1d0 2024-09-16T01:40:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.432+0000 7fbe6f7fe640 1 -- 172.21.15.139:0/3372948530 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80035040 con 0x55f4f9f2e1d0 2024-09-16T01:40:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.432+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/3372948530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f2e1d0 msgr2=0x55f4f9f3ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.432+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/3372948530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f3ca80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009900 tx=0x7fbe800316e0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.433+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/3372948530 shutdown_connections 2024-09-16T01:40:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.433+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/3372948530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f3f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.433+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/3372948530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f3ca80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.433+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/3372948530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f2d8b0 0x55f4f9f2dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.433+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/3372948530 >> 172.21.15.139:0/3372948530 conn(0x55f4f9f29f60 msgr2=0x55f4f9f2a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.433+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/3372948530 shutdown_connections 2024-09-16T01:40:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.433+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/3372948530 wait complete. 2024-09-16T01:40:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.433+0000 7fbe8d6a3740 1 -- start start 2024-09-16T01:40:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.433+0000 7fbe8d6a3740 1 Processor -- start 2024-09-16T01:40:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe8d6a3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f2d8b0 0x55f4f9f3c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe8d6a3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f39b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe8d6a3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f3a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe8d6a3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f4f9f3fb90 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe8d6a3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f4f9e4ef20 con 0x55f4f9f2e1d0 2024-09-16T01:40:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe8d6a3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f4f9f3fd10 con 0x55f4f9f2d8b0 2024-09-16T01:40:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe8d6a3740 10 client.0 _ll_register_callbacks cb 0x55f4f9e18030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe88865640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f3a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe88865640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f3a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46590/0 (socket says 172.21.15.139:46590) 2024-09-16T01:40:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe88865640 1 -- 172.21.15.139:0/2677274425 learned_addr learned my addr 172.21.15.139:0/2677274425 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe89066640 1 --2- 172.21.15.139:0/2677274425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f39b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe88865640 1 -- 172.21.15.139:0/2677274425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f2d8b0 msgr2=0x55f4f9f3c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe88865640 1 --2- 172.21.15.139:0/2677274425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f2d8b0 0x55f4f9f3c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe88865640 1 -- 172.21.15.139:0/2677274425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f2e1d0 msgr2=0x55f4f9f39b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe88865640 1 --2- 172.21.15.139:0/2677274425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f39b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe88865640 1 -- 172.21.15.139:0/2677274425 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f9c052b0 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe88865640 1 --2- 172.21.15.139:0/2677274425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f3a090 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbe74006cd0 tx=0x7fbe74017df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16e945f5fe78be4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe6cff9640 1 -- 172.21.15.139:0/2677274425 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe74015a30 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe6cff9640 1 -- 172.21.15.139:0/2677274425 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fbe74015bd0 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2677274425 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe80005020 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe6cff9640 1 -- 172.21.15.139:0/2677274425 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbe74015dd0 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe8d6a3740 10 client.15489 fetch_fsmap learned FSMap version 29 2024-09-16T01:40:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.434+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2677274425 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f4f9f4b4f0 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.435+0000 7fbe6cff9640 1 -- 172.21.15.139:0/2677274425 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe80005020 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.435+0000 7fbe6cff9640 1 -- 172.21.15.139:0/2677274425 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbe740065b0 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.435+0000 7fbe8d6a3740 10 client.15489 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:40:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.435+0000 7fbe8d6a3740 10 client.15489 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T01:40:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.435+0000 7fbe8d6a3740 20 client.15489 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.435+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2677274425 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55f4f9f4dae0 0x55f4f9f4ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.435+0000 7fbe8d6a3740 4 client.15489 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:40:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.435+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2677274425 --> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4f9ea1600 con 0x55f4f9f4dae0 2024-09-16T01:40:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.435+0000 7fbe89066640 1 --2- 172.21.15.139:0/2677274425 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55f4f9f4dae0 0x55f4f9f4ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.436+0000 7fbe6cff9640 10 client.15489 ms_handle_connect on v2:172.21.15.176:6834/3060423566 2024-09-16T01:40:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.436+0000 7fbe89066640 1 --2- 172.21.15.139:0/2677274425 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55f4f9f4dae0 0x55f4f9f4ef50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4ce038a575ce65cb server_cookie=9d2d174581f3b75b in_seq=0 out_seq=0 2024-09-16T01:40:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.436+0000 7fbe6cff9640 1 -- 172.21.15.139:0/2677274425 <== mds.4 v2:172.21.15.176:6834/3060423566 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4f9ea1600 con 0x55f4f9f4dae0 2024-09-16T01:40:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.436+0000 7fbe6cff9640 10 client.15489 handle_command_reply: tid=0 2024-09-16T01:40:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.436+0000 7fbe8d6a3740 1 client.15489 shutdown 2024-09-16T01:40:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.437+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2677274425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f3cfc0 msgr2=0x55f4f9f3a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.437+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2677274425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f3a090 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbe74006cd0 tx=0x7fbe74017df0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.437+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2677274425 shutdown_connections 2024-09-16T01:40:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.437+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2677274425 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55f4f9f4dae0 0x55f4f9f4ef50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.437+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2677274425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f3a090 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.437+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2677274425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f39b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.437+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2677274425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f2d8b0 0x55f4f9f3c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.437+0000 7fbe6cff9640 0 client.15489 ms_handle_reset on v2:172.21.15.176:6834/3060423566 2024-09-16T01:40:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.437+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2677274425 >> 172.21.15.139:0/2677274425 conn(0x55f4f9f29f60 msgr2=0x55f4f9f2fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.437+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2677274425 shutdown_connections 2024-09-16T01:40:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.437+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2677274425 wait complete. 2024-09-16T01:40:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.437+0000 7fbe8d6a3740 20 client.15489 trim_cache size 0 max 16384 2024-09-16T01:40:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.454+0000 7fbe8d6a3740 1 Processor -- start 2024-09-16T01:40:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe8d6a3740 1 -- start start 2024-09-16T01:40:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe8d6a3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f2d8b0 0x55f4f9f4ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe8d6a3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f57230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe8d6a3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f57770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe8d6a3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f4f9f4d290 con 0x55f4f9f2e1d0 2024-09-16T01:40:40.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe8d6a3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f4f9f4d960 con 0x55f4f9f2d8b0 2024-09-16T01:40:40.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe8d6a3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f4f9f4e030 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe89066640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f57230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe89066640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f57230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46596/0 (socket says 172.21.15.139:46596) 2024-09-16T01:40:40.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe89066640 1 -- 172.21.15.139:0/2614368218 learned_addr learned my addr 172.21.15.139:0/2614368218 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:40.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe89066640 1 -- 172.21.15.139:0/2614368218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f3cfc0 msgr2=0x55f4f9f57770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:40.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe89066640 1 --2- 172.21.15.139:0/2614368218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f57770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe89066640 1 -- 172.21.15.139:0/2614368218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f2d8b0 msgr2=0x55f4f9f4ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:40.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe89066640 1 --2- 172.21.15.139:0/2614368218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f2d8b0 0x55f4f9f4ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe89066640 1 -- 172.21.15.139:0/2614368218 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f9f4e4e0 con 0x55f4f9f2e1d0 2024-09-16T01:40:40.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe89066640 1 --2- 172.21.15.139:0/2614368218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f57230 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55f4f9f305c0 tx=0x7fbe8002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762d1025b84bde73 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:40.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe6effd640 1 -- 172.21.15.139:0/2614368218 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80040050 con 0x55f4f9f2e1d0 2024-09-16T01:40:40.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe6effd640 1 -- 172.21.15.139:0/2614368218 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe80004240 con 0x55f4f9f2e1d0 2024-09-16T01:40:40.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.455+0000 7fbe6effd640 1 -- 172.21.15.139:0/2614368218 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80035040 con 0x55f4f9f2e1d0 2024-09-16T01:40:40.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.456+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2614368218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f2e1d0 msgr2=0x55f4f9f57230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:40.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.456+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2614368218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f57230 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55f4f9f305c0 tx=0x7fbe8002a3f0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.456+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2614368218 shutdown_connections 2024-09-16T01:40:40.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.456+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2614368218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f57770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.456+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2614368218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f57230 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.456+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2614368218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f2d8b0 0x55f4f9f4ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.456+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2614368218 >> 172.21.15.139:0/2614368218 conn(0x55f4f9f29f60 msgr2=0x55f4f9f38420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:40.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.456+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2614368218 shutdown_connections 2024-09-16T01:40:40.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.456+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2614368218 wait complete. 2024-09-16T01:40:40.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.456+0000 7fbe8d6a3740 1 -- start start 2024-09-16T01:40:40.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe8d6a3740 1 Processor -- start 2024-09-16T01:40:40.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe8d6a3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f2d8b0 0x55f4f9f57360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe8d6a3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f578a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe8d6a3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f50760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe8d6a3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f4f9f4edd0 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe8d6a3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f4f9f4e700 con 0x55f4f9f2e1d0 2024-09-16T01:40:40.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe8d6a3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f4f9f4f4a0 con 0x55f4f9f2d8b0 2024-09-16T01:40:40.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe8d6a3740 10 client.0 _ll_register_callbacks cb 0x7fbe74009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:40.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe88865640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f50760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe88865640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f50760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46610/0 (socket says 172.21.15.139:46610) 2024-09-16T01:40:40.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe88865640 1 -- 172.21.15.139:0/2453052643 learned_addr learned my addr 172.21.15.139:0/2453052643 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:40.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe89066640 1 --2- 172.21.15.139:0/2453052643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe88865640 1 --2- 172.21.15.139:0/2453052643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f2d8b0 0x55f4f9f57360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe88865640 1 -- 172.21.15.139:0/2453052643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f2d8b0 msgr2=0x55f4f9f57360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:40.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe88865640 1 --2- 172.21.15.139:0/2453052643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f2d8b0 0x55f4f9f57360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe88865640 1 -- 172.21.15.139:0/2453052643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f2e1d0 msgr2=0x55f4f9f578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:40.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe88865640 1 --2- 172.21.15.139:0/2453052643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.457+0000 7fbe88865640 1 -- 172.21.15.139:0/2453052643 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f9f50f10 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe88865640 1 --2- 172.21.15.139:0/2453052643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f50760 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbe74015da0 tx=0x7fbe74008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fd0f32a5ef8f07 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:40.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:40.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe6cff9640 1 -- 172.21.15.139:0/2453052643 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe740093a0 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe6cff9640 1 -- 172.21.15.139:0/2453052643 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fbe740059c0 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2453052643 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe80004240 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe6cff9640 1 -- 172.21.15.139:0/2453052643 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbe74006a00 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe8d6a3740 10 client.15495 fetch_fsmap learned FSMap version 29 2024-09-16T01:40:40.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2453052643 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f4f9f55860 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe6cff9640 1 -- 172.21.15.139:0/2453052643 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe80004240 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe6cff9640 1 -- 172.21.15.139:0/2453052643 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbe740064a0 con 0x55f4f9f3cfc0 2024-09-16T01:40:40.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe8d6a3740 10 client.15495 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:40:40.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe8d6a3740 10 client.15495 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T01:40:40.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe8d6a3740 20 client.15495 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:40.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2453052643 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55f4f9fbc690 0x55f4f9fbb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:40.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe8d6a3740 4 client.15495 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:40:40.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.458+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2453052643 --> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] -- command(tid 0: {"prefix": "status"}) -- 0x55f4f9e9bb10 con 0x55f4f9fbc690 2024-09-16T01:40:40.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.459+0000 7fbe89066640 1 --2- 172.21.15.139:0/2453052643 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55f4f9fbc690 0x55f4f9fbb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:40.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.459+0000 7fbe89066640 1 --2- 172.21.15.139:0/2453052643 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55f4f9fbc690 0x55f4f9fbb3a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5bca4c448d827e0d server_cookie=2d0250ed321663a in_seq=0 out_seq=0 2024-09-16T01:40:40.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.459+0000 7fbe6cff9640 10 client.15495 ms_handle_connect on v2:172.21.15.176:6834/3060423566 2024-09-16T01:40:40.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.459+0000 7fbe6cff9640 1 -- 172.21.15.139:0/2453052643 <== mds.4 v2:172.21.15.176:6834/3060423566 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f4f9e9bb10 con 0x55f4f9fbc690 2024-09-16T01:40:40.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.459+0000 7fbe6cff9640 10 client.15495 handle_command_reply: tid=0 2024-09-16T01:40:40.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.459+0000 7fbe8d6a3740 1 client.15495 shutdown 2024-09-16T01:40:40.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2453052643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f3cfc0 msgr2=0x55f4f9f50760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:40.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2453052643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f50760 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbe74015da0 tx=0x7fbe74008040 comp rx=0 tx=0).stop 2024-09-16T01:40:40.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2453052643 shutdown_connections 2024-09-16T01:40:40.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2453052643 >> [v2:172.21.15.176:6834/3060423566,v1:172.21.15.176:6835/3060423566] conn(0x55f4f9fbc690 0x55f4f9fbb3a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe6cff9640 0 client.15495 ms_handle_reset on v2:172.21.15.176:6834/3060423566 2024-09-16T01:40:40.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2453052643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4f9f3cfc0 0x55f4f9f50760 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2453052643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4f9f2e1d0 0x55f4f9f578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8d6a3740 1 --2- 172.21.15.139:0/2453052643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f9f2d8b0 0x55f4f9f57360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2453052643 >> 172.21.15.139:0/2453052643 conn(0x55f4f9f29f60 msgr2=0x55f4f9f34a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:40.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2453052643 shutdown_connections 2024-09-16T01:40:40.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8d6a3740 1 -- 172.21.15.139:0/2453052643 wait complete. 2024-09-16T01:40:40.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8d6a3740 20 client.15495 trim_cache size 0 max 16384 2024-09-16T01:40:40.478 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:40:40.478 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:40:40.479 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 4, 2024-09-16T01:40:40.479 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24433, 2024-09-16T01:40:40.479 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:40:40.479 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:40:40.479 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:40:40.479 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 120.909454328, 2024-09-16T01:40:40.479 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 29, 2024-09-16T01:40:40.479 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 86, 2024-09-16T01:40:40.479 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 85, 2024-09-16T01:40:40.479 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 187.12625068200001 2024-09-16T01:40:40.480 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:40:40.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8b2ca640 1 -- 172.21.15.139:0/3842428244 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbe700a34c0 msgr2=0x7fbe700a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:40.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8b2ca640 1 --2- 172.21.15.139:0/3842428244 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbe700a34c0 0x7fbe700a5980 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c0cd9a0 tx=0x7fbe7c018400 comp rx=0 tx=0).stop 2024-09-16T01:40:40.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8b2ca640 1 -- 172.21.15.139:0/3842428244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe84106f10 msgr2=0x7fbe8419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:40.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.460+0000 7fbe8b2ca640 1 --2- 172.21.15.139:0/3842428244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe84106f10 0x7fbe8419cd20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00cce0 tx=0x7fbe7c00a620 comp rx=0 tx=0).stop 2024-09-16T01:40:40.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.461+0000 7fbe8b2ca640 1 -- 172.21.15.139:0/3842428244 shutdown_connections 2024-09-16T01:40:40.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.461+0000 7fbe8b2ca640 1 --2- 172.21.15.139:0/3842428244 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbe700a34c0 0x7fbe700a5980 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.461+0000 7fbe8b2ca640 1 --2- 172.21.15.139:0/3842428244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe84106f10 0x7fbe8419cd20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.461+0000 7fbe8b2ca640 1 --2- 172.21.15.139:0/3842428244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe84102790 0x7fbe84198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.461+0000 7fbe8b2ca640 1 --2- 172.21.15.139:0/3842428244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe84101de0 0x7fbe84198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:40.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.461+0000 7fbe8b2ca640 1 -- 172.21.15.139:0/3842428244 >> 172.21.15.139:0/3842428244 conn(0x7fbe840fd8c0 msgr2=0x7fbe840fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:40.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.461+0000 7fbe8b2ca640 1 -- 172.21.15.139:0/3842428244 shutdown_connections 2024-09-16T01:40:40.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:40.461+0000 7fbe8b2ca640 1 -- 172.21.15.139:0/3842428244 wait complete. 2024-09-16T01:40:41.180 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-16T01:40:41.425 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:41.864 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:41 smithi139 ceph-mon[31175]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-09-16T01:40:41.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:41 smithi176 ceph-mon[35324]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-09-16T01:40:41.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.944+0000 7f3d003ac640 1 -- 172.21.15.139:0/3600322231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cf8101df0 msgr2=0x7f3cf81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:41.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.944+0000 7f3d003ac640 1 --2- 172.21.15.139:0/3600322231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cf8101df0 0x7f3cf81021d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009900 tx=0x7f3cf402a410 comp rx=0 tx=0).stop 2024-09-16T01:40:41.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.944+0000 7f3d003ac640 1 -- 172.21.15.139:0/3600322231 shutdown_connections 2024-09-16T01:40:41.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.944+0000 7f3d003ac640 1 --2- 172.21.15.139:0/3600322231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cf8106f20 0x7f3cf810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.944+0000 7f3d003ac640 1 --2- 172.21.15.139:0/3600322231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cf81027a0 0x7f3cf81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.944+0000 7f3d003ac640 1 --2- 172.21.15.139:0/3600322231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cf8101df0 0x7f3cf81021d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.944+0000 7f3d003ac640 1 -- 172.21.15.139:0/3600322231 >> 172.21.15.139:0/3600322231 conn(0x7f3cf80fd8b0 msgr2=0x7f3cf80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:41.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.944+0000 7f3d003ac640 1 -- 172.21.15.139:0/3600322231 shutdown_connections 2024-09-16T01:40:41.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.944+0000 7f3d003ac640 1 -- 172.21.15.139:0/3600322231 wait complete. 2024-09-16T01:40:41.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3d003ac640 1 Processor -- start 2024-09-16T01:40:41.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3d003ac640 1 -- start start 2024-09-16T01:40:41.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3d003ac640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cf8101df0 0x7f3cf8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3d003ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cf81027a0 0x7f3cf8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3d003ac640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cf8106f20 0x7f3cf819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3d003ac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf8111f80 con 0x7f3cf81027a0 2024-09-16T01:40:41.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3d003ac640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf8111e00 con 0x7f3cf8101df0 2024-09-16T01:40:41.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3d003ac640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf8112100 con 0x7f3cf8106f20 2024-09-16T01:40:41.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3cfd947640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cf81027a0 0x7f3cf8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3cfd947640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cf81027a0 0x7f3cf8198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46616/0 (socket says 172.21.15.139:46616) 2024-09-16T01:40:41.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3cfd947640 1 -- 172.21.15.139:0/2777624039 learned_addr learned my addr 172.21.15.139:0/2777624039 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:41.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3cfe148640 1 --2- 172.21.15.139:0/2777624039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cf8101df0 0x7f3cf8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3cfe949640 1 --2- 172.21.15.139:0/2777624039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cf8106f20 0x7f3cf819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3cfd947640 1 -- 172.21.15.139:0/2777624039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cf8106f20 msgr2=0x7f3cf819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:41.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3cfd947640 1 --2- 172.21.15.139:0/2777624039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cf8106f20 0x7f3cf819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3cfd947640 1 -- 172.21.15.139:0/2777624039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cf8101df0 msgr2=0x7f3cf8198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:41.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3cfd947640 1 --2- 172.21.15.139:0/2777624039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cf8101df0 0x7f3cf8198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.945+0000 7f3cfd947640 1 -- 172.21.15.139:0/2777624039 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf819d4d0 con 0x7f3cf81027a0 2024-09-16T01:40:41.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.946+0000 7f3cfd947640 1 --2- 172.21.15.139:0/2777624039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cf81027a0 0x7f3cf8198a60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800c970 tx=0x7f3ce800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:41.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.946+0000 7f3cef7fe640 1 -- 172.21.15.139:0/2777624039 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8007bb0 con 0x7f3cf81027a0 2024-09-16T01:40:41.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.946+0000 7f3cef7fe640 1 -- 172.21.15.139:0/2777624039 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce8007d50 con 0x7f3cf81027a0 2024-09-16T01:40:41.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.946+0000 7f3d003ac640 1 -- 172.21.15.139:0/2777624039 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf819d7c0 con 0x7f3cf81027a0 2024-09-16T01:40:41.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.946+0000 7f3cef7fe640 1 -- 172.21.15.139:0/2777624039 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8018800 con 0x7f3cf81027a0 2024-09-16T01:40:41.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.946+0000 7f3d003ac640 1 -- 172.21.15.139:0/2777624039 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf81a50a0 con 0x7f3cf81027a0 2024-09-16T01:40:41.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.949+0000 7f3cef7fe640 1 -- 172.21.15.139:0/2777624039 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 23) ==== 138532+0+0 (secure 0 0 0) 0x7f3ce800f450 con 0x7f3cf81027a0 2024-09-16T01:40:41.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.949+0000 7f3d02785740 1 Processor -- start 2024-09-16T01:40:41.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.949+0000 7f3d02785740 1 -- start start 2024-09-16T01:40:41.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.949+0000 7f3d02785740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f579d0 0x564d13f57db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.949+0000 7f3d02785740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f582f0 0x564d13f62670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.949+0000 7f3d02785740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13f64fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.949+0000 7f3d02785740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564d13ebe090 con 0x564d13f62bb0 2024-09-16T01:40:41.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.949+0000 7f3d02785740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564d13e65a70 con 0x564d13f579d0 2024-09-16T01:40:41.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.949+0000 7f3d02785740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564d13eba050 con 0x564d13f582f0 2024-09-16T01:40:41.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.949+0000 7f3cfd947640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13f64fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.949+0000 7f3cfd947640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13f64fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46626/0 (socket says 172.21.15.139:46626) 2024-09-16T01:40:41.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.949+0000 7f3cfd947640 1 -- 172.21.15.139:0/3593783055 learned_addr learned my addr 172.21.15.139:0/3593783055 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:41.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cfe148640 1 --2- 172.21.15.139:0/3593783055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f582f0 0x564d13f62670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cfe949640 1 --2- 172.21.15.139:0/3593783055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f579d0 0x564d13f57db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cef7fe640 1 --2- 172.21.15.139:0/2777624039 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f3ce40a34c0 0x7f3ce40a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cfd947640 1 -- 172.21.15.139:0/3593783055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f582f0 msgr2=0x564d13f62670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:41.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cfd947640 1 --2- 172.21.15.139:0/3593783055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f582f0 0x564d13f62670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cfd947640 1 -- 172.21.15.139:0/3593783055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f579d0 msgr2=0x564d13f57db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:41.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cfd947640 1 --2- 172.21.15.139:0/3593783055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f579d0 0x564d13f57db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cfd947640 1 -- 172.21.15.139:0/3593783055 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d13e72130 con 0x564d13f62bb0 2024-09-16T01:40:41.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cfe949640 1 --2- 172.21.15.139:0/2777624039 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f3ce40a34c0 0x7f3ce40a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cfd947640 1 --2- 172.21.15.139:0/3593783055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13f64fa0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80cd4c0 tx=0x7f3ce80ab870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53db2ed0e651d807 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:41.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cef7fe640 1 -- 172.21.15.139:0/2777624039 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f3ce80d02d0 con 0x7f3cf81027a0 2024-09-16T01:40:41.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cecdea640 1 -- 172.21.15.139:0/3593783055 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce809be00 con 0x564d13f62bb0 2024-09-16T01:40:41.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cecdea640 1 -- 172.21.15.139:0/3593783055 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce809c730 con 0x564d13f62bb0 2024-09-16T01:40:41.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cecdea640 1 -- 172.21.15.139:0/3593783055 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce809b990 con 0x564d13f62bb0 2024-09-16T01:40:41.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.950+0000 7f3cfe949640 1 --2- 172.21.15.139:0/2777624039 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f3ce40a34c0 0x7f3ce40a5980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0005fd0 tx=0x7f3cf0004a40 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:41.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.951+0000 7f3d02785740 1 -- 172.21.15.139:0/3593783055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 msgr2=0x564d13f64fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:41.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.951+0000 7f3d02785740 1 --2- 172.21.15.139:0/3593783055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13f64fa0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80cd4c0 tx=0x7f3ce80ab870 comp rx=0 tx=0).stop 2024-09-16T01:40:41.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.951+0000 7f3d02785740 1 -- 172.21.15.139:0/3593783055 shutdown_connections 2024-09-16T01:40:41.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.951+0000 7f3d02785740 1 --2- 172.21.15.139:0/3593783055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13f64fa0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.951+0000 7f3d02785740 1 --2- 172.21.15.139:0/3593783055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f582f0 0x564d13f62670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.951+0000 7f3d02785740 1 --2- 172.21.15.139:0/3593783055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f579d0 0x564d13f57db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.951+0000 7f3d02785740 1 -- 172.21.15.139:0/3593783055 >> 172.21.15.139:0/3593783055 conn(0x564d13f53f60 msgr2=0x564d13f54340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:41.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.951+0000 7f3d02785740 1 -- 172.21.15.139:0/3593783055 shutdown_connections 2024-09-16T01:40:41.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.951+0000 7f3d02785740 1 -- 172.21.15.139:0/3593783055 wait complete. 2024-09-16T01:40:41.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3d02785740 1 -- start start 2024-09-16T01:40:41.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3d02785740 1 Processor -- start 2024-09-16T01:40:41.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3d02785740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f579d0 0x564d13f62450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3d02785740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f582f0 0x564d13f5f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3d02785740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13f5fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3d02785740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564d13f65780 con 0x564d13f62bb0 2024-09-16T01:40:41.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3d02785740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564d13e78f20 con 0x564d13f579d0 2024-09-16T01:40:41.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3d02785740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564d13f65900 con 0x564d13f582f0 2024-09-16T01:40:41.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3d02785740 10 client.0 _ll_register_callbacks cb 0x564d13e42030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:41.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3cfe949640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13f5fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3cfe949640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13f5fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46642/0 (socket says 172.21.15.139:46642) 2024-09-16T01:40:41.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3cfe949640 1 -- 172.21.15.139:0/1896418255 learned_addr learned my addr 172.21.15.139:0/1896418255 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:41.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3cfe148640 1 --2- 172.21.15.139:0/1896418255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f579d0 0x564d13f62450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3cfd947640 1 --2- 172.21.15.139:0/1896418255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f582f0 0x564d13f5f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3cfe949640 1 -- 172.21.15.139:0/1896418255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f582f0 msgr2=0x564d13f5f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:41.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3cfe949640 1 --2- 172.21.15.139:0/1896418255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f582f0 0x564d13f5f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3cfe949640 1 -- 172.21.15.139:0/1896418255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f579d0 msgr2=0x564d13f62450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:41.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3cfe949640 1 --2- 172.21.15.139:0/1896418255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f579d0 0x564d13f62450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.952+0000 7f3cfe949640 1 -- 172.21.15.139:0/1896418255 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d13c2f2b0 con 0x564d13f62bb0 2024-09-16T01:40:41.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3cfe949640 1 --2- 172.21.15.139:0/1896418255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13f5fce0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0012910 tx=0x7f3cf0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53db94059a6032f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:41.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3ce1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:41.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3ce1ffb640 1 -- 172.21.15.139:0/1896418255 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0023070 con 0x564d13f62bb0 2024-09-16T01:40:41.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3ce1ffb640 1 -- 172.21.15.139:0/1896418255 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f3cf0011ae0 con 0x564d13f62bb0 2024-09-16T01:40:41.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3d02785740 1 -- 172.21.15.139:0/1896418255 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ce809c730 con 0x564d13f62bb0 2024-09-16T01:40:41.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3ce1ffb640 1 -- 172.21.15.139:0/1896418255 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3cf002f050 con 0x564d13f62bb0 2024-09-16T01:40:41.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3d02785740 10 client.15507 fetch_fsmap learned FSMap version 29 2024-09-16T01:40:41.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3d02785740 1 -- 172.21.15.139:0/1896418255 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564d13f71100 con 0x564d13f62bb0 2024-09-16T01:40:41.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3ce1ffb640 1 -- 172.21.15.139:0/1896418255 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce809c730 con 0x564d13f62bb0 2024-09-16T01:40:41.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3ce1ffb640 1 -- 172.21.15.139:0/1896418255 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3cf000f040 con 0x564d13f62bb0 2024-09-16T01:40:41.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3d02785740 10 client.15507 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:40:41.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3d02785740 10 client.15507 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T01:40:41.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3d02785740 20 client.15507 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:41.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3d02785740 1 --2- 172.21.15.139:0/1896418255 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x564d13f73730 0x564d13f74ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3d02785740 4 client.15507 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:40:41.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.953+0000 7f3d02785740 1 -- 172.21.15.139:0/1896418255 --> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d13ecb600 con 0x564d13f73730 2024-09-16T01:40:41.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.954+0000 7f3cfe148640 1 --2- 172.21.15.139:0/1896418255 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x564d13f73730 0x564d13f74ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.954+0000 7f3cfe148640 1 --2- 172.21.15.139:0/1896418255 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x564d13f73730 0x564d13f74ba0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=37585625339ed369 server_cookie=ebe9a9d643775dfb in_seq=0 out_seq=0 2024-09-16T01:40:41.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.954+0000 7f3ce1ffb640 10 client.15507 ms_handle_connect on v2:172.21.15.176:6836/1942543693 2024-09-16T01:40:41.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3ce1ffb640 1 -- 172.21.15.139:0/1896418255 <== mds.3 v2:172.21.15.176:6836/1942543693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d13ecb600 con 0x564d13f73730 2024-09-16T01:40:41.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3ce1ffb640 10 client.15507 handle_command_reply: tid=0 2024-09-16T01:40:41.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3d02785740 1 client.15507 shutdown 2024-09-16T01:40:41.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3d02785740 1 -- 172.21.15.139:0/1896418255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 msgr2=0x564d13f5fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:41.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3d02785740 1 --2- 172.21.15.139:0/1896418255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13f5fce0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0012910 tx=0x7f3cf0004ab0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3d02785740 1 -- 172.21.15.139:0/1896418255 shutdown_connections 2024-09-16T01:40:41.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3d02785740 1 --2- 172.21.15.139:0/1896418255 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x564d13f73730 0x564d13f74ba0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3d02785740 1 --2- 172.21.15.139:0/1896418255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13f5fce0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3d02785740 1 --2- 172.21.15.139:0/1896418255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f582f0 0x564d13f5f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3d02785740 1 --2- 172.21.15.139:0/1896418255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f579d0 0x564d13f62450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3d02785740 1 -- 172.21.15.139:0/1896418255 >> 172.21.15.139:0/1896418255 conn(0x564d13f53f60 msgr2=0x564d13f59c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:41.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3ce1ffb640 0 client.15507 ms_handle_reset on v2:172.21.15.176:6836/1942543693 2024-09-16T01:40:41.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3d02785740 1 -- 172.21.15.139:0/1896418255 shutdown_connections 2024-09-16T01:40:41.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3d02785740 1 -- 172.21.15.139:0/1896418255 wait complete. 2024-09-16T01:40:41.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.955+0000 7f3d02785740 20 client.15507 trim_cache size 0 max 16384 2024-09-16T01:40:41.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.972+0000 7f3d02785740 1 Processor -- start 2024-09-16T01:40:41.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3d02785740 1 -- start start 2024-09-16T01:40:41.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3d02785740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f579d0 0x564d13f73730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3d02785740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f582f0 0x564d13f5f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3d02785740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f62bb0 0x564d13f7d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3d02785740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564d13f72ee0 con 0x564d13f582f0 2024-09-16T01:40:41.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3d02785740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564d13f735b0 con 0x564d13f62bb0 2024-09-16T01:40:41.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3d02785740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564d13f73c80 con 0x564d13f579d0 2024-09-16T01:40:41.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3cfd947640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f582f0 0x564d13f5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3cfd947640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f582f0 0x564d13f5f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46654/0 (socket says 172.21.15.139:46654) 2024-09-16T01:40:41.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3cfd947640 1 -- 172.21.15.139:0/63918697 learned_addr learned my addr 172.21.15.139:0/63918697 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:41.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3cfd947640 1 -- 172.21.15.139:0/63918697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f579d0 msgr2=0x564d13f73730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:41.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3cfd947640 1 --2- 172.21.15.139:0/63918697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f579d0 0x564d13f73730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3cfd947640 1 -- 172.21.15.139:0/63918697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f62bb0 msgr2=0x564d13f7d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:41.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3cfd947640 1 --2- 172.21.15.139:0/63918697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f62bb0 0x564d13f7d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3cfd947640 1 -- 172.21.15.139:0/63918697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d13f57750 con 0x564d13f582f0 2024-09-16T01:40:41.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3cfd947640 1 --2- 172.21.15.139:0/63918697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f582f0 0x564d13f5f080 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3ce809a2e0 tx=0x7f3ce800fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87801b27b9243a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:41.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3ce3fff640 1 -- 172.21.15.139:0/63918697 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce809bb70 con 0x564d13f582f0 2024-09-16T01:40:41.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3ce3fff640 1 -- 172.21.15.139:0/63918697 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce809c730 con 0x564d13f582f0 2024-09-16T01:40:41.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.973+0000 7f3ce3fff640 1 -- 172.21.15.139:0/63918697 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce809be00 con 0x564d13f582f0 2024-09-16T01:40:41.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.974+0000 7f3d02785740 1 -- 172.21.15.139:0/63918697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f582f0 msgr2=0x564d13f5f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:41.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.974+0000 7f3d02785740 1 --2- 172.21.15.139:0/63918697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f582f0 0x564d13f5f080 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3ce809a2e0 tx=0x7f3ce800fda0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.974+0000 7f3d02785740 1 -- 172.21.15.139:0/63918697 shutdown_connections 2024-09-16T01:40:41.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.974+0000 7f3d02785740 1 --2- 172.21.15.139:0/63918697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f62bb0 0x564d13f7d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.974+0000 7f3d02785740 1 --2- 172.21.15.139:0/63918697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f582f0 0x564d13f5f080 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.974+0000 7f3d02785740 1 --2- 172.21.15.139:0/63918697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f579d0 0x564d13f73730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.974+0000 7f3d02785740 1 -- 172.21.15.139:0/63918697 >> 172.21.15.139:0/63918697 conn(0x564d13f53f60 msgr2=0x564d13f54340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:41.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.974+0000 7f3d02785740 1 -- 172.21.15.139:0/63918697 shutdown_connections 2024-09-16T01:40:41.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.974+0000 7f3d02785740 1 -- 172.21.15.139:0/63918697 wait complete. 2024-09-16T01:40:41.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.974+0000 7f3d02785740 1 -- start start 2024-09-16T01:40:41.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.974+0000 7f3d02785740 1 Processor -- start 2024-09-16T01:40:41.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3d02785740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f579d0 0x564d13f7a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3d02785740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f582f0 0x564d13f7a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3d02785740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13fdfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3d02785740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564d13f74a20 con 0x564d13f62bb0 2024-09-16T01:40:41.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3d02785740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564d13f74350 con 0x564d13f582f0 2024-09-16T01:40:41.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3d02785740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564d13f750f0 con 0x564d13f579d0 2024-09-16T01:40:41.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3d02785740 10 client.0 _ll_register_callbacks cb 0x7f3cf0009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:41.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3cfe949640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13fdfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3cfe949640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13fdfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46660/0 (socket says 172.21.15.139:46660) 2024-09-16T01:40:41.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3cfe949640 1 -- 172.21.15.139:0/1853160485 learned_addr learned my addr 172.21.15.139:0/1853160485 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:41.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3cfe148640 1 --2- 172.21.15.139:0/1853160485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f579d0 0x564d13f7a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3cfd947640 1 --2- 172.21.15.139:0/1853160485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f582f0 0x564d13f7a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3cfe949640 1 -- 172.21.15.139:0/1853160485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f579d0 msgr2=0x564d13f7a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:41.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3cfe949640 1 --2- 172.21.15.139:0/1853160485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f579d0 0x564d13f7a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3cfe949640 1 -- 172.21.15.139:0/1853160485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f582f0 msgr2=0x564d13f7a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:41.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3cfe949640 1 --2- 172.21.15.139:0/1853160485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f582f0 0x564d13f7a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3cfe949640 1 -- 172.21.15.139:0/1853160485 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d13f7b070 con 0x564d13f62bb0 2024-09-16T01:40:41.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3cfe949640 1 --2- 172.21.15.139:0/1853160485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13fdfe60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3cf001d450 tx=0x7f3cf0011040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c25b3f6ce355e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:41.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3ce1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:41.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3ce1ffb640 1 -- 172.21.15.139:0/1853160485 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0023070 con 0x564d13f62bb0 2024-09-16T01:40:41.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3ce1ffb640 1 -- 172.21.15.139:0/1853160485 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f3cf001da50 con 0x564d13f62bb0 2024-09-16T01:40:41.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.975+0000 7f3d02785740 1 -- 172.21.15.139:0/1853160485 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ce809c730 con 0x564d13f62bb0 2024-09-16T01:40:41.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.976+0000 7f3ce1ffb640 1 -- 172.21.15.139:0/1853160485 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3cf0011230 con 0x564d13f62bb0 2024-09-16T01:40:41.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.976+0000 7f3d02785740 10 client.15513 fetch_fsmap learned FSMap version 29 2024-09-16T01:40:41.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.976+0000 7f3d02785740 1 -- 172.21.15.139:0/1853160485 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564d13fe1470 con 0x564d13f62bb0 2024-09-16T01:40:41.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.976+0000 7f3ce1ffb640 1 -- 172.21.15.139:0/1853160485 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce809c730 con 0x564d13f62bb0 2024-09-16T01:40:41.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.976+0000 7f3ce1ffb640 1 -- 172.21.15.139:0/1853160485 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3cf00346e0 con 0x564d13f62bb0 2024-09-16T01:40:41.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.976+0000 7f3d02785740 10 client.15513 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:40:41.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.976+0000 7f3d02785740 10 client.15513 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T01:40:41.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.976+0000 7f3d02785740 20 client.15513 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:41.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.976+0000 7f3d02785740 1 --2- 172.21.15.139:0/1853160485 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x564d13fe6040 0x564d13fe4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:41.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.976+0000 7f3d02785740 4 client.15513 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:40:41.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.976+0000 7f3d02785740 1 -- 172.21.15.139:0/1853160485 --> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] -- command(tid 0: {"prefix": "status"}) -- 0x564d13ec5b10 con 0x564d13fe6040 2024-09-16T01:40:41.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.976+0000 7f3cfe148640 1 --2- 172.21.15.139:0/1853160485 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x564d13fe6040 0x564d13fe4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:41.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3cfe148640 1 --2- 172.21.15.139:0/1853160485 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x564d13fe6040 0x564d13fe4d50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d6560a59fc4aadb server_cookie=a55abb1339f7834b in_seq=0 out_seq=0 2024-09-16T01:40:41.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3ce1ffb640 10 client.15513 ms_handle_connect on v2:172.21.15.176:6836/1942543693 2024-09-16T01:40:41.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3ce1ffb640 1 -- 172.21.15.139:0/1853160485 <== mds.3 v2:172.21.15.176:6836/1942543693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564d13ec5b10 con 0x564d13fe6040 2024-09-16T01:40:41.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3ce1ffb640 10 client.15513 handle_command_reply: tid=0 2024-09-16T01:40:41.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3d02785740 1 client.15513 shutdown 2024-09-16T01:40:41.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3d02785740 1 -- 172.21.15.139:0/1853160485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 msgr2=0x564d13fdfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:41.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3d02785740 1 --2- 172.21.15.139:0/1853160485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13fdfe60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3cf001d450 tx=0x7f3cf0011040 comp rx=0 tx=0).stop 2024-09-16T01:40:41.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3d02785740 1 -- 172.21.15.139:0/1853160485 shutdown_connections 2024-09-16T01:40:41.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3d02785740 1 --2- 172.21.15.139:0/1853160485 >> [v2:172.21.15.176:6836/1942543693,v1:172.21.15.176:6837/1942543693] conn(0x564d13fe6040 0x564d13fe4d50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3d02785740 1 --2- 172.21.15.139:0/1853160485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d13f62bb0 0x564d13fdfe60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3d02785740 1 --2- 172.21.15.139:0/1853160485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d13f582f0 0x564d13f7a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3d02785740 1 --2- 172.21.15.139:0/1853160485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d13f579d0 0x564d13f7a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3ce1ffb640 0 client.15513 ms_handle_reset on v2:172.21.15.176:6836/1942543693 2024-09-16T01:40:41.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3d02785740 1 -- 172.21.15.139:0/1853160485 >> 172.21.15.139:0/1853160485 conn(0x564d13f53f60 msgr2=0x564d13f54340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:41.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3d02785740 1 -- 172.21.15.139:0/1853160485 shutdown_connections 2024-09-16T01:40:41.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3d02785740 1 -- 172.21.15.139:0/1853160485 wait complete. 2024-09-16T01:40:41.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.977+0000 7f3d02785740 20 client.15513 trim_cache size 0 max 16384 2024-09-16T01:40:41.984 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:40:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:40:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 3, 2024-09-16T01:40:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24442, 2024-09-16T01:40:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:40:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:40:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:40:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 123.43930726000001, 2024-09-16T01:40:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 29, 2024-09-16T01:40:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 86, 2024-09-16T01:40:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 85, 2024-09-16T01:40:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 186.21386036499999 2024-09-16T01:40:41.985 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:40:41.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.978+0000 7f3d003ac640 1 -- 172.21.15.139:0/2777624039 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f3ce40a34c0 msgr2=0x7f3ce40a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:41.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.978+0000 7f3d003ac640 1 --2- 172.21.15.139:0/2777624039 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f3ce40a34c0 0x7f3ce40a5980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0005fd0 tx=0x7f3cf0004a40 comp rx=0 tx=0).stop 2024-09-16T01:40:41.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.978+0000 7f3d003ac640 1 -- 172.21.15.139:0/2777624039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cf81027a0 msgr2=0x7f3cf8198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:41.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.978+0000 7f3d003ac640 1 --2- 172.21.15.139:0/2777624039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cf81027a0 0x7f3cf8198a60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800c970 tx=0x7f3ce800ce30 comp rx=0 tx=0).stop 2024-09-16T01:40:41.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.978+0000 7f3d003ac640 1 -- 172.21.15.139:0/2777624039 shutdown_connections 2024-09-16T01:40:41.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.978+0000 7f3d003ac640 1 --2- 172.21.15.139:0/2777624039 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f3ce40a34c0 0x7f3ce40a5980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.978+0000 7f3d003ac640 1 --2- 172.21.15.139:0/2777624039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cf8106f20 0x7f3cf819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.978+0000 7f3d003ac640 1 --2- 172.21.15.139:0/2777624039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cf81027a0 0x7f3cf8198a60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.978+0000 7f3d003ac640 1 --2- 172.21.15.139:0/2777624039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cf8101df0 0x7f3cf8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:41.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.978+0000 7f3d003ac640 1 -- 172.21.15.139:0/2777624039 >> 172.21.15.139:0/2777624039 conn(0x7f3cf80fd8b0 msgr2=0x7f3cf80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:41.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.978+0000 7f3d003ac640 1 -- 172.21.15.139:0/2777624039 shutdown_connections 2024-09-16T01:40:41.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:41.978+0000 7f3d003ac640 1 -- 172.21.15.139:0/2777624039 wait complete. 2024-09-16T01:40:42.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:41 smithi164 ceph-mon[35121]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-09-16T01:40:42.693 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T01:40:42.939 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:43.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.562+0000 7f8126ffc640 1 -- 172.21.15.139:0/3787219052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8120101de0 msgr2=0x7f81201021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:43.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.562+0000 7f8126ffc640 1 --2- 172.21.15.139:0/3787219052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8120101de0 0x7f81201021c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8114009900 tx=0x7f811402a410 comp rx=0 tx=0).stop 2024-09-16T01:40:43.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.562+0000 7f8126ffc640 1 -- 172.21.15.139:0/3787219052 shutdown_connections 2024-09-16T01:40:43.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.562+0000 7f8126ffc640 1 --2- 172.21.15.139:0/3787219052 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8120106f10 0x7f812010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.562+0000 7f8126ffc640 1 --2- 172.21.15.139:0/3787219052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8120102790 0x7f81201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.562+0000 7f8126ffc640 1 --2- 172.21.15.139:0/3787219052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8120101de0 0x7f81201021c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.562+0000 7f8126ffc640 1 -- 172.21.15.139:0/3787219052 >> 172.21.15.139:0/3787219052 conn(0x7f81200fd8c0 msgr2=0x7f81200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:43.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.562+0000 7f8126ffc640 1 -- 172.21.15.139:0/3787219052 shutdown_connections 2024-09-16T01:40:43.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.562+0000 7f8126ffc640 1 -- 172.21.15.139:0/3787219052 wait complete. 2024-09-16T01:40:43.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.562+0000 7f8126ffc640 1 Processor -- start 2024-09-16T01:40:43.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.562+0000 7f8126ffc640 1 -- start start 2024-09-16T01:40:43.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f8126ffc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8120101de0 0x7f8120198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f8126ffc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8120102790 0x7f81201989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f8126ffc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8120106f10 0x7f812019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f8126ffc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8120111f30 con 0x7f8120102790 2024-09-16T01:40:43.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f8126ffc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8120111db0 con 0x7f8120106f10 2024-09-16T01:40:43.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f8126ffc640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f81201120b0 con 0x7f8120101de0 2024-09-16T01:40:43.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f811ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8120102790 0x7f81201989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f811ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8120102790 0x7f81201989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:32984/0 (socket says 172.21.15.139:32984) 2024-09-16T01:40:43.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f811ffff640 1 -- 172.21.15.139:0/1498878633 learned_addr learned my addr 172.21.15.139:0/1498878633 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:43.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f8124d98640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8120101de0 0x7f8120198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f811ffff640 1 -- 172.21.15.139:0/1498878633 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8120101de0 msgr2=0x7f8120198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:43.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f8125599640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8120106f10 0x7f812019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f811ffff640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8120101de0 0x7f8120198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f811ffff640 1 -- 172.21.15.139:0/1498878633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8120106f10 msgr2=0x7f812019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:43.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f811ffff640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8120106f10 0x7f812019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f811ffff640 1 -- 172.21.15.139:0/1498878633 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812019d440 con 0x7f8120102790 2024-09-16T01:40:43.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f8125599640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8120106f10 0x7f812019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:43.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f811ffff640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8120102790 0x7f81201989d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f810c00d950 tx=0x7f810c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:43.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f811dffb640 1 -- 172.21.15.139:0/1498878633 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c017030 con 0x7f8120102790 2024-09-16T01:40:43.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f811dffb640 1 -- 172.21.15.139:0/1498878633 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f810c004020 con 0x7f8120102790 2024-09-16T01:40:43.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f8126ffc640 1 -- 172.21.15.139:0/1498878633 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812019d730 con 0x7f8120102790 2024-09-16T01:40:43.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.563+0000 7f811dffb640 1 -- 172.21.15.139:0/1498878633 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c0047a0 con 0x7f8120102790 2024-09-16T01:40:43.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.564+0000 7f8126ffc640 1 -- 172.21.15.139:0/1498878633 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81201a5010 con 0x7f8120102790 2024-09-16T01:40:43.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.566+0000 7f811dffb640 1 -- 172.21.15.139:0/1498878633 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 23) ==== 138532+0+0 (secure 0 0 0) 0x7f810c0041d0 con 0x7f8120102790 2024-09-16T01:40:43.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.566+0000 7f81293d5740 1 Processor -- start 2024-09-16T01:40:43.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.566+0000 7f81293d5740 1 -- start start 2024-09-16T01:40:43.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.566+0000 7f81293d5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be30c9d0 0x55f3be30cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.566+0000 7f81293d5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 0x55f3be317670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.566+0000 7f81293d5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be317bb0 0x55f3be319fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.566+0000 7f811dffb640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f81100a3440 0x7f81100a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.566+0000 7f81293d5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f3be273090 con 0x55f3be30c9d0 2024-09-16T01:40:43.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.566+0000 7f81293d5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f3be21aa70 con 0x55f3be30d2f0 2024-09-16T01:40:43.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.566+0000 7f81293d5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f3be26f050 con 0x55f3be317bb0 2024-09-16T01:40:43.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f8125599640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be30c9d0 0x55f3be30cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f8125599640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be30c9d0 0x55f3be30cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:32986/0 (socket says 172.21.15.139:32986) 2024-09-16T01:40:43.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f8125599640 1 -- 172.21.15.139:0/3684290777 learned_addr learned my addr 172.21.15.139:0/3684290777 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:43.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f8125599640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f81100a3440 0x7f81100a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f811dffb640 1 -- 172.21.15.139:0/1498878633 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f810c0cfd10 con 0x7f8120102790 2024-09-16T01:40:43.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f811ffff640 1 --2- 172.21.15.139:0/3684290777 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be317bb0 0x55f3be319fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f8124d98640 1 --2- 172.21.15.139:0/3684290777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 0x55f3be317670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f8125599640 1 -- 172.21.15.139:0/3684290777 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be317bb0 msgr2=0x55f3be319fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:43.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f8125599640 1 --2- 172.21.15.139:0/3684290777 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be317bb0 0x55f3be319fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f8125599640 1 -- 172.21.15.139:0/3684290777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 msgr2=0x55f3be317670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:43.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f8125599640 1 --2- 172.21.15.139:0/3684290777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 0x55f3be317670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f8125599640 1 -- 172.21.15.139:0/3684290777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3be227130 con 0x55f3be30c9d0 2024-09-16T01:40:43.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f8124d98640 1 --2- 172.21.15.139:0/3684290777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 0x55f3be317670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:43.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f8125599640 1 --2- 172.21.15.139:0/3684290777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be30c9d0 0x55f3be30cdb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8118012db0 tx=0x7f8118010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53dc14efdc91b87 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:43.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f8125599640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f81100a3440 0x7f81100a5900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8118010c40 tx=0x7f811801b000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:43.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f810affd640 1 -- 172.21.15.139:0/3684290777 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118004d20 con 0x55f3be30c9d0 2024-09-16T01:40:43.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f810affd640 1 -- 172.21.15.139:0/3684290777 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8118005020 con 0x55f3be30c9d0 2024-09-16T01:40:43.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.567+0000 7f810affd640 1 -- 172.21.15.139:0/3684290777 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811801d680 con 0x55f3be30c9d0 2024-09-16T01:40:43.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.568+0000 7f81293d5740 1 -- 172.21.15.139:0/3684290777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be30c9d0 msgr2=0x55f3be30cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:43.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.568+0000 7f81293d5740 1 --2- 172.21.15.139:0/3684290777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be30c9d0 0x55f3be30cdb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8118012db0 tx=0x7f8118010740 comp rx=0 tx=0).stop 2024-09-16T01:40:43.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.568+0000 7f81293d5740 1 -- 172.21.15.139:0/3684290777 shutdown_connections 2024-09-16T01:40:43.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.568+0000 7f81293d5740 1 --2- 172.21.15.139:0/3684290777 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be317bb0 0x55f3be319fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.568+0000 7f81293d5740 1 --2- 172.21.15.139:0/3684290777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 0x55f3be317670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.568+0000 7f81293d5740 1 --2- 172.21.15.139:0/3684290777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be30c9d0 0x55f3be30cdb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.568+0000 7f81293d5740 1 -- 172.21.15.139:0/3684290777 >> 172.21.15.139:0/3684290777 conn(0x55f3be308f60 msgr2=0x55f3be309340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:43.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.568+0000 7f81293d5740 1 -- 172.21.15.139:0/3684290777 shutdown_connections 2024-09-16T01:40:43.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.568+0000 7f81293d5740 1 -- 172.21.15.139:0/3684290777 wait complete. 2024-09-16T01:40:43.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.568+0000 7f81293d5740 1 -- start start 2024-09-16T01:40:43.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f81293d5740 1 Processor -- start 2024-09-16T01:40:43.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f81293d5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30c9d0 0x55f3be317520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f81293d5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 0x55f3be314870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f81293d5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be314dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f81293d5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f3be31a780 con 0x55f3be317bb0 2024-09-16T01:40:43.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f81293d5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f3be22df20 con 0x55f3be30d2f0 2024-09-16T01:40:43.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f81293d5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f3be31a900 con 0x55f3be30c9d0 2024-09-16T01:40:43.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f81293d5740 10 client.0 _ll_register_callbacks cb 0x55f3be1f7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:43.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f8125599640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be314dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f8124d98640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30c9d0 0x55f3be317520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f8125599640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be314dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:32996/0 (socket says 172.21.15.139:32996) 2024-09-16T01:40:43.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f8125599640 1 -- 172.21.15.139:0/1273142183 learned_addr learned my addr 172.21.15.139:0/1273142183 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:43.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f811ffff640 1 --2- 172.21.15.139:0/1273142183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 0x55f3be314870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f8125599640 1 -- 172.21.15.139:0/1273142183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30c9d0 msgr2=0x55f3be317520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:43.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f8125599640 1 --2- 172.21.15.139:0/1273142183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30c9d0 0x55f3be317520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f8125599640 1 -- 172.21.15.139:0/1273142183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 msgr2=0x55f3be314870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:43.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f8125599640 1 --2- 172.21.15.139:0/1273142183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 0x55f3be314870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f8125599640 1 -- 172.21.15.139:0/1273142183 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3bdfe42b0 con 0x55f3be317bb0 2024-09-16T01:40:43.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.569+0000 7f811ffff640 1 --2- 172.21.15.139:0/1273142183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 0x55f3be314870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:43.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f8125599640 1 --2- 172.21.15.139:0/1273142183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be314dd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8118006010 tx=0x7f811800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8ea35b4ad58571 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:43.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f80fffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:43.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f80fffff640 1 -- 172.21.15.139:0/1273142183 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118003d80 con 0x55f3be317bb0 2024-09-16T01:40:43.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f81293d5740 1 -- 172.21.15.139:0/1273142183 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8118005020 con 0x55f3be317bb0 2024-09-16T01:40:43.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f80fffff640 1 -- 172.21.15.139:0/1273142183 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f81180051d0 con 0x55f3be317bb0 2024-09-16T01:40:43.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f80fffff640 1 -- 172.21.15.139:0/1273142183 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8118038050 con 0x55f3be317bb0 2024-09-16T01:40:43.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f81293d5740 10 client.15525 fetch_fsmap learned FSMap version 29 2024-09-16T01:40:43.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f81293d5740 1 -- 172.21.15.139:0/1273142183 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3be326150 con 0x55f3be317bb0 2024-09-16T01:40:43.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f80fffff640 1 -- 172.21.15.139:0/1273142183 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8118005020 con 0x55f3be317bb0 2024-09-16T01:40:43.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f80fffff640 1 -- 172.21.15.139:0/1273142183 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f811800f040 con 0x55f3be317bb0 2024-09-16T01:40:43.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f81293d5740 10 client.15525 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:40:43.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f81293d5740 10 client.15525 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T01:40:43.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f81293d5740 20 client.15525 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:43.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f81293d5740 1 --2- 172.21.15.139:0/1273142183 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x55f3be3285a0 0x55f3be329a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f81293d5740 4 client.15525 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:40:43.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.570+0000 7f81293d5740 1 -- 172.21.15.139:0/1273142183 --> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3be280600 con 0x55f3be3285a0 2024-09-16T01:40:43.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.571+0000 7f8124d98640 1 --2- 172.21.15.139:0/1273142183 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x55f3be3285a0 0x55f3be329a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.571+0000 7f80fffff640 10 client.15525 ms_handle_connect on v2:172.21.15.164:6836/1270226509 2024-09-16T01:40:43.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.571+0000 7f8124d98640 1 --2- 172.21.15.139:0/1273142183 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x55f3be3285a0 0x55f3be329a10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8555b5aae6bf1718 server_cookie=184f010af45f48c4 in_seq=0 out_seq=0 2024-09-16T01:40:43.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f80fffff640 1 -- 172.21.15.139:0/1273142183 <== mds.2 v2:172.21.15.164:6836/1270226509 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3be280600 con 0x55f3be3285a0 2024-09-16T01:40:43.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f80fffff640 10 client.15525 handle_command_reply: tid=0 2024-09-16T01:40:43.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f81293d5740 1 client.15525 shutdown 2024-09-16T01:40:43.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f81293d5740 1 -- 172.21.15.139:0/1273142183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 msgr2=0x55f3be314dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:43.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f81293d5740 1 --2- 172.21.15.139:0/1273142183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be314dd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8118006010 tx=0x7f811800bfd0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f81293d5740 1 -- 172.21.15.139:0/1273142183 shutdown_connections 2024-09-16T01:40:43.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f81293d5740 1 --2- 172.21.15.139:0/1273142183 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x55f3be3285a0 0x55f3be329a10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f81293d5740 1 --2- 172.21.15.139:0/1273142183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be314dd0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f81293d5740 1 --2- 172.21.15.139:0/1273142183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 0x55f3be314870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f81293d5740 1 --2- 172.21.15.139:0/1273142183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30c9d0 0x55f3be317520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f81293d5740 1 -- 172.21.15.139:0/1273142183 >> 172.21.15.139:0/1273142183 conn(0x55f3be308f60 msgr2=0x55f3be30ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:43.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f80fffff640 0 client.15525 ms_handle_reset on v2:172.21.15.164:6836/1270226509 2024-09-16T01:40:43.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f81293d5740 1 -- 172.21.15.139:0/1273142183 shutdown_connections 2024-09-16T01:40:43.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f81293d5740 1 -- 172.21.15.139:0/1273142183 wait complete. 2024-09-16T01:40:43.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.572+0000 7f81293d5740 20 client.15525 trim_cache size 0 max 16384 2024-09-16T01:40:43.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f81293d5740 1 Processor -- start 2024-09-16T01:40:43.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f81293d5740 1 -- start start 2024-09-16T01:40:43.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f81293d5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30c9d0 0x55f3be314dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f81293d5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30d2f0 0x55f3be329a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f81293d5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be3166d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f81293d5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f3be327d50 con 0x55f3be317bb0 2024-09-16T01:40:43.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f81293d5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f3be328420 con 0x55f3be30c9d0 2024-09-16T01:40:43.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f81293d5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f3be328af0 con 0x55f3be30d2f0 2024-09-16T01:40:43.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f8125599640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be3166d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f8125599640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be3166d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:32998/0 (socket says 172.21.15.139:32998) 2024-09-16T01:40:43.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f8125599640 1 -- 172.21.15.139:0/1016199940 learned_addr learned my addr 172.21.15.139:0/1016199940 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:43.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f811ffff640 1 --2- 172.21.15.139:0/1016199940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30d2f0 0x55f3be329a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f8124d98640 1 --2- 172.21.15.139:0/1016199940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30c9d0 0x55f3be314dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f8125599640 1 -- 172.21.15.139:0/1016199940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30d2f0 msgr2=0x55f3be329a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:43.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f8125599640 1 --2- 172.21.15.139:0/1016199940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30d2f0 0x55f3be329a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f8125599640 1 -- 172.21.15.139:0/1016199940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30c9d0 msgr2=0x55f3be314dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:43.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f8125599640 1 --2- 172.21.15.139:0/1016199940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30c9d0 0x55f3be314dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f8125599640 1 -- 172.21.15.139:0/1016199940 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3be317140 con 0x55f3be317bb0 2024-09-16T01:40:43.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.590+0000 7f8125599640 1 --2- 172.21.15.139:0/1016199940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be3166d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8118012c80 tx=0x7f8118009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bbc7f17d991dcd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:43.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.591+0000 7f810a7fc640 1 -- 172.21.15.139:0/1016199940 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118003d80 con 0x55f3be317bb0 2024-09-16T01:40:43.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.591+0000 7f810a7fc640 1 -- 172.21.15.139:0/1016199940 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f811802a070 con 0x55f3be317bb0 2024-09-16T01:40:43.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.591+0000 7f810a7fc640 1 -- 172.21.15.139:0/1016199940 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118005500 con 0x55f3be317bb0 2024-09-16T01:40:43.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.591+0000 7f81293d5740 1 -- 172.21.15.139:0/1016199940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 msgr2=0x55f3be3166d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:43.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.591+0000 7f81293d5740 1 --2- 172.21.15.139:0/1016199940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be3166d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8118012c80 tx=0x7f8118009f90 comp rx=0 tx=0).stop 2024-09-16T01:40:43.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.591+0000 7f81293d5740 1 -- 172.21.15.139:0/1016199940 shutdown_connections 2024-09-16T01:40:43.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.591+0000 7f81293d5740 1 --2- 172.21.15.139:0/1016199940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be3166d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.591+0000 7f81293d5740 1 --2- 172.21.15.139:0/1016199940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30d2f0 0x55f3be329a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.591+0000 7f81293d5740 1 --2- 172.21.15.139:0/1016199940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30c9d0 0x55f3be314dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.591+0000 7f81293d5740 1 -- 172.21.15.139:0/1016199940 >> 172.21.15.139:0/1016199940 conn(0x55f3be308f60 msgr2=0x55f3be315fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:43.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.591+0000 7f81293d5740 1 -- 172.21.15.139:0/1016199940 shutdown_connections 2024-09-16T01:40:43.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.591+0000 7f81293d5740 1 -- 172.21.15.139:0/1016199940 wait complete. 2024-09-16T01:40:43.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.592+0000 7f81293d5740 1 -- start start 2024-09-16T01:40:43.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.592+0000 7f81293d5740 1 Processor -- start 2024-09-16T01:40:43.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.592+0000 7f81293d5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30c9d0 0x55f3be331d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.592+0000 7f81293d5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 0x55f3be3322d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.592+0000 7f81293d5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be32b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.592+0000 7f81293d5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f3be329890 con 0x55f3be317bb0 2024-09-16T01:40:43.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.592+0000 7f81293d5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f3be3291c0 con 0x55f3be30d2f0 2024-09-16T01:40:43.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.592+0000 7f81293d5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f3be329f60 con 0x55f3be30c9d0 2024-09-16T01:40:43.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.592+0000 7f81293d5740 10 client.0 _ll_register_callbacks cb 0x7f8118012bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:43.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.592+0000 7f8125599640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be32b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.592+0000 7f8125599640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be32b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33010/0 (socket says 172.21.15.139:33010) 2024-09-16T01:40:43.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.592+0000 7f8125599640 1 -- 172.21.15.139:0/1121459372 learned_addr learned my addr 172.21.15.139:0/1121459372 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:43.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.592+0000 7f811ffff640 1 --2- 172.21.15.139:0/1121459372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 0x55f3be3322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.592+0000 7f8124d98640 1 --2- 172.21.15.139:0/1121459372 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30c9d0 0x55f3be331d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f8125599640 1 -- 172.21.15.139:0/1121459372 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30c9d0 msgr2=0x55f3be331d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:43.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f8125599640 1 --2- 172.21.15.139:0/1121459372 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30c9d0 0x55f3be331d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f8125599640 1 -- 172.21.15.139:0/1121459372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 msgr2=0x55f3be3322d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:43.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f8125599640 1 --2- 172.21.15.139:0/1121459372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 0x55f3be3322d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f8125599640 1 -- 172.21.15.139:0/1121459372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3be32b870 con 0x55f3be317bb0 2024-09-16T01:40:43.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f8125599640 1 --2- 172.21.15.139:0/1121459372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be32b1b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f811800c010 tx=0x7f811802f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d13d3f34d32a76e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:43.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f80fffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:43.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f80fffff640 1 -- 172.21.15.139:0/1121459372 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118003d80 con 0x55f3be317bb0 2024-09-16T01:40:43.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f80fffff640 1 -- 172.21.15.139:0/1121459372 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f8118005280 con 0x55f3be317bb0 2024-09-16T01:40:43.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f81293d5740 1 -- 172.21.15.139:0/1121459372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f811802a070 con 0x55f3be317bb0 2024-09-16T01:40:43.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f80fffff640 1 -- 172.21.15.139:0/1121459372 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f811804b050 con 0x55f3be317bb0 2024-09-16T01:40:43.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f81293d5740 10 client.15531 fetch_fsmap learned FSMap version 29 2024-09-16T01:40:43.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f81293d5740 1 -- 172.21.15.139:0/1121459372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3be330230 con 0x55f3be317bb0 2024-09-16T01:40:43.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f80fffff640 1 -- 172.21.15.139:0/1121459372 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f811802a070 con 0x55f3be317bb0 2024-09-16T01:40:43.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.593+0000 7f80fffff640 1 -- 172.21.15.139:0/1121459372 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8118026780 con 0x55f3be317bb0 2024-09-16T01:40:43.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.594+0000 7f81293d5740 10 client.15531 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:40:43.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.594+0000 7f81293d5740 10 client.15531 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T01:40:43.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.594+0000 7f81293d5740 20 client.15531 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:43.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.594+0000 7f81293d5740 1 --2- 172.21.15.139:0/1121459372 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x55f3be396f50 0x55f3be395c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:43.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.594+0000 7f81293d5740 4 client.15531 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:40:43.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.594+0000 7f81293d5740 1 -- 172.21.15.139:0/1121459372 --> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] -- command(tid 0: {"prefix": "status"}) -- 0x55f3be27ab10 con 0x55f3be396f50 2024-09-16T01:40:43.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.594+0000 7f8124d98640 1 --2- 172.21.15.139:0/1121459372 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x55f3be396f50 0x55f3be395c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:43.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.594+0000 7f8124d98640 1 --2- 172.21.15.139:0/1121459372 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x55f3be396f50 0x55f3be395c60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1978c6e42ae9b55 server_cookie=91423f4a638bcb94 in_seq=0 out_seq=0 2024-09-16T01:40:43.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.594+0000 7f80fffff640 10 client.15531 ms_handle_connect on v2:172.21.15.164:6836/1270226509 2024-09-16T01:40:43.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.594+0000 7f80fffff640 1 -- 172.21.15.139:0/1121459372 <== mds.2 v2:172.21.15.164:6836/1270226509 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f3be27ab10 con 0x55f3be396f50 2024-09-16T01:40:43.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.594+0000 7f80fffff640 10 client.15531 handle_command_reply: tid=0 2024-09-16T01:40:43.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.594+0000 7f81293d5740 1 client.15531 shutdown 2024-09-16T01:40:43.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.595+0000 7f81293d5740 1 -- 172.21.15.139:0/1121459372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 msgr2=0x55f3be32b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:43.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.595+0000 7f81293d5740 1 --2- 172.21.15.139:0/1121459372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be32b1b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f811800c010 tx=0x7f811802f6a0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.595+0000 7f81293d5740 1 -- 172.21.15.139:0/1121459372 shutdown_connections 2024-09-16T01:40:43.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.595+0000 7f81293d5740 1 --2- 172.21.15.139:0/1121459372 >> [v2:172.21.15.164:6836/1270226509,v1:172.21.15.164:6837/1270226509] conn(0x55f3be396f50 0x55f3be395c60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.595+0000 7f81293d5740 1 --2- 172.21.15.139:0/1121459372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f3be317bb0 0x55f3be32b1b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.595+0000 7f81293d5740 1 --2- 172.21.15.139:0/1121459372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3be30d2f0 0x55f3be3322d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.595+0000 7f80fffff640 0 client.15531 ms_handle_reset on v2:172.21.15.164:6836/1270226509 2024-09-16T01:40:43.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.595+0000 7f81293d5740 1 --2- 172.21.15.139:0/1121459372 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3be30c9d0 0x55f3be331d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.595+0000 7f81293d5740 1 -- 172.21.15.139:0/1121459372 >> 172.21.15.139:0/1121459372 conn(0x55f3be308f60 msgr2=0x55f3be3166c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:43.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.595+0000 7f81293d5740 1 -- 172.21.15.139:0/1121459372 shutdown_connections 2024-09-16T01:40:43.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.595+0000 7f81293d5740 1 -- 172.21.15.139:0/1121459372 wait complete. 2024-09-16T01:40:43.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.595+0000 7f81293d5740 20 client.15531 trim_cache size 0 max 16384 2024-09-16T01:40:43.604 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:40:43.604 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:40:43.604 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 2, 2024-09-16T01:40:43.604 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24443, 2024-09-16T01:40:43.604 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:40:43.604 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:40:43.604 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:40:43.605 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 126.065329451, 2024-09-16T01:40:43.605 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 29, 2024-09-16T01:40:43.605 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 86, 2024-09-16T01:40:43.605 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 85, 2024-09-16T01:40:43.605 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 185.30440848200001 2024-09-16T01:40:43.605 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:40:43.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.595+0000 7f8126ffc640 1 -- 172.21.15.139:0/1498878633 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f81100a3440 msgr2=0x7f81100a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:43.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.596+0000 7f8126ffc640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f81100a3440 0x7f81100a5900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8118010c40 tx=0x7f811801b000 comp rx=0 tx=0).stop 2024-09-16T01:40:43.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.596+0000 7f8126ffc640 1 -- 172.21.15.139:0/1498878633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8120102790 msgr2=0x7f81201989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:43.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.596+0000 7f8126ffc640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8120102790 0x7f81201989d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f810c00d950 tx=0x7f810c00de10 comp rx=0 tx=0).stop 2024-09-16T01:40:43.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.596+0000 7f8126ffc640 1 -- 172.21.15.139:0/1498878633 shutdown_connections 2024-09-16T01:40:43.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.596+0000 7f8126ffc640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f81100a3440 0x7f81100a5900 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.596+0000 7f8126ffc640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8120106f10 0x7f812019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.596+0000 7f8126ffc640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8120102790 0x7f81201989d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.596+0000 7f8126ffc640 1 --2- 172.21.15.139:0/1498878633 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8120101de0 0x7f8120198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:43.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.596+0000 7f8126ffc640 1 -- 172.21.15.139:0/1498878633 >> 172.21.15.139:0/1498878633 conn(0x7f81200fd8c0 msgr2=0x7f81200fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:43.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.596+0000 7f8126ffc640 1 -- 172.21.15.139:0/1498878633 shutdown_connections 2024-09-16T01:40:43.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:43.596+0000 7f8126ffc640 1 -- 172.21.15.139:0/1498878633 wait complete. 2024-09-16T01:40:43.607 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:43 smithi139 ceph-mon[31175]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 60 op/s 2024-09-16T01:40:43.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:43 smithi176 ceph-mon[35324]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 60 op/s 2024-09-16T01:40:44.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:43 smithi164 ceph-mon[35121]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 60 op/s 2024-09-16T01:40:44.424 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-16T01:40:44.669 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:45.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.160+0000 7fa6b4b8e640 1 -- 172.21.15.139:0/834232908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6b0102570 msgr2=0x7fa6b01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:45.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.160+0000 7fa6b4b8e640 1 --2- 172.21.15.139:0/834232908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6b0102570 0x7fa6b01029b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009900 tx=0x7fa6a402a480 comp rx=0 tx=0).stop 2024-09-16T01:40:45.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.160+0000 7fa6b4b8e640 1 -- 172.21.15.139:0/834232908 shutdown_connections 2024-09-16T01:40:45.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.160+0000 7fa6b4b8e640 1 --2- 172.21.15.139:0/834232908 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6b0102ef0 0x7fa6b010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.160+0000 7fa6b4b8e640 1 --2- 172.21.15.139:0/834232908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6b0102570 0x7fa6b01029b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.160+0000 7fa6b4b8e640 1 --2- 172.21.15.139:0/834232908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6b0100e20 0x7fa6b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.160+0000 7fa6b4b8e640 1 -- 172.21.15.139:0/834232908 >> 172.21.15.139:0/834232908 conn(0x7fa6b00fc7d0 msgr2=0x7fa6b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:45.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.160+0000 7fa6b4b8e640 1 -- 172.21.15.139:0/834232908 shutdown_connections 2024-09-16T01:40:45.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.160+0000 7fa6b4b8e640 1 -- 172.21.15.139:0/834232908 wait complete. 2024-09-16T01:40:45.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6b4b8e640 1 Processor -- start 2024-09-16T01:40:45.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6b4b8e640 1 -- start start 2024-09-16T01:40:45.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6b4b8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6b0100e20 0x7fa6b019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6b4b8e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6b0102570 0x7fa6b019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6b4b8e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6b0102ef0 0x7fa6b019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6b4b8e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b01120a0 con 0x7fa6b0100e20 2024-09-16T01:40:45.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6b4b8e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b0111f20 con 0x7fa6b0102ef0 2024-09-16T01:40:45.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6b4b8e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b0112220 con 0x7fa6b0102570 2024-09-16T01:40:45.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6ae59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6b0100e20 0x7fa6b019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6ae59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6b0100e20 0x7fa6b019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33034/0 (socket says 172.21.15.139:33034) 2024-09-16T01:40:45.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6ae59c640 1 -- 172.21.15.139:0/4130429179 learned_addr learned my addr 172.21.15.139:0/4130429179 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:45.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6add9b640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6b0102570 0x7fa6b019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6aed9d640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6b0102ef0 0x7fa6b019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6ae59c640 1 -- 172.21.15.139:0/4130429179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6b0102570 msgr2=0x7fa6b019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:45.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6ae59c640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6b0102570 0x7fa6b019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6ae59c640 1 -- 172.21.15.139:0/4130429179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6b0102ef0 msgr2=0x7fa6b019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:45.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6ae59c640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6b0102ef0 0x7fa6b019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6ae59c640 1 -- 172.21.15.139:0/4130429179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b019f6a0 con 0x7fa6b0100e20 2024-09-16T01:40:45.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.161+0000 7fa6aed9d640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6b0102ef0 0x7fa6b019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:45.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.162+0000 7fa6ae59c640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6b0100e20 0x7fa6b019a6f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000d950 tx=0x7fa6a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:45.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.162+0000 7fa69f7fe640 1 -- 172.21.15.139:0/4130429179 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0017070 con 0x7fa6b0100e20 2024-09-16T01:40:45.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.162+0000 7fa69f7fe640 1 -- 172.21.15.139:0/4130429179 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a0009c30 con 0x7fa6b0100e20 2024-09-16T01:40:45.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.162+0000 7fa6b4b8e640 1 -- 172.21.15.139:0/4130429179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b019f990 con 0x7fa6b0100e20 2024-09-16T01:40:45.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.162+0000 7fa69f7fe640 1 -- 172.21.15.139:0/4130429179 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a000f6b0 con 0x7fa6b0100e20 2024-09-16T01:40:45.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.162+0000 7fa6b4b8e640 1 -- 172.21.15.139:0/4130429179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b01a7270 con 0x7fa6b0100e20 2024-09-16T01:40:45.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.164+0000 7fa69f7fe640 1 -- 172.21.15.139:0/4130429179 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 23) ==== 138532+0+0 (secure 0 0 0) 0x7fa6a00109d0 con 0x7fa6b0100e20 2024-09-16T01:40:45.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.164+0000 7fa6b6f67740 1 Processor -- start 2024-09-16T01:40:45.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6b6f67740 1 -- start start 2024-09-16T01:40:45.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa69f7fe640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa6980a34c0 0x7fa6980a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6b6f67740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f3370d9d0 0x556f3370ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6b6f67740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370e2f0 0x556f33718670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6b6f67740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f33718bb0 0x556f3371afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6b6f67740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556f33674090 con 0x556f3370e2f0 2024-09-16T01:40:45.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6b6f67740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556f3361ba70 con 0x556f3370d9d0 2024-09-16T01:40:45.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6b6f67740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556f33670050 con 0x556f33718bb0 2024-09-16T01:40:45.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6aed9d640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa6980a34c0 0x7fa6980a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa69f7fe640 1 -- 172.21.15.139:0/4130429179 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fa6a00cfe60 con 0x7fa6b0100e20 2024-09-16T01:40:45.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6add9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370e2f0 0x556f33718670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6add9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370e2f0 0x556f33718670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33048/0 (socket says 172.21.15.139:33048) 2024-09-16T01:40:45.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6add9b640 1 -- 172.21.15.139:0/1803152537 learned_addr learned my addr 172.21.15.139:0/1803152537 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:45.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6ae59c640 1 --2- 172.21.15.139:0/1803152537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f3370d9d0 0x556f3370ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6aed9d640 1 --2- 172.21.15.139:0/1803152537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f33718bb0 0x556f3371afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6add9b640 1 -- 172.21.15.139:0/1803152537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f33718bb0 msgr2=0x556f3371afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:45.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6add9b640 1 --2- 172.21.15.139:0/1803152537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f33718bb0 0x556f3371afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6add9b640 1 -- 172.21.15.139:0/1803152537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f3370d9d0 msgr2=0x556f3370ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:45.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6add9b640 1 --2- 172.21.15.139:0/1803152537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f3370d9d0 0x556f3370ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6add9b640 1 -- 172.21.15.139:0/1803152537 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f33628130 con 0x556f3370e2f0 2024-09-16T01:40:45.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.165+0000 7fa6aed9d640 1 --2- 172.21.15.139:0/1803152537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f33718bb0 0x556f3371afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:45.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.166+0000 7fa6aed9d640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa6980a34c0 0x7fa6980a5980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8005fd0 tx=0x7fa6a8014b70 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:45.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.166+0000 7fa6add9b640 1 --2- 172.21.15.139:0/1803152537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370e2f0 0x556f33718670 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa6a402a9a0 tx=0x7fa6a40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c122ae8a94ae73 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:45.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.166+0000 7fa69cff9640 1 -- 172.21.15.139:0/1803152537 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4035030 con 0x556f3370e2f0 2024-09-16T01:40:45.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.166+0000 7fa69cff9640 1 -- 172.21.15.139:0/1803152537 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a4040070 con 0x556f3370e2f0 2024-09-16T01:40:45.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.166+0000 7fa69cff9640 1 -- 172.21.15.139:0/1803152537 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4031970 con 0x556f3370e2f0 2024-09-16T01:40:45.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.166+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1803152537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370e2f0 msgr2=0x556f33718670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:45.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.166+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/1803152537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370e2f0 0x556f33718670 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa6a402a9a0 tx=0x7fa6a40040c0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1803152537 shutdown_connections 2024-09-16T01:40:45.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/1803152537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f33718bb0 0x556f3371afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/1803152537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370e2f0 0x556f33718670 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/1803152537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f3370d9d0 0x556f3370ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1803152537 >> 172.21.15.139:0/1803152537 conn(0x556f33709f60 msgr2=0x556f3370a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:45.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1803152537 shutdown_connections 2024-09-16T01:40:45.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1803152537 wait complete. 2024-09-16T01:40:45.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 -- start start 2024-09-16T01:40:45.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 Processor -- start 2024-09-16T01:40:45.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f3370d9d0 0x556f33718430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370e2f0 0x556f33715780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f33718bb0 0x556f33715cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556f3371b780 con 0x556f3370e2f0 2024-09-16T01:40:45.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556f3362ef20 con 0x556f33718bb0 2024-09-16T01:40:45.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556f3371b900 con 0x556f3370d9d0 2024-09-16T01:40:45.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.167+0000 7fa6b6f67740 10 client.0 _ll_register_callbacks cb 0x556f335f8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:45.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa6add9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370e2f0 0x556f33715780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa6add9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370e2f0 0x556f33715780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33060/0 (socket says 172.21.15.139:33060) 2024-09-16T01:40:45.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa6add9b640 1 -- 172.21.15.139:0/901643794 learned_addr learned my addr 172.21.15.139:0/901643794 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:45.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa6ae59c640 1 --2- 172.21.15.139:0/901643794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f3370d9d0 0x556f33718430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa6aed9d640 1 --2- 172.21.15.139:0/901643794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f33718bb0 0x556f33715cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa6add9b640 1 -- 172.21.15.139:0/901643794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f3370d9d0 msgr2=0x556f33718430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:45.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa6add9b640 1 --2- 172.21.15.139:0/901643794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f3370d9d0 0x556f33718430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa6add9b640 1 -- 172.21.15.139:0/901643794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f33718bb0 msgr2=0x556f33715cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:45.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa6add9b640 1 --2- 172.21.15.139:0/901643794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f33718bb0 0x556f33715cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa6add9b640 1 -- 172.21.15.139:0/901643794 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f333e52b0 con 0x556f3370e2f0 2024-09-16T01:40:45.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa6add9b640 1 --2- 172.21.15.139:0/901643794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370e2f0 0x556f33715780 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa6a402a4c0 tx=0x7fa6a4009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e722af65902e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:45.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa695ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:45.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa695ffb640 1 -- 172.21.15.139:0/901643794 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4052020 con 0x556f3370e2f0 2024-09-16T01:40:45.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa695ffb640 1 -- 172.21.15.139:0/901643794 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fa6a4032cb0 con 0x556f3370e2f0 2024-09-16T01:40:45.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.168+0000 7fa6b6f67740 1 -- 172.21.15.139:0/901643794 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a4040070 con 0x556f3370e2f0 2024-09-16T01:40:45.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.169+0000 7fa695ffb640 1 -- 172.21.15.139:0/901643794 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa6a405a050 con 0x556f3370e2f0 2024-09-16T01:40:45.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.169+0000 7fa6b6f67740 10 client.15543 fetch_fsmap learned FSMap version 29 2024-09-16T01:40:45.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.169+0000 7fa6b6f67740 1 -- 172.21.15.139:0/901643794 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556f33727090 con 0x556f3370e2f0 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.169+0000 7fa695ffb640 1 -- 172.21.15.139:0/901643794 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a4040070 con 0x556f3370e2f0 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.169+0000 7fa695ffb640 1 -- 172.21.15.139:0/901643794 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa6a404d070 con 0x556f3370e2f0 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.169+0000 7fa6b6f67740 10 client.15543 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.169+0000 7fa6b6f67740 10 client.15543 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.169+0000 7fa6b6f67740 20 client.15543 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.169+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/901643794 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x556f337292a0 0x556f3372a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.169+0000 7fa6b6f67740 4 client.15543 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.169+0000 7fa6b6f67740 1 -- 172.21.15.139:0/901643794 --> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f33681600 con 0x556f337292a0 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.169+0000 7fa6ae59c640 1 --2- 172.21.15.139:0/901643794 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x556f337292a0 0x556f3372a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.170+0000 7fa695ffb640 10 client.15543 ms_handle_connect on v2:172.21.15.176:6838/3912805068 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.170+0000 7fa6ae59c640 1 --2- 172.21.15.139:0/901643794 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x556f337292a0 0x556f3372a710 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8274bb3bf18fe30e server_cookie=bc3fd3ee93e95e60 in_seq=0 out_seq=0 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.170+0000 7fa695ffb640 1 -- 172.21.15.139:0/901643794 <== mds.1 v2:172.21.15.176:6838/3912805068 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f33681600 con 0x556f337292a0 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.170+0000 7fa695ffb640 10 client.15543 handle_command_reply: tid=0 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.170+0000 7fa6b6f67740 1 client.15543 shutdown 2024-09-16T01:40:45.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.170+0000 7fa6b6f67740 1 -- 172.21.15.139:0/901643794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370e2f0 msgr2=0x556f33715780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:45.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.170+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/901643794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370e2f0 0x556f33715780 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa6a402a4c0 tx=0x7fa6a4009520 comp rx=0 tx=0).stop 2024-09-16T01:40:45.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.170+0000 7fa6b6f67740 1 -- 172.21.15.139:0/901643794 shutdown_connections 2024-09-16T01:40:45.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.170+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/901643794 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x556f337292a0 0x556f3372a710 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.170+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/901643794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f33718bb0 0x556f33715cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.170+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/901643794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370e2f0 0x556f33715780 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.170+0000 7fa695ffb640 0 client.15543 ms_handle_reset on v2:172.21.15.176:6838/3912805068 2024-09-16T01:40:45.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.170+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/901643794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f3370d9d0 0x556f33718430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.170+0000 7fa6b6f67740 1 -- 172.21.15.139:0/901643794 >> 172.21.15.139:0/901643794 conn(0x556f33709f60 msgr2=0x556f3370fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:45.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.171+0000 7fa6b6f67740 1 -- 172.21.15.139:0/901643794 shutdown_connections 2024-09-16T01:40:45.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.171+0000 7fa6b6f67740 1 -- 172.21.15.139:0/901643794 wait complete. 2024-09-16T01:40:45.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.171+0000 7fa6b6f67740 20 client.15543 trim_cache size 0 max 16384 2024-09-16T01:40:45.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.188+0000 7fa6b6f67740 1 Processor -- start 2024-09-16T01:40:45.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.188+0000 7fa6b6f67740 1 -- start start 2024-09-16T01:40:45.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.188+0000 7fa6b6f67740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370d9d0 0x556f33728bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.188+0000 7fa6b6f67740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f3370e2f0 0x556f33729970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.188+0000 7fa6b6f67740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f33718bb0 0x556f33708ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.188+0000 7fa6b6f67740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556f33728a50 con 0x556f3370d9d0 2024-09-16T01:40:45.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.188+0000 7fa6b6f67740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556f33729120 con 0x556f33718bb0 2024-09-16T01:40:45.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.188+0000 7fa6b6f67740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556f337297f0 con 0x556f3370e2f0 2024-09-16T01:40:45.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.189+0000 7fa6ae59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370d9d0 0x556f33728bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.189+0000 7fa6ae59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370d9d0 0x556f33728bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33072/0 (socket says 172.21.15.139:33072) 2024-09-16T01:40:45.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.189+0000 7fa6ae59c640 1 -- 172.21.15.139:0/1777046248 learned_addr learned my addr 172.21.15.139:0/1777046248 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:45.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.189+0000 7fa6ae59c640 1 -- 172.21.15.139:0/1777046248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f3370e2f0 msgr2=0x556f33729970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:45.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.189+0000 7fa6add9b640 1 --2- 172.21.15.139:0/1777046248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f3370e2f0 0x556f33729970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.189+0000 7fa6aed9d640 1 --2- 172.21.15.139:0/1777046248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f33718bb0 0x556f33708ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.189+0000 7fa6ae59c640 1 --2- 172.21.15.139:0/1777046248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f3370e2f0 0x556f33729970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.189+0000 7fa6ae59c640 1 -- 172.21.15.139:0/1777046248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f33718bb0 msgr2=0x556f33708ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:45.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.189+0000 7fa6ae59c640 1 --2- 172.21.15.139:0/1777046248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f33718bb0 0x556f33708ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.189+0000 7fa6ae59c640 1 -- 172.21.15.139:0/1777046248 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f33709a00 con 0x556f3370d9d0 2024-09-16T01:40:45.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.189+0000 7fa6ae59c640 1 --2- 172.21.15.139:0/1777046248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370d9d0 0x556f33728bd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00058c0 tx=0x7fa6a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b074a01ab051a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:45.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.189+0000 7fa697fff640 1 -- 172.21.15.139:0/1777046248 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0020040 con 0x556f3370d9d0 2024-09-16T01:40:45.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.189+0000 7fa697fff640 1 -- 172.21.15.139:0/1777046248 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a009c320 con 0x556f3370d9d0 2024-09-16T01:40:45.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.189+0000 7fa697fff640 1 -- 172.21.15.139:0/1777046248 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0098290 con 0x556f3370d9d0 2024-09-16T01:40:45.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1777046248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370d9d0 msgr2=0x556f33728bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:45.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/1777046248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370d9d0 0x556f33728bd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00058c0 tx=0x7fa6a0000bc0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1777046248 shutdown_connections 2024-09-16T01:40:45.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/1777046248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f33718bb0 0x556f33708ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/1777046248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f3370e2f0 0x556f33729970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/1777046248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370d9d0 0x556f33728bd0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1777046248 >> 172.21.15.139:0/1777046248 conn(0x556f33709f60 msgr2=0x556f337087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:45.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1777046248 shutdown_connections 2024-09-16T01:40:45.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1777046248 wait complete. 2024-09-16T01:40:45.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 -- start start 2024-09-16T01:40:45.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 Processor -- start 2024-09-16T01:40:45.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370d9d0 0x556f33732ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f3370e2f0 0x556f337330e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f33718bb0 0x556f3372df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556f3372a590 con 0x556f3370d9d0 2024-09-16T01:40:45.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556f33729ec0 con 0x556f3370e2f0 2024-09-16T01:40:45.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556f3372ac60 con 0x556f33718bb0 2024-09-16T01:40:45.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6b6f67740 10 client.0 _ll_register_callbacks cb 0x7fa6a00ccfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:45.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6ae59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370d9d0 0x556f33732ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6ae59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370d9d0 0x556f33732ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33078/0 (socket says 172.21.15.139:33078) 2024-09-16T01:40:45.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.190+0000 7fa6ae59c640 1 -- 172.21.15.139:0/1092422688 learned_addr learned my addr 172.21.15.139:0/1092422688 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:45.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa6aed9d640 1 --2- 172.21.15.139:0/1092422688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f33718bb0 0x556f3372df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa6add9b640 1 --2- 172.21.15.139:0/1092422688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f3370e2f0 0x556f337330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa6ae59c640 1 -- 172.21.15.139:0/1092422688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f33718bb0 msgr2=0x556f3372df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:45.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa6ae59c640 1 --2- 172.21.15.139:0/1092422688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f33718bb0 0x556f3372df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa6ae59c640 1 -- 172.21.15.139:0/1092422688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f3370e2f0 msgr2=0x556f337330e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:45.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa6ae59c640 1 --2- 172.21.15.139:0/1092422688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f3370e2f0 0x556f337330e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa6ae59c640 1 -- 172.21.15.139:0/1092422688 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3372e790 con 0x556f3370d9d0 2024-09-16T01:40:45.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa6aed9d640 1 --2- 172.21.15.139:0/1092422688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f33718bb0 0x556f3372df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:45.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa6ae59c640 1 --2- 172.21.15.139:0/1092422688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370d9d0 0x556f33732ba0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0098080 tx=0x7fa6a009ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ebbf05a6e7ba58 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:45.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa695ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:45.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa695ffb640 1 -- 172.21.15.139:0/1092422688 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0020030 con 0x556f3370d9d0 2024-09-16T01:40:45.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa695ffb640 1 -- 172.21.15.139:0/1092422688 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fa6a00cd3f0 con 0x556f3370d9d0 2024-09-16T01:40:45.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1092422688 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a009c320 con 0x556f3370d9d0 2024-09-16T01:40:45.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa695ffb640 1 -- 172.21.15.139:0/1092422688 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa6a0099fe0 con 0x556f3370d9d0 2024-09-16T01:40:45.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa6b6f67740 10 client.15549 fetch_fsmap learned FSMap version 29 2024-09-16T01:40:45.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1092422688 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556f337934e0 con 0x556f3370d9d0 2024-09-16T01:40:45.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa695ffb640 1 -- 172.21.15.139:0/1092422688 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a009c320 con 0x556f3370d9d0 2024-09-16T01:40:45.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa695ffb640 1 -- 172.21.15.139:0/1092422688 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa6a009aae0 con 0x556f3370d9d0 2024-09-16T01:40:45.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa6b6f67740 10 client.15549 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:40:45.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.191+0000 7fa6b6f67740 10 client.15549 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T01:40:45.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6b6f67740 20 client.15549 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:45.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/1092422688 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x556f337979b0 0x556f337966c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6b6f67740 4 client.15549 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:40:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1092422688 --> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] -- command(tid 0: {"prefix": "status"}) -- 0x556f3367bb10 con 0x556f337979b0 2024-09-16T01:40:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6add9b640 1 --2- 172.21.15.139:0/1092422688 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x556f337979b0 0x556f337966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa695ffb640 10 client.15549 ms_handle_connect on v2:172.21.15.176:6838/3912805068 2024-09-16T01:40:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6add9b640 1 --2- 172.21.15.139:0/1092422688 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x556f337979b0 0x556f337966c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d258de8e4796a8f2 server_cookie=a574ed632948fe86 in_seq=0 out_seq=0 2024-09-16T01:40:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa695ffb640 1 -- 172.21.15.139:0/1092422688 <== mds.1 v2:172.21.15.176:6838/3912805068 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556f3367bb10 con 0x556f337979b0 2024-09-16T01:40:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa695ffb640 10 client.15549 handle_command_reply: tid=0 2024-09-16T01:40:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6b6f67740 1 client.15549 shutdown 2024-09-16T01:40:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1092422688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370d9d0 msgr2=0x556f33732ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/1092422688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370d9d0 0x556f33732ba0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0098080 tx=0x7fa6a009ffd0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1092422688 shutdown_connections 2024-09-16T01:40:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/1092422688 >> [v2:172.21.15.176:6838/3912805068,v1:172.21.15.176:6839/3912805068] conn(0x556f337979b0 0x556f337966c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/1092422688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f33718bb0 0x556f3372df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa695ffb640 0 client.15549 ms_handle_reset on v2:172.21.15.176:6838/3912805068 2024-09-16T01:40:45.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/1092422688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f3370e2f0 0x556f337330e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6b6f67740 1 --2- 172.21.15.139:0/1092422688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556f3370d9d0 0x556f33732ba0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.192+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1092422688 >> 172.21.15.139:0/1092422688 conn(0x556f33709f60 msgr2=0x556f33709000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:45.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1092422688 shutdown_connections 2024-09-16T01:40:45.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b6f67740 1 -- 172.21.15.139:0/1092422688 wait complete. 2024-09-16T01:40:45.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b6f67740 20 client.15549 trim_cache size 0 max 16384 2024-09-16T01:40:45.210 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:40:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:40:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 1, 2024-09-16T01:40:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24454, 2024-09-16T01:40:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:40:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:40:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:40:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 128.67355976100001, 2024-09-16T01:40:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 29, 2024-09-16T01:40:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 86, 2024-09-16T01:40:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 85, 2024-09-16T01:40:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 184.34944116200001 2024-09-16T01:40:45.211 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:40:45.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b4b8e640 1 -- 172.21.15.139:0/4130429179 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa6980a34c0 msgr2=0x7fa6980a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:45.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b4b8e640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa6980a34c0 0x7fa6980a5980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8005fd0 tx=0x7fa6a8014b70 comp rx=0 tx=0).stop 2024-09-16T01:40:45.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b4b8e640 1 -- 172.21.15.139:0/4130429179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6b0100e20 msgr2=0x7fa6b019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:45.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b4b8e640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6b0100e20 0x7fa6b019a6f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000d950 tx=0x7fa6a000de10 comp rx=0 tx=0).stop 2024-09-16T01:40:45.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b4b8e640 1 -- 172.21.15.139:0/4130429179 shutdown_connections 2024-09-16T01:40:45.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b4b8e640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa6980a34c0 0x7fa6980a5980 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b4b8e640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6b0102ef0 0x7fa6b019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b4b8e640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6b0102570 0x7fa6b019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b4b8e640 1 --2- 172.21.15.139:0/4130429179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6b0100e20 0x7fa6b019a6f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:45.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b4b8e640 1 -- 172.21.15.139:0/4130429179 >> 172.21.15.139:0/4130429179 conn(0x7fa6b00fc7d0 msgr2=0x7fa6b0105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:45.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b4b8e640 1 -- 172.21.15.139:0/4130429179 shutdown_connections 2024-09-16T01:40:45.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:45.193+0000 7fa6b4b8e640 1 -- 172.21.15.139:0/4130429179 wait complete. 2024-09-16T01:40:45.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:45 smithi139 ceph-mon[31175]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-16T01:40:45.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:45 smithi176 ceph-mon[35324]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-16T01:40:45.950 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T01:40:46.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:45 smithi164 ceph-mon[35121]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-16T01:40:46.194 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:40:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.695+0000 7ff9ee15e640 1 -- 172.21.15.139:0/2361437944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e8101df0 msgr2=0x7ff9e81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.695+0000 7ff9ee15e640 1 --2- 172.21.15.139:0/2361437944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e8101df0 0x7ff9e81021d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009900 tx=0x7ff9d802a410 comp rx=0 tx=0).stop 2024-09-16T01:40:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.695+0000 7ff9ee15e640 1 -- 172.21.15.139:0/2361437944 shutdown_connections 2024-09-16T01:40:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.695+0000 7ff9ee15e640 1 --2- 172.21.15.139:0/2361437944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9e8106f20 0x7ff9e810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.695+0000 7ff9ee15e640 1 --2- 172.21.15.139:0/2361437944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff9e81027a0 0x7ff9e81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.695+0000 7ff9ee15e640 1 --2- 172.21.15.139:0/2361437944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e8101df0 0x7ff9e81021d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.695+0000 7ff9ee15e640 1 -- 172.21.15.139:0/2361437944 >> 172.21.15.139:0/2361437944 conn(0x7ff9e80fd8b0 msgr2=0x7ff9e80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.695+0000 7ff9ee15e640 1 -- 172.21.15.139:0/2361437944 shutdown_connections 2024-09-16T01:40:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.695+0000 7ff9ee15e640 1 -- 172.21.15.139:0/2361437944 wait complete. 2024-09-16T01:40:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.696+0000 7ff9ee15e640 1 Processor -- start 2024-09-16T01:40:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.696+0000 7ff9ee15e640 1 -- start start 2024-09-16T01:40:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9ee15e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff9e8101df0 0x7ff9e81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9ee15e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e81027a0 0x7ff9e8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9ee15e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9e8106f20 0x7ff9e819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9ee15e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e8111f80 con 0x7ff9e81027a0 2024-09-16T01:40:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9ee15e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e8111e00 con 0x7ff9e8106f20 2024-09-16T01:40:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9ee15e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e8112100 con 0x7ff9e8101df0 2024-09-16T01:40:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9dffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e81027a0 0x7ff9e8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9dffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e81027a0 0x7ff9e8198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33104/0 (socket says 172.21.15.139:33104) 2024-09-16T01:40:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9dffff640 1 -- 172.21.15.139:0/2594242469 learned_addr learned my addr 172.21.15.139:0/2594242469 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9e77fe640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff9e8101df0 0x7ff9e81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9e7fff640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9e8106f20 0x7ff9e819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9dffff640 1 -- 172.21.15.139:0/2594242469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff9e8101df0 msgr2=0x7ff9e81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9dffff640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff9e8101df0 0x7ff9e81983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9dffff640 1 -- 172.21.15.139:0/2594242469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9e8106f20 msgr2=0x7ff9e819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9dffff640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9e8106f20 0x7ff9e819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9dffff640 1 -- 172.21.15.139:0/2594242469 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e819d370 con 0x7ff9e81027a0 2024-09-16T01:40:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9e7fff640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9e8106f20 0x7ff9e819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9dffff640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e81027a0 0x7ff9e8198900 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000c970 tx=0x7ff9d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9e57fa640 1 -- 172.21.15.139:0/2594242469 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0007bb0 con 0x7ff9e81027a0 2024-09-16T01:40:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9e57fa640 1 -- 172.21.15.139:0/2594242469 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d0007d50 con 0x7ff9e81027a0 2024-09-16T01:40:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9ee15e640 1 -- 172.21.15.139:0/2594242469 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e819d660 con 0x7ff9e81027a0 2024-09-16T01:40:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9e57fa640 1 -- 172.21.15.139:0/2594242469 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000f450 con 0x7ff9e81027a0 2024-09-16T01:40:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.697+0000 7ff9ee15e640 1 -- 172.21.15.139:0/2594242469 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e81a4f40 con 0x7ff9e81027a0 2024-09-16T01:40:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.700+0000 7ff9e57fa640 1 -- 172.21.15.139:0/2594242469 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 23) ==== 138532+0+0 (secure 0 0 0) 0x7ff9d0005020 con 0x7ff9e81027a0 2024-09-16T01:40:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.700+0000 7ff9f0537740 1 Processor -- start 2024-09-16T01:40:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.700+0000 7ff9f0537740 1 -- start start 2024-09-16T01:40:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.700+0000 7ff9f0537740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b7fa9d0 0x55c83b7fadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.700+0000 7ff9f0537740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b7fb2f0 0x55c83b805670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.700+0000 7ff9f0537740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b805bb0 0x55c83b807fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.700+0000 7ff9f0537740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c83b761090 con 0x55c83b7fb2f0 2024-09-16T01:40:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.700+0000 7ff9f0537740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c83b708a70 con 0x55c83b805bb0 2024-09-16T01:40:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.700+0000 7ff9f0537740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c83b75d050 con 0x55c83b7fa9d0 2024-09-16T01:40:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.700+0000 7ff9e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b7fb2f0 0x55c83b805670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.700+0000 7ff9e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b7fb2f0 0x55c83b805670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33110/0 (socket says 172.21.15.139:33110) 2024-09-16T01:40:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.700+0000 7ff9e77fe640 1 -- 172.21.15.139:0/2827335092 learned_addr learned my addr 172.21.15.139:0/2827335092 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.700+0000 7ff9e57fa640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff9d40a34c0 0x7ff9d40a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9dffff640 1 --2- 172.21.15.139:0/2827335092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b805bb0 0x55c83b807fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9e7fff640 1 --2- 172.21.15.139:0/2827335092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b7fa9d0 0x55c83b7fadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9e7fff640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff9d40a34c0 0x7ff9d40a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9e77fe640 1 -- 172.21.15.139:0/2827335092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b7fa9d0 msgr2=0x55c83b7fadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9e77fe640 1 --2- 172.21.15.139:0/2827335092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b7fa9d0 0x55c83b7fadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9e77fe640 1 -- 172.21.15.139:0/2827335092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b805bb0 msgr2=0x55c83b807fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9e57fa640 1 -- 172.21.15.139:0/2594242469 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7ff9d00cffd0 con 0x7ff9e81027a0 2024-09-16T01:40:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9e77fe640 1 --2- 172.21.15.139:0/2827335092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b805bb0 0x55c83b807fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9e77fe640 1 -- 172.21.15.139:0/2827335092 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83b715130 con 0x55c83b7fb2f0 2024-09-16T01:40:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9dffff640 1 --2- 172.21.15.139:0/2827335092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b805bb0 0x55c83b807fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:40:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9e77fe640 1 --2- 172.21.15.139:0/2827335092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b7fb2f0 0x55c83b805670 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009900 tx=0x7ff9d80316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37344407b42151e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9e7fff640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff9d40a34c0 0x7ff9d40a5980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00124b0 tx=0x7ff9e0005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9ddffb640 1 -- 172.21.15.139:0/2827335092 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8040070 con 0x55c83b7fb2f0 2024-09-16T01:40:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9ddffb640 1 -- 172.21.15.139:0/2827335092 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d8005020 con 0x55c83b7fb2f0 2024-09-16T01:40:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.701+0000 7ff9ddffb640 1 -- 172.21.15.139:0/2827335092 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8035040 con 0x55c83b7fb2f0 2024-09-16T01:40:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.702+0000 7ff9f0537740 1 -- 172.21.15.139:0/2827335092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b7fb2f0 msgr2=0x55c83b805670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.702+0000 7ff9f0537740 1 --2- 172.21.15.139:0/2827335092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b7fb2f0 0x55c83b805670 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009900 tx=0x7ff9d80316e0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.702+0000 7ff9f0537740 1 -- 172.21.15.139:0/2827335092 shutdown_connections 2024-09-16T01:40:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.702+0000 7ff9f0537740 1 --2- 172.21.15.139:0/2827335092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b805bb0 0x55c83b807fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.702+0000 7ff9f0537740 1 --2- 172.21.15.139:0/2827335092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b7fb2f0 0x55c83b805670 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.702+0000 7ff9f0537740 1 --2- 172.21.15.139:0/2827335092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b7fa9d0 0x55c83b7fadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.702+0000 7ff9f0537740 1 -- 172.21.15.139:0/2827335092 >> 172.21.15.139:0/2827335092 conn(0x55c83b7f6f60 msgr2=0x55c83b7f7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.702+0000 7ff9f0537740 1 -- 172.21.15.139:0/2827335092 shutdown_connections 2024-09-16T01:40:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.702+0000 7ff9f0537740 1 -- 172.21.15.139:0/2827335092 wait complete. 2024-09-16T01:40:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.702+0000 7ff9f0537740 1 -- start start 2024-09-16T01:40:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9f0537740 1 Processor -- start 2024-09-16T01:40:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9f0537740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fa9d0 0x55c83b805380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9f0537740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b7fb2f0 0x55c83b8026d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9f0537740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b805bb0 0x55c83b802c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9f0537740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c83b808780 con 0x55c83b805bb0 2024-09-16T01:40:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9f0537740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c83b71bf20 con 0x55c83b7fa9d0 2024-09-16T01:40:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9f0537740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c83b808900 con 0x55c83b7fb2f0 2024-09-16T01:40:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9f0537740 10 client.0 _ll_register_callbacks cb 0x55c83b6e5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:46.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b805bb0 0x55c83b802c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b805bb0 0x55c83b802c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33116/0 (socket says 172.21.15.139:33116) 2024-09-16T01:40:46.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9e7fff640 1 -- 172.21.15.139:0/1883703828 learned_addr learned my addr 172.21.15.139:0/1883703828 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9dffff640 1 --2- 172.21.15.139:0/1883703828 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b7fb2f0 0x55c83b8026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9e77fe640 1 --2- 172.21.15.139:0/1883703828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fa9d0 0x55c83b805380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9e7fff640 1 -- 172.21.15.139:0/1883703828 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b7fb2f0 msgr2=0x55c83b8026d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9e7fff640 1 --2- 172.21.15.139:0/1883703828 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b7fb2f0 0x55c83b8026d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9e7fff640 1 -- 172.21.15.139:0/1883703828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fa9d0 msgr2=0x55c83b805380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9e7fff640 1 --2- 172.21.15.139:0/1883703828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fa9d0 0x55c83b805380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.703+0000 7ff9e7fff640 1 -- 172.21.15.139:0/1883703828 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83b4d22b0 con 0x55c83b805bb0 2024-09-16T01:40:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9e7fff640 1 --2- 172.21.15.139:0/1883703828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b805bb0 0x55c83b802c10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000dc30 tx=0x7ff9e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab090fff13e3d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9c6ffd640 1 -- 172.21.15.139:0/1883703828 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e001d610 con 0x55c83b805bb0 2024-09-16T01:40:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9c6ffd640 1 -- 172.21.15.139:0/1883703828 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7ff9e001d7b0 con 0x55c83b805bb0 2024-09-16T01:40:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9f0537740 1 -- 172.21.15.139:0/1883703828 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d8005020 con 0x55c83b805bb0 2024-09-16T01:40:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9c6ffd640 1 -- 172.21.15.139:0/1883703828 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9e002e050 con 0x55c83b805bb0 2024-09-16T01:40:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9f0537740 10 client.15561 fetch_fsmap learned FSMap version 29 2024-09-16T01:40:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9f0537740 1 -- 172.21.15.139:0/1883703828 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c83b814010 con 0x55c83b805bb0 2024-09-16T01:40:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9c6ffd640 1 -- 172.21.15.139:0/1883703828 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d8005020 con 0x55c83b805bb0 2024-09-16T01:40:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9c6ffd640 1 -- 172.21.15.139:0/1883703828 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff9e0019070 con 0x55c83b805bb0 2024-09-16T01:40:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9f0537740 10 client.15561 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:40:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9f0537740 10 client.15561 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T01:40:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9f0537740 20 client.15561 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9f0537740 1 --2- 172.21.15.139:0/1883703828 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c83b816670 0x55c83b817ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9f0537740 4 client.15561 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:40:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.704+0000 7ff9f0537740 1 -- 172.21.15.139:0/1883703828 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c83b76e600 con 0x55c83b816670 2024-09-16T01:40:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.705+0000 7ff9e77fe640 1 --2- 172.21.15.139:0/1883703828 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c83b816670 0x55c83b817ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.705+0000 7ff9c6ffd640 10 client.15561 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.705+0000 7ff9e77fe640 1 --2- 172.21.15.139:0/1883703828 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c83b816670 0x55c83b817ae0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=789d33b0cee64821 server_cookie=23b2a05939b08ca4 in_seq=0 out_seq=0 2024-09-16T01:40:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9c6ffd640 1 -- 172.21.15.139:0/1883703828 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c83b76e600 con 0x55c83b816670 2024-09-16T01:40:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9c6ffd640 10 client.15561 handle_command_reply: tid=0 2024-09-16T01:40:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9f0537740 1 client.15561 shutdown 2024-09-16T01:40:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9f0537740 1 -- 172.21.15.139:0/1883703828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b805bb0 msgr2=0x55c83b802c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9f0537740 1 --2- 172.21.15.139:0/1883703828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b805bb0 0x55c83b802c10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000dc30 tx=0x7ff9e0004a40 comp rx=0 tx=0).stop 2024-09-16T01:40:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9f0537740 1 -- 172.21.15.139:0/1883703828 shutdown_connections 2024-09-16T01:40:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9f0537740 1 --2- 172.21.15.139:0/1883703828 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c83b816670 0x55c83b817ae0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9f0537740 1 --2- 172.21.15.139:0/1883703828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b805bb0 0x55c83b802c10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9f0537740 1 --2- 172.21.15.139:0/1883703828 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b7fb2f0 0x55c83b8026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9c6ffd640 0 client.15561 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9f0537740 1 --2- 172.21.15.139:0/1883703828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fa9d0 0x55c83b805380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9f0537740 1 -- 172.21.15.139:0/1883703828 >> 172.21.15.139:0/1883703828 conn(0x55c83b7f6f60 msgr2=0x55c83b7fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9f0537740 1 -- 172.21.15.139:0/1883703828 shutdown_connections 2024-09-16T01:40:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9f0537740 1 -- 172.21.15.139:0/1883703828 wait complete. 2024-09-16T01:40:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.706+0000 7ff9f0537740 20 client.15561 trim_cache size 0 max 16384 2024-09-16T01:40:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.723+0000 7ff9f0537740 1 Processor -- start 2024-09-16T01:40:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9f0537740 1 -- start start 2024-09-16T01:40:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9f0537740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fa9d0 0x55c83b817ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9f0537740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b7fb2f0 0x55c83b8201b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9f0537740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b805bb0 0x55c83b802690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9f0537740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c83b815e20 con 0x55c83b805bb0 2024-09-16T01:40:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9f0537740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c83b8164f0 con 0x55c83b7fa9d0 2024-09-16T01:40:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9f0537740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c83b816bc0 con 0x55c83b7fb2f0 2024-09-16T01:40:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b805bb0 0x55c83b802690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b805bb0 0x55c83b802690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33118/0 (socket says 172.21.15.139:33118) 2024-09-16T01:40:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9e7fff640 1 -- 172.21.15.139:0/3359306332 learned_addr learned my addr 172.21.15.139:0/3359306332 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9e77fe640 1 --2- 172.21.15.139:0/3359306332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fa9d0 0x55c83b817ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9e7fff640 1 -- 172.21.15.139:0/3359306332 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b7fb2f0 msgr2=0x55c83b8201b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9e7fff640 1 --2- 172.21.15.139:0/3359306332 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b7fb2f0 0x55c83b8201b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9e7fff640 1 -- 172.21.15.139:0/3359306332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fa9d0 msgr2=0x55c83b817ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9e7fff640 1 --2- 172.21.15.139:0/3359306332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fa9d0 0x55c83b817ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9e7fff640 1 -- 172.21.15.139:0/3359306332 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83b7f6d90 con 0x55c83b805bb0 2024-09-16T01:40:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9e77fe640 1 --2- 172.21.15.139:0/3359306332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fa9d0 0x55c83b817ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9e7fff640 1 --2- 172.21.15.139:0/3359306332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b805bb0 0x55c83b802690 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000dc00 tx=0x7ff9e000a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d39d35a49f809b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9dd7fa640 1 -- 172.21.15.139:0/3359306332 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e001bdd0 con 0x55c83b805bb0 2024-09-16T01:40:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9dd7fa640 1 -- 172.21.15.139:0/3359306332 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e0016070 con 0x55c83b805bb0 2024-09-16T01:40:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.724+0000 7ff9dd7fa640 1 -- 172.21.15.139:0/3359306332 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0005290 con 0x55c83b805bb0 2024-09-16T01:40:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.725+0000 7ff9f0537740 1 -- 172.21.15.139:0/3359306332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b805bb0 msgr2=0x55c83b802690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.725+0000 7ff9f0537740 1 --2- 172.21.15.139:0/3359306332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b805bb0 0x55c83b802690 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000dc00 tx=0x7ff9e000a4c0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.725+0000 7ff9f0537740 1 -- 172.21.15.139:0/3359306332 shutdown_connections 2024-09-16T01:40:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.725+0000 7ff9f0537740 1 --2- 172.21.15.139:0/3359306332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b805bb0 0x55c83b802690 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.725+0000 7ff9f0537740 1 --2- 172.21.15.139:0/3359306332 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b7fb2f0 0x55c83b8201b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.725+0000 7ff9f0537740 1 --2- 172.21.15.139:0/3359306332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fa9d0 0x55c83b817ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.725+0000 7ff9f0537740 1 -- 172.21.15.139:0/3359306332 >> 172.21.15.139:0/3359306332 conn(0x55c83b7f6f60 msgr2=0x55c83b801c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.725+0000 7ff9f0537740 1 -- 172.21.15.139:0/3359306332 shutdown_connections 2024-09-16T01:40:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.725+0000 7ff9f0537740 1 -- 172.21.15.139:0/3359306332 wait complete. 2024-09-16T01:40:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.725+0000 7ff9f0537740 1 -- start start 2024-09-16T01:40:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9f0537740 1 Processor -- start 2024-09-16T01:40:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9f0537740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b7fa9d0 0x55c83b820420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9f0537740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fb2f0 0x55c83b81d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9f0537740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b805bb0 0x55c83b81dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9f0537740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c83b817960 con 0x55c83b7fa9d0 2024-09-16T01:40:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9f0537740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c83b817290 con 0x55c83b7fb2f0 2024-09-16T01:40:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9f0537740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c83b818030 con 0x55c83b805bb0 2024-09-16T01:40:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9f0537740 10 client.0 _ll_register_callbacks cb 0x7ff9e000d8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b7fa9d0 0x55c83b820420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b7fa9d0 0x55c83b820420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33132/0 (socket says 172.21.15.139:33132) 2024-09-16T01:40:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9e77fe640 1 -- 172.21.15.139:0/2262240897 learned_addr learned my addr 172.21.15.139:0/2262240897 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9dffff640 1 --2- 172.21.15.139:0/2262240897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fb2f0 0x55c83b81d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9e7fff640 1 --2- 172.21.15.139:0/2262240897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b805bb0 0x55c83b81dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9e77fe640 1 -- 172.21.15.139:0/2262240897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b805bb0 msgr2=0x55c83b81dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9e77fe640 1 --2- 172.21.15.139:0/2262240897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b805bb0 0x55c83b81dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9e77fe640 1 -- 172.21.15.139:0/2262240897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fb2f0 msgr2=0x55c83b81d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9e77fe640 1 --2- 172.21.15.139:0/2262240897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fb2f0 0x55c83b81d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9e77fe640 1 -- 172.21.15.139:0/2262240897 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83b81e580 con 0x55c83b7fa9d0 2024-09-16T01:40:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9e7fff640 1 --2- 172.21.15.139:0/2262240897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b805bb0 0x55c83b81dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9e77fe640 1 --2- 172.21.15.139:0/2262240897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b7fa9d0 0x55c83b820420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x55c83b7f7dc0 tx=0x7ff9d802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc520e816d27444b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.726+0000 7ff9c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.727+0000 7ff9c6ffd640 1 -- 172.21.15.139:0/2262240897 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8054060 con 0x55c83b7fa9d0 2024-09-16T01:40:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.727+0000 7ff9c6ffd640 1 -- 172.21.15.139:0/2262240897 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7ff9d803c650 con 0x55c83b7fa9d0 2024-09-16T01:40:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.727+0000 7ff9f0537740 1 -- 172.21.15.139:0/2262240897 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e0016070 con 0x55c83b7fa9d0 2024-09-16T01:40:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.727+0000 7ff9c6ffd640 1 -- 172.21.15.139:0/2262240897 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9d805b050 con 0x55c83b7fa9d0 2024-09-16T01:40:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.727+0000 7ff9f0537740 10 client.15567 fetch_fsmap learned FSMap version 29 2024-09-16T01:40:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.727+0000 7ff9f0537740 1 -- 172.21.15.139:0/2262240897 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c83b884c60 con 0x55c83b7fa9d0 2024-09-16T01:40:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.727+0000 7ff9c6ffd640 1 -- 172.21.15.139:0/2262240897 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e0016070 con 0x55c83b7fa9d0 2024-09-16T01:40:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.727+0000 7ff9c6ffd640 1 -- 172.21.15.139:0/2262240897 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff9d804d070 con 0x55c83b7fa9d0 2024-09-16T01:40:46.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.727+0000 7ff9f0537740 10 client.15567 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:40:46.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.727+0000 7ff9f0537740 10 client.15567 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T01:40:46.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.727+0000 7ff9f0537740 20 client.15567 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:46.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.727+0000 7ff9f0537740 1 --2- 172.21.15.139:0/2262240897 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c83b889360 0x55c83b888070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:46.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.727+0000 7ff9f0537740 4 client.15567 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T01:40:46.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.727+0000 7ff9f0537740 1 -- 172.21.15.139:0/2262240897 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "status"}) -- 0x55c83b768b10 con 0x55c83b889360 2024-09-16T01:40:46.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9dffff640 1 --2- 172.21.15.139:0/2262240897 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c83b889360 0x55c83b888070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:46.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9dffff640 1 --2- 172.21.15.139:0/2262240897 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c83b889360 0x55c83b888070 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6653057a0fd0801 server_cookie=175962317d19244 in_seq=0 out_seq=0 2024-09-16T01:40:46.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9c6ffd640 10 client.15567 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:46.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9c6ffd640 1 -- 172.21.15.139:0/2262240897 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c83b768b10 con 0x55c83b889360 2024-09-16T01:40:46.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9c6ffd640 10 client.15567 handle_command_reply: tid=0 2024-09-16T01:40:46.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9f0537740 1 client.15567 shutdown 2024-09-16T01:40:46.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9f0537740 1 -- 172.21.15.139:0/2262240897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b7fa9d0 msgr2=0x55c83b820420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:46.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9f0537740 1 --2- 172.21.15.139:0/2262240897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b7fa9d0 0x55c83b820420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x55c83b7f7dc0 tx=0x7ff9d802a410 comp rx=0 tx=0).stop 2024-09-16T01:40:46.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9f0537740 1 -- 172.21.15.139:0/2262240897 shutdown_connections 2024-09-16T01:40:46.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9f0537740 1 --2- 172.21.15.139:0/2262240897 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c83b889360 0x55c83b888070 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9f0537740 1 --2- 172.21.15.139:0/2262240897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c83b805bb0 0x55c83b81dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9f0537740 1 --2- 172.21.15.139:0/2262240897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c83b7fb2f0 0x55c83b81d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9f0537740 1 --2- 172.21.15.139:0/2262240897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c83b7fa9d0 0x55c83b820420 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9f0537740 1 -- 172.21.15.139:0/2262240897 >> 172.21.15.139:0/2262240897 conn(0x55c83b7f6f60 msgr2=0x55c83b802690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:46.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9c6ffd640 0 client.15567 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:46.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.728+0000 7ff9f0537740 1 -- 172.21.15.139:0/2262240897 shutdown_connections 2024-09-16T01:40:46.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.729+0000 7ff9f0537740 1 -- 172.21.15.139:0/2262240897 wait complete. 2024-09-16T01:40:46.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.729+0000 7ff9f0537740 20 client.15567 trim_cache size 0 max 16384 2024-09-16T01:40:46.748 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:40:46.748 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "1d90dd3a-73cb-11ef-bceb-c7b262605968", 2024-09-16T01:40:46.748 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-09-16T01:40:46.748 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 24470, 2024-09-16T01:40:46.748 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T01:40:46.748 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T01:40:46.748 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T01:40:46.748 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 164.215051587, 2024-09-16T01:40:46.748 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 29, 2024-09-16T01:40:46.748 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 86, 2024-09-16T01:40:46.749 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 85, 2024-09-16T01:40:46.749 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 176.27449911599999 2024-09-16T01:40:46.749 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:40:46.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.729+0000 7ff9ee15e640 1 -- 172.21.15.139:0/2594242469 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff9d40a34c0 msgr2=0x7ff9d40a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:46.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.729+0000 7ff9ee15e640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff9d40a34c0 0x7ff9d40a5980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00124b0 tx=0x7ff9e0005f90 comp rx=0 tx=0).stop 2024-09-16T01:40:46.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.729+0000 7ff9ee15e640 1 -- 172.21.15.139:0/2594242469 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e81027a0 msgr2=0x7ff9e8198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:46.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.729+0000 7ff9ee15e640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e81027a0 0x7ff9e8198900 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000c970 tx=0x7ff9d000ce30 comp rx=0 tx=0).stop 2024-09-16T01:40:46.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.729+0000 7ff9ee15e640 1 -- 172.21.15.139:0/2594242469 shutdown_connections 2024-09-16T01:40:46.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.729+0000 7ff9ee15e640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff9d40a34c0 0x7ff9d40a5980 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.729+0000 7ff9ee15e640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9e8106f20 0x7ff9e819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.729+0000 7ff9ee15e640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e81027a0 0x7ff9e8198900 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.729+0000 7ff9ee15e640 1 --2- 172.21.15.139:0/2594242469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff9e8101df0 0x7ff9e81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:46.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.729+0000 7ff9ee15e640 1 -- 172.21.15.139:0/2594242469 >> 172.21.15.139:0/2594242469 conn(0x7ff9e80fd8b0 msgr2=0x7ff9e80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:46.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.729+0000 7ff9ee15e640 1 -- 172.21.15.139:0/2594242469 shutdown_connections 2024-09-16T01:40:46.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:40:46.729+0000 7ff9ee15e640 1 -- 172.21.15.139:0/2594242469 wait complete. 2024-09-16T01:40:47.579 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-16T01:40:47.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:47 smithi139 ceph-mon[31175]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-09-16T01:40:47.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:47 smithi176 ceph-mon[35324]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-09-16T01:40:48.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:47 smithi164 ceph-mon[35121]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-09-16T01:40:49.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:49 smithi139 ceph-mon[31175]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-16T01:40:49.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:49 smithi176 ceph-mon[35324]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-16T01:40:50.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:49 smithi164 ceph-mon[35121]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-16T01:40:51.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:51 smithi139 ceph-mon[31175]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-16T01:40:51.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:51 smithi176 ceph-mon[35324]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-16T01:40:52.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:51 smithi164 ceph-mon[35121]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-09-16T01:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.192+0000 7f00487b7640 1 -- 172.21.15.139:0/2194153529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0040101350 msgr2=0x7f004010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.192+0000 7f00487b7640 1 --2- 172.21.15.139:0/2194153529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0040101350 0x7f004010d880 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0038009cd0 tx=0x7f003802a8e0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.192+0000 7f00487b7640 1 -- 172.21.15.139:0/2194153529 shutdown_connections 2024-09-16T01:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.192+0000 7f00487b7640 1 --2- 172.21.15.139:0/2194153529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0040101350 0x7f004010d880 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.192+0000 7f00487b7640 1 --2- 172.21.15.139:0/2194153529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00401009b0 0x7f0040100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.192+0000 7f00487b7640 1 --2- 172.21.15.139:0/2194153529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00401069c0 0x7f0040106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.192+0000 7f00487b7640 1 -- 172.21.15.139:0/2194153529 >> 172.21.15.139:0/2194153529 conn(0x7f0040077990 msgr2=0x7f00400fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.192+0000 7f00487b7640 1 -- 172.21.15.139:0/2194153529 shutdown_connections 2024-09-16T01:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.192+0000 7f00487b7640 1 -- 172.21.15.139:0/2194153529 wait complete. 2024-09-16T01:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f00487b7640 1 Processor -- start 2024-09-16T01:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f00487b7640 1 -- start start 2024-09-16T01:40:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f00487b7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00401009b0 0x7f0040199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f00487b7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0040101350 0x7f0040199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f00487b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00401069c0 0x7f00401a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f00487b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0040110e00 con 0x7f00401069c0 2024-09-16T01:40:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f00487b7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0040110c80 con 0x7f0040101350 2024-09-16T01:40:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f00487b7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0040110f80 con 0x7f00401009b0 2024-09-16T01:40:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f0046d54640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00401069c0 0x7f00401a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f0046d54640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00401069c0 0x7f00401a41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33148/0 (socket says 172.21.15.139:33148) 2024-09-16T01:40:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f0046d54640 1 -- 172.21.15.139:0/2058573994 learned_addr learned my addr 172.21.15.139:0/2058573994 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f0046553640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00401009b0 0x7f0040199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f0045d52640 1 --2- 172.21.15.139:0/2058573994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0040101350 0x7f0040199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f0046d54640 1 -- 172.21.15.139:0/2058573994 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00401009b0 msgr2=0x7f0040199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f0046d54640 1 --2- 172.21.15.139:0/2058573994 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00401009b0 0x7f0040199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f0046d54640 1 -- 172.21.15.139:0/2058573994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0040101350 msgr2=0x7f0040199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f0046d54640 1 --2- 172.21.15.139:0/2058573994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0040101350 0x7f0040199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.193+0000 7f0046d54640 1 -- 172.21.15.139:0/2058573994 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00401a4710 con 0x7f00401069c0 2024-09-16T01:40:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.194+0000 7f0046d54640 1 --2- 172.21.15.139:0/2058573994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00401069c0 0x7f00401a41d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0038005cc0 tx=0x7f0038032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.194+0000 7f003f7fe640 1 -- 172.21.15.139:0/2058573994 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038004040 con 0x7f00401069c0 2024-09-16T01:40:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.194+0000 7f003f7fe640 1 -- 172.21.15.139:0/2058573994 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00380046c0 con 0x7f00401069c0 2024-09-16T01:40:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.194+0000 7f00487b7640 1 -- 172.21.15.139:0/2058573994 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00401a48e0 con 0x7f00401069c0 2024-09-16T01:40:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.194+0000 7f003f7fe640 1 -- 172.21.15.139:0/2058573994 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038030e00 con 0x7f00401069c0 2024-09-16T01:40:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.194+0000 7f00487b7640 1 -- 172.21.15.139:0/2058573994 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00401a4df0 con 0x7f00401069c0 2024-09-16T01:40:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.196+0000 7f003f7fe640 1 -- 172.21.15.139:0/2058573994 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 23) ==== 138532+0+0 (secure 0 0 0) 0x7f00380305a0 con 0x7f00401069c0 2024-09-16T01:40:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.196+0000 7f004b62b740 1 Processor -- start 2024-09-16T01:40:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f004b62b740 1 -- start start 2024-09-16T01:40:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f004b62b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 0x558e3c445ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f004b62b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c446410 0x558e3c450770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f004b62b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c4530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f004b62b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558e3c3aa2f0 con 0x558e3c450cb0 2024-09-16T01:40:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f004b62b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558e3c352270 con 0x558e3c445af0 2024-09-16T01:40:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f004b62b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558e3c3a62b0 con 0x558e3c446410 2024-09-16T01:40:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f003f7fe640 1 --2- 172.21.15.139:0/2058573994 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f00280a3580 0x7f00280a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f0045d52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c4530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f0046553640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c446410 0x558e3c450770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f0045d52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c4530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33154/0 (socket says 172.21.15.139:33154) 2024-09-16T01:40:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f0045d52640 1 -- 172.21.15.139:0/3600173016 learned_addr learned my addr 172.21.15.139:0/3600173016 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f0045d52640 1 --2- 172.21.15.139:0/3600173016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 0x558e3c445ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f0046d54640 1 --2- 172.21.15.139:0/2058573994 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f00280a3580 0x7f00280a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f003f7fe640 1 -- 172.21.15.139:0/2058573994 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f003802e030 con 0x7f00401069c0 2024-09-16T01:40:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f0045d52640 1 -- 172.21.15.139:0/3600173016 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c446410 msgr2=0x558e3c450770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f0045d52640 1 --2- 172.21.15.139:0/3600173016 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c446410 0x558e3c450770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f0045d52640 1 -- 172.21.15.139:0/3600173016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 msgr2=0x558e3c445ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.197+0000 7f0045d52640 1 --2- 172.21.15.139:0/3600173016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 0x558e3c445ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.198+0000 7f0045d52640 1 -- 172.21.15.139:0/3600173016 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e3c351340 con 0x558e3c450cb0 2024-09-16T01:40:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.198+0000 7f0046553640 1 --2- 172.21.15.139:0/3600173016 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c446410 0x558e3c450770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:40:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.198+0000 7f0045d52640 1 --2- 172.21.15.139:0/3600173016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c4530a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f003400b6c0 tx=0x7f003400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401f185bf2752551 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.198+0000 7f0046d54640 1 --2- 172.21.15.139:0/2058573994 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f00280a3580 0x7f00280a5a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f00380eeca0 tx=0x7f0038090550 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.198+0000 7f003cff9640 1 -- 172.21.15.139:0/3600173016 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034012d00 con 0x558e3c450cb0 2024-09-16T01:40:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.198+0000 7f003cff9640 1 -- 172.21.15.139:0/3600173016 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0034010770 con 0x558e3c450cb0 2024-09-16T01:40:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.198+0000 7f003cff9640 1 -- 172.21.15.139:0/3600173016 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034003850 con 0x558e3c450cb0 2024-09-16T01:40:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.199+0000 7f004b62b740 1 -- 172.21.15.139:0/3600173016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 msgr2=0x558e3c4530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.199+0000 7f004b62b740 1 --2- 172.21.15.139:0/3600173016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c4530a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f003400b6c0 tx=0x7f003400b650 comp rx=0 tx=0).stop 2024-09-16T01:40:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.199+0000 7f004b62b740 1 -- 172.21.15.139:0/3600173016 shutdown_connections 2024-09-16T01:40:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.199+0000 7f004b62b740 1 --2- 172.21.15.139:0/3600173016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c4530a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.199+0000 7f004b62b740 1 --2- 172.21.15.139:0/3600173016 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c446410 0x558e3c450770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.199+0000 7f004b62b740 1 --2- 172.21.15.139:0/3600173016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 0x558e3c445ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.199+0000 7f004b62b740 1 -- 172.21.15.139:0/3600173016 >> 172.21.15.139:0/3600173016 conn(0x558e3c442060 msgr2=0x558e3c442440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.199+0000 7f004b62b740 1 -- 172.21.15.139:0/3600173016 shutdown_connections 2024-09-16T01:40:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.199+0000 7f004b62b740 1 -- 172.21.15.139:0/3600173016 wait complete. 2024-09-16T01:40:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.199+0000 7f004b62b740 1 -- start start 2024-09-16T01:40:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.199+0000 7f004b62b740 1 Processor -- start 2024-09-16T01:40:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f004b62b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c445af0 0x558e3c44dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f004b62b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c446410 0x558e3c44e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f004b62b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c44e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f004b62b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558e3c453880 con 0x558e3c450cb0 2024-09-16T01:40:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f004b62b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558e3c333290 con 0x558e3c446410 2024-09-16T01:40:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f004b62b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558e3c453a00 con 0x558e3c445af0 2024-09-16T01:40:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f004b62b740 10 client.0 _ll_register_callbacks cb 0x558e3c364140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0045d52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c44e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0045d52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c44e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33166/0 (socket says 172.21.15.139:33166) 2024-09-16T01:40:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0045d52640 1 -- 172.21.15.139:0/384211843 learned_addr learned my addr 172.21.15.139:0/384211843 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0046553640 1 --2- 172.21.15.139:0/384211843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c446410 0x558e3c44e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0045d52640 1 --2- 172.21.15.139:0/384211843 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c445af0 0x558e3c44dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0045d52640 1 -- 172.21.15.139:0/384211843 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c445af0 msgr2=0x558e3c44dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0045d52640 1 --2- 172.21.15.139:0/384211843 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c445af0 0x558e3c44dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0045d52640 1 -- 172.21.15.139:0/384211843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c446410 msgr2=0x558e3c44e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0045d52640 1 --2- 172.21.15.139:0/384211843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c446410 0x558e3c44e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0045d52640 1 -- 172.21.15.139:0/384211843 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e3c0bd210 con 0x558e3c450cb0 2024-09-16T01:40:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0046553640 1 --2- 172.21.15.139:0/384211843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c446410 0x558e3c44e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0045d52640 1 --2- 172.21.15.139:0/384211843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c44e6c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0034010740 tx=0x7f0034017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed45e4675a18c1f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0025ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0025ffb640 1 -- 172.21.15.139:0/384211843 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003401f470 con 0x558e3c450cb0 2024-09-16T01:40:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0025ffb640 1 -- 172.21.15.139:0/384211843 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f003401f610 con 0x558e3c450cb0 2024-09-16T01:40:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f004b62b740 1 -- 172.21.15.139:0/384211843 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0034010770 con 0x558e3c450cb0 2024-09-16T01:40:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f0025ffb640 1 -- 172.21.15.139:0/384211843 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0034030050 con 0x558e3c450cb0 2024-09-16T01:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.200+0000 7f004b62b740 10 client.15579 fetch_fsmap learned FSMap version 29 2024-09-16T01:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.201+0000 7f004b62b740 1 -- 172.21.15.139:0/384211843 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558e3c189620 con 0x558e3c450cb0 2024-09-16T01:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.201+0000 7f0025ffb640 1 -- 172.21.15.139:0/384211843 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0034010770 con 0x558e3c450cb0 2024-09-16T01:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.201+0000 7f0025ffb640 1 -- 172.21.15.139:0/384211843 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f003402bab0 con 0x558e3c450cb0 2024-09-16T01:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.201+0000 7f004b62b740 10 client.15579 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.201+0000 7f004b62b740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.201+0000 7f004b62b740 20 client.15579 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.201+0000 7f004b62b740 1 --2- 172.21.15.139:0/384211843 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e3c460bd0 0x558e3c462fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.201+0000 7f004b62b740 4 client.15579 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.201+0000 7f004b62b740 1 -- 172.21.15.139:0/384211843 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e3c3b7a20 con 0x558e3c460bd0 2024-09-16T01:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.201+0000 7f0046553640 1 --2- 172.21.15.139:0/384211843 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e3c460bd0 0x558e3c462fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.202+0000 7f0025ffb640 10 client.15579 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.202+0000 7f0046553640 1 --2- 172.21.15.139:0/384211843 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e3c460bd0 0x558e3c462fc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e0f5cfc6d2f2cb server_cookie=f79c275d7b4c09a6 in_seq=0 out_seq=0 2024-09-16T01:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.202+0000 7f0025ffb640 1 -- 172.21.15.139:0/384211843 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e3c3b7a20 con 0x558e3c460bd0 2024-09-16T01:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.202+0000 7f0025ffb640 10 client.15579 handle_command_reply: tid=0 2024-09-16T01:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.202+0000 7f004b62b740 1 client.15579 shutdown 2024-09-16T01:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.202+0000 7f004b62b740 1 -- 172.21.15.139:0/384211843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 msgr2=0x558e3c44e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.202+0000 7f004b62b740 1 --2- 172.21.15.139:0/384211843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c44e6c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0034010740 tx=0x7f0034017040 comp rx=0 tx=0).stop 2024-09-16T01:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.202+0000 7f004b62b740 1 -- 172.21.15.139:0/384211843 shutdown_connections 2024-09-16T01:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.202+0000 7f004b62b740 1 --2- 172.21.15.139:0/384211843 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e3c460bd0 0x558e3c462fc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.202+0000 7f004b62b740 1 --2- 172.21.15.139:0/384211843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c44e6c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.202+0000 7f0025ffb640 0 client.15579 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.202+0000 7f004b62b740 1 --2- 172.21.15.139:0/384211843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c446410 0x558e3c44e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.203+0000 7f004b62b740 1 --2- 172.21.15.139:0/384211843 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c445af0 0x558e3c44dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.203+0000 7f004b62b740 1 -- 172.21.15.139:0/384211843 >> 172.21.15.139:0/384211843 conn(0x558e3c442060 msgr2=0x558e3c447d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.203+0000 7f004b62b740 1 -- 172.21.15.139:0/384211843 shutdown_connections 2024-09-16T01:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.203+0000 7f004b62b740 1 -- 172.21.15.139:0/384211843 wait complete. 2024-09-16T01:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.203+0000 7f004b62b740 20 client.15579 trim_cache size 0 max 16384 2024-09-16T01:40:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.217+0000 7f004b62b740 1 Processor -- start 2024-09-16T01:40:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.217+0000 7f004b62b740 1 -- start start 2024-09-16T01:40:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.217+0000 7f004b62b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 0x558e3c448560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.217+0000 7f004b62b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c446410 0x558e3c44e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.217+0000 7f004b62b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c450cb0 0x558e3c468700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.217+0000 7f004b62b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f00401aeeb0 con 0x558e3c446410 2024-09-16T01:40:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.217+0000 7f004b62b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00401b09f0 con 0x558e3c445af0 2024-09-16T01:40:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.217+0000 7f004b62b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f00401af580 con 0x558e3c450cb0 2024-09-16T01:40:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.217+0000 7f0046553640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c446410 0x558e3c44e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.217+0000 7f0046553640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c446410 0x558e3c44e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33172/0 (socket says 172.21.15.139:33172) 2024-09-16T01:40:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.217+0000 7f0046553640 1 -- 172.21.15.139:0/4101590430 learned_addr learned my addr 172.21.15.139:0/4101590430 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.218+0000 7f0045d52640 1 --2- 172.21.15.139:0/4101590430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c450cb0 0x558e3c468700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.218+0000 7f0046553640 1 -- 172.21.15.139:0/4101590430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c450cb0 msgr2=0x558e3c468700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.218+0000 7f0045d52640 1 --2- 172.21.15.139:0/4101590430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 0x558e3c448560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.218+0000 7f0046553640 1 --2- 172.21.15.139:0/4101590430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c450cb0 0x558e3c468700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.218+0000 7f0046553640 1 -- 172.21.15.139:0/4101590430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 msgr2=0x558e3c448560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.218+0000 7f0046553640 1 --2- 172.21.15.139:0/4101590430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 0x558e3c448560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.218+0000 7f0046553640 1 -- 172.21.15.139:0/4101590430 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e3c45d660 con 0x558e3c446410 2024-09-16T01:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.218+0000 7f0045d52640 1 --2- 172.21.15.139:0/4101590430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 0x558e3c448560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.218+0000 7f0046553640 1 --2- 172.21.15.139:0/4101590430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c446410 0x558e3c44e4f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x558e3c442ec0 tx=0x7f00300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f803d385aa1096 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.218+0000 7f0027fff640 1 -- 172.21.15.139:0/4101590430 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030003e10 con 0x558e3c446410 2024-09-16T01:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.218+0000 7f0027fff640 1 -- 172.21.15.139:0/4101590430 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0030010910 con 0x558e3c446410 2024-09-16T01:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.218+0000 7f0027fff640 1 -- 172.21.15.139:0/4101590430 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003001ea70 con 0x558e3c446410 2024-09-16T01:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 -- 172.21.15.139:0/4101590430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c446410 msgr2=0x558e3c44e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 --2- 172.21.15.139:0/4101590430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c446410 0x558e3c44e4f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x558e3c442ec0 tx=0x7f00300046c0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 -- 172.21.15.139:0/4101590430 shutdown_connections 2024-09-16T01:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 --2- 172.21.15.139:0/4101590430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c450cb0 0x558e3c468700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 --2- 172.21.15.139:0/4101590430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c446410 0x558e3c44e4f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 --2- 172.21.15.139:0/4101590430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 0x558e3c448560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 -- 172.21.15.139:0/4101590430 >> 172.21.15.139:0/4101590430 conn(0x558e3c442060 msgr2=0x558e3c44fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 -- 172.21.15.139:0/4101590430 shutdown_connections 2024-09-16T01:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 -- 172.21.15.139:0/4101590430 wait complete. 2024-09-16T01:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 -- start start 2024-09-16T01:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 Processor -- start 2024-09-16T01:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 0x558e3c458c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c446410 0x558e3c4591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c449ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f00401b0320 con 0x558e3c450cb0 2024-09-16T01:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00401afc50 con 0x558e3c445af0 2024-09-16T01:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f00401ace90 con 0x558e3c446410 2024-09-16T01:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f004b62b740 10 client.0 _ll_register_callbacks cb 0x7f003400bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.219+0000 7f0045d52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c449ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0045d52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c449ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33182/0 (socket says 172.21.15.139:33182) 2024-09-16T01:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0045d52640 1 -- 172.21.15.139:0/178193089 learned_addr learned my addr 172.21.15.139:0/178193089 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0046553640 1 --2- 172.21.15.139:0/178193089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c446410 0x558e3c4591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0045d52640 1 --2- 172.21.15.139:0/178193089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 0x558e3c458c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0045d52640 1 -- 172.21.15.139:0/178193089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c446410 msgr2=0x558e3c4591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0045d52640 1 --2- 172.21.15.139:0/178193089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c446410 0x558e3c4591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0045d52640 1 -- 172.21.15.139:0/178193089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 msgr2=0x558e3c458c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0045d52640 1 --2- 172.21.15.139:0/178193089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 0x558e3c458c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0045d52640 1 -- 172.21.15.139:0/178193089 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e3c44bad0 con 0x558e3c450cb0 2024-09-16T01:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0045d52640 1 --2- 172.21.15.139:0/178193089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c449ed0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f003402b4d0 tx=0x7f003400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da145ef976a749c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0025ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0025ffb640 1 -- 172.21.15.139:0/178193089 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003403b030 con 0x558e3c450cb0 2024-09-16T01:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0025ffb640 1 -- 172.21.15.139:0/178193089 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f0034010e00 con 0x558e3c450cb0 2024-09-16T01:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f004b62b740 1 -- 172.21.15.139:0/178193089 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0030010910 con 0x558e3c450cb0 2024-09-16T01:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0025ffb640 1 -- 172.21.15.139:0/178193089 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0034047050 con 0x558e3c450cb0 2024-09-16T01:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f004b62b740 10 client.15585 fetch_fsmap learned FSMap version 29 2024-09-16T01:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f004b62b740 1 -- 172.21.15.139:0/178193089 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558e3c4caa90 con 0x558e3c450cb0 2024-09-16T01:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.220+0000 7f0025ffb640 1 -- 172.21.15.139:0/178193089 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0030010910 con 0x558e3c450cb0 2024-09-16T01:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.221+0000 7f0025ffb640 1 -- 172.21.15.139:0/178193089 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f003402a7e0 con 0x558e3c450cb0 2024-09-16T01:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.221+0000 7f004b62b740 10 client.15585 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.221+0000 7f004b62b740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.221+0000 7f004b62b740 20 client.15585 populate_metadata read hostname 'smithi139' 2024-09-16T01:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.221+0000 7f004b62b740 1 --2- 172.21.15.139:0/178193089 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e3c460bd0 0x558e3c4cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.221+0000 7f004b62b740 4 client.15585 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:40:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.221+0000 7f004b62b740 1 -- 172.21.15.139:0/178193089 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e3c3b1d70 con 0x558e3c460bd0 2024-09-16T01:40:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.221+0000 7f0046553640 1 --2- 172.21.15.139:0/178193089 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e3c460bd0 0x558e3c4cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:40:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.221+0000 7f0046553640 1 --2- 172.21.15.139:0/178193089 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e3c460bd0 0x558e3c4cd200 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126fc2e8050b63a0 server_cookie=f751e6efe6dd6133 in_seq=0 out_seq=0 2024-09-16T01:40:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.221+0000 7f0025ffb640 10 client.15585 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f0025ffb640 1 -- 172.21.15.139:0/178193089 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e3c3b1d70 con 0x558e3c460bd0 2024-09-16T01:40:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f0025ffb640 10 client.15585 handle_command_reply: tid=0 2024-09-16T01:40:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f004b62b740 1 client.15585 shutdown 2024-09-16T01:40:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f004b62b740 1 -- 172.21.15.139:0/178193089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 msgr2=0x558e3c449ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f004b62b740 1 --2- 172.21.15.139:0/178193089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c449ed0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f003402b4d0 tx=0x7f003400bf50 comp rx=0 tx=0).stop 2024-09-16T01:40:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f004b62b740 1 -- 172.21.15.139:0/178193089 shutdown_connections 2024-09-16T01:40:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f004b62b740 1 --2- 172.21.15.139:0/178193089 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e3c460bd0 0x558e3c4cd200 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f004b62b740 1 --2- 172.21.15.139:0/178193089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e3c450cb0 0x558e3c449ed0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f004b62b740 1 --2- 172.21.15.139:0/178193089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e3c446410 0x558e3c4591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f004b62b740 1 --2- 172.21.15.139:0/178193089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3c445af0 0x558e3c458c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f0025ffb640 0 client.15585 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:40:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f004b62b740 1 -- 172.21.15.139:0/178193089 >> 172.21.15.139:0/178193089 conn(0x558e3c442060 msgr2=0x558e3c4409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f004b62b740 1 -- 172.21.15.139:0/178193089 shutdown_connections 2024-09-16T01:40:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f004b62b740 1 -- 172.21.15.139:0/178193089 wait complete. 2024-09-16T01:40:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.229+0000 7f004b62b740 20 client.15585 trim_cache size 0 max 16384 2024-09-16T01:40:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:40:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:40:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:40:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:40:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.230+0000 7f00487b7640 1 -- 172.21.15.139:0/2058573994 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f00280a3580 msgr2=0x7f00280a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.230+0000 7f00487b7640 1 --2- 172.21.15.139:0/2058573994 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f00280a3580 0x7f00280a5a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f00380eeca0 tx=0x7f0038090550 comp rx=0 tx=0).stop 2024-09-16T01:40:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.230+0000 7f00487b7640 1 -- 172.21.15.139:0/2058573994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00401069c0 msgr2=0x7f00401a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:40:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.230+0000 7f00487b7640 1 --2- 172.21.15.139:0/2058573994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00401069c0 0x7f00401a41d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0038005cc0 tx=0x7f0038032f40 comp rx=0 tx=0).stop 2024-09-16T01:40:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.230+0000 7f00487b7640 1 -- 172.21.15.139:0/2058573994 shutdown_connections 2024-09-16T01:40:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.230+0000 7f00487b7640 1 --2- 172.21.15.139:0/2058573994 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f00280a3580 0x7f00280a5a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.230+0000 7f00487b7640 1 --2- 172.21.15.139:0/2058573994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00401069c0 0x7f00401a41d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.230+0000 7f00487b7640 1 --2- 172.21.15.139:0/2058573994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0040101350 0x7f0040199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.230+0000 7f00487b7640 1 --2- 172.21.15.139:0/2058573994 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00401009b0 0x7f0040199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:40:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.230+0000 7f00487b7640 1 -- 172.21.15.139:0/2058573994 >> 172.21.15.139:0/2058573994 conn(0x7f0040077990 msgr2=0x7f0040103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:40:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.230+0000 7f00487b7640 1 -- 172.21.15.139:0/2058573994 shutdown_connections 2024-09-16T01:40:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:40:52.230+0000 7f00487b7640 1 -- 172.21.15.139:0/2058573994 wait complete. 2024-09-16T01:40:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:53 smithi139 ceph-mon[31175]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-09-16T01:40:53.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:53 smithi176 ceph-mon[35324]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-09-16T01:40:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:53 smithi164 ceph-mon[35121]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-09-16T01:40:55.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:55 smithi139 ceph-mon[31175]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-16T01:40:55.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:55 smithi176 ceph-mon[35324]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-16T01:40:56.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:55 smithi164 ceph-mon[35121]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-16T01:40:57.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:57 smithi139 ceph-mon[31175]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-16T01:40:57.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:57 smithi176 ceph-mon[35324]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-16T01:40:58.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:57 smithi164 ceph-mon[35121]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-16T01:40:59.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:40:59 smithi139 ceph-mon[31175]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T01:40:59.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:40:59 smithi176 ceph-mon[35324]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T01:41:00.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:40:59 smithi164 ceph-mon[35121]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T01:41:00.423 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:41:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:41:00.144+0000 7fe1a3182640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:41:00.423 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:41:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:41:00.144+0000 7fe1a3182640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:41:00.423 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:41:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:41:00.144+0000 7fe1a3182640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:41:00.423 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:41:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:41:00.144+0000 7fe1a3182640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:41:00.423 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:41:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:41:00.144+0000 7fe1a3182640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T01:41:01.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:01 smithi139 ceph-mon[31175]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-16T01:41:01.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:01 smithi176 ceph-mon[35324]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-16T01:41:02.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:01 smithi164 ceph-mon[35121]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-16T01:41:02.581 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.2 2024-09-16T01:41:02.581 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:41:02.866 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:41:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.361+0000 7f42fcd1c640 1 -- 172.21.15.139:0/2616606126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f8102d30 msgr2=0x7f42f8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:03.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.361+0000 7f42fcd1c640 1 --2- 172.21.15.139:0/2616606126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f8102d30 0x7f42f8103110 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009930 tx=0x7f42e802a420 comp rx=0 tx=0).stop 2024-09-16T01:41:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.361+0000 7f42fcd1c640 1 -- 172.21.15.139:0/2616606126 shutdown_connections 2024-09-16T01:41:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.361+0000 7f42fcd1c640 1 --2- 172.21.15.139:0/2616606126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42f81082a0 0x7f42f810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.361+0000 7f42fcd1c640 1 --2- 172.21.15.139:0/2616606126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42f81036e0 0x7f42f8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.361+0000 7f42fcd1c640 1 --2- 172.21.15.139:0/2616606126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f8102d30 0x7f42f8103110 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.361+0000 7f42fcd1c640 1 -- 172.21.15.139:0/2616606126 >> 172.21.15.139:0/2616606126 conn(0x7f42f80fc7d0 msgr2=0x7f42f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.361+0000 7f42fcd1c640 1 -- 172.21.15.139:0/2616606126 shutdown_connections 2024-09-16T01:41:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.361+0000 7f42fcd1c640 1 -- 172.21.15.139:0/2616606126 wait complete. 2024-09-16T01:41:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.362+0000 7f42fcd1c640 1 Processor -- start 2024-09-16T01:41:03.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.362+0000 7f42fcd1c640 1 -- start start 2024-09-16T01:41:03.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.362+0000 7f42fcd1c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42f8102d30 0x7f42f81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:03.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42fcd1c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f81036e0 0x7f42f8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:03.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42fcd1c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42f81082a0 0x7f42f819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:03.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42fcd1c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42f81120a0 con 0x7f42f81036e0 2024-09-16T01:41:03.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42fcd1c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8111f20 con 0x7f42f8102d30 2024-09-16T01:41:03.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42fcd1c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8112220 con 0x7f42f81082a0 2024-09-16T01:41:03.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42effff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f81036e0 0x7f42f8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:03.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42effff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f81036e0 0x7f42f8198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36638/0 (socket says 172.21.15.139:36638) 2024-09-16T01:41:03.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42effff640 1 -- 172.21.15.139:0/3445617019 learned_addr learned my addr 172.21.15.139:0/3445617019 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:03.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42f6d9d640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42f81082a0 0x7f42f819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:03.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42f659c640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42f8102d30 0x7f42f81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:03.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42effff640 1 -- 172.21.15.139:0/3445617019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42f81082a0 msgr2=0x7f42f819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:03.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42effff640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42f81082a0 0x7f42f819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:03.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42effff640 1 -- 172.21.15.139:0/3445617019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42f8102d30 msgr2=0x7f42f81985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:03.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42effff640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42f8102d30 0x7f42f81985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:03.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42effff640 1 -- 172.21.15.139:0/3445617019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f819d570 con 0x7f42f81036e0 2024-09-16T01:41:03.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42f659c640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42f8102d30 0x7f42f81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:41:03.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42effff640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f81036e0 0x7f42f8198b00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f42e000c970 tx=0x7f42e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:03.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42ef7fe640 1 -- 172.21.15.139:0/3445617019 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0007bb0 con 0x7f42f81036e0 2024-09-16T01:41:03.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42ef7fe640 1 -- 172.21.15.139:0/3445617019 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42e0007d50 con 0x7f42f81036e0 2024-09-16T01:41:03.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.363+0000 7f42fcd1c640 1 -- 172.21.15.139:0/3445617019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f819d860 con 0x7f42f81036e0 2024-09-16T01:41:03.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.364+0000 7f42ef7fe640 1 -- 172.21.15.139:0/3445617019 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e000f450 con 0x7f42f81036e0 2024-09-16T01:41:03.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.364+0000 7f42fcd1c640 1 -- 172.21.15.139:0/3445617019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f81a5140 con 0x7f42f81036e0 2024-09-16T01:41:03.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.365+0000 7f42fcd1c640 1 -- 172.21.15.139:0/3445617019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f806a850 con 0x7f42f81036e0 2024-09-16T01:41:03.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.366+0000 7f42ef7fe640 1 -- 172.21.15.139:0/3445617019 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 24) ==== 138589+0+0 (secure 0 0 0) 0x7f42e0004f60 con 0x7f42f81036e0 2024-09-16T01:41:03.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.366+0000 7f42ef7fe640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f42e40a34d0 0x7f42e40a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:03.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.366+0000 7f42f659c640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f42e40a34d0 0x7f42e40a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:03.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.366+0000 7f42ef7fe640 1 -- 172.21.15.139:0/3445617019 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f42e00d0180 con 0x7f42f81036e0 2024-09-16T01:41:03.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.367+0000 7f42f659c640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f42e40a34d0 0x7f42e40a5990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42e8000c00 tx=0x7f42e8032c70 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:03.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.371+0000 7f42ef7fe640 1 -- 172.21.15.139:0/3445617019 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42e0014030 con 0x7f42f81036e0 2024-09-16T01:41:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.598+0000 7f42fcd1c640 1 -- 172.21.15.139:0/3445617019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42f810a2b0 con 0x7f42f81036e0 2024-09-16T01:41:03.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.599+0000 7f42ef7fe640 1 -- 172.21.15.139:0/3445617019 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11995 (secure 0 0 0) 0x7f42e009c560 con 0x7f42f81036e0 2024-09-16T01:41:03.601 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:41:03.601 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":29,"btime":"2024-09-16T01:40:22:367807+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:40:22.317639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:41:03.602 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 29 2024-09-16T01:41:03.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.601+0000 7f42fcd1c640 1 -- 172.21.15.139:0/3445617019 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f42e40a34d0 msgr2=0x7f42e40a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:03.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.601+0000 7f42fcd1c640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f42e40a34d0 0x7f42e40a5990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42e8000c00 tx=0x7f42e8032c70 comp rx=0 tx=0).stop 2024-09-16T01:41:03.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.601+0000 7f42fcd1c640 1 -- 172.21.15.139:0/3445617019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f81036e0 msgr2=0x7f42f8198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:03.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.601+0000 7f42fcd1c640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f81036e0 0x7f42f8198b00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f42e000c970 tx=0x7f42e000ce30 comp rx=0 tx=0).stop 2024-09-16T01:41:03.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.601+0000 7f42fcd1c640 1 -- 172.21.15.139:0/3445617019 shutdown_connections 2024-09-16T01:41:03.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.601+0000 7f42fcd1c640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f42e40a34d0 0x7f42e40a5990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:03.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.601+0000 7f42fcd1c640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42f81082a0 0x7f42f819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:03.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.601+0000 7f42fcd1c640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f81036e0 0x7f42f8198b00 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:03.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.601+0000 7f42fcd1c640 1 --2- 172.21.15.139:0/3445617019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42f8102d30 0x7f42f81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:03.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.601+0000 7f42fcd1c640 1 -- 172.21.15.139:0/3445617019 >> 172.21.15.139:0/3445617019 conn(0x7f42f80fc7d0 msgr2=0x7f42f8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:03.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.602+0000 7f42fcd1c640 1 -- 172.21.15.139:0/3445617019 shutdown_connections 2024-09-16T01:41:03.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:03.602+0000 7f42fcd1c640 1 -- 172.21.15.139:0/3445617019 wait complete. 2024-09-16T01:41:03.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:03 smithi139 ceph-mon[31175]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-16T01:41:03.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:03 smithi139 ceph-mon[31175]: mgrmap e24: x(active, since 38s), standbys: z, y 2024-09-16T01:41:03.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:03 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3445617019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:41:03.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:03 smithi176 ceph-mon[35324]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-16T01:41:03.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:03 smithi176 ceph-mon[35324]: mgrmap e24: x(active, since 38s), standbys: z, y 2024-09-16T01:41:03.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:03 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3445617019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:41:04.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:03 smithi164 ceph-mon[35121]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-16T01:41:04.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:03 smithi164 ceph-mon[35121]: mgrmap e24: x(active, since 38s), standbys: z, y 2024-09-16T01:41:04.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:03 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3445617019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:41:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.321+0000 7f033a065640 1 -- 172.21.15.139:0/1135671952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0334101350 msgr2=0x7f033410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.321+0000 7f033a065640 1 --2- 172.21.15.139:0/1135671952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0334101350 0x7f033410d880 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f032c009cd0 tx=0x7f032c02a8e0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.321+0000 7f033a065640 1 -- 172.21.15.139:0/1135671952 shutdown_connections 2024-09-16T01:41:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.321+0000 7f033a065640 1 --2- 172.21.15.139:0/1135671952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0334101350 0x7f033410d880 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.321+0000 7f033a065640 1 --2- 172.21.15.139:0/1135671952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03341009b0 0x7f0334100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.321+0000 7f033a065640 1 --2- 172.21.15.139:0/1135671952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03341069c0 0x7f0334106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.321+0000 7f033a065640 1 -- 172.21.15.139:0/1135671952 >> 172.21.15.139:0/1135671952 conn(0x7f0334077990 msgr2=0x7f03340fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.321+0000 7f033a065640 1 -- 172.21.15.139:0/1135671952 shutdown_connections 2024-09-16T01:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.321+0000 7f033a065640 1 -- 172.21.15.139:0/1135671952 wait complete. 2024-09-16T01:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.321+0000 7f033a065640 1 Processor -- start 2024-09-16T01:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f033a065640 1 -- start start 2024-09-16T01:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f033a065640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03341009b0 0x7f03341982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f033a065640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0334101350 0x7f0334198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f033a065640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03341069c0 0x7f033419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f033a065640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0334110e00 con 0x7f0334101350 2024-09-16T01:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f033a065640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0334110c80 con 0x7f03341009b0 2024-09-16T01:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f033a065640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0334110f80 con 0x7f03341069c0 2024-09-16T01:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f0332ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0334101350 0x7f0334198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f0332ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0334101350 0x7f0334198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36654/0 (socket says 172.21.15.139:36654) 2024-09-16T01:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f0332ffd640 1 -- 172.21.15.139:0/71526412 learned_addr learned my addr 172.21.15.139:0/71526412 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f0333fff640 1 --2- 172.21.15.139:0/71526412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03341069c0 0x7f033419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f03337fe640 1 --2- 172.21.15.139:0/71526412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03341009b0 0x7f03341982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f0332ffd640 1 -- 172.21.15.139:0/71526412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03341069c0 msgr2=0x7f033419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f0332ffd640 1 --2- 172.21.15.139:0/71526412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03341069c0 0x7f033419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f0332ffd640 1 -- 172.21.15.139:0/71526412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03341009b0 msgr2=0x7f03341982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f0332ffd640 1 --2- 172.21.15.139:0/71526412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03341009b0 0x7f03341982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f0332ffd640 1 -- 172.21.15.139:0/71526412 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033419d2c0 con 0x7f0334101350 2024-09-16T01:41:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f0332ffd640 1 --2- 172.21.15.139:0/71526412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0334101350 0x7f0334198810 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f032800d770 tx=0x7f032800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.322+0000 7f0330ff9640 1 -- 172.21.15.139:0/71526412 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328017070 con 0x7f0334101350 2024-09-16T01:41:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.323+0000 7f0330ff9640 1 -- 172.21.15.139:0/71526412 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0328005500 con 0x7f0334101350 2024-09-16T01:41:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.323+0000 7f033a065640 1 -- 172.21.15.139:0/71526412 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033419d5b0 con 0x7f0334101350 2024-09-16T01:41:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.323+0000 7f0330ff9640 1 -- 172.21.15.139:0/71526412 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328004e00 con 0x7f0334101350 2024-09-16T01:41:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.323+0000 7f033a065640 1 -- 172.21.15.139:0/71526412 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03341a4e40 con 0x7f0334101350 2024-09-16T01:41:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.326+0000 7f0330ff9640 1 -- 172.21.15.139:0/71526412 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 24) ==== 138589+0+0 (secure 0 0 0) 0x7f0328004fa0 con 0x7f0334101350 2024-09-16T01:41:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.326+0000 7f033ce05740 1 Processor -- start 2024-09-16T01:41:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.326+0000 7f033ce05740 1 -- start start 2024-09-16T01:41:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.326+0000 7f033ce05740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d6001a10 0x55c8d6001df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.326+0000 7f033ce05740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d6010b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.326+0000 7f033ce05740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d60110c0 0x55c8d60134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.326+0000 7f033ce05740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c8d5f662f0 con 0x55c8d6002330 2024-09-16T01:41:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.326+0000 7f033ce05740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c8d5f0e270 con 0x55c8d6001a10 2024-09-16T01:41:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.326+0000 7f033ce05740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c8d5f622b0 con 0x55c8d60110c0 2024-09-16T01:41:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.326+0000 7f03337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d6010b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.326+0000 7f03337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d6010b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36656/0 (socket says 172.21.15.139:36656) 2024-09-16T01:41:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.326+0000 7f03337fe640 1 -- 172.21.15.139:0/829874685 learned_addr learned my addr 172.21.15.139:0/829874685 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.326+0000 7f0333fff640 1 --2- 172.21.15.139:0/829874685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d6001a10 0x55c8d6001df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.326+0000 7f0332ffd640 1 --2- 172.21.15.139:0/829874685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d60110c0 0x55c8d60134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.327+0000 7f0330ff9640 1 --2- 172.21.15.139:0/71526412 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f031c0a3580 0x7f031c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.327+0000 7f03337fe640 1 -- 172.21.15.139:0/829874685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d60110c0 msgr2=0x55c8d60134b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.327+0000 7f03337fe640 1 --2- 172.21.15.139:0/829874685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d60110c0 0x55c8d60134b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.327+0000 7f03337fe640 1 -- 172.21.15.139:0/829874685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d6001a10 msgr2=0x55c8d6001df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.327+0000 7f03337fe640 1 --2- 172.21.15.139:0/829874685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d6001a10 0x55c8d6001df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.327+0000 7f03337fe640 1 -- 172.21.15.139:0/829874685 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8d5f0d340 con 0x55c8d6002330 2024-09-16T01:41:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.327+0000 7f0333fff640 1 --2- 172.21.15.139:0/71526412 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f031c0a3580 0x7f031c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.327+0000 7f0330ff9640 1 -- 172.21.15.139:0/71526412 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f03280cfe00 con 0x7f0334101350 2024-09-16T01:41:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.327+0000 7f03337fe640 1 --2- 172.21.15.139:0/829874685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d6010b80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f032400bc60 tx=0x7f032400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c42a658266dada server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.327+0000 7f0319ffb640 1 -- 172.21.15.139:0/829874685 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324015070 con 0x55c8d6002330 2024-09-16T01:41:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.327+0000 7f0319ffb640 1 -- 172.21.15.139:0/829874685 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0324011920 con 0x55c8d6002330 2024-09-16T01:41:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.327+0000 7f0319ffb640 1 -- 172.21.15.139:0/829874685 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03240107f0 con 0x55c8d6002330 2024-09-16T01:41:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.328+0000 7f033ce05740 1 -- 172.21.15.139:0/829874685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 msgr2=0x55c8d6010b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.328+0000 7f0333fff640 1 --2- 172.21.15.139:0/71526412 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f031c0a3580 0x7f031c0a5a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f032c004ab0 tx=0x7f032c0095c0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.328+0000 7f033ce05740 1 --2- 172.21.15.139:0/829874685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d6010b80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f032400bc60 tx=0x7f032400f730 comp rx=0 tx=0).stop 2024-09-16T01:41:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.328+0000 7f033ce05740 1 -- 172.21.15.139:0/829874685 shutdown_connections 2024-09-16T01:41:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.328+0000 7f033ce05740 1 --2- 172.21.15.139:0/829874685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d60110c0 0x55c8d60134b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.328+0000 7f033ce05740 1 --2- 172.21.15.139:0/829874685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d6010b80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.328+0000 7f033ce05740 1 --2- 172.21.15.139:0/829874685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d6001a10 0x55c8d6001df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.328+0000 7f033ce05740 1 -- 172.21.15.139:0/829874685 >> 172.21.15.139:0/829874685 conn(0x55c8d5ffe060 msgr2=0x55c8d5ffe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.328+0000 7f033ce05740 1 -- 172.21.15.139:0/829874685 shutdown_connections 2024-09-16T01:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.328+0000 7f033ce05740 1 -- 172.21.15.139:0/829874685 wait complete. 2024-09-16T01:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.328+0000 7f033ce05740 1 -- start start 2024-09-16T01:41:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f033ce05740 1 Processor -- start 2024-09-16T01:41:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f033ce05740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6001a10 0x55c8d600df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f033ce05740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d6002330 0x55c8d600e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f033ce05740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d60110c0 0x55c8d600e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f033ce05740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c8d6013c90 con 0x55c8d6001a10 2024-09-16T01:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f033ce05740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c8d5eef290 con 0x55c8d60110c0 2024-09-16T01:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f033ce05740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c8d6013e10 con 0x55c8d6002330 2024-09-16T01:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f033ce05740 10 client.0 _ll_register_callbacks cb 0x55c8d5f20140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f03337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6001a10 0x55c8d600df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f03337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6001a10 0x55c8d600df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36660/0 (socket says 172.21.15.139:36660) 2024-09-16T01:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f03337fe640 1 -- 172.21.15.139:0/4215295874 learned_addr learned my addr 172.21.15.139:0/4215295874 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f0332ffd640 1 --2- 172.21.15.139:0/4215295874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d6002330 0x55c8d600e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f0333fff640 1 --2- 172.21.15.139:0/4215295874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d60110c0 0x55c8d600e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f03337fe640 1 -- 172.21.15.139:0/4215295874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d6002330 msgr2=0x55c8d600e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f03337fe640 1 --2- 172.21.15.139:0/4215295874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d6002330 0x55c8d600e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f03337fe640 1 -- 172.21.15.139:0/4215295874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d60110c0 msgr2=0x55c8d600e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f03337fe640 1 --2- 172.21.15.139:0/4215295874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d60110c0 0x55c8d600e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f03337fe640 1 -- 172.21.15.139:0/4215295874 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8d5c79210 con 0x55c8d6001a10 2024-09-16T01:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f0333fff640 1 --2- 172.21.15.139:0/4215295874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d60110c0 0x55c8d600e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f03337fe640 1 --2- 172.21.15.139:0/4215295874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6001a10 0x55c8d600df60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0324005f70 tx=0x7f032400b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d04a619c46aeca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.329+0000 7f0312ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.330+0000 7f0312ffd640 1 -- 172.21.15.139:0/4215295874 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324028060 con 0x55c8d6001a10 2024-09-16T01:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.330+0000 7f0312ffd640 1 -- 172.21.15.139:0/4215295874 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f0324011ad0 con 0x55c8d6001a10 2024-09-16T01:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.330+0000 7f033ce05740 1 -- 172.21.15.139:0/4215295874 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0324011920 con 0x55c8d6001a10 2024-09-16T01:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.330+0000 7f0312ffd640 1 -- 172.21.15.139:0/4215295874 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f032402f050 con 0x55c8d6001a10 2024-09-16T01:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.330+0000 7f033ce05740 10 client.15609 fetch_fsmap learned FSMap version 29 2024-09-16T01:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.330+0000 7f033ce05740 1 -- 172.21.15.139:0/4215295874 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c8d5d45620 con 0x55c8d6001a10 2024-09-16T01:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.330+0000 7f0312ffd640 1 -- 172.21.15.139:0/4215295874 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0324011920 con 0x55c8d6001a10 2024-09-16T01:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.330+0000 7f0312ffd640 1 -- 172.21.15.139:0/4215295874 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f03240048c0 con 0x55c8d6001a10 2024-09-16T01:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.330+0000 7f033ce05740 10 client.15609 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.330+0000 7f033ce05740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.330+0000 7f033ce05740 20 client.15609 populate_metadata read hostname 'smithi139' 2024-09-16T01:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.330+0000 7f033ce05740 1 --2- 172.21.15.139:0/4215295874 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c8d6021010 0x55c8d6023400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.330+0000 7f033ce05740 4 client.15609 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.330+0000 7f033ce05740 1 -- 172.21.15.139:0/4215295874 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8d5f73a20 con 0x55c8d6021010 2024-09-16T01:41:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.331+0000 7f0332ffd640 1 --2- 172.21.15.139:0/4215295874 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c8d6021010 0x55c8d6023400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.331+0000 7f0332ffd640 1 --2- 172.21.15.139:0/4215295874 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c8d6021010 0x55c8d6023400 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b10db77cb2de824a server_cookie=7b52f0e0973e8ac9 in_seq=0 out_seq=0 2024-09-16T01:41:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.331+0000 7f0312ffd640 10 client.15609 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f0312ffd640 1 -- 172.21.15.139:0/4215295874 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8d5f73a20 con 0x55c8d6021010 2024-09-16T01:41:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f0312ffd640 10 client.15609 handle_command_reply: tid=0 2024-09-16T01:41:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f033ce05740 1 client.15609 shutdown 2024-09-16T01:41:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f033ce05740 1 -- 172.21.15.139:0/4215295874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6001a10 msgr2=0x55c8d600df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f033ce05740 1 --2- 172.21.15.139:0/4215295874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6001a10 0x55c8d600df60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0324005f70 tx=0x7f032400b9b0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f033ce05740 1 -- 172.21.15.139:0/4215295874 shutdown_connections 2024-09-16T01:41:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f033ce05740 1 --2- 172.21.15.139:0/4215295874 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c8d6021010 0x55c8d6023400 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f033ce05740 1 --2- 172.21.15.139:0/4215295874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d60110c0 0x55c8d600e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f0312ffd640 0 client.15609 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f033ce05740 1 --2- 172.21.15.139:0/4215295874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d6002330 0x55c8d600e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f033ce05740 1 --2- 172.21.15.139:0/4215295874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6001a10 0x55c8d600df60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f033ce05740 1 -- 172.21.15.139:0/4215295874 >> 172.21.15.139:0/4215295874 conn(0x55c8d5ffe060 msgr2=0x55c8d6003d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f033ce05740 1 -- 172.21.15.139:0/4215295874 shutdown_connections 2024-09-16T01:41:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f033ce05740 1 -- 172.21.15.139:0/4215295874 wait complete. 2024-09-16T01:41:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.332+0000 7f033ce05740 20 client.15609 trim_cache size 0 max 16384 2024-09-16T01:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f033ce05740 1 Processor -- start 2024-09-16T01:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f033ce05740 1 -- start start 2024-09-16T01:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f033ce05740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d6001a10 0x55c8d600d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f033ce05740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d60289a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f033ce05740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d60110c0 0x55c8d6028ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f033ce05740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f03341ace30 con 0x55c8d6002330 2024-09-16T01:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f033ce05740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f03341ae970 con 0x55c8d60110c0 2024-09-16T01:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f033ce05740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f03341ad500 con 0x55c8d6001a10 2024-09-16T01:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f0332ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d60289a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f0332ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d60289a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36662/0 (socket says 172.21.15.139:36662) 2024-09-16T01:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f0332ffd640 1 -- 172.21.15.139:0/3622986066 learned_addr learned my addr 172.21.15.139:0/3622986066 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f0333fff640 1 --2- 172.21.15.139:0/3622986066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d60110c0 0x55c8d6028ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f03337fe640 1 --2- 172.21.15.139:0/3622986066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d6001a10 0x55c8d600d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f0332ffd640 1 -- 172.21.15.139:0/3622986066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d6001a10 msgr2=0x55c8d600d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f0332ffd640 1 --2- 172.21.15.139:0/3622986066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d6001a10 0x55c8d600d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f0332ffd640 1 -- 172.21.15.139:0/3622986066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d60110c0 msgr2=0x55c8d6028ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f0332ffd640 1 --2- 172.21.15.139:0/3622986066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d60110c0 0x55c8d6028ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f0332ffd640 1 -- 172.21.15.139:0/3622986066 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8d5ffde90 con 0x55c8d6002330 2024-09-16T01:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.349+0000 7f03337fe640 1 --2- 172.21.15.139:0/3622986066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d6001a10 0x55c8d600d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.350+0000 7f0332ffd640 1 --2- 172.21.15.139:0/3622986066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d60289a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f03280cd130 tx=0x7f0328086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c244cab55ccd1c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.350+0000 7f03197fa640 1 -- 172.21.15.139:0/3622986066 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03280b3070 con 0x55c8d6002330 2024-09-16T01:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.350+0000 7f03197fa640 1 -- 172.21.15.139:0/3622986066 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03280046c0 con 0x55c8d6002330 2024-09-16T01:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.350+0000 7f03197fa640 1 -- 172.21.15.139:0/3622986066 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328020040 con 0x55c8d6002330 2024-09-16T01:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.350+0000 7f033ce05740 1 -- 172.21.15.139:0/3622986066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 msgr2=0x55c8d60289a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.350+0000 7f033ce05740 1 --2- 172.21.15.139:0/3622986066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d60289a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f03280cd130 tx=0x7f0328086040 comp rx=0 tx=0).stop 2024-09-16T01:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 -- 172.21.15.139:0/3622986066 shutdown_connections 2024-09-16T01:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 --2- 172.21.15.139:0/3622986066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d60110c0 0x55c8d6028ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 --2- 172.21.15.139:0/3622986066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d60289a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 --2- 172.21.15.139:0/3622986066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d6001a10 0x55c8d600d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 -- 172.21.15.139:0/3622986066 >> 172.21.15.139:0/3622986066 conn(0x55c8d5ffe060 msgr2=0x55c8d600e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 -- 172.21.15.139:0/3622986066 shutdown_connections 2024-09-16T01:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 -- 172.21.15.139:0/3622986066 wait complete. 2024-09-16T01:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 -- start start 2024-09-16T01:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 Processor -- start 2024-09-16T01:41:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d6001a10 0x55c8d60198a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d600acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d60110c0 0x55c8d600b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f03341ae2a0 con 0x55c8d6002330 2024-09-16T01:41:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f03341adbd0 con 0x55c8d6001a10 2024-09-16T01:41:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f03341aae10 con 0x55c8d60110c0 2024-09-16T01:41:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f033ce05740 10 client.0 _ll_register_callbacks cb 0x7f032c009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:41:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.351+0000 7f0332ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d600acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f0332ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d600acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36674/0 (socket says 172.21.15.139:36674) 2024-09-16T01:41:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f0332ffd640 1 -- 172.21.15.139:0/899384757 learned_addr learned my addr 172.21.15.139:0/899384757 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f0333fff640 1 --2- 172.21.15.139:0/899384757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d60110c0 0x55c8d600b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f03337fe640 1 --2- 172.21.15.139:0/899384757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d6001a10 0x55c8d60198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f0332ffd640 1 -- 172.21.15.139:0/899384757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d60110c0 msgr2=0x55c8d600b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f0332ffd640 1 --2- 172.21.15.139:0/899384757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d60110c0 0x55c8d600b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f0332ffd640 1 -- 172.21.15.139:0/899384757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d6001a10 msgr2=0x55c8d60198a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f0332ffd640 1 --2- 172.21.15.139:0/899384757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d6001a10 0x55c8d60198a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f0332ffd640 1 -- 172.21.15.139:0/899384757 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8d600ba80 con 0x55c8d6002330 2024-09-16T01:41:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f03337fe640 1 --2- 172.21.15.139:0/899384757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d6001a10 0x55c8d60198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:41:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f0332ffd640 1 --2- 172.21.15.139:0/899384757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d600acf0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f03280cd7a0 tx=0x7f032809b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc4f70079581daa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f0312ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:41:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f0312ffd640 1 -- 172.21.15.139:0/899384757 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03280b3070 con 0x55c8d6002330 2024-09-16T01:41:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f0312ffd640 1 -- 172.21.15.139:0/899384757 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f0328099de0 con 0x55c8d6002330 2024-09-16T01:41:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f033ce05740 1 -- 172.21.15.139:0/899384757 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c8d5f6dd70 con 0x55c8d6002330 2024-09-16T01:41:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f0312ffd640 1 -- 172.21.15.139:0/899384757 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0328020040 con 0x55c8d6002330 2024-09-16T01:41:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f033ce05740 10 client.15615 fetch_fsmap learned FSMap version 29 2024-09-16T01:41:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f033ce05740 1 -- 172.21.15.139:0/899384757 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c8d608ba10 con 0x55c8d6002330 2024-09-16T01:41:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.352+0000 7f0312ffd640 1 -- 172.21.15.139:0/899384757 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c8d5f6dd70 con 0x55c8d6002330 2024-09-16T01:41:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.353+0000 7f0312ffd640 1 -- 172.21.15.139:0/899384757 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f03280b4ce0 con 0x55c8d6002330 2024-09-16T01:41:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.353+0000 7f033ce05740 10 client.15615 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:41:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.353+0000 7f033ce05740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:41:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.353+0000 7f033ce05740 20 client.15615 populate_metadata read hostname 'smithi139' 2024-09-16T01:41:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.353+0000 7f033ce05740 1 --2- 172.21.15.139:0/899384757 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c8d608c080 0x55c8d608e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.353+0000 7f033ce05740 4 client.15615 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:41:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.353+0000 7f033ce05740 1 -- 172.21.15.139:0/899384757 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8d5eea900 con 0x55c8d608c080 2024-09-16T01:41:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.353+0000 7f03337fe640 1 --2- 172.21.15.139:0/899384757 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c8d608c080 0x55c8d608e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.354+0000 7f0312ffd640 10 client.15615 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.354+0000 7f03337fe640 1 --2- 172.21.15.139:0/899384757 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c8d608c080 0x55c8d608e4a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a6510c8f8cf6a7b server_cookie=22f6b69edac48c7a in_seq=0 out_seq=0 2024-09-16T01:41:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f0312ffd640 1 -- 172.21.15.139:0/899384757 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8d5eea900 con 0x55c8d608c080 2024-09-16T01:41:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f0312ffd640 10 client.15615 handle_command_reply: tid=0 2024-09-16T01:41:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f033ce05740 1 client.15615 shutdown 2024-09-16T01:41:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f033ce05740 1 -- 172.21.15.139:0/899384757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 msgr2=0x55c8d600acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f033ce05740 1 --2- 172.21.15.139:0/899384757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d600acf0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f03280cd7a0 tx=0x7f032809b4c0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f033ce05740 1 -- 172.21.15.139:0/899384757 shutdown_connections 2024-09-16T01:41:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f033ce05740 1 --2- 172.21.15.139:0/899384757 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c8d608c080 0x55c8d608e4a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f033ce05740 1 --2- 172.21.15.139:0/899384757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8d60110c0 0x55c8d600b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f033ce05740 1 --2- 172.21.15.139:0/899384757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8d6002330 0x55c8d600acf0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f0312ffd640 0 client.15615 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f033ce05740 1 --2- 172.21.15.139:0/899384757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8d6001a10 0x55c8d60198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f033ce05740 1 -- 172.21.15.139:0/899384757 >> 172.21.15.139:0/899384757 conn(0x55c8d5ffe060 msgr2=0x55c8d60033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f033ce05740 1 -- 172.21.15.139:0/899384757 shutdown_connections 2024-09-16T01:41:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f033ce05740 1 -- 172.21.15.139:0/899384757 wait complete. 2024-09-16T01:41:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.361+0000 7f033ce05740 20 client.15615 trim_cache size 0 max 16384 2024-09-16T01:41:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:41:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:41:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:41:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:41:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.362+0000 7f033a065640 1 -- 172.21.15.139:0/71526412 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f031c0a3580 msgr2=0x7f031c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.362+0000 7f033a065640 1 --2- 172.21.15.139:0/71526412 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f031c0a3580 0x7f031c0a5a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f032c004ab0 tx=0x7f032c0095c0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.362+0000 7f033a065640 1 -- 172.21.15.139:0/71526412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0334101350 msgr2=0x7f0334198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.362+0000 7f033a065640 1 --2- 172.21.15.139:0/71526412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0334101350 0x7f0334198810 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f032800d770 tx=0x7f032800dc30 comp rx=0 tx=0).stop 2024-09-16T01:41:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.362+0000 7f033a065640 1 -- 172.21.15.139:0/71526412 shutdown_connections 2024-09-16T01:41:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.362+0000 7f033a065640 1 --2- 172.21.15.139:0/71526412 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f031c0a3580 0x7f031c0a5a40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.362+0000 7f033a065640 1 --2- 172.21.15.139:0/71526412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03341069c0 0x7f033419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.362+0000 7f033a065640 1 --2- 172.21.15.139:0/71526412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0334101350 0x7f0334198810 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.362+0000 7f033a065640 1 --2- 172.21.15.139:0/71526412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03341009b0 0x7f03341982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.362+0000 7f033a065640 1 -- 172.21.15.139:0/71526412 >> 172.21.15.139:0/71526412 conn(0x7f0334077990 msgr2=0x7f0334069a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.362+0000 7f033a065640 1 -- 172.21.15.139:0/71526412 shutdown_connections 2024-09-16T01:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:05.362+0000 7f033a065640 1 -- 172.21.15.139:0/71526412 wait complete. 2024-09-16T01:41:05.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:05 smithi139 ceph-mon[31175]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T01:41:05.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:05 smithi176 ceph-mon[35324]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T01:41:06.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:05 smithi164 ceph-mon[35121]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T01:41:07.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:07 smithi139 ceph-mon[31175]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-16T01:41:07.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:07 smithi176 ceph-mon[35324]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-16T01:41:08.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:07 smithi164 ceph-mon[35121]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-16T01:41:08.448 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0' 2024-09-16T01:41:08.448 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0 2024-09-16T01:41:08.720 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:41:09.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.236+0000 7f2d2ef50640 1 -- 172.21.15.139:0/2663649048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d28102ef0 msgr2=0x7f2d2810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:09.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.236+0000 7f2d2ef50640 1 --2- 172.21.15.139:0/2663649048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d28102ef0 0x7f2d2810da40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009ca0 tx=0x7f2d2002a950 comp rx=0 tx=0).stop 2024-09-16T01:41:09.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.236+0000 7f2d2ef50640 1 -- 172.21.15.139:0/2663649048 shutdown_connections 2024-09-16T01:41:09.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.236+0000 7f2d2ef50640 1 --2- 172.21.15.139:0/2663649048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d28102ef0 0x7f2d2810da40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.236+0000 7f2d2ef50640 1 --2- 172.21.15.139:0/2663649048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d28102570 0x7f2d281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.236+0000 7f2d2ef50640 1 --2- 172.21.15.139:0/2663649048 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d28100e20 0x7f2d28101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.236+0000 7f2d2ef50640 1 -- 172.21.15.139:0/2663649048 >> 172.21.15.139:0/2663649048 conn(0x7f2d280fc7d0 msgr2=0x7f2d280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:09.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.236+0000 7f2d2ef50640 1 -- 172.21.15.139:0/2663649048 shutdown_connections 2024-09-16T01:41:09.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.236+0000 7f2d2ef50640 1 -- 172.21.15.139:0/2663649048 wait complete. 2024-09-16T01:41:09.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.237+0000 7f2d2ef50640 1 Processor -- start 2024-09-16T01:41:09.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.237+0000 7f2d2ef50640 1 -- start start 2024-09-16T01:41:09.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.237+0000 7f2d2ef50640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d28100e20 0x7f2d28199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.237+0000 7f2d2ef50640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d28102570 0x7f2d28199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2ef50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d28102ef0 0x7f2d281a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2ef50640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28110f80 con 0x7f2d28102ef0 2024-09-16T01:41:09.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2ef50640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28110e00 con 0x7f2d28100e20 2024-09-16T01:41:09.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2ccec640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d28100e20 0x7f2d28199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2ef50640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28111100 con 0x7f2d28102570 2024-09-16T01:41:09.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2ccec640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d28100e20 0x7f2d28199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:38406/0 (socket says 172.21.15.139:38406) 2024-09-16T01:41:09.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2ccec640 1 -- 172.21.15.139:0/1325690799 learned_addr learned my addr 172.21.15.139:0/1325690799 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:09.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2d4ed640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d28102ef0 0x7f2d281a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d27fff640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d28102570 0x7f2d28199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2d4ed640 1 -- 172.21.15.139:0/1325690799 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d28102570 msgr2=0x7f2d28199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:09.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2d4ed640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d28102570 0x7f2d28199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2d4ed640 1 -- 172.21.15.139:0/1325690799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d28100e20 msgr2=0x7f2d28199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:09.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2d4ed640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d28100e20 0x7f2d28199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2d4ed640 1 -- 172.21.15.139:0/1325690799 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d281a47d0 con 0x7f2d28102ef0 2024-09-16T01:41:09.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d28100e20 0x7f2d28199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:41:09.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.238+0000 7f2d2d4ed640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d28102ef0 0x7f2d281a4290 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2d20005cc0 tx=0x7f2d2002f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:09.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.239+0000 7f2d25ffb640 1 -- 172.21.15.139:0/1325690799 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20034040 con 0x7f2d28102ef0 2024-09-16T01:41:09.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.239+0000 7f2d25ffb640 1 -- 172.21.15.139:0/1325690799 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d20030920 con 0x7f2d28102ef0 2024-09-16T01:41:09.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.239+0000 7f2d2ef50640 1 -- 172.21.15.139:0/1325690799 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d281a49a0 con 0x7f2d28102ef0 2024-09-16T01:41:09.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.239+0000 7f2d25ffb640 1 -- 172.21.15.139:0/1325690799 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d200046f0 con 0x7f2d28102ef0 2024-09-16T01:41:09.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.239+0000 7f2d2ef50640 1 -- 172.21.15.139:0/1325690799 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d281972e0 con 0x7f2d28102ef0 2024-09-16T01:41:09.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.241+0000 7f2d25ffb640 1 -- 172.21.15.139:0/1325690799 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 24) ==== 138589+0+0 (secure 0 0 0) 0x7f2d20037020 con 0x7f2d28102ef0 2024-09-16T01:41:09.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.242+0000 7f2d25ffb640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2d180a35d0 0x7f2d180a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.242+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2d180a35d0 0x7f2d180a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.242+0000 7f2d25ffb640 1 -- 172.21.15.139:0/1325690799 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f2d200f1100 con 0x7f2d28102ef0 2024-09-16T01:41:09.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.242+0000 7f2d31329740 1 Processor -- start 2024-09-16T01:41:09.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.242+0000 7f2d31329740 1 -- start start 2024-09-16T01:41:09.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.242+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2d180a35d0 0x7f2d180a5a90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0104e0 tx=0x7f2d1c00b410 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:09.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d31329740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb3760 0x55b865fb3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d31329740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 0x55b865fbe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d31329740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fbe940 0x55b865fc0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d31329740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b865f192a0 con 0x55b865fbe940 2024-09-16T01:41:09.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d31329740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b865ec0b10 con 0x55b865fb3760 2024-09-16T01:41:09.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d31329740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b865f15260 con 0x55b865fb4080 2024-09-16T01:41:09.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d2ccec640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb3760 0x55b865fb3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d2ccec640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb3760 0x55b865fb3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:38416/0 (socket says 172.21.15.139:38416) 2024-09-16T01:41:09.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d2ccec640 1 -- 172.21.15.139:0/212854533 learned_addr learned my addr 172.21.15.139:0/212854533 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:09.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d27fff640 1 --2- 172.21.15.139:0/212854533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 0x55b865fbe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d2d4ed640 1 --2- 172.21.15.139:0/212854533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fbe940 0x55b865fc0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d2ccec640 1 -- 172.21.15.139:0/212854533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 msgr2=0x55b865fbe400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:09.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/212854533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 0x55b865fbe400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d2ccec640 1 -- 172.21.15.139:0/212854533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fbe940 msgr2=0x55b865fc0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:09.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/212854533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fbe940 0x55b865fc0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.243+0000 7f2d2ccec640 1 -- 172.21.15.139:0/212854533 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b865c8ab90 con 0x55b865fb3760 2024-09-16T01:41:09.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.244+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/212854533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb3760 0x55b865fb3b40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0118f0 tx=0x7f2d1c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd039120764f6a8f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:09.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.244+0000 7f2d12ffd640 1 -- 172.21.15.139:0/212854533 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c01cac0 con 0x55b865fb3760 2024-09-16T01:41:09.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.244+0000 7f2d12ffd640 1 -- 172.21.15.139:0/212854533 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d1c01cc60 con 0x55b865fb3760 2024-09-16T01:41:09.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.244+0000 7f2d12ffd640 1 -- 172.21.15.139:0/212854533 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c01d610 con 0x55b865fb3760 2024-09-16T01:41:09.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.245+0000 7f2d31329740 1 -- 172.21.15.139:0/212854533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb3760 msgr2=0x55b865fb3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:09.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.245+0000 7f2d31329740 1 --2- 172.21.15.139:0/212854533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb3760 0x55b865fb3b40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0118f0 tx=0x7f2d1c013000 comp rx=0 tx=0).stop 2024-09-16T01:41:09.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.245+0000 7f2d31329740 1 -- 172.21.15.139:0/212854533 shutdown_connections 2024-09-16T01:41:09.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.245+0000 7f2d31329740 1 --2- 172.21.15.139:0/212854533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fbe940 0x55b865fc0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.245+0000 7f2d31329740 1 --2- 172.21.15.139:0/212854533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 0x55b865fbe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.245+0000 7f2d31329740 1 --2- 172.21.15.139:0/212854533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb3760 0x55b865fb3b40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.245+0000 7f2d31329740 1 -- 172.21.15.139:0/212854533 >> 172.21.15.139:0/212854533 conn(0x55b865fafcf0 msgr2=0x55b865fb00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:09.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.245+0000 7f2d31329740 1 -- 172.21.15.139:0/212854533 shutdown_connections 2024-09-16T01:41:09.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.245+0000 7f2d31329740 1 -- 172.21.15.139:0/212854533 wait complete. 2024-09-16T01:41:09.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.245+0000 7f2d31329740 1 -- start start 2024-09-16T01:41:09.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.245+0000 7f2d31329740 1 Processor -- start 2024-09-16T01:41:09.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.246+0000 7f2d31329740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb3760 0x55b865fbb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.246+0000 7f2d31329740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 0x55b865fbbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.246+0000 7f2d31329740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fbe940 0x55b865fbc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.246+0000 7f2d31329740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b865fc1510 con 0x55b865fbe940 2024-09-16T01:41:09.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.246+0000 7f2d31329740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b865ea16e0 con 0x55b865fb3760 2024-09-16T01:41:09.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.246+0000 7f2d31329740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b865fc1690 con 0x55b865fb4080 2024-09-16T01:41:09.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.246+0000 7f2d2ccec640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb3760 0x55b865fbb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.246+0000 7f2d2ccec640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb3760 0x55b865fbb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:38432/0 (socket says 172.21.15.139:38432) 2024-09-16T01:41:09.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.246+0000 7f2d2ccec640 1 -- 172.21.15.139:0/1024194421 learned_addr learned my addr 172.21.15.139:0/1024194421 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:09.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.246+0000 7f2d27fff640 1 --2- 172.21.15.139:0/1024194421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 0x55b865fbbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.246+0000 7f2d31329740 10 client.0 _ll_register_callbacks cb 0x55b865ec5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:41:09.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.246+0000 7f2d2ccec640 1 -- 172.21.15.139:0/1024194421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 msgr2=0x55b865fbbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:09.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.246+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/1024194421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 0x55b865fbbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.247+0000 7f2d2ccec640 1 -- 172.21.15.139:0/1024194421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fbe940 msgr2=0x55b865fbc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:09.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.247+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/1024194421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fbe940 0x55b865fbc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.247+0000 7f2d2ccec640 1 -- 172.21.15.139:0/1024194421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b865fcaa00 con 0x55b865fb3760 2024-09-16T01:41:09.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.247+0000 7f2d27fff640 1 --2- 172.21.15.139:0/1024194421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 0x55b865fbbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:41:09.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.247+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/1024194421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb3760 0x55b865fbb830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c004730 tx=0x7f2d1c0115a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faae16098b9d0e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:09.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.247+0000 7f2d07fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:41:09.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.247+0000 7f2d07fff640 1 -- 172.21.15.139:0/1024194421 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c037060 con 0x55b865fb3760 2024-09-16T01:41:09.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.247+0000 7f2d31329740 1 -- 172.21.15.139:0/1024194421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d1c01cc60 con 0x55b865fb3760 2024-09-16T01:41:09.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.247+0000 7f2d07fff640 1 -- 172.21.15.139:0/1024194421 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f2d1c01e630 con 0x55b865fb3760 2024-09-16T01:41:09.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.247+0000 7f2d07fff640 1 -- 172.21.15.139:0/1024194421 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d1c03e050 con 0x55b865fb3760 2024-09-16T01:41:09.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.247+0000 7f2d31329740 10 client.25151 fetch_fsmap learned FSMap version 29 2024-09-16T01:41:09.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.247+0000 7f2d31329740 1 -- 172.21.15.139:0/1024194421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b865fcd3d0 con 0x55b865fb3760 2024-09-16T01:41:09.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.248+0000 7f2d07fff640 1 -- 172.21.15.139:0/1024194421 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d1c01cc60 con 0x55b865fb3760 2024-09-16T01:41:09.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.248+0000 7f2d07fff640 1 -- 172.21.15.139:0/1024194421 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2d1c027480 con 0x55b865fb3760 2024-09-16T01:41:09.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.249+0000 7f2d31329740 10 client.25151 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:41:09.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.249+0000 7f2d31329740 10 client.25151 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:41:09.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.249+0000 7f2d31329740 20 client.25151 populate_metadata read hostname 'smithi139' 2024-09-16T01:41:09.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.249+0000 7f2d31329740 1 --2- 172.21.15.139:0/1024194421 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b865fce7e0 0x55b865fd0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.249+0000 7f2d31329740 4 client.25151 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:41:09.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.249+0000 7f2d31329740 1 -- 172.21.15.139:0/1024194421 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b865f269d0 con 0x55b865fce7e0 2024-09-16T01:41:09.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.249+0000 7f2d27fff640 1 --2- 172.21.15.139:0/1024194421 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b865fce7e0 0x55b865fd0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.249+0000 7f2d07fff640 10 client.25151 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:09.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.249+0000 7f2d27fff640 1 --2- 172.21.15.139:0/1024194421 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b865fce7e0 0x55b865fd0bd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac7b4ab6f306662 server_cookie=82fce711c75b1176 in_seq=0 out_seq=0 2024-09-16T01:41:09.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.250+0000 7f2d07fff640 1 -- 172.21.15.139:0/1024194421 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b865f269d0 con 0x55b865fce7e0 2024-09-16T01:41:09.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.250+0000 7f2d07fff640 10 client.25151 handle_command_reply: tid=0 2024-09-16T01:41:09.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.250+0000 7f2d31329740 1 client.25151 shutdown 2024-09-16T01:41:09.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.250+0000 7f2d31329740 1 -- 172.21.15.139:0/1024194421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb3760 msgr2=0x55b865fbb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:09.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.250+0000 7f2d31329740 1 --2- 172.21.15.139:0/1024194421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb3760 0x55b865fbb830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c004730 tx=0x7f2d1c0115a0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.251+0000 7f2d31329740 1 -- 172.21.15.139:0/1024194421 shutdown_connections 2024-09-16T01:41:09.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.251+0000 7f2d31329740 1 --2- 172.21.15.139:0/1024194421 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b865fce7e0 0x55b865fd0bd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.251+0000 7f2d31329740 1 --2- 172.21.15.139:0/1024194421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fbe940 0x55b865fbc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.251+0000 7f2d31329740 1 --2- 172.21.15.139:0/1024194421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 0x55b865fbbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.251+0000 7f2d07fff640 0 client.25151 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:09.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.251+0000 7f2d31329740 1 --2- 172.21.15.139:0/1024194421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb3760 0x55b865fbb830 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.251+0000 7f2d31329740 1 -- 172.21.15.139:0/1024194421 >> 172.21.15.139:0/1024194421 conn(0x55b865fafcf0 msgr2=0x55b865faee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:09.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.251+0000 7f2d31329740 1 -- 172.21.15.139:0/1024194421 shutdown_connections 2024-09-16T01:41:09.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.251+0000 7f2d31329740 1 -- 172.21.15.139:0/1024194421 wait complete. 2024-09-16T01:41:09.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.252+0000 7f2d31329740 20 client.25151 trim_cache size 0 max 16384 2024-09-16T01:41:09.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.283+0000 7f2d31329740 1 Processor -- start 2024-09-16T01:41:09.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.283+0000 7f2d31329740 1 -- start start 2024-09-16T01:41:09.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.283+0000 7f2d31329740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fb3760 0x55b865fb58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.283+0000 7f2d31329740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 0x55b865fd6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.283+0000 7f2d31329740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fbe940 0x55b865fd6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.283+0000 7f2d31329740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281b11f0 con 0x55b865fb3760 2024-09-16T01:41:09.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.283+0000 7f2d31329740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281b2d30 con 0x55b865fbe940 2024-09-16T01:41:09.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.283+0000 7f2d31329740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281b18c0 con 0x55b865fb4080 2024-09-16T01:41:09.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.283+0000 7f2d2ccec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fb3760 0x55b865fb58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.283+0000 7f2d2ccec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fb3760 0x55b865fb58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36728/0 (socket says 172.21.15.139:36728) 2024-09-16T01:41:09.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.283+0000 7f2d2ccec640 1 -- 172.21.15.139:0/4110299345 learned_addr learned my addr 172.21.15.139:0/4110299345 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:09.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.283+0000 7f2d27fff640 1 --2- 172.21.15.139:0/4110299345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 0x55b865fd6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.283+0000 7f2d2d4ed640 1 --2- 172.21.15.139:0/4110299345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fbe940 0x55b865fd6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.284+0000 7f2d2ccec640 1 -- 172.21.15.139:0/4110299345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 msgr2=0x55b865fd6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:09.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.284+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/4110299345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 0x55b865fd6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.284+0000 7f2d2ccec640 1 -- 172.21.15.139:0/4110299345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fbe940 msgr2=0x55b865fd6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:09.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.284+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/4110299345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fbe940 0x55b865fd6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.284+0000 7f2d2ccec640 1 -- 172.21.15.139:0/4110299345 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b865fab260 con 0x55b865fb3760 2024-09-16T01:41:09.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.284+0000 7f2d27fff640 1 --2- 172.21.15.139:0/4110299345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 0x55b865fd6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:41:09.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.284+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/4110299345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fb3760 0x55b865fb58c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c010e10 tx=0x7f2d1c01ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c718efb44fe1c2be server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:09.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.284+0000 7f2d127fc640 1 -- 172.21.15.139:0/4110299345 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c01c710 con 0x55b865fb3760 2024-09-16T01:41:09.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.284+0000 7f2d127fc640 1 -- 172.21.15.139:0/4110299345 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d1c01cad0 con 0x55b865fb3760 2024-09-16T01:41:09.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.284+0000 7f2d127fc640 1 -- 172.21.15.139:0/4110299345 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c01e920 con 0x55b865fb3760 2024-09-16T01:41:09.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.285+0000 7f2d31329740 1 -- 172.21.15.139:0/4110299345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fb3760 msgr2=0x55b865fb58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:09.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.285+0000 7f2d31329740 1 --2- 172.21.15.139:0/4110299345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fb3760 0x55b865fb58c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c010e10 tx=0x7f2d1c01ede0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.285+0000 7f2d31329740 1 -- 172.21.15.139:0/4110299345 shutdown_connections 2024-09-16T01:41:09.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.285+0000 7f2d31329740 1 --2- 172.21.15.139:0/4110299345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fbe940 0x55b865fd6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.285+0000 7f2d31329740 1 --2- 172.21.15.139:0/4110299345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fb4080 0x55b865fd6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.285+0000 7f2d31329740 1 --2- 172.21.15.139:0/4110299345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fb3760 0x55b865fb58c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.285+0000 7f2d31329740 1 -- 172.21.15.139:0/4110299345 >> 172.21.15.139:0/4110299345 conn(0x55b865fafcf0 msgr2=0x55b865fb51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:09.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.285+0000 7f2d31329740 1 -- 172.21.15.139:0/4110299345 shutdown_connections 2024-09-16T01:41:09.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.285+0000 7f2d31329740 1 -- 172.21.15.139:0/4110299345 wait complete. 2024-09-16T01:41:09.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.285+0000 7f2d31329740 1 -- start start 2024-09-16T01:41:09.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.286+0000 7f2d31329740 1 Processor -- start 2024-09-16T01:41:09.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.286+0000 7f2d31329740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fb3760 0x55b865fc2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.286+0000 7f2d31329740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb4080 0x55b865fc2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.286+0000 7f2d31329740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fbe940 0x55b865fc3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.286+0000 7f2d31329740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281b2660 con 0x55b865fb3760 2024-09-16T01:41:09.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.286+0000 7f2d31329740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281b1f90 con 0x55b865fb4080 2024-09-16T01:41:09.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.286+0000 7f2d31329740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281af1d0 con 0x55b865fbe940 2024-09-16T01:41:09.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.286+0000 7f2d31329740 10 client.0 _ll_register_callbacks cb 0x7f2d1c011f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:41:09.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.286+0000 7f2d2ccec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fb3760 0x55b865fc2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.286+0000 7f2d27fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb4080 0x55b865fc2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.286+0000 7f2d27fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb4080 0x55b865fc2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:38452/0 (socket says 172.21.15.139:38452) 2024-09-16T01:41:09.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.286+0000 7f2d27fff640 1 -- 172.21.15.139:0/4025109664 learned_addr learned my addr 172.21.15.139:0/4025109664 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:09.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.286+0000 7f2d2d4ed640 1 --2- 172.21.15.139:0/4025109664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fbe940 0x55b865fc3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.287+0000 7f2d27fff640 1 -- 172.21.15.139:0/4025109664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fbe940 msgr2=0x55b865fc3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:09.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.287+0000 7f2d27fff640 1 --2- 172.21.15.139:0/4025109664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fbe940 0x55b865fc3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.287+0000 7f2d27fff640 1 -- 172.21.15.139:0/4025109664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fb3760 msgr2=0x55b865fc2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:09.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.287+0000 7f2d27fff640 1 --2- 172.21.15.139:0/4025109664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fb3760 0x55b865fc2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.287+0000 7f2d27fff640 1 -- 172.21.15.139:0/4025109664 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b865fb65d0 con 0x55b865fb4080 2024-09-16T01:41:09.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.287+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/4025109664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fb3760 0x55b865fc2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:41:09.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.287+0000 7f2d07fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:41:09.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.287+0000 7f2d27fff640 1 --2- 172.21.15.139:0/4025109664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb4080 0x55b865fc2d40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55b865fb0b50 tx=0x7f2d14012790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b1d8f6596ed135d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:09.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.287+0000 7f2d07fff640 1 -- 172.21.15.139:0/4025109664 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14010430 con 0x55b865fb4080 2024-09-16T01:41:09.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.287+0000 7f2d31329740 1 -- 172.21.15.139:0/4025109664 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d1c01cad0 con 0x55b865fb4080 2024-09-16T01:41:09.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.287+0000 7f2d07fff640 1 -- 172.21.15.139:0/4025109664 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f2d140105d0 con 0x55b865fb4080 2024-09-16T01:41:09.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.288+0000 7f2d07fff640 1 -- 172.21.15.139:0/4025109664 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d14032050 con 0x55b865fb4080 2024-09-16T01:41:09.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.288+0000 7f2d31329740 10 client.25157 fetch_fsmap learned FSMap version 29 2024-09-16T01:41:09.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.288+0000 7f2d31329740 1 -- 172.21.15.139:0/4025109664 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b865fb0980 con 0x55b865fb4080 2024-09-16T01:41:09.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.288+0000 7f2d07fff640 1 -- 172.21.15.139:0/4025109664 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d1c01cad0 con 0x55b865fb4080 2024-09-16T01:41:09.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.288+0000 7f2d07fff640 1 -- 172.21.15.139:0/4025109664 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2d1402d030 con 0x55b865fb4080 2024-09-16T01:41:09.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.288+0000 7f2d31329740 10 client.25157 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:41:09.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.288+0000 7f2d31329740 10 client.25157 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:41:09.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.288+0000 7f2d31329740 20 client.25157 populate_metadata read hostname 'smithi139' 2024-09-16T01:41:09.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.288+0000 7f2d31329740 1 --2- 172.21.15.139:0/4025109664 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b865fce7e0 0x55b86603bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:09.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.288+0000 7f2d31329740 4 client.25157 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}] 2024-09-16T01:41:09.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.288+0000 7f2d31329740 1 -- 172.21.15.139:0/4025109664 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}) -- 0x55b865f20d20 con 0x55b865fce7e0 2024-09-16T01:41:09.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.288+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/4025109664 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b865fce7e0 0x55b86603bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:09.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.289+0000 7f2d07fff640 10 client.25157 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:09.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.289+0000 7f2d2ccec640 1 --2- 172.21.15.139:0/4025109664 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b865fce7e0 0x55b86603bbb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d8cf06d09c7a33f server_cookie=e69ea08ad605063a in_seq=0 out_seq=0 2024-09-16T01:41:09.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.301+0000 7f2d07fff640 1 -- 172.21.15.139:0/4025109664 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b865f20d20 con 0x55b865fce7e0 2024-09-16T01:41:09.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.301+0000 7f2d07fff640 10 client.25157 handle_command_reply: tid=0 2024-09-16T01:41:09.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.301+0000 7f2d31329740 1 client.25157 shutdown 2024-09-16T01:41:09.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.301+0000 7f2d31329740 1 -- 172.21.15.139:0/4025109664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb4080 msgr2=0x55b865fc2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:09.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.301+0000 7f2d31329740 1 --2- 172.21.15.139:0/4025109664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb4080 0x55b865fc2d40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55b865fb0b50 tx=0x7f2d14012790 comp rx=0 tx=0).stop 2024-09-16T01:41:09.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.301+0000 7f2d31329740 1 -- 172.21.15.139:0/4025109664 shutdown_connections 2024-09-16T01:41:09.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.301+0000 7f2d31329740 1 --2- 172.21.15.139:0/4025109664 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b865fce7e0 0x55b86603bbb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.301+0000 7f2d07fff640 0 client.25157 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:09.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.301+0000 7f2d31329740 1 --2- 172.21.15.139:0/4025109664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b865fbe940 0x55b865fc3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.301+0000 7f2d31329740 1 --2- 172.21.15.139:0/4025109664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b865fb4080 0x55b865fc2d40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.301+0000 7f2d31329740 1 --2- 172.21.15.139:0/4025109664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b865fb3760 0x55b865fc2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.301+0000 7f2d31329740 1 -- 172.21.15.139:0/4025109664 >> 172.21.15.139:0/4025109664 conn(0x55b865fafcf0 msgr2=0x55b865fbbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:09.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.301+0000 7f2d31329740 1 -- 172.21.15.139:0/4025109664 shutdown_connections 2024-09-16T01:41:09.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.301+0000 7f2d31329740 1 -- 172.21.15.139:0/4025109664 wait complete. 2024-09-16T01:41:09.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.302+0000 7f2d31329740 20 client.25157 trim_cache size 0 max 16384 2024-09-16T01:41:09.305 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:41:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 29, 2024-09-16T01:41:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:41:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 2, 2024-09-16T01:41:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:41:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "72f21021": { 2024-09-16T01:41:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 2, 2024-09-16T01:41:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:41:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:41:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:41:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:41:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:41:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:41:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 150.2, 2024-09-16T01:41:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:41:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:41:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:41:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:41:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:41:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:41:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:41:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:41:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:41:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:41:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:41:09.307 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:41:09.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.302+0000 7f2d2ef50640 1 -- 172.21.15.139:0/1325690799 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2d180a35d0 msgr2=0x7f2d180a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:09.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.302+0000 7f2d2ef50640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2d180a35d0 0x7f2d180a5a90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0104e0 tx=0x7f2d1c00b410 comp rx=0 tx=0).stop 2024-09-16T01:41:09.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.302+0000 7f2d2ef50640 1 -- 172.21.15.139:0/1325690799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d28102ef0 msgr2=0x7f2d281a4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:09.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.302+0000 7f2d2ef50640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d28102ef0 0x7f2d281a4290 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2d20005cc0 tx=0x7f2d2002f630 comp rx=0 tx=0).stop 2024-09-16T01:41:09.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.303+0000 7f2d2ef50640 1 -- 172.21.15.139:0/1325690799 shutdown_connections 2024-09-16T01:41:09.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.303+0000 7f2d2ef50640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2d180a35d0 0x7f2d180a5a90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.303+0000 7f2d2ef50640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d28102ef0 0x7f2d281a4290 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.303+0000 7f2d2ef50640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d28102570 0x7f2d28199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.303+0000 7f2d2ef50640 1 --2- 172.21.15.139:0/1325690799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d28100e20 0x7f2d28199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:09.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.303+0000 7f2d2ef50640 1 -- 172.21.15.139:0/1325690799 >> 172.21.15.139:0/1325690799 conn(0x7f2d280fc7d0 msgr2=0x7f2d28106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:09.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.303+0000 7f2d2ef50640 1 -- 172.21.15.139:0/1325690799 shutdown_connections 2024-09-16T01:41:09.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:09.303+0000 7f2d2ef50640 1 -- 172.21.15.139:0/1325690799 wait complete. 2024-09-16T01:41:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:09 smithi139 ceph-mon[31175]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-16T01:41:09.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:09 smithi176 ceph-mon[35324]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-16T01:41:10.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:09 smithi164 ceph-mon[35121]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-16T01:41:10.242 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '72f21021', quiesce took 7.7 seconds. Will release after: 22.5 2024-09-16T01:41:11.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:11 smithi139 ceph-mon[31175]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-09-16T01:41:11.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:11 smithi176 ceph-mon[35324]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-09-16T01:41:12.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:11 smithi164 ceph-mon[35121]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-09-16T01:41:13.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:13 smithi139 ceph-mon[31175]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-09-16T01:41:13.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:13 smithi176 ceph-mon[35324]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-09-16T01:41:14.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:13 smithi164 ceph-mon[35121]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-09-16T01:41:15.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:15 smithi139 ceph-mon[31175]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-16T01:41:15.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:15 smithi176 ceph-mon[35324]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-16T01:41:16.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:15 smithi164 ceph-mon[35121]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-16T01:41:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:17 smithi139 ceph-mon[31175]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-09-16T01:41:17.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:17 smithi176 ceph-mon[35324]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-09-16T01:41:18.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:17 smithi164 ceph-mon[35121]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-09-16T01:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.492+0000 7fd0fcca4640 1 -- 172.21.15.139:0/1227850620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0f81009b0 msgr2=0x7fd0f8100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.492+0000 7fd0fcca4640 1 --2- 172.21.15.139:0/1227850620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0f81009b0 0x7fd0f8100e10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009930 tx=0x7fd0ec02a420 comp rx=0 tx=0).stop 2024-09-16T01:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.492+0000 7fd0fcca4640 1 -- 172.21.15.139:0/1227850620 shutdown_connections 2024-09-16T01:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.492+0000 7fd0fcca4640 1 --2- 172.21.15.139:0/1227850620 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0f8101350 0x7fd0f810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.492+0000 7fd0fcca4640 1 --2- 172.21.15.139:0/1227850620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0f81009b0 0x7fd0f8100e10 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.492+0000 7fd0fcca4640 1 --2- 172.21.15.139:0/1227850620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f81069c0 0x7fd0f8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.492+0000 7fd0fcca4640 1 -- 172.21.15.139:0/1227850620 >> 172.21.15.139:0/1227850620 conn(0x7fd0f8077990 msgr2=0x7fd0f80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.492+0000 7fd0fcca4640 1 -- 172.21.15.139:0/1227850620 shutdown_connections 2024-09-16T01:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.492+0000 7fd0fcca4640 1 -- 172.21.15.139:0/1227850620 wait complete. 2024-09-16T01:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.493+0000 7fd0fcca4640 1 Processor -- start 2024-09-16T01:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.493+0000 7fd0fcca4640 1 -- start start 2024-09-16T01:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.493+0000 7fd0fcca4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0f81009b0 0x7fd0f8199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0fcca4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f8101350 0x7fd0f81996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0fcca4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0f81069c0 0x7fd0f81a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0fcca4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f8110e00 con 0x7fd0f81009b0 2024-09-16T01:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0fcca4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f8110c80 con 0x7fd0f8101350 2024-09-16T01:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0fcca4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f8110f80 con 0x7fd0f81069c0 2024-09-16T01:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0f659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0f81009b0 0x7fd0f8199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0f659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0f81009b0 0x7fd0f8199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43602/0 (socket says 172.21.15.139:43602) 2024-09-16T01:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0f659c640 1 -- 172.21.15.139:0/1715494502 learned_addr learned my addr 172.21.15.139:0/1715494502 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0f5d9b640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f8101350 0x7fd0f81996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0f81069c0 0x7fd0f81a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0f659c640 1 -- 172.21.15.139:0/1715494502 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0f81069c0 msgr2=0x7fd0f81a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0f659c640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0f81069c0 0x7fd0f81a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0f659c640 1 -- 172.21.15.139:0/1715494502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f8101350 msgr2=0x7fd0f81996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0f659c640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f8101350 0x7fd0f81996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0f659c640 1 -- 172.21.15.139:0/1715494502 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f81a4720 con 0x7fd0f81009b0 2024-09-16T01:41:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0f81069c0 0x7fd0f81a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:41:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.494+0000 7fd0f5d9b640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f8101350 0x7fd0f81996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:41:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.495+0000 7fd0f659c640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0f81009b0 0x7fd0f8199160 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800ea60 tx=0x7fd0e800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.495+0000 7fd0e77fe640 1 -- 172.21.15.139:0/1715494502 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e800cde0 con 0x7fd0f81009b0 2024-09-16T01:41:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.495+0000 7fd0e77fe640 1 -- 172.21.15.139:0/1715494502 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0e80035f0 con 0x7fd0f81009b0 2024-09-16T01:41:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.495+0000 7fd0e77fe640 1 -- 172.21.15.139:0/1715494502 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8010670 con 0x7fd0f81009b0 2024-09-16T01:41:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.495+0000 7fd0fcca4640 1 -- 172.21.15.139:0/1715494502 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f81a48f0 con 0x7fd0f81009b0 2024-09-16T01:41:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.495+0000 7fd0fcca4640 1 -- 172.21.15.139:0/1715494502 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f81a4d60 con 0x7fd0f81009b0 2024-09-16T01:41:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.498+0000 7fd0e77fe640 1 -- 172.21.15.139:0/1715494502 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 24) ==== 138589+0+0 (secure 0 0 0) 0x7fd0e80037a0 con 0x7fd0f81009b0 2024-09-16T01:41:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.498+0000 7fd0ffa37740 1 Processor -- start 2024-09-16T01:41:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.499+0000 7fd0ffa37740 1 -- start start 2024-09-16T01:41:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.499+0000 7fd0ffa37740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5645112a3cc0 0x5645112a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.499+0000 7fd0ffa37740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56451129f660 0x56451129fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.499+0000 7fd0ffa37740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a0000 0x5645112ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.499+0000 7fd0ffa37740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5645111fa2f0 con 0x5645112a0000 2024-09-16T01:41:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.499+0000 7fd0ffa37740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5645111a2270 con 0x56451129f660 2024-09-16T01:41:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.499+0000 7fd0ffa37740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5645111f62b0 con 0x5645112a3cc0 2024-09-16T01:41:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.499+0000 7fd0f5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a0000 0x5645112ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.499+0000 7fd0f6d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5645112a3cc0 0x5645112a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.499+0000 7fd0f5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a0000 0x5645112ab9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43604/0 (socket says 172.21.15.139:43604) 2024-09-16T01:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.499+0000 7fd0f5d9b640 1 -- 172.21.15.139:0/3576258408 learned_addr learned my addr 172.21.15.139:0/3576258408 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.499+0000 7fd0f659c640 1 --2- 172.21.15.139:0/3576258408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56451129f660 0x56451129fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.500+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/3576258408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56451129f660 msgr2=0x56451129fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.500+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/3576258408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56451129f660 0x56451129fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.500+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/3576258408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a0000 msgr2=0x5645112ab9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.500+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/3576258408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a0000 0x5645112ab9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.500+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/3576258408 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645111a1340 con 0x5645112a3cc0 2024-09-16T01:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.500+0000 7fd0f5d9b640 1 --2- 172.21.15.139:0/3576258408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a0000 0x5645112ab9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.500+0000 7fd0e77fe640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd0e00a3680 0x7fd0e00a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.501+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/3576258408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5645112a3cc0 0x5645112a40a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0005fd0 tx=0x7fd0f0011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3bda071bdee20a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.501+0000 7fd0f659c640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd0e00a3680 0x7fd0e00a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.501+0000 7fd0e77fe640 1 -- 172.21.15.139:0/1715494502 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fd0e80d0670 con 0x7fd0f81009b0 2024-09-16T01:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.501+0000 7fd0e4ff9640 1 -- 172.21.15.139:0/3576258408 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0004d20 con 0x5645112a3cc0 2024-09-16T01:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.501+0000 7fd0e4ff9640 1 -- 172.21.15.139:0/3576258408 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f000f9a0 con 0x5645112a3cc0 2024-09-16T01:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.501+0000 7fd0e4ff9640 1 -- 172.21.15.139:0/3576258408 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00044a0 con 0x5645112a3cc0 2024-09-16T01:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.502+0000 7fd0ffa37740 1 -- 172.21.15.139:0/3576258408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5645112a3cc0 msgr2=0x5645112a40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.502+0000 7fd0f659c640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd0e00a3680 0x7fd0e00a5b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80cd890 tx=0x7fd0e8003f30 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.502+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/3576258408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5645112a3cc0 0x5645112a40a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0005fd0 tx=0x7fd0f0011bf0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.502+0000 7fd0ffa37740 1 -- 172.21.15.139:0/3576258408 shutdown_connections 2024-09-16T01:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.502+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/3576258408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a0000 0x5645112ab9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.502+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/3576258408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56451129f660 0x56451129fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.502+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/3576258408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5645112a3cc0 0x5645112a40a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.502+0000 7fd0ffa37740 1 -- 172.21.15.139:0/3576258408 >> 172.21.15.139:0/3576258408 conn(0x564511291b10 msgr2=0x564511291ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.502+0000 7fd0ffa37740 1 -- 172.21.15.139:0/3576258408 shutdown_connections 2024-09-16T01:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.502+0000 7fd0ffa37740 1 -- 172.21.15.139:0/3576258408 wait complete. 2024-09-16T01:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.502+0000 7fd0ffa37740 1 -- start start 2024-09-16T01:41:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.503+0000 7fd0ffa37740 1 Processor -- start 2024-09-16T01:41:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.503+0000 7fd0ffa37740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56451129f660 0x5645112a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.503+0000 7fd0ffa37740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 0x5645112a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.503+0000 7fd0ffa37740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5645112a3cc0 0x5645112a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.503+0000 7fd0ffa37740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5645112ac0b0 con 0x56451129f660 2024-09-16T01:41:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.503+0000 7fd0ffa37740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564511183290 con 0x5645112a0000 2024-09-16T01:41:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.503+0000 7fd0ffa37740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5645112ac230 con 0x5645112a3cc0 2024-09-16T01:41:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.503+0000 7fd0f6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56451129f660 0x5645112a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.503+0000 7fd0f6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56451129f660 0x5645112a8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43616/0 (socket says 172.21.15.139:43616) 2024-09-16T01:41:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.503+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/2233521378 learned_addr learned my addr 172.21.15.139:0/2233521378 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.503+0000 7fd0ffa37740 10 client.0 _ll_register_callbacks cb 0x5645111b4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:41:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.503+0000 7fd0f5d9b640 1 --2- 172.21.15.139:0/2233521378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 0x5645112a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.503+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/2233521378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5645112a3cc0 0x5645112a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.504+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/2233521378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5645112a3cc0 msgr2=0x5645112a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.504+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/2233521378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5645112a3cc0 0x5645112a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.504+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/2233521378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 msgr2=0x5645112a93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.504+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/2233521378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 0x5645112a93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.504+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/2233521378 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564510f0d210 con 0x56451129f660 2024-09-16T01:41:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.504+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/2233521378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56451129f660 0x5645112a8e60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00115a0 tx=0x7fd0f00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54faaea8d7d8559e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.504+0000 7fd0ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:41:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.504+0000 7fd0ddffb640 1 -- 172.21.15.139:0/2233521378 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0005080 con 0x56451129f660 2024-09-16T01:41:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.504+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2233521378 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f000f9a0 con 0x56451129f660 2024-09-16T01:41:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.504+0000 7fd0ddffb640 1 -- 172.21.15.139:0/2233521378 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fd0f0005220 con 0x56451129f660 2024-09-16T01:41:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.504+0000 7fd0ddffb640 1 -- 172.21.15.139:0/2233521378 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0f0034050 con 0x56451129f660 2024-09-16T01:41:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.504+0000 7fd0ffa37740 10 client.15639 fetch_fsmap learned FSMap version 29 2024-09-16T01:41:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.504+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2233521378 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564510fd9620 con 0x56451129f660 2024-09-16T01:41:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.506+0000 7fd0ddffb640 1 -- 172.21.15.139:0/2233521378 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f000f9a0 con 0x56451129f660 2024-09-16T01:41:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.506+0000 7fd0ddffb640 1 -- 172.21.15.139:0/2233521378 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd0f0018070 con 0x56451129f660 2024-09-16T01:41:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.506+0000 7fd0ffa37740 10 client.15639 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:41:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.506+0000 7fd0ffa37740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:41:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.506+0000 7fd0ffa37740 20 client.15639 populate_metadata read hostname 'smithi139' 2024-09-16T01:41:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.506+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/2233521378 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5645112b97b0 0x5645112bbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.506+0000 7fd0ffa37740 4 client.15639 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:41:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.506+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2233521378 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564511207a20 con 0x5645112b97b0 2024-09-16T01:41:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.507+0000 7fd0f5d9b640 1 --2- 172.21.15.139:0/2233521378 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5645112b97b0 0x5645112bbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.507+0000 7fd0f5d9b640 1 --2- 172.21.15.139:0/2233521378 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5645112b97b0 0x5645112bbbd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c7f20a92a91691 server_cookie=8588c0ae94235275 in_seq=0 out_seq=0 2024-09-16T01:41:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.507+0000 7fd0ddffb640 10 client.15639 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.508+0000 7fd0ddffb640 1 -- 172.21.15.139:0/2233521378 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564511207a20 con 0x5645112b97b0 2024-09-16T01:41:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.508+0000 7fd0ddffb640 10 client.15639 handle_command_reply: tid=0 2024-09-16T01:41:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.508+0000 7fd0ffa37740 1 client.15639 shutdown 2024-09-16T01:41:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.509+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2233521378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56451129f660 msgr2=0x5645112a8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.509+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/2233521378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56451129f660 0x5645112a8e60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00115a0 tx=0x7fd0f00040c0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.509+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2233521378 shutdown_connections 2024-09-16T01:41:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.509+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/2233521378 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5645112b97b0 0x5645112bbbd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.509+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/2233521378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5645112a3cc0 0x5645112a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.509+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/2233521378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 0x5645112a93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.509+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/2233521378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56451129f660 0x5645112a8e60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.509+0000 7fd0ddffb640 0 client.15639 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.509+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2233521378 >> 172.21.15.139:0/2233521378 conn(0x564511291b10 msgr2=0x56451129ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.509+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2233521378 shutdown_connections 2024-09-16T01:41:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.509+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2233521378 wait complete. 2024-09-16T01:41:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.509+0000 7fd0ffa37740 20 client.15639 trim_cache size 0 max 16384 2024-09-16T01:41:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.535+0000 7fd0ffa37740 1 Processor -- start 2024-09-16T01:41:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.535+0000 7fd0ffa37740 1 -- start start 2024-09-16T01:41:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.535+0000 7fd0ffa37740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56451129f660 0x5645112a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.535+0000 7fd0ffa37740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 0x5645112c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.535+0000 7fd0ffa37740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a3cc0 0x5645112c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.535+0000 7fd0ffa37740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81acda0 con 0x5645112a3cc0 2024-09-16T01:41:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.536+0000 7fd0ffa37740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81ae8e0 con 0x5645112a0000 2024-09-16T01:41:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.536+0000 7fd0ffa37740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81ad470 con 0x56451129f660 2024-09-16T01:41:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.536+0000 7fd0f6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a3cc0 0x5645112c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.536+0000 7fd0f6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a3cc0 0x5645112c1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43628/0 (socket says 172.21.15.139:43628) 2024-09-16T01:41:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.536+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/352074244 learned_addr learned my addr 172.21.15.139:0/352074244 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.536+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/352074244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56451129f660 0x5645112a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.536+0000 7fd0f5d9b640 1 --2- 172.21.15.139:0/352074244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 0x5645112c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.536+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/352074244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56451129f660 msgr2=0x5645112a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.536+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/352074244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56451129f660 0x5645112a26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.536+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/352074244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 msgr2=0x5645112c14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.536+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/352074244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 0x5645112c14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.536+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/352074244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645112a4410 con 0x5645112a3cc0 2024-09-16T01:41:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.536+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/352074244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a3cc0 0x5645112c1a20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009fd0 tx=0x7fd0f000ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb7fc62e9565e2c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.536+0000 7fd0dffff640 1 -- 172.21.15.139:0/352074244 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0027cd0 con 0x5645112a3cc0 2024-09-16T01:41:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.537+0000 7fd0dffff640 1 -- 172.21.15.139:0/352074244 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f001f040 con 0x5645112a3cc0 2024-09-16T01:41:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.537+0000 7fd0dffff640 1 -- 172.21.15.139:0/352074244 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00297c0 con 0x5645112a3cc0 2024-09-16T01:41:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.537+0000 7fd0ffa37740 1 -- 172.21.15.139:0/352074244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a3cc0 msgr2=0x5645112c1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.537+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/352074244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a3cc0 0x5645112c1a20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009fd0 tx=0x7fd0f000ff80 comp rx=0 tx=0).stop 2024-09-16T01:41:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.538+0000 7fd0ffa37740 1 -- 172.21.15.139:0/352074244 shutdown_connections 2024-09-16T01:41:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.538+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/352074244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a3cc0 0x5645112c1a20 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.538+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/352074244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 0x5645112c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.538+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/352074244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56451129f660 0x5645112a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.538+0000 7fd0ffa37740 1 -- 172.21.15.139:0/352074244 >> 172.21.15.139:0/352074244 conn(0x564511291b10 msgr2=0x5645112a8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.538+0000 7fd0ffa37740 1 -- 172.21.15.139:0/352074244 shutdown_connections 2024-09-16T01:41:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.538+0000 7fd0ffa37740 1 -- 172.21.15.139:0/352074244 wait complete. 2024-09-16T01:41:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.538+0000 7fd0ffa37740 1 -- start start 2024-09-16T01:41:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.538+0000 7fd0ffa37740 1 Processor -- start 2024-09-16T01:41:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.539+0000 7fd0ffa37740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56451129f660 0x5645112b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.539+0000 7fd0ffa37740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 0x5645112ac480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.539+0000 7fd0ffa37740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a3cc0 0x5645112acc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.539+0000 7fd0f6d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56451129f660 0x5645112b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.539+0000 7fd0f6d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56451129f660 0x5645112b1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:38824/0 (socket says 172.21.15.139:38824) 2024-09-16T01:41:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.539+0000 7fd0f5d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 0x5645112ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.539+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/2234587336 learned_addr learned my addr 172.21.15.139:0/2234587336 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.539+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2234587336 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81ae210 con 0x5645112a3cc0 2024-09-16T01:41:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.539+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2234587336 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81adb40 con 0x5645112a0000 2024-09-16T01:41:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.539+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2234587336 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81aad80 con 0x56451129f660 2024-09-16T01:41:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.539+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/2234587336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a3cc0 0x5645112acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.540+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/2234587336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 msgr2=0x5645112ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.540+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/2234587336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 0x5645112ac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.540+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/2234587336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a3cc0 msgr2=0x5645112acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.540+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/2234587336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a3cc0 0x5645112acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.540+0000 7fd0f6d9d640 1 -- 172.21.15.139:0/2234587336 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645113224a0 con 0x56451129f660 2024-09-16T01:41:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.540+0000 7fd0ffa37740 10 client.0 _ll_register_callbacks cb 0x7fd0f00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:41:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.540+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2234587336 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f001f040 con 0x56451129f660 2024-09-16T01:41:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.540+0000 7fd0f6d9d640 1 --2- 172.21.15.139:0/2234587336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56451129f660 0x5645112b1d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0005d70 tx=0x7fd0f000f8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a250d802c9ea6be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.540+0000 7fd0ddffb640 10 client.25144 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.540+0000 7fd0ddffb640 1 -- 172.21.15.139:0/2234587336 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0027450 con 0x56451129f660 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.540+0000 7fd0ddffb640 1 -- 172.21.15.139:0/2234587336 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fd0f00275f0 con 0x56451129f660 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.540+0000 7fd0ddffb640 1 -- 172.21.15.139:0/2234587336 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0f0027830 con 0x56451129f660 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.540+0000 7fd0ffa37740 10 client.25144 fetch_fsmap learned FSMap version 29 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.540+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2234587336 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5645113234c0 con 0x56451129f660 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.541+0000 7fd0ddffb640 1 -- 172.21.15.139:0/2234587336 <== mon.1 v2:172.21.15.176:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd0f0018070 con 0x56451129f660 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.541+0000 7fd0ffa37740 10 client.25144 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.541+0000 7fd0ffa37740 10 client.25144 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.541+0000 7fd0ffa37740 20 client.25144 populate_metadata read hostname 'smithi139' 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.541+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/2234587336 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564511323a70 0x564511325e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.541+0000 7fd0ffa37740 4 client.25144 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.541+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2234587336 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564511201d70 con 0x564511323a70 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.541+0000 7fd0f5d9b640 1 --2- 172.21.15.139:0/2234587336 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564511323a70 0x564511325e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.541+0000 7fd0ddffb640 10 client.25144 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.541+0000 7fd0f5d9b640 1 --2- 172.21.15.139:0/2234587336 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564511323a70 0x564511325e60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3812dbdc5bf6200e server_cookie=82679c89b70fed8d in_seq=0 out_seq=0 2024-09-16T01:41:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ddffb640 1 -- 172.21.15.139:0/2234587336 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564511201d70 con 0x564511323a70 2024-09-16T01:41:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ddffb640 10 client.25144 handle_command_reply: tid=0 2024-09-16T01:41:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ffa37740 1 client.25144 shutdown 2024-09-16T01:41:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2234587336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56451129f660 msgr2=0x5645112b1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/2234587336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56451129f660 0x5645112b1d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0005d70 tx=0x7fd0f000f8d0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2234587336 shutdown_connections 2024-09-16T01:41:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/2234587336 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564511323a70 0x564511325e60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/2234587336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5645112a3cc0 0x5645112acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ddffb640 0 client.25144 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/2234587336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5645112a0000 0x5645112ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ffa37740 1 --2- 172.21.15.139:0/2234587336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56451129f660 0x5645112b1d80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2234587336 >> 172.21.15.139:0/2234587336 conn(0x564511291b10 msgr2=0x56451129e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2234587336 shutdown_connections 2024-09-16T01:41:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ffa37740 1 -- 172.21.15.139:0/2234587336 wait complete. 2024-09-16T01:41:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.550+0000 7fd0ffa37740 20 client.25144 trim_cache size 0 max 16384 2024-09-16T01:41:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:41:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:41:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:41:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:41:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.551+0000 7fd0fcca4640 1 -- 172.21.15.139:0/1715494502 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd0e00a3680 msgr2=0x7fd0e00a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.551+0000 7fd0fcca4640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd0e00a3680 0x7fd0e00a5b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80cd890 tx=0x7fd0e8003f30 comp rx=0 tx=0).stop 2024-09-16T01:41:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.551+0000 7fd0fcca4640 1 -- 172.21.15.139:0/1715494502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0f81009b0 msgr2=0x7fd0f8199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.551+0000 7fd0fcca4640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0f81009b0 0x7fd0f8199160 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800ea60 tx=0x7fd0e800ef20 comp rx=0 tx=0).stop 2024-09-16T01:41:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.551+0000 7fd0fcca4640 1 -- 172.21.15.139:0/1715494502 shutdown_connections 2024-09-16T01:41:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.551+0000 7fd0fcca4640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd0e00a3680 0x7fd0e00a5b40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.551+0000 7fd0fcca4640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0f81069c0 0x7fd0f81a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.551+0000 7fd0fcca4640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f8101350 0x7fd0f81996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.551+0000 7fd0fcca4640 1 --2- 172.21.15.139:0/1715494502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0f81009b0 0x7fd0f8199160 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.551+0000 7fd0fcca4640 1 -- 172.21.15.139:0/1715494502 >> 172.21.15.139:0/1715494502 conn(0x7fd0f8077990 msgr2=0x7fd0f81037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.551+0000 7fd0fcca4640 1 -- 172.21.15.139:0/1715494502 shutdown_connections 2024-09-16T01:41:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:18.551+0000 7fd0fcca4640 1 -- 172.21.15.139:0/1715494502 wait complete. 2024-09-16T01:41:19.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:19 smithi139 ceph-mon[31175]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-16T01:41:19.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:19 smithi176 ceph-mon[35324]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-16T01:41:20.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:19 smithi164 ceph-mon[35121]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-16T01:41:21.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:21 smithi139 ceph-mon[31175]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-09-16T01:41:21.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:21 smithi176 ceph-mon[35324]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-09-16T01:41:22.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:21 smithi164 ceph-mon[35121]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-09-16T01:41:23.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:23 smithi139 ceph-mon[31175]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T01:41:23.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:23 smithi176 ceph-mon[35324]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T01:41:24.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:23 smithi164 ceph-mon[35121]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T01:41:25.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:25 smithi139 ceph-mon[31175]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T01:41:25.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:25 smithi176 ceph-mon[35324]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T01:41:26.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:25 smithi164 ceph-mon[35121]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T01:41:27.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:27 smithi139 ceph-mon[31175]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-16T01:41:27.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:27 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:41:27.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:27 smithi176 ceph-mon[35324]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-16T01:41:27.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:27 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:41:28.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:27 smithi164 ceph-mon[35121]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-16T01:41:28.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:27 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:41:29.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:28 smithi164 ceph-mon[35121]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T01:41:29.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:29.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:29.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:29.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:29.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:41:29.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:41:29.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:28 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:28 smithi139 ceph-mon[31175]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T01:41:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:41:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:41:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:28 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:29.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:28 smithi176 ceph-mon[35324]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-16T01:41:29.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:29.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:29.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:29.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:29.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:41:29.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:41:29.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:28 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:41:31.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:30 smithi164 ceph-mon[35121]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-16T01:41:31.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:30 smithi139 ceph-mon[31175]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-16T01:41:31.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:30 smithi176 ceph-mon[35324]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-16T01:41:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.686+0000 7f4d32f03640 1 -- 172.21.15.139:0/3501266904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c1009b0 msgr2=0x7f4d2c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.686+0000 7f4d32f03640 1 --2- 172.21.15.139:0/3501266904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c1009b0 0x7f4d2c100e10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009930 tx=0x7f4d1802a420 comp rx=0 tx=0).stop 2024-09-16T01:41:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.687+0000 7f4d32f03640 1 -- 172.21.15.139:0/3501266904 shutdown_connections 2024-09-16T01:41:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.687+0000 7f4d32f03640 1 --2- 172.21.15.139:0/3501266904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d2c101350 0x7f4d2c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.687+0000 7f4d32f03640 1 --2- 172.21.15.139:0/3501266904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c1009b0 0x7f4d2c100e10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.687+0000 7f4d32f03640 1 --2- 172.21.15.139:0/3501266904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d2c1069c0 0x7f4d2c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.687+0000 7f4d32f03640 1 -- 172.21.15.139:0/3501266904 >> 172.21.15.139:0/3501266904 conn(0x7f4d2c077990 msgr2=0x7f4d2c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.687+0000 7f4d32f03640 1 -- 172.21.15.139:0/3501266904 shutdown_connections 2024-09-16T01:41:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.687+0000 7f4d32f03640 1 -- 172.21.15.139:0/3501266904 wait complete. 2024-09-16T01:41:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.687+0000 7f4d32f03640 1 Processor -- start 2024-09-16T01:41:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d32f03640 1 -- start start 2024-09-16T01:41:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d32f03640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d2c1009b0 0x7f4d2c196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d32f03640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101350 0x7f4d2c197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d32f03640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d2c1069c0 0x7f4d2c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d2bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101350 0x7f4d2c197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d2bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101350 0x7f4d2c197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43168/0 (socket says 172.21.15.139:43168) 2024-09-16T01:41:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d2bfff640 1 -- 172.21.15.139:0/1189908970 learned_addr learned my addr 172.21.15.139:0/1189908970 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d32f03640 1 -- 172.21.15.139:0/1189908970 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c110e50 con 0x7f4d2c101350 2024-09-16T01:41:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d32f03640 1 -- 172.21.15.139:0/1189908970 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c110cd0 con 0x7f4d2c1009b0 2024-09-16T01:41:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d32f03640 1 -- 172.21.15.139:0/1189908970 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c110fd0 con 0x7f4d2c1069c0 2024-09-16T01:41:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d30c9f640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d2c1009b0 0x7f4d2c196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d314a0640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d2c1069c0 0x7f4d2c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d30c9f640 1 -- 172.21.15.139:0/1189908970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d2c1069c0 msgr2=0x7f4d2c197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d30c9f640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d2c1069c0 0x7f4d2c197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d30c9f640 1 -- 172.21.15.139:0/1189908970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101350 msgr2=0x7f4d2c197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d30c9f640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101350 0x7f4d2c197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d30c9f640 1 -- 172.21.15.139:0/1189908970 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c1a6840 con 0x7f4d2c1009b0 2024-09-16T01:41:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.688+0000 7f4d2bfff640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101350 0x7f4d2c197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:41:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.689+0000 7f4d30c9f640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d2c1009b0 0x7f4d2c196fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000e9f0 tx=0x7f4d2000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.689+0000 7f4d29ffb640 1 -- 172.21.15.139:0/1189908970 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2000ccc0 con 0x7f4d2c1009b0 2024-09-16T01:41:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.689+0000 7f4d32f03640 1 -- 172.21.15.139:0/1189908970 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2c1a6b30 con 0x7f4d2c1009b0 2024-09-16T01:41:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.689+0000 7f4d29ffb640 1 -- 172.21.15.139:0/1189908970 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d20005020 con 0x7f4d2c1009b0 2024-09-16T01:41:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.689+0000 7f4d29ffb640 1 -- 172.21.15.139:0/1189908970 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20004a00 con 0x7f4d2c1009b0 2024-09-16T01:41:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.689+0000 7f4d32f03640 1 -- 172.21.15.139:0/1189908970 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2c1a7080 con 0x7f4d2c1009b0 2024-09-16T01:41:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.692+0000 7f4d29ffb640 1 -- 172.21.15.139:0/1189908970 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 24) ==== 138589+0+0 (secure 0 0 0) 0x7f4d200051d0 con 0x7f4d2c1009b0 2024-09-16T01:41:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.692+0000 7f4d35c9c740 1 Processor -- start 2024-09-16T01:41:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.693+0000 7f4d35c9c740 1 -- start start 2024-09-16T01:41:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.693+0000 7f4d35c9c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8eaf0 0x561d21d8eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.693+0000 7f4d35c9c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8f410 0x561d21d99770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.693+0000 7f4d35c9c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21d9c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.693+0000 7f4d35c9c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561d21cf32f0 con 0x561d21d99cb0 2024-09-16T01:41:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.693+0000 7f4d35c9c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561d21c9b270 con 0x561d21d8f410 2024-09-16T01:41:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.693+0000 7f4d35c9c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561d21cef2b0 con 0x561d21d8eaf0 2024-09-16T01:41:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.693+0000 7f4d314a0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8eaf0 0x561d21d8eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.693+0000 7f4d314a0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8eaf0 0x561d21d8eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:34004/0 (socket says 172.21.15.139:34004) 2024-09-16T01:41:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.693+0000 7f4d314a0640 1 -- 172.21.15.139:0/934757905 learned_addr learned my addr 172.21.15.139:0/934757905 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.693+0000 7f4d30c9f640 1 --2- 172.21.15.139:0/934757905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8f410 0x561d21d99770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.693+0000 7f4d2bfff640 1 --2- 172.21.15.139:0/934757905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21d9c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.694+0000 7f4d29ffb640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f4d1c0a3600 0x7f4d1c0a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.694+0000 7f4d314a0640 1 -- 172.21.15.139:0/934757905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8f410 msgr2=0x561d21d99770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.694+0000 7f4d314a0640 1 --2- 172.21.15.139:0/934757905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8f410 0x561d21d99770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.694+0000 7f4d29ffb640 1 -- 172.21.15.139:0/1189908970 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f4d200d01e0 con 0x7f4d2c1009b0 2024-09-16T01:41:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.694+0000 7f4d314a0640 1 -- 172.21.15.139:0/934757905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 msgr2=0x561d21d9c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.694+0000 7f4d314a0640 1 --2- 172.21.15.139:0/934757905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21d9c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.694+0000 7f4d314a0640 1 -- 172.21.15.139:0/934757905 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d21c9a340 con 0x561d21d8eaf0 2024-09-16T01:41:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.694+0000 7f4d2bfff640 1 --2- 172.21.15.139:0/934757905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21d9c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:41:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.694+0000 7f4d30c9f640 1 --2- 172.21.15.139:0/934757905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8f410 0x561d21d99770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:41:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.694+0000 7f4d314a0640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f4d1c0a3600 0x7f4d1c0a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.694+0000 7f4d314a0640 1 --2- 172.21.15.139:0/934757905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8eaf0 0x561d21d8eed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4d240114e0 tx=0x7f4d240119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c382a37a5d8a3da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.695+0000 7f4d314a0640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f4d1c0a3600 0x7f4d1c0a5ac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4d24023000 tx=0x7f4d24024000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.695+0000 7f4d16ffd640 1 -- 172.21.15.139:0/934757905 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24017050 con 0x561d21d8eaf0 2024-09-16T01:41:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.695+0000 7f4d16ffd640 1 -- 172.21.15.139:0/934757905 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d24004de0 con 0x561d21d8eaf0 2024-09-16T01:41:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.695+0000 7f4d16ffd640 1 -- 172.21.15.139:0/934757905 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24005270 con 0x561d21d8eaf0 2024-09-16T01:41:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.696+0000 7f4d35c9c740 1 -- 172.21.15.139:0/934757905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8eaf0 msgr2=0x561d21d8eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.696+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/934757905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8eaf0 0x561d21d8eed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4d240114e0 tx=0x7f4d240119a0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.696+0000 7f4d35c9c740 1 -- 172.21.15.139:0/934757905 shutdown_connections 2024-09-16T01:41:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.696+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/934757905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21d9c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.696+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/934757905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8f410 0x561d21d99770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.696+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/934757905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8eaf0 0x561d21d8eed0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.696+0000 7f4d35c9c740 1 -- 172.21.15.139:0/934757905 >> 172.21.15.139:0/934757905 conn(0x561d21d8b060 msgr2=0x561d21d8b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.696+0000 7f4d35c9c740 1 -- 172.21.15.139:0/934757905 shutdown_connections 2024-09-16T01:41:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.696+0000 7f4d35c9c740 1 -- 172.21.15.139:0/934757905 wait complete. 2024-09-16T01:41:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.697+0000 7f4d35c9c740 1 -- start start 2024-09-16T01:41:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.697+0000 7f4d35c9c740 1 Processor -- start 2024-09-16T01:41:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.697+0000 7f4d35c9c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8eaf0 0x561d21d96c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d35c9c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 0x561d21d97140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d35c9c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21d97680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d314a0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21d97680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d314a0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21d97680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43188/0 (socket says 172.21.15.139:43188) 2024-09-16T01:41:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d314a0640 1 -- 172.21.15.139:0/2537104543 learned_addr learned my addr 172.21.15.139:0/2537104543 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d30c9f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8eaf0 0x561d21d96c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d35c9c740 1 -- 172.21.15.139:0/2537104543 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561d21d9c880 con 0x561d21d99cb0 2024-09-16T01:41:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d35c9c740 1 -- 172.21.15.139:0/2537104543 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561d21c7c290 con 0x561d21d8eaf0 2024-09-16T01:41:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d35c9c740 1 -- 172.21.15.139:0/2537104543 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561d21d9ca00 con 0x561d21d8f410 2024-09-16T01:41:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d35c9c740 10 client.0 _ll_register_callbacks cb 0x561d21cad140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:41:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d2bfff640 1 --2- 172.21.15.139:0/2537104543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 0x561d21d97140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d30c9f640 1 -- 172.21.15.139:0/2537104543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 msgr2=0x561d21d97140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d30c9f640 1 --2- 172.21.15.139:0/2537104543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 0x561d21d97140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d30c9f640 1 -- 172.21.15.139:0/2537104543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 msgr2=0x561d21d97680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d30c9f640 1 --2- 172.21.15.139:0/2537104543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21d97680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.698+0000 7f4d30c9f640 1 -- 172.21.15.139:0/2537104543 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d21a06210 con 0x561d21d8eaf0 2024-09-16T01:41:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.699+0000 7f4d314a0640 1 --2- 172.21.15.139:0/2537104543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21d97680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:41:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.699+0000 7f4d35c9c740 1 -- 172.21.15.139:0/2537104543 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d24004de0 con 0x561d21d8eaf0 2024-09-16T01:41:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.699+0000 7f4d30c9f640 1 --2- 172.21.15.139:0/2537104543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8eaf0 0x561d21d96c00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4d200cd3b0 tx=0x7f4d2009c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b96785799154665 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.699+0000 7f4d0bfff640 10 client.25184 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:41:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.699+0000 7f4d0bfff640 1 -- 172.21.15.139:0/2537104543 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2001c040 con 0x561d21d8eaf0 2024-09-16T01:41:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.699+0000 7f4d0bfff640 1 -- 172.21.15.139:0/2537104543 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f4d200a4680 con 0x561d21d8eaf0 2024-09-16T01:41:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.699+0000 7f4d0bfff640 1 -- 172.21.15.139:0/2537104543 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4d200a4880 con 0x561d21d8eaf0 2024-09-16T01:41:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.699+0000 7f4d35c9c740 10 client.25184 fetch_fsmap learned FSMap version 29 2024-09-16T01:41:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.699+0000 7f4d35c9c740 1 -- 172.21.15.139:0/2537104543 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=29}) -- 0x561d21ad2620 con 0x561d21d8eaf0 2024-09-16T01:41:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.700+0000 7f4d0bfff640 1 -- 172.21.15.139:0/2537104543 <== mon.2 v2:172.21.15.164:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4d20004ba0 con 0x561d21d8eaf0 2024-09-16T01:41:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.700+0000 7f4d35c9c740 10 client.25184 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:41:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.700+0000 7f4d35c9c740 10 client.25184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:41:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.700+0000 7f4d35c9c740 20 client.25184 populate_metadata read hostname 'smithi139' 2024-09-16T01:41:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.700+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/2537104543 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d21da9a30 0x561d21dabe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.700+0000 7f4d35c9c740 4 client.25184 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:41:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.700+0000 7f4d35c9c740 1 -- 172.21.15.139:0/2537104543 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d21d00a20 con 0x561d21da9a30 2024-09-16T01:41:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.700+0000 7f4d2bfff640 1 --2- 172.21.15.139:0/2537104543 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d21da9a30 0x561d21dabe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.701+0000 7f4d0bfff640 10 client.25184 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.701+0000 7f4d2bfff640 1 --2- 172.21.15.139:0/2537104543 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d21da9a30 0x561d21dabe20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b50c3e28995b781b server_cookie=b9fcfc5e38481ac9 in_seq=0 out_seq=0 2024-09-16T01:41:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.701+0000 7f4d0bfff640 1 -- 172.21.15.139:0/2537104543 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d21d00a20 con 0x561d21da9a30 2024-09-16T01:41:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.701+0000 7f4d0bfff640 10 client.25184 handle_command_reply: tid=0 2024-09-16T01:41:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.701+0000 7f4d35c9c740 1 client.25184 shutdown 2024-09-16T01:41:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.701+0000 7f4d35c9c740 1 -- 172.21.15.139:0/2537104543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8eaf0 msgr2=0x561d21d96c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.701+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/2537104543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8eaf0 0x561d21d96c00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4d200cd3b0 tx=0x7f4d2009c660 comp rx=0 tx=0).stop 2024-09-16T01:41:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.701+0000 7f4d35c9c740 1 -- 172.21.15.139:0/2537104543 shutdown_connections 2024-09-16T01:41:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.701+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/2537104543 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d21da9a30 0x561d21dabe20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.701+0000 7f4d0bfff640 0 client.25184 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.701+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/2537104543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21d97680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.701+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/2537104543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 0x561d21d97140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.702+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/2537104543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8eaf0 0x561d21d96c00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.702+0000 7f4d35c9c740 1 -- 172.21.15.139:0/2537104543 >> 172.21.15.139:0/2537104543 conn(0x561d21d8b060 msgr2=0x561d21d90d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.702+0000 7f4d35c9c740 1 -- 172.21.15.139:0/2537104543 shutdown_connections 2024-09-16T01:41:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.702+0000 7f4d35c9c740 1 -- 172.21.15.139:0/2537104543 wait complete. 2024-09-16T01:41:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.702+0000 7f4d35c9c740 20 client.25184 trim_cache size 0 max 16384 2024-09-16T01:41:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.730+0000 7f4d35c9c740 1 Processor -- start 2024-09-16T01:41:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.730+0000 7f4d35c9c740 1 -- start start 2024-09-16T01:41:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.730+0000 7f4d35c9c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8eaf0 0x561d21d974b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.730+0000 7f4d35c9c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 0x561d21db1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.730+0000 7f4d35c9c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21db1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.730+0000 7f4d35c9c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1b1140 con 0x561d21d99cb0 2024-09-16T01:41:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.730+0000 7f4d35c9c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1b2c80 con 0x561d21d8eaf0 2024-09-16T01:41:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.730+0000 7f4d35c9c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1b1810 con 0x561d21d8f410 2024-09-16T01:41:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.730+0000 7f4d314a0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21db1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.730+0000 7f4d314a0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21db1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43198/0 (socket says 172.21.15.139:43198) 2024-09-16T01:41:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.730+0000 7f4d314a0640 1 -- 172.21.15.139:0/691313002 learned_addr learned my addr 172.21.15.139:0/691313002 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.730+0000 7f4d30c9f640 1 --2- 172.21.15.139:0/691313002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8eaf0 0x561d21d974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.731+0000 7f4d2bfff640 1 --2- 172.21.15.139:0/691313002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 0x561d21db1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.731+0000 7f4d314a0640 1 -- 172.21.15.139:0/691313002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 msgr2=0x561d21db1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.731+0000 7f4d314a0640 1 --2- 172.21.15.139:0/691313002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 0x561d21db1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.731+0000 7f4d314a0640 1 -- 172.21.15.139:0/691313002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8eaf0 msgr2=0x561d21d974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.731+0000 7f4d314a0640 1 --2- 172.21.15.139:0/691313002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8eaf0 0x561d21d974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.731+0000 7f4d314a0640 1 -- 172.21.15.139:0/691313002 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d21d8bcf0 con 0x561d21d99cb0 2024-09-16T01:41:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.731+0000 7f4d30c9f640 1 --2- 172.21.15.139:0/691313002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8eaf0 0x561d21d974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:41:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.731+0000 7f4d314a0640 1 --2- 172.21.15.139:0/691313002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21db1c40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4d24006010 tx=0x7f4d2400ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd963c9d6e7591cf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.731+0000 7f4d167fc640 1 -- 172.21.15.139:0/691313002 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d240049e0 con 0x561d21d99cb0 2024-09-16T01:41:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.731+0000 7f4d167fc640 1 -- 172.21.15.139:0/691313002 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d24005050 con 0x561d21d99cb0 2024-09-16T01:41:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.731+0000 7f4d167fc640 1 -- 172.21.15.139:0/691313002 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24004b80 con 0x561d21d99cb0 2024-09-16T01:41:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.732+0000 7f4d35c9c740 1 -- 172.21.15.139:0/691313002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 msgr2=0x561d21db1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.732+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/691313002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21db1c40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4d24006010 tx=0x7f4d2400ffd0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.732+0000 7f4d35c9c740 1 -- 172.21.15.139:0/691313002 shutdown_connections 2024-09-16T01:41:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.732+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/691313002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d99cb0 0x561d21db1c40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.732+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/691313002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 0x561d21db1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.732+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/691313002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d8eaf0 0x561d21d974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.732+0000 7f4d35c9c740 1 -- 172.21.15.139:0/691313002 >> 172.21.15.139:0/691313002 conn(0x561d21d8b060 msgr2=0x561d21d98c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.733+0000 7f4d35c9c740 1 -- 172.21.15.139:0/691313002 shutdown_connections 2024-09-16T01:41:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.733+0000 7f4d35c9c740 1 -- 172.21.15.139:0/691313002 wait complete. 2024-09-16T01:41:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.733+0000 7f4d35c9c740 1 -- start start 2024-09-16T01:41:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.733+0000 7f4d35c9c740 1 Processor -- start 2024-09-16T01:41:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d35c9c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d8eaf0 0x561d21d93c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d35c9c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 0x561d21d94180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d35c9c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d99cb0 0x561d21d946c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d30c9f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d8eaf0 0x561d21d93c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d35c9c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1b25b0 con 0x561d21d8eaf0 2024-09-16T01:41:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d35c9c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1b1ee0 con 0x561d21d99cb0 2024-09-16T01:41:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d35c9c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1af120 con 0x561d21d8f410 2024-09-16T01:41:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d30c9f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d8eaf0 0x561d21d93c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43210/0 (socket says 172.21.15.139:43210) 2024-09-16T01:41:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d30c9f640 1 -- 172.21.15.139:0/172214710 learned_addr learned my addr 172.21.15.139:0/172214710 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d35c9c740 10 client.0 _ll_register_callbacks cb 0x7f4d20009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:41:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d2bfff640 1 --2- 172.21.15.139:0/172214710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 0x561d21d94180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d314a0640 1 --2- 172.21.15.139:0/172214710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d99cb0 0x561d21d946c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d30c9f640 1 -- 172.21.15.139:0/172214710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 msgr2=0x561d21d94180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d30c9f640 1 --2- 172.21.15.139:0/172214710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 0x561d21d94180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d30c9f640 1 -- 172.21.15.139:0/172214710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d99cb0 msgr2=0x561d21d946c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d30c9f640 1 --2- 172.21.15.139:0/172214710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d99cb0 0x561d21d946c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d30c9f640 1 -- 172.21.15.139:0/172214710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d21e136d0 con 0x561d21d8eaf0 2024-09-16T01:41:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d314a0640 1 --2- 172.21.15.139:0/172214710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d99cb0 0x561d21d946c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:41:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d30c9f640 1 --2- 172.21.15.139:0/172214710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d8eaf0 0x561d21d93c40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4d2009a150 tx=0x7f4d200d2670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cb7da09a1c2a52 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:41:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.734+0000 7f4d0bfff640 1 -- 172.21.15.139:0/172214710 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2001c040 con 0x561d21d8eaf0 2024-09-16T01:41:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.735+0000 7f4d0bfff640 1 -- 172.21.15.139:0/172214710 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f4d200a4680 con 0x561d21d8eaf0 2024-09-16T01:41:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.735+0000 7f4d35c9c740 1 -- 172.21.15.139:0/172214710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d24005050 con 0x561d21d8eaf0 2024-09-16T01:41:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.735+0000 7f4d0bfff640 1 -- 172.21.15.139:0/172214710 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4d2009b800 con 0x561d21d8eaf0 2024-09-16T01:41:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.735+0000 7f4d35c9c740 10 client.15663 fetch_fsmap learned FSMap version 29 2024-09-16T01:41:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.735+0000 7f4d35c9c740 1 -- 172.21.15.139:0/172214710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561d21e136d0 con 0x561d21d8eaf0 2024-09-16T01:41:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.735+0000 7f4d0bfff640 1 -- 172.21.15.139:0/172214710 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d24005050 con 0x561d21d8eaf0 2024-09-16T01:41:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.735+0000 7f4d0bfff640 1 -- 172.21.15.139:0/172214710 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4d200baae0 con 0x561d21d8eaf0 2024-09-16T01:41:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.735+0000 7f4d35c9c740 10 client.15663 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:41:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.735+0000 7f4d35c9c740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:41:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.735+0000 7f4d35c9c740 20 client.15663 populate_metadata read hostname 'smithi139' 2024-09-16T01:41:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.735+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/172214710 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d21e14d80 0x561d21e17170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.735+0000 7f4d35c9c740 4 client.15663 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:41:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.735+0000 7f4d35c9c740 1 -- 172.21.15.139:0/172214710 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d21cfad70 con 0x561d21e14d80 2024-09-16T01:41:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.736+0000 7f4d2bfff640 1 --2- 172.21.15.139:0/172214710 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d21e14d80 0x561d21e17170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.736+0000 7f4d2bfff640 1 --2- 172.21.15.139:0/172214710 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d21e14d80 0x561d21e17170 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93d822fed664fde8 server_cookie=87f63d9d4eb77d63 in_seq=0 out_seq=0 2024-09-16T01:41:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.736+0000 7f4d0bfff640 10 client.15663 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d0bfff640 1 -- 172.21.15.139:0/172214710 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d21cfad70 con 0x561d21e14d80 2024-09-16T01:41:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d0bfff640 10 client.15663 handle_command_reply: tid=0 2024-09-16T01:41:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d35c9c740 1 client.15663 shutdown 2024-09-16T01:41:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d35c9c740 1 -- 172.21.15.139:0/172214710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d8eaf0 msgr2=0x561d21d93c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/172214710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d8eaf0 0x561d21d93c40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4d2009a150 tx=0x7f4d200d2670 comp rx=0 tx=0).stop 2024-09-16T01:41:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d35c9c740 1 -- 172.21.15.139:0/172214710 shutdown_connections 2024-09-16T01:41:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/172214710 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d21e14d80 0x561d21e17170 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/172214710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d21d99cb0 0x561d21d946c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/172214710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d21d8f410 0x561d21d94180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d0bfff640 0 client.15663 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d35c9c740 1 --2- 172.21.15.139:0/172214710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d21d8eaf0 0x561d21d93c40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d35c9c740 1 -- 172.21.15.139:0/172214710 >> 172.21.15.139:0/172214710 conn(0x561d21d8b060 msgr2=0x561d21d90550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d35c9c740 1 -- 172.21.15.139:0/172214710 shutdown_connections 2024-09-16T01:41:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d35c9c740 1 -- 172.21.15.139:0/172214710 wait complete. 2024-09-16T01:41:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.740+0000 7f4d35c9c740 20 client.15663 trim_cache size 0 max 16384 2024-09-16T01:41:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:41:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:41:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:41:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:41:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.741+0000 7f4d32f03640 1 -- 172.21.15.139:0/1189908970 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f4d1c0a3600 msgr2=0x7f4d1c0a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.741+0000 7f4d32f03640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f4d1c0a3600 0x7f4d1c0a5ac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4d24023000 tx=0x7f4d24024000 comp rx=0 tx=0).stop 2024-09-16T01:41:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.741+0000 7f4d32f03640 1 -- 172.21.15.139:0/1189908970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d2c1009b0 msgr2=0x7f4d2c196fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.741+0000 7f4d32f03640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d2c1009b0 0x7f4d2c196fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000e9f0 tx=0x7f4d2000edf0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.741+0000 7f4d32f03640 1 -- 172.21.15.139:0/1189908970 shutdown_connections 2024-09-16T01:41:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.741+0000 7f4d32f03640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f4d1c0a3600 0x7f4d1c0a5ac0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.741+0000 7f4d32f03640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d2c1069c0 0x7f4d2c197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.741+0000 7f4d32f03640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2c101350 0x7f4d2c197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.741+0000 7f4d32f03640 1 --2- 172.21.15.139:0/1189908970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d2c1009b0 0x7f4d2c196fd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.741+0000 7f4d32f03640 1 -- 172.21.15.139:0/1189908970 >> 172.21.15.139:0/1189908970 conn(0x7f4d2c077990 msgr2=0x7f4d2c1037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.741+0000 7f4d32f03640 1 -- 172.21.15.139:0/1189908970 shutdown_connections 2024-09-16T01:41:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:31.741+0000 7f4d32f03640 1 -- 172.21.15.139:0/1189908970 wait complete. 2024-09-16T01:41:32.744 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:41:32.994 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:41:33.115 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:32 smithi139 ceph-mon[31175]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T01:41:33.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:32 smithi164 ceph-mon[35121]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T01:41:33.505 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:32 smithi176 ceph-mon[35324]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T01:41:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.503+0000 7f9a2730b640 1 -- 172.21.15.139:0/1946374990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a20102ef0 msgr2=0x7f9a2010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.503+0000 7f9a2730b640 1 --2- 172.21.15.139:0/1946374990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a20102ef0 0x7f9a2010da40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009ca0 tx=0x7f9a1802a950 comp rx=0 tx=0).stop 2024-09-16T01:41:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.503+0000 7f9a2730b640 1 -- 172.21.15.139:0/1946374990 shutdown_connections 2024-09-16T01:41:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.503+0000 7f9a2730b640 1 --2- 172.21.15.139:0/1946374990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a20102ef0 0x7f9a2010da40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.503+0000 7f9a2730b640 1 --2- 172.21.15.139:0/1946374990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a20102570 0x7f9a201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.503+0000 7f9a2730b640 1 --2- 172.21.15.139:0/1946374990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20100e20 0x7f9a20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.504+0000 7f9a2730b640 1 -- 172.21.15.139:0/1946374990 >> 172.21.15.139:0/1946374990 conn(0x7f9a200fc7d0 msgr2=0x7f9a200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.504+0000 7f9a2730b640 1 -- 172.21.15.139:0/1946374990 shutdown_connections 2024-09-16T01:41:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.504+0000 7f9a2730b640 1 -- 172.21.15.139:0/1946374990 wait complete. 2024-09-16T01:41:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.504+0000 7f9a2730b640 1 Processor -- start 2024-09-16T01:41:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.504+0000 7f9a2730b640 1 -- start start 2024-09-16T01:41:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.504+0000 7f9a2730b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20100e20 0x7f9a201984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.504+0000 7f9a2730b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a20102570 0x7f9a20198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.504+0000 7f9a2730b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a20102ef0 0x7f9a2019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.504+0000 7f9a2730b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20110f80 con 0x7f9a20102ef0 2024-09-16T01:41:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.504+0000 7f9a2730b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20110e00 con 0x7f9a20102570 2024-09-16T01:41:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.504+0000 7f9a2730b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20111100 con 0x7f9a20100e20 2024-09-16T01:41:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a258a8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a20102ef0 0x7f9a2019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a258a8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a20102ef0 0x7f9a2019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:32922/0 (socket says 172.21.15.139:32922) 2024-09-16T01:41:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a258a8640 1 -- 172.21.15.139:0/353236608 learned_addr learned my addr 172.21.15.139:0/353236608 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a250a7640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20100e20 0x7f9a201984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a248a6640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a20102570 0x7f9a20198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a258a8640 1 -- 172.21.15.139:0/353236608 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20100e20 msgr2=0x7f9a201984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a258a8640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20100e20 0x7f9a201984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a258a8640 1 -- 172.21.15.139:0/353236608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a20102570 msgr2=0x7f9a20198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a258a8640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a20102570 0x7f9a20198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a258a8640 1 -- 172.21.15.139:0/353236608 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2019d480 con 0x7f9a20102ef0 2024-09-16T01:41:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a248a6640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a20102570 0x7f9a20198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:41:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a258a8640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a20102ef0 0x7f9a2019cda0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9a18004af0 tx=0x7f9a1802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a167fc640 1 -- 172.21.15.139:0/353236608 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18034040 con 0x7f9a20102ef0 2024-09-16T01:41:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a167fc640 1 -- 172.21.15.139:0/353236608 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a18030920 con 0x7f9a20102ef0 2024-09-16T01:41:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a167fc640 1 -- 172.21.15.139:0/353236608 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18004820 con 0x7f9a20102ef0 2024-09-16T01:41:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a2730b640 1 -- 172.21.15.139:0/353236608 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2019d710 con 0x7f9a20102ef0 2024-09-16T01:41:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.505+0000 7f9a2730b640 1 -- 172.21.15.139:0/353236608 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a201a5050 con 0x7f9a20102ef0 2024-09-16T01:41:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.507+0000 7f9a2730b640 1 -- 172.21.15.139:0/353236608 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a201021b0 con 0x7f9a20102ef0 2024-09-16T01:41:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.508+0000 7f9a167fc640 1 -- 172.21.15.139:0/353236608 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 24) ==== 138589+0+0 (secure 0 0 0) 0x7f9a18037020 con 0x7f9a20102ef0 2024-09-16T01:41:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.509+0000 7f9a167fc640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9a0c0a3550 0x7f9a0c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.509+0000 7f9a250a7640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9a0c0a3550 0x7f9a0c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.510+0000 7f9a167fc640 1 -- 172.21.15.139:0/353236608 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f9a180f10e0 con 0x7f9a20102ef0 2024-09-16T01:41:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.510+0000 7f9a250a7640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9a0c0a3550 0x7f9a0c0a5a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c005fd0 tx=0x7f9a1c00b390 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.514+0000 7f9a167fc640 1 -- 172.21.15.139:0/353236608 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a180bd5d0 con 0x7f9a20102ef0 2024-09-16T01:41:33.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.729+0000 7f9a2730b640 1 -- 172.21.15.139:0/353236608 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a2010a0a0 con 0x7f9a20102ef0 2024-09-16T01:41:33.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.730+0000 7f9a167fc640 1 -- 172.21.15.139:0/353236608 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11995 (secure 0 0 0) 0x7f9a180bced0 con 0x7f9a20102ef0 2024-09-16T01:41:33.731 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:41:33.731 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":29,"btime":"2024-09-16T01:40:22:367807+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:40:22.317639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:41:33.731 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 29 2024-09-16T01:41:33.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.732+0000 7f9a2730b640 1 -- 172.21.15.139:0/353236608 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9a0c0a3550 msgr2=0x7f9a0c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:33.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.732+0000 7f9a2730b640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9a0c0a3550 0x7f9a0c0a5a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c005fd0 tx=0x7f9a1c00b390 comp rx=0 tx=0).stop 2024-09-16T01:41:33.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.732+0000 7f9a2730b640 1 -- 172.21.15.139:0/353236608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a20102ef0 msgr2=0x7f9a2019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:33.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.732+0000 7f9a2730b640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a20102ef0 0x7f9a2019cda0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9a18004af0 tx=0x7f9a1802f5b0 comp rx=0 tx=0).stop 2024-09-16T01:41:33.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.732+0000 7f9a2730b640 1 -- 172.21.15.139:0/353236608 shutdown_connections 2024-09-16T01:41:33.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.732+0000 7f9a2730b640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9a0c0a3550 0x7f9a0c0a5a10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:33.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.732+0000 7f9a2730b640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a20102ef0 0x7f9a2019cda0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:33.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.732+0000 7f9a2730b640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a20102570 0x7f9a20198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:33.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.732+0000 7f9a2730b640 1 --2- 172.21.15.139:0/353236608 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20100e20 0x7f9a201984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:33.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.732+0000 7f9a2730b640 1 -- 172.21.15.139:0/353236608 >> 172.21.15.139:0/353236608 conn(0x7f9a200fc7d0 msgr2=0x7f9a2006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:33.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.732+0000 7f9a2730b640 1 -- 172.21.15.139:0/353236608 shutdown_connections 2024-09-16T01:41:33.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:33.732+0000 7f9a2730b640 1 -- 172.21.15.139:0/353236608 wait complete. 2024-09-16T01:41:34.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:33 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/353236608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:41:34.330 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:33 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/353236608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:41:34.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:33 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/353236608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:41:34.633 INFO:tasks.workunit.client.0.smithi139.stderr:Updating files: 95% (12257/12889) Updating files: 96% (12374/12889) Updating files: 97% (12503/12889) Updating files: 98% (12632/12889) Updating files: 99% (12761/12889) Updating files: 100% (12889/12889) Updating files: 100% (12889/12889), done. 2024-09-16T01:41:34.824 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 72f21021 --release --await' 2024-09-16T01:41:34.824 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 72f21021 --release --await 2024-09-16T01:41:35.078 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:41:35.249 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:34 smithi139 ceph-mon[31175]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T01:41:35.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:34 smithi164 ceph-mon[35121]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T01:41:35.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:34 smithi176 ceph-mon[35324]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T01:41:35.456 INFO:tasks.workunit.client.0.smithi139.stderr:Note: switching to 'ad35db06e8924912f01e32b35576297c0f1352b4'. 2024-09-16T01:41:35.456 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T01:41:35.456 INFO:tasks.workunit.client.0.smithi139.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T01:41:35.456 INFO:tasks.workunit.client.0.smithi139.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T01:41:35.456 INFO:tasks.workunit.client.0.smithi139.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T01:41:35.457 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T01:41:35.457 INFO:tasks.workunit.client.0.smithi139.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T01:41:35.457 INFO:tasks.workunit.client.0.smithi139.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T01:41:35.457 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T01:41:35.457 INFO:tasks.workunit.client.0.smithi139.stderr: git switch -c 2024-09-16T01:41:35.457 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T01:41:35.457 INFO:tasks.workunit.client.0.smithi139.stderr:Or undo this operation with: 2024-09-16T01:41:35.457 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T01:41:35.457 INFO:tasks.workunit.client.0.smithi139.stderr: git switch - 2024-09-16T01:41:35.457 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T01:41:35.457 INFO:tasks.workunit.client.0.smithi139.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T01:41:35.457 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T01:41:35.457 INFO:tasks.workunit.client.0.smithi139.stderr:HEAD is now at ad35db06e89 Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid 2024-09-16T01:41:35.461 DEBUG:teuthology.orchestra.run.smithi139:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-16T01:41:35.531 INFO:tasks.workunit.client.0.smithi139.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T01:41:35.533 INFO:tasks.workunit.client.0.smithi139.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T01:41:35.533 INFO:tasks.workunit.client.0.smithi139.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T01:41:35.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.678+0000 7fcb4f43d640 1 -- 172.21.15.139:0/621953917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb48106f50 msgr2=0x7fcb4810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:35.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.678+0000 7fcb4f43d640 1 --2- 172.21.15.139:0/621953917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb48106f50 0x7fcb4810eaf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009cd0 tx=0x7fcb4002a8e0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.678+0000 7fcb4f43d640 1 -- 172.21.15.139:0/621953917 shutdown_connections 2024-09-16T01:41:35.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.678+0000 7fcb4f43d640 1 --2- 172.21.15.139:0/621953917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb48106f50 0x7fcb4810eaf0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.678+0000 7fcb4f43d640 1 --2- 172.21.15.139:0/621953917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb48100ca0 0x7fcb48106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.678+0000 7fcb4f43d640 1 --2- 172.21.15.139:0/621953917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb481002f0 0x7fcb481006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.678+0000 7fcb4f43d640 1 -- 172.21.15.139:0/621953917 >> 172.21.15.139:0/621953917 conn(0x7fcb48077210 msgr2=0x7fcb480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:35.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.678+0000 7fcb4f43d640 1 -- 172.21.15.139:0/621953917 shutdown_connections 2024-09-16T01:41:35.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.678+0000 7fcb4f43d640 1 -- 172.21.15.139:0/621953917 wait complete. 2024-09-16T01:41:35.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.679+0000 7fcb4f43d640 1 Processor -- start 2024-09-16T01:41:35.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.679+0000 7fcb4f43d640 1 -- start start 2024-09-16T01:41:35.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.679+0000 7fcb4f43d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb481002f0 0x7fcb481983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4f43d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb48100ca0 0x7fcb48198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4f43d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb48106f50 0x7fcb4819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4f43d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcb48111fa0 con 0x7fcb48100ca0 2024-09-16T01:41:35.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4f43d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb48111e20 con 0x7fcb481002f0 2024-09-16T01:41:35.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4f43d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb48112120 con 0x7fcb48106f50 2024-09-16T01:41:35.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4c9d8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb48100ca0 0x7fcb48198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4c9d8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb48100ca0 0x7fcb48198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:32938/0 (socket says 172.21.15.139:32938) 2024-09-16T01:41:35.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4c9d8640 1 -- 172.21.15.139:0/2195541690 learned_addr learned my addr 172.21.15.139:0/2195541690 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:35.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4d9da640 1 --2- 172.21.15.139:0/2195541690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb48106f50 0x7fcb4819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4d1d9640 1 --2- 172.21.15.139:0/2195541690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb481002f0 0x7fcb481983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4c9d8640 1 -- 172.21.15.139:0/2195541690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb48106f50 msgr2=0x7fcb4819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:35.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4c9d8640 1 --2- 172.21.15.139:0/2195541690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb48106f50 0x7fcb4819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4c9d8640 1 -- 172.21.15.139:0/2195541690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb481002f0 msgr2=0x7fcb481983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:35.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4c9d8640 1 --2- 172.21.15.139:0/2195541690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb481002f0 0x7fcb481983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4c9d8640 1 -- 172.21.15.139:0/2195541690 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4819d3a0 con 0x7fcb48100ca0 2024-09-16T01:41:35.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4c9d8640 1 --2- 172.21.15.139:0/2195541690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb48100ca0 0x7fcb48198930 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400d740 tx=0x7fcb4400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:35.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb3a7fc640 1 -- 172.21.15.139:0/2195541690 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44017070 con 0x7fcb48100ca0 2024-09-16T01:41:35.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4f43d640 1 -- 172.21.15.139:0/2195541690 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4819d690 con 0x7fcb48100ca0 2024-09-16T01:41:35.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb3a7fc640 1 -- 172.21.15.139:0/2195541690 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb4400de50 con 0x7fcb48100ca0 2024-09-16T01:41:35.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb3a7fc640 1 -- 172.21.15.139:0/2195541690 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44004e00 con 0x7fcb48100ca0 2024-09-16T01:41:35.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.680+0000 7fcb4f43d640 1 -- 172.21.15.139:0/2195541690 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb481a4f70 con 0x7fcb48100ca0 2024-09-16T01:41:35.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.683+0000 7fcb3a7fc640 1 -- 172.21.15.139:0/2195541690 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 24) ==== 138589+0+0 (secure 0 0 0) 0x7fcb44004fa0 con 0x7fcb48100ca0 2024-09-16T01:41:35.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.683+0000 7fcb51816740 1 Processor -- start 2024-09-16T01:41:35.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.683+0000 7fcb51816740 1 -- start start 2024-09-16T01:41:35.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb3a7fc640 1 --2- 172.21.15.139:0/2195541690 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcb340a34d0 0x7fcb340a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb51816740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555060 0x55b9b7555440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb51816740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7555980 0x55b9b755fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb51816740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7560240 0x55b9b7562630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb4d9da640 1 --2- 172.21.15.139:0/2195541690 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcb340a34d0 0x7fcb340a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb51816740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b9b74ba620 con 0x55b9b7560240 2024-09-16T01:41:35.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb51816740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b9b7461e50 con 0x55b9b7555060 2024-09-16T01:41:35.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb51816740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b9b74b65e0 con 0x55b9b7555980 2024-09-16T01:41:35.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb3a7fc640 1 -- 172.21.15.139:0/2195541690 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fcb440cfcc0 con 0x7fcb48100ca0 2024-09-16T01:41:35.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb4d9da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7560240 0x55b9b7562630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb4d9da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7560240 0x55b9b7562630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:32954/0 (socket says 172.21.15.139:32954) 2024-09-16T01:41:35.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb4d9da640 1 -- 172.21.15.139:0/1187756157 learned_addr learned my addr 172.21.15.139:0/1187756157 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:35.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb4c9d8640 1 --2- 172.21.15.139:0/1187756157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7555980 0x55b9b755fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb4d1d9640 1 --2- 172.21.15.139:0/1187756157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555060 0x55b9b7555440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb4d9da640 1 --2- 172.21.15.139:0/2195541690 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcb340a34d0 0x7fcb340a5990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcb4002ae00 tx=0x7fcb400317a0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:35.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb4c9d8640 1 -- 172.21.15.139:0/1187756157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555060 msgr2=0x55b9b7555440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:35.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb4c9d8640 1 --2- 172.21.15.139:0/1187756157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555060 0x55b9b7555440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb4c9d8640 1 -- 172.21.15.139:0/1187756157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7560240 msgr2=0x55b9b7562630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb4c9d8640 1 --2- 172.21.15.139:0/1187756157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7560240 0x55b9b7562630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb4c9d8640 1 -- 172.21.15.139:0/1187756157 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9b7452d40 con 0x55b9b7555980 2024-09-16T01:41:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.684+0000 7fcb4c9d8640 1 --2- 172.21.15.139:0/1187756157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7555980 0x55b9b755fd00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb440ccee0 tx=0x7fcb4409c120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d98c22b718491d89 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.685+0000 7fcb337fe640 1 -- 172.21.15.139:0/1187756157 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44020070 con 0x55b9b7555980 2024-09-16T01:41:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.685+0000 7fcb337fe640 1 -- 172.21.15.139:0/1187756157 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb440a4610 con 0x55b9b7555980 2024-09-16T01:41:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.685+0000 7fcb337fe640 1 -- 172.21.15.139:0/1187756157 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb440cd420 con 0x55b9b7555980 2024-09-16T01:41:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.685+0000 7fcb51816740 1 -- 172.21.15.139:0/1187756157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7555980 msgr2=0x55b9b755fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.685+0000 7fcb51816740 1 --2- 172.21.15.139:0/1187756157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7555980 0x55b9b755fd00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb440ccee0 tx=0x7fcb4409c120 comp rx=0 tx=0).stop 2024-09-16T01:41:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.685+0000 7fcb51816740 1 -- 172.21.15.139:0/1187756157 shutdown_connections 2024-09-16T01:41:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.685+0000 7fcb51816740 1 --2- 172.21.15.139:0/1187756157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7560240 0x55b9b7562630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.685+0000 7fcb51816740 1 --2- 172.21.15.139:0/1187756157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7555980 0x55b9b755fd00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.685+0000 7fcb51816740 1 --2- 172.21.15.139:0/1187756157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555060 0x55b9b7555440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.685+0000 7fcb51816740 1 -- 172.21.15.139:0/1187756157 >> 172.21.15.139:0/1187756157 conn(0x55b9b75515f0 msgr2=0x55b9b75519d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.685+0000 7fcb51816740 1 -- 172.21.15.139:0/1187756157 shutdown_connections 2024-09-16T01:41:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.685+0000 7fcb51816740 1 -- 172.21.15.139:0/1187756157 wait complete. 2024-09-16T01:41:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.686+0000 7fcb51816740 1 -- start start 2024-09-16T01:41:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.686+0000 7fcb51816740 1 Processor -- start 2024-09-16T01:41:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.686+0000 7fcb51816740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b755d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.686+0000 7fcb51816740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555980 0x55b9b755d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.686+0000 7fcb51816740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7560240 0x55b9b755dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.686+0000 7fcb51816740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b9b7562e10 con 0x55b9b7555060 2024-09-16T01:41:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.686+0000 7fcb51816740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b9b7442f10 con 0x55b9b7555980 2024-09-16T01:41:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.686+0000 7fcb51816740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b9b7562f90 con 0x55b9b7560240 2024-09-16T01:41:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.686+0000 7fcb51816740 10 client.0 _ll_register_callbacks cb 0x55b9b7466db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:41:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.686+0000 7fcb4d1d9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b755d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.686+0000 7fcb4d1d9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b755d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:32970/0 (socket says 172.21.15.139:32970) 2024-09-16T01:41:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.686+0000 7fcb4d1d9640 1 -- 172.21.15.139:0/403531137 learned_addr learned my addr 172.21.15.139:0/403531137 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb4d9da640 1 --2- 172.21.15.139:0/403531137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7560240 0x55b9b755dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb4c9d8640 1 --2- 172.21.15.139:0/403531137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555980 0x55b9b755d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb4d1d9640 1 -- 172.21.15.139:0/403531137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7560240 msgr2=0x55b9b755dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb4d1d9640 1 --2- 172.21.15.139:0/403531137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7560240 0x55b9b755dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb4d1d9640 1 -- 172.21.15.139:0/403531137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555980 msgr2=0x55b9b755d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb4d1d9640 1 --2- 172.21.15.139:0/403531137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555980 0x55b9b755d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb4d1d9640 1 -- 172.21.15.139:0/403531137 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9b71c9580 con 0x55b9b7555060 2024-09-16T01:41:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb4d1d9640 1 --2- 172.21.15.139:0/403531137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b755d1d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c010a70 tx=0x7fcb3c010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca9005264070da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:41:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb30ff9640 1 -- 172.21.15.139:0/403531137 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c00eda0 con 0x55b9b7555060 2024-09-16T01:41:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb30ff9640 1 -- 172.21.15.139:0/403531137 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fcb3c01c480 con 0x55b9b7555060 2024-09-16T01:41:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb51816740 1 -- 172.21.15.139:0/403531137 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb440a4610 con 0x55b9b7555060 2024-09-16T01:41:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb30ff9640 1 -- 172.21.15.139:0/403531137 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcb3c025050 con 0x55b9b7555060 2024-09-16T01:41:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb51816740 10 client.15681 fetch_fsmap learned FSMap version 29 2024-09-16T01:41:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.687+0000 7fcb51816740 1 -- 172.21.15.139:0/403531137 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9b7293960 con 0x55b9b7555060 2024-09-16T01:41:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.688+0000 7fcb30ff9640 1 -- 172.21.15.139:0/403531137 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb440a4610 con 0x55b9b7555060 2024-09-16T01:41:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.688+0000 7fcb30ff9640 1 -- 172.21.15.139:0/403531137 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcb3c0117c0 con 0x55b9b7555060 2024-09-16T01:41:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.688+0000 7fcb51816740 10 client.15681 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:41:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.688+0000 7fcb51816740 10 client.15681 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:41:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.688+0000 7fcb51816740 20 client.15681 populate_metadata read hostname 'smithi139' 2024-09-16T01:41:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.688+0000 7fcb51816740 1 --2- 172.21.15.139:0/403531137 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b9b756feb0 0x55b9b75722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.688+0000 7fcb51816740 4 client.15681 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:41:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.688+0000 7fcb51816740 1 -- 172.21.15.139:0/403531137 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9b74c7b90 con 0x55b9b756feb0 2024-09-16T01:41:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.688+0000 7fcb4c9d8640 1 --2- 172.21.15.139:0/403531137 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b9b756feb0 0x55b9b75722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.688+0000 7fcb30ff9640 10 client.15681 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.688+0000 7fcb4c9d8640 1 --2- 172.21.15.139:0/403531137 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b9b756feb0 0x55b9b75722c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db1988dae66d1189 server_cookie=d5995ae437da9f24 in_seq=0 out_seq=0 2024-09-16T01:41:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.689+0000 7fcb30ff9640 1 -- 172.21.15.139:0/403531137 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9b74c7b90 con 0x55b9b756feb0 2024-09-16T01:41:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.689+0000 7fcb30ff9640 10 client.15681 handle_command_reply: tid=0 2024-09-16T01:41:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.689+0000 7fcb51816740 1 client.15681 shutdown 2024-09-16T01:41:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.689+0000 7fcb51816740 1 -- 172.21.15.139:0/403531137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 msgr2=0x55b9b755d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.689+0000 7fcb51816740 1 --2- 172.21.15.139:0/403531137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b755d1d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c010a70 tx=0x7fcb3c010f30 comp rx=0 tx=0).stop 2024-09-16T01:41:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.689+0000 7fcb51816740 1 -- 172.21.15.139:0/403531137 shutdown_connections 2024-09-16T01:41:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.689+0000 7fcb51816740 1 --2- 172.21.15.139:0/403531137 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b9b756feb0 0x55b9b75722c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.689+0000 7fcb51816740 1 --2- 172.21.15.139:0/403531137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7560240 0x55b9b755dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.689+0000 7fcb51816740 1 --2- 172.21.15.139:0/403531137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555980 0x55b9b755d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.689+0000 7fcb51816740 1 --2- 172.21.15.139:0/403531137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b755d1d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.689+0000 7fcb51816740 1 -- 172.21.15.139:0/403531137 >> 172.21.15.139:0/403531137 conn(0x55b9b75515f0 msgr2=0x55b9b75572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.689+0000 7fcb30ff9640 0 client.15681 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.689+0000 7fcb51816740 1 -- 172.21.15.139:0/403531137 shutdown_connections 2024-09-16T01:41:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.690+0000 7fcb51816740 1 -- 172.21.15.139:0/403531137 wait complete. 2024-09-16T01:41:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.690+0000 7fcb51816740 20 client.15681 trim_cache size 0 max 16384 2024-09-16T01:41:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.706+0000 7fcb51816740 1 Processor -- start 2024-09-16T01:41:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.706+0000 7fcb51816740 1 -- start start 2024-09-16T01:41:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.706+0000 7fcb51816740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b7557ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.706+0000 7fcb51816740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7555980 0x55b9b7550640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.706+0000 7fcb51816740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7560240 0x55b9b7577b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.706+0000 7fcb51816740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481af110 con 0x55b9b7555060 2024-09-16T01:41:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.706+0000 7fcb51816740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481b0c50 con 0x55b9b7560240 2024-09-16T01:41:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.706+0000 7fcb51816740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481af7e0 con 0x55b9b7555980 2024-09-16T01:41:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.706+0000 7fcb4d1d9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b7557ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.707+0000 7fcb4d1d9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b7557ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:32978/0 (socket says 172.21.15.139:32978) 2024-09-16T01:41:35.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.707+0000 7fcb4d1d9640 1 -- 172.21.15.139:0/2695051523 learned_addr learned my addr 172.21.15.139:0/2695051523 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.707+0000 7fcb4c9d8640 1 --2- 172.21.15.139:0/2695051523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7555980 0x55b9b7550640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.707+0000 7fcb4d1d9640 1 -- 172.21.15.139:0/2695051523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7555980 msgr2=0x55b9b7550640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.707+0000 7fcb4d1d9640 1 --2- 172.21.15.139:0/2695051523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7555980 0x55b9b7550640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.707+0000 7fcb4d1d9640 1 -- 172.21.15.139:0/2695051523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7560240 msgr2=0x55b9b7577b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.707+0000 7fcb4d9da640 1 --2- 172.21.15.139:0/2695051523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7560240 0x55b9b7577b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.707+0000 7fcb4d1d9640 1 --2- 172.21.15.139:0/2695051523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7560240 0x55b9b7577b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.707+0000 7fcb4d1d9640 1 -- 172.21.15.139:0/2695051523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9b754fbb0 con 0x55b9b7555060 2024-09-16T01:41:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.707+0000 7fcb4d1d9640 1 --2- 172.21.15.139:0/2695051523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b7557ac0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0058c0 tx=0x7fcb3c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2723abca93cfd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.707+0000 7fcb32ffd640 1 -- 172.21.15.139:0/2695051523 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c02d070 con 0x55b9b7555060 2024-09-16T01:41:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.707+0000 7fcb32ffd640 1 -- 172.21.15.139:0/2695051523 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb3c00e720 con 0x55b9b7555060 2024-09-16T01:41:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.707+0000 7fcb32ffd640 1 -- 172.21.15.139:0/2695051523 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c005240 con 0x55b9b7555060 2024-09-16T01:41:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.708+0000 7fcb51816740 1 -- 172.21.15.139:0/2695051523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 msgr2=0x55b9b7557ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.708+0000 7fcb51816740 1 --2- 172.21.15.139:0/2695051523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b7557ac0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0058c0 tx=0x7fcb3c00c330 comp rx=0 tx=0).stop 2024-09-16T01:41:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.708+0000 7fcb51816740 1 -- 172.21.15.139:0/2695051523 shutdown_connections 2024-09-16T01:41:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.708+0000 7fcb51816740 1 --2- 172.21.15.139:0/2695051523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7560240 0x55b9b7577b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.708+0000 7fcb51816740 1 --2- 172.21.15.139:0/2695051523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7555980 0x55b9b7550640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.708+0000 7fcb51816740 1 --2- 172.21.15.139:0/2695051523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b7557ac0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.708+0000 7fcb51816740 1 -- 172.21.15.139:0/2695051523 >> 172.21.15.139:0/2695051523 conn(0x55b9b75515f0 msgr2=0x55b9b754ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.708+0000 7fcb51816740 1 -- 172.21.15.139:0/2695051523 shutdown_connections 2024-09-16T01:41:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.708+0000 7fcb51816740 1 -- 172.21.15.139:0/2695051523 wait complete. 2024-09-16T01:41:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.708+0000 7fcb51816740 1 -- start start 2024-09-16T01:41:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.708+0000 7fcb51816740 1 Processor -- start 2024-09-16T01:41:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb51816740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b75683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb51816740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555980 0x55b9b7568900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb51816740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7560240 0x55b9b75651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb51816740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481b0580 con 0x55b9b7555060 2024-09-16T01:41:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb51816740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481afeb0 con 0x55b9b7555980 2024-09-16T01:41:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb51816740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481ad0f0 con 0x55b9b7560240 2024-09-16T01:41:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb51816740 10 client.0 _ll_register_callbacks cb 0x7fcb3c010600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:41:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb4d1d9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b75683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb4d1d9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b75683c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:32994/0 (socket says 172.21.15.139:32994) 2024-09-16T01:41:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb4d1d9640 1 -- 172.21.15.139:0/218643989 learned_addr learned my addr 172.21.15.139:0/218643989 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:35.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb4d9da640 1 --2- 172.21.15.139:0/218643989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7560240 0x55b9b75651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb4c9d8640 1 --2- 172.21.15.139:0/218643989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555980 0x55b9b7568900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb4d1d9640 1 -- 172.21.15.139:0/218643989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7560240 msgr2=0x55b9b75651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb4d1d9640 1 --2- 172.21.15.139:0/218643989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7560240 0x55b9b75651b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb4d1d9640 1 -- 172.21.15.139:0/218643989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555980 msgr2=0x55b9b7568900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb4d1d9640 1 --2- 172.21.15.139:0/218643989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555980 0x55b9b7568900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb4d1d9640 1 -- 172.21.15.139:0/218643989 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9b75659b0 con 0x55b9b7555060 2024-09-16T01:41:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb4d1d9640 1 --2- 172.21.15.139:0/218643989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b75683c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c005b90 tx=0x7fcb3c010a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f26b0f7add2e4d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.709+0000 7fcb30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:41:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.710+0000 7fcb30ff9640 1 -- 172.21.15.139:0/218643989 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c036020 con 0x55b9b7555060 2024-09-16T01:41:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.710+0000 7fcb30ff9640 1 -- 172.21.15.139:0/218643989 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fcb3c005350 con 0x55b9b7555060 2024-09-16T01:41:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.710+0000 7fcb51816740 1 -- 172.21.15.139:0/218643989 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb3c00e720 con 0x55b9b7555060 2024-09-16T01:41:35.765 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:41:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 29, 2024-09-16T01:41:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:41:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 4, 2024-09-16T01:41:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:41:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "72f21021": { 2024-09-16T01:41:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 4, 2024-09-16T01:41:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:41:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:41:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:41:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:41:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:41:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:41:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 150.2, 2024-09-16T01:41:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:41:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:41:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:41:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:41:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:41:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:41:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:41:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:41:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:41:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:41:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:41:35.766 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:41:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.710+0000 7fcb30ff9640 1 -- 172.21.15.139:0/218643989 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcb3c041050 con 0x55b9b7555060 2024-09-16T01:41:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.710+0000 7fcb51816740 10 client.15687 fetch_fsmap learned FSMap version 29 2024-09-16T01:41:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.710+0000 7fcb51816740 1 -- 172.21.15.139:0/218643989 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9b75db630 con 0x55b9b7555060 2024-09-16T01:41:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.710+0000 7fcb30ff9640 1 -- 172.21.15.139:0/218643989 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb3c00e720 con 0x55b9b7555060 2024-09-16T01:41:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.710+0000 7fcb30ff9640 1 -- 172.21.15.139:0/218643989 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fcb3c003890 con 0x55b9b7555060 2024-09-16T01:41:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.710+0000 7fcb51816740 10 client.15687 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:41:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.710+0000 7fcb51816740 10 client.15687 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:41:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.710+0000 7fcb51816740 20 client.15687 populate_metadata read hostname 'smithi139' 2024-09-16T01:41:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.710+0000 7fcb51816740 1 --2- 172.21.15.139:0/218643989 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b9b75dbce0 0x55b9b75de0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.710+0000 7fcb51816740 4 client.15687 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "72f21021", "release": true, "await": true}] 2024-09-16T01:41:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.710+0000 7fcb51816740 1 -- 172.21.15.139:0/218643989 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "72f21021", "release": true, "await": true}) -- 0x55b9b74c20a0 con 0x55b9b75dbce0 2024-09-16T01:41:35.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.711+0000 7fcb4c9d8640 1 --2- 172.21.15.139:0/218643989 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b9b75dbce0 0x55b9b75de0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.711+0000 7fcb30ff9640 10 client.15687 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.711+0000 7fcb4c9d8640 1 --2- 172.21.15.139:0/218643989 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b9b75dbce0 0x55b9b75de0d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfbb009b3d60e35f server_cookie=662f7cd7d6a121b in_seq=0 out_seq=0 2024-09-16T01:41:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb30ff9640 1 -- 172.21.15.139:0/218643989 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b9b74c20a0 con 0x55b9b75dbce0 2024-09-16T01:41:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb30ff9640 10 client.15687 handle_command_reply: tid=0 2024-09-16T01:41:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb51816740 1 client.15687 shutdown 2024-09-16T01:41:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb51816740 1 -- 172.21.15.139:0/218643989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 msgr2=0x55b9b75683c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb51816740 1 --2- 172.21.15.139:0/218643989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b75683c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c005b90 tx=0x7fcb3c010a30 comp rx=0 tx=0).stop 2024-09-16T01:41:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb51816740 1 -- 172.21.15.139:0/218643989 shutdown_connections 2024-09-16T01:41:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb51816740 1 --2- 172.21.15.139:0/218643989 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b9b75dbce0 0x55b9b75de0d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb51816740 1 --2- 172.21.15.139:0/218643989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b9b7560240 0x55b9b75651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb30ff9640 0 client.15687 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb51816740 1 --2- 172.21.15.139:0/218643989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9b7555980 0x55b9b7568900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb51816740 1 --2- 172.21.15.139:0/218643989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b9b7555060 0x55b9b75683c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb51816740 1 -- 172.21.15.139:0/218643989 >> 172.21.15.139:0/218643989 conn(0x55b9b75515f0 msgr2=0x55b9b7550750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb51816740 1 -- 172.21.15.139:0/218643989 shutdown_connections 2024-09-16T01:41:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb51816740 1 -- 172.21.15.139:0/218643989 wait complete. 2024-09-16T01:41:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.713+0000 7fcb51816740 20 client.15687 trim_cache size 0 max 16384 2024-09-16T01:41:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.714+0000 7fcb4f43d640 1 -- 172.21.15.139:0/2195541690 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcb340a34d0 msgr2=0x7fcb340a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.714+0000 7fcb4f43d640 1 --2- 172.21.15.139:0/2195541690 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcb340a34d0 0x7fcb340a5990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcb4002ae00 tx=0x7fcb400317a0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.714+0000 7fcb4f43d640 1 -- 172.21.15.139:0/2195541690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb48100ca0 msgr2=0x7fcb48198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.714+0000 7fcb4f43d640 1 --2- 172.21.15.139:0/2195541690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb48100ca0 0x7fcb48198930 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400d740 tx=0x7fcb4400dc00 comp rx=0 tx=0).stop 2024-09-16T01:41:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.714+0000 7fcb4f43d640 1 -- 172.21.15.139:0/2195541690 shutdown_connections 2024-09-16T01:41:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.714+0000 7fcb4f43d640 1 --2- 172.21.15.139:0/2195541690 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcb340a34d0 0x7fcb340a5990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.714+0000 7fcb4f43d640 1 --2- 172.21.15.139:0/2195541690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb48106f50 0x7fcb4819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.714+0000 7fcb4f43d640 1 --2- 172.21.15.139:0/2195541690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb48100ca0 0x7fcb48198930 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.714+0000 7fcb4f43d640 1 --2- 172.21.15.139:0/2195541690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb481002f0 0x7fcb481983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.714+0000 7fcb4f43d640 1 -- 172.21.15.139:0/2195541690 >> 172.21.15.139:0/2195541690 conn(0x7fcb48077210 msgr2=0x7fcb48105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:35.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.714+0000 7fcb4f43d640 1 -- 172.21.15.139:0/2195541690 shutdown_connections 2024-09-16T01:41:35.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:41:35.714+0000 7fcb4f43d640 1 -- 172.21.15.139:0/2195541690 wait complete. 2024-09-16T01:41:36.694 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '72f21021', total seconds elapsed: 34.1 2024-09-16T01:41:37.069 INFO:tasks.workunit.client.0.smithi139.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T01:41:37.146 INFO:tasks.workunit.client.0.smithi139.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T01:41:37.186 INFO:tasks.workunit.client.0.smithi139.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T01:41:37.188 INFO:tasks.workunit.client.0.smithi139.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T01:41:37.188 INFO:tasks.workunit.client.0.smithi139.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T01:41:37.226 INFO:tasks.workunit.client.0.smithi139.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T01:41:37.229 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:41:37.230 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-16T01:41:37.245 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-09-16T01:41:37.246 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-09-16T01:41:37.246 DEBUG:teuthology.orchestra.run.smithi139:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-09-16T01:41:37.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:36 smithi164 ceph-mon[35121]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-16T01:41:37.308 INFO:tasks.workunit.client.0.smithi139.stdout:getting blogbench 2024-09-16T01:41:37.308 INFO:tasks.workunit.client.0.smithi139.stderr:+ echo 'getting blogbench' 2024-09-16T01:41:37.308 INFO:tasks.workunit.client.0.smithi139.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-16T01:41:37.328 INFO:tasks.workunit.client.0.smithi139.stderr:--2024-09-16 01:41:37-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-16T01:41:37.329 INFO:tasks.workunit.client.0.smithi139.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-16T01:41:37.351 INFO:tasks.workunit.client.0.smithi139.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-16T01:41:37.371 INFO:tasks.workunit.client.0.smithi139.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-16T01:41:37.372 INFO:tasks.workunit.client.0.smithi139.stderr:Length: 103355 (101K) [application/octet-stream] 2024-09-16T01:41:37.372 INFO:tasks.workunit.client.0.smithi139.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-09-16T01:41:37.375 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T01:41:37.413 INFO:tasks.workunit.client.0.smithi139.stderr: 0K .......... .......... .......... .......... .......... 49% 1.22M 0s 2024-09-16T01:41:37.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:36 smithi139 ceph-mon[31175]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-16T01:41:37.433 INFO:tasks.workunit.client.0.smithi139.stderr: 50K .......... .......... .......... .......... .......... 99% 2.37M 0s 2024-09-16T01:41:37.434 INFO:tasks.workunit.client.0.smithi139.stderr: 100K 100% 1.74T=0.06s 2024-09-16T01:41:37.434 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T01:41:37.434 INFO:tasks.workunit.client.0.smithi139.stderr:2024-09-16 01:41:37 (1.63 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-09-16T01:41:37.434 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T01:41:37.434 INFO:tasks.workunit.client.0.smithi139.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-09-16T01:41:37.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:36 smithi176 ceph-mon[35324]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-16T01:41:37.477 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/ 2024-09-16T01:41:37.478 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/m4/ 2024-09-16T01:41:37.478 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-09-16T01:41:37.479 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/man/ 2024-09-16T01:41:37.480 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/man/blogbench.8 2024-09-16T01:41:37.481 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/man/Makefile.am 2024-09-16T01:41:37.482 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/man/Makefile.in 2024-09-16T01:41:37.483 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/ 2024-09-16T01:41:37.484 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/mysnprintf.c 2024-09-16T01:41:37.485 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/mysnprintf.h 2024-09-16T01:41:37.486 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/blogbench.c 2024-09-16T01:41:37.487 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/blogbench.h 2024-09-16T01:41:37.488 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/fakesnprintf.c 2024-09-16T01:41:37.489 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/fakesnprintf.h 2024-09-16T01:41:37.490 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/blogbench_p.h 2024-09-16T01:41:37.491 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-09-16T01:41:37.492 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-09-16T01:41:37.493 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/Makefile.am 2024-09-16T01:41:37.494 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/Makefile.in 2024-09-16T01:41:37.495 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/process.c 2024-09-16T01:41:37.496 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/globals.h 2024-09-16T01:41:37.497 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/commenter.c 2024-09-16T01:41:37.498 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/rewriter.c 2024-09-16T01:41:37.499 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/reader.c 2024-09-16T01:41:37.500 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/writer.c 2024-09-16T01:41:37.501 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/src/helpers.c 2024-09-16T01:41:37.502 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/NEWS 2024-09-16T01:41:37.503 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/compile 2024-09-16T01:41:37.503 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/depcomp 2024-09-16T01:41:37.504 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/aclocal.m4 2024-09-16T01:41:37.505 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/README 2024-09-16T01:41:37.506 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/configure 2024-09-16T01:41:37.509 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/configure.ac 2024-09-16T01:41:37.510 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/config.guess 2024-09-16T01:41:37.511 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/install-sh 2024-09-16T01:41:37.512 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/config.sub 2024-09-16T01:41:37.513 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/missing 2024-09-16T01:41:37.514 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/mkinstalldirs 2024-09-16T01:41:37.515 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/Makefile.am 2024-09-16T01:41:37.516 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/Makefile.in 2024-09-16T01:41:37.517 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/config.h.in 2024-09-16T01:41:37.517 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/AUTHORS 2024-09-16T01:41:37.518 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/INSTALL 2024-09-16T01:41:37.519 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/ChangeLog 2024-09-16T01:41:37.520 INFO:tasks.workunit.client.0.smithi139.stdout:blogbench-1.0/COPYING 2024-09-16T01:41:37.521 INFO:tasks.workunit.client.0.smithi139.stderr:+ cd blogbench-1.0/ 2024-09-16T01:41:37.521 INFO:tasks.workunit.client.0.smithi139.stdout:making blogbench 2024-09-16T01:41:37.522 INFO:tasks.workunit.client.0.smithi139.stderr:+ echo 'making blogbench' 2024-09-16T01:41:37.522 INFO:tasks.workunit.client.0.smithi139.stderr:+ ./configure 2024-09-16T01:41:37.615 INFO:tasks.workunit.client.0.smithi139.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-16T01:41:38.623 INFO:tasks.workunit.client.0.smithi139.stdout:checking whether build environment is sane... yes 2024-09-16T01:41:38.633 INFO:tasks.workunit.client.0.smithi139.stdout:checking for gawk... gawk 2024-09-16T01:41:38.649 INFO:tasks.workunit.client.0.smithi139.stdout:checking whether make sets $(MAKE)... yes 2024-09-16T01:41:38.659 INFO:tasks.workunit.client.0.smithi139.stdout:checking for gcc... gcc 2024-09-16T01:41:38.709 INFO:tasks.workunit.client.0.smithi139.stdout:checking for C compiler default output file name... a.out 2024-09-16T01:41:38.710 INFO:tasks.workunit.client.0.smithi139.stdout:checking whether the C compiler works... yes 2024-09-16T01:41:38.713 INFO:tasks.workunit.client.0.smithi139.stdout:checking whether we are cross compiling... no 2024-09-16T01:41:38.749 INFO:tasks.workunit.client.0.smithi139.stdout:checking for suffix of executables... 2024-09-16T01:41:38.784 INFO:tasks.workunit.client.0.smithi139.stdout:checking for suffix of object files... o 2024-09-16T01:41:38.816 INFO:tasks.workunit.client.0.smithi139.stdout:checking whether we are using the GNU C compiler... yes 2024-09-16T01:41:38.848 INFO:tasks.workunit.client.0.smithi139.stdout:checking whether gcc accepts -g... yes 2024-09-16T01:41:38.923 INFO:tasks.workunit.client.0.smithi139.stdout:checking for gcc option to accept ANSI C... none needed 2024-09-16T01:41:38.965 INFO:tasks.workunit.client.0.smithi139.stdout:checking for style of include used by make... GNU 2024-09-16T01:41:39.058 INFO:tasks.workunit.client.0.smithi139.stdout:checking dependency style of gcc... gcc3 2024-09-16T01:41:39.058 INFO:tasks.workunit.client.0.smithi139.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-16T01:41:39.106 INFO:tasks.workunit.client.0.smithi139.stdout:checking how to run the C preprocessor... gcc -E 2024-09-16T01:41:39.146 INFO:tasks.workunit.client.0.smithi139.stdout:checking for egrep... grep -E 2024-09-16T01:41:39.155 INFO:tasks.workunit.client.0.smithi139.stdout:checking for AIX... no 2024-09-16T01:41:39.227 INFO:tasks.workunit.client.0.smithi139.stdout:checking for library containing strerror... none required 2024-09-16T01:41:39.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:38 smithi164 ceph-mon[35121]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-16T01:41:39.380 INFO:tasks.workunit.client.0.smithi139.stdout:checking for ANSI C header files... yes 2024-09-16T01:41:39.416 INFO:tasks.workunit.client.0.smithi139.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-16T01:41:39.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:38 smithi139 ceph-mon[31175]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-16T01:41:39.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:38 smithi176 ceph-mon[35324]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-16T01:41:39.461 INFO:tasks.workunit.client.0.smithi139.stdout:checking for sys/types.h... yes 2024-09-16T01:41:39.508 INFO:tasks.workunit.client.0.smithi139.stdout:checking for sys/stat.h... yes 2024-09-16T01:41:39.554 INFO:tasks.workunit.client.0.smithi139.stdout:checking for stdlib.h... yes 2024-09-16T01:41:39.602 INFO:tasks.workunit.client.0.smithi139.stdout:checking for string.h... yes 2024-09-16T01:41:39.656 INFO:tasks.workunit.client.0.smithi139.stdout:checking for memory.h... yes 2024-09-16T01:41:39.704 INFO:tasks.workunit.client.0.smithi139.stdout:checking for strings.h... yes 2024-09-16T01:41:39.752 INFO:tasks.workunit.client.0.smithi139.stdout:checking for inttypes.h... yes 2024-09-16T01:41:39.800 INFO:tasks.workunit.client.0.smithi139.stdout:checking for stdint.h... yes 2024-09-16T01:41:39.850 INFO:tasks.workunit.client.0.smithi139.stdout:checking for unistd.h... yes 2024-09-16T01:41:39.862 INFO:tasks.workunit.client.0.smithi139.stdout:checking for memory.h... (cached) yes 2024-09-16T01:41:39.867 INFO:tasks.workunit.client.0.smithi139.stdout:checking for strings.h... (cached) yes 2024-09-16T01:41:39.872 INFO:tasks.workunit.client.0.smithi139.stdout:checking for unistd.h... (cached) yes 2024-09-16T01:41:39.916 INFO:tasks.workunit.client.0.smithi139.stdout:checking locale.h usability... yes 2024-09-16T01:41:39.935 INFO:tasks.workunit.client.0.smithi139.stdout:checking locale.h presence... yes 2024-09-16T01:41:39.936 INFO:tasks.workunit.client.0.smithi139.stdout:checking for locale.h... yes 2024-09-16T01:41:39.949 INFO:tasks.workunit.client.0.smithi139.stdout:checking for stdlib.h... (cached) yes 2024-09-16T01:41:39.954 INFO:tasks.workunit.client.0.smithi139.stdout:checking for string.h... (cached) yes 2024-09-16T01:41:40.002 INFO:tasks.workunit.client.0.smithi139.stdout:checking sys/socket.h usability... yes 2024-09-16T01:41:40.023 INFO:tasks.workunit.client.0.smithi139.stdout:checking sys/socket.h presence... yes 2024-09-16T01:41:40.024 INFO:tasks.workunit.client.0.smithi139.stdout:checking for sys/socket.h... yes 2024-09-16T01:41:40.075 INFO:tasks.workunit.client.0.smithi139.stdout:checking sys/time.h usability... yes 2024-09-16T01:41:40.094 INFO:tasks.workunit.client.0.smithi139.stdout:checking sys/time.h presence... yes 2024-09-16T01:41:40.095 INFO:tasks.workunit.client.0.smithi139.stdout:checking for sys/time.h... yes 2024-09-16T01:41:40.152 INFO:tasks.workunit.client.0.smithi139.stdout:checking fcntl.h usability... yes 2024-09-16T01:41:40.175 INFO:tasks.workunit.client.0.smithi139.stdout:checking fcntl.h presence... yes 2024-09-16T01:41:40.175 INFO:tasks.workunit.client.0.smithi139.stdout:checking for fcntl.h... yes 2024-09-16T01:41:40.238 INFO:tasks.workunit.client.0.smithi139.stdout:checking sys/fcntl.h usability... yes 2024-09-16T01:41:40.262 INFO:tasks.workunit.client.0.smithi139.stdout:checking sys/fcntl.h presence... yes 2024-09-16T01:41:40.262 INFO:tasks.workunit.client.0.smithi139.stdout:checking for sys/fcntl.h... yes 2024-09-16T01:41:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:checking sys/param.h usability... yes 2024-09-16T01:41:40.363 INFO:tasks.workunit.client.0.smithi139.stdout:checking sys/param.h presence... yes 2024-09-16T01:41:40.363 INFO:tasks.workunit.client.0.smithi139.stdout:checking for sys/param.h... yes 2024-09-16T01:41:40.408 INFO:tasks.workunit.client.0.smithi139.stdout:checking ioctl.h usability... no 2024-09-16T01:41:40.428 INFO:tasks.workunit.client.0.smithi139.stdout:checking ioctl.h presence... no 2024-09-16T01:41:40.429 INFO:tasks.workunit.client.0.smithi139.stdout:checking for ioctl.h... no 2024-09-16T01:41:40.478 INFO:tasks.workunit.client.0.smithi139.stdout:checking sys/ioctl.h usability... yes 2024-09-16T01:41:40.497 INFO:tasks.workunit.client.0.smithi139.stdout:checking sys/ioctl.h presence... yes 2024-09-16T01:41:40.498 INFO:tasks.workunit.client.0.smithi139.stdout:checking for sys/ioctl.h... yes 2024-09-16T01:41:40.546 INFO:tasks.workunit.client.0.smithi139.stdout:checking return type of signal handlers... void 2024-09-16T01:41:40.589 INFO:tasks.workunit.client.0.smithi139.stdout:checking for size_t... yes 2024-09-16T01:41:40.627 INFO:tasks.workunit.client.0.smithi139.stdout:checking for dev_t... yes 2024-09-16T01:41:40.671 INFO:tasks.workunit.client.0.smithi139.stdout:checking for short... yes 2024-09-16T01:41:40.747 INFO:tasks.workunit.client.0.smithi139.stdout:checking size of short... 2 2024-09-16T01:41:40.790 INFO:tasks.workunit.client.0.smithi139.stdout:checking for int... yes 2024-09-16T01:41:40.863 INFO:tasks.workunit.client.0.smithi139.stdout:checking size of int... 4 2024-09-16T01:41:40.914 INFO:tasks.workunit.client.0.smithi139.stdout:checking for long... yes 2024-09-16T01:41:40.989 INFO:tasks.workunit.client.0.smithi139.stdout:checking size of long... 8 2024-09-16T01:41:41.031 INFO:tasks.workunit.client.0.smithi139.stdout:checking for long long... yes 2024-09-16T01:41:41.106 INFO:tasks.workunit.client.0.smithi139.stdout:checking size of long long... 8 2024-09-16T01:41:41.149 INFO:tasks.workunit.client.0.smithi139.stdout:checking for ssize_t... yes 2024-09-16T01:41:41.189 INFO:tasks.workunit.client.0.smithi139.stdout:checking for sig_atomic_t... yes 2024-09-16T01:41:41.253 INFO:tasks.workunit.client.0.smithi139.stdout:checking whether gcc needs -traditional... no 2024-09-16T01:41:41.288 INFO:tasks.workunit.client.0.smithi139.stdout:checking for an ANSI C-conforming const... yes 2024-09-16T01:41:41.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:40 smithi164 ceph-mon[35121]: pgmap v41: 129 pgs: 129 active+clean; 2.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 70 KiB/s wr, 15 op/s 2024-09-16T01:41:41.312 INFO:tasks.workunit.client.0.smithi139.stdout:checking for inline... inline 2024-09-16T01:41:41.380 INFO:tasks.workunit.client.0.smithi139.stdout:checking for working alloca.h... yes 2024-09-16T01:41:41.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:40 smithi139 ceph-mon[31175]: pgmap v41: 129 pgs: 129 active+clean; 2.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 70 KiB/s wr, 15 op/s 2024-09-16T01:41:41.426 INFO:tasks.workunit.client.0.smithi139.stdout:checking for alloca... yes 2024-09-16T01:41:41.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:40 smithi176 ceph-mon[35324]: pgmap v41: 129 pgs: 129 active+clean; 2.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 70 KiB/s wr, 15 op/s 2024-09-16T01:41:41.493 INFO:tasks.workunit.client.0.smithi139.stdout:checking for working memcmp... yes 2024-09-16T01:41:41.549 INFO:tasks.workunit.client.0.smithi139.stdout:checking for strftime... yes 2024-09-16T01:41:41.619 INFO:tasks.workunit.client.0.smithi139.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-09-16T01:41:41.690 INFO:tasks.workunit.client.0.smithi139.stdout:checking whether stat accepts an empty string... no 2024-09-16T01:41:41.744 INFO:tasks.workunit.client.0.smithi139.stdout:checking for vprintf... yes 2024-09-16T01:41:41.800 INFO:tasks.workunit.client.0.smithi139.stdout:checking for _doprnt... no 2024-09-16T01:41:41.821 INFO:tasks.workunit.client.0.smithi139.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-16T01:41:41.833 INFO:tasks.workunit.client.0.smithi139.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-16T01:41:41.881 INFO:tasks.workunit.client.0.smithi139.stdout:checking for the pthreads library -lpthreads... no 2024-09-16T01:41:41.934 INFO:tasks.workunit.client.0.smithi139.stdout:checking whether pthreads work without any flags... yes 2024-09-16T01:41:41.979 INFO:tasks.workunit.client.0.smithi139.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-09-16T01:41:41.979 INFO:tasks.workunit.client.0.smithi139.stdout:checking if more special flags are required for pthreads... no 2024-09-16T01:41:41.979 INFO:tasks.workunit.client.0.smithi139.stdout:checking for cc_r... gcc 2024-09-16T01:41:42.050 INFO:tasks.workunit.client.0.smithi139.stdout:checking for setrlimit... yes 2024-09-16T01:41:42.110 INFO:tasks.workunit.client.0.smithi139.stdout:checking for setproctitle... no 2024-09-16T01:41:42.175 INFO:tasks.workunit.client.0.smithi139.stdout:checking for getopt_long... yes 2024-09-16T01:41:42.235 INFO:tasks.workunit.client.0.smithi139.stdout:checking for strtoull... yes 2024-09-16T01:41:42.294 INFO:tasks.workunit.client.0.smithi139.stdout:checking for strtoq... yes 2024-09-16T01:41:42.355 INFO:tasks.workunit.client.0.smithi139.stdout:checking for snprintf... yes 2024-09-16T01:41:42.463 INFO:tasks.workunit.client.0.smithi139.stdout:checking for vsnprintf... yes 2024-09-16T01:41:42.538 INFO:tasks.workunit.client.0.smithi139.stdout:checking for vfprintf... yes 2024-09-16T01:41:42.601 INFO:tasks.workunit.client.0.smithi139.stdout:checking for gethostname... yes 2024-09-16T01:41:42.678 INFO:tasks.workunit.client.0.smithi139.stdout:checking for setlocale... yes 2024-09-16T01:41:42.772 INFO:tasks.workunit.client.0.smithi139.stdout:checking for sync... yes 2024-09-16T01:41:42.836 INFO:tasks.workunit.client.0.smithi139.stdout:checking whether snprintf is C99 conformant... done 2024-09-16T01:41:42.846 INFO:tasks.workunit.client.0.smithi139.stdout:configure: creating ./config.status 2024-09-16T01:41:42.994 INFO:tasks.workunit.client.0.smithi139.stdout:config.status: creating Makefile 2024-09-16T01:41:43.011 INFO:tasks.workunit.client.0.smithi139.stdout:config.status: creating src/Makefile 2024-09-16T01:41:43.030 INFO:tasks.workunit.client.0.smithi139.stdout:config.status: creating man/Makefile 2024-09-16T01:41:43.042 INFO:tasks.workunit.client.0.smithi139.stdout:config.status: creating config.h 2024-09-16T01:41:43.095 INFO:tasks.workunit.client.0.smithi139.stdout:config.status: executing depfiles commands 2024-09-16T01:41:43.222 INFO:tasks.workunit.client.0.smithi139.stderr:+ make 2024-09-16T01:41:43.231 INFO:tasks.workunit.client.0.smithi139.stdout:make all-recursive 2024-09-16T01:41:43.234 INFO:tasks.workunit.client.0.smithi139.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-16T01:41:43.236 INFO:tasks.workunit.client.0.smithi139.stdout:Making all in src 2024-09-16T01:41:43.242 INFO:tasks.workunit.client.0.smithi139.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-16T01:41:43.242 INFO:tasks.workunit.client.0.smithi139.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-09-16T01:41:43.242 INFO:tasks.workunit.client.0.smithi139.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-09-16T01:41:43.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:42 smithi164 ceph-mon[35121]: pgmap v42: 129 pgs: 129 active+clean; 2.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 70 KiB/s wr, 11 op/s 2024-09-16T01:41:43.337 INFO:tasks.workunit.client.0.smithi139.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-09-16T01:41:43.337 INFO:tasks.workunit.client.0.smithi139.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-09-16T01:41:43.403 INFO:tasks.workunit.client.0.smithi139.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-09-16T01:41:43.404 INFO:tasks.workunit.client.0.smithi139.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-09-16T01:41:43.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:42 smithi139 ceph-mon[31175]: pgmap v42: 129 pgs: 129 active+clean; 2.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 70 KiB/s wr, 11 op/s 2024-09-16T01:41:43.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:42 smithi176 ceph-mon[35324]: pgmap v42: 129 pgs: 129 active+clean; 2.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 70 KiB/s wr, 11 op/s 2024-09-16T01:41:43.459 INFO:tasks.workunit.client.0.smithi139.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-09-16T01:41:43.460 INFO:tasks.workunit.client.0.smithi139.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-09-16T01:41:43.508 INFO:tasks.workunit.client.0.smithi139.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-09-16T01:41:43.508 INFO:tasks.workunit.client.0.smithi139.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-09-16T01:41:43.558 INFO:tasks.workunit.client.0.smithi139.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-09-16T01:41:43.559 INFO:tasks.workunit.client.0.smithi139.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-09-16T01:41:43.602 INFO:tasks.workunit.client.0.smithi139.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-09-16T01:41:43.602 INFO:tasks.workunit.client.0.smithi139.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-09-16T01:41:43.675 INFO:tasks.workunit.client.0.smithi139.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-09-16T01:41:43.675 INFO:tasks.workunit.client.0.smithi139.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-09-16T01:41:43.708 INFO:tasks.workunit.client.0.smithi139.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-09-16T01:41:43.708 INFO:tasks.workunit.client.0.smithi139.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-09-16T01:41:43.743 INFO:tasks.workunit.client.0.smithi139.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-09-16T01:41:43.743 INFO:tasks.workunit.client.0.smithi139.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-09-16T01:41:43.779 INFO:tasks.workunit.client.0.smithi139.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-09-16T01:41:43.798 INFO:tasks.workunit.client.0.smithi139.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-16T01:41:43.798 INFO:tasks.workunit.client.0.smithi139.stdout:Making all in man 2024-09-16T01:41:43.822 INFO:tasks.workunit.client.0.smithi139.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-16T01:41:43.822 INFO:tasks.workunit.client.0.smithi139.stdout:make[2]: Nothing to be done for 'all'. 2024-09-16T01:41:43.823 INFO:tasks.workunit.client.0.smithi139.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-16T01:41:43.824 INFO:tasks.workunit.client.0.smithi139.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-16T01:41:43.825 INFO:tasks.workunit.client.0.smithi139.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-16T01:41:43.825 INFO:tasks.workunit.client.0.smithi139.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-16T01:41:43.826 INFO:tasks.workunit.client.0.smithi139.stderr:+ cd src 2024-09-16T01:41:43.826 INFO:tasks.workunit.client.0.smithi139.stderr:+ mkdir blogtest_in 2024-09-16T01:41:43.828 INFO:tasks.workunit.client.0.smithi139.stdout:running blogbench 2024-09-16T01:41:43.828 INFO:tasks.workunit.client.0.smithi139.stderr:+ echo 'running blogbench' 2024-09-16T01:41:43.828 INFO:tasks.workunit.client.0.smithi139.stderr:+ ./blogbench -d blogtest_in 2024-09-16T01:41:43.828 INFO:tasks.workunit.client.0.smithi139.stdout: 2024-09-16T01:41:43.828 INFO:tasks.workunit.client.0.smithi139.stdout:Frequency = 10 secs 2024-09-16T01:41:43.828 INFO:tasks.workunit.client.0.smithi139.stdout:Scratch dir = [blogtest_in] 2024-09-16T01:41:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.834+0000 7f3392fe4640 1 -- 172.21.15.139:0/2929156726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f338c102c80 msgr2=0x7f338c103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.834+0000 7f3392fe4640 1 --2- 172.21.15.139:0/2929156726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f338c102c80 0x7f338c103060 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3388009900 tx=0x7f338802a480 comp rx=0 tx=0).stop 2024-09-16T01:41:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.834+0000 7f3392fe4640 1 -- 172.21.15.139:0/2929156726 shutdown_connections 2024-09-16T01:41:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.834+0000 7f3392fe4640 1 --2- 172.21.15.139:0/2929156726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f338c10c0d0 0x7f338c10e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3392fe4640 1 --2- 172.21.15.139:0/2929156726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f338c1035a0 0x7f338c10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3392fe4640 1 --2- 172.21.15.139:0/2929156726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f338c102c80 0x7f338c103060 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3392fe4640 1 -- 172.21.15.139:0/2929156726 >> 172.21.15.139:0/2929156726 conn(0x7f338c077990 msgr2=0x7f338c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3392fe4640 1 -- 172.21.15.139:0/2929156726 shutdown_connections 2024-09-16T01:41:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3392fe4640 1 -- 172.21.15.139:0/2929156726 wait complete. 2024-09-16T01:41:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3392fe4640 1 Processor -- start 2024-09-16T01:41:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3392fe4640 1 -- start start 2024-09-16T01:41:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3392fe4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f338c102c80 0x7f338c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3392fe4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f338c1035a0 0x7f338c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3392fe4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f338c10c0d0 0x7f338c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3392fe4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f338c111eb0 con 0x7f338c1035a0 2024-09-16T01:41:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3392fe4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f338c111d30 con 0x7f338c102c80 2024-09-16T01:41:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3392fe4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f338c112030 con 0x7f338c10c0d0 2024-09-16T01:41:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3383fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f338c1035a0 0x7f338c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3383fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f338c1035a0 0x7f338c1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39770/0 (socket says 172.21.15.139:39770) 2024-09-16T01:41:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.835+0000 7f3383fff640 1 -- 172.21.15.139:0/2427739190 learned_addr learned my addr 172.21.15.139:0/2427739190 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.836+0000 7f3390d80640 1 --2- 172.21.15.139:0/2427739190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f338c102c80 0x7f338c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.836+0000 7f3391581640 1 --2- 172.21.15.139:0/2427739190 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f338c10c0d0 0x7f338c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.836+0000 7f3383fff640 1 -- 172.21.15.139:0/2427739190 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f338c10c0d0 msgr2=0x7f338c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.836+0000 7f3383fff640 1 --2- 172.21.15.139:0/2427739190 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f338c10c0d0 0x7f338c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.836+0000 7f3383fff640 1 -- 172.21.15.139:0/2427739190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f338c102c80 msgr2=0x7f338c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.836+0000 7f3383fff640 1 --2- 172.21.15.139:0/2427739190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f338c102c80 0x7f338c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.836+0000 7f3383fff640 1 -- 172.21.15.139:0/2427739190 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338c19d370 con 0x7f338c1035a0 2024-09-16T01:41:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.836+0000 7f3383fff640 1 --2- 172.21.15.139:0/2427739190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f338c1035a0 0x7f338c1988c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f337c00e9f0 tx=0x7f337c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.836+0000 7f3381ffb640 1 -- 172.21.15.139:0/2427739190 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c00ccc0 con 0x7f338c1035a0 2024-09-16T01:41:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.836+0000 7f3381ffb640 1 -- 172.21.15.139:0/2427739190 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f337c00ce60 con 0x7f338c1035a0 2024-09-16T01:41:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.836+0000 7f3392fe4640 1 -- 172.21.15.139:0/2427739190 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338c19d660 con 0x7f338c1035a0 2024-09-16T01:41:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.836+0000 7f3381ffb640 1 -- 172.21.15.139:0/2427739190 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c010690 con 0x7f338c1035a0 2024-09-16T01:41:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.836+0000 7f3392fe4640 1 -- 172.21.15.139:0/2427739190 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f338c1a4ef0 con 0x7f338c1035a0 2024-09-16T01:41:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.838+0000 7f3381ffb640 1 -- 172.21.15.139:0/2427739190 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 24) ==== 138589+0+0 (secure 0 0 0) 0x7f337c0046c0 con 0x7f338c1035a0 2024-09-16T01:41:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3395e5a740 1 Processor -- start 2024-09-16T01:41:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3395e5a740 1 -- start start 2024-09-16T01:41:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3395e5a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77baf0 0x5606cd77bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3381ffb640 1 --2- 172.21.15.139:0/2427739190 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f33740a3600 0x7f33740a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3395e5a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd77c410 0x5606cd786770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3395e5a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd786cb0 0x5606cd7890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3395e5a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5606cd6e02f0 con 0x5606cd786cb0 2024-09-16T01:41:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3395e5a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5606cd688270 con 0x5606cd77baf0 2024-09-16T01:41:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3395e5a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5606cd6dc2b0 con 0x5606cd77c410 2024-09-16T01:41:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3381ffb640 1 -- 172.21.15.139:0/2427739190 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f337c0d0630 con 0x7f338c1035a0 2024-09-16T01:41:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3390d80640 1 --2- 172.21.15.139:0/2427739190 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f33740a3600 0x7f33740a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3391581640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd786cb0 0x5606cd7890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3391581640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd786cb0 0x5606cd7890a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39778/0 (socket says 172.21.15.139:39778) 2024-09-16T01:41:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3391581640 1 -- 172.21.15.139:0/385961511 learned_addr learned my addr 172.21.15.139:0/385961511 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3383fff640 1 --2- 172.21.15.139:0/385961511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd77c410 0x5606cd786770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.839+0000 7f3391581640 1 --2- 172.21.15.139:0/385961511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77baf0 0x5606cd77bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.840+0000 7f3391581640 1 -- 172.21.15.139:0/385961511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd77c410 msgr2=0x5606cd786770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.840+0000 7f3391581640 1 --2- 172.21.15.139:0/385961511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd77c410 0x5606cd786770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.840+0000 7f3391581640 1 -- 172.21.15.139:0/385961511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77baf0 msgr2=0x5606cd77bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.840+0000 7f3391581640 1 --2- 172.21.15.139:0/385961511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77baf0 0x5606cd77bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.840+0000 7f3391581640 1 -- 172.21.15.139:0/385961511 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606cd687340 con 0x5606cd786cb0 2024-09-16T01:41:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.840+0000 7f3383fff640 1 --2- 172.21.15.139:0/385961511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd77c410 0x5606cd786770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:41:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.840+0000 7f3391581640 1 --2- 172.21.15.139:0/385961511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd786cb0 0x5606cd7890a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3384012a20 tx=0x7f3384009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa77ba548b18676 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.840+0000 7f3390d80640 1 --2- 172.21.15.139:0/2427739190 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f33740a3600 0x7f33740a5ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3388009230 tx=0x7f33880058d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.840+0000 7f3372ffd640 1 -- 172.21.15.139:0/385961511 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384004b20 con 0x5606cd786cb0 2024-09-16T01:41:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.840+0000 7f3372ffd640 1 -- 172.21.15.139:0/385961511 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3384003e50 con 0x5606cd786cb0 2024-09-16T01:41:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.840+0000 7f3372ffd640 1 -- 172.21.15.139:0/385961511 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384004030 con 0x5606cd786cb0 2024-09-16T01:41:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 -- 172.21.15.139:0/385961511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd786cb0 msgr2=0x5606cd7890a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 --2- 172.21.15.139:0/385961511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd786cb0 0x5606cd7890a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3384012a20 tx=0x7f3384009a50 comp rx=0 tx=0).stop 2024-09-16T01:41:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 -- 172.21.15.139:0/385961511 shutdown_connections 2024-09-16T01:41:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 --2- 172.21.15.139:0/385961511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd786cb0 0x5606cd7890a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 --2- 172.21.15.139:0/385961511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd77c410 0x5606cd786770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 --2- 172.21.15.139:0/385961511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77baf0 0x5606cd77bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 -- 172.21.15.139:0/385961511 >> 172.21.15.139:0/385961511 conn(0x5606cd778060 msgr2=0x5606cd778440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 -- 172.21.15.139:0/385961511 shutdown_connections 2024-09-16T01:41:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 -- 172.21.15.139:0/385961511 wait complete. 2024-09-16T01:41:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 -- start start 2024-09-16T01:41:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 Processor -- start 2024-09-16T01:41:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd77baf0 0x5606cd783ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 0x5606cd7840e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd786cb0 0x5606cd784620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5606cd789880 con 0x5606cd786cb0 2024-09-16T01:41:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5606cd669290 con 0x5606cd77c410 2024-09-16T01:41:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5606cd789a00 con 0x5606cd77baf0 2024-09-16T01:41:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.841+0000 7f3395e5a740 10 client.0 _ll_register_callbacks cb 0x5606cd69a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:41:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f3391581640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd786cb0 0x5606cd784620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f3391581640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd786cb0 0x5606cd784620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39790/0 (socket says 172.21.15.139:39790) 2024-09-16T01:41:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f3391581640 1 -- 172.21.15.139:0/2257365393 learned_addr learned my addr 172.21.15.139:0/2257365393 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f3390d80640 1 --2- 172.21.15.139:0/2257365393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd77baf0 0x5606cd783ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f3383fff640 1 --2- 172.21.15.139:0/2257365393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 0x5606cd7840e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f3391581640 1 -- 172.21.15.139:0/2257365393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd77baf0 msgr2=0x5606cd783ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f3391581640 1 --2- 172.21.15.139:0/2257365393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd77baf0 0x5606cd783ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f3391581640 1 -- 172.21.15.139:0/2257365393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 msgr2=0x5606cd7840e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f3391581640 1 --2- 172.21.15.139:0/2257365393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 0x5606cd7840e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f3391581640 1 -- 172.21.15.139:0/2257365393 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606cd3f3210 con 0x5606cd786cb0 2024-09-16T01:41:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f3391581640 1 --2- 172.21.15.139:0/2257365393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd786cb0 0x5606cd784620 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3384010c10 tx=0x7f3384010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647528857d9bb661 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f336bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:41:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f336bfff640 1 -- 172.21.15.139:0/2257365393 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384003b80 con 0x5606cd786cb0 2024-09-16T01:41:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f336bfff640 1 -- 172.21.15.139:0/2257365393 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f3384004a40 con 0x5606cd786cb0 2024-09-16T01:41:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f3395e5a740 1 -- 172.21.15.139:0/2257365393 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3384003e50 con 0x5606cd786cb0 2024-09-16T01:41:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f336bfff640 1 -- 172.21.15.139:0/2257365393 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3384004c40 con 0x5606cd786cb0 2024-09-16T01:41:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f3395e5a740 10 client.15699 fetch_fsmap learned FSMap version 29 2024-09-16T01:41:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f3395e5a740 1 -- 172.21.15.139:0/2257365393 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5606cd4bf620 con 0x5606cd786cb0 2024-09-16T01:41:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.842+0000 7f336bfff640 1 -- 172.21.15.139:0/2257365393 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3384003e50 con 0x5606cd786cb0 2024-09-16T01:41:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.843+0000 7f336bfff640 1 -- 172.21.15.139:0/2257365393 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3384026020 con 0x5606cd786cb0 2024-09-16T01:41:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.843+0000 7f3395e5a740 10 client.15699 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:41:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.843+0000 7f3395e5a740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:41:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.843+0000 7f3395e5a740 20 client.15699 populate_metadata read hostname 'smithi139' 2024-09-16T01:41:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.843+0000 7f3395e5a740 1 --2- 172.21.15.139:0/2257365393 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606cd796a90 0x5606cd798e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.843+0000 7f3395e5a740 4 client.15699 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:41:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.843+0000 7f3395e5a740 1 -- 172.21.15.139:0/2257365393 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606cd6eda20 con 0x5606cd796a90 2024-09-16T01:41:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.843+0000 7f3390d80640 1 --2- 172.21.15.139:0/2257365393 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606cd796a90 0x5606cd798e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f336bfff640 10 client.15699 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f3390d80640 1 --2- 172.21.15.139:0/2257365393 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606cd796a90 0x5606cd798e80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b798d5f047bce7a server_cookie=673b27491c35683c in_seq=0 out_seq=0 2024-09-16T01:41:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f336bfff640 1 -- 172.21.15.139:0/2257365393 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606cd6eda20 con 0x5606cd796a90 2024-09-16T01:41:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f336bfff640 10 client.15699 handle_command_reply: tid=0 2024-09-16T01:41:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f3395e5a740 1 client.15699 shutdown 2024-09-16T01:41:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f3395e5a740 1 -- 172.21.15.139:0/2257365393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd786cb0 msgr2=0x5606cd784620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f3395e5a740 1 --2- 172.21.15.139:0/2257365393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd786cb0 0x5606cd784620 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3384010c10 tx=0x7f3384010fa0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f3395e5a740 1 -- 172.21.15.139:0/2257365393 shutdown_connections 2024-09-16T01:41:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f3395e5a740 1 --2- 172.21.15.139:0/2257365393 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606cd796a90 0x5606cd798e80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f3395e5a740 1 --2- 172.21.15.139:0/2257365393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd786cb0 0x5606cd784620 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f3395e5a740 1 --2- 172.21.15.139:0/2257365393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 0x5606cd7840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f336bfff640 0 client.15699 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f3395e5a740 1 --2- 172.21.15.139:0/2257365393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd77baf0 0x5606cd783ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f3395e5a740 1 -- 172.21.15.139:0/2257365393 >> 172.21.15.139:0/2257365393 conn(0x5606cd778060 msgr2=0x5606cd77dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f3395e5a740 1 -- 172.21.15.139:0/2257365393 shutdown_connections 2024-09-16T01:41:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.844+0000 7f3395e5a740 1 -- 172.21.15.139:0/2257365393 wait complete. 2024-09-16T01:41:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.845+0000 7f3395e5a740 20 client.15699 trim_cache size 0 max 16384 2024-09-16T01:41:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.858+0000 7f3395e5a740 1 Processor -- start 2024-09-16T01:41:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3395e5a740 1 -- start start 2024-09-16T01:41:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3395e5a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd77baf0 0x5606cd77e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3395e5a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 0x5606cd7840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3395e5a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd786cb0 0x5606cd784610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3395e5a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1af030 con 0x5606cd77baf0 2024-09-16T01:41:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3395e5a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1b0b70 con 0x5606cd77c410 2024-09-16T01:41:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3395e5a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1af700 con 0x5606cd786cb0 2024-09-16T01:41:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3390d80640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd77baf0 0x5606cd77e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3390d80640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd77baf0 0x5606cd77e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39806/0 (socket says 172.21.15.139:39806) 2024-09-16T01:41:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3390d80640 1 -- 172.21.15.139:0/3341593801 learned_addr learned my addr 172.21.15.139:0/3341593801 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3383fff640 1 --2- 172.21.15.139:0/3341593801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 0x5606cd7840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3390d80640 1 -- 172.21.15.139:0/3341593801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd786cb0 msgr2=0x5606cd784610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3390d80640 1 --2- 172.21.15.139:0/3341593801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd786cb0 0x5606cd784610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3390d80640 1 -- 172.21.15.139:0/3341593801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 msgr2=0x5606cd7840d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3390d80640 1 --2- 172.21.15.139:0/3341593801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 0x5606cd7840d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3390d80640 1 -- 172.21.15.139:0/3341593801 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606cd777e90 con 0x5606cd77baf0 2024-09-16T01:41:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f3390d80640 1 --2- 172.21.15.139:0/3341593801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd77baf0 0x5606cd77e560 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3388009900 tx=0x7f3388040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e994678f75e81fb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f33727fc640 1 -- 172.21.15.139:0/3341593801 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388035040 con 0x5606cd77baf0 2024-09-16T01:41:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f33727fc640 1 -- 172.21.15.139:0/3341593801 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3388057070 con 0x5606cd77baf0 2024-09-16T01:41:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.859+0000 7f33727fc640 1 -- 172.21.15.139:0/3341593801 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33880514f0 con 0x5606cd77baf0 2024-09-16T01:41:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.860+0000 7f3395e5a740 1 -- 172.21.15.139:0/3341593801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd77baf0 msgr2=0x5606cd77e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.860+0000 7f3395e5a740 1 --2- 172.21.15.139:0/3341593801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd77baf0 0x5606cd77e560 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3388009900 tx=0x7f3388040000 comp rx=0 tx=0).stop 2024-09-16T01:41:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.860+0000 7f3395e5a740 1 -- 172.21.15.139:0/3341593801 shutdown_connections 2024-09-16T01:41:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.860+0000 7f3395e5a740 1 --2- 172.21.15.139:0/3341593801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd786cb0 0x5606cd784610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.860+0000 7f3395e5a740 1 --2- 172.21.15.139:0/3341593801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 0x5606cd7840d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.860+0000 7f3395e5a740 1 --2- 172.21.15.139:0/3341593801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd77baf0 0x5606cd77e560 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.860+0000 7f3395e5a740 1 -- 172.21.15.139:0/3341593801 >> 172.21.15.139:0/3341593801 conn(0x5606cd778060 msgr2=0x5606cd785c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.860+0000 7f3395e5a740 1 -- 172.21.15.139:0/3341593801 shutdown_connections 2024-09-16T01:41:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.860+0000 7f3395e5a740 1 -- 172.21.15.139:0/3341593801 wait complete. 2024-09-16T01:41:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.860+0000 7f3395e5a740 1 -- start start 2024-09-16T01:41:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.860+0000 7f3395e5a740 1 Processor -- start 2024-09-16T01:41:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3395e5a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd77baf0 0x5606cd78a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3395e5a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 0x5606cd78a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3395e5a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd786cb0 0x5606cd7f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3395e5a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1b04a0 con 0x5606cd77baf0 2024-09-16T01:41:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3395e5a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1afdd0 con 0x5606cd77c410 2024-09-16T01:41:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3395e5a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1ad010 con 0x5606cd786cb0 2024-09-16T01:41:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3395e5a740 10 client.0 _ll_register_callbacks cb 0x7f3384012510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:41:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3390d80640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd77baf0 0x5606cd78a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3390d80640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd77baf0 0x5606cd78a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39818/0 (socket says 172.21.15.139:39818) 2024-09-16T01:41:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3390d80640 1 -- 172.21.15.139:0/2822004483 learned_addr learned my addr 172.21.15.139:0/2822004483 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3391581640 1 --2- 172.21.15.139:0/2822004483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd786cb0 0x5606cd7f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3383fff640 1 --2- 172.21.15.139:0/2822004483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 0x5606cd78a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3390d80640 1 -- 172.21.15.139:0/2822004483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd786cb0 msgr2=0x5606cd7f94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3390d80640 1 --2- 172.21.15.139:0/2822004483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd786cb0 0x5606cd7f94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3390d80640 1 -- 172.21.15.139:0/2822004483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 msgr2=0x5606cd78a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3390d80640 1 --2- 172.21.15.139:0/2822004483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 0x5606cd78a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3390d80640 1 -- 172.21.15.139:0/2822004483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606cd7f9b40 con 0x5606cd77baf0 2024-09-16T01:41:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3383fff640 1 --2- 172.21.15.139:0/2822004483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 0x5606cd78a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:41:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3390d80640 1 --2- 172.21.15.139:0/2822004483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd77baf0 0x5606cd78a400 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f338802a4c0 tx=0x7f33880045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c8c74148012d0b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f336bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:41:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f336bfff640 1 -- 172.21.15.139:0/2822004483 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388035040 con 0x5606cd77baf0 2024-09-16T01:41:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f336bfff640 1 -- 172.21.15.139:0/2822004483 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f33880036f0 con 0x5606cd77baf0 2024-09-16T01:41:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.861+0000 7f3395e5a740 1 -- 172.21.15.139:0/2822004483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3388057070 con 0x5606cd77baf0 2024-09-16T01:41:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.862+0000 7f336bfff640 1 -- 172.21.15.139:0/2822004483 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3388066050 con 0x5606cd77baf0 2024-09-16T01:41:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.862+0000 7f3395e5a740 10 client.15705 fetch_fsmap learned FSMap version 29 2024-09-16T01:41:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.862+0000 7f3395e5a740 1 -- 172.21.15.139:0/2822004483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5606cd800ed0 con 0x5606cd77baf0 2024-09-16T01:41:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.862+0000 7f336bfff640 1 -- 172.21.15.139:0/2822004483 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3388057070 con 0x5606cd77baf0 2024-09-16T01:41:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.862+0000 7f336bfff640 1 -- 172.21.15.139:0/2822004483 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f338803c440 con 0x5606cd77baf0 2024-09-16T01:41:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.862+0000 7f3395e5a740 10 client.15705 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:41:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.862+0000 7f3395e5a740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:41:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.862+0000 7f3395e5a740 20 client.15705 populate_metadata read hostname 'smithi139' 2024-09-16T01:41:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.862+0000 7f3395e5a740 1 --2- 172.21.15.139:0/2822004483 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606cd796a90 0x5606cd803620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.862+0000 7f3395e5a740 4 client.15705 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:41:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.862+0000 7f3395e5a740 1 -- 172.21.15.139:0/2822004483 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606cd6e7d70 con 0x5606cd796a90 2024-09-16T01:41:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.862+0000 7f3383fff640 1 --2- 172.21.15.139:0/2822004483 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606cd796a90 0x5606cd803620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.862+0000 7f336bfff640 10 client.15705 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.862+0000 7f3383fff640 1 --2- 172.21.15.139:0/2822004483 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606cd796a90 0x5606cd803620 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=595309b9cffc5ffd server_cookie=3f93f25a15c9222d in_seq=0 out_seq=0 2024-09-16T01:41:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f336bfff640 1 -- 172.21.15.139:0/2822004483 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5606cd6e7d70 con 0x5606cd796a90 2024-09-16T01:41:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f336bfff640 10 client.15705 handle_command_reply: tid=0 2024-09-16T01:41:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f3395e5a740 1 client.15705 shutdown 2024-09-16T01:41:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f3395e5a740 1 -- 172.21.15.139:0/2822004483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd77baf0 msgr2=0x5606cd78a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f3395e5a740 1 --2- 172.21.15.139:0/2822004483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd77baf0 0x5606cd78a400 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f338802a4c0 tx=0x7f33880045d0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f3395e5a740 1 -- 172.21.15.139:0/2822004483 shutdown_connections 2024-09-16T01:41:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f3395e5a740 1 --2- 172.21.15.139:0/2822004483 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5606cd796a90 0x5606cd803620 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f3395e5a740 1 --2- 172.21.15.139:0/2822004483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606cd786cb0 0x5606cd7f94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f336bfff640 0 client.15705 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f3395e5a740 1 --2- 172.21.15.139:0/2822004483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606cd77c410 0x5606cd78a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f3395e5a740 1 --2- 172.21.15.139:0/2822004483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5606cd77baf0 0x5606cd78a400 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f3395e5a740 1 -- 172.21.15.139:0/2822004483 >> 172.21.15.139:0/2822004483 conn(0x5606cd778060 msgr2=0x5606cd7769a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f3395e5a740 1 -- 172.21.15.139:0/2822004483 shutdown_connections 2024-09-16T01:41:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f3395e5a740 1 -- 172.21.15.139:0/2822004483 wait complete. 2024-09-16T01:41:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.909+0000 7f3395e5a740 20 client.15705 trim_cache size 0 max 16384 2024-09-16T01:41:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:41:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:41:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:41:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:41:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.910+0000 7f3392fe4640 1 -- 172.21.15.139:0/2427739190 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f33740a3600 msgr2=0x7f33740a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.910+0000 7f3392fe4640 1 --2- 172.21.15.139:0/2427739190 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f33740a3600 0x7f33740a5ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3388009230 tx=0x7f33880058d0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.910+0000 7f3392fe4640 1 -- 172.21.15.139:0/2427739190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f338c1035a0 msgr2=0x7f338c1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.910+0000 7f3392fe4640 1 --2- 172.21.15.139:0/2427739190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f338c1035a0 0x7f338c1988c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f337c00e9f0 tx=0x7f337c00eeb0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.910+0000 7f3392fe4640 1 -- 172.21.15.139:0/2427739190 shutdown_connections 2024-09-16T01:41:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.910+0000 7f3392fe4640 1 --2- 172.21.15.139:0/2427739190 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f33740a3600 0x7f33740a5ac0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.910+0000 7f3392fe4640 1 --2- 172.21.15.139:0/2427739190 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f338c10c0d0 0x7f338c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.910+0000 7f3392fe4640 1 --2- 172.21.15.139:0/2427739190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f338c1035a0 0x7f338c1988c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.910+0000 7f3392fe4640 1 --2- 172.21.15.139:0/2427739190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f338c102c80 0x7f338c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.910+0000 7f3392fe4640 1 -- 172.21.15.139:0/2427739190 >> 172.21.15.139:0/2427739190 conn(0x7f338c077990 msgr2=0x7f338c0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.910+0000 7f3392fe4640 1 -- 172.21.15.139:0/2427739190 shutdown_connections 2024-09-16T01:41:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:44.910+0000 7f3392fe4640 1 -- 172.21.15.139:0/2427739190 wait complete. 2024-09-16T01:41:45.293 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:44 smithi139 ceph-mon[31175]: pgmap v43: 129 pgs: 129 active+clean; 2.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 70 KiB/s wr, 11 op/s 2024-09-16T01:41:45.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:44 smithi164 ceph-mon[35121]: pgmap v43: 129 pgs: 129 active+clean; 2.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 70 KiB/s wr, 11 op/s 2024-09-16T01:41:45.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:44 smithi176 ceph-mon[35324]: pgmap v43: 129 pgs: 129 active+clean; 2.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 70 KiB/s wr, 11 op/s 2024-09-16T01:41:45.963 INFO:tasks.workunit.client.0.smithi139.stdout:Spawning 3 writers... 2024-09-16T01:41:45.963 INFO:tasks.workunit.client.0.smithi139.stdout:Spawning 1 rewriters... 2024-09-16T01:41:45.963 INFO:tasks.workunit.client.0.smithi139.stdout:Spawning 5 commenters... 2024-09-16T01:41:45.963 INFO:tasks.workunit.client.0.smithi139.stdout:Spawning 100 readers... 2024-09-16T01:41:45.966 INFO:tasks.workunit.client.0.smithi139.stdout:Benchmarking for 30 iterations. 2024-09-16T01:41:45.966 INFO:tasks.workunit.client.0.smithi139.stdout:The test will run during 5 minutes. 2024-09-16T01:41:45.966 INFO:tasks.workunit.client.0.smithi139.stdout: 2024-09-16T01:41:45.966 INFO:tasks.workunit.client.0.smithi139.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-09-16T01:41:47.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:46 smithi164 ceph-mon[35121]: pgmap v44: 129 pgs: 129 active+clean; 5.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 430 KiB/s wr, 62 op/s 2024-09-16T01:41:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:46 smithi139 ceph-mon[31175]: pgmap v44: 129 pgs: 129 active+clean; 5.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 430 KiB/s wr, 62 op/s 2024-09-16T01:41:47.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:47 smithi176 ceph-mon[35324]: pgmap v44: 129 pgs: 129 active+clean; 5.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 430 KiB/s wr, 62 op/s 2024-09-16T01:41:49.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:49 smithi164 ceph-mon[35121]: pgmap v45: 129 pgs: 129 active+clean; 5.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 430 KiB/s wr, 58 op/s 2024-09-16T01:41:49.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:49 smithi139 ceph-mon[31175]: pgmap v45: 129 pgs: 129 active+clean; 5.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 430 KiB/s wr, 58 op/s 2024-09-16T01:41:49.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:49 smithi176 ceph-mon[35324]: pgmap v45: 129 pgs: 129 active+clean; 5.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 430 KiB/s wr, 58 op/s 2024-09-16T01:41:51.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:51 smithi164 ceph-mon[35121]: pgmap v46: 129 pgs: 129 active+clean; 12 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-09-16T01:41:51.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:51 smithi139 ceph-mon[31175]: pgmap v46: 129 pgs: 129 active+clean; 12 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-09-16T01:41:51.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:51 smithi176 ceph-mon[35324]: pgmap v46: 129 pgs: 129 active+clean; 12 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-09-16T01:41:53.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:53 smithi164 ceph-mon[35121]: pgmap v47: 129 pgs: 129 active+clean; 12 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1015 KiB/s wr, 89 op/s 2024-09-16T01:41:53.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:53 smithi139 ceph-mon[31175]: pgmap v47: 129 pgs: 129 active+clean; 12 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1015 KiB/s wr, 89 op/s 2024-09-16T01:41:53.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:53 smithi176 ceph-mon[35324]: pgmap v47: 129 pgs: 129 active+clean; 12 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1015 KiB/s wr, 89 op/s 2024-09-16T01:41:55.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:55 smithi164 ceph-mon[35121]: pgmap v48: 129 pgs: 129 active+clean; 12 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1015 KiB/s wr, 89 op/s 2024-09-16T01:41:55.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:55 smithi139 ceph-mon[31175]: pgmap v48: 129 pgs: 129 active+clean; 12 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1015 KiB/s wr, 89 op/s 2024-09-16T01:41:55.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:55 smithi176 ceph-mon[35324]: pgmap v48: 129 pgs: 129 active+clean; 12 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1015 KiB/s wr, 89 op/s 2024-09-16T01:41:55.966 INFO:tasks.workunit.client.0.smithi139.stdout: 24 419522 1303 267635 1186 72950 660 2024-09-16T01:41:57.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:57 smithi164 ceph-mon[35121]: pgmap v49: 129 pgs: 129 active+clean; 65 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 281 op/s 2024-09-16T01:41:57.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:57 smithi164 ceph-mon[35121]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-16T01:41:57.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:57 smithi139 ceph-mon[31175]: pgmap v49: 129 pgs: 129 active+clean; 65 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 281 op/s 2024-09-16T01:41:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:57 smithi139 ceph-mon[31175]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-16T01:41:57.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:57 smithi176 ceph-mon[35324]: pgmap v49: 129 pgs: 129 active+clean; 65 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 281 op/s 2024-09-16T01:41:57.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:57 smithi176 ceph-mon[35324]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-16T01:41:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.111+0000 7fc1bc598640 1 -- 172.21.15.139:0/1350054463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1b4101ac0 msgr2=0x7fc1b4101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.111+0000 7fc1bc598640 1 --2- 172.21.15.139:0/1350054463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1b4101ac0 0x7fc1b4101f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009900 tx=0x7fc1a402a410 comp rx=0 tx=0).stop 2024-09-16T01:41:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.111+0000 7fc1bc598640 1 -- 172.21.15.139:0/1350054463 shutdown_connections 2024-09-16T01:41:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.111+0000 7fc1bc598640 1 --2- 172.21.15.139:0/1350054463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b4102460 0x7fc1b410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.111+0000 7fc1bc598640 1 --2- 172.21.15.139:0/1350054463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1b4101ac0 0x7fc1b4101f20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.111+0000 7fc1bc598640 1 --2- 172.21.15.139:0/1350054463 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1b4107ad0 0x7fc1b4107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.111+0000 7fc1bc598640 1 -- 172.21.15.139:0/1350054463 >> 172.21.15.139:0/1350054463 conn(0x7fc1b40fd820 msgr2=0x7fc1b40ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.111+0000 7fc1bc598640 1 -- 172.21.15.139:0/1350054463 shutdown_connections 2024-09-16T01:41:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.111+0000 7fc1bc598640 1 -- 172.21.15.139:0/1350054463 wait complete. 2024-09-16T01:41:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.112+0000 7fc1bc598640 1 Processor -- start 2024-09-16T01:41:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.113+0000 7fc1bc598640 1 -- start start 2024-09-16T01:41:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.113+0000 7fc1bc598640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b4101ac0 0x7fc1b4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.113+0000 7fc1bc598640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1b4102460 0x7fc1b41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.113+0000 7fc1ba334640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b4101ac0 0x7fc1b4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.113+0000 7fc1ba334640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b4101ac0 0x7fc1b4198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54556/0 (socket says 172.21.15.139:54556) 2024-09-16T01:41:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.113+0000 7fc1bc598640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1b4107ad0 0x7fc1b419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.113+0000 7fc1ba334640 1 -- 172.21.15.139:0/347629788 learned_addr learned my addr 172.21.15.139:0/347629788 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.113+0000 7fc1b9b33640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1b4102460 0x7fc1b41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.114+0000 7fc1bc598640 1 -- 172.21.15.139:0/347629788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4111f10 con 0x7fc1b4101ac0 2024-09-16T01:41:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.114+0000 7fc1bc598640 1 -- 172.21.15.139:0/347629788 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4111d90 con 0x7fc1b4102460 2024-09-16T01:41:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.114+0000 7fc1bc598640 1 -- 172.21.15.139:0/347629788 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4112090 con 0x7fc1b4107ad0 2024-09-16T01:41:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.114+0000 7fc1bab35640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1b4107ad0 0x7fc1b419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.114+0000 7fc1ba334640 1 -- 172.21.15.139:0/347629788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1b4107ad0 msgr2=0x7fc1b419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.114+0000 7fc1ba334640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1b4107ad0 0x7fc1b419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.114+0000 7fc1ba334640 1 -- 172.21.15.139:0/347629788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1b4102460 msgr2=0x7fc1b41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.114+0000 7fc1ba334640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1b4102460 0x7fc1b41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.114+0000 7fc1ba334640 1 -- 172.21.15.139:0/347629788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b419d350 con 0x7fc1b4101ac0 2024-09-16T01:41:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.114+0000 7fc1b9b33640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1b4102460 0x7fc1b41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:41:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.114+0000 7fc1ba334640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b4101ac0 0x7fc1b4198360 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000d770 tx=0x7fc1b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.114+0000 7fc1ab7fe640 1 -- 172.21.15.139:0/347629788 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0017070 con 0x7fc1b4101ac0 2024-09-16T01:41:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.115+0000 7fc1bc598640 1 -- 172.21.15.139:0/347629788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b419d640 con 0x7fc1b4101ac0 2024-09-16T01:41:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.115+0000 7fc1ab7fe640 1 -- 172.21.15.139:0/347629788 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b0005500 con 0x7fc1b4101ac0 2024-09-16T01:41:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.115+0000 7fc1ab7fe640 1 -- 172.21.15.139:0/347629788 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0020060 con 0x7fc1b4101ac0 2024-09-16T01:41:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.116+0000 7fc1bc598640 1 -- 172.21.15.139:0/347629788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b41a4ed0 con 0x7fc1b4101ac0 2024-09-16T01:41:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.118+0000 7fc1ab7fe640 1 -- 172.21.15.139:0/347629788 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc1b0009cc0 con 0x7fc1b4101ac0 2024-09-16T01:41:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.119+0000 7fc1ab7fe640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc1a00a3660 0x7fc1a00a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.119+0000 7fc1ab7fe640 1 -- 172.21.15.139:0/347629788 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fc1b00cf390 con 0x7fc1b4101ac0 2024-09-16T01:41:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.119+0000 7fc1b9b33640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc1a00a3660 0x7fc1a00a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.123 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.120+0000 7fc1bf411740 1 Processor -- start 2024-09-16T01:41:58.123 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.120+0000 7fc1bf411740 1 -- start start 2024-09-16T01:41:58.123 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.120+0000 7fc1bf411740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd511eaf0 0x563dd511eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.123 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.120+0000 7fc1bf411740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511f410 0x563dd5129770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.123 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.120+0000 7fc1bf411740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd5129cb0 0x563dd512c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1bf411740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563dd50832f0 con 0x563dd511f410 2024-09-16T01:41:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1bf411740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563dd502b270 con 0x563dd511eaf0 2024-09-16T01:41:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1bf411740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563dd507f2b0 con 0x563dd5129cb0 2024-09-16T01:41:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1ba334640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd511eaf0 0x563dd511eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1b9b33640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc1a00a3660 0x7fc1a00a5b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009230 tx=0x7fc1a40058d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1ba334640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd511eaf0 0x563dd511eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:33064/0 (socket says 172.21.15.139:33064) 2024-09-16T01:41:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1ba334640 1 -- 172.21.15.139:0/2659971701 learned_addr learned my addr 172.21.15.139:0/2659971701 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1bab35640 1 --2- 172.21.15.139:0/2659971701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd5129cb0 0x563dd512c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1b9b33640 1 --2- 172.21.15.139:0/2659971701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511f410 0x563dd5129770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1ba334640 1 -- 172.21.15.139:0/2659971701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd5129cb0 msgr2=0x563dd512c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1ba334640 1 --2- 172.21.15.139:0/2659971701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd5129cb0 0x563dd512c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1ba334640 1 -- 172.21.15.139:0/2659971701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511f410 msgr2=0x563dd5129770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1ba334640 1 --2- 172.21.15.139:0/2659971701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511f410 0x563dd5129770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1ba334640 1 -- 172.21.15.139:0/2659971701 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dd502a340 con 0x563dd511eaf0 2024-09-16T01:41:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1ba334640 1 --2- 172.21.15.139:0/2659971701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd511eaf0 0x563dd511eed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc1b009b2a0 tx=0x7fc1b00975b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b80aa422a514a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1a8bfe640 1 -- 172.21.15.139:0/2659971701 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0017070 con 0x563dd511eaf0 2024-09-16T01:41:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.121+0000 7fc1a8bfe640 1 -- 172.21.15.139:0/2659971701 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b00cf170 con 0x563dd511eaf0 2024-09-16T01:41:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.122+0000 7fc1a8bfe640 1 -- 172.21.15.139:0/2659971701 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00bb310 con 0x563dd511eaf0 2024-09-16T01:41:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.122+0000 7fc1bf411740 1 -- 172.21.15.139:0/2659971701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd511eaf0 msgr2=0x563dd511eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.122+0000 7fc1bf411740 1 --2- 172.21.15.139:0/2659971701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd511eaf0 0x563dd511eed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc1b009b2a0 tx=0x7fc1b00975b0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.122+0000 7fc1bf411740 1 -- 172.21.15.139:0/2659971701 shutdown_connections 2024-09-16T01:41:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.122+0000 7fc1bf411740 1 --2- 172.21.15.139:0/2659971701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd5129cb0 0x563dd512c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.122+0000 7fc1bf411740 1 --2- 172.21.15.139:0/2659971701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511f410 0x563dd5129770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.122+0000 7fc1bf411740 1 --2- 172.21.15.139:0/2659971701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd511eaf0 0x563dd511eed0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.122+0000 7fc1bf411740 1 -- 172.21.15.139:0/2659971701 >> 172.21.15.139:0/2659971701 conn(0x563dd511b060 msgr2=0x563dd511b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.122+0000 7fc1bf411740 1 -- 172.21.15.139:0/2659971701 shutdown_connections 2024-09-16T01:41:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.123+0000 7fc1bf411740 1 -- 172.21.15.139:0/2659971701 wait complete. 2024-09-16T01:41:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.123+0000 7fc1bf411740 1 -- start start 2024-09-16T01:41:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.123+0000 7fc1bf411740 1 Processor -- start 2024-09-16T01:41:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.123+0000 7fc1bf411740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd5126c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.123+0000 7fc1bf411740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 0x563dd5127160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.123+0000 7fc1bf411740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 0x563dd51276a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.123+0000 7fc1bf411740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563dd512c820 con 0x563dd511eaf0 2024-09-16T01:41:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.123+0000 7fc1bf411740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563dd500c290 con 0x563dd5129cb0 2024-09-16T01:41:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.123+0000 7fc1bf411740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563dd512c9a0 con 0x563dd511f410 2024-09-16T01:41:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.123+0000 7fc1bf411740 10 client.0 _ll_register_callbacks cb 0x563dd503d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:41:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.123+0000 7fc1ba334640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd5126c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.123+0000 7fc1ba334640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd5126c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54574/0 (socket says 172.21.15.139:54574) 2024-09-16T01:41:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.123+0000 7fc1ba334640 1 -- 172.21.15.139:0/3404408147 learned_addr learned my addr 172.21.15.139:0/3404408147 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.123+0000 7fc1bab35640 1 --2- 172.21.15.139:0/3404408147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 0x563dd51276a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc1b9b33640 1 --2- 172.21.15.139:0/3404408147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 0x563dd5127160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc1ba334640 1 -- 172.21.15.139:0/3404408147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 msgr2=0x563dd5127160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc1ba334640 1 --2- 172.21.15.139:0/3404408147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 0x563dd5127160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc1ba334640 1 -- 172.21.15.139:0/3404408147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 msgr2=0x563dd51276a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc1ba334640 1 --2- 172.21.15.139:0/3404408147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 0x563dd51276a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc1ba334640 1 -- 172.21.15.139:0/3404408147 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dd4d96210 con 0x563dd511eaf0 2024-09-16T01:41:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc1bab35640 1 --2- 172.21.15.139:0/3404408147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 0x563dd51276a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:41:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc1ba334640 1 --2- 172.21.15.139:0/3404408147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd5126c20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0097ce0 tx=0x7fc1b00ccbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf369b81b75f4ae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc19dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:41:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc19dffb640 1 -- 172.21.15.139:0/3404408147 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00b3070 con 0x563dd511eaf0 2024-09-16T01:41:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc19dffb640 1 -- 172.21.15.139:0/3404408147 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fc1b00bb2c0 con 0x563dd511eaf0 2024-09-16T01:41:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc1bf411740 1 -- 172.21.15.139:0/3404408147 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563dd5090a20 con 0x563dd511eaf0 2024-09-16T01:41:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc19dffb640 1 -- 172.21.15.139:0/3404408147 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc1b0017050 con 0x563dd511eaf0 2024-09-16T01:41:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc1bf411740 10 client.15717 fetch_fsmap learned FSMap version 29 2024-09-16T01:41:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.124+0000 7fc1bf411740 1 -- 172.21.15.139:0/3404408147 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563dd4e62620 con 0x563dd511eaf0 2024-09-16T01:41:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.125+0000 7fc19dffb640 1 -- 172.21.15.139:0/3404408147 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563dd5090a20 con 0x563dd511eaf0 2024-09-16T01:41:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.125+0000 7fc19dffb640 1 -- 172.21.15.139:0/3404408147 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc1b0023060 con 0x563dd511eaf0 2024-09-16T01:41:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.125+0000 7fc1bf411740 10 client.15717 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:41:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.125+0000 7fc1bf411740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:41:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.125+0000 7fc1bf411740 20 client.15717 populate_metadata read hostname 'smithi139' 2024-09-16T01:41:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.125+0000 7fc1bf411740 1 --2- 172.21.15.139:0/3404408147 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563dd5139a80 0x563dd513be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.125+0000 7fc1bf411740 4 client.15717 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:41:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.125+0000 7fc1bf411740 1 -- 172.21.15.139:0/3404408147 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dd508ad70 con 0x563dd5139a80 2024-09-16T01:41:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.125+0000 7fc1b9b33640 1 --2- 172.21.15.139:0/3404408147 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563dd5139a80 0x563dd513be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.125+0000 7fc19dffb640 10 client.15717 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.126+0000 7fc1b9b33640 1 --2- 172.21.15.139:0/3404408147 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563dd5139a80 0x563dd513be70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3eb3b54c15f1fd9 server_cookie=5ec3457dd102f839 in_seq=0 out_seq=0 2024-09-16T01:41:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.128+0000 7fc19dffb640 1 -- 172.21.15.139:0/3404408147 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563dd508ad70 con 0x563dd5139a80 2024-09-16T01:41:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.128+0000 7fc19dffb640 10 client.15717 handle_command_reply: tid=0 2024-09-16T01:41:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.129+0000 7fc1bf411740 1 client.15717 shutdown 2024-09-16T01:41:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.129+0000 7fc1bf411740 1 -- 172.21.15.139:0/3404408147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 msgr2=0x563dd5126c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.129+0000 7fc1bf411740 1 --2- 172.21.15.139:0/3404408147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd5126c20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0097ce0 tx=0x7fc1b00ccbb0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.129+0000 7fc1bf411740 1 -- 172.21.15.139:0/3404408147 shutdown_connections 2024-09-16T01:41:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.129+0000 7fc1bf411740 1 --2- 172.21.15.139:0/3404408147 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563dd5139a80 0x563dd513be70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.129+0000 7fc1bf411740 1 --2- 172.21.15.139:0/3404408147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 0x563dd51276a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.129+0000 7fc19dffb640 0 client.15717 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.129+0000 7fc1bf411740 1 --2- 172.21.15.139:0/3404408147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 0x563dd5127160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.129+0000 7fc1bf411740 1 --2- 172.21.15.139:0/3404408147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd5126c20 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.129+0000 7fc1bf411740 1 -- 172.21.15.139:0/3404408147 >> 172.21.15.139:0/3404408147 conn(0x563dd511b060 msgr2=0x563dd5120dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.129+0000 7fc1bf411740 1 -- 172.21.15.139:0/3404408147 shutdown_connections 2024-09-16T01:41:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.129+0000 7fc1bf411740 1 -- 172.21.15.139:0/3404408147 wait complete. 2024-09-16T01:41:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.129+0000 7fc1bf411740 20 client.15717 trim_cache size 0 max 16384 2024-09-16T01:41:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.143+0000 7fc1bf411740 1 Processor -- start 2024-09-16T01:41:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.143+0000 7fc1bf411740 1 -- start start 2024-09-16T01:41:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1bf411740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd51254d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1bf411740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 0x563dd51274a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1bf411740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 0x563dd5141410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1bf411740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41acdf0 con 0x563dd511eaf0 2024-09-16T01:41:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1bf411740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41ae930 con 0x563dd5129cb0 2024-09-16T01:41:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1bf411740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41ad4c0 con 0x563dd511f410 2024-09-16T01:41:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1ba334640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd51254d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1ba334640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd51254d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54580/0 (socket says 172.21.15.139:54580) 2024-09-16T01:41:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1ba334640 1 -- 172.21.15.139:0/518140186 learned_addr learned my addr 172.21.15.139:0/518140186 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1bab35640 1 --2- 172.21.15.139:0/518140186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 0x563dd5141410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1b9b33640 1 --2- 172.21.15.139:0/518140186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 0x563dd51274a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1ba334640 1 -- 172.21.15.139:0/518140186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 msgr2=0x563dd51274a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1ba334640 1 --2- 172.21.15.139:0/518140186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 0x563dd51274a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1ba334640 1 -- 172.21.15.139:0/518140186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 msgr2=0x563dd5141410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1ba334640 1 --2- 172.21.15.139:0/518140186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 0x563dd5141410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1ba334640 1 -- 172.21.15.139:0/518140186 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dd513c1e0 con 0x563dd511eaf0 2024-09-16T01:41:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc1ba334640 1 --2- 172.21.15.139:0/518140186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd51254d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00995c0 tx=0x7fc1b0084520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1bcc13c579a367 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc19ffff640 1 -- 172.21.15.139:0/518140186 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00b3030 con 0x563dd511eaf0 2024-09-16T01:41:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc19ffff640 1 -- 172.21.15.139:0/518140186 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b0005a70 con 0x563dd511eaf0 2024-09-16T01:41:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.144+0000 7fc19ffff640 1 -- 172.21.15.139:0/518140186 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00a6030 con 0x563dd511eaf0 2024-09-16T01:41:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.145+0000 7fc1bf411740 1 -- 172.21.15.139:0/518140186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 msgr2=0x563dd51254d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:41:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.145+0000 7fc1bf411740 1 --2- 172.21.15.139:0/518140186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd51254d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00995c0 tx=0x7fc1b0084520 comp rx=0 tx=0).stop 2024-09-16T01:41:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.145+0000 7fc1bf411740 1 -- 172.21.15.139:0/518140186 shutdown_connections 2024-09-16T01:41:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.145+0000 7fc1bf411740 1 --2- 172.21.15.139:0/518140186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 0x563dd5141410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.145+0000 7fc1bf411740 1 --2- 172.21.15.139:0/518140186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 0x563dd51274a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.145+0000 7fc1bf411740 1 --2- 172.21.15.139:0/518140186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd51254d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.145+0000 7fc1bf411740 1 -- 172.21.15.139:0/518140186 >> 172.21.15.139:0/518140186 conn(0x563dd511b060 msgr2=0x563dd5128ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:41:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.145+0000 7fc1bf411740 1 -- 172.21.15.139:0/518140186 shutdown_connections 2024-09-16T01:41:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.145+0000 7fc1bf411740 1 -- 172.21.15.139:0/518140186 wait complete. 2024-09-16T01:41:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1bf411740 1 -- start start 2024-09-16T01:41:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1bf411740 1 Processor -- start 2024-09-16T01:41:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1bf411740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd5132090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1bf411740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 0x563dd512d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1bf411740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 0x563dd512d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1bf411740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41ae260 con 0x563dd511eaf0 2024-09-16T01:41:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1bf411740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41adb90 con 0x563dd5129cb0 2024-09-16T01:41:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1bf411740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41aadd0 con 0x563dd511f410 2024-09-16T01:41:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1bf411740 10 client.0 _ll_register_callbacks cb 0x7fc1ac009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:41:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1ba334640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd5132090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1ba334640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd5132090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54588/0 (socket says 172.21.15.139:54588) 2024-09-16T01:41:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1ba334640 1 -- 172.21.15.139:0/2209129746 learned_addr learned my addr 172.21.15.139:0/2209129746 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:41:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1bab35640 1 --2- 172.21.15.139:0/2209129746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 0x563dd512d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1b9b33640 1 --2- 172.21.15.139:0/2209129746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 0x563dd512d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1ba334640 1 -- 172.21.15.139:0/2209129746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 msgr2=0x563dd512d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1ba334640 1 --2- 172.21.15.139:0/2209129746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 0x563dd512d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1ba334640 1 -- 172.21.15.139:0/2209129746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 msgr2=0x563dd512d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:41:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1ba334640 1 --2- 172.21.15.139:0/2209129746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 0x563dd512d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:41:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1ba334640 1 -- 172.21.15.139:0/2209129746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dd512de30 con 0x563dd511eaf0 2024-09-16T01:41:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1b9b33640 1 --2- 172.21.15.139:0/2209129746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 0x563dd512d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:41:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc1ba334640 1 --2- 172.21.15.139:0/2209129746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd5132090 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00ccbb0 tx=0x7fc1b0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcecab64f824c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:41:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.146+0000 7fc19dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:41:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc19dffb640 1 -- 172.21.15.139:0/2209129746 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00a6070 con 0x563dd511eaf0 2024-09-16T01:41:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc19dffb640 1 -- 172.21.15.139:0/2209129746 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fc1b0099610 con 0x563dd511eaf0 2024-09-16T01:41:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc1bf411740 1 -- 172.21.15.139:0/2209129746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1b0005a70 con 0x563dd511eaf0 2024-09-16T01:41:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc19dffb640 1 -- 172.21.15.139:0/2209129746 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc1b00b3070 con 0x563dd511eaf0 2024-09-16T01:41:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc1bf411740 10 client.15723 fetch_fsmap learned FSMap version 29 2024-09-16T01:41:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc1bf411740 1 -- 172.21.15.139:0/2209129746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563dd512de30 con 0x563dd511eaf0 2024-09-16T01:41:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc19dffb640 1 -- 172.21.15.139:0/2209129746 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b0005a70 con 0x563dd511eaf0 2024-09-16T01:41:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc19dffb640 1 -- 172.21.15.139:0/2209129746 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc1b009a1d0 con 0x563dd511eaf0 2024-09-16T01:41:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc1bf411740 10 client.15723 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T01:41:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc1bf411740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:41:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc1bf411740 20 client.15723 populate_metadata read hostname 'smithi139' 2024-09-16T01:41:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc1bf411740 1 --2- 172.21.15.139:0/2209129746 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563dd51a4460 0x563dd51a6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:41:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc1bf411740 4 client.15723 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:41:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc1bf411740 1 -- 172.21.15.139:0/2209129746 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563dd5007900 con 0x563dd51a4460 2024-09-16T01:41:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.147+0000 7fc1b9b33640 1 --2- 172.21.15.139:0/2209129746 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563dd51a4460 0x563dd51a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:41:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.148+0000 7fc19dffb640 10 client.15723 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:41:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:41:58.148+0000 7fc1b9b33640 1 --2- 172.21.15.139:0/2209129746 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563dd51a4460 0x563dd51a6850 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e288f804128577a9 server_cookie=7c5258c1133a7434 in_seq=0 out_seq=0 2024-09-16T01:41:59.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:41:59 smithi164 ceph-mon[35121]: pgmap v50: 129 pgs: 129 active+clean; 65 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 230 op/s 2024-09-16T01:41:59.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:41:59 smithi176 ceph-mon[35324]: pgmap v50: 129 pgs: 129 active+clean; 65 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 230 op/s 2024-09-16T01:41:59.446 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:41:59 smithi139 ceph-mon[31175]: pgmap v50: 129 pgs: 129 active+clean; 65 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 230 op/s 2024-09-16T01:42:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc19dffb640 1 -- 172.21.15.139:0/2209129746 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563dd5007900 con 0x563dd51a4460 2024-09-16T01:42:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc19dffb640 10 client.15723 handle_command_reply: tid=0 2024-09-16T01:42:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc1bf411740 1 client.15723 shutdown 2024-09-16T01:42:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc1bf411740 1 -- 172.21.15.139:0/2209129746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 msgr2=0x563dd5132090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc1bf411740 1 --2- 172.21.15.139:0/2209129746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd5132090 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00ccbb0 tx=0x7fc1b0009c30 comp rx=0 tx=0).stop 2024-09-16T01:42:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc1bf411740 1 -- 172.21.15.139:0/2209129746 shutdown_connections 2024-09-16T01:42:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc1bf411740 1 --2- 172.21.15.139:0/2209129746 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563dd51a4460 0x563dd51a6850 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc1bf411740 1 --2- 172.21.15.139:0/2209129746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563dd5129cb0 0x563dd512d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc1bf411740 1 --2- 172.21.15.139:0/2209129746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dd511f410 0x563dd512d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc1bf411740 1 --2- 172.21.15.139:0/2209129746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563dd511eaf0 0x563dd5132090 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc1bf411740 1 -- 172.21.15.139:0/2209129746 >> 172.21.15.139:0/2209129746 conn(0x563dd511b060 msgr2=0x563dd51199a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc19dffb640 0 client.15723 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc1bf411740 1 -- 172.21.15.139:0/2209129746 shutdown_connections 2024-09-16T01:42:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc1bf411740 1 -- 172.21.15.139:0/2209129746 wait complete. 2024-09-16T01:42:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.775+0000 7fc1bf411740 20 client.15723 trim_cache size 0 max 16384 2024-09-16T01:42:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:42:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:42:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:42:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:42:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.776+0000 7fc1bc598640 1 -- 172.21.15.139:0/347629788 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc1a00a3660 msgr2=0x7fc1a00a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.776+0000 7fc1bc598640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc1a00a3660 0x7fc1a00a5b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009230 tx=0x7fc1a40058d0 comp rx=0 tx=0).stop 2024-09-16T01:42:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.777+0000 7fc1bc598640 1 -- 172.21.15.139:0/347629788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b4101ac0 msgr2=0x7fc1b4198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.777+0000 7fc1bc598640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b4101ac0 0x7fc1b4198360 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000d770 tx=0x7fc1b000dc30 comp rx=0 tx=0).stop 2024-09-16T01:42:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.777+0000 7fc1bc598640 1 -- 172.21.15.139:0/347629788 shutdown_connections 2024-09-16T01:42:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.777+0000 7fc1bc598640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc1a00a3660 0x7fc1a00a5b20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.777+0000 7fc1bc598640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1b4107ad0 0x7fc1b419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.777+0000 7fc1bc598640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1b4102460 0x7fc1b41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.777+0000 7fc1bc598640 1 --2- 172.21.15.139:0/347629788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b4101ac0 0x7fc1b4198360 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.777+0000 7fc1bc598640 1 -- 172.21.15.139:0/347629788 >> 172.21.15.139:0/347629788 conn(0x7fc1b40fd820 msgr2=0x7fc1b40fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.777+0000 7fc1bc598640 1 -- 172.21.15.139:0/347629788 shutdown_connections 2024-09-16T01:42:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:00.777+0000 7fc1bc598640 1 -- 172.21.15.139:0/347629788 wait complete. 2024-09-16T01:42:01.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:01 smithi164 ceph-mon[35121]: pgmap v51: 129 pgs: 129 active+clean; 69 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 430 op/s 2024-09-16T01:42:01.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:01 smithi139 ceph-mon[31175]: pgmap v51: 129 pgs: 129 active+clean; 69 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 430 op/s 2024-09-16T01:42:01.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:01 smithi176 ceph-mon[35324]: pgmap v51: 129 pgs: 129 active+clean; 69 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 430 op/s 2024-09-16T01:42:02.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:02 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:42:02.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:02 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:42:02.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:02 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:42:03.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:03 smithi164 ceph-mon[35121]: pgmap v52: 129 pgs: 129 active+clean; 71 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 764 op/s 2024-09-16T01:42:03.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:03 smithi139 ceph-mon[31175]: pgmap v52: 129 pgs: 129 active+clean; 71 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 764 op/s 2024-09-16T01:42:03.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:03 smithi176 ceph-mon[35324]: pgmap v52: 129 pgs: 129 active+clean; 71 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 764 op/s 2024-09-16T01:42:05.297 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:05 smithi164 ceph-mon[35121]: pgmap v53: 129 pgs: 129 active+clean; 71 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 763 op/s 2024-09-16T01:42:05.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:05 smithi139 ceph-mon[31175]: pgmap v53: 129 pgs: 129 active+clean; 71 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 763 op/s 2024-09-16T01:42:05.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:05 smithi176 ceph-mon[35324]: pgmap v53: 129 pgs: 129 active+clean; 71 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.4 MiB/s wr, 763 op/s 2024-09-16T01:42:05.967 INFO:tasks.workunit.client.0.smithi139.stdout: 36 773533 435 500844 640 184785 1065 2024-09-16T01:42:07.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:07 smithi139 ceph-mon[31175]: pgmap v54: 129 pgs: 129 active+clean; 53 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.1 MiB/s wr, 1.16k op/s 2024-09-16T01:42:07.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:07 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:42:07.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:07 smithi176 ceph-mon[35324]: pgmap v54: 129 pgs: 129 active+clean; 53 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.1 MiB/s wr, 1.16k op/s 2024-09-16T01:42:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:07 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:42:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:07 smithi164 ceph-mon[35121]: pgmap v54: 129 pgs: 129 active+clean; 53 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.1 MiB/s wr, 1.16k op/s 2024-09-16T01:42:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:07 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:42:09.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:09 smithi139 ceph-mon[31175]: pgmap v55: 129 pgs: 129 active+clean; 53 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 970 op/s 2024-09-16T01:42:09.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:09 smithi176 ceph-mon[35324]: pgmap v55: 129 pgs: 129 active+clean; 53 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 970 op/s 2024-09-16T01:42:09.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:09 smithi164 ceph-mon[35121]: pgmap v55: 129 pgs: 129 active+clean; 53 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 970 op/s 2024-09-16T01:42:11.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:11 smithi139 ceph-mon[31175]: pgmap v56: 129 pgs: 129 active+clean; 67 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 1.04k op/s 2024-09-16T01:42:11.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:11 smithi176 ceph-mon[35324]: pgmap v56: 129 pgs: 129 active+clean; 67 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 1.04k op/s 2024-09-16T01:42:11.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:11 smithi164 ceph-mon[35121]: pgmap v56: 129 pgs: 129 active+clean; 67 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 1.04k op/s 2024-09-16T01:42:13.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:13 smithi139 ceph-mon[31175]: pgmap v57: 129 pgs: 129 active+clean; 127 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 1.09k op/s 2024-09-16T01:42:13.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:13 smithi176 ceph-mon[35324]: pgmap v57: 129 pgs: 129 active+clean; 127 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 1.09k op/s 2024-09-16T01:42:13.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:13 smithi164 ceph-mon[35121]: pgmap v57: 129 pgs: 129 active+clean; 127 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 1.09k op/s 2024-09-16T01:42:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.891+0000 7f07fac53640 1 -- 172.21.15.139:0/3499263386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f41026f0 msgr2=0x7f07f410d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.891+0000 7f07fac53640 1 --2- 172.21.15.139:0/3499263386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f41026f0 0x7f07f410d110 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009cd0 tx=0x7f07ec02a8e0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.891+0000 7f07fac53640 1 -- 172.21.15.139:0/3499263386 shutdown_connections 2024-09-16T01:42:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.891+0000 7f07fac53640 1 --2- 172.21.15.139:0/3499263386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f41026f0 0x7f07f410d110 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.891+0000 7f07fac53640 1 --2- 172.21.15.139:0/3499263386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07f4101d70 0x7f07f41021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.891+0000 7f07fac53640 1 --2- 172.21.15.139:0/3499263386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f4100490 0x7f07f4100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.891+0000 7f07fac53640 1 -- 172.21.15.139:0/3499263386 >> 172.21.15.139:0/3499263386 conn(0x7f07f40770f0 msgr2=0x7f07f40fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.891+0000 7f07fac53640 1 -- 172.21.15.139:0/3499263386 shutdown_connections 2024-09-16T01:42:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.891+0000 7f07fac53640 1 -- 172.21.15.139:0/3499263386 wait complete. 2024-09-16T01:42:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.892+0000 7f07fac53640 1 Processor -- start 2024-09-16T01:42:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.892+0000 7f07fac53640 1 -- start start 2024-09-16T01:42:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.892+0000 7f07fac53640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07f4100490 0x7f07f40fc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.892+0000 7f07fac53640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f4101d70 0x7f07f40fca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.892+0000 7f07fac53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f41026f0 0x7f07f40fcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.892+0000 7f07fac53640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f07f410fdd0 con 0x7f07f41026f0 2024-09-16T01:42:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.892+0000 7f07fac53640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07f410fc50 con 0x7f07f4101d70 2024-09-16T01:42:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.892+0000 7f07fac53640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f07f410ff50 con 0x7f07f4100490 2024-09-16T01:42:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.892+0000 7f07f89ef640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07f4100490 0x7f07f40fc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.892+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f4101d70 0x7f07f40fca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.892+0000 7f07f89ef640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07f4100490 0x7f07f40fc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:38762/0 (socket says 172.21.15.139:38762) 2024-09-16T01:42:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.892+0000 7f07f89ef640 1 -- 172.21.15.139:0/1456062261 learned_addr learned my addr 172.21.15.139:0/1456062261 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.892+0000 7f07f91f0640 1 --2- 172.21.15.139:0/1456062261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f41026f0 0x7f07f40fcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.893+0000 7f07f3fff640 1 -- 172.21.15.139:0/1456062261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07f4100490 msgr2=0x7f07f40fc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.893+0000 7f07f3fff640 1 --2- 172.21.15.139:0/1456062261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07f4100490 0x7f07f40fc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.893+0000 7f07f3fff640 1 -- 172.21.15.139:0/1456062261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f41026f0 msgr2=0x7f07f40fcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.893+0000 7f07f3fff640 1 --2- 172.21.15.139:0/1456062261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f41026f0 0x7f07f40fcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.893+0000 7f07f3fff640 1 -- 172.21.15.139:0/1456062261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f41a6900 con 0x7f07f4101d70 2024-09-16T01:42:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.893+0000 7f07f89ef640 1 --2- 172.21.15.139:0/1456062261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07f4100490 0x7f07f40fc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.893+0000 7f07f3fff640 1 --2- 172.21.15.139:0/1456062261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f4101d70 0x7f07f40fca80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f07e000ea90 tx=0x7f07e000ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.893+0000 7f07f1ffb640 1 -- 172.21.15.139:0/1456062261 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e000cc80 con 0x7f07f4101d70 2024-09-16T01:42:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.893+0000 7f07fac53640 1 -- 172.21.15.139:0/1456062261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f41a6bf0 con 0x7f07f4101d70 2024-09-16T01:42:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.893+0000 7f07f1ffb640 1 -- 172.21.15.139:0/1456062261 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07e000ce20 con 0x7f07f4101d70 2024-09-16T01:42:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.893+0000 7f07f1ffb640 1 -- 172.21.15.139:0/1456062261 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0004a00 con 0x7f07f4101d70 2024-09-16T01:42:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.893+0000 7f07fac53640 1 -- 172.21.15.139:0/1456062261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f41a7140 con 0x7f07f4101d70 2024-09-16T01:42:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.895+0000 7f07f1ffb640 1 -- 172.21.15.139:0/1456062261 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f07e001d020 con 0x7f07f4101d70 2024-09-16T01:42:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.896+0000 7f07fd9eb740 1 Processor -- start 2024-09-16T01:42:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.896+0000 7f07fd9eb740 1 -- start start 2024-09-16T01:42:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.896+0000 7f07fd9eb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.896+0000 7f07fd9eb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676ef410 0x564d676f9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07fd9eb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676f9cb0 0x564d676fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07fd9eb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564d676532f0 con 0x564d676f9cb0 2024-09-16T01:42:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07fd9eb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564d675fb270 con 0x564d676eeaf0 2024-09-16T01:42:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07fd9eb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564d6764f2b0 con 0x564d676ef410 2024-09-16T01:42:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07f89ef640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676ef410 0x564d676f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07f91f0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07f91f0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676eeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39094/0 (socket says 172.21.15.139:39094) 2024-09-16T01:42:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07f91f0640 1 -- 172.21.15.139:0/1257923330 learned_addr learned my addr 172.21.15.139:0/1257923330 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07f3fff640 1 --2- 172.21.15.139:0/1257923330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676f9cb0 0x564d676fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07f1ffb640 1 --2- 172.21.15.139:0/1456062261 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f07e40a34e0 0x7f07e40a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07f91f0640 1 -- 172.21.15.139:0/1257923330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676ef410 msgr2=0x564d676f9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07f91f0640 1 --2- 172.21.15.139:0/1257923330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676ef410 0x564d676f9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07f91f0640 1 -- 172.21.15.139:0/1257923330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676f9cb0 msgr2=0x564d676fc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07f91f0640 1 --2- 172.21.15.139:0/1257923330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676f9cb0 0x564d676fc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07f91f0640 1 -- 172.21.15.139:0/1257923330 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d675fa340 con 0x564d676eeaf0 2024-09-16T01:42:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07f89ef640 1 --2- 172.21.15.139:0/1257923330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676ef410 0x564d676f9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07f1ffb640 1 -- 172.21.15.139:0/1456062261 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f07e00cfb30 con 0x7f07f4101d70 2024-09-16T01:42:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.897+0000 7f07f91f0640 1 --2- 172.21.15.139:0/1456062261 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f07e40a34e0 0x7f07e40a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.898+0000 7f07f91f0640 1 --2- 172.21.15.139:0/1257923330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676eeed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009cd0 tx=0x7f07ec009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c7d64a1e6bf624c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.898+0000 7f07deffd640 1 -- 172.21.15.139:0/1257923330 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec003e40 con 0x564d676eeaf0 2024-09-16T01:42:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.898+0000 7f07deffd640 1 -- 172.21.15.139:0/1257923330 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ec036070 con 0x564d676eeaf0 2024-09-16T01:42:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.898+0000 7f07deffd640 1 -- 172.21.15.139:0/1257923330 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec030b10 con 0x564d676eeaf0 2024-09-16T01:42:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.899+0000 7f07fd9eb740 1 -- 172.21.15.139:0/1257923330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 msgr2=0x564d676eeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.899+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/1257923330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676eeed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009cd0 tx=0x7f07ec009a20 comp rx=0 tx=0).stop 2024-09-16T01:42:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.899+0000 7f07f91f0640 1 --2- 172.21.15.139:0/1456062261 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f07e40a34e0 0x7f07e40a59a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f07ec049000 tx=0x7f07ec04a000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.899+0000 7f07fd9eb740 1 -- 172.21.15.139:0/1257923330 shutdown_connections 2024-09-16T01:42:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.899+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/1257923330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676f9cb0 0x564d676fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.899+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/1257923330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676ef410 0x564d676f9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.899+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/1257923330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676eeed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.899+0000 7f07fd9eb740 1 -- 172.21.15.139:0/1257923330 >> 172.21.15.139:0/1257923330 conn(0x564d676eb060 msgr2=0x564d676eb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.899+0000 7f07fd9eb740 1 -- 172.21.15.139:0/1257923330 shutdown_connections 2024-09-16T01:42:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.899+0000 7f07fd9eb740 1 -- 172.21.15.139:0/1257923330 wait complete. 2024-09-16T01:42:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.899+0000 7f07fd9eb740 1 -- start start 2024-09-16T01:42:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.899+0000 7f07fd9eb740 1 Processor -- start 2024-09-16T01:42:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07fd9eb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07fd9eb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d676f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07fd9eb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676f9cb0 0x564d676f7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07fd9eb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564d676fc880 con 0x564d676ef410 2024-09-16T01:42:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07fd9eb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564d675dc290 con 0x564d676eeaf0 2024-09-16T01:42:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07fd9eb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564d676fca00 con 0x564d676f9cb0 2024-09-16T01:42:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07fd9eb740 10 client.0 _ll_register_callbacks cb 0x564d6760d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:42:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d676f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07f89ef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07f89ef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676f6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39098/0 (socket says 172.21.15.139:39098) 2024-09-16T01:42:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d676f70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59606/0 (socket says 172.21.15.139:59606) 2024-09-16T01:42:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07f91f0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676f9cb0 0x564d676f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07f89ef640 1 -- 172.21.15.139:0/313456450 learned_addr learned my addr 172.21.15.139:0/313456450 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07f89ef640 1 -- 172.21.15.139:0/313456450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676f9cb0 msgr2=0x564d676f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07f89ef640 1 --2- 172.21.15.139:0/313456450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676f9cb0 0x564d676f7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07f89ef640 1 -- 172.21.15.139:0/313456450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 msgr2=0x564d676f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07f89ef640 1 --2- 172.21.15.139:0/313456450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d676f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07f89ef640 1 -- 172.21.15.139:0/313456450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d67366210 con 0x564d676eeaf0 2024-09-16T01:42:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07f89ef640 1 --2- 172.21.15.139:0/313456450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676f6ba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f07e8012d50 tx=0x7f07e800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51eedbbb697dd157 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.900+0000 7f07d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:42:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.901+0000 7f07d3fff640 1 -- 172.21.15.139:0/313456450 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e800fe20 con 0x564d676eeaf0 2024-09-16T01:42:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.901+0000 7f07fd9eb740 1 -- 172.21.15.139:0/313456450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07ec036070 con 0x564d676eeaf0 2024-09-16T01:42:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.901+0000 7f07d3fff640 1 -- 172.21.15.139:0/313456450 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f07e8005020 con 0x564d676eeaf0 2024-09-16T01:42:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.901+0000 7f07d3fff640 1 -- 172.21.15.139:0/313456450 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f07e8005220 con 0x564d676eeaf0 2024-09-16T01:42:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.901+0000 7f07fd9eb740 10 client.25247 fetch_fsmap learned FSMap version 31 2024-09-16T01:42:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.901+0000 7f07fd9eb740 1 -- 172.21.15.139:0/313456450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564d67432620 con 0x564d676eeaf0 2024-09-16T01:42:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.901+0000 7f07d3fff640 1 -- 172.21.15.139:0/313456450 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ec036070 con 0x564d676eeaf0 2024-09-16T01:42:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.901+0000 7f07d3fff640 1 -- 172.21.15.139:0/313456450 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7f07e801c500 con 0x564d676eeaf0 2024-09-16T01:42:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.901+0000 7f07fd9eb740 10 client.25247 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:42:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.901+0000 7f07fd9eb740 10 client.25247 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:42:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.901+0000 7f07fd9eb740 20 client.25247 populate_metadata read hostname 'smithi139' 2024-09-16T01:42:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.902+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/313456450 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d67709cb0 0x564d6770c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.902+0000 7f07fd9eb740 4 client.25247 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:42:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.902+0000 7f07fd9eb740 1 -- 172.21.15.139:0/313456450 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d67660a20 con 0x564d67709cb0 2024-09-16T01:42:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.902+0000 7f07f3fff640 1 --2- 172.21.15.139:0/313456450 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d67709cb0 0x564d6770c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.902+0000 7f07d3fff640 10 client.25247 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.902+0000 7f07f3fff640 1 --2- 172.21.15.139:0/313456450 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d67709cb0 0x564d6770c120 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf1170b9c7bd28a8 server_cookie=8f05386e0364f80b in_seq=0 out_seq=0 2024-09-16T01:42:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07d3fff640 1 -- 172.21.15.139:0/313456450 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d67660a20 con 0x564d67709cb0 2024-09-16T01:42:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07d3fff640 10 client.25247 handle_command_reply: tid=0 2024-09-16T01:42:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07fd9eb740 1 client.25247 shutdown 2024-09-16T01:42:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07fd9eb740 1 -- 172.21.15.139:0/313456450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 msgr2=0x564d676f6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/313456450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676f6ba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f07e8012d50 tx=0x7f07e800f790 comp rx=0 tx=0).stop 2024-09-16T01:42:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07fd9eb740 1 -- 172.21.15.139:0/313456450 shutdown_connections 2024-09-16T01:42:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/313456450 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d67709cb0 0x564d6770c120 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/313456450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676f9cb0 0x564d676f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/313456450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d676f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07d3fff640 0 client.25247 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/313456450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676f6ba0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07fd9eb740 1 -- 172.21.15.139:0/313456450 >> 172.21.15.139:0/313456450 conn(0x564d676eb060 msgr2=0x564d676f0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07fd9eb740 1 -- 172.21.15.139:0/313456450 shutdown_connections 2024-09-16T01:42:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07fd9eb740 1 -- 172.21.15.139:0/313456450 wait complete. 2024-09-16T01:42:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.904+0000 7f07fd9eb740 20 client.25247 trim_cache size 0 max 16384 2024-09-16T01:42:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.922+0000 7f07fd9eb740 1 Processor -- start 2024-09-16T01:42:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.922+0000 7f07fd9eb740 1 -- start start 2024-09-16T01:42:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.922+0000 7f07fd9eb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676eeaf0 0x564d676f7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.922+0000 7f07fd9eb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d67711a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.922+0000 7f07fd9eb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676f9cb0 0x564d67711f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.922+0000 7f07fd9eb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f07f41b11f0 con 0x564d676ef410 2024-09-16T01:42:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.922+0000 7f07fd9eb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07f41b2df0 con 0x564d676f9cb0 2024-09-16T01:42:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.922+0000 7f07fd9eb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f07f41b18c0 con 0x564d676eeaf0 2024-09-16T01:42:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.923+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d67711a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.923+0000 7f07f89ef640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676eeaf0 0x564d676f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.923+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d67711a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59616/0 (socket says 172.21.15.139:59616) 2024-09-16T01:42:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.923+0000 7f07f3fff640 1 -- 172.21.15.139:0/827104797 learned_addr learned my addr 172.21.15.139:0/827104797 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.923+0000 7f07f91f0640 1 --2- 172.21.15.139:0/827104797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676f9cb0 0x564d67711f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.923+0000 7f07f3fff640 1 -- 172.21.15.139:0/827104797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676eeaf0 msgr2=0x564d676f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.923+0000 7f07f3fff640 1 --2- 172.21.15.139:0/827104797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676eeaf0 0x564d676f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.923+0000 7f07f3fff640 1 -- 172.21.15.139:0/827104797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676f9cb0 msgr2=0x564d67711f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.923+0000 7f07f3fff640 1 --2- 172.21.15.139:0/827104797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676f9cb0 0x564d67711f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.923+0000 7f07f3fff640 1 -- 172.21.15.139:0/827104797 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d676eae90 con 0x564d676ef410 2024-09-16T01:42:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.923+0000 7f07f3fff640 1 --2- 172.21.15.139:0/827104797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d67711a30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c7a0 tx=0x7f07e0086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9573a29bdfa539f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.923+0000 7f07de7fc640 1 -- 172.21.15.139:0/827104797 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e001a040 con 0x564d676ef410 2024-09-16T01:42:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.924+0000 7f07de7fc640 1 -- 172.21.15.139:0/827104797 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07e00d5050 con 0x564d676ef410 2024-09-16T01:42:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.924+0000 7f07de7fc640 1 -- 172.21.15.139:0/827104797 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e00860b0 con 0x564d676ef410 2024-09-16T01:42:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.924+0000 7f07fd9eb740 1 -- 172.21.15.139:0/827104797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 msgr2=0x564d67711a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.924+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/827104797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d67711a30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c7a0 tx=0x7f07e0086040 comp rx=0 tx=0).stop 2024-09-16T01:42:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.924+0000 7f07fd9eb740 1 -- 172.21.15.139:0/827104797 shutdown_connections 2024-09-16T01:42:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.924+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/827104797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676f9cb0 0x564d67711f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.924+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/827104797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d67711a30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.924+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/827104797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676eeaf0 0x564d676f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.924+0000 7f07fd9eb740 1 -- 172.21.15.139:0/827104797 >> 172.21.15.139:0/827104797 conn(0x564d676eb060 msgr2=0x564d676f8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.925+0000 7f07fd9eb740 1 -- 172.21.15.139:0/827104797 shutdown_connections 2024-09-16T01:42:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.925+0000 7f07fd9eb740 1 -- 172.21.15.139:0/827104797 wait complete. 2024-09-16T01:42:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.925+0000 7f07fd9eb740 1 -- start start 2024-09-16T01:42:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.925+0000 7f07fd9eb740 1 Processor -- start 2024-09-16T01:42:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.925+0000 7f07fd9eb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676fdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.925+0000 7f07fd9eb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d676fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.925+0000 7f07fd9eb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676f9cb0 0x564d67773a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.925+0000 7f07fd9eb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f07f41b2660 con 0x564d676ef410 2024-09-16T01:42:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.925+0000 7f07fd9eb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07f41b1f90 con 0x564d676eeaf0 2024-09-16T01:42:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.925+0000 7f07fd9eb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f07f41af1d0 con 0x564d676f9cb0 2024-09-16T01:42:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07fd9eb740 10 client.0 _ll_register_callbacks cb 0x7f07e8009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:42:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d676fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07f89ef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676fdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d676fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59628/0 (socket says 172.21.15.139:59628) 2024-09-16T01:42:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07f3fff640 1 -- 172.21.15.139:0/1725231166 learned_addr learned my addr 172.21.15.139:0/1725231166 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07f91f0640 1 --2- 172.21.15.139:0/1725231166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676f9cb0 0x564d67773a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07f3fff640 1 -- 172.21.15.139:0/1725231166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676f9cb0 msgr2=0x564d67773a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07f3fff640 1 --2- 172.21.15.139:0/1725231166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676f9cb0 0x564d67773a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07f3fff640 1 -- 172.21.15.139:0/1725231166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 msgr2=0x564d676fdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07f3fff640 1 --2- 172.21.15.139:0/1725231166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676fdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07f3fff640 1 -- 172.21.15.139:0/1725231166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d676feb80 con 0x564d676ef410 2024-09-16T01:42:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07f91f0640 1 --2- 172.21.15.139:0/1725231166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676f9cb0 0x564d67773a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:42:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07f89ef640 1 --2- 172.21.15.139:0/1725231166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676fdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07f3fff640 1 --2- 172.21.15.139:0/1725231166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d676fe2a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f07e00983d0 tx=0x7f07e00a4670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d492c0868b180d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:42:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07d3fff640 1 -- 172.21.15.139:0/1725231166 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e00b4060 con 0x564d676ef410 2024-09-16T01:42:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07d3fff640 1 -- 172.21.15.139:0/1725231166 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f07e0098070 con 0x564d676ef410 2024-09-16T01:42:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.926+0000 7f07fd9eb740 1 -- 172.21.15.139:0/1725231166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07e00d5050 con 0x564d676ef410 2024-09-16T01:42:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.927+0000 7f07d3fff640 1 -- 172.21.15.139:0/1725231166 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f07e001a030 con 0x564d676ef410 2024-09-16T01:42:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.927+0000 7f07fd9eb740 10 client.15741 fetch_fsmap learned FSMap version 31 2024-09-16T01:42:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.927+0000 7f07fd9eb740 1 -- 172.21.15.139:0/1725231166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564d67775060 con 0x564d676ef410 2024-09-16T01:42:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.927+0000 7f07d3fff640 1 -- 172.21.15.139:0/1725231166 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07e00d5050 con 0x564d676ef410 2024-09-16T01:42:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.927+0000 7f07d3fff640 1 -- 172.21.15.139:0/1725231166 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7f07e000b040 con 0x564d676ef410 2024-09-16T01:42:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.927+0000 7f07fd9eb740 10 client.15741 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:42:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.927+0000 7f07fd9eb740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:42:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.927+0000 7f07fd9eb740 20 client.15741 populate_metadata read hostname 'smithi139' 2024-09-16T01:42:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.927+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/1725231166 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d67775880 0x564d67777ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.927+0000 7f07fd9eb740 4 client.15741 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:42:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.927+0000 7f07fd9eb740 1 -- 172.21.15.139:0/1725231166 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d6765ad70 con 0x564d67775880 2024-09-16T01:42:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.927+0000 7f07f89ef640 1 --2- 172.21.15.139:0/1725231166 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d67775880 0x564d67777ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.928+0000 7f07d3fff640 10 client.15741 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:13.928+0000 7f07f89ef640 1 --2- 172.21.15.139:0/1725231166 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d67775880 0x564d67777ca0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8163f26d8bf46c6b server_cookie=d28cbf1ce27e7dd8 in_seq=0 out_seq=0 2024-09-16T01:42:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.546+0000 7f07d3fff640 1 -- 172.21.15.139:0/1725231166 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d6765ad70 con 0x564d67775880 2024-09-16T01:42:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.546+0000 7f07d3fff640 10 client.15741 handle_command_reply: tid=0 2024-09-16T01:42:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.546+0000 7f07fd9eb740 1 client.15741 shutdown 2024-09-16T01:42:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.546+0000 7f07fd9eb740 1 -- 172.21.15.139:0/1725231166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 msgr2=0x564d676fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.546+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/1725231166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d676fe2a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f07e00983d0 tx=0x7f07e00a4670 comp rx=0 tx=0).stop 2024-09-16T01:42:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.546+0000 7f07fd9eb740 1 -- 172.21.15.139:0/1725231166 shutdown_connections 2024-09-16T01:42:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.546+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/1725231166 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d67775880 0x564d67777ca0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.546+0000 7f07d3fff640 0 client.15741 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.546+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/1725231166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d676f9cb0 0x564d67773a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.546+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/1725231166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d676ef410 0x564d676fe2a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.547+0000 7f07fd9eb740 1 --2- 172.21.15.139:0/1725231166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d676eeaf0 0x564d676fdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.547+0000 7f07fd9eb740 1 -- 172.21.15.139:0/1725231166 >> 172.21.15.139:0/1725231166 conn(0x564d676eb060 msgr2=0x564d676f05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.547+0000 7f07fd9eb740 1 -- 172.21.15.139:0/1725231166 shutdown_connections 2024-09-16T01:42:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.547+0000 7f07fd9eb740 1 -- 172.21.15.139:0/1725231166 wait complete. 2024-09-16T01:42:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.547+0000 7f07fd9eb740 20 client.15741 trim_cache size 0 max 16384 2024-09-16T01:42:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:42:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:42:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:42:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:42:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.548+0000 7f07fac53640 1 -- 172.21.15.139:0/1456062261 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f07e40a34e0 msgr2=0x7f07e40a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.548+0000 7f07fac53640 1 --2- 172.21.15.139:0/1456062261 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f07e40a34e0 0x7f07e40a59a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f07ec049000 tx=0x7f07ec04a000 comp rx=0 tx=0).stop 2024-09-16T01:42:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.548+0000 7f07fac53640 1 -- 172.21.15.139:0/1456062261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f4101d70 msgr2=0x7f07f40fca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.548+0000 7f07fac53640 1 --2- 172.21.15.139:0/1456062261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f4101d70 0x7f07f40fca80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f07e000ea90 tx=0x7f07e000ef50 comp rx=0 tx=0).stop 2024-09-16T01:42:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.548+0000 7f07fac53640 1 -- 172.21.15.139:0/1456062261 shutdown_connections 2024-09-16T01:42:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.548+0000 7f07fac53640 1 --2- 172.21.15.139:0/1456062261 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f07e40a34e0 0x7f07e40a59a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.548+0000 7f07fac53640 1 --2- 172.21.15.139:0/1456062261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f41026f0 0x7f07f40fcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.548+0000 7f07fac53640 1 --2- 172.21.15.139:0/1456062261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f4101d70 0x7f07f40fca80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.548+0000 7f07fac53640 1 --2- 172.21.15.139:0/1456062261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07f4100490 0x7f07f40fc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.548+0000 7f07fac53640 1 -- 172.21.15.139:0/1456062261 >> 172.21.15.139:0/1456062261 conn(0x7f07f40770f0 msgr2=0x7f07f40fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.548+0000 7f07fac53640 1 -- 172.21.15.139:0/1456062261 shutdown_connections 2024-09-16T01:42:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:14.548+0000 7f07fac53640 1 -- 172.21.15.139:0/1456062261 wait complete. 2024-09-16T01:42:15.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:15 smithi139 ceph-mon[31175]: pgmap v58: 129 pgs: 129 active+clean; 127 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.1 MiB/s wr, 717 op/s 2024-09-16T01:42:15.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:15 smithi176 ceph-mon[35324]: pgmap v58: 129 pgs: 129 active+clean; 127 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.1 MiB/s wr, 717 op/s 2024-09-16T01:42:15.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:15 smithi164 ceph-mon[35121]: pgmap v58: 129 pgs: 129 active+clean; 127 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.1 MiB/s wr, 717 op/s 2024-09-16T01:42:15.966 INFO:tasks.workunit.client.0.smithi139.stdout: 40 91785 233 66799 201 31916 586 2024-09-16T01:42:17.346 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:17 smithi164 ceph-mon[35121]: pgmap v59: 129 pgs: 129 active+clean; 107 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.4 MiB/s wr, 1.09k op/s 2024-09-16T01:42:17.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:17 smithi139 ceph-mon[31175]: pgmap v59: 129 pgs: 129 active+clean; 107 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.4 MiB/s wr, 1.09k op/s 2024-09-16T01:42:17.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:17 smithi176 ceph-mon[35324]: pgmap v59: 129 pgs: 129 active+clean; 107 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.4 MiB/s wr, 1.09k op/s 2024-09-16T01:42:19.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:19 smithi139 ceph-mon[31175]: pgmap v60: 129 pgs: 129 active+clean; 107 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 694 op/s 2024-09-16T01:42:19.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:19 smithi176 ceph-mon[35324]: pgmap v60: 129 pgs: 129 active+clean; 107 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 694 op/s 2024-09-16T01:42:19.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:19 smithi164 ceph-mon[35121]: pgmap v60: 129 pgs: 129 active+clean; 107 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 694 op/s 2024-09-16T01:42:21.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:21 smithi139 ceph-mon[31175]: pgmap v61: 129 pgs: 129 active+clean; 110 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.0 MiB/s wr, 712 op/s 2024-09-16T01:42:21.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:21 smithi176 ceph-mon[35324]: pgmap v61: 129 pgs: 129 active+clean; 110 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.0 MiB/s wr, 712 op/s 2024-09-16T01:42:21.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:21 smithi164 ceph-mon[35121]: pgmap v61: 129 pgs: 129 active+clean; 110 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.0 MiB/s wr, 712 op/s 2024-09-16T01:42:23.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:23 smithi139 ceph-mon[31175]: pgmap v62: 129 pgs: 129 active+clean; 114 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 669 op/s 2024-09-16T01:42:23.502 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:23 smithi176 ceph-mon[35324]: pgmap v62: 129 pgs: 129 active+clean; 114 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 669 op/s 2024-09-16T01:42:23.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:23 smithi164 ceph-mon[35121]: pgmap v62: 129 pgs: 129 active+clean; 114 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 669 op/s 2024-09-16T01:42:25.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:25 smithi139 ceph-mon[31175]: pgmap v63: 129 pgs: 129 active+clean; 114 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 420 op/s 2024-09-16T01:42:25.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:25 smithi176 ceph-mon[35324]: pgmap v63: 129 pgs: 129 active+clean; 114 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 420 op/s 2024-09-16T01:42:25.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:25 smithi164 ceph-mon[35121]: pgmap v63: 129 pgs: 129 active+clean; 114 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 420 op/s 2024-09-16T01:42:25.966 INFO:tasks.workunit.client.0.smithi139.stdout: 61 723353 1123 521851 1218 289732 509 2024-09-16T01:42:27.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:27 smithi139 ceph-mon[31175]: pgmap v64: 129 pgs: 129 active+clean; 181 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.8 MiB/s wr, 723 op/s 2024-09-16T01:42:27.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:27 smithi176 ceph-mon[35324]: pgmap v64: 129 pgs: 129 active+clean; 181 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.8 MiB/s wr, 723 op/s 2024-09-16T01:42:27.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:27 smithi164 ceph-mon[35121]: pgmap v64: 129 pgs: 129 active+clean; 181 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.8 MiB/s wr, 723 op/s 2024-09-16T01:42:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.646+0000 7ff141386640 1 -- 172.21.15.139:0/1220869289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c1007f0 msgr2=0x7ff13c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.646+0000 7ff141386640 1 --2- 172.21.15.139:0/1220869289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c1007f0 0x7ff13c100c50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff124009930 tx=0x7ff12402a420 comp rx=0 tx=0).stop 2024-09-16T01:42:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.646+0000 7ff141386640 1 -- 172.21.15.139:0/1220869289 shutdown_connections 2024-09-16T01:42:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.646+0000 7ff141386640 1 --2- 172.21.15.139:0/1220869289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff13c101190 0x7ff13c10d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.646+0000 7ff141386640 1 --2- 172.21.15.139:0/1220869289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c1007f0 0x7ff13c100c50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.646+0000 7ff141386640 1 --2- 172.21.15.139:0/1220869289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff13c106800 0x7ff13c106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.646+0000 7ff141386640 1 -- 172.21.15.139:0/1220869289 >> 172.21.15.139:0/1220869289 conn(0x7ff13c0777c0 msgr2=0x7ff13c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.646+0000 7ff141386640 1 -- 172.21.15.139:0/1220869289 shutdown_connections 2024-09-16T01:42:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.646+0000 7ff141386640 1 -- 172.21.15.139:0/1220869289 wait complete. 2024-09-16T01:42:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.647+0000 7ff141386640 1 Processor -- start 2024-09-16T01:42:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.647+0000 7ff141386640 1 -- start start 2024-09-16T01:42:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.647+0000 7ff141386640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff13c1007f0 0x7ff13c196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.647+0000 7ff141386640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c101190 0x7ff13c197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.647+0000 7ff141386640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff13c106800 0x7ff13c19bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.647+0000 7ff141386640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c110c40 con 0x7ff13c101190 2024-09-16T01:42:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.647+0000 7ff141386640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c110ac0 con 0x7ff13c106800 2024-09-16T01:42:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.647+0000 7ff141386640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c110dc0 con 0x7ff13c1007f0 2024-09-16T01:42:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.647+0000 7ff13affd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff13c1007f0 0x7ff13c196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.647+0000 7ff133fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c101190 0x7ff13c197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.647+0000 7ff133fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c101190 0x7ff13c197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53980/0 (socket says 172.21.15.139:53980) 2024-09-16T01:42:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.647+0000 7ff133fff640 1 -- 172.21.15.139:0/1395857823 learned_addr learned my addr 172.21.15.139:0/1395857823 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.648+0000 7ff13b7fe640 1 --2- 172.21.15.139:0/1395857823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff13c106800 0x7ff13c19bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.648+0000 7ff133fff640 1 -- 172.21.15.139:0/1395857823 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff13c1007f0 msgr2=0x7ff13c196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.648+0000 7ff133fff640 1 --2- 172.21.15.139:0/1395857823 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff13c1007f0 0x7ff13c196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.648+0000 7ff133fff640 1 -- 172.21.15.139:0/1395857823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff13c106800 msgr2=0x7ff13c19bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.648+0000 7ff133fff640 1 --2- 172.21.15.139:0/1395857823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff13c106800 0x7ff13c19bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.648+0000 7ff133fff640 1 -- 172.21.15.139:0/1395857823 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c19c4b0 con 0x7ff13c101190 2024-09-16T01:42:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.648+0000 7ff133fff640 1 --2- 172.21.15.139:0/1395857823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c101190 0x7ff13c197460 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff124009230 tx=0x7ff124004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.649+0000 7ff138ff9640 1 -- 172.21.15.139:0/1395857823 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12403f030 con 0x7ff13c101190 2024-09-16T01:42:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.649+0000 7ff138ff9640 1 -- 172.21.15.139:0/1395857823 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff124004440 con 0x7ff13c101190 2024-09-16T01:42:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.649+0000 7ff141386640 1 -- 172.21.15.139:0/1395857823 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13c19c740 con 0x7ff13c101190 2024-09-16T01:42:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.649+0000 7ff138ff9640 1 -- 172.21.15.139:0/1395857823 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124035070 con 0x7ff13c101190 2024-09-16T01:42:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.649+0000 7ff141386640 1 -- 172.21.15.139:0/1395857823 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13c19cc80 con 0x7ff13c101190 2024-09-16T01:42:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.653+0000 7ff1441fc740 1 Processor -- start 2024-09-16T01:42:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.654+0000 7ff138ff9640 1 -- 172.21.15.139:0/1395857823 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff124042030 con 0x7ff13c101190 2024-09-16T01:42:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.654+0000 7ff1441fc740 1 -- start start 2024-09-16T01:42:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.654+0000 7ff1441fc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee043cc0 0x5609ee0440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.654+0000 7ff1441fc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee0445e0 0x5609ee04e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.654+0000 7ff1441fc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee04ee80 0x5609ee051270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.654+0000 7ff1441fc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5609edfa82f0 con 0x5609ee04ee80 2024-09-16T01:42:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.654+0000 7ff1441fc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5609edf50270 con 0x5609ee0445e0 2024-09-16T01:42:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.654+0000 7ff1441fc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5609edfa42b0 con 0x5609ee043cc0 2024-09-16T01:42:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.654+0000 7ff133fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee04ee80 0x5609ee051270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.654+0000 7ff133fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee04ee80 0x5609ee051270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53982/0 (socket says 172.21.15.139:53982) 2024-09-16T01:42:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.654+0000 7ff133fff640 1 -- 172.21.15.139:0/2971044532 learned_addr learned my addr 172.21.15.139:0/2971044532 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.654+0000 7ff13b7fe640 1 --2- 172.21.15.139:0/2971044532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee043cc0 0x5609ee0440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.655+0000 7ff13affd640 1 --2- 172.21.15.139:0/2971044532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee0445e0 0x5609ee04e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.655+0000 7ff133fff640 1 -- 172.21.15.139:0/2971044532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee043cc0 msgr2=0x5609ee0440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.655+0000 7ff133fff640 1 --2- 172.21.15.139:0/2971044532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee043cc0 0x5609ee0440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.655+0000 7ff133fff640 1 -- 172.21.15.139:0/2971044532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee0445e0 msgr2=0x5609ee04e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.655+0000 7ff133fff640 1 --2- 172.21.15.139:0/2971044532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee0445e0 0x5609ee04e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.655+0000 7ff138ff9640 1 --2- 172.21.15.139:0/1395857823 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff1280a34b0 0x7ff1280a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.655+0000 7ff133fff640 1 -- 172.21.15.139:0/2971044532 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609edf4f340 con 0x5609ee04ee80 2024-09-16T01:42:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.655+0000 7ff133fff640 1 --2- 172.21.15.139:0/2971044532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee04ee80 0x5609ee051270 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff1240ed680 tx=0x7ff1240b8630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df776644b1e06f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.655+0000 7ff13b7fe640 1 --2- 172.21.15.139:0/1395857823 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff1280a34b0 0x7ff1280a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.656+0000 7ff138ff9640 1 -- 172.21.15.139:0/1395857823 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7ff1240f0400 con 0x7ff13c101190 2024-09-16T01:42:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.656+0000 7ff1317fa640 1 -- 172.21.15.139:0/2971044532 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124049070 con 0x5609ee04ee80 2024-09-16T01:42:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.656+0000 7ff1317fa640 1 -- 172.21.15.139:0/2971044532 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1240bc2d0 con 0x5609ee04ee80 2024-09-16T01:42:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.656+0000 7ff1317fa640 1 -- 172.21.15.139:0/2971044532 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240bbe10 con 0x5609ee04ee80 2024-09-16T01:42:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.656+0000 7ff13b7fe640 1 --2- 172.21.15.139:0/1395857823 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff1280a34b0 0x7ff1280a5970 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff134005fd0 tx=0x7ff134009500 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.657+0000 7ff1441fc740 1 -- 172.21.15.139:0/2971044532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee04ee80 msgr2=0x5609ee051270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.657+0000 7ff1441fc740 1 --2- 172.21.15.139:0/2971044532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee04ee80 0x5609ee051270 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff1240ed680 tx=0x7ff1240b8630 comp rx=0 tx=0).stop 2024-09-16T01:42:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.657+0000 7ff1441fc740 1 -- 172.21.15.139:0/2971044532 shutdown_connections 2024-09-16T01:42:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.657+0000 7ff1441fc740 1 --2- 172.21.15.139:0/2971044532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee04ee80 0x5609ee051270 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.657+0000 7ff1441fc740 1 --2- 172.21.15.139:0/2971044532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee0445e0 0x5609ee04e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.657+0000 7ff1441fc740 1 --2- 172.21.15.139:0/2971044532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee043cc0 0x5609ee0440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.657+0000 7ff1441fc740 1 -- 172.21.15.139:0/2971044532 >> 172.21.15.139:0/2971044532 conn(0x5609ee040230 msgr2=0x5609ee040610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.657+0000 7ff1441fc740 1 -- 172.21.15.139:0/2971044532 shutdown_connections 2024-09-16T01:42:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.657+0000 7ff1441fc740 1 -- 172.21.15.139:0/2971044532 wait complete. 2024-09-16T01:42:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.657+0000 7ff1441fc740 1 -- start start 2024-09-16T01:42:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.658+0000 7ff1441fc740 1 Processor -- start 2024-09-16T01:42:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.658+0000 7ff1441fc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee043cc0 0x5609ee04bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.658+0000 7ff1441fc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee0445e0 0x5609ee04c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.658+0000 7ff13affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee043cc0 0x5609ee04bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.658+0000 7ff13affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee043cc0 0x5609ee04bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47224/0 (socket says 172.21.15.139:47224) 2024-09-16T01:42:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.658+0000 7ff133fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee0445e0 0x5609ee04c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.658+0000 7ff133fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee0445e0 0x5609ee04c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53992/0 (socket says 172.21.15.139:53992) 2024-09-16T01:42:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.658+0000 7ff13affd640 1 -- 172.21.15.139:0/1745494324 learned_addr learned my addr 172.21.15.139:0/1745494324 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.658+0000 7ff1441fc740 1 --2- 172.21.15.139:0/1745494324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee04ee80 0x5609ee04c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.658+0000 7ff1441fc740 1 -- 172.21.15.139:0/1745494324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5609ee051810 con 0x5609ee0445e0 2024-09-16T01:42:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.658+0000 7ff1441fc740 1 -- 172.21.15.139:0/1745494324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5609edf31290 con 0x5609ee043cc0 2024-09-16T01:42:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.658+0000 7ff1441fc740 1 -- 172.21.15.139:0/1745494324 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5609ee051990 con 0x5609ee04ee80 2024-09-16T01:42:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff1441fc740 10 client.0 _ll_register_callbacks cb 0x5609edf62140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:42:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff13b7fe640 1 --2- 172.21.15.139:0/1745494324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee04ee80 0x5609ee04c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff133fff640 1 -- 172.21.15.139:0/1745494324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee04ee80 msgr2=0x5609ee04c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff133fff640 1 --2- 172.21.15.139:0/1745494324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee04ee80 0x5609ee04c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff133fff640 1 -- 172.21.15.139:0/1745494324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee043cc0 msgr2=0x5609ee04bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff133fff640 1 --2- 172.21.15.139:0/1745494324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee043cc0 0x5609ee04bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff133fff640 1 -- 172.21.15.139:0/1745494324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609edcbb210 con 0x5609ee0445e0 2024-09-16T01:42:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff13b7fe640 1 --2- 172.21.15.139:0/1745494324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee04ee80 0x5609ee04c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:42:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff133fff640 1 --2- 172.21.15.139:0/1745494324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee0445e0 0x5609ee04c2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff124009900 tx=0x7ff12403b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd143d988362ad50 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff1227fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:42:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff1227fc640 1 -- 172.21.15.139:0/1745494324 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124049040 con 0x5609ee0445e0 2024-09-16T01:42:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff1227fc640 1 -- 172.21.15.139:0/1745494324 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7ff1240b8940 con 0x5609ee0445e0 2024-09-16T01:42:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff1441fc740 1 -- 172.21.15.139:0/1745494324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5609edfb5a20 con 0x5609ee0445e0 2024-09-16T01:42:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff1227fc640 1 -- 172.21.15.139:0/1745494324 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff1240bbc30 con 0x5609ee0445e0 2024-09-16T01:42:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff1441fc740 10 client.15753 fetch_fsmap learned FSMap version 31 2024-09-16T01:42:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.659+0000 7ff1441fc740 1 -- 172.21.15.139:0/1745494324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5609edd87620 con 0x5609ee0445e0 2024-09-16T01:42:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.660+0000 7ff1227fc640 1 -- 172.21.15.139:0/1745494324 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5609edfb5a20 con 0x5609ee0445e0 2024-09-16T01:42:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.661+0000 7ff1227fc640 1 -- 172.21.15.139:0/1745494324 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7ff1240bc530 con 0x5609ee0445e0 2024-09-16T01:42:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.661+0000 7ff1441fc740 10 client.15753 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:42:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.661+0000 7ff1441fc740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:42:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.661+0000 7ff1441fc740 20 client.15753 populate_metadata read hostname 'smithi139' 2024-09-16T01:42:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.661+0000 7ff1441fc740 1 --2- 172.21.15.139:0/1745494324 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5609ee05ebf0 0x5609ee061030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.661+0000 7ff1441fc740 4 client.15753 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:42:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.661+0000 7ff1441fc740 1 -- 172.21.15.139:0/1745494324 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609edfafd70 con 0x5609ee05ebf0 2024-09-16T01:42:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.662+0000 7ff13affd640 1 --2- 172.21.15.139:0/1745494324 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5609ee05ebf0 0x5609ee061030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.663+0000 7ff13affd640 1 --2- 172.21.15.139:0/1745494324 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5609ee05ebf0 0x5609ee061030 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfc54ff426f6b033 server_cookie=8a1e8cb38ed6df9b in_seq=0 out_seq=0 2024-09-16T01:42:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.663+0000 7ff1227fc640 10 client.15753 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.665+0000 7ff1227fc640 1 -- 172.21.15.139:0/1745494324 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609edfafd70 con 0x5609ee05ebf0 2024-09-16T01:42:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.666+0000 7ff1227fc640 10 client.15753 handle_command_reply: tid=0 2024-09-16T01:42:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.666+0000 7ff1441fc740 1 client.15753 shutdown 2024-09-16T01:42:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.669+0000 7ff1441fc740 1 -- 172.21.15.139:0/1745494324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee0445e0 msgr2=0x5609ee04c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.669+0000 7ff1441fc740 1 --2- 172.21.15.139:0/1745494324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee0445e0 0x5609ee04c2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff124009900 tx=0x7ff12403b560 comp rx=0 tx=0).stop 2024-09-16T01:42:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.669+0000 7ff1441fc740 1 -- 172.21.15.139:0/1745494324 shutdown_connections 2024-09-16T01:42:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.669+0000 7ff1441fc740 1 --2- 172.21.15.139:0/1745494324 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5609ee05ebf0 0x5609ee061030 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.669+0000 7ff1441fc740 1 --2- 172.21.15.139:0/1745494324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee04ee80 0x5609ee04c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.669+0000 7ff1441fc740 1 --2- 172.21.15.139:0/1745494324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee0445e0 0x5609ee04c2f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.669+0000 7ff1441fc740 1 --2- 172.21.15.139:0/1745494324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee043cc0 0x5609ee04bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.670+0000 7ff1441fc740 1 -- 172.21.15.139:0/1745494324 >> 172.21.15.139:0/1745494324 conn(0x5609ee040230 msgr2=0x5609ee045f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.670+0000 7ff1227fc640 0 client.15753 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.670+0000 7ff1441fc740 1 -- 172.21.15.139:0/1745494324 shutdown_connections 2024-09-16T01:42:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.670+0000 7ff1441fc740 1 -- 172.21.15.139:0/1745494324 wait complete. 2024-09-16T01:42:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.670+0000 7ff1441fc740 20 client.15753 trim_cache size 0 max 16384 2024-09-16T01:42:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.688+0000 7ff1441fc740 1 Processor -- start 2024-09-16T01:42:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.688+0000 7ff1441fc740 1 -- start start 2024-09-16T01:42:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff1441fc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee043cc0 0x5609ee04c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff1441fc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee0445e0 0x5609ee066900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff1441fc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee04ee80 0x5609ee066e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff1441fc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1bdce0 con 0x5609ee04ee80 2024-09-16T01:42:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff1441fc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1bf8e0 con 0x5609ee043cc0 2024-09-16T01:42:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff1441fc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1be3b0 con 0x5609ee0445e0 2024-09-16T01:42:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee04ee80 0x5609ee066e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff13affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee043cc0 0x5609ee04c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff133fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee0445e0 0x5609ee066900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee04ee80 0x5609ee066e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54000/0 (socket says 172.21.15.139:54000) 2024-09-16T01:42:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff13affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee043cc0 0x5609ee04c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47226/0 (socket says 172.21.15.139:47226) 2024-09-16T01:42:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff13b7fe640 1 -- 172.21.15.139:0/1115764865 learned_addr learned my addr 172.21.15.139:0/1115764865 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff13b7fe640 1 -- 172.21.15.139:0/1115764865 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee0445e0 msgr2=0x5609ee066900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff13b7fe640 1 --2- 172.21.15.139:0/1115764865 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee0445e0 0x5609ee066900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff13b7fe640 1 -- 172.21.15.139:0/1115764865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee043cc0 msgr2=0x5609ee04c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff13b7fe640 1 --2- 172.21.15.139:0/1115764865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee043cc0 0x5609ee04c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff13b7fe640 1 -- 172.21.15.139:0/1115764865 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609ee04e900 con 0x5609ee04ee80 2024-09-16T01:42:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff13affd640 1 --2- 172.21.15.139:0/1115764865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee043cc0 0x5609ee04c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.689+0000 7ff13b7fe640 1 --2- 172.21.15.139:0/1115764865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee04ee80 0x5609ee066e40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff1340049c0 tx=0x7ff134015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e93ed09090fc8dc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.690+0000 7ff130ff9640 1 -- 172.21.15.139:0/1115764865 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134027470 con 0x5609ee04ee80 2024-09-16T01:42:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.690+0000 7ff130ff9640 1 -- 172.21.15.139:0/1115764865 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff134027af0 con 0x5609ee04ee80 2024-09-16T01:42:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.690+0000 7ff130ff9640 1 -- 172.21.15.139:0/1115764865 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13401bce0 con 0x5609ee04ee80 2024-09-16T01:42:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.690+0000 7ff1441fc740 1 -- 172.21.15.139:0/1115764865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee04ee80 msgr2=0x5609ee066e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.690+0000 7ff1441fc740 1 --2- 172.21.15.139:0/1115764865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee04ee80 0x5609ee066e40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff1340049c0 tx=0x7ff134015000 comp rx=0 tx=0).stop 2024-09-16T01:42:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 -- 172.21.15.139:0/1115764865 shutdown_connections 2024-09-16T01:42:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 --2- 172.21.15.139:0/1115764865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee04ee80 0x5609ee066e40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 --2- 172.21.15.139:0/1115764865 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee0445e0 0x5609ee066900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 --2- 172.21.15.139:0/1115764865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee043cc0 0x5609ee04c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 -- 172.21.15.139:0/1115764865 >> 172.21.15.139:0/1115764865 conn(0x5609ee040230 msgr2=0x5609ee047020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 -- 172.21.15.139:0/1115764865 shutdown_connections 2024-09-16T01:42:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 -- 172.21.15.139:0/1115764865 wait complete. 2024-09-16T01:42:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 -- start start 2024-09-16T01:42:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 Processor -- start 2024-09-16T01:42:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee043cc0 0x5609ee051a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee0445e0 0x5609ee051fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee04ee80 0x5609ee0524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1bf150 con 0x5609ee043cc0 2024-09-16T01:42:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1bea80 con 0x5609ee0445e0 2024-09-16T01:42:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1a4c80 con 0x5609ee04ee80 2024-09-16T01:42:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff1441fc740 10 client.0 _ll_register_callbacks cb 0x7ff1240f83c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:42:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff13affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee043cc0 0x5609ee051a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff13affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee043cc0 0x5609ee051a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54012/0 (socket says 172.21.15.139:54012) 2024-09-16T01:42:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff13affd640 1 -- 172.21.15.139:0/3723196013 learned_addr learned my addr 172.21.15.139:0/3723196013 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff133fff640 1 --2- 172.21.15.139:0/3723196013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee0445e0 0x5609ee051fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.691+0000 7ff13b7fe640 1 --2- 172.21.15.139:0/3723196013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee04ee80 0x5609ee0524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff13affd640 1 -- 172.21.15.139:0/3723196013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee04ee80 msgr2=0x5609ee0524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff13affd640 1 --2- 172.21.15.139:0/3723196013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee04ee80 0x5609ee0524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff13affd640 1 -- 172.21.15.139:0/3723196013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee0445e0 msgr2=0x5609ee051fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff13affd640 1 --2- 172.21.15.139:0/3723196013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee0445e0 0x5609ee051fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff13affd640 1 -- 172.21.15.139:0/3723196013 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609ee0c7a70 con 0x5609ee043cc0 2024-09-16T01:42:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff133fff640 1 --2- 172.21.15.139:0/3723196013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee0445e0 0x5609ee051fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff13affd640 1 --2- 172.21.15.139:0/3723196013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee043cc0 0x5609ee051a70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x5609ee0618a0 tx=0x7ff12c0041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561ce373acbf7a76 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff1227fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:42:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff1227fc640 1 -- 172.21.15.139:0/3723196013 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c02b060 con 0x5609ee043cc0 2024-09-16T01:42:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff1227fc640 1 -- 172.21.15.139:0/3723196013 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7ff12c020410 con 0x5609ee043cc0 2024-09-16T01:42:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff1441fc740 1 -- 172.21.15.139:0/3723196013 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff134027af0 con 0x5609ee043cc0 2024-09-16T01:42:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff1227fc640 1 -- 172.21.15.139:0/3723196013 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff12c020610 con 0x5609ee043cc0 2024-09-16T01:42:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff1441fc740 10 client.15759 fetch_fsmap learned FSMap version 31 2024-09-16T01:42:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff1441fc740 1 -- 172.21.15.139:0/3723196013 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5609ee0c8d80 con 0x5609ee043cc0 2024-09-16T01:42:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.692+0000 7ff1227fc640 1 -- 172.21.15.139:0/3723196013 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff134027af0 con 0x5609ee043cc0 2024-09-16T01:42:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.693+0000 7ff1227fc640 1 -- 172.21.15.139:0/3723196013 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7ff12c025030 con 0x5609ee043cc0 2024-09-16T01:42:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.693+0000 7ff1441fc740 10 client.15759 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:42:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.693+0000 7ff1441fc740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:42:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.693+0000 7ff1441fc740 20 client.15759 populate_metadata read hostname 'smithi139' 2024-09-16T01:42:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.693+0000 7ff1441fc740 1 --2- 172.21.15.139:0/3723196013 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5609ee0c9510 0x5609ee0cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.693+0000 7ff1441fc740 4 client.15759 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:42:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.693+0000 7ff1441fc740 1 -- 172.21.15.139:0/3723196013 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609edf2c900 con 0x5609ee0c9510 2024-09-16T01:42:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.693+0000 7ff133fff640 1 --2- 172.21.15.139:0/3723196013 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5609ee0c9510 0x5609ee0cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.693+0000 7ff1227fc640 10 client.15759 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:27.693+0000 7ff133fff640 1 --2- 172.21.15.139:0/3723196013 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5609ee0c9510 0x5609ee0cb930 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0808d98cda2020f server_cookie=799e56d5ae3660db in_seq=0 out_seq=0 2024-09-16T01:42:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.978+0000 7ff1227fc640 1 -- 172.21.15.139:0/3723196013 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609edf2c900 con 0x5609ee0c9510 2024-09-16T01:42:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.978+0000 7ff1227fc640 10 client.15759 handle_command_reply: tid=0 2024-09-16T01:42:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.978+0000 7ff1441fc740 1 client.15759 shutdown 2024-09-16T01:42:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.979+0000 7ff1441fc740 1 -- 172.21.15.139:0/3723196013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee043cc0 msgr2=0x5609ee051a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.979+0000 7ff1441fc740 1 --2- 172.21.15.139:0/3723196013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee043cc0 0x5609ee051a70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x5609ee0618a0 tx=0x7ff12c0041e0 comp rx=0 tx=0).stop 2024-09-16T01:42:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.980+0000 7ff1441fc740 1 -- 172.21.15.139:0/3723196013 shutdown_connections 2024-09-16T01:42:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.980+0000 7ff1441fc740 1 --2- 172.21.15.139:0/3723196013 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5609ee0c9510 0x5609ee0cb930 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.980+0000 7ff1441fc740 1 --2- 172.21.15.139:0/3723196013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609ee04ee80 0x5609ee0524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.980+0000 7ff1441fc740 1 --2- 172.21.15.139:0/3723196013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609ee0445e0 0x5609ee051fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.980+0000 7ff1441fc740 1 --2- 172.21.15.139:0/3723196013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609ee043cc0 0x5609ee051a70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.980+0000 7ff1441fc740 1 -- 172.21.15.139:0/3723196013 >> 172.21.15.139:0/3723196013 conn(0x5609ee040230 msgr2=0x5609ee045720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.980+0000 7ff1227fc640 0 client.15759 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.980+0000 7ff1441fc740 1 -- 172.21.15.139:0/3723196013 shutdown_connections 2024-09-16T01:42:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.980+0000 7ff1441fc740 1 -- 172.21.15.139:0/3723196013 wait complete. 2024-09-16T01:42:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.980+0000 7ff1441fc740 20 client.15759 trim_cache size 0 max 16384 2024-09-16T01:42:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:42:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:42:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:42:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:42:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.982+0000 7ff141386640 1 -- 172.21.15.139:0/1395857823 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff1280a34b0 msgr2=0x7ff1280a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.982+0000 7ff141386640 1 --2- 172.21.15.139:0/1395857823 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff1280a34b0 0x7ff1280a5970 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff134005fd0 tx=0x7ff134009500 comp rx=0 tx=0).stop 2024-09-16T01:42:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.982+0000 7ff141386640 1 -- 172.21.15.139:0/1395857823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c101190 msgr2=0x7ff13c197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.982+0000 7ff141386640 1 --2- 172.21.15.139:0/1395857823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c101190 0x7ff13c197460 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff124009230 tx=0x7ff124004040 comp rx=0 tx=0).stop 2024-09-16T01:42:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.982+0000 7ff141386640 1 -- 172.21.15.139:0/1395857823 shutdown_connections 2024-09-16T01:42:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.982+0000 7ff141386640 1 --2- 172.21.15.139:0/1395857823 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff1280a34b0 0x7ff1280a5970 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.982+0000 7ff141386640 1 --2- 172.21.15.139:0/1395857823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff13c106800 0x7ff13c19bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.982+0000 7ff141386640 1 --2- 172.21.15.139:0/1395857823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c101190 0x7ff13c197460 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.982+0000 7ff141386640 1 --2- 172.21.15.139:0/1395857823 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff13c1007f0 0x7ff13c196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.982+0000 7ff141386640 1 -- 172.21.15.139:0/1395857823 >> 172.21.15.139:0/1395857823 conn(0x7ff13c0777c0 msgr2=0x7ff13c103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.982+0000 7ff141386640 1 -- 172.21.15.139:0/1395857823 shutdown_connections 2024-09-16T01:42:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:28.982+0000 7ff141386640 1 -- 172.21.15.139:0/1395857823 wait complete. 2024-09-16T01:42:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:29 smithi139 ceph-mon[31175]: pgmap v65: 129 pgs: 129 active+clean; 181 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.5 MiB/s wr, 347 op/s 2024-09-16T01:42:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:29 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:42:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:29 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:42:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:29 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:42:29.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:29 smithi176 ceph-mon[35324]: pgmap v65: 129 pgs: 129 active+clean; 181 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.5 MiB/s wr, 347 op/s 2024-09-16T01:42:29.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:29 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:42:29.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:29 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:42:29.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:29 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:42:29.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:29 smithi164 ceph-mon[35121]: pgmap v65: 129 pgs: 129 active+clean; 181 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.5 MiB/s wr, 347 op/s 2024-09-16T01:42:29.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:29 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:42:29.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:29 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:42:29.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:29 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:42:30.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:30 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:42:30.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:30 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:42:30.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:30 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:42:30.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:30 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:42:30.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:30 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:42:30.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:30 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:42:30.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:30 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:42:30.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:30 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:42:30.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:30 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:42:31.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:31 smithi164 ceph-mon[35121]: pgmap v66: 129 pgs: 129 active+clean; 182 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 519 op/s 2024-09-16T01:42:31.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:31 smithi139 ceph-mon[31175]: pgmap v66: 129 pgs: 129 active+clean; 182 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 519 op/s 2024-09-16T01:42:31.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:31 smithi176 ceph-mon[35324]: pgmap v66: 129 pgs: 129 active+clean; 182 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 519 op/s 2024-09-16T01:42:32.781 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T01:42:32.781 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.55 2024-09-16T01:42:32.782 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:42:33.056 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:42:33.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.678+0000 7faed213d640 1 -- 172.21.15.139:0/2535042632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faecc06ea30 msgr2=0x7faecc06ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:33.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.679+0000 7faed213d640 1 --2- 172.21.15.139:0/2535042632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faecc06ea30 0x7faecc06ee90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7faec0009900 tx=0x7faec002a440 comp rx=0 tx=0).stop 2024-09-16T01:42:33.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.680+0000 7faed213d640 1 -- 172.21.15.139:0/2535042632 shutdown_connections 2024-09-16T01:42:33.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.680+0000 7faed213d640 1 --2- 172.21.15.139:0/2535042632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faecc11b140 0x7faecc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:33.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.680+0000 7faed213d640 1 --2- 172.21.15.139:0/2535042632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faecc06ea30 0x7faecc06ee90 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:33.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.680+0000 7faed213d640 1 --2- 172.21.15.139:0/2535042632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faecc110220 0x7faecc06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:33.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.680+0000 7faed213d640 1 -- 172.21.15.139:0/2535042632 >> 172.21.15.139:0/2535042632 conn(0x7faecc073210 msgr2=0x7faecc073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:33.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.685+0000 7faed213d640 1 -- 172.21.15.139:0/2535042632 shutdown_connections 2024-09-16T01:42:33.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.685+0000 7faed213d640 1 -- 172.21.15.139:0/2535042632 wait complete. 2024-09-16T01:42:33.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.686+0000 7faed213d640 1 Processor -- start 2024-09-16T01:42:33.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.689+0000 7faed213d640 1 -- start start 2024-09-16T01:42:33.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.690+0000 7faed213d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faecc06ea30 0x7faecc1a72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:33.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.690+0000 7faed213d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faecc110220 0x7faecc1a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:33.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.690+0000 7faed213d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faecc11b140 0x7faecc1abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:33.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.690+0000 7faed213d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faecc120db0 con 0x7faecc06ea30 2024-09-16T01:42:33.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.690+0000 7faed213d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faecc120c30 con 0x7faecc110220 2024-09-16T01:42:33.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.691+0000 7faed213d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7faecc120f30 con 0x7faecc11b140 2024-09-16T01:42:33.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.691+0000 7faecb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faecc06ea30 0x7faecc1a72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:33.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.691+0000 7faecb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faecc06ea30 0x7faecc1a72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33274/0 (socket says 172.21.15.139:33274) 2024-09-16T01:42:33.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.691+0000 7faecb7fe640 1 -- 172.21.15.139:0/3825110267 learned_addr learned my addr 172.21.15.139:0/3825110267 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:33.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.692+0000 7faecbfff640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faecc11b140 0x7faecc1abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:33.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.692+0000 7faecb7fe640 1 -- 172.21.15.139:0/3825110267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faecc11b140 msgr2=0x7faecc1abb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:33.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.692+0000 7faecb7fe640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faecc11b140 0x7faecc1abb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:33.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.692+0000 7faecb7fe640 1 -- 172.21.15.139:0/3825110267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faecc110220 msgr2=0x7faecc1a77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:33.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.692+0000 7faecaffd640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faecc110220 0x7faecc1a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:33.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.692+0000 7faecb7fe640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faecc110220 0x7faecc1a77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:33.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.692+0000 7faecb7fe640 1 -- 172.21.15.139:0/3825110267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc1ac260 con 0x7faecc06ea30 2024-09-16T01:42:33.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.692+0000 7faecbfff640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faecc11b140 0x7faecc1abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:42:33.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.693+0000 7faecb7fe640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faecc06ea30 0x7faecc1a72b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7faebc00b450 tx=0x7faebc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:33.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.694+0000 7faec8ff9640 1 -- 172.21.15.139:0/3825110267 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc015020 con 0x7faecc06ea30 2024-09-16T01:42:33.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.694+0000 7faec8ff9640 1 -- 172.21.15.139:0/3825110267 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faebc011d70 con 0x7faecc06ea30 2024-09-16T01:42:33.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.694+0000 7faec8ff9640 1 -- 172.21.15.139:0/3825110267 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc00f340 con 0x7faecc06ea30 2024-09-16T01:42:33.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.694+0000 7faed213d640 1 -- 172.21.15.139:0/3825110267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faecc1ac550 con 0x7faecc06ea30 2024-09-16T01:42:33.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.694+0000 7faed213d640 1 -- 172.21.15.139:0/3825110267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faecc1b3e30 con 0x7faecc06ea30 2024-09-16T01:42:33.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.699+0000 7faed213d640 1 -- 172.21.15.139:0/3825110267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faecc109740 con 0x7faecc06ea30 2024-09-16T01:42:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.703+0000 7faec8ff9640 1 -- 172.21.15.139:0/3825110267 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7faebc018020 con 0x7faecc06ea30 2024-09-16T01:42:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.704+0000 7faec8ff9640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faeb40a3580 0x7faeb40a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.704+0000 7faec8ff9640 1 -- 172.21.15.139:0/3825110267 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7faebc0cfaa0 con 0x7faecc06ea30 2024-09-16T01:42:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.710+0000 7faecaffd640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faeb40a3580 0x7faeb40a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:33.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.711+0000 7faecaffd640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faeb40a3580 0x7faeb40a5a40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faec00096c0 tx=0x7faec0032d10 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:33.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:33.714+0000 7faec8ff9640 1 -- 172.21.15.139:0/3825110267 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faebc01f020 con 0x7faecc06ea30 2024-09-16T01:42:33.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:33 smithi164 ceph-mon[35121]: pgmap v67: 129 pgs: 129 active+clean; 170 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.2 MiB/s wr, 806 op/s 2024-09-16T01:42:33.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:33 smithi139 ceph-mon[31175]: pgmap v67: 129 pgs: 129 active+clean; 170 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.2 MiB/s wr, 806 op/s 2024-09-16T01:42:33.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:33 smithi176 ceph-mon[35324]: pgmap v67: 129 pgs: 129 active+clean; 170 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.2 MiB/s wr, 806 op/s 2024-09-16T01:42:34.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:34.001+0000 7faed213d640 1 -- 172.21.15.139:0/3825110267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faecc06dbf0 con 0x7faecc06ea30 2024-09-16T01:42:34.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:34.001+0000 7faec8ff9640 1 -- 172.21.15.139:0/3825110267 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12002 (secure 0 0 0) 0x7faebc00d040 con 0x7faecc06ea30 2024-09-16T01:42:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:42:34.003 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":31,"btime":"2024-09-16T01:42:06:037308+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:42:05.468117+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:42:34.004 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 31 2024-09-16T01:42:34.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:34.004+0000 7faed213d640 1 -- 172.21.15.139:0/3825110267 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faeb40a3580 msgr2=0x7faeb40a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:34.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:34.004+0000 7faed213d640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faeb40a3580 0x7faeb40a5a40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faec00096c0 tx=0x7faec0032d10 comp rx=0 tx=0).stop 2024-09-16T01:42:34.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:34.004+0000 7faed213d640 1 -- 172.21.15.139:0/3825110267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faecc06ea30 msgr2=0x7faecc1a72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:34.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:34.004+0000 7faed213d640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faecc06ea30 0x7faecc1a72b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7faebc00b450 tx=0x7faebc00b910 comp rx=0 tx=0).stop 2024-09-16T01:42:34.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:34.004+0000 7faed213d640 1 -- 172.21.15.139:0/3825110267 shutdown_connections 2024-09-16T01:42:34.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:34.004+0000 7faed213d640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faeb40a3580 0x7faeb40a5a40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:34.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:34.004+0000 7faed213d640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faecc11b140 0x7faecc1abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:34.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:34.004+0000 7faed213d640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faecc110220 0x7faecc1a77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:34.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:34.004+0000 7faed213d640 1 --2- 172.21.15.139:0/3825110267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faecc06ea30 0x7faecc1a72b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:34.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:34.004+0000 7faed213d640 1 -- 172.21.15.139:0/3825110267 >> 172.21.15.139:0/3825110267 conn(0x7faecc073210 msgr2=0x7faecc1192d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:34.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:34.004+0000 7faed213d640 1 -- 172.21.15.139:0/3825110267 shutdown_connections 2024-09-16T01:42:34.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:34.004+0000 7faed213d640 1 -- 172.21.15.139:0/3825110267 wait complete. 2024-09-16T01:42:34.608 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:34 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3825110267' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:42:34.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:34 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3825110267' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:42:34.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:34 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3825110267' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:42:35.470 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0' 2024-09-16T01:42:35.470 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0 2024-09-16T01:42:35.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:35 smithi139 ceph-mon[31175]: pgmap v68: 129 pgs: 129 active+clean; 170 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 780 op/s 2024-09-16T01:42:35.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:35 smithi164 ceph-mon[35121]: pgmap v68: 129 pgs: 129 active+clean; 170 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 780 op/s 2024-09-16T01:42:35.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:35 smithi176 ceph-mon[35324]: pgmap v68: 129 pgs: 129 active+clean; 170 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 780 op/s 2024-09-16T01:42:35.967 INFO:tasks.workunit.client.0.smithi139.stdout: 88 697673 1416 523294 1390 165294 177 2024-09-16T01:42:36.018 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:42:36.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.572+0000 7fc4a62ba640 1 -- 172.21.15.139:0/2951297569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a0102ef0 msgr2=0x7fc4a010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:36.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.572+0000 7fc4a62ba640 1 --2- 172.21.15.139:0/2951297569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a0102ef0 0x7fc4a010da40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc498009ca0 tx=0x7fc49802a950 comp rx=0 tx=0).stop 2024-09-16T01:42:36.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.572+0000 7fc4a62ba640 1 -- 172.21.15.139:0/2951297569 shutdown_connections 2024-09-16T01:42:36.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.572+0000 7fc4a62ba640 1 --2- 172.21.15.139:0/2951297569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a0102ef0 0x7fc4a010da40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.572+0000 7fc4a62ba640 1 --2- 172.21.15.139:0/2951297569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a0102570 0x7fc4a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.572+0000 7fc4a62ba640 1 --2- 172.21.15.139:0/2951297569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4a0100e20 0x7fc4a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.572+0000 7fc4a62ba640 1 -- 172.21.15.139:0/2951297569 >> 172.21.15.139:0/2951297569 conn(0x7fc4a00fc7d0 msgr2=0x7fc4a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:36.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.572+0000 7fc4a62ba640 1 -- 172.21.15.139:0/2951297569 shutdown_connections 2024-09-16T01:42:36.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.572+0000 7fc4a62ba640 1 -- 172.21.15.139:0/2951297569 wait complete. 2024-09-16T01:42:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.573+0000 7fc4a62ba640 1 Processor -- start 2024-09-16T01:42:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.573+0000 7fc4a62ba640 1 -- start start 2024-09-16T01:42:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc4a62ba640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4a0100e20 0x7fc4a0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc4a62ba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a0102570 0x7fc4a01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc4a62ba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a0102ef0 0x7fc4a019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc4a62ba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0110fd0 con 0x7fc4a0102570 2024-09-16T01:42:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc4a62ba640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0110e50 con 0x7fc4a0102ef0 2024-09-16T01:42:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc4a62ba640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0111150 con 0x7fc4a0100e20 2024-09-16T01:42:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc497fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a0102570 0x7fc4a01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc497fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a0102570 0x7fc4a01989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33294/0 (socket says 172.21.15.139:33294) 2024-09-16T01:42:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc497fff640 1 -- 172.21.15.139:0/3785330006 learned_addr learned my addr 172.21.15.139:0/3785330006 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc4a4857640 1 --2- 172.21.15.139:0/3785330006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a0102ef0 0x7fc4a019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc49ffff640 1 --2- 172.21.15.139:0/3785330006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4a0100e20 0x7fc4a0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc497fff640 1 -- 172.21.15.139:0/3785330006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4a0100e20 msgr2=0x7fc4a0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:36.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc497fff640 1 --2- 172.21.15.139:0/3785330006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4a0100e20 0x7fc4a0198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc497fff640 1 -- 172.21.15.139:0/3785330006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a0102ef0 msgr2=0x7fc4a019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:36.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc497fff640 1 --2- 172.21.15.139:0/3785330006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a0102ef0 0x7fc4a019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.574+0000 7fc497fff640 1 -- 172.21.15.139:0/3785330006 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a019d420 con 0x7fc4a0102570 2024-09-16T01:42:36.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.575+0000 7fc497fff640 1 --2- 172.21.15.139:0/3785330006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a0102570 0x7fc4a01989b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc48800e9f0 tx=0x7fc48800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:36.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.575+0000 7fc49dffb640 1 -- 172.21.15.139:0/3785330006 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48800cd80 con 0x7fc4a0102570 2024-09-16T01:42:36.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.575+0000 7fc4a62ba640 1 -- 172.21.15.139:0/3785330006 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a019d710 con 0x7fc4a0102570 2024-09-16T01:42:36.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.575+0000 7fc49dffb640 1 -- 172.21.15.139:0/3785330006 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc488010910 con 0x7fc4a0102570 2024-09-16T01:42:36.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.575+0000 7fc49dffb640 1 -- 172.21.15.139:0/3785330006 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4880188f0 con 0x7fc4a0102570 2024-09-16T01:42:36.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.575+0000 7fc4a62ba640 1 -- 172.21.15.139:0/3785330006 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a01a4ff0 con 0x7fc4a0102570 2024-09-16T01:42:36.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.577+0000 7fc49dffb640 1 -- 172.21.15.139:0/3785330006 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc488018b10 con 0x7fc4a0102570 2024-09-16T01:42:36.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.578+0000 7fc4a8693740 1 Processor -- start 2024-09-16T01:42:36.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.578+0000 7fc4a8693740 1 -- start start 2024-09-16T01:42:36.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.578+0000 7fc4a8693740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cdc6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.578+0000 7fc4a8693740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc7090 0x55a64cdd5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.578+0000 7fc4a8693740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdd5e40 0x55a64cdd8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.578+0000 7fc4a8693740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a64cd2b160 con 0x55a64cdc7090 2024-09-16T01:42:36.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.578+0000 7fc4a8693740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a64ccd2720 con 0x55a64cdd5e40 2024-09-16T01:42:36.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.578+0000 7fc4a8693740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a64cd27120 con 0x55a64cdc6770 2024-09-16T01:42:36.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.578+0000 7fc4a4857640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cdc6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.578+0000 7fc4a4857640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cdc6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:37588/0 (socket says 172.21.15.139:37588) 2024-09-16T01:42:36.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.578+0000 7fc4a4857640 1 -- 172.21.15.139:0/2176218144 learned_addr learned my addr 172.21.15.139:0/2176218144 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:36.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.578+0000 7fc497fff640 1 --2- 172.21.15.139:0/2176218144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdd5e40 0x55a64cdd8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.578+0000 7fc49ffff640 1 --2- 172.21.15.139:0/2176218144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc7090 0x55a64cdd5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.578+0000 7fc49dffb640 1 --2- 172.21.15.139:0/3785330006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc48c0a33b0 0x7fc48c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.579+0000 7fc4a4857640 1 --2- 172.21.15.139:0/3785330006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc48c0a33b0 0x7fc48c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.579+0000 7fc497fff640 1 -- 172.21.15.139:0/2176218144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 msgr2=0x55a64cdc6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.579+0000 7fc497fff640 1 --2- 172.21.15.139:0/2176218144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cdc6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.579+0000 7fc497fff640 1 -- 172.21.15.139:0/2176218144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc7090 msgr2=0x55a64cdd5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.579+0000 7fc497fff640 1 --2- 172.21.15.139:0/2176218144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc7090 0x55a64cdd5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.579+0000 7fc497fff640 1 -- 172.21.15.139:0/2176218144 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a64ca9cba0 con 0x55a64cdd5e40 2024-09-16T01:42:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.579+0000 7fc49ffff640 1 --2- 172.21.15.139:0/2176218144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc7090 0x55a64cdd5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.579+0000 7fc497fff640 1 --2- 172.21.15.139:0/2176218144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdd5e40 0x55a64cdd8230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4880cd8a0 tx=0x7fc4880d6b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc7436f0f1b909c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.579+0000 7fc49dffb640 1 -- 172.21.15.139:0/3785330006 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fc4880d0500 con 0x7fc4a0102570 2024-09-16T01:42:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.579+0000 7fc4a4857640 1 --2- 172.21.15.139:0/3785330006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc48c0a33b0 0x7fc48c0a5870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc498005fe0 tx=0x7fc49802a950 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.579+0000 7fc4967fc640 1 -- 172.21.15.139:0/2176218144 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4880986d0 con 0x55a64cdd5e40 2024-09-16T01:42:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.579+0000 7fc4967fc640 1 -- 172.21.15.139:0/2176218144 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc48809cb30 con 0x55a64cdd5e40 2024-09-16T01:42:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.579+0000 7fc4967fc640 1 -- 172.21.15.139:0/2176218144 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48809bd90 con 0x55a64cdd5e40 2024-09-16T01:42:36.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.580+0000 7fc4a8693740 1 -- 172.21.15.139:0/2176218144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdd5e40 msgr2=0x55a64cdd8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:36.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.580+0000 7fc4a8693740 1 --2- 172.21.15.139:0/2176218144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdd5e40 0x55a64cdd8230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4880cd8a0 tx=0x7fc4880d6b80 comp rx=0 tx=0).stop 2024-09-16T01:42:36.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.580+0000 7fc4a8693740 1 -- 172.21.15.139:0/2176218144 shutdown_connections 2024-09-16T01:42:36.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.580+0000 7fc4a8693740 1 --2- 172.21.15.139:0/2176218144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdd5e40 0x55a64cdd8230 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.580+0000 7fc4a8693740 1 --2- 172.21.15.139:0/2176218144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc7090 0x55a64cdd5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.580+0000 7fc4a8693740 1 --2- 172.21.15.139:0/2176218144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cdc6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.580+0000 7fc4a8693740 1 -- 172.21.15.139:0/2176218144 >> 172.21.15.139:0/2176218144 conn(0x55a64cdc2de0 msgr2=0x55a64cdc31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:36.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.580+0000 7fc4a8693740 1 -- 172.21.15.139:0/2176218144 shutdown_connections 2024-09-16T01:42:36.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.580+0000 7fc4a8693740 1 -- 172.21.15.139:0/2176218144 wait complete. 2024-09-16T01:42:36.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.581+0000 7fc4a8693740 1 -- start start 2024-09-16T01:42:36.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.581+0000 7fc4a8693740 1 Processor -- start 2024-09-16T01:42:36.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.581+0000 7fc4a8693740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc6770 0x55a64cdd2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.581+0000 7fc4a8693740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdc7090 0x55a64cdd3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.581+0000 7fc4a8693740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdd5e40 0x55a64cdd3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.581+0000 7fc4a8693740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a64cdd8950 con 0x55a64cdc6770 2024-09-16T01:42:36.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.581+0000 7fc4a8693740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a64ccb3900 con 0x55a64cdc7090 2024-09-16T01:42:36.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.581+0000 7fc4a8693740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a64cdd8ad0 con 0x55a64cdd5e40 2024-09-16T01:42:36.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.581+0000 7fc4a8693740 10 client.0 _ll_register_callbacks cb 0x55a64ccb1780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:42:36.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.581+0000 7fc497fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdc7090 0x55a64cdd3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.581+0000 7fc49ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc6770 0x55a64cdd2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.581+0000 7fc497fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdc7090 0x55a64cdd3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37658/0 (socket says 172.21.15.139:37658) 2024-09-16T01:42:36.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.581+0000 7fc497fff640 1 -- 172.21.15.139:0/271392422 learned_addr learned my addr 172.21.15.139:0/271392422 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:36.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.582+0000 7fc4a4857640 1 --2- 172.21.15.139:0/271392422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdd5e40 0x55a64cdd3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.582+0000 7fc497fff640 1 -- 172.21.15.139:0/271392422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdd5e40 msgr2=0x55a64cdd3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:36.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.582+0000 7fc497fff640 1 --2- 172.21.15.139:0/271392422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdd5e40 0x55a64cdd3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.582+0000 7fc497fff640 1 -- 172.21.15.139:0/271392422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc6770 msgr2=0x55a64cdd2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:36.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.582+0000 7fc497fff640 1 --2- 172.21.15.139:0/271392422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc6770 0x55a64cdd2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.582+0000 7fc497fff640 1 -- 172.21.15.139:0/271392422 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a64ccf3c50 con 0x55a64cdc7090 2024-09-16T01:42:36.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.582+0000 7fc49ffff640 1 --2- 172.21.15.139:0/271392422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc6770 0x55a64cdd2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:36.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.582+0000 7fc497fff640 1 --2- 172.21.15.139:0/271392422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdc7090 0x55a64cdd3320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc4880cd950 tx=0x7fc4880d3b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b61f839bc577a719 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:36.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.582+0000 7fc47f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:42:36.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.582+0000 7fc47f7fe640 1 -- 172.21.15.139:0/271392422 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48809bf70 con 0x55a64cdc7090 2024-09-16T01:42:36.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.582+0000 7fc47f7fe640 1 -- 172.21.15.139:0/271392422 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fc48809a4d0 con 0x55a64cdc7090 2024-09-16T01:42:36.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.582+0000 7fc4a8693740 1 -- 172.21.15.139:0/271392422 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc48809cb30 con 0x55a64cdc7090 2024-09-16T01:42:36.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.583+0000 7fc47f7fe640 1 -- 172.21.15.139:0/271392422 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc4880b2e60 con 0x55a64cdc7090 2024-09-16T01:42:36.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.583+0000 7fc4a8693740 10 client.25283 fetch_fsmap learned FSMap version 31 2024-09-16T01:42:36.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.583+0000 7fc4a8693740 1 -- 172.21.15.139:0/271392422 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a64cdd8950 con 0x55a64cdc7090 2024-09-16T01:42:36.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.583+0000 7fc47f7fe640 1 -- 172.21.15.139:0/271392422 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc48809cb30 con 0x55a64cdc7090 2024-09-16T01:42:36.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.583+0000 7fc47f7fe640 1 -- 172.21.15.139:0/271392422 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7fc488098aa0 con 0x55a64cdc7090 2024-09-16T01:42:36.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.583+0000 7fc4a8693740 10 client.25283 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:42:36.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.583+0000 7fc4a8693740 10 client.25283 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:42:36.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.583+0000 7fc4a8693740 20 client.25283 populate_metadata read hostname 'smithi139' 2024-09-16T01:42:36.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.583+0000 7fc4a8693740 1 --2- 172.21.15.139:0/271392422 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a64cde5860 0x55a64cde7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.583+0000 7fc4a8693740 4 client.25283 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:42:36.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.583+0000 7fc4a8693740 1 -- 172.21.15.139:0/271392422 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a64cd38890 con 0x55a64cde5860 2024-09-16T01:42:36.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.583+0000 7fc49ffff640 1 --2- 172.21.15.139:0/271392422 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a64cde5860 0x55a64cde7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.584+0000 7fc47f7fe640 10 client.25283 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:36.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.584+0000 7fc49ffff640 1 --2- 172.21.15.139:0/271392422 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a64cde5860 0x55a64cde7c50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e49e32648987bb31 server_cookie=5a39d93faee5efa6 in_seq=0 out_seq=0 2024-09-16T01:42:36.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.586+0000 7fc47f7fe640 1 -- 172.21.15.139:0/271392422 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a64cd38890 con 0x55a64cde5860 2024-09-16T01:42:36.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.586+0000 7fc47f7fe640 10 client.25283 handle_command_reply: tid=0 2024-09-16T01:42:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.586+0000 7fc4a8693740 1 client.25283 shutdown 2024-09-16T01:42:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.586+0000 7fc4a8693740 1 -- 172.21.15.139:0/271392422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdc7090 msgr2=0x55a64cdd3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.586+0000 7fc4a8693740 1 --2- 172.21.15.139:0/271392422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdc7090 0x55a64cdd3320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc4880cd950 tx=0x7fc4880d3b60 comp rx=0 tx=0).stop 2024-09-16T01:42:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.586+0000 7fc4a8693740 1 -- 172.21.15.139:0/271392422 shutdown_connections 2024-09-16T01:42:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.586+0000 7fc4a8693740 1 --2- 172.21.15.139:0/271392422 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a64cde5860 0x55a64cde7c50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.586+0000 7fc4a8693740 1 --2- 172.21.15.139:0/271392422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdd5e40 0x55a64cdd3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.586+0000 7fc4a8693740 1 --2- 172.21.15.139:0/271392422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdc7090 0x55a64cdd3320 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.586+0000 7fc4a8693740 1 --2- 172.21.15.139:0/271392422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc6770 0x55a64cdd2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.586+0000 7fc47f7fe640 0 client.25283 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.586+0000 7fc4a8693740 1 -- 172.21.15.139:0/271392422 >> 172.21.15.139:0/271392422 conn(0x55a64cdc2de0 msgr2=0x55a64cdc8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.586+0000 7fc4a8693740 1 -- 172.21.15.139:0/271392422 shutdown_connections 2024-09-16T01:42:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.586+0000 7fc4a8693740 1 -- 172.21.15.139:0/271392422 wait complete. 2024-09-16T01:42:36.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.587+0000 7fc4a8693740 20 client.25283 trim_cache size 0 max 16384 2024-09-16T01:42:36.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.606+0000 7fc4a8693740 1 Processor -- start 2024-09-16T01:42:36.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.606+0000 7fc4a8693740 1 -- start start 2024-09-16T01:42:36.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.606+0000 7fc4a8693740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cded370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.606+0000 7fc4a8693740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc7090 0x55a64cded8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.606+0000 7fc4a8693740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdd5e40 0x55a64cdd2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.606+0000 7fc4a8693740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01acef0 con 0x55a64cdc7090 2024-09-16T01:42:36.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.606+0000 7fc4a8693740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01aeaf0 con 0x55a64cdd5e40 2024-09-16T01:42:36.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.606+0000 7fc4a8693740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01ad5c0 con 0x55a64cdc6770 2024-09-16T01:42:36.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.606+0000 7fc497fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc7090 0x55a64cded8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.606+0000 7fc497fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc7090 0x55a64cded8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33328/0 (socket says 172.21.15.139:33328) 2024-09-16T01:42:36.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.606+0000 7fc497fff640 1 -- 172.21.15.139:0/3578035306 learned_addr learned my addr 172.21.15.139:0/3578035306 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:36.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.606+0000 7fc49ffff640 1 --2- 172.21.15.139:0/3578035306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cded370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.606+0000 7fc4a4857640 1 --2- 172.21.15.139:0/3578035306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdd5e40 0x55a64cdd2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.607+0000 7fc497fff640 1 -- 172.21.15.139:0/3578035306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 msgr2=0x55a64cded370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:36.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.607+0000 7fc497fff640 1 --2- 172.21.15.139:0/3578035306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cded370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.607+0000 7fc497fff640 1 -- 172.21.15.139:0/3578035306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdd5e40 msgr2=0x55a64cdd2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:36.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.607+0000 7fc497fff640 1 --2- 172.21.15.139:0/3578035306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdd5e40 0x55a64cdd2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.607+0000 7fc497fff640 1 -- 172.21.15.139:0/3578035306 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a64cdd1330 con 0x55a64cdc7090 2024-09-16T01:42:36.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.607+0000 7fc49ffff640 1 --2- 172.21.15.139:0/3578035306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cded370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:36.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.607+0000 7fc497fff640 1 --2- 172.21.15.139:0/3578035306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc7090 0x55a64cded8b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc48809c130 tx=0x7fc4880d6b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a243c1fa13dbb3e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:36.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.607+0000 7fc495ffb640 1 -- 172.21.15.139:0/3578035306 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48809c200 con 0x55a64cdc7090 2024-09-16T01:42:36.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.607+0000 7fc495ffb640 1 -- 172.21.15.139:0/3578035306 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4880d6300 con 0x55a64cdc7090 2024-09-16T01:42:36.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.607+0000 7fc495ffb640 1 -- 172.21.15.139:0/3578035306 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48809bd90 con 0x55a64cdc7090 2024-09-16T01:42:36.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.608+0000 7fc4a8693740 1 -- 172.21.15.139:0/3578035306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc7090 msgr2=0x55a64cded8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:36.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.608+0000 7fc4a8693740 1 --2- 172.21.15.139:0/3578035306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc7090 0x55a64cded8b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc48809c130 tx=0x7fc4880d6b80 comp rx=0 tx=0).stop 2024-09-16T01:42:36.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.608+0000 7fc4a8693740 1 -- 172.21.15.139:0/3578035306 shutdown_connections 2024-09-16T01:42:36.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.608+0000 7fc4a8693740 1 --2- 172.21.15.139:0/3578035306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdd5e40 0x55a64cdd2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.608+0000 7fc4a8693740 1 --2- 172.21.15.139:0/3578035306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdc7090 0x55a64cded8b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.608+0000 7fc4a8693740 1 --2- 172.21.15.139:0/3578035306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cded370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.608+0000 7fc4a8693740 1 -- 172.21.15.139:0/3578035306 >> 172.21.15.139:0/3578035306 conn(0x55a64cdc2de0 msgr2=0x55a64cdd1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:36.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.608+0000 7fc4a8693740 1 -- 172.21.15.139:0/3578035306 shutdown_connections 2024-09-16T01:42:36.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.608+0000 7fc4a8693740 1 -- 172.21.15.139:0/3578035306 wait complete. 2024-09-16T01:42:36.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.609+0000 7fc4a8693740 1 -- start start 2024-09-16T01:42:36.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.609+0000 7fc4a8693740 1 Processor -- start 2024-09-16T01:42:36.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.609+0000 7fc4a8693740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cdd9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.609+0000 7fc4a8693740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdc7090 0x55a64cdd9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.609+0000 7fc4a8693740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdd5e40 0x55a64cdddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.609+0000 7fc4a8693740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01ae360 con 0x55a64cdd5e40 2024-09-16T01:42:36.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.609+0000 7fc4a8693740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01adc90 con 0x55a64cdc7090 2024-09-16T01:42:36.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.609+0000 7fc4a8693740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01aaed0 con 0x55a64cdc6770 2024-09-16T01:42:36.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc4a8693740 10 client.0 _ll_register_callbacks cb 0x7fc490009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:42:36.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc4a4857640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdd5e40 0x55a64cdddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc497fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdc7090 0x55a64cdd9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc49ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cdd9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc497fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdc7090 0x55a64cdd9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37668/0 (socket says 172.21.15.139:37668) 2024-09-16T01:42:36.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc4a4857640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdd5e40 0x55a64cdddb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33338/0 (socket says 172.21.15.139:33338) 2024-09-16T01:42:36.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc497fff640 1 -- 172.21.15.139:0/1988778271 learned_addr learned my addr 172.21.15.139:0/1988778271 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:36.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc49ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cdd9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:37610/0 (socket says 172.21.15.139:37610) 2024-09-16T01:42:36.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc4a4857640 1 -- 172.21.15.139:0/1988778271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 msgr2=0x55a64cdd9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:36.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc4a4857640 1 --2- 172.21.15.139:0/1988778271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cdd9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc4a4857640 1 -- 172.21.15.139:0/1988778271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdc7090 msgr2=0x55a64cdd9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:36.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc4a4857640 1 --2- 172.21.15.139:0/1988778271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdc7090 0x55a64cdd9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:36.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc4a4857640 1 -- 172.21.15.139:0/1988778271 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a64cdde1e0 con 0x55a64cdd5e40 2024-09-16T01:42:36.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc49ffff640 1 --2- 172.21.15.139:0/1988778271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cdd9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:36.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc497fff640 1 --2- 172.21.15.139:0/1988778271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdc7090 0x55a64cdd9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:36.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc4a4857640 1 --2- 172.21.15.139:0/1988778271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdd5e40 0x55a64cdddb70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc498009a60 tx=0x7fc498005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d58a68c1908302 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:36.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.610+0000 7fc47f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:42:36.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.611+0000 7fc47f7fe640 1 -- 172.21.15.139:0/1988778271 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498058060 con 0x55a64cdd5e40 2024-09-16T01:42:36.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.611+0000 7fc47f7fe640 1 -- 172.21.15.139:0/1988778271 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fc49803ad40 con 0x55a64cdd5e40 2024-09-16T01:42:36.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.611+0000 7fc4a8693740 1 -- 172.21.15.139:0/1988778271 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a64cd32be0 con 0x55a64cdd5e40 2024-09-16T01:42:36.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.611+0000 7fc47f7fe640 1 -- 172.21.15.139:0/1988778271 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc49805f050 con 0x55a64cdd5e40 2024-09-16T01:42:36.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.611+0000 7fc4a8693740 10 client.15783 fetch_fsmap learned FSMap version 31 2024-09-16T01:42:36.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.611+0000 7fc4a8693740 1 -- 172.21.15.139:0/1988778271 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a64ce4fe40 con 0x55a64cdd5e40 2024-09-16T01:42:36.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.611+0000 7fc47f7fe640 1 -- 172.21.15.139:0/1988778271 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a64cd32be0 con 0x55a64cdd5e40 2024-09-16T01:42:36.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.611+0000 7fc47f7fe640 1 -- 172.21.15.139:0/1988778271 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7fc498030440 con 0x55a64cdd5e40 2024-09-16T01:42:36.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.611+0000 7fc4a8693740 10 client.15783 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:42:36.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.611+0000 7fc4a8693740 10 client.15783 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:42:36.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.611+0000 7fc4a8693740 20 client.15783 populate_metadata read hostname 'smithi139' 2024-09-16T01:42:36.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.611+0000 7fc4a8693740 1 --2- 172.21.15.139:0/1988778271 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a64ce506f0 0x55a64ce52b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:36.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.611+0000 7fc4a8693740 4 client.15783 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}] 2024-09-16T01:42:36.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.611+0000 7fc4a8693740 1 -- 172.21.15.139:0/1988778271 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}) -- 0x55a64ccaef40 con 0x55a64ce506f0 2024-09-16T01:42:36.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.612+0000 7fc49ffff640 1 --2- 172.21.15.139:0/1988778271 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a64ce506f0 0x55a64ce52b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:36.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.612+0000 7fc47f7fe640 10 client.15783 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:36.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:36.612+0000 7fc49ffff640 1 --2- 172.21.15.139:0/1988778271 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a64ce506f0 0x55a64ce52b10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2a5c9f45a46f08e server_cookie=6b5c5c6c4bffb196 in_seq=0 out_seq=0 2024-09-16T01:42:37.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:37 smithi164 ceph-mon[35121]: pgmap v69: 129 pgs: 129 active+clean; 179 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.7 MiB/s wr, 839 op/s 2024-09-16T01:42:37.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:37 smithi139 ceph-mon[31175]: pgmap v69: 129 pgs: 129 active+clean; 179 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.7 MiB/s wr, 839 op/s 2024-09-16T01:42:37.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:37 smithi176 ceph-mon[35324]: pgmap v69: 129 pgs: 129 active+clean; 179 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.7 MiB/s wr, 839 op/s 2024-09-16T01:42:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:39 smithi164 ceph-mon[35121]: pgmap v70: 129 pgs: 129 active+clean; 179 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 536 op/s 2024-09-16T01:42:39.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:39 smithi139 ceph-mon[31175]: pgmap v70: 129 pgs: 129 active+clean; 179 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 536 op/s 2024-09-16T01:42:39.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:39 smithi176 ceph-mon[35324]: pgmap v70: 129 pgs: 129 active+clean; 179 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 536 op/s 2024-09-16T01:42:41.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:41 smithi164 ceph-mon[35121]: pgmap v71: 129 pgs: 129 active+clean; 190 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 716 op/s 2024-09-16T01:42:41.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:41 smithi139 ceph-mon[31175]: pgmap v71: 129 pgs: 129 active+clean; 190 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 716 op/s 2024-09-16T01:42:41.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:41 smithi176 ceph-mon[35324]: pgmap v71: 129 pgs: 129 active+clean; 190 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 716 op/s 2024-09-16T01:42:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.072+0000 7fc7a2c23640 1 -- 172.21.15.139:0/3367639034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc79c1009f0 msgr2=0x7fc79c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.072+0000 7fc7a2c23640 1 --2- 172.21.15.139:0/3367639034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc79c1009f0 0x7fc79c100e50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc788009930 tx=0x7fc78802a420 comp rx=0 tx=0).stop 2024-09-16T01:42:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.072+0000 7fc7a2c23640 1 -- 172.21.15.139:0/3367639034 shutdown_connections 2024-09-16T01:42:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.072+0000 7fc7a2c23640 1 --2- 172.21.15.139:0/3367639034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc79c101390 0x7fc79c10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.072+0000 7fc7a2c23640 1 --2- 172.21.15.139:0/3367639034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc79c1009f0 0x7fc79c100e50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.072+0000 7fc7a2c23640 1 --2- 172.21.15.139:0/3367639034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc79c106a00 0x7fc79c106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.072+0000 7fc7a2c23640 1 -- 172.21.15.139:0/3367639034 >> 172.21.15.139:0/3367639034 conn(0x7fc79c0fc6b0 msgr2=0x7fc79c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.072+0000 7fc7a2c23640 1 -- 172.21.15.139:0/3367639034 shutdown_connections 2024-09-16T01:42:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.072+0000 7fc7a2c23640 1 -- 172.21.15.139:0/3367639034 wait complete. 2024-09-16T01:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.073+0000 7fc7a2c23640 1 Processor -- start 2024-09-16T01:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.073+0000 7fc7a2c23640 1 -- start start 2024-09-16T01:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.073+0000 7fc7a2c23640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc79c1009f0 0x7fc79c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.073+0000 7fc7a2c23640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc79c101390 0x7fc79c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.073+0000 7fc7a2c23640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc79c106a00 0x7fc79c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.073+0000 7fc7a2c23640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c111f30 con 0x7fc79c106a00 2024-09-16T01:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.073+0000 7fc7a2c23640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c111db0 con 0x7fc79c101390 2024-09-16T01:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc7a2c23640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1120b0 con 0x7fc79c1009f0 2024-09-16T01:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc7a11c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc79c106a00 0x7fc79c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc79bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc79c101390 0x7fc79c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc7a11c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc79c106a00 0x7fc79c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33346/0 (socket says 172.21.15.139:33346) 2024-09-16T01:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc79bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc79c101390 0x7fc79c198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37694/0 (socket says 172.21.15.139:37694) 2024-09-16T01:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc7a11c0640 1 -- 172.21.15.139:0/118808440 learned_addr learned my addr 172.21.15.139:0/118808440 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc7a09bf640 1 --2- 172.21.15.139:0/118808440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc79c1009f0 0x7fc79c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc7a11c0640 1 -- 172.21.15.139:0/118808440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc79c1009f0 msgr2=0x7fc79c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc7a11c0640 1 --2- 172.21.15.139:0/118808440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc79c1009f0 0x7fc79c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc7a11c0640 1 -- 172.21.15.139:0/118808440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc79c101390 msgr2=0x7fc79c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc7a11c0640 1 --2- 172.21.15.139:0/118808440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc79c101390 0x7fc79c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc7a11c0640 1 -- 172.21.15.139:0/118808440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79c19d3f0 con 0x7fc79c106a00 2024-09-16T01:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc7a11c0640 1 --2- 172.21.15.139:0/118808440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc79c106a00 0x7fc79c19ccc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc79400eb10 tx=0x7fc79400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc799ffb640 1 -- 172.21.15.139:0/118808440 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794004b20 con 0x7fc79c106a00 2024-09-16T01:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc799ffb640 1 -- 172.21.15.139:0/118808440 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc794004cc0 con 0x7fc79c106a00 2024-09-16T01:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc799ffb640 1 -- 172.21.15.139:0/118808440 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794010760 con 0x7fc79c106a00 2024-09-16T01:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc7a2c23640 1 -- 172.21.15.139:0/118808440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79c19d6e0 con 0x7fc79c106a00 2024-09-16T01:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.074+0000 7fc7a2c23640 1 -- 172.21.15.139:0/118808440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79c1a4f70 con 0x7fc79c106a00 2024-09-16T01:42:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.076+0000 7fc799ffb640 1 -- 172.21.15.139:0/118808440 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc794005020 con 0x7fc79c106a00 2024-09-16T01:42:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.077+0000 7fc7a59ba740 1 Processor -- start 2024-09-16T01:42:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.077+0000 7fc7a59ba740 1 -- start start 2024-09-16T01:42:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.077+0000 7fc799ffb640 1 --2- 172.21.15.139:0/118808440 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc78c0a3460 0x7fc78c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.077+0000 7fc7a59ba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d2f9d0 0x557120d2fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.077+0000 7fc7a59ba740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d3eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.077+0000 7fc7a59ba740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d3f0c0 0x557120d414b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.077+0000 7fc7a59ba740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557120c942f0 con 0x557120d302f0 2024-09-16T01:42:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.077+0000 7fc7a59ba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557120c3c270 con 0x557120d2f9d0 2024-09-16T01:42:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.077+0000 7fc7a59ba740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557120c902b0 con 0x557120d3f0c0 2024-09-16T01:42:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.077+0000 7fc79bfff640 1 --2- 172.21.15.139:0/118808440 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc78c0a3460 0x7fc78c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.077+0000 7fc7a09bf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d2f9d0 0x557120d2fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.077+0000 7fc799ffb640 1 -- 172.21.15.139:0/118808440 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fc7940d06e0 con 0x7fc79c106a00 2024-09-16T01:42:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.077+0000 7fc7a09bf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d2f9d0 0x557120d2fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37708/0 (socket says 172.21.15.139:37708) 2024-09-16T01:42:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.077+0000 7fc7a09bf640 1 -- 172.21.15.139:0/2079914597 learned_addr learned my addr 172.21.15.139:0/2079914597 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.078+0000 7fc7a11c0640 1 --2- 172.21.15.139:0/2079914597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d3f0c0 0x557120d414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.078+0000 7fc79bfff640 1 --2- 172.21.15.139:0/2079914597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d3eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.078+0000 7fc7a09bf640 1 -- 172.21.15.139:0/2079914597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d3f0c0 msgr2=0x557120d414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.078+0000 7fc7a09bf640 1 --2- 172.21.15.139:0/2079914597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d3f0c0 0x557120d414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.078+0000 7fc7a09bf640 1 -- 172.21.15.139:0/2079914597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 msgr2=0x557120d3eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.078+0000 7fc7a09bf640 1 --2- 172.21.15.139:0/2079914597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d3eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.078+0000 7fc7a09bf640 1 -- 172.21.15.139:0/2079914597 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557120c3b340 con 0x557120d2f9d0 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.078+0000 7fc7a09bf640 1 --2- 172.21.15.139:0/2079914597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d2f9d0 0x557120d2fdb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc79000b740 tx=0x7fc790004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace8995501b5901f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.078+0000 7fc79bfff640 1 --2- 172.21.15.139:0/118808440 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc78c0a3460 0x7fc78c0a5920 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc788009900 tx=0x7fc788005fd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.078+0000 7fc786ffd640 1 -- 172.21.15.139:0/2079914597 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790003bd0 con 0x557120d2f9d0 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.078+0000 7fc786ffd640 1 -- 172.21.15.139:0/2079914597 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc790015070 con 0x557120d2f9d0 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.078+0000 7fc786ffd640 1 -- 172.21.15.139:0/2079914597 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790004d80 con 0x557120d2f9d0 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.079+0000 7fc7a59ba740 1 -- 172.21.15.139:0/2079914597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d2f9d0 msgr2=0x557120d2fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.079+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/2079914597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d2f9d0 0x557120d2fdb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc79000b740 tx=0x7fc790004610 comp rx=0 tx=0).stop 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.079+0000 7fc7a59ba740 1 -- 172.21.15.139:0/2079914597 shutdown_connections 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.079+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/2079914597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d3f0c0 0x557120d414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.079+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/2079914597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d3eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.079+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/2079914597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d2f9d0 0x557120d2fdb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.079+0000 7fc7a59ba740 1 -- 172.21.15.139:0/2079914597 >> 172.21.15.139:0/2079914597 conn(0x557120d2c060 msgr2=0x557120d2c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.079+0000 7fc7a59ba740 1 -- 172.21.15.139:0/2079914597 shutdown_connections 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.079+0000 7fc7a59ba740 1 -- 172.21.15.139:0/2079914597 wait complete. 2024-09-16T01:42:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.079+0000 7fc7a59ba740 1 -- start start 2024-09-16T01:42:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.079+0000 7fc7a59ba740 1 Processor -- start 2024-09-16T01:42:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a59ba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d2f9d0 0x557120d3c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a59ba740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d302f0 0x557120d3c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a59ba740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d3f0c0 0x557120d3caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a59ba740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557120d41cf0 con 0x557120d3f0c0 2024-09-16T01:42:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a59ba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557120c1d290 con 0x557120d2f9d0 2024-09-16T01:42:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a59ba740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557120d41e70 con 0x557120d302f0 2024-09-16T01:42:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a59ba740 10 client.0 _ll_register_callbacks cb 0x557120c4e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:42:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a11c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d3f0c0 0x557120d3caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a09bf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d2f9d0 0x557120d3c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a09bf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d2f9d0 0x557120d3c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37718/0 (socket says 172.21.15.139:37718) 2024-09-16T01:42:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a09bf640 1 -- 172.21.15.139:0/1488479011 learned_addr learned my addr 172.21.15.139:0/1488479011 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc79bfff640 1 --2- 172.21.15.139:0/1488479011 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d302f0 0x557120d3c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a11c0640 1 -- 172.21.15.139:0/1488479011 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d302f0 msgr2=0x557120d3c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a11c0640 1 --2- 172.21.15.139:0/1488479011 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d302f0 0x557120d3c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a11c0640 1 -- 172.21.15.139:0/1488479011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d2f9d0 msgr2=0x557120d3c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a11c0640 1 --2- 172.21.15.139:0/1488479011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d2f9d0 0x557120d3c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a11c0640 1 -- 172.21.15.139:0/1488479011 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571209a7210 con 0x557120d3f0c0 2024-09-16T01:42:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a11c0640 1 --2- 172.21.15.139:0/1488479011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d3f0c0 0x557120d3caa0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc7940cda20 tx=0x7fc79409bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26795d1439b6ce8b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc77bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:42:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc77bfff640 1 -- 172.21.15.139:0/1488479011 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7940cf030 con 0x557120d3f0c0 2024-09-16T01:42:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc77bfff640 1 -- 172.21.15.139:0/1488479011 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fc7940cdca0 con 0x557120d3f0c0 2024-09-16T01:42:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.080+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1488479011 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc790015070 con 0x557120d3f0c0 2024-09-16T01:42:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.081+0000 7fc77bfff640 1 -- 172.21.15.139:0/1488479011 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc7940cdea0 con 0x557120d3f0c0 2024-09-16T01:42:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.081+0000 7fc7a59ba740 10 client.15795 fetch_fsmap learned FSMap version 31 2024-09-16T01:42:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.081+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1488479011 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557120a73620 con 0x557120d3f0c0 2024-09-16T01:42:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.081+0000 7fc77bfff640 1 -- 172.21.15.139:0/1488479011 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc790015070 con 0x557120d3f0c0 2024-09-16T01:42:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.081+0000 7fc77bfff640 1 -- 172.21.15.139:0/1488479011 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7fc7940b0ad0 con 0x557120d3f0c0 2024-09-16T01:42:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.081+0000 7fc7a59ba740 10 client.15795 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:42:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.081+0000 7fc7a59ba740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:42:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.081+0000 7fc7a59ba740 20 client.15795 populate_metadata read hostname 'smithi139' 2024-09-16T01:42:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.081+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/1488479011 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557120d4f0e0 0x557120d51550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.081+0000 7fc7a59ba740 4 client.15795 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:42:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.081+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1488479011 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557120ca1a20 con 0x557120d4f0e0 2024-09-16T01:42:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.081+0000 7fc7a09bf640 1 --2- 172.21.15.139:0/1488479011 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557120d4f0e0 0x557120d51550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.082+0000 7fc7a09bf640 1 --2- 172.21.15.139:0/1488479011 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557120d4f0e0 0x557120d51550 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e23fbb7e036166a2 server_cookie=f5331fe00a1becb in_seq=0 out_seq=0 2024-09-16T01:42:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.082+0000 7fc77bfff640 10 client.15795 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc77bfff640 1 -- 172.21.15.139:0/1488479011 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557120ca1a20 con 0x557120d4f0e0 2024-09-16T01:42:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc77bfff640 10 client.15795 handle_command_reply: tid=0 2024-09-16T01:42:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc7a59ba740 1 client.15795 shutdown 2024-09-16T01:42:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1488479011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d3f0c0 msgr2=0x557120d3caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/1488479011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d3f0c0 0x557120d3caa0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc7940cda20 tx=0x7fc79409bf10 comp rx=0 tx=0).stop 2024-09-16T01:42:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1488479011 shutdown_connections 2024-09-16T01:42:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/1488479011 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557120d4f0e0 0x557120d51550 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/1488479011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d3f0c0 0x557120d3caa0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/1488479011 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d302f0 0x557120d3c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/1488479011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d2f9d0 0x557120d3c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc77bfff640 0 client.15795 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1488479011 >> 172.21.15.139:0/1488479011 conn(0x557120d2c060 msgr2=0x557120d2b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1488479011 shutdown_connections 2024-09-16T01:42:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1488479011 wait complete. 2024-09-16T01:42:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.156+0000 7fc7a59ba740 20 client.15795 trim_cache size 0 max 16384 2024-09-16T01:42:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.170+0000 7fc7a59ba740 1 Processor -- start 2024-09-16T01:42:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.170+0000 7fc7a59ba740 1 -- start start 2024-09-16T01:42:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.170+0000 7fc7a59ba740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d2f9d0 0x557120d3df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.170+0000 7fc7a59ba740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d56e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.170+0000 7fc7a59ba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d3f0c0 0x557120d573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.170+0000 7fc7a59ba740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1ace30 con 0x557120d302f0 2024-09-16T01:42:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.170+0000 7fc7a59ba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1aea30 con 0x557120d3f0c0 2024-09-16T01:42:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.170+0000 7fc7a59ba740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1ad500 con 0x557120d2f9d0 2024-09-16T01:42:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.170+0000 7fc79bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d56e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.170+0000 7fc79bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d56e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33386/0 (socket says 172.21.15.139:33386) 2024-09-16T01:42:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.170+0000 7fc79bfff640 1 -- 172.21.15.139:0/2905802172 learned_addr learned my addr 172.21.15.139:0/2905802172 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.170+0000 7fc7a11c0640 1 --2- 172.21.15.139:0/2905802172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d3f0c0 0x557120d573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.170+0000 7fc7a09bf640 1 --2- 172.21.15.139:0/2905802172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d2f9d0 0x557120d3df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.171+0000 7fc79bfff640 1 -- 172.21.15.139:0/2905802172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d2f9d0 msgr2=0x557120d3df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.171+0000 7fc79bfff640 1 --2- 172.21.15.139:0/2905802172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d2f9d0 0x557120d3df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.171+0000 7fc79bfff640 1 -- 172.21.15.139:0/2905802172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d3f0c0 msgr2=0x557120d573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.171+0000 7fc79bfff640 1 --2- 172.21.15.139:0/2905802172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d3f0c0 0x557120d573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.171+0000 7fc79bfff640 1 -- 172.21.15.139:0/2905802172 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557120d30120 con 0x557120d302f0 2024-09-16T01:42:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.171+0000 7fc7a09bf640 1 --2- 172.21.15.139:0/2905802172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d2f9d0 0x557120d3df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:42:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.171+0000 7fc79bfff640 1 --2- 172.21.15.139:0/2905802172 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d56e60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc788009930 tx=0x7fc78802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63075dcee5237703 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.171+0000 7fc7867fc640 1 -- 172.21.15.139:0/2905802172 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788050070 con 0x557120d302f0 2024-09-16T01:42:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.171+0000 7fc7867fc640 1 -- 172.21.15.139:0/2905802172 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc788003cd0 con 0x557120d302f0 2024-09-16T01:42:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.171+0000 7fc7867fc640 1 -- 172.21.15.139:0/2905802172 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78804cc60 con 0x557120d302f0 2024-09-16T01:42:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 -- 172.21.15.139:0/2905802172 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 msgr2=0x557120d56e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/2905802172 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d56e60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc788009930 tx=0x7fc78802a420 comp rx=0 tx=0).stop 2024-09-16T01:42:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 -- 172.21.15.139:0/2905802172 shutdown_connections 2024-09-16T01:42:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/2905802172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d3f0c0 0x557120d573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/2905802172 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d56e60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/2905802172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d2f9d0 0x557120d3df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 -- 172.21.15.139:0/2905802172 >> 172.21.15.139:0/2905802172 conn(0x557120d2c060 msgr2=0x557120d3d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 -- 172.21.15.139:0/2905802172 shutdown_connections 2024-09-16T01:42:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 -- 172.21.15.139:0/2905802172 wait complete. 2024-09-16T01:42:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 -- start start 2024-09-16T01:42:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 Processor -- start 2024-09-16T01:42:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d2f9d0 0x557120d42d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d3f0c0 0x557120d437e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1ae2a0 con 0x557120d302f0 2024-09-16T01:42:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1adbd0 con 0x557120d3f0c0 2024-09-16T01:42:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.172+0000 7fc7a59ba740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1aae10 con 0x557120d2f9d0 2024-09-16T01:42:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc7a59ba740 10 client.0 _ll_register_callbacks cb 0x7fc794004e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:42:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc79bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc79bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d432a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33400/0 (socket says 172.21.15.139:33400) 2024-09-16T01:42:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc79bfff640 1 -- 172.21.15.139:0/1783727523 learned_addr learned my addr 172.21.15.139:0/1783727523 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc7a11c0640 1 --2- 172.21.15.139:0/1783727523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d3f0c0 0x557120d437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc7a09bf640 1 --2- 172.21.15.139:0/1783727523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d2f9d0 0x557120d42d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc79bfff640 1 -- 172.21.15.139:0/1783727523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d2f9d0 msgr2=0x557120d42d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc79bfff640 1 --2- 172.21.15.139:0/1783727523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d2f9d0 0x557120d42d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc79bfff640 1 -- 172.21.15.139:0/1783727523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d3f0c0 msgr2=0x557120d437e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc79bfff640 1 --2- 172.21.15.139:0/1783727523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d3f0c0 0x557120d437e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc79bfff640 1 -- 172.21.15.139:0/1783727523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557120db8e30 con 0x557120d302f0 2024-09-16T01:42:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc7a09bf640 1 --2- 172.21.15.139:0/1783727523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d2f9d0 0x557120d42d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:42:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc79bfff640 1 --2- 172.21.15.139:0/1783727523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d432a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc78804c940 tx=0x7fc78804cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ca944a7473b75d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc77bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:42:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc77bfff640 1 -- 172.21.15.139:0/1783727523 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788050040 con 0x557120d302f0 2024-09-16T01:42:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1783727523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc788003cd0 con 0x557120d302f0 2024-09-16T01:42:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc77bfff640 1 -- 172.21.15.139:0/1783727523 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fc788003800 con 0x557120d302f0 2024-09-16T01:42:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc77bfff640 1 -- 172.21.15.139:0/1783727523 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc788066050 con 0x557120d302f0 2024-09-16T01:42:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc7a59ba740 10 client.15801 fetch_fsmap learned FSMap version 31 2024-09-16T01:42:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1783727523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557120dba080 con 0x557120d302f0 2024-09-16T01:42:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.173+0000 7fc77bfff640 1 -- 172.21.15.139:0/1783727523 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc788003cd0 con 0x557120d302f0 2024-09-16T01:42:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.174+0000 7fc77bfff640 1 -- 172.21.15.139:0/1783727523 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7fc78803d930 con 0x557120d302f0 2024-09-16T01:42:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.174+0000 7fc7a59ba740 10 client.15801 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:42:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.174+0000 7fc7a59ba740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:42:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.174+0000 7fc7a59ba740 20 client.15801 populate_metadata read hostname 'smithi139' 2024-09-16T01:42:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.174+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/1783727523 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557120dba8a0 0x557120dbccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.174+0000 7fc7a59ba740 4 client.15801 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:42:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.174+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1783727523 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557120c9bd70 con 0x557120dba8a0 2024-09-16T01:42:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.174+0000 7fc7a09bf640 1 --2- 172.21.15.139:0/1783727523 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557120dba8a0 0x557120dbccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.174+0000 7fc77bfff640 10 client.15801 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:42.174+0000 7fc7a09bf640 1 --2- 172.21.15.139:0/1783727523 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557120dba8a0 0x557120dbccc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ff741bdc41d9d5d server_cookie=f4dcf7165a53bd4f in_seq=0 out_seq=0 2024-09-16T01:42:43.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.300+0000 7fc47f7fe640 1 -- 172.21.15.139:0/1988778271 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a64ccaef40 con 0x55a64ce506f0 2024-09-16T01:42:43.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.300+0000 7fc47f7fe640 10 client.15783 handle_command_reply: tid=0 2024-09-16T01:42:43.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.300+0000 7fc4a8693740 1 client.15783 shutdown 2024-09-16T01:42:43.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.300+0000 7fc4a8693740 1 -- 172.21.15.139:0/1988778271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdd5e40 msgr2=0x55a64cdddb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:43.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.300+0000 7fc4a8693740 1 --2- 172.21.15.139:0/1988778271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdd5e40 0x55a64cdddb70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc498009a60 tx=0x7fc498005d80 comp rx=0 tx=0).stop 2024-09-16T01:42:43.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.300+0000 7fc4a8693740 1 -- 172.21.15.139:0/1988778271 shutdown_connections 2024-09-16T01:42:43.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.300+0000 7fc4a8693740 1 --2- 172.21.15.139:0/1988778271 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a64ce506f0 0x55a64ce52b10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.300+0000 7fc4a8693740 1 --2- 172.21.15.139:0/1988778271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a64cdd5e40 0x55a64cdddb70 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.300+0000 7fc4a8693740 1 --2- 172.21.15.139:0/1988778271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a64cdc7090 0x55a64cdd9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.300+0000 7fc4a8693740 1 --2- 172.21.15.139:0/1988778271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a64cdc6770 0x55a64cdd9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.300+0000 7fc4a8693740 1 -- 172.21.15.139:0/1988778271 >> 172.21.15.139:0/1988778271 conn(0x55a64cdc2de0 msgr2=0x55a64cdd4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:43.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.300+0000 7fc47f7fe640 0 client.15783 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:43.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.301+0000 7fc4a8693740 1 -- 172.21.15.139:0/1988778271 shutdown_connections 2024-09-16T01:42:43.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.301+0000 7fc4a8693740 1 -- 172.21.15.139:0/1988778271 wait complete. 2024-09-16T01:42:43.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.301+0000 7fc4a8693740 20 client.15783 trim_cache size 0 max 16384 2024-09-16T01:42:43.304 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:42:43.304 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 31, 2024-09-16T01:42:43.304 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:42:43.304 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 6, 2024-09-16T01:42:43.304 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:42:43.304 INFO:teuthology.orchestra.run.smithi139.stdout: "f292b9e9": { 2024-09-16T01:42:43.304 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 6, 2024-09-16T01:42:43.305 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:42:43.305 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:42:43.305 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:42:43.305 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:42:43.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:42:43.305 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:42:43.305 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 139.6, 2024-09-16T01:42:43.305 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:42:43.305 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:42:43.305 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:42:43.305 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:42:43.305 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:42:43.305 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:42:43.306 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:42:43.306 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:42:43.306 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:42:43.306 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:42:43.306 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:42:43.306 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:42:43.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.302+0000 7fc4a62ba640 1 -- 172.21.15.139:0/3785330006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc48c0a33b0 msgr2=0x7fc48c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:43.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.302+0000 7fc4a62ba640 1 --2- 172.21.15.139:0/3785330006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc48c0a33b0 0x7fc48c0a5870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc498005fe0 tx=0x7fc49802a950 comp rx=0 tx=0).stop 2024-09-16T01:42:43.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.302+0000 7fc4a62ba640 1 -- 172.21.15.139:0/3785330006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a0102570 msgr2=0x7fc4a01989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:43.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.302+0000 7fc4a62ba640 1 --2- 172.21.15.139:0/3785330006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a0102570 0x7fc4a01989b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc48800e9f0 tx=0x7fc48800eeb0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.302+0000 7fc4a62ba640 1 -- 172.21.15.139:0/3785330006 shutdown_connections 2024-09-16T01:42:43.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.302+0000 7fc4a62ba640 1 --2- 172.21.15.139:0/3785330006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc48c0a33b0 0x7fc48c0a5870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.302+0000 7fc4a62ba640 1 --2- 172.21.15.139:0/3785330006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a0102ef0 0x7fc4a019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.302+0000 7fc4a62ba640 1 --2- 172.21.15.139:0/3785330006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a0102570 0x7fc4a01989b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.302+0000 7fc4a62ba640 1 --2- 172.21.15.139:0/3785330006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4a0100e20 0x7fc4a0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.302+0000 7fc4a62ba640 1 -- 172.21.15.139:0/3785330006 >> 172.21.15.139:0/3785330006 conn(0x7fc4a00fc7d0 msgr2=0x7fc4a0103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:43.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.302+0000 7fc4a62ba640 1 -- 172.21.15.139:0/3785330006 shutdown_connections 2024-09-16T01:42:43.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:43.302+0000 7fc4a62ba640 1 -- 172.21.15.139:0/3785330006 wait complete. 2024-09-16T01:42:43.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:43 smithi139 ceph-mon[31175]: pgmap v72: 129 pgs: 129 active+clean; 205 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 748 op/s 2024-09-16T01:42:43.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:43 smithi164 ceph-mon[35121]: pgmap v72: 129 pgs: 129 active+clean; 205 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 748 op/s 2024-09-16T01:42:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc77bfff640 1 -- 172.21.15.139:0/1783727523 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557120c9bd70 con 0x557120dba8a0 2024-09-16T01:42:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc77bfff640 10 client.15801 handle_command_reply: tid=0 2024-09-16T01:42:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc7a59ba740 1 client.15801 shutdown 2024-09-16T01:42:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1783727523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 msgr2=0x557120d432a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/1783727523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d432a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc78804c940 tx=0x7fc78804cc10 comp rx=0 tx=0).stop 2024-09-16T01:42:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1783727523 shutdown_connections 2024-09-16T01:42:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/1783727523 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557120dba8a0 0x557120dbccc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/1783727523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557120d3f0c0 0x557120d437e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/1783727523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557120d302f0 0x557120d432a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc7a59ba740 1 --2- 172.21.15.139:0/1783727523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557120d2f9d0 0x557120d42d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc77bfff640 0 client.15801 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1783727523 >> 172.21.15.139:0/1783727523 conn(0x557120d2c060 msgr2=0x557120d3d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1783727523 shutdown_connections 2024-09-16T01:42:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc7a59ba740 1 -- 172.21.15.139:0/1783727523 wait complete. 2024-09-16T01:42:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.864+0000 7fc7a59ba740 20 client.15801 trim_cache size 0 max 16384 2024-09-16T01:42:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:42:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:42:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:42:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:42:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.865+0000 7fc7a2c23640 1 -- 172.21.15.139:0/118808440 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc78c0a3460 msgr2=0x7fc78c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.865+0000 7fc7a2c23640 1 --2- 172.21.15.139:0/118808440 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc78c0a3460 0x7fc78c0a5920 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc788009900 tx=0x7fc788005fd0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.865+0000 7fc7a2c23640 1 -- 172.21.15.139:0/118808440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc79c106a00 msgr2=0x7fc79c19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.865+0000 7fc7a2c23640 1 --2- 172.21.15.139:0/118808440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc79c106a00 0x7fc79c19ccc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc79400eb10 tx=0x7fc79400c2d0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.865+0000 7fc7a2c23640 1 -- 172.21.15.139:0/118808440 shutdown_connections 2024-09-16T01:42:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.865+0000 7fc7a2c23640 1 --2- 172.21.15.139:0/118808440 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc78c0a3460 0x7fc78c0a5920 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.865+0000 7fc7a2c23640 1 --2- 172.21.15.139:0/118808440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc79c106a00 0x7fc79c19ccc0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.865+0000 7fc7a2c23640 1 --2- 172.21.15.139:0/118808440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc79c101390 0x7fc79c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.865+0000 7fc7a2c23640 1 --2- 172.21.15.139:0/118808440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc79c1009f0 0x7fc79c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.865+0000 7fc7a2c23640 1 -- 172.21.15.139:0/118808440 >> 172.21.15.139:0/118808440 conn(0x7fc79c0fc6b0 msgr2=0x7fc79c0fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.866+0000 7fc7a2c23640 1 -- 172.21.15.139:0/118808440 shutdown_connections 2024-09-16T01:42:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:43.866+0000 7fc7a2c23640 1 -- 172.21.15.139:0/118808440 wait complete. 2024-09-16T01:42:43.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:43 smithi176 ceph-mon[35324]: pgmap v72: 129 pgs: 129 active+clean; 205 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 748 op/s 2024-09-16T01:42:44.918 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f292b9e9', quiesce took 12.1 seconds. Will release after: 7.450000000000001 2024-09-16T01:42:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:45 smithi164 ceph-mon[35121]: pgmap v73: 129 pgs: 129 active+clean; 205 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 442 op/s 2024-09-16T01:42:45.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:45 smithi139 ceph-mon[31175]: pgmap v73: 129 pgs: 129 active+clean; 205 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 442 op/s 2024-09-16T01:42:45.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:45 smithi176 ceph-mon[35324]: pgmap v73: 129 pgs: 129 active+clean; 205 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 442 op/s 2024-09-16T01:42:45.966 INFO:tasks.workunit.client.0.smithi139.stdout: 92 45498 188 34515 150 8924 17 2024-09-16T01:42:47.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:47 smithi164 ceph-mon[35121]: pgmap v74: 129 pgs: 129 active+clean; 202 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 965 op/s 2024-09-16T01:42:47.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:47 smithi139 ceph-mon[31175]: pgmap v74: 129 pgs: 129 active+clean; 202 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 965 op/s 2024-09-16T01:42:47.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:47 smithi176 ceph-mon[35324]: pgmap v74: 129 pgs: 129 active+clean; 202 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 965 op/s 2024-09-16T01:42:49.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:49 smithi164 ceph-mon[35121]: pgmap v75: 129 pgs: 129 active+clean; 202 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 906 op/s 2024-09-16T01:42:49.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:49 smithi139 ceph-mon[31175]: pgmap v75: 129 pgs: 129 active+clean; 202 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 906 op/s 2024-09-16T01:42:49.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:49 smithi176 ceph-mon[35324]: pgmap v75: 129 pgs: 129 active+clean; 202 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 906 op/s 2024-09-16T01:42:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:51 smithi164 ceph-mon[35121]: pgmap v76: 129 pgs: 129 active+clean; 222 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 1.02k op/s 2024-09-16T01:42:51.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:51 smithi139 ceph-mon[31175]: pgmap v76: 129 pgs: 129 active+clean; 222 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 1.02k op/s 2024-09-16T01:42:51.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:51 smithi176 ceph-mon[35324]: pgmap v76: 129 pgs: 129 active+clean; 222 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 1.02k op/s 2024-09-16T01:42:52.432 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:42:52.876 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:42:53.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.429+0000 7f992a560640 1 -- 172.21.15.139:0/3792883604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9924101de0 msgr2=0x7f99241021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:53.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.429+0000 7f992a560640 1 --2- 172.21.15.139:0/3792883604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9924101de0 0x7f99241021c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9914009900 tx=0x7f991402a410 comp rx=0 tx=0).stop 2024-09-16T01:42:53.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.429+0000 7f992a560640 1 -- 172.21.15.139:0/3792883604 shutdown_connections 2024-09-16T01:42:53.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.429+0000 7f992a560640 1 --2- 172.21.15.139:0/3792883604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9924106f10 0x7f992410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:53.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.429+0000 7f992a560640 1 --2- 172.21.15.139:0/3792883604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9924102790 0x7f99241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:53.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.429+0000 7f992a560640 1 --2- 172.21.15.139:0/3792883604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9924101de0 0x7f99241021c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:53.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.429+0000 7f992a560640 1 -- 172.21.15.139:0/3792883604 >> 172.21.15.139:0/3792883604 conn(0x7f99240fd8c0 msgr2=0x7f99240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:53.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.429+0000 7f992a560640 1 -- 172.21.15.139:0/3792883604 shutdown_connections 2024-09-16T01:42:53.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.429+0000 7f992a560640 1 -- 172.21.15.139:0/3792883604 wait complete. 2024-09-16T01:42:53.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.430+0000 7f992a560640 1 Processor -- start 2024-09-16T01:42:53.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.430+0000 7f992a560640 1 -- start start 2024-09-16T01:42:53.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.430+0000 7f992a560640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9924101de0 0x7f99241984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:53.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.430+0000 7f992a560640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9924102790 0x7f9924198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:53.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.430+0000 7f992a560640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9924106f10 0x7f992419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:53.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.430+0000 7f992a560640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9924111f70 con 0x7f9924106f10 2024-09-16T01:42:53.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.430+0000 7f992a560640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9924111df0 con 0x7f9924102790 2024-09-16T01:42:53.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.430+0000 7f992a560640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f99241120f0 con 0x7f9924101de0 2024-09-16T01:42:53.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.430+0000 7f9928afd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9924106f10 0x7f992419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:53.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.430+0000 7f9923fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9924101de0 0x7f99241984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:53.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.430+0000 7f9928afd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9924106f10 0x7f992419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58688/0 (socket says 172.21.15.139:58688) 2024-09-16T01:42:53.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.430+0000 7f9928afd640 1 -- 172.21.15.139:0/2919378536 learned_addr learned my addr 172.21.15.139:0/2919378536 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:53.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.430+0000 7f991bfff640 1 --2- 172.21.15.139:0/2919378536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9924102790 0x7f9924198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:53.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.431+0000 7f9928afd640 1 -- 172.21.15.139:0/2919378536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9924101de0 msgr2=0x7f99241984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:53.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.431+0000 7f9928afd640 1 --2- 172.21.15.139:0/2919378536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9924101de0 0x7f99241984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:53.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.431+0000 7f9928afd640 1 -- 172.21.15.139:0/2919378536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9924102790 msgr2=0x7f9924198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:53.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.431+0000 7f9928afd640 1 --2- 172.21.15.139:0/2919378536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9924102790 0x7f9924198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:53.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.431+0000 7f9928afd640 1 -- 172.21.15.139:0/2919378536 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992419d480 con 0x7f9924106f10 2024-09-16T01:42:53.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.431+0000 7f9923fff640 1 --2- 172.21.15.139:0/2919378536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9924101de0 0x7f99241984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:53.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.431+0000 7f9928afd640 1 --2- 172.21.15.139:0/2919378536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9924106f10 0x7f992419cda0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f991c00eaa0 tx=0x7f991c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:53.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.431+0000 7f9921ffb640 1 -- 172.21.15.139:0/2919378536 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c00cdb0 con 0x7f9924106f10 2024-09-16T01:42:53.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.431+0000 7f9921ffb640 1 -- 172.21.15.139:0/2919378536 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f991c005020 con 0x7f9924106f10 2024-09-16T01:42:53.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.431+0000 7f992a560640 1 -- 172.21.15.139:0/2919378536 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992419d770 con 0x7f9924106f10 2024-09-16T01:42:53.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.431+0000 7f9921ffb640 1 -- 172.21.15.139:0/2919378536 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c0116b0 con 0x7f9924106f10 2024-09-16T01:42:53.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.431+0000 7f992a560640 1 -- 172.21.15.139:0/2919378536 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99241a5050 con 0x7f9924106f10 2024-09-16T01:42:53.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.432+0000 7f992a560640 1 -- 172.21.15.139:0/2919378536 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99241021c0 con 0x7f9924106f10 2024-09-16T01:42:53.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.433+0000 7f9921ffb640 1 -- 172.21.15.139:0/2919378536 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f991c005560 con 0x7f9924106f10 2024-09-16T01:42:53.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.434+0000 7f9921ffb640 1 --2- 172.21.15.139:0/2919378536 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f99100a3430 0x7f99100a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:53.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.434+0000 7f9923fff640 1 --2- 172.21.15.139:0/2919378536 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f99100a3430 0x7f99100a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:53.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.434+0000 7f9921ffb640 1 -- 172.21.15.139:0/2919378536 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f991c0d01a0 con 0x7f9924106f10 2024-09-16T01:42:53.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.434+0000 7f9923fff640 1 --2- 172.21.15.139:0/2919378536 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f99100a3430 0x7f99100a58f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9914009230 tx=0x7f99140058d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:53.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.438+0000 7f9921ffb640 1 -- 172.21.15.139:0/2919378536 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f991c016030 con 0x7f9924106f10 2024-09-16T01:42:53.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.642+0000 7f992a560640 1 -- 172.21.15.139:0/2919378536 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9924109b60 con 0x7f9924106f10 2024-09-16T01:42:53.645 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:53 smithi139 ceph-mon[31175]: pgmap v77: 129 pgs: 129 active+clean; 260 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.1 MiB/s wr, 1.05k op/s 2024-09-16T01:42:53.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.643+0000 7f9921ffb640 1 -- 172.21.15.139:0/2919378536 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12002 (secure 0 0 0) 0x7f991c09c180 con 0x7f9924106f10 2024-09-16T01:42:53.645 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:42:53.646 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":31,"btime":"2024-09-16T01:42:06:037308+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:42:05.468117+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:42:53.646 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 31 2024-09-16T01:42:53.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.646+0000 7f992a560640 1 -- 172.21.15.139:0/2919378536 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f99100a3430 msgr2=0x7f99100a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:53.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.646+0000 7f992a560640 1 --2- 172.21.15.139:0/2919378536 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f99100a3430 0x7f99100a58f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9914009230 tx=0x7f99140058d0 comp rx=0 tx=0).stop 2024-09-16T01:42:53.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.646+0000 7f992a560640 1 -- 172.21.15.139:0/2919378536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9924106f10 msgr2=0x7f992419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:53.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.646+0000 7f992a560640 1 --2- 172.21.15.139:0/2919378536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9924106f10 0x7f992419cda0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f991c00eaa0 tx=0x7f991c00ef60 comp rx=0 tx=0).stop 2024-09-16T01:42:53.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.646+0000 7f992a560640 1 -- 172.21.15.139:0/2919378536 shutdown_connections 2024-09-16T01:42:53.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.646+0000 7f992a560640 1 --2- 172.21.15.139:0/2919378536 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f99100a3430 0x7f99100a58f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:53.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.646+0000 7f992a560640 1 --2- 172.21.15.139:0/2919378536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9924106f10 0x7f992419cda0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:53.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.646+0000 7f992a560640 1 --2- 172.21.15.139:0/2919378536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9924102790 0x7f9924198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:53.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.646+0000 7f992a560640 1 --2- 172.21.15.139:0/2919378536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9924101de0 0x7f99241984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:53.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.646+0000 7f992a560640 1 -- 172.21.15.139:0/2919378536 >> 172.21.15.139:0/2919378536 conn(0x7f99240fd8c0 msgr2=0x7f99240fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:53.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.646+0000 7f992a560640 1 -- 172.21.15.139:0/2919378536 shutdown_connections 2024-09-16T01:42:53.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:53.646+0000 7f992a560640 1 -- 172.21.15.139:0/2919378536 wait complete. 2024-09-16T01:42:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:53 smithi164 ceph-mon[35121]: pgmap v77: 129 pgs: 129 active+clean; 260 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.1 MiB/s wr, 1.05k op/s 2024-09-16T01:42:53.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:53 smithi176 ceph-mon[35324]: pgmap v77: 129 pgs: 129 active+clean; 260 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.1 MiB/s wr, 1.05k op/s 2024-09-16T01:42:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:54 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2919378536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:42:54.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:54 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2919378536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:42:54.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:54 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2919378536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:42:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:55 smithi164 ceph-mon[35121]: pgmap v78: 129 pgs: 129 active+clean; 260 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.7 MiB/s wr, 849 op/s 2024-09-16T01:42:55.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:55 smithi139 ceph-mon[31175]: pgmap v78: 129 pgs: 129 active+clean; 260 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.7 MiB/s wr, 849 op/s 2024-09-16T01:42:55.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:55 smithi176 ceph-mon[35324]: pgmap v78: 129 pgs: 129 active+clean; 260 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.7 MiB/s wr, 849 op/s 2024-09-16T01:42:55.967 INFO:tasks.workunit.client.0.smithi139.stdout: 92 107915 0 79384 0 20851 0 2024-09-16T01:42:56.068 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id f292b9e9 --release --await' 2024-09-16T01:42:56.068 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id f292b9e9 --release --await 2024-09-16T01:42:56.323 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:42:56.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.782+0000 7f88c1db6640 1 -- 172.21.15.139:0/3982509631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc103760 msgr2=0x7f88bc107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:56.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.782+0000 7f88c1db6640 1 --2- 172.21.15.139:0/3982509631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc103760 0x7f88bc107ba0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009930 tx=0x7f88a402a420 comp rx=0 tx=0).stop 2024-09-16T01:42:56.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.782+0000 7f88c1db6640 1 -- 172.21.15.139:0/3982509631 shutdown_connections 2024-09-16T01:42:56.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.782+0000 7f88c1db6640 1 --2- 172.21.15.139:0/3982509631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f88bc1082e0 0x7f88bc10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.782+0000 7f88c1db6640 1 --2- 172.21.15.139:0/3982509631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc103760 0x7f88bc107ba0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.782+0000 7f88c1db6640 1 --2- 172.21.15.139:0/3982509631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88bc102db0 0x7f88bc103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.782+0000 7f88c1db6640 1 -- 172.21.15.139:0/3982509631 >> 172.21.15.139:0/3982509631 conn(0x7f88bc0fc7d0 msgr2=0x7f88bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:56.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.782+0000 7f88c1db6640 1 -- 172.21.15.139:0/3982509631 shutdown_connections 2024-09-16T01:42:56.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.782+0000 7f88c1db6640 1 -- 172.21.15.139:0/3982509631 wait complete. 2024-09-16T01:42:56.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.782+0000 7f88c1db6640 1 Processor -- start 2024-09-16T01:42:56.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.782+0000 7f88c1db6640 1 -- start start 2024-09-16T01:42:56.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88c1db6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc102db0 0x7f88bc1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88c1db6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f88bc103760 0x7f88bc198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88c1db6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88bc1082e0 0x7f88bc19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88c1db6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc112050 con 0x7f88bc102db0 2024-09-16T01:42:56.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88c1db6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc111ed0 con 0x7f88bc1082e0 2024-09-16T01:42:56.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88c1db6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1121d0 con 0x7f88bc103760 2024-09-16T01:42:56.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88bb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc102db0 0x7f88bc1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88bb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc102db0 0x7f88bc1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58704/0 (socket says 172.21.15.139:58704) 2024-09-16T01:42:56.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88bb7fe640 1 -- 172.21.15.139:0/1165005707 learned_addr learned my addr 172.21.15.139:0/1165005707 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:56.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88b3fff640 1 --2- 172.21.15.139:0/1165005707 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f88bc103760 0x7f88bc198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88bb7fe640 1 -- 172.21.15.139:0/1165005707 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f88bc103760 msgr2=0x7f88bc198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:56.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88bb7fe640 1 --2- 172.21.15.139:0/1165005707 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f88bc103760 0x7f88bc198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88bb7fe640 1 -- 172.21.15.139:0/1165005707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88bc1082e0 msgr2=0x7f88bc19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:56.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88bbfff640 1 --2- 172.21.15.139:0/1165005707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88bc1082e0 0x7f88bc19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88bb7fe640 1 --2- 172.21.15.139:0/1165005707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88bc1082e0 0x7f88bc19cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88bb7fe640 1 -- 172.21.15.139:0/1165005707 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc19d5a0 con 0x7f88bc102db0 2024-09-16T01:42:56.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88bb7fe640 1 --2- 172.21.15.139:0/1165005707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc102db0 0x7f88bc1985f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00e9f0 tx=0x7f88ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:56.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88b97fa640 1 -- 172.21.15.139:0/1165005707 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac00cd80 con 0x7f88bc102db0 2024-09-16T01:42:56.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88c1db6640 1 -- 172.21.15.139:0/1165005707 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88bc19d890 con 0x7f88bc102db0 2024-09-16T01:42:56.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88b97fa640 1 -- 172.21.15.139:0/1165005707 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88ac010910 con 0x7f88bc102db0 2024-09-16T01:42:56.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88b97fa640 1 -- 172.21.15.139:0/1165005707 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac0188f0 con 0x7f88bc102db0 2024-09-16T01:42:56.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.783+0000 7f88c1db6640 1 -- 172.21.15.139:0/1165005707 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88bc1a5170 con 0x7f88bc102db0 2024-09-16T01:42:56.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.786+0000 7f88c418f740 1 Processor -- start 2024-09-16T01:42:56.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.786+0000 7f88b97fa640 1 -- 172.21.15.139:0/1165005707 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f88ac010ac0 con 0x7f88bc102db0 2024-09-16T01:42:56.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.786+0000 7f88c418f740 1 -- start start 2024-09-16T01:42:56.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.786+0000 7f88c418f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e060 0x55fd6f50e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.786+0000 7f88c418f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e980 0x55fd6f518d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.786+0000 7f88c418f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f51b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.786+0000 7f88c418f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55fd6f473620 con 0x55fd6f519240 2024-09-16T01:42:56.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.786+0000 7f88c418f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fd6f41ae50 con 0x55fd6f50e980 2024-09-16T01:42:56.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.786+0000 7f88c418f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fd6f46f5e0 con 0x55fd6f50e060 2024-09-16T01:42:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.786+0000 7f88b3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f51b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.786+0000 7f88bb7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e980 0x55fd6f518d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.786+0000 7f88b3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f51b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58714/0 (socket says 172.21.15.139:58714) 2024-09-16T01:42:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.786+0000 7f88bbfff640 1 --2- 172.21.15.139:0/2100276342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e060 0x55fd6f50e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.786+0000 7f88b3fff640 1 -- 172.21.15.139:0/2100276342 learned_addr learned my addr 172.21.15.139:0/2100276342 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.787+0000 7f88b3fff640 1 -- 172.21.15.139:0/2100276342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e060 msgr2=0x55fd6f50e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.787+0000 7f88b3fff640 1 --2- 172.21.15.139:0/2100276342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e060 0x55fd6f50e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.787+0000 7f88b3fff640 1 -- 172.21.15.139:0/2100276342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e980 msgr2=0x55fd6f518d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.787+0000 7f88b3fff640 1 --2- 172.21.15.139:0/2100276342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e980 0x55fd6f518d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.787+0000 7f88b97fa640 1 --2- 172.21.15.139:0/1165005707 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f88a80a33b0 0x7f88a80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.787+0000 7f88b3fff640 1 -- 172.21.15.139:0/2100276342 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd6f40bd40 con 0x55fd6f519240 2024-09-16T01:42:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.787+0000 7f88bb7fe640 1 --2- 172.21.15.139:0/2100276342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e980 0x55fd6f518d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.787+0000 7f88bbfff640 1 --2- 172.21.15.139:0/1165005707 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f88a80a33b0 0x7f88a80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.787+0000 7f88b3fff640 1 --2- 172.21.15.139:0/2100276342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f51b630 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0697d0 tx=0x7f88a4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3b2c70fad96dc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.787+0000 7f88b97fa640 1 -- 172.21.15.139:0/1165005707 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f88ac0d05e0 con 0x7f88bc102db0 2024-09-16T01:42:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.787+0000 7f88b1ffb640 1 -- 172.21.15.139:0/2100276342 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4035030 con 0x55fd6f519240 2024-09-16T01:42:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.787+0000 7f88b1ffb640 1 -- 172.21.15.139:0/2100276342 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a4005520 con 0x55fd6f519240 2024-09-16T01:42:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.787+0000 7f88b1ffb640 1 -- 172.21.15.139:0/2100276342 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a40035f0 con 0x55fd6f519240 2024-09-16T01:42:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.788+0000 7f88c418f740 1 -- 172.21.15.139:0/2100276342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 msgr2=0x55fd6f51b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.788+0000 7f88bbfff640 1 --2- 172.21.15.139:0/1165005707 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f88a80a33b0 0x7f88a80a5870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f88b4005fd0 tx=0x7f88b40095a0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.788+0000 7f88c418f740 1 --2- 172.21.15.139:0/2100276342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f51b630 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0697d0 tx=0x7f88a4005e50 comp rx=0 tx=0).stop 2024-09-16T01:42:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.788+0000 7f88c418f740 1 -- 172.21.15.139:0/2100276342 shutdown_connections 2024-09-16T01:42:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.788+0000 7f88c418f740 1 --2- 172.21.15.139:0/2100276342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f51b630 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.788+0000 7f88c418f740 1 --2- 172.21.15.139:0/2100276342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e980 0x55fd6f518d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.788+0000 7f88c418f740 1 --2- 172.21.15.139:0/2100276342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e060 0x55fd6f50e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.788+0000 7f88c418f740 1 -- 172.21.15.139:0/2100276342 >> 172.21.15.139:0/2100276342 conn(0x55fd6f50a5f0 msgr2=0x55fd6f50a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:56.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.788+0000 7f88c418f740 1 -- 172.21.15.139:0/2100276342 shutdown_connections 2024-09-16T01:42:56.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.788+0000 7f88c418f740 1 -- 172.21.15.139:0/2100276342 wait complete. 2024-09-16T01:42:56.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.788+0000 7f88c418f740 1 -- start start 2024-09-16T01:42:56.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.788+0000 7f88c418f740 1 Processor -- start 2024-09-16T01:42:56.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88c418f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e060 0x55fd6f516100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88c418f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e980 0x55fd6f516640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88c418f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f516b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88c418f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55fd6f51be10 con 0x55fd6f519240 2024-09-16T01:42:56.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88c418f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fd6f3fbf10 con 0x55fd6f50e060 2024-09-16T01:42:56.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88c418f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fd6f51bf90 con 0x55fd6f50e980 2024-09-16T01:42:56.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88c418f740 10 client.0 _ll_register_callbacks cb 0x55fd6f41fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:42:56.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88bbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f516b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88bbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f516b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58730/0 (socket says 172.21.15.139:58730) 2024-09-16T01:42:56.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88bbfff640 1 -- 172.21.15.139:0/3402269422 learned_addr learned my addr 172.21.15.139:0/3402269422 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:56.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88b3fff640 1 --2- 172.21.15.139:0/3402269422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e980 0x55fd6f516640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88bb7fe640 1 --2- 172.21.15.139:0/3402269422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e060 0x55fd6f516100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88bbfff640 1 -- 172.21.15.139:0/3402269422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e980 msgr2=0x55fd6f516640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:56.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88bbfff640 1 --2- 172.21.15.139:0/3402269422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e980 0x55fd6f516640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88bbfff640 1 -- 172.21.15.139:0/3402269422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e060 msgr2=0x55fd6f516100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:56.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88bbfff640 1 --2- 172.21.15.139:0/3402269422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e060 0x55fd6f516100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88bbfff640 1 -- 172.21.15.139:0/3402269422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd6f182580 con 0x55fd6f519240 2024-09-16T01:42:56.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88b3fff640 1 --2- 172.21.15.139:0/3402269422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e980 0x55fd6f516640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:42:56.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88bbfff640 1 --2- 172.21.15.139:0/3402269422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f516b80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f88b4003e20 tx=0x7f88b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a6614099ded74d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:56.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f889affd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:42:56.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f889affd640 1 -- 172.21.15.139:0/3402269422 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b401f830 con 0x55fd6f519240 2024-09-16T01:42:56.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f889affd640 1 -- 172.21.15.139:0/3402269422 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f88b401f9d0 con 0x55fd6f519240 2024-09-16T01:42:56.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.789+0000 7f88c418f740 1 -- 172.21.15.139:0/3402269422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a4005520 con 0x55fd6f519240 2024-09-16T01:42:56.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.790+0000 7f889affd640 1 -- 172.21.15.139:0/3402269422 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f88b401fbd0 con 0x55fd6f519240 2024-09-16T01:42:56.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.790+0000 7f88c418f740 10 client.15819 fetch_fsmap learned FSMap version 31 2024-09-16T01:42:56.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.790+0000 7f88c418f740 1 -- 172.21.15.139:0/3402269422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fd6f24c960 con 0x55fd6f519240 2024-09-16T01:42:56.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.790+0000 7f889affd640 1 -- 172.21.15.139:0/3402269422 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a4005520 con 0x55fd6f519240 2024-09-16T01:42:56.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.790+0000 7f889affd640 1 -- 172.21.15.139:0/3402269422 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7f88b400f3c0 con 0x55fd6f519240 2024-09-16T01:42:56.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.790+0000 7f88c418f740 10 client.15819 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:42:56.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.790+0000 7f88c418f740 10 client.15819 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:42:56.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.790+0000 7f88c418f740 20 client.15819 populate_metadata read hostname 'smithi139' 2024-09-16T01:42:56.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.790+0000 7f88c418f740 1 --2- 172.21.15.139:0/3402269422 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fd6f528e80 0x55fd6f52b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.790+0000 7f88c418f740 4 client.15819 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:42:56.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.790+0000 7f88c418f740 1 -- 172.21.15.139:0/3402269422 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd6f480b90 con 0x55fd6f528e80 2024-09-16T01:42:56.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.790+0000 7f88bb7fe640 1 --2- 172.21.15.139:0/3402269422 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fd6f528e80 0x55fd6f52b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.791+0000 7f889affd640 10 client.15819 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:56.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.791+0000 7f88bb7fe640 1 --2- 172.21.15.139:0/3402269422 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fd6f528e80 0x55fd6f52b2b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b331ea945785985 server_cookie=235d5ea2806b2a0c in_seq=0 out_seq=0 2024-09-16T01:42:56.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.791+0000 7f889affd640 1 -- 172.21.15.139:0/3402269422 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd6f480b90 con 0x55fd6f528e80 2024-09-16T01:42:56.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.791+0000 7f889affd640 10 client.15819 handle_command_reply: tid=0 2024-09-16T01:42:56.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.791+0000 7f88c418f740 1 client.15819 shutdown 2024-09-16T01:42:56.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.792+0000 7f88c418f740 1 -- 172.21.15.139:0/3402269422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 msgr2=0x55fd6f516b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:56.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.792+0000 7f88c418f740 1 --2- 172.21.15.139:0/3402269422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f516b80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f88b4003e20 tx=0x7f88b4004a40 comp rx=0 tx=0).stop 2024-09-16T01:42:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.792+0000 7f88c418f740 1 -- 172.21.15.139:0/3402269422 shutdown_connections 2024-09-16T01:42:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.792+0000 7f88c418f740 1 --2- 172.21.15.139:0/3402269422 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fd6f528e80 0x55fd6f52b2b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.792+0000 7f88c418f740 1 --2- 172.21.15.139:0/3402269422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f516b80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.792+0000 7f88c418f740 1 --2- 172.21.15.139:0/3402269422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e980 0x55fd6f516640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.792+0000 7f889affd640 0 client.15819 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.792+0000 7f88c418f740 1 --2- 172.21.15.139:0/3402269422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e060 0x55fd6f516100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.792+0000 7f88c418f740 1 -- 172.21.15.139:0/3402269422 >> 172.21.15.139:0/3402269422 conn(0x55fd6f50a5f0 msgr2=0x55fd6f5102d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.792+0000 7f88c418f740 1 -- 172.21.15.139:0/3402269422 shutdown_connections 2024-09-16T01:42:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.792+0000 7f88c418f740 1 -- 172.21.15.139:0/3402269422 wait complete. 2024-09-16T01:42:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.792+0000 7f88c418f740 20 client.15819 trim_cache size 0 max 16384 2024-09-16T01:42:56.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88c418f740 1 Processor -- start 2024-09-16T01:42:56.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88c418f740 1 -- start start 2024-09-16T01:42:56.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88c418f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f50e060 0x55fd6f5215c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88c418f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e980 0x55fd6f521b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88c418f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f519240 0x55fd6f530bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88c418f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1af190 con 0x55fd6f50e060 2024-09-16T01:42:56.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88c418f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1b0d90 con 0x55fd6f519240 2024-09-16T01:42:56.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88c418f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1af860 con 0x55fd6f50e980 2024-09-16T01:42:56.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88bb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f50e060 0x55fd6f5215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88bb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f50e060 0x55fd6f5215c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58734/0 (socket says 172.21.15.139:58734) 2024-09-16T01:42:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88bb7fe640 1 -- 172.21.15.139:0/1931294728 learned_addr learned my addr 172.21.15.139:0/1931294728 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88b3fff640 1 --2- 172.21.15.139:0/1931294728 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e980 0x55fd6f521b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88bb7fe640 1 -- 172.21.15.139:0/1931294728 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e980 msgr2=0x55fd6f521b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88bb7fe640 1 --2- 172.21.15.139:0/1931294728 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e980 0x55fd6f521b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88bbfff640 1 --2- 172.21.15.139:0/1931294728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f519240 0x55fd6f530bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88bb7fe640 1 -- 172.21.15.139:0/1931294728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f519240 msgr2=0x55fd6f530bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88bb7fe640 1 --2- 172.21.15.139:0/1931294728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f519240 0x55fd6f530bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88bb7fe640 1 -- 172.21.15.139:0/1931294728 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd6f50e7b0 con 0x55fd6f50e060 2024-09-16T01:42:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.809+0000 7f88bb7fe640 1 --2- 172.21.15.139:0/1931294728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f50e060 0x55fd6f5215c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00c6a0 tx=0x7f88ac0d6550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30280790679d2c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.810+0000 7f88b17fa640 1 -- 172.21.15.139:0/1931294728 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac0d0070 con 0x55fd6f50e060 2024-09-16T01:42:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.810+0000 7f88b17fa640 1 -- 172.21.15.139:0/1931294728 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88ac09cc10 con 0x55fd6f50e060 2024-09-16T01:42:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.810+0000 7f88b17fa640 1 -- 172.21.15.139:0/1931294728 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac0cdc00 con 0x55fd6f50e060 2024-09-16T01:42:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.810+0000 7f88c418f740 1 -- 172.21.15.139:0/1931294728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f50e060 msgr2=0x55fd6f5215c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:56.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.810+0000 7f88c418f740 1 --2- 172.21.15.139:0/1931294728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f50e060 0x55fd6f5215c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00c6a0 tx=0x7f88ac0d6550 comp rx=0 tx=0).stop 2024-09-16T01:42:56.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.810+0000 7f88c418f740 1 -- 172.21.15.139:0/1931294728 shutdown_connections 2024-09-16T01:42:56.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.810+0000 7f88c418f740 1 --2- 172.21.15.139:0/1931294728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f519240 0x55fd6f530bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.810+0000 7f88c418f740 1 --2- 172.21.15.139:0/1931294728 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e980 0x55fd6f521b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.810+0000 7f88c418f740 1 --2- 172.21.15.139:0/1931294728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f50e060 0x55fd6f5215c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.810+0000 7f88c418f740 1 -- 172.21.15.139:0/1931294728 >> 172.21.15.139:0/1931294728 conn(0x55fd6f50a5f0 msgr2=0x55fd6f5149d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:56.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.810+0000 7f88c418f740 1 -- 172.21.15.139:0/1931294728 shutdown_connections 2024-09-16T01:42:56.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.810+0000 7f88c418f740 1 -- 172.21.15.139:0/1931294728 wait complete. 2024-09-16T01:42:56.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.811+0000 7f88c418f740 1 -- start start 2024-09-16T01:42:56.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.811+0000 7f88c418f740 1 Processor -- start 2024-09-16T01:42:56.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.811+0000 7f88c418f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e060 0x55fd6f512370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.811+0000 7f88c418f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e980 0x55fd6f5128b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.811+0000 7f88c418f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f512df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.811+0000 7f88c418f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1b0600 con 0x55fd6f519240 2024-09-16T01:42:56.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.811+0000 7f88c418f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1aff30 con 0x55fd6f50e980 2024-09-16T01:42:56.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.811+0000 7f88c418f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1ad170 con 0x55fd6f50e060 2024-09-16T01:42:56.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.811+0000 7f88bbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f512df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.811+0000 7f88c418f740 10 client.0 _ll_register_callbacks cb 0x7f88b4003ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:42:56.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.811+0000 7f88bbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f512df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58742/0 (socket says 172.21.15.139:58742) 2024-09-16T01:42:56.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.811+0000 7f88bbfff640 1 -- 172.21.15.139:0/3635256921 learned_addr learned my addr 172.21.15.139:0/3635256921 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:56.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.811+0000 7f88b3fff640 1 --2- 172.21.15.139:0/3635256921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e980 0x55fd6f5128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.811+0000 7f88bb7fe640 1 --2- 172.21.15.139:0/3635256921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e060 0x55fd6f512370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f88bbfff640 1 -- 172.21.15.139:0/3635256921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e060 msgr2=0x55fd6f512370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:56.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f88bbfff640 1 --2- 172.21.15.139:0/3635256921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e060 0x55fd6f512370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f88bbfff640 1 -- 172.21.15.139:0/3635256921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e980 msgr2=0x55fd6f5128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f88bbfff640 1 --2- 172.21.15.139:0/3635256921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e980 0x55fd6f5128b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f88bbfff640 1 -- 172.21.15.139:0/3635256921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd6f597c00 con 0x55fd6f519240 2024-09-16T01:42:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f88bbfff640 1 --2- 172.21.15.139:0/3635256921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f512df0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f88b4000f80 tx=0x7f88b400f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279630914c44e8d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f889affd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:42:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f889affd640 1 -- 172.21.15.139:0/3635256921 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b403b070 con 0x55fd6f519240 2024-09-16T01:42:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f889affd640 1 -- 172.21.15.139:0/3635256921 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f88b401f630 con 0x55fd6f519240 2024-09-16T01:42:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f88c418f740 1 -- 172.21.15.139:0/3635256921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88ac09cc10 con 0x55fd6f519240 2024-09-16T01:42:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f889affd640 1 -- 172.21.15.139:0/3635256921 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f88b4012d30 con 0x55fd6f519240 2024-09-16T01:42:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f88c418f740 10 client.15825 fetch_fsmap learned FSMap version 31 2024-09-16T01:42:56.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f88c418f740 1 -- 172.21.15.139:0/3635256921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fd6f598f40 con 0x55fd6f519240 2024-09-16T01:42:56.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f889affd640 1 -- 172.21.15.139:0/3635256921 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88ac09cc10 con 0x55fd6f519240 2024-09-16T01:42:56.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f889affd640 1 -- 172.21.15.139:0/3635256921 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7f88b4005720 con 0x55fd6f519240 2024-09-16T01:42:56.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f88c418f740 10 client.15825 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:42:56.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f88c418f740 10 client.15825 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:42:56.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f88c418f740 20 client.15825 populate_metadata read hostname 'smithi139' 2024-09-16T01:42:56.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.812+0000 7f88c418f740 1 --2- 172.21.15.139:0/3635256921 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fd6f5997a0 0x55fd6f59bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.813+0000 7f88c418f740 4 client.15825 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f292b9e9", "release": true, "await": true}] 2024-09-16T01:42:56.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.813+0000 7f88c418f740 1 -- 172.21.15.139:0/3635256921 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f292b9e9", "release": true, "await": true}) -- 0x55fd6f47b0a0 con 0x55fd6f5997a0 2024-09-16T01:42:56.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.813+0000 7f88bb7fe640 1 --2- 172.21.15.139:0/3635256921 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fd6f5997a0 0x55fd6f59bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.813+0000 7f889affd640 10 client.15825 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:56.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:56.813+0000 7f88bb7fe640 1 --2- 172.21.15.139:0/3635256921 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fd6f5997a0 0x55fd6f59bbc0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=917ef5d506816f5b server_cookie=75458d0c7d4a6f0 in_seq=0 out_seq=0 2024-09-16T01:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.958+0000 7efce7a5e640 1 -- 172.21.15.139:0/2106445227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce0107b70 msgr2=0x7efce0107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.958+0000 7efce7a5e640 1 --2- 172.21.15.139:0/2106445227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce0107b70 0x7efce0107f50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009930 tx=0x7efcdc02a420 comp rx=0 tx=0).stop 2024-09-16T01:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 -- 172.21.15.139:0/2106445227 shutdown_connections 2024-09-16T01:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 --2- 172.21.15.139:0/2106445227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efce0102470 0x7efce010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 --2- 172.21.15.139:0/2106445227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efce0101ad0 0x7efce0101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 --2- 172.21.15.139:0/2106445227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce0107b70 0x7efce0107f50 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 -- 172.21.15.139:0/2106445227 >> 172.21.15.139:0/2106445227 conn(0x7efce00fd7d0 msgr2=0x7efce00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 -- 172.21.15.139:0/2106445227 shutdown_connections 2024-09-16T01:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 -- 172.21.15.139:0/2106445227 wait complete. 2024-09-16T01:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 Processor -- start 2024-09-16T01:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 -- start start 2024-09-16T01:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efce0101ad0 0x7efce0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce0102470 0x7efce01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efce0107b70 0x7efce019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efce0111f20 con 0x7efce0102470 2024-09-16T01:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efce0111da0 con 0x7efce0107b70 2024-09-16T01:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.959+0000 7efce7a5e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efce01120a0 con 0x7efce0101ad0 2024-09-16T01:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efce4ff9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce0102470 0x7efce01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efce57fa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efce0101ad0 0x7efce0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efce4ff9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce0102470 0x7efce01988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58760/0 (socket says 172.21.15.139:58760) 2024-09-16T01:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efce4ff9640 1 -- 172.21.15.139:0/4031839748 learned_addr learned my addr 172.21.15.139:0/4031839748 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efce5ffb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efce0107b70 0x7efce019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efce4ff9640 1 -- 172.21.15.139:0/4031839748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efce0101ad0 msgr2=0x7efce0198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efce4ff9640 1 --2- 172.21.15.139:0/4031839748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efce0101ad0 0x7efce0198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efce4ff9640 1 -- 172.21.15.139:0/4031839748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efce0107b70 msgr2=0x7efce019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efce4ff9640 1 --2- 172.21.15.139:0/4031839748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efce0107b70 0x7efce019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efce4ff9640 1 -- 172.21.15.139:0/4031839748 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce019d380 con 0x7efce0102470 2024-09-16T01:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efce4ff9640 1 --2- 172.21.15.139:0/4031839748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce0102470 0x7efce01988d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7efcd000e9f0 tx=0x7efcd000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efcd67fc640 1 -- 172.21.15.139:0/4031839748 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd000cd80 con 0x7efce0102470 2024-09-16T01:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efcd67fc640 1 -- 172.21.15.139:0/4031839748 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcd0005500 con 0x7efce0102470 2024-09-16T01:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efcd67fc640 1 -- 172.21.15.139:0/4031839748 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0010690 con 0x7efce0102470 2024-09-16T01:42:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efce7a5e640 1 -- 172.21.15.139:0/4031839748 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce019d670 con 0x7efce0102470 2024-09-16T01:42:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.960+0000 7efce7a5e640 1 -- 172.21.15.139:0/4031839748 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce01a4f00 con 0x7efce0102470 2024-09-16T01:42:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efcd67fc640 1 -- 172.21.15.139:0/4031839748 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7efcd00046c0 con 0x7efce0102470 2024-09-16T01:42:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efcea7fe740 1 Processor -- start 2024-09-16T01:42:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efcea7fe740 1 -- start start 2024-09-16T01:42:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efcea7fe740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a30af0 0x55b763a30ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efcea7fe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a31410 0x55b763a3b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efcea7fe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a3bcb0 0x55b763a3e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efcea7fe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b7639952f0 con 0x55b763a3bcb0 2024-09-16T01:42:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efcea7fe740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b76393d270 con 0x55b763a30af0 2024-09-16T01:42:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efcea7fe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b7639912b0 con 0x55b763a31410 2024-09-16T01:42:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efce5ffb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a30af0 0x55b763a30ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efce57fa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a31410 0x55b763a3b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efce4ff9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a3bcb0 0x55b763a3e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efce5ffb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a30af0 0x55b763a30ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:53238/0 (socket says 172.21.15.139:53238) 2024-09-16T01:42:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efce5ffb640 1 -- 172.21.15.139:0/3630641763 learned_addr learned my addr 172.21.15.139:0/3630641763 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efce57fa640 1 -- 172.21.15.139:0/3630641763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a30af0 msgr2=0x55b763a30ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efce57fa640 1 --2- 172.21.15.139:0/3630641763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a30af0 0x55b763a30ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efce57fa640 1 -- 172.21.15.139:0/3630641763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a3bcb0 msgr2=0x55b763a3e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efce57fa640 1 --2- 172.21.15.139:0/3630641763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a3bcb0 0x55b763a3e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efce57fa640 1 -- 172.21.15.139:0/3630641763 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b76393c340 con 0x55b763a31410 2024-09-16T01:42:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efce5ffb640 1 --2- 172.21.15.139:0/3630641763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a30af0 0x55b763a30ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.964+0000 7efcd67fc640 1 --2- 172.21.15.139:0/4031839748 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7efccc0a34e0 0x7efccc0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.965+0000 7efce57fa640 1 --2- 172.21.15.139:0/3630641763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a31410 0x55b763a3b770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efcdc005940 tx=0x7efcdc005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=169baea3335e7e9b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.965+0000 7efce5ffb640 1 --2- 172.21.15.139:0/4031839748 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7efccc0a34e0 0x7efccc0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.965+0000 7efcd67fc640 1 -- 172.21.15.139:0/4031839748 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7efcd00d0450 con 0x7efce0102470 2024-09-16T01:42:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.965+0000 7efccb7fe640 1 -- 172.21.15.139:0/3630641763 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc040070 con 0x55b763a31410 2024-09-16T01:42:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.965+0000 7efccb7fe640 1 -- 172.21.15.139:0/3630641763 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcdc031870 con 0x55b763a31410 2024-09-16T01:42:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.965+0000 7efccb7fe640 1 -- 172.21.15.139:0/3630641763 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc035040 con 0x55b763a31410 2024-09-16T01:42:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efcea7fe740 1 -- 172.21.15.139:0/3630641763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a31410 msgr2=0x55b763a3b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efce5ffb640 1 --2- 172.21.15.139:0/4031839748 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7efccc0a34e0 0x7efccc0a59a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009fd0 tx=0x7efcd8009500 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efcea7fe740 1 --2- 172.21.15.139:0/3630641763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a31410 0x55b763a3b770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efcdc005940 tx=0x7efcdc005bd0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efcea7fe740 1 -- 172.21.15.139:0/3630641763 shutdown_connections 2024-09-16T01:42:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efcea7fe740 1 --2- 172.21.15.139:0/3630641763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a3bcb0 0x55b763a3e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efcea7fe740 1 --2- 172.21.15.139:0/3630641763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a31410 0x55b763a3b770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efcea7fe740 1 --2- 172.21.15.139:0/3630641763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a30af0 0x55b763a30ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efcea7fe740 1 -- 172.21.15.139:0/3630641763 >> 172.21.15.139:0/3630641763 conn(0x55b763a2d060 msgr2=0x55b763a2d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efcea7fe740 1 -- 172.21.15.139:0/3630641763 shutdown_connections 2024-09-16T01:42:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efcea7fe740 1 -- 172.21.15.139:0/3630641763 wait complete. 2024-09-16T01:42:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efcea7fe740 1 -- start start 2024-09-16T01:42:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efcea7fe740 1 Processor -- start 2024-09-16T01:42:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efcea7fe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a38bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efcea7fe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a31410 0x55b763a390f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.966+0000 7efcea7fe740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a3bcb0 0x55b763a39630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efcea7fe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b763a3e880 con 0x55b763a31410 2024-09-16T01:42:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efcea7fe740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b76391e290 con 0x55b763a3bcb0 2024-09-16T01:42:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efcea7fe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b763a3ea00 con 0x55b763a30af0 2024-09-16T01:42:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efcea7fe740 10 client.0 _ll_register_callbacks cb 0x55b76394f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:42:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efce57fa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a38bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efce4ff9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a31410 0x55b763a390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efce57fa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a38bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:59574/0 (socket says 172.21.15.139:59574) 2024-09-16T01:42:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efce57fa640 1 -- 172.21.15.139:0/2231438675 learned_addr learned my addr 172.21.15.139:0/2231438675 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efce5ffb640 1 --2- 172.21.15.139:0/2231438675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a3bcb0 0x55b763a39630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efce4ff9640 1 -- 172.21.15.139:0/2231438675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 msgr2=0x55b763a38bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efce4ff9640 1 --2- 172.21.15.139:0/2231438675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a38bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efce4ff9640 1 -- 172.21.15.139:0/2231438675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a3bcb0 msgr2=0x55b763a39630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efce4ff9640 1 --2- 172.21.15.139:0/2231438675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a3bcb0 0x55b763a39630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efce4ff9640 1 -- 172.21.15.139:0/2231438675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7636a8210 con 0x55b763a31410 2024-09-16T01:42:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efce4ff9640 1 --2- 172.21.15.139:0/2231438675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a31410 0x55b763a390f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7efcd00cd7f0 tx=0x7efcd00d3b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3f255cc66074fe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efcc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:42:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efcc8ff9640 1 -- 172.21.15.139:0/2231438675 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd009bec0 con 0x55b763a31410 2024-09-16T01:42:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efcc8ff9640 1 -- 172.21.15.139:0/2231438675 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7efcd00a4b60 con 0x55b763a31410 2024-09-16T01:42:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efcea7fe740 1 -- 172.21.15.139:0/2231438675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcdc031870 con 0x55b763a31410 2024-09-16T01:42:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efcc8ff9640 1 -- 172.21.15.139:0/2231438675 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efcd009bce0 con 0x55b763a31410 2024-09-16T01:42:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efcea7fe740 10 client.15837 fetch_fsmap learned FSMap version 31 2024-09-16T01:42:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efcea7fe740 1 -- 172.21.15.139:0/2231438675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b763774620 con 0x55b763a31410 2024-09-16T01:42:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.967+0000 7efcc8ff9640 1 -- 172.21.15.139:0/2231438675 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcdc031870 con 0x55b763a31410 2024-09-16T01:42:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.968+0000 7efcc8ff9640 1 -- 172.21.15.139:0/2231438675 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7efcd009b7d0 con 0x55b763a31410 2024-09-16T01:42:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.968+0000 7efcea7fe740 10 client.15837 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:42:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.968+0000 7efcea7fe740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:42:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.968+0000 7efcea7fe740 20 client.15837 populate_metadata read hostname 'smithi139' 2024-09-16T01:42:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.968+0000 7efcea7fe740 1 --2- 172.21.15.139:0/2231438675 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b763a4c240 0x55b763a4e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.968+0000 7efcea7fe740 4 client.15837 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:42:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.968+0000 7efcea7fe740 1 -- 172.21.15.139:0/2231438675 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7639a2a20 con 0x55b763a4c240 2024-09-16T01:42:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.968+0000 7efce57fa640 1 --2- 172.21.15.139:0/2231438675 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b763a4c240 0x55b763a4e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.968+0000 7efcc8ff9640 10 client.15837 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:56.968+0000 7efce57fa640 1 --2- 172.21.15.139:0/2231438675 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b763a4c240 0x55b763a4e6a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49235c423c6fc2b2 server_cookie=451c7c400dd1f7fd in_seq=0 out_seq=0 2024-09-16T01:42:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcc8ff9640 1 -- 172.21.15.139:0/2231438675 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7639a2a20 con 0x55b763a4c240 2024-09-16T01:42:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcc8ff9640 10 client.15837 handle_command_reply: tid=0 2024-09-16T01:42:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcea7fe740 1 client.15837 shutdown 2024-09-16T01:42:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcea7fe740 1 -- 172.21.15.139:0/2231438675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a31410 msgr2=0x55b763a390f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcea7fe740 1 --2- 172.21.15.139:0/2231438675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a31410 0x55b763a390f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7efcd00cd7f0 tx=0x7efcd00d3b90 comp rx=0 tx=0).stop 2024-09-16T01:42:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcea7fe740 1 -- 172.21.15.139:0/2231438675 shutdown_connections 2024-09-16T01:42:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcea7fe740 1 --2- 172.21.15.139:0/2231438675 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b763a4c240 0x55b763a4e6a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcea7fe740 1 --2- 172.21.15.139:0/2231438675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a3bcb0 0x55b763a39630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcc8ff9640 0 client.15837 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcea7fe740 1 --2- 172.21.15.139:0/2231438675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a31410 0x55b763a390f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcea7fe740 1 --2- 172.21.15.139:0/2231438675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a38bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcea7fe740 1 -- 172.21.15.139:0/2231438675 >> 172.21.15.139:0/2231438675 conn(0x55b763a2d060 msgr2=0x55b763a32d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcea7fe740 1 -- 172.21.15.139:0/2231438675 shutdown_connections 2024-09-16T01:42:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcea7fe740 1 -- 172.21.15.139:0/2231438675 wait complete. 2024-09-16T01:42:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.062+0000 7efcea7fe740 20 client.15837 trim_cache size 0 max 16384 2024-09-16T01:42:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f889affd640 1 -- 172.21.15.139:0/3635256921 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55fd6f47b0a0 con 0x55fd6f5997a0 2024-09-16T01:42:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f889affd640 10 client.15825 handle_command_reply: tid=0 2024-09-16T01:42:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f88c418f740 1 client.15825 shutdown 2024-09-16T01:42:57.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f88c418f740 1 -- 172.21.15.139:0/3635256921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 msgr2=0x55fd6f512df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:57.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f88c418f740 1 --2- 172.21.15.139:0/3635256921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f512df0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f88b4000f80 tx=0x7f88b400f5f0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f88c418f740 1 -- 172.21.15.139:0/3635256921 shutdown_connections 2024-09-16T01:42:57.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f88c418f740 1 --2- 172.21.15.139:0/3635256921 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fd6f5997a0 0x55fd6f59bbc0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f88c418f740 1 --2- 172.21.15.139:0/3635256921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fd6f519240 0x55fd6f512df0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f889affd640 0 client.15825 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:57.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f88c418f740 1 --2- 172.21.15.139:0/3635256921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd6f50e980 0x55fd6f5128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f88c418f740 1 --2- 172.21.15.139:0/3635256921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd6f50e060 0x55fd6f512370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f88c418f740 1 -- 172.21.15.139:0/3635256921 >> 172.21.15.139:0/3635256921 conn(0x55fd6f50a5f0 msgr2=0x55fd6f50fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:57.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f88c418f740 1 -- 172.21.15.139:0/3635256921 shutdown_connections 2024-09-16T01:42:57.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f88c418f740 1 -- 172.21.15.139:0/3635256921 wait complete. 2024-09-16T01:42:57.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.065+0000 7f88c418f740 20 client.15825 trim_cache size 0 max 16384 2024-09-16T01:42:57.069 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:42:57.069 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 31, 2024-09-16T01:42:57.069 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:42:57.069 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 8, 2024-09-16T01:42:57.069 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:42:57.069 INFO:teuthology.orchestra.run.smithi139.stdout: "f292b9e9": { 2024-09-16T01:42:57.069 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 8, 2024-09-16T01:42:57.069 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:42:57.070 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:42:57.070 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:42:57.070 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:42:57.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:42:57.070 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:42:57.070 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 139.6, 2024-09-16T01:42:57.070 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:42:57.070 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:42:57.070 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:42:57.070 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:42:57.070 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:42:57.070 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:42:57.071 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:42:57.071 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:42:57.071 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:42:57.071 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:42:57.071 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:42:57.071 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:42:57.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.066+0000 7f88c1db6640 1 -- 172.21.15.139:0/1165005707 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f88a80a33b0 msgr2=0x7f88a80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:57.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.066+0000 7f88c1db6640 1 --2- 172.21.15.139:0/1165005707 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f88a80a33b0 0x7f88a80a5870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f88b4005fd0 tx=0x7f88b40095a0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.066+0000 7f88c1db6640 1 -- 172.21.15.139:0/1165005707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc102db0 msgr2=0x7f88bc1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:57.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.066+0000 7f88c1db6640 1 --2- 172.21.15.139:0/1165005707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc102db0 0x7f88bc1985f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00e9f0 tx=0x7f88ac00eeb0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.066+0000 7f88c1db6640 1 -- 172.21.15.139:0/1165005707 shutdown_connections 2024-09-16T01:42:57.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.066+0000 7f88c1db6640 1 --2- 172.21.15.139:0/1165005707 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f88a80a33b0 0x7f88a80a5870 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.066+0000 7f88c1db6640 1 --2- 172.21.15.139:0/1165005707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88bc1082e0 0x7f88bc19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.066+0000 7f88c1db6640 1 --2- 172.21.15.139:0/1165005707 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f88bc103760 0x7f88bc198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.066+0000 7f88c1db6640 1 --2- 172.21.15.139:0/1165005707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc102db0 0x7f88bc1985f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.066+0000 7f88c1db6640 1 -- 172.21.15.139:0/1165005707 >> 172.21.15.139:0/1165005707 conn(0x7f88bc0fc7d0 msgr2=0x7f88bc105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.066+0000 7f88c1db6640 1 -- 172.21.15.139:0/1165005707 shutdown_connections 2024-09-16T01:42:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:57.066+0000 7f88c1db6640 1 -- 172.21.15.139:0/1165005707 wait complete. 2024-09-16T01:42:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.079+0000 7efcea7fe740 1 Processor -- start 2024-09-16T01:42:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.079+0000 7efcea7fe740 1 -- start start 2024-09-16T01:42:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.079+0000 7efcea7fe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a3ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.079+0000 7efcea7fe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a31410 0x55b763a39460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.079+0000 7efcea7fe740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a3bcb0 0x55b763a53f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.079+0000 7efcea7fe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efce01aefd0 con 0x55b763a31410 2024-09-16T01:42:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.079+0000 7efcea7fe740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efce01b0bd0 con 0x55b763a3bcb0 2024-09-16T01:42:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.079+0000 7efcea7fe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efce01af6a0 con 0x55b763a30af0 2024-09-16T01:42:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.079+0000 7efce4ff9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a31410 0x55b763a39460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.079+0000 7efce5ffb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a3bcb0 0x55b763a53f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.079+0000 7efce4ff9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a31410 0x55b763a39460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58784/0 (socket says 172.21.15.139:58784) 2024-09-16T01:42:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.079+0000 7efce4ff9640 1 -- 172.21.15.139:0/1391039417 learned_addr learned my addr 172.21.15.139:0/1391039417 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.079+0000 7efce57fa640 1 --2- 172.21.15.139:0/1391039417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a3ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.080+0000 7efce4ff9640 1 -- 172.21.15.139:0/1391039417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 msgr2=0x55b763a3ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.080+0000 7efce4ff9640 1 --2- 172.21.15.139:0/1391039417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a3ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.080+0000 7efce4ff9640 1 -- 172.21.15.139:0/1391039417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a3bcb0 msgr2=0x55b763a53f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.080+0000 7efce4ff9640 1 --2- 172.21.15.139:0/1391039417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a3bcb0 0x55b763a53f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.080+0000 7efce4ff9640 1 -- 172.21.15.139:0/1391039417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b763a30870 con 0x55b763a31410 2024-09-16T01:42:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.080+0000 7efce5ffb640 1 --2- 172.21.15.139:0/1391039417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a3bcb0 0x55b763a53f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.080+0000 7efce4ff9640 1 --2- 172.21.15.139:0/1391039417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a31410 0x55b763a39460 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efcd00049d0 tx=0x7efcd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da074dd8c2f7028 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.080+0000 7efccaffd640 1 -- 172.21.15.139:0/1391039417 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd009bec0 con 0x55b763a31410 2024-09-16T01:42:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.080+0000 7efccaffd640 1 -- 172.21.15.139:0/1391039417 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcd009ca80 con 0x55b763a31410 2024-09-16T01:42:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.080+0000 7efccaffd640 1 -- 172.21.15.139:0/1391039417 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd00a0730 con 0x55b763a31410 2024-09-16T01:42:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.081+0000 7efcea7fe740 1 -- 172.21.15.139:0/1391039417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a31410 msgr2=0x55b763a39460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.081+0000 7efcea7fe740 1 --2- 172.21.15.139:0/1391039417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a31410 0x55b763a39460 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efcd00049d0 tx=0x7efcd0004a40 comp rx=0 tx=0).stop 2024-09-16T01:42:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.081+0000 7efcea7fe740 1 -- 172.21.15.139:0/1391039417 shutdown_connections 2024-09-16T01:42:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.081+0000 7efcea7fe740 1 --2- 172.21.15.139:0/1391039417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a3bcb0 0x55b763a53f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.081+0000 7efcea7fe740 1 --2- 172.21.15.139:0/1391039417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a31410 0x55b763a39460 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.081+0000 7efcea7fe740 1 --2- 172.21.15.139:0/1391039417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a3ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.081+0000 7efcea7fe740 1 -- 172.21.15.139:0/1391039417 >> 172.21.15.139:0/1391039417 conn(0x55b763a2d060 msgr2=0x55b763a3a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.081+0000 7efcea7fe740 1 -- 172.21.15.139:0/1391039417 shutdown_connections 2024-09-16T01:42:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.081+0000 7efcea7fe740 1 -- 172.21.15.139:0/1391039417 wait complete. 2024-09-16T01:42:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.081+0000 7efcea7fe740 1 -- start start 2024-09-16T01:42:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.081+0000 7efcea7fe740 1 Processor -- start 2024-09-16T01:42:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efcea7fe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a43c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efcea7fe740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a31410 0x55b763a441d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efcea7fe740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a3bcb0 0x55b763a351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efcea7fe740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efce01b0440 con 0x55b763a3bcb0 2024-09-16T01:42:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efcea7fe740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efce01afd70 con 0x55b763a31410 2024-09-16T01:42:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efcea7fe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efce01acfb0 con 0x55b763a30af0 2024-09-16T01:42:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efcea7fe740 10 client.0 _ll_register_callbacks cb 0x7efcd00d3c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:42:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce57fa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a43c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce5ffb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a3bcb0 0x55b763a351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce4ff9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a31410 0x55b763a441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce57fa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a43c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:59590/0 (socket says 172.21.15.139:59590) 2024-09-16T01:42:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce5ffb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a3bcb0 0x55b763a351d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58786/0 (socket says 172.21.15.139:58786) 2024-09-16T01:42:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce4ff9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a31410 0x55b763a441d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:53270/0 (socket says 172.21.15.139:53270) 2024-09-16T01:42:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce57fa640 1 -- 172.21.15.139:0/718643051 learned_addr learned my addr 172.21.15.139:0/718643051 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce57fa640 1 -- 172.21.15.139:0/718643051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a31410 msgr2=0x55b763a441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce57fa640 1 --2- 172.21.15.139:0/718643051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a31410 0x55b763a441d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce57fa640 1 -- 172.21.15.139:0/718643051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a3bcb0 msgr2=0x55b763a351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:42:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce57fa640 1 --2- 172.21.15.139:0/718643051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a3bcb0 0x55b763a351d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce57fa640 1 -- 172.21.15.139:0/718643051 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b763a35810 con 0x55b763a30af0 2024-09-16T01:42:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce4ff9640 1 --2- 172.21.15.139:0/718643051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a31410 0x55b763a441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce5ffb640 1 --2- 172.21.15.139:0/718643051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a3bcb0 0x55b763a351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:42:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efce57fa640 1 --2- 172.21.15.139:0/718643051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a43c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efcdc005c40 tx=0x7efcdc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=806c57e8104f7056 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.082+0000 7efcc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:42:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efcc8ff9640 1 -- 172.21.15.139:0/718643051 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc040050 con 0x55b763a30af0 2024-09-16T01:42:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efcc8ff9640 1 -- 172.21.15.139:0/718643051 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7efcdc004040 con 0x55b763a30af0 2024-09-16T01:42:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efcea7fe740 1 -- 172.21.15.139:0/718643051 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcd009ca80 con 0x55b763a30af0 2024-09-16T01:42:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efcc8ff9640 1 -- 172.21.15.139:0/718643051 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efcdc05d050 con 0x55b763a30af0 2024-09-16T01:42:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efcea7fe740 10 client.25327 fetch_fsmap learned FSMap version 31 2024-09-16T01:42:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efcea7fe740 1 -- 172.21.15.139:0/718643051 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b763ab68c0 con 0x55b763a30af0 2024-09-16T01:42:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efcc8ff9640 1 -- 172.21.15.139:0/718643051 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcd009ca80 con 0x55b763a30af0 2024-09-16T01:42:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efcc8ff9640 1 -- 172.21.15.139:0/718643051 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7efcdc04f3c0 con 0x55b763a30af0 2024-09-16T01:42:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efcea7fe740 10 client.25327 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:42:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efcea7fe740 10 client.25327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:42:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efcea7fe740 20 client.25327 populate_metadata read hostname 'smithi139' 2024-09-16T01:42:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efcea7fe740 1 --2- 172.21.15.139:0/718643051 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b763ab6d10 0x55b763ab9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efcea7fe740 4 client.25327 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:42:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efcea7fe740 1 -- 172.21.15.139:0/718643051 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b76399cd70 con 0x55b763ab6d10 2024-09-16T01:42:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.083+0000 7efce4ff9640 1 --2- 172.21.15.139:0/718643051 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b763ab6d10 0x55b763ab9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.084+0000 7efcc8ff9640 10 client.25327 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.084+0000 7efce4ff9640 1 --2- 172.21.15.139:0/718643051 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b763ab6d10 0x55b763ab9100 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=780b4173f5c10ae9 server_cookie=83530b5977334 in_seq=0 out_seq=0 2024-09-16T01:42:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcc8ff9640 1 -- 172.21.15.139:0/718643051 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b76399cd70 con 0x55b763ab6d10 2024-09-16T01:42:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcc8ff9640 10 client.25327 handle_command_reply: tid=0 2024-09-16T01:42:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcea7fe740 1 client.25327 shutdown 2024-09-16T01:42:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcea7fe740 1 -- 172.21.15.139:0/718643051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 msgr2=0x55b763a43c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcea7fe740 1 --2- 172.21.15.139:0/718643051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a43c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efcdc005c40 tx=0x7efcdc000bc0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcea7fe740 1 -- 172.21.15.139:0/718643051 shutdown_connections 2024-09-16T01:42:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcea7fe740 1 --2- 172.21.15.139:0/718643051 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b763ab6d10 0x55b763ab9100 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcea7fe740 1 --2- 172.21.15.139:0/718643051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b763a3bcb0 0x55b763a351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcea7fe740 1 --2- 172.21.15.139:0/718643051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b763a31410 0x55b763a441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcea7fe740 1 --2- 172.21.15.139:0/718643051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b763a30af0 0x55b763a43c90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcc8ff9640 0 client.25327 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:42:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcea7fe740 1 -- 172.21.15.139:0/718643051 >> 172.21.15.139:0/718643051 conn(0x55b763a2d060 msgr2=0x55b763a32d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcea7fe740 1 -- 172.21.15.139:0/718643051 shutdown_connections 2024-09-16T01:42:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcea7fe740 1 -- 172.21.15.139:0/718643051 wait complete. 2024-09-16T01:42:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.189+0000 7efcea7fe740 20 client.25327 trim_cache size 0 max 16384 2024-09-16T01:42:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:42:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:42:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:42:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:42:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.190+0000 7efce7a5e640 1 -- 172.21.15.139:0/4031839748 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7efccc0a34e0 msgr2=0x7efccc0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.190+0000 7efce7a5e640 1 --2- 172.21.15.139:0/4031839748 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7efccc0a34e0 0x7efccc0a59a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009fd0 tx=0x7efcd8009500 comp rx=0 tx=0).stop 2024-09-16T01:42:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.190+0000 7efce7a5e640 1 -- 172.21.15.139:0/4031839748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce0102470 msgr2=0x7efce01988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.190+0000 7efce7a5e640 1 --2- 172.21.15.139:0/4031839748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce0102470 0x7efce01988d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7efcd000e9f0 tx=0x7efcd000edf0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.190+0000 7efce7a5e640 1 -- 172.21.15.139:0/4031839748 shutdown_connections 2024-09-16T01:42:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.190+0000 7efce7a5e640 1 --2- 172.21.15.139:0/4031839748 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7efccc0a34e0 0x7efccc0a59a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.190+0000 7efce7a5e640 1 --2- 172.21.15.139:0/4031839748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efce0107b70 0x7efce019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.190+0000 7efce7a5e640 1 --2- 172.21.15.139:0/4031839748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce0102470 0x7efce01988d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.190+0000 7efce7a5e640 1 --2- 172.21.15.139:0/4031839748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efce0101ad0 0x7efce0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.190+0000 7efce7a5e640 1 -- 172.21.15.139:0/4031839748 >> 172.21.15.139:0/4031839748 conn(0x7efce00fd7d0 msgr2=0x7efce00fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.190+0000 7efce7a5e640 1 -- 172.21.15.139:0/4031839748 shutdown_connections 2024-09-16T01:42:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:42:57.190+0000 7efce7a5e640 1 -- 172.21.15.139:0/4031839748 wait complete. 2024-09-16T01:42:57.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:57 smithi164 ceph-mon[35121]: pgmap v79: 129 pgs: 129 active+clean; 287 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.0 MiB/s wr, 953 op/s 2024-09-16T01:42:57.897 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:57 smithi139 ceph-mon[31175]: pgmap v79: 129 pgs: 129 active+clean; 287 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.0 MiB/s wr, 953 op/s 2024-09-16T01:42:57.925 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f292b9e9', total seconds elapsed: 25.1 2024-09-16T01:42:57.925 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.55 2024-09-16T01:42:57.926 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:42:57.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:57 smithi176 ceph-mon[35324]: pgmap v79: 129 pgs: 129 active+clean; 287 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.0 MiB/s wr, 953 op/s 2024-09-16T01:42:58.204 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:42:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.695+0000 7f47ed0bc640 1 -- 172.21.15.139:0/130439459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47e8100e20 msgr2=0x7f47e8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.695+0000 7f47ed0bc640 1 --2- 172.21.15.139:0/130439459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47e8100e20 0x7f47e8101200 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009900 tx=0x7f47d802a480 comp rx=0 tx=0).stop 2024-09-16T01:42:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.695+0000 7f47ed0bc640 1 -- 172.21.15.139:0/130439459 shutdown_connections 2024-09-16T01:42:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.695+0000 7f47ed0bc640 1 --2- 172.21.15.139:0/130439459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47e8102ef0 0x7f47e810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.695+0000 7f47ed0bc640 1 --2- 172.21.15.139:0/130439459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47e8102570 0x7f47e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.695+0000 7f47ed0bc640 1 --2- 172.21.15.139:0/130439459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47e8100e20 0x7f47e8101200 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.695+0000 7f47ed0bc640 1 -- 172.21.15.139:0/130439459 >> 172.21.15.139:0/130439459 conn(0x7f47e80fc7d0 msgr2=0x7f47e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.695+0000 7f47ed0bc640 1 -- 172.21.15.139:0/130439459 shutdown_connections 2024-09-16T01:42:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.695+0000 7f47ed0bc640 1 -- 172.21.15.139:0/130439459 wait complete. 2024-09-16T01:42:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.695+0000 7f47ed0bc640 1 Processor -- start 2024-09-16T01:42:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.695+0000 7f47ed0bc640 1 -- start start 2024-09-16T01:42:58.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47ed0bc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47e8100e20 0x7f47e81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:58.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47ed0bc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47e8102570 0x7f47e8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:58.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47ed0bc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47e8102ef0 0x7f47e819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:58.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47ed0bc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8110f80 con 0x7f47e8100e20 2024-09-16T01:42:58.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47ed0bc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8110e00 con 0x7f47e8102570 2024-09-16T01:42:58.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47ed0bc640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8111100 con 0x7f47e8102ef0 2024-09-16T01:42:58.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47e6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47e8100e20 0x7f47e81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:58.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47e6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47e8100e20 0x7f47e81984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58800/0 (socket says 172.21.15.139:58800) 2024-09-16T01:42:58.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47dffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47e8102570 0x7f47e8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:58.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47e6d9d640 1 -- 172.21.15.139:0/4055711013 learned_addr learned my addr 172.21.15.139:0/4055711013 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:42:58.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47e759e640 1 --2- 172.21.15.139:0/4055711013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47e8102ef0 0x7f47e819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:58.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47dffff640 1 -- 172.21.15.139:0/4055711013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47e8102ef0 msgr2=0x7f47e819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:58.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47dffff640 1 --2- 172.21.15.139:0/4055711013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47e8102ef0 0x7f47e819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47dffff640 1 -- 172.21.15.139:0/4055711013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47e8100e20 msgr2=0x7f47e81984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47dffff640 1 --2- 172.21.15.139:0/4055711013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47e8100e20 0x7f47e81984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47dffff640 1 -- 172.21.15.139:0/4055711013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e819d470 con 0x7f47e8102570 2024-09-16T01:42:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47e759e640 1 --2- 172.21.15.139:0/4055711013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47e8102ef0 0x7f47e819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:42:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47dffff640 1 --2- 172.21.15.139:0/4055711013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47e8102570 0x7f47e8198a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47d000e9f0 tx=0x7f47d000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.696+0000 7f47e4d99640 1 -- 172.21.15.139:0/4055711013 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d000ccc0 con 0x7f47e8102570 2024-09-16T01:42:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.697+0000 7f47ed0bc640 1 -- 172.21.15.139:0/4055711013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e819d760 con 0x7f47e8102570 2024-09-16T01:42:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.697+0000 7f47e4d99640 1 -- 172.21.15.139:0/4055711013 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d0010910 con 0x7f47e8102570 2024-09-16T01:42:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.697+0000 7f47e4d99640 1 -- 172.21.15.139:0/4055711013 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d0018960 con 0x7f47e8102570 2024-09-16T01:42:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.697+0000 7f47ed0bc640 1 -- 172.21.15.139:0/4055711013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e81a5040 con 0x7f47e8102570 2024-09-16T01:42:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.698+0000 7f47ed0bc640 1 -- 172.21.15.139:0/4055711013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e8101200 con 0x7f47e8102570 2024-09-16T01:42:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.699+0000 7f47e4d99640 1 -- 172.21.15.139:0/4055711013 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f47d0010ac0 con 0x7f47e8102570 2024-09-16T01:42:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.700+0000 7f47e4d99640 1 --2- 172.21.15.139:0/4055711013 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f47d40a33b0 0x7f47d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:42:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.700+0000 7f47e6d9d640 1 --2- 172.21.15.139:0/4055711013 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f47d40a33b0 0x7f47d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:42:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.700+0000 7f47e4d99640 1 -- 172.21.15.139:0/4055711013 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f47d00d0600 con 0x7f47e8102570 2024-09-16T01:42:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.700+0000 7f47e6d9d640 1 --2- 172.21.15.139:0/4055711013 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f47d40a33b0 0x7f47d40a5870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f47d8005770 tx=0x7f47d8005a20 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:42:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.707+0000 7f47e4d99640 1 -- 172.21.15.139:0/4055711013 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47d009cc30 con 0x7f47e8102570 2024-09-16T01:42:58.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.933+0000 7f47ed0bc640 1 -- 172.21.15.139:0/4055711013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47e810a0d0 con 0x7f47e8102570 2024-09-16T01:42:58.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.934+0000 7f47e4d99640 1 -- 172.21.15.139:0/4055711013 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12002 (secure 0 0 0) 0x7f47d009c530 con 0x7f47e8102570 2024-09-16T01:42:58.935 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:42:58.935 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":31,"btime":"2024-09-16T01:42:06:037308+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:42:05.468117+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:42:58.935 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 31 2024-09-16T01:42:58.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.936+0000 7f47ed0bc640 1 -- 172.21.15.139:0/4055711013 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f47d40a33b0 msgr2=0x7f47d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:58.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.936+0000 7f47ed0bc640 1 --2- 172.21.15.139:0/4055711013 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f47d40a33b0 0x7f47d40a5870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f47d8005770 tx=0x7f47d8005a20 comp rx=0 tx=0).stop 2024-09-16T01:42:58.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.936+0000 7f47ed0bc640 1 -- 172.21.15.139:0/4055711013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47e8102570 msgr2=0x7f47e8198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:42:58.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.936+0000 7f47ed0bc640 1 --2- 172.21.15.139:0/4055711013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47e8102570 0x7f47e8198a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47d000e9f0 tx=0x7f47d000eeb0 comp rx=0 tx=0).stop 2024-09-16T01:42:58.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.936+0000 7f47ed0bc640 1 -- 172.21.15.139:0/4055711013 shutdown_connections 2024-09-16T01:42:58.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.936+0000 7f47ed0bc640 1 --2- 172.21.15.139:0/4055711013 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f47d40a33b0 0x7f47d40a5870 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:58.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.936+0000 7f47ed0bc640 1 --2- 172.21.15.139:0/4055711013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47e8102ef0 0x7f47e819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:58.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.936+0000 7f47ed0bc640 1 --2- 172.21.15.139:0/4055711013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47e8102570 0x7f47e8198a00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:58.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.936+0000 7f47ed0bc640 1 --2- 172.21.15.139:0/4055711013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47e8100e20 0x7f47e81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:42:58.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.936+0000 7f47ed0bc640 1 -- 172.21.15.139:0/4055711013 >> 172.21.15.139:0/4055711013 conn(0x7f47e80fc7d0 msgr2=0x7f47e806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:42:58.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.936+0000 7f47ed0bc640 1 -- 172.21.15.139:0/4055711013 shutdown_connections 2024-09-16T01:42:58.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:42:58.936+0000 7f47ed0bc640 1 -- 172.21.15.139:0/4055711013 wait complete. 2024-09-16T01:42:59.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:59 smithi139 ceph-mon[31175]: pgmap v80: 129 pgs: 129 active+clean; 287 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.3 MiB/s wr, 430 op/s 2024-09-16T01:42:59.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:42:59 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/4055711013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:42:59.791 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0' 2024-09-16T01:42:59.791 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0 2024-09-16T01:42:59.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:59 smithi164 ceph-mon[35121]: pgmap v80: 129 pgs: 129 active+clean; 287 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.3 MiB/s wr, 430 op/s 2024-09-16T01:42:59.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:42:59 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/4055711013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:42:59.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:59 smithi176 ceph-mon[35324]: pgmap v80: 129 pgs: 129 active+clean; 287 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.3 MiB/s wr, 430 op/s 2024-09-16T01:42:59.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:42:59 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/4055711013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:43:00.229 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:43:00.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.842+0000 7fbb4965d640 1 -- 172.21.15.139:0/1452085749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb44074eb0 msgr2=0x7fbb44112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:00.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.842+0000 7fbb4965d640 1 --2- 172.21.15.139:0/1452085749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb44074eb0 0x7fbb44112650 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009900 tx=0x7fbb3802a4a0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.842+0000 7fbb4965d640 1 -- 172.21.15.139:0/1452085749 shutdown_connections 2024-09-16T01:43:00.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.842+0000 7fbb4965d640 1 --2- 172.21.15.139:0/1452085749 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb44112b90 0x7fbb44114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.842+0000 7fbb4965d640 1 --2- 172.21.15.139:0/1452085749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb44074eb0 0x7fbb44112650 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.842+0000 7fbb4965d640 1 --2- 172.21.15.139:0/1452085749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb44074590 0x7fbb44074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.842+0000 7fbb4965d640 1 -- 172.21.15.139:0/1452085749 >> 172.21.15.139:0/1452085749 conn(0x7fbb440fd880 msgr2=0x7fbb440ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:00.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.842+0000 7fbb4965d640 1 -- 172.21.15.139:0/1452085749 shutdown_connections 2024-09-16T01:43:00.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.842+0000 7fbb4965d640 1 -- 172.21.15.139:0/1452085749 wait complete. 2024-09-16T01:43:00.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.843+0000 7fbb4965d640 1 Processor -- start 2024-09-16T01:43:00.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.843+0000 7fbb4965d640 1 -- start start 2024-09-16T01:43:00.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb4965d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb44074590 0x7fbb4419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb4965d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb44074eb0 0x7fbb4419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb4965d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb44112b90 0x7fbb441a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb4965d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbb44118650 con 0x7fbb44112b90 2024-09-16T01:43:00.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb4965d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb441184d0 con 0x7fbb44074590 2024-09-16T01:43:00.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb4965d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbb441187d0 con 0x7fbb44074eb0 2024-09-16T01:43:00.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb42ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb44074590 0x7fbb4419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb42ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb44074590 0x7fbb4419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:53288/0 (socket says 172.21.15.139:53288) 2024-09-16T01:43:00.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb42ffd640 1 -- 172.21.15.139:0/869928364 learned_addr learned my addr 172.21.15.139:0/869928364 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:00.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb427fc640 1 --2- 172.21.15.139:0/869928364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb44074eb0 0x7fbb4419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb437fe640 1 --2- 172.21.15.139:0/869928364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb44112b90 0x7fbb441a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb42ffd640 1 -- 172.21.15.139:0/869928364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb44074eb0 msgr2=0x7fbb4419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:00.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb42ffd640 1 --2- 172.21.15.139:0/869928364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb44074eb0 0x7fbb4419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb42ffd640 1 -- 172.21.15.139:0/869928364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb44112b90 msgr2=0x7fbb441a34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:00.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb42ffd640 1 --2- 172.21.15.139:0/869928364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb44112b90 0x7fbb441a34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.844+0000 7fbb42ffd640 1 -- 172.21.15.139:0/869928364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb441a3bf0 con 0x7fbb44074590 2024-09-16T01:43:00.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.845+0000 7fbb42ffd640 1 --2- 172.21.15.139:0/869928364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb44074590 0x7fbb4419ebc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00ba50 tx=0x7fbb3c00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:00.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.845+0000 7fbb2bfff640 1 -- 172.21.15.139:0/869928364 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c019070 con 0x7fbb44074590 2024-09-16T01:43:00.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.845+0000 7fbb4965d640 1 -- 172.21.15.139:0/869928364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb441a3ee0 con 0x7fbb44074590 2024-09-16T01:43:00.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.845+0000 7fbb2bfff640 1 -- 172.21.15.139:0/869928364 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb3c00cbe0 con 0x7fbb44074590 2024-09-16T01:43:00.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.845+0000 7fbb2bfff640 1 -- 172.21.15.139:0/869928364 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c019070 con 0x7fbb44074590 2024-09-16T01:43:00.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.845+0000 7fbb4965d640 1 -- 172.21.15.139:0/869928364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb441ab770 con 0x7fbb44074590 2024-09-16T01:43:00.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.848+0000 7fbb4ba36740 1 Processor -- start 2024-09-16T01:43:00.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.848+0000 7fbb2bfff640 1 -- 172.21.15.139:0/869928364 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fbb3c01c070 con 0x7fbb44074590 2024-09-16T01:43:00.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.848+0000 7fbb4ba36740 1 -- start start 2024-09-16T01:43:00.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.848+0000 7fbb4ba36740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c51a850 0x55605c51ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.848+0000 7fbb437fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c51a850 0x55605c51ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb437fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c51a850 0x55605c51ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:59650/0 (socket says 172.21.15.139:59650) 2024-09-16T01:43:00.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb437fe640 1 -- 172.21.15.139:0/4285142284 learned_addr learned my addr 172.21.15.139:0/4285142284 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:00.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/4285142284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c5254f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/4285142284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c525a30 0x55605c527e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb4ba36740 1 -- 172.21.15.139:0/4285142284 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55605c47f160 con 0x55605c51b170 2024-09-16T01:43:00.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb4ba36740 1 -- 172.21.15.139:0/4285142284 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55605c426720 con 0x55605c525a30 2024-09-16T01:43:00.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb4ba36740 1 -- 172.21.15.139:0/4285142284 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55605c47b120 con 0x55605c51a850 2024-09-16T01:43:00.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb42ffd640 1 --2- 172.21.15.139:0/4285142284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c5254f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb427fc640 1 --2- 172.21.15.139:0/4285142284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c525a30 0x55605c527e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb437fe640 1 -- 172.21.15.139:0/4285142284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c525a30 msgr2=0x55605c527e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:00.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb437fe640 1 --2- 172.21.15.139:0/4285142284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c525a30 0x55605c527e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb437fe640 1 -- 172.21.15.139:0/4285142284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 msgr2=0x55605c5254f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:00.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb437fe640 1 --2- 172.21.15.139:0/4285142284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c5254f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb437fe640 1 -- 172.21.15.139:0/4285142284 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55605c1f0ba0 con 0x55605c51a850 2024-09-16T01:43:00.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb2bfff640 1 --2- 172.21.15.139:0/869928364 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbb2c0a37b0 0x7fbb2c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.849+0000 7fbb2bfff640 1 -- 172.21.15.139:0/869928364 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fbb3c00c9e0 con 0x7fbb44074590 2024-09-16T01:43:00.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.850+0000 7fbb42ffd640 1 --2- 172.21.15.139:0/869928364 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbb2c0a37b0 0x7fbb2c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.850+0000 7fbb437fe640 1 --2- 172.21.15.139:0/4285142284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c51a850 0x55605c51ac30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009c20 tx=0x7fbb34011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbe3d632267b3424 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:00.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.850+0000 7fbb297fa640 1 -- 172.21.15.139:0/4285142284 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3401e070 con 0x55605c51a850 2024-09-16T01:43:00.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.850+0000 7fbb297fa640 1 -- 172.21.15.139:0/4285142284 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb34019b10 con 0x55605c51a850 2024-09-16T01:43:00.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.850+0000 7fbb297fa640 1 -- 172.21.15.139:0/4285142284 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3400e9a0 con 0x55605c51a850 2024-09-16T01:43:00.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.851+0000 7fbb4ba36740 1 -- 172.21.15.139:0/4285142284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c51a850 msgr2=0x55605c51ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:00.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.851+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/4285142284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c51a850 0x55605c51ac30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009c20 tx=0x7fbb34011810 comp rx=0 tx=0).stop 2024-09-16T01:43:00.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.851+0000 7fbb42ffd640 1 --2- 172.21.15.139:0/869928364 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbb2c0a37b0 0x7fbb2c0a5c70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0cc5b0 tx=0x7fbb3c014d30 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:00.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.852+0000 7fbb4ba36740 1 -- 172.21.15.139:0/4285142284 shutdown_connections 2024-09-16T01:43:00.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.852+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/4285142284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c525a30 0x55605c527e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.852+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/4285142284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c5254f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.852+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/4285142284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c51a850 0x55605c51ac30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.852+0000 7fbb4ba36740 1 -- 172.21.15.139:0/4285142284 >> 172.21.15.139:0/4285142284 conn(0x55605c516de0 msgr2=0x55605c5171c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:00.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.852+0000 7fbb4ba36740 1 -- 172.21.15.139:0/4285142284 shutdown_connections 2024-09-16T01:43:00.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.852+0000 7fbb4ba36740 1 -- 172.21.15.139:0/4285142284 wait complete. 2024-09-16T01:43:00.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.852+0000 7fbb4ba36740 1 -- start start 2024-09-16T01:43:00.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.852+0000 7fbb4ba36740 1 Processor -- start 2024-09-16T01:43:00.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.853+0000 7fbb4ba36740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51a850 0x55605c522980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.853+0000 7fbb4ba36740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51b170 0x55605c522ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.853+0000 7fbb4ba36740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 0x55605c523400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.853+0000 7fbb4ba36740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55605c528600 con 0x55605c51a850 2024-09-16T01:43:00.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.853+0000 7fbb4ba36740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55605c407900 con 0x55605c51b170 2024-09-16T01:43:00.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.853+0000 7fbb4ba36740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55605c528780 con 0x55605c525a30 2024-09-16T01:43:00.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.853+0000 7fbb4ba36740 10 client.0 _ll_register_callbacks cb 0x55605c405780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:43:00.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.854+0000 7fbb427fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51b170 0x55605c522ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.854+0000 7fbb427fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51b170 0x55605c522ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:53310/0 (socket says 172.21.15.139:53310) 2024-09-16T01:43:00.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.854+0000 7fbb427fc640 1 -- 172.21.15.139:0/281232939 learned_addr learned my addr 172.21.15.139:0/281232939 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:00.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.854+0000 7fbb437fe640 1 --2- 172.21.15.139:0/281232939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51a850 0x55605c522980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.854+0000 7fbb437fe640 1 --2- 172.21.15.139:0/281232939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 0x55605c523400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.854+0000 7fbb437fe640 1 -- 172.21.15.139:0/281232939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 msgr2=0x55605c523400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:00.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.854+0000 7fbb437fe640 1 --2- 172.21.15.139:0/281232939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 0x55605c523400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.854+0000 7fbb437fe640 1 -- 172.21.15.139:0/281232939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51b170 msgr2=0x55605c522ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:00.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.854+0000 7fbb437fe640 1 --2- 172.21.15.139:0/281232939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51b170 0x55605c522ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.854+0000 7fbb437fe640 1 -- 172.21.15.139:0/281232939 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55605c447c50 con 0x55605c51a850 2024-09-16T01:43:00.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.854+0000 7fbb437fe640 1 --2- 172.21.15.139:0/281232939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51a850 0x55605c522980 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009bf0 tx=0x7fbb340096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8828940c1fb0c4e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:00.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.854+0000 7fbb26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:43:00.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.855+0000 7fbb26ff5640 1 -- 172.21.15.139:0/281232939 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3401e040 con 0x55605c51a850 2024-09-16T01:43:00.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.855+0000 7fbb26ff5640 1 -- 172.21.15.139:0/281232939 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fbb34019cc0 con 0x55605c51a850 2024-09-16T01:43:00.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.855+0000 7fbb4ba36740 1 -- 172.21.15.139:0/281232939 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb34019b10 con 0x55605c51a850 2024-09-16T01:43:00.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.856+0000 7fbb26ff5640 1 -- 172.21.15.139:0/281232939 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbb34035050 con 0x55605c51a850 2024-09-16T01:43:00.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.856+0000 7fbb4ba36740 10 client.15858 fetch_fsmap learned FSMap version 31 2024-09-16T01:43:00.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.856+0000 7fbb4ba36740 1 -- 172.21.15.139:0/281232939 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55605c534310 con 0x55605c51a850 2024-09-16T01:43:00.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.856+0000 7fbb26ff5640 1 -- 172.21.15.139:0/281232939 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb34005040 con 0x55605c51a850 2024-09-16T01:43:00.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.856+0000 7fbb26ff5640 1 -- 172.21.15.139:0/281232939 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7fbb340046c0 con 0x55605c51a850 2024-09-16T01:43:00.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.856+0000 7fbb4ba36740 10 client.15858 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:43:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.856+0000 7fbb4ba36740 10 client.15858 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:43:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.856+0000 7fbb4ba36740 20 client.15858 populate_metadata read hostname 'smithi139' 2024-09-16T01:43:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.856+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/281232939 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55605c535770 0x55605c537b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.856+0000 7fbb4ba36740 4 client.15858 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:43:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.856+0000 7fbb4ba36740 1 -- 172.21.15.139:0/281232939 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbb34019b10 con 0x55605c535770 2024-09-16T01:43:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.856+0000 7fbb427fc640 1 --2- 172.21.15.139:0/281232939 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55605c535770 0x55605c537b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.857+0000 7fbb26ff5640 10 client.15858 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.857+0000 7fbb427fc640 1 --2- 172.21.15.139:0/281232939 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55605c535770 0x55605c537b60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69cd25d58faae4fc server_cookie=f0bf362d7002babc in_seq=0 out_seq=0 2024-09-16T01:43:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.858+0000 7fbb26ff5640 1 -- 172.21.15.139:0/281232939 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fbb34019b10 con 0x55605c535770 2024-09-16T01:43:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.858+0000 7fbb26ff5640 10 client.15858 handle_command_reply: tid=0 2024-09-16T01:43:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.858+0000 7fbb4ba36740 1 client.15858 shutdown 2024-09-16T01:43:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.858+0000 7fbb4ba36740 1 -- 172.21.15.139:0/281232939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51a850 msgr2=0x55605c522980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.858+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/281232939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51a850 0x55605c522980 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009bf0 tx=0x7fbb340096d0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.858+0000 7fbb4ba36740 1 -- 172.21.15.139:0/281232939 shutdown_connections 2024-09-16T01:43:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.858+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/281232939 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55605c535770 0x55605c537b60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.859+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/281232939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 0x55605c523400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.859+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/281232939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51b170 0x55605c522ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.859+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/281232939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51a850 0x55605c522980 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.859+0000 7fbb26ff5640 0 client.15858 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.859+0000 7fbb4ba36740 1 -- 172.21.15.139:0/281232939 >> 172.21.15.139:0/281232939 conn(0x55605c516de0 msgr2=0x55605c51cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.859+0000 7fbb4ba36740 1 -- 172.21.15.139:0/281232939 shutdown_connections 2024-09-16T01:43:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.859+0000 7fbb4ba36740 1 -- 172.21.15.139:0/281232939 wait complete. 2024-09-16T01:43:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.859+0000 7fbb4ba36740 20 client.15858 trim_cache size 0 max 16384 2024-09-16T01:43:00.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.877+0000 7fbb4ba36740 1 Processor -- start 2024-09-16T01:43:00.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.877+0000 7fbb4ba36740 1 -- start start 2024-09-16T01:43:00.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.877+0000 7fbb4ba36740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51a850 0x55605c51d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.877+0000 7fbb4ba36740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c53d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.877+0000 7fbb4ba36740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 0x55605c53d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.877+0000 7fbb4ba36740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbb441b5820 con 0x55605c51b170 2024-09-16T01:43:00.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.877+0000 7fbb4ba36740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb441b7420 con 0x55605c51a850 2024-09-16T01:43:00.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.877+0000 7fbb4ba36740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbb441b5ef0 con 0x55605c525a30 2024-09-16T01:43:00.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.877+0000 7fbb427fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c53d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.877+0000 7fbb427fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c53d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58820/0 (socket says 172.21.15.139:58820) 2024-09-16T01:43:00.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.877+0000 7fbb427fc640 1 -- 172.21.15.139:0/4142276709 learned_addr learned my addr 172.21.15.139:0/4142276709 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:00.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.878+0000 7fbb437fe640 1 --2- 172.21.15.139:0/4142276709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51a850 0x55605c51d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.878+0000 7fbb437fe640 1 --2- 172.21.15.139:0/4142276709 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 0x55605c53d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.878+0000 7fbb427fc640 1 -- 172.21.15.139:0/4142276709 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 msgr2=0x55605c53d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:00.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.878+0000 7fbb427fc640 1 --2- 172.21.15.139:0/4142276709 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 0x55605c53d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.878+0000 7fbb427fc640 1 -- 172.21.15.139:0/4142276709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51a850 msgr2=0x55605c51d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:00.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.878+0000 7fbb427fc640 1 --2- 172.21.15.139:0/4142276709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51a850 0x55605c51d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.878+0000 7fbb427fc640 1 -- 172.21.15.139:0/4142276709 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55605c5123f0 con 0x55605c51b170 2024-09-16T01:43:00.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.878+0000 7fbb427fc640 1 --2- 172.21.15.139:0/4142276709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c53d310 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbb38006010 tx=0x7fbb380092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1eb4b00b567a2ec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:00.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.878+0000 7fbb28ff9640 1 -- 172.21.15.139:0/4142276709 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38035040 con 0x55605c51b170 2024-09-16T01:43:00.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.878+0000 7fbb28ff9640 1 -- 172.21.15.139:0/4142276709 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb3804a070 con 0x55605c51b170 2024-09-16T01:43:00.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.878+0000 7fbb28ff9640 1 -- 172.21.15.139:0/4142276709 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38031810 con 0x55605c51b170 2024-09-16T01:43:00.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.879+0000 7fbb4ba36740 1 -- 172.21.15.139:0/4142276709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 msgr2=0x55605c53d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:00.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.879+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/4142276709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c53d310 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbb38006010 tx=0x7fbb380092d0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.879+0000 7fbb4ba36740 1 -- 172.21.15.139:0/4142276709 shutdown_connections 2024-09-16T01:43:00.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.879+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/4142276709 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 0x55605c53d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.879+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/4142276709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c53d310 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.879+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/4142276709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51a850 0x55605c51d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.879+0000 7fbb4ba36740 1 -- 172.21.15.139:0/4142276709 >> 172.21.15.139:0/4142276709 conn(0x55605c516de0 msgr2=0x55605c531070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:00.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.879+0000 7fbb4ba36740 1 -- 172.21.15.139:0/4142276709 shutdown_connections 2024-09-16T01:43:00.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.879+0000 7fbb4ba36740 1 -- 172.21.15.139:0/4142276709 wait complete. 2024-09-16T01:43:00.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.879+0000 7fbb4ba36740 1 -- start start 2024-09-16T01:43:00.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.879+0000 7fbb4ba36740 1 Processor -- start 2024-09-16T01:43:00.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb4ba36740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51a850 0x55605c52e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb4ba36740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c51f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb4ba36740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 0x55605c51fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb4ba36740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbb441b6c90 con 0x55605c51b170 2024-09-16T01:43:00.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb4ba36740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb441b65c0 con 0x55605c51a850 2024-09-16T01:43:00.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb4ba36740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbb441b3800 con 0x55605c525a30 2024-09-16T01:43:00.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb4ba36740 10 client.0 _ll_register_callbacks cb 0x7fbb3401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:43:00.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb427fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c51f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb427fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c51f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58834/0 (socket says 172.21.15.139:58834) 2024-09-16T01:43:00.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb427fc640 1 -- 172.21.15.139:0/649625842 learned_addr learned my addr 172.21.15.139:0/649625842 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:00.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb437fe640 1 --2- 172.21.15.139:0/649625842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51a850 0x55605c52e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb437fe640 1 --2- 172.21.15.139:0/649625842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 0x55605c51fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb427fc640 1 -- 172.21.15.139:0/649625842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 msgr2=0x55605c51fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:00.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb427fc640 1 --2- 172.21.15.139:0/649625842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 0x55605c51fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb427fc640 1 -- 172.21.15.139:0/649625842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51a850 msgr2=0x55605c52e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:00.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb427fc640 1 --2- 172.21.15.139:0/649625842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51a850 0x55605c52e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:00.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb427fc640 1 -- 172.21.15.139:0/649625842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55605c520590 con 0x55605c51b170 2024-09-16T01:43:00.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb427fc640 1 --2- 172.21.15.139:0/649625842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c51f7c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009610 tx=0x7fbb38005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414394b472a26192 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:00.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:43:00.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb26ff5640 1 -- 172.21.15.139:0/649625842 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38035040 con 0x55605c51b170 2024-09-16T01:43:00.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb26ff5640 1 -- 172.21.15.139:0/649625842 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fbb38005040 con 0x55605c51b170 2024-09-16T01:43:00.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb4ba36740 1 -- 172.21.15.139:0/649625842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb3804a070 con 0x55605c51b170 2024-09-16T01:43:00.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb26ff5640 1 -- 172.21.15.139:0/649625842 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbb3805f050 con 0x55605c51b170 2024-09-16T01:43:00.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb4ba36740 10 client.15864 fetch_fsmap learned FSMap version 31 2024-09-16T01:43:00.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.880+0000 7fbb4ba36740 1 -- 172.21.15.139:0/649625842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55605c5a0300 con 0x55605c51b170 2024-09-16T01:43:00.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.881+0000 7fbb26ff5640 1 -- 172.21.15.139:0/649625842 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb3804a070 con 0x55605c51b170 2024-09-16T01:43:00.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.881+0000 7fbb26ff5640 1 -- 172.21.15.139:0/649625842 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7fbb38031c90 con 0x55605c51b170 2024-09-16T01:43:00.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.881+0000 7fbb4ba36740 10 client.15864 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:43:00.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.881+0000 7fbb4ba36740 10 client.15864 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:43:00.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.881+0000 7fbb4ba36740 20 client.15864 populate_metadata read hostname 'smithi139' 2024-09-16T01:43:00.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.881+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/649625842 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55605c5a0ad0 0x55605c5a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:00.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.881+0000 7fbb4ba36740 4 client.15864 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}] 2024-09-16T01:43:00.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.881+0000 7fbb4ba36740 1 -- 172.21.15.139:0/649625842 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}) -- 0x55605c48c890 con 0x55605c5a0ad0 2024-09-16T01:43:00.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.881+0000 7fbb437fe640 1 --2- 172.21.15.139:0/649625842 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55605c5a0ad0 0x55605c5a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:00.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.881+0000 7fbb26ff5640 10 client.15864 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:00.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:00.881+0000 7fbb437fe640 1 --2- 172.21.15.139:0/649625842 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55605c5a0ad0 0x55605c5a2ef0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c68b8292c82853b server_cookie=5edd14580d5292fd in_seq=0 out_seq=0 2024-09-16T01:43:01.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:01 smithi139 ceph-mon[31175]: pgmap v81: 129 pgs: 129 active+clean; 285 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 462 op/s 2024-09-16T01:43:01.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:01 smithi176 ceph-mon[35324]: pgmap v81: 129 pgs: 129 active+clean; 285 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 462 op/s 2024-09-16T01:43:02.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:01 smithi164 ceph-mon[35121]: pgmap v81: 129 pgs: 129 active+clean; 285 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 462 op/s 2024-09-16T01:43:03.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:03 smithi139 ceph-mon[31175]: pgmap v82: 129 pgs: 129 active+clean; 289 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 371 op/s 2024-09-16T01:43:03.954 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:03 smithi176 ceph-mon[35324]: pgmap v82: 129 pgs: 129 active+clean; 289 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 371 op/s 2024-09-16T01:43:04.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:03 smithi164 ceph-mon[35121]: pgmap v82: 129 pgs: 129 active+clean; 289 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 371 op/s 2024-09-16T01:43:05.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:05 smithi139 ceph-mon[31175]: pgmap v83: 129 pgs: 129 active+clean; 289 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 2.8 MiB/s wr, 158 op/s 2024-09-16T01:43:05.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:05 smithi176 ceph-mon[35324]: pgmap v83: 129 pgs: 129 active+clean; 289 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 2.8 MiB/s wr, 158 op/s 2024-09-16T01:43:05.967 INFO:tasks.workunit.client.0.smithi139.stdout: 105 216120 691 152486 626 45515 366 2024-09-16T01:43:06.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:05 smithi164 ceph-mon[35121]: pgmap v83: 129 pgs: 129 active+clean; 289 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 2.8 MiB/s wr, 158 op/s 2024-09-16T01:43:06.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.650+0000 7fbb26ff5640 1 -- 172.21.15.139:0/649625842 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55605c48c890 con 0x55605c5a0ad0 2024-09-16T01:43:06.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.650+0000 7fbb26ff5640 10 client.15864 handle_command_reply: tid=0 2024-09-16T01:43:06.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.650+0000 7fbb4ba36740 1 client.15864 shutdown 2024-09-16T01:43:06.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.650+0000 7fbb4ba36740 1 -- 172.21.15.139:0/649625842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 msgr2=0x55605c51f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:06.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.650+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/649625842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c51f7c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009610 tx=0x7fbb38005d10 comp rx=0 tx=0).stop 2024-09-16T01:43:06.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.651+0000 7fbb4ba36740 1 -- 172.21.15.139:0/649625842 shutdown_connections 2024-09-16T01:43:06.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.651+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/649625842 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55605c5a0ad0 0x55605c5a2ef0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:06.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.651+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/649625842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55605c525a30 0x55605c51fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:06.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.651+0000 7fbb26ff5640 0 client.15864 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:06.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.651+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/649625842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55605c51b170 0x55605c51f7c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:06.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.651+0000 7fbb4ba36740 1 --2- 172.21.15.139:0/649625842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55605c51a850 0x55605c52e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:06.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.651+0000 7fbb4ba36740 1 -- 172.21.15.139:0/649625842 >> 172.21.15.139:0/649625842 conn(0x55605c516de0 msgr2=0x55605c5241e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:06.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.651+0000 7fbb4ba36740 1 -- 172.21.15.139:0/649625842 shutdown_connections 2024-09-16T01:43:06.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.651+0000 7fbb4ba36740 1 -- 172.21.15.139:0/649625842 wait complete. 2024-09-16T01:43:06.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.651+0000 7fbb4ba36740 20 client.15864 trim_cache size 0 max 16384 2024-09-16T01:43:06.654 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:43:06.654 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 31, 2024-09-16T01:43:06.654 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:43:06.654 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 10, 2024-09-16T01:43:06.654 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:43:06.654 INFO:teuthology.orchestra.run.smithi139.stdout: "130f58d1": { 2024-09-16T01:43:06.654 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 10, 2024-09-16T01:43:06.654 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:43:06.655 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:43:06.655 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:43:06.655 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:43:06.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:43:06.655 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:43:06.655 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 139.6, 2024-09-16T01:43:06.655 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:43:06.655 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:43:06.655 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:43:06.655 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:43:06.655 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:43:06.655 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:43:06.655 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:43:06.655 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:43:06.656 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:43:06.656 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:43:06.656 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:43:06.656 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:43:06.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.652+0000 7fbb4965d640 1 -- 172.21.15.139:0/869928364 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbb2c0a37b0 msgr2=0x7fbb2c0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:06.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.652+0000 7fbb4965d640 1 --2- 172.21.15.139:0/869928364 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbb2c0a37b0 0x7fbb2c0a5c70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0cc5b0 tx=0x7fbb3c014d30 comp rx=0 tx=0).stop 2024-09-16T01:43:06.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.652+0000 7fbb4965d640 1 -- 172.21.15.139:0/869928364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb44074590 msgr2=0x7fbb4419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:06.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.652+0000 7fbb4965d640 1 --2- 172.21.15.139:0/869928364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb44074590 0x7fbb4419ebc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00ba50 tx=0x7fbb3c00bf10 comp rx=0 tx=0).stop 2024-09-16T01:43:06.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.652+0000 7fbb4965d640 1 -- 172.21.15.139:0/869928364 shutdown_connections 2024-09-16T01:43:06.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.652+0000 7fbb4965d640 1 --2- 172.21.15.139:0/869928364 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbb2c0a37b0 0x7fbb2c0a5c70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:06.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.652+0000 7fbb4965d640 1 --2- 172.21.15.139:0/869928364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb44112b90 0x7fbb441a34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:06.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.652+0000 7fbb4965d640 1 --2- 172.21.15.139:0/869928364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb44074eb0 0x7fbb4419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:06.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.652+0000 7fbb4965d640 1 --2- 172.21.15.139:0/869928364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb44074590 0x7fbb4419ebc0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00ba50 tx=0x7fbb3c00bf10 comp rx=0 tx=0).stop 2024-09-16T01:43:06.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.652+0000 7fbb4965d640 1 -- 172.21.15.139:0/869928364 >> 172.21.15.139:0/869928364 conn(0x7fbb440fd880 msgr2=0x7fbb440ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:06.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.652+0000 7fbb4965d640 1 -- 172.21.15.139:0/869928364 shutdown_connections 2024-09-16T01:43:06.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:06.652+0000 7fbb4965d640 1 -- 172.21.15.139:0/869928364 wait complete. 2024-09-16T01:43:07.479 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '130f58d1', quiesce took 9.6 seconds. Will release after: 9.950000000000001 2024-09-16T01:43:07.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:07 smithi139 ceph-mon[31175]: pgmap v84: 129 pgs: 129 active+clean; 296 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 330 op/s 2024-09-16T01:43:07.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:07 smithi176 ceph-mon[35324]: pgmap v84: 129 pgs: 129 active+clean; 296 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 330 op/s 2024-09-16T01:43:08.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:07 smithi164 ceph-mon[35121]: pgmap v84: 129 pgs: 129 active+clean; 296 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 330 op/s 2024-09-16T01:43:09.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:09 smithi139 ceph-mon[31175]: pgmap v85: 129 pgs: 129 active+clean; 296 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-09-16T01:43:09.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:09 smithi176 ceph-mon[35324]: pgmap v85: 129 pgs: 129 active+clean; 296 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-09-16T01:43:10.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:09 smithi164 ceph-mon[35121]: pgmap v85: 129 pgs: 129 active+clean; 296 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-09-16T01:43:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.281+0000 7fe7bc28d640 1 -- 172.21.15.139:0/2843340117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b41069c0 msgr2=0x7fe7b4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.281+0000 7fe7bc28d640 1 --2- 172.21.15.139:0/2843340117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b41069c0 0x7fe7b4106da0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009930 tx=0x7fe7b002a420 comp rx=0 tx=0).stop 2024-09-16T01:43:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.281+0000 7fe7bc28d640 1 -- 172.21.15.139:0/2843340117 shutdown_connections 2024-09-16T01:43:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.281+0000 7fe7bc28d640 1 --2- 172.21.15.139:0/2843340117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7b4101350 0x7fe7b410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.281+0000 7fe7bc28d640 1 --2- 172.21.15.139:0/2843340117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7b41009b0 0x7fe7b4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.281+0000 7fe7bc28d640 1 --2- 172.21.15.139:0/2843340117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b41069c0 0x7fe7b4106da0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.281+0000 7fe7bc28d640 1 -- 172.21.15.139:0/2843340117 >> 172.21.15.139:0/2843340117 conn(0x7fe7b4077990 msgr2=0x7fe7b40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.281+0000 7fe7bc28d640 1 -- 172.21.15.139:0/2843340117 shutdown_connections 2024-09-16T01:43:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.281+0000 7fe7bc28d640 1 -- 172.21.15.139:0/2843340117 wait complete. 2024-09-16T01:43:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.282+0000 7fe7bc28d640 1 Processor -- start 2024-09-16T01:43:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.282+0000 7fe7bc28d640 1 -- start start 2024-09-16T01:43:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.282+0000 7fe7bc28d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7b41009b0 0x7fe7b41991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.282+0000 7fe7bc28d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7b4101350 0x7fe7b4199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.282+0000 7fe7bc28d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b41069c0 0x7fe7b41a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.282+0000 7fe7bc28d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b4110dc0 con 0x7fe7b41069c0 2024-09-16T01:43:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.282+0000 7fe7bc28d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b4110c40 con 0x7fe7b41009b0 2024-09-16T01:43:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.282+0000 7fe7bc28d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b4110f40 con 0x7fe7b4101350 2024-09-16T01:43:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.282+0000 7fe7ba82a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b41069c0 0x7fe7b41a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.282+0000 7fe7ba029640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7b41009b0 0x7fe7b41991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.282+0000 7fe7ba82a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b41069c0 0x7fe7b41a4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52306/0 (socket says 172.21.15.139:52306) 2024-09-16T01:43:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.282+0000 7fe7ba029640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7b41009b0 0x7fe7b41991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:54174/0 (socket says 172.21.15.139:54174) 2024-09-16T01:43:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.282+0000 7fe7ba82a640 1 -- 172.21.15.139:0/1893583737 learned_addr learned my addr 172.21.15.139:0/1893583737 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.283+0000 7fe7b9828640 1 --2- 172.21.15.139:0/1893583737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7b4101350 0x7fe7b4199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.283+0000 7fe7ba82a640 1 -- 172.21.15.139:0/1893583737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7b4101350 msgr2=0x7fe7b4199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.283+0000 7fe7ba82a640 1 --2- 172.21.15.139:0/1893583737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7b4101350 0x7fe7b4199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.283+0000 7fe7ba82a640 1 -- 172.21.15.139:0/1893583737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7b41009b0 msgr2=0x7fe7b41991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.283+0000 7fe7ba82a640 1 --2- 172.21.15.139:0/1893583737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7b41009b0 0x7fe7b41991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.283+0000 7fe7ba82a640 1 -- 172.21.15.139:0/1893583737 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b41a4780 con 0x7fe7b41069c0 2024-09-16T01:43:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.283+0000 7fe7ba029640 1 --2- 172.21.15.139:0/1893583737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7b41009b0 0x7fe7b41991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:43:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.283+0000 7fe7ba82a640 1 --2- 172.21.15.139:0/1893583737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b41069c0 0x7fe7b41a4240 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00eb10 tx=0x7fe7ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.283+0000 7fe7ab7fe640 1 -- 172.21.15.139:0/1893583737 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac004b20 con 0x7fe7b41069c0 2024-09-16T01:43:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.283+0000 7fe7ab7fe640 1 -- 172.21.15.139:0/1893583737 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7ac004cc0 con 0x7fe7b41069c0 2024-09-16T01:43:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.283+0000 7fe7bc28d640 1 -- 172.21.15.139:0/1893583737 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b41a4950 con 0x7fe7b41069c0 2024-09-16T01:43:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.283+0000 7fe7ab7fe640 1 -- 172.21.15.139:0/1893583737 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac010760 con 0x7fe7b41069c0 2024-09-16T01:43:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.283+0000 7fe7bc28d640 1 -- 172.21.15.139:0/1893583737 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b41a4e20 con 0x7fe7b41069c0 2024-09-16T01:43:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.285+0000 7fe7ab7fe640 1 -- 172.21.15.139:0/1893583737 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe7ac005020 con 0x7fe7b41069c0 2024-09-16T01:43:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.286+0000 7fe7bf027740 1 Processor -- start 2024-09-16T01:43:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.286+0000 7fe7bf027740 1 -- start start 2024-09-16T01:43:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.286+0000 7fe7bf027740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaeb9faf0 0x55beaeb9fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.286+0000 7fe7bf027740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 0x55beaebaa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.286+0000 7fe7bf027740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaebaacb0 0x55beaebad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.286+0000 7fe7bf027740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55beaeb042f0 con 0x55beaeb9faf0 2024-09-16T01:43:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.286+0000 7fe7bf027740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55beaeaac270 con 0x55beaebaacb0 2024-09-16T01:43:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.286+0000 7fe7bf027740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55beaeb002b0 con 0x55beaeba0410 2024-09-16T01:43:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.286+0000 7fe7b9828640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaeb9faf0 0x55beaeb9fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.286+0000 7fe7ab7fe640 1 --2- 172.21.15.139:0/1893583737 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe7a00a3460 0x7fe7a00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.286+0000 7fe7b9828640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaeb9faf0 0x55beaeb9fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52314/0 (socket says 172.21.15.139:52314) 2024-09-16T01:43:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.286+0000 7fe7b9828640 1 -- 172.21.15.139:0/1103266159 learned_addr learned my addr 172.21.15.139:0/1103266159 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.286+0000 7fe7ba029640 1 --2- 172.21.15.139:0/1103266159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 0x55beaebaa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.286+0000 7fe7b9828640 1 --2- 172.21.15.139:0/1103266159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaebaacb0 0x55beaebad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.287+0000 7fe7ba82a640 1 --2- 172.21.15.139:0/1893583737 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe7a00a3460 0x7fe7a00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.287+0000 7fe7b9828640 1 -- 172.21.15.139:0/1103266159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 msgr2=0x55beaebaa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.287+0000 7fe7b9828640 1 --2- 172.21.15.139:0/1103266159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 0x55beaebaa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.287+0000 7fe7b9828640 1 -- 172.21.15.139:0/1103266159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaebaacb0 msgr2=0x55beaebad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.287+0000 7fe7b9828640 1 --2- 172.21.15.139:0/1103266159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaebaacb0 0x55beaebad0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.287+0000 7fe7b9828640 1 -- 172.21.15.139:0/1103266159 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beaeaab340 con 0x55beaeb9faf0 2024-09-16T01:43:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.287+0000 7fe7ab7fe640 1 -- 172.21.15.139:0/1893583737 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fe7ac0d0660 con 0x7fe7b41069c0 2024-09-16T01:43:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.287+0000 7fe7b9828640 1 --2- 172.21.15.139:0/1103266159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaeb9faf0 0x55beaeb9fed0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4013930 tx=0x7fe7a400b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfe8609ea2d5117 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.287+0000 7fe7ba82a640 1 --2- 172.21.15.139:0/1893583737 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe7a00a3460 0x7fe7a00a5920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0ce000 tx=0x7fe7ac06b280 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.287+0000 7fe7a8bfe640 1 -- 172.21.15.139:0/1103266159 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a400d8e0 con 0x55beaeb9faf0 2024-09-16T01:43:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.287+0000 7fe7a8bfe640 1 -- 172.21.15.139:0/1103266159 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7a4011400 con 0x55beaeb9faf0 2024-09-16T01:43:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.287+0000 7fe7a8bfe640 1 -- 172.21.15.139:0/1103266159 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a400f630 con 0x55beaeb9faf0 2024-09-16T01:43:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.288+0000 7fe7bf027740 1 -- 172.21.15.139:0/1103266159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaeb9faf0 msgr2=0x55beaeb9fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.288+0000 7fe7bf027740 1 --2- 172.21.15.139:0/1103266159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaeb9faf0 0x55beaeb9fed0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4013930 tx=0x7fe7a400b450 comp rx=0 tx=0).stop 2024-09-16T01:43:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.288+0000 7fe7bf027740 1 -- 172.21.15.139:0/1103266159 shutdown_connections 2024-09-16T01:43:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.288+0000 7fe7bf027740 1 --2- 172.21.15.139:0/1103266159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaebaacb0 0x55beaebad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.288+0000 7fe7bf027740 1 --2- 172.21.15.139:0/1103266159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 0x55beaebaa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.288+0000 7fe7bf027740 1 --2- 172.21.15.139:0/1103266159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaeb9faf0 0x55beaeb9fed0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.288+0000 7fe7bf027740 1 -- 172.21.15.139:0/1103266159 >> 172.21.15.139:0/1103266159 conn(0x55beaeb9c060 msgr2=0x55beaeb9c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.288+0000 7fe7bf027740 1 -- 172.21.15.139:0/1103266159 shutdown_connections 2024-09-16T01:43:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.288+0000 7fe7bf027740 1 -- 172.21.15.139:0/1103266159 wait complete. 2024-09-16T01:43:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.288+0000 7fe7bf027740 1 -- start start 2024-09-16T01:43:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.288+0000 7fe7bf027740 1 Processor -- start 2024-09-16T01:43:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7bf027740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaeb9faf0 0x55beaeba7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7bf027740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 0x55beaeba8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7bf027740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaebaacb0 0x55beaeba8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7bf027740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55beaebad880 con 0x55beaebaacb0 2024-09-16T01:43:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7bf027740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55beaea8d290 con 0x55beaeb9faf0 2024-09-16T01:43:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7bf027740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55beaebada00 con 0x55beaeba0410 2024-09-16T01:43:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7bf027740 10 client.0 _ll_register_callbacks cb 0x55beaeabe140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:43:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7b9828640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaebaacb0 0x55beaeba8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7b9828640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaebaacb0 0x55beaeba8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52322/0 (socket says 172.21.15.139:52322) 2024-09-16T01:43:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7b9828640 1 -- 172.21.15.139:0/1535885836 learned_addr learned my addr 172.21.15.139:0/1535885836 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7ba029640 1 --2- 172.21.15.139:0/1535885836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 0x55beaeba8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7b9828640 1 --2- 172.21.15.139:0/1535885836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaeb9faf0 0x55beaeba7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7b9828640 1 -- 172.21.15.139:0/1535885836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 msgr2=0x55beaeba8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7b9828640 1 --2- 172.21.15.139:0/1535885836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 0x55beaeba8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7b9828640 1 -- 172.21.15.139:0/1535885836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaeb9faf0 msgr2=0x55beaeba7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7b9828640 1 --2- 172.21.15.139:0/1535885836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaeb9faf0 0x55beaeba7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7b9828640 1 -- 172.21.15.139:0/1535885836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beae817210 con 0x55beaebaacb0 2024-09-16T01:43:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe79dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:43:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.289+0000 7fe7b9828640 1 --2- 172.21.15.139:0/1535885836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaebaacb0 0x55beaeba8550 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400b330 tx=0x7fe7a4011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35439c28fe74d8c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe79dffb640 1 -- 172.21.15.139:0/1535885836 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a40119f0 con 0x55beaebaacb0 2024-09-16T01:43:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe79dffb640 1 -- 172.21.15.139:0/1535885836 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fe7a4004020 con 0x55beaebaacb0 2024-09-16T01:43:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe7bf027740 1 -- 172.21.15.139:0/1535885836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7a4011400 con 0x55beaebaacb0 2024-09-16T01:43:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe79dffb640 1 -- 172.21.15.139:0/1535885836 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe7a4004220 con 0x55beaebaacb0 2024-09-16T01:43:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe7bf027740 10 client.15876 fetch_fsmap learned FSMap version 31 2024-09-16T01:43:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe7bf027740 1 -- 172.21.15.139:0/1535885836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55beae8e3620 con 0x55beaebaacb0 2024-09-16T01:43:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe79dffb640 1 -- 172.21.15.139:0/1535885836 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7a4011400 con 0x55beaebaacb0 2024-09-16T01:43:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe79dffb640 1 -- 172.21.15.139:0/1535885836 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7fe7a4003680 con 0x55beaebaacb0 2024-09-16T01:43:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe7bf027740 10 client.15876 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:43:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe7bf027740 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:43:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe7bf027740 20 client.15876 populate_metadata read hostname 'smithi139' 2024-09-16T01:43:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe7bf027740 1 --2- 172.21.15.139:0/1535885836 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55beaebbaa00 0x55beaebbcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe7bf027740 4 client.15876 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:43:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe7bf027740 1 -- 172.21.15.139:0/1535885836 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beaeb11a20 con 0x55beaebbaa00 2024-09-16T01:43:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.290+0000 7fe7ba029640 1 --2- 172.21.15.139:0/1535885836 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55beaebbaa00 0x55beaebbcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.291+0000 7fe79dffb640 10 client.15876 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.291+0000 7fe7ba029640 1 --2- 172.21.15.139:0/1535885836 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55beaebbaa00 0x55beaebbcdf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c94ddfe10a2dd0b server_cookie=b7727e229da6e6d5 in_seq=0 out_seq=0 2024-09-16T01:43:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe79dffb640 1 -- 172.21.15.139:0/1535885836 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55beaeb11a20 con 0x55beaebbaa00 2024-09-16T01:43:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe79dffb640 10 client.15876 handle_command_reply: tid=0 2024-09-16T01:43:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe7bf027740 1 client.15876 shutdown 2024-09-16T01:43:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe7bf027740 1 -- 172.21.15.139:0/1535885836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaebaacb0 msgr2=0x55beaeba8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe7bf027740 1 --2- 172.21.15.139:0/1535885836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaebaacb0 0x55beaeba8550 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400b330 tx=0x7fe7a4011da0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe7bf027740 1 -- 172.21.15.139:0/1535885836 shutdown_connections 2024-09-16T01:43:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe7bf027740 1 --2- 172.21.15.139:0/1535885836 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55beaebbaa00 0x55beaebbcdf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe7bf027740 1 --2- 172.21.15.139:0/1535885836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaebaacb0 0x55beaeba8550 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe7bf027740 1 --2- 172.21.15.139:0/1535885836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 0x55beaeba8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe79dffb640 0 client.15876 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe7bf027740 1 --2- 172.21.15.139:0/1535885836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaeb9faf0 0x55beaeba7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe7bf027740 1 -- 172.21.15.139:0/1535885836 >> 172.21.15.139:0/1535885836 conn(0x55beaeb9c060 msgr2=0x55beaeba1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe7bf027740 1 -- 172.21.15.139:0/1535885836 shutdown_connections 2024-09-16T01:43:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe7bf027740 1 -- 172.21.15.139:0/1535885836 wait complete. 2024-09-16T01:43:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.292+0000 7fe7bf027740 20 client.15876 trim_cache size 0 max 16384 2024-09-16T01:43:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.308+0000 7fe7bf027740 1 Processor -- start 2024-09-16T01:43:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.308+0000 7fe7bf027740 1 -- start start 2024-09-16T01:43:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.308+0000 7fe7bf027740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeb9faf0 0x55beaeba6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.308+0000 7fe7bf027740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaeba0410 0x55beaeba8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.308+0000 7fe7bf027740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaebaacb0 0x55beaebc25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.308+0000 7fe7bf027740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41af0a0 con 0x55beaeba0410 2024-09-16T01:43:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.308+0000 7fe7bf027740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41b0ca0 con 0x55beaebaacb0 2024-09-16T01:43:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.308+0000 7fe7bf027740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41af770 con 0x55beaeb9faf0 2024-09-16T01:43:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.308+0000 7fe7ba029640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaeba0410 0x55beaeba8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.308+0000 7fe7ba029640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaeba0410 0x55beaeba8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52330/0 (socket says 172.21.15.139:52330) 2024-09-16T01:43:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.308+0000 7fe7ba029640 1 -- 172.21.15.139:0/3503594087 learned_addr learned my addr 172.21.15.139:0/3503594087 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.308+0000 7fe7b9828640 1 --2- 172.21.15.139:0/3503594087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaebaacb0 0x55beaebc25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.308+0000 7fe7b9828640 1 --2- 172.21.15.139:0/3503594087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeb9faf0 0x55beaeba6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.309+0000 7fe7ba029640 1 -- 172.21.15.139:0/3503594087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeb9faf0 msgr2=0x55beaeba6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.309+0000 7fe7ba029640 1 --2- 172.21.15.139:0/3503594087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeb9faf0 0x55beaeba6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.309+0000 7fe7ba029640 1 -- 172.21.15.139:0/3503594087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaebaacb0 msgr2=0x55beaebc25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.309+0000 7fe7ba029640 1 --2- 172.21.15.139:0/3503594087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaebaacb0 0x55beaebc25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.309+0000 7fe7ba029640 1 -- 172.21.15.139:0/3503594087 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beaebbd380 con 0x55beaeba0410 2024-09-16T01:43:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.309+0000 7fe7b9828640 1 --2- 172.21.15.139:0/3503594087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeb9faf0 0x55beaeba6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:43:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.309+0000 7fe7ba029640 1 --2- 172.21.15.139:0/3503594087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaeba0410 0x55beaeba8380 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009850 tx=0x7fe7b0032970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3233e7e512e979a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.309+0000 7fe79ffff640 1 -- 172.21.15.139:0/3503594087 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0035040 con 0x55beaeba0410 2024-09-16T01:43:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.309+0000 7fe79ffff640 1 -- 172.21.15.139:0/3503594087 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b0005520 con 0x55beaeba0410 2024-09-16T01:43:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.309+0000 7fe79ffff640 1 -- 172.21.15.139:0/3503594087 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0004bb0 con 0x55beaeba0410 2024-09-16T01:43:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 -- 172.21.15.139:0/3503594087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaeba0410 msgr2=0x55beaeba8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 --2- 172.21.15.139:0/3503594087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaeba0410 0x55beaeba8380 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009850 tx=0x7fe7b0032970 comp rx=0 tx=0).stop 2024-09-16T01:43:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 -- 172.21.15.139:0/3503594087 shutdown_connections 2024-09-16T01:43:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 --2- 172.21.15.139:0/3503594087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaebaacb0 0x55beaebc25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 --2- 172.21.15.139:0/3503594087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaeba0410 0x55beaeba8380 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 --2- 172.21.15.139:0/3503594087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeb9faf0 0x55beaeba6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 -- 172.21.15.139:0/3503594087 >> 172.21.15.139:0/3503594087 conn(0x55beaeb9c060 msgr2=0x55beaeba9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 -- 172.21.15.139:0/3503594087 shutdown_connections 2024-09-16T01:43:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 -- 172.21.15.139:0/3503594087 wait complete. 2024-09-16T01:43:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 -- start start 2024-09-16T01:43:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 Processor -- start 2024-09-16T01:43:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaeb9faf0 0x55beaebb3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 0x55beaebabcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaebaacb0 0x55beaebac410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41b0510 con 0x55beaebaacb0 2024-09-16T01:43:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41afe40 con 0x55beaeb9faf0 2024-09-16T01:43:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41ad080 con 0x55beaeba0410 2024-09-16T01:43:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.310+0000 7fe7bf027740 10 client.0 _ll_register_callbacks cb 0x7fe7a400be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:43:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe7b9828640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaebaacb0 0x55beaebac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe7b9828640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaebaacb0 0x55beaebac410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52340/0 (socket says 172.21.15.139:52340) 2024-09-16T01:43:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe7b9828640 1 -- 172.21.15.139:0/32707472 learned_addr learned my addr 172.21.15.139:0/32707472 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe7b9828640 1 --2- 172.21.15.139:0/32707472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaeb9faf0 0x55beaebb3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe7ba029640 1 --2- 172.21.15.139:0/32707472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 0x55beaebabcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe7b9828640 1 -- 172.21.15.139:0/32707472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 msgr2=0x55beaebabcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe7b9828640 1 --2- 172.21.15.139:0/32707472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 0x55beaebabcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe7b9828640 1 -- 172.21.15.139:0/32707472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaeb9faf0 msgr2=0x55beaebb3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe7b9828640 1 --2- 172.21.15.139:0/32707472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaeb9faf0 0x55beaebb3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe7b9828640 1 -- 172.21.15.139:0/32707472 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beaeb972d0 con 0x55beaebaacb0 2024-09-16T01:43:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe7b9828640 1 --2- 172.21.15.139:0/32707472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaebaacb0 0x55beaebac410 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4011ef0 tx=0x7fe7a400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0f410f6ab52202 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe79dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:43:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe79dffb640 1 -- 172.21.15.139:0/32707472 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4036070 con 0x55beaebaacb0 2024-09-16T01:43:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe7bf027740 1 -- 172.21.15.139:0/32707472 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b0005520 con 0x55beaebaacb0 2024-09-16T01:43:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe79dffb640 1 -- 172.21.15.139:0/32707472 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fe7a4013500 con 0x55beaebaacb0 2024-09-16T01:43:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe79dffb640 1 -- 172.21.15.139:0/32707472 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe7a4043050 con 0x55beaebaacb0 2024-09-16T01:43:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe7bf027740 10 client.15882 fetch_fsmap learned FSMap version 31 2024-09-16T01:43:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.311+0000 7fe7bf027740 1 -- 172.21.15.139:0/32707472 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55beaeb9be90 con 0x55beaebaacb0 2024-09-16T01:43:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.312+0000 7fe79dffb640 1 -- 172.21.15.139:0/32707472 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b0005520 con 0x55beaebaacb0 2024-09-16T01:43:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.312+0000 7fe79dffb640 1 -- 172.21.15.139:0/32707472 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7fe7a400db90 con 0x55beaebaacb0 2024-09-16T01:43:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.312+0000 7fe7bf027740 10 client.15882 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:43:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.312+0000 7fe7bf027740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:43:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.312+0000 7fe7bf027740 20 client.15882 populate_metadata read hostname 'smithi139' 2024-09-16T01:43:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.312+0000 7fe7bf027740 1 --2- 172.21.15.139:0/32707472 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55beaec25a50 0x55beaec27e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.312+0000 7fe7bf027740 4 client.15882 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:43:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.312+0000 7fe7bf027740 1 -- 172.21.15.139:0/32707472 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55beaeb0bd70 con 0x55beaec25a50 2024-09-16T01:43:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.312+0000 7fe7ba029640 1 --2- 172.21.15.139:0/32707472 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55beaec25a50 0x55beaec27e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.312+0000 7fe79dffb640 10 client.15882 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:10.312+0000 7fe7ba029640 1 --2- 172.21.15.139:0/32707472 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55beaec25a50 0x55beaec27e70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d319fa01507471f6 server_cookie=54f168a716571300 in_seq=0 out_seq=0 2024-09-16T01:43:11.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:11 smithi139 ceph-mon[31175]: pgmap v86: 129 pgs: 129 active+clean; 302 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 251 op/s 2024-09-16T01:43:11.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:11 smithi176 ceph-mon[35324]: pgmap v86: 129 pgs: 129 active+clean; 302 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 251 op/s 2024-09-16T01:43:12.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:11 smithi164 ceph-mon[35121]: pgmap v86: 129 pgs: 129 active+clean; 302 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 251 op/s 2024-09-16T01:43:13.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:13 smithi139 ceph-mon[31175]: pgmap v87: 129 pgs: 129 active+clean; 304 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 254 op/s 2024-09-16T01:43:13.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:13 smithi176 ceph-mon[35324]: pgmap v87: 129 pgs: 129 active+clean; 304 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 254 op/s 2024-09-16T01:43:14.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:13 smithi164 ceph-mon[35121]: pgmap v87: 129 pgs: 129 active+clean; 304 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 254 op/s 2024-09-16T01:43:15.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:15 smithi139 ceph-mon[31175]: pgmap v88: 129 pgs: 129 active+clean; 304 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 232 op/s 2024-09-16T01:43:15.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:15 smithi176 ceph-mon[35324]: pgmap v88: 129 pgs: 129 active+clean; 304 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 232 op/s 2024-09-16T01:43:15.967 INFO:tasks.workunit.client.0.smithi139.stdout: 105 145832 0 105341 0 25867 0 2024-09-16T01:43:16.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:15 smithi164 ceph-mon[35121]: pgmap v88: 129 pgs: 129 active+clean; 304 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 232 op/s 2024-09-16T01:43:17.430 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:43:17.683 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:43:17.804 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:17 smithi139 ceph-mon[31175]: pgmap v89: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 441 op/s 2024-09-16T01:43:17.846 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:17 smithi164 ceph-mon[35121]: pgmap v89: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 441 op/s 2024-09-16T01:43:17.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:17 smithi176 ceph-mon[35324]: pgmap v89: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 441 op/s 2024-09-16T01:43:18.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.170+0000 7f4248423640 1 -- 172.21.15.139:0/1521866091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4240102d30 msgr2=0x7f4240103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:18.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.170+0000 7f4248423640 1 --2- 172.21.15.139:0/1521866091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4240102d30 0x7f4240103110 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f423c009930 tx=0x7f423c02a420 comp rx=0 tx=0).stop 2024-09-16T01:43:18.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.170+0000 7f4248423640 1 -- 172.21.15.139:0/1521866091 shutdown_connections 2024-09-16T01:43:18.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.170+0000 7f4248423640 1 --2- 172.21.15.139:0/1521866091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42401082a0 0x7f424010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:18.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.170+0000 7f4248423640 1 --2- 172.21.15.139:0/1521866091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42401036e0 0x7f4240107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:18.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.170+0000 7f4248423640 1 --2- 172.21.15.139:0/1521866091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4240102d30 0x7f4240103110 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:18.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.170+0000 7f4248423640 1 -- 172.21.15.139:0/1521866091 >> 172.21.15.139:0/1521866091 conn(0x7f42400fc7d0 msgr2=0x7f42400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:18.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.170+0000 7f4248423640 1 -- 172.21.15.139:0/1521866091 shutdown_connections 2024-09-16T01:43:18.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.170+0000 7f4248423640 1 -- 172.21.15.139:0/1521866091 wait complete. 2024-09-16T01:43:18.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.170+0000 7f4248423640 1 Processor -- start 2024-09-16T01:43:18.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.170+0000 7f4248423640 1 -- start start 2024-09-16T01:43:18.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f4248423640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4240102d30 0x7f42401985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:18.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f4248423640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42401036e0 0x7f4240198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:18.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f4248423640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42401082a0 0x7f424019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:18.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f4248423640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42401120a0 con 0x7f42401082a0 2024-09-16T01:43:18.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f4248423640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4240111f20 con 0x7f42401036e0 2024-09-16T01:43:18.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f4248423640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4240112220 con 0x7f4240102d30 2024-09-16T01:43:18.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42469c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42401082a0 0x7f424019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:18.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42469c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42401082a0 0x7f424019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47462/0 (socket says 172.21.15.139:47462) 2024-09-16T01:43:18.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42469c0640 1 -- 172.21.15.139:0/3619915595 learned_addr learned my addr 172.21.15.139:0/3619915595 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:18.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42461bf640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4240102d30 0x7f42401985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:18.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42459be640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42401036e0 0x7f4240198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:18.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42469c0640 1 -- 172.21.15.139:0/3619915595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4240102d30 msgr2=0x7f42401985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:18.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42469c0640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4240102d30 0x7f42401985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:18.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42469c0640 1 -- 172.21.15.139:0/3619915595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42401036e0 msgr2=0x7f4240198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:18.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42469c0640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42401036e0 0x7f4240198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:18.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42469c0640 1 -- 172.21.15.139:0/3619915595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424019d570 con 0x7f42401082a0 2024-09-16T01:43:18.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42459be640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42401036e0 0x7f4240198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:43:18.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42469c0640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42401082a0 0x7f424019ce90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f423800eaa0 tx=0x7f423800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:18.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42337fe640 1 -- 172.21.15.139:0/3619915595 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423800cdb0 con 0x7f42401082a0 2024-09-16T01:43:18.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42337fe640 1 -- 172.21.15.139:0/3619915595 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4238005020 con 0x7f42401082a0 2024-09-16T01:43:18.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f42337fe640 1 -- 172.21.15.139:0/3619915595 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380116b0 con 0x7f42401082a0 2024-09-16T01:43:18.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.171+0000 7f4248423640 1 -- 172.21.15.139:0/3619915595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424019d860 con 0x7f42401082a0 2024-09-16T01:43:18.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.172+0000 7f4248423640 1 -- 172.21.15.139:0/3619915595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42401a5140 con 0x7f42401082a0 2024-09-16T01:43:18.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.173+0000 7f4248423640 1 -- 172.21.15.139:0/3619915595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f424006a850 con 0x7f42401082a0 2024-09-16T01:43:18.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.174+0000 7f42337fe640 1 -- 172.21.15.139:0/3619915595 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4238005560 con 0x7f42401082a0 2024-09-16T01:43:18.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.175+0000 7f42337fe640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f422c0a33b0 0x7f422c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:18.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.175+0000 7f42461bf640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f422c0a33b0 0x7f422c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:18.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.175+0000 7f42337fe640 1 -- 172.21.15.139:0/3619915595 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f42380d0330 con 0x7f42401082a0 2024-09-16T01:43:18.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.175+0000 7f42461bf640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f422c0a33b0 0x7f422c0a5870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f423c000c00 tx=0x7f423c005a20 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:18.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.179+0000 7f42337fe640 1 -- 172.21.15.139:0/3619915595 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4238016030 con 0x7f42401082a0 2024-09-16T01:43:18.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.389+0000 7f4248423640 1 -- 172.21.15.139:0/3619915595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f424010a2b0 con 0x7f42401082a0 2024-09-16T01:43:18.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.390+0000 7f42337fe640 1 -- 172.21.15.139:0/3619915595 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12002 (secure 0 0 0) 0x7f423809c960 con 0x7f42401082a0 2024-09-16T01:43:18.392 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:43:18.392 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":31,"btime":"2024-09-16T01:42:06:037308+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:42:05.468117+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:43:18.392 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 31 2024-09-16T01:43:18.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.392+0000 7f4248423640 1 -- 172.21.15.139:0/3619915595 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f422c0a33b0 msgr2=0x7f422c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:18.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.392+0000 7f4248423640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f422c0a33b0 0x7f422c0a5870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f423c000c00 tx=0x7f423c005a20 comp rx=0 tx=0).stop 2024-09-16T01:43:18.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.392+0000 7f4248423640 1 -- 172.21.15.139:0/3619915595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42401082a0 msgr2=0x7f424019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:18.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.392+0000 7f4248423640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42401082a0 0x7f424019ce90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f423800eaa0 tx=0x7f423800ef60 comp rx=0 tx=0).stop 2024-09-16T01:43:18.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.392+0000 7f4248423640 1 -- 172.21.15.139:0/3619915595 shutdown_connections 2024-09-16T01:43:18.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.392+0000 7f4248423640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f422c0a33b0 0x7f422c0a5870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:18.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.392+0000 7f4248423640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42401082a0 0x7f424019ce90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:18.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.392+0000 7f4248423640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42401036e0 0x7f4240198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:18.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.392+0000 7f4248423640 1 --2- 172.21.15.139:0/3619915595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4240102d30 0x7f42401985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:18.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.392+0000 7f4248423640 1 -- 172.21.15.139:0/3619915595 >> 172.21.15.139:0/3619915595 conn(0x7f42400fc7d0 msgr2=0x7f4240105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:18.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.392+0000 7f4248423640 1 -- 172.21.15.139:0/3619915595 shutdown_connections 2024-09-16T01:43:18.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:18.392+0000 7f4248423640 1 -- 172.21.15.139:0/3619915595 wait complete. 2024-09-16T01:43:18.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:18 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3619915595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:43:18.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:18 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3619915595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:43:19.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:18 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3619915595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:43:19.223 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 130f58d1 --release --await' 2024-09-16T01:43:19.223 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 130f58d1 --release --await 2024-09-16T01:43:19.478 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:43:19.900 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:19 smithi139 ceph-mon[31175]: pgmap v90: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 961 KiB/s wr, 270 op/s 2024-09-16T01:43:19.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:19 smithi176 ceph-mon[35324]: pgmap v90: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 961 KiB/s wr, 270 op/s 2024-09-16T01:43:20.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.984+0000 7fbdad9f0640 1 -- 172.21.15.139:0/1937183976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbda81082a0 msgr2=0x7fbda810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:20.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.984+0000 7fbdad9f0640 1 --2- 172.21.15.139:0/1937183976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbda81082a0 0x7fbda810ec40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009cd0 tx=0x7fbda002a8e0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.985+0000 7fbdad9f0640 1 -- 172.21.15.139:0/1937183976 shutdown_connections 2024-09-16T01:43:20.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.985+0000 7fbdad9f0640 1 --2- 172.21.15.139:0/1937183976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbda81082a0 0x7fbda810ec40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.985+0000 7fbdad9f0640 1 --2- 172.21.15.139:0/1937183976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbda81036e0 0x7fbda8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.985+0000 7fbdad9f0640 1 --2- 172.21.15.139:0/1937183976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbda8102d30 0x7fbda8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.985+0000 7fbdad9f0640 1 -- 172.21.15.139:0/1937183976 >> 172.21.15.139:0/1937183976 conn(0x7fbda80fc7d0 msgr2=0x7fbda80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:20.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.985+0000 7fbdad9f0640 1 -- 172.21.15.139:0/1937183976 shutdown_connections 2024-09-16T01:43:20.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.985+0000 7fbdad9f0640 1 -- 172.21.15.139:0/1937183976 wait complete. 2024-09-16T01:43:20.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.985+0000 7fbdad9f0640 1 Processor -- start 2024-09-16T01:43:20.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.985+0000 7fbdad9f0640 1 -- start start 2024-09-16T01:43:20.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbdad9f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbda8102d30 0x7fbda81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbdad9f0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbda81036e0 0x7fbda8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbdad9f0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbda81082a0 0x7fbda819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbdad9f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbda8112100 con 0x7fbda8102d30 2024-09-16T01:43:20.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbdad9f0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbda8111f80 con 0x7fbda81036e0 2024-09-16T01:43:20.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbdad9f0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbda8112280 con 0x7fbda81082a0 2024-09-16T01:43:20.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbda6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbda8102d30 0x7fbda81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbda6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbda8102d30 0x7fbda81985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47466/0 (socket says 172.21.15.139:47466) 2024-09-16T01:43:20.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbda6ffd640 1 -- 172.21.15.139:0/149721602 learned_addr learned my addr 172.21.15.139:0/149721602 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:20.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbd9e7fc640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbda81036e0 0x7fbda8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbda77fe640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbda81082a0 0x7fbda819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbda6ffd640 1 -- 172.21.15.139:0/149721602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbda81082a0 msgr2=0x7fbda819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:20.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbda6ffd640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbda81082a0 0x7fbda819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbda6ffd640 1 -- 172.21.15.139:0/149721602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbda81036e0 msgr2=0x7fbda8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:20.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbda6ffd640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbda81036e0 0x7fbda8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbda6ffd640 1 -- 172.21.15.139:0/149721602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda819d560 con 0x7fbda8102d30 2024-09-16T01:43:20.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbda77fe640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbda81082a0 0x7fbda819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:43:20.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbd9e7fc640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbda81036e0 0x7fbda8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:43:20.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbda6ffd640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbda8102d30 0x7fbda81985b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800b4a0 tx=0x7fbd9800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:20.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbdac9ee640 1 -- 172.21.15.139:0/149721602 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98014070 con 0x7fbda8102d30 2024-09-16T01:43:20.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbdac9ee640 1 -- 172.21.15.139:0/149721602 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd98010af0 con 0x7fbda8102d30 2024-09-16T01:43:20.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbdac9ee640 1 -- 172.21.15.139:0/149721602 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98017d10 con 0x7fbda8102d30 2024-09-16T01:43:20.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbdad9f0640 1 -- 172.21.15.139:0/149721602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda819d850 con 0x7fbda8102d30 2024-09-16T01:43:20.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.986+0000 7fbdad9f0640 1 -- 172.21.15.139:0/149721602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda81a5130 con 0x7fbda8102d30 2024-09-16T01:43:20.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.989+0000 7fbdac9ee640 1 -- 172.21.15.139:0/149721602 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fbd98017070 con 0x7fbda8102d30 2024-09-16T01:43:20.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.989+0000 7fbdafdc9740 1 Processor -- start 2024-09-16T01:43:20.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.989+0000 7fbdafdc9740 1 -- start start 2024-09-16T01:43:20.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.989+0000 7fbdac9ee640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbd940a33b0 0x7fbd940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.989+0000 7fbdafdc9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561850581060 0x561850581440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.989+0000 7fbdafdc9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581980 0x56185058bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.989+0000 7fbdafdc9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185058c240 0x56185058e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.989+0000 7fbdafdc9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5618504e6620 con 0x56185058c240 2024-09-16T01:43:20.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.989+0000 7fbdafdc9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56185048de50 con 0x561850581980 2024-09-16T01:43:20.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.989+0000 7fbdafdc9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5618504e25e0 con 0x561850581060 2024-09-16T01:43:20.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.989+0000 7fbda77fe640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbd940a33b0 0x7fbd940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.989+0000 7fbdac9ee640 1 -- 172.21.15.139:0/149721602 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fbd980cf510 con 0x7fbda8102d30 2024-09-16T01:43:20.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbda77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185058c240 0x56185058e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbda6ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561850581060 0x561850581440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbd9e7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581980 0x56185058bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbda77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185058c240 0x56185058e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47470/0 (socket says 172.21.15.139:47470) 2024-09-16T01:43:20.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbda6ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561850581060 0x561850581440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:33912/0 (socket says 172.21.15.139:33912) 2024-09-16T01:43:20.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbda77fe640 1 -- 172.21.15.139:0/3035064481 learned_addr learned my addr 172.21.15.139:0/3035064481 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:20.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbda77fe640 1 -- 172.21.15.139:0/3035064481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561850581060 msgr2=0x561850581440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:20.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbda77fe640 1 --2- 172.21.15.139:0/3035064481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561850581060 0x561850581440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbda77fe640 1 -- 172.21.15.139:0/3035064481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581980 msgr2=0x56185058bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:20.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbda77fe640 1 --2- 172.21.15.139:0/3035064481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581980 0x56185058bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbda77fe640 1 -- 172.21.15.139:0/3035064481 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56185047ed40 con 0x56185058c240 2024-09-16T01:43:20.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbda6ffd640 1 --2- 172.21.15.139:0/3035064481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561850581060 0x561850581440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:43:20.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbda77fe640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbd940a33b0 0x7fbd940a5870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009cd0 tx=0x7fbda0031430 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:20.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbda77fe640 1 --2- 172.21.15.139:0/3035064481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185058c240 0x56185058e630 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009a50 tx=0x7fbda0047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faaea28c53acb563 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:20.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbd9dffb640 1 -- 172.21.15.139:0/3035064481 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda004d070 con 0x56185058c240 2024-09-16T01:43:20.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbd9dffb640 1 -- 172.21.15.139:0/3035064481 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda0036040 con 0x56185058c240 2024-09-16T01:43:20.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.990+0000 7fbd9dffb640 1 -- 172.21.15.139:0/3035064481 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda00308e0 con 0x56185058c240 2024-09-16T01:43:20.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.991+0000 7fbdafdc9740 1 -- 172.21.15.139:0/3035064481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185058c240 msgr2=0x56185058e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:20.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.991+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/3035064481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185058c240 0x56185058e630 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009a50 tx=0x7fbda0047000 comp rx=0 tx=0).stop 2024-09-16T01:43:20.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.991+0000 7fbdafdc9740 1 -- 172.21.15.139:0/3035064481 shutdown_connections 2024-09-16T01:43:20.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.991+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/3035064481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185058c240 0x56185058e630 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.991+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/3035064481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581980 0x56185058bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.991+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/3035064481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561850581060 0x561850581440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.991+0000 7fbdafdc9740 1 -- 172.21.15.139:0/3035064481 >> 172.21.15.139:0/3035064481 conn(0x56185057d5f0 msgr2=0x56185057d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:20.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.991+0000 7fbdafdc9740 1 -- 172.21.15.139:0/3035064481 shutdown_connections 2024-09-16T01:43:20.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.991+0000 7fbdafdc9740 1 -- 172.21.15.139:0/3035064481 wait complete. 2024-09-16T01:43:20.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.991+0000 7fbdafdc9740 1 -- start start 2024-09-16T01:43:20.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbdafdc9740 1 Processor -- start 2024-09-16T01:43:20.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbdafdc9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581060 0x561850589150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbdafdc9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561850581980 0x561850589690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbdafdc9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56185058c240 0x561850589bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbdafdc9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56185058ee10 con 0x561850581060 2024-09-16T01:43:20.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbdafdc9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56185046ef10 con 0x56185058c240 2024-09-16T01:43:20.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbdafdc9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56185058ef90 con 0x561850581980 2024-09-16T01:43:20.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbdafdc9740 10 client.0 _ll_register_callbacks cb 0x561850492db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:43:20.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbda6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581060 0x561850589150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbda6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581060 0x561850589150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47484/0 (socket says 172.21.15.139:47484) 2024-09-16T01:43:20.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbda6ffd640 1 -- 172.21.15.139:0/3682132311 learned_addr learned my addr 172.21.15.139:0/3682132311 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:20.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbda77fe640 1 --2- 172.21.15.139:0/3682132311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56185058c240 0x561850589bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbd9e7fc640 1 --2- 172.21.15.139:0/3682132311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561850581980 0x561850589690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbda6ffd640 1 -- 172.21.15.139:0/3682132311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561850581980 msgr2=0x561850589690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:20.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbda6ffd640 1 --2- 172.21.15.139:0/3682132311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561850581980 0x561850589690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbda6ffd640 1 -- 172.21.15.139:0/3682132311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56185058c240 msgr2=0x561850589bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:20.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbda6ffd640 1 --2- 172.21.15.139:0/3682132311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56185058c240 0x561850589bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbda6ffd640 1 -- 172.21.15.139:0/3682132311 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618501f5580 con 0x561850581060 2024-09-16T01:43:20.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbda6ffd640 1 --2- 172.21.15.139:0/3682132311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581060 0x561850589150 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbd9809b1f0 tx=0x7fbd98078ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ab51de506b3d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:20.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbd86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:43:20.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.992+0000 7fbd86ffd640 1 -- 172.21.15.139:0/3682132311 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9801e040 con 0x561850581060 2024-09-16T01:43:20.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.993+0000 7fbd86ffd640 1 -- 172.21.15.139:0/3682132311 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fbd98097b60 con 0x561850581060 2024-09-16T01:43:20.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.993+0000 7fbdafdc9740 1 -- 172.21.15.139:0/3682132311 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbda0036040 con 0x561850581060 2024-09-16T01:43:20.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.993+0000 7fbd86ffd640 1 -- 172.21.15.139:0/3682132311 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbd980784c0 con 0x561850581060 2024-09-16T01:43:20.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.993+0000 7fbdafdc9740 10 client.15900 fetch_fsmap learned FSMap version 31 2024-09-16T01:43:20.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.993+0000 7fbdafdc9740 1 -- 172.21.15.139:0/3682132311 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5618502bf960 con 0x561850581060 2024-09-16T01:43:20.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.993+0000 7fbd86ffd640 1 -- 172.21.15.139:0/3682132311 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda0036040 con 0x561850581060 2024-09-16T01:43:20.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.993+0000 7fbd86ffd640 1 -- 172.21.15.139:0/3682132311 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7fbd98010ca0 con 0x561850581060 2024-09-16T01:43:20.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.993+0000 7fbdafdc9740 10 client.15900 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:43:20.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.993+0000 7fbdafdc9740 10 client.15900 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:43:20.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.993+0000 7fbdafdc9740 20 client.15900 populate_metadata read hostname 'smithi139' 2024-09-16T01:43:20.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.993+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/3682132311 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56185059beb0 0x56185059e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.993+0000 7fbdafdc9740 4 client.15900 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:43:20.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.993+0000 7fbdafdc9740 1 -- 172.21.15.139:0/3682132311 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618504f3b90 con 0x56185059beb0 2024-09-16T01:43:20.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.993+0000 7fbd9e7fc640 1 --2- 172.21.15.139:0/3682132311 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56185059beb0 0x56185059e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.994+0000 7fbd86ffd640 10 client.15900 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:20.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.994+0000 7fbd9e7fc640 1 --2- 172.21.15.139:0/3682132311 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56185059beb0 0x56185059e2d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9b429d75f802a78 server_cookie=23f1359752266ad0 in_seq=0 out_seq=0 2024-09-16T01:43:20.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.994+0000 7fbd86ffd640 1 -- 172.21.15.139:0/3682132311 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618504f3b90 con 0x56185059beb0 2024-09-16T01:43:20.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.994+0000 7fbd86ffd640 10 client.15900 handle_command_reply: tid=0 2024-09-16T01:43:20.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.994+0000 7fbdafdc9740 1 client.15900 shutdown 2024-09-16T01:43:20.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.995+0000 7fbdafdc9740 1 -- 172.21.15.139:0/3682132311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581060 msgr2=0x561850589150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:20.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.995+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/3682132311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581060 0x561850589150 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbd9809b1f0 tx=0x7fbd98078ba0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.995+0000 7fbdafdc9740 1 -- 172.21.15.139:0/3682132311 shutdown_connections 2024-09-16T01:43:20.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.995+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/3682132311 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56185059beb0 0x56185059e2d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.995+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/3682132311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56185058c240 0x561850589bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.995+0000 7fbd86ffd640 0 client.15900 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:20.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.995+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/3682132311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561850581980 0x561850589690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.995+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/3682132311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581060 0x561850589150 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.995+0000 7fbdafdc9740 1 -- 172.21.15.139:0/3682132311 >> 172.21.15.139:0/3682132311 conn(0x56185057d5f0 msgr2=0x561850583320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:20.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.995+0000 7fbdafdc9740 1 -- 172.21.15.139:0/3682132311 shutdown_connections 2024-09-16T01:43:20.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.995+0000 7fbdafdc9740 1 -- 172.21.15.139:0/3682132311 wait complete. 2024-09-16T01:43:20.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:19.995+0000 7fbdafdc9740 20 client.15900 trim_cache size 0 max 16384 2024-09-16T01:43:20.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbdafdc9740 1 Processor -- start 2024-09-16T01:43:20.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbdafdc9740 1 -- start start 2024-09-16T01:43:20.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbdafdc9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581060 0x561850594610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbdafdc9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581980 0x561850594b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbdafdc9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56185058c240 0x5618505a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbdafdc9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81af0c0 con 0x561850581980 2024-09-16T01:43:20.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbdafdc9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81b0cc0 con 0x561850581060 2024-09-16T01:43:20.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbdafdc9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81af790 con 0x56185058c240 2024-09-16T01:43:20.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbd9e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581980 0x561850594b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbd9e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581980 0x561850594b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47494/0 (socket says 172.21.15.139:47494) 2024-09-16T01:43:20.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbd9e7fc640 1 -- 172.21.15.139:0/1994282251 learned_addr learned my addr 172.21.15.139:0/1994282251 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:20.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbda6ffd640 1 --2- 172.21.15.139:0/1994282251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581060 0x561850594610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbda77fe640 1 --2- 172.21.15.139:0/1994282251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56185058c240 0x5618505a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbd9e7fc640 1 -- 172.21.15.139:0/1994282251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56185058c240 msgr2=0x5618505a3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:20.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbd9e7fc640 1 --2- 172.21.15.139:0/1994282251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56185058c240 0x5618505a3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbd9e7fc640 1 -- 172.21.15.139:0/1994282251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581060 msgr2=0x561850594610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:20.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbd9e7fc640 1 --2- 172.21.15.139:0/1994282251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581060 0x561850594610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbd9e7fc640 1 -- 172.21.15.139:0/1994282251 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561850588100 con 0x561850581980 2024-09-16T01:43:20.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbda77fe640 1 --2- 172.21.15.139:0/1994282251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56185058c240 0x5618505a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:43:20.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbda6ffd640 1 --2- 172.21.15.139:0/1994282251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581060 0x561850594610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:43:20.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.012+0000 7fbd9e7fc640 1 --2- 172.21.15.139:0/1994282251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581980 0x561850594b50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbda806a370 tx=0x7fbd90010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2633f97e23b242e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:20.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.013+0000 7fbd9d7fa640 1 -- 172.21.15.139:0/1994282251 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90021030 con 0x561850581980 2024-09-16T01:43:20.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.013+0000 7fbd9d7fa640 1 -- 172.21.15.139:0/1994282251 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd9001c630 con 0x561850581980 2024-09-16T01:43:20.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.013+0000 7fbd9d7fa640 1 -- 172.21.15.139:0/1994282251 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90003a10 con 0x561850581980 2024-09-16T01:43:20.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.013+0000 7fbdafdc9740 1 -- 172.21.15.139:0/1994282251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581980 msgr2=0x561850594b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:20.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.013+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/1994282251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581980 0x561850594b50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbda806a370 tx=0x7fbd90010800 comp rx=0 tx=0).stop 2024-09-16T01:43:20.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.013+0000 7fbdafdc9740 1 -- 172.21.15.139:0/1994282251 shutdown_connections 2024-09-16T01:43:20.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.013+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/1994282251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56185058c240 0x5618505a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.013+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/1994282251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581980 0x561850594b50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.013+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/1994282251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581060 0x561850594610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.013+0000 7fbdafdc9740 1 -- 172.21.15.139:0/1994282251 >> 172.21.15.139:0/1994282251 conn(0x56185057d5f0 msgr2=0x56185058a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:20.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.013+0000 7fbdafdc9740 1 -- 172.21.15.139:0/1994282251 shutdown_connections 2024-09-16T01:43:20.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.013+0000 7fbdafdc9740 1 -- 172.21.15.139:0/1994282251 wait complete. 2024-09-16T01:43:20.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.014+0000 7fbdafdc9740 1 -- start start 2024-09-16T01:43:20.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.014+0000 7fbdafdc9740 1 Processor -- start 2024-09-16T01:43:20.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.014+0000 7fbdafdc9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581060 0x5618506046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.014+0000 7fbdafdc9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581980 0x561850604c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.014+0000 7fbdafdc9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56185058c240 0x5618505feb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.014+0000 7fbdafdc9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81b0530 con 0x561850581980 2024-09-16T01:43:20.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.014+0000 7fbdafdc9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81afe60 con 0x561850581060 2024-09-16T01:43:20.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.014+0000 7fbdafdc9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81ad0a0 con 0x56185058c240 2024-09-16T01:43:20.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.014+0000 7fbdafdc9740 10 client.0 _ll_register_callbacks cb 0x7fbd9809aee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:43:20.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.014+0000 7fbd9e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581980 0x561850604c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.014+0000 7fbd9e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581980 0x561850604c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47508/0 (socket says 172.21.15.139:47508) 2024-09-16T01:43:20.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.014+0000 7fbd9e7fc640 1 -- 172.21.15.139:0/507271327 learned_addr learned my addr 172.21.15.139:0/507271327 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:20.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.014+0000 7fbda6ffd640 1 --2- 172.21.15.139:0/507271327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581060 0x5618506046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.015+0000 7fbd9e7fc640 1 -- 172.21.15.139:0/507271327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56185058c240 msgr2=0x5618505feb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:20.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.015+0000 7fbd9e7fc640 1 --2- 172.21.15.139:0/507271327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56185058c240 0x5618505feb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.015+0000 7fbd9e7fc640 1 -- 172.21.15.139:0/507271327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581060 msgr2=0x5618506046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:20.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.015+0000 7fbd9e7fc640 1 --2- 172.21.15.139:0/507271327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581060 0x5618506046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.015+0000 7fbd9e7fc640 1 -- 172.21.15.139:0/507271327 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618505ff270 con 0x561850581980 2024-09-16T01:43:20.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.015+0000 7fbd9e7fc640 1 --2- 172.21.15.139:0/507271327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581980 0x561850604c10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbd90008b70 tx=0x7fbd900042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0687b82d56a2cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:20.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.015+0000 7fbd86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:43:20.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.015+0000 7fbd86ffd640 1 -- 172.21.15.139:0/507271327 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9001cbf0 con 0x561850581980 2024-09-16T01:43:20.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.015+0000 7fbdafdc9740 1 -- 172.21.15.139:0/507271327 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd9001c630 con 0x561850581980 2024-09-16T01:43:20.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.015+0000 7fbd86ffd640 1 -- 172.21.15.139:0/507271327 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fbd90003690 con 0x561850581980 2024-09-16T01:43:20.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.015+0000 7fbd86ffd640 1 -- 172.21.15.139:0/507271327 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbd90038050 con 0x561850581980 2024-09-16T01:43:20.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.015+0000 7fbdafdc9740 10 client.15906 fetch_fsmap learned FSMap version 31 2024-09-16T01:43:20.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.015+0000 7fbdafdc9740 1 -- 172.21.15.139:0/507271327 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56185060bec0 con 0x561850581980 2024-09-16T01:43:20.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.015+0000 7fbd86ffd640 1 -- 172.21.15.139:0/507271327 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd9001c630 con 0x561850581980 2024-09-16T01:43:20.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.016+0000 7fbd86ffd640 1 -- 172.21.15.139:0/507271327 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7fbd900269b0 con 0x561850581980 2024-09-16T01:43:20.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.016+0000 7fbdafdc9740 10 client.15906 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:43:20.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.016+0000 7fbdafdc9740 10 client.15906 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:43:20.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.016+0000 7fbdafdc9740 20 client.15906 populate_metadata read hostname 'smithi139' 2024-09-16T01:43:20.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.016+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/507271327 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56185059beb0 0x56185060e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:20.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.016+0000 7fbdafdc9740 4 client.15906 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "130f58d1", "release": true, "await": true}] 2024-09-16T01:43:20.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.016+0000 7fbdafdc9740 1 -- 172.21.15.139:0/507271327 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "130f58d1", "release": true, "await": true}) -- 0x5618504ee0a0 con 0x56185059beb0 2024-09-16T01:43:20.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.016+0000 7fbda6ffd640 1 --2- 172.21.15.139:0/507271327 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56185059beb0 0x56185060e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:20.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.016+0000 7fbd86ffd640 10 client.15906 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:20.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.016+0000 7fbda6ffd640 1 --2- 172.21.15.139:0/507271327 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56185059beb0 0x56185060e710 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1d951871d8c69a4 server_cookie=7249096a68eff5c2 in_seq=0 out_seq=0 2024-09-16T01:43:20.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:19 smithi164 ceph-mon[35121]: pgmap v90: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 961 KiB/s wr, 270 op/s 2024-09-16T01:43:20.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbd86ffd640 1 -- 172.21.15.139:0/507271327 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5618504ee0a0 con 0x56185059beb0 2024-09-16T01:43:20.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbd86ffd640 10 client.15906 handle_command_reply: tid=0 2024-09-16T01:43:20.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbdafdc9740 1 client.15906 shutdown 2024-09-16T01:43:20.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbdafdc9740 1 -- 172.21.15.139:0/507271327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581980 msgr2=0x561850604c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:20.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/507271327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581980 0x561850604c10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbd90008b70 tx=0x7fbd900042d0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbdafdc9740 1 -- 172.21.15.139:0/507271327 shutdown_connections 2024-09-16T01:43:20.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/507271327 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56185059beb0 0x56185060e710 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/507271327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56185058c240 0x5618505feb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/507271327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561850581980 0x561850604c10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbdafdc9740 1 --2- 172.21.15.139:0/507271327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561850581060 0x5618506046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbdafdc9740 1 -- 172.21.15.139:0/507271327 >> 172.21.15.139:0/507271327 conn(0x56185057d5f0 msgr2=0x56185057bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:20.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbd86ffd640 0 client.15906 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:20.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbdafdc9740 1 -- 172.21.15.139:0/507271327 shutdown_connections 2024-09-16T01:43:20.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbdafdc9740 1 -- 172.21.15.139:0/507271327 wait complete. 2024-09-16T01:43:20.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.295+0000 7fbdafdc9740 20 client.15906 trim_cache size 0 max 16384 2024-09-16T01:43:20.299 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:43:20.299 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 31, 2024-09-16T01:43:20.299 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:43:20.299 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 12, 2024-09-16T01:43:20.299 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:43:20.299 INFO:teuthology.orchestra.run.smithi139.stdout: "130f58d1": { 2024-09-16T01:43:20.299 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 12, 2024-09-16T01:43:20.299 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:43:20.299 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:43:20.299 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:43:20.300 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:43:20.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:43:20.300 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:43:20.300 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 139.6, 2024-09-16T01:43:20.300 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:43:20.300 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:43:20.300 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:43:20.300 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:43:20.300 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:43:20.300 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:43:20.300 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:43:20.300 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:43:20.300 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:43:20.301 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:43:20.301 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:43:20.301 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:43:20.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.296+0000 7fbdad9f0640 1 -- 172.21.15.139:0/149721602 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbd940a33b0 msgr2=0x7fbd940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:20.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.296+0000 7fbdad9f0640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbd940a33b0 0x7fbd940a5870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009cd0 tx=0x7fbda0031430 comp rx=0 tx=0).stop 2024-09-16T01:43:20.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.296+0000 7fbdad9f0640 1 -- 172.21.15.139:0/149721602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbda8102d30 msgr2=0x7fbda81985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:20.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.296+0000 7fbdad9f0640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbda8102d30 0x7fbda81985b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800b4a0 tx=0x7fbd9800b960 comp rx=0 tx=0).stop 2024-09-16T01:43:20.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.296+0000 7fbdad9f0640 1 -- 172.21.15.139:0/149721602 shutdown_connections 2024-09-16T01:43:20.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.296+0000 7fbdad9f0640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbd940a33b0 0x7fbd940a5870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.296+0000 7fbdad9f0640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbda81082a0 0x7fbda819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.296+0000 7fbdad9f0640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbda81036e0 0x7fbda8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.296+0000 7fbdad9f0640 1 --2- 172.21.15.139:0/149721602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbda8102d30 0x7fbda81985b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.296+0000 7fbdad9f0640 1 -- 172.21.15.139:0/149721602 >> 172.21.15.139:0/149721602 conn(0x7fbda80fc7d0 msgr2=0x7fbda80fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:20.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.296+0000 7fbdad9f0640 1 -- 172.21.15.139:0/149721602 shutdown_connections 2024-09-16T01:43:20.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:43:20.296+0000 7fbdad9f0640 1 -- 172.21.15.139:0/149721602 wait complete. 2024-09-16T01:43:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.787+0000 7fe79dffb640 1 -- 172.21.15.139:0/32707472 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55beaeb0bd70 con 0x55beaec25a50 2024-09-16T01:43:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.787+0000 7fe79dffb640 10 client.15882 handle_command_reply: tid=0 2024-09-16T01:43:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.787+0000 7fe7bf027740 1 client.15882 shutdown 2024-09-16T01:43:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.788+0000 7fe7bf027740 1 -- 172.21.15.139:0/32707472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaebaacb0 msgr2=0x55beaebac410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.788+0000 7fe7bf027740 1 --2- 172.21.15.139:0/32707472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaebaacb0 0x55beaebac410 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4011ef0 tx=0x7fe7a400b990 comp rx=0 tx=0).stop 2024-09-16T01:43:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.788+0000 7fe7bf027740 1 -- 172.21.15.139:0/32707472 shutdown_connections 2024-09-16T01:43:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.788+0000 7fe7bf027740 1 --2- 172.21.15.139:0/32707472 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55beaec25a50 0x55beaec27e70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.788+0000 7fe7bf027740 1 --2- 172.21.15.139:0/32707472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55beaebaacb0 0x55beaebac410 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.788+0000 7fe7bf027740 1 --2- 172.21.15.139:0/32707472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beaeba0410 0x55beaebabcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.788+0000 7fe79dffb640 0 client.15882 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.788+0000 7fe7bf027740 1 --2- 172.21.15.139:0/32707472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55beaeb9faf0 0x55beaebb3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.788+0000 7fe7bf027740 1 -- 172.21.15.139:0/32707472 >> 172.21.15.139:0/32707472 conn(0x55beaeb9c060 msgr2=0x55beaeb9a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.788+0000 7fe7bf027740 1 -- 172.21.15.139:0/32707472 shutdown_connections 2024-09-16T01:43:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.788+0000 7fe7bf027740 1 -- 172.21.15.139:0/32707472 wait complete. 2024-09-16T01:43:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.788+0000 7fe7bf027740 20 client.15882 trim_cache size 0 max 16384 2024-09-16T01:43:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:43:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:43:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:43:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:43:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.822+0000 7fe7bc28d640 1 -- 172.21.15.139:0/1893583737 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe7a00a3460 msgr2=0x7fe7a00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.822+0000 7fe7bc28d640 1 --2- 172.21.15.139:0/1893583737 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe7a00a3460 0x7fe7a00a5920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0ce000 tx=0x7fe7ac06b280 comp rx=0 tx=0).stop 2024-09-16T01:43:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.822+0000 7fe7bc28d640 1 -- 172.21.15.139:0/1893583737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b41069c0 msgr2=0x7fe7b41a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.822+0000 7fe7bc28d640 1 --2- 172.21.15.139:0/1893583737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b41069c0 0x7fe7b41a4240 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00eb10 tx=0x7fe7ac00c2d0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.822+0000 7fe7bc28d640 1 -- 172.21.15.139:0/1893583737 shutdown_connections 2024-09-16T01:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.823+0000 7fe7bc28d640 1 --2- 172.21.15.139:0/1893583737 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe7a00a3460 0x7fe7a00a5920 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.823+0000 7fe7bc28d640 1 --2- 172.21.15.139:0/1893583737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b41069c0 0x7fe7b41a4240 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.823+0000 7fe7bc28d640 1 --2- 172.21.15.139:0/1893583737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7b4101350 0x7fe7b4199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.823+0000 7fe7bc28d640 1 --2- 172.21.15.139:0/1893583737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7b41009b0 0x7fe7b41991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.823+0000 7fe7bc28d640 1 -- 172.21.15.139:0/1893583737 >> 172.21.15.139:0/1893583737 conn(0x7fe7b4077990 msgr2=0x7fe7b4103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.823+0000 7fe7bc28d640 1 -- 172.21.15.139:0/1893583737 shutdown_connections 2024-09-16T01:43:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:20.823+0000 7fe7bc28d640 1 -- 172.21.15.139:0/1893583737 wait complete. 2024-09-16T01:43:21.161 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '130f58d1', total seconds elapsed: 23.2 2024-09-16T01:43:21.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:21 smithi139 ceph-mon[31175]: pgmap v91: 129 pgs: 129 active+clean; 309 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.3 MiB/s wr, 289 op/s 2024-09-16T01:43:21.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:21 smithi176 ceph-mon[35324]: pgmap v91: 129 pgs: 129 active+clean; 309 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.3 MiB/s wr, 289 op/s 2024-09-16T01:43:22.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:21 smithi164 ceph-mon[35121]: pgmap v91: 129 pgs: 129 active+clean; 309 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.3 MiB/s wr, 289 op/s 2024-09-16T01:43:23.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:23 smithi139 ceph-mon[31175]: pgmap v92: 129 pgs: 129 active+clean; 314 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.2 MiB/s wr, 284 op/s 2024-09-16T01:43:23.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:23 smithi176 ceph-mon[35324]: pgmap v92: 129 pgs: 129 active+clean; 314 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.2 MiB/s wr, 284 op/s 2024-09-16T01:43:24.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:23 smithi164 ceph-mon[35121]: pgmap v92: 129 pgs: 129 active+clean; 314 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.2 MiB/s wr, 284 op/s 2024-09-16T01:43:25.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:25 smithi139 ceph-mon[31175]: pgmap v93: 129 pgs: 129 active+clean; 314 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.1 MiB/s wr, 248 op/s 2024-09-16T01:43:25.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:25 smithi176 ceph-mon[35324]: pgmap v93: 129 pgs: 129 active+clean; 314 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.1 MiB/s wr, 248 op/s 2024-09-16T01:43:25.967 INFO:tasks.workunit.client.0.smithi139.stdout: 119 694382 755 498114 565 132830 385 2024-09-16T01:43:26.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:25 smithi164 ceph-mon[35121]: pgmap v93: 129 pgs: 129 active+clean; 314 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.1 MiB/s wr, 248 op/s 2024-09-16T01:43:27.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:27 smithi139 ceph-mon[31175]: pgmap v94: 129 pgs: 129 active+clean; 310 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 320 op/s 2024-09-16T01:43:27.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:27 smithi176 ceph-mon[35324]: pgmap v94: 129 pgs: 129 active+clean; 310 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 320 op/s 2024-09-16T01:43:28.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:27 smithi164 ceph-mon[35121]: pgmap v94: 129 pgs: 129 active+clean; 310 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 320 op/s 2024-09-16T01:43:29.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:29 smithi139 ceph-mon[31175]: pgmap v95: 129 pgs: 129 active+clean; 310 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.9 MiB/s wr, 110 op/s 2024-09-16T01:43:29.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:29 smithi176 ceph-mon[35324]: pgmap v95: 129 pgs: 129 active+clean; 310 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.9 MiB/s wr, 110 op/s 2024-09-16T01:43:30.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:29 smithi164 ceph-mon[35121]: pgmap v95: 129 pgs: 129 active+clean; 310 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.9 MiB/s wr, 110 op/s 2024-09-16T01:43:30.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:30 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:43:30.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:30 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:43:30.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:30 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:43:30.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:30 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:43:30.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:30 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:43:30.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:30 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:43:30.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:30 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:43:30.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:30 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:43:31.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:30 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:43:31.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:30 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:43:31.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:30 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:43:31.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:30 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:43:31.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:31 smithi139 ceph-mon[31175]: pgmap v96: 129 pgs: 129 active+clean; 317 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 148 op/s 2024-09-16T01:43:31.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:31 smithi176 ceph-mon[35324]: pgmap v96: 129 pgs: 129 active+clean; 317 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 148 op/s 2024-09-16T01:43:32.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:31 smithi164 ceph-mon[35121]: pgmap v96: 129 pgs: 129 active+clean; 317 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 148 op/s 2024-09-16T01:43:33.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:33 smithi139 ceph-mon[31175]: pgmap v97: 129 pgs: 129 active+clean; 320 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-09-16T01:43:33.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:33 smithi176 ceph-mon[35324]: pgmap v97: 129 pgs: 129 active+clean; 320 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-09-16T01:43:34.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:33 smithi164 ceph-mon[35121]: pgmap v97: 129 pgs: 129 active+clean; 320 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-09-16T01:43:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.802+0000 7f039f578640 1 -- 172.21.15.139:0/4289018323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0398109b70 msgr2=0x7f039807ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.802+0000 7f039f578640 1 --2- 172.21.15.139:0/4289018323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0398109b70 0x7f039807ae80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0394008410 tx=0x7f039402a120 comp rx=0 tx=0).stop 2024-09-16T01:43:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.806+0000 7f039f578640 1 -- 172.21.15.139:0/4289018323 shutdown_connections 2024-09-16T01:43:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.806+0000 7f039f578640 1 --2- 172.21.15.139:0/4289018323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0398109b70 0x7f039807ae80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.806+0000 7f039f578640 1 --2- 172.21.15.139:0/4289018323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03981091f0 0x7f0398109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.806+0000 7f039f578640 1 --2- 172.21.15.139:0/4289018323 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f039810d610 0x7f039810d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.806+0000 7f039f578640 1 -- 172.21.15.139:0/4289018323 >> 172.21.15.139:0/4289018323 conn(0x7f039806cc60 msgr2=0x7f039806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.807+0000 7f039f578640 1 -- 172.21.15.139:0/4289018323 shutdown_connections 2024-09-16T01:43:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.807+0000 7f039f578640 1 -- 172.21.15.139:0/4289018323 wait complete. 2024-09-16T01:43:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.808+0000 7f039f578640 1 Processor -- start 2024-09-16T01:43:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.808+0000 7f039f578640 1 -- start start 2024-09-16T01:43:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.808+0000 7f039f578640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03981091f0 0x7f039807a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.808+0000 7f039f578640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0398109b70 0x7f039807ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.808+0000 7f039f578640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f039810d610 0x7f0398075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.808+0000 7f039f578640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f039807e4a0 con 0x7f03981091f0 2024-09-16T01:43:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.808+0000 7f039f578640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f039807e320 con 0x7f0398109b70 2024-09-16T01:43:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.808+0000 7f039f578640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f039807e620 con 0x7f039810d610 2024-09-16T01:43:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.809+0000 7f039d314640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03981091f0 0x7f039807a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.809+0000 7f039d314640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03981091f0 0x7f039807a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40004/0 (socket says 172.21.15.139:40004) 2024-09-16T01:43:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.809+0000 7f039d314640 1 -- 172.21.15.139:0/1109682674 learned_addr learned my addr 172.21.15.139:0/1109682674 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.809+0000 7f039d314640 1 -- 172.21.15.139:0/1109682674 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f039810d610 msgr2=0x7f0398075b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.809+0000 7f039d314640 1 --2- 172.21.15.139:0/1109682674 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f039810d610 0x7f0398075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.809+0000 7f039d314640 1 -- 172.21.15.139:0/1109682674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0398109b70 msgr2=0x7f039807ae20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:43:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.809+0000 7f039d314640 1 --2- 172.21.15.139:0/1109682674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0398109b70 0x7f039807ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.809+0000 7f039d314640 1 -- 172.21.15.139:0/1109682674 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0398076350 con 0x7f03981091f0 2024-09-16T01:43:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.810+0000 7f039d314640 1 --2- 172.21.15.139:0/1109682674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03981091f0 0x7f039807a8e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f038800da90 tx=0x7f038800df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.824+0000 7f038e7fc640 1 -- 172.21.15.139:0/1109682674 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388017070 con 0x7f03981091f0 2024-09-16T01:43:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.824+0000 7f038e7fc640 1 -- 172.21.15.139:0/1109682674 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0388009c00 con 0x7f03981091f0 2024-09-16T01:43:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.824+0000 7f039f578640 1 -- 172.21.15.139:0/1109682674 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0398076640 con 0x7f03981091f0 2024-09-16T01:43:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.824+0000 7f039f578640 1 -- 172.21.15.139:0/1109682674 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0398130480 con 0x7f03981091f0 2024-09-16T01:43:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.828+0000 7f03a23ee740 1 Processor -- start 2024-09-16T01:43:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.829+0000 7f03a23ee740 1 -- start start 2024-09-16T01:43:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.829+0000 7f03a23ee740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fd9cc0 0x55d207fda0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.829+0000 7f03a23ee740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fda5e0 0x55d207fe4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.829+0000 7f03a23ee740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fe4e80 0x55d207fe7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.829+0000 7f03a23ee740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d207f3e2f0 con 0x55d207fe4e80 2024-09-16T01:43:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.829+0000 7f03a23ee740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d207ee6270 con 0x55d207fd9cc0 2024-09-16T01:43:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.829+0000 7f03a23ee740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d207f3a2b0 con 0x55d207fda5e0 2024-09-16T01:43:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.829+0000 7f039db15640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fd9cc0 0x55d207fda0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.829+0000 7f039db15640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fd9cc0 0x55d207fda0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37142/0 (socket says 172.21.15.139:37142) 2024-09-16T01:43:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.829+0000 7f039db15640 1 -- 172.21.15.139:0/3017262800 learned_addr learned my addr 172.21.15.139:0/3017262800 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.829+0000 7f038e7fc640 1 -- 172.21.15.139:0/1109682674 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038800f620 con 0x7f03981091f0 2024-09-16T01:43:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.830+0000 7f039d314640 1 --2- 172.21.15.139:0/3017262800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fda5e0 0x55d207fe4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.830+0000 7f039db15640 1 -- 172.21.15.139:0/3017262800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fda5e0 msgr2=0x55d207fe4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.830+0000 7f039db15640 1 --2- 172.21.15.139:0/3017262800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fda5e0 0x55d207fe4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.830+0000 7f039db15640 1 -- 172.21.15.139:0/3017262800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fe4e80 msgr2=0x55d207fe7270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:43:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.830+0000 7f039db15640 1 --2- 172.21.15.139:0/3017262800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fe4e80 0x55d207fe7270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.830+0000 7f039db15640 1 -- 172.21.15.139:0/3017262800 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d207ee5340 con 0x55d207fd9cc0 2024-09-16T01:43:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.830+0000 7f039db15640 1 --2- 172.21.15.139:0/3017262800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fd9cc0 0x55d207fda0a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f039402a690 tx=0x7f039400aee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc5cdf207909f5e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.835+0000 7f038e7fc640 1 -- 172.21.15.139:0/1109682674 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f038800fa20 con 0x7f03981091f0 2024-09-16T01:43:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.836+0000 7f038e7fc640 1 --2- 172.21.15.139:0/1109682674 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f03840a35c0 0x7f03840a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.837+0000 7f039d314640 1 --2- 172.21.15.139:0/1109682674 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f03840a35c0 0x7f03840a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.838+0000 7f039d314640 1 --2- 172.21.15.139:0/1109682674 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f03840a35c0 0x7f03840a5a80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55d207fd7090 tx=0x7f0388000bc0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.838+0000 7f038e7fc640 1 -- 172.21.15.139:0/1109682674 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f03880d6830 con 0x7f03981091f0 2024-09-16T01:43:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.841+0000 7f03837fe640 1 -- 172.21.15.139:0/3017262800 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039402ac60 con 0x55d207fd9cc0 2024-09-16T01:43:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.841+0000 7f03837fe640 1 -- 172.21.15.139:0/3017262800 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0394040070 con 0x55d207fd9cc0 2024-09-16T01:43:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.841+0000 7f03837fe640 1 -- 172.21.15.139:0/3017262800 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03940042a0 con 0x55d207fd9cc0 2024-09-16T01:43:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.843+0000 7f03a23ee740 1 -- 172.21.15.139:0/3017262800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fd9cc0 msgr2=0x55d207fda0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.843+0000 7f03a23ee740 1 --2- 172.21.15.139:0/3017262800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fd9cc0 0x55d207fda0a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f039402a690 tx=0x7f039400aee0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.846+0000 7f03a23ee740 1 -- 172.21.15.139:0/3017262800 shutdown_connections 2024-09-16T01:43:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.846+0000 7f03a23ee740 1 --2- 172.21.15.139:0/3017262800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fe4e80 0x55d207fe7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.846+0000 7f03a23ee740 1 --2- 172.21.15.139:0/3017262800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fda5e0 0x55d207fe4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.846+0000 7f03a23ee740 1 --2- 172.21.15.139:0/3017262800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fd9cc0 0x55d207fda0a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.846+0000 7f03a23ee740 1 -- 172.21.15.139:0/3017262800 >> 172.21.15.139:0/3017262800 conn(0x55d207fd6230 msgr2=0x55d207fd6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.851+0000 7f03a23ee740 1 -- 172.21.15.139:0/3017262800 shutdown_connections 2024-09-16T01:43:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.852+0000 7f03a23ee740 1 -- 172.21.15.139:0/3017262800 wait complete. 2024-09-16T01:43:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.853+0000 7f03a23ee740 1 -- start start 2024-09-16T01:43:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.853+0000 7f03a23ee740 1 Processor -- start 2024-09-16T01:43:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.854+0000 7f03a23ee740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fd9cc0 0x55d207fe1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.854+0000 7f03a23ee740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fda5e0 0x55d207fe22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.854+0000 7f03a23ee740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fe4e80 0x55d207fe2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.854+0000 7f03a23ee740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d207fe7a50 con 0x55d207fd9cc0 2024-09-16T01:43:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.854+0000 7f03a23ee740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d207ec7290 con 0x55d207fda5e0 2024-09-16T01:43:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.854+0000 7f03a23ee740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d207fe7bd0 con 0x55d207fe4e80 2024-09-16T01:43:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.854+0000 7f03a23ee740 10 client.0 _ll_register_callbacks cb 0x55d207ef8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:43:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.854+0000 7f039cb13640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fda5e0 0x55d207fe22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.854+0000 7f039cb13640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fda5e0 0x55d207fe22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37152/0 (socket says 172.21.15.139:37152) 2024-09-16T01:43:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.854+0000 7f039cb13640 1 -- 172.21.15.139:0/3880520714 learned_addr learned my addr 172.21.15.139:0/3880520714 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.854+0000 7f039db15640 1 --2- 172.21.15.139:0/3880520714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fd9cc0 0x55d207fe1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.855+0000 7f039cb13640 1 -- 172.21.15.139:0/3880520714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fe4e80 msgr2=0x55d207fe2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.855+0000 7f039cb13640 1 --2- 172.21.15.139:0/3880520714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fe4e80 0x55d207fe2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.855+0000 7f039cb13640 1 -- 172.21.15.139:0/3880520714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fd9cc0 msgr2=0x55d207fe1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.855+0000 7f039cb13640 1 --2- 172.21.15.139:0/3880520714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fd9cc0 0x55d207fe1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.855+0000 7f039cb13640 1 -- 172.21.15.139:0/3880520714 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d207c51210 con 0x55d207fda5e0 2024-09-16T01:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.855+0000 7f039cb13640 1 --2- 172.21.15.139:0/3880520714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fda5e0 0x55d207fe22e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f039000a640 tx=0x7f039000ab00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=366182711df042be server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.856+0000 7f039db15640 1 --2- 172.21.15.139:0/3880520714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fd9cc0 0x55d207fe1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:43:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.903+0000 7f0380ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:43:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.903+0000 7f0380ff9640 1 -- 172.21.15.139:0/3880520714 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390019020 con 0x55d207fda5e0 2024-09-16T01:43:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.903+0000 7f0380ff9640 1 -- 172.21.15.139:0/3880520714 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f0390004d20 con 0x55d207fda5e0 2024-09-16T01:43:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.903+0000 7f03a23ee740 1 -- 172.21.15.139:0/3880520714 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0394040070 con 0x55d207fda5e0 2024-09-16T01:43:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.905+0000 7f0380ff9640 1 -- 172.21.15.139:0/3880520714 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0390021050 con 0x55d207fda5e0 2024-09-16T01:43:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.912+0000 7f03a23ee740 10 client.25406 fetch_fsmap learned FSMap version 31 2024-09-16T01:43:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.912+0000 7f03a23ee740 1 -- 172.21.15.139:0/3880520714 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d207d1d620 con 0x55d207fda5e0 2024-09-16T01:43:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.912+0000 7f0380ff9640 1 -- 172.21.15.139:0/3880520714 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0394040070 con 0x55d207fda5e0 2024-09-16T01:43:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.913+0000 7f0380ff9640 1 -- 172.21.15.139:0/3880520714 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7f0390015990 con 0x55d207fda5e0 2024-09-16T01:43:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.913+0000 7f03a23ee740 10 client.25406 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:43:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.913+0000 7f03a23ee740 10 client.25406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:43:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.913+0000 7f03a23ee740 20 client.25406 populate_metadata read hostname 'smithi139' 2024-09-16T01:43:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.913+0000 7f03a23ee740 1 --2- 172.21.15.139:0/3880520714 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d207ff50a0 0x55d207ff74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.913+0000 7f03a23ee740 4 client.25406 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:43:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.913+0000 7f03a23ee740 1 -- 172.21.15.139:0/3880520714 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d207f4ba20 con 0x55d207ff50a0 2024-09-16T01:43:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.913+0000 7f039db15640 1 --2- 172.21.15.139:0/3880520714 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d207ff50a0 0x55d207ff74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.914+0000 7f039db15640 1 --2- 172.21.15.139:0/3880520714 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d207ff50a0 0x55d207ff74e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d380781ce09cf8d2 server_cookie=a583ff31e3c0cfc6 in_seq=0 out_seq=0 2024-09-16T01:43:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.914+0000 7f0380ff9640 10 client.25406 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.918+0000 7f0380ff9640 1 -- 172.21.15.139:0/3880520714 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d207f4ba20 con 0x55d207ff50a0 2024-09-16T01:43:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.918+0000 7f0380ff9640 10 client.25406 handle_command_reply: tid=0 2024-09-16T01:43:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.918+0000 7f03a23ee740 1 client.25406 shutdown 2024-09-16T01:43:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.935+0000 7f03a23ee740 1 -- 172.21.15.139:0/3880520714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fda5e0 msgr2=0x55d207fe22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.935+0000 7f03a23ee740 1 --2- 172.21.15.139:0/3880520714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fda5e0 0x55d207fe22e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f039000a640 tx=0x7f039000ab00 comp rx=0 tx=0).stop 2024-09-16T01:43:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.941+0000 7f03a23ee740 1 -- 172.21.15.139:0/3880520714 shutdown_connections 2024-09-16T01:43:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.941+0000 7f03a23ee740 1 --2- 172.21.15.139:0/3880520714 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d207ff50a0 0x55d207ff74e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.941+0000 7f03a23ee740 1 --2- 172.21.15.139:0/3880520714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fe4e80 0x55d207fe2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.941+0000 7f03a23ee740 1 --2- 172.21.15.139:0/3880520714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fda5e0 0x55d207fe22e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.941+0000 7f03a23ee740 1 --2- 172.21.15.139:0/3880520714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fd9cc0 0x55d207fe1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.941+0000 7f0380ff9640 0 client.25406 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.941+0000 7f03a23ee740 1 -- 172.21.15.139:0/3880520714 >> 172.21.15.139:0/3880520714 conn(0x55d207fd6230 msgr2=0x55d207fdbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.941+0000 7f03a23ee740 1 -- 172.21.15.139:0/3880520714 shutdown_connections 2024-09-16T01:43:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.941+0000 7f03a23ee740 1 -- 172.21.15.139:0/3880520714 wait complete. 2024-09-16T01:43:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:34.942+0000 7f03a23ee740 20 client.25406 trim_cache size 0 max 16384 2024-09-16T01:43:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.198+0000 7f03a23ee740 1 Processor -- start 2024-09-16T01:43:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.198+0000 7f03a23ee740 1 -- start start 2024-09-16T01:43:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.198+0000 7f03a23ee740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fd9cc0 0x55d207fdc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.198+0000 7f03a23ee740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fda5e0 0x55d207fe2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.199+0000 7f03a23ee740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fe4e80 0x55d207fe3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.199+0000 7f039db15640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fd9cc0 0x55d207fdc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.199+0000 7f039db15640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fd9cc0 0x55d207fdc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:60538/0 (socket says 172.21.15.139:60538) 2024-09-16T01:43:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.199+0000 7f039db15640 1 -- 172.21.15.139:0/2697970731 learned_addr learned my addr 172.21.15.139:0/2697970731 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.199+0000 7f039cb13640 1 --2- 172.21.15.139:0/2697970731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fda5e0 0x55d207fe2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.203+0000 7f03a23ee740 1 -- 172.21.15.139:0/2697970731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f037c010070 con 0x55d207fe4e80 2024-09-16T01:43:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.203+0000 7f03a23ee740 1 -- 172.21.15.139:0/2697970731 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f037c011c70 con 0x55d207fda5e0 2024-09-16T01:43:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.203+0000 7f03a23ee740 1 -- 172.21.15.139:0/2697970731 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f037c010740 con 0x55d207fd9cc0 2024-09-16T01:43:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.204+0000 7f039db15640 1 --2- 172.21.15.139:0/2697970731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fe4e80 0x55d207fe3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.204+0000 7f039cb13640 1 -- 172.21.15.139:0/2697970731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fd9cc0 msgr2=0x55d207fdc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.204+0000 7f039cb13640 1 --2- 172.21.15.139:0/2697970731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fd9cc0 0x55d207fdc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.204+0000 7f039cb13640 1 -- 172.21.15.139:0/2697970731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fe4e80 msgr2=0x55d207fe3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.204+0000 7f039cb13640 1 --2- 172.21.15.139:0/2697970731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fe4e80 0x55d207fe3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.204+0000 7f039cb13640 1 -- 172.21.15.139:0/2697970731 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d207fe02d0 con 0x55d207fda5e0 2024-09-16T01:43:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.204+0000 7f039db15640 1 --2- 172.21.15.139:0/2697970731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fe4e80 0x55d207fe3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:43:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.205+0000 7f039cb13640 1 --2- 172.21.15.139:0/2697970731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fda5e0 0x55d207fe2650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f03900071c0 tx=0x7f03900055a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33a5a4e0b16b92b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.222+0000 7f0382ffd640 1 -- 172.21.15.139:0/2697970731 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390019030 con 0x55d207fda5e0 2024-09-16T01:43:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.222+0000 7f0382ffd640 1 -- 172.21.15.139:0/2697970731 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0390029070 con 0x55d207fda5e0 2024-09-16T01:43:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.222+0000 7f0382ffd640 1 -- 172.21.15.139:0/2697970731 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390004f80 con 0x55d207fda5e0 2024-09-16T01:43:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.224+0000 7f03a23ee740 1 -- 172.21.15.139:0/2697970731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fda5e0 msgr2=0x55d207fe2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.224+0000 7f03a23ee740 1 --2- 172.21.15.139:0/2697970731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fda5e0 0x55d207fe2650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f03900071c0 tx=0x7f03900055a0 comp rx=0 tx=0).stop 2024-09-16T01:43:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.244+0000 7f03a23ee740 1 -- 172.21.15.139:0/2697970731 shutdown_connections 2024-09-16T01:43:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.244+0000 7f03a23ee740 1 --2- 172.21.15.139:0/2697970731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fe4e80 0x55d207fe3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.244+0000 7f03a23ee740 1 --2- 172.21.15.139:0/2697970731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fda5e0 0x55d207fe2650 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.244+0000 7f03a23ee740 1 --2- 172.21.15.139:0/2697970731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fd9cc0 0x55d207fdc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.244+0000 7f03a23ee740 1 -- 172.21.15.139:0/2697970731 >> 172.21.15.139:0/2697970731 conn(0x55d207fd6230 msgr2=0x55d207fe3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.244+0000 7f03a23ee740 1 -- 172.21.15.139:0/2697970731 shutdown_connections 2024-09-16T01:43:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.244+0000 7f03a23ee740 1 -- 172.21.15.139:0/2697970731 wait complete. 2024-09-16T01:43:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.245+0000 7f03a23ee740 1 -- start start 2024-09-16T01:43:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.245+0000 7f03a23ee740 1 Processor -- start 2024-09-16T01:43:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.245+0000 7f03a23ee740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fd9cc0 0x55d207fed540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.245+0000 7f03a23ee740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fda5e0 0x55d207fe8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.245+0000 7f03a23ee740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fe4e80 0x55d207fe8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.245+0000 7f03a23ee740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f037c0114e0 con 0x55d207fda5e0 2024-09-16T01:43:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.245+0000 7f03a23ee740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f037c010e10 con 0x55d207fe4e80 2024-09-16T01:43:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.245+0000 7f03a23ee740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f037c00e050 con 0x55d207fd9cc0 2024-09-16T01:43:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.245+0000 7f03a23ee740 10 client.0 _ll_register_callbacks cb 0x7f039000d8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:43:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.246+0000 7f039db15640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fd9cc0 0x55d207fed540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.246+0000 7f039db15640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fe4e80 0x55d207fe8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.246+0000 7f039db15640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fd9cc0 0x55d207fed540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:60544/0 (socket says 172.21.15.139:60544) 2024-09-16T01:43:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.246+0000 7f039db15640 1 -- 172.21.15.139:0/816772152 learned_addr learned my addr 172.21.15.139:0/816772152 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.246+0000 7f039cb13640 1 --2- 172.21.15.139:0/816772152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fda5e0 0x55d207fe8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.247+0000 7f039cb13640 1 -- 172.21.15.139:0/816772152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fd9cc0 msgr2=0x55d207fed540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.247+0000 7f039cb13640 1 --2- 172.21.15.139:0/816772152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fd9cc0 0x55d207fed540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.247+0000 7f039cb13640 1 -- 172.21.15.139:0/816772152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fe4e80 msgr2=0x55d207fe8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.247+0000 7f039cb13640 1 --2- 172.21.15.139:0/816772152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fe4e80 0x55d207fe8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.247+0000 7f039cb13640 1 -- 172.21.15.139:0/816772152 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d207fe95d0 con 0x55d207fda5e0 2024-09-16T01:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.248+0000 7f039cb13640 1 --2- 172.21.15.139:0/816772152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fda5e0 0x55d207fe8860 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55d207fe2ff0 tx=0x7f0390005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3029e58b5c937187 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.249+0000 7f03a23ee740 1 -- 172.21.15.139:0/816772152 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0390029070 con 0x55d207fda5e0 2024-09-16T01:43:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.259+0000 7f0380ff9640 10 client.15921 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:43:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.260+0000 7f0380ff9640 1 -- 172.21.15.139:0/816772152 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390019070 con 0x55d207fda5e0 2024-09-16T01:43:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.260+0000 7f0380ff9640 1 -- 172.21.15.139:0/816772152 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f0390004c00 con 0x55d207fda5e0 2024-09-16T01:43:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.260+0000 7f0380ff9640 1 -- 172.21.15.139:0/816772152 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f039003d050 con 0x55d207fda5e0 2024-09-16T01:43:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.260+0000 7f03a23ee740 10 client.15921 fetch_fsmap learned FSMap version 31 2024-09-16T01:43:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.260+0000 7f03a23ee740 1 -- 172.21.15.139:0/816772152 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55d20805fb50 con 0x55d207fda5e0 2024-09-16T01:43:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.261+0000 7f0380ff9640 1 -- 172.21.15.139:0/816772152 <== mon.0 v2:172.21.15.139:3300/0 4 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7f0390004e00 con 0x55d207fda5e0 2024-09-16T01:43:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.262+0000 7f03a23ee740 10 client.15921 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:43:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.262+0000 7f03a23ee740 10 client.15921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:43:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.262+0000 7f03a23ee740 20 client.15921 populate_metadata read hostname 'smithi139' 2024-09-16T01:43:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.262+0000 7f03a23ee740 1 --2- 172.21.15.139:0/816772152 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d208060120 0x55d208062540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.262+0000 7f03a23ee740 4 client.15921 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:43:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.262+0000 7f03a23ee740 1 -- 172.21.15.139:0/816772152 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d207f45d70 con 0x55d208060120 2024-09-16T01:43:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.262+0000 7f039db15640 1 --2- 172.21.15.139:0/816772152 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d208060120 0x55d208062540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.262+0000 7f039db15640 1 --2- 172.21.15.139:0/816772152 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d208060120 0x55d208062540 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98ba862f18973eca server_cookie=e3c3e1f113415e6e in_seq=0 out_seq=0 2024-09-16T01:43:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:35.263+0000 7f0380ff9640 10 client.15921 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:35.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:35 smithi139 ceph-mon[31175]: pgmap v98: 129 pgs: 129 active+clean; 320 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 131 op/s 2024-09-16T01:43:35.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:35 smithi176 ceph-mon[35324]: pgmap v98: 129 pgs: 129 active+clean; 320 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 131 op/s 2024-09-16T01:43:35.967 INFO:tasks.workunit.client.0.smithi139.stdout: 146 1190536 1511 830285 1553 213065 399 2024-09-16T01:43:36.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:35 smithi164 ceph-mon[35121]: pgmap v98: 129 pgs: 129 active+clean; 320 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 131 op/s 2024-09-16T01:43:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.605+0000 7f0380ff9640 1 -- 172.21.15.139:0/816772152 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d207f45d70 con 0x55d208060120 2024-09-16T01:43:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.605+0000 7f0380ff9640 10 client.15921 handle_command_reply: tid=0 2024-09-16T01:43:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.605+0000 7f03a23ee740 1 client.15921 shutdown 2024-09-16T01:43:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.606+0000 7f03a23ee740 1 -- 172.21.15.139:0/816772152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fda5e0 msgr2=0x55d207fe8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.606+0000 7f03a23ee740 1 --2- 172.21.15.139:0/816772152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fda5e0 0x55d207fe8860 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55d207fe2ff0 tx=0x7f0390005b00 comp rx=0 tx=0).stop 2024-09-16T01:43:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.606+0000 7f03a23ee740 1 -- 172.21.15.139:0/816772152 shutdown_connections 2024-09-16T01:43:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.606+0000 7f03a23ee740 1 --2- 172.21.15.139:0/816772152 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d208060120 0x55d208062540 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.606+0000 7f03a23ee740 1 --2- 172.21.15.139:0/816772152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d207fe4e80 0x55d207fe8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.606+0000 7f03a23ee740 1 --2- 172.21.15.139:0/816772152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d207fda5e0 0x55d207fe8860 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.606+0000 7f03a23ee740 1 --2- 172.21.15.139:0/816772152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d207fd9cc0 0x55d207fed540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.606+0000 7f03a23ee740 1 -- 172.21.15.139:0/816772152 >> 172.21.15.139:0/816772152 conn(0x55d207fd6230 msgr2=0x55d207fe3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.606+0000 7f0380ff9640 0 client.15921 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.606+0000 7f03a23ee740 1 -- 172.21.15.139:0/816772152 shutdown_connections 2024-09-16T01:43:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.606+0000 7f03a23ee740 1 -- 172.21.15.139:0/816772152 wait complete. 2024-09-16T01:43:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.607+0000 7f03a23ee740 20 client.15921 trim_cache size 0 max 16384 2024-09-16T01:43:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:43:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:43:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:43:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:43:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.631+0000 7f039f578640 1 -- 172.21.15.139:0/1109682674 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f03840a35c0 msgr2=0x7f03840a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.631+0000 7f039f578640 1 --2- 172.21.15.139:0/1109682674 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f03840a35c0 0x7f03840a5a80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55d207fd7090 tx=0x7f0388000bc0 comp rx=0 tx=0).stop 2024-09-16T01:43:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.631+0000 7f039f578640 1 -- 172.21.15.139:0/1109682674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03981091f0 msgr2=0x7f039807a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.631+0000 7f039f578640 1 --2- 172.21.15.139:0/1109682674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03981091f0 0x7f039807a8e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f038800da90 tx=0x7f038800df50 comp rx=0 tx=0).stop 2024-09-16T01:43:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.631+0000 7f039f578640 1 -- 172.21.15.139:0/1109682674 shutdown_connections 2024-09-16T01:43:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.631+0000 7f039f578640 1 --2- 172.21.15.139:0/1109682674 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f03840a35c0 0x7f03840a5a80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.631+0000 7f039f578640 1 --2- 172.21.15.139:0/1109682674 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f039810d610 0x7f0398075b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.631+0000 7f039f578640 1 --2- 172.21.15.139:0/1109682674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0398109b70 0x7f039807ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.631+0000 7f039f578640 1 --2- 172.21.15.139:0/1109682674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03981091f0 0x7f039807a8e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.631+0000 7f039f578640 1 -- 172.21.15.139:0/1109682674 >> 172.21.15.139:0/1109682674 conn(0x7f039806cc60 msgr2=0x7f03980722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.634+0000 7f039f578640 1 -- 172.21.15.139:0/1109682674 shutdown_connections 2024-09-16T01:43:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:37.634+0000 7f039f578640 1 -- 172.21.15.139:0/1109682674 wait complete. 2024-09-16T01:43:37.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:37 smithi139 ceph-mon[31175]: pgmap v99: 129 pgs: 129 active+clean; 355 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 277 op/s 2024-09-16T01:43:37.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:37 smithi176 ceph-mon[35324]: pgmap v99: 129 pgs: 129 active+clean; 355 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 277 op/s 2024-09-16T01:43:38.047 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:37 smithi164 ceph-mon[35121]: pgmap v99: 129 pgs: 129 active+clean; 355 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 277 op/s 2024-09-16T01:43:39.628 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:39 smithi139 ceph-mon[31175]: pgmap v100: 129 pgs: 129 active+clean; 355 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 206 op/s 2024-09-16T01:43:39.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:39 smithi176 ceph-mon[35324]: pgmap v100: 129 pgs: 129 active+clean; 355 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 206 op/s 2024-09-16T01:43:40.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:39 smithi164 ceph-mon[35121]: pgmap v100: 129 pgs: 129 active+clean; 355 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 206 op/s 2024-09-16T01:43:41.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:41 smithi139 ceph-mon[31175]: pgmap v101: 129 pgs: 129 active+clean; 342 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 502 op/s 2024-09-16T01:43:41.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:41 smithi176 ceph-mon[35324]: pgmap v101: 129 pgs: 129 active+clean; 342 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 502 op/s 2024-09-16T01:43:42.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:41 smithi164 ceph-mon[35121]: pgmap v101: 129 pgs: 129 active+clean; 342 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 502 op/s 2024-09-16T01:43:43.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:43 smithi139 ceph-mon[31175]: pgmap v102: 129 pgs: 129 active+clean; 349 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 701 op/s 2024-09-16T01:43:43.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:43 smithi176 ceph-mon[35324]: pgmap v102: 129 pgs: 129 active+clean; 349 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 701 op/s 2024-09-16T01:43:44.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:43 smithi164 ceph-mon[35121]: pgmap v102: 129 pgs: 129 active+clean; 349 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 701 op/s 2024-09-16T01:43:45.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:45 smithi139 ceph-mon[31175]: pgmap v103: 129 pgs: 129 active+clean; 349 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 679 op/s 2024-09-16T01:43:45.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:45 smithi176 ceph-mon[35324]: pgmap v103: 129 pgs: 129 active+clean; 349 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 679 op/s 2024-09-16T01:43:45.967 INFO:tasks.workunit.client.0.smithi139.stdout: 183 1589950 1577 1118635 1757 297337 1267 2024-09-16T01:43:46.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:45 smithi164 ceph-mon[35121]: pgmap v103: 129 pgs: 129 active+clean; 349 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 679 op/s 2024-09-16T01:43:47.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:47 smithi139 ceph-mon[31175]: pgmap v104: 129 pgs: 129 active+clean; 473 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-09-16T01:43:47.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:47 smithi176 ceph-mon[35324]: pgmap v104: 129 pgs: 129 active+clean; 473 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-09-16T01:43:48.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:47 smithi164 ceph-mon[35121]: pgmap v104: 129 pgs: 129 active+clean; 473 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-09-16T01:43:49.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:49 smithi139 ceph-mon[31175]: pgmap v105: 129 pgs: 129 active+clean; 473 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 12 MiB/s wr, 992 op/s 2024-09-16T01:43:49.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:49 smithi176 ceph-mon[35324]: pgmap v105: 129 pgs: 129 active+clean; 473 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 12 MiB/s wr, 992 op/s 2024-09-16T01:43:50.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:49 smithi164 ceph-mon[35121]: pgmap v105: 129 pgs: 129 active+clean; 473 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 12 MiB/s wr, 992 op/s 2024-09-16T01:43:51.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:51 smithi139 ceph-mon[31175]: pgmap v106: 129 pgs: 129 active+clean; 477 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-16T01:43:51.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:51 smithi176 ceph-mon[35324]: pgmap v106: 129 pgs: 129 active+clean; 477 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-16T01:43:52.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:51 smithi164 ceph-mon[35121]: pgmap v106: 129 pgs: 129 active+clean; 477 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-16T01:43:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.402+0000 7f0ae5077640 1 -- 172.21.15.139:0/1011718782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae010a300 msgr2=0x7f0ae010a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.402+0000 7f0ae5077640 1 --2- 172.21.15.139:0/1011718782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae010a300 0x7f0ae010a6e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009ca0 tx=0x7f0ad802cb90 comp rx=0 tx=0).stop 2024-09-16T01:43:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.403+0000 7f0ae5077640 1 -- 172.21.15.139:0/1011718782 shutdown_connections 2024-09-16T01:43:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.403+0000 7f0ae5077640 1 --2- 172.21.15.139:0/1011718782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ae01138b0 0x7f0ae0115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.403+0000 7f0ae5077640 1 --2- 172.21.15.139:0/1011718782 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ae010ac20 0x7f0ae0113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.403+0000 7f0ae5077640 1 --2- 172.21.15.139:0/1011718782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae010a300 0x7f0ae010a6e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.403+0000 7f0ae5077640 1 -- 172.21.15.139:0/1011718782 >> 172.21.15.139:0/1011718782 conn(0x7f0ae006cc60 msgr2=0x7f0ae006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.403+0000 7f0ae5077640 1 -- 172.21.15.139:0/1011718782 shutdown_connections 2024-09-16T01:43:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.403+0000 7f0ae5077640 1 -- 172.21.15.139:0/1011718782 wait complete. 2024-09-16T01:43:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.405+0000 7f0ae5077640 1 Processor -- start 2024-09-16T01:43:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.405+0000 7f0ae5077640 1 -- start start 2024-09-16T01:43:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.406+0000 7f0ae5077640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ae010a300 0x7f0ae01a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.406+0000 7f0ae5077640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ae010ac20 0x7f0ae01a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.406+0000 7f0ae5077640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae01138b0 0x7f0ae019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.406+0000 7f0ae5077640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0118380 con 0x7f0ae01138b0 2024-09-16T01:43:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.406+0000 7f0ae5077640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0118200 con 0x7f0ae010ac20 2024-09-16T01:43:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.406+0000 7f0ae5077640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0118500 con 0x7f0ae010a300 2024-09-16T01:43:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.407+0000 7f0adf59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae01138b0 0x7f0ae019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.407+0000 7f0adf59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae01138b0 0x7f0ae019e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56606/0 (socket says 172.21.15.139:56606) 2024-09-16T01:43:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.407+0000 7f0adf59e640 1 -- 172.21.15.139:0/4245705359 learned_addr learned my addr 172.21.15.139:0/4245705359 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.407+0000 7f0adf59e640 1 -- 172.21.15.139:0/4245705359 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ae010a300 msgr2=0x7f0ae01a45d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:43:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.407+0000 7f0adf59e640 1 --2- 172.21.15.139:0/4245705359 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ae010a300 0x7f0ae01a45d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.407+0000 7f0adf59e640 1 -- 172.21.15.139:0/4245705359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ae010ac20 msgr2=0x7f0ae01a4b10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:43:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.407+0000 7f0adf59e640 1 --2- 172.21.15.139:0/4245705359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ae010ac20 0x7f0ae01a4b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.407+0000 7f0adf59e640 1 -- 172.21.15.139:0/4245705359 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae019ef50 con 0x7f0ae01138b0 2024-09-16T01:43:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.408+0000 7f0adf59e640 1 --2- 172.21.15.139:0/4245705359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae01138b0 0x7f0ae019e690 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000d950 tx=0x7f0ad000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.427+0000 7f0ac7fff640 1 -- 172.21.15.139:0/4245705359 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0017030 con 0x7f0ae01138b0 2024-09-16T01:43:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.427+0000 7f0ae5077640 1 -- 172.21.15.139:0/4245705359 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae019f240 con 0x7f0ae01138b0 2024-09-16T01:43:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.427+0000 7f0ae5077640 1 -- 172.21.15.139:0/4245705359 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae01ab550 con 0x7f0ae01138b0 2024-09-16T01:43:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.432+0000 7f0ae7e0d740 1 Processor -- start 2024-09-16T01:43:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.437+0000 7f0ac7fff640 1 -- 172.21.15.139:0/4245705359 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad0004020 con 0x7f0ae01138b0 2024-09-16T01:43:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.437+0000 7f0ac7fff640 1 -- 172.21.15.139:0/4245705359 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0003c70 con 0x7f0ae01138b0 2024-09-16T01:43:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.437+0000 7f0ae7e0d740 1 -- start start 2024-09-16T01:43:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.439+0000 7f0ac7fff640 1 -- 172.21.15.139:0/4245705359 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0ad001a070 con 0x7f0ae01138b0 2024-09-16T01:43:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.440+0000 7f0ac7fff640 1 --2- 172.21.15.139:0/4245705359 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0ac80a35f0 0x7f0ac80a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.440+0000 7f0ade59c640 1 --2- 172.21.15.139:0/4245705359 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0ac80a35f0 0x7f0ac80a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.440+0000 7f0ac7fff640 1 -- 172.21.15.139:0/4245705359 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f0ad00d0c60 con 0x7f0ae01138b0 2024-09-16T01:43:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.445+0000 7f0ae7e0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415b0f00 0x563c415b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.445+0000 7f0ae7e0d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415ab9a0 0x563c415abe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.445+0000 7f0ae7e0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ac340 0x563c415b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.445+0000 7f0ae7e0d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563c415062f0 con 0x563c415b0f00 2024-09-16T01:43:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.445+0000 7f0ae7e0d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563c414ae270 con 0x563c415ac340 2024-09-16T01:43:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.445+0000 7f0ae7e0d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563c415022b0 con 0x563c415ab9a0 2024-09-16T01:43:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.445+0000 7f0ade59c640 1 --2- 172.21.15.139:0/4245705359 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0ac80a35f0 0x7f0ac80a5ab0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0ae019fee0 tx=0x7f0ad4006bd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.445+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415ab9a0 0x563c415abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.446+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415ab9a0 0x563c415abe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:50984/0 (socket says 172.21.15.139:50984) 2024-09-16T01:43:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.446+0000 7f0ade59c640 1 -- 172.21.15.139:0/1737971491 learned_addr learned my addr 172.21.15.139:0/1737971491 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.446+0000 7f0aded9d640 1 --2- 172.21.15.139:0/1737971491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415b0f00 0x563c415b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.447+0000 7f0adf59e640 1 --2- 172.21.15.139:0/1737971491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ac340 0x563c415b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.447+0000 7f0adf59e640 1 -- 172.21.15.139:0/1737971491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415ab9a0 msgr2=0x563c415abe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.447+0000 7f0adf59e640 1 --2- 172.21.15.139:0/1737971491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415ab9a0 0x563c415abe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.447+0000 7f0adf59e640 1 -- 172.21.15.139:0/1737971491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415b0f00 msgr2=0x563c415b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.447+0000 7f0adf59e640 1 --2- 172.21.15.139:0/1737971491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415b0f00 0x563c415b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.447+0000 7f0adf59e640 1 -- 172.21.15.139:0/1737971491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c414ad340 con 0x563c415ac340 2024-09-16T01:43:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.447+0000 7f0aded9d640 1 --2- 172.21.15.139:0/1737971491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415b0f00 0x563c415b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:43:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.448+0000 7f0adf59e640 1 --2- 172.21.15.139:0/1737971491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ac340 0x563c415b7d40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00ce020 tx=0x7f0ad009cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5947f7c7a2c56ef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.478+0000 7f0ae5077640 1 -- 172.21.15.139:0/1737971491 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad00b3030 con 0x563c415ac340 2024-09-16T01:43:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.478+0000 7f0ae5077640 1 -- 172.21.15.139:0/1737971491 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad009aeb0 con 0x563c415ac340 2024-09-16T01:43:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.479+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/1737971491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ac340 msgr2=0x563c415b7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.479+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/1737971491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ac340 0x563c415b7d40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00ce020 tx=0x7f0ad009cd30 comp rx=0 tx=0).stop 2024-09-16T01:43:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.483+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/1737971491 shutdown_connections 2024-09-16T01:43:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.483+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/1737971491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ac340 0x563c415b7d40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.483+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/1737971491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415ab9a0 0x563c415abe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.483+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/1737971491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415b0f00 0x563c415b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.483+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/1737971491 >> 172.21.15.139:0/1737971491 conn(0x563c4159d000 msgr2=0x563c4159d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.484+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/1737971491 shutdown_connections 2024-09-16T01:43:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.484+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/1737971491 wait complete. 2024-09-16T01:43:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.484+0000 7f0ae7e0d740 1 -- start start 2024-09-16T01:43:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.484+0000 7f0ae7e0d740 1 Processor -- start 2024-09-16T01:43:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.485+0000 7f0ae7e0d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415ab9a0 0x563c415a6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.485+0000 7f0ae7e0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ac340 0x563c415a7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.485+0000 7f0ae7e0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415b0f00 0x563c415b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.485+0000 7f0ae7e0d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x563c415a22a0 con 0x563c415ac340 2024-09-16T01:43:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.485+0000 7f0ae7e0d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563c4148f290 con 0x563c415b0f00 2024-09-16T01:43:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.485+0000 7f0ae7e0d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563c415a2420 con 0x563c415ab9a0 2024-09-16T01:43:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.485+0000 7f0ae7e0d740 10 client.0 _ll_register_callbacks cb 0x563c414c0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:43:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.485+0000 7f0adf59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415b0f00 0x563c415b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.485+0000 7f0adf59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415b0f00 0x563c415b8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44378/0 (socket says 172.21.15.139:44378) 2024-09-16T01:43:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.485+0000 7f0adf59e640 1 -- 172.21.15.139:0/1957703329 learned_addr learned my addr 172.21.15.139:0/1957703329 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.486+0000 7f0aded9d640 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415ab9a0 0x563c415a6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.486+0000 7f0ade59c640 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ac340 0x563c415a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.486+0000 7f0adf59e640 1 -- 172.21.15.139:0/1957703329 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415ab9a0 msgr2=0x563c415a6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.486+0000 7f0adf59e640 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415ab9a0 0x563c415a6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.486+0000 7f0adf59e640 1 -- 172.21.15.139:0/1957703329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ac340 msgr2=0x563c415a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.486+0000 7f0adf59e640 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ac340 0x563c415a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.486+0000 7f0adf59e640 1 -- 172.21.15.139:0/1957703329 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c41219210 con 0x563c415b0f00 2024-09-16T01:43:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.486+0000 7f0ade59c640 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ac340 0x563c415a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:43:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.486+0000 7f0aded9d640 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415ab9a0 0x563c415a6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:43:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.486+0000 7f0adf59e640 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415b0f00 0x563c415b8ca0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0099010 tx=0x7f0ad0098e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35c3e9aacb36d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.506+0000 7f0ac2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:43:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.506+0000 7f0ac2ff5640 1 -- 172.21.15.139:0/1957703329 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad00a9020 con 0x563c415b0f00 2024-09-16T01:43:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.506+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/1957703329 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad009aeb0 con 0x563c415b0f00 2024-09-16T01:43:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.507+0000 7f0ac2ff5640 1 -- 172.21.15.139:0/1957703329 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f0ad009d010 con 0x563c415b0f00 2024-09-16T01:43:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.508+0000 7f0ac2ff5640 1 -- 172.21.15.139:0/1957703329 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0ad00b3070 con 0x563c415b0f00 2024-09-16T01:43:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.509+0000 7f0ae7e0d740 10 client.25421 fetch_fsmap learned FSMap version 31 2024-09-16T01:43:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.509+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/1957703329 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563c412e5620 con 0x563c415b0f00 2024-09-16T01:43:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.510+0000 7f0ac2ff5640 1 -- 172.21.15.139:0/1957703329 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad009aeb0 con 0x563c415b0f00 2024-09-16T01:43:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.512+0000 7f0ac2ff5640 1 -- 172.21.15.139:0/1957703329 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7f0ad0024060 con 0x563c415b0f00 2024-09-16T01:43:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.513+0000 7f0ae7e0d740 10 client.25421 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:43:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.513+0000 7f0ae7e0d740 10 client.25421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:43:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.513+0000 7f0ae7e0d740 20 client.25421 populate_metadata read hostname 'smithi139' 2024-09-16T01:43:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.513+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563c415bd040 0x563c415bf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.513+0000 7f0ae7e0d740 4 client.25421 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:43:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.513+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/1957703329 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c41513a20 con 0x563c415bd040 2024-09-16T01:43:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.513+0000 7f0aded9d640 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563c415bd040 0x563c415bf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.514+0000 7f0aded9d640 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563c415bd040 0x563c415bf460 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd37bc12575d37d7 server_cookie=6386fc3bd8190a08 in_seq=0 out_seq=0 2024-09-16T01:43:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.514+0000 7f0ac2ff5640 10 client.25421 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.514+0000 7f0ac2ff5640 1 -- 172.21.15.139:0/1957703329 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c41513a20 con 0x563c415bd040 2024-09-16T01:43:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.514+0000 7f0ac2ff5640 10 client.25421 handle_command_reply: tid=0 2024-09-16T01:43:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.514+0000 7f0ae7e0d740 1 client.25421 shutdown 2024-09-16T01:43:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.516+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/1957703329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415b0f00 msgr2=0x563c415b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.516+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415b0f00 0x563c415b8ca0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0099010 tx=0x7f0ad0098e50 comp rx=0 tx=0).stop 2024-09-16T01:43:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.517+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/1957703329 shutdown_connections 2024-09-16T01:43:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.517+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563c415bd040 0x563c415bf460 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.517+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415b0f00 0x563c415b8ca0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.517+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ac340 0x563c415a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.517+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/1957703329 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415ab9a0 0x563c415a6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.517+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/1957703329 >> 172.21.15.139:0/1957703329 conn(0x563c4159d000 msgr2=0x563c415a1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.525+0000 7f0ac2ff5640 0 client.25421 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.525+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/1957703329 shutdown_connections 2024-09-16T01:43:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.525+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/1957703329 wait complete. 2024-09-16T01:43:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.525+0000 7f0ae7e0d740 20 client.25421 trim_cache size 0 max 16384 2024-09-16T01:43:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.942+0000 7f0ae7e0d740 1 Processor -- start 2024-09-16T01:43:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.942+0000 7f0ae7e0d740 1 -- start start 2024-09-16T01:43:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.943+0000 7f0ae7e0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ab9a0 0x563c415a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.943+0000 7f0aded9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ab9a0 0x563c415a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.943+0000 7f0aded9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ab9a0 0x563c415a5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56632/0 (socket says 172.21.15.139:56632) 2024-09-16T01:43:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.943+0000 7f0aded9d640 1 -- 172.21.15.139:0/199414541 learned_addr learned my addr 172.21.15.139:0/199414541 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.971+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/199414541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ac340 0x563c415aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.971+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/199414541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415b0f00 0x563c415c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.971+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/199414541 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc00fef0 con 0x563c415ab9a0 2024-09-16T01:43:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.971+0000 7f0ade59c640 1 --2- 172.21.15.139:0/199414541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ac340 0x563c415aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.971+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/199414541 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc011af0 con 0x563c415ac340 2024-09-16T01:43:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.971+0000 7f0adf59e640 1 --2- 172.21.15.139:0/199414541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415b0f00 0x563c415c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.971+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/199414541 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc0105c0 con 0x563c415b0f00 2024-09-16T01:43:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.972+0000 7f0adf59e640 1 -- 172.21.15.139:0/199414541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ac340 msgr2=0x563c415aacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.972+0000 7f0adf59e640 1 --2- 172.21.15.139:0/199414541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ac340 0x563c415aacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.972+0000 7f0adf59e640 1 -- 172.21.15.139:0/199414541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ab9a0 msgr2=0x563c415a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.972+0000 7f0adf59e640 1 --2- 172.21.15.139:0/199414541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ab9a0 0x563c415a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.972+0000 7f0adf59e640 1 -- 172.21.15.139:0/199414541 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c415a0e70 con 0x563c415b0f00 2024-09-16T01:43:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.972+0000 7f0ade59c640 1 --2- 172.21.15.139:0/199414541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ac340 0x563c415aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:43:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.972+0000 7f0aded9d640 1 --2- 172.21.15.139:0/199414541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ab9a0 0x563c415a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:43:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.972+0000 7f0adf59e640 1 --2- 172.21.15.139:0/199414541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415b0f00 0x563c415c4d30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0ad009cb40 tx=0x7f0ad0005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59bf42d8507f7ebd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.977+0000 7f0ac4ff9640 1 -- 172.21.15.139:0/199414541 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad00a9070 con 0x563c415b0f00 2024-09-16T01:43:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.977+0000 7f0ac4ff9640 1 -- 172.21.15.139:0/199414541 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad009c000 con 0x563c415b0f00 2024-09-16T01:43:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.977+0000 7f0ac4ff9640 1 -- 172.21.15.139:0/199414541 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad009d070 con 0x563c415b0f00 2024-09-16T01:43:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.978+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/199414541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415b0f00 msgr2=0x563c415c4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.978+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/199414541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415b0f00 0x563c415c4d30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0ad009cb40 tx=0x7f0ad0005ab0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.983+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/199414541 shutdown_connections 2024-09-16T01:43:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.983+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/199414541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415b0f00 0x563c415c4d30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.983+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/199414541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ac340 0x563c415aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.983+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/199414541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ab9a0 0x563c415a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.983+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/199414541 >> 172.21.15.139:0/199414541 conn(0x563c4159d000 msgr2=0x563c415aa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.983+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/199414541 shutdown_connections 2024-09-16T01:43:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.983+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/199414541 wait complete. 2024-09-16T01:43:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.984+0000 7f0ae7e0d740 1 -- start start 2024-09-16T01:43:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.984+0000 7f0ae7e0d740 1 Processor -- start 2024-09-16T01:43:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.985+0000 7f0ae7e0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ab9a0 0x563c415b8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.985+0000 7f0ae7e0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ac340 0x563c415b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.985+0000 7f0aded9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ab9a0 0x563c415b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.985+0000 7f0aded9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ab9a0 0x563c415b8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44406/0 (socket says 172.21.15.139:44406) 2024-09-16T01:43:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.985+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ac340 0x563c415b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.986+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ac340 0x563c415b8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56634/0 (socket says 172.21.15.139:56634) 2024-09-16T01:43:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:52.986+0000 7f0aded9d640 1 -- 172.21.15.139:0/4059406130 learned_addr learned my addr 172.21.15.139:0/4059406130 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:43:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.004+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/4059406130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415b0f00 0x563c415b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.004+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/4059406130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc011360 con 0x563c415ac340 2024-09-16T01:43:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.004+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/4059406130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc010c90 con 0x563c415ab9a0 2024-09-16T01:43:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.004+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/4059406130 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc00df80 con 0x563c415b0f00 2024-09-16T01:43:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.004+0000 7f0adf59e640 1 --2- 172.21.15.139:0/4059406130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415b0f00 0x563c415b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.005+0000 7f0aded9d640 1 -- 172.21.15.139:0/4059406130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415b0f00 msgr2=0x563c415b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.005+0000 7f0aded9d640 1 --2- 172.21.15.139:0/4059406130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415b0f00 0x563c415b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.005+0000 7f0aded9d640 1 -- 172.21.15.139:0/4059406130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ac340 msgr2=0x563c415b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:43:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.005+0000 7f0aded9d640 1 --2- 172.21.15.139:0/4059406130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ac340 0x563c415b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.005+0000 7f0aded9d640 1 -- 172.21.15.139:0/4059406130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c415b42a0 con 0x563c415ab9a0 2024-09-16T01:43:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.005+0000 7f0ade59c640 1 --2- 172.21.15.139:0/4059406130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ac340 0x563c415b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:43:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.005+0000 7f0ae7e0d740 10 client.0 _ll_register_callbacks cb 0x7f0ad8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:43:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.005+0000 7f0aded9d640 1 --2- 172.21.15.139:0/4059406130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ab9a0 0x563c415b8650 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009430 tx=0x7f0ad8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=443f9aeb46aa5588 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:43:53.007 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.005+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/4059406130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad009c000 con 0x563c415ab9a0 2024-09-16T01:43:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.020+0000 7f0ac2ff5640 10 client.25424 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:43:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.020+0000 7f0ac2ff5640 1 -- 172.21.15.139:0/4059406130 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad804d070 con 0x563c415ab9a0 2024-09-16T01:43:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.020+0000 7f0ac2ff5640 1 -- 172.21.15.139:0/4059406130 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f0ad8003b70 con 0x563c415ab9a0 2024-09-16T01:43:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.020+0000 7f0ac2ff5640 1 -- 172.21.15.139:0/4059406130 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0ad8058050 con 0x563c415ab9a0 2024-09-16T01:43:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.021+0000 7f0ae7e0d740 10 client.25424 fetch_fsmap learned FSMap version 31 2024-09-16T01:43:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.021+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/4059406130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=31}) -- 0x563c41627710 con 0x563c415ab9a0 2024-09-16T01:43:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.027+0000 7f0ac2ff5640 1 -- 172.21.15.139:0/4059406130 <== mon.2 v2:172.21.15.164:3300/0 4 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7f0ad802d8d0 con 0x563c415ab9a0 2024-09-16T01:43:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.032+0000 7f0ae7e0d740 10 client.25424 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:43:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.032+0000 7f0ae7e0d740 10 client.25424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:43:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.032+0000 7f0ae7e0d740 20 client.25424 populate_metadata read hostname 'smithi139' 2024-09-16T01:43:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.032+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/4059406130 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563c41627e00 0x563c4162a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:43:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.032+0000 7f0ae7e0d740 4 client.25424 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:43:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.032+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/4059406130 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0ad009c000 con 0x563c41627e00 2024-09-16T01:43:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.032+0000 7f0ade59c640 1 --2- 172.21.15.139:0/4059406130 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563c41627e00 0x563c4162a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:43:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.033+0000 7f0ade59c640 1 --2- 172.21.15.139:0/4059406130 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563c41627e00 0x563c4162a240 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ec620d2e879c4 server_cookie=6a6b871cc93a5924 in_seq=0 out_seq=0 2024-09-16T01:43:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:53.036+0000 7f0ac2ff5640 10 client.25424 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:53.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:53 smithi176 ceph-mon[35324]: pgmap v107: 129 pgs: 129 active+clean; 554 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-09-16T01:43:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:53 smithi164 ceph-mon[35121]: pgmap v107: 129 pgs: 129 active+clean; 554 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-09-16T01:43:54.192 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:53 smithi139 ceph-mon[31175]: pgmap v107: 129 pgs: 129 active+clean; 554 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-09-16T01:43:55.967 INFO:tasks.workunit.client.0.smithi139.stdout: 225 2179100 2061 1545496 2281 421653 623 2024-09-16T01:43:56.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:55 smithi164 ceph-mon[35121]: pgmap v108: 129 pgs: 129 active+clean; 554 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 789 op/s 2024-09-16T01:43:56.175 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:55 smithi139 ceph-mon[31175]: pgmap v108: 129 pgs: 129 active+clean; 554 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 789 op/s 2024-09-16T01:43:56.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:55 smithi176 ceph-mon[35324]: pgmap v108: 129 pgs: 129 active+clean; 554 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 789 op/s 2024-09-16T01:43:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.384+0000 7f0ac2ff5640 1 -- 172.21.15.139:0/4059406130 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f0ad009c000 con 0x563c41627e00 2024-09-16T01:43:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.384+0000 7f0ac2ff5640 10 client.25424 handle_command_reply: tid=0 2024-09-16T01:43:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.384+0000 7f0ae7e0d740 1 client.25424 shutdown 2024-09-16T01:43:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.385+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/4059406130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ab9a0 msgr2=0x563c415b8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.385+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/4059406130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ab9a0 0x563c415b8650 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009430 tx=0x7f0ad8009780 comp rx=0 tx=0).stop 2024-09-16T01:43:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.385+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/4059406130 shutdown_connections 2024-09-16T01:43:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.385+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/4059406130 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x563c41627e00 0x563c4162a240 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.385+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/4059406130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563c415b0f00 0x563c415b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.385+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/4059406130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563c415ac340 0x563c415b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.385+0000 7f0ae7e0d740 1 --2- 172.21.15.139:0/4059406130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c415ab9a0 0x563c415b8650 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.385+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/4059406130 >> 172.21.15.139:0/4059406130 conn(0x563c4159d000 msgr2=0x563c415a1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.385+0000 7f0ac2ff5640 0 client.25424 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:43:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.385+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/4059406130 shutdown_connections 2024-09-16T01:43:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.385+0000 7f0ae7e0d740 1 -- 172.21.15.139:0/4059406130 wait complete. 2024-09-16T01:43:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.386+0000 7f0ae7e0d740 20 client.25424 trim_cache size 0 max 16384 2024-09-16T01:43:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:43:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:43:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:43:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:43:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.433+0000 7f0ac5ffb640 1 -- 172.21.15.139:0/4245705359 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0ac80a35f0 msgr2=0x7f0ac80a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.434+0000 7f0ac5ffb640 1 --2- 172.21.15.139:0/4245705359 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0ac80a35f0 0x7f0ac80a5ab0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0ae019fee0 tx=0x7f0ad4006bd0 comp rx=0 tx=0).stop 2024-09-16T01:43:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.434+0000 7f0ac5ffb640 1 -- 172.21.15.139:0/4245705359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae01138b0 msgr2=0x7f0ae019e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:43:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.434+0000 7f0ac5ffb640 1 --2- 172.21.15.139:0/4245705359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae01138b0 0x7f0ae019e690 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000d950 tx=0x7f0ad000de10 comp rx=0 tx=0).stop 2024-09-16T01:43:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.435+0000 7f0ac5ffb640 1 -- 172.21.15.139:0/4245705359 shutdown_connections 2024-09-16T01:43:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.435+0000 7f0ac5ffb640 1 --2- 172.21.15.139:0/4245705359 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0ac80a35f0 0x7f0ac80a5ab0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.435+0000 7f0ac5ffb640 1 --2- 172.21.15.139:0/4245705359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ae01138b0 0x7f0ae019e690 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.435+0000 7f0ac5ffb640 1 --2- 172.21.15.139:0/4245705359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ae010ac20 0x7f0ae01a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.435+0000 7f0ac5ffb640 1 --2- 172.21.15.139:0/4245705359 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ae010a300 0x7f0ae01a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:43:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.435+0000 7f0ac5ffb640 1 -- 172.21.15.139:0/4245705359 >> 172.21.15.139:0/4245705359 conn(0x7f0ae006cc60 msgr2=0x7f0ae00706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:43:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.437+0000 7f0ac5ffb640 1 -- 172.21.15.139:0/4245705359 shutdown_connections 2024-09-16T01:43:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:43:56.438+0000 7f0ac5ffb640 1 -- 172.21.15.139:0/4245705359 wait complete. 2024-09-16T01:43:58.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:57 smithi164 ceph-mon[35121]: pgmap v109: 129 pgs: 129 active+clean; 568 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 19 MiB/s wr, 1.33k op/s 2024-09-16T01:43:58.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:57 smithi139 ceph-mon[31175]: pgmap v109: 129 pgs: 129 active+clean; 568 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 19 MiB/s wr, 1.33k op/s 2024-09-16T01:43:58.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:57 smithi176 ceph-mon[35324]: pgmap v109: 129 pgs: 129 active+clean; 568 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 19 MiB/s wr, 1.33k op/s 2024-09-16T01:43:59.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:43:58 smithi164 ceph-mon[35121]: pgmap v110: 129 pgs: 129 active+clean; 568 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.5 MiB/s wr, 867 op/s 2024-09-16T01:43:59.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:43:58 smithi139 ceph-mon[31175]: pgmap v110: 129 pgs: 129 active+clean; 568 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.5 MiB/s wr, 867 op/s 2024-09-16T01:43:59.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:43:58 smithi176 ceph-mon[35324]: pgmap v110: 129 pgs: 129 active+clean; 568 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.5 MiB/s wr, 867 op/s 2024-09-16T01:44:01.177 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:00 smithi139 ceph-mon[31175]: pgmap v111: 129 pgs: 129 active+clean; 578 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.5 MiB/s wr, 910 op/s 2024-09-16T01:44:01.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:00 smithi176 ceph-mon[35324]: pgmap v111: 129 pgs: 129 active+clean; 578 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.5 MiB/s wr, 910 op/s 2024-09-16T01:44:01.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:00 smithi164 ceph-mon[35121]: pgmap v111: 129 pgs: 129 active+clean; 578 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.5 MiB/s wr, 910 op/s 2024-09-16T01:44:03.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:02 smithi139 ceph-mon[31175]: pgmap v112: 129 pgs: 129 active+clean; 599 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 13 MiB/s wr, 1.10k op/s 2024-09-16T01:44:03.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:02 smithi176 ceph-mon[35324]: pgmap v112: 129 pgs: 129 active+clean; 599 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 13 MiB/s wr, 1.10k op/s 2024-09-16T01:44:03.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:02 smithi164 ceph-mon[35121]: pgmap v112: 129 pgs: 129 active+clean; 599 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 13 MiB/s wr, 1.10k op/s 2024-09-16T01:44:05.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:04 smithi139 ceph-mon[31175]: pgmap v113: 129 pgs: 129 active+clean; 599 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 788 op/s 2024-09-16T01:44:05.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:04 smithi176 ceph-mon[35324]: pgmap v113: 129 pgs: 129 active+clean; 599 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 788 op/s 2024-09-16T01:44:05.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:04 smithi164 ceph-mon[35121]: pgmap v113: 129 pgs: 129 active+clean; 599 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 788 op/s 2024-09-16T01:44:05.967 INFO:tasks.workunit.client.0.smithi139.stdout: 257 4448264 1564 1618666 1732 377030 428 2024-09-16T01:44:07.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:06 smithi139 ceph-mon[31175]: pgmap v114: 129 pgs: 129 active+clean; 655 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 989 op/s 2024-09-16T01:44:07.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:06 smithi176 ceph-mon[35324]: pgmap v114: 129 pgs: 129 active+clean; 655 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 989 op/s 2024-09-16T01:44:07.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:06 smithi164 ceph-mon[35121]: pgmap v114: 129 pgs: 129 active+clean; 655 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 989 op/s 2024-09-16T01:44:09.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:08 smithi139 ceph-mon[31175]: pgmap v115: 129 pgs: 129 active+clean; 670 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 524 op/s 2024-09-16T01:44:09.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:08 smithi176 ceph-mon[35324]: pgmap v115: 129 pgs: 129 active+clean; 670 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 524 op/s 2024-09-16T01:44:09.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:08 smithi164 ceph-mon[35121]: pgmap v115: 129 pgs: 129 active+clean; 670 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 524 op/s 2024-09-16T01:44:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.691+0000 7ffaeb4d7640 1 -- 172.21.15.139:0/4034594933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffae4101ad0 msgr2=0x7ffae4101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.691+0000 7ffaeb4d7640 1 --2- 172.21.15.139:0/4034594933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffae4101ad0 0x7ffae4101f30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009cd0 tx=0x7ffad802a8e0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.691+0000 7ffaeb4d7640 1 -- 172.21.15.139:0/4034594933 shutdown_connections 2024-09-16T01:44:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.691+0000 7ffaeb4d7640 1 --2- 172.21.15.139:0/4034594933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffae4102470 0x7ffae410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.691+0000 7ffaeb4d7640 1 --2- 172.21.15.139:0/4034594933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffae4101ad0 0x7ffae4101f30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.691+0000 7ffaeb4d7640 1 --2- 172.21.15.139:0/4034594933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffae4107b70 0x7ffae4107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.691+0000 7ffaeb4d7640 1 -- 172.21.15.139:0/4034594933 >> 172.21.15.139:0/4034594933 conn(0x7ffae40fd7d0 msgr2=0x7ffae40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.691+0000 7ffaeb4d7640 1 -- 172.21.15.139:0/4034594933 shutdown_connections 2024-09-16T01:44:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.691+0000 7ffaeb4d7640 1 -- 172.21.15.139:0/4034594933 wait complete. 2024-09-16T01:44:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.691+0000 7ffaeb4d7640 1 Processor -- start 2024-09-16T01:44:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.691+0000 7ffaeb4d7640 1 -- start start 2024-09-16T01:44:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffaeb4d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffae4101ad0 0x7ffae40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffaeb4d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffae4102470 0x7ffae4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffaeb4d7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffae4107b70 0x7ffae4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffaeb4d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffae4111f20 con 0x7ffae4102470 2024-09-16T01:44:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffaeb4d7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffae4111da0 con 0x7ffae4101ad0 2024-09-16T01:44:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffaeb4d7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41120a0 con 0x7ffae4107b70 2024-09-16T01:44:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffae8a72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffae4102470 0x7ffae4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffae8a72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffae4102470 0x7ffae4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60246/0 (socket says 172.21.15.139:60246) 2024-09-16T01:44:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffae8a72640 1 -- 172.21.15.139:0/2726033084 learned_addr learned my addr 172.21.15.139:0/2726033084 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffae9273640 1 --2- 172.21.15.139:0/2726033084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffae4101ad0 0x7ffae40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffae9a74640 1 --2- 172.21.15.139:0/2726033084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffae4107b70 0x7ffae4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffae8a72640 1 -- 172.21.15.139:0/2726033084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffae4107b70 msgr2=0x7ffae4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffae8a72640 1 --2- 172.21.15.139:0/2726033084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffae4107b70 0x7ffae4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffae8a72640 1 -- 172.21.15.139:0/2726033084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffae4101ad0 msgr2=0x7ffae40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffae8a72640 1 --2- 172.21.15.139:0/2726033084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffae4101ad0 0x7ffae40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffae8a72640 1 -- 172.21.15.139:0/2726033084 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae41a68b0 con 0x7ffae4102470 2024-09-16T01:44:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffae8a72640 1 --2- 172.21.15.139:0/2726033084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffae4102470 0x7ffae4074a30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ffad802ae00 tx=0x7ffad8031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.692+0000 7ffae27fc640 1 -- 172.21.15.139:0/2726033084 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8003e00 con 0x7ffae4102470 2024-09-16T01:44:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.693+0000 7ffaeb4d7640 1 -- 172.21.15.139:0/2726033084 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae41a6b40 con 0x7ffae4102470 2024-09-16T01:44:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.693+0000 7ffae27fc640 1 -- 172.21.15.139:0/2726033084 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffad8004520 con 0x7ffae4102470 2024-09-16T01:44:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.693+0000 7ffaeb4d7640 1 -- 172.21.15.139:0/2726033084 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae41a70e0 con 0x7ffae4102470 2024-09-16T01:44:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.693+0000 7ffae27fc640 1 -- 172.21.15.139:0/2726033084 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8030de0 con 0x7ffae4102470 2024-09-16T01:44:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.695+0000 7ffae27fc640 1 -- 172.21.15.139:0/2726033084 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ffad80046d0 con 0x7ffae4102470 2024-09-16T01:44:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.695+0000 7ffaee272740 1 Processor -- start 2024-09-16T01:44:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.695+0000 7ffaee272740 1 -- start start 2024-09-16T01:44:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffaee272740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf4faf0 0x5579cdf4fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffaee272740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf50410 0x5579cdf5a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffaee272740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf5acb0 0x5579cdf5d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffaee272740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5579cdeb42f0 con 0x5579cdf50410 2024-09-16T01:44:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffaee272740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5579cde5c270 con 0x5579cdf4faf0 2024-09-16T01:44:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffaee272740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5579cdeb02b0 con 0x5579cdf5acb0 2024-09-16T01:44:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffae9a74640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf4faf0 0x5579cdf4fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffae27fc640 1 --2- 172.21.15.139:0/2726033084 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ffacc0a3460 0x7ffacc0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffae8a72640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf5acb0 0x5579cdf5d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffae9273640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf50410 0x5579cdf5a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffae9a74640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf4faf0 0x5579cdf4fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:59332/0 (socket says 172.21.15.139:59332) 2024-09-16T01:44:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffae9a74640 1 -- 172.21.15.139:0/4100283020 learned_addr learned my addr 172.21.15.139:0/4100283020 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffae9a74640 1 --2- 172.21.15.139:0/2726033084 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ffacc0a3460 0x7ffacc0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffae9a74640 1 -- 172.21.15.139:0/4100283020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf5acb0 msgr2=0x5579cdf5d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffae9a74640 1 --2- 172.21.15.139:0/4100283020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf5acb0 0x5579cdf5d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffae9a74640 1 -- 172.21.15.139:0/4100283020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf50410 msgr2=0x5579cdf5a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffae9a74640 1 --2- 172.21.15.139:0/4100283020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf50410 0x5579cdf5a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffae9a74640 1 -- 172.21.15.139:0/4100283020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579cde5b340 con 0x5579cdf4faf0 2024-09-16T01:44:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffae9273640 1 --2- 172.21.15.139:0/4100283020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf50410 0x5579cdf5a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:44:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.696+0000 7ffae27fc640 1 -- 172.21.15.139:0/2726033084 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7ffad802e030 con 0x7ffae4102470 2024-09-16T01:44:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.697+0000 7ffae9a74640 1 --2- 172.21.15.139:0/4100283020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf4faf0 0x5579cdf4fed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffad40138b0 tx=0x7ffad4013d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f232d2b40491523 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.697+0000 7ffae9a74640 1 --2- 172.21.15.139:0/2726033084 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ffacc0a3460 0x7ffacc0a5920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffad401e000 tx=0x7ffad4020000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.697+0000 7ffacb7fe640 1 -- 172.21.15.139:0/4100283020 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4012280 con 0x5579cdf4faf0 2024-09-16T01:44:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.697+0000 7ffacb7fe640 1 -- 172.21.15.139:0/4100283020 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffad4023070 con 0x5579cdf4faf0 2024-09-16T01:44:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.697+0000 7ffacb7fe640 1 -- 172.21.15.139:0/4100283020 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad40035f0 con 0x5579cdf4faf0 2024-09-16T01:44:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 -- 172.21.15.139:0/4100283020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf4faf0 msgr2=0x5579cdf4fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 --2- 172.21.15.139:0/4100283020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf4faf0 0x5579cdf4fed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffad40138b0 tx=0x7ffad4013d70 comp rx=0 tx=0).stop 2024-09-16T01:44:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 -- 172.21.15.139:0/4100283020 shutdown_connections 2024-09-16T01:44:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 --2- 172.21.15.139:0/4100283020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf5acb0 0x5579cdf5d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 --2- 172.21.15.139:0/4100283020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf50410 0x5579cdf5a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 --2- 172.21.15.139:0/4100283020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf4faf0 0x5579cdf4fed0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 -- 172.21.15.139:0/4100283020 >> 172.21.15.139:0/4100283020 conn(0x5579cdf4c060 msgr2=0x5579cdf4c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 -- 172.21.15.139:0/4100283020 shutdown_connections 2024-09-16T01:44:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 -- 172.21.15.139:0/4100283020 wait complete. 2024-09-16T01:44:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 -- start start 2024-09-16T01:44:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 Processor -- start 2024-09-16T01:44:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf4faf0 0x5579cdf57bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf50410 0x5579cdf58100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf58640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5579cdf5d880 con 0x5579cdf5acb0 2024-09-16T01:44:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5579cde3d290 con 0x5579cdf50410 2024-09-16T01:44:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.698+0000 7ffaee272740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5579cdf5da00 con 0x5579cdf4faf0 2024-09-16T01:44:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffaee272740 10 client.0 _ll_register_callbacks cb 0x5579cde6e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:44:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffae9273640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf4faf0 0x5579cdf57bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffae9a74640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf58640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffae9273640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf4faf0 0x5579cdf57bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:55690/0 (socket says 172.21.15.139:55690) 2024-09-16T01:44:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffae9273640 1 -- 172.21.15.139:0/4094635478 learned_addr learned my addr 172.21.15.139:0/4094635478 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffae8a72640 1 --2- 172.21.15.139:0/4094635478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf50410 0x5579cdf58100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffae9a74640 1 -- 172.21.15.139:0/4094635478 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf4faf0 msgr2=0x5579cdf57bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffae9a74640 1 --2- 172.21.15.139:0/4094635478 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf4faf0 0x5579cdf57bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffae9a74640 1 -- 172.21.15.139:0/4094635478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf50410 msgr2=0x5579cdf58100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffae9a74640 1 --2- 172.21.15.139:0/4094635478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf50410 0x5579cdf58100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffae9a74640 1 -- 172.21.15.139:0/4094635478 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579cdbc7210 con 0x5579cdf5acb0 2024-09-16T01:44:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffae9273640 1 --2- 172.21.15.139:0/4094635478 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf4faf0 0x5579cdf57bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:44:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffae9a74640 1 --2- 172.21.15.139:0/4094635478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf58640 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ffad400a010 tx=0x7ffad401ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09dd8e2539f5b54 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffac8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:44:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffac8ff9640 1 -- 172.21.15.139:0/4094635478 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4005250 con 0x5579cdf5acb0 2024-09-16T01:44:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffaee272740 1 -- 172.21.15.139:0/4094635478 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffad4023070 con 0x5579cdf5acb0 2024-09-16T01:44:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.699+0000 7ffac8ff9640 1 -- 172.21.15.139:0/4094635478 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7ffad40053f0 con 0x5579cdf5acb0 2024-09-16T01:44:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.700+0000 7ffac8ff9640 1 -- 172.21.15.139:0/4094635478 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffad4012290 con 0x5579cdf5acb0 2024-09-16T01:44:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.700+0000 7ffaee272740 10 client.15948 fetch_fsmap learned FSMap version 31 2024-09-16T01:44:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.700+0000 7ffaee272740 1 -- 172.21.15.139:0/4094635478 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5579cdc93620 con 0x5579cdf5acb0 2024-09-16T01:44:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.700+0000 7ffac8ff9640 1 -- 172.21.15.139:0/4094635478 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffad4023070 con 0x5579cdf5acb0 2024-09-16T01:44:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.700+0000 7ffac8ff9640 1 -- 172.21.15.139:0/4094635478 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7ffad4027450 con 0x5579cdf5acb0 2024-09-16T01:44:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.700+0000 7ffaee272740 10 client.15948 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:44:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.700+0000 7ffaee272740 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:44:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.700+0000 7ffaee272740 20 client.15948 populate_metadata read hostname 'smithi139' 2024-09-16T01:44:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.700+0000 7ffaee272740 1 --2- 172.21.15.139:0/4094635478 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5579cdf6aab0 0x5579cdf6cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.700+0000 7ffaee272740 4 client.15948 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:44:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.700+0000 7ffaee272740 1 -- 172.21.15.139:0/4094635478 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579cdec1a20 con 0x5579cdf6aab0 2024-09-16T01:44:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.700+0000 7ffae9273640 1 --2- 172.21.15.139:0/4094635478 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5579cdf6aab0 0x5579cdf6cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.701+0000 7ffac8ff9640 10 client.15948 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.701+0000 7ffae9273640 1 --2- 172.21.15.139:0/4094635478 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5579cdf6aab0 0x5579cdf6cea0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b2e997a143d553 server_cookie=9249dd6f15ac30af in_seq=0 out_seq=0 2024-09-16T01:44:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.702+0000 7ffac8ff9640 1 -- 172.21.15.139:0/4094635478 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579cdec1a20 con 0x5579cdf6aab0 2024-09-16T01:44:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.702+0000 7ffac8ff9640 10 client.15948 handle_command_reply: tid=0 2024-09-16T01:44:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.702+0000 7ffaee272740 1 client.15948 shutdown 2024-09-16T01:44:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.702+0000 7ffaee272740 1 -- 172.21.15.139:0/4094635478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 msgr2=0x5579cdf58640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.702+0000 7ffaee272740 1 --2- 172.21.15.139:0/4094635478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf58640 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ffad400a010 tx=0x7ffad401ff70 comp rx=0 tx=0).stop 2024-09-16T01:44:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.702+0000 7ffaee272740 1 -- 172.21.15.139:0/4094635478 shutdown_connections 2024-09-16T01:44:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.702+0000 7ffaee272740 1 --2- 172.21.15.139:0/4094635478 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5579cdf6aab0 0x5579cdf6cea0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.702+0000 7ffaee272740 1 --2- 172.21.15.139:0/4094635478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf58640 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.702+0000 7ffaee272740 1 --2- 172.21.15.139:0/4094635478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf50410 0x5579cdf58100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.702+0000 7ffaee272740 1 --2- 172.21.15.139:0/4094635478 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf4faf0 0x5579cdf57bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.702+0000 7ffaee272740 1 -- 172.21.15.139:0/4094635478 >> 172.21.15.139:0/4094635478 conn(0x5579cdf4c060 msgr2=0x5579cdf51dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.702+0000 7ffac8ff9640 0 client.15948 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.703+0000 7ffaee272740 1 -- 172.21.15.139:0/4094635478 shutdown_connections 2024-09-16T01:44:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.703+0000 7ffaee272740 1 -- 172.21.15.139:0/4094635478 wait complete. 2024-09-16T01:44:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.703+0000 7ffaee272740 20 client.15948 trim_cache size 0 max 16384 2024-09-16T01:44:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffaee272740 1 Processor -- start 2024-09-16T01:44:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffaee272740 1 -- start start 2024-09-16T01:44:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffaee272740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf4faf0 0x5579cdf58440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffaee272740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf50410 0x5579cdf4b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffaee272740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf72710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffaee272740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41b1240 con 0x5579cdf5acb0 2024-09-16T01:44:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffaee272740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41b2e40 con 0x5579cdf50410 2024-09-16T01:44:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffaee272740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41b1910 con 0x5579cdf4faf0 2024-09-16T01:44:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffae9a74640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf72710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffae9a74640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf72710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60272/0 (socket says 172.21.15.139:60272) 2024-09-16T01:44:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffae8a72640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf50410 0x5579cdf4b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffae9273640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf4faf0 0x5579cdf58440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffae9a74640 1 -- 172.21.15.139:0/2350520365 learned_addr learned my addr 172.21.15.139:0/2350520365 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffae9a74640 1 -- 172.21.15.139:0/2350520365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf4faf0 msgr2=0x5579cdf58440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffae9a74640 1 --2- 172.21.15.139:0/2350520365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf4faf0 0x5579cdf58440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffae9a74640 1 -- 172.21.15.139:0/2350520365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf50410 msgr2=0x5579cdf4b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffae9a74640 1 --2- 172.21.15.139:0/2350520365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf50410 0x5579cdf4b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffae9a74640 1 -- 172.21.15.139:0/2350520365 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579cdf50240 con 0x5579cdf5acb0 2024-09-16T01:44:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.720+0000 7ffae9a74640 1 --2- 172.21.15.139:0/2350520365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf72710 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ffad4027820 tx=0x7ffad40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9818b9c91e39f6a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.721+0000 7ffacaffd640 1 -- 172.21.15.139:0/2350520365 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad401f850 con 0x5579cdf5acb0 2024-09-16T01:44:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.721+0000 7ffacaffd640 1 -- 172.21.15.139:0/2350520365 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffad402c070 con 0x5579cdf5acb0 2024-09-16T01:44:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.721+0000 7ffacaffd640 1 -- 172.21.15.139:0/2350520365 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4005340 con 0x5579cdf5acb0 2024-09-16T01:44:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 -- 172.21.15.139:0/2350520365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 msgr2=0x5579cdf72710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 --2- 172.21.15.139:0/2350520365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf72710 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ffad4027820 tx=0x7ffad40096a0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 -- 172.21.15.139:0/2350520365 shutdown_connections 2024-09-16T01:44:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 --2- 172.21.15.139:0/2350520365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf72710 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 --2- 172.21.15.139:0/2350520365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf50410 0x5579cdf4b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 --2- 172.21.15.139:0/2350520365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf4faf0 0x5579cdf58440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 -- 172.21.15.139:0/2350520365 >> 172.21.15.139:0/2350520365 conn(0x5579cdf4c060 msgr2=0x5579cdf4a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 -- 172.21.15.139:0/2350520365 shutdown_connections 2024-09-16T01:44:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 -- 172.21.15.139:0/2350520365 wait complete. 2024-09-16T01:44:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 -- start start 2024-09-16T01:44:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 Processor -- start 2024-09-16T01:44:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf4faf0 0x5579cdf5ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf50410 0x5579cdf5ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf5f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41b26b0 con 0x5579cdf5acb0 2024-09-16T01:44:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41b1fe0 con 0x5579cdf4faf0 2024-09-16T01:44:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41af220 con 0x5579cdf50410 2024-09-16T01:44:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.722+0000 7ffaee272740 10 client.0 _ll_register_callbacks cb 0x7ffadc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:44:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffae9273640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf4faf0 0x5579cdf5ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffae9a74640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf5f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffae8a72640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf50410 0x5579cdf5ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffae9273640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf4faf0 0x5579cdf5ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:59376/0 (socket says 172.21.15.139:59376) 2024-09-16T01:44:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffae9273640 1 -- 172.21.15.139:0/3649413986 learned_addr learned my addr 172.21.15.139:0/3649413986 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffae9273640 1 -- 172.21.15.139:0/3649413986 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf50410 msgr2=0x5579cdf5ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffae9273640 1 --2- 172.21.15.139:0/3649413986 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf50410 0x5579cdf5ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffae9273640 1 -- 172.21.15.139:0/3649413986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 msgr2=0x5579cdf5f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffae9273640 1 --2- 172.21.15.139:0/3649413986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf5f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffae9273640 1 -- 172.21.15.139:0/3649413986 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579cdfd4710 con 0x5579cdf4faf0 2024-09-16T01:44:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffae9a74640 1 --2- 172.21.15.139:0/3649413986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf5f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:44:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffae8a72640 1 --2- 172.21.15.139:0/3649413986 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf50410 0x5579cdf5ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:44:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffae9273640 1 --2- 172.21.15.139:0/3649413986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf4faf0 0x5579cdf5ea20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x5579cdf6d6b0 tx=0x7ffadc012b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64489c34a4edb3c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffac8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:44:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffac8ff9640 1 -- 172.21.15.139:0/3649413986 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc02b060 con 0x5579cdf4faf0 2024-09-16T01:44:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffaee272740 1 -- 172.21.15.139:0/3649413986 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffad402c070 con 0x5579cdf4faf0 2024-09-16T01:44:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffac8ff9640 1 -- 172.21.15.139:0/3649413986 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7ffadc004230 con 0x5579cdf4faf0 2024-09-16T01:44:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffac8ff9640 1 -- 172.21.15.139:0/3649413986 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffadc032050 con 0x5579cdf4faf0 2024-09-16T01:44:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffaee272740 10 client.25442 fetch_fsmap learned FSMap version 31 2024-09-16T01:44:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.723+0000 7ffaee272740 1 -- 172.21.15.139:0/3649413986 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5579cdfd5ab0 con 0x5579cdf4faf0 2024-09-16T01:44:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.724+0000 7ffac8ff9640 1 -- 172.21.15.139:0/3649413986 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffad402c070 con 0x5579cdf4faf0 2024-09-16T01:44:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.724+0000 7ffac8ff9640 1 -- 172.21.15.139:0/3649413986 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7ffadc025070 con 0x5579cdf4faf0 2024-09-16T01:44:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.724+0000 7ffaee272740 10 client.25442 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T01:44:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.724+0000 7ffaee272740 10 client.25442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:44:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.724+0000 7ffaee272740 20 client.25442 populate_metadata read hostname 'smithi139' 2024-09-16T01:44:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.724+0000 7ffaee272740 1 --2- 172.21.15.139:0/3649413986 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5579cdfd62a0 0x5579cdfd86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.724+0000 7ffaee272740 4 client.25442 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:44:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.724+0000 7ffaee272740 1 -- 172.21.15.139:0/3649413986 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579cdebbd70 con 0x5579cdfd62a0 2024-09-16T01:44:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.724+0000 7ffae8a72640 1 --2- 172.21.15.139:0/3649413986 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5579cdfd62a0 0x5579cdfd86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.724+0000 7ffac8ff9640 10 client.25442 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:09.724+0000 7ffae8a72640 1 --2- 172.21.15.139:0/3649413986 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5579cdfd62a0 0x5579cdfd86c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72fa9a3af95e843d server_cookie=cc8754ac5f755cc in_seq=0 out_seq=0 2024-09-16T01:44:11.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:10 smithi139 ceph-mon[31175]: pgmap v116: 129 pgs: 129 active+clean; 673 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 539 op/s 2024-09-16T01:44:11.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:10 smithi176 ceph-mon[35324]: pgmap v116: 129 pgs: 129 active+clean; 673 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 539 op/s 2024-09-16T01:44:11.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:10 smithi164 ceph-mon[35121]: pgmap v116: 129 pgs: 129 active+clean; 673 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 539 op/s 2024-09-16T01:44:13.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:12 smithi139 ceph-mon[31175]: pgmap v117: 129 pgs: 129 active+clean; 819 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 23 MiB/s wr, 948 op/s 2024-09-16T01:44:13.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:12 smithi176 ceph-mon[35324]: pgmap v117: 129 pgs: 129 active+clean; 819 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 23 MiB/s wr, 948 op/s 2024-09-16T01:44:13.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:12 smithi164 ceph-mon[35121]: pgmap v117: 129 pgs: 129 active+clean; 819 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 23 MiB/s wr, 948 op/s 2024-09-16T01:44:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.656+0000 7ffac8ff9640 1 -- 172.21.15.139:0/3649413986 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579cdebbd70 con 0x5579cdfd62a0 2024-09-16T01:44:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.656+0000 7ffac8ff9640 10 client.25442 handle_command_reply: tid=0 2024-09-16T01:44:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.656+0000 7ffaee272740 1 client.25442 shutdown 2024-09-16T01:44:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.656+0000 7ffaee272740 1 -- 172.21.15.139:0/3649413986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf4faf0 msgr2=0x5579cdf5ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.656+0000 7ffaee272740 1 --2- 172.21.15.139:0/3649413986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf4faf0 0x5579cdf5ea20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x5579cdf6d6b0 tx=0x7ffadc012b50 comp rx=0 tx=0).stop 2024-09-16T01:44:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.656+0000 7ffaee272740 1 -- 172.21.15.139:0/3649413986 shutdown_connections 2024-09-16T01:44:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.656+0000 7ffaee272740 1 --2- 172.21.15.139:0/3649413986 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5579cdfd62a0 0x5579cdfd86c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.656+0000 7ffaee272740 1 --2- 172.21.15.139:0/3649413986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5579cdf5acb0 0x5579cdf5f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.656+0000 7ffaee272740 1 --2- 172.21.15.139:0/3649413986 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579cdf50410 0x5579cdf5ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.656+0000 7ffaee272740 1 --2- 172.21.15.139:0/3649413986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579cdf4faf0 0x5579cdf5ea20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.656+0000 7ffaee272740 1 -- 172.21.15.139:0/3649413986 >> 172.21.15.139:0/3649413986 conn(0x5579cdf4c060 msgr2=0x5579cdf51d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.656+0000 7ffac8ff9640 0 client.25442 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.657+0000 7ffaee272740 1 -- 172.21.15.139:0/3649413986 shutdown_connections 2024-09-16T01:44:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.657+0000 7ffaee272740 1 -- 172.21.15.139:0/3649413986 wait complete. 2024-09-16T01:44:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.657+0000 7ffaee272740 20 client.25442 trim_cache size 0 max 16384 2024-09-16T01:44:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:44:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:44:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:44:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:44:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.657+0000 7ffaeb4d7640 1 -- 172.21.15.139:0/2726033084 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ffacc0a3460 msgr2=0x7ffacc0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.657+0000 7ffaeb4d7640 1 --2- 172.21.15.139:0/2726033084 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ffacc0a3460 0x7ffacc0a5920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffad401e000 tx=0x7ffad4020000 comp rx=0 tx=0).stop 2024-09-16T01:44:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.657+0000 7ffaeb4d7640 1 -- 172.21.15.139:0/2726033084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffae4102470 msgr2=0x7ffae4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.657+0000 7ffaeb4d7640 1 --2- 172.21.15.139:0/2726033084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffae4102470 0x7ffae4074a30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ffad802ae00 tx=0x7ffad8031e60 comp rx=0 tx=0).stop 2024-09-16T01:44:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.658+0000 7ffaeb4d7640 1 -- 172.21.15.139:0/2726033084 shutdown_connections 2024-09-16T01:44:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.658+0000 7ffaeb4d7640 1 --2- 172.21.15.139:0/2726033084 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ffacc0a3460 0x7ffacc0a5920 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.658+0000 7ffaeb4d7640 1 --2- 172.21.15.139:0/2726033084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffae4107b70 0x7ffae4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.658+0000 7ffaeb4d7640 1 --2- 172.21.15.139:0/2726033084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffae4102470 0x7ffae4074a30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.658+0000 7ffaeb4d7640 1 --2- 172.21.15.139:0/2726033084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffae4101ad0 0x7ffae40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.658+0000 7ffaeb4d7640 1 -- 172.21.15.139:0/2726033084 >> 172.21.15.139:0/2726033084 conn(0x7ffae40fd7d0 msgr2=0x7ffae40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.658+0000 7ffaeb4d7640 1 -- 172.21.15.139:0/2726033084 shutdown_connections 2024-09-16T01:44:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:13.658+0000 7ffaeb4d7640 1 -- 172.21.15.139:0/2726033084 wait complete. 2024-09-16T01:44:15.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:14 smithi139 ceph-mon[31175]: pgmap v118: 129 pgs: 129 active+clean; 819 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 19 MiB/s wr, 739 op/s 2024-09-16T01:44:15.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:14 smithi176 ceph-mon[35324]: pgmap v118: 129 pgs: 129 active+clean; 819 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 19 MiB/s wr, 739 op/s 2024-09-16T01:44:15.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:14 smithi164 ceph-mon[35121]: pgmap v118: 129 pgs: 129 active+clean; 819 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 19 MiB/s wr, 739 op/s 2024-09-16T01:44:15.968 INFO:tasks.workunit.client.0.smithi139.stdout: 285 820314 1651 577288 1299 132701 195 2024-09-16T01:44:17.099 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:16 smithi164 ceph-mon[35121]: pgmap v119: 129 pgs: 129 active+clean; 822 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 20 MiB/s wr, 984 op/s 2024-09-16T01:44:17.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:16 smithi139 ceph-mon[31175]: pgmap v119: 129 pgs: 129 active+clean; 822 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 20 MiB/s wr, 984 op/s 2024-09-16T01:44:17.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:16 smithi176 ceph-mon[35324]: pgmap v119: 129 pgs: 129 active+clean; 822 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 20 MiB/s wr, 984 op/s 2024-09-16T01:44:19.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:18 smithi139 ceph-mon[31175]: pgmap v120: 129 pgs: 129 active+clean; 816 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 16 MiB/s wr, 833 op/s 2024-09-16T01:44:19.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:18 smithi176 ceph-mon[35324]: pgmap v120: 129 pgs: 129 active+clean; 816 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 16 MiB/s wr, 833 op/s 2024-09-16T01:44:19.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:18 smithi164 ceph-mon[35121]: pgmap v120: 129 pgs: 129 active+clean; 816 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 16 MiB/s wr, 833 op/s 2024-09-16T01:44:21.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:20 smithi139 ceph-mon[31175]: pgmap v121: 129 pgs: 129 active+clean; 816 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 14 MiB/s wr, 768 op/s 2024-09-16T01:44:21.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:20 smithi176 ceph-mon[35324]: pgmap v121: 129 pgs: 129 active+clean; 816 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 14 MiB/s wr, 768 op/s 2024-09-16T01:44:21.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:20 smithi164 ceph-mon[35121]: pgmap v121: 129 pgs: 129 active+clean; 816 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 14 MiB/s wr, 768 op/s 2024-09-16T01:44:23.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:22 smithi139 ceph-mon[31175]: pgmap v122: 129 pgs: 129 active+clean; 845 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 862 op/s 2024-09-16T01:44:23.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:22 smithi176 ceph-mon[35324]: pgmap v122: 129 pgs: 129 active+clean; 845 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 862 op/s 2024-09-16T01:44:23.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:22 smithi164 ceph-mon[35121]: pgmap v122: 129 pgs: 129 active+clean; 845 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 17 MiB/s wr, 862 op/s 2024-09-16T01:44:25.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:24 smithi139 ceph-mon[31175]: pgmap v123: 129 pgs: 129 active+clean; 845 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 411 op/s 2024-09-16T01:44:25.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:24 smithi176 ceph-mon[35324]: pgmap v123: 129 pgs: 129 active+clean; 845 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 411 op/s 2024-09-16T01:44:25.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:24 smithi164 ceph-mon[35121]: pgmap v123: 129 pgs: 129 active+clean; 845 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 411 op/s 2024-09-16T01:44:25.968 INFO:tasks.workunit.client.0.smithi139.stdout: 313 2172463 1668 1535064 1635 339425 1079 2024-09-16T01:44:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.858+0000 7feb23949640 1 -- 172.21.15.139:0/841655146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1c101370 msgr2=0x7feb1c10d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.858+0000 7feb23949640 1 --2- 172.21.15.139:0/841655146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1c101370 0x7feb1c10d8a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feb1400bc30 tx=0x7feb1402e690 comp rx=0 tx=0).stop 2024-09-16T01:44:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.865+0000 7feb23949640 1 -- 172.21.15.139:0/841655146 shutdown_connections 2024-09-16T01:44:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.865+0000 7feb23949640 1 --2- 172.21.15.139:0/841655146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1c101370 0x7feb1c10d8a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.866+0000 7feb23949640 1 --2- 172.21.15.139:0/841655146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb1c1009d0 0x7feb1c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.866+0000 7feb23949640 1 --2- 172.21.15.139:0/841655146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb1c1069e0 0x7feb1c106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.866+0000 7feb23949640 1 -- 172.21.15.139:0/841655146 >> 172.21.15.139:0/841655146 conn(0x7feb1c0fc6b0 msgr2=0x7feb1c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.871+0000 7feb23949640 1 -- 172.21.15.139:0/841655146 shutdown_connections 2024-09-16T01:44:26.875 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.874+0000 7feb23949640 1 -- 172.21.15.139:0/841655146 wait complete. 2024-09-16T01:44:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.875+0000 7feb23949640 1 Processor -- start 2024-09-16T01:44:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.877+0000 7feb23949640 1 -- start start 2024-09-16T01:44:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.878+0000 7feb23949640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1c1009d0 0x7feb1c196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.878+0000 7feb23949640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb1c101370 0x7feb1c196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.878+0000 7feb23949640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb1c1069e0 0x7feb1c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.878+0000 7feb23949640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c10fd20 con 0x7feb1c1069e0 2024-09-16T01:44:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.878+0000 7feb23949640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c10fba0 con 0x7feb1c101370 2024-09-16T01:44:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.878+0000 7feb23949640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c10fea0 con 0x7feb1c1009d0 2024-09-16T01:44:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.880+0000 7feb20ee4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb1c101370 0x7feb1c196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.880+0000 7feb20ee4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb1c101370 0x7feb1c196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37824/0 (socket says 172.21.15.139:37824) 2024-09-16T01:44:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.880+0000 7feb20ee4640 1 -- 172.21.15.139:0/2281018180 learned_addr learned my addr 172.21.15.139:0/2281018180 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.880+0000 7feb216e5640 1 --2- 172.21.15.139:0/2281018180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1c1009d0 0x7feb1c196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.881+0000 7feb21ee6640 1 --2- 172.21.15.139:0/2281018180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb1c1069e0 0x7feb1c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.882+0000 7feb20ee4640 1 -- 172.21.15.139:0/2281018180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1c1009d0 msgr2=0x7feb1c196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.882+0000 7feb20ee4640 1 --2- 172.21.15.139:0/2281018180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1c1009d0 0x7feb1c196140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.882+0000 7feb20ee4640 1 -- 172.21.15.139:0/2281018180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb1c1069e0 msgr2=0x7feb1c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.882+0000 7feb20ee4640 1 --2- 172.21.15.139:0/2281018180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb1c1069e0 0x7feb1c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.882+0000 7feb20ee4640 1 -- 172.21.15.139:0/2281018180 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c19b130 con 0x7feb1c101370 2024-09-16T01:44:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.883+0000 7feb20ee4640 1 --2- 172.21.15.139:0/2281018180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb1c101370 0x7feb1c196680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feb1800c890 tx=0x7feb1800cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.884+0000 7feb0e7fc640 1 -- 172.21.15.139:0/2281018180 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb18004220 con 0x7feb1c101370 2024-09-16T01:44:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.884+0000 7feb0e7fc640 1 -- 172.21.15.139:0/2281018180 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb180043c0 con 0x7feb1c101370 2024-09-16T01:44:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.884+0000 7feb0e7fc640 1 -- 172.21.15.139:0/2281018180 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb180049c0 con 0x7feb1c101370 2024-09-16T01:44:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.887+0000 7feb23949640 1 -- 172.21.15.139:0/2281018180 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1c19b420 con 0x7feb1c101370 2024-09-16T01:44:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.887+0000 7feb23949640 1 -- 172.21.15.139:0/2281018180 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1c1a2cb0 con 0x7feb1c101370 2024-09-16T01:44:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.894+0000 7feb267bd740 1 Processor -- start 2024-09-16T01:44:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.900+0000 7feb0e7fc640 1 -- 172.21.15.139:0/2281018180 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7feb18004b60 con 0x7feb1c101370 2024-09-16T01:44:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.902+0000 7feb267bd740 1 -- start start 2024-09-16T01:44:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.905+0000 7feb267bd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d746e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.906+0000 7feb267bd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d74c20 0x555a95d7ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.906+0000 7feb267bd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d7f4c0 0x555a95d818b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.906+0000 7feb267bd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555a95cd02f0 con 0x555a95d74300 2024-09-16T01:44:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.906+0000 7feb267bd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555a95c78270 con 0x555a95d74c20 2024-09-16T01:44:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.906+0000 7feb267bd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555a95ccc2b0 con 0x555a95d7f4c0 2024-09-16T01:44:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.906+0000 7feb0e7fc640 1 --2- 172.21.15.139:0/2281018180 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feb080a3690 0x7feb080a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.906+0000 7feb20ee4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d7f4c0 0x555a95d818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.906+0000 7feb0e7fc640 1 -- 172.21.15.139:0/2281018180 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7feb180d00a0 con 0x7feb1c101370 2024-09-16T01:44:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.907+0000 7feb20ee4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d7f4c0 0x555a95d818b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:48334/0 (socket says 172.21.15.139:48334) 2024-09-16T01:44:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.907+0000 7feb20ee4640 1 -- 172.21.15.139:0/3167938297 learned_addr learned my addr 172.21.15.139:0/3167938297 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.907+0000 7feb20ee4640 1 -- 172.21.15.139:0/3167938297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d74c20 msgr2=0x555a95d7ef80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:44:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.907+0000 7feb20ee4640 1 --2- 172.21.15.139:0/3167938297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d74c20 0x555a95d7ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.907+0000 7feb20ee4640 1 -- 172.21.15.139:0/3167938297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 msgr2=0x555a95d746e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:44:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.907+0000 7feb20ee4640 1 --2- 172.21.15.139:0/3167938297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d746e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.907+0000 7feb20ee4640 1 -- 172.21.15.139:0/3167938297 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a95c77340 con 0x555a95d7f4c0 2024-09-16T01:44:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.908+0000 7feb20ee4640 1 --2- 172.21.15.139:0/3167938297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d7f4c0 0x555a95d818b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb18007590 tx=0x7feb180054c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e210ffa20be3ba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.909+0000 7feb077fe640 1 -- 172.21.15.139:0/3167938297 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb180b4070 con 0x555a95d7f4c0 2024-09-16T01:44:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.909+0000 7feb077fe640 1 -- 172.21.15.139:0/3167938297 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb180a03a0 con 0x555a95d7f4c0 2024-09-16T01:44:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.909+0000 7feb077fe640 1 -- 172.21.15.139:0/3167938297 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb180a47d0 con 0x555a95d7f4c0 2024-09-16T01:44:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.912+0000 7feb267bd740 1 -- 172.21.15.139:0/3167938297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d7f4c0 msgr2=0x555a95d818b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.912+0000 7feb267bd740 1 --2- 172.21.15.139:0/3167938297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d7f4c0 0x555a95d818b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb18007590 tx=0x7feb180054c0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.914+0000 7feb21ee6640 1 --2- 172.21.15.139:0/2281018180 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feb080a3690 0x7feb080a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.920+0000 7feb267bd740 1 -- 172.21.15.139:0/3167938297 shutdown_connections 2024-09-16T01:44:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.920+0000 7feb267bd740 1 --2- 172.21.15.139:0/3167938297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d7f4c0 0x555a95d818b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.920+0000 7feb267bd740 1 --2- 172.21.15.139:0/3167938297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d74c20 0x555a95d7ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.920+0000 7feb267bd740 1 --2- 172.21.15.139:0/3167938297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.920+0000 7feb267bd740 1 -- 172.21.15.139:0/3167938297 >> 172.21.15.139:0/3167938297 conn(0x555a95d70870 msgr2=0x555a95d70c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.924+0000 7feb21ee6640 1 --2- 172.21.15.139:0/2281018180 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feb080a3690 0x7feb080a5b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feb14006010 tx=0x7feb1400d040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.929+0000 7feb267bd740 1 -- 172.21.15.139:0/3167938297 shutdown_connections 2024-09-16T01:44:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.932+0000 7feb267bd740 1 -- 172.21.15.139:0/3167938297 wait complete. 2024-09-16T01:44:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.932+0000 7feb267bd740 1 -- start start 2024-09-16T01:44:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.932+0000 7feb267bd740 1 Processor -- start 2024-09-16T01:44:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.936+0000 7feb267bd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d7c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.936+0000 7feb267bd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d74c20 0x555a95d7c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.936+0000 7feb267bd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d7f4c0 0x555a95d7cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.936+0000 7feb267bd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555a95d82090 con 0x555a95d74300 2024-09-16T01:44:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.936+0000 7feb267bd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555a95c59290 con 0x555a95d7f4c0 2024-09-16T01:44:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.936+0000 7feb267bd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555a95d82210 con 0x555a95d74c20 2024-09-16T01:44:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.936+0000 7feb267bd740 10 client.0 _ll_register_callbacks cb 0x555a95c8a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:44:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.937+0000 7feb216e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d7c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.937+0000 7feb216e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d7c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49976/0 (socket says 172.21.15.139:49976) 2024-09-16T01:44:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.937+0000 7feb216e5640 1 -- 172.21.15.139:0/400721178 learned_addr learned my addr 172.21.15.139:0/400721178 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.938+0000 7feb21ee6640 1 --2- 172.21.15.139:0/400721178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d7f4c0 0x555a95d7cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.938+0000 7feb216e5640 1 -- 172.21.15.139:0/400721178 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d74c20 msgr2=0x555a95d7c9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:44:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.938+0000 7feb216e5640 1 --2- 172.21.15.139:0/400721178 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d74c20 0x555a95d7c9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.938+0000 7feb216e5640 1 -- 172.21.15.139:0/400721178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d7f4c0 msgr2=0x555a95d7cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.938+0000 7feb216e5640 1 --2- 172.21.15.139:0/400721178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d7f4c0 0x555a95d7cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.938+0000 7feb216e5640 1 -- 172.21.15.139:0/400721178 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a959e3210 con 0x555a95d74300 2024-09-16T01:44:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.939+0000 7feb216e5640 1 --2- 172.21.15.139:0/400721178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d7c480 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7feb10021690 tx=0x7feb10021b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f07d0cff81f3acb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.940+0000 7feb04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:44:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.940+0000 7feb04ff9640 1 -- 172.21.15.139:0/400721178 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10028070 con 0x555a95d74300 2024-09-16T01:44:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.940+0000 7feb267bd740 1 -- 172.21.15.139:0/400721178 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb180a03a0 con 0x555a95d74300 2024-09-16T01:44:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.940+0000 7feb04ff9640 1 -- 172.21.15.139:0/400721178 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7feb1001fac0 con 0x555a95d74300 2024-09-16T01:44:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.942+0000 7feb04ff9640 1 -- 172.21.15.139:0/400721178 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb10037050 con 0x555a95d74300 2024-09-16T01:44:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.943+0000 7feb267bd740 10 client.15963 fetch_fsmap learned FSMap version 32 2024-09-16T01:44:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.944+0000 7feb267bd740 1 -- 172.21.15.139:0/400721178 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555a95aaf620 con 0x555a95d74300 2024-09-16T01:44:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.945+0000 7feb04ff9640 1 -- 172.21.15.139:0/400721178 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb180a03a0 con 0x555a95d74300 2024-09-16T01:44:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.948+0000 7feb04ff9640 1 -- 172.21.15.139:0/400721178 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7feb10022450 con 0x555a95d74300 2024-09-16T01:44:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.948+0000 7feb267bd740 10 client.15963 fetch_fsmap finished waiting for FSMap version 32 2024-09-16T01:44:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.948+0000 7feb267bd740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:44:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.948+0000 7feb267bd740 20 client.15963 populate_metadata read hostname 'smithi139' 2024-09-16T01:44:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.948+0000 7feb267bd740 1 --2- 172.21.15.139:0/400721178 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555a95d68e00 0x555a95d6b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.948+0000 7feb267bd740 4 client.15963 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:44:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.948+0000 7feb267bd740 1 -- 172.21.15.139:0/400721178 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a95cdda20 con 0x555a95d68e00 2024-09-16T01:44:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.949+0000 7feb20ee4640 1 --2- 172.21.15.139:0/400721178 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555a95d68e00 0x555a95d6b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.949+0000 7feb20ee4640 1 --2- 172.21.15.139:0/400721178 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555a95d68e00 0x555a95d6b280 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6d2eb13a746ad0a server_cookie=c6ff805fc48f56b7 in_seq=0 out_seq=0 2024-09-16T01:44:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.950+0000 7feb04ff9640 10 client.15963 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.950+0000 7feb04ff9640 1 -- 172.21.15.139:0/400721178 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a95cdda20 con 0x555a95d68e00 2024-09-16T01:44:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.950+0000 7feb04ff9640 10 client.15963 handle_command_reply: tid=0 2024-09-16T01:44:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.950+0000 7feb267bd740 1 client.15963 shutdown 2024-09-16T01:44:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.980+0000 7feb267bd740 1 -- 172.21.15.139:0/400721178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 msgr2=0x555a95d7c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.980+0000 7feb267bd740 1 --2- 172.21.15.139:0/400721178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d7c480 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7feb10021690 tx=0x7feb10021b50 comp rx=0 tx=0).stop 2024-09-16T01:44:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.982+0000 7feb267bd740 1 -- 172.21.15.139:0/400721178 shutdown_connections 2024-09-16T01:44:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.982+0000 7feb267bd740 1 --2- 172.21.15.139:0/400721178 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555a95d68e00 0x555a95d6b280 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.982+0000 7feb267bd740 1 --2- 172.21.15.139:0/400721178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d7f4c0 0x555a95d7cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.982+0000 7feb267bd740 1 --2- 172.21.15.139:0/400721178 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d74c20 0x555a95d7c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.982+0000 7feb267bd740 1 --2- 172.21.15.139:0/400721178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d7c480 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.982+0000 7feb267bd740 1 -- 172.21.15.139:0/400721178 >> 172.21.15.139:0/400721178 conn(0x555a95d70870 msgr2=0x555a95d671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.983+0000 7feb04ff9640 0 client.15963 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.983+0000 7feb267bd740 1 -- 172.21.15.139:0/400721178 shutdown_connections 2024-09-16T01:44:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.985+0000 7feb267bd740 1 -- 172.21.15.139:0/400721178 wait complete. 2024-09-16T01:44:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:26.986+0000 7feb267bd740 20 client.15963 trim_cache size 0 max 16384 2024-09-16T01:44:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.049+0000 7feb267bd740 1 Processor -- start 2024-09-16T01:44:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.049+0000 7feb267bd740 1 -- start start 2024-09-16T01:44:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.050+0000 7feb267bd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d796c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.050+0000 7feb267bd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d74c20 0x555a95d79c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.050+0000 7feb267bd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d7f4c0 0x555a95d930d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.050+0000 7feb267bd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1acdf0 con 0x555a95d74300 2024-09-16T01:44:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.050+0000 7feb267bd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1ae9f0 con 0x555a95d74c20 2024-09-16T01:44:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.050+0000 7feb267bd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1ad4c0 con 0x555a95d7f4c0 2024-09-16T01:44:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.052+0000 7feb216e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d796c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.052+0000 7feb216e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d796c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49982/0 (socket says 172.21.15.139:49982) 2024-09-16T01:44:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.052+0000 7feb216e5640 1 -- 172.21.15.139:0/27091282 learned_addr learned my addr 172.21.15.139:0/27091282 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.056+0000 7feb20ee4640 1 --2- 172.21.15.139:0/27091282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d74c20 0x555a95d79c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.056+0000 7feb21ee6640 1 --2- 172.21.15.139:0/27091282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d7f4c0 0x555a95d930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.058+0000 7feb216e5640 1 -- 172.21.15.139:0/27091282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d7f4c0 msgr2=0x555a95d930d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.058+0000 7feb216e5640 1 --2- 172.21.15.139:0/27091282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d7f4c0 0x555a95d930d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.058+0000 7feb216e5640 1 -- 172.21.15.139:0/27091282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d74c20 msgr2=0x555a95d79c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.058+0000 7feb216e5640 1 --2- 172.21.15.139:0/27091282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d74c20 0x555a95d79c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.058+0000 7feb216e5640 1 -- 172.21.15.139:0/27091282 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a95d74130 con 0x555a95d74300 2024-09-16T01:44:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.061+0000 7feb216e5640 1 --2- 172.21.15.139:0/27091282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d796c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7feb10005c80 tx=0x7feb10022700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb67a176b3aa9a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.063+0000 7feb06ffd640 1 -- 172.21.15.139:0/27091282 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10041030 con 0x555a95d74300 2024-09-16T01:44:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.064+0000 7feb06ffd640 1 -- 172.21.15.139:0/27091282 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb10004020 con 0x555a95d74300 2024-09-16T01:44:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.064+0000 7feb06ffd640 1 -- 172.21.15.139:0/27091282 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10028040 con 0x555a95d74300 2024-09-16T01:44:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.066+0000 7feb267bd740 1 -- 172.21.15.139:0/27091282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 msgr2=0x555a95d796c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.067+0000 7feb267bd740 1 --2- 172.21.15.139:0/27091282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d796c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7feb10005c80 tx=0x7feb10022700 comp rx=0 tx=0).stop 2024-09-16T01:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.067+0000 7feb267bd740 1 -- 172.21.15.139:0/27091282 shutdown_connections 2024-09-16T01:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.067+0000 7feb267bd740 1 --2- 172.21.15.139:0/27091282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d7f4c0 0x555a95d930d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.067+0000 7feb267bd740 1 --2- 172.21.15.139:0/27091282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d74c20 0x555a95d79c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.067+0000 7feb267bd740 1 --2- 172.21.15.139:0/27091282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d74300 0x555a95d796c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.067+0000 7feb267bd740 1 -- 172.21.15.139:0/27091282 >> 172.21.15.139:0/27091282 conn(0x555a95d70870 msgr2=0x555a95d77740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.072+0000 7feb267bd740 1 -- 172.21.15.139:0/27091282 shutdown_connections 2024-09-16T01:44:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.072+0000 7feb267bd740 1 -- 172.21.15.139:0/27091282 wait complete. 2024-09-16T01:44:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.073+0000 7feb267bd740 1 -- start start 2024-09-16T01:44:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.073+0000 7feb267bd740 1 Processor -- start 2024-09-16T01:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.074+0000 7feb267bd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d74300 0x555a95d90740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.074+0000 7feb267bd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d74c20 0x555a95d90c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.074+0000 7feb267bd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d7f4c0 0x555a95df01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.075+0000 7feb267bd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1ae260 con 0x555a95d7f4c0 2024-09-16T01:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.075+0000 7feb267bd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1adb90 con 0x555a95d74c20 2024-09-16T01:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.075+0000 7feb267bd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1aae80 con 0x555a95d74300 2024-09-16T01:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.075+0000 7feb267bd740 10 client.0 _ll_register_callbacks cb 0x7feb10009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:44:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.076+0000 7feb21ee6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d7f4c0 0x555a95df01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.076+0000 7feb21ee6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d7f4c0 0x555a95df01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49992/0 (socket says 172.21.15.139:49992) 2024-09-16T01:44:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.076+0000 7feb21ee6640 1 -- 172.21.15.139:0/233391067 learned_addr learned my addr 172.21.15.139:0/233391067 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.077+0000 7feb20ee4640 1 --2- 172.21.15.139:0/233391067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d74c20 0x555a95d90c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.077+0000 7feb216e5640 1 --2- 172.21.15.139:0/233391067 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d74300 0x555a95d90740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.077+0000 7feb21ee6640 1 -- 172.21.15.139:0/233391067 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d74300 msgr2=0x555a95d90740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.077+0000 7feb21ee6640 1 --2- 172.21.15.139:0/233391067 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d74300 0x555a95d90740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.077+0000 7feb21ee6640 1 -- 172.21.15.139:0/233391067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d74c20 msgr2=0x555a95d90c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.077+0000 7feb21ee6640 1 --2- 172.21.15.139:0/233391067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d74c20 0x555a95d90c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.077+0000 7feb21ee6640 1 -- 172.21.15.139:0/233391067 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a95d82200 con 0x555a95d7f4c0 2024-09-16T01:44:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.080+0000 7feb21ee6640 1 --2- 172.21.15.139:0/233391067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d7f4c0 0x555a95df01c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7feb1402ec80 tx=0x7feb14009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825fb358287230b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.081+0000 7feb04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:44:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.081+0000 7feb04ff9640 1 -- 172.21.15.139:0/233391067 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14050070 con 0x555a95d7f4c0 2024-09-16T01:44:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.081+0000 7feb04ff9640 1 -- 172.21.15.139:0/233391067 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7feb140101a0 con 0x555a95d7f4c0 2024-09-16T01:44:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.081+0000 7feb267bd740 1 -- 172.21.15.139:0/233391067 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb10004020 con 0x555a95d7f4c0 2024-09-16T01:44:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.088+0000 7feb04ff9640 1 -- 172.21.15.139:0/233391067 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb1405b050 con 0x555a95d7f4c0 2024-09-16T01:44:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.091+0000 7feb267bd740 10 client.15969 fetch_fsmap learned FSMap version 32 2024-09-16T01:44:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.091+0000 7feb267bd740 1 -- 172.21.15.139:0/233391067 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555a95df1890 con 0x555a95d7f4c0 2024-09-16T01:44:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.093+0000 7feb04ff9640 1 -- 172.21.15.139:0/233391067 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb10004020 con 0x555a95d7f4c0 2024-09-16T01:44:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.093+0000 7feb04ff9640 1 -- 172.21.15.139:0/233391067 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7feb1403d030 con 0x555a95d7f4c0 2024-09-16T01:44:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.094+0000 7feb267bd740 10 client.15969 fetch_fsmap finished waiting for FSMap version 32 2024-09-16T01:44:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.094+0000 7feb267bd740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:44:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.094+0000 7feb267bd740 20 client.15969 populate_metadata read hostname 'smithi139' 2024-09-16T01:44:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.094+0000 7feb267bd740 1 --2- 172.21.15.139:0/233391067 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555a95df2050 0x555a95df4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.094+0000 7feb267bd740 4 client.15969 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:44:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.094+0000 7feb267bd740 1 -- 172.21.15.139:0/233391067 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a95cd7d70 con 0x555a95df2050 2024-09-16T01:44:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.095+0000 7feb216e5640 1 --2- 172.21.15.139:0/233391067 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555a95df2050 0x555a95df4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.108+0000 7feb04ff9640 10 client.15969 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:27.108+0000 7feb216e5640 1 --2- 172.21.15.139:0/233391067 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555a95df2050 0x555a95df4470 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=402c5fb5c1ced363 server_cookie=e70c0977f256e3d2 in_seq=0 out_seq=0 2024-09-16T01:44:27.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:26 smithi139 ceph-mon[31175]: pgmap v124: 129 pgs: 129 active+clean; 889 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.0 MiB/s wr, 593 op/s 2024-09-16T01:44:27.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:26 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:44:27.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:26 smithi176 ceph-mon[35324]: pgmap v124: 129 pgs: 129 active+clean; 889 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.0 MiB/s wr, 593 op/s 2024-09-16T01:44:27.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:26 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:44:27.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:26 smithi164 ceph-mon[35121]: pgmap v124: 129 pgs: 129 active+clean; 889 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.0 MiB/s wr, 593 op/s 2024-09-16T01:44:27.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:26 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:44:29.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:28 smithi176 ceph-mon[35324]: pgmap v125: 129 pgs: 129 active+clean; 946 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 12 MiB/s wr, 547 op/s 2024-09-16T01:44:29.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:28 smithi164 ceph-mon[35121]: pgmap v125: 129 pgs: 129 active+clean; 946 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 12 MiB/s wr, 547 op/s 2024-09-16T01:44:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:28 smithi139 ceph-mon[31175]: pgmap v125: 129 pgs: 129 active+clean; 946 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 12 MiB/s wr, 547 op/s 2024-09-16T01:44:31.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:30 smithi164 ceph-mon[35121]: pgmap v126: 129 pgs: 129 active+clean; 959 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 13 MiB/s wr, 554 op/s 2024-09-16T01:44:31.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:30 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:44:31.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:30 smithi139 ceph-mon[31175]: pgmap v126: 129 pgs: 129 active+clean; 959 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 13 MiB/s wr, 554 op/s 2024-09-16T01:44:31.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:30 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:44:31.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:30 smithi176 ceph-mon[35324]: pgmap v126: 129 pgs: 129 active+clean; 959 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 13 MiB/s wr, 554 op/s 2024-09-16T01:44:31.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:30 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:44:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:44:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:44:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:44:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:44:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.525+0000 7feb04ff9640 1 -- 172.21.15.139:0/233391067 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a95cd7d70 con 0x555a95df2050 2024-09-16T01:44:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.525+0000 7feb04ff9640 10 client.15969 handle_command_reply: tid=0 2024-09-16T01:44:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.526+0000 7feb267bd740 1 client.15969 shutdown 2024-09-16T01:44:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.528+0000 7feb267bd740 1 -- 172.21.15.139:0/233391067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d7f4c0 msgr2=0x555a95df01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.528+0000 7feb267bd740 1 --2- 172.21.15.139:0/233391067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d7f4c0 0x555a95df01c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7feb1402ec80 tx=0x7feb14009260 comp rx=0 tx=0).stop 2024-09-16T01:44:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.528+0000 7feb267bd740 1 -- 172.21.15.139:0/233391067 shutdown_connections 2024-09-16T01:44:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.528+0000 7feb267bd740 1 --2- 172.21.15.139:0/233391067 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555a95df2050 0x555a95df4470 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.528+0000 7feb267bd740 1 --2- 172.21.15.139:0/233391067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a95d7f4c0 0x555a95df01c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.528+0000 7feb267bd740 1 --2- 172.21.15.139:0/233391067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555a95d74c20 0x555a95d90c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.528+0000 7feb267bd740 1 --2- 172.21.15.139:0/233391067 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a95d74300 0x555a95d90740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.528+0000 7feb267bd740 1 -- 172.21.15.139:0/233391067 >> 172.21.15.139:0/233391067 conn(0x555a95d70870 msgr2=0x555a95d669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.528+0000 7feb04ff9640 0 client.15969 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.528+0000 7feb267bd740 1 -- 172.21.15.139:0/233391067 shutdown_connections 2024-09-16T01:44:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.528+0000 7feb267bd740 1 -- 172.21.15.139:0/233391067 wait complete. 2024-09-16T01:44:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.529+0000 7feb267bd740 20 client.15969 trim_cache size 0 max 16384 2024-09-16T01:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.566+0000 7feb23949640 1 -- 172.21.15.139:0/2281018180 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feb080a3690 msgr2=0x7feb080a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.566+0000 7feb23949640 1 --2- 172.21.15.139:0/2281018180 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feb080a3690 0x7feb080a5b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feb14006010 tx=0x7feb1400d040 comp rx=0 tx=0).stop 2024-09-16T01:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.566+0000 7feb23949640 1 -- 172.21.15.139:0/2281018180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb1c101370 msgr2=0x7feb1c196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.566+0000 7feb23949640 1 --2- 172.21.15.139:0/2281018180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb1c101370 0x7feb1c196680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feb1800c890 tx=0x7feb1800cd50 comp rx=0 tx=0).stop 2024-09-16T01:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.567+0000 7feb23949640 1 -- 172.21.15.139:0/2281018180 shutdown_connections 2024-09-16T01:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.567+0000 7feb23949640 1 --2- 172.21.15.139:0/2281018180 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feb080a3690 0x7feb080a5b50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.567+0000 7feb23949640 1 --2- 172.21.15.139:0/2281018180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb1c1069e0 0x7feb1c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.567+0000 7feb23949640 1 --2- 172.21.15.139:0/2281018180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb1c101370 0x7feb1c196680 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.568+0000 7feb23949640 1 --2- 172.21.15.139:0/2281018180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1c1009d0 0x7feb1c196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.568+0000 7feb23949640 1 -- 172.21.15.139:0/2281018180 >> 172.21.15.139:0/2281018180 conn(0x7feb1c0fc6b0 msgr2=0x7feb1c0fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.569+0000 7feb23949640 1 -- 172.21.15.139:0/2281018180 shutdown_connections 2024-09-16T01:44:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:31.576+0000 7feb23949640 1 -- 172.21.15.139:0/2281018180 wait complete. 2024-09-16T01:44:32.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:31 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:44:32.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:31 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:44:32.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:31 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:44:33.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:32 smithi164 ceph-mon[35121]: pgmap v127: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-09-16T01:44:33.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:32 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:44:33.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:32 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:44:33.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:32 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:44:33.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:32 smithi139 ceph-mon[31175]: pgmap v127: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-09-16T01:44:33.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:32 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:44:33.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:32 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:44:33.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:32 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:44:33.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:32 smithi176 ceph-mon[35324]: pgmap v127: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-09-16T01:44:33.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:32 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:44:33.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:32 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:44:33.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:32 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:44:33.776 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.8 2024-09-16T01:44:33.777 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:44:35.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:34 smithi164 ceph-mon[35121]: pgmap v128: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-09-16T01:44:35.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:34 smithi139 ceph-mon[31175]: pgmap v128: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-09-16T01:44:35.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:34 smithi176 ceph-mon[35324]: pgmap v128: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-09-16T01:44:35.450 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:44:35.967 INFO:tasks.workunit.client.0.smithi139.stdout: 341 1644491 1296 1164833 1596 280219 935 2024-09-16T01:44:36.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.646+0000 7ff814853640 1 -- 172.21.15.139:0/1092566991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff810101e10 msgr2=0x7ff8101021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:36.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.646+0000 7ff814853640 1 --2- 172.21.15.139:0/1092566991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff810101e10 0x7ff8101021f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff800009860 tx=0x7ff80002a340 comp rx=0 tx=0).stop 2024-09-16T01:44:36.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.646+0000 7ff814853640 1 -- 172.21.15.139:0/1092566991 shutdown_connections 2024-09-16T01:44:36.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.646+0000 7ff814853640 1 --2- 172.21.15.139:0/1092566991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff810106f40 0x7ff81010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:36.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.646+0000 7ff814853640 1 --2- 172.21.15.139:0/1092566991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8101027c0 0x7ff810106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:36.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.646+0000 7ff814853640 1 --2- 172.21.15.139:0/1092566991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff810101e10 0x7ff8101021f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:36.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.646+0000 7ff814853640 1 -- 172.21.15.139:0/1092566991 >> 172.21.15.139:0/1092566991 conn(0x7ff8100fd8d0 msgr2=0x7ff8100ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:36.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.646+0000 7ff814853640 1 -- 172.21.15.139:0/1092566991 shutdown_connections 2024-09-16T01:44:36.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.646+0000 7ff814853640 1 -- 172.21.15.139:0/1092566991 wait complete. 2024-09-16T01:44:36.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.647+0000 7ff814853640 1 Processor -- start 2024-09-16T01:44:36.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.647+0000 7ff814853640 1 -- start start 2024-09-16T01:44:36.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.647+0000 7ff814853640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff810101e10 0x7ff81019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:36.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.647+0000 7ff814853640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8101027c0 0x7ff81019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:36.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.647+0000 7ff814853640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff810106f40 0x7ff81019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:36.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.647+0000 7ff814853640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff810111fa0 con 0x7ff810106f40 2024-09-16T01:44:36.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff814853640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff810111e20 con 0x7ff810101e10 2024-09-16T01:44:36.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff814853640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff810112120 con 0x7ff8101027c0 2024-09-16T01:44:36.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff80ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff810106f40 0x7ff81019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:36.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff80ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff810106f40 0x7ff81019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48734/0 (socket says 172.21.15.139:48734) 2024-09-16T01:44:36.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff80ed9d640 1 -- 172.21.15.139:0/1975970864 learned_addr learned my addr 172.21.15.139:0/1975970864 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:36.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff80dd9b640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8101027c0 0x7ff81019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:36.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff80e59c640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff810101e10 0x7ff81019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:36.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff80ed9d640 1 -- 172.21.15.139:0/1975970864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8101027c0 msgr2=0x7ff81019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:36.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff80ed9d640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8101027c0 0x7ff81019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:36.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff80ed9d640 1 -- 172.21.15.139:0/1975970864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff810101e10 msgr2=0x7ff81019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:36.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff80ed9d640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff810101e10 0x7ff81019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:36.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff80ed9d640 1 -- 172.21.15.139:0/1975970864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81019f620 con 0x7ff810106f40 2024-09-16T01:44:36.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff80e59c640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff810101e10 0x7ff81019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:44:36.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff80ed9d640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff810106f40 0x7ff81019ef40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff80800eb10 tx=0x7ff80800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:36.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff7ff7fe640 1 -- 172.21.15.139:0/1975970864 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80800ce20 con 0x7ff810106f40 2024-09-16T01:44:36.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff7ff7fe640 1 -- 172.21.15.139:0/1975970864 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff808005020 con 0x7ff810106f40 2024-09-16T01:44:36.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff814853640 1 -- 172.21.15.139:0/1975970864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81019f910 con 0x7ff810106f40 2024-09-16T01:44:36.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.648+0000 7ff7ff7fe640 1 -- 172.21.15.139:0/1975970864 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808010690 con 0x7ff810106f40 2024-09-16T01:44:36.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.649+0000 7ff814853640 1 -- 172.21.15.139:0/1975970864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8101a71f0 con 0x7ff810106f40 2024-09-16T01:44:36.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.650+0000 7ff814853640 1 -- 172.21.15.139:0/1975970864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8101021f0 con 0x7ff810106f40 2024-09-16T01:44:36.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.651+0000 7ff7ff7fe640 1 -- 172.21.15.139:0/1975970864 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff808005540 con 0x7ff810106f40 2024-09-16T01:44:36.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.651+0000 7ff7ff7fe640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff7f80a3430 0x7ff7f80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:36.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.652+0000 7ff80e59c640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff7f80a3430 0x7ff7f80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:36.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.652+0000 7ff7ff7fe640 1 -- 172.21.15.139:0/1975970864 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7ff8080d0830 con 0x7ff810106f40 2024-09-16T01:44:36.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.652+0000 7ff80e59c640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff7f80a3430 0x7ff7f80a58f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff800000c00 tx=0x7ff8000058b0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:36.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:36.659+0000 7ff7ff7fe640 1 -- 172.21.15.139:0/1975970864 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8080541b0 con 0x7ff810106f40 2024-09-16T01:44:37.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:37.026+0000 7ff814853640 1 -- 172.21.15.139:0/1975970864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8101a74d0 con 0x7ff810106f40 2024-09-16T01:44:37.035 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:44:37.035 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":33,"btime":"2024-09-16T01:44:30:937847+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:44:30.781836+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:44:37.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:37.033+0000 7ff7ff7fe640 1 -- 172.21.15.139:0/1975970864 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+12009 (secure 0 0 0) 0x7ff80809ce60 con 0x7ff810106f40 2024-09-16T01:44:37.036 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 33 2024-09-16T01:44:37.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:37.039+0000 7ff814853640 1 -- 172.21.15.139:0/1975970864 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff7f80a3430 msgr2=0x7ff7f80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:37.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:37.040+0000 7ff814853640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff7f80a3430 0x7ff7f80a58f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff800000c00 tx=0x7ff8000058b0 comp rx=0 tx=0).stop 2024-09-16T01:44:37.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:37.040+0000 7ff814853640 1 -- 172.21.15.139:0/1975970864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff810106f40 msgr2=0x7ff81019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:37.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:37.040+0000 7ff814853640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff810106f40 0x7ff81019ef40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff80800eb10 tx=0x7ff80800ef10 comp rx=0 tx=0).stop 2024-09-16T01:44:37.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:37.044+0000 7ff814853640 1 -- 172.21.15.139:0/1975970864 shutdown_connections 2024-09-16T01:44:37.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:37.044+0000 7ff814853640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff7f80a3430 0x7ff7f80a58f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:37.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:37.044+0000 7ff814853640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff810106f40 0x7ff81019ef40 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:37.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:37.044+0000 7ff814853640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8101027c0 0x7ff81019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:37.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:37.044+0000 7ff814853640 1 --2- 172.21.15.139:0/1975970864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff810101e10 0x7ff81019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:37.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:37.044+0000 7ff814853640 1 -- 172.21.15.139:0/1975970864 >> 172.21.15.139:0/1975970864 conn(0x7ff8100fd8d0 msgr2=0x7ff8100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:37.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:37.050+0000 7ff814853640 1 -- 172.21.15.139:0/1975970864 shutdown_connections 2024-09-16T01:44:37.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:37.054+0000 7ff814853640 1 -- 172.21.15.139:0/1975970864 wait complete. 2024-09-16T01:44:37.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:36 smithi164 ceph-mon[35121]: pgmap v129: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 18 MiB/s wr, 1.10k op/s 2024-09-16T01:44:37.398 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:36 smithi139 ceph-mon[31175]: pgmap v129: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 18 MiB/s wr, 1.10k op/s 2024-09-16T01:44:37.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:36 smithi176 ceph-mon[35324]: pgmap v129: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 18 MiB/s wr, 1.10k op/s 2024-09-16T01:44:38.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:37 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1975970864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:44:38.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:37 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1975970864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:44:38.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:37 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1975970864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:44:38.759 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 133.8 --await-for 45.0' 2024-09-16T01:44:38.760 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 133.8 --await-for 45.0 2024-09-16T01:44:39.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:38 smithi164 ceph-mon[35121]: pgmap v130: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 15 MiB/s wr, 975 op/s 2024-09-16T01:44:39.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:38 smithi139 ceph-mon[31175]: pgmap v130: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 15 MiB/s wr, 975 op/s 2024-09-16T01:44:39.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:38 smithi176 ceph-mon[35324]: pgmap v130: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 15 MiB/s wr, 975 op/s 2024-09-16T01:44:39.561 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:44:40.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.312+0000 7fb042b5d640 1 -- 172.21.15.139:0/3550685394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb03c105f50 msgr2=0x7fb03c1063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:40.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.312+0000 7fb042b5d640 1 --2- 172.21.15.139:0/3550685394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb03c105f50 0x7fb03c1063e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb034009860 tx=0x7fb03402a3d0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.313+0000 7fb042b5d640 1 -- 172.21.15.139:0/3550685394 shutdown_connections 2024-09-16T01:44:40.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.313+0000 7fb042b5d640 1 --2- 172.21.15.139:0/3550685394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb03c105f50 0x7fb03c1063e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.313+0000 7fb042b5d640 1 --2- 172.21.15.139:0/3550685394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb03c101460 0x7fb03c105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.313+0000 7fb042b5d640 1 --2- 172.21.15.139:0/3550685394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb03c100b40 0x7fb03c100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.313+0000 7fb042b5d640 1 -- 172.21.15.139:0/3550685394 >> 172.21.15.139:0/3550685394 conn(0x7fb03c0fc7d0 msgr2=0x7fb03c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:40.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.314+0000 7fb042b5d640 1 -- 172.21.15.139:0/3550685394 shutdown_connections 2024-09-16T01:44:40.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.315+0000 7fb042b5d640 1 -- 172.21.15.139:0/3550685394 wait complete. 2024-09-16T01:44:40.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.316+0000 7fb042b5d640 1 Processor -- start 2024-09-16T01:44:40.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.317+0000 7fb042b5d640 1 -- start start 2024-09-16T01:44:40.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.317+0000 7fb042b5d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb03c100b40 0x7fb03c114cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.317+0000 7fb042b5d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb03c101460 0x7fb03c115200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.317+0000 7fb042b5d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb03c105f50 0x7fb03c10df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.317+0000 7fb042b5d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c118510 con 0x7fb03c105f50 2024-09-16T01:44:40.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.317+0000 7fb042b5d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c118390 con 0x7fb03c100b40 2024-09-16T01:44:40.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.317+0000 7fb042b5d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c118690 con 0x7fb03c101460 2024-09-16T01:44:40.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.318+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb03c101460 0x7fb03c115200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:40.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.318+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb03c101460 0x7fb03c115200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:33238/0 (socket says 172.21.15.139:33238) 2024-09-16T01:44:40.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.318+0000 7fb03bfff640 1 -- 172.21.15.139:0/1889254180 learned_addr learned my addr 172.21.15.139:0/1889254180 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:40.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.318+0000 7fb0408f9640 1 --2- 172.21.15.139:0/1889254180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb03c100b40 0x7fb03c114cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:40.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.319+0000 7fb03bfff640 1 -- 172.21.15.139:0/1889254180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb03c100b40 msgr2=0x7fb03c114cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:40.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.319+0000 7fb03bfff640 1 --2- 172.21.15.139:0/1889254180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb03c100b40 0x7fb03c114cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.319+0000 7fb03bfff640 1 -- 172.21.15.139:0/1889254180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb03c105f50 msgr2=0x7fb03c10df00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:44:40.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.319+0000 7fb03bfff640 1 --2- 172.21.15.139:0/1889254180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb03c105f50 0x7fb03c10df00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.319+0000 7fb03bfff640 1 -- 172.21.15.139:0/1889254180 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03c10e7c0 con 0x7fb03c101460 2024-09-16T01:44:40.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.322+0000 7fb03bfff640 1 --2- 172.21.15.139:0/1889254180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb03c101460 0x7fb03c115200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00eea0 tx=0x7fb02c00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:40.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.323+0000 7fb039ffb640 1 -- 172.21.15.139:0/1889254180 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c01b070 con 0x7fb03c101460 2024-09-16T01:44:40.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.323+0000 7fb039ffb640 1 -- 172.21.15.139:0/1889254180 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb02c00ba50 con 0x7fb03c101460 2024-09-16T01:44:40.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.323+0000 7fb039ffb640 1 -- 172.21.15.139:0/1889254180 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c016a80 con 0x7fb03c101460 2024-09-16T01:44:40.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.323+0000 7fb042b5d640 1 -- 172.21.15.139:0/1889254180 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03c10eab0 con 0x7fb03c101460 2024-09-16T01:44:40.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.323+0000 7fb042b5d640 1 -- 172.21.15.139:0/1889254180 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03c112a80 con 0x7fb03c101460 2024-09-16T01:44:40.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.329+0000 7fb044f36740 1 Processor -- start 2024-09-16T01:44:40.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.330+0000 7fb044f36740 1 -- start start 2024-09-16T01:44:40.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.330+0000 7fb044f36740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 0x55572a04bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.330+0000 7fb044f36740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04c080 0x55572a056400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.330+0000 7fb044f36740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 0x55572a058d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.330+0000 7fb044f36740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555729fb12a0 con 0x55572a04c080 2024-09-16T01:44:40.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.330+0000 7fb044f36740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555729f58b10 con 0x55572a04b760 2024-09-16T01:44:40.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.330+0000 7fb044f36740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555729fad260 con 0x55572a056940 2024-09-16T01:44:40.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.331+0000 7fb0408f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04c080 0x55572a056400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:40.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.331+0000 7fb0408f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04c080 0x55572a056400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48742/0 (socket says 172.21.15.139:48742) 2024-09-16T01:44:40.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.331+0000 7fb0408f9640 1 -- 172.21.15.139:0/3688376680 learned_addr learned my addr 172.21.15.139:0/3688376680 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:40.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.331+0000 7fb0410fa640 1 --2- 172.21.15.139:0/3688376680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 0x55572a04bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:40.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.332+0000 7fb0408f9640 1 -- 172.21.15.139:0/3688376680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 msgr2=0x55572a058d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:44:40.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.332+0000 7fb0408f9640 1 --2- 172.21.15.139:0/3688376680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 0x55572a058d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.332+0000 7fb0408f9640 1 -- 172.21.15.139:0/3688376680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 msgr2=0x55572a04bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:40.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.334+0000 7fb039ffb640 1 -- 172.21.15.139:0/1889254180 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb02c01e070 con 0x7fb03c101460 2024-09-16T01:44:40.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.335+0000 7fb039ffb640 1 --2- 172.21.15.139:0/1889254180 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0280a35c0 0x7fb0280a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.335+0000 7fb0408f9640 1 --2- 172.21.15.139:0/3688376680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 0x55572a04bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.335+0000 7fb0408f9640 1 -- 172.21.15.139:0/3688376680 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555729d22b90 con 0x55572a04c080 2024-09-16T01:44:40.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.335+0000 7fb0410fa640 1 --2- 172.21.15.139:0/3688376680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 0x55572a04bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:44:40.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.335+0000 7fb03bfff640 1 --2- 172.21.15.139:0/1889254180 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0280a35c0 0x7fb0280a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:40.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.336+0000 7fb0408f9640 1 --2- 172.21.15.139:0/3688376680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04c080 0x55572a056400 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb02400cbf0 tx=0x7fb02400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e950e785db32dd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.336+0000 7fb03bfff640 1 --2- 172.21.15.139:0/1889254180 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0280a35c0 0x7fb0280a5a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55572a049070 tx=0x7fb02c0d61a0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.337+0000 7fb039ffb640 1 -- 172.21.15.139:0/1889254180 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fb02c0d07c0 con 0x7fb03c101460 2024-09-16T01:44:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.337+0000 7fb032ffd640 1 -- 172.21.15.139:0/3688376680 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024010630 con 0x55572a04c080 2024-09-16T01:44:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.337+0000 7fb032ffd640 1 -- 172.21.15.139:0/3688376680 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb024010cb0 con 0x55572a04c080 2024-09-16T01:44:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.338+0000 7fb032ffd640 1 -- 172.21.15.139:0/3688376680 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024019ca0 con 0x55572a04c080 2024-09-16T01:44:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.338+0000 7fb044f36740 1 -- 172.21.15.139:0/3688376680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04c080 msgr2=0x55572a056400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.338+0000 7fb044f36740 1 --2- 172.21.15.139:0/3688376680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04c080 0x55572a056400 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb02400cbf0 tx=0x7fb02400e720 comp rx=0 tx=0).stop 2024-09-16T01:44:40.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.338+0000 7fb044f36740 1 -- 172.21.15.139:0/3688376680 shutdown_connections 2024-09-16T01:44:40.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.338+0000 7fb044f36740 1 --2- 172.21.15.139:0/3688376680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 0x55572a058d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.338+0000 7fb044f36740 1 --2- 172.21.15.139:0/3688376680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04c080 0x55572a056400 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.338+0000 7fb044f36740 1 --2- 172.21.15.139:0/3688376680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 0x55572a04bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.338+0000 7fb044f36740 1 -- 172.21.15.139:0/3688376680 >> 172.21.15.139:0/3688376680 conn(0x55572a047cf0 msgr2=0x55572a0480d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.338+0000 7fb044f36740 1 -- 172.21.15.139:0/3688376680 shutdown_connections 2024-09-16T01:44:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.339+0000 7fb044f36740 1 -- 172.21.15.139:0/3688376680 wait complete. 2024-09-16T01:44:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.339+0000 7fb044f36740 1 -- start start 2024-09-16T01:44:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.340+0000 7fb044f36740 1 Processor -- start 2024-09-16T01:44:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.341+0000 7fb044f36740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04b760 0x55572a0537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.341+0000 7fb044f36740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04c080 0x55572a053d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.341+0000 7fb044f36740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 0x55572a054270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.341+0000 7fb044f36740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55572a059560 con 0x55572a04b760 2024-09-16T01:44:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.341+0000 7fb044f36740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555729f396e0 con 0x55572a04c080 2024-09-16T01:44:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.341+0000 7fb044f36740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55572a0596e0 con 0x55572a056940 2024-09-16T01:44:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.341+0000 7fb044f36740 10 client.0 _ll_register_callbacks cb 0x555729f5da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:44:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.341+0000 7fb0408f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04c080 0x55572a053d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.341+0000 7fb0408f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04c080 0x55572a053d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37430/0 (socket says 172.21.15.139:37430) 2024-09-16T01:44:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.341+0000 7fb0408f9640 1 -- 172.21.15.139:0/765338320 learned_addr learned my addr 172.21.15.139:0/765338320 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.341+0000 7fb0410fa640 1 --2- 172.21.15.139:0/765338320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04b760 0x55572a0537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.342+0000 7fb0410fa640 1 -- 172.21.15.139:0/765338320 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 msgr2=0x55572a054270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:44:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.342+0000 7fb0410fa640 1 --2- 172.21.15.139:0/765338320 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 0x55572a054270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.342+0000 7fb0410fa640 1 -- 172.21.15.139:0/765338320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04c080 msgr2=0x55572a053d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.342+0000 7fb0410fa640 1 --2- 172.21.15.139:0/765338320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04c080 0x55572a053d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.342+0000 7fb0410fa640 1 -- 172.21.15.139:0/765338320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55572a084b10 con 0x55572a04b760 2024-09-16T01:44:40.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.343+0000 7fb0410fa640 1 --2- 172.21.15.139:0/765338320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04b760 0x55572a0537f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb034034000 tx=0x7fb034030990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a76269b1c230cd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:40.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.343+0000 7fb01bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:44:40.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.343+0000 7fb01bfff640 1 -- 172.21.15.139:0/765338320 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034005240 con 0x55572a04b760 2024-09-16T01:44:40.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.343+0000 7fb01bfff640 1 -- 172.21.15.139:0/765338320 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fb0340053e0 con 0x55572a04b760 2024-09-16T01:44:40.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.344+0000 7fb044f36740 1 -- 172.21.15.139:0/765338320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb024010cb0 con 0x55572a04b760 2024-09-16T01:44:40.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.344+0000 7fb01bfff640 1 -- 172.21.15.139:0/765338320 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb0340055e0 con 0x55572a04b760 2024-09-16T01:44:40.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.345+0000 7fb044f36740 10 client.15984 fetch_fsmap learned FSMap version 33 2024-09-16T01:44:40.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.345+0000 7fb044f36740 1 -- 172.21.15.139:0/765338320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55572a0874a0 con 0x55572a04b760 2024-09-16T01:44:40.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.346+0000 7fb01bfff640 1 -- 172.21.15.139:0/765338320 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb024010cb0 con 0x55572a04b760 2024-09-16T01:44:40.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.347+0000 7fb01bfff640 1 -- 172.21.15.139:0/765338320 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7fb0340046c0 con 0x55572a04b760 2024-09-16T01:44:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.348+0000 7fb044f36740 10 client.15984 fetch_fsmap finished waiting for FSMap version 33 2024-09-16T01:44:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.348+0000 7fb044f36740 10 client.15984 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:44:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.348+0000 7fb044f36740 20 client.15984 populate_metadata read hostname 'smithi139' 2024-09-16T01:44:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.348+0000 7fb044f36740 1 --2- 172.21.15.139:0/765338320 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55572a088a00 0x55572a08ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.348+0000 7fb044f36740 4 client.15984 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:44:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.348+0000 7fb044f36740 1 -- 172.21.15.139:0/765338320 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555729fbe9d0 con 0x55572a088a00 2024-09-16T01:44:40.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.348+0000 7fb0408f9640 1 --2- 172.21.15.139:0/765338320 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55572a088a00 0x55572a08ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:40.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.350+0000 7fb0408f9640 1 --2- 172.21.15.139:0/765338320 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55572a088a00 0x55572a08ae30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0e46c37856a712e server_cookie=b6df6fb0b2c0a4f0 in_seq=0 out_seq=0 2024-09-16T01:44:40.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.350+0000 7fb01bfff640 10 client.15984 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:40.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.351+0000 7fb01bfff640 1 -- 172.21.15.139:0/765338320 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555729fbe9d0 con 0x55572a088a00 2024-09-16T01:44:40.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.352+0000 7fb01bfff640 10 client.15984 handle_command_reply: tid=0 2024-09-16T01:44:40.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.352+0000 7fb044f36740 1 client.15984 shutdown 2024-09-16T01:44:40.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.353+0000 7fb044f36740 1 -- 172.21.15.139:0/765338320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04b760 msgr2=0x55572a0537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:40.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.353+0000 7fb044f36740 1 --2- 172.21.15.139:0/765338320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04b760 0x55572a0537f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb034034000 tx=0x7fb034030990 comp rx=0 tx=0).stop 2024-09-16T01:44:40.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.354+0000 7fb044f36740 1 -- 172.21.15.139:0/765338320 shutdown_connections 2024-09-16T01:44:40.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.354+0000 7fb044f36740 1 --2- 172.21.15.139:0/765338320 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55572a088a00 0x55572a08ae30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.354+0000 7fb044f36740 1 --2- 172.21.15.139:0/765338320 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 0x55572a054270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.354+0000 7fb044f36740 1 --2- 172.21.15.139:0/765338320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04c080 0x55572a053d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.354+0000 7fb044f36740 1 --2- 172.21.15.139:0/765338320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04b760 0x55572a0537f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.354+0000 7fb044f36740 1 -- 172.21.15.139:0/765338320 >> 172.21.15.139:0/765338320 conn(0x55572a047cf0 msgr2=0x55572a04da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:40.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.355+0000 7fb01bfff640 0 client.15984 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:40.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.355+0000 7fb044f36740 1 -- 172.21.15.139:0/765338320 shutdown_connections 2024-09-16T01:44:40.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.356+0000 7fb044f36740 1 -- 172.21.15.139:0/765338320 wait complete. 2024-09-16T01:44:40.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.357+0000 7fb044f36740 20 client.15984 trim_cache size 0 max 16384 2024-09-16T01:44:40.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.398+0000 7fb044f36740 1 Processor -- start 2024-09-16T01:44:40.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.402+0000 7fb044f36740 1 -- start start 2024-09-16T01:44:40.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.402+0000 7fb044f36740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 0x55572a04e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.402+0000 7fb044f36740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04c080 0x55572a090740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.402+0000 7fb044f36740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 0x55572a090c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.402+0000 7fb044f36740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c010080 con 0x55572a04c080 2024-09-16T01:44:40.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.402+0000 7fb044f36740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c011c80 con 0x55572a04b760 2024-09-16T01:44:40.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.402+0000 7fb044f36740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c010750 con 0x55572a056940 2024-09-16T01:44:40.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.403+0000 7fb0408f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04c080 0x55572a090740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:40.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.403+0000 7fb0408f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04c080 0x55572a090740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48760/0 (socket says 172.21.15.139:48760) 2024-09-16T01:44:40.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.403+0000 7fb0408f9640 1 -- 172.21.15.139:0/2371677439 learned_addr learned my addr 172.21.15.139:0/2371677439 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:40.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.403+0000 7fb0408f9640 1 -- 172.21.15.139:0/2371677439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 msgr2=0x55572a090c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:44:40.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.403+0000 7fb0408f9640 1 --2- 172.21.15.139:0/2371677439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 0x55572a090c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.403+0000 7fb0408f9640 1 -- 172.21.15.139:0/2371677439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 msgr2=0x55572a04e1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:44:40.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.403+0000 7fb0408f9640 1 --2- 172.21.15.139:0/2371677439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 0x55572a04e1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.403+0000 7fb0408f9640 1 -- 172.21.15.139:0/2371677439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55572a047790 con 0x55572a04c080 2024-09-16T01:44:40.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.404+0000 7fb0408f9640 1 --2- 172.21.15.139:0/2371677439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04c080 0x55572a090740 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb02400f010 tx=0x7fb02400c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bfaa36db81c459 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:40.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.404+0000 7fb0327fc640 1 -- 172.21.15.139:0/2371677439 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024010e60 con 0x55572a04c080 2024-09-16T01:44:40.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.404+0000 7fb0327fc640 1 -- 172.21.15.139:0/2371677439 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb024019e50 con 0x55572a04c080 2024-09-16T01:44:40.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.404+0000 7fb0327fc640 1 -- 172.21.15.139:0/2371677439 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024005580 con 0x55572a04c080 2024-09-16T01:44:40.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.405+0000 7fb044f36740 1 -- 172.21.15.139:0/2371677439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04c080 msgr2=0x55572a090740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:40.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.405+0000 7fb044f36740 1 --2- 172.21.15.139:0/2371677439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04c080 0x55572a090740 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb02400f010 tx=0x7fb02400c4f0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.406+0000 7fb044f36740 1 -- 172.21.15.139:0/2371677439 shutdown_connections 2024-09-16T01:44:40.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.406+0000 7fb044f36740 1 --2- 172.21.15.139:0/2371677439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 0x55572a090c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.406+0000 7fb044f36740 1 --2- 172.21.15.139:0/2371677439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a04c080 0x55572a090740 secure :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb02400f010 tx=0x7fb02400c4f0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.406+0000 7fb044f36740 1 --2- 172.21.15.139:0/2371677439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 0x55572a04e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.406+0000 7fb044f36740 1 -- 172.21.15.139:0/2371677439 >> 172.21.15.139:0/2371677439 conn(0x55572a047cf0 msgr2=0x55572a055870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:40.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.410+0000 7fb044f36740 1 -- 172.21.15.139:0/2371677439 shutdown_connections 2024-09-16T01:44:40.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.410+0000 7fb044f36740 1 -- 172.21.15.139:0/2371677439 wait complete. 2024-09-16T01:44:40.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.411+0000 7fb044f36740 1 -- start start 2024-09-16T01:44:40.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.411+0000 7fb044f36740 1 Processor -- start 2024-09-16T01:44:40.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.412+0000 7fb044f36740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 0x55572a05a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.412+0000 7fb044f36740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 0x55572a05ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.412+0000 7fb044f36740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a05b380 0x55572a0f2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.412+0000 7fb044f36740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c0114f0 con 0x55572a05b380 2024-09-16T01:44:40.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.412+0000 7fb0408f9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 0x55572a05ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:40.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.412+0000 7fb0408f9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 0x55572a05ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:33256/0 (socket says 172.21.15.139:33256) 2024-09-16T01:44:40.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.412+0000 7fb0408f9640 1 -- 172.21.15.139:0/2419414613 learned_addr learned my addr 172.21.15.139:0/2419414613 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:40.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.412+0000 7fb044f36740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c010e20 con 0x55572a04b760 2024-09-16T01:44:40.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.413+0000 7fb044f36740 1 -- 172.21.15.139:0/2419414613 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c00e020 con 0x55572a056940 2024-09-16T01:44:40.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.413+0000 7fb0410fa640 1 --2- 172.21.15.139:0/2419414613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a05b380 0x55572a0f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:40.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.413+0000 7fb0410fa640 1 --2- 172.21.15.139:0/2419414613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 0x55572a05a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:40.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.413+0000 7fb044f36740 10 client.0 _ll_register_callbacks cb 0x7fb0340305d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:44:40.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.414+0000 7fb0410fa640 1 -- 172.21.15.139:0/2419414613 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 msgr2=0x55572a05ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:40.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.414+0000 7fb0410fa640 1 --2- 172.21.15.139:0/2419414613 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 0x55572a05ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.414+0000 7fb0410fa640 1 -- 172.21.15.139:0/2419414613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 msgr2=0x55572a05a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:40.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.414+0000 7fb0410fa640 1 --2- 172.21.15.139:0/2419414613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 0x55572a05a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:40.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.414+0000 7fb0410fa640 1 -- 172.21.15.139:0/2419414613 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55572a0f2d50 con 0x55572a05b380 2024-09-16T01:44:40.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.417+0000 7fb0410fa640 1 --2- 172.21.15.139:0/2419414613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a05b380 0x55572a0f2710 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb0340393b0 tx=0x7fb03403d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf9ace0f9a6e03a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:40.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.419+0000 7fb01bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:44:40.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.419+0000 7fb01bfff640 1 -- 172.21.15.139:0/2419414613 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0340055e0 con 0x55572a05b380 2024-09-16T01:44:40.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.419+0000 7fb044f36740 1 -- 172.21.15.139:0/2419414613 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb024019e50 con 0x55572a05b380 2024-09-16T01:44:40.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.419+0000 7fb01bfff640 1 -- 172.21.15.139:0/2419414613 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fb0340053e0 con 0x55572a05b380 2024-09-16T01:44:40.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.422+0000 7fb01bfff640 1 -- 172.21.15.139:0/2419414613 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb0340057d0 con 0x55572a05b380 2024-09-16T01:44:40.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.422+0000 7fb044f36740 10 client.15990 fetch_fsmap learned FSMap version 33 2024-09-16T01:44:40.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.422+0000 7fb044f36740 1 -- 172.21.15.139:0/2419414613 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55572a0f3fb0 con 0x55572a05b380 2024-09-16T01:44:40.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.423+0000 7fb01bfff640 1 -- 172.21.15.139:0/2419414613 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb024019e50 con 0x55572a05b380 2024-09-16T01:44:40.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.423+0000 7fb01bfff640 1 -- 172.21.15.139:0/2419414613 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7fb0340046c0 con 0x55572a05b380 2024-09-16T01:44:40.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.424+0000 7fb044f36740 10 client.15990 fetch_fsmap finished waiting for FSMap version 33 2024-09-16T01:44:40.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.424+0000 7fb044f36740 10 client.15990 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:44:40.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.424+0000 7fb044f36740 20 client.15990 populate_metadata read hostname 'smithi139' 2024-09-16T01:44:40.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.424+0000 7fb044f36740 1 --2- 172.21.15.139:0/2419414613 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55572a0f4720 0x55572a0f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:40.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.424+0000 7fb044f36740 4 client.15990 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.8, "await_for": 45.0}] 2024-09-16T01:44:40.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.424+0000 7fb044f36740 1 -- 172.21.15.139:0/2419414613 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.8, "await_for": 45.0}) -- 0x555729fb8d20 con 0x55572a0f4720 2024-09-16T01:44:40.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.426+0000 7fb0408f9640 1 --2- 172.21.15.139:0/2419414613 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55572a0f4720 0x55572a0f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:40.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.428+0000 7fb0408f9640 1 --2- 172.21.15.139:0/2419414613 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55572a0f4720 0x55572a0f6b40 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e95a35149fe8ab server_cookie=67044a96796a44eb in_seq=0 out_seq=0 2024-09-16T01:44:40.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:40.428+0000 7fb01bfff640 10 client.15990 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:41.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:40 smithi164 ceph-mon[35121]: pgmap v131: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 788 op/s 2024-09-16T01:44:41.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:40 smithi139 ceph-mon[31175]: pgmap v131: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 788 op/s 2024-09-16T01:44:41.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:40 smithi176 ceph-mon[35324]: pgmap v131: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 788 op/s 2024-09-16T01:44:43.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:42 smithi164 ceph-mon[35121]: pgmap v132: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 831 op/s 2024-09-16T01:44:43.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:42 smithi139 ceph-mon[31175]: pgmap v132: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 831 op/s 2024-09-16T01:44:43.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:42 smithi176 ceph-mon[35324]: pgmap v132: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 831 op/s 2024-09-16T01:44:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.727+0000 7f8bcf59e640 1 -- 172.21.15.139:0/3254824205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc8095a50 msgr2=0x7f8bc8095e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.727+0000 7f8bcf59e640 1 --2- 172.21.15.139:0/3254824205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc8095a50 0x7f8bc8095e90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009890 tx=0x7f8bc402a410 comp rx=0 tx=0).stop 2024-09-16T01:44:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.727+0000 7f8bcf59e640 1 -- 172.21.15.139:0/3254824205 shutdown_connections 2024-09-16T01:44:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.727+0000 7f8bcf59e640 1 --2- 172.21.15.139:0/3254824205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bc80963d0 0x7f8bc80a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.727+0000 7f8bcf59e640 1 --2- 172.21.15.139:0/3254824205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc8095a50 0x7f8bc8095e90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.727+0000 7f8bcf59e640 1 --2- 172.21.15.139:0/3254824205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8bc809b6a0 0x7f8bc809ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.727+0000 7f8bcf59e640 1 -- 172.21.15.139:0/3254824205 >> 172.21.15.139:0/3254824205 conn(0x7f8bc800b350 msgr2=0x7f8bc800b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.727+0000 7f8bcf59e640 1 -- 172.21.15.139:0/3254824205 shutdown_connections 2024-09-16T01:44:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.727+0000 7f8bcf59e640 1 -- 172.21.15.139:0/3254824205 wait complete. 2024-09-16T01:44:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.728+0000 7f8bcf59e640 1 Processor -- start 2024-09-16T01:44:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.728+0000 7f8bcf59e640 1 -- start start 2024-09-16T01:44:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.729+0000 7f8bcf59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc8095a50 0x7f8bc812bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.729+0000 7f8bcf59e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8bc80963d0 0x7f8bc812c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.729+0000 7f8bcf59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bc809b6a0 0x7f8bc81307e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.729+0000 7f8bcf59e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc80a59f0 con 0x7f8bc809b6a0 2024-09-16T01:44:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.729+0000 7f8bcf59e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc80a5870 con 0x7f8bc8095a50 2024-09-16T01:44:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.729+0000 7f8bcf59e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc80a5b70 con 0x7f8bc80963d0 2024-09-16T01:44:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.729+0000 7f8bce59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc8095a50 0x7f8bc812bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.729+0000 7f8bce59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc8095a50 0x7f8bc812bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:58156/0 (socket says 172.21.15.139:58156) 2024-09-16T01:44:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.729+0000 7f8bce59c640 1 -- 172.21.15.139:0/1756600025 learned_addr learned my addr 172.21.15.139:0/1756600025 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.729+0000 7f8bced9d640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bc809b6a0 0x7f8bc81307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.729+0000 7f8bcdd9b640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8bc80963d0 0x7f8bc812c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.730+0000 7f8bce59c640 1 -- 172.21.15.139:0/1756600025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8bc80963d0 msgr2=0x7f8bc812c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.730+0000 7f8bce59c640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8bc80963d0 0x7f8bc812c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.730+0000 7f8bce59c640 1 -- 172.21.15.139:0/1756600025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bc809b6a0 msgr2=0x7f8bc81307e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.730+0000 7f8bce59c640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bc809b6a0 0x7f8bc81307e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.730+0000 7f8bce59c640 1 -- 172.21.15.139:0/1756600025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc8130f10 con 0x7f8bc8095a50 2024-09-16T01:44:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.730+0000 7f8bced9d640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bc809b6a0 0x7f8bc81307e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:44:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.730+0000 7f8bce59c640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc8095a50 0x7f8bc812bf20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000e9f0 tx=0x7f8bc000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.730+0000 7f8bbf7fe640 1 -- 172.21.15.139:0/1756600025 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000cd80 con 0x7f8bc8095a50 2024-09-16T01:44:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.730+0000 7f8bcf59e640 1 -- 172.21.15.139:0/1756600025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc8131200 con 0x7f8bc8095a50 2024-09-16T01:44:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.730+0000 7f8bbf7fe640 1 -- 172.21.15.139:0/1756600025 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc0005500 con 0x7f8bc8095a50 2024-09-16T01:44:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.730+0000 7f8bbf7fe640 1 -- 172.21.15.139:0/1756600025 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0010690 con 0x7f8bc8095a50 2024-09-16T01:44:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.730+0000 7f8bcf59e640 1 -- 172.21.15.139:0/1756600025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc8007220 con 0x7f8bc8095a50 2024-09-16T01:44:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.732+0000 7f8bd7c1d740 1 Processor -- start 2024-09-16T01:44:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.732+0000 7f8bbf7fe640 1 -- 172.21.15.139:0/1756600025 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8bc0010830 con 0x7f8bc8095a50 2024-09-16T01:44:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.732+0000 7f8bd7c1d740 1 -- start start 2024-09-16T01:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.732+0000 7f8bd7c1d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046b3cc0 0x558a046b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.732+0000 7f8bd7c1d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b45e0 0x558a046be940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.732+0000 7f8bd7c1d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046bee80 0x558a046c1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.732+0000 7f8bd7c1d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558a046182f0 con 0x558a046b45e0 2024-09-16T01:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.732+0000 7f8bd7c1d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558a045c0270 con 0x558a046b3cc0 2024-09-16T01:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.732+0000 7f8bd7c1d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558a046142b0 con 0x558a046bee80 2024-09-16T01:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.733+0000 7f8bced9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046b3cc0 0x558a046b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.733+0000 7f8bced9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046b3cc0 0x558a046b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:58168/0 (socket says 172.21.15.139:58168) 2024-09-16T01:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.733+0000 7f8bced9d640 1 -- 172.21.15.139:0/4035947106 learned_addr learned my addr 172.21.15.139:0/4035947106 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.733+0000 7f8bcdd9b640 1 --2- 172.21.15.139:0/4035947106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046bee80 0x558a046c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.733+0000 7f8bce59c640 1 --2- 172.21.15.139:0/4035947106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b45e0 0x558a046be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.733+0000 7f8bced9d640 1 -- 172.21.15.139:0/4035947106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046bee80 msgr2=0x558a046c1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.733+0000 7f8bced9d640 1 --2- 172.21.15.139:0/4035947106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046bee80 0x558a046c1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.733+0000 7f8bced9d640 1 -- 172.21.15.139:0/4035947106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b45e0 msgr2=0x558a046be940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.733+0000 7f8bced9d640 1 --2- 172.21.15.139:0/4035947106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b45e0 0x558a046be940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.733+0000 7f8bced9d640 1 -- 172.21.15.139:0/4035947106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a045bf340 con 0x558a046b3cc0 2024-09-16T01:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.733+0000 7f8bcdd9b640 1 --2- 172.21.15.139:0/4035947106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046bee80 0x558a046c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.733+0000 7f8bce59c640 1 --2- 172.21.15.139:0/4035947106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b45e0 0x558a046be940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.733+0000 7f8bbf7fe640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f8bb80a33c0 0x7f8bb80a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.734+0000 7f8bced9d640 1 --2- 172.21.15.139:0/4035947106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046b3cc0 0x558a046b40a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0065360 tx=0x7f8bd00505a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64acbe60ff5c5a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.734+0000 7f8bce59c640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f8bb80a33c0 0x7f8bb80a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.734+0000 7f8bbcff9640 1 -- 172.21.15.139:0/4035947106 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd008f070 con 0x558a046b3cc0 2024-09-16T01:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.734+0000 7f8bbcff9640 1 -- 172.21.15.139:0/4035947106 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd006b7a0 con 0x558a046b3cc0 2024-09-16T01:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.734+0000 7f8bbcff9640 1 -- 172.21.15.139:0/4035947106 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0074e60 con 0x558a046b3cc0 2024-09-16T01:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.734+0000 7f8bbf7fe640 1 -- 172.21.15.139:0/1756600025 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f8bc00d04e0 con 0x7f8bc8095a50 2024-09-16T01:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.735+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/4035947106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046b3cc0 msgr2=0x558a046b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.735+0000 7f8bce59c640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f8bb80a33c0 0x7f8bb80a5880 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00cd930 tx=0x7f8bc0004a40 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.735+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/4035947106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046b3cc0 0x558a046b40a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0065360 tx=0x7f8bd00505a0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.735+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/4035947106 shutdown_connections 2024-09-16T01:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.735+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/4035947106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046bee80 0x558a046c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.735+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/4035947106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b45e0 0x558a046be940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.735+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/4035947106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046b3cc0 0x558a046b40a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.735+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/4035947106 >> 172.21.15.139:0/4035947106 conn(0x558a046b0230 msgr2=0x558a046b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.735+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/4035947106 shutdown_connections 2024-09-16T01:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.735+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/4035947106 wait complete. 2024-09-16T01:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.736+0000 7f8bd7c1d740 1 -- start start 2024-09-16T01:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.736+0000 7f8bd7c1d740 1 Processor -- start 2024-09-16T01:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.736+0000 7f8bd7c1d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b3cc0 0x558a046bbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.736+0000 7f8bd7c1d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b45e0 0x558a046bc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.736+0000 7f8bd7c1d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046bee80 0x558a046bc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.736+0000 7f8bd7c1d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558a046c1a50 con 0x558a046b45e0 2024-09-16T01:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.736+0000 7f8bd7c1d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558a045a1290 con 0x558a046bee80 2024-09-16T01:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.736+0000 7f8bd7c1d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558a046c1bd0 con 0x558a046b3cc0 2024-09-16T01:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.736+0000 7f8bd7c1d740 10 client.0 _ll_register_callbacks cb 0x558a045d2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.736+0000 7f8bcdd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b45e0 0x558a046bc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.736+0000 7f8bcdd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b45e0 0x558a046bc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35314/0 (socket says 172.21.15.139:35314) 2024-09-16T01:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.736+0000 7f8bced9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046bee80 0x558a046bc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.736+0000 7f8bcdd9b640 1 -- 172.21.15.139:0/170904001 learned_addr learned my addr 172.21.15.139:0/170904001 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.736+0000 7f8bced9d640 1 --2- 172.21.15.139:0/170904001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b3cc0 0x558a046bbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.737+0000 7f8bcdd9b640 1 -- 172.21.15.139:0/170904001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b3cc0 msgr2=0x558a046bbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.737+0000 7f8bcdd9b640 1 --2- 172.21.15.139:0/170904001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b3cc0 0x558a046bbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.737+0000 7f8bcdd9b640 1 -- 172.21.15.139:0/170904001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046bee80 msgr2=0x558a046bc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.737+0000 7f8bcdd9b640 1 --2- 172.21.15.139:0/170904001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046bee80 0x558a046bc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.737+0000 7f8bcdd9b640 1 -- 172.21.15.139:0/170904001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a0432b210 con 0x558a046b45e0 2024-09-16T01:44:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.737+0000 7f8bced9d640 1 --2- 172.21.15.139:0/170904001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046bee80 0x558a046bc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:44:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.737+0000 7f8bcdd9b640 1 --2- 172.21.15.139:0/170904001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b45e0 0x558a046bc320 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009890 tx=0x7f8bc4005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4fa0607817862d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.737+0000 7f8bb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:44:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.737+0000 7f8bb5ffb640 1 -- 172.21.15.139:0/170904001 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40308b0 con 0x558a046b45e0 2024-09-16T01:44:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.737+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/170904001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd006b7a0 con 0x558a046b45e0 2024-09-16T01:44:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.737+0000 7f8bb5ffb640 1 -- 172.21.15.139:0/170904001 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f8bc4005040 con 0x558a046b45e0 2024-09-16T01:44:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.737+0000 7f8bb5ffb640 1 -- 172.21.15.139:0/170904001 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8bc404e050 con 0x558a046b45e0 2024-09-16T01:44:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.738+0000 7f8bd7c1d740 10 client.16002 fetch_fsmap learned FSMap version 33 2024-09-16T01:44:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.738+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/170904001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558a043f7620 con 0x558a046b45e0 2024-09-16T01:44:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.738+0000 7f8bb5ffb640 1 -- 172.21.15.139:0/170904001 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd006b7a0 con 0x558a046b45e0 2024-09-16T01:44:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.738+0000 7f8bb5ffb640 1 -- 172.21.15.139:0/170904001 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7f8bc402f570 con 0x558a046b45e0 2024-09-16T01:44:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.738+0000 7f8bd7c1d740 10 client.16002 fetch_fsmap finished waiting for FSMap version 33 2024-09-16T01:44:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.738+0000 7f8bd7c1d740 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:44:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.738+0000 7f8bd7c1d740 20 client.16002 populate_metadata read hostname 'smithi139' 2024-09-16T01:44:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.738+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/170904001 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558a046d7790 0x558a046d9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.738+0000 7f8bd7c1d740 4 client.16002 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:44:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.738+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/170904001 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a04625a20 con 0x558a046d7790 2024-09-16T01:44:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.739+0000 7f8bced9d640 1 --2- 172.21.15.139:0/170904001 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558a046d7790 0x558a046d9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.739+0000 7f8bb5ffb640 10 client.16002 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.739+0000 7f8bced9d640 1 --2- 172.21.15.139:0/170904001 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558a046d7790 0x558a046d9bb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=664505a8e1da8d5a server_cookie=f5f812dc0b5fd608 in_seq=0 out_seq=0 2024-09-16T01:44:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bb5ffb640 1 -- 172.21.15.139:0/170904001 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a04625a20 con 0x558a046d7790 2024-09-16T01:44:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bb5ffb640 10 client.16002 handle_command_reply: tid=0 2024-09-16T01:44:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bd7c1d740 1 client.16002 shutdown 2024-09-16T01:44:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/170904001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b45e0 msgr2=0x558a046bc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/170904001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b45e0 0x558a046bc320 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009890 tx=0x7f8bc4005cd0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/170904001 shutdown_connections 2024-09-16T01:44:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/170904001 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558a046d7790 0x558a046d9bb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/170904001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046bee80 0x558a046bc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bb5ffb640 0 client.16002 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/170904001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b45e0 0x558a046bc320 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/170904001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b3cc0 0x558a046bbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/170904001 >> 172.21.15.139:0/170904001 conn(0x558a046b0230 msgr2=0x558a046b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/170904001 shutdown_connections 2024-09-16T01:44:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/170904001 wait complete. 2024-09-16T01:44:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.764+0000 7f8bd7c1d740 20 client.16002 trim_cache size 0 max 16384 2024-09-16T01:44:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.784+0000 7f8bd7c1d740 1 Processor -- start 2024-09-16T01:44:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.784+0000 7f8bd7c1d740 1 -- start start 2024-09-16T01:44:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.784+0000 7f8bd7c1d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046b3cc0 0x558a046df480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.784+0000 7f8bd7c1d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b45e0 0x558a046df9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.784+0000 7f8bd7c1d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046bee80 0x558a046b5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.784+0000 7f8bd7c1d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8140930 con 0x558a046bee80 2024-09-16T01:44:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.784+0000 7f8bd7c1d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8142530 con 0x558a046b3cc0 2024-09-16T01:44:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.784+0000 7f8bd7c1d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8141000 con 0x558a046b45e0 2024-09-16T01:44:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.785+0000 7f8bced9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046bee80 0x558a046b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.785+0000 7f8bced9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046bee80 0x558a046b5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35324/0 (socket says 172.21.15.139:35324) 2024-09-16T01:44:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.785+0000 7f8bced9d640 1 -- 172.21.15.139:0/2114025778 learned_addr learned my addr 172.21.15.139:0/2114025778 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.785+0000 7f8bcdd9b640 1 --2- 172.21.15.139:0/2114025778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b45e0 0x558a046df9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.785+0000 7f8bced9d640 1 --2- 172.21.15.139:0/2114025778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046b3cc0 0x558a046df480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.785+0000 7f8bced9d640 1 -- 172.21.15.139:0/2114025778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b45e0 msgr2=0x558a046df9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.785+0000 7f8bced9d640 1 --2- 172.21.15.139:0/2114025778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b45e0 0x558a046df9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.785+0000 7f8bced9d640 1 -- 172.21.15.139:0/2114025778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046b3cc0 msgr2=0x558a046df480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.785+0000 7f8bced9d640 1 --2- 172.21.15.139:0/2114025778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046b3cc0 0x558a046df480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.785+0000 7f8bced9d640 1 -- 172.21.15.139:0/2114025778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a046d6220 con 0x558a046bee80 2024-09-16T01:44:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.785+0000 7f8bced9d640 1 --2- 172.21.15.139:0/2114025778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046bee80 0x558a046b5720 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0067960 tx=0x7f8bd0067b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3fa52667ec4375 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.785+0000 7f8bb7fff640 1 -- 172.21.15.139:0/2114025778 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd008f070 con 0x558a046bee80 2024-09-16T01:44:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.785+0000 7f8bb7fff640 1 -- 172.21.15.139:0/2114025778 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd0090040 con 0x558a046bee80 2024-09-16T01:44:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.785+0000 7f8bb7fff640 1 -- 172.21.15.139:0/2114025778 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0062630 con 0x558a046bee80 2024-09-16T01:44:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.786+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/2114025778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046bee80 msgr2=0x558a046b5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.786+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/2114025778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046bee80 0x558a046b5720 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0067960 tx=0x7f8bd0067b50 comp rx=0 tx=0).stop 2024-09-16T01:44:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.786+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/2114025778 shutdown_connections 2024-09-16T01:44:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.786+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/2114025778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046bee80 0x558a046b5720 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.786+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/2114025778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b45e0 0x558a046df9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.786+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/2114025778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046b3cc0 0x558a046df480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.786+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/2114025778 >> 172.21.15.139:0/2114025778 conn(0x558a046b0230 msgr2=0x558a046b7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.786+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/2114025778 shutdown_connections 2024-09-16T01:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.786+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/2114025778 wait complete. 2024-09-16T01:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.787+0000 7f8bd7c1d740 1 -- start start 2024-09-16T01:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.787+0000 7f8bd7c1d740 1 Processor -- start 2024-09-16T01:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.787+0000 7f8bd7c1d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b3cc0 0x558a046cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.787+0000 7f8bd7c1d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b45e0 0x558a046cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.787+0000 7f8bd7c1d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046bee80 0x558a046b8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.787+0000 7f8bd7c1d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8141da0 con 0x558a046b3cc0 2024-09-16T01:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.787+0000 7f8bd7c1d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81416d0 con 0x558a046bee80 2024-09-16T01:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.787+0000 7f8bd7c1d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc813e910 con 0x558a046b45e0 2024-09-16T01:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.787+0000 7f8bd7c1d740 10 client.0 _ll_register_callbacks cb 0x7f8bd00636c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.787+0000 7f8bced9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b3cc0 0x558a046cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.787+0000 7f8bcdd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b45e0 0x558a046cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.787+0000 7f8bced9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b3cc0 0x558a046cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35326/0 (socket says 172.21.15.139:35326) 2024-09-16T01:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.788+0000 7f8bced9d640 1 -- 172.21.15.139:0/3822984659 learned_addr learned my addr 172.21.15.139:0/3822984659 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.788+0000 7f8bced9d640 1 --2- 172.21.15.139:0/3822984659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046bee80 0x558a046b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.788+0000 7f8bcdd9b640 1 -- 172.21.15.139:0/3822984659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046bee80 msgr2=0x558a046b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.788+0000 7f8bcdd9b640 1 --2- 172.21.15.139:0/3822984659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046bee80 0x558a046b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.788+0000 7f8bcdd9b640 1 -- 172.21.15.139:0/3822984659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b3cc0 msgr2=0x558a046cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.788+0000 7f8bcdd9b640 1 --2- 172.21.15.139:0/3822984659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b3cc0 0x558a046cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.788+0000 7f8bcdd9b640 1 -- 172.21.15.139:0/3822984659 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a046b9040 con 0x558a046b45e0 2024-09-16T01:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.788+0000 7f8bced9d640 1 --2- 172.21.15.139:0/3822984659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b3cc0 0x558a046cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.788+0000 7f8bcdd9b640 1 --2- 172.21.15.139:0/3822984659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b45e0 0x558a046cff60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8bc402f9c0 tx=0x7f8bc40308e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff61ba13d47bdde0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.788+0000 7f8bb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.789+0000 7f8bb5ffb640 1 -- 172.21.15.139:0/3822984659 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4057060 con 0x558a046b45e0 2024-09-16T01:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.789+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/3822984659 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd0090040 con 0x558a046b45e0 2024-09-16T01:44:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.789+0000 7f8bb5ffb640 1 -- 172.21.15.139:0/3822984659 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f8bc402f730 con 0x558a046b45e0 2024-09-16T01:44:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.789+0000 7f8bb5ffb640 1 -- 172.21.15.139:0/3822984659 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8bc40096b0 con 0x558a046b45e0 2024-09-16T01:44:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.789+0000 7f8bd7c1d740 10 client.25471 fetch_fsmap learned FSMap version 33 2024-09-16T01:44:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.789+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/3822984659 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558a047420c0 con 0x558a046b45e0 2024-09-16T01:44:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.789+0000 7f8bb5ffb640 1 -- 172.21.15.139:0/3822984659 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd0090040 con 0x558a046b45e0 2024-09-16T01:44:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.790+0000 7f8bb5ffb640 1 -- 172.21.15.139:0/3822984659 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7f8bc4004750 con 0x558a046b45e0 2024-09-16T01:44:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.790+0000 7f8bd7c1d740 10 client.25471 fetch_fsmap finished waiting for FSMap version 33 2024-09-16T01:44:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.790+0000 7f8bd7c1d740 10 client.25471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:44:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.790+0000 7f8bd7c1d740 20 client.25471 populate_metadata read hostname 'smithi139' 2024-09-16T01:44:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.790+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/3822984659 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558a04742890 0x558a04744cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.790+0000 7f8bd7c1d740 4 client.25471 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:44:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.790+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/3822984659 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a0461fd70 con 0x558a04742890 2024-09-16T01:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.790+0000 7f8bced9d640 1 --2- 172.21.15.139:0/3822984659 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558a04742890 0x558a04744cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.790+0000 7f8bb5ffb640 10 client.25471 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:44.790+0000 7f8bced9d640 1 --2- 172.21.15.139:0/3822984659 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558a04742890 0x558a04744cb0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d5a175595fe2dcf server_cookie=3505061fe596deef in_seq=0 out_seq=0 2024-09-16T01:44:45.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:45 smithi164 ceph-mon[35121]: pgmap v133: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 320 op/s 2024-09-16T01:44:45.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:45 smithi139 ceph-mon[31175]: pgmap v133: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 320 op/s 2024-09-16T01:44:45.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:45 smithi176 ceph-mon[35324]: pgmap v133: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 320 op/s 2024-09-16T01:44:45.967 INFO:tasks.workunit.client.0.smithi139.stdout: 365 805720 1160 565803 1008 145351 678 2024-09-16T01:44:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.597+0000 7f8bb5ffb640 1 -- 172.21.15.139:0/3822984659 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a0461fd70 con 0x558a04742890 2024-09-16T01:44:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.597+0000 7f8bb5ffb640 10 client.25471 handle_command_reply: tid=0 2024-09-16T01:44:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.597+0000 7f8bd7c1d740 1 client.25471 shutdown 2024-09-16T01:44:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.597+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/3822984659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b45e0 msgr2=0x558a046cff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.597+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/3822984659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b45e0 0x558a046cff60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8bc402f9c0 tx=0x7f8bc40308e0 comp rx=0 tx=0).stop 2024-09-16T01:44:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.597+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/3822984659 shutdown_connections 2024-09-16T01:44:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.597+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/3822984659 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558a04742890 0x558a04744cb0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.597+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/3822984659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a046bee80 0x558a046b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.597+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/3822984659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a046b45e0 0x558a046cff60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.597+0000 7f8bd7c1d740 1 --2- 172.21.15.139:0/3822984659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558a046b3cc0 0x558a046cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.597+0000 7f8bb5ffb640 0 client.25471 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.597+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/3822984659 >> 172.21.15.139:0/3822984659 conn(0x558a046b0230 msgr2=0x558a046b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.597+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/3822984659 shutdown_connections 2024-09-16T01:44:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.597+0000 7f8bd7c1d740 1 -- 172.21.15.139:0/3822984659 wait complete. 2024-09-16T01:44:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.598+0000 7f8bd7c1d740 20 client.25471 trim_cache size 0 max 16384 2024-09-16T01:44:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:44:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:44:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:44:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:44:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.598+0000 7f8bcf59e640 1 -- 172.21.15.139:0/1756600025 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f8bb80a33c0 msgr2=0x7f8bb80a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.598+0000 7f8bcf59e640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f8bb80a33c0 0x7f8bb80a5880 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00cd930 tx=0x7f8bc0004a40 comp rx=0 tx=0).stop 2024-09-16T01:44:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.598+0000 7f8bcf59e640 1 -- 172.21.15.139:0/1756600025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc8095a50 msgr2=0x7f8bc812bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.598+0000 7f8bcf59e640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc8095a50 0x7f8bc812bf20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000e9f0 tx=0x7f8bc000edf0 comp rx=0 tx=0).stop 2024-09-16T01:44:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.598+0000 7f8bcf59e640 1 -- 172.21.15.139:0/1756600025 shutdown_connections 2024-09-16T01:44:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.598+0000 7f8bcf59e640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f8bb80a33c0 0x7f8bb80a5880 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.598+0000 7f8bcf59e640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bc809b6a0 0x7f8bc81307e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.598+0000 7f8bcf59e640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8bc80963d0 0x7f8bc812c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.598+0000 7f8bcf59e640 1 --2- 172.21.15.139:0/1756600025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc8095a50 0x7f8bc812bf20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.598+0000 7f8bcf59e640 1 -- 172.21.15.139:0/1756600025 >> 172.21.15.139:0/1756600025 conn(0x7f8bc800b350 msgr2=0x7f8bc8094800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.598+0000 7f8bcf59e640 1 -- 172.21.15.139:0/1756600025 shutdown_connections 2024-09-16T01:44:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:46.598+0000 7f8bcf59e640 1 -- 172.21.15.139:0/1756600025 wait complete. 2024-09-16T01:44:47.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:47 smithi164 ceph-mon[35121]: pgmap v134: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.3 MiB/s wr, 483 op/s 2024-09-16T01:44:47.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:47 smithi139 ceph-mon[31175]: pgmap v134: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.3 MiB/s wr, 483 op/s 2024-09-16T01:44:47.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:47 smithi176 ceph-mon[35324]: pgmap v134: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.3 MiB/s wr, 483 op/s 2024-09-16T01:44:49.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:49 smithi139 ceph-mon[31175]: pgmap v135: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 843 op/s 2024-09-16T01:44:49.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:49 smithi164 ceph-mon[35121]: pgmap v135: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 843 op/s 2024-09-16T01:44:49.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:49 smithi176 ceph-mon[35324]: pgmap v135: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 843 op/s 2024-09-16T01:44:51.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:51 smithi164 ceph-mon[35121]: pgmap v136: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 937 op/s 2024-09-16T01:44:51.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:51 smithi139 ceph-mon[31175]: pgmap v136: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 937 op/s 2024-09-16T01:44:51.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:51 smithi176 ceph-mon[35324]: pgmap v136: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 937 op/s 2024-09-16T01:44:53.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:53 smithi164 ceph-mon[35121]: pgmap v137: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-09-16T01:44:53.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:53 smithi139 ceph-mon[31175]: pgmap v137: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-09-16T01:44:53.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:53 smithi176 ceph-mon[35324]: pgmap v137: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-09-16T01:44:55.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:55 smithi164 ceph-mon[35121]: pgmap v138: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 17 MiB/s wr, 1.62k op/s 2024-09-16T01:44:55.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:55 smithi139 ceph-mon[31175]: pgmap v138: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 17 MiB/s wr, 1.62k op/s 2024-09-16T01:44:55.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:55 smithi176 ceph-mon[35324]: pgmap v138: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 17 MiB/s wr, 1.62k op/s 2024-09-16T01:44:55.968 INFO:tasks.workunit.client.0.smithi139.stdout: 365 14 0 13 0 115 0 2024-09-16T01:44:57.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:57 smithi164 ceph-mon[35121]: pgmap v139: 129 pgs: 129 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 18 MiB/s wr, 1.89k op/s 2024-09-16T01:44:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:57 smithi139 ceph-mon[31175]: pgmap v139: 129 pgs: 129 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 18 MiB/s wr, 1.89k op/s 2024-09-16T01:44:57.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:57 smithi176 ceph-mon[35324]: pgmap v139: 129 pgs: 129 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 18 MiB/s wr, 1.89k op/s 2024-09-16T01:44:59.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.235+0000 7fb01bfff640 1 -- 172.21.15.139:0/2419414613 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555729fb8d20 con 0x55572a0f4720 2024-09-16T01:44:59.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.235+0000 7fb01bfff640 10 client.15990 handle_command_reply: tid=0 2024-09-16T01:44:59.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.235+0000 7fb044f36740 1 client.15990 shutdown 2024-09-16T01:44:59.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.235+0000 7fb044f36740 1 -- 172.21.15.139:0/2419414613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a05b380 msgr2=0x55572a0f2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:59.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.235+0000 7fb044f36740 1 --2- 172.21.15.139:0/2419414613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a05b380 0x55572a0f2710 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb0340393b0 tx=0x7fb03403d4a0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.235+0000 7fb044f36740 1 -- 172.21.15.139:0/2419414613 shutdown_connections 2024-09-16T01:44:59.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.235+0000 7fb044f36740 1 --2- 172.21.15.139:0/2419414613 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55572a0f4720 0x55572a0f6b40 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.235+0000 7fb044f36740 1 --2- 172.21.15.139:0/2419414613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55572a05b380 0x55572a0f2710 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.235+0000 7fb044f36740 1 --2- 172.21.15.139:0/2419414613 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55572a056940 0x55572a05ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.235+0000 7fb01bfff640 0 client.15990 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:59.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.235+0000 7fb044f36740 1 --2- 172.21.15.139:0/2419414613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572a04b760 0x55572a05a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.235+0000 7fb044f36740 1 -- 172.21.15.139:0/2419414613 >> 172.21.15.139:0/2419414613 conn(0x55572a047cf0 msgr2=0x55572a046650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:59.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.236+0000 7fb044f36740 1 -- 172.21.15.139:0/2419414613 shutdown_connections 2024-09-16T01:44:59.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.236+0000 7fb044f36740 1 -- 172.21.15.139:0/2419414613 wait complete. 2024-09-16T01:44:59.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.236+0000 7fb044f36740 20 client.15990 trim_cache size 0 max 16384 2024-09-16T01:44:59.239 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:44:59.239 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 33, 2024-09-16T01:44:59.239 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:44:59.240 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 14, 2024-09-16T01:44:59.240 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:44:59.240 INFO:teuthology.orchestra.run.smithi139.stdout: "f152ec88": { 2024-09-16T01:44:59.240 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 14, 2024-09-16T01:44:59.240 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:44:59.240 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:44:59.240 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:44:59.240 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:44:59.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:44:59.240 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:44:59.240 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 133.8, 2024-09-16T01:44:59.240 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:44:59.240 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:44:59.240 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:44:59.241 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:44:59.241 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:44:59.241 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:44:59.241 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:44:59.241 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:44:59.241 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:44:59.241 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:44:59.241 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:44:59.241 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:44:59.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.236+0000 7fb042b5d640 1 -- 172.21.15.139:0/1889254180 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0280a35c0 msgr2=0x7fb0280a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:59.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.236+0000 7fb042b5d640 1 --2- 172.21.15.139:0/1889254180 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0280a35c0 0x7fb0280a5a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55572a049070 tx=0x7fb02c0d61a0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.236+0000 7fb042b5d640 1 -- 172.21.15.139:0/1889254180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb03c101460 msgr2=0x7fb03c115200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:59.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.236+0000 7fb042b5d640 1 --2- 172.21.15.139:0/1889254180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb03c101460 0x7fb03c115200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00eea0 tx=0x7fb02c00ef90 comp rx=0 tx=0).stop 2024-09-16T01:44:59.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.237+0000 7fb042b5d640 1 -- 172.21.15.139:0/1889254180 shutdown_connections 2024-09-16T01:44:59.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.237+0000 7fb042b5d640 1 --2- 172.21.15.139:0/1889254180 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0280a35c0 0x7fb0280a5a80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.237+0000 7fb042b5d640 1 --2- 172.21.15.139:0/1889254180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb03c105f50 0x7fb03c10df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.237+0000 7fb042b5d640 1 --2- 172.21.15.139:0/1889254180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb03c101460 0x7fb03c115200 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.237+0000 7fb042b5d640 1 --2- 172.21.15.139:0/1889254180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb03c100b40 0x7fb03c114cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.237+0000 7fb042b5d640 1 -- 172.21.15.139:0/1889254180 >> 172.21.15.139:0/1889254180 conn(0x7fb03c0fc7d0 msgr2=0x7fb03c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:59.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.237+0000 7fb042b5d640 1 -- 172.21.15.139:0/1889254180 shutdown_connections 2024-09-16T01:44:59.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:44:59.237+0000 7fb042b5d640 1 -- 172.21.15.139:0/1889254180 wait complete. 2024-09-16T01:44:59.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:44:59 smithi164 ceph-mon[35121]: pgmap v140: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 15 MiB/s wr, 1.71k op/s 2024-09-16T01:44:59.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:44:59 smithi139 ceph-mon[31175]: pgmap v140: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 15 MiB/s wr, 1.71k op/s 2024-09-16T01:44:59.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:44:59 smithi176 ceph-mon[35324]: pgmap v140: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 15 MiB/s wr, 1.71k op/s 2024-09-16T01:44:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.712+0000 7feba805e640 1 -- 172.21.15.139:0/3451622007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feba0102470 msgr2=0x7feba010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.712+0000 7feba805e640 1 --2- 172.21.15.139:0/3451622007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feba0102470 0x7feba010e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feb98009be0 tx=0x7feb9802a6c0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.712+0000 7feba805e640 1 -- 172.21.15.139:0/3451622007 shutdown_connections 2024-09-16T01:44:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.712+0000 7feba805e640 1 --2- 172.21.15.139:0/3451622007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feba0102470 0x7feba010e9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.712+0000 7feba805e640 1 --2- 172.21.15.139:0/3451622007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba0101ad0 0x7feba0101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.712+0000 7feba805e640 1 --2- 172.21.15.139:0/3451622007 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feba0107b70 0x7feba0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.712+0000 7feba805e640 1 -- 172.21.15.139:0/3451622007 >> 172.21.15.139:0/3451622007 conn(0x7feba00fd7d0 msgr2=0x7feba00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.712+0000 7feba805e640 1 -- 172.21.15.139:0/3451622007 shutdown_connections 2024-09-16T01:44:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.712+0000 7feba805e640 1 -- 172.21.15.139:0/3451622007 wait complete. 2024-09-16T01:44:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba805e640 1 Processor -- start 2024-09-16T01:44:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba805e640 1 -- start start 2024-09-16T01:44:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba805e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feba0101ad0 0x7feba01a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba805e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feba0102470 0x7feba01a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba805e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba0107b70 0x7feba010dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba805e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feba0111e90 con 0x7feba0107b70 2024-09-16T01:44:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba805e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feba0111d10 con 0x7feba0102470 2024-09-16T01:44:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba805e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feba0112010 con 0x7feba0101ad0 2024-09-16T01:44:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba65fb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba0107b70 0x7feba010dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba55f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feba0102470 0x7feba01a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba65fb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba0107b70 0x7feba010dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52656/0 (socket says 172.21.15.139:52656) 2024-09-16T01:44:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba65fb640 1 -- 172.21.15.139:0/858987888 learned_addr learned my addr 172.21.15.139:0/858987888 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba5dfa640 1 --2- 172.21.15.139:0/858987888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feba0101ad0 0x7feba01a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba55f9640 1 -- 172.21.15.139:0/858987888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feba0101ad0 msgr2=0x7feba01a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.713+0000 7feba55f9640 1 --2- 172.21.15.139:0/858987888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feba0101ad0 0x7feba01a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.714+0000 7feba55f9640 1 -- 172.21.15.139:0/858987888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba0107b70 msgr2=0x7feba010dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.714+0000 7feba55f9640 1 --2- 172.21.15.139:0/858987888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba0107b70 0x7feba010dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.714+0000 7feba55f9640 1 -- 172.21.15.139:0/858987888 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba010e380 con 0x7feba0102470 2024-09-16T01:44:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.714+0000 7feba55f9640 1 --2- 172.21.15.139:0/858987888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feba0102470 0x7feba01a4b90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feb90005fd0 tx=0x7feb9000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.714+0000 7feb96ffd640 1 -- 172.21.15.139:0/858987888 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9000cd00 con 0x7feba0102470 2024-09-16T01:44:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.714+0000 7feba805e640 1 -- 172.21.15.139:0/858987888 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba010e670 con 0x7feba0102470 2024-09-16T01:44:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.714+0000 7feb96ffd640 1 -- 172.21.15.139:0/858987888 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb900035f0 con 0x7feba0102470 2024-09-16T01:44:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.714+0000 7feb96ffd640 1 -- 172.21.15.139:0/858987888 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90010690 con 0x7feba0102470 2024-09-16T01:44:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.714+0000 7feba805e640 1 -- 172.21.15.139:0/858987888 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba01ab5d0 con 0x7feba0102470 2024-09-16T01:44:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.717+0000 7feb96ffd640 1 -- 172.21.15.139:0/858987888 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7feb90010830 con 0x7feba0102470 2024-09-16T01:44:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.717+0000 7febaadf9740 1 Processor -- start 2024-09-16T01:44:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.717+0000 7febaadf9740 1 -- start start 2024-09-16T01:44:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.717+0000 7febaadf9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f8af0 0x562a3a6f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.717+0000 7febaadf9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a6f9410 0x562a3a703770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.717+0000 7febaadf9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a703cb0 0x562a3a7060a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.717+0000 7febaadf9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562a3a65d2f0 con 0x562a3a6f9410 2024-09-16T01:44:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.717+0000 7febaadf9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562a3a605270 con 0x562a3a703cb0 2024-09-16T01:44:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.717+0000 7febaadf9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562a3a6592b0 con 0x562a3a6f8af0 2024-09-16T01:44:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.718+0000 7feba65fb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f8af0 0x562a3a6f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.718+0000 7feba65fb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f8af0 0x562a3a6f8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:50452/0 (socket says 172.21.15.139:50452) 2024-09-16T01:44:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.718+0000 7feba65fb640 1 -- 172.21.15.139:0/2395286515 learned_addr learned my addr 172.21.15.139:0/2395286515 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.718+0000 7feba55f9640 1 --2- 172.21.15.139:0/2395286515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a703cb0 0x562a3a7060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.718+0000 7feba5dfa640 1 --2- 172.21.15.139:0/2395286515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a6f9410 0x562a3a703770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.718+0000 7feb96ffd640 1 --2- 172.21.15.139:0/858987888 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feb8c0a3460 0x7feb8c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.718+0000 7feba65fb640 1 -- 172.21.15.139:0/2395286515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a703cb0 msgr2=0x562a3a7060a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.718+0000 7feba65fb640 1 --2- 172.21.15.139:0/2395286515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a703cb0 0x562a3a7060a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.718+0000 7feba65fb640 1 -- 172.21.15.139:0/2395286515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a6f9410 msgr2=0x562a3a703770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.718+0000 7feba65fb640 1 --2- 172.21.15.139:0/2395286515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a6f9410 0x562a3a703770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.718+0000 7feba65fb640 1 -- 172.21.15.139:0/2395286515 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a3a604340 con 0x562a3a6f8af0 2024-09-16T01:44:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.718+0000 7feba65fb640 1 --2- 172.21.15.139:0/858987888 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feb8c0a3460 0x7feb8c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.718+0000 7feb96ffd640 1 -- 172.21.15.139:0/858987888 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7feb900d0370 con 0x7feba0102470 2024-09-16T01:44:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.719+0000 7feba65fb640 1 --2- 172.21.15.139:0/2395286515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f8af0 0x562a3a6f8ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feb98006010 tx=0x7feb98005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70484d8b7496883e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.719+0000 7feba65fb640 1 --2- 172.21.15.139:0/858987888 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feb8c0a3460 0x7feb8c0a5920 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb9803f000 tx=0x7feb98040000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.719+0000 7feb8bfff640 1 -- 172.21.15.139:0/2395286515 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98042040 con 0x562a3a6f8af0 2024-09-16T01:44:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.719+0000 7feb8bfff640 1 -- 172.21.15.139:0/2395286515 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb98037040 con 0x562a3a6f8af0 2024-09-16T01:44:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.719+0000 7feb8bfff640 1 -- 172.21.15.139:0/2395286515 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98036070 con 0x562a3a6f8af0 2024-09-16T01:44:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.720+0000 7febaadf9740 1 -- 172.21.15.139:0/2395286515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f8af0 msgr2=0x562a3a6f8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.720+0000 7febaadf9740 1 --2- 172.21.15.139:0/2395286515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f8af0 0x562a3a6f8ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feb98006010 tx=0x7feb98005ec0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.720+0000 7febaadf9740 1 -- 172.21.15.139:0/2395286515 shutdown_connections 2024-09-16T01:44:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.720+0000 7febaadf9740 1 --2- 172.21.15.139:0/2395286515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a703cb0 0x562a3a7060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.720+0000 7febaadf9740 1 --2- 172.21.15.139:0/2395286515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a6f9410 0x562a3a703770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.720+0000 7febaadf9740 1 --2- 172.21.15.139:0/2395286515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f8af0 0x562a3a6f8ed0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.720+0000 7febaadf9740 1 -- 172.21.15.139:0/2395286515 >> 172.21.15.139:0/2395286515 conn(0x562a3a6f5060 msgr2=0x562a3a6f5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.720+0000 7febaadf9740 1 -- 172.21.15.139:0/2395286515 shutdown_connections 2024-09-16T01:44:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.720+0000 7febaadf9740 1 -- 172.21.15.139:0/2395286515 wait complete. 2024-09-16T01:44:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.720+0000 7febaadf9740 1 -- start start 2024-09-16T01:44:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.720+0000 7febaadf9740 1 Processor -- start 2024-09-16T01:44:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7febaadf9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f8af0 0x562a3a700c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7febaadf9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f9410 0x562a3a701150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7febaadf9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a703cb0 0x562a3a701690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7febaadf9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562a3a706880 con 0x562a3a703cb0 2024-09-16T01:44:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7febaadf9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562a3a5e6290 con 0x562a3a6f8af0 2024-09-16T01:44:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7febaadf9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562a3a706a00 con 0x562a3a6f9410 2024-09-16T01:44:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7febaadf9740 10 client.0 _ll_register_callbacks cb 0x562a3a617140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:44:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7feba55f9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f9410 0x562a3a701150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7feba5dfa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f8af0 0x562a3a700c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7feba65fb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a703cb0 0x562a3a701690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7feba55f9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f9410 0x562a3a701150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:50454/0 (socket says 172.21.15.139:50454) 2024-09-16T01:44:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7feba5dfa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f8af0 0x562a3a700c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:38768/0 (socket says 172.21.15.139:38768) 2024-09-16T01:44:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7feba55f9640 1 -- 172.21.15.139:0/2920449057 learned_addr learned my addr 172.21.15.139:0/2920449057 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7feba5dfa640 1 -- 172.21.15.139:0/2920449057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f9410 msgr2=0x562a3a701150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7feba5dfa640 1 --2- 172.21.15.139:0/2920449057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f9410 0x562a3a701150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7feba5dfa640 1 -- 172.21.15.139:0/2920449057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a703cb0 msgr2=0x562a3a701690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7feba5dfa640 1 --2- 172.21.15.139:0/2920449057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a703cb0 0x562a3a701690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7feba5dfa640 1 -- 172.21.15.139:0/2920449057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a3a370210 con 0x562a3a6f8af0 2024-09-16T01:44:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.721+0000 7feba65fb640 1 --2- 172.21.15.139:0/2920449057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a703cb0 0x562a3a701690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:44:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.722+0000 7feba5dfa640 1 --2- 172.21.15.139:0/2920449057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f8af0 0x562a3a700c10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feb9c012700 tx=0x7feb9c012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eccdff41dd2e95d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.722+0000 7feb897fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:44:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.722+0000 7feb897fa640 1 -- 172.21.15.139:0/2920449057 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00f8a0 con 0x562a3a6f8af0 2024-09-16T01:44:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.722+0000 7febaadf9740 1 -- 172.21.15.139:0/2920449057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb98037040 con 0x562a3a6f8af0 2024-09-16T01:44:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.722+0000 7feb897fa640 1 -- 172.21.15.139:0/2920449057 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7feb9c00fa40 con 0x562a3a6f8af0 2024-09-16T01:44:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.722+0000 7feb897fa640 1 -- 172.21.15.139:0/2920449057 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7feb9c025050 con 0x562a3a6f8af0 2024-09-16T01:44:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.722+0000 7febaadf9740 10 client.25499 fetch_fsmap learned FSMap version 33 2024-09-16T01:44:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.722+0000 7febaadf9740 1 -- 172.21.15.139:0/2920449057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562a3a43c620 con 0x562a3a6f8af0 2024-09-16T01:44:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.722+0000 7feb897fa640 1 -- 172.21.15.139:0/2920449057 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb98037040 con 0x562a3a6f8af0 2024-09-16T01:44:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.722+0000 7feb897fa640 1 -- 172.21.15.139:0/2920449057 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7feb9c0051c0 con 0x562a3a6f8af0 2024-09-16T01:44:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.722+0000 7febaadf9740 10 client.25499 fetch_fsmap finished waiting for FSMap version 33 2024-09-16T01:44:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.722+0000 7febaadf9740 10 client.25499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:44:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.722+0000 7febaadf9740 20 client.25499 populate_metadata read hostname 'smithi139' 2024-09-16T01:44:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.723+0000 7febaadf9740 1 --2- 172.21.15.139:0/2920449057 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562a3a713d20 0x562a3a716160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.723+0000 7febaadf9740 4 client.25499 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:44:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.723+0000 7febaadf9740 1 -- 172.21.15.139:0/2920449057 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a3a66aa20 con 0x562a3a713d20 2024-09-16T01:44:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.723+0000 7feba55f9640 1 --2- 172.21.15.139:0/2920449057 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562a3a713d20 0x562a3a716160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.723+0000 7feba55f9640 1 --2- 172.21.15.139:0/2920449057 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562a3a713d20 0x562a3a716160 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e8121608569c0fd server_cookie=3b073757c168c749 in_seq=0 out_seq=0 2024-09-16T01:44:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.723+0000 7feb897fa640 10 client.25499 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7feb897fa640 1 -- 172.21.15.139:0/2920449057 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a3a66aa20 con 0x562a3a713d20 2024-09-16T01:44:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7feb897fa640 10 client.25499 handle_command_reply: tid=0 2024-09-16T01:44:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7febaadf9740 1 client.25499 shutdown 2024-09-16T01:44:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7febaadf9740 1 -- 172.21.15.139:0/2920449057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f8af0 msgr2=0x562a3a700c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7febaadf9740 1 --2- 172.21.15.139:0/2920449057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f8af0 0x562a3a700c10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feb9c012700 tx=0x7feb9c012bc0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7febaadf9740 1 -- 172.21.15.139:0/2920449057 shutdown_connections 2024-09-16T01:44:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7febaadf9740 1 --2- 172.21.15.139:0/2920449057 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562a3a713d20 0x562a3a716160 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7febaadf9740 1 --2- 172.21.15.139:0/2920449057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a703cb0 0x562a3a701690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7febaadf9740 1 --2- 172.21.15.139:0/2920449057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f9410 0x562a3a701150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7febaadf9740 1 --2- 172.21.15.139:0/2920449057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f8af0 0x562a3a700c10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7feb897fa640 0 client.25499 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:44:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7febaadf9740 1 -- 172.21.15.139:0/2920449057 >> 172.21.15.139:0/2920449057 conn(0x562a3a6f5060 msgr2=0x562a3a6fad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7febaadf9740 1 -- 172.21.15.139:0/2920449057 shutdown_connections 2024-09-16T01:44:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7febaadf9740 1 -- 172.21.15.139:0/2920449057 wait complete. 2024-09-16T01:44:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.724+0000 7febaadf9740 20 client.25499 trim_cache size 0 max 16384 2024-09-16T01:44:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.742+0000 7febaadf9740 1 Processor -- start 2024-09-16T01:44:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.742+0000 7febaadf9740 1 -- start start 2024-09-16T01:44:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.742+0000 7febaadf9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f8af0 0x562a3a6ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.742+0000 7febaadf9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a6f9410 0x562a3a7014c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.742+0000 7febaadf9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a703cb0 0x562a3a71ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.742+0000 7febaadf9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feba01b5630 con 0x562a3a6f9410 2024-09-16T01:44:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.742+0000 7febaadf9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feba01b7230 con 0x562a3a6f8af0 2024-09-16T01:44:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.742+0000 7febaadf9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feba01b5d00 con 0x562a3a703cb0 2024-09-16T01:44:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.742+0000 7feba55f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a6f9410 0x562a3a7014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.742+0000 7feba5dfa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f8af0 0x562a3a6ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.742+0000 7feba5dfa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f8af0 0x562a3a6ff4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:38776/0 (socket says 172.21.15.139:38776) 2024-09-16T01:44:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.743+0000 7feba5dfa640 1 -- 172.21.15.139:0/2485616361 learned_addr learned my addr 172.21.15.139:0/2485616361 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.743+0000 7feba65fb640 1 --2- 172.21.15.139:0/2485616361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a703cb0 0x562a3a71ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.743+0000 7feba5dfa640 1 -- 172.21.15.139:0/2485616361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a703cb0 msgr2=0x562a3a71ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.743+0000 7feba5dfa640 1 --2- 172.21.15.139:0/2485616361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a703cb0 0x562a3a71ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.743+0000 7feba5dfa640 1 -- 172.21.15.139:0/2485616361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a6f9410 msgr2=0x562a3a7014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.743+0000 7feba5dfa640 1 --2- 172.21.15.139:0/2485616361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a6f9410 0x562a3a7014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.743+0000 7feba5dfa640 1 -- 172.21.15.139:0/2485616361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a3a6f8920 con 0x562a3a6f8af0 2024-09-16T01:44:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.743+0000 7feba55f9640 1 --2- 172.21.15.139:0/2485616361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a6f9410 0x562a3a7014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:44:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.743+0000 7feba5dfa640 1 --2- 172.21.15.139:0/2485616361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f8af0 0x562a3a6ff4c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009d20 tx=0x7feb9c009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47bdc7e5da278906 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.743+0000 7feb8b7fe640 1 -- 172.21.15.139:0/2485616361 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c02e070 con 0x562a3a6f8af0 2024-09-16T01:44:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.743+0000 7feb8b7fe640 1 -- 172.21.15.139:0/2485616361 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb9c009dc0 con 0x562a3a6f8af0 2024-09-16T01:44:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.743+0000 7feb8b7fe640 1 -- 172.21.15.139:0/2485616361 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c010b10 con 0x562a3a6f8af0 2024-09-16T01:44:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.744+0000 7febaadf9740 1 -- 172.21.15.139:0/2485616361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f8af0 msgr2=0x562a3a6ff4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:44:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.744+0000 7febaadf9740 1 --2- 172.21.15.139:0/2485616361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f8af0 0x562a3a6ff4c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009d20 tx=0x7feb9c009370 comp rx=0 tx=0).stop 2024-09-16T01:44:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.744+0000 7febaadf9740 1 -- 172.21.15.139:0/2485616361 shutdown_connections 2024-09-16T01:44:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.744+0000 7febaadf9740 1 --2- 172.21.15.139:0/2485616361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a703cb0 0x562a3a71ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.744+0000 7febaadf9740 1 --2- 172.21.15.139:0/2485616361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a6f9410 0x562a3a7014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.744+0000 7febaadf9740 1 --2- 172.21.15.139:0/2485616361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f8af0 0x562a3a6ff4c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.744+0000 7febaadf9740 1 -- 172.21.15.139:0/2485616361 >> 172.21.15.139:0/2485616361 conn(0x562a3a6f5060 msgr2=0x562a3a702c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:44:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.744+0000 7febaadf9740 1 -- 172.21.15.139:0/2485616361 shutdown_connections 2024-09-16T01:44:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.744+0000 7febaadf9740 1 -- 172.21.15.139:0/2485616361 wait complete. 2024-09-16T01:44:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7febaadf9740 1 -- start start 2024-09-16T01:44:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7febaadf9740 1 Processor -- start 2024-09-16T01:44:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7febaadf9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f8af0 0x562a3a70c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7febaadf9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f9410 0x562a3a70c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7febaadf9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a703cb0 0x562a3a6fd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7febaadf9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feba01b6aa0 con 0x562a3a703cb0 2024-09-16T01:44:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7febaadf9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feba01b63d0 con 0x562a3a6f9410 2024-09-16T01:44:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7febaadf9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feba01b3610 con 0x562a3a6f8af0 2024-09-16T01:44:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7febaadf9740 10 client.0 _ll_register_callbacks cb 0x7feb9c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:44:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7feba5dfa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f8af0 0x562a3a70c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7feba65fb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a703cb0 0x562a3a6fd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7feba55f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f9410 0x562a3a70c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7feba5dfa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f8af0 0x562a3a70c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:50470/0 (socket says 172.21.15.139:50470) 2024-09-16T01:44:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7feba65fb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a703cb0 0x562a3a6fd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52700/0 (socket says 172.21.15.139:52700) 2024-09-16T01:44:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.745+0000 7feba5dfa640 1 -- 172.21.15.139:0/1708195218 learned_addr learned my addr 172.21.15.139:0/1708195218 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:44:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.746+0000 7feba55f9640 1 -- 172.21.15.139:0/1708195218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f8af0 msgr2=0x562a3a70c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.746+0000 7feba55f9640 1 --2- 172.21.15.139:0/1708195218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f8af0 0x562a3a70c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.746+0000 7feba55f9640 1 -- 172.21.15.139:0/1708195218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a703cb0 msgr2=0x562a3a6fd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:44:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.746+0000 7feba55f9640 1 --2- 172.21.15.139:0/1708195218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a703cb0 0x562a3a6fd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:44:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.746+0000 7feba55f9640 1 -- 172.21.15.139:0/1708195218 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a3a6fe020 con 0x562a3a6f9410 2024-09-16T01:44:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.746+0000 7feba55f9640 1 --2- 172.21.15.139:0/1708195218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f9410 0x562a3a70c590 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feb900cdd10 tx=0x7feb900d3ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ef69e47a64a00d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:44:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.746+0000 7feb897fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:44:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.746+0000 7feb897fa640 1 -- 172.21.15.139:0/1708195218 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb900cde60 con 0x562a3a6f9410 2024-09-16T01:44:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.746+0000 7feb897fa640 1 -- 172.21.15.139:0/1708195218 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7feb90003b10 con 0x562a3a6f9410 2024-09-16T01:44:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.746+0000 7febaadf9740 1 -- 172.21.15.139:0/1708195218 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb9c009dc0 con 0x562a3a6f9410 2024-09-16T01:44:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.746+0000 7feb897fa640 1 -- 172.21.15.139:0/1708195218 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7feb900037a0 con 0x562a3a6f9410 2024-09-16T01:44:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.746+0000 7febaadf9740 10 client.25505 fetch_fsmap learned FSMap version 33 2024-09-16T01:44:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.746+0000 7febaadf9740 1 -- 172.21.15.139:0/1708195218 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562a3a77e7a0 con 0x562a3a6f9410 2024-09-16T01:44:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.746+0000 7feb897fa640 1 -- 172.21.15.139:0/1708195218 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb9c009dc0 con 0x562a3a6f9410 2024-09-16T01:44:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.747+0000 7feb897fa640 1 -- 172.21.15.139:0/1708195218 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7feb900cd990 con 0x562a3a6f9410 2024-09-16T01:44:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.747+0000 7febaadf9740 10 client.25505 fetch_fsmap finished waiting for FSMap version 33 2024-09-16T01:44:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.747+0000 7febaadf9740 10 client.25505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:44:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.747+0000 7febaadf9740 20 client.25505 populate_metadata read hostname 'smithi139' 2024-09-16T01:44:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.747+0000 7febaadf9740 1 --2- 172.21.15.139:0/1708195218 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562a3a77ef50 0x562a3a781370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:44:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.747+0000 7febaadf9740 4 client.25505 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:44:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.747+0000 7febaadf9740 1 -- 172.21.15.139:0/1708195218 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a3a664d70 con 0x562a3a77ef50 2024-09-16T01:44:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.747+0000 7feba5dfa640 1 --2- 172.21.15.139:0/1708195218 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562a3a77ef50 0x562a3a781370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:44:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.747+0000 7feba5dfa640 1 --2- 172.21.15.139:0/1708195218 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562a3a77ef50 0x562a3a781370 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a21cdc9d213ecfdc server_cookie=7f86d3f75b72b719 in_seq=0 out_seq=0 2024-09-16T01:44:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:44:59.747+0000 7feb897fa640 10 client.25505 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.344+0000 7feb897fa640 1 -- 172.21.15.139:0/1708195218 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a3a664d70 con 0x562a3a77ef50 2024-09-16T01:45:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.344+0000 7feb897fa640 10 client.25505 handle_command_reply: tid=0 2024-09-16T01:45:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.344+0000 7febaadf9740 1 client.25505 shutdown 2024-09-16T01:45:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.345+0000 7febaadf9740 1 -- 172.21.15.139:0/1708195218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f9410 msgr2=0x562a3a70c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.345+0000 7febaadf9740 1 --2- 172.21.15.139:0/1708195218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f9410 0x562a3a70c590 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feb900cdd10 tx=0x7feb900d3ec0 comp rx=0 tx=0).stop 2024-09-16T01:45:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.345+0000 7febaadf9740 1 -- 172.21.15.139:0/1708195218 shutdown_connections 2024-09-16T01:45:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.345+0000 7febaadf9740 1 --2- 172.21.15.139:0/1708195218 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562a3a77ef50 0x562a3a781370 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.345+0000 7febaadf9740 1 --2- 172.21.15.139:0/1708195218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562a3a703cb0 0x562a3a6fd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.345+0000 7febaadf9740 1 --2- 172.21.15.139:0/1708195218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3a6f9410 0x562a3a70c590 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.345+0000 7feb897fa640 0 client.25505 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.345+0000 7febaadf9740 1 --2- 172.21.15.139:0/1708195218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a3a6f8af0 0x562a3a70c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.345+0000 7febaadf9740 1 -- 172.21.15.139:0/1708195218 >> 172.21.15.139:0/1708195218 conn(0x562a3a6f5060 msgr2=0x562a3a6fa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.345+0000 7febaadf9740 1 -- 172.21.15.139:0/1708195218 shutdown_connections 2024-09-16T01:45:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.345+0000 7febaadf9740 1 -- 172.21.15.139:0/1708195218 wait complete. 2024-09-16T01:45:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.345+0000 7febaadf9740 20 client.25505 trim_cache size 0 max 16384 2024-09-16T01:45:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:45:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:45:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:45:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:45:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.346+0000 7feba805e640 1 -- 172.21.15.139:0/858987888 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feb8c0a3460 msgr2=0x7feb8c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.346+0000 7feba805e640 1 --2- 172.21.15.139:0/858987888 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feb8c0a3460 0x7feb8c0a5920 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb9803f000 tx=0x7feb98040000 comp rx=0 tx=0).stop 2024-09-16T01:45:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.346+0000 7feba805e640 1 -- 172.21.15.139:0/858987888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feba0102470 msgr2=0x7feba01a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.346+0000 7feba805e640 1 --2- 172.21.15.139:0/858987888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feba0102470 0x7feba01a4b90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feb90005fd0 tx=0x7feb9000ee50 comp rx=0 tx=0).stop 2024-09-16T01:45:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.346+0000 7feba805e640 1 -- 172.21.15.139:0/858987888 shutdown_connections 2024-09-16T01:45:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.346+0000 7feba805e640 1 --2- 172.21.15.139:0/858987888 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feb8c0a3460 0x7feb8c0a5920 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.346+0000 7feba805e640 1 --2- 172.21.15.139:0/858987888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba0107b70 0x7feba010dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.346+0000 7feba805e640 1 --2- 172.21.15.139:0/858987888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feba0102470 0x7feba01a4b90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.346+0000 7feba805e640 1 --2- 172.21.15.139:0/858987888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feba0101ad0 0x7feba01a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.346+0000 7feba805e640 1 -- 172.21.15.139:0/858987888 >> 172.21.15.139:0/858987888 conn(0x7feba00fd7d0 msgr2=0x7feba00fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.346+0000 7feba805e640 1 -- 172.21.15.139:0/858987888 shutdown_connections 2024-09-16T01:45:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:00.346+0000 7feba805e640 1 -- 172.21.15.139:0/858987888 wait complete. 2024-09-16T01:45:00.860 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f152ec88', quiesce took 27.1 seconds. Will release after: -13.3 2024-09-16T01:45:00.860 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:45:01.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:01 smithi139 ceph-mon[31175]: pgmap v141: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 13 MiB/s wr, 1.31k op/s 2024-09-16T01:45:01.177 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:45:01.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:01 smithi164 ceph-mon[35121]: pgmap v141: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 13 MiB/s wr, 1.31k op/s 2024-09-16T01:45:01.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:01 smithi176 ceph-mon[35324]: pgmap v141: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 13 MiB/s wr, 1.31k op/s 2024-09-16T01:45:01.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.709+0000 7fd4980a2640 1 -- 172.21.15.139:0/2539405154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd490109120 msgr2=0x7fd490110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:01.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.709+0000 7fd4980a2640 1 --2- 172.21.15.139:0/2539405154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd490109120 0x7fd490110ca0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd488009ca0 tx=0x7fd48802a8c0 comp rx=0 tx=0).stop 2024-09-16T01:45:01.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.709+0000 7fd4980a2640 1 -- 172.21.15.139:0/2539405154 shutdown_connections 2024-09-16T01:45:01.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.709+0000 7fd4980a2640 1 --2- 172.21.15.139:0/2539405154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd490109120 0x7fd490110ca0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:01.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.709+0000 7fd4980a2640 1 --2- 172.21.15.139:0/2539405154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4901049a0 0x7fd4901089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:01.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.709+0000 7fd4980a2640 1 --2- 172.21.15.139:0/2539405154 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd490103ff0 0x7fd4901043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:01.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.709+0000 7fd4980a2640 1 -- 172.21.15.139:0/2539405154 >> 172.21.15.139:0/2539405154 conn(0x7fd4900ffab0 msgr2=0x7fd490101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:01.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.709+0000 7fd4980a2640 1 -- 172.21.15.139:0/2539405154 shutdown_connections 2024-09-16T01:45:01.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.709+0000 7fd4980a2640 1 -- 172.21.15.139:0/2539405154 wait complete. 2024-09-16T01:45:01.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.710+0000 7fd4980a2640 1 Processor -- start 2024-09-16T01:45:01.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.710+0000 7fd4980a2640 1 -- start start 2024-09-16T01:45:01.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.710+0000 7fd4980a2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd490103ff0 0x7fd49019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:01.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd4980a2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4901049a0 0x7fd49019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:01.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd4980a2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd490109120 0x7fd49019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:01.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd4980a2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd4901141a0 con 0x7fd490103ff0 2024-09-16T01:45:01.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd4980a2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd490114020 con 0x7fd4901049a0 2024-09-16T01:45:01.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd4980a2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd490114320 con 0x7fd490109120 2024-09-16T01:45:01.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd495e3e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd490103ff0 0x7fd49019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:01.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd495e3e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd490103ff0 0x7fd49019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52724/0 (socket says 172.21.15.139:52724) 2024-09-16T01:45:01.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd495e3e640 1 -- 172.21.15.139:0/558859913 learned_addr learned my addr 172.21.15.139:0/558859913 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:01.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd49563d640 1 --2- 172.21.15.139:0/558859913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4901049a0 0x7fd49019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:01.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd49663f640 1 --2- 172.21.15.139:0/558859913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd490109120 0x7fd49019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:01.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd495e3e640 1 -- 172.21.15.139:0/558859913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd490109120 msgr2=0x7fd49019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:01.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd495e3e640 1 --2- 172.21.15.139:0/558859913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd490109120 0x7fd49019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:01.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd495e3e640 1 -- 172.21.15.139:0/558859913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4901049a0 msgr2=0x7fd49019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:01.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd495e3e640 1 --2- 172.21.15.139:0/558859913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4901049a0 0x7fd49019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:01.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd495e3e640 1 -- 172.21.15.139:0/558859913 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49019f590 con 0x7fd490103ff0 2024-09-16T01:45:01.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd495e3e640 1 --2- 172.21.15.139:0/558859913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd490103ff0 0x7fd49019a5e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00e9f0 tx=0x7fd48c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:01.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.711+0000 7fd486ffd640 1 -- 172.21.15.139:0/558859913 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c00cd80 con 0x7fd490103ff0 2024-09-16T01:45:01.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.712+0000 7fd4980a2640 1 -- 172.21.15.139:0/558859913 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49019f880 con 0x7fd490103ff0 2024-09-16T01:45:01.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.712+0000 7fd4980a2640 1 -- 172.21.15.139:0/558859913 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4901a7160 con 0x7fd490103ff0 2024-09-16T01:45:01.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.712+0000 7fd486ffd640 1 -- 172.21.15.139:0/558859913 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd48c005500 con 0x7fd490103ff0 2024-09-16T01:45:01.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.712+0000 7fd486ffd640 1 -- 172.21.15.139:0/558859913 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c010690 con 0x7fd490103ff0 2024-09-16T01:45:01.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.713+0000 7fd4980a2640 1 -- 172.21.15.139:0/558859913 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd474005290 con 0x7fd490103ff0 2024-09-16T01:45:01.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.714+0000 7fd486ffd640 1 -- 172.21.15.139:0/558859913 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd48c0046c0 con 0x7fd490103ff0 2024-09-16T01:45:01.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.715+0000 7fd486ffd640 1 --2- 172.21.15.139:0/558859913 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd47c0a3430 0x7fd47c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:01.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.715+0000 7fd49563d640 1 --2- 172.21.15.139:0/558859913 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd47c0a3430 0x7fd47c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:01.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.715+0000 7fd486ffd640 1 -- 172.21.15.139:0/558859913 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fd48c0d0410 con 0x7fd490103ff0 2024-09-16T01:45:01.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.715+0000 7fd49563d640 1 --2- 172.21.15.139:0/558859913 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd47c0a3430 0x7fd47c0a58f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd4900ffed0 tx=0x7fd480005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:01.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.720+0000 7fd486ffd640 1 -- 172.21.15.139:0/558859913 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd48c09ca40 con 0x7fd490103ff0 2024-09-16T01:45:01.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.991+0000 7fd4980a2640 1 -- 172.21.15.139:0/558859913 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd474005850 con 0x7fd490103ff0 2024-09-16T01:45:01.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.992+0000 7fd486ffd640 1 -- 172.21.15.139:0/558859913 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+12009 (secure 0 0 0) 0x7fd48c09c340 con 0x7fd490103ff0 2024-09-16T01:45:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:45:01.995 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":33,"btime":"2024-09-16T01:44:30:937847+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:44:30.781836+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:45:01.995 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 33 2024-09-16T01:45:01.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.995+0000 7fd4980a2640 1 -- 172.21.15.139:0/558859913 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd47c0a3430 msgr2=0x7fd47c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:01.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.995+0000 7fd4980a2640 1 --2- 172.21.15.139:0/558859913 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd47c0a3430 0x7fd47c0a58f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd4900ffed0 tx=0x7fd480005f90 comp rx=0 tx=0).stop 2024-09-16T01:45:01.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.995+0000 7fd4980a2640 1 -- 172.21.15.139:0/558859913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd490103ff0 msgr2=0x7fd49019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:01.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.995+0000 7fd4980a2640 1 --2- 172.21.15.139:0/558859913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd490103ff0 0x7fd49019a5e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00e9f0 tx=0x7fd48c00edf0 comp rx=0 tx=0).stop 2024-09-16T01:45:01.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.995+0000 7fd4980a2640 1 -- 172.21.15.139:0/558859913 shutdown_connections 2024-09-16T01:45:01.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.995+0000 7fd4980a2640 1 --2- 172.21.15.139:0/558859913 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd47c0a3430 0x7fd47c0a58f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:01.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.995+0000 7fd4980a2640 1 --2- 172.21.15.139:0/558859913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd490109120 0x7fd49019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:01.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.995+0000 7fd4980a2640 1 --2- 172.21.15.139:0/558859913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4901049a0 0x7fd49019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:01.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.995+0000 7fd4980a2640 1 --2- 172.21.15.139:0/558859913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd490103ff0 0x7fd49019a5e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:01.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.996+0000 7fd4980a2640 1 -- 172.21.15.139:0/558859913 >> 172.21.15.139:0/558859913 conn(0x7fd4900ffab0 msgr2=0x7fd490100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:01.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.996+0000 7fd4980a2640 1 -- 172.21.15.139:0/558859913 shutdown_connections 2024-09-16T01:45:01.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:01.996+0000 7fd4980a2640 1 -- 172.21.15.139:0/558859913 wait complete. 2024-09-16T01:45:02.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:02 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/558859913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:45:02.375 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:02 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/558859913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:45:02.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:02 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/558859913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:45:02.828 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id f152ec88 --release --await' 2024-09-16T01:45:02.828 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id f152ec88 --release --await 2024-09-16T01:45:03.139 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:45:03.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:03 smithi139 ceph-mon[31175]: pgmap v142: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 1.26k op/s 2024-09-16T01:45:03.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:03 smithi164 ceph-mon[35121]: pgmap v142: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 1.26k op/s 2024-09-16T01:45:03.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:03 smithi176 ceph-mon[35324]: pgmap v142: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 1.26k op/s 2024-09-16T01:45:03.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.670+0000 7f3dad3d7640 1 -- 172.21.15.139:0/2295741945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3da8100860 msgr2=0x7f3da81045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:03.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.670+0000 7f3dad3d7640 1 --2- 172.21.15.139:0/2295741945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3da8100860 0x7f3da81045a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009900 tx=0x7f3da002a410 comp rx=0 tx=0).stop 2024-09-16T01:45:03.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.670+0000 7f3dad3d7640 1 -- 172.21.15.139:0/2295741945 shutdown_connections 2024-09-16T01:45:03.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.670+0000 7f3dad3d7640 1 --2- 172.21.15.139:0/2295741945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3da8100860 0x7f3da81045a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.670+0000 7f3dad3d7640 1 --2- 172.21.15.139:0/2295741945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3da8107440 0x7f3da8100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.670+0000 7f3dad3d7640 1 --2- 172.21.15.139:0/2295741945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3da8106a90 0x7f3da8106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.670+0000 7f3dad3d7640 1 -- 172.21.15.139:0/2295741945 >> 172.21.15.139:0/2295741945 conn(0x7f3da8077230 msgr2=0x7f3da80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:03.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.670+0000 7f3dad3d7640 1 -- 172.21.15.139:0/2295741945 shutdown_connections 2024-09-16T01:45:03.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.670+0000 7f3dad3d7640 1 -- 172.21.15.139:0/2295741945 wait complete. 2024-09-16T01:45:03.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.671+0000 7f3dad3d7640 1 Processor -- start 2024-09-16T01:45:03.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.671+0000 7f3dad3d7640 1 -- start start 2024-09-16T01:45:03.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3dad3d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3da8100860 0x7f3da81a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3dad3d7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3da8106a90 0x7f3da81a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3dad3d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3da819e860 0x7f3da819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3dad3d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3da8118590 con 0x7f3da819e860 2024-09-16T01:45:03.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3dad3d7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3da8118410 con 0x7f3da8100860 2024-09-16T01:45:03.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3dad3d7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3da8118710 con 0x7f3da8106a90 2024-09-16T01:45:03.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3da77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3da819e860 0x7f3da819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3da77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3da819e860 0x7f3da819ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44968/0 (socket says 172.21.15.139:44968) 2024-09-16T01:45:03.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3da77fe640 1 -- 172.21.15.139:0/2063975282 learned_addr learned my addr 172.21.15.139:0/2063975282 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:03.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3d9e7fc640 1 --2- 172.21.15.139:0/2063975282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3da8106a90 0x7f3da81a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3d9e7fc640 1 -- 172.21.15.139:0/2063975282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3da8100860 msgr2=0x7f3da81a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:03.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3d9e7fc640 1 --2- 172.21.15.139:0/2063975282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3da8100860 0x7f3da81a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3d9e7fc640 1 -- 172.21.15.139:0/2063975282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3da819e860 msgr2=0x7f3da819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:03.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3d9e7fc640 1 --2- 172.21.15.139:0/2063975282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3da819e860 0x7f3da819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.672+0000 7f3d9e7fc640 1 -- 172.21.15.139:0/2063975282 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da819f380 con 0x7f3da8106a90 2024-09-16T01:45:03.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.673+0000 7f3d9e7fc640 1 --2- 172.21.15.139:0/2063975282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3da8106a90 0x7f3da81a4be0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000e9c0 tx=0x7f3d9000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:03.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.673+0000 7f3da4ff9640 1 -- 172.21.15.139:0/2063975282 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000cd80 con 0x7f3da8106a90 2024-09-16T01:45:03.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.673+0000 7f3da4ff9640 1 -- 172.21.15.139:0/2063975282 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d900035f0 con 0x7f3da8106a90 2024-09-16T01:45:03.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.673+0000 7f3dad3d7640 1 -- 172.21.15.139:0/2063975282 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da819f670 con 0x7f3da8106a90 2024-09-16T01:45:03.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.673+0000 7f3dad3d7640 1 -- 172.21.15.139:0/2063975282 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da81ab900 con 0x7f3da8106a90 2024-09-16T01:45:03.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.674+0000 7f3da4ff9640 1 -- 172.21.15.139:0/2063975282 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90010690 con 0x7f3da8106a90 2024-09-16T01:45:03.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.676+0000 7f3da4ff9640 1 -- 172.21.15.139:0/2063975282 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3d90003fa0 con 0x7f3da8106a90 2024-09-16T01:45:03.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.677+0000 7f3daf7b0740 1 Processor -- start 2024-09-16T01:45:03.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.677+0000 7f3da4ff9640 1 --2- 172.21.15.139:0/2063975282 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f3d940a3460 0x7f3d940a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.677+0000 7f3daf7b0740 1 -- start start 2024-09-16T01:45:03.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.677+0000 7f3da77fe640 1 --2- 172.21.15.139:0/2063975282 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f3d940a3460 0x7f3d940a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.677+0000 7f3da4ff9640 1 -- 172.21.15.139:0/2063975282 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f3d900d0260 con 0x7f3da8106a90 2024-09-16T01:45:03.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3daf7b0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265060 0x55faca265440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3daf7b0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265980 0x55faca26fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3daf7b0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca270240 0x55faca272630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3daf7b0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55faca1ca620 con 0x55faca265980 2024-09-16T01:45:03.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3daf7b0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55faca171e50 con 0x55faca265060 2024-09-16T01:45:03.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3daf7b0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55faca1c65e0 con 0x55faca270240 2024-09-16T01:45:03.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3da6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265060 0x55faca265440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3da6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265060 0x55faca265440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:57086/0 (socket says 172.21.15.139:57086) 2024-09-16T01:45:03.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3da6ffd640 1 -- 172.21.15.139:0/1961450321 learned_addr learned my addr 172.21.15.139:0/1961450321 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:03.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3da77fe640 1 --2- 172.21.15.139:0/1961450321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca270240 0x55faca272630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3d9e7fc640 1 --2- 172.21.15.139:0/1961450321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265980 0x55faca26fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3da77fe640 1 --2- 172.21.15.139:0/2063975282 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f3d940a3460 0x7f3d940a5920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3da00096c0 tx=0x7f3da0005d70 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:03.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3da6ffd640 1 -- 172.21.15.139:0/1961450321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca270240 msgr2=0x55faca272630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:03.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3da6ffd640 1 --2- 172.21.15.139:0/1961450321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca270240 0x55faca272630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3da6ffd640 1 -- 172.21.15.139:0/1961450321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265980 msgr2=0x55faca26fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:03.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3da6ffd640 1 --2- 172.21.15.139:0/1961450321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265980 0x55faca26fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3da6ffd640 1 -- 172.21.15.139:0/1961450321 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faca162d40 con 0x55faca265060 2024-09-16T01:45:03.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.678+0000 7f3d9e7fc640 1 --2- 172.21.15.139:0/1961450321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265980 0x55faca26fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:45:03.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.679+0000 7f3da6ffd640 1 --2- 172.21.15.139:0/1961450321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265060 0x55faca265440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009e70 tx=0x7f3d9800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52d179fba1fa50bf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:03.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.679+0000 7f3d9d7fa640 1 -- 172.21.15.139:0/1961450321 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98004d20 con 0x55faca265060 2024-09-16T01:45:03.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.679+0000 7f3d9d7fa640 1 -- 172.21.15.139:0/1961450321 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d98003e50 con 0x55faca265060 2024-09-16T01:45:03.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.679+0000 7f3d9d7fa640 1 -- 172.21.15.139:0/1961450321 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98005280 con 0x55faca265060 2024-09-16T01:45:03.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.680+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1961450321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265060 msgr2=0x55faca265440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:03.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.680+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/1961450321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265060 0x55faca265440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009e70 tx=0x7f3d9800e720 comp rx=0 tx=0).stop 2024-09-16T01:45:03.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.680+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1961450321 shutdown_connections 2024-09-16T01:45:03.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.680+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/1961450321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca270240 0x55faca272630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.680+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/1961450321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265980 0x55faca26fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.680+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/1961450321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265060 0x55faca265440 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.680+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1961450321 >> 172.21.15.139:0/1961450321 conn(0x55faca2615f0 msgr2=0x55faca2619d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:03.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.680+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1961450321 shutdown_connections 2024-09-16T01:45:03.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.680+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1961450321 wait complete. 2024-09-16T01:45:03.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.680+0000 7f3daf7b0740 1 -- start start 2024-09-16T01:45:03.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.680+0000 7f3daf7b0740 1 Processor -- start 2024-09-16T01:45:03.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3daf7b0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca265060 0x55faca26cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3daf7b0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265980 0x55faca26d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3daf7b0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca270240 0x55faca26da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3daf7b0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55faca272e10 con 0x55faca270240 2024-09-16T01:45:03.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3daf7b0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55faca152f10 con 0x55faca265980 2024-09-16T01:45:03.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3daf7b0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55faca272f90 con 0x55faca265060 2024-09-16T01:45:03.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3daf7b0740 10 client.0 _ll_register_callbacks cb 0x55faca176db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:45:03.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3da6ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca265060 0x55faca26cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3da6ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca265060 0x55faca26cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:32920/0 (socket says 172.21.15.139:32920) 2024-09-16T01:45:03.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3da6ffd640 1 -- 172.21.15.139:0/553075197 learned_addr learned my addr 172.21.15.139:0/553075197 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:03.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3d9e7fc640 1 --2- 172.21.15.139:0/553075197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265980 0x55faca26d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3da77fe640 1 --2- 172.21.15.139:0/553075197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca270240 0x55faca26da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3da6ffd640 1 -- 172.21.15.139:0/553075197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265980 msgr2=0x55faca26d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:03.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3da6ffd640 1 --2- 172.21.15.139:0/553075197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265980 0x55faca26d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3da6ffd640 1 -- 172.21.15.139:0/553075197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca270240 msgr2=0x55faca26da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:03.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3da6ffd640 1 --2- 172.21.15.139:0/553075197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca270240 0x55faca26da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3da6ffd640 1 -- 172.21.15.139:0/553075197 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac9ed9580 con 0x55faca265060 2024-09-16T01:45:03.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.681+0000 7f3d9e7fc640 1 --2- 172.21.15.139:0/553075197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265980 0x55faca26d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:45:03.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.682+0000 7f3da6ffd640 1 --2- 172.21.15.139:0/553075197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca265060 0x55faca26cff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d98004730 tx=0x7f3d98003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd9c6f13dc132e0f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:03.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.682+0000 7f3d8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:45:03.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.682+0000 7f3d8e7fc640 1 -- 172.21.15.139:0/553075197 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98009ac0 con 0x55faca265060 2024-09-16T01:45:03.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.682+0000 7f3daf7b0740 1 -- 172.21.15.139:0/553075197 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d98003e50 con 0x55faca265060 2024-09-16T01:45:03.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.682+0000 7f3d8e7fc640 1 -- 172.21.15.139:0/553075197 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f3d98018680 con 0x55faca265060 2024-09-16T01:45:03.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.682+0000 7f3d8e7fc640 1 -- 172.21.15.139:0/553075197 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3d98018880 con 0x55faca265060 2024-09-16T01:45:03.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.682+0000 7f3daf7b0740 10 client.25507 fetch_fsmap learned FSMap version 33 2024-09-16T01:45:03.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.683+0000 7f3daf7b0740 1 -- 172.21.15.139:0/553075197 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fac9fa3960 con 0x55faca265060 2024-09-16T01:45:03.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.683+0000 7f3d8e7fc640 1 -- 172.21.15.139:0/553075197 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d98003e50 con 0x55faca265060 2024-09-16T01:45:03.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.683+0000 7f3d8e7fc640 1 -- 172.21.15.139:0/553075197 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7f3d980109c0 con 0x55faca265060 2024-09-16T01:45:03.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.683+0000 7f3daf7b0740 10 client.25507 fetch_fsmap finished waiting for FSMap version 33 2024-09-16T01:45:03.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.683+0000 7f3daf7b0740 10 client.25507 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:45:03.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.683+0000 7f3daf7b0740 20 client.25507 populate_metadata read hostname 'smithi139' 2024-09-16T01:45:03.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.683+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/553075197 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55faca280190 0x55faca282650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.683+0000 7f3daf7b0740 4 client.25507 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:45:03.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.683+0000 7f3daf7b0740 1 -- 172.21.15.139:0/553075197 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faca1d7b90 con 0x55faca280190 2024-09-16T01:45:03.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.684+0000 7f3d9e7fc640 1 --2- 172.21.15.139:0/553075197 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55faca280190 0x55faca282650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.684+0000 7f3d8e7fc640 10 client.25507 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:03.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.684+0000 7f3d9e7fc640 1 --2- 172.21.15.139:0/553075197 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55faca280190 0x55faca282650 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7359bd0843b04d1 server_cookie=a939626fc7f83d80 in_seq=0 out_seq=0 2024-09-16T01:45:03.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.685+0000 7f3d8e7fc640 1 -- 172.21.15.139:0/553075197 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55faca1d7b90 con 0x55faca280190 2024-09-16T01:45:03.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.685+0000 7f3d8e7fc640 10 client.25507 handle_command_reply: tid=0 2024-09-16T01:45:03.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.685+0000 7f3daf7b0740 1 client.25507 shutdown 2024-09-16T01:45:03.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.685+0000 7f3daf7b0740 1 -- 172.21.15.139:0/553075197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca265060 msgr2=0x55faca26cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.685+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/553075197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca265060 0x55faca26cff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d98004730 tx=0x7f3d98003ba0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.685+0000 7f3daf7b0740 1 -- 172.21.15.139:0/553075197 shutdown_connections 2024-09-16T01:45:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.685+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/553075197 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55faca280190 0x55faca282650 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.685+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/553075197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca270240 0x55faca26da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.685+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/553075197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265980 0x55faca26d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.685+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/553075197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca265060 0x55faca26cff0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.685+0000 7f3daf7b0740 1 -- 172.21.15.139:0/553075197 >> 172.21.15.139:0/553075197 conn(0x55faca2615f0 msgr2=0x55faca260750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.685+0000 7f3d8e7fc640 0 client.25507 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.685+0000 7f3daf7b0740 1 -- 172.21.15.139:0/553075197 shutdown_connections 2024-09-16T01:45:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.685+0000 7f3daf7b0740 1 -- 172.21.15.139:0/553075197 wait complete. 2024-09-16T01:45:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.686+0000 7f3daf7b0740 20 client.25507 trim_cache size 0 max 16384 2024-09-16T01:45:03.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.709+0000 7f3daf7b0740 1 Processor -- start 2024-09-16T01:45:03.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.709+0000 7f3daf7b0740 1 -- start start 2024-09-16T01:45:03.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.709+0000 7f3daf7b0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265060 0x55faca26bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.709+0000 7f3daf7b0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca265980 0x55faca287f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.709+0000 7f3daf7b0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca270240 0x55faca288460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.709+0000 7f3daf7b0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8800ff70 con 0x55faca265060 2024-09-16T01:45:03.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.709+0000 7f3daf7b0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d88011b70 con 0x55faca270240 2024-09-16T01:45:03.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.709+0000 7f3daf7b0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3d88010640 con 0x55faca265980 2024-09-16T01:45:03.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.709+0000 7f3da6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265060 0x55faca26bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.709+0000 7f3d9e7fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca265980 0x55faca287f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.709+0000 7f3da77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca270240 0x55faca288460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.709+0000 7f3da6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265060 0x55faca26bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44988/0 (socket says 172.21.15.139:44988) 2024-09-16T01:45:03.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.710+0000 7f3da6ffd640 1 -- 172.21.15.139:0/1870916219 learned_addr learned my addr 172.21.15.139:0/1870916219 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:03.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.710+0000 7f3da6ffd640 1 -- 172.21.15.139:0/1870916219 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca265980 msgr2=0x55faca287f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:03.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.710+0000 7f3da6ffd640 1 --2- 172.21.15.139:0/1870916219 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca265980 0x55faca287f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.710+0000 7f3da6ffd640 1 -- 172.21.15.139:0/1870916219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca270240 msgr2=0x55faca288460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:03.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.710+0000 7f3da6ffd640 1 --2- 172.21.15.139:0/1870916219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca270240 0x55faca288460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.710+0000 7f3da6ffd640 1 -- 172.21.15.139:0/1870916219 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faca261420 con 0x55faca265060 2024-09-16T01:45:03.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.710+0000 7f3d9e7fc640 1 --2- 172.21.15.139:0/1870916219 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca265980 0x55faca287f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:45:03.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.710+0000 7f3da77fe640 1 --2- 172.21.15.139:0/1870916219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca270240 0x55faca288460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:45:03.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.710+0000 7f3da6ffd640 1 --2- 172.21.15.139:0/1870916219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265060 0x55faca26bea0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3d98006010 tx=0x7f3d98009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f35002504a359e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:03.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.711+0000 7f3d9cff9640 1 -- 172.21.15.139:0/1870916219 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98003c10 con 0x55faca265060 2024-09-16T01:45:03.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.711+0000 7f3d9cff9640 1 -- 172.21.15.139:0/1870916219 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d98009ac0 con 0x55faca265060 2024-09-16T01:45:03.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.711+0000 7f3d9cff9640 1 -- 172.21.15.139:0/1870916219 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98018e20 con 0x55faca265060 2024-09-16T01:45:03.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.711+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1870916219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265060 msgr2=0x55faca26bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:03.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.711+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/1870916219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265060 0x55faca26bea0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3d98006010 tx=0x7f3d98009650 comp rx=0 tx=0).stop 2024-09-16T01:45:03.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.711+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1870916219 shutdown_connections 2024-09-16T01:45:03.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.711+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/1870916219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca270240 0x55faca288460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.712+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/1870916219 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca265980 0x55faca287f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.712+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/1870916219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265060 0x55faca26bea0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.712+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1870916219 >> 172.21.15.139:0/1870916219 conn(0x55faca2615f0 msgr2=0x55faca26b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:03.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.712+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1870916219 shutdown_connections 2024-09-16T01:45:03.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.712+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1870916219 wait complete. 2024-09-16T01:45:03.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.712+0000 7f3daf7b0740 1 -- start start 2024-09-16T01:45:03.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.712+0000 7f3daf7b0740 1 Processor -- start 2024-09-16T01:45:03.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3daf7b0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265060 0x55faca273200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3daf7b0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265980 0x55faca273740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3daf7b0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca270240 0x55faca273c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3da6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265060 0x55faca273200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3daf7b0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d880113e0 con 0x55faca265060 2024-09-16T01:45:03.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3daf7b0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d88010d10 con 0x55faca265980 2024-09-16T01:45:03.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3daf7b0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8800df50 con 0x55faca270240 2024-09-16T01:45:03.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3da6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265060 0x55faca273200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45004/0 (socket says 172.21.15.139:45004) 2024-09-16T01:45:03.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3da6ffd640 1 -- 172.21.15.139:0/1983794746 learned_addr learned my addr 172.21.15.139:0/1983794746 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:03.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3daf7b0740 10 client.0 _ll_register_callbacks cb 0x7f3d980092f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:45:03.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3d9e7fc640 1 --2- 172.21.15.139:0/1983794746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265980 0x55faca273740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3da6ffd640 1 -- 172.21.15.139:0/1983794746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca270240 msgr2=0x55faca273c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:03.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3da6ffd640 1 --2- 172.21.15.139:0/1983794746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca270240 0x55faca273c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3da6ffd640 1 -- 172.21.15.139:0/1983794746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265980 msgr2=0x55faca273740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:03.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3da6ffd640 1 --2- 172.21.15.139:0/1983794746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265980 0x55faca273740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:03.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3da6ffd640 1 -- 172.21.15.139:0/1983794746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faca2e8ee0 con 0x55faca265060 2024-09-16T01:45:03.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3da6ffd640 1 --2- 172.21.15.139:0/1983794746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265060 0x55faca273200 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3d98018a60 tx=0x7f3d98004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fadd6c67d4b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:03.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.713+0000 7f3d8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:45:03.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.714+0000 7f3d8e7fc640 1 -- 172.21.15.139:0/1983794746 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98036070 con 0x55faca265060 2024-09-16T01:45:03.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.714+0000 7f3d8e7fc640 1 -- 172.21.15.139:0/1983794746 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f3d98010640 con 0x55faca265060 2024-09-16T01:45:03.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.714+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1983794746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d98009ac0 con 0x55faca265060 2024-09-16T01:45:03.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.714+0000 7f3d8e7fc640 1 -- 172.21.15.139:0/1983794746 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3d980043e0 con 0x55faca265060 2024-09-16T01:45:03.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.714+0000 7f3daf7b0740 10 client.16050 fetch_fsmap learned FSMap version 33 2024-09-16T01:45:03.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.714+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1983794746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55faca2ea1a0 con 0x55faca265060 2024-09-16T01:45:03.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.714+0000 7f3d8e7fc640 1 -- 172.21.15.139:0/1983794746 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d98009ac0 con 0x55faca265060 2024-09-16T01:45:03.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.714+0000 7f3d8e7fc640 1 -- 172.21.15.139:0/1983794746 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7f3d98030020 con 0x55faca265060 2024-09-16T01:45:03.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.714+0000 7f3daf7b0740 10 client.16050 fetch_fsmap finished waiting for FSMap version 33 2024-09-16T01:45:03.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.714+0000 7f3daf7b0740 10 client.16050 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:45:03.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.714+0000 7f3daf7b0740 20 client.16050 populate_metadata read hostname 'smithi139' 2024-09-16T01:45:03.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.715+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/1983794746 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55faca2ea7a0 0x55faca2ecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:03.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.715+0000 7f3daf7b0740 4 client.16050 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f152ec88", "release": true, "await": true}] 2024-09-16T01:45:03.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.715+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1983794746 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f152ec88", "release": true, "await": true}) -- 0x55faca1d20a0 con 0x55faca2ea7a0 2024-09-16T01:45:03.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.715+0000 7f3d9e7fc640 1 --2- 172.21.15.139:0/1983794746 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55faca2ea7a0 0x55faca2ecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:03.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.715+0000 7f3d8e7fc640 10 client.16050 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:03.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:03.715+0000 7f3d9e7fc640 1 --2- 172.21.15.139:0/1983794746 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55faca2ea7a0 0x55faca2ecbe0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcebddcb8eb0f412 server_cookie=f0386c464fbb286c in_seq=0 out_seq=0 2024-09-16T01:45:04.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.534+0000 7f3d8e7fc640 1 -- 172.21.15.139:0/1983794746 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55faca1d20a0 con 0x55faca2ea7a0 2024-09-16T01:45:04.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.534+0000 7f3d8e7fc640 10 client.16050 handle_command_reply: tid=0 2024-09-16T01:45:04.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.535+0000 7f3daf7b0740 1 client.16050 shutdown 2024-09-16T01:45:04.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.535+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1983794746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265060 msgr2=0x55faca273200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:04.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.535+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/1983794746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265060 0x55faca273200 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3d98018a60 tx=0x7f3d98004690 comp rx=0 tx=0).stop 2024-09-16T01:45:04.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.535+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1983794746 shutdown_connections 2024-09-16T01:45:04.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.535+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/1983794746 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55faca2ea7a0 0x55faca2ecbe0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:04.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.535+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/1983794746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55faca270240 0x55faca273c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:04.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.535+0000 7f3d8e7fc640 0 client.16050 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:04.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.535+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/1983794746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faca265980 0x55faca273740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:04.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.535+0000 7f3daf7b0740 1 --2- 172.21.15.139:0/1983794746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55faca265060 0x55faca273200 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:04.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.535+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1983794746 >> 172.21.15.139:0/1983794746 conn(0x55faca2615f0 msgr2=0x55faca260750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:04.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.535+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1983794746 shutdown_connections 2024-09-16T01:45:04.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.535+0000 7f3daf7b0740 1 -- 172.21.15.139:0/1983794746 wait complete. 2024-09-16T01:45:04.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.535+0000 7f3daf7b0740 20 client.16050 trim_cache size 0 max 16384 2024-09-16T01:45:04.538 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:45:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 33, 2024-09-16T01:45:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:45:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 16, 2024-09-16T01:45:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:45:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "f152ec88": { 2024-09-16T01:45:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 16, 2024-09-16T01:45:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:45:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:45:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:45:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:45:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:45:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:45:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 133.8, 2024-09-16T01:45:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:45:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:45:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:45:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:45:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:45:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:45:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:45:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:45:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:45:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:45:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:45:04.540 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:45:04.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.536+0000 7f3dad3d7640 1 -- 172.21.15.139:0/2063975282 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f3d940a3460 msgr2=0x7f3d940a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:04.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.536+0000 7f3dad3d7640 1 --2- 172.21.15.139:0/2063975282 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f3d940a3460 0x7f3d940a5920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3da00096c0 tx=0x7f3da0005d70 comp rx=0 tx=0).stop 2024-09-16T01:45:04.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.536+0000 7f3dad3d7640 1 -- 172.21.15.139:0/2063975282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3da8106a90 msgr2=0x7f3da81a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:04.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.536+0000 7f3dad3d7640 1 --2- 172.21.15.139:0/2063975282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3da8106a90 0x7f3da81a4be0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000e9c0 tx=0x7f3d9000ee80 comp rx=0 tx=0).stop 2024-09-16T01:45:04.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.536+0000 7f3dad3d7640 1 -- 172.21.15.139:0/2063975282 shutdown_connections 2024-09-16T01:45:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.536+0000 7f3dad3d7640 1 --2- 172.21.15.139:0/2063975282 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f3d940a3460 0x7f3d940a5920 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.536+0000 7f3dad3d7640 1 --2- 172.21.15.139:0/2063975282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3da819e860 0x7f3da819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.536+0000 7f3dad3d7640 1 --2- 172.21.15.139:0/2063975282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3da8106a90 0x7f3da81a4be0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.536+0000 7f3dad3d7640 1 --2- 172.21.15.139:0/2063975282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3da8100860 0x7f3da81a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.536+0000 7f3dad3d7640 1 -- 172.21.15.139:0/2063975282 >> 172.21.15.139:0/2063975282 conn(0x7f3da8077230 msgr2=0x7f3da81026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.536+0000 7f3dad3d7640 1 -- 172.21.15.139:0/2063975282 shutdown_connections 2024-09-16T01:45:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:04.536+0000 7f3dad3d7640 1 -- 172.21.15.139:0/2063975282 wait complete. 2024-09-16T01:45:05.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:05 smithi164 ceph-mon[35121]: pgmap v143: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.6 MiB/s wr, 598 op/s 2024-09-16T01:45:05.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:05 smithi139 ceph-mon[31175]: pgmap v143: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.6 MiB/s wr, 598 op/s 2024-09-16T01:45:05.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:05 smithi176 ceph-mon[35324]: pgmap v143: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.6 MiB/s wr, 598 op/s 2024-09-16T01:45:05.524 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f152ec88', total seconds elapsed: 31.7 2024-09-16T01:45:05.968 INFO:tasks.workunit.client.0.smithi139.stdout: 369 27869 121 20367 156 4757 143 2024-09-16T01:45:07.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:07 smithi139 ceph-mon[31175]: pgmap v144: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.1 MiB/s wr, 815 op/s 2024-09-16T01:45:07.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:07 smithi176 ceph-mon[35324]: pgmap v144: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.1 MiB/s wr, 815 op/s 2024-09-16T01:45:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:07 smithi164 ceph-mon[35121]: pgmap v144: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.1 MiB/s wr, 815 op/s 2024-09-16T01:45:09.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:09 smithi139 ceph-mon[31175]: pgmap v145: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 5.2 MiB/s wr, 684 op/s 2024-09-16T01:45:09.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:09 smithi176 ceph-mon[35324]: pgmap v145: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 5.2 MiB/s wr, 684 op/s 2024-09-16T01:45:09.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:09 smithi164 ceph-mon[35121]: pgmap v145: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 5.2 MiB/s wr, 684 op/s 2024-09-16T01:45:11.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:11 smithi139 ceph-mon[31175]: pgmap v146: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 4.5 MiB/s wr, 724 op/s 2024-09-16T01:45:11.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:11 smithi176 ceph-mon[35324]: pgmap v146: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 4.5 MiB/s wr, 724 op/s 2024-09-16T01:45:11.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:11 smithi164 ceph-mon[35121]: pgmap v146: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 4.5 MiB/s wr, 724 op/s 2024-09-16T01:45:13.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:13 smithi139 ceph-mon[31175]: pgmap v147: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 4.2 MiB/s wr, 993 op/s 2024-09-16T01:45:13.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:13 smithi176 ceph-mon[35324]: pgmap v147: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 4.2 MiB/s wr, 993 op/s 2024-09-16T01:45:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.445+0000 7fb29f8af640 1 -- 172.21.15.139:0/3003451504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb298101370 msgr2=0x7fb29810d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.445+0000 7fb29f8af640 1 --2- 172.21.15.139:0/3003451504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb298101370 0x7fb29810d8a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb290009c10 tx=0x7fb29002a6e0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.445+0000 7fb29f8af640 1 -- 172.21.15.139:0/3003451504 shutdown_connections 2024-09-16T01:45:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.445+0000 7fb29f8af640 1 --2- 172.21.15.139:0/3003451504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb298101370 0x7fb29810d8a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.445+0000 7fb29f8af640 1 --2- 172.21.15.139:0/3003451504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2981009d0 0x7fb298100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.445+0000 7fb29f8af640 1 --2- 172.21.15.139:0/3003451504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2981069e0 0x7fb298106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.445+0000 7fb29f8af640 1 -- 172.21.15.139:0/3003451504 >> 172.21.15.139:0/3003451504 conn(0x7fb2980fc6b0 msgr2=0x7fb2980fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.445+0000 7fb29f8af640 1 -- 172.21.15.139:0/3003451504 shutdown_connections 2024-09-16T01:45:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.445+0000 7fb29f8af640 1 -- 172.21.15.139:0/3003451504 wait complete. 2024-09-16T01:45:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29f8af640 1 Processor -- start 2024-09-16T01:45:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29f8af640 1 -- start start 2024-09-16T01:45:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29f8af640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2981009d0 0x7fb298198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29f8af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb298101370 0x7fb298198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29f8af640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2981069e0 0x7fb29819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29f8af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb298110dd0 con 0x7fb298101370 2024-09-16T01:45:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29f8af640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb298110c50 con 0x7fb2981009d0 2024-09-16T01:45:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29f8af640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb298110f50 con 0x7fb2981069e0 2024-09-16T01:45:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29ce4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb298101370 0x7fb298198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29d64b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2981009d0 0x7fb298198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29de4c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2981069e0 0x7fb29819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29ce4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb298101370 0x7fb298198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53726/0 (socket says 172.21.15.139:53726) 2024-09-16T01:45:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29ce4a640 1 -- 172.21.15.139:0/2764653224 learned_addr learned my addr 172.21.15.139:0/2764653224 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29de4c640 1 -- 172.21.15.139:0/2764653224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2981009d0 msgr2=0x7fb298198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29de4c640 1 --2- 172.21.15.139:0/2764653224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2981009d0 0x7fb298198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29de4c640 1 -- 172.21.15.139:0/2764653224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb298101370 msgr2=0x7fb298198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29de4c640 1 --2- 172.21.15.139:0/2764653224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb298101370 0x7fb298198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29de4c640 1 -- 172.21.15.139:0/2764653224 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29819d300 con 0x7fb2981069e0 2024-09-16T01:45:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.446+0000 7fb29ce4a640 1 --2- 172.21.15.139:0/2764653224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb298101370 0x7fb298198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:45:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.447+0000 7fb29de4c640 1 --2- 172.21.15.139:0/2764653224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2981069e0 0x7fb29819cbd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb290034040 tx=0x7fb2900305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.447+0000 7fb28e7fc640 1 -- 172.21.15.139:0/2764653224 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290041020 con 0x7fb2981069e0 2024-09-16T01:45:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.447+0000 7fb29f8af640 1 -- 172.21.15.139:0/2764653224 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29819d590 con 0x7fb2981069e0 2024-09-16T01:45:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.447+0000 7fb28e7fc640 1 -- 172.21.15.139:0/2764653224 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb290037040 con 0x7fb2981069e0 2024-09-16T01:45:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.447+0000 7fb28e7fc640 1 -- 172.21.15.139:0/2764653224 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290036070 con 0x7fb2981069e0 2024-09-16T01:45:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.447+0000 7fb29f8af640 1 -- 172.21.15.139:0/2764653224 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2981a4e80 con 0x7fb2981069e0 2024-09-16T01:45:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.450+0000 7fb2a2642740 1 Processor -- start 2024-09-16T01:45:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.450+0000 7fb28e7fc640 1 -- 172.21.15.139:0/2764653224 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb290044020 con 0x7fb2981069e0 2024-09-16T01:45:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.450+0000 7fb2a2642740 1 -- start start 2024-09-16T01:45:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.450+0000 7fb2a2642740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97aaf0 0x56032c97aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.450+0000 7fb2a2642740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c985770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.450+0000 7fb2a2642740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c9880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.450+0000 7fb2a2642740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56032c8df2f0 con 0x56032c985cb0 2024-09-16T01:45:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.450+0000 7fb2a2642740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56032c887270 con 0x56032c97aaf0 2024-09-16T01:45:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.450+0000 7fb2a2642740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56032c8db2b0 con 0x56032c97b410 2024-09-16T01:45:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb29ce4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c9880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb29d64b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c985770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb29d64b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c985770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:33932/0 (socket says 172.21.15.139:33932) 2024-09-16T01:45:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb29ce4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c9880a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53728/0 (socket says 172.21.15.139:53728) 2024-09-16T01:45:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb29d64b640 1 -- 172.21.15.139:0/2359375569 learned_addr learned my addr 172.21.15.139:0/2359375569 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb29ce4a640 1 --2- 172.21.15.139:0/2359375569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97aaf0 0x56032c97aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb28e7fc640 1 --2- 172.21.15.139:0/2764653224 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb2840a3460 0x7fb2840a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb29ce4a640 1 -- 172.21.15.139:0/2359375569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 msgr2=0x56032c985770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb29ce4a640 1 --2- 172.21.15.139:0/2359375569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c985770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb29ce4a640 1 -- 172.21.15.139:0/2359375569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97aaf0 msgr2=0x56032c97aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb29ce4a640 1 --2- 172.21.15.139:0/2359375569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97aaf0 0x56032c97aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb29ce4a640 1 -- 172.21.15.139:0/2359375569 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56032c886340 con 0x56032c985cb0 2024-09-16T01:45:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb29de4c640 1 --2- 172.21.15.139:0/2764653224 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb2840a3460 0x7fb2840a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb29d64b640 1 --2- 172.21.15.139:0/2359375569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c985770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:45:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb28e7fc640 1 -- 172.21.15.139:0/2764653224 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fb2900f0b30 con 0x7fb2981069e0 2024-09-16T01:45:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb29ce4a640 1 --2- 172.21.15.139:0/2359375569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c9880a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb298105060 tx=0x7fb288009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f268d9a3bd69c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb2837fe640 1 -- 172.21.15.139:0/2359375569 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28800fc50 con 0x56032c985cb0 2024-09-16T01:45:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb2837fe640 1 -- 172.21.15.139:0/2359375569 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb28800fdf0 con 0x56032c985cb0 2024-09-16T01:45:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.451+0000 7fb2837fe640 1 -- 172.21.15.139:0/2359375569 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28801e3f0 con 0x56032c985cb0 2024-09-16T01:45:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.452+0000 7fb2a2642740 1 -- 172.21.15.139:0/2359375569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 msgr2=0x56032c9880a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.452+0000 7fb29de4c640 1 --2- 172.21.15.139:0/2764653224 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb2840a3460 0x7fb2840a5920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb2900ee4d0 tx=0x7fb29008e7e0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.452+0000 7fb2a2642740 1 --2- 172.21.15.139:0/2359375569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c9880a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb298105060 tx=0x7fb288009590 comp rx=0 tx=0).stop 2024-09-16T01:45:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.452+0000 7fb2a2642740 1 -- 172.21.15.139:0/2359375569 shutdown_connections 2024-09-16T01:45:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.452+0000 7fb2a2642740 1 --2- 172.21.15.139:0/2359375569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c9880a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.452+0000 7fb2a2642740 1 --2- 172.21.15.139:0/2359375569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c985770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.452+0000 7fb2a2642740 1 --2- 172.21.15.139:0/2359375569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97aaf0 0x56032c97aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.452+0000 7fb2a2642740 1 -- 172.21.15.139:0/2359375569 >> 172.21.15.139:0/2359375569 conn(0x56032c977060 msgr2=0x56032c977440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.452+0000 7fb2a2642740 1 -- 172.21.15.139:0/2359375569 shutdown_connections 2024-09-16T01:45:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.452+0000 7fb2a2642740 1 -- 172.21.15.139:0/2359375569 wait complete. 2024-09-16T01:45:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.453+0000 7fb2a2642740 1 -- start start 2024-09-16T01:45:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.453+0000 7fb2a2642740 1 Processor -- start 2024-09-16T01:45:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.453+0000 7fb2a2642740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c97aaf0 0x56032c982c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.453+0000 7fb2a2642740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c983180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.453+0000 7fb2a2642740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c985cb0 0x56032c9836c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.453+0000 7fb2a2642740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56032c988880 con 0x56032c97aaf0 2024-09-16T01:45:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.453+0000 7fb2a2642740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56032c868290 con 0x56032c985cb0 2024-09-16T01:45:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.453+0000 7fb2a2642740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56032c988a00 con 0x56032c97b410 2024-09-16T01:45:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.453+0000 7fb2a2642740 10 client.0 _ll_register_callbacks cb 0x56032c899140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:45:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.453+0000 7fb29ce4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c97aaf0 0x56032c982c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.453+0000 7fb29ce4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c97aaf0 0x56032c982c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53732/0 (socket says 172.21.15.139:53732) 2024-09-16T01:45:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.453+0000 7fb29ce4a640 1 -- 172.21.15.139:0/1415739377 learned_addr learned my addr 172.21.15.139:0/1415739377 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.453+0000 7fb29d64b640 1 --2- 172.21.15.139:0/1415739377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c983180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.453+0000 7fb29ce4a640 1 --2- 172.21.15.139:0/1415739377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c985cb0 0x56032c9836c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.454+0000 7fb29ce4a640 1 -- 172.21.15.139:0/1415739377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 msgr2=0x56032c983180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.454+0000 7fb29ce4a640 1 --2- 172.21.15.139:0/1415739377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c983180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.454+0000 7fb29ce4a640 1 -- 172.21.15.139:0/1415739377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c985cb0 msgr2=0x56032c9836c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.454+0000 7fb29ce4a640 1 --2- 172.21.15.139:0/1415739377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c985cb0 0x56032c9836c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.454+0000 7fb29ce4a640 1 -- 172.21.15.139:0/1415739377 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56032c5f2210 con 0x56032c97aaf0 2024-09-16T01:45:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.454+0000 7fb29ce4a640 1 --2- 172.21.15.139:0/1415739377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c97aaf0 0x56032c982c40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb288004b00 tx=0x7fb288005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca5d99ac64ba741 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.454+0000 7fb280ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:45:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.454+0000 7fb280ff9640 1 -- 172.21.15.139:0/1415739377 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288022070 con 0x56032c97aaf0 2024-09-16T01:45:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.454+0000 7fb280ff9640 1 -- 172.21.15.139:0/1415739377 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fb2880046c0 con 0x56032c97aaf0 2024-09-16T01:45:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.454+0000 7fb2a2642740 1 -- 172.21.15.139:0/1415739377 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb28800fdf0 con 0x56032c97aaf0 2024-09-16T01:45:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.454+0000 7fb280ff9640 1 -- 172.21.15.139:0/1415739377 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb288033050 con 0x56032c97aaf0 2024-09-16T01:45:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.454+0000 7fb2a2642740 10 client.16062 fetch_fsmap learned FSMap version 33 2024-09-16T01:45:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.454+0000 7fb2a2642740 1 -- 172.21.15.139:0/1415739377 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56032c6be620 con 0x56032c97aaf0 2024-09-16T01:45:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.454+0000 7fb280ff9640 1 -- 172.21.15.139:0/1415739377 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb28800fdf0 con 0x56032c97aaf0 2024-09-16T01:45:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.455+0000 7fb280ff9640 1 -- 172.21.15.139:0/1415739377 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7fb28802b020 con 0x56032c97aaf0 2024-09-16T01:45:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.455+0000 7fb2a2642740 10 client.16062 fetch_fsmap finished waiting for FSMap version 33 2024-09-16T01:45:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.455+0000 7fb2a2642740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:45:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.455+0000 7fb2a2642740 20 client.16062 populate_metadata read hostname 'smithi139' 2024-09-16T01:45:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.455+0000 7fb2a2642740 1 --2- 172.21.15.139:0/1415739377 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56032c995b50 0x56032c997f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.455+0000 7fb2a2642740 4 client.16062 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:45:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.455+0000 7fb2a2642740 1 -- 172.21.15.139:0/1415739377 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56032c8eca20 con 0x56032c995b50 2024-09-16T01:45:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.455+0000 7fb29d64b640 1 --2- 172.21.15.139:0/1415739377 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56032c995b50 0x56032c997f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.455+0000 7fb29d64b640 1 --2- 172.21.15.139:0/1415739377 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56032c995b50 0x56032c997f40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5bff333a2f2e1f server_cookie=686de2b0eac5cd87 in_seq=0 out_seq=0 2024-09-16T01:45:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.455+0000 7fb280ff9640 10 client.16062 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb280ff9640 1 -- 172.21.15.139:0/1415739377 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56032c8eca20 con 0x56032c995b50 2024-09-16T01:45:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb280ff9640 10 client.16062 handle_command_reply: tid=0 2024-09-16T01:45:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb2a2642740 1 client.16062 shutdown 2024-09-16T01:45:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb2a2642740 1 -- 172.21.15.139:0/1415739377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c97aaf0 msgr2=0x56032c982c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb2a2642740 1 --2- 172.21.15.139:0/1415739377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c97aaf0 0x56032c982c40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb288004b00 tx=0x7fb288005f90 comp rx=0 tx=0).stop 2024-09-16T01:45:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb2a2642740 1 -- 172.21.15.139:0/1415739377 shutdown_connections 2024-09-16T01:45:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb2a2642740 1 --2- 172.21.15.139:0/1415739377 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56032c995b50 0x56032c997f40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb2a2642740 1 --2- 172.21.15.139:0/1415739377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c985cb0 0x56032c9836c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb2a2642740 1 --2- 172.21.15.139:0/1415739377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c983180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb2a2642740 1 --2- 172.21.15.139:0/1415739377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c97aaf0 0x56032c982c40 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb280ff9640 0 client.16062 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb2a2642740 1 -- 172.21.15.139:0/1415739377 >> 172.21.15.139:0/1415739377 conn(0x56032c977060 msgr2=0x56032c97cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb2a2642740 1 -- 172.21.15.139:0/1415739377 shutdown_connections 2024-09-16T01:45:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb2a2642740 1 -- 172.21.15.139:0/1415739377 wait complete. 2024-09-16T01:45:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.461+0000 7fb2a2642740 20 client.16062 trim_cache size 0 max 16384 2024-09-16T01:45:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.484+0000 7fb2a2642740 1 Processor -- start 2024-09-16T01:45:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.484+0000 7fb2a2642740 1 -- start start 2024-09-16T01:45:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.484+0000 7fb2a2642740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97aaf0 0x56032c9844a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.484+0000 7fb2a2642740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97b410 0x56032c9834f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.484+0000 7fb2a2642740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c99d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.484+0000 7fb2a2642740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981aefe0 con 0x56032c985cb0 2024-09-16T01:45:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.484+0000 7fb2a2642740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981b0be0 con 0x56032c97b410 2024-09-16T01:45:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.484+0000 7fb2a2642740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981af6b0 con 0x56032c97aaf0 2024-09-16T01:45:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.485+0000 7fb29ce4a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97aaf0 0x56032c9844a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.485+0000 7fb29d64b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97b410 0x56032c9834f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.485+0000 7fb29ce4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c99d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.485+0000 7fb29d64b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97b410 0x56032c9834f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:59210/0 (socket says 172.21.15.139:59210) 2024-09-16T01:45:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.485+0000 7fb29d64b640 1 -- 172.21.15.139:0/3320498053 learned_addr learned my addr 172.21.15.139:0/3320498053 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.485+0000 7fb29ce4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c99d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53746/0 (socket says 172.21.15.139:53746) 2024-09-16T01:45:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.485+0000 7fb29d64b640 1 -- 172.21.15.139:0/3320498053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97aaf0 msgr2=0x56032c9844a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.485+0000 7fb29d64b640 1 --2- 172.21.15.139:0/3320498053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97aaf0 0x56032c9844a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.485+0000 7fb29d64b640 1 -- 172.21.15.139:0/3320498053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 msgr2=0x56032c99d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.485+0000 7fb29d64b640 1 --2- 172.21.15.139:0/3320498053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c99d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.485+0000 7fb29d64b640 1 -- 172.21.15.139:0/3320498053 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56032c97b240 con 0x56032c97b410 2024-09-16T01:45:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.485+0000 7fb29d64b640 1 --2- 172.21.15.139:0/3320498053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97b410 0x56032c9834f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x56032c977ec0 tx=0x7fb2940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8b1fe4ef8c2873 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.486+0000 7fb282ffd640 1 -- 172.21.15.139:0/3320498053 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294021020 con 0x56032c97b410 2024-09-16T01:45:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.486+0000 7fb282ffd640 1 -- 172.21.15.139:0/3320498053 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb294003e20 con 0x56032c97b410 2024-09-16T01:45:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.486+0000 7fb282ffd640 1 -- 172.21.15.139:0/3320498053 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294010930 con 0x56032c97b410 2024-09-16T01:45:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.487+0000 7fb2a2642740 1 -- 172.21.15.139:0/3320498053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97b410 msgr2=0x56032c9834f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.487+0000 7fb2a2642740 1 --2- 172.21.15.139:0/3320498053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97b410 0x56032c9834f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x56032c977ec0 tx=0x7fb2940046c0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.487+0000 7fb2a2642740 1 -- 172.21.15.139:0/3320498053 shutdown_connections 2024-09-16T01:45:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.487+0000 7fb2a2642740 1 --2- 172.21.15.139:0/3320498053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c99d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.487+0000 7fb2a2642740 1 --2- 172.21.15.139:0/3320498053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97b410 0x56032c9834f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.487+0000 7fb2a2642740 1 --2- 172.21.15.139:0/3320498053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97aaf0 0x56032c9844a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.487+0000 7fb2a2642740 1 -- 172.21.15.139:0/3320498053 >> 172.21.15.139:0/3320498053 conn(0x56032c977060 msgr2=0x56032c97cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.487+0000 7fb2a2642740 1 -- 172.21.15.139:0/3320498053 shutdown_connections 2024-09-16T01:45:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.487+0000 7fb2a2642740 1 -- 172.21.15.139:0/3320498053 wait complete. 2024-09-16T01:45:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.487+0000 7fb2a2642740 1 -- start start 2024-09-16T01:45:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.488+0000 7fb2a2642740 1 Processor -- start 2024-09-16T01:45:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.488+0000 7fb2a2642740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97aaf0 0x56032c98e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.488+0000 7fb2a2642740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c989880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.488+0000 7fb2a2642740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c98a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.488+0000 7fb2a2642740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981b0450 con 0x56032c985cb0 2024-09-16T01:45:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.488+0000 7fb2a2642740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981afd80 con 0x56032c97aaf0 2024-09-16T01:45:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.488+0000 7fb2a2642740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981acfc0 con 0x56032c97b410 2024-09-16T01:45:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.488+0000 7fb2a2642740 10 client.0 _ll_register_callbacks cb 0x7fb288011730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:45:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.489+0000 7fb29ce4a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97aaf0 0x56032c98e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.489+0000 7fb29d64b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c989880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.489+0000 7fb29d64b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c989880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:33968/0 (socket says 172.21.15.139:33968) 2024-09-16T01:45:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.489+0000 7fb29d64b640 1 -- 172.21.15.139:0/3974918544 learned_addr learned my addr 172.21.15.139:0/3974918544 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.489+0000 7fb29ce4a640 1 --2- 172.21.15.139:0/3974918544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c98a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.489+0000 7fb29d64b640 1 -- 172.21.15.139:0/3974918544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97aaf0 msgr2=0x56032c98e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.489+0000 7fb29d64b640 1 --2- 172.21.15.139:0/3974918544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97aaf0 0x56032c98e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.489+0000 7fb29d64b640 1 -- 172.21.15.139:0/3974918544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 msgr2=0x56032c98a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.489+0000 7fb29d64b640 1 --2- 172.21.15.139:0/3974918544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c98a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.489+0000 7fb29d64b640 1 -- 172.21.15.139:0/3974918544 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56032ca07fc0 con 0x56032c97b410 2024-09-16T01:45:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.489+0000 7fb29d64b640 1 --2- 172.21.15.139:0/3974918544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c989880 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb294005c50 tx=0x7fb294009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8c17033a45879dc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.489+0000 7fb280ff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:45:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.489+0000 7fb280ff9640 1 -- 172.21.15.139:0/3974918544 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294021040 con 0x56032c97b410 2024-09-16T01:45:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.490+0000 7fb280ff9640 1 -- 172.21.15.139:0/3974918544 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fb294010980 con 0x56032c97b410 2024-09-16T01:45:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.490+0000 7fb2a2642740 1 -- 172.21.15.139:0/3974918544 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb294003e20 con 0x56032c97b410 2024-09-16T01:45:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.490+0000 7fb280ff9640 1 -- 172.21.15.139:0/3974918544 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb294038050 con 0x56032c97b410 2024-09-16T01:45:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.490+0000 7fb2a2642740 10 client.25528 fetch_fsmap learned FSMap version 33 2024-09-16T01:45:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.490+0000 7fb2a2642740 1 -- 172.21.15.139:0/3974918544 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56032ca09120 con 0x56032c97b410 2024-09-16T01:45:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.490+0000 7fb280ff9640 1 -- 172.21.15.139:0/3974918544 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb294003e20 con 0x56032c97b410 2024-09-16T01:45:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.490+0000 7fb280ff9640 1 -- 172.21.15.139:0/3974918544 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 33) ==== 8402+0+0 (secure 0 0 0) 0x7fb294032510 con 0x56032c97b410 2024-09-16T01:45:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.490+0000 7fb2a2642740 10 client.25528 fetch_fsmap finished waiting for FSMap version 33 2024-09-16T01:45:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.490+0000 7fb2a2642740 10 client.25528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:45:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.490+0000 7fb2a2642740 20 client.25528 populate_metadata read hostname 'smithi139' 2024-09-16T01:45:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.490+0000 7fb2a2642740 1 --2- 172.21.15.139:0/3974918544 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56032ca09880 0x56032ca0bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.490+0000 7fb2a2642740 4 client.25528 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:45:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.490+0000 7fb2a2642740 1 -- 172.21.15.139:0/3974918544 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56032c8e6d70 con 0x56032ca09880 2024-09-16T01:45:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.491+0000 7fb29ce4a640 1 --2- 172.21.15.139:0/3974918544 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56032ca09880 0x56032ca0bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.491+0000 7fb280ff9640 10 client.25528 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:13.491+0000 7fb29ce4a640 1 --2- 172.21.15.139:0/3974918544 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56032ca09880 0x56032ca0bca0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39277cb0ba395c17 server_cookie=ded9dfd7e889b184 in_seq=0 out_seq=0 2024-09-16T01:45:13.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:13 smithi164 ceph-mon[35121]: pgmap v147: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 4.2 MiB/s wr, 993 op/s 2024-09-16T01:45:15.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:15 smithi139 ceph-mon[31175]: pgmap v148: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.9 MiB/s wr, 922 op/s 2024-09-16T01:45:15.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:15 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:45:15.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:15 smithi176 ceph-mon[35324]: pgmap v148: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.9 MiB/s wr, 922 op/s 2024-09-16T01:45:15.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:15 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:45:15.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:15 smithi164 ceph-mon[35121]: pgmap v148: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.9 MiB/s wr, 922 op/s 2024-09-16T01:45:15.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:15 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:45:15.968 INFO:tasks.workunit.client.0.smithi139.stdout: 393 1137539 1386 791083 1213 217743 1593 2024-09-16T01:45:17.430 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:17 smithi139 ceph-mon[31175]: pgmap v149: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 4.0 MiB/s wr, 983 op/s 2024-09-16T01:45:17.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:17 smithi176 ceph-mon[35324]: pgmap v149: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 4.0 MiB/s wr, 983 op/s 2024-09-16T01:45:17.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:17 smithi164 ceph-mon[35121]: pgmap v149: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 4.0 MiB/s wr, 983 op/s 2024-09-16T01:45:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.853+0000 7fb280ff9640 1 -- 172.21.15.139:0/3974918544 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56032c8e6d70 con 0x56032ca09880 2024-09-16T01:45:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.853+0000 7fb280ff9640 10 client.25528 handle_command_reply: tid=0 2024-09-16T01:45:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.853+0000 7fb2a2642740 1 client.25528 shutdown 2024-09-16T01:45:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.854+0000 7fb2a2642740 1 -- 172.21.15.139:0/3974918544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 msgr2=0x56032c989880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.854+0000 7fb2a2642740 1 --2- 172.21.15.139:0/3974918544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c989880 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb294005c50 tx=0x7fb294009f80 comp rx=0 tx=0).stop 2024-09-16T01:45:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.854+0000 7fb2a2642740 1 -- 172.21.15.139:0/3974918544 shutdown_connections 2024-09-16T01:45:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.854+0000 7fb2a2642740 1 --2- 172.21.15.139:0/3974918544 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56032ca09880 0x56032ca0bca0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.854+0000 7fb2a2642740 1 --2- 172.21.15.139:0/3974918544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56032c985cb0 0x56032c98a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.854+0000 7fb2a2642740 1 --2- 172.21.15.139:0/3974918544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032c97b410 0x56032c989880 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.854+0000 7fb2a2642740 1 --2- 172.21.15.139:0/3974918544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56032c97aaf0 0x56032c98e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.854+0000 7fb2a2642740 1 -- 172.21.15.139:0/3974918544 >> 172.21.15.139:0/3974918544 conn(0x56032c977060 msgr2=0x56032c9759a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.854+0000 7fb280ff9640 0 client.25528 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.854+0000 7fb2a2642740 1 -- 172.21.15.139:0/3974918544 shutdown_connections 2024-09-16T01:45:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.854+0000 7fb2a2642740 1 -- 172.21.15.139:0/3974918544 wait complete. 2024-09-16T01:45:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.855+0000 7fb2a2642740 20 client.25528 trim_cache size 0 max 16384 2024-09-16T01:45:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:45:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:45:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:45:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:45:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.890+0000 7fb29f8af640 1 -- 172.21.15.139:0/2764653224 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb2840a3460 msgr2=0x7fb2840a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.890+0000 7fb29f8af640 1 --2- 172.21.15.139:0/2764653224 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb2840a3460 0x7fb2840a5920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb2900ee4d0 tx=0x7fb29008e7e0 comp rx=0 tx=0).stop 2024-09-16T01:45:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.890+0000 7fb29f8af640 1 -- 172.21.15.139:0/2764653224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2981069e0 msgr2=0x7fb29819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.890+0000 7fb29f8af640 1 --2- 172.21.15.139:0/2764653224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2981069e0 0x7fb29819cbd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb290034040 tx=0x7fb2900305d0 comp rx=0 tx=0).stop 2024-09-16T01:45:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.894+0000 7fb29f8af640 1 -- 172.21.15.139:0/2764653224 shutdown_connections 2024-09-16T01:45:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.894+0000 7fb29f8af640 1 --2- 172.21.15.139:0/2764653224 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb2840a3460 0x7fb2840a5920 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.894+0000 7fb29f8af640 1 --2- 172.21.15.139:0/2764653224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2981069e0 0x7fb29819cbd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.894+0000 7fb29f8af640 1 --2- 172.21.15.139:0/2764653224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb298101370 0x7fb298198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.894+0000 7fb29f8af640 1 --2- 172.21.15.139:0/2764653224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2981009d0 0x7fb298198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.894+0000 7fb29f8af640 1 -- 172.21.15.139:0/2764653224 >> 172.21.15.139:0/2764653224 conn(0x7fb2980fc6b0 msgr2=0x7fb29806a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.894+0000 7fb29f8af640 1 -- 172.21.15.139:0/2764653224 shutdown_connections 2024-09-16T01:45:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:18.895+0000 7fb29f8af640 1 -- 172.21.15.139:0/2764653224 wait complete. 2024-09-16T01:45:19.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:19 smithi139 ceph-mon[31175]: pgmap v150: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.7 MiB/s wr, 895 op/s 2024-09-16T01:45:19.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:19 smithi176 ceph-mon[35324]: pgmap v150: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.7 MiB/s wr, 895 op/s 2024-09-16T01:45:19.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:19 smithi164 ceph-mon[35121]: pgmap v150: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.7 MiB/s wr, 895 op/s 2024-09-16T01:45:20.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:20 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:45:20.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:20 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:45:20.676 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:20 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:45:21.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:21 smithi164 ceph-mon[35121]: pgmap v151: 129 pgs: 129 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.8 MiB/s wr, 826 op/s 2024-09-16T01:45:21.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:21 smithi139 ceph-mon[31175]: pgmap v151: 129 pgs: 129 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.8 MiB/s wr, 826 op/s 2024-09-16T01:45:21.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:21 smithi176 ceph-mon[35324]: pgmap v151: 129 pgs: 129 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.8 MiB/s wr, 826 op/s 2024-09-16T01:45:23.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:23 smithi164 ceph-mon[35121]: pgmap v152: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 11 MiB/s wr, 964 op/s 2024-09-16T01:45:23.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:23 smithi139 ceph-mon[31175]: pgmap v152: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 11 MiB/s wr, 964 op/s 2024-09-16T01:45:23.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:23 smithi176 ceph-mon[35324]: pgmap v152: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 11 MiB/s wr, 964 op/s 2024-09-16T01:45:25.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:25 smithi164 ceph-mon[35121]: pgmap v153: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 758 op/s 2024-09-16T01:45:25.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:25 smithi139 ceph-mon[31175]: pgmap v153: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 758 op/s 2024-09-16T01:45:25.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:25 smithi176 ceph-mon[35324]: pgmap v153: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 758 op/s 2024-09-16T01:45:25.968 INFO:tasks.workunit.client.0.smithi139.stdout: 427 1979045 1660 1368486 1529 410225 1679 2024-09-16T01:45:27.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:27 smithi164 ceph-mon[35121]: pgmap v154: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 12 MiB/s wr, 784 op/s 2024-09-16T01:45:27.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:27 smithi139 ceph-mon[31175]: pgmap v154: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 12 MiB/s wr, 784 op/s 2024-09-16T01:45:27.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:27 smithi176 ceph-mon[35324]: pgmap v154: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 12 MiB/s wr, 784 op/s 2024-09-16T01:45:29.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:29 smithi164 ceph-mon[35121]: pgmap v155: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 11 MiB/s wr, 661 op/s 2024-09-16T01:45:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:29 smithi139 ceph-mon[31175]: pgmap v155: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 11 MiB/s wr, 661 op/s 2024-09-16T01:45:29.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:29 smithi176 ceph-mon[35324]: pgmap v155: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 11 MiB/s wr, 661 op/s 2024-09-16T01:45:31.153 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T01:45:31.154 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-09-16T01:45:31.154 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:45:31.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:31 smithi164 ceph-mon[35121]: pgmap v156: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.7 MiB/s wr, 525 op/s 2024-09-16T01:45:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:31 smithi139 ceph-mon[31175]: pgmap v156: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.7 MiB/s wr, 525 op/s 2024-09-16T01:45:31.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:31 smithi176 ceph-mon[35324]: pgmap v156: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.7 MiB/s wr, 525 op/s 2024-09-16T01:45:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.856+0000 7fcfc5082640 1 -- 172.21.15.139:0/158759152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfc01091f0 msgr2=0x7fcfc0110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.856+0000 7fcfc5082640 1 --2- 172.21.15.139:0/158759152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfc01091f0 0x7fcfc0110250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009930 tx=0x7fcfb002a480 comp rx=0 tx=0).stop 2024-09-16T01:45:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.856+0000 7fcfc5082640 1 -- 172.21.15.139:0/158759152 shutdown_connections 2024-09-16T01:45:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.857+0000 7fcfc5082640 1 --2- 172.21.15.139:0/158759152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfc011b140 0x7fcfc011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.857+0000 7fcfc5082640 1 --2- 172.21.15.139:0/158759152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfc01095d0 0x7fcfc0109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.857+0000 7fcfc5082640 1 --2- 172.21.15.139:0/158759152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfc01091f0 0x7fcfc0110250 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.857+0000 7fcfc5082640 1 -- 172.21.15.139:0/158759152 >> 172.21.15.139:0/158759152 conn(0x7fcfc006ce00 msgr2=0x7fcfc006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.900+0000 7fcfc5082640 1 -- 172.21.15.139:0/158759152 shutdown_connections 2024-09-16T01:45:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.900+0000 7fcfc5082640 1 -- 172.21.15.139:0/158759152 wait complete. 2024-09-16T01:45:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.901+0000 7fcfc5082640 1 Processor -- start 2024-09-16T01:45:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.901+0000 7fcfc5082640 1 -- start start 2024-09-16T01:45:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.902+0000 7fcfc5082640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfc01095d0 0x7fcfc0118390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.902+0000 7fcfc5082640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfc011b140 0x7fcfc01188d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.902+0000 7fcfc5082640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfc0111410 0x7fcfc01118c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.902+0000 7fcfc5082640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0120db0 con 0x7fcfc0111410 2024-09-16T01:45:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.902+0000 7fcfc5082640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0120c30 con 0x7fcfc01095d0 2024-09-16T01:45:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.902+0000 7fcfc5082640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0120f30 con 0x7fcfc011b140 2024-09-16T01:45:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.905+0000 7fcfbf59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfc0111410 0x7fcfc01118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.905+0000 7fcfbf59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfc0111410 0x7fcfc01118c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49266/0 (socket says 172.21.15.139:49266) 2024-09-16T01:45:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.905+0000 7fcfbf59e640 1 -- 172.21.15.139:0/1331215977 learned_addr learned my addr 172.21.15.139:0/1331215977 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.905+0000 7fcfbf59e640 1 -- 172.21.15.139:0/1331215977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfc011b140 msgr2=0x7fcfc01188d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:45:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.905+0000 7fcfbf59e640 1 --2- 172.21.15.139:0/1331215977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfc011b140 0x7fcfc01188d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.905+0000 7fcfbf59e640 1 -- 172.21.15.139:0/1331215977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfc01095d0 msgr2=0x7fcfc0118390 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:45:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.905+0000 7fcfbf59e640 1 --2- 172.21.15.139:0/1331215977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfc01095d0 0x7fcfc0118390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.905+0000 7fcfbf59e640 1 -- 172.21.15.139:0/1331215977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc0112040 con 0x7fcfc0111410 2024-09-16T01:45:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.906+0000 7fcfbf59e640 1 --2- 172.21.15.139:0/1331215977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfc0111410 0x7fcfc01118c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800b970 tx=0x7fcfb800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.916+0000 7fcfa7fff640 1 -- 172.21.15.139:0/1331215977 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8019070 con 0x7fcfc0111410 2024-09-16T01:45:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.916+0000 7fcfc5082640 1 -- 172.21.15.139:0/1331215977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc0115eb0 con 0x7fcfc0111410 2024-09-16T01:45:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.916+0000 7fcfc5082640 1 -- 172.21.15.139:0/1331215977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc0116400 con 0x7fcfc0111410 2024-09-16T01:45:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.920+0000 7fcfc7e21740 1 Processor -- start 2024-09-16T01:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.921+0000 7fcfc7e21740 1 -- start start 2024-09-16T01:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.921+0000 7fcfc7e21740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd7af0 0x56371ccd7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.921+0000 7fcfc7e21740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371ccd8410 0x56371cce2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.921+0000 7fcfc7e21740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371cce2cb0 0x56371cce50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.921+0000 7fcfc7e21740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56371cc3c2f0 con 0x56371ccd7af0 2024-09-16T01:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.921+0000 7fcfc7e21740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56371cbe4270 con 0x56371ccd8410 2024-09-16T01:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.921+0000 7fcfc7e21740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56371cc382b0 con 0x56371cce2cb0 2024-09-16T01:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.921+0000 7fcfa7fff640 1 -- 172.21.15.139:0/1331215977 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb800cb40 con 0x7fcfc0111410 2024-09-16T01:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.921+0000 7fcfa7fff640 1 -- 172.21.15.139:0/1331215977 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80076a0 con 0x7fcfc0111410 2024-09-16T01:45:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.926+0000 7fcfbe59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371cce2cb0 0x56371cce50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.926+0000 7fcfbe59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371cce2cb0 0x56371cce50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:58240/0 (socket says 172.21.15.139:58240) 2024-09-16T01:45:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.926+0000 7fcfbe59c640 1 -- 172.21.15.139:0/3576884281 learned_addr learned my addr 172.21.15.139:0/3576884281 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.926+0000 7fcfbed9d640 1 --2- 172.21.15.139:0/3576884281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371ccd8410 0x56371cce2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.927+0000 7fcfbe59c640 1 -- 172.21.15.139:0/3576884281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371ccd8410 msgr2=0x56371cce2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.927+0000 7fcfbe59c640 1 --2- 172.21.15.139:0/3576884281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371ccd8410 0x56371cce2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.927+0000 7fcfbe59c640 1 -- 172.21.15.139:0/3576884281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd7af0 msgr2=0x56371ccd7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.927+0000 7fcfbe59c640 1 --2- 172.21.15.139:0/3576884281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd7af0 0x56371ccd7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.927+0000 7fcfbe59c640 1 -- 172.21.15.139:0/3576884281 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56371cbe3340 con 0x56371cce2cb0 2024-09-16T01:45:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.928+0000 7fcfbe59c640 1 --2- 172.21.15.139:0/3576884281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371cce2cb0 0x56371cce50a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400b880 tx=0x7fcfb400bd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c64016541a0ee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.929+0000 7fcfa7fff640 1 -- 172.21.15.139:0/1331215977 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fcfb801c070 con 0x7fcfc0111410 2024-09-16T01:45:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.930+0000 7fcfa7fff640 1 --2- 172.21.15.139:0/1331215977 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcfa80a3680 0x7fcfa80a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.930+0000 7fcfbed9d640 1 --2- 172.21.15.139:0/1331215977 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcfa80a3680 0x7fcfa80a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.931+0000 7fcfbed9d640 1 --2- 172.21.15.139:0/1331215977 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcfa80a3680 0x7fcfa80a5b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009850 tx=0x7fcfb0005850 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.931+0000 7fcfa7fff640 1 -- 172.21.15.139:0/1331215977 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fcfb80cf4f0 con 0x7fcfc0111410 2024-09-16T01:45:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.933+0000 7fcfa57fa640 1 -- 172.21.15.139:0/3576884281 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4018070 con 0x56371cce2cb0 2024-09-16T01:45:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.934+0000 7fcfa57fa640 1 -- 172.21.15.139:0/3576884281 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb400ddb0 con 0x56371cce2cb0 2024-09-16T01:45:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.936+0000 7fcfc7e21740 1 -- 172.21.15.139:0/3576884281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371cce2cb0 msgr2=0x56371cce50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.936+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/3576884281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371cce2cb0 0x56371cce50a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400b880 tx=0x7fcfb400bd40 comp rx=0 tx=0).stop 2024-09-16T01:45:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.936+0000 7fcfc7e21740 1 -- 172.21.15.139:0/3576884281 shutdown_connections 2024-09-16T01:45:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.936+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/3576884281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371cce2cb0 0x56371cce50a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.936+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/3576884281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371ccd8410 0x56371cce2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.936+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/3576884281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd7af0 0x56371ccd7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.936+0000 7fcfc7e21740 1 -- 172.21.15.139:0/3576884281 >> 172.21.15.139:0/3576884281 conn(0x56371ccd4060 msgr2=0x56371ccd4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.945+0000 7fcfc7e21740 1 -- 172.21.15.139:0/3576884281 shutdown_connections 2024-09-16T01:45:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.947+0000 7fcfc7e21740 1 -- 172.21.15.139:0/3576884281 wait complete. 2024-09-16T01:45:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.947+0000 7fcfc7e21740 1 -- start start 2024-09-16T01:45:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.947+0000 7fcfc7e21740 1 Processor -- start 2024-09-16T01:45:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.948+0000 7fcfc7e21740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371ccd7af0 0x56371ccdfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.948+0000 7fcfc7e21740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd8410 0x56371cce0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.948+0000 7fcfc7e21740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371cce2cb0 0x56371cce0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.948+0000 7fcfc7e21740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56371cce58d0 con 0x56371ccd8410 2024-09-16T01:45:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.948+0000 7fcfc7e21740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56371cbc5290 con 0x56371cce2cb0 2024-09-16T01:45:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.948+0000 7fcfc7e21740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56371cce5a50 con 0x56371ccd7af0 2024-09-16T01:45:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.948+0000 7fcfc7e21740 10 client.0 _ll_register_callbacks cb 0x56371cbf6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:45:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.948+0000 7fcfbe59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd8410 0x56371cce0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.948+0000 7fcfbe59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd8410 0x56371cce0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49276/0 (socket says 172.21.15.139:49276) 2024-09-16T01:45:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.948+0000 7fcfbe59c640 1 -- 172.21.15.139:0/433818635 learned_addr learned my addr 172.21.15.139:0/433818635 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.949+0000 7fcfbe59c640 1 -- 172.21.15.139:0/433818635 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371ccd7af0 msgr2=0x56371ccdfc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:45:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.949+0000 7fcfbe59c640 1 --2- 172.21.15.139:0/433818635 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371ccd7af0 0x56371ccdfc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.949+0000 7fcfbe59c640 1 -- 172.21.15.139:0/433818635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371cce2cb0 msgr2=0x56371cce0680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:45:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.949+0000 7fcfbe59c640 1 --2- 172.21.15.139:0/433818635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371cce2cb0 0x56371cce0680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.949+0000 7fcfbe59c640 1 -- 172.21.15.139:0/433818635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56371c94f210 con 0x56371ccd8410 2024-09-16T01:45:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.956+0000 7fcfbe59c640 1 --2- 172.21.15.139:0/433818635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd8410 0x56371cce0140 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4004700 tx=0x7fcfb4008d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8507da0878d9b3f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.960+0000 7fcfa2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:45:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.973+0000 7fcfa2ff5640 1 -- 172.21.15.139:0/433818635 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4018040 con 0x56371ccd8410 2024-09-16T01:45:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.973+0000 7fcfc7e21740 1 -- 172.21.15.139:0/433818635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb400ddb0 con 0x56371ccd8410 2024-09-16T01:45:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.973+0000 7fcfa2ff5640 1 -- 172.21.15.139:0/433818635 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fcfb4004c80 con 0x56371ccd8410 2024-09-16T01:45:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.976+0000 7fcfa2ff5640 1 -- 172.21.15.139:0/433818635 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcfb400b4d0 con 0x56371ccd8410 2024-09-16T01:45:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.977+0000 7fcfc7e21740 10 client.16074 fetch_fsmap learned FSMap version 35 2024-09-16T01:45:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.977+0000 7fcfc7e21740 1 -- 172.21.15.139:0/433818635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56371ca1b620 con 0x56371ccd8410 2024-09-16T01:45:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.987+0000 7fcfa2ff5640 1 -- 172.21.15.139:0/433818635 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb400ddb0 con 0x56371ccd8410 2024-09-16T01:45:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.987+0000 7fcfa2ff5640 1 -- 172.21.15.139:0/433818635 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7fcfb4024030 con 0x56371ccd8410 2024-09-16T01:45:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.988+0000 7fcfc7e21740 10 client.16074 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:45:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.988+0000 7fcfc7e21740 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:45:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.988+0000 7fcfc7e21740 20 client.16074 populate_metadata read hostname 'smithi139' 2024-09-16T01:45:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.988+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/433818635 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56371ccf2cf0 0x56371ccf5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.988+0000 7fcfc7e21740 4 client.16074 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:45:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.988+0000 7fcfc7e21740 1 -- 172.21.15.139:0/433818635 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56371cc49a20 con 0x56371ccf2cf0 2024-09-16T01:45:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:32.988+0000 7fcfbed9d640 1 --2- 172.21.15.139:0/433818635 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56371ccf2cf0 0x56371ccf5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.000+0000 7fcfa2ff5640 10 client.16074 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.000+0000 7fcfbed9d640 1 --2- 172.21.15.139:0/433818635 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56371ccf2cf0 0x56371ccf5110 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce601164d0af1a60 server_cookie=2db1f70cc6dd8148 in_seq=0 out_seq=0 2024-09-16T01:45:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.005+0000 7fcfa2ff5640 1 -- 172.21.15.139:0/433818635 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56371cc49a20 con 0x56371ccf2cf0 2024-09-16T01:45:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.005+0000 7fcfa2ff5640 10 client.16074 handle_command_reply: tid=0 2024-09-16T01:45:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.005+0000 7fcfc7e21740 1 client.16074 shutdown 2024-09-16T01:45:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.005+0000 7fcfc7e21740 1 -- 172.21.15.139:0/433818635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd8410 msgr2=0x56371cce0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.005+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/433818635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd8410 0x56371cce0140 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4004700 tx=0x7fcfb4008d50 comp rx=0 tx=0).stop 2024-09-16T01:45:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.011+0000 7fcfc7e21740 1 -- 172.21.15.139:0/433818635 shutdown_connections 2024-09-16T01:45:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.011+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/433818635 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56371ccf2cf0 0x56371ccf5110 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.011+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/433818635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371cce2cb0 0x56371cce0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.011+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/433818635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd8410 0x56371cce0140 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.011+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/433818635 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371ccd7af0 0x56371ccdfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.011+0000 7fcfa2ff5640 0 client.16074 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.011+0000 7fcfc7e21740 1 -- 172.21.15.139:0/433818635 >> 172.21.15.139:0/433818635 conn(0x56371ccd4060 msgr2=0x56371ccd9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.013+0000 7fcfc7e21740 1 -- 172.21.15.139:0/433818635 shutdown_connections 2024-09-16T01:45:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.013+0000 7fcfc7e21740 1 -- 172.21.15.139:0/433818635 wait complete. 2024-09-16T01:45:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.013+0000 7fcfc7e21740 20 client.16074 trim_cache size 0 max 16384 2024-09-16T01:45:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.069+0000 7fcfc7e21740 1 Processor -- start 2024-09-16T01:45:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.069+0000 7fcfc7e21740 1 -- start start 2024-09-16T01:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.070+0000 7fcfc7e21740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371ccd7af0 0x56371ccda590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.070+0000 7fcfc7e21740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371ccd8410 0x56371cce04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.070+0000 7fcfc7e21740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371cce2cb0 0x56371ccfaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.070+0000 7fcfc7e21740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac010080 con 0x56371cce2cb0 2024-09-16T01:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.070+0000 7fcfbe59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371ccd8410 0x56371cce04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.070+0000 7fcfc7e21740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac011d40 con 0x56371ccd8410 2024-09-16T01:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.070+0000 7fcfc7e21740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac010750 con 0x56371ccd7af0 2024-09-16T01:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.070+0000 7fcfbe59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371ccd8410 0x56371cce04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56302/0 (socket says 172.21.15.139:56302) 2024-09-16T01:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.070+0000 7fcfbe59c640 1 -- 172.21.15.139:0/2018103311 learned_addr learned my addr 172.21.15.139:0/2018103311 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.071+0000 7fcfbed9d640 1 --2- 172.21.15.139:0/2018103311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371ccd7af0 0x56371ccda590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.071+0000 7fcfbe59c640 1 -- 172.21.15.139:0/2018103311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371ccd7af0 msgr2=0x56371ccda590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.071+0000 7fcfbe59c640 1 --2- 172.21.15.139:0/2018103311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371ccd7af0 0x56371ccda590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.071+0000 7fcfbe59c640 1 -- 172.21.15.139:0/2018103311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371cce2cb0 msgr2=0x56371ccfaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.071+0000 7fcfbe59c640 1 --2- 172.21.15.139:0/2018103311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371cce2cb0 0x56371ccfaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.071+0000 7fcfbe59c640 1 -- 172.21.15.139:0/2018103311 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56371ccd8240 con 0x56371ccd8410 2024-09-16T01:45:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.077+0000 7fcfbe59c640 1 --2- 172.21.15.139:0/2018103311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371ccd8410 0x56371cce04b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4003f10 tx=0x7fcfb400b2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a73968cd373beafa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.081+0000 7fcfa4ff9640 1 -- 172.21.15.139:0/2018103311 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb402d070 con 0x56371ccd8410 2024-09-16T01:45:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.081+0000 7fcfa4ff9640 1 -- 172.21.15.139:0/2018103311 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb40035f0 con 0x56371ccd8410 2024-09-16T01:45:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.082+0000 7fcfc7e21740 1 -- 172.21.15.139:0/2018103311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371ccd8410 msgr2=0x56371cce04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.082+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/2018103311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371ccd8410 0x56371cce04b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4003f10 tx=0x7fcfb400b2a0 comp rx=0 tx=0).stop 2024-09-16T01:45:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.083+0000 7fcfc7e21740 1 -- 172.21.15.139:0/2018103311 shutdown_connections 2024-09-16T01:45:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.083+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/2018103311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371cce2cb0 0x56371ccfaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.083+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/2018103311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371ccd8410 0x56371cce04b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.083+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/2018103311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371ccd7af0 0x56371ccda590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.083+0000 7fcfc7e21740 1 -- 172.21.15.139:0/2018103311 >> 172.21.15.139:0/2018103311 conn(0x56371ccd4060 msgr2=0x56371cce1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.087+0000 7fcfc7e21740 1 -- 172.21.15.139:0/2018103311 shutdown_connections 2024-09-16T01:45:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.089+0000 7fcfc7e21740 1 -- 172.21.15.139:0/2018103311 wait complete. 2024-09-16T01:45:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.089+0000 7fcfc7e21740 1 -- start start 2024-09-16T01:45:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.089+0000 7fcfc7e21740 1 Processor -- start 2024-09-16T01:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfc7e21740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371ccd7af0 0x56371cceb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfc7e21740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd8410 0x56371cce6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfc7e21740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371cce2cb0 0x56371cce6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfc7e21740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac0115b0 con 0x56371ccd8410 2024-09-16T01:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfc7e21740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac010ee0 con 0x56371cce2cb0 2024-09-16T01:45:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfc7e21740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac00e060 con 0x56371ccd7af0 2024-09-16T01:45:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfc7e21740 10 client.0 _ll_register_callbacks cb 0x7fcfb4016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:45:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfbf59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371cce2cb0 0x56371cce6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfbf59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371cce2cb0 0x56371cce6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56308/0 (socket says 172.21.15.139:56308) 2024-09-16T01:45:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfbf59e640 1 -- 172.21.15.139:0/42536716 learned_addr learned my addr 172.21.15.139:0/42536716 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfbf59e640 1 -- 172.21.15.139:0/42536716 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371ccd7af0 msgr2=0x56371cceb400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfbf59e640 1 --2- 172.21.15.139:0/42536716 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371ccd7af0 0x56371cceb400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfbf59e640 1 -- 172.21.15.139:0/42536716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd8410 msgr2=0x56371cce6720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfbf59e640 1 --2- 172.21.15.139:0/42536716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd8410 0x56371cce6720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.090+0000 7fcfbf59e640 1 -- 172.21.15.139:0/42536716 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56371cce7350 con 0x56371cce2cb0 2024-09-16T01:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.091+0000 7fcfbf59e640 1 --2- 172.21.15.139:0/42536716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371cce2cb0 0x56371cce6c60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8014e90 tx=0x7fcfb809cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1301263094e030aa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.091+0000 7fcfa2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.092+0000 7fcfa2ff5640 1 -- 172.21.15.139:0/42536716 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80a5280 con 0x56371cce2cb0 2024-09-16T01:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.092+0000 7fcfc7e21740 1 -- 172.21.15.139:0/42536716 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb40035f0 con 0x56371cce2cb0 2024-09-16T01:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.092+0000 7fcfa2ff5640 1 -- 172.21.15.139:0/42536716 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fcfb80ccab0 con 0x56371cce2cb0 2024-09-16T01:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.092+0000 7fcfa2ff5640 1 -- 172.21.15.139:0/42536716 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcfb80a4df0 con 0x56371cce2cb0 2024-09-16T01:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.093+0000 7fcfc7e21740 10 client.25556 fetch_fsmap learned FSMap version 35 2024-09-16T01:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.093+0000 7fcfc7e21740 1 -- 172.21.15.139:0/42536716 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56371cd5d8c0 con 0x56371cce2cb0 2024-09-16T01:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.094+0000 7fcfa2ff5640 1 -- 172.21.15.139:0/42536716 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb40035f0 con 0x56371cce2cb0 2024-09-16T01:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.094+0000 7fcfa2ff5640 1 -- 172.21.15.139:0/42536716 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7fcfb80b0cc0 con 0x56371cce2cb0 2024-09-16T01:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.096+0000 7fcfc7e21740 10 client.25556 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.096+0000 7fcfc7e21740 10 client.25556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.096+0000 7fcfc7e21740 20 client.25556 populate_metadata read hostname 'smithi139' 2024-09-16T01:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.096+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/42536716 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56371cd5e060 0x56371cd60480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.096+0000 7fcfc7e21740 4 client.25556 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.096+0000 7fcfc7e21740 1 -- 172.21.15.139:0/42536716 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56371cc43d70 con 0x56371cd5e060 2024-09-16T01:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.098+0000 7fcfbed9d640 1 --2- 172.21.15.139:0/42536716 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56371cd5e060 0x56371cd60480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.099+0000 7fcfa2ff5640 10 client.25556 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:33.099+0000 7fcfbed9d640 1 --2- 172.21.15.139:0/42536716 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56371cd5e060 0x56371cd60480 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b78defd8b0c62933 server_cookie=e3e7e6368bda2b2a in_seq=0 out_seq=0 2024-09-16T01:45:33.290 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:45:33.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:33 smithi164 ceph-mon[35121]: pgmap v157: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.0 MiB/s wr, 470 op/s 2024-09-16T01:45:33.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:33 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:45:33.637 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:33 smithi139 ceph-mon[31175]: pgmap v157: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.0 MiB/s wr, 470 op/s 2024-09-16T01:45:33.638 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:33 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:45:33.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:33 smithi176 ceph-mon[35324]: pgmap v157: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.0 MiB/s wr, 470 op/s 2024-09-16T01:45:33.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:33 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:45:34.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.948+0000 7fd55a9bc640 1 -- 172.21.15.139:0/298493962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd54c0952d0 msgr2=0x7fd54c0a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:34.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.948+0000 7fd55a9bc640 1 --2- 172.21.15.139:0/298493962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd54c0952d0 0x7fd54c0a1320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd554051500 tx=0x7fd554096d90 comp rx=0 tx=0).stop 2024-09-16T01:45:34.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.948+0000 7fd55a9bc640 1 -- 172.21.15.139:0/298493962 shutdown_connections 2024-09-16T01:45:34.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.948+0000 7fd55a9bc640 1 --2- 172.21.15.139:0/298493962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd54c0952d0 0x7fd54c0a1320 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:34.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.948+0000 7fd55a9bc640 1 --2- 172.21.15.139:0/298493962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd54c094930 0x7fd54c094d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:34.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.948+0000 7fd55a9bc640 1 --2- 172.21.15.139:0/298493962 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd54c09a550 0x7fd54c09a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:34.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.948+0000 7fd55a9bc640 1 -- 172.21.15.139:0/298493962 >> 172.21.15.139:0/298493962 conn(0x7fd54c00a1f0 msgr2=0x7fd54c00a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:34.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.949+0000 7fd55a9bc640 1 -- 172.21.15.139:0/298493962 shutdown_connections 2024-09-16T01:45:34.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.949+0000 7fd55a9bc640 1 -- 172.21.15.139:0/298493962 wait complete. 2024-09-16T01:45:34.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.951+0000 7fd55a9bc640 1 Processor -- start 2024-09-16T01:45:34.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.951+0000 7fd55a9bc640 1 -- start start 2024-09-16T01:45:34.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.951+0000 7fd55a9bc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd54c094930 0x7fd54c13cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:34.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.951+0000 7fd55a9bc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd54c0952d0 0x7fd54c13d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:34.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.951+0000 7fd55a9bc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd54c09a550 0x7fd54c141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:34.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.951+0000 7fd55a9bc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c0a47b0 con 0x7fd54c0952d0 2024-09-16T01:45:34.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.951+0000 7fd55a9bc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c0a4630 con 0x7fd54c09a550 2024-09-16T01:45:34.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.951+0000 7fd55a9bc640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c0a4930 con 0x7fd54c094930 2024-09-16T01:45:34.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.951+0000 7fd55a1bb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd54c09a550 0x7fd54c141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:34.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.952+0000 7fd55a1bb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd54c09a550 0x7fd54c141640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56316/0 (socket says 172.21.15.139:56316) 2024-09-16T01:45:34.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.952+0000 7fd55a1bb640 1 -- 172.21.15.139:0/3761977948 learned_addr learned my addr 172.21.15.139:0/3761977948 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:34.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.952+0000 7fd5599ba640 1 --2- 172.21.15.139:0/3761977948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd54c094930 0x7fd54c13cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:34.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.952+0000 7fd5599ba640 1 -- 172.21.15.139:0/3761977948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd54c09a550 msgr2=0x7fd54c141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:34.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.952+0000 7fd5599ba640 1 --2- 172.21.15.139:0/3761977948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd54c09a550 0x7fd54c141640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:34.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.952+0000 7fd5599ba640 1 -- 172.21.15.139:0/3761977948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd54c0952d0 msgr2=0x7fd54c13d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:34.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.952+0000 7fd5599ba640 1 --2- 172.21.15.139:0/3761977948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd54c0952d0 0x7fd54c13d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:34.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.952+0000 7fd5599ba640 1 -- 172.21.15.139:0/3761977948 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c141d20 con 0x7fd54c094930 2024-09-16T01:45:34.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.955+0000 7fd5599ba640 1 --2- 172.21.15.139:0/3761977948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd54c094930 0x7fd54c13cd70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd55000e9f0 tx=0x7fd55000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:34.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.957+0000 7fd54affd640 1 -- 172.21.15.139:0/3761977948 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55000ccc0 con 0x7fd54c094930 2024-09-16T01:45:34.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.957+0000 7fd54affd640 1 -- 172.21.15.139:0/3761977948 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd550010910 con 0x7fd54c094930 2024-09-16T01:45:34.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.957+0000 7fd55a9bc640 1 -- 172.21.15.139:0/3761977948 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54c142010 con 0x7fd54c094930 2024-09-16T01:45:34.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.957+0000 7fd55a9bc640 1 -- 172.21.15.139:0/3761977948 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54c1498f0 con 0x7fd54c094930 2024-09-16T01:45:34.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.957+0000 7fd54affd640 1 -- 172.21.15.139:0/3761977948 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550018960 con 0x7fd54c094930 2024-09-16T01:45:34.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.965+0000 7fd54affd640 1 -- 172.21.15.139:0/3761977948 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd550018b00 con 0x7fd54c094930 2024-09-16T01:45:34.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.966+0000 7fd54affd640 1 --2- 172.21.15.139:0/3761977948 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd5400a3440 0x7fd5400a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:34.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.966+0000 7fd55a9bc640 1 -- 172.21.15.139:0/3761977948 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd538005290 con 0x7fd54c094930 2024-09-16T01:45:34.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.967+0000 7fd54affd640 1 -- 172.21.15.139:0/3761977948 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fd5500d1400 con 0x7fd54c094930 2024-09-16T01:45:34.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.967+0000 7fd5591b9640 1 --2- 172.21.15.139:0/3761977948 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd5400a3440 0x7fd5400a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:34.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.970+0000 7fd5591b9640 1 --2- 172.21.15.139:0/3761977948 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd5400a3440 0x7fd5400a5900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd544005d00 tx=0x7fd544005c50 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:34.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:33.980+0000 7fd54affd640 1 -- 172.21.15.139:0/3761977948 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd55009da30 con 0x7fd54c094930 2024-09-16T01:45:34.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:34.424+0000 7fd55a9bc640 1 -- 172.21.15.139:0/3761977948 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd538005850 con 0x7fd54c094930 2024-09-16T01:45:34.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:34.427+0000 7fd54affd640 1 -- 172.21.15.139:0/3761977948 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+12016 (secure 0 0 0) 0x7fd55009d330 con 0x7fd54c094930 2024-09-16T01:45:34.429 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:45:34.429 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":35,"btime":"2024-09-16T01:45:19:161714+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:45:18.328499+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:45:34.429 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 35 2024-09-16T01:45:34.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:34.430+0000 7fd55a9bc640 1 -- 172.21.15.139:0/3761977948 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd5400a3440 msgr2=0x7fd5400a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:34.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:34.430+0000 7fd55a9bc640 1 --2- 172.21.15.139:0/3761977948 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd5400a3440 0x7fd5400a5900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd544005d00 tx=0x7fd544005c50 comp rx=0 tx=0).stop 2024-09-16T01:45:34.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:34.430+0000 7fd55a9bc640 1 -- 172.21.15.139:0/3761977948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd54c094930 msgr2=0x7fd54c13cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:34.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:34.430+0000 7fd55a9bc640 1 --2- 172.21.15.139:0/3761977948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd54c094930 0x7fd54c13cd70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd55000e9f0 tx=0x7fd55000eeb0 comp rx=0 tx=0).stop 2024-09-16T01:45:34.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:34.430+0000 7fd55a9bc640 1 -- 172.21.15.139:0/3761977948 shutdown_connections 2024-09-16T01:45:34.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:34.430+0000 7fd55a9bc640 1 --2- 172.21.15.139:0/3761977948 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd5400a3440 0x7fd5400a5900 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:34.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:34.430+0000 7fd55a9bc640 1 --2- 172.21.15.139:0/3761977948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd54c09a550 0x7fd54c141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:34.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:34.431+0000 7fd55a9bc640 1 --2- 172.21.15.139:0/3761977948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd54c0952d0 0x7fd54c13d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:34.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:34.431+0000 7fd55a9bc640 1 --2- 172.21.15.139:0/3761977948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd54c094930 0x7fd54c13cd70 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd55000e9f0 tx=0x7fd55000eeb0 comp rx=0 tx=0).stop 2024-09-16T01:45:34.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:34.431+0000 7fd55a9bc640 1 -- 172.21.15.139:0/3761977948 >> 172.21.15.139:0/3761977948 conn(0x7fd54c00a1f0 msgr2=0x7fd54c092200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:34.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:34.434+0000 7fd55a9bc640 1 -- 172.21.15.139:0/3761977948 shutdown_connections 2024-09-16T01:45:34.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:34.435+0000 7fd55a9bc640 1 -- 172.21.15.139:0/3761977948 wait complete. 2024-09-16T01:45:34.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:34 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:45:34.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:34 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:45:34.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:34 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:45:34.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:34 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:45:34.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:34 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:45:34.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:34 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:45:34.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:34 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:45:34.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:34 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:45:34.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:34 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:45:35.303 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0' 2024-09-16T01:45:35.303 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0 2024-09-16T01:45:35.392 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:35 smithi139 ceph-mon[31175]: pgmap v158: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.9 MiB/s wr, 370 op/s 2024-09-16T01:45:35.392 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:35 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3761977948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:45:35.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:35 smithi176 ceph-mon[35324]: pgmap v158: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.9 MiB/s wr, 370 op/s 2024-09-16T01:45:35.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:35 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3761977948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:45:35.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:35 smithi164 ceph-mon[35121]: pgmap v158: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.9 MiB/s wr, 370 op/s 2024-09-16T01:45:35.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:35 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3761977948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:45:35.931 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:45:35.968 INFO:tasks.workunit.client.0.smithi139.stdout: 459 1656160 1879 1136796 1584 352477 1078 2024-09-16T01:45:36.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.731+0000 7f5740e0e640 1 -- 172.21.15.139:0/252069952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f573c100430 msgr2=0x7f573c100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:36.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.731+0000 7f5740e0e640 1 --2- 172.21.15.139:0/252069952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f573c100430 0x7f573c100810 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f572c00dbf0 tx=0x7f572c0326c0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.733+0000 7f5740e0e640 1 -- 172.21.15.139:0/252069952 shutdown_connections 2024-09-16T01:45:36.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.733+0000 7f5740e0e640 1 --2- 172.21.15.139:0/252069952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f573c105560 0x7f573c10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.733+0000 7f5740e0e640 1 --2- 172.21.15.139:0/252069952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f573c100de0 0x7f573c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.733+0000 7f5740e0e640 1 --2- 172.21.15.139:0/252069952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f573c100430 0x7f573c100810 secure :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f572c00dbf0 tx=0x7f572c0326c0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.733+0000 7f5740e0e640 1 -- 172.21.15.139:0/252069952 >> 172.21.15.139:0/252069952 conn(0x7f573c077230 msgr2=0x7f573c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:36.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.740+0000 7f5740e0e640 1 -- 172.21.15.139:0/252069952 shutdown_connections 2024-09-16T01:45:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.744+0000 7f5740e0e640 1 -- 172.21.15.139:0/252069952 wait complete. 2024-09-16T01:45:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.745+0000 7f5740e0e640 1 Processor -- start 2024-09-16T01:45:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.745+0000 7f5740e0e640 1 -- start start 2024-09-16T01:45:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.745+0000 7f5740e0e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f573c100de0 0x7f573c073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.745+0000 7f5740e0e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f573c105560 0x7f573c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.745+0000 7f5740e0e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f573c06d230 0x7f573c06d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.745+0000 7f5740e0e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f573c10fdd0 con 0x7f573c105560 2024-09-16T01:45:36.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.745+0000 7f5740e0e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f573c10fc50 con 0x7f573c100de0 2024-09-16T01:45:36.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.745+0000 7f5740e0e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f573c10ff50 con 0x7f573c06d230 2024-09-16T01:45:36.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.747+0000 7f573a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f573c100de0 0x7f573c073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:36.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.747+0000 7f573a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f573c100de0 0x7f573c073c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56320/0 (socket says 172.21.15.139:56320) 2024-09-16T01:45:36.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.747+0000 7f573a59c640 1 -- 172.21.15.139:0/2505582919 learned_addr learned my addr 172.21.15.139:0/2505582919 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:36.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.748+0000 7f573a59c640 1 -- 172.21.15.139:0/2505582919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f573c06d230 msgr2=0x7f573c06d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:36.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.748+0000 7f573a59c640 1 --2- 172.21.15.139:0/2505582919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f573c06d230 0x7f573c06d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.748+0000 7f573a59c640 1 -- 172.21.15.139:0/2505582919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f573c105560 msgr2=0x7f573c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:36.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.748+0000 7f573a59c640 1 --2- 172.21.15.139:0/2505582919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f573c105560 0x7f573c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.748+0000 7f573a59c640 1 -- 172.21.15.139:0/2505582919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573c0718f0 con 0x7f573c100de0 2024-09-16T01:45:36.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.749+0000 7f573a59c640 1 --2- 172.21.15.139:0/2505582919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f573c100de0 0x7f573c073c60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f572c005d00 tx=0x7f572c00bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:36.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.751+0000 7f572b7fe640 1 -- 172.21.15.139:0/2505582919 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c049020 con 0x7f573c100de0 2024-09-16T01:45:36.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.751+0000 7f5740e0e640 1 -- 172.21.15.139:0/2505582919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573c071b80 con 0x7f573c100de0 2024-09-16T01:45:36.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.751+0000 7f5740e0e640 1 -- 172.21.15.139:0/2505582919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f573c071fe0 con 0x7f573c100de0 2024-09-16T01:45:36.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.751+0000 7f572b7fe640 1 -- 172.21.15.139:0/2505582919 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c03f040 con 0x7f573c100de0 2024-09-16T01:45:36.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.751+0000 7f572b7fe640 1 -- 172.21.15.139:0/2505582919 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c03e050 con 0x7f573c100de0 2024-09-16T01:45:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.756+0000 7f57431e7740 1 Processor -- start 2024-09-16T01:45:36.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.758+0000 7f57431e7740 1 -- start start 2024-09-16T01:45:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.759+0000 7f57431e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3d850 0x55e618a3dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.759+0000 7f57431e7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a3e170 0x55e618a484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.759+0000 7f57431e7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a48a30 0x55e618a4ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.759+0000 7f57431e7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e6189a2160 con 0x55e618a48a30 2024-09-16T01:45:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.759+0000 7f57431e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e618949720 con 0x55e618a3d850 2024-09-16T01:45:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.759+0000 7f57431e7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e61899e120 con 0x55e618a3e170 2024-09-16T01:45:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.759+0000 7f572b7fe640 1 -- 172.21.15.139:0/2505582919 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f572c04c070 con 0x7f573c100de0 2024-09-16T01:45:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.761+0000 7f572b7fe640 1 --2- 172.21.15.139:0/2505582919 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f57240a35c0 0x7f57240a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.761+0000 7f572b7fe640 1 -- 172.21.15.139:0/2505582919 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f572c0f7cd0 con 0x7f573c100de0 2024-09-16T01:45:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.762+0000 7f5739d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a48a30 0x55e618a4ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.762+0000 7f5739d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a48a30 0x55e618a4ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49332/0 (socket says 172.21.15.139:49332) 2024-09-16T01:45:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.762+0000 7f5739d9b640 1 -- 172.21.15.139:0/3846154034 learned_addr learned my addr 172.21.15.139:0/3846154034 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:36.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.762+0000 7f573ad9d640 1 --2- 172.21.15.139:0/3846154034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3d850 0x55e618a3dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:36.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.763+0000 7f573ad9d640 1 --2- 172.21.15.139:0/2505582919 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f57240a35c0 0x7f57240a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:36.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.764+0000 7f573a59c640 1 --2- 172.21.15.139:0/3846154034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a3e170 0x55e618a484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:36.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.764+0000 7f573ad9d640 1 -- 172.21.15.139:0/3846154034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a3e170 msgr2=0x55e618a484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:36.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.764+0000 7f573ad9d640 1 --2- 172.21.15.139:0/3846154034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a3e170 0x55e618a484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.764+0000 7f573ad9d640 1 -- 172.21.15.139:0/3846154034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a48a30 msgr2=0x55e618a4ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:36.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.764+0000 7f573ad9d640 1 --2- 172.21.15.139:0/3846154034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a48a30 0x55e618a4ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.764+0000 7f573ad9d640 1 -- 172.21.15.139:0/3846154034 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e618713ba0 con 0x55e618a3d850 2024-09-16T01:45:36.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.770+0000 7f573ad9d640 1 --2- 172.21.15.139:0/3846154034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3d850 0x55e618a3dc30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f57340123c0 tx=0x7f5734012880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=581c54d277999d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:36.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.771+0000 7f5728ff9640 1 -- 172.21.15.139:0/3846154034 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573401b070 con 0x55e618a3d850 2024-09-16T01:45:36.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.771+0000 7f5728ff9640 1 -- 172.21.15.139:0/3846154034 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f573400b100 con 0x55e618a3d850 2024-09-16T01:45:36.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.773+0000 7f57431e7740 1 -- 172.21.15.139:0/3846154034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3d850 msgr2=0x55e618a3dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:36.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.773+0000 7f57431e7740 1 --2- 172.21.15.139:0/3846154034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3d850 0x55e618a3dc30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f57340123c0 tx=0x7f5734012880 comp rx=0 tx=0).stop 2024-09-16T01:45:36.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.776+0000 7f57431e7740 1 -- 172.21.15.139:0/3846154034 shutdown_connections 2024-09-16T01:45:36.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.776+0000 7f57431e7740 1 --2- 172.21.15.139:0/3846154034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a48a30 0x55e618a4ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.776+0000 7f57431e7740 1 --2- 172.21.15.139:0/3846154034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a3e170 0x55e618a484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.776+0000 7f57431e7740 1 --2- 172.21.15.139:0/3846154034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3d850 0x55e618a3dc30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.776+0000 7f57431e7740 1 -- 172.21.15.139:0/3846154034 >> 172.21.15.139:0/3846154034 conn(0x55e618a39de0 msgr2=0x55e618a3a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:36.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.776+0000 7f573ad9d640 1 --2- 172.21.15.139:0/2505582919 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f57240a35c0 0x7f57240a5a80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f573401d000 tx=0x7f57340177d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.783+0000 7f57431e7740 1 -- 172.21.15.139:0/3846154034 shutdown_connections 2024-09-16T01:45:36.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.785+0000 7f57431e7740 1 -- 172.21.15.139:0/3846154034 wait complete. 2024-09-16T01:45:36.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.785+0000 7f57431e7740 1 -- start start 2024-09-16T01:45:36.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.785+0000 7f57431e7740 1 Processor -- start 2024-09-16T01:45:36.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.787+0000 7f57431e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3d850 0x55e618a457b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.787+0000 7f57431e7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a3e170 0x55e618a45cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.787+0000 7f57431e7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a48a30 0x55e618a46230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.787+0000 7f57431e7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e618a4b650 con 0x55e618a3e170 2024-09-16T01:45:36.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.787+0000 7f57431e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e61892a900 con 0x55e618a3d850 2024-09-16T01:45:36.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.787+0000 7f57431e7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e618a4b7d0 con 0x55e618a48a30 2024-09-16T01:45:36.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.787+0000 7f57431e7740 10 client.0 _ll_register_callbacks cb 0x55e618928780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:45:36.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.788+0000 7f573a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3d850 0x55e618a457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:36.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.788+0000 7f573a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3d850 0x55e618a457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56334/0 (socket says 172.21.15.139:56334) 2024-09-16T01:45:36.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.788+0000 7f573a59c640 1 -- 172.21.15.139:0/3877240528 learned_addr learned my addr 172.21.15.139:0/3877240528 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:36.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.788+0000 7f5739d9b640 1 --2- 172.21.15.139:0/3877240528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a3e170 0x55e618a45cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:36.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.789+0000 7f5739d9b640 1 -- 172.21.15.139:0/3877240528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a48a30 msgr2=0x55e618a46230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:36.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.789+0000 7f5739d9b640 1 --2- 172.21.15.139:0/3877240528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a48a30 0x55e618a46230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.789+0000 7f5739d9b640 1 -- 172.21.15.139:0/3877240528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3d850 msgr2=0x55e618a457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:36.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.789+0000 7f5739d9b640 1 --2- 172.21.15.139:0/3877240528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3d850 0x55e618a457b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.789+0000 7f5739d9b640 1 -- 172.21.15.139:0/3877240528 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61896ac50 con 0x55e618a3e170 2024-09-16T01:45:36.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.790+0000 7f5739d9b640 1 --2- 172.21.15.139:0/3877240528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a3e170 0x55e618a45cf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f57300058c0 tx=0x7f57300039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcadc9cf78ad9ca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:36.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.793+0000 7f5721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:45:36.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.793+0000 7f5721ffb640 1 -- 172.21.15.139:0/3877240528 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573000a890 con 0x55e618a3e170 2024-09-16T01:45:36.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.793+0000 7f5721ffb640 1 -- 172.21.15.139:0/3877240528 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f573000aa30 con 0x55e618a3e170 2024-09-16T01:45:36.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.795+0000 7f57431e7740 1 -- 172.21.15.139:0/3877240528 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f573400b100 con 0x55e618a3e170 2024-09-16T01:45:36.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.798+0000 7f5721ffb640 1 -- 172.21.15.139:0/3877240528 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f573000ac30 con 0x55e618a3e170 2024-09-16T01:45:36.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.798+0000 7f57431e7740 10 client.16086 fetch_fsmap learned FSMap version 35 2024-09-16T01:45:36.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.798+0000 7f57431e7740 1 -- 172.21.15.139:0/3877240528 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e618a572e0 con 0x55e618a3e170 2024-09-16T01:45:36.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.803+0000 7f5721ffb640 1 -- 172.21.15.139:0/3877240528 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f573400b100 con 0x55e618a3e170 2024-09-16T01:45:36.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.803+0000 7f5721ffb640 1 -- 172.21.15.139:0/3877240528 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7f573000b040 con 0x55e618a3e170 2024-09-16T01:45:36.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.803+0000 7f57431e7740 10 client.16086 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:45:36.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.803+0000 7f57431e7740 10 client.16086 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:45:36.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.803+0000 7f57431e7740 20 client.16086 populate_metadata read hostname 'smithi139' 2024-09-16T01:45:36.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.803+0000 7f57431e7740 1 --2- 172.21.15.139:0/3877240528 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e618a588b0 0x55e618a5aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.803+0000 7f57431e7740 4 client.16086 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:45:36.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.803+0000 7f57431e7740 1 -- 172.21.15.139:0/3877240528 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6189af890 con 0x55e618a588b0 2024-09-16T01:45:36.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.806+0000 7f573a59c640 1 --2- 172.21.15.139:0/3877240528 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e618a588b0 0x55e618a5aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:36.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.811+0000 7f573a59c640 1 --2- 172.21.15.139:0/3877240528 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e618a588b0 0x55e618a5aca0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3c2d562c9866f43 server_cookie=ff5b6ecf9a265032 in_seq=0 out_seq=0 2024-09-16T01:45:36.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.812+0000 7f5721ffb640 10 client.16086 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:36.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.820+0000 7f5721ffb640 1 -- 172.21.15.139:0/3877240528 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6189af890 con 0x55e618a588b0 2024-09-16T01:45:36.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.821+0000 7f5721ffb640 10 client.16086 handle_command_reply: tid=0 2024-09-16T01:45:36.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.822+0000 7f57431e7740 1 client.16086 shutdown 2024-09-16T01:45:36.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.826+0000 7f57431e7740 1 -- 172.21.15.139:0/3877240528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a3e170 msgr2=0x55e618a45cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:36.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.826+0000 7f57431e7740 1 --2- 172.21.15.139:0/3877240528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a3e170 0x55e618a45cf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f57300058c0 tx=0x7f57300039c0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.828+0000 7f57431e7740 1 -- 172.21.15.139:0/3877240528 shutdown_connections 2024-09-16T01:45:36.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.828+0000 7f57431e7740 1 --2- 172.21.15.139:0/3877240528 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e618a588b0 0x55e618a5aca0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.828+0000 7f57431e7740 1 --2- 172.21.15.139:0/3877240528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a48a30 0x55e618a46230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.828+0000 7f57431e7740 1 --2- 172.21.15.139:0/3877240528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a3e170 0x55e618a45cf0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.828+0000 7f57431e7740 1 --2- 172.21.15.139:0/3877240528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3d850 0x55e618a457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.828+0000 7f57431e7740 1 -- 172.21.15.139:0/3877240528 >> 172.21.15.139:0/3877240528 conn(0x55e618a39de0 msgr2=0x55e618a3fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:36.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.828+0000 7f5721ffb640 0 client.16086 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:36.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.830+0000 7f57431e7740 1 -- 172.21.15.139:0/3877240528 shutdown_connections 2024-09-16T01:45:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.830+0000 7f57431e7740 1 -- 172.21.15.139:0/3877240528 wait complete. 2024-09-16T01:45:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.830+0000 7f57431e7740 20 client.16086 trim_cache size 0 max 16384 2024-09-16T01:45:36.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.869+0000 7f57431e7740 1 Processor -- start 2024-09-16T01:45:36.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.870+0000 7f57431e7740 1 -- start start 2024-09-16T01:45:36.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.870+0000 7f57431e7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a3d850 0x55e618a46080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.870+0000 7f57431e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3e170 0x55e618a605b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.870+0000 7f57431e7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a48a30 0x55e618a60af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.871+0000 7f573a59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a3d850 0x55e618a46080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:36.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.871+0000 7f57431e7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f571c0100d0 con 0x55e618a48a30 2024-09-16T01:45:36.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.871+0000 7f57431e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f571c011d90 con 0x55e618a3e170 2024-09-16T01:45:36.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.871+0000 7f57431e7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f571c0107a0 con 0x55e618a3d850 2024-09-16T01:45:36.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.871+0000 7f573ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a48a30 0x55e618a60af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:36.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.871+0000 7f573ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a48a30 0x55e618a60af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49336/0 (socket says 172.21.15.139:49336) 2024-09-16T01:45:36.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.871+0000 7f573ad9d640 1 -- 172.21.15.139:0/2405750306 learned_addr learned my addr 172.21.15.139:0/2405750306 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:36.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.874+0000 7f573a59c640 1 -- 172.21.15.139:0/2405750306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3e170 msgr2=0x55e618a605b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:45:36.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.874+0000 7f573a59c640 1 --2- 172.21.15.139:0/2405750306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3e170 0x55e618a605b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.874+0000 7f573a59c640 1 -- 172.21.15.139:0/2405750306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a48a30 msgr2=0x55e618a60af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:36.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.874+0000 7f573a59c640 1 --2- 172.21.15.139:0/2405750306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a48a30 0x55e618a60af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.874+0000 7f573a59c640 1 -- 172.21.15.139:0/2405750306 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e618a3dfa0 con 0x55e618a3d850 2024-09-16T01:45:36.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.878+0000 7f573a59c640 1 --2- 172.21.15.139:0/2405750306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a3d850 0x55e618a46080 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f572c0c38a0 tx=0x7f572c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d73af63fe3115bc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:36.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.880+0000 7f5723fff640 1 -- 172.21.15.139:0/2405750306 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c00ba80 con 0x55e618a3d850 2024-09-16T01:45:36.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.880+0000 7f5723fff640 1 -- 172.21.15.139:0/2405750306 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c0c3ba0 con 0x55e618a3d850 2024-09-16T01:45:36.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.880+0000 7f5723fff640 1 -- 172.21.15.139:0/2405750306 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c004240 con 0x55e618a3d850 2024-09-16T01:45:36.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.881+0000 7f57431e7740 1 -- 172.21.15.139:0/2405750306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a3d850 msgr2=0x55e618a46080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:36.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.881+0000 7f57431e7740 1 --2- 172.21.15.139:0/2405750306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a3d850 0x55e618a46080 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f572c0c38a0 tx=0x7f572c005f90 comp rx=0 tx=0).stop 2024-09-16T01:45:36.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.881+0000 7f57431e7740 1 -- 172.21.15.139:0/2405750306 shutdown_connections 2024-09-16T01:45:36.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.881+0000 7f57431e7740 1 --2- 172.21.15.139:0/2405750306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a48a30 0x55e618a60af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.881+0000 7f57431e7740 1 --2- 172.21.15.139:0/2405750306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3e170 0x55e618a605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.881+0000 7f57431e7740 1 --2- 172.21.15.139:0/2405750306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a3d850 0x55e618a46080 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f572c0c38a0 tx=0x7f572c005f90 comp rx=0 tx=0).stop 2024-09-16T01:45:36.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.881+0000 7f57431e7740 1 -- 172.21.15.139:0/2405750306 >> 172.21.15.139:0/2405750306 conn(0x55e618a39de0 msgr2=0x55e618a442b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:36.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.884+0000 7f57431e7740 1 -- 172.21.15.139:0/2405750306 shutdown_connections 2024-09-16T01:45:36.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.884+0000 7f57431e7740 1 -- 172.21.15.139:0/2405750306 wait complete. 2024-09-16T01:45:36.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.885+0000 7f57431e7740 1 -- start start 2024-09-16T01:45:36.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.885+0000 7f57431e7740 1 Processor -- start 2024-09-16T01:45:36.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.885+0000 7f57431e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3e170 0x55e618a50d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.885+0000 7f57431e7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a48a30 0x55e618a51280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.885+0000 7f57431e7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a41a20 0x55e618a41e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.885+0000 7f57431e7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f571c011600 con 0x55e618a41a20 2024-09-16T01:45:36.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.885+0000 7f57431e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f571c010f30 con 0x55e618a3e170 2024-09-16T01:45:36.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.885+0000 7f57431e7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f571c00e0b0 con 0x55e618a48a30 2024-09-16T01:45:36.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.885+0000 7f57431e7740 10 client.0 _ll_register_callbacks cb 0x7f573000d550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:45:36.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.888+0000 7f573a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3e170 0x55e618a50d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:36.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.888+0000 7f5739d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a48a30 0x55e618a51280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:36.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.888+0000 7f5739d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a48a30 0x55e618a51280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:58332/0 (socket says 172.21.15.139:58332) 2024-09-16T01:45:36.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.888+0000 7f5739d9b640 1 -- 172.21.15.139:0/1186050597 learned_addr learned my addr 172.21.15.139:0/1186050597 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:36.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.890+0000 7f573ad9d640 1 --2- 172.21.15.139:0/1186050597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a41a20 0x55e618a41e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:36.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.893+0000 7f5739d9b640 1 -- 172.21.15.139:0/1186050597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3e170 msgr2=0x55e618a50d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:36.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.893+0000 7f5739d9b640 1 --2- 172.21.15.139:0/1186050597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3e170 0x55e618a50d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.893+0000 7f5739d9b640 1 -- 172.21.15.139:0/1186050597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a41a20 msgr2=0x55e618a41e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:36.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.893+0000 7f5739d9b640 1 --2- 172.21.15.139:0/1186050597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a41a20 0x55e618a41e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:36.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.893+0000 7f5739d9b640 1 -- 172.21.15.139:0/1186050597 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e618a42660 con 0x55e618a48a30 2024-09-16T01:45:36.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.897+0000 7f5739d9b640 1 --2- 172.21.15.139:0/1186050597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a48a30 0x55e618a51280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55e618a45040 tx=0x7f573000a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c10b2a9e86aef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:36.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.897+0000 7f5721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:45:36.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.901+0000 7f5721ffb640 1 -- 172.21.15.139:0/1186050597 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573002f020 con 0x55e618a48a30 2024-09-16T01:45:36.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.901+0000 7f57431e7740 1 -- 172.21.15.139:0/1186050597 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f572c0c3ba0 con 0x55e618a48a30 2024-09-16T01:45:36.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.901+0000 7f5721ffb640 1 -- 172.21.15.139:0/1186050597 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f5730022c40 con 0x55e618a48a30 2024-09-16T01:45:36.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.906+0000 7f5721ffb640 1 -- 172.21.15.139:0/1186050597 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5730005020 con 0x55e618a48a30 2024-09-16T01:45:36.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.907+0000 7f57431e7740 10 client.25549 fetch_fsmap learned FSMap version 35 2024-09-16T01:45:36.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.907+0000 7f57431e7740 1 -- 172.21.15.139:0/1186050597 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e618ac4590 con 0x55e618a48a30 2024-09-16T01:45:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.911+0000 7f5721ffb640 1 -- 172.21.15.139:0/1186050597 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c0c3ba0 con 0x55e618a48a30 2024-09-16T01:45:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.911+0000 7f5721ffb640 1 -- 172.21.15.139:0/1186050597 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7f573002a070 con 0x55e618a48a30 2024-09-16T01:45:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.911+0000 7f57431e7740 10 client.25549 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:45:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.911+0000 7f57431e7740 10 client.25549 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:45:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.911+0000 7f57431e7740 20 client.25549 populate_metadata read hostname 'smithi139' 2024-09-16T01:45:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.911+0000 7f57431e7740 1 --2- 172.21.15.139:0/1186050597 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e618ac4cf0 0x55e618ac7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.911+0000 7f57431e7740 4 client.25549 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}] 2024-09-16T01:45:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.911+0000 7f57431e7740 1 -- 172.21.15.139:0/1186050597 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}) -- 0x55e6189a9be0 con 0x55e618ac4cf0 2024-09-16T01:45:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.911+0000 7f573a59c640 1 --2- 172.21.15.139:0/1186050597 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e618ac4cf0 0x55e618ac7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:36.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.913+0000 7f573a59c640 1 --2- 172.21.15.139:0/1186050597 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e618ac4cf0 0x55e618ac7110 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c652e217bdad8f6 server_cookie=bd9228e0172d820a in_seq=0 out_seq=0 2024-09-16T01:45:36.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:36.914+0000 7f5721ffb640 10 client.25549 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:37.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:37 smithi139 ceph-mon[31175]: pgmap v159: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 606 op/s 2024-09-16T01:45:37.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:37 smithi176 ceph-mon[35324]: pgmap v159: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 606 op/s 2024-09-16T01:45:37.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:37 smithi164 ceph-mon[35121]: pgmap v159: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 606 op/s 2024-09-16T01:45:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.370+0000 7fcfa2ff5640 1 -- 172.21.15.139:0/42536716 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56371cc43d70 con 0x56371cd5e060 2024-09-16T01:45:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.370+0000 7fcfa2ff5640 10 client.25556 handle_command_reply: tid=0 2024-09-16T01:45:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.370+0000 7fcfc7e21740 1 client.25556 shutdown 2024-09-16T01:45:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.371+0000 7fcfc7e21740 1 -- 172.21.15.139:0/42536716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371cce2cb0 msgr2=0x56371cce6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.371+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/42536716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371cce2cb0 0x56371cce6c60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8014e90 tx=0x7fcfb809cdd0 comp rx=0 tx=0).stop 2024-09-16T01:45:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.371+0000 7fcfc7e21740 1 -- 172.21.15.139:0/42536716 shutdown_connections 2024-09-16T01:45:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.371+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/42536716 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56371cd5e060 0x56371cd60480 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.371+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/42536716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56371cce2cb0 0x56371cce6c60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.371+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/42536716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371ccd8410 0x56371cce6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.371+0000 7fcfc7e21740 1 --2- 172.21.15.139:0/42536716 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56371ccd7af0 0x56371cceb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.371+0000 7fcfc7e21740 1 -- 172.21.15.139:0/42536716 >> 172.21.15.139:0/42536716 conn(0x56371ccd4060 msgr2=0x56371ccd29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.371+0000 7fcfa2ff5640 0 client.25556 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.371+0000 7fcfc7e21740 1 -- 172.21.15.139:0/42536716 shutdown_connections 2024-09-16T01:45:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.371+0000 7fcfc7e21740 1 -- 172.21.15.139:0/42536716 wait complete. 2024-09-16T01:45:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.371+0000 7fcfc7e21740 20 client.25556 trim_cache size 0 max 16384 2024-09-16T01:45:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:45:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:45:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:45:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:45:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.372+0000 7fcfc5082640 1 -- 172.21.15.139:0/1331215977 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcfa80a3680 msgr2=0x7fcfa80a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.372+0000 7fcfc5082640 1 --2- 172.21.15.139:0/1331215977 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcfa80a3680 0x7fcfa80a5b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009850 tx=0x7fcfb0005850 comp rx=0 tx=0).stop 2024-09-16T01:45:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.372+0000 7fcfc5082640 1 -- 172.21.15.139:0/1331215977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfc0111410 msgr2=0x7fcfc01118c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.372+0000 7fcfc5082640 1 --2- 172.21.15.139:0/1331215977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfc0111410 0x7fcfc01118c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800b970 tx=0x7fcfb800be30 comp rx=0 tx=0).stop 2024-09-16T01:45:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.372+0000 7fcfc5082640 1 -- 172.21.15.139:0/1331215977 shutdown_connections 2024-09-16T01:45:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.372+0000 7fcfc5082640 1 --2- 172.21.15.139:0/1331215977 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcfa80a3680 0x7fcfa80a5b40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.372+0000 7fcfc5082640 1 --2- 172.21.15.139:0/1331215977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfc0111410 0x7fcfc01118c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.372+0000 7fcfc5082640 1 --2- 172.21.15.139:0/1331215977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfc011b140 0x7fcfc01188d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.372+0000 7fcfc5082640 1 --2- 172.21.15.139:0/1331215977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfc01095d0 0x7fcfc0118390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.372+0000 7fcfc5082640 1 -- 172.21.15.139:0/1331215977 >> 172.21.15.139:0/1331215977 conn(0x7fcfc006ce00 msgr2=0x7fcfc010a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.372+0000 7fcfc5082640 1 -- 172.21.15.139:0/1331215977 shutdown_connections 2024-09-16T01:45:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:39.372+0000 7fcfc5082640 1 -- 172.21.15.139:0/1331215977 wait complete. 2024-09-16T01:45:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:39 smithi139 ceph-mon[31175]: pgmap v160: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 13 MiB/s wr, 886 op/s 2024-09-16T01:45:39.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:39 smithi176 ceph-mon[35324]: pgmap v160: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 13 MiB/s wr, 886 op/s 2024-09-16T01:45:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:39 smithi164 ceph-mon[35121]: pgmap v160: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 13 MiB/s wr, 886 op/s 2024-09-16T01:45:41.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:41 smithi139 ceph-mon[31175]: pgmap v161: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-09-16T01:45:41.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:41 smithi176 ceph-mon[35324]: pgmap v161: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-09-16T01:45:41.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:41 smithi164 ceph-mon[35121]: pgmap v161: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-09-16T01:45:43.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:43 smithi139 ceph-mon[31175]: pgmap v162: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 16 MiB/s wr, 1.44k op/s 2024-09-16T01:45:43.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:43 smithi176 ceph-mon[35324]: pgmap v162: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 16 MiB/s wr, 1.44k op/s 2024-09-16T01:45:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:43 smithi164 ceph-mon[35121]: pgmap v162: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 16 MiB/s wr, 1.44k op/s 2024-09-16T01:45:45.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:45 smithi139 ceph-mon[31175]: pgmap v163: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 17 MiB/s wr, 1.60k op/s 2024-09-16T01:45:45.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:45 smithi176 ceph-mon[35324]: pgmap v163: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 17 MiB/s wr, 1.60k op/s 2024-09-16T01:45:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:45 smithi164 ceph-mon[35121]: pgmap v163: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 17 MiB/s wr, 1.60k op/s 2024-09-16T01:45:45.968 INFO:tasks.workunit.client.0.smithi139.stdout: 462 35640 19 24095 162 6971 9 2024-09-16T01:45:47.597 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:47 smithi164 ceph-mon[35121]: pgmap v164: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 15 MiB/s wr, 1.57k op/s 2024-09-16T01:45:47.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:47 smithi139 ceph-mon[31175]: pgmap v164: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 15 MiB/s wr, 1.57k op/s 2024-09-16T01:45:47.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:47 smithi176 ceph-mon[35324]: pgmap v164: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 15 MiB/s wr, 1.57k op/s 2024-09-16T01:45:49.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:49 smithi139 ceph-mon[31175]: pgmap v165: 129 pgs: 129 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.33k op/s 2024-09-16T01:45:49.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:49 smithi176 ceph-mon[35324]: pgmap v165: 129 pgs: 129 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.33k op/s 2024-09-16T01:45:49.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:49 smithi164 ceph-mon[35121]: pgmap v165: 129 pgs: 129 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.33k op/s 2024-09-16T01:45:51.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:51 smithi139 ceph-mon[31175]: pgmap v166: 129 pgs: 129 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.2 MiB/s wr, 1.28k op/s 2024-09-16T01:45:51.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:51 smithi176 ceph-mon[35324]: pgmap v166: 129 pgs: 129 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.2 MiB/s wr, 1.28k op/s 2024-09-16T01:45:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:51 smithi164 ceph-mon[35121]: pgmap v166: 129 pgs: 129 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.2 MiB/s wr, 1.28k op/s 2024-09-16T01:45:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.471+0000 7f44ee647640 1 -- 172.21.15.139:0/1211840774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44e8102470 msgr2=0x7f44e810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.471+0000 7f44ee647640 1 --2- 172.21.15.139:0/1211840774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44e8102470 0x7f44e810e9a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009cd0 tx=0x7f44e002a8e0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.471+0000 7f44ee647640 1 -- 172.21.15.139:0/1211840774 shutdown_connections 2024-09-16T01:45:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.471+0000 7f44ee647640 1 --2- 172.21.15.139:0/1211840774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44e8102470 0x7f44e810e9a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.471+0000 7f44ee647640 1 --2- 172.21.15.139:0/1211840774 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44e8101ad0 0x7f44e8101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.471+0000 7f44ee647640 1 --2- 172.21.15.139:0/1211840774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44e8107b70 0x7f44e8107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.471+0000 7f44ee647640 1 -- 172.21.15.139:0/1211840774 >> 172.21.15.139:0/1211840774 conn(0x7f44e80fd7d0 msgr2=0x7f44e80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.472+0000 7f44ee647640 1 -- 172.21.15.139:0/1211840774 shutdown_connections 2024-09-16T01:45:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.472+0000 7f44ee647640 1 -- 172.21.15.139:0/1211840774 wait complete. 2024-09-16T01:45:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.472+0000 7f44ee647640 1 Processor -- start 2024-09-16T01:45:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.472+0000 7f44ee647640 1 -- start start 2024-09-16T01:45:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ee647640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44e8101ad0 0x7f44e81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ee647640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44e8102470 0x7f44e81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ee647640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44e8107b70 0x7f44e819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ee647640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8111f20 con 0x7f44e8107b70 2024-09-16T01:45:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ee647640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8111da0 con 0x7f44e8101ad0 2024-09-16T01:45:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ee647640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81120a0 con 0x7f44e8102470 2024-09-16T01:45:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ecbe4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44e8107b70 0x7f44e819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ecbe4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44e8107b70 0x7f44e819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51890/0 (socket says 172.21.15.139:51890) 2024-09-16T01:45:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ecbe4640 1 -- 172.21.15.139:0/4099275561 learned_addr learned my addr 172.21.15.139:0/4099275561 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44e7fff640 1 --2- 172.21.15.139:0/4099275561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44e8101ad0 0x7f44e81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44e77fe640 1 --2- 172.21.15.139:0/4099275561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44e8102470 0x7f44e81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ecbe4640 1 -- 172.21.15.139:0/4099275561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44e8102470 msgr2=0x7f44e81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ecbe4640 1 --2- 172.21.15.139:0/4099275561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44e8102470 0x7f44e81988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ecbe4640 1 -- 172.21.15.139:0/4099275561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44e8101ad0 msgr2=0x7f44e81983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ecbe4640 1 --2- 172.21.15.139:0/4099275561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44e8101ad0 0x7f44e81983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ecbe4640 1 -- 172.21.15.139:0/4099275561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e819d3a0 con 0x7f44e8107b70 2024-09-16T01:45:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.473+0000 7f44ecbe4640 1 --2- 172.21.15.139:0/4099275561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44e8107b70 0x7f44e819cc70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f44e0005cc0 tx=0x7f44e00093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.474+0000 7f44e57fa640 1 -- 172.21.15.139:0/4099275561 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0003e00 con 0x7f44e8107b70 2024-09-16T01:45:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.474+0000 7f44e57fa640 1 -- 172.21.15.139:0/4099275561 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e0004520 con 0x7f44e8107b70 2024-09-16T01:45:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.474+0000 7f44ee647640 1 -- 172.21.15.139:0/4099275561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e819d630 con 0x7f44e8107b70 2024-09-16T01:45:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.474+0000 7f44e57fa640 1 -- 172.21.15.139:0/4099275561 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0030db0 con 0x7f44e8107b70 2024-09-16T01:45:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.474+0000 7f44ee647640 1 -- 172.21.15.139:0/4099275561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e81a4f20 con 0x7f44e8107b70 2024-09-16T01:45:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.476+0000 7f44e57fa640 1 -- 172.21.15.139:0/4099275561 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f44e00046d0 con 0x7f44e8107b70 2024-09-16T01:45:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.476+0000 7f44f13db740 1 Processor -- start 2024-09-16T01:45:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.476+0000 7f44f13db740 1 -- start start 2024-09-16T01:45:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.476+0000 7f44f13db740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae3ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.476+0000 7f44f13db740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae3e410 0x561daae48770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.476+0000 7f44f13db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae48cb0 0x561daae4b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.476+0000 7f44f13db740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561daada22f0 con 0x561daae3daf0 2024-09-16T01:45:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.476+0000 7f44f13db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561daad4a270 con 0x561daae48cb0 2024-09-16T01:45:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.476+0000 7f44f13db740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561daad9e2b0 con 0x561daae3e410 2024-09-16T01:45:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44e57fa640 1 --2- 172.21.15.139:0/4099275561 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f44d00a3460 0x7f44d00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae3ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44e7fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae3e410 0x561daae48770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44e7fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae3e410 0x561daae48770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:56738/0 (socket says 172.21.15.139:56738) 2024-09-16T01:45:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44e7fff640 1 -- 172.21.15.139:0/3790098313 learned_addr learned my addr 172.21.15.139:0/3790098313 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44ecbe4640 1 --2- 172.21.15.139:0/4099275561 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f44d00a3460 0x7f44d00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44e77fe640 1 --2- 172.21.15.139:0/3790098313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae48cb0 0x561daae4b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44e57fa640 1 -- 172.21.15.139:0/4099275561 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f44e002e030 con 0x7f44e8107b70 2024-09-16T01:45:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44e7fff640 1 -- 172.21.15.139:0/3790098313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae48cb0 msgr2=0x561daae4b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44e7fff640 1 --2- 172.21.15.139:0/3790098313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae48cb0 0x561daae4b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44e7fff640 1 -- 172.21.15.139:0/3790098313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 msgr2=0x561daae3ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44e7fff640 1 --2- 172.21.15.139:0/3790098313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae3ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44e7fff640 1 -- 172.21.15.139:0/3790098313 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561daad49340 con 0x561daae3e410 2024-09-16T01:45:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44e77fe640 1 --2- 172.21.15.139:0/3790098313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae48cb0 0x561daae4b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:45:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44ecbe4640 1 --2- 172.21.15.139:0/4099275561 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f44d00a3460 0x7f44d00a5920 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f44e00eeb10 tx=0x7f44e0090420 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44e7fff640 1 --2- 172.21.15.139:0/3790098313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae3e410 0x561daae48770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009c70 tx=0x7f44d8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=756aa1db81a7dca2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44ce7fc640 1 -- 172.21.15.139:0/3790098313 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8004230 con 0x561daae3e410 2024-09-16T01:45:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.477+0000 7f44ce7fc640 1 -- 172.21.15.139:0/3790098313 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d80043d0 con 0x561daae3e410 2024-09-16T01:45:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.478+0000 7f44ce7fc640 1 -- 172.21.15.139:0/3790098313 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8019740 con 0x561daae3e410 2024-09-16T01:45:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.478+0000 7f44f13db740 1 -- 172.21.15.139:0/3790098313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae3e410 msgr2=0x561daae48770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.478+0000 7f44f13db740 1 --2- 172.21.15.139:0/3790098313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae3e410 0x561daae48770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009c70 tx=0x7f44d8011810 comp rx=0 tx=0).stop 2024-09-16T01:45:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.478+0000 7f44f13db740 1 -- 172.21.15.139:0/3790098313 shutdown_connections 2024-09-16T01:45:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.478+0000 7f44f13db740 1 --2- 172.21.15.139:0/3790098313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae48cb0 0x561daae4b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.478+0000 7f44f13db740 1 --2- 172.21.15.139:0/3790098313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae3e410 0x561daae48770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.478+0000 7f44f13db740 1 --2- 172.21.15.139:0/3790098313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae3ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.478+0000 7f44f13db740 1 -- 172.21.15.139:0/3790098313 >> 172.21.15.139:0/3790098313 conn(0x561daae3a060 msgr2=0x561daae3a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.478+0000 7f44f13db740 1 -- 172.21.15.139:0/3790098313 shutdown_connections 2024-09-16T01:45:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.478+0000 7f44f13db740 1 -- 172.21.15.139:0/3790098313 wait complete. 2024-09-16T01:45:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44f13db740 1 -- start start 2024-09-16T01:45:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44f13db740 1 Processor -- start 2024-09-16T01:45:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44f13db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3daf0 0x561daae45b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44f13db740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3e410 0x561daae460b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44f13db740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 0x561daae465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44f13db740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561daae4b880 con 0x561daae3e410 2024-09-16T01:45:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44f13db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561daad2b290 con 0x561daae3daf0 2024-09-16T01:45:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44f13db740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561daae4ba00 con 0x561daae48cb0 2024-09-16T01:45:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44f13db740 10 client.0 _ll_register_callbacks cb 0x561daad5c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:45:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3e410 0x561daae460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3e410 0x561daae460b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51910/0 (socket says 172.21.15.139:51910) 2024-09-16T01:45:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44e7fff640 1 -- 172.21.15.139:0/3508039853 learned_addr learned my addr 172.21.15.139:0/3508039853 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44e77fe640 1 --2- 172.21.15.139:0/3508039853 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 0x561daae465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44e77fe640 1 --2- 172.21.15.139:0/3508039853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3daf0 0x561daae45b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44e7fff640 1 -- 172.21.15.139:0/3508039853 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 msgr2=0x561daae465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44e7fff640 1 --2- 172.21.15.139:0/3508039853 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 0x561daae465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44e7fff640 1 -- 172.21.15.139:0/3508039853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3daf0 msgr2=0x561daae45b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.479+0000 7f44e7fff640 1 --2- 172.21.15.139:0/3508039853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3daf0 0x561daae45b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44e7fff640 1 -- 172.21.15.139:0/3508039853 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561daaab5210 con 0x561daae3e410 2024-09-16T01:45:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44e7fff640 1 --2- 172.21.15.139:0/3508039853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3e410 0x561daae460b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f44d8000c00 tx=0x7f44d8009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360f1e87dcff2610 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:45:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44c77fe640 1 -- 172.21.15.139:0/3508039853 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8027070 con 0x561daae3e410 2024-09-16T01:45:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44c77fe640 1 -- 172.21.15.139:0/3508039853 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f44d8003800 con 0x561daae3e410 2024-09-16T01:45:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44f13db740 1 -- 172.21.15.139:0/3508039853 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44d80043d0 con 0x561daae3e410 2024-09-16T01:45:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44c77fe640 1 -- 172.21.15.139:0/3508039853 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f44d8033050 con 0x561daae3e410 2024-09-16T01:45:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44f13db740 10 client.16104 fetch_fsmap learned FSMap version 35 2024-09-16T01:45:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44f13db740 1 -- 172.21.15.139:0/3508039853 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561daab81620 con 0x561daae3e410 2024-09-16T01:45:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44c77fe640 1 -- 172.21.15.139:0/3508039853 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d80043d0 con 0x561daae3e410 2024-09-16T01:45:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44c77fe640 1 -- 172.21.15.139:0/3508039853 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7f44d8018070 con 0x561daae3e410 2024-09-16T01:45:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44f13db740 10 client.16104 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:45:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44f13db740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:45:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44f13db740 20 client.16104 populate_metadata read hostname 'smithi139' 2024-09-16T01:45:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44f13db740 1 --2- 172.21.15.139:0/3508039853 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561daae58b20 0x561daae5af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44f13db740 4 client.16104 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:45:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.480+0000 7f44f13db740 1 -- 172.21.15.139:0/3508039853 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561daadafa20 con 0x561daae58b20 2024-09-16T01:45:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.481+0000 7f44e77fe640 1 --2- 172.21.15.139:0/3508039853 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561daae58b20 0x561daae5af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.481+0000 7f44c77fe640 10 client.16104 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.481+0000 7f44e77fe640 1 --2- 172.21.15.139:0/3508039853 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561daae58b20 0x561daae5af10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f7b792048d9226 server_cookie=9ef3554c76ff3c33 in_seq=0 out_seq=0 2024-09-16T01:45:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44c77fe640 1 -- 172.21.15.139:0/3508039853 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561daadafa20 con 0x561daae58b20 2024-09-16T01:45:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44c77fe640 10 client.16104 handle_command_reply: tid=0 2024-09-16T01:45:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44f13db740 1 client.16104 shutdown 2024-09-16T01:45:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44f13db740 1 -- 172.21.15.139:0/3508039853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3e410 msgr2=0x561daae460b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44f13db740 1 --2- 172.21.15.139:0/3508039853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3e410 0x561daae460b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f44d8000c00 tx=0x7f44d8009ee0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44f13db740 1 -- 172.21.15.139:0/3508039853 shutdown_connections 2024-09-16T01:45:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44f13db740 1 --2- 172.21.15.139:0/3508039853 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561daae58b20 0x561daae5af10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44f13db740 1 --2- 172.21.15.139:0/3508039853 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 0x561daae465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44c77fe640 0 client.16104 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44f13db740 1 --2- 172.21.15.139:0/3508039853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3e410 0x561daae460b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44f13db740 1 --2- 172.21.15.139:0/3508039853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3daf0 0x561daae45b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44f13db740 1 -- 172.21.15.139:0/3508039853 >> 172.21.15.139:0/3508039853 conn(0x561daae3a060 msgr2=0x561daae3fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44f13db740 1 -- 172.21.15.139:0/3508039853 shutdown_connections 2024-09-16T01:45:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44f13db740 1 -- 172.21.15.139:0/3508039853 wait complete. 2024-09-16T01:45:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.482+0000 7f44f13db740 20 client.16104 trim_cache size 0 max 16384 2024-09-16T01:45:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.497+0000 7f44f13db740 1 Processor -- start 2024-09-16T01:45:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.497+0000 7f44f13db740 1 -- start start 2024-09-16T01:45:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.497+0000 7f44f13db740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae46420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.497+0000 7f44f13db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3e410 0x561daae47ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.497+0000 7f44f13db740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 0x561daae60740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.497+0000 7f44f13db740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81aef50 con 0x561daae3daf0 2024-09-16T01:45:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.497+0000 7f44f13db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81b0c10 con 0x561daae3e410 2024-09-16T01:45:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.497+0000 7f44f13db740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81af620 con 0x561daae48cb0 2024-09-16T01:45:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae46420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae46420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51916/0 (socket says 172.21.15.139:51916) 2024-09-16T01:45:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44e77fe640 1 -- 172.21.15.139:0/664424670 learned_addr learned my addr 172.21.15.139:0/664424670 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44e7fff640 1 --2- 172.21.15.139:0/664424670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3e410 0x561daae47ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44e77fe640 1 --2- 172.21.15.139:0/664424670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 0x561daae60740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44e77fe640 1 -- 172.21.15.139:0/664424670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 msgr2=0x561daae60740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44e77fe640 1 --2- 172.21.15.139:0/664424670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 0x561daae60740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44e77fe640 1 -- 172.21.15.139:0/664424670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3e410 msgr2=0x561daae47ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44e77fe640 1 --2- 172.21.15.139:0/664424670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3e410 0x561daae47ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44e77fe640 1 -- 172.21.15.139:0/664424670 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561daae5b4d0 con 0x561daae3daf0 2024-09-16T01:45:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44e7fff640 1 --2- 172.21.15.139:0/664424670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3e410 0x561daae47ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:45:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44e77fe640 1 --2- 172.21.15.139:0/664424670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae46420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0105b0 tx=0x7f44dc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a194ecd7ea65983 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44cdffb640 1 -- 172.21.15.139:0/664424670 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc004230 con 0x561daae3daf0 2024-09-16T01:45:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44cdffb640 1 -- 172.21.15.139:0/664424670 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44dc00fe30 con 0x561daae3daf0 2024-09-16T01:45:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.498+0000 7f44cdffb640 1 -- 172.21.15.139:0/664424670 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc022340 con 0x561daae3daf0 2024-09-16T01:45:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.499+0000 7f44f13db740 1 -- 172.21.15.139:0/664424670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 msgr2=0x561daae46420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.499+0000 7f44f13db740 1 --2- 172.21.15.139:0/664424670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae46420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0105b0 tx=0x7f44dc005850 comp rx=0 tx=0).stop 2024-09-16T01:45:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.499+0000 7f44f13db740 1 -- 172.21.15.139:0/664424670 shutdown_connections 2024-09-16T01:45:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.499+0000 7f44f13db740 1 --2- 172.21.15.139:0/664424670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 0x561daae60740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.499+0000 7f44f13db740 1 --2- 172.21.15.139:0/664424670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3e410 0x561daae47ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.499+0000 7f44f13db740 1 --2- 172.21.15.139:0/664424670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae46420 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.499+0000 7f44f13db740 1 -- 172.21.15.139:0/664424670 >> 172.21.15.139:0/664424670 conn(0x561daae3a060 msgr2=0x561daae473d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.499+0000 7f44f13db740 1 -- 172.21.15.139:0/664424670 shutdown_connections 2024-09-16T01:45:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.499+0000 7f44f13db740 1 -- 172.21.15.139:0/664424670 wait complete. 2024-09-16T01:45:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.499+0000 7f44f13db740 1 -- start start 2024-09-16T01:45:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.499+0000 7f44f13db740 1 Processor -- start 2024-09-16T01:45:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44f13db740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae50cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44f13db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3e410 0x561daae51200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44f13db740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 0x561daae41e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44f13db740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81b0480 con 0x561daae3daf0 2024-09-16T01:45:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44f13db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81afdb0 con 0x561daae3e410 2024-09-16T01:45:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44f13db740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81acf30 con 0x561daae48cb0 2024-09-16T01:45:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44f13db740 10 client.0 _ll_register_callbacks cb 0x7f44dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:45:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae50cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae50cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51918/0 (socket says 172.21.15.139:51918) 2024-09-16T01:45:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44e77fe640 1 -- 172.21.15.139:0/2826188762 learned_addr learned my addr 172.21.15.139:0/2826188762 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44e77fe640 1 --2- 172.21.15.139:0/2826188762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 0x561daae41e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44e7fff640 1 --2- 172.21.15.139:0/2826188762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3e410 0x561daae51200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44e77fe640 1 -- 172.21.15.139:0/2826188762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 msgr2=0x561daae41e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44e77fe640 1 --2- 172.21.15.139:0/2826188762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 0x561daae41e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44e77fe640 1 -- 172.21.15.139:0/2826188762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3e410 msgr2=0x561daae51200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44e77fe640 1 --2- 172.21.15.139:0/2826188762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3e410 0x561daae51200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44e77fe640 1 -- 172.21.15.139:0/2826188762 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561daae51940 con 0x561daae3daf0 2024-09-16T01:45:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44e77fe640 1 --2- 172.21.15.139:0/2826188762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae50cc0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f44dc005820 tx=0x7f44dc003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7875718eeb1419a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:45:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44c77fe640 1 -- 172.21.15.139:0/2826188762 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc037020 con 0x561daae3daf0 2024-09-16T01:45:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44c77fe640 1 -- 172.21.15.139:0/2826188762 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f44dc00fb50 con 0x561daae3daf0 2024-09-16T01:45:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.500+0000 7f44f13db740 1 -- 172.21.15.139:0/2826188762 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44dc00fe30 con 0x561daae3daf0 2024-09-16T01:45:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.501+0000 7f44c77fe640 1 -- 172.21.15.139:0/2826188762 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f44dc005020 con 0x561daae3daf0 2024-09-16T01:45:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.501+0000 7f44f13db740 10 client.16110 fetch_fsmap learned FSMap version 35 2024-09-16T01:45:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.501+0000 7f44f13db740 1 -- 172.21.15.139:0/2826188762 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561daaec2d20 con 0x561daae3daf0 2024-09-16T01:45:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.501+0000 7f44c77fe640 1 -- 172.21.15.139:0/2826188762 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44dc00fe30 con 0x561daae3daf0 2024-09-16T01:45:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.501+0000 7f44c77fe640 1 -- 172.21.15.139:0/2826188762 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7f44dc003b70 con 0x561daae3daf0 2024-09-16T01:45:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.501+0000 7f44f13db740 10 client.16110 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:45:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.501+0000 7f44f13db740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:45:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.501+0000 7f44f13db740 20 client.16110 populate_metadata read hostname 'smithi139' 2024-09-16T01:45:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.501+0000 7f44f13db740 1 --2- 172.21.15.139:0/2826188762 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561daaec3420 0x561daaec5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.501+0000 7f44f13db740 4 client.16110 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:45:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.501+0000 7f44f13db740 1 -- 172.21.15.139:0/2826188762 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561daada9d70 con 0x561daaec3420 2024-09-16T01:45:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.501+0000 7f44e7fff640 1 --2- 172.21.15.139:0/2826188762 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561daaec3420 0x561daaec5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.501+0000 7f44c77fe640 10 client.16110 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:52.501+0000 7f44e7fff640 1 --2- 172.21.15.139:0/2826188762 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561daaec3420 0x561daaec5840 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d81cd79ebab01f21 server_cookie=eddfa74e35470cae in_seq=0 out_seq=0 2024-09-16T01:45:53.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:53 smithi139 ceph-mon[31175]: pgmap v167: 129 pgs: 129 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 1.19k op/s 2024-09-16T01:45:53.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:53 smithi176 ceph-mon[35324]: pgmap v167: 129 pgs: 129 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 1.19k op/s 2024-09-16T01:45:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:53 smithi164 ceph-mon[35121]: pgmap v167: 129 pgs: 129 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 1.19k op/s 2024-09-16T01:45:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.058+0000 7f44c77fe640 1 -- 172.21.15.139:0/2826188762 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561daada9d70 con 0x561daaec3420 2024-09-16T01:45:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.058+0000 7f44c77fe640 10 client.16110 handle_command_reply: tid=0 2024-09-16T01:45:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.058+0000 7f44f13db740 1 client.16110 shutdown 2024-09-16T01:45:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.059+0000 7f44f13db740 1 -- 172.21.15.139:0/2826188762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 msgr2=0x561daae50cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.059+0000 7f44f13db740 1 --2- 172.21.15.139:0/2826188762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae50cc0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f44dc005820 tx=0x7f44dc003ad0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.059+0000 7f44f13db740 1 -- 172.21.15.139:0/2826188762 shutdown_connections 2024-09-16T01:45:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.059+0000 7f44f13db740 1 --2- 172.21.15.139:0/2826188762 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561daaec3420 0x561daaec5840 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.059+0000 7f44f13db740 1 --2- 172.21.15.139:0/2826188762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561daae48cb0 0x561daae41e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.059+0000 7f44f13db740 1 --2- 172.21.15.139:0/2826188762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daae3e410 0x561daae51200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.059+0000 7f44c77fe640 0 client.16110 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.059+0000 7f44f13db740 1 --2- 172.21.15.139:0/2826188762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561daae3daf0 0x561daae50cc0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.059+0000 7f44f13db740 1 -- 172.21.15.139:0/2826188762 >> 172.21.15.139:0/2826188762 conn(0x561daae3a060 msgr2=0x561daae3fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.059+0000 7f44f13db740 1 -- 172.21.15.139:0/2826188762 shutdown_connections 2024-09-16T01:45:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.059+0000 7f44f13db740 1 -- 172.21.15.139:0/2826188762 wait complete. 2024-09-16T01:45:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.059+0000 7f44f13db740 20 client.16110 trim_cache size 0 max 16384 2024-09-16T01:45:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:45:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:45:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:45:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:45:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.059+0000 7f44ee647640 1 -- 172.21.15.139:0/4099275561 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f44d00a3460 msgr2=0x7f44d00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.059+0000 7f44ee647640 1 --2- 172.21.15.139:0/4099275561 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f44d00a3460 0x7f44d00a5920 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f44e00eeb10 tx=0x7f44e0090420 comp rx=0 tx=0).stop 2024-09-16T01:45:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.060+0000 7f44ee647640 1 -- 172.21.15.139:0/4099275561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44e8107b70 msgr2=0x7f44e819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.060+0000 7f44ee647640 1 --2- 172.21.15.139:0/4099275561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44e8107b70 0x7f44e819cc70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f44e0005cc0 tx=0x7f44e00093d0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.060+0000 7f44ee647640 1 -- 172.21.15.139:0/4099275561 shutdown_connections 2024-09-16T01:45:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.060+0000 7f44ee647640 1 --2- 172.21.15.139:0/4099275561 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f44d00a3460 0x7f44d00a5920 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.060+0000 7f44ee647640 1 --2- 172.21.15.139:0/4099275561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44e8107b70 0x7f44e819cc70 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.060+0000 7f44ee647640 1 --2- 172.21.15.139:0/4099275561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44e8102470 0x7f44e81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.060+0000 7f44ee647640 1 --2- 172.21.15.139:0/4099275561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44e8101ad0 0x7f44e81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.060+0000 7f44ee647640 1 -- 172.21.15.139:0/4099275561 >> 172.21.15.139:0/4099275561 conn(0x7f44e80fd7d0 msgr2=0x7f44e80fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.060+0000 7f44ee647640 1 -- 172.21.15.139:0/4099275561 shutdown_connections 2024-09-16T01:45:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:45:54.060+0000 7f44ee647640 1 -- 172.21.15.139:0/4099275561 wait complete. 2024-09-16T01:45:54.293 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:45:54.293 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 35, 2024-09-16T01:45:54.293 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:45:54.293 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 18, 2024-09-16T01:45:54.293 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:45:54.294 INFO:teuthology.orchestra.run.smithi139.stdout: "2dbb8969": { 2024-09-16T01:45:54.294 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 18, 2024-09-16T01:45:54.294 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:45:54.294 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:45:54.294 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:45:54.294 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:45:54.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:45:54.294 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:45:54.294 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 145.3, 2024-09-16T01:45:54.294 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:45:54.294 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:45:54.294 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:45:54.295 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:45:54.295 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:45:54.295 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:45:54.295 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:45:54.295 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:45:54.295 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:45:54.295 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:45:54.295 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:45:54.295 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:45:54.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f5721ffb640 1 -- 172.21.15.139:0/1186050597 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e6189a9be0 con 0x55e618ac4cf0 2024-09-16T01:45:54.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f5721ffb640 10 client.25549 handle_command_reply: tid=0 2024-09-16T01:45:54.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f57431e7740 1 client.25549 shutdown 2024-09-16T01:45:54.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f57431e7740 1 -- 172.21.15.139:0/1186050597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a48a30 msgr2=0x55e618a51280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:54.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f57431e7740 1 --2- 172.21.15.139:0/1186050597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a48a30 0x55e618a51280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55e618a45040 tx=0x7f573000a8c0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f57431e7740 1 -- 172.21.15.139:0/1186050597 shutdown_connections 2024-09-16T01:45:54.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f57431e7740 1 --2- 172.21.15.139:0/1186050597 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e618ac4cf0 0x55e618ac7110 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f57431e7740 1 --2- 172.21.15.139:0/1186050597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e618a41a20 0x55e618a41e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f57431e7740 1 --2- 172.21.15.139:0/1186050597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e618a48a30 0x55e618a51280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f57431e7740 1 --2- 172.21.15.139:0/1186050597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e618a3e170 0x55e618a50d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f5721ffb640 0 client.25549 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:54.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f57431e7740 1 -- 172.21.15.139:0/1186050597 >> 172.21.15.139:0/1186050597 conn(0x55e618a39de0 msgr2=0x55e618a386c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:54.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f57431e7740 1 -- 172.21.15.139:0/1186050597 shutdown_connections 2024-09-16T01:45:54.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f57431e7740 1 -- 172.21.15.139:0/1186050597 wait complete. 2024-09-16T01:45:54.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.290+0000 7f57431e7740 20 client.25549 trim_cache size 0 max 16384 2024-09-16T01:45:54.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.291+0000 7f5740e0e640 1 -- 172.21.15.139:0/2505582919 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f57240a35c0 msgr2=0x7f57240a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:54.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.291+0000 7f5740e0e640 1 --2- 172.21.15.139:0/2505582919 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f57240a35c0 0x7f57240a5a80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f573401d000 tx=0x7f57340177d0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.291+0000 7f5740e0e640 1 -- 172.21.15.139:0/2505582919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f573c100de0 msgr2=0x7f573c073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:54.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.291+0000 7f5740e0e640 1 --2- 172.21.15.139:0/2505582919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f573c100de0 0x7f573c073c60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f572c005d00 tx=0x7f572c00bf80 comp rx=0 tx=0).stop 2024-09-16T01:45:54.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.291+0000 7f5740e0e640 1 -- 172.21.15.139:0/2505582919 shutdown_connections 2024-09-16T01:45:54.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.291+0000 7f5740e0e640 1 --2- 172.21.15.139:0/2505582919 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f57240a35c0 0x7f57240a5a80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.291+0000 7f5740e0e640 1 --2- 172.21.15.139:0/2505582919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f573c06d230 0x7f573c06d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.291+0000 7f5740e0e640 1 --2- 172.21.15.139:0/2505582919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f573c105560 0x7f573c06ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.291+0000 7f5740e0e640 1 --2- 172.21.15.139:0/2505582919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f573c100de0 0x7f573c073c60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:54.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.291+0000 7f5740e0e640 1 -- 172.21.15.139:0/2505582919 >> 172.21.15.139:0/2505582919 conn(0x7f573c077230 msgr2=0x7f573c103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:54.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.291+0000 7f5740e0e640 1 -- 172.21.15.139:0/2505582919 shutdown_connections 2024-09-16T01:45:54.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:54.292+0000 7f5740e0e640 1 -- 172.21.15.139:0/2505582919 wait complete. 2024-09-16T01:45:55.152 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2dbb8969', quiesce took 24.0 seconds. Will release after: 1.3500000000000014 2024-09-16T01:45:55.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:55 smithi139 ceph-mon[31175]: pgmap v168: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.1 MiB/s wr, 868 op/s 2024-09-16T01:45:55.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:55 smithi176 ceph-mon[35324]: pgmap v168: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.1 MiB/s wr, 868 op/s 2024-09-16T01:45:55.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:55 smithi164 ceph-mon[35121]: pgmap v168: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.1 MiB/s wr, 868 op/s 2024-09-16T01:45:55.968 INFO:tasks.workunit.client.0.smithi139.stdout: 462 14197 0 9626 0 2944 1 2024-09-16T01:45:56.503 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:45:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:45:57.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.327+0000 7f84feb18640 1 -- 172.21.15.139:0/3681096078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f810c170 msgr2=0x7f84f810e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:57.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.327+0000 7f84feb18640 1 --2- 172.21.15.139:0/3681096078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f810c170 0x7f84f810e5f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009ca0 tx=0x7f84f002a950 comp rx=0 tx=0).stop 2024-09-16T01:45:57.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.327+0000 7f84feb18640 1 -- 172.21.15.139:0/3681096078 shutdown_connections 2024-09-16T01:45:57.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.327+0000 7f84feb18640 1 --2- 172.21.15.139:0/3681096078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f810c170 0x7f84f810e5f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:57.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.327+0000 7f84feb18640 1 --2- 172.21.15.139:0/3681096078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84f81036b0 0x7f84f810bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:57.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.327+0000 7f84feb18640 1 --2- 172.21.15.139:0/3681096078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84f8102d90 0x7f84f8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:57.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.327+0000 7f84feb18640 1 -- 172.21.15.139:0/3681096078 >> 172.21.15.139:0/3681096078 conn(0x7f84f80fc7d0 msgr2=0x7f84f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:57.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.327+0000 7f84feb18640 1 -- 172.21.15.139:0/3681096078 shutdown_connections 2024-09-16T01:45:57.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.327+0000 7f84feb18640 1 -- 172.21.15.139:0/3681096078 wait complete. 2024-09-16T01:45:57.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.327+0000 7f84feb18640 1 Processor -- start 2024-09-16T01:45:57.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.327+0000 7f84feb18640 1 -- start start 2024-09-16T01:45:57.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84feb18640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84f8102d90 0x7f84f8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:57.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84feb18640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84f81036b0 0x7f84f81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:57.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84feb18640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f810c170 0x7f84f819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:57.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84feb18640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f84f8111fa0 con 0x7f84f810c170 2024-09-16T01:45:57.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84feb18640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84f8111e20 con 0x7f84f8102d90 2024-09-16T01:45:57.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84feb18640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f84f8112120 con 0x7f84f81036b0 2024-09-16T01:45:57.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84fd0b5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f810c170 0x7f84f819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:57.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84effff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84f81036b0 0x7f84f81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:57.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84fc8b4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84f8102d90 0x7f84f8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:57.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84fd0b5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f810c170 0x7f84f819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51946/0 (socket says 172.21.15.139:51946) 2024-09-16T01:45:57.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84fc8b4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84f8102d90 0x7f84f8198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:50774/0 (socket says 172.21.15.139:50774) 2024-09-16T01:45:57.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84effff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84f81036b0 0x7f84f81989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:56794/0 (socket says 172.21.15.139:56794) 2024-09-16T01:45:57.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84fd0b5640 1 -- 172.21.15.139:0/1289682449 learned_addr learned my addr 172.21.15.139:0/1289682449 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:57.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84fd0b5640 1 -- 172.21.15.139:0/1289682449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84f81036b0 msgr2=0x7f84f81989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:57.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84fd0b5640 1 --2- 172.21.15.139:0/1289682449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84f81036b0 0x7f84f81989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:57.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84fd0b5640 1 -- 172.21.15.139:0/1289682449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84f8102d90 msgr2=0x7f84f8198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:57.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84fd0b5640 1 --2- 172.21.15.139:0/1289682449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84f8102d90 0x7f84f8198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:57.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84fd0b5640 1 -- 172.21.15.139:0/1289682449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f819d420 con 0x7f84f810c170 2024-09-16T01:45:57.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84effff640 1 --2- 172.21.15.139:0/1289682449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84f81036b0 0x7f84f81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:45:57.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84fc8b4640 1 --2- 172.21.15.139:0/1289682449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84f8102d90 0x7f84f8198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:45:57.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84fd0b5640 1 --2- 172.21.15.139:0/1289682449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f810c170 0x7f84f819cd40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009dc0 tx=0x7f84f0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:57.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.328+0000 7f84edffb640 1 -- 172.21.15.139:0/1289682449 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0034040 con 0x7f84f810c170 2024-09-16T01:45:57.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.329+0000 7f84feb18640 1 -- 172.21.15.139:0/1289682449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f819d6b0 con 0x7f84f810c170 2024-09-16T01:45:57.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.329+0000 7f84feb18640 1 -- 172.21.15.139:0/1289682449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f81a4ff0 con 0x7f84f810c170 2024-09-16T01:45:57.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.329+0000 7f84edffb640 1 -- 172.21.15.139:0/1289682449 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f002f940 con 0x7f84f810c170 2024-09-16T01:45:57.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.329+0000 7f84edffb640 1 -- 172.21.15.139:0/1289682449 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0004600 con 0x7f84f810c170 2024-09-16T01:45:57.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.330+0000 7f84feb18640 1 -- 172.21.15.139:0/1289682449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e4005290 con 0x7f84f810c170 2024-09-16T01:45:57.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.331+0000 7f84edffb640 1 -- 172.21.15.139:0/1289682449 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f84f0037020 con 0x7f84f810c170 2024-09-16T01:45:57.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.332+0000 7f84edffb640 1 --2- 172.21.15.139:0/1289682449 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f84e00a33b0 0x7f84e00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:57.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.332+0000 7f84fc8b4640 1 --2- 172.21.15.139:0/1289682449 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f84e00a33b0 0x7f84e00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:57.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.332+0000 7f84edffb640 1 -- 172.21.15.139:0/1289682449 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f84f00f0b40 con 0x7f84f810c170 2024-09-16T01:45:57.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.333+0000 7f84fc8b4640 1 --2- 172.21.15.139:0/1289682449 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f84e00a33b0 0x7f84e00a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009fc0 tx=0x7f84f4005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:57.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.337+0000 7f84edffb640 1 -- 172.21.15.139:0/1289682449 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84f00bd280 con 0x7f84f810c170 2024-09-16T01:45:57.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.589+0000 7f84feb18640 1 -- 172.21.15.139:0/1289682449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84e4005580 con 0x7f84f810c170 2024-09-16T01:45:57.592 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:57 smithi139 ceph-mon[31175]: pgmap v169: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 824 op/s 2024-09-16T01:45:57.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.590+0000 7f84edffb640 1 -- 172.21.15.139:0/1289682449 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+12016 (secure 0 0 0) 0x7f84f00bcb80 con 0x7f84f810c170 2024-09-16T01:45:57.593 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:45:57.593 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":35,"btime":"2024-09-16T01:45:19:161714+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:45:18.328499+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:45:57.593 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 35 2024-09-16T01:45:57.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.593+0000 7f84feb18640 1 -- 172.21.15.139:0/1289682449 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f84e00a33b0 msgr2=0x7f84e00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:57.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.593+0000 7f84feb18640 1 --2- 172.21.15.139:0/1289682449 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f84e00a33b0 0x7f84e00a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009fc0 tx=0x7f84f4005f90 comp rx=0 tx=0).stop 2024-09-16T01:45:57.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.593+0000 7f84feb18640 1 -- 172.21.15.139:0/1289682449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f810c170 msgr2=0x7f84f819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:57.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.593+0000 7f84feb18640 1 --2- 172.21.15.139:0/1289682449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f810c170 0x7f84f819cd40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009dc0 tx=0x7f84f0003c10 comp rx=0 tx=0).stop 2024-09-16T01:45:57.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.593+0000 7f84feb18640 1 -- 172.21.15.139:0/1289682449 shutdown_connections 2024-09-16T01:45:57.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.593+0000 7f84feb18640 1 --2- 172.21.15.139:0/1289682449 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f84e00a33b0 0x7f84e00a5870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:57.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.593+0000 7f84feb18640 1 --2- 172.21.15.139:0/1289682449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f810c170 0x7f84f819cd40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:57.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.593+0000 7f84feb18640 1 --2- 172.21.15.139:0/1289682449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84f81036b0 0x7f84f81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:57.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.593+0000 7f84feb18640 1 --2- 172.21.15.139:0/1289682449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84f8102d90 0x7f84f8198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:57.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.593+0000 7f84feb18640 1 -- 172.21.15.139:0/1289682449 >> 172.21.15.139:0/1289682449 conn(0x7f84f80fc7d0 msgr2=0x7f84f80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:57.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.593+0000 7f84feb18640 1 -- 172.21.15.139:0/1289682449 shutdown_connections 2024-09-16T01:45:57.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:57.594+0000 7f84feb18640 1 -- 172.21.15.139:0/1289682449 wait complete. 2024-09-16T01:45:57.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:57 smithi176 ceph-mon[35324]: pgmap v169: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 824 op/s 2024-09-16T01:45:57.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:57 smithi164 ceph-mon[35121]: pgmap v169: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 824 op/s 2024-09-16T01:45:58.241 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 2dbb8969 --release --await' 2024-09-16T01:45:58.242 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 2dbb8969 --release --await 2024-09-16T01:45:58.528 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:45:58.653 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:58 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1289682449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:45:58.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:58 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1289682449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:45:58.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:58 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1289682449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:45:59.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.068+0000 7f6599540640 1 -- 172.21.15.139:0/532520677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6594106a90 msgr2=0x7f6594106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:59.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.068+0000 7f6599540640 1 --2- 172.21.15.139:0/532520677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6594106a90 0x7f6594106e70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6584009930 tx=0x7f658402a420 comp rx=0 tx=0).stop 2024-09-16T01:45:59.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.069+0000 7f6599540640 1 -- 172.21.15.139:0/532520677 shutdown_connections 2024-09-16T01:45:59.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.069+0000 7f6599540640 1 --2- 172.21.15.139:0/532520677 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6594101430 0x7f659410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.069+0000 7f6599540640 1 --2- 172.21.15.139:0/532520677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6594100a90 0x7f6594100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.069+0000 7f6599540640 1 --2- 172.21.15.139:0/532520677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6594106a90 0x7f6594106e70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.069+0000 7f6599540640 1 -- 172.21.15.139:0/532520677 >> 172.21.15.139:0/532520677 conn(0x7f65940fc7d0 msgr2=0x7f65940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:59.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.069+0000 7f6599540640 1 -- 172.21.15.139:0/532520677 shutdown_connections 2024-09-16T01:45:59.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.069+0000 7f6599540640 1 -- 172.21.15.139:0/532520677 wait complete. 2024-09-16T01:45:59.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.070+0000 7f6599540640 1 Processor -- start 2024-09-16T01:45:59.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.070+0000 7f6599540640 1 -- start start 2024-09-16T01:45:59.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.070+0000 7f6599540640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6594100a90 0x7f6594198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.070+0000 7f6599540640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6594101430 0x7f6594198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.070+0000 7f6599540640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6594106a90 0x7f659419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.070+0000 7f6599540640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6594110e90 con 0x7f6594100a90 2024-09-16T01:45:59.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.070+0000 7f6599540640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6594110d10 con 0x7f6594101430 2024-09-16T01:45:59.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.070+0000 7f6599540640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6594111010 con 0x7f6594106a90 2024-09-16T01:45:59.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.070+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6594100a90 0x7f6594198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.070+0000 7f65937fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6594106a90 0x7f659419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.070+0000 7f65927fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6594101430 0x7f6594198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.071+0000 7f65927fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6594101430 0x7f6594198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:50804/0 (socket says 172.21.15.139:50804) 2024-09-16T01:45:59.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.071+0000 7f65927fc640 1 -- 172.21.15.139:0/1974942628 learned_addr learned my addr 172.21.15.139:0/1974942628 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.071+0000 7f65937fe640 1 -- 172.21.15.139:0/1974942628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6594101430 msgr2=0x7f6594198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.071+0000 7f65937fe640 1 --2- 172.21.15.139:0/1974942628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6594101430 0x7f6594198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.071+0000 7f65937fe640 1 -- 172.21.15.139:0/1974942628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6594100a90 msgr2=0x7f6594198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.071+0000 7f65937fe640 1 --2- 172.21.15.139:0/1974942628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6594100a90 0x7f6594198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.071+0000 7f65937fe640 1 -- 172.21.15.139:0/1974942628 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659419d3f0 con 0x7f6594106a90 2024-09-16T01:45:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.071+0000 7f6592ffd640 1 --2- 172.21.15.139:0/1974942628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6594100a90 0x7f6594198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:45:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.071+0000 7f65937fe640 1 --2- 172.21.15.139:0/1974942628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6594106a90 0x7f659419cd10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f658c009fd0 tx=0x7f658c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.071+0000 7f657bfff640 1 -- 172.21.15.139:0/1974942628 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c00ce20 con 0x7f6594106a90 2024-09-16T01:45:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.071+0000 7f6599540640 1 -- 172.21.15.139:0/1974942628 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659419d6e0 con 0x7f6594106a90 2024-09-16T01:45:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.072+0000 7f657bfff640 1 -- 172.21.15.139:0/1974942628 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f658c010430 con 0x7f6594106a90 2024-09-16T01:45:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.072+0000 7f6599540640 1 -- 172.21.15.139:0/1974942628 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65941a4fc0 con 0x7f6594106a90 2024-09-16T01:45:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.072+0000 7f657bfff640 1 -- 172.21.15.139:0/1974942628 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c00f620 con 0x7f6594106a90 2024-09-16T01:45:59.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.075+0000 7f657bfff640 1 -- 172.21.15.139:0/1974942628 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f658c003970 con 0x7f6594106a90 2024-09-16T01:45:59.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.075+0000 7f659b919740 1 Processor -- start 2024-09-16T01:45:59.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.075+0000 7f659b919740 1 -- start start 2024-09-16T01:45:59.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.075+0000 7f659b919740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe4c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.075+0000 7f657bfff640 1 --2- 172.21.15.139:0/1974942628 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f657c0a33b0 0x7f657c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.075+0000 7f659b919740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4c880 0x55d2dbe5b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.075+0000 7f659b919740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe5b650 0x55d2dbe5da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.075+0000 7f659b919740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d2dbdb1620 con 0x55d2dbe4bf60 2024-09-16T01:45:59.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.075+0000 7f659b919740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d2dbd58e50 con 0x55d2dbe5b650 2024-09-16T01:45:59.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.075+0000 7f659b919740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d2dbdad5e0 con 0x55d2dbe4c880 2024-09-16T01:45:59.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.076+0000 7f65937fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe5b650 0x55d2dbe5da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.076+0000 7f65937fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe5b650 0x55d2dbe5da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:50816/0 (socket says 172.21.15.139:50816) 2024-09-16T01:45:59.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.076+0000 7f65937fe640 1 -- 172.21.15.139:0/2566627283 learned_addr learned my addr 172.21.15.139:0/2566627283 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:59.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.076+0000 7f6592ffd640 1 --2- 172.21.15.139:0/1974942628 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f657c0a33b0 0x7f657c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.076+0000 7f65927fc640 1 --2- 172.21.15.139:0/2566627283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe4c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.076+0000 7f65927fc640 1 --2- 172.21.15.139:0/2566627283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4c880 0x55d2dbe5b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.076+0000 7f657bfff640 1 -- 172.21.15.139:0/1974942628 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f658c0d0280 con 0x7f6594106a90 2024-09-16T01:45:59.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.076+0000 7f65937fe640 1 -- 172.21.15.139:0/2566627283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4c880 msgr2=0x55d2dbe5b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:59.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.076+0000 7f65937fe640 1 --2- 172.21.15.139:0/2566627283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4c880 0x55d2dbe5b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.076+0000 7f65937fe640 1 -- 172.21.15.139:0/2566627283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 msgr2=0x55d2dbe4c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:59.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.076+0000 7f65937fe640 1 --2- 172.21.15.139:0/2566627283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe4c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.076+0000 7f65937fe640 1 -- 172.21.15.139:0/2566627283 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2dbd49d40 con 0x55d2dbe5b650 2024-09-16T01:45:59.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.076+0000 7f65937fe640 1 --2- 172.21.15.139:0/2566627283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe5b650 0x55d2dbe5da40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f658c0cd5c0 tx=0x7f658c097e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d58d2cbef8cb951 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:59.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.076+0000 7f6592ffd640 1 --2- 172.21.15.139:0/1974942628 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f657c0a33b0 0x7f657c0a5870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6584009900 tx=0x7f65840058d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:59.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.077+0000 7f65797fa640 1 -- 172.21.15.139:0/2566627283 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c0aeed0 con 0x55d2dbe5b650 2024-09-16T01:45:59.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.077+0000 7f65797fa640 1 -- 172.21.15.139:0/2566627283 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f658c09c8b0 con 0x55d2dbe5b650 2024-09-16T01:45:59.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.077+0000 7f65797fa640 1 -- 172.21.15.139:0/2566627283 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c0aecf0 con 0x55d2dbe5b650 2024-09-16T01:45:59.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.078+0000 7f659b919740 1 -- 172.21.15.139:0/2566627283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe5b650 msgr2=0x55d2dbe5da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:59.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.078+0000 7f659b919740 1 --2- 172.21.15.139:0/2566627283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe5b650 0x55d2dbe5da40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f658c0cd5c0 tx=0x7f658c097e50 comp rx=0 tx=0).stop 2024-09-16T01:45:59.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.078+0000 7f659b919740 1 -- 172.21.15.139:0/2566627283 shutdown_connections 2024-09-16T01:45:59.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.078+0000 7f659b919740 1 --2- 172.21.15.139:0/2566627283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe5b650 0x55d2dbe5da40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.078+0000 7f659b919740 1 --2- 172.21.15.139:0/2566627283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4c880 0x55d2dbe5b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.078+0000 7f659b919740 1 --2- 172.21.15.139:0/2566627283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe4c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.078+0000 7f659b919740 1 -- 172.21.15.139:0/2566627283 >> 172.21.15.139:0/2566627283 conn(0x55d2dbe485f0 msgr2=0x55d2dbe489d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:59.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.078+0000 7f659b919740 1 -- 172.21.15.139:0/2566627283 shutdown_connections 2024-09-16T01:45:59.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.078+0000 7f659b919740 1 -- 172.21.15.139:0/2566627283 wait complete. 2024-09-16T01:45:59.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.078+0000 7f659b919740 1 -- start start 2024-09-16T01:45:59.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f659b919740 1 Processor -- start 2024-09-16T01:45:59.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f659b919740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe585a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f659b919740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4c880 0x55d2dbe58ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f659b919740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe5b650 0x55d2dbe59020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f659b919740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d2dbe5e280 con 0x55d2dbe4bf60 2024-09-16T01:45:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f659b919740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d2dbd39f10 con 0x55d2dbe5b650 2024-09-16T01:45:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f659b919740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d2dbe5e400 con 0x55d2dbe4c880 2024-09-16T01:45:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f659b919740 10 client.0 _ll_register_callbacks cb 0x55d2dbd5ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:45:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f65937fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe5b650 0x55d2dbe59020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f65937fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe5b650 0x55d2dbe59020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:50824/0 (socket says 172.21.15.139:50824) 2024-09-16T01:45:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f65937fe640 1 -- 172.21.15.139:0/1700158296 learned_addr learned my addr 172.21.15.139:0/1700158296 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f65927fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4c880 0x55d2dbe58ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51998/0 (socket says 172.21.15.139:51998) 2024-09-16T01:45:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f65937fe640 1 -- 172.21.15.139:0/1700158296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4c880 msgr2=0x55d2dbe58ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f65937fe640 1 --2- 172.21.15.139:0/1700158296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4c880 0x55d2dbe58ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f65937fe640 1 -- 172.21.15.139:0/1700158296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 msgr2=0x55d2dbe585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f65937fe640 1 --2- 172.21.15.139:0/1700158296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.079+0000 7f65937fe640 1 -- 172.21.15.139:0/1700158296 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2dbac0580 con 0x55d2dbe5b650 2024-09-16T01:45:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.080+0000 7f65937fe640 1 --2- 172.21.15.139:0/1700158296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe5b650 0x55d2dbe59020 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f658c09beb0 tx=0x7f658c0d0120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de2c8abd83e287a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.080+0000 7f6576ff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:45:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.080+0000 7f6576ff5640 1 -- 172.21.15.139:0/1700158296 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c0b5060 con 0x55d2dbe5b650 2024-09-16T01:45:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.080+0000 7f6576ff5640 1 -- 172.21.15.139:0/1700158296 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f658c0bc110 con 0x55d2dbe5b650 2024-09-16T01:45:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.080+0000 7f659b919740 1 -- 172.21.15.139:0/1700158296 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f658c09c8b0 con 0x55d2dbe5b650 2024-09-16T01:45:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.080+0000 7f6576ff5640 1 -- 172.21.15.139:0/1700158296 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f658c0a9290 con 0x55d2dbe5b650 2024-09-16T01:45:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.080+0000 7f659b919740 10 client.25607 fetch_fsmap learned FSMap version 35 2024-09-16T01:45:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.080+0000 7f659b919740 1 -- 172.21.15.139:0/1700158296 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d2dbb8a960 con 0x55d2dbe5b650 2024-09-16T01:45:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.080+0000 7f6576ff5640 1 -- 172.21.15.139:0/1700158296 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f658c09c8b0 con 0x55d2dbe5b650 2024-09-16T01:45:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.080+0000 7f6576ff5640 1 -- 172.21.15.139:0/1700158296 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7f658c01f020 con 0x55d2dbe5b650 2024-09-16T01:45:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.080+0000 7f659b919740 10 client.25607 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:45:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.080+0000 7f659b919740 10 client.25607 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:45:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.081+0000 7f659b919740 20 client.25607 populate_metadata read hostname 'smithi139' 2024-09-16T01:45:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.081+0000 7f659b919740 1 --2- 172.21.15.139:0/1700158296 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d2dbe6b1e0 0x55d2dbe6d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.081+0000 7f659b919740 4 client.25607 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:45:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.081+0000 7f659b919740 1 -- 172.21.15.139:0/1700158296 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2dbdbeb90 con 0x55d2dbe6b1e0 2024-09-16T01:45:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.081+0000 7f6592ffd640 1 --2- 172.21.15.139:0/1700158296 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d2dbe6b1e0 0x55d2dbe6d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.081+0000 7f6576ff5640 10 client.25607 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.081+0000 7f6592ffd640 1 --2- 172.21.15.139:0/1700158296 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d2dbe6b1e0 0x55d2dbe6d5d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d3ae6e3172d87e6 server_cookie=476e843e339f43ca in_seq=0 out_seq=0 2024-09-16T01:45:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.083+0000 7f6576ff5640 1 -- 172.21.15.139:0/1700158296 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2dbdbeb90 con 0x55d2dbe6b1e0 2024-09-16T01:45:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.083+0000 7f6576ff5640 10 client.25607 handle_command_reply: tid=0 2024-09-16T01:45:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.083+0000 7f659b919740 1 client.25607 shutdown 2024-09-16T01:45:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.083+0000 7f659b919740 1 -- 172.21.15.139:0/1700158296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe5b650 msgr2=0x55d2dbe59020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.083+0000 7f659b919740 1 --2- 172.21.15.139:0/1700158296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe5b650 0x55d2dbe59020 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f658c09beb0 tx=0x7f658c0d0120 comp rx=0 tx=0).stop 2024-09-16T01:45:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.083+0000 7f659b919740 1 -- 172.21.15.139:0/1700158296 shutdown_connections 2024-09-16T01:45:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.083+0000 7f659b919740 1 --2- 172.21.15.139:0/1700158296 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d2dbe6b1e0 0x55d2dbe6d5d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.083+0000 7f659b919740 1 --2- 172.21.15.139:0/1700158296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe5b650 0x55d2dbe59020 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.083+0000 7f659b919740 1 --2- 172.21.15.139:0/1700158296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4c880 0x55d2dbe58ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.083+0000 7f6576ff5640 0 client.25607 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:59.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.083+0000 7f659b919740 1 --2- 172.21.15.139:0/1700158296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.083+0000 7f659b919740 1 -- 172.21.15.139:0/1700158296 >> 172.21.15.139:0/1700158296 conn(0x55d2dbe485f0 msgr2=0x55d2dbe4e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:59.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.083+0000 7f659b919740 1 -- 172.21.15.139:0/1700158296 shutdown_connections 2024-09-16T01:45:59.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.083+0000 7f659b919740 1 -- 172.21.15.139:0/1700158296 wait complete. 2024-09-16T01:45:59.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.084+0000 7f659b919740 20 client.25607 trim_cache size 0 max 16384 2024-09-16T01:45:59.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.106+0000 7f659b919740 1 Processor -- start 2024-09-16T01:45:59.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.106+0000 7f659b919740 1 -- start start 2024-09-16T01:45:59.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.106+0000 7f659b919740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe72ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.106+0000 7f659b919740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe4c880 0x55d2dbe730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.106+0000 7f659b919740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe5b650 0x55d2dbe63710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.106+0000 7f659b919740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f658000ffa0 con 0x55d2dbe5b650 2024-09-16T01:45:59.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.106+0000 7f659b919740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6580011c60 con 0x55d2dbe4c880 2024-09-16T01:45:59.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.106+0000 7f659b919740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6580010670 con 0x55d2dbe4bf60 2024-09-16T01:45:59.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.106+0000 7f65937fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe5b650 0x55d2dbe63710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.106+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe72ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.106+0000 7f65927fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe4c880 0x55d2dbe730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.106+0000 7f65937fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe5b650 0x55d2dbe63710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52008/0 (socket says 172.21.15.139:52008) 2024-09-16T01:45:59.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.106+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe72ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:56842/0 (socket says 172.21.15.139:56842) 2024-09-16T01:45:59.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.106+0000 7f65937fe640 1 -- 172.21.15.139:0/1282749944 learned_addr learned my addr 172.21.15.139:0/1282749944 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:59.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.107+0000 7f65927fc640 1 -- 172.21.15.139:0/1282749944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4bf60 msgr2=0x55d2dbe72ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:59.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.107+0000 7f65927fc640 1 --2- 172.21.15.139:0/1282749944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe72ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.107+0000 7f65927fc640 1 -- 172.21.15.139:0/1282749944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe5b650 msgr2=0x55d2dbe63710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:59.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.107+0000 7f65927fc640 1 --2- 172.21.15.139:0/1282749944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe5b650 0x55d2dbe63710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.107+0000 7f65927fc640 1 -- 172.21.15.139:0/1282749944 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2dbe57550 con 0x55d2dbe4c880 2024-09-16T01:45:59.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.107+0000 7f65937fe640 1 --2- 172.21.15.139:0/1282749944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe5b650 0x55d2dbe63710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:45:59.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.107+0000 7f65927fc640 1 --2- 172.21.15.139:0/1282749944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe4c880 0x55d2dbe730e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55d2dbe4ec50 tx=0x7f6588004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c41cecef7f031b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:59.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.107+0000 7f6578ff9640 1 -- 172.21.15.139:0/1282749944 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658800f680 con 0x55d2dbe4c880 2024-09-16T01:45:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.107+0000 7f6578ff9640 1 -- 172.21.15.139:0/1282749944 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f658800fd00 con 0x55d2dbe4c880 2024-09-16T01:45:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.107+0000 7f6578ff9640 1 -- 172.21.15.139:0/1282749944 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658801cde0 con 0x55d2dbe4c880 2024-09-16T01:45:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.108+0000 7f659b919740 1 -- 172.21.15.139:0/1282749944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe4c880 msgr2=0x55d2dbe730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:45:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.108+0000 7f659b919740 1 --2- 172.21.15.139:0/1282749944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe4c880 0x55d2dbe730e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55d2dbe4ec50 tx=0x7f6588004730 comp rx=0 tx=0).stop 2024-09-16T01:45:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.108+0000 7f659b919740 1 -- 172.21.15.139:0/1282749944 shutdown_connections 2024-09-16T01:45:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.108+0000 7f659b919740 1 --2- 172.21.15.139:0/1282749944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe5b650 0x55d2dbe63710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.108+0000 7f659b919740 1 --2- 172.21.15.139:0/1282749944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe4c880 0x55d2dbe730e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.108+0000 7f659b919740 1 --2- 172.21.15.139:0/1282749944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe72ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.108+0000 7f659b919740 1 -- 172.21.15.139:0/1282749944 >> 172.21.15.139:0/1282749944 conn(0x55d2dbe485f0 msgr2=0x55d2dbe59e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:45:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.108+0000 7f659b919740 1 -- 172.21.15.139:0/1282749944 shutdown_connections 2024-09-16T01:45:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.108+0000 7f659b919740 1 -- 172.21.15.139:0/1282749944 wait complete. 2024-09-16T01:45:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.109+0000 7f659b919740 1 -- start start 2024-09-16T01:45:59.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.109+0000 7f659b919740 1 Processor -- start 2024-09-16T01:45:59.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.109+0000 7f659b919740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe63e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.109+0000 7f659b919740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe4c880 0x55d2dbe601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.109+0000 7f659b919740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe5b650 0x55d2dbe60710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.109+0000 7f659b919740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f65800114d0 con 0x55d2dbe4bf60 2024-09-16T01:45:59.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.109+0000 7f659b919740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6580010e00 con 0x55d2dbe4c880 2024-09-16T01:45:59.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.109+0000 7f659b919740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f658000df80 con 0x55d2dbe5b650 2024-09-16T01:45:59.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.109+0000 7f659b919740 10 client.0 _ll_register_callbacks cb 0x7f658c01ce10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:45:59.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.109+0000 7f65927fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe4c880 0x55d2dbe601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.109+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe63e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.110+0000 7f65927fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe4c880 0x55d2dbe601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:50840/0 (socket says 172.21.15.139:50840) 2024-09-16T01:45:59.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.110+0000 7f65927fc640 1 -- 172.21.15.139:0/902205740 learned_addr learned my addr 172.21.15.139:0/902205740 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:45:59.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.110+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe63e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52020/0 (socket says 172.21.15.139:52020) 2024-09-16T01:45:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.110+0000 7f65937fe640 1 --2- 172.21.15.139:0/902205740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe5b650 0x55d2dbe60710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.110+0000 7f6592ffd640 1 -- 172.21.15.139:0/902205740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe5b650 msgr2=0x55d2dbe60710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.110+0000 7f6592ffd640 1 --2- 172.21.15.139:0/902205740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe5b650 0x55d2dbe60710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.110+0000 7f6592ffd640 1 -- 172.21.15.139:0/902205740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe4c880 msgr2=0x55d2dbe601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:45:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.110+0000 7f6592ffd640 1 --2- 172.21.15.139:0/902205740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe4c880 0x55d2dbe601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:45:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.110+0000 7f6592ffd640 1 -- 172.21.15.139:0/902205740 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2dbe60f60 con 0x55d2dbe4bf60 2024-09-16T01:45:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.110+0000 7f6592ffd640 1 --2- 172.21.15.139:0/902205740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe63e90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6584005940 tx=0x7f658402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15656a3b14ddb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:45:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.110+0000 7f6576ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:45:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.110+0000 7f6576ff5640 1 -- 172.21.15.139:0/902205740 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584056070 con 0x55d2dbe4bf60 2024-09-16T01:45:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.110+0000 7f6576ff5640 1 -- 172.21.15.139:0/902205740 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f6584005220 con 0x55d2dbe4bf60 2024-09-16T01:45:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.110+0000 7f659b919740 1 -- 172.21.15.139:0/902205740 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f658800fd00 con 0x55d2dbe4bf60 2024-09-16T01:45:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.111+0000 7f6576ff5640 1 -- 172.21.15.139:0/902205740 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6584035040 con 0x55d2dbe4bf60 2024-09-16T01:45:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.111+0000 7f659b919740 10 client.16134 fetch_fsmap learned FSMap version 35 2024-09-16T01:45:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.111+0000 7f659b919740 1 -- 172.21.15.139:0/902205740 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d2dbed6c50 con 0x55d2dbe4bf60 2024-09-16T01:45:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.111+0000 7f6576ff5640 1 -- 172.21.15.139:0/902205740 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f658800fd00 con 0x55d2dbe4bf60 2024-09-16T01:45:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.111+0000 7f6576ff5640 1 -- 172.21.15.139:0/902205740 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7f6584004800 con 0x55d2dbe4bf60 2024-09-16T01:45:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.111+0000 7f659b919740 10 client.16134 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:45:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.111+0000 7f659b919740 10 client.16134 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:45:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.111+0000 7f659b919740 20 client.16134 populate_metadata read hostname 'smithi139' 2024-09-16T01:45:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.111+0000 7f659b919740 1 --2- 172.21.15.139:0/902205740 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d2dbed73e0 0x55d2dbed9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:45:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.111+0000 7f659b919740 4 client.16134 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2dbb8969", "release": true, "await": true}] 2024-09-16T01:45:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.111+0000 7f659b919740 1 -- 172.21.15.139:0/902205740 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2dbb8969", "release": true, "await": true}) -- 0x55d2dbdb90a0 con 0x55d2dbed73e0 2024-09-16T01:45:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.112+0000 7f65927fc640 1 --2- 172.21.15.139:0/902205740 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d2dbed73e0 0x55d2dbed9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:45:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.112+0000 7f65927fc640 1 --2- 172.21.15.139:0/902205740 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d2dbed73e0 0x55d2dbed9800 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e24b212ddbb599e8 server_cookie=8d427de25ba5a76a in_seq=0 out_seq=0 2024-09-16T01:45:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:45:59.112+0000 7f6576ff5640 10 client.16134 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:45:59.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:45:59 smithi139 ceph-mon[31175]: pgmap v170: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.3 MiB/s wr, 846 op/s 2024-09-16T01:45:59.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:45:59 smithi176 ceph-mon[35324]: pgmap v170: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.3 MiB/s wr, 846 op/s 2024-09-16T01:45:59.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:45:59 smithi164 ceph-mon[35121]: pgmap v170: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.3 MiB/s wr, 846 op/s 2024-09-16T01:46:00.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.031+0000 7f6576ff5640 1 -- 172.21.15.139:0/902205740 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d2dbdb90a0 con 0x55d2dbed73e0 2024-09-16T01:46:00.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.031+0000 7f6576ff5640 10 client.16134 handle_command_reply: tid=0 2024-09-16T01:46:00.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.031+0000 7f659b919740 1 client.16134 shutdown 2024-09-16T01:46:00.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.031+0000 7f659b919740 1 -- 172.21.15.139:0/902205740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 msgr2=0x55d2dbe63e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:00.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.031+0000 7f659b919740 1 --2- 172.21.15.139:0/902205740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe63e90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6584005940 tx=0x7f658402a420 comp rx=0 tx=0).stop 2024-09-16T01:46:00.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.032+0000 7f659b919740 1 -- 172.21.15.139:0/902205740 shutdown_connections 2024-09-16T01:46:00.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.032+0000 7f659b919740 1 --2- 172.21.15.139:0/902205740 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d2dbed73e0 0x55d2dbed9800 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:00.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.032+0000 7f659b919740 1 --2- 172.21.15.139:0/902205740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2dbe5b650 0x55d2dbe60710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:00.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.032+0000 7f6576ff5640 0 client.16134 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:00.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.032+0000 7f659b919740 1 --2- 172.21.15.139:0/902205740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2dbe4c880 0x55d2dbe601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:00.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.032+0000 7f659b919740 1 --2- 172.21.15.139:0/902205740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d2dbe4bf60 0x55d2dbe63e90 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:00.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.032+0000 7f659b919740 1 -- 172.21.15.139:0/902205740 >> 172.21.15.139:0/902205740 conn(0x55d2dbe485f0 msgr2=0x55d2dbe46f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:00.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.032+0000 7f659b919740 1 -- 172.21.15.139:0/902205740 shutdown_connections 2024-09-16T01:46:00.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.032+0000 7f659b919740 1 -- 172.21.15.139:0/902205740 wait complete. 2024-09-16T01:46:00.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.032+0000 7f659b919740 20 client.16134 trim_cache size 0 max 16384 2024-09-16T01:46:00.035 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:46:00.035 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 35, 2024-09-16T01:46:00.035 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:46:00.035 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 20, 2024-09-16T01:46:00.035 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:46:00.035 INFO:teuthology.orchestra.run.smithi139.stdout: "2dbb8969": { 2024-09-16T01:46:00.036 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 20, 2024-09-16T01:46:00.036 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:46:00.036 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:46:00.036 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:46:00.036 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:46:00.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:46:00.036 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:46:00.036 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 145.3, 2024-09-16T01:46:00.036 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:46:00.036 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:46:00.036 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:46:00.036 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:46:00.036 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:46:00.037 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:46:00.037 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:00.037 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:00.037 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:00.037 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:00.037 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:00.037 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:46:00.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.033+0000 7f6599540640 1 -- 172.21.15.139:0/1974942628 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f657c0a33b0 msgr2=0x7f657c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:00.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.033+0000 7f6599540640 1 --2- 172.21.15.139:0/1974942628 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f657c0a33b0 0x7f657c0a5870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6584009900 tx=0x7f65840058d0 comp rx=0 tx=0).stop 2024-09-16T01:46:00.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.033+0000 7f6599540640 1 -- 172.21.15.139:0/1974942628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6594106a90 msgr2=0x7f659419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:00.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.033+0000 7f6599540640 1 --2- 172.21.15.139:0/1974942628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6594106a90 0x7f659419cd10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f658c009fd0 tx=0x7f658c00ef10 comp rx=0 tx=0).stop 2024-09-16T01:46:00.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.033+0000 7f6599540640 1 -- 172.21.15.139:0/1974942628 shutdown_connections 2024-09-16T01:46:00.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.033+0000 7f6599540640 1 --2- 172.21.15.139:0/1974942628 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f657c0a33b0 0x7f657c0a5870 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:00.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.033+0000 7f6599540640 1 --2- 172.21.15.139:0/1974942628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6594106a90 0x7f659419cd10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:00.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.033+0000 7f6599540640 1 --2- 172.21.15.139:0/1974942628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6594101430 0x7f6594198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:00.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.033+0000 7f6599540640 1 --2- 172.21.15.139:0/1974942628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6594100a90 0x7f6594198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:00.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.033+0000 7f6599540640 1 -- 172.21.15.139:0/1974942628 >> 172.21.15.139:0/1974942628 conn(0x7f65940fc7d0 msgr2=0x7f659406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:00.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.033+0000 7f6599540640 1 -- 172.21.15.139:0/1974942628 shutdown_connections 2024-09-16T01:46:00.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:00.033+0000 7f6599540640 1 -- 172.21.15.139:0/1974942628 wait complete. 2024-09-16T01:46:00.782 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2dbb8969', total seconds elapsed: 29.6 2024-09-16T01:46:00.782 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-09-16T01:46:00.782 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:46:01.093 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:46:01.521 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:01 smithi139 ceph-mon[31175]: pgmap v171: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 819 op/s 2024-09-16T01:46:01.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.621+0000 7fde0dce6640 1 -- 172.21.15.139:0/537681620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde08100930 msgr2=0x7fde08100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:01.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.621+0000 7fde0dce6640 1 --2- 172.21.15.139:0/537681620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde08100930 0x7fde08100d10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009900 tx=0x7fddf802a410 comp rx=0 tx=0).stop 2024-09-16T01:46:01.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.621+0000 7fde0dce6640 1 -- 172.21.15.139:0/537681620 shutdown_connections 2024-09-16T01:46:01.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.621+0000 7fde0dce6640 1 --2- 172.21.15.139:0/537681620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde0810e4e0 0x7fde081108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:01.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.621+0000 7fde0dce6640 1 --2- 172.21.15.139:0/537681620 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde081012e0 0x7fde0810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:01.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.621+0000 7fde0dce6640 1 --2- 172.21.15.139:0/537681620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde08100930 0x7fde08100d10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:01.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.621+0000 7fde0dce6640 1 -- 172.21.15.139:0/537681620 >> 172.21.15.139:0/537681620 conn(0x7fde080fc7a0 msgr2=0x7fde080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:01.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.621+0000 7fde0dce6640 1 -- 172.21.15.139:0/537681620 shutdown_connections 2024-09-16T01:46:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.621+0000 7fde0dce6640 1 -- 172.21.15.139:0/537681620 wait complete. 2024-09-16T01:46:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.622+0000 7fde0dce6640 1 Processor -- start 2024-09-16T01:46:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.622+0000 7fde0dce6640 1 -- start start 2024-09-16T01:46:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.622+0000 7fde0dce6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde08100930 0x7fde0819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.622+0000 7fde0dce6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde081012e0 0x7fde0819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.622+0000 7fde0dce6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde0810e4e0 0x7fde0819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.622+0000 7fde0dce6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fde08114190 con 0x7fde08100930 2024-09-16T01:46:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.622+0000 7fde0dce6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fde08114010 con 0x7fde081012e0 2024-09-16T01:46:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.622+0000 7fde0dce6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fde08114310 con 0x7fde0810e4e0 2024-09-16T01:46:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.623+0000 7fde06ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde081012e0 0x7fde0819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.623+0000 7fde07fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde0810e4e0 0x7fde0819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.623+0000 7fde06ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde081012e0 0x7fde0819abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:50852/0 (socket says 172.21.15.139:50852) 2024-09-16T01:46:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.623+0000 7fde06ffd640 1 -- 172.21.15.139:0/2592521251 learned_addr learned my addr 172.21.15.139:0/2592521251 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.623+0000 7fde077fe640 1 --2- 172.21.15.139:0/2592521251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde08100930 0x7fde0819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:01.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.623+0000 7fde06ffd640 1 -- 172.21.15.139:0/2592521251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde0810e4e0 msgr2=0x7fde0819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:01.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.623+0000 7fde06ffd640 1 --2- 172.21.15.139:0/2592521251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde0810e4e0 0x7fde0819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:01.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.623+0000 7fde06ffd640 1 -- 172.21.15.139:0/2592521251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde08100930 msgr2=0x7fde0819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:01.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.623+0000 7fde06ffd640 1 --2- 172.21.15.139:0/2592521251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde08100930 0x7fde0819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:01.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.623+0000 7fde06ffd640 1 -- 172.21.15.139:0/2592521251 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0819f660 con 0x7fde081012e0 2024-09-16T01:46:01.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.623+0000 7fde07fff640 1 --2- 172.21.15.139:0/2592521251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde0810e4e0 0x7fde0819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:46:01.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.623+0000 7fde06ffd640 1 --2- 172.21.15.139:0/2592521251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde081012e0 0x7fde0819abf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00e9f0 tx=0x7fddfc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:01.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.624+0000 7fde04ff9640 1 -- 172.21.15.139:0/2592521251 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00cd80 con 0x7fde081012e0 2024-09-16T01:46:01.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.624+0000 7fde04ff9640 1 -- 172.21.15.139:0/2592521251 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddfc005500 con 0x7fde081012e0 2024-09-16T01:46:01.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.624+0000 7fde0dce6640 1 -- 172.21.15.139:0/2592521251 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0819f950 con 0x7fde081012e0 2024-09-16T01:46:01.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.624+0000 7fde04ff9640 1 -- 172.21.15.139:0/2592521251 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc010690 con 0x7fde081012e0 2024-09-16T01:46:01.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.624+0000 7fde0dce6640 1 -- 172.21.15.139:0/2592521251 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde08102b40 con 0x7fde081012e0 2024-09-16T01:46:01.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.625+0000 7fde0dce6640 1 -- 172.21.15.139:0/2592521251 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde08100d10 con 0x7fde081012e0 2024-09-16T01:46:01.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.627+0000 7fde04ff9640 1 -- 172.21.15.139:0/2592521251 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fddfc0046c0 con 0x7fde081012e0 2024-09-16T01:46:01.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.627+0000 7fde04ff9640 1 --2- 172.21.15.139:0/2592521251 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fddf00a3430 0x7fddf00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:01.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.628+0000 7fde077fe640 1 --2- 172.21.15.139:0/2592521251 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fddf00a3430 0x7fddf00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:01.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.628+0000 7fde04ff9640 1 -- 172.21.15.139:0/2592521251 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fddfc0d0660 con 0x7fde081012e0 2024-09-16T01:46:01.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.628+0000 7fde077fe640 1 --2- 172.21.15.139:0/2592521251 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fddf00a3430 0x7fddf00a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009230 tx=0x7fddf8040040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:01.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.634+0000 7fde04ff9640 1 -- 172.21.15.139:0/2592521251 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fddfc0541b0 con 0x7fde081012e0 2024-09-16T01:46:01.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:01 smithi176 ceph-mon[35324]: pgmap v171: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 819 op/s 2024-09-16T01:46:01.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:01 smithi164 ceph-mon[35121]: pgmap v171: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 819 op/s 2024-09-16T01:46:01.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.893+0000 7fde0dce6640 1 -- 172.21.15.139:0/2592521251 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde0819ba10 con 0x7fde081012e0 2024-09-16T01:46:01.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.894+0000 7fde04ff9640 1 -- 172.21.15.139:0/2592521251 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+12016 (secure 0 0 0) 0x7fde0819ba10 con 0x7fde081012e0 2024-09-16T01:46:01.896 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:46:01.897 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":35,"btime":"2024-09-16T01:45:19:161714+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:45:18.328499+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:46:01.897 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 35 2024-09-16T01:46:01.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.897+0000 7fde0dce6640 1 -- 172.21.15.139:0/2592521251 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fddf00a3430 msgr2=0x7fddf00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:01.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.897+0000 7fde0dce6640 1 --2- 172.21.15.139:0/2592521251 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fddf00a3430 0x7fddf00a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009230 tx=0x7fddf8040040 comp rx=0 tx=0).stop 2024-09-16T01:46:01.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.897+0000 7fde0dce6640 1 -- 172.21.15.139:0/2592521251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde081012e0 msgr2=0x7fde0819abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:01.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.897+0000 7fde0dce6640 1 --2- 172.21.15.139:0/2592521251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde081012e0 0x7fde0819abf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00e9f0 tx=0x7fddfc00edf0 comp rx=0 tx=0).stop 2024-09-16T01:46:01.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.897+0000 7fde0dce6640 1 -- 172.21.15.139:0/2592521251 shutdown_connections 2024-09-16T01:46:01.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.897+0000 7fde0dce6640 1 --2- 172.21.15.139:0/2592521251 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fddf00a3430 0x7fddf00a58f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:01.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.897+0000 7fde0dce6640 1 --2- 172.21.15.139:0/2592521251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde0810e4e0 0x7fde0819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:01.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.897+0000 7fde0dce6640 1 --2- 172.21.15.139:0/2592521251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde081012e0 0x7fde0819abf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:01.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.897+0000 7fde0dce6640 1 --2- 172.21.15.139:0/2592521251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde08100930 0x7fde0819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:01.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.897+0000 7fde0dce6640 1 -- 172.21.15.139:0/2592521251 >> 172.21.15.139:0/2592521251 conn(0x7fde080fc7a0 msgr2=0x7fde080feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:01.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.897+0000 7fde0dce6640 1 -- 172.21.15.139:0/2592521251 shutdown_connections 2024-09-16T01:46:01.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:01.897+0000 7fde0dce6640 1 -- 172.21.15.139:0/2592521251 wait complete. 2024-09-16T01:46:02.607 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0' 2024-09-16T01:46:02.607 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0 2024-09-16T01:46:02.631 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:02 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2592521251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:46:02.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:02 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2592521251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:46:02.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:02 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2592521251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:46:02.893 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:46:03.548 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:03 smithi139 ceph-mon[31175]: pgmap v172: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.4 MiB/s wr, 600 op/s 2024-09-16T01:46:03.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.553+0000 7f71b77fe640 1 -- 172.21.15.139:0/4211932692 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a40319b0 con 0x7f71b8103740 2024-09-16T01:46:03.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.554+0000 7f71becec640 1 -- 172.21.15.139:0/4211932692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71b8103740 msgr2=0x7f71b8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:03.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.554+0000 7f71becec640 1 --2- 172.21.15.139:0/4211932692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71b8103740 0x7f71b8107b80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009930 tx=0x7f71a402a420 comp rx=0 tx=0).stop 2024-09-16T01:46:03.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.554+0000 7f71becec640 1 -- 172.21.15.139:0/4211932692 shutdown_connections 2024-09-16T01:46:03.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.554+0000 7f71becec640 1 --2- 172.21.15.139:0/4211932692 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b8108290 0x7f71b810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.554+0000 7f71becec640 1 --2- 172.21.15.139:0/4211932692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71b8103740 0x7f71b8107b80 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.554+0000 7f71becec640 1 --2- 172.21.15.139:0/4211932692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71b8102d90 0x7f71b8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.554+0000 7f71becec640 1 -- 172.21.15.139:0/4211932692 >> 172.21.15.139:0/4211932692 conn(0x7f71b80fc7d0 msgr2=0x7f71b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:03.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.554+0000 7f71becec640 1 -- 172.21.15.139:0/4211932692 shutdown_connections 2024-09-16T01:46:03.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.554+0000 7f71becec640 1 -- 172.21.15.139:0/4211932692 wait complete. 2024-09-16T01:46:03.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.555+0000 7f71becec640 1 Processor -- start 2024-09-16T01:46:03.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.555+0000 7f71becec640 1 -- start start 2024-09-16T01:46:03.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.555+0000 7f71becec640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71b8102d90 0x7f71b81967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.555+0000 7f71becec640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b8103740 0x7f71b8196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.555+0000 7f71becec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71b8108290 0x7f71b819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71becec640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f71b8101080 con 0x7f71b8108290 2024-09-16T01:46:03.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71becec640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71b8100f00 con 0x7f71b8103740 2024-09-16T01:46:03.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71becec640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f71b8101200 con 0x7f71b8102d90 2024-09-16T01:46:03.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71bca88640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71b8102d90 0x7f71b81967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71b7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b8103740 0x7f71b8196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71bca88640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71b8102d90 0x7f71b81967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:55164/0 (socket says 172.21.15.139:55164) 2024-09-16T01:46:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71bca88640 1 -- 172.21.15.139:0/1055128424 learned_addr learned my addr 172.21.15.139:0/1055128424 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71b7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b8103740 0x7f71b8196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44782/0 (socket says 172.21.15.139:44782) 2024-09-16T01:46:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71bd289640 1 --2- 172.21.15.139:0/1055128424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71b8108290 0x7f71b819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71b7fff640 1 -- 172.21.15.139:0/1055128424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71b8102d90 msgr2=0x7f71b81967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71b7fff640 1 --2- 172.21.15.139:0/1055128424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71b8102d90 0x7f71b81967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71b7fff640 1 -- 172.21.15.139:0/1055128424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71b8108290 msgr2=0x7f71b819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71b7fff640 1 --2- 172.21.15.139:0/1055128424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71b8108290 0x7f71b819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71b7fff640 1 -- 172.21.15.139:0/1055128424 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b819b3f0 con 0x7f71b8103740 2024-09-16T01:46:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71bd289640 1 --2- 172.21.15.139:0/1055128424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71b8108290 0x7f71b819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:46:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.556+0000 7f71b7fff640 1 --2- 172.21.15.139:0/1055128424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b8103740 0x7f71b8196cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009230 tx=0x7f71a4031b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.557+0000 7f71b5ffb640 1 -- 172.21.15.139:0/1055128424 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a403f070 con 0x7f71b8103740 2024-09-16T01:46:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.557+0000 7f71becec640 1 -- 172.21.15.139:0/1055128424 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b819b680 con 0x7f71b8103740 2024-09-16T01:46:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.557+0000 7f71b5ffb640 1 -- 172.21.15.139:0/1055128424 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a4003ad0 con 0x7f71b8103740 2024-09-16T01:46:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.557+0000 7f71b5ffb640 1 -- 172.21.15.139:0/1055128424 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4035070 con 0x7f71b8103740 2024-09-16T01:46:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.557+0000 7f71becec640 1 -- 172.21.15.139:0/1055128424 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b81a2fc0 con 0x7f71b8103740 2024-09-16T01:46:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.560+0000 7f71c10c5740 1 Processor -- start 2024-09-16T01:46:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.560+0000 7f71b5ffb640 1 -- 172.21.15.139:0/1055128424 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f71a40035f0 con 0x7f71b8103740 2024-09-16T01:46:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.560+0000 7f71c10c5740 1 -- start start 2024-09-16T01:46:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71c10c5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b8850 0x55a4832b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71c10c5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832b9170 0x55a4832c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71c10c5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832c3a30 0x55a4832c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71c10c5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a48321d160 con 0x55a4832c3a30 2024-09-16T01:46:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71c10c5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a4831c4720 con 0x55a4832b9170 2024-09-16T01:46:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71c10c5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a483219120 con 0x55a4832b8850 2024-09-16T01:46:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71bca88640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832b9170 0x55a4832c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71bd289640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b8850 0x55a4832b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71bca88640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832b9170 0x55a4832c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44796/0 (socket says 172.21.15.139:44796) 2024-09-16T01:46:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71bd289640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b8850 0x55a4832b8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:55170/0 (socket says 172.21.15.139:55170) 2024-09-16T01:46:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71bca88640 1 -- 172.21.15.139:0/3090642543 learned_addr learned my addr 172.21.15.139:0/3090642543 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71b7fff640 1 --2- 172.21.15.139:0/3090642543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832c3a30 0x55a4832c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71bca88640 1 -- 172.21.15.139:0/3090642543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b8850 msgr2=0x55a4832b8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71bca88640 1 --2- 172.21.15.139:0/3090642543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b8850 0x55a4832b8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71bca88640 1 -- 172.21.15.139:0/3090642543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832c3a30 msgr2=0x55a4832c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:03.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.561+0000 7f71b5ffb640 1 --2- 172.21.15.139:0/1055128424 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f71a80a7840 0x7f71a80a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.562+0000 7f71bd289640 1 --2- 172.21.15.139:0/1055128424 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f71a80a7840 0x7f71a80a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.562+0000 7f71bca88640 1 --2- 172.21.15.139:0/3090642543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832c3a30 0x55a4832c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.562+0000 7f71bca88640 1 -- 172.21.15.139:0/3090642543 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a482f8eba0 con 0x55a4832b9170 2024-09-16T01:46:03.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.562+0000 7f71b5ffb640 1 -- 172.21.15.139:0/1055128424 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f71a40f0490 con 0x7f71b8103740 2024-09-16T01:46:03.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.562+0000 7f71bca88640 1 --2- 172.21.15.139:0/3090642543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832b9170 0x55a4832c34f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009dc0 tx=0x7f71ac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f71563a45e2fcaa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:03.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.562+0000 7f71b7fff640 1 --2- 172.21.15.139:0/3090642543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832c3a30 0x55a4832c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:46:03.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.562+0000 7f71a2ffd640 1 -- 172.21.15.139:0/3090642543 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac005220 con 0x55a4832b9170 2024-09-16T01:46:03.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.562+0000 7f71bd289640 1 --2- 172.21.15.139:0/1055128424 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f71a80a7840 0x7f71a80a9d00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009fd0 tx=0x7f71b0009500 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:03.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.562+0000 7f71a2ffd640 1 -- 172.21.15.139:0/3090642543 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac004c80 con 0x55a4832b9170 2024-09-16T01:46:03.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.562+0000 7f71a2ffd640 1 -- 172.21.15.139:0/3090642543 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac019970 con 0x55a4832b9170 2024-09-16T01:46:03.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.563+0000 7f71c10c5740 1 -- 172.21.15.139:0/3090642543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832b9170 msgr2=0x55a4832c34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:03.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.563+0000 7f71c10c5740 1 --2- 172.21.15.139:0/3090642543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832b9170 0x55a4832c34f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009dc0 tx=0x7f71ac011810 comp rx=0 tx=0).stop 2024-09-16T01:46:03.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.563+0000 7f71c10c5740 1 -- 172.21.15.139:0/3090642543 shutdown_connections 2024-09-16T01:46:03.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.563+0000 7f71c10c5740 1 --2- 172.21.15.139:0/3090642543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832c3a30 0x55a4832c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.563+0000 7f71c10c5740 1 --2- 172.21.15.139:0/3090642543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832b9170 0x55a4832c34f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.563+0000 7f71c10c5740 1 --2- 172.21.15.139:0/3090642543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b8850 0x55a4832b8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.563+0000 7f71c10c5740 1 -- 172.21.15.139:0/3090642543 >> 172.21.15.139:0/3090642543 conn(0x55a4832b4de0 msgr2=0x55a4832b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:03.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.564+0000 7f71c10c5740 1 -- 172.21.15.139:0/3090642543 shutdown_connections 2024-09-16T01:46:03.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.564+0000 7f71c10c5740 1 -- 172.21.15.139:0/3090642543 wait complete. 2024-09-16T01:46:03.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.564+0000 7f71c10c5740 1 -- start start 2024-09-16T01:46:03.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.564+0000 7f71c10c5740 1 Processor -- start 2024-09-16T01:46:03.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71c10c5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832b8850 0x55a4832c0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71c10c5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b9170 0x55a4832c0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71c10c5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832c3a30 0x55a4832c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71c10c5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a4832c6600 con 0x55a4832b9170 2024-09-16T01:46:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71c10c5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a4831a5900 con 0x55a4832b8850 2024-09-16T01:46:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71c10c5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a4832c6780 con 0x55a4832c3a30 2024-09-16T01:46:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71c10c5740 10 client.0 _ll_register_callbacks cb 0x55a4831a3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:46:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71b7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b9170 0x55a4832c0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71bd289640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832c3a30 0x55a4832c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71bca88640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832b8850 0x55a4832c0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71bd289640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832c3a30 0x55a4832c13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:55178/0 (socket says 172.21.15.139:55178) 2024-09-16T01:46:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71b7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b9170 0x55a4832c0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48514/0 (socket says 172.21.15.139:48514) 2024-09-16T01:46:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71bca88640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832b8850 0x55a4832c0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44812/0 (socket says 172.21.15.139:44812) 2024-09-16T01:46:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71bd289640 1 -- 172.21.15.139:0/3159212287 learned_addr learned my addr 172.21.15.139:0/3159212287 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71bca88640 1 -- 172.21.15.139:0/3159212287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832c3a30 msgr2=0x55a4832c13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71bca88640 1 --2- 172.21.15.139:0/3159212287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832c3a30 0x55a4832c13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71bca88640 1 -- 172.21.15.139:0/3159212287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b9170 msgr2=0x55a4832c0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71bca88640 1 --2- 172.21.15.139:0/3159212287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b9170 0x55a4832c0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71bca88640 1 -- 172.21.15.139:0/3159212287 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4831e5c50 con 0x55a4832b8850 2024-09-16T01:46:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f71bca88640 1 --2- 172.21.15.139:0/3159212287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832b8850 0x55a4832c0960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71ac019470 tx=0x7f71ac005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23aa89ef93e2159d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.565+0000 7f7197fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:46:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.566+0000 7f7197fff640 1 -- 172.21.15.139:0/3159212287 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0099d0 con 0x55a4832b8850 2024-09-16T01:46:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.566+0000 7f71c10c5740 1 -- 172.21.15.139:0/3159212287 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ac004c80 con 0x55a4832b8850 2024-09-16T01:46:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.566+0000 7f7197fff640 1 -- 172.21.15.139:0/3159212287 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f71ac004250 con 0x55a4832b8850 2024-09-16T01:46:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.566+0000 7f7197fff640 1 -- 172.21.15.139:0/3159212287 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f71ac032050 con 0x55a4832b8850 2024-09-16T01:46:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.566+0000 7f71c10c5740 10 client.25631 fetch_fsmap learned FSMap version 35 2024-09-16T01:46:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.566+0000 7f71c10c5740 1 -- 172.21.15.139:0/3159212287 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a4832d2270 con 0x55a4832b8850 2024-09-16T01:46:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.566+0000 7f7197fff640 1 -- 172.21.15.139:0/3159212287 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac004c80 con 0x55a4832b8850 2024-09-16T01:46:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.566+0000 7f7197fff640 1 -- 172.21.15.139:0/3159212287 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7f71ac019cd0 con 0x55a4832b8850 2024-09-16T01:46:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.566+0000 7f71c10c5740 10 client.25631 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:46:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.566+0000 7f71c10c5740 10 client.25631 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:46:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.566+0000 7f71c10c5740 20 client.25631 populate_metadata read hostname 'smithi139' 2024-09-16T01:46:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.566+0000 7f71c10c5740 1 --2- 172.21.15.139:0/3159212287 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a4832d3680 0x55a4832d5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.566+0000 7f71c10c5740 4 client.25631 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:46:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.566+0000 7f71c10c5740 1 -- 172.21.15.139:0/3159212287 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a48322a890 con 0x55a4832d3680 2024-09-16T01:46:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.567+0000 7f71b7fff640 1 --2- 172.21.15.139:0/3159212287 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a4832d3680 0x55a4832d5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.567+0000 7f7197fff640 10 client.25631 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.567+0000 7f71b7fff640 1 --2- 172.21.15.139:0/3159212287 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a4832d3680 0x55a4832d5a70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57d9346b0a9d48a3 server_cookie=c00b89a16ac68104 in_seq=0 out_seq=0 2024-09-16T01:46:03.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.568+0000 7f7197fff640 1 -- 172.21.15.139:0/3159212287 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a48322a890 con 0x55a4832d3680 2024-09-16T01:46:03.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.568+0000 7f7197fff640 10 client.25631 handle_command_reply: tid=0 2024-09-16T01:46:03.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.568+0000 7f71c10c5740 1 client.25631 shutdown 2024-09-16T01:46:03.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.569+0000 7f71c10c5740 1 -- 172.21.15.139:0/3159212287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832b8850 msgr2=0x55a4832c0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:03.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.569+0000 7f71c10c5740 1 --2- 172.21.15.139:0/3159212287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832b8850 0x55a4832c0960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71ac019470 tx=0x7f71ac005230 comp rx=0 tx=0).stop 2024-09-16T01:46:03.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.569+0000 7f71c10c5740 1 -- 172.21.15.139:0/3159212287 shutdown_connections 2024-09-16T01:46:03.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.569+0000 7f71c10c5740 1 --2- 172.21.15.139:0/3159212287 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a4832d3680 0x55a4832d5a70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.569+0000 7f71c10c5740 1 --2- 172.21.15.139:0/3159212287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832c3a30 0x55a4832c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.569+0000 7f71c10c5740 1 --2- 172.21.15.139:0/3159212287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b9170 0x55a4832c0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.569+0000 7f7197fff640 0 client.25631 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:03.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.569+0000 7f71c10c5740 1 --2- 172.21.15.139:0/3159212287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832b8850 0x55a4832c0960 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.569+0000 7f71c10c5740 1 -- 172.21.15.139:0/3159212287 >> 172.21.15.139:0/3159212287 conn(0x55a4832b4de0 msgr2=0x55a4832baac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:03.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.569+0000 7f71c10c5740 1 -- 172.21.15.139:0/3159212287 shutdown_connections 2024-09-16T01:46:03.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.569+0000 7f71c10c5740 1 -- 172.21.15.139:0/3159212287 wait complete. 2024-09-16T01:46:03.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.569+0000 7f71c10c5740 20 client.25631 trim_cache size 0 max 16384 2024-09-16T01:46:03.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.589+0000 7f71c10c5740 1 Processor -- start 2024-09-16T01:46:03.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.589+0000 7f71c10c5740 1 -- start start 2024-09-16T01:46:03.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.589+0000 7f71c10c5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b8850 0x55a4832bf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.589+0000 7f71c10c5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b9170 0x55a4832db270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.589+0000 7f71c10c5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832c3a30 0x55a4832db7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.589+0000 7f71c10c5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81acf10 con 0x55a4832b8850 2024-09-16T01:46:03.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.589+0000 7f71c10c5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81aebd0 con 0x55a4832c3a30 2024-09-16T01:46:03.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.589+0000 7f71c10c5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81ad5e0 con 0x55a4832b9170 2024-09-16T01:46:03.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.590+0000 7f71bca88640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b8850 0x55a4832bf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.590+0000 7f71bca88640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b8850 0x55a4832bf230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48530/0 (socket says 172.21.15.139:48530) 2024-09-16T01:46:03.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.590+0000 7f71bca88640 1 -- 172.21.15.139:0/3895965744 learned_addr learned my addr 172.21.15.139:0/3895965744 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:03.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.590+0000 7f71b7fff640 1 --2- 172.21.15.139:0/3895965744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b9170 0x55a4832db270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.590+0000 7f71bca88640 1 -- 172.21.15.139:0/3895965744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b9170 msgr2=0x55a4832db270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:03.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.590+0000 7f71bca88640 1 --2- 172.21.15.139:0/3895965744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b9170 0x55a4832db270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.590+0000 7f71bca88640 1 -- 172.21.15.139:0/3895965744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832c3a30 msgr2=0x55a4832db7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:03.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.590+0000 7f71bd289640 1 --2- 172.21.15.139:0/3895965744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832c3a30 0x55a4832db7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.590+0000 7f71bca88640 1 --2- 172.21.15.139:0/3895965744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832c3a30 0x55a4832db7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.590+0000 7f71bca88640 1 -- 172.21.15.139:0/3895965744 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4832b4c10 con 0x55a4832b8850 2024-09-16T01:46:03.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.590+0000 7f71bca88640 1 --2- 172.21.15.139:0/3895965744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b8850 0x55a4832bf230 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f71ac000c00 tx=0x7f71ac009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a824c7ffecfc89aa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:03.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.590+0000 7f71a27fc640 1 -- 172.21.15.139:0/3895965744 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac03b070 con 0x55a4832b8850 2024-09-16T01:46:03.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.591+0000 7f71a27fc640 1 -- 172.21.15.139:0/3895965744 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac003a10 con 0x55a4832b8850 2024-09-16T01:46:03.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.591+0000 7f71a27fc640 1 -- 172.21.15.139:0/3895965744 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0044b0 con 0x55a4832b8850 2024-09-16T01:46:03.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.592+0000 7f71c10c5740 1 -- 172.21.15.139:0/3895965744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b8850 msgr2=0x55a4832bf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:03.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.592+0000 7f71c10c5740 1 --2- 172.21.15.139:0/3895965744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b8850 0x55a4832bf230 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f71ac000c00 tx=0x7f71ac009f70 comp rx=0 tx=0).stop 2024-09-16T01:46:03.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.592+0000 7f71c10c5740 1 -- 172.21.15.139:0/3895965744 shutdown_connections 2024-09-16T01:46:03.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.592+0000 7f71c10c5740 1 --2- 172.21.15.139:0/3895965744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832c3a30 0x55a4832db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.592+0000 7f71c10c5740 1 --2- 172.21.15.139:0/3895965744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b9170 0x55a4832db270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.592+0000 7f71c10c5740 1 --2- 172.21.15.139:0/3895965744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b8850 0x55a4832bf230 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.592+0000 7f71c10c5740 1 -- 172.21.15.139:0/3895965744 >> 172.21.15.139:0/3895965744 conn(0x55a4832b4de0 msgr2=0x55a4832baac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:03.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.592+0000 7f71c10c5740 1 -- 172.21.15.139:0/3895965744 shutdown_connections 2024-09-16T01:46:03.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.592+0000 7f71c10c5740 1 -- 172.21.15.139:0/3895965744 wait complete. 2024-09-16T01:46:03.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.593+0000 7f71c10c5740 1 -- start start 2024-09-16T01:46:03.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.593+0000 7f71c10c5740 1 Processor -- start 2024-09-16T01:46:03.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.593+0000 7f71c10c5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b8850 0x55a4832bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71c10c5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b9170 0x55a4832be050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71c10c5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832c3a30 0x55a4832be590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71c10c5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81ae440 con 0x55a4832b8850 2024-09-16T01:46:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71c10c5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81add70 con 0x55a4832c3a30 2024-09-16T01:46:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71c10c5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81aaef0 con 0x55a4832b9170 2024-09-16T01:46:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71c10c5740 10 client.0 _ll_register_callbacks cb 0x7f71b0009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:46:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71bd289640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832c3a30 0x55a4832be590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71bd289640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832c3a30 0x55a4832be590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44832/0 (socket says 172.21.15.139:44832) 2024-09-16T01:46:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71bd289640 1 -- 172.21.15.139:0/4271594887 learned_addr learned my addr 172.21.15.139:0/4271594887 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71b7fff640 1 --2- 172.21.15.139:0/4271594887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b9170 0x55a4832be050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71bd289640 1 -- 172.21.15.139:0/4271594887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b9170 msgr2=0x55a4832be050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71bd289640 1 --2- 172.21.15.139:0/4271594887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b9170 0x55a4832be050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71bd289640 1 -- 172.21.15.139:0/4271594887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b8850 msgr2=0x55a4832bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71bd289640 1 --2- 172.21.15.139:0/4271594887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b8850 0x55a4832bdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.594+0000 7f71bd289640 1 -- 172.21.15.139:0/4271594887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4832b5f70 con 0x55a4832c3a30 2024-09-16T01:46:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.595+0000 7f71bd289640 1 --2- 172.21.15.139:0/4271594887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832c3a30 0x55a4832be590 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71b0003fc0 tx=0x7f71b0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93d17c57ed315c9b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.595+0000 7f7197fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:46:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.595+0000 7f7197fff640 1 -- 172.21.15.139:0/4271594887 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0004b20 con 0x55a4832c3a30 2024-09-16T01:46:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.595+0000 7f7197fff640 1 -- 172.21.15.139:0/4271594887 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f71b0004cc0 con 0x55a4832c3a30 2024-09-16T01:46:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.595+0000 7f71c10c5740 1 -- 172.21.15.139:0/4271594887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ac003a10 con 0x55a4832c3a30 2024-09-16T01:46:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.595+0000 7f7197fff640 1 -- 172.21.15.139:0/4271594887 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f71b0004310 con 0x55a4832c3a30 2024-09-16T01:46:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.595+0000 7f71c10c5740 10 client.25634 fetch_fsmap learned FSMap version 35 2024-09-16T01:46:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.595+0000 7f71c10c5740 1 -- 172.21.15.139:0/4271594887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a483346d40 con 0x55a4832c3a30 2024-09-16T01:46:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.595+0000 7f7197fff640 1 -- 172.21.15.139:0/4271594887 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac003a10 con 0x55a4832c3a30 2024-09-16T01:46:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.596+0000 7f7197fff640 1 -- 172.21.15.139:0/4271594887 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7f71b0028070 con 0x55a4832c3a30 2024-09-16T01:46:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.596+0000 7f71c10c5740 10 client.25634 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:46:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.596+0000 7f71c10c5740 10 client.25634 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:46:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.596+0000 7f71c10c5740 20 client.25634 populate_metadata read hostname 'smithi139' 2024-09-16T01:46:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.596+0000 7f71c10c5740 1 --2- 172.21.15.139:0/4271594887 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a4832d3680 0x55a4833495e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.596+0000 7f71c10c5740 4 client.25634 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}] 2024-09-16T01:46:03.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.596+0000 7f71c10c5740 1 -- 172.21.15.139:0/4271594887 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}) -- 0x55a483224be0 con 0x55a4832d3680 2024-09-16T01:46:03.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.596+0000 7f71bca88640 1 --2- 172.21.15.139:0/4271594887 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a4832d3680 0x55a4833495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:03.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.596+0000 7f7197fff640 10 client.25634 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:03.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:03.597+0000 7f71bca88640 1 --2- 172.21.15.139:0/4271594887 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a4832d3680 0x55a4833495e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f779190500d29a6d server_cookie=c10970c12170b149 in_seq=0 out_seq=0 2024-09-16T01:46:03.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:03 smithi176 ceph-mon[35324]: pgmap v172: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.4 MiB/s wr, 600 op/s 2024-09-16T01:46:03.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:03 smithi164 ceph-mon[35121]: pgmap v172: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.4 MiB/s wr, 600 op/s 2024-09-16T01:46:05.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:05 smithi139 ceph-mon[31175]: pgmap v173: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.3 MiB/s wr, 561 op/s 2024-09-16T01:46:05.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:05 smithi176 ceph-mon[35324]: pgmap v173: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.3 MiB/s wr, 561 op/s 2024-09-16T01:46:05.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:05 smithi164 ceph-mon[35121]: pgmap v173: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.3 MiB/s wr, 561 op/s 2024-09-16T01:46:05.968 INFO:tasks.workunit.client.0.smithi139.stdout: 467 85877 384 59375 237 17736 244 2024-09-16T01:46:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.195+0000 7fce0e00a640 1 -- 172.21.15.139:0/3492523973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce081009b0 msgr2=0x7fce08100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.195+0000 7fce0e00a640 1 --2- 172.21.15.139:0/3492523973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce081009b0 0x7fce08100e10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009860 tx=0x7fcdf802a340 comp rx=0 tx=0).stop 2024-09-16T01:46:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.203+0000 7fce0e00a640 1 -- 172.21.15.139:0/3492523973 shutdown_connections 2024-09-16T01:46:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.203+0000 7fce0e00a640 1 --2- 172.21.15.139:0/3492523973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce08101350 0x7fce080698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.203+0000 7fce0e00a640 1 --2- 172.21.15.139:0/3492523973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce081009b0 0x7fce08100e10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.203+0000 7fce0e00a640 1 --2- 172.21.15.139:0/3492523973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce081069c0 0x7fce08106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.203+0000 7fce0e00a640 1 -- 172.21.15.139:0/3492523973 >> 172.21.15.139:0/3492523973 conn(0x7fce08077990 msgr2=0x7fce080fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.210+0000 7fce0e00a640 1 -- 172.21.15.139:0/3492523973 shutdown_connections 2024-09-16T01:46:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.212+0000 7fce0e00a640 1 -- 172.21.15.139:0/3492523973 wait complete. 2024-09-16T01:46:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.213+0000 7fce0e00a640 1 Processor -- start 2024-09-16T01:46:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.216+0000 7fce0e00a640 1 -- start start 2024-09-16T01:46:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.219+0000 7fce0e00a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce081009b0 0x7fce0810fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.219+0000 7fce0e00a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce08101350 0x7fce081100a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.219+0000 7fce0e00a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce081069c0 0x7fce081105e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.219+0000 7fce0e00a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fce08114110 con 0x7fce081069c0 2024-09-16T01:46:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.219+0000 7fce0e00a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce08113f90 con 0x7fce08101350 2024-09-16T01:46:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.219+0000 7fce0e00a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fce08114290 con 0x7fce081009b0 2024-09-16T01:46:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.220+0000 7fce077fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce081009b0 0x7fce0810fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.220+0000 7fce077fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce081009b0 0x7fce0810fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:55204/0 (socket says 172.21.15.139:55204) 2024-09-16T01:46:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.220+0000 7fce077fe640 1 -- 172.21.15.139:0/1247438869 learned_addr learned my addr 172.21.15.139:0/1247438869 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.221+0000 7fce07fff640 1 --2- 172.21.15.139:0/1247438869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce081069c0 0x7fce081105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.221+0000 7fce06ffd640 1 --2- 172.21.15.139:0/1247438869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce08101350 0x7fce081100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.221+0000 7fce077fe640 1 -- 172.21.15.139:0/1247438869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce08101350 msgr2=0x7fce081100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.221+0000 7fce077fe640 1 --2- 172.21.15.139:0/1247438869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce08101350 0x7fce081100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.221+0000 7fce077fe640 1 -- 172.21.15.139:0/1247438869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce081069c0 msgr2=0x7fce081105e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.221+0000 7fce077fe640 1 --2- 172.21.15.139:0/1247438869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce081069c0 0x7fce081105e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.221+0000 7fce077fe640 1 -- 172.21.15.139:0/1247438869 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce081aab60 con 0x7fce081009b0 2024-09-16T01:46:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.225+0000 7fce077fe640 1 --2- 172.21.15.139:0/1247438869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce081009b0 0x7fce0810fb60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce0000ebc0 tx=0x7fce0000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.227+0000 7fce04ff9640 1 -- 172.21.15.139:0/1247438869 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00010630 con 0x7fce081009b0 2024-09-16T01:46:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.227+0000 7fce04ff9640 1 -- 172.21.15.139:0/1247438869 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce00010cb0 con 0x7fce081009b0 2024-09-16T01:46:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.227+0000 7fce04ff9640 1 -- 172.21.15.139:0/1247438869 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00018c00 con 0x7fce081009b0 2024-09-16T01:46:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.228+0000 7fce0e00a640 1 -- 172.21.15.139:0/1247438869 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce081aae50 con 0x7fce081009b0 2024-09-16T01:46:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.229+0000 7fce0e00a640 1 -- 172.21.15.139:0/1247438869 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce081ab2c0 con 0x7fce081009b0 2024-09-16T01:46:07.238 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.237+0000 7fce10e77740 1 Processor -- start 2024-09-16T01:46:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.238+0000 7fce10e77740 1 -- start start 2024-09-16T01:46:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.238+0000 7fce10e77740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a6af0 0x55598e6a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.238+0000 7fce10e77740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6a7410 0x55598e6b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.238+0000 7fce10e77740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6b1cb0 0x55598e6b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.238+0000 7fce10e77740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55598e60b2f0 con 0x55598e6a6af0 2024-09-16T01:46:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.238+0000 7fce10e77740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55598e5b3270 con 0x55598e6b1cb0 2024-09-16T01:46:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.238+0000 7fce10e77740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55598e6072b0 con 0x55598e6a7410 2024-09-16T01:46:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.238+0000 7fce04ff9640 1 -- 172.21.15.139:0/1247438869 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fce000107d0 con 0x7fce081009b0 2024-09-16T01:46:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.239+0000 7fce06ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6b1cb0 0x55598e6b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.239+0000 7fce06ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6b1cb0 0x55598e6b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44850/0 (socket says 172.21.15.139:44850) 2024-09-16T01:46:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.239+0000 7fce06ffd640 1 -- 172.21.15.139:0/163627930 learned_addr learned my addr 172.21.15.139:0/163627930 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.240+0000 7fce04ff9640 1 --2- 172.21.15.139:0/1247438869 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcdf00a3640 0x7fcdf00a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.240+0000 7fce04ff9640 1 -- 172.21.15.139:0/1247438869 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fce000d0690 con 0x7fce081009b0 2024-09-16T01:46:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.240+0000 7fce077fe640 1 --2- 172.21.15.139:0/163627930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6a7410 0x55598e6b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.242+0000 7fce077fe640 1 -- 172.21.15.139:0/163627930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6b1cb0 msgr2=0x55598e6b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.242+0000 7fce077fe640 1 --2- 172.21.15.139:0/163627930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6b1cb0 0x55598e6b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.242+0000 7fce077fe640 1 -- 172.21.15.139:0/163627930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a6af0 msgr2=0x55598e6a6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.242+0000 7fce077fe640 1 --2- 172.21.15.139:0/163627930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a6af0 0x55598e6a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.242+0000 7fce077fe640 1 -- 172.21.15.139:0/163627930 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55598e5b2340 con 0x55598e6a7410 2024-09-16T01:46:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.243+0000 7fce077fe640 1 --2- 172.21.15.139:0/163627930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6a7410 0x55598e6b1770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fce000cd9d0 tx=0x7fce0000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86c5adae1d09e99 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.243+0000 7fce07fff640 1 --2- 172.21.15.139:0/1247438869 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcdf00a3640 0x7fcdf00a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.245+0000 7fcdedffb640 1 -- 172.21.15.139:0/163627930 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00010e60 con 0x55598e6a7410 2024-09-16T01:46:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.245+0000 7fcdedffb640 1 -- 172.21.15.139:0/163627930 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce0009cc60 con 0x55598e6a7410 2024-09-16T01:46:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.246+0000 7fce10e77740 1 -- 172.21.15.139:0/163627930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6a7410 msgr2=0x55598e6b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.246+0000 7fce10e77740 1 --2- 172.21.15.139:0/163627930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6a7410 0x55598e6b1770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fce000cd9d0 tx=0x7fce0000cf90 comp rx=0 tx=0).stop 2024-09-16T01:46:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.246+0000 7fce07fff640 1 --2- 172.21.15.139:0/1247438869 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcdf00a3640 0x7fcdf00a5b00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc005ab0 tx=0x7fcdfc005a00 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.248+0000 7fce10e77740 1 -- 172.21.15.139:0/163627930 shutdown_connections 2024-09-16T01:46:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.248+0000 7fce10e77740 1 --2- 172.21.15.139:0/163627930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6b1cb0 0x55598e6b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.248+0000 7fce10e77740 1 --2- 172.21.15.139:0/163627930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6a7410 0x55598e6b1770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.248+0000 7fce10e77740 1 --2- 172.21.15.139:0/163627930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a6af0 0x55598e6a6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.248+0000 7fce10e77740 1 -- 172.21.15.139:0/163627930 >> 172.21.15.139:0/163627930 conn(0x55598e6a3060 msgr2=0x55598e6a3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.249+0000 7fce10e77740 1 -- 172.21.15.139:0/163627930 shutdown_connections 2024-09-16T01:46:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.249+0000 7fce10e77740 1 -- 172.21.15.139:0/163627930 wait complete. 2024-09-16T01:46:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.249+0000 7fce10e77740 1 -- start start 2024-09-16T01:46:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.250+0000 7fce10e77740 1 Processor -- start 2024-09-16T01:46:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.251+0000 7fce10e77740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a6af0 0x55598e6aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.251+0000 7fce10e77740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6a7410 0x55598e6af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.251+0000 7fce10e77740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6b1cb0 0x55598e6af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.251+0000 7fce10e77740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55598e6c58e0 con 0x55598e6a6af0 2024-09-16T01:46:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.251+0000 7fce10e77740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55598e594290 con 0x55598e6a7410 2024-09-16T01:46:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.251+0000 7fce10e77740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55598e6c5a60 con 0x55598e6b1cb0 2024-09-16T01:46:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.251+0000 7fce10e77740 10 client.0 _ll_register_callbacks cb 0x55598e5c5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:46:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.251+0000 7fce077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a6af0 0x55598e6aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.251+0000 7fce077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a6af0 0x55598e6aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48594/0 (socket says 172.21.15.139:48594) 2024-09-16T01:46:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.251+0000 7fce077fe640 1 -- 172.21.15.139:0/502778343 learned_addr learned my addr 172.21.15.139:0/502778343 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.251+0000 7fce06ffd640 1 --2- 172.21.15.139:0/502778343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6a7410 0x55598e6af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.251+0000 7fce07fff640 1 --2- 172.21.15.139:0/502778343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6b1cb0 0x55598e6af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.252+0000 7fce06ffd640 1 -- 172.21.15.139:0/502778343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6b1cb0 msgr2=0x55598e6af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.252+0000 7fce06ffd640 1 --2- 172.21.15.139:0/502778343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6b1cb0 0x55598e6af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.252+0000 7fce06ffd640 1 -- 172.21.15.139:0/502778343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a6af0 msgr2=0x55598e6aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.252+0000 7fce06ffd640 1 --2- 172.21.15.139:0/502778343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a6af0 0x55598e6aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.252+0000 7fce06ffd640 1 -- 172.21.15.139:0/502778343 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55598e31e210 con 0x55598e6a7410 2024-09-16T01:46:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.252+0000 7fce06ffd640 1 --2- 172.21.15.139:0/502778343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6a7410 0x55598e6af050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55598e6a43e0 tx=0x7fcdf80052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=964edbebcd57bbf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.252+0000 7fcde6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:46:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.253+0000 7fcde6ffd640 1 -- 172.21.15.139:0/502778343 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8041020 con 0x55598e6a7410 2024-09-16T01:46:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.253+0000 7fcde6ffd640 1 -- 172.21.15.139:0/502778343 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fcdf8030820 con 0x55598e6a7410 2024-09-16T01:46:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.254+0000 7fce10e77740 1 -- 172.21.15.139:0/502778343 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce0009cc60 con 0x55598e6a7410 2024-09-16T01:46:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.254+0000 7fcde6ffd640 1 -- 172.21.15.139:0/502778343 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcdf8037070 con 0x55598e6a7410 2024-09-16T01:46:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.254+0000 7fce10e77740 10 client.25646 fetch_fsmap learned FSMap version 35 2024-09-16T01:46:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.254+0000 7fce10e77740 1 -- 172.21.15.139:0/502778343 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55598e3ea620 con 0x55598e6a7410 2024-09-16T01:46:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.255+0000 7fcde6ffd640 1 -- 172.21.15.139:0/502778343 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce0009cc60 con 0x55598e6a7410 2024-09-16T01:46:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.255+0000 7fcde6ffd640 1 -- 172.21.15.139:0/502778343 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7fcdf80048d0 con 0x55598e6a7410 2024-09-16T01:46:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.255+0000 7fce10e77740 10 client.25646 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:46:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.255+0000 7fce10e77740 10 client.25646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:46:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.255+0000 7fce10e77740 20 client.25646 populate_metadata read hostname 'smithi139' 2024-09-16T01:46:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.255+0000 7fce10e77740 1 --2- 172.21.15.139:0/502778343 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55598e6d2c70 0x55598e6d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.255+0000 7fce10e77740 4 client.25646 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:46:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.255+0000 7fce10e77740 1 -- 172.21.15.139:0/502778343 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55598e618a20 con 0x55598e6d2c70 2024-09-16T01:46:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.255+0000 7fce077fe640 1 --2- 172.21.15.139:0/502778343 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55598e6d2c70 0x55598e6d50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.256+0000 7fcde6ffd640 10 client.25646 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.256+0000 7fce077fe640 1 --2- 172.21.15.139:0/502778343 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55598e6d2c70 0x55598e6d50e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa2a58767b86087f server_cookie=a82bb4917a0b6040 in_seq=0 out_seq=0 2024-09-16T01:46:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.294+0000 7fcde6ffd640 1 -- 172.21.15.139:0/502778343 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55598e618a20 con 0x55598e6d2c70 2024-09-16T01:46:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.294+0000 7fcde6ffd640 10 client.25646 handle_command_reply: tid=0 2024-09-16T01:46:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.294+0000 7fce10e77740 1 client.25646 shutdown 2024-09-16T01:46:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.294+0000 7fce10e77740 1 -- 172.21.15.139:0/502778343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6a7410 msgr2=0x55598e6af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.294+0000 7fce10e77740 1 --2- 172.21.15.139:0/502778343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6a7410 0x55598e6af050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55598e6a43e0 tx=0x7fcdf80052f0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.295+0000 7fce10e77740 1 -- 172.21.15.139:0/502778343 shutdown_connections 2024-09-16T01:46:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.295+0000 7fce10e77740 1 --2- 172.21.15.139:0/502778343 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55598e6d2c70 0x55598e6d50e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.295+0000 7fce10e77740 1 --2- 172.21.15.139:0/502778343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6b1cb0 0x55598e6af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.295+0000 7fce10e77740 1 --2- 172.21.15.139:0/502778343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6a7410 0x55598e6af050 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.295+0000 7fce10e77740 1 --2- 172.21.15.139:0/502778343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a6af0 0x55598e6aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.295+0000 7fcde6ffd640 0 client.25646 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.295+0000 7fce10e77740 1 -- 172.21.15.139:0/502778343 >> 172.21.15.139:0/502778343 conn(0x55598e6a3060 msgr2=0x55598e6a8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.295+0000 7fce10e77740 1 -- 172.21.15.139:0/502778343 shutdown_connections 2024-09-16T01:46:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.295+0000 7fce10e77740 1 -- 172.21.15.139:0/502778343 wait complete. 2024-09-16T01:46:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.295+0000 7fce10e77740 20 client.25646 trim_cache size 0 max 16384 2024-09-16T01:46:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce10e77740 1 Processor -- start 2024-09-16T01:46:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce10e77740 1 -- start start 2024-09-16T01:46:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce10e77740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6a6af0 0x55598e6da9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce10e77740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a7410 0x55598e6daef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce10e77740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6b1cb0 0x55598e6abd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce10e77740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fce081ace70 con 0x55598e6a7410 2024-09-16T01:46:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce10e77740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce081aeb30 con 0x55598e6a6af0 2024-09-16T01:46:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce10e77740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fce081ad540 con 0x55598e6b1cb0 2024-09-16T01:46:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce06ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a7410 0x55598e6daef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce077fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6a6af0 0x55598e6da9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce077fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6a6af0 0x55598e6da9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44862/0 (socket says 172.21.15.139:44862) 2024-09-16T01:46:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce077fe640 1 -- 172.21.15.139:0/2361361443 learned_addr learned my addr 172.21.15.139:0/2361361443 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce07fff640 1 --2- 172.21.15.139:0/2361361443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6b1cb0 0x55598e6abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce077fe640 1 -- 172.21.15.139:0/2361361443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6b1cb0 msgr2=0x55598e6abd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce077fe640 1 --2- 172.21.15.139:0/2361361443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6b1cb0 0x55598e6abd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce077fe640 1 -- 172.21.15.139:0/2361361443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a7410 msgr2=0x55598e6daef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce077fe640 1 --2- 172.21.15.139:0/2361361443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a7410 0x55598e6daef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce077fe640 1 -- 172.21.15.139:0/2361361443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55598e6a6870 con 0x55598e6a6af0 2024-09-16T01:46:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.313+0000 7fce06ffd640 1 --2- 172.21.15.139:0/2361361443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a7410 0x55598e6daef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:46:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.314+0000 7fce077fe640 1 --2- 172.21.15.139:0/2361361443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6a6af0 0x55598e6da9b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fce000a4ca0 tx=0x7fce0001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65627a3733032923 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.314+0000 7fcded7fa640 1 -- 172.21.15.139:0/2361361443 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00010e60 con 0x55598e6a6af0 2024-09-16T01:46:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.314+0000 7fcded7fa640 1 -- 172.21.15.139:0/2361361443 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce000b4040 con 0x55598e6a6af0 2024-09-16T01:46:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.314+0000 7fcded7fa640 1 -- 172.21.15.139:0/2361361443 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0009b9b0 con 0x55598e6a6af0 2024-09-16T01:46:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.315+0000 7fce10e77740 1 -- 172.21.15.139:0/2361361443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6a6af0 msgr2=0x55598e6da9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.315+0000 7fce10e77740 1 --2- 172.21.15.139:0/2361361443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6a6af0 0x55598e6da9b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fce000a4ca0 tx=0x7fce0001f000 comp rx=0 tx=0).stop 2024-09-16T01:46:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.315+0000 7fce10e77740 1 -- 172.21.15.139:0/2361361443 shutdown_connections 2024-09-16T01:46:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.315+0000 7fce10e77740 1 --2- 172.21.15.139:0/2361361443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6b1cb0 0x55598e6abd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.315+0000 7fce10e77740 1 --2- 172.21.15.139:0/2361361443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a7410 0x55598e6daef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.315+0000 7fce10e77740 1 --2- 172.21.15.139:0/2361361443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6a6af0 0x55598e6da9b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.315+0000 7fce10e77740 1 -- 172.21.15.139:0/2361361443 >> 172.21.15.139:0/2361361443 conn(0x55598e6a3060 msgr2=0x55598e6b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.315+0000 7fce10e77740 1 -- 172.21.15.139:0/2361361443 shutdown_connections 2024-09-16T01:46:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.315+0000 7fce10e77740 1 -- 172.21.15.139:0/2361361443 wait complete. 2024-09-16T01:46:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.315+0000 7fce10e77740 1 -- start start 2024-09-16T01:46:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce10e77740 1 Processor -- start 2024-09-16T01:46:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce10e77740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6a6af0 0x55598e6abb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce10e77740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a7410 0x55598e6ac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce10e77740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6b1cb0 0x55598e6ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce10e77740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fce081ae3a0 con 0x55598e6a7410 2024-09-16T01:46:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce10e77740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce081adcd0 con 0x55598e6b1cb0 2024-09-16T01:46:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce10e77740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fce081a22a0 con 0x55598e6a6af0 2024-09-16T01:46:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce10e77740 10 client.0 _ll_register_callbacks cb 0x7fcdfc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:46:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce06ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a7410 0x55598e6ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce07fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6b1cb0 0x55598e6ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce06ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a7410 0x55598e6ac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48610/0 (socket says 172.21.15.139:48610) 2024-09-16T01:46:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce06ffd640 1 -- 172.21.15.139:0/1433888434 learned_addr learned my addr 172.21.15.139:0/1433888434 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce077fe640 1 --2- 172.21.15.139:0/1433888434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6a6af0 0x55598e6abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce07fff640 1 -- 172.21.15.139:0/1433888434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6a6af0 msgr2=0x55598e6abb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce07fff640 1 --2- 172.21.15.139:0/1433888434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6a6af0 0x55598e6abb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce07fff640 1 -- 172.21.15.139:0/1433888434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a7410 msgr2=0x55598e6ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce07fff640 1 --2- 172.21.15.139:0/1433888434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a7410 0x55598e6ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.316+0000 7fce07fff640 1 -- 172.21.15.139:0/1433888434 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55598e73c980 con 0x55598e6b1cb0 2024-09-16T01:46:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.317+0000 7fce07fff640 1 --2- 172.21.15.139:0/1433888434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6b1cb0 0x55598e6ac590 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc004700 tx=0x7fcdfc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38d144b982a987b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.317+0000 7fcde6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:46:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.317+0000 7fcde6ffd640 1 -- 172.21.15.139:0/1433888434 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc003d90 con 0x55598e6b1cb0 2024-09-16T01:46:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.317+0000 7fce10e77740 1 -- 172.21.15.139:0/1433888434 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce000b4040 con 0x55598e6b1cb0 2024-09-16T01:46:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.317+0000 7fcde6ffd640 1 -- 172.21.15.139:0/1433888434 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fcdfc02a420 con 0x55598e6b1cb0 2024-09-16T01:46:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.317+0000 7fcde6ffd640 1 -- 172.21.15.139:0/1433888434 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcdfc032050 con 0x55598e6b1cb0 2024-09-16T01:46:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.317+0000 7fce10e77740 10 client.25652 fetch_fsmap learned FSMap version 35 2024-09-16T01:46:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.317+0000 7fce10e77740 1 -- 172.21.15.139:0/1433888434 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55598e73dd20 con 0x55598e6b1cb0 2024-09-16T01:46:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.317+0000 7fcde6ffd640 1 -- 172.21.15.139:0/1433888434 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce000b4040 con 0x55598e6b1cb0 2024-09-16T01:46:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.317+0000 7fcde6ffd640 1 -- 172.21.15.139:0/1433888434 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7fcdfc0108a0 con 0x55598e6b1cb0 2024-09-16T01:46:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.318+0000 7fce10e77740 10 client.25652 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:46:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.318+0000 7fce10e77740 10 client.25652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:46:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.318+0000 7fce10e77740 20 client.25652 populate_metadata read hostname 'smithi139' 2024-09-16T01:46:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.318+0000 7fce10e77740 1 --2- 172.21.15.139:0/1433888434 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55598e73e560 0x55598e740980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.318+0000 7fce10e77740 4 client.25652 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:46:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.318+0000 7fce10e77740 1 -- 172.21.15.139:0/1433888434 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55598e612d70 con 0x55598e73e560 2024-09-16T01:46:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.318+0000 7fce077fe640 1 --2- 172.21.15.139:0/1433888434 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55598e73e560 0x55598e740980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.318+0000 7fcde6ffd640 10 client.25652 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:07.318+0000 7fce077fe640 1 --2- 172.21.15.139:0/1433888434 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55598e73e560 0x55598e740980 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e906c901d4d6237 server_cookie=90511dba39ab2557 in_seq=0 out_seq=0 2024-09-16T01:46:07.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:07 smithi139 ceph-mon[31175]: pgmap v174: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 2.3 MiB/s wr, 557 op/s 2024-09-16T01:46:07.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:07 smithi176 ceph-mon[35324]: pgmap v174: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 2.3 MiB/s wr, 557 op/s 2024-09-16T01:46:07.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:07 smithi164 ceph-mon[35121]: pgmap v174: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 2.3 MiB/s wr, 557 op/s 2024-09-16T01:46:09.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:09 smithi139 ceph-mon[31175]: pgmap v175: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.8 MiB/s wr, 423 op/s 2024-09-16T01:46:09.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:09 smithi176 ceph-mon[35324]: pgmap v175: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.8 MiB/s wr, 423 op/s 2024-09-16T01:46:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:09 smithi164 ceph-mon[35121]: pgmap v175: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.8 MiB/s wr, 423 op/s 2024-09-16T01:46:11.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:11 smithi139 ceph-mon[31175]: pgmap v176: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.7 MiB/s wr, 345 op/s 2024-09-16T01:46:11.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:11 smithi176 ceph-mon[35324]: pgmap v176: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.7 MiB/s wr, 345 op/s 2024-09-16T01:46:11.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:11 smithi164 ceph-mon[35121]: pgmap v176: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.7 MiB/s wr, 345 op/s 2024-09-16T01:46:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fcde6ffd640 1 -- 172.21.15.139:0/1433888434 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55598e612d70 con 0x55598e73e560 2024-09-16T01:46:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fcde6ffd640 10 client.25652 handle_command_reply: tid=0 2024-09-16T01:46:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fce10e77740 1 client.25652 shutdown 2024-09-16T01:46:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fce10e77740 1 -- 172.21.15.139:0/1433888434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6b1cb0 msgr2=0x55598e6ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fce10e77740 1 --2- 172.21.15.139:0/1433888434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6b1cb0 0x55598e6ac590 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc004700 tx=0x7fcdfc009260 comp rx=0 tx=0).stop 2024-09-16T01:46:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fce10e77740 1 -- 172.21.15.139:0/1433888434 shutdown_connections 2024-09-16T01:46:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fce10e77740 1 --2- 172.21.15.139:0/1433888434 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55598e73e560 0x55598e740980 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fce10e77740 1 --2- 172.21.15.139:0/1433888434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55598e6b1cb0 0x55598e6ac590 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fce10e77740 1 --2- 172.21.15.139:0/1433888434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55598e6a7410 0x55598e6ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fce10e77740 1 --2- 172.21.15.139:0/1433888434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55598e6a6af0 0x55598e6abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fcde6ffd640 0 client.25652 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fce10e77740 1 -- 172.21.15.139:0/1433888434 >> 172.21.15.139:0/1433888434 conn(0x55598e6a3060 msgr2=0x55598e6b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fce10e77740 1 -- 172.21.15.139:0/1433888434 shutdown_connections 2024-09-16T01:46:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fce10e77740 1 -- 172.21.15.139:0/1433888434 wait complete. 2024-09-16T01:46:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.984+0000 7fce10e77740 20 client.25652 trim_cache size 0 max 16384 2024-09-16T01:46:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:46:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:46:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:46:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:46:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.985+0000 7fce0e00a640 1 -- 172.21.15.139:0/1247438869 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcdf00a3640 msgr2=0x7fcdf00a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.985+0000 7fce0e00a640 1 --2- 172.21.15.139:0/1247438869 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcdf00a3640 0x7fcdf00a5b00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc005ab0 tx=0x7fcdfc005a00 comp rx=0 tx=0).stop 2024-09-16T01:46:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.985+0000 7fce0e00a640 1 -- 172.21.15.139:0/1247438869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce081009b0 msgr2=0x7fce0810fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.985+0000 7fce0e00a640 1 --2- 172.21.15.139:0/1247438869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce081009b0 0x7fce0810fb60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce0000ebc0 tx=0x7fce0000c6a0 comp rx=0 tx=0).stop 2024-09-16T01:46:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.985+0000 7fce0e00a640 1 -- 172.21.15.139:0/1247438869 shutdown_connections 2024-09-16T01:46:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.985+0000 7fce0e00a640 1 --2- 172.21.15.139:0/1247438869 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcdf00a3640 0x7fcdf00a5b00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.985+0000 7fce0e00a640 1 --2- 172.21.15.139:0/1247438869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce081069c0 0x7fce081105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.985+0000 7fce0e00a640 1 --2- 172.21.15.139:0/1247438869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce08101350 0x7fce081100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.985+0000 7fce0e00a640 1 --2- 172.21.15.139:0/1247438869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce081009b0 0x7fce0810fb60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.985+0000 7fce0e00a640 1 -- 172.21.15.139:0/1247438869 >> 172.21.15.139:0/1247438869 conn(0x7fce08077990 msgr2=0x7fce0810ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.985+0000 7fce0e00a640 1 -- 172.21.15.139:0/1247438869 shutdown_connections 2024-09-16T01:46:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:12.985+0000 7fce0e00a640 1 -- 172.21.15.139:0/1247438869 wait complete. 2024-09-16T01:46:13.515 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:46:13.516 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 35, 2024-09-16T01:46:13.516 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:46:13.516 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 22, 2024-09-16T01:46:13.516 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:46:13.516 INFO:teuthology.orchestra.run.smithi139.stdout: "63cecc48": { 2024-09-16T01:46:13.516 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 22, 2024-09-16T01:46:13.516 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:46:13.516 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:46:13.516 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:46:13.516 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:46:13.516 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:46:13.516 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:46:13.517 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 145.3, 2024-09-16T01:46:13.517 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:46:13.517 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:46:13.517 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:46:13.517 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:46:13.517 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:46:13.517 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:46:13.517 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:13.517 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:13.517 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:13.517 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:13.517 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:13.518 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:46:13.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.513+0000 7f7197fff640 1 -- 172.21.15.139:0/4271594887 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a483224be0 con 0x55a4832d3680 2024-09-16T01:46:13.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.513+0000 7f7197fff640 10 client.25634 handle_command_reply: tid=0 2024-09-16T01:46:13.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.513+0000 7f71c10c5740 1 client.25634 shutdown 2024-09-16T01:46:13.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71c10c5740 1 -- 172.21.15.139:0/4271594887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832c3a30 msgr2=0x55a4832be590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:13.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71c10c5740 1 --2- 172.21.15.139:0/4271594887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832c3a30 0x55a4832be590 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71b0003fc0 tx=0x7f71b0004ab0 comp rx=0 tx=0).stop 2024-09-16T01:46:13.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71c10c5740 1 -- 172.21.15.139:0/4271594887 shutdown_connections 2024-09-16T01:46:13.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71c10c5740 1 --2- 172.21.15.139:0/4271594887 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a4832d3680 0x55a4833495e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:13.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71c10c5740 1 --2- 172.21.15.139:0/4271594887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4832c3a30 0x55a4832be590 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:13.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71c10c5740 1 --2- 172.21.15.139:0/4271594887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4832b9170 0x55a4832be050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:13.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71c10c5740 1 --2- 172.21.15.139:0/4271594887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a4832b8850 0x55a4832bdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:13.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f7197fff640 0 client.25634 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:13.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71c10c5740 1 -- 172.21.15.139:0/4271594887 >> 172.21.15.139:0/4271594887 conn(0x55a4832b4de0 msgr2=0x55a4832b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:13.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71c10c5740 1 -- 172.21.15.139:0/4271594887 shutdown_connections 2024-09-16T01:46:13.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71c10c5740 1 -- 172.21.15.139:0/4271594887 wait complete. 2024-09-16T01:46:13.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71c10c5740 20 client.25634 trim_cache size 0 max 16384 2024-09-16T01:46:13.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71becec640 1 -- 172.21.15.139:0/1055128424 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f71a80a7840 msgr2=0x7f71a80a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:13.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71becec640 1 --2- 172.21.15.139:0/1055128424 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f71a80a7840 0x7f71a80a9d00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009fd0 tx=0x7f71b0009500 comp rx=0 tx=0).stop 2024-09-16T01:46:13.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71becec640 1 -- 172.21.15.139:0/1055128424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b8103740 msgr2=0x7f71b8196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:13.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.514+0000 7f71becec640 1 --2- 172.21.15.139:0/1055128424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b8103740 0x7f71b8196cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009230 tx=0x7f71a4031b50 comp rx=0 tx=0).stop 2024-09-16T01:46:13.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.515+0000 7f71becec640 1 -- 172.21.15.139:0/1055128424 shutdown_connections 2024-09-16T01:46:13.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.515+0000 7f71becec640 1 --2- 172.21.15.139:0/1055128424 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f71a80a7840 0x7f71a80a9d00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:13.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.515+0000 7f71becec640 1 --2- 172.21.15.139:0/1055128424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71b8108290 0x7f71b819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:13.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.515+0000 7f71becec640 1 --2- 172.21.15.139:0/1055128424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b8103740 0x7f71b8196cf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:13.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.515+0000 7f71becec640 1 --2- 172.21.15.139:0/1055128424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71b8102d90 0x7f71b81967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:13.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.515+0000 7f71becec640 1 -- 172.21.15.139:0/1055128424 >> 172.21.15.139:0/1055128424 conn(0x7f71b80fc7d0 msgr2=0x7f71b80fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:13.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.515+0000 7f71becec640 1 -- 172.21.15.139:0/1055128424 shutdown_connections 2024-09-16T01:46:13.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:13.515+0000 7f71becec640 1 -- 172.21.15.139:0/1055128424 wait complete. 2024-09-16T01:46:13.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:13 smithi139 ceph-mon[31175]: pgmap v177: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-16T01:46:13.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:13 smithi176 ceph-mon[35324]: pgmap v177: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-16T01:46:13.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:13 smithi164 ceph-mon[35121]: pgmap v177: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-16T01:46:14.364 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '63cecc48', quiesce took 13.6 seconds. Will release after: 11.750000000000002 2024-09-16T01:46:15.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:15 smithi139 ceph-mon[31175]: pgmap v178: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 349 op/s 2024-09-16T01:46:15.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:15 smithi176 ceph-mon[35324]: pgmap v178: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 349 op/s 2024-09-16T01:46:15.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:15 smithi164 ceph-mon[35121]: pgmap v178: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 349 op/s 2024-09-16T01:46:15.968 INFO:tasks.workunit.client.0.smithi139.stdout: 467 8795 1 5638 0 2370 2 2024-09-16T01:46:17.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:17 smithi139 ceph-mon[31175]: pgmap v179: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2024-09-16T01:46:17.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:17 smithi176 ceph-mon[35324]: pgmap v179: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2024-09-16T01:46:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:17 smithi164 ceph-mon[35121]: pgmap v179: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2024-09-16T01:46:19.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:19 smithi139 ceph-mon[31175]: pgmap v180: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.6 MiB/s wr, 426 op/s 2024-09-16T01:46:19.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:19 smithi176 ceph-mon[35324]: pgmap v180: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.6 MiB/s wr, 426 op/s 2024-09-16T01:46:19.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:19 smithi164 ceph-mon[35121]: pgmap v180: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.6 MiB/s wr, 426 op/s 2024-09-16T01:46:21.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:21 smithi139 ceph-mon[31175]: pgmap v181: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.2 MiB/s wr, 482 op/s 2024-09-16T01:46:21.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:21 smithi176 ceph-mon[35324]: pgmap v181: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.2 MiB/s wr, 482 op/s 2024-09-16T01:46:21.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:21 smithi164 ceph-mon[35121]: pgmap v181: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.2 MiB/s wr, 482 op/s 2024-09-16T01:46:23.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:23 smithi139 ceph-mon[31175]: pgmap v182: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.1 MiB/s wr, 719 op/s 2024-09-16T01:46:23.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:23 smithi176 ceph-mon[35324]: pgmap v182: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.1 MiB/s wr, 719 op/s 2024-09-16T01:46:23.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:23 smithi164 ceph-mon[35121]: pgmap v182: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.1 MiB/s wr, 719 op/s 2024-09-16T01:46:25.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:25 smithi139 ceph-mon[31175]: pgmap v183: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.1 MiB/s wr, 904 op/s 2024-09-16T01:46:25.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:25 smithi176 ceph-mon[35324]: pgmap v183: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.1 MiB/s wr, 904 op/s 2024-09-16T01:46:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:25 smithi164 ceph-mon[35121]: pgmap v183: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.1 MiB/s wr, 904 op/s 2024-09-16T01:46:25.968 INFO:tasks.workunit.client.0.smithi139.stdout: 467 438667 0 300976 0 92190 0 2024-09-16T01:46:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.071+0000 7fb8610f1640 1 -- 172.21.15.139:0/1455584732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb85c101390 msgr2=0x7fb85c10d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.071+0000 7fb8610f1640 1 --2- 172.21.15.139:0/1455584732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb85c101390 0x7fb85c10d8c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb854009cd0 tx=0x7fb85402a8e0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.071+0000 7fb8610f1640 1 -- 172.21.15.139:0/1455584732 shutdown_connections 2024-09-16T01:46:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.071+0000 7fb8610f1640 1 --2- 172.21.15.139:0/1455584732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb85c101390 0x7fb85c10d8c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.071+0000 7fb8610f1640 1 --2- 172.21.15.139:0/1455584732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c1009f0 0x7fb85c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.071+0000 7fb8610f1640 1 --2- 172.21.15.139:0/1455584732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb85c106a00 0x7fb85c106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.071+0000 7fb8610f1640 1 -- 172.21.15.139:0/1455584732 >> 172.21.15.139:0/1455584732 conn(0x7fb85c0fc6b0 msgr2=0x7fb85c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.071+0000 7fb8610f1640 1 -- 172.21.15.139:0/1455584732 shutdown_connections 2024-09-16T01:46:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.071+0000 7fb8610f1640 1 -- 172.21.15.139:0/1455584732 wait complete. 2024-09-16T01:46:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb8610f1640 1 Processor -- start 2024-09-16T01:46:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb8610f1640 1 -- start start 2024-09-16T01:46:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb8610f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c1009f0 0x7fb85c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb8610f1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb85c101390 0x7fb85c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb8610f1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb85c106a00 0x7fb85c19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb8610f1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c110e40 con 0x7fb85c101390 2024-09-16T01:46:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb8610f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c110cc0 con 0x7fb85c1009f0 2024-09-16T01:46:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb8610f1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c110fc0 con 0x7fb85c106a00 2024-09-16T01:46:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb85a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb85c101390 0x7fb85c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb85a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb85c101390 0x7fb85c1977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56202/0 (socket says 172.21.15.139:56202) 2024-09-16T01:46:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb85a59c640 1 -- 172.21.15.139:0/2396149019 learned_addr learned my addr 172.21.15.139:0/2396149019 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb85ad9d640 1 --2- 172.21.15.139:0/2396149019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c1009f0 0x7fb85c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb85b59e640 1 --2- 172.21.15.139:0/2396149019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb85c106a00 0x7fb85c19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb85a59c640 1 -- 172.21.15.139:0/2396149019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb85c106a00 msgr2=0x7fb85c19bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb85a59c640 1 --2- 172.21.15.139:0/2396149019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb85c106a00 0x7fb85c19bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb85a59c640 1 -- 172.21.15.139:0/2396149019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c1009f0 msgr2=0x7fb85c197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.072+0000 7fb85a59c640 1 --2- 172.21.15.139:0/2396149019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c1009f0 0x7fb85c197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.073+0000 7fb85a59c640 1 -- 172.21.15.139:0/2396149019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85c19c260 con 0x7fb85c101390 2024-09-16T01:46:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.073+0000 7fb85a59c640 1 --2- 172.21.15.139:0/2396149019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb85c101390 0x7fb85c1977b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb85000d950 tx=0x7fb85000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.073+0000 7fb843fff640 1 -- 172.21.15.139:0/2396149019 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850017070 con 0x7fb85c101390 2024-09-16T01:46:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.073+0000 7fb843fff640 1 -- 172.21.15.139:0/2396149019 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb850009c30 con 0x7fb85c101390 2024-09-16T01:46:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.073+0000 7fb8610f1640 1 -- 172.21.15.139:0/2396149019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85c19c550 con 0x7fb85c101390 2024-09-16T01:46:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.073+0000 7fb843fff640 1 -- 172.21.15.139:0/2396149019 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85000f6b0 con 0x7fb85c101390 2024-09-16T01:46:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.073+0000 7fb8610f1640 1 -- 172.21.15.139:0/2396149019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb85c1a3de0 con 0x7fb85c101390 2024-09-16T01:46:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.075+0000 7fb843fff640 1 -- 172.21.15.139:0/2396149019 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb8500109d0 con 0x7fb85c101390 2024-09-16T01:46:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.075+0000 7fb863e91740 1 Processor -- start 2024-09-16T01:46:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.075+0000 7fb863e91740 1 -- start start 2024-09-16T01:46:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb863e91740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a485899990 0x55a485899d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb863e91740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a48589a2b0 0x55a4858a6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb863e91740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 0x55a4858a92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb863e91740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a4857fe2f0 con 0x55a48589a2b0 2024-09-16T01:46:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb863e91740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a4857a6270 con 0x55a4858a6ec0 2024-09-16T01:46:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb863e91740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a4857fa2b0 con 0x55a485899990 2024-09-16T01:46:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb85ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a48589a2b0 0x55a4858a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb85ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a48589a2b0 0x55a4858a6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56214/0 (socket says 172.21.15.139:56214) 2024-09-16T01:46:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb85ad9d640 1 -- 172.21.15.139:0/964840512 learned_addr learned my addr 172.21.15.139:0/964840512 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb843fff640 1 --2- 172.21.15.139:0/2396149019 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb8440a34e0 0x7fb8440a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb85b59e640 1 --2- 172.21.15.139:0/964840512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a485899990 0x55a485899d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb85ad9d640 1 -- 172.21.15.139:0/964840512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a485899990 msgr2=0x55a485899d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb85b59e640 1 --2- 172.21.15.139:0/2396149019 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb8440a34e0 0x7fb8440a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb85ad9d640 1 --2- 172.21.15.139:0/964840512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a485899990 0x55a485899d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb85ad9d640 1 -- 172.21.15.139:0/964840512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 msgr2=0x55a4858a92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb843fff640 1 -- 172.21.15.139:0/2396149019 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fb8500cfd80 con 0x7fb85c101390 2024-09-16T01:46:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb85a59c640 1 --2- 172.21.15.139:0/964840512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 0x55a4858a92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb85ad9d640 1 --2- 172.21.15.139:0/964840512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 0x55a4858a92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb85ad9d640 1 -- 172.21.15.139:0/964840512 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4857a5340 con 0x55a48589a2b0 2024-09-16T01:46:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb85a59c640 1 --2- 172.21.15.139:0/964840512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 0x55a4858a92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:46:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb85ad9d640 1 --2- 172.21.15.139:0/964840512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a48589a2b0 0x55a4858a6980 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b710 tx=0x7fb84c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7762b96007677874 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb8417fa640 1 -- 172.21.15.139:0/964840512 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c003bd0 con 0x55a48589a2b0 2024-09-16T01:46:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.076+0000 7fb85b59e640 1 --2- 172.21.15.139:0/2396149019 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb8440a34e0 0x7fb8440a59a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb854005e30 tx=0x7fb8540095c0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.077+0000 7fb8417fa640 1 -- 172.21.15.139:0/964840512 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84c015070 con 0x55a48589a2b0 2024-09-16T01:46:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.077+0000 7fb8417fa640 1 -- 172.21.15.139:0/964840512 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c004e40 con 0x55a48589a2b0 2024-09-16T01:46:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.077+0000 7fb863e91740 1 -- 172.21.15.139:0/964840512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a48589a2b0 msgr2=0x55a4858a6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.077+0000 7fb863e91740 1 --2- 172.21.15.139:0/964840512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a48589a2b0 0x55a4858a6980 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b710 tx=0x7fb84c004610 comp rx=0 tx=0).stop 2024-09-16T01:46:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.077+0000 7fb863e91740 1 -- 172.21.15.139:0/964840512 shutdown_connections 2024-09-16T01:46:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.077+0000 7fb863e91740 1 --2- 172.21.15.139:0/964840512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 0x55a4858a92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.077+0000 7fb863e91740 1 --2- 172.21.15.139:0/964840512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a48589a2b0 0x55a4858a6980 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.077+0000 7fb863e91740 1 --2- 172.21.15.139:0/964840512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a485899990 0x55a485899d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.077+0000 7fb863e91740 1 -- 172.21.15.139:0/964840512 >> 172.21.15.139:0/964840512 conn(0x55a485896060 msgr2=0x55a485896440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.077+0000 7fb863e91740 1 -- 172.21.15.139:0/964840512 shutdown_connections 2024-09-16T01:46:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.077+0000 7fb863e91740 1 -- 172.21.15.139:0/964840512 wait complete. 2024-09-16T01:46:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb863e91740 1 -- start start 2024-09-16T01:46:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb863e91740 1 Processor -- start 2024-09-16T01:46:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb863e91740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb863e91740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a48589a2b0 0x55a4858a4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb863e91740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 0x55a4858a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb863e91740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a4858a9af0 con 0x55a485899990 2024-09-16T01:46:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb863e91740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a485787290 con 0x55a4858a6ec0 2024-09-16T01:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb863e91740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a4858a9c70 con 0x55a48589a2b0 2024-09-16T01:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb863e91740 10 client.0 _ll_register_callbacks cb 0x55a4857b8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb85ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb85ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56224/0 (socket says 172.21.15.139:56224) 2024-09-16T01:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb85ad9d640 1 -- 172.21.15.139:0/2487976227 learned_addr learned my addr 172.21.15.139:0/2487976227 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb85b59e640 1 --2- 172.21.15.139:0/2487976227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 0x55a4858a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb85a59c640 1 --2- 172.21.15.139:0/2487976227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a48589a2b0 0x55a4858a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb85ad9d640 1 -- 172.21.15.139:0/2487976227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a48589a2b0 msgr2=0x55a4858a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb85ad9d640 1 --2- 172.21.15.139:0/2487976227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a48589a2b0 0x55a4858a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb85ad9d640 1 -- 172.21.15.139:0/2487976227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 msgr2=0x55a4858a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb85ad9d640 1 --2- 172.21.15.139:0/2487976227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 0x55a4858a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.078+0000 7fb85ad9d640 1 -- 172.21.15.139:0/2487976227 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a485511210 con 0x55a485899990 2024-09-16T01:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb85ad9d640 1 --2- 172.21.15.139:0/2487976227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a3e20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0039c0 tx=0x7fb84c004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f71b32ea58fef07 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb83eff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:46:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb83eff5640 1 -- 172.21.15.139:0/2487976227 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c028060 con 0x55a485899990 2024-09-16T01:46:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb83eff5640 1 -- 172.21.15.139:0/2487976227 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fb84c011440 con 0x55a485899990 2024-09-16T01:46:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb863e91740 1 -- 172.21.15.139:0/2487976227 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb84c015070 con 0x55a485899990 2024-09-16T01:46:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb83eff5640 1 -- 172.21.15.139:0/2487976227 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb84c02f050 con 0x55a485899990 2024-09-16T01:46:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb863e91740 10 client.16179 fetch_fsmap learned FSMap version 35 2024-09-16T01:46:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb863e91740 1 -- 172.21.15.139:0/2487976227 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a4855dd620 con 0x55a485899990 2024-09-16T01:46:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb83eff5640 1 -- 172.21.15.139:0/2487976227 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84c015070 con 0x55a485899990 2024-09-16T01:46:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb83eff5640 1 -- 172.21.15.139:0/2487976227 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7fb84c022070 con 0x55a485899990 2024-09-16T01:46:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb863e91740 10 client.16179 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:46:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb863e91740 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:46:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb863e91740 20 client.16179 populate_metadata read hostname 'smithi139' 2024-09-16T01:46:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb863e91740 1 --2- 172.21.15.139:0/2487976227 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a4858b6d90 0x55a4858b9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb863e91740 4 client.16179 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:46:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.079+0000 7fb863e91740 1 -- 172.21.15.139:0/2487976227 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a48580ba20 con 0x55a4858b6d90 2024-09-16T01:46:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.080+0000 7fb85a59c640 1 --2- 172.21.15.139:0/2487976227 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a4858b6d90 0x55a4858b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.080+0000 7fb85a59c640 1 --2- 172.21.15.139:0/2487976227 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a4858b6d90 0x55a4858b9180 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab65d0a8a55aec6 server_cookie=143ff4ca14c076d0 in_seq=0 out_seq=0 2024-09-16T01:46:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.080+0000 7fb83eff5640 10 client.16179 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb83eff5640 1 -- 172.21.15.139:0/2487976227 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a48580ba20 con 0x55a4858b6d90 2024-09-16T01:46:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb83eff5640 10 client.16179 handle_command_reply: tid=0 2024-09-16T01:46:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb863e91740 1 client.16179 shutdown 2024-09-16T01:46:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb863e91740 1 -- 172.21.15.139:0/2487976227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 msgr2=0x55a4858a3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb863e91740 1 --2- 172.21.15.139:0/2487976227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a3e20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0039c0 tx=0x7fb84c004800 comp rx=0 tx=0).stop 2024-09-16T01:46:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb863e91740 1 -- 172.21.15.139:0/2487976227 shutdown_connections 2024-09-16T01:46:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb863e91740 1 --2- 172.21.15.139:0/2487976227 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a4858b6d90 0x55a4858b9180 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb863e91740 1 --2- 172.21.15.139:0/2487976227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 0x55a4858a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb83eff5640 0 client.16179 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb863e91740 1 --2- 172.21.15.139:0/2487976227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a48589a2b0 0x55a4858a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb863e91740 1 --2- 172.21.15.139:0/2487976227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a3e20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb863e91740 1 -- 172.21.15.139:0/2487976227 >> 172.21.15.139:0/2487976227 conn(0x55a485896060 msgr2=0x55a48589bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb863e91740 1 -- 172.21.15.139:0/2487976227 shutdown_connections 2024-09-16T01:46:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb863e91740 1 -- 172.21.15.139:0/2487976227 wait complete. 2024-09-16T01:46:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.081+0000 7fb863e91740 20 client.16179 trim_cache size 0 max 16384 2024-09-16T01:46:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.095+0000 7fb863e91740 1 Processor -- start 2024-09-16T01:46:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.095+0000 7fb863e91740 1 -- start start 2024-09-16T01:46:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.095+0000 7fb863e91740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.095+0000 7fb863e91740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a48589a2b0 0x55a4858be800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb863e91740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4858a6ec0 0x55a4858bed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb863e91740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1ace50 con 0x55a485899990 2024-09-16T01:46:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb863e91740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1aeb10 con 0x55a48589a2b0 2024-09-16T01:46:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb863e91740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1ad520 con 0x55a4858a6ec0 2024-09-16T01:46:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb85ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb85ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56226/0 (socket says 172.21.15.139:56226) 2024-09-16T01:46:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb85ad9d640 1 -- 172.21.15.139:0/2897174833 learned_addr learned my addr 172.21.15.139:0/2897174833 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb85a59c640 1 --2- 172.21.15.139:0/2897174833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a48589a2b0 0x55a4858be800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb85b59e640 1 --2- 172.21.15.139:0/2897174833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4858a6ec0 0x55a4858bed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb85ad9d640 1 -- 172.21.15.139:0/2897174833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4858a6ec0 msgr2=0x55a4858bed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb85ad9d640 1 --2- 172.21.15.139:0/2897174833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4858a6ec0 0x55a4858bed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb85ad9d640 1 -- 172.21.15.139:0/2897174833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a48589a2b0 msgr2=0x55a4858be800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb85ad9d640 1 --2- 172.21.15.139:0/2897174833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a48589a2b0 0x55a4858be800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb85ad9d640 1 -- 172.21.15.139:0/2897174833 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4858902f0 con 0x55a485899990 2024-09-16T01:46:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb85b59e640 1 --2- 172.21.15.139:0/2897174833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4858a6ec0 0x55a4858bed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:46:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb85ad9d640 1 --2- 172.21.15.139:0/2897174833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a5d90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b3d0 tx=0x7fb84c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6d8aa10f9b1706 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb840ff9640 1 -- 172.21.15.139:0/2897174833 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c028070 con 0x55a485899990 2024-09-16T01:46:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb840ff9640 1 -- 172.21.15.139:0/2897174833 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84c015070 con 0x55a485899990 2024-09-16T01:46:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.096+0000 7fb840ff9640 1 -- 172.21.15.139:0/2897174833 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c011b80 con 0x55a485899990 2024-09-16T01:46:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.097+0000 7fb863e91740 1 -- 172.21.15.139:0/2897174833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 msgr2=0x55a4858a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.097+0000 7fb863e91740 1 --2- 172.21.15.139:0/2897174833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a5d90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b3d0 tx=0x7fb84c004270 comp rx=0 tx=0).stop 2024-09-16T01:46:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.097+0000 7fb863e91740 1 -- 172.21.15.139:0/2897174833 shutdown_connections 2024-09-16T01:46:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.097+0000 7fb863e91740 1 --2- 172.21.15.139:0/2897174833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a4858a6ec0 0x55a4858bed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.097+0000 7fb863e91740 1 --2- 172.21.15.139:0/2897174833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a48589a2b0 0x55a4858be800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.097+0000 7fb863e91740 1 --2- 172.21.15.139:0/2897174833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a5d90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.097+0000 7fb863e91740 1 -- 172.21.15.139:0/2897174833 >> 172.21.15.139:0/2897174833 conn(0x55a485896060 msgr2=0x55a4858a5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.097+0000 7fb863e91740 1 -- 172.21.15.139:0/2897174833 shutdown_connections 2024-09-16T01:46:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.097+0000 7fb863e91740 1 -- 172.21.15.139:0/2897174833 wait complete. 2024-09-16T01:46:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.097+0000 7fb863e91740 1 -- start start 2024-09-16T01:46:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb863e91740 1 Processor -- start 2024-09-16T01:46:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb863e91740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb863e91740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a48589a2b0 0x55a4858a1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb863e91740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 0x55a4858a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb863e91740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1ae380 con 0x55a485899990 2024-09-16T01:46:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb863e91740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1adcb0 con 0x55a4858a6ec0 2024-09-16T01:46:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb863e91740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1aae30 con 0x55a48589a2b0 2024-09-16T01:46:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb863e91740 10 client.0 _ll_register_callbacks cb 0x7fb84c004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:46:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb85ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb85ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56232/0 (socket says 172.21.15.139:56232) 2024-09-16T01:46:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb85ad9d640 1 -- 172.21.15.139:0/4039136048 learned_addr learned my addr 172.21.15.139:0/4039136048 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb85a59c640 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a48589a2b0 0x55a4858a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb85b59e640 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 0x55a4858a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb85ad9d640 1 -- 172.21.15.139:0/4039136048 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a48589a2b0 msgr2=0x55a4858a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb85ad9d640 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a48589a2b0 0x55a4858a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb85ad9d640 1 -- 172.21.15.139:0/4039136048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 msgr2=0x55a4858a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb85ad9d640 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 0x55a4858a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb85ad9d640 1 -- 172.21.15.139:0/4039136048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4859207d0 con 0x55a485899990 2024-09-16T01:46:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb85a59c640 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a48589a2b0 0x55a4858a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:46:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb85b59e640 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 0x55a4858a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:46:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb85ad9d640 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a0d40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb84c004070 tx=0x7fb84c019420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83dfcc4e243f4b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb83eff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:46:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb83eff5640 1 -- 172.21.15.139:0/4039136048 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c028040 con 0x55a485899990 2024-09-16T01:46:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb83eff5640 1 -- 172.21.15.139:0/4039136048 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fb84c019af0 con 0x55a485899990 2024-09-16T01:46:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.098+0000 7fb863e91740 1 -- 172.21.15.139:0/4039136048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb84c015070 con 0x55a485899990 2024-09-16T01:46:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.099+0000 7fb83eff5640 1 -- 172.21.15.139:0/4039136048 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb84c019cf0 con 0x55a485899990 2024-09-16T01:46:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.099+0000 7fb863e91740 10 client.16185 fetch_fsmap learned FSMap version 35 2024-09-16T01:46:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.099+0000 7fb863e91740 1 -- 172.21.15.139:0/4039136048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a4859219c0 con 0x55a485899990 2024-09-16T01:46:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.099+0000 7fb83eff5640 1 -- 172.21.15.139:0/4039136048 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84c015070 con 0x55a485899990 2024-09-16T01:46:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.099+0000 7fb83eff5640 1 -- 172.21.15.139:0/4039136048 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7fb84c022040 con 0x55a485899990 2024-09-16T01:46:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.099+0000 7fb863e91740 10 client.16185 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:46:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.099+0000 7fb863e91740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:46:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.099+0000 7fb863e91740 20 client.16185 populate_metadata read hostname 'smithi139' 2024-09-16T01:46:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.099+0000 7fb863e91740 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a485922110 0x55a485924530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.099+0000 7fb863e91740 4 client.16185 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:46:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.099+0000 7fb863e91740 1 -- 172.21.15.139:0/4039136048 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a485805d70 con 0x55a485922110 2024-09-16T01:46:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.099+0000 7fb85a59c640 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a485922110 0x55a485924530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.100+0000 7fb85a59c640 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a485922110 0x55a485924530 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c32ebc41580a11 server_cookie=9ebd1401a3e73cf3 in_seq=0 out_seq=0 2024-09-16T01:46:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.100+0000 7fb83eff5640 10 client.16185 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:26.115 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:46:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb83eff5640 1 -- 172.21.15.139:0/4039136048 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a485805d70 con 0x55a485922110 2024-09-16T01:46:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb83eff5640 10 client.16185 handle_command_reply: tid=0 2024-09-16T01:46:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb863e91740 1 client.16185 shutdown 2024-09-16T01:46:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:46:26.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:46:26.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:46:26.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:46:26.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb863e91740 1 -- 172.21.15.139:0/4039136048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 msgr2=0x55a4858a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:26.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb863e91740 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a0d40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb84c004070 tx=0x7fb84c019420 comp rx=0 tx=0).stop 2024-09-16T01:46:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb863e91740 1 -- 172.21.15.139:0/4039136048 shutdown_connections 2024-09-16T01:46:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb863e91740 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a485922110 0x55a485924530 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb863e91740 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4858a6ec0 0x55a4858a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb863e91740 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a48589a2b0 0x55a4858a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb83eff5640 0 client.16185 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb863e91740 1 --2- 172.21.15.139:0/4039136048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a485899990 0x55a4858a0d40 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb863e91740 1 -- 172.21.15.139:0/4039136048 >> 172.21.15.139:0/4039136048 conn(0x55a485896060 msgr2=0x55a4858951c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb863e91740 1 -- 172.21.15.139:0/4039136048 shutdown_connections 2024-09-16T01:46:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb863e91740 1 -- 172.21.15.139:0/4039136048 wait complete. 2024-09-16T01:46:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.132+0000 7fb863e91740 20 client.16185 trim_cache size 0 max 16384 2024-09-16T01:46:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.133+0000 7fb8610f1640 1 -- 172.21.15.139:0/2396149019 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb8440a34e0 msgr2=0x7fb8440a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.133+0000 7fb8610f1640 1 --2- 172.21.15.139:0/2396149019 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb8440a34e0 0x7fb8440a59a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb854005e30 tx=0x7fb8540095c0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.133+0000 7fb8610f1640 1 -- 172.21.15.139:0/2396149019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb85c101390 msgr2=0x7fb85c1977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.133+0000 7fb8610f1640 1 --2- 172.21.15.139:0/2396149019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb85c101390 0x7fb85c1977b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb85000d950 tx=0x7fb85000de10 comp rx=0 tx=0).stop 2024-09-16T01:46:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.133+0000 7fb8610f1640 1 -- 172.21.15.139:0/2396149019 shutdown_connections 2024-09-16T01:46:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.133+0000 7fb8610f1640 1 --2- 172.21.15.139:0/2396149019 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb8440a34e0 0x7fb8440a59a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.133+0000 7fb8610f1640 1 --2- 172.21.15.139:0/2396149019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb85c106a00 0x7fb85c19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.133+0000 7fb8610f1640 1 --2- 172.21.15.139:0/2396149019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb85c101390 0x7fb85c1977b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.133+0000 7fb8610f1640 1 --2- 172.21.15.139:0/2396149019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c1009f0 0x7fb85c197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.133+0000 7fb8610f1640 1 -- 172.21.15.139:0/2396149019 >> 172.21.15.139:0/2396149019 conn(0x7fb85c0fc6b0 msgr2=0x7fb85c0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.133+0000 7fb8610f1640 1 -- 172.21.15.139:0/2396149019 shutdown_connections 2024-09-16T01:46:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:26.133+0000 7fb8610f1640 1 -- 172.21.15.139:0/2396149019 wait complete. 2024-09-16T01:46:26.368 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:46:27.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.078+0000 7f9844cbe640 1 -- 172.21.15.139:0/465658593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9840106f20 msgr2=0x7f984010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:27.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.078+0000 7f9844cbe640 1 --2- 172.21.15.139:0/465658593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9840106f20 0x7f984010eaa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9838009900 tx=0x7f983802a410 comp rx=0 tx=0).stop 2024-09-16T01:46:27.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.078+0000 7f9844cbe640 1 -- 172.21.15.139:0/465658593 shutdown_connections 2024-09-16T01:46:27.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.078+0000 7f9844cbe640 1 --2- 172.21.15.139:0/465658593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9840106f20 0x7f984010eaa0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:27.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.078+0000 7f9844cbe640 1 --2- 172.21.15.139:0/465658593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98401027a0 0x7f98401067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:27.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.078+0000 7f9844cbe640 1 --2- 172.21.15.139:0/465658593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9840101df0 0x7f98401021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:27.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.078+0000 7f9844cbe640 1 -- 172.21.15.139:0/465658593 >> 172.21.15.139:0/465658593 conn(0x7f98400fd8f0 msgr2=0x7f98400ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:27.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.078+0000 7f9844cbe640 1 -- 172.21.15.139:0/465658593 shutdown_connections 2024-09-16T01:46:27.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.078+0000 7f9844cbe640 1 -- 172.21.15.139:0/465658593 wait complete. 2024-09-16T01:46:27.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f9844cbe640 1 Processor -- start 2024-09-16T01:46:27.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f9844cbe640 1 -- start start 2024-09-16T01:46:27.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f9844cbe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9840101df0 0x7f9840198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:27.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f9844cbe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98401027a0 0x7f9840198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:27.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f9844cbe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9840106f20 0x7f984019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:27.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f9844cbe640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9840111f80 con 0x7f9840101df0 2024-09-16T01:46:27.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f9844cbe640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9840111e00 con 0x7f98401027a0 2024-09-16T01:46:27.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f9844cbe640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9840112100 con 0x7f9840106f20 2024-09-16T01:46:27.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f983e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9840101df0 0x7f9840198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:27.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f983e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9840101df0 0x7f9840198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56242/0 (socket says 172.21.15.139:56242) 2024-09-16T01:46:27.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f983e59c640 1 -- 172.21.15.139:0/2338595426 learned_addr learned my addr 172.21.15.139:0/2338595426 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:27.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f983dd9b640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98401027a0 0x7f9840198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:27.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f983ed9d640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9840106f20 0x7f984019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:27.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f983e59c640 1 -- 172.21.15.139:0/2338595426 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9840106f20 msgr2=0x7f984019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:27.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f983e59c640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9840106f20 0x7f984019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:27.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f983e59c640 1 -- 172.21.15.139:0/2338595426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98401027a0 msgr2=0x7f9840198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:27.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f983e59c640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98401027a0 0x7f9840198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:27.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f983e59c640 1 -- 172.21.15.139:0/2338595426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984019d3d0 con 0x7f9840101df0 2024-09-16T01:46:27.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f983dd9b640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98401027a0 0x7f9840198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:46:27.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.079+0000 7f983e59c640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9840101df0 0x7f9840198420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f983000cce0 tx=0x7f983000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:27.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.080+0000 7f98377fe640 1 -- 172.21.15.139:0/2338595426 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983000f650 con 0x7f9840101df0 2024-09-16T01:46:27.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.080+0000 7f9844cbe640 1 -- 172.21.15.139:0/2338595426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984019d6c0 con 0x7f9840101df0 2024-09-16T01:46:27.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.080+0000 7f98377fe640 1 -- 172.21.15.139:0/2338595426 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983000fcd0 con 0x7f9840101df0 2024-09-16T01:46:27.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.080+0000 7f98377fe640 1 -- 172.21.15.139:0/2338595426 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830018c70 con 0x7f9840101df0 2024-09-16T01:46:27.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.080+0000 7f9844cbe640 1 -- 172.21.15.139:0/2338595426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98401a4fa0 con 0x7f9840101df0 2024-09-16T01:46:27.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.081+0000 7f9844cbe640 1 -- 172.21.15.139:0/2338595426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98401021d0 con 0x7f9840101df0 2024-09-16T01:46:27.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.082+0000 7f98377fe640 1 -- 172.21.15.139:0/2338595426 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f983000f7f0 con 0x7f9840101df0 2024-09-16T01:46:27.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.082+0000 7f98377fe640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f982c0a33b0 0x7f982c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:27.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.083+0000 7f983dd9b640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f982c0a33b0 0x7f982c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:27.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.083+0000 7f98377fe640 1 -- 172.21.15.139:0/2338595426 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f98300d0640 con 0x7f9840101df0 2024-09-16T01:46:27.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.083+0000 7f983dd9b640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f982c0a33b0 0x7f982c0a5870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9828005fd0 tx=0x7f982800b5d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:27.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.087+0000 7f98377fe640 1 -- 172.21.15.139:0/2338595426 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9830014030 con 0x7f9840101df0 2024-09-16T01:46:27.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.288+0000 7f9844cbe640 1 -- 172.21.15.139:0/2338595426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9840199780 con 0x7f9840101df0 2024-09-16T01:46:27.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.288+0000 7f98377fe640 1 -- 172.21.15.139:0/2338595426 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+12016 (secure 0 0 0) 0x7f98300bc090 con 0x7f9840101df0 2024-09-16T01:46:27.291 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:46:27.291 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":35,"btime":"2024-09-16T01:45:19:161714+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:45:18.328499+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:46:27.291 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 35 2024-09-16T01:46:27.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.290+0000 7f9844cbe640 1 -- 172.21.15.139:0/2338595426 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f982c0a33b0 msgr2=0x7f982c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:27.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.290+0000 7f9844cbe640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f982c0a33b0 0x7f982c0a5870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9828005fd0 tx=0x7f982800b5d0 comp rx=0 tx=0).stop 2024-09-16T01:46:27.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.291+0000 7f9844cbe640 1 -- 172.21.15.139:0/2338595426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9840101df0 msgr2=0x7f9840198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:27.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.291+0000 7f9844cbe640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9840101df0 0x7f9840198420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f983000cce0 tx=0x7f983000a620 comp rx=0 tx=0).stop 2024-09-16T01:46:27.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.291+0000 7f9844cbe640 1 -- 172.21.15.139:0/2338595426 shutdown_connections 2024-09-16T01:46:27.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.291+0000 7f9844cbe640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f982c0a33b0 0x7f982c0a5870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:27.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.291+0000 7f9844cbe640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9840106f20 0x7f984019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:27.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.291+0000 7f9844cbe640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98401027a0 0x7f9840198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:27.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.291+0000 7f9844cbe640 1 --2- 172.21.15.139:0/2338595426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9840101df0 0x7f9840198420 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:27.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.291+0000 7f9844cbe640 1 -- 172.21.15.139:0/2338595426 >> 172.21.15.139:0/2338595426 conn(0x7f98400fd8f0 msgr2=0x7f98400fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:27.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.291+0000 7f9844cbe640 1 -- 172.21.15.139:0/2338595426 shutdown_connections 2024-09-16T01:46:27.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:27.291+0000 7f9844cbe640 1 -- 172.21.15.139:0/2338595426 wait complete. 2024-09-16T01:46:27.613 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:27 smithi139 ceph-mon[31175]: pgmap v184: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.8 MiB/s wr, 1.03k op/s 2024-09-16T01:46:27.613 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:27 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2338595426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:46:27.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:27 smithi176 ceph-mon[35324]: pgmap v184: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.8 MiB/s wr, 1.03k op/s 2024-09-16T01:46:27.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:27 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2338595426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:46:27.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:27 smithi164 ceph-mon[35121]: pgmap v184: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.8 MiB/s wr, 1.03k op/s 2024-09-16T01:46:27.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:27 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2338595426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:46:27.960 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 63cecc48 --release --await' 2024-09-16T01:46:27.961 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 63cecc48 --release --await 2024-09-16T01:46:28.242 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:46:28.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.736+0000 7fecf70bb640 1 -- 172.21.15.139:0/3064836541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecf0101420 msgr2=0x7fecf010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:28.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.736+0000 7fecf70bb640 1 --2- 172.21.15.139:0/3064836541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecf0101420 0x7fecf010d950 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fece8009cd0 tx=0x7fece802a8e0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.736+0000 7fecf70bb640 1 -- 172.21.15.139:0/3064836541 shutdown_connections 2024-09-16T01:46:28.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.736+0000 7fecf70bb640 1 --2- 172.21.15.139:0/3064836541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecf0101420 0x7fecf010d950 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.736+0000 7fecf70bb640 1 --2- 172.21.15.139:0/3064836541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecf0100a80 0x7fecf0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.736+0000 7fecf70bb640 1 --2- 172.21.15.139:0/3064836541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecf0106a80 0x7fecf0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.736+0000 7fecf70bb640 1 -- 172.21.15.139:0/3064836541 >> 172.21.15.139:0/3064836541 conn(0x7fecf00fc7a0 msgr2=0x7fecf00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:28.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.736+0000 7fecf70bb640 1 -- 172.21.15.139:0/3064836541 shutdown_connections 2024-09-16T01:46:28.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.736+0000 7fecf70bb640 1 -- 172.21.15.139:0/3064836541 wait complete. 2024-09-16T01:46:28.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7fecf70bb640 1 Processor -- start 2024-09-16T01:46:28.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7fecf70bb640 1 -- start start 2024-09-16T01:46:28.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7fecf70bb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecf0100a80 0x7fecf0197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7fecf70bb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecf0101420 0x7fecf0197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7fecf70bb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecf0106a80 0x7fecf019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7fecf70bb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fecf0110e80 con 0x7fecf0101420 2024-09-16T01:46:28.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7fecf70bb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fecf0110d00 con 0x7fecf0106a80 2024-09-16T01:46:28.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7fecf70bb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fecf0111000 con 0x7fecf0100a80 2024-09-16T01:46:28.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7feceffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecf0101420 0x7fecf0197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7feceffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecf0101420 0x7fecf0197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56270/0 (socket says 172.21.15.139:56270) 2024-09-16T01:46:28.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7feceffff640 1 -- 172.21.15.139:0/1510403281 learned_addr learned my addr 172.21.15.139:0/1510403281 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:28.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7fecf4e57640 1 --2- 172.21.15.139:0/1510403281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecf0100a80 0x7fecf0197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7fecf5658640 1 --2- 172.21.15.139:0/1510403281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecf0106a80 0x7fecf019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7feceffff640 1 -- 172.21.15.139:0/1510403281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecf0100a80 msgr2=0x7fecf0197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:28.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7feceffff640 1 --2- 172.21.15.139:0/1510403281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecf0100a80 0x7fecf0197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7feceffff640 1 -- 172.21.15.139:0/1510403281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecf0106a80 msgr2=0x7fecf019bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:28.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7feceffff640 1 --2- 172.21.15.139:0/1510403281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecf0106a80 0x7fecf019bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.737+0000 7feceffff640 1 -- 172.21.15.139:0/1510403281 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf019c2b0 con 0x7fecf0101420 2024-09-16T01:46:28.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.738+0000 7feceffff640 1 --2- 172.21.15.139:0/1510403281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecf0101420 0x7fecf0197840 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00d770 tx=0x7fecdc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:28.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.738+0000 7fecedffb640 1 -- 172.21.15.139:0/1510403281 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc017070 con 0x7fecf0101420 2024-09-16T01:46:28.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.738+0000 7fecedffb640 1 -- 172.21.15.139:0/1510403281 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecdc005500 con 0x7fecf0101420 2024-09-16T01:46:28.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.738+0000 7fecf70bb640 1 -- 172.21.15.139:0/1510403281 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf019c5a0 con 0x7fecf0101420 2024-09-16T01:46:28.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.738+0000 7fecedffb640 1 -- 172.21.15.139:0/1510403281 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc004e00 con 0x7fecf0101420 2024-09-16T01:46:28.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.738+0000 7fecf70bb640 1 -- 172.21.15.139:0/1510403281 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf01a3e80 con 0x7fecf0101420 2024-09-16T01:46:28.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.740+0000 7fecedffb640 1 -- 172.21.15.139:0/1510403281 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fecdc004fa0 con 0x7fecf0101420 2024-09-16T01:46:28.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.740+0000 7fecf9494740 1 Processor -- start 2024-09-16T01:46:28.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.740+0000 7fecf9494740 1 -- start start 2024-09-16T01:46:28.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf9494740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679eedf20 0x55f679eee300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf9494740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679ee8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecedffb640 1 --2- 172.21.15.139:0/1510403281 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fece00a33b0 0x7fece00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf9494740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 0x55f679ef4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf9494740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f679e48620 con 0x55f679ee8620 2024-09-16T01:46:28.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf9494740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f679defe50 con 0x55f679ee8fe0 2024-09-16T01:46:28.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf9494740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f679e445e0 con 0x55f679eedf20 2024-09-16T01:46:28.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf4e57640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679ee8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf4e57640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679ee8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56274/0 (socket says 172.21.15.139:56274) 2024-09-16T01:46:28.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf4e57640 1 -- 172.21.15.139:0/2104051494 learned_addr learned my addr 172.21.15.139:0/2104051494 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:28.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecedffb640 1 -- 172.21.15.139:0/1510403281 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fecdc0cfb50 con 0x7fecf0101420 2024-09-16T01:46:28.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf5658640 1 --2- 172.21.15.139:0/2104051494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679eedf20 0x55f679eee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7feceffff640 1 --2- 172.21.15.139:0/1510403281 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fece00a33b0 0x7fece00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf5658640 1 --2- 172.21.15.139:0/2104051494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 0x55f679ef4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf4e57640 1 -- 172.21.15.139:0/2104051494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679eedf20 msgr2=0x55f679eee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:28.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf4e57640 1 --2- 172.21.15.139:0/2104051494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679eedf20 0x55f679eee300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf4e57640 1 -- 172.21.15.139:0/2104051494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 msgr2=0x55f679ef4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:28.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf4e57640 1 --2- 172.21.15.139:0/2104051494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 0x55f679ef4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf4e57640 1 -- 172.21.15.139:0/2104051494 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f679de0d40 con 0x55f679ee8620 2024-09-16T01:46:28.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf5658640 1 --2- 172.21.15.139:0/2104051494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679eedf20 0x55f679eee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:46:28.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7feceffff640 1 --2- 172.21.15.139:0/1510403281 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fece00a33b0 0x7fece00a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fecdc09ba10 tx=0x7fecdc005bb0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:28.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecf4e57640 1 --2- 172.21.15.139:0/2104051494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679ee8aa0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fece400bc80 tx=0x7fece40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6227310c7180e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:28.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.741+0000 7fecdaffd640 1 -- 172.21.15.139:0/2104051494 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece401c480 con 0x55f679ee8620 2024-09-16T01:46:28.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.742+0000 7fecdaffd640 1 -- 172.21.15.139:0/2104051494 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece401cb00 con 0x55f679ee8620 2024-09-16T01:46:28.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.742+0000 7fecdaffd640 1 -- 172.21.15.139:0/2104051494 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4011a90 con 0x55f679ee8620 2024-09-16T01:46:28.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.742+0000 7fecf9494740 1 -- 172.21.15.139:0/2104051494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 msgr2=0x55f679ee8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:28.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.742+0000 7fecf9494740 1 --2- 172.21.15.139:0/2104051494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679ee8aa0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fece400bc80 tx=0x7fece40107a0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.742+0000 7fecf9494740 1 -- 172.21.15.139:0/2104051494 shutdown_connections 2024-09-16T01:46:28.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.742+0000 7fecf9494740 1 --2- 172.21.15.139:0/2104051494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 0x55f679ef4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.742+0000 7fecf9494740 1 --2- 172.21.15.139:0/2104051494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679ee8aa0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.742+0000 7fecf9494740 1 --2- 172.21.15.139:0/2104051494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679eedf20 0x55f679eee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.742+0000 7fecf9494740 1 -- 172.21.15.139:0/2104051494 >> 172.21.15.139:0/2104051494 conn(0x55f679edf0e0 msgr2=0x55f679edf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:28.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.742+0000 7fecf9494740 1 -- 172.21.15.139:0/2104051494 shutdown_connections 2024-09-16T01:46:28.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.742+0000 7fecf9494740 1 -- 172.21.15.139:0/2104051494 wait complete. 2024-09-16T01:46:28.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.743+0000 7fecf9494740 1 -- start start 2024-09-16T01:46:28.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.743+0000 7fecf9494740 1 Processor -- start 2024-09-16T01:46:28.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.743+0000 7fecf9494740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679ef2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.743+0000 7fecf9494740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 0x55f679ef2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.743+0000 7fecf9494740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679eedf20 0x55f679ef2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.743+0000 7fecf9494740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f679ef5260 con 0x55f679ee8620 2024-09-16T01:46:28.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.743+0000 7fecf9494740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f679dd0f10 con 0x55f679ee8fe0 2024-09-16T01:46:28.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.743+0000 7fecf9494740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f679ef53e0 con 0x55f679eedf20 2024-09-16T01:46:28.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.743+0000 7fecf9494740 10 client.0 _ll_register_callbacks cb 0x55f679df4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:46:28.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.743+0000 7fecf5658640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679ef2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.743+0000 7fecf5658640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679ef2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56276/0 (socket says 172.21.15.139:56276) 2024-09-16T01:46:28.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.743+0000 7fecf5658640 1 -- 172.21.15.139:0/1561548063 learned_addr learned my addr 172.21.15.139:0/1561548063 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:28.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.743+0000 7fecf4e57640 1 --2- 172.21.15.139:0/1561548063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 0x55f679ef2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.744+0000 7fecf5658640 1 -- 172.21.15.139:0/1561548063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679eedf20 msgr2=0x55f679ef2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:28.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.744+0000 7fecf5658640 1 --2- 172.21.15.139:0/1561548063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679eedf20 0x55f679ef2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.744+0000 7fecf5658640 1 -- 172.21.15.139:0/1561548063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 msgr2=0x55f679ef2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:28.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.744+0000 7fecf5658640 1 --2- 172.21.15.139:0/1561548063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 0x55f679ef2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.744+0000 7fecf5658640 1 -- 172.21.15.139:0/1561548063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f679b57580 con 0x55f679ee8620 2024-09-16T01:46:28.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.744+0000 7fecf5658640 1 --2- 172.21.15.139:0/1561548063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679ef2130 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fece8035000 tx=0x7fece8031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662fc7ce7d101e41 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:28.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.744+0000 7feccffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:46:28.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.744+0000 7feccffff640 1 -- 172.21.15.139:0/1561548063 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8003970 con 0x55f679ee8620 2024-09-16T01:46:28.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.744+0000 7fecf9494740 1 -- 172.21.15.139:0/1561548063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fece401cb00 con 0x55f679ee8620 2024-09-16T01:46:28.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.744+0000 7feccffff640 1 -- 172.21.15.139:0/1561548063 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fece8003b10 con 0x55f679ee8620 2024-09-16T01:46:28.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.744+0000 7feccffff640 1 -- 172.21.15.139:0/1561548063 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fece8053050 con 0x55f679ee8620 2024-09-16T01:46:28.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.744+0000 7fecf9494740 10 client.16203 fetch_fsmap learned FSMap version 35 2024-09-16T01:46:28.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.744+0000 7fecf9494740 1 -- 172.21.15.139:0/1561548063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f679c21960 con 0x55f679ee8620 2024-09-16T01:46:28.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.744+0000 7feccffff640 1 -- 172.21.15.139:0/1561548063 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece401cb00 con 0x55f679ee8620 2024-09-16T01:46:28.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.745+0000 7feccffff640 1 -- 172.21.15.139:0/1561548063 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7fece8030070 con 0x55f679ee8620 2024-09-16T01:46:28.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.745+0000 7fecf9494740 10 client.16203 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:46:28.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.745+0000 7fecf9494740 10 client.16203 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:46:28.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.745+0000 7fecf9494740 20 client.16203 populate_metadata read hostname 'smithi139' 2024-09-16T01:46:28.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.745+0000 7fecf9494740 1 --2- 172.21.15.139:0/1561548063 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f679f022d0 0x55f679f046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.745+0000 7fecf9494740 4 client.16203 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:46:28.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.745+0000 7fecf9494740 1 -- 172.21.15.139:0/1561548063 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f679e55b90 con 0x55f679f022d0 2024-09-16T01:46:28.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.745+0000 7fecf4e57640 1 --2- 172.21.15.139:0/1561548063 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f679f022d0 0x55f679f046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.745+0000 7fecf4e57640 1 --2- 172.21.15.139:0/1561548063 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f679f022d0 0x55f679f046f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1255c375afcff4ec server_cookie=ef99c254f0e7737a in_seq=0 out_seq=0 2024-09-16T01:46:28.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.745+0000 7feccffff640 10 client.16203 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:28.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.746+0000 7feccffff640 1 -- 172.21.15.139:0/1561548063 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f679e55b90 con 0x55f679f022d0 2024-09-16T01:46:28.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.746+0000 7feccffff640 10 client.16203 handle_command_reply: tid=0 2024-09-16T01:46:28.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.746+0000 7fecf9494740 1 client.16203 shutdown 2024-09-16T01:46:28.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.746+0000 7fecf9494740 1 -- 172.21.15.139:0/1561548063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 msgr2=0x55f679ef2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:28.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.746+0000 7fecf9494740 1 --2- 172.21.15.139:0/1561548063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679ef2130 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fece8035000 tx=0x7fece8031b60 comp rx=0 tx=0).stop 2024-09-16T01:46:28.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.746+0000 7fecf9494740 1 -- 172.21.15.139:0/1561548063 shutdown_connections 2024-09-16T01:46:28.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.746+0000 7fecf9494740 1 --2- 172.21.15.139:0/1561548063 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f679f022d0 0x55f679f046f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.746+0000 7fecf9494740 1 --2- 172.21.15.139:0/1561548063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679eedf20 0x55f679ef2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.746+0000 7fecf9494740 1 --2- 172.21.15.139:0/1561548063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 0x55f679ef2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.746+0000 7feccffff640 0 client.16203 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:28.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.746+0000 7fecf9494740 1 --2- 172.21.15.139:0/1561548063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679ef2130 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.746+0000 7fecf9494740 1 -- 172.21.15.139:0/1561548063 >> 172.21.15.139:0/1561548063 conn(0x55f679edf0e0 msgr2=0x55f679eed210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:28.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.747+0000 7fecf9494740 1 -- 172.21.15.139:0/1561548063 shutdown_connections 2024-09-16T01:46:28.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.747+0000 7fecf9494740 1 -- 172.21.15.139:0/1561548063 wait complete. 2024-09-16T01:46:28.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.747+0000 7fecf9494740 20 client.16203 trim_cache size 0 max 16384 2024-09-16T01:46:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.763+0000 7fecf9494740 1 Processor -- start 2024-09-16T01:46:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.763+0000 7fecf9494740 1 -- start start 2024-09-16T01:46:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.763+0000 7fecf9494740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679ee8620 0x55f679eea690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.763+0000 7fecf9494740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 0x55f679ee7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.763+0000 7fecf9494740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679eedf20 0x55f679ef08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.763+0000 7fecf9494740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01acea0 con 0x55f679eedf20 2024-09-16T01:46:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.763+0000 7fecf9494740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01aeb60 con 0x55f679ee8fe0 2024-09-16T01:46:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.763+0000 7fecf9494740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01ad570 con 0x55f679ee8620 2024-09-16T01:46:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.763+0000 7fecf5658640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679eedf20 0x55f679ef08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.763+0000 7fecf5658640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679eedf20 0x55f679ef08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56286/0 (socket says 172.21.15.139:56286) 2024-09-16T01:46:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.763+0000 7fecf5658640 1 -- 172.21.15.139:0/1524910582 learned_addr learned my addr 172.21.15.139:0/1524910582 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.763+0000 7fecf5658640 1 --2- 172.21.15.139:0/1524910582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679ee8620 0x55f679eea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.764+0000 7fecf5658640 1 -- 172.21.15.139:0/1524910582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679ee8620 msgr2=0x55f679eea690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.764+0000 7fecf5658640 1 --2- 172.21.15.139:0/1524910582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679ee8620 0x55f679eea690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.764+0000 7fecf4e57640 1 --2- 172.21.15.139:0/1524910582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 0x55f679ee7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.764+0000 7fecf5658640 1 -- 172.21.15.139:0/1524910582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 msgr2=0x55f679ee7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.764+0000 7fecf5658640 1 --2- 172.21.15.139:0/1524910582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 0x55f679ee7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.764+0000 7fecf5658640 1 -- 172.21.15.139:0/1524910582 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f679eddbb0 con 0x55f679eedf20 2024-09-16T01:46:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.764+0000 7fecf5658640 1 --2- 172.21.15.139:0/1524910582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679eedf20 0x55f679ef08d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fece8000f80 tx=0x7fece802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afde962dbd3cf04 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.764+0000 7fecda7fc640 1 -- 172.21.15.139:0/1524910582 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8004720 con 0x55f679eedf20 2024-09-16T01:46:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.764+0000 7fecda7fc640 1 -- 172.21.15.139:0/1524910582 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece8036040 con 0x55f679eedf20 2024-09-16T01:46:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.764+0000 7fecda7fc640 1 -- 172.21.15.139:0/1524910582 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece803bcd0 con 0x55f679eedf20 2024-09-16T01:46:28.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 -- 172.21.15.139:0/1524910582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679eedf20 msgr2=0x55f679ef08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 --2- 172.21.15.139:0/1524910582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679eedf20 0x55f679ef08d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fece8000f80 tx=0x7fece802a8e0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 -- 172.21.15.139:0/1524910582 shutdown_connections 2024-09-16T01:46:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 --2- 172.21.15.139:0/1524910582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679eedf20 0x55f679ef08d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 --2- 172.21.15.139:0/1524910582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679ee8fe0 0x55f679ee7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 --2- 172.21.15.139:0/1524910582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679ee8620 0x55f679eea690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 -- 172.21.15.139:0/1524910582 >> 172.21.15.139:0/1524910582 conn(0x55f679edf0e0 msgr2=0x55f679ee7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 -- 172.21.15.139:0/1524910582 shutdown_connections 2024-09-16T01:46:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 -- 172.21.15.139:0/1524910582 wait complete. 2024-09-16T01:46:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 -- start start 2024-09-16T01:46:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 Processor -- start 2024-09-16T01:46:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679efa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679ee8fe0 0x55f679efaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679eedf20 0x55f679f08ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01ae3d0 con 0x55f679ee8620 2024-09-16T01:46:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01add00 con 0x55f679eedf20 2024-09-16T01:46:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01aae80 con 0x55f679ee8fe0 2024-09-16T01:46:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.765+0000 7fecf9494740 10 client.0 _ll_register_callbacks cb 0x7fece80317b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:46:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7fecf5658640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679efa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7fecf5658640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679efa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56294/0 (socket says 172.21.15.139:56294) 2024-09-16T01:46:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7fecf5658640 1 -- 172.21.15.139:0/208507141 learned_addr learned my addr 172.21.15.139:0/208507141 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7fecf4e57640 1 --2- 172.21.15.139:0/208507141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679ee8fe0 0x55f679efaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7fecf5658640 1 --2- 172.21.15.139:0/208507141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679eedf20 0x55f679f08ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7fecf5658640 1 -- 172.21.15.139:0/208507141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679ee8fe0 msgr2=0x55f679efaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7fecf5658640 1 --2- 172.21.15.139:0/208507141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679ee8fe0 0x55f679efaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7fecf5658640 1 -- 172.21.15.139:0/208507141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679eedf20 msgr2=0x55f679f08ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7fecf5658640 1 --2- 172.21.15.139:0/208507141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679eedf20 0x55f679f08ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7fecf5658640 1 -- 172.21.15.139:0/208507141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f679f09530 con 0x55f679ee8620 2024-09-16T01:46:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7fecf5658640 1 --2- 172.21.15.139:0/208507141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679efa660 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fece80098d0 tx=0x7fece8031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0df568a09daf838 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7feccffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:46:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7feccffff640 1 -- 172.21.15.139:0/208507141 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece803b480 con 0x55f679ee8620 2024-09-16T01:46:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7feccffff640 1 -- 172.21.15.139:0/208507141 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fece805d040 con 0x55f679ee8620 2024-09-16T01:46:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7fecf9494740 1 -- 172.21.15.139:0/208507141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fece8036040 con 0x55f679ee8620 2024-09-16T01:46:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7feccffff640 1 -- 172.21.15.139:0/208507141 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fece803b620 con 0x55f679ee8620 2024-09-16T01:46:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7fecf9494740 10 client.16209 fetch_fsmap learned FSMap version 35 2024-09-16T01:46:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.766+0000 7fecf9494740 1 -- 172.21.15.139:0/208507141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f679f0a830 con 0x55f679ee8620 2024-09-16T01:46:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.767+0000 7feccffff640 1 -- 172.21.15.139:0/208507141 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece8036040 con 0x55f679ee8620 2024-09-16T01:46:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.767+0000 7feccffff640 1 -- 172.21.15.139:0/208507141 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7fece8063900 con 0x55f679ee8620 2024-09-16T01:46:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.767+0000 7fecf9494740 10 client.16209 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:46:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.767+0000 7fecf9494740 10 client.16209 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:46:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.767+0000 7fecf9494740 20 client.16209 populate_metadata read hostname 'smithi139' 2024-09-16T01:46:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.767+0000 7fecf9494740 1 --2- 172.21.15.139:0/208507141 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f679f6dfe0 0x55f679f703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.767+0000 7fecf9494740 4 client.16209 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "63cecc48", "release": true, "await": true}] 2024-09-16T01:46:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.767+0000 7fecf9494740 1 -- 172.21.15.139:0/208507141 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "63cecc48", "release": true, "await": true}) -- 0x55f679e500a0 con 0x55f679f6dfe0 2024-09-16T01:46:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.767+0000 7fecf4e57640 1 --2- 172.21.15.139:0/208507141 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f679f6dfe0 0x55f679f703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.768+0000 7fecf4e57640 1 --2- 172.21.15.139:0/208507141 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f679f6dfe0 0x55f679f703d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d94dcb479f8ec24 server_cookie=bba72fd6cbd1a6d3 in_seq=0 out_seq=0 2024-09-16T01:46:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:28.768+0000 7feccffff640 10 client.16209 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:29 smithi139 ceph-mon[31175]: pgmap v185: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 1.4 MiB/s wr, 1.03k op/s 2024-09-16T01:46:29.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:29 smithi176 ceph-mon[35324]: pgmap v185: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 1.4 MiB/s wr, 1.03k op/s 2024-09-16T01:46:29.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.756+0000 7feccffff640 1 -- 172.21.15.139:0/208507141 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f679e500a0 con 0x55f679f6dfe0 2024-09-16T01:46:29.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.756+0000 7feccffff640 10 client.16209 handle_command_reply: tid=0 2024-09-16T01:46:29.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.756+0000 7fecf9494740 1 client.16209 shutdown 2024-09-16T01:46:29.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.756+0000 7fecf9494740 1 -- 172.21.15.139:0/208507141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 msgr2=0x55f679efa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:29.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.756+0000 7fecf9494740 1 --2- 172.21.15.139:0/208507141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679efa660 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fece80098d0 tx=0x7fece8031740 comp rx=0 tx=0).stop 2024-09-16T01:46:29.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.756+0000 7fecf9494740 1 -- 172.21.15.139:0/208507141 shutdown_connections 2024-09-16T01:46:29.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.756+0000 7fecf9494740 1 --2- 172.21.15.139:0/208507141 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f679f6dfe0 0x55f679f703d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:29.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.756+0000 7fecf9494740 1 --2- 172.21.15.139:0/208507141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f679eedf20 0x55f679f08ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:29.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.756+0000 7fecf9494740 1 --2- 172.21.15.139:0/208507141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f679ee8fe0 0x55f679efaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:29.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.756+0000 7fecf9494740 1 --2- 172.21.15.139:0/208507141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f679ee8620 0x55f679efa660 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:29.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.756+0000 7feccffff640 0 client.16209 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:29.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.756+0000 7fecf9494740 1 -- 172.21.15.139:0/208507141 >> 172.21.15.139:0/208507141 conn(0x55f679edf0e0 msgr2=0x55f679ee7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:29.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.756+0000 7fecf9494740 1 -- 172.21.15.139:0/208507141 shutdown_connections 2024-09-16T01:46:29.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.756+0000 7fecf9494740 1 -- 172.21.15.139:0/208507141 wait complete. 2024-09-16T01:46:29.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.757+0000 7fecf9494740 20 client.16209 trim_cache size 0 max 16384 2024-09-16T01:46:29.760 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:46:29.760 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 35, 2024-09-16T01:46:29.760 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:46:29.760 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 24, 2024-09-16T01:46:29.760 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:46:29.760 INFO:teuthology.orchestra.run.smithi139.stdout: "63cecc48": { 2024-09-16T01:46:29.760 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 24, 2024-09-16T01:46:29.761 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:46:29.761 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:46:29.761 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:46:29.761 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:46:29.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:46:29.761 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:46:29.761 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 145.3, 2024-09-16T01:46:29.761 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:46:29.761 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:46:29.761 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:46:29.761 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:46:29.761 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:46:29.761 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:46:29.761 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:29.762 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:29.762 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:29.762 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:29.762 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:46:29.762 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:46:29.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.757+0000 7fecf70bb640 1 -- 172.21.15.139:0/1510403281 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fece00a33b0 msgr2=0x7fece00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:29.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.757+0000 7fecf70bb640 1 --2- 172.21.15.139:0/1510403281 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fece00a33b0 0x7fece00a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fecdc09ba10 tx=0x7fecdc005bb0 comp rx=0 tx=0).stop 2024-09-16T01:46:29.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.757+0000 7fecf70bb640 1 -- 172.21.15.139:0/1510403281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecf0101420 msgr2=0x7fecf0197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:29.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.757+0000 7fecf70bb640 1 --2- 172.21.15.139:0/1510403281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecf0101420 0x7fecf0197840 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00d770 tx=0x7fecdc00dc30 comp rx=0 tx=0).stop 2024-09-16T01:46:29.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.757+0000 7fecf70bb640 1 -- 172.21.15.139:0/1510403281 shutdown_connections 2024-09-16T01:46:29.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.758+0000 7fecf70bb640 1 --2- 172.21.15.139:0/1510403281 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fece00a33b0 0x7fece00a5870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:29.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.758+0000 7fecf70bb640 1 --2- 172.21.15.139:0/1510403281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecf0106a80 0x7fecf019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:29.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.758+0000 7fecf70bb640 1 --2- 172.21.15.139:0/1510403281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecf0101420 0x7fecf0197840 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:29.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.758+0000 7fecf70bb640 1 --2- 172.21.15.139:0/1510403281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecf0100a80 0x7fecf0197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:29.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.758+0000 7fecf70bb640 1 -- 172.21.15.139:0/1510403281 >> 172.21.15.139:0/1510403281 conn(0x7fecf00fc7a0 msgr2=0x7fecf00fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:29.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.758+0000 7fecf70bb640 1 -- 172.21.15.139:0/1510403281 shutdown_connections 2024-09-16T01:46:29.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:46:29.758+0000 7fecf70bb640 1 -- 172.21.15.139:0/1510403281 wait complete. 2024-09-16T01:46:29.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:29 smithi164 ceph-mon[35121]: pgmap v185: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 1.4 MiB/s wr, 1.03k op/s 2024-09-16T01:46:30.501 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '63cecc48', total seconds elapsed: 29.7 2024-09-16T01:46:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:31 smithi139 ceph-mon[31175]: pgmap v186: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 1.3 MiB/s wr, 909 op/s 2024-09-16T01:46:31.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:31 smithi176 ceph-mon[35324]: pgmap v186: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 1.3 MiB/s wr, 909 op/s 2024-09-16T01:46:31.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:31 smithi164 ceph-mon[35121]: pgmap v186: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 1.3 MiB/s wr, 909 op/s 2024-09-16T01:46:33.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:33 smithi139 ceph-mon[31175]: pgmap v187: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.2 MiB/s wr, 827 op/s 2024-09-16T01:46:33.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:33 smithi176 ceph-mon[35324]: pgmap v187: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.2 MiB/s wr, 827 op/s 2024-09-16T01:46:33.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:33 smithi164 ceph-mon[35121]: pgmap v187: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.2 MiB/s wr, 827 op/s 2024-09-16T01:46:35.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:35 smithi139 ceph-mon[31175]: pgmap v188: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 583 op/s 2024-09-16T01:46:35.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:35 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:46:35.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:35 smithi176 ceph-mon[35324]: pgmap v188: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 583 op/s 2024-09-16T01:46:35.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:35 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:46:35.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:35 smithi164 ceph-mon[35121]: pgmap v188: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 583 op/s 2024-09-16T01:46:35.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:35 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:46:35.968 INFO:tasks.workunit.client.0.smithi139.stdout: 486 952014 743 647580 976 208142 465 2024-09-16T01:46:37.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:37 smithi139 ceph-mon[31175]: pgmap v189: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.4 MiB/s wr, 402 op/s 2024-09-16T01:46:37.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:37 smithi176 ceph-mon[35324]: pgmap v189: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.4 MiB/s wr, 402 op/s 2024-09-16T01:46:37.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:37 smithi164 ceph-mon[35121]: pgmap v189: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.4 MiB/s wr, 402 op/s 2024-09-16T01:46:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.603+0000 7f81a98ac640 1 -- 172.21.15.139:0/1825821642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a411b140 msgr2=0x7f81a411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.603+0000 7f81a98ac640 1 --2- 172.21.15.139:0/1825821642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a411b140 0x7f81a411d530 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8194009900 tx=0x7f819402a410 comp rx=0 tx=0).stop 2024-09-16T01:46:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.606+0000 7f81a98ac640 1 -- 172.21.15.139:0/1825821642 shutdown_connections 2024-09-16T01:46:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.606+0000 7f81a98ac640 1 --2- 172.21.15.139:0/1825821642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a411b140 0x7f81a411d530 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.606+0000 7f81a98ac640 1 --2- 172.21.15.139:0/1825821642 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a41095d0 0x7f81a4109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.606+0000 7f81a98ac640 1 --2- 172.21.15.139:0/1825821642 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81a41091f0 0x7f81a4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.606+0000 7f81a98ac640 1 -- 172.21.15.139:0/1825821642 >> 172.21.15.139:0/1825821642 conn(0x7f81a406cc60 msgr2=0x7f81a406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.606+0000 7f81a98ac640 1 -- 172.21.15.139:0/1825821642 shutdown_connections 2024-09-16T01:46:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.606+0000 7f81a98ac640 1 -- 172.21.15.139:0/1825821642 wait complete. 2024-09-16T01:46:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.607+0000 7f81a98ac640 1 Processor -- start 2024-09-16T01:46:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.608+0000 7f81a98ac640 1 -- start start 2024-09-16T01:46:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.608+0000 7f81a98ac640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a41091f0 0x7f81a41a6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.608+0000 7f81a98ac640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a41095d0 0x7f81a41a74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.608+0000 7f81a98ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81a411b140 0x7f81a41ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.608+0000 7f81a98ac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4120b20 con 0x7f81a411b140 2024-09-16T01:46:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.608+0000 7f81a98ac640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41209a0 con 0x7f81a41095d0 2024-09-16T01:46:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.608+0000 7f81a98ac640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4120ca0 con 0x7f81a41091f0 2024-09-16T01:46:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.608+0000 7f81a27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a41095d0 0x7f81a41a74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.608+0000 7f81a27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a41095d0 0x7f81a41a74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:55294/0 (socket says 172.21.15.139:55294) 2024-09-16T01:46:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.608+0000 7f81a27fc640 1 -- 172.21.15.139:0/3881058388 learned_addr learned my addr 172.21.15.139:0/3881058388 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.608+0000 7f81a2ffd640 1 --2- 172.21.15.139:0/3881058388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a41091f0 0x7f81a41a6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.609+0000 7f81a27fc640 1 -- 172.21.15.139:0/3881058388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a41091f0 msgr2=0x7f81a41a6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.609+0000 7f81a27fc640 1 --2- 172.21.15.139:0/3881058388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a41091f0 0x7f81a41a6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.609+0000 7f81a27fc640 1 -- 172.21.15.139:0/3881058388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81a411b140 msgr2=0x7f81a41ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.609+0000 7f81a27fc640 1 --2- 172.21.15.139:0/3881058388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81a411b140 0x7f81a41ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.609+0000 7f81a27fc640 1 -- 172.21.15.139:0/3881058388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a41abfa0 con 0x7f81a41095d0 2024-09-16T01:46:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.609+0000 7f81a27fc640 1 --2- 172.21.15.139:0/3881058388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a41095d0 0x7f81a41a74f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f819800d950 tx=0x7f819800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.612+0000 7f81a88aa640 1 -- 172.21.15.139:0/3881058388 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198017030 con 0x7f81a41095d0 2024-09-16T01:46:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.613+0000 7f81a88aa640 1 -- 172.21.15.139:0/3881058388 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8198004020 con 0x7f81a41095d0 2024-09-16T01:46:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.614+0000 7f81a88aa640 1 -- 172.21.15.139:0/3881058388 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81980047a0 con 0x7f81a41095d0 2024-09-16T01:46:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.614+0000 7f81a98ac640 1 -- 172.21.15.139:0/3881058388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a41ac290 con 0x7f81a41095d0 2024-09-16T01:46:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.614+0000 7f81a98ac640 1 -- 172.21.15.139:0/3881058388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a41b3b20 con 0x7f81a41095d0 2024-09-16T01:46:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.621+0000 7f81ac64d740 1 Processor -- start 2024-09-16T01:46:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.621+0000 7f81ac64d740 1 -- start start 2024-09-16T01:46:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.621+0000 7f81ac64d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef98cc0 0x564fdef990a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.622+0000 7f81ac64d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 0x564fdefa3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.622+0000 7f81a37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef98cc0 0x564fdef990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.622+0000 7f81a37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef98cc0 0x564fdef990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:38916/0 (socket says 172.21.15.139:38916) 2024-09-16T01:46:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.622+0000 7f81a37fe640 1 -- 172.21.15.139:0/4096005356 learned_addr learned my addr 172.21.15.139:0/4096005356 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.622+0000 7f81ac64d740 1 --2- 172.21.15.139:0/4096005356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa3e80 0x564fdefa6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.622+0000 7f81ac64d740 1 -- 172.21.15.139:0/4096005356 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564fdeefd2f0 con 0x564fdef995e0 2024-09-16T01:46:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.622+0000 7f81ac64d740 1 -- 172.21.15.139:0/4096005356 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564fdeea5270 con 0x564fdefa3e80 2024-09-16T01:46:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.622+0000 7f81ac64d740 1 -- 172.21.15.139:0/4096005356 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564fdeef92b0 con 0x564fdef98cc0 2024-09-16T01:46:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.622+0000 7f81a27fc640 1 --2- 172.21.15.139:0/4096005356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa3e80 0x564fdefa6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.623+0000 7f81a37fe640 1 -- 172.21.15.139:0/4096005356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa3e80 msgr2=0x564fdefa6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.623+0000 7f81a37fe640 1 --2- 172.21.15.139:0/4096005356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa3e80 0x564fdefa6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.623+0000 7f81a37fe640 1 -- 172.21.15.139:0/4096005356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 msgr2=0x564fdefa3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.623+0000 7f81a37fe640 1 --2- 172.21.15.139:0/4096005356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 0x564fdefa3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.623+0000 7f81a37fe640 1 -- 172.21.15.139:0/4096005356 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fdeea4340 con 0x564fdef98cc0 2024-09-16T01:46:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.623+0000 7f81a37fe640 1 --2- 172.21.15.139:0/4096005356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef98cc0 0x564fdef990a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8194005e70 tx=0x7f8194032c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3321316fc3e5521f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.623+0000 7f81a88aa640 1 -- 172.21.15.139:0/3881058388 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f81980041d0 con 0x7f81a41095d0 2024-09-16T01:46:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.625+0000 7f81a88aa640 1 --2- 172.21.15.139:0/3881058388 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f818c0a3420 0x7f818c0a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.626+0000 7f81a88aa640 1 -- 172.21.15.139:0/3881058388 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f81980cf060 con 0x7f81a41095d0 2024-09-16T01:46:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.626+0000 7f81a2ffd640 1 --2- 172.21.15.139:0/3881058388 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f818c0a3420 0x7f818c0a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.627+0000 7f81a2ffd640 1 --2- 172.21.15.139:0/3881058388 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f818c0a3420 0x7f818c0a58e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f819c009950 tx=0x7f819c0098e0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.627+0000 7f8191ffb640 1 -- 172.21.15.139:0/4096005356 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194040070 con 0x564fdef98cc0 2024-09-16T01:46:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.628+0000 7f8191ffb640 1 -- 172.21.15.139:0/4096005356 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8194004040 con 0x564fdef98cc0 2024-09-16T01:46:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.629+0000 7f81ac64d740 1 -- 172.21.15.139:0/4096005356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef98cc0 msgr2=0x564fdef990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.629+0000 7f81ac64d740 1 --2- 172.21.15.139:0/4096005356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef98cc0 0x564fdef990a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8194005e70 tx=0x7f8194032c20 comp rx=0 tx=0).stop 2024-09-16T01:46:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.629+0000 7f81ac64d740 1 -- 172.21.15.139:0/4096005356 shutdown_connections 2024-09-16T01:46:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.629+0000 7f81ac64d740 1 --2- 172.21.15.139:0/4096005356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa3e80 0x564fdefa6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.629+0000 7f81ac64d740 1 --2- 172.21.15.139:0/4096005356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 0x564fdefa3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.629+0000 7f81ac64d740 1 --2- 172.21.15.139:0/4096005356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef98cc0 0x564fdef990a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.629+0000 7f81ac64d740 1 -- 172.21.15.139:0/4096005356 >> 172.21.15.139:0/4096005356 conn(0x564fdef95230 msgr2=0x564fdef95610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.631+0000 7f81ac64d740 1 -- 172.21.15.139:0/4096005356 shutdown_connections 2024-09-16T01:46:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.631+0000 7f81ac64d740 1 -- 172.21.15.139:0/4096005356 wait complete. 2024-09-16T01:46:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.632+0000 7f81ac64d740 1 -- start start 2024-09-16T01:46:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.633+0000 7f81ac64d740 1 Processor -- start 2024-09-16T01:46:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.633+0000 7f81ac64d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 0x564fdefa38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.633+0000 7f81ac64d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa3e80 0x564fdefa0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.633+0000 7f81ac64d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdefa1270 0x564fdefaffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.633+0000 7f81ac64d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564fdefa6aa0 con 0x564fdef995e0 2024-09-16T01:46:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.633+0000 7f81ac64d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564fdee86290 con 0x564fdefa3e80 2024-09-16T01:46:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.633+0000 7f81ac64d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564fdefa6c20 con 0x564fdefa1270 2024-09-16T01:46:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.633+0000 7f81ac64d740 10 client.0 _ll_register_callbacks cb 0x564fdeeb7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:46:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.634+0000 7f81a37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdefa1270 0x564fdefaffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.634+0000 7f81a27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa3e80 0x564fdefa0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.634+0000 7f81a2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 0x564fdefa38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.634+0000 7f81a2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 0x564fdefa38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58800/0 (socket says 172.21.15.139:58800) 2024-09-16T01:46:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.634+0000 7f81a2ffd640 1 -- 172.21.15.139:0/3052225457 learned_addr learned my addr 172.21.15.139:0/3052225457 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.634+0000 7f81a27fc640 1 -- 172.21.15.139:0/3052225457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdefa1270 msgr2=0x564fdefaffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.634+0000 7f81a27fc640 1 --2- 172.21.15.139:0/3052225457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdefa1270 0x564fdefaffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.634+0000 7f81a27fc640 1 -- 172.21.15.139:0/3052225457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 msgr2=0x564fdefa38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.634+0000 7f81a27fc640 1 --2- 172.21.15.139:0/3052225457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 0x564fdefa38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.634+0000 7f81a27fc640 1 -- 172.21.15.139:0/3052225457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fdec10210 con 0x564fdefa3e80 2024-09-16T01:46:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.635+0000 7f81a27fc640 1 --2- 172.21.15.139:0/3052225457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa3e80 0x564fdefa0b80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f81980cc3a0 tx=0x7f81980045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6561dd4ee28ad46d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.639+0000 7f8186ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:46:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.639+0000 7f8186ffd640 1 -- 172.21.15.139:0/3052225457 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819809aaf0 con 0x564fdefa3e80 2024-09-16T01:46:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.639+0000 7f8186ffd640 1 -- 172.21.15.139:0/3052225457 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f81980cc620 con 0x564fdefa3e80 2024-09-16T01:46:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.639+0000 7f81ac64d740 1 -- 172.21.15.139:0/3052225457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564fdef0aa20 con 0x564fdefa3e80 2024-09-16T01:46:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.640+0000 7f8186ffd640 1 -- 172.21.15.139:0/3052225457 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f819809a910 con 0x564fdefa3e80 2024-09-16T01:46:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.640+0000 7f81ac64d740 10 client.25694 fetch_fsmap learned FSMap version 35 2024-09-16T01:46:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.640+0000 7f81ac64d740 1 -- 172.21.15.139:0/3052225457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564fdecdc620 con 0x564fdefa3e80 2024-09-16T01:46:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.640+0000 7f8186ffd640 1 -- 172.21.15.139:0/3052225457 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564fdef0aa20 con 0x564fdefa3e80 2024-09-16T01:46:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.640+0000 7f8186ffd640 1 -- 172.21.15.139:0/3052225457 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7f81980d10f0 con 0x564fdefa3e80 2024-09-16T01:46:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.640+0000 7f81ac64d740 10 client.25694 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:46:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.640+0000 7f81ac64d740 10 client.25694 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:46:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.640+0000 7f81ac64d740 20 client.25694 populate_metadata read hostname 'smithi139' 2024-09-16T01:46:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.641+0000 7f81ac64d740 1 --2- 172.21.15.139:0/3052225457 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564fdefb46f0 0x564fdefb6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.641+0000 7f81ac64d740 4 client.25694 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:46:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.641+0000 7f81ac64d740 1 -- 172.21.15.139:0/3052225457 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fdef04d70 con 0x564fdefb46f0 2024-09-16T01:46:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.641+0000 7f81a2ffd640 1 --2- 172.21.15.139:0/3052225457 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564fdefb46f0 0x564fdefb6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.642+0000 7f8186ffd640 10 client.25694 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.643+0000 7f81a2ffd640 1 --2- 172.21.15.139:0/3052225457 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564fdefb46f0 0x564fdefb6b10 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54c819ab66fb8ba3 server_cookie=48365ef3324f1be9 in_seq=0 out_seq=0 2024-09-16T01:46:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.643+0000 7f8186ffd640 1 -- 172.21.15.139:0/3052225457 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fdef04d70 con 0x564fdefb46f0 2024-09-16T01:46:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.644+0000 7f8186ffd640 10 client.25694 handle_command_reply: tid=0 2024-09-16T01:46:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.644+0000 7f81ac64d740 1 client.25694 shutdown 2024-09-16T01:46:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.668+0000 7f81ac64d740 1 -- 172.21.15.139:0/3052225457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa3e80 msgr2=0x564fdefa0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.668+0000 7f81ac64d740 1 --2- 172.21.15.139:0/3052225457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa3e80 0x564fdefa0b80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f81980cc3a0 tx=0x7f81980045b0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.668+0000 7f81ac64d740 1 -- 172.21.15.139:0/3052225457 shutdown_connections 2024-09-16T01:46:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.668+0000 7f81ac64d740 1 --2- 172.21.15.139:0/3052225457 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564fdefb46f0 0x564fdefb6b10 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.668+0000 7f81ac64d740 1 --2- 172.21.15.139:0/3052225457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdefa1270 0x564fdefaffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.668+0000 7f81ac64d740 1 --2- 172.21.15.139:0/3052225457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa3e80 0x564fdefa0b80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.668+0000 7f81ac64d740 1 --2- 172.21.15.139:0/3052225457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 0x564fdefa38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.668+0000 7f81ac64d740 1 -- 172.21.15.139:0/3052225457 >> 172.21.15.139:0/3052225457 conn(0x564fdef95230 msgr2=0x564fdef9a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.668+0000 7f8186ffd640 0 client.25694 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.669+0000 7f81ac64d740 1 -- 172.21.15.139:0/3052225457 shutdown_connections 2024-09-16T01:46:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.669+0000 7f81ac64d740 1 -- 172.21.15.139:0/3052225457 wait complete. 2024-09-16T01:46:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.669+0000 7f81ac64d740 20 client.25694 trim_cache size 0 max 16384 2024-09-16T01:46:39.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:39 smithi176 ceph-mon[35324]: pgmap v190: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-09-16T01:46:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.705+0000 7f81ac64d740 1 Processor -- start 2024-09-16T01:46:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.705+0000 7f81ac64d740 1 -- start start 2024-09-16T01:46:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.706+0000 7f81ac64d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef95230 0x564fdefa0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.706+0000 7f81ac64d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 0x564fdefafd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.707+0000 7f81a2ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef95230 0x564fdefa0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.707+0000 7f81a2ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef95230 0x564fdefa0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:38932/0 (socket says 172.21.15.139:38932) 2024-09-16T01:46:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.707+0000 7f81a27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 0x564fdefafd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.707+0000 7f81a27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 0x564fdefafd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58808/0 (socket says 172.21.15.139:58808) 2024-09-16T01:46:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.707+0000 7f81a2ffd640 1 -- 172.21.15.139:0/1152573790 learned_addr learned my addr 172.21.15.139:0/1152573790 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.707+0000 7f81ac64d740 1 --2- 172.21.15.139:0/1152573790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa1270 0x564fdefa5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.707+0000 7f81ac64d740 1 -- 172.21.15.139:0/1152573790 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41bd990 con 0x564fdef995e0 2024-09-16T01:46:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.707+0000 7f81ac64d740 1 -- 172.21.15.139:0/1152573790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41bf650 con 0x564fdefa1270 2024-09-16T01:46:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.707+0000 7f81a37fe640 1 --2- 172.21.15.139:0/1152573790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa1270 0x564fdefa5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.707+0000 7f81ac64d740 1 -- 172.21.15.139:0/1152573790 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41be060 con 0x564fdef95230 2024-09-16T01:46:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.708+0000 7f81a2ffd640 1 -- 172.21.15.139:0/1152573790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa1270 msgr2=0x564fdefa5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.708+0000 7f81a2ffd640 1 --2- 172.21.15.139:0/1152573790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa1270 0x564fdefa5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.708+0000 7f81a2ffd640 1 -- 172.21.15.139:0/1152573790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 msgr2=0x564fdefafd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.708+0000 7f81a2ffd640 1 --2- 172.21.15.139:0/1152573790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 0x564fdefafd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.708+0000 7f81a2ffd640 1 -- 172.21.15.139:0/1152573790 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fdefb02d0 con 0x564fdef95230 2024-09-16T01:46:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.708+0000 7f81a27fc640 1 --2- 172.21.15.139:0/1152573790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 0x564fdefafd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:46:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.709+0000 7f81a2ffd640 1 --2- 172.21.15.139:0/1152573790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef95230 0x564fdefa0b30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f819c005fd0 tx=0x7f819c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d31e74159c743c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.709+0000 7f81917fa640 1 -- 172.21.15.139:0/1152573790 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c012e20 con 0x564fdef95230 2024-09-16T01:46:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.709+0000 7f81917fa640 1 -- 172.21.15.139:0/1152573790 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819c02d070 con 0x564fdef95230 2024-09-16T01:46:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.710+0000 7f81ac64d740 1 -- 172.21.15.139:0/1152573790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef95230 msgr2=0x564fdefa0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.710+0000 7f81ac64d740 1 --2- 172.21.15.139:0/1152573790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef95230 0x564fdefa0b30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f819c005fd0 tx=0x7f819c013000 comp rx=0 tx=0).stop 2024-09-16T01:46:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.711+0000 7f81ac64d740 1 -- 172.21.15.139:0/1152573790 shutdown_connections 2024-09-16T01:46:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.711+0000 7f81ac64d740 1 --2- 172.21.15.139:0/1152573790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa1270 0x564fdefa5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.711+0000 7f81ac64d740 1 --2- 172.21.15.139:0/1152573790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdef995e0 0x564fdefafd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.711+0000 7f81ac64d740 1 --2- 172.21.15.139:0/1152573790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef95230 0x564fdefa0b30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.711+0000 7f81ac64d740 1 -- 172.21.15.139:0/1152573790 >> 172.21.15.139:0/1152573790 conn(0x564fdefa63d0 msgr2=0x564fdefa0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.711+0000 7f81ac64d740 1 -- 172.21.15.139:0/1152573790 shutdown_connections 2024-09-16T01:46:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.711+0000 7f81ac64d740 1 -- 172.21.15.139:0/1152573790 wait complete. 2024-09-16T01:46:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.712+0000 7f81ac64d740 1 -- start start 2024-09-16T01:46:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.712+0000 7f81ac64d740 1 Processor -- start 2024-09-16T01:46:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.713+0000 7f81ac64d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef995e0 0x564fdefbb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.713+0000 7f81ac64d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa1270 0x564fdefbb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.713+0000 7f81ac64d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdefa3e80 0x564fdefbbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.713+0000 7f81ac64d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41beec0 con 0x564fdefa3e80 2024-09-16T01:46:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.713+0000 7f81ac64d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41be7f0 con 0x564fdefa1270 2024-09-16T01:46:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.713+0000 7f81ac64d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41bb970 con 0x564fdef995e0 2024-09-16T01:46:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.713+0000 7f81ac64d740 10 client.0 _ll_register_callbacks cb 0x7f8198004660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:46:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.714+0000 7f81a27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa1270 0x564fdefbb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.714+0000 7f81a2ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef995e0 0x564fdefbb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.714+0000 7f81a27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa1270 0x564fdefbb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:55342/0 (socket says 172.21.15.139:55342) 2024-09-16T01:46:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.714+0000 7f81a27fc640 1 -- 172.21.15.139:0/2613634788 learned_addr learned my addr 172.21.15.139:0/2613634788 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.714+0000 7f81a37fe640 1 --2- 172.21.15.139:0/2613634788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdefa3e80 0x564fdefbbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.714+0000 7f81a27fc640 1 -- 172.21.15.139:0/2613634788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef995e0 msgr2=0x564fdefbb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.714+0000 7f81a27fc640 1 --2- 172.21.15.139:0/2613634788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef995e0 0x564fdefbb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.714+0000 7f81a27fc640 1 -- 172.21.15.139:0/2613634788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdefa3e80 msgr2=0x564fdefbbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.714+0000 7f81a27fc640 1 --2- 172.21.15.139:0/2613634788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdefa3e80 0x564fdefbbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.714+0000 7f81a27fc640 1 -- 172.21.15.139:0/2613634788 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fdefab210 con 0x564fdefa1270 2024-09-16T01:46:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.714+0000 7f81a37fe640 1 --2- 172.21.15.139:0/2613634788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdefa3e80 0x564fdefbbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:46:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.715+0000 7f81a27fc640 1 --2- 172.21.15.139:0/2613634788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa1270 0x564fdefbb910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f819809acb0 tx=0x7f819809f450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97c07d5a5dcbf2dd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.715+0000 7f8186ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:46:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.715+0000 7f8186ffd640 1 -- 172.21.15.139:0/2613634788 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819809a730 con 0x564fdefa1270 2024-09-16T01:46:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.715+0000 7f81ac64d740 1 -- 172.21.15.139:0/2613634788 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564fdef99430 con 0x564fdefa1270 2024-09-16T01:46:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.715+0000 7f8186ffd640 1 -- 172.21.15.139:0/2613634788 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f81980cc620 con 0x564fdefa1270 2024-09-16T01:46:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.715+0000 7f8186ffd640 1 -- 172.21.15.139:0/2613634788 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8198097250 con 0x564fdefa1270 2024-09-16T01:46:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.715+0000 7f81ac64d740 10 client.25700 fetch_fsmap learned FSMap version 35 2024-09-16T01:46:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.715+0000 7f81ac64d740 1 -- 172.21.15.139:0/2613634788 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564fdefac550 con 0x564fdefa1270 2024-09-16T01:46:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.717+0000 7f8186ffd640 1 -- 172.21.15.139:0/2613634788 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564fdef99430 con 0x564fdefa1270 2024-09-16T01:46:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.717+0000 7f8186ffd640 1 -- 172.21.15.139:0/2613634788 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 35) ==== 8418+0+0 (secure 0 0 0) 0x7f81980ccaf0 con 0x564fdefa1270 2024-09-16T01:46:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.720+0000 7f81ac64d740 10 client.25700 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T01:46:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.720+0000 7f81ac64d740 10 client.25700 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:46:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.720+0000 7f81ac64d740 20 client.25700 populate_metadata read hostname 'smithi139' 2024-09-16T01:46:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.720+0000 7f81ac64d740 1 --2- 172.21.15.139:0/2613634788 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564fdf01f3b0 0x564fdf0217a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.720+0000 7f81ac64d740 4 client.25700 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:46:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.720+0000 7f81ac64d740 1 -- 172.21.15.139:0/2613634788 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564fdee81900 con 0x564fdf01f3b0 2024-09-16T01:46:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.720+0000 7f81a2ffd640 1 --2- 172.21.15.139:0/2613634788 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564fdf01f3b0 0x564fdf0217a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.721+0000 7f81a2ffd640 1 --2- 172.21.15.139:0/2613634788 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564fdf01f3b0 0x564fdf0217a0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e388a7a1334b9311 server_cookie=359680230113ad62 in_seq=0 out_seq=0 2024-09-16T01:46:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:39.721+0000 7f8186ffd640 10 client.25700 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:39 smithi164 ceph-mon[35121]: pgmap v190: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-09-16T01:46:39.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:39 smithi139 ceph-mon[31175]: pgmap v190: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-09-16T01:46:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.430+0000 7f8186ffd640 1 -- 172.21.15.139:0/2613634788 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564fdee81900 con 0x564fdf01f3b0 2024-09-16T01:46:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.430+0000 7f8186ffd640 10 client.25700 handle_command_reply: tid=0 2024-09-16T01:46:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.430+0000 7f81ac64d740 1 client.25700 shutdown 2024-09-16T01:46:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.430+0000 7f81ac64d740 1 -- 172.21.15.139:0/2613634788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa1270 msgr2=0x564fdefbb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.430+0000 7f81ac64d740 1 --2- 172.21.15.139:0/2613634788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa1270 0x564fdefbb910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f819809acb0 tx=0x7f819809f450 comp rx=0 tx=0).stop 2024-09-16T01:46:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.430+0000 7f81ac64d740 1 -- 172.21.15.139:0/2613634788 shutdown_connections 2024-09-16T01:46:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.430+0000 7f81ac64d740 1 --2- 172.21.15.139:0/2613634788 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564fdf01f3b0 0x564fdf0217a0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.430+0000 7f81ac64d740 1 --2- 172.21.15.139:0/2613634788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdefa3e80 0x564fdefbbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.431+0000 7f81ac64d740 1 --2- 172.21.15.139:0/2613634788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fdefa1270 0x564fdefbb910 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.431+0000 7f8186ffd640 0 client.25700 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.431+0000 7f81ac64d740 1 --2- 172.21.15.139:0/2613634788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564fdef995e0 0x564fdefbb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.431+0000 7f81ac64d740 1 -- 172.21.15.139:0/2613634788 >> 172.21.15.139:0/2613634788 conn(0x564fdefa63d0 msgr2=0x564fdef93a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.431+0000 7f81ac64d740 1 -- 172.21.15.139:0/2613634788 shutdown_connections 2024-09-16T01:46:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.431+0000 7f81ac64d740 1 -- 172.21.15.139:0/2613634788 wait complete. 2024-09-16T01:46:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.431+0000 7f81ac64d740 20 client.25700 trim_cache size 0 max 16384 2024-09-16T01:46:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:46:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:46:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:46:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:46:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.432+0000 7f81a98ac640 1 -- 172.21.15.139:0/3881058388 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f818c0a3420 msgr2=0x7f818c0a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.432+0000 7f81a98ac640 1 --2- 172.21.15.139:0/3881058388 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f818c0a3420 0x7f818c0a58e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f819c009950 tx=0x7f819c0098e0 comp rx=0 tx=0).stop 2024-09-16T01:46:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.432+0000 7f81a98ac640 1 -- 172.21.15.139:0/3881058388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a41095d0 msgr2=0x7f81a41a74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.432+0000 7f81a98ac640 1 --2- 172.21.15.139:0/3881058388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a41095d0 0x7f81a41a74f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f819800d950 tx=0x7f819800de10 comp rx=0 tx=0).stop 2024-09-16T01:46:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.432+0000 7f81a98ac640 1 -- 172.21.15.139:0/3881058388 shutdown_connections 2024-09-16T01:46:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.432+0000 7f81a98ac640 1 --2- 172.21.15.139:0/3881058388 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f818c0a3420 0x7f818c0a58e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.432+0000 7f81a98ac640 1 --2- 172.21.15.139:0/3881058388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81a411b140 0x7f81a41ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.432+0000 7f81a98ac640 1 --2- 172.21.15.139:0/3881058388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a41095d0 0x7f81a41a74f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.432+0000 7f81a98ac640 1 --2- 172.21.15.139:0/3881058388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a41091f0 0x7f81a41a6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.432+0000 7f81a98ac640 1 -- 172.21.15.139:0/3881058388 >> 172.21.15.139:0/3881058388 conn(0x7f81a406cc60 msgr2=0x7f81a411b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.432+0000 7f81a98ac640 1 -- 172.21.15.139:0/3881058388 shutdown_connections 2024-09-16T01:46:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:40.432+0000 7f81a98ac640 1 -- 172.21.15.139:0/3881058388 wait complete. 2024-09-16T01:46:40.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:40 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:46:40.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:40 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:46:40.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:40 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:46:40.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:40 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:46:40.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:40 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:46:40.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:40 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:46:40.829 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:40 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:46:40.829 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:40 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:46:40.829 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:40 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:46:41.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:41 smithi164 ceph-mon[35121]: pgmap v191: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 152 op/s 2024-09-16T01:46:41.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:41 smithi139 ceph-mon[31175]: pgmap v191: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 152 op/s 2024-09-16T01:46:41.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:41 smithi176 ceph-mon[35324]: pgmap v191: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 152 op/s 2024-09-16T01:46:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:43 smithi164 ceph-mon[35121]: pgmap v192: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 228 op/s 2024-09-16T01:46:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:43 smithi139 ceph-mon[31175]: pgmap v192: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 228 op/s 2024-09-16T01:46:43.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:43 smithi176 ceph-mon[35324]: pgmap v192: 129 pgs: 129 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 228 op/s 2024-09-16T01:46:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:45 smithi164 ceph-mon[35121]: pgmap v193: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 298 op/s 2024-09-16T01:46:45.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:45 smithi139 ceph-mon[31175]: pgmap v193: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 298 op/s 2024-09-16T01:46:45.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:45 smithi176 ceph-mon[35324]: pgmap v193: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 298 op/s 2024-09-16T01:46:45.969 INFO:tasks.workunit.client.0.smithi139.stdout: 510 2312898 1353 1595034 1000 502832 1054 2024-09-16T01:46:47.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:47 smithi164 ceph-mon[35121]: pgmap v194: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 342 op/s 2024-09-16T01:46:47.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:47 smithi139 ceph-mon[31175]: pgmap v194: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 342 op/s 2024-09-16T01:46:47.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:47 smithi176 ceph-mon[35324]: pgmap v194: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 342 op/s 2024-09-16T01:46:49.738 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:49 smithi176 ceph-mon[35324]: pgmap v195: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 314 op/s 2024-09-16T01:46:49.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:49 smithi164 ceph-mon[35121]: pgmap v195: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 314 op/s 2024-09-16T01:46:49.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:49 smithi139 ceph-mon[31175]: pgmap v195: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 314 op/s 2024-09-16T01:46:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:51 smithi164 ceph-mon[35121]: pgmap v196: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 334 op/s 2024-09-16T01:46:51.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:51 smithi139 ceph-mon[31175]: pgmap v196: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 334 op/s 2024-09-16T01:46:51.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:51 smithi176 ceph-mon[35324]: pgmap v196: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 334 op/s 2024-09-16T01:46:52.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:52 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:46:52.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:52 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:46:52.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:52 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.535+0000 7fe891d91640 1 -- 172.21.15.139:0/2974056462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88c1009b0 msgr2=0x7fe88c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.535+0000 7fe891d91640 1 --2- 172.21.15.139:0/2974056462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88c1009b0 0x7fe88c100e10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe874009930 tx=0x7fe87402a420 comp rx=0 tx=0).stop 2024-09-16T01:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.535+0000 7fe891d91640 1 -- 172.21.15.139:0/2974056462 shutdown_connections 2024-09-16T01:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.535+0000 7fe891d91640 1 --2- 172.21.15.139:0/2974056462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe88c101350 0x7fe88c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.535+0000 7fe891d91640 1 --2- 172.21.15.139:0/2974056462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88c1009b0 0x7fe88c100e10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.535+0000 7fe891d91640 1 --2- 172.21.15.139:0/2974056462 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe88c1069c0 0x7fe88c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.535+0000 7fe891d91640 1 -- 172.21.15.139:0/2974056462 >> 172.21.15.139:0/2974056462 conn(0x7fe88c077990 msgr2=0x7fe88c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.535+0000 7fe891d91640 1 -- 172.21.15.139:0/2974056462 shutdown_connections 2024-09-16T01:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.535+0000 7fe891d91640 1 -- 172.21.15.139:0/2974056462 wait complete. 2024-09-16T01:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe891d91640 1 Processor -- start 2024-09-16T01:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe891d91640 1 -- start start 2024-09-16T01:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe891d91640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe88c1009b0 0x7fe88c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe891d91640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe88c101350 0x7fe88c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe891d91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88c1069c0 0x7fe88c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe891d91640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c110e00 con 0x7fe88c1069c0 2024-09-16T01:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe891d91640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c110c80 con 0x7fe88c1009b0 2024-09-16T01:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe891d91640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c110f80 con 0x7fe88c101350 2024-09-16T01:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe88bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88c1069c0 0x7fe88c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe88b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe88c1009b0 0x7fe88c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe88bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88c1069c0 0x7fe88c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55824/0 (socket says 172.21.15.139:55824) 2024-09-16T01:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe88bfff640 1 -- 172.21.15.139:0/1898530280 learned_addr learned my addr 172.21.15.139:0/1898530280 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe88b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe88c1009b0 0x7fe88c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:45974/0 (socket says 172.21.15.139:45974) 2024-09-16T01:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe883fff640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe88c101350 0x7fe88c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe88bfff640 1 -- 172.21.15.139:0/1898530280 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe88c101350 msgr2=0x7fe88c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe88bfff640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe88c101350 0x7fe88c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe88bfff640 1 -- 172.21.15.139:0/1898530280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe88c1009b0 msgr2=0x7fe88c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe88bfff640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe88c1009b0 0x7fe88c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe88bfff640 1 -- 172.21.15.139:0/1898530280 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c19d350 con 0x7fe88c1069c0 2024-09-16T01:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe88b7fe640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe88c1009b0 0x7fe88c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe883fff640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe88c101350 0x7fe88c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.536+0000 7fe88bfff640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88c1069c0 0x7fe88c19cc20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe88400eb10 tx=0x7fe88400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.537+0000 7fe8897fa640 1 -- 172.21.15.139:0/1898530280 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884004b20 con 0x7fe88c1069c0 2024-09-16T01:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.537+0000 7fe891d91640 1 -- 172.21.15.139:0/1898530280 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88c19d640 con 0x7fe88c1069c0 2024-09-16T01:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.537+0000 7fe8897fa640 1 -- 172.21.15.139:0/1898530280 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe884004cc0 con 0x7fe88c1069c0 2024-09-16T01:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.537+0000 7fe8897fa640 1 -- 172.21.15.139:0/1898530280 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884010760 con 0x7fe88c1069c0 2024-09-16T01:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.537+0000 7fe891d91640 1 -- 172.21.15.139:0/1898530280 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88c1a4ed0 con 0x7fe88c1069c0 2024-09-16T01:46:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.539+0000 7fe8897fa640 1 -- 172.21.15.139:0/1898530280 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe884005020 con 0x7fe88c1069c0 2024-09-16T01:46:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.539+0000 7fe894c09740 1 Processor -- start 2024-09-16T01:46:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.539+0000 7fe894c09740 1 -- start start 2024-09-16T01:46:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe894c09740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 0x561af565ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe894c09740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565f2d0 0x561af566db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe894c09740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 0x561af56704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe894c09740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561af55c32f0 con 0x561af565f2d0 2024-09-16T01:46:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe894c09740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561af556b270 con 0x561af565e9b0 2024-09-16T01:46:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe894c09740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561af55bf2b0 con 0x561af566e0c0 2024-09-16T01:46:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe88b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565f2d0 0x561af566db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe88b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565f2d0 0x561af566db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55834/0 (socket says 172.21.15.139:55834) 2024-09-16T01:46:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe883fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 0x561af565ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe88b7fe640 1 -- 172.21.15.139:0/4167863493 learned_addr learned my addr 172.21.15.139:0/4167863493 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe8897fa640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe8780a3460 0x7fe8780a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe883fff640 1 --2- 172.21.15.139:0/4167863493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 0x561af56704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe88b7fe640 1 -- 172.21.15.139:0/4167863493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 msgr2=0x561af56704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe88b7fe640 1 --2- 172.21.15.139:0/4167863493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 0x561af56704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe88b7fe640 1 -- 172.21.15.139:0/4167863493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 msgr2=0x561af565ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe88b7fe640 1 --2- 172.21.15.139:0/4167863493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 0x561af565ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe88b7fe640 1 -- 172.21.15.139:0/4167863493 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af556a340 con 0x561af565f2d0 2024-09-16T01:46:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe883fff640 1 --2- 172.21.15.139:0/4167863493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 0x561af565ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:46:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe88bfff640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe8780a3460 0x7fe8780a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe8897fa640 1 -- 172.21.15.139:0/1898530280 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fe8840d0790 con 0x7fe88c1069c0 2024-09-16T01:46:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.540+0000 7fe88b7fe640 1 --2- 172.21.15.139:0/4167863493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565f2d0 0x561af566db80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe87c010480 tx=0x7fe87c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d360bcfd9aafc316 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.541+0000 7fe881ffb640 1 -- 172.21.15.139:0/4167863493 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c011760 con 0x561af565f2d0 2024-09-16T01:46:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.541+0000 7fe88bfff640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe8780a3460 0x7fe8780a5920 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe884099030 tx=0x7fe88400efd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.541+0000 7fe881ffb640 1 -- 172.21.15.139:0/4167863493 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe87c011de0 con 0x561af565f2d0 2024-09-16T01:46:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.541+0000 7fe881ffb640 1 -- 172.21.15.139:0/4167863493 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c01a380 con 0x561af565f2d0 2024-09-16T01:46:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.541+0000 7fe894c09740 1 -- 172.21.15.139:0/4167863493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565f2d0 msgr2=0x561af566db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.541+0000 7fe894c09740 1 --2- 172.21.15.139:0/4167863493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565f2d0 0x561af566db80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe87c010480 tx=0x7fe87c00bf90 comp rx=0 tx=0).stop 2024-09-16T01:46:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.541+0000 7fe894c09740 1 -- 172.21.15.139:0/4167863493 shutdown_connections 2024-09-16T01:46:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.541+0000 7fe894c09740 1 --2- 172.21.15.139:0/4167863493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 0x561af56704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.541+0000 7fe894c09740 1 --2- 172.21.15.139:0/4167863493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565f2d0 0x561af566db80 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.541+0000 7fe894c09740 1 --2- 172.21.15.139:0/4167863493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 0x561af565ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.541+0000 7fe894c09740 1 -- 172.21.15.139:0/4167863493 >> 172.21.15.139:0/4167863493 conn(0x561af565b060 msgr2=0x561af565b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.541+0000 7fe894c09740 1 -- 172.21.15.139:0/4167863493 shutdown_connections 2024-09-16T01:46:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.541+0000 7fe894c09740 1 -- 172.21.15.139:0/4167863493 wait complete. 2024-09-16T01:46:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.542+0000 7fe894c09740 1 -- start start 2024-09-16T01:46:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.542+0000 7fe894c09740 1 Processor -- start 2024-09-16T01:46:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.542+0000 7fe894c09740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565e9b0 0x561af566b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.542+0000 7fe894c09740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565f2d0 0x561af566b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.542+0000 7fe894c09740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 0x561af566bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.542+0000 7fe894c09740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561af5670c90 con 0x561af565e9b0 2024-09-16T01:46:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.542+0000 7fe894c09740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561af554c290 con 0x561af565f2d0 2024-09-16T01:46:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.542+0000 7fe894c09740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561af5670e10 con 0x561af566e0c0 2024-09-16T01:46:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.542+0000 7fe894c09740 10 client.0 _ll_register_callbacks cb 0x561af557d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:46:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.542+0000 7fe88b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565f2d0 0x561af566b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.542+0000 7fe88b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565f2d0 0x561af566b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:45994/0 (socket says 172.21.15.139:45994) 2024-09-16T01:46:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.542+0000 7fe88b7fe640 1 -- 172.21.15.139:0/914651402 learned_addr learned my addr 172.21.15.139:0/914651402 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe883fff640 1 --2- 172.21.15.139:0/914651402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565e9b0 0x561af566b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe883fff640 1 --2- 172.21.15.139:0/914651402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 0x561af566bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe88b7fe640 1 -- 172.21.15.139:0/914651402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 msgr2=0x561af566bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe88b7fe640 1 --2- 172.21.15.139:0/914651402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 0x561af566bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe88b7fe640 1 -- 172.21.15.139:0/914651402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565e9b0 msgr2=0x561af566b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe88b7fe640 1 --2- 172.21.15.139:0/914651402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565e9b0 0x561af566b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe88b7fe640 1 -- 172.21.15.139:0/914651402 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af52d6210 con 0x561af565f2d0 2024-09-16T01:46:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe883fff640 1 --2- 172.21.15.139:0/914651402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 0x561af566bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:46:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe88b7fe640 1 --2- 172.21.15.139:0/914651402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565f2d0 0x561af566b570 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe87c010dd0 tx=0x7fe87c01c6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26df461f7d6d0ec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe86affd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:46:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe86affd640 1 -- 172.21.15.139:0/914651402 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c02a060 con 0x561af565f2d0 2024-09-16T01:46:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe894c09740 1 -- 172.21.15.139:0/914651402 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe87c011de0 con 0x561af565f2d0 2024-09-16T01:46:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe86affd640 1 -- 172.21.15.139:0/914651402 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fe87c004250 con 0x561af565f2d0 2024-09-16T01:46:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe86affd640 1 -- 172.21.15.139:0/914651402 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe87c031050 con 0x561af565f2d0 2024-09-16T01:46:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe894c09740 10 client.25712 fetch_fsmap learned FSMap version 36 2024-09-16T01:46:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe894c09740 1 -- 172.21.15.139:0/914651402 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561af53a2620 con 0x561af565f2d0 2024-09-16T01:46:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.543+0000 7fe86affd640 1 -- 172.21.15.139:0/914651402 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe87c011de0 con 0x561af565f2d0 2024-09-16T01:46:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.544+0000 7fe86affd640 1 -- 172.21.15.139:0/914651402 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 36) ==== 8414+0+0 (secure 0 0 0) 0x7fe87c0118f0 con 0x561af565f2d0 2024-09-16T01:46:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.544+0000 7fe894c09740 10 client.25712 fetch_fsmap finished waiting for FSMap version 36 2024-09-16T01:46:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.544+0000 7fe894c09740 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:46:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.544+0000 7fe894c09740 20 client.25712 populate_metadata read hostname 'smithi139' 2024-09-16T01:46:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.544+0000 7fe894c09740 1 --2- 172.21.15.139:0/914651402 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561af567e000 0x561af56803f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.544+0000 7fe894c09740 4 client.25712 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:46:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.544+0000 7fe894c09740 1 -- 172.21.15.139:0/914651402 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561af55d0a20 con 0x561af567e000 2024-09-16T01:46:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.544+0000 7fe883fff640 1 --2- 172.21.15.139:0/914651402 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561af567e000 0x561af56803f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.544+0000 7fe86affd640 10 client.25712 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.544+0000 7fe883fff640 1 --2- 172.21.15.139:0/914651402 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561af567e000 0x561af56803f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd81f073459d6b77 server_cookie=cce4942131e31732 in_seq=0 out_seq=0 2024-09-16T01:46:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe86affd640 1 -- 172.21.15.139:0/914651402 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561af55d0a20 con 0x561af567e000 2024-09-16T01:46:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe86affd640 10 client.25712 handle_command_reply: tid=0 2024-09-16T01:46:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe894c09740 1 client.25712 shutdown 2024-09-16T01:46:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe894c09740 1 -- 172.21.15.139:0/914651402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565f2d0 msgr2=0x561af566b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe894c09740 1 --2- 172.21.15.139:0/914651402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565f2d0 0x561af566b570 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe87c010dd0 tx=0x7fe87c01c6b0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe894c09740 1 -- 172.21.15.139:0/914651402 shutdown_connections 2024-09-16T01:46:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe894c09740 1 --2- 172.21.15.139:0/914651402 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561af567e000 0x561af56803f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe894c09740 1 --2- 172.21.15.139:0/914651402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 0x561af566bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe86affd640 0 client.25712 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe894c09740 1 --2- 172.21.15.139:0/914651402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565f2d0 0x561af566b570 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe894c09740 1 --2- 172.21.15.139:0/914651402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565e9b0 0x561af566b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe894c09740 1 -- 172.21.15.139:0/914651402 >> 172.21.15.139:0/914651402 conn(0x561af565b060 msgr2=0x561af5660d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe894c09740 1 -- 172.21.15.139:0/914651402 shutdown_connections 2024-09-16T01:46:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe894c09740 1 -- 172.21.15.139:0/914651402 wait complete. 2024-09-16T01:46:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.545+0000 7fe894c09740 20 client.25712 trim_cache size 0 max 16384 2024-09-16T01:46:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.559+0000 7fe894c09740 1 Processor -- start 2024-09-16T01:46:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.559+0000 7fe894c09740 1 -- start start 2024-09-16T01:46:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.559+0000 7fe894c09740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 0x561af5685a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.559+0000 7fe894c09740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af565f2d0 0x561af5685f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.559+0000 7fe894c09740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af566e0c0 0x561af5668270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.559+0000 7fe894c09740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c1acd70 con 0x561af566e0c0 2024-09-16T01:46:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.559+0000 7fe894c09740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c1aea00 con 0x561af565e9b0 2024-09-16T01:46:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.559+0000 7fe894c09740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c1ad440 con 0x561af565f2d0 2024-09-16T01:46:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.560+0000 7fe883fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af566e0c0 0x561af5668270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.560+0000 7fe883fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af566e0c0 0x561af5668270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55848/0 (socket says 172.21.15.139:55848) 2024-09-16T01:46:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.560+0000 7fe883fff640 1 -- 172.21.15.139:0/4241003713 learned_addr learned my addr 172.21.15.139:0/4241003713 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.560+0000 7fe883fff640 1 --2- 172.21.15.139:0/4241003713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 0x561af5685a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.560+0000 7fe88b7fe640 1 --2- 172.21.15.139:0/4241003713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af565f2d0 0x561af5685f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.560+0000 7fe883fff640 1 -- 172.21.15.139:0/4241003713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af565f2d0 msgr2=0x561af5685f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.560+0000 7fe883fff640 1 --2- 172.21.15.139:0/4241003713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af565f2d0 0x561af5685f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.560+0000 7fe883fff640 1 -- 172.21.15.139:0/4241003713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 msgr2=0x561af5685a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.560+0000 7fe883fff640 1 --2- 172.21.15.139:0/4241003713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 0x561af5685a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.560+0000 7fe883fff640 1 -- 172.21.15.139:0/4241003713 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af565bcf0 con 0x561af566e0c0 2024-09-16T01:46:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.560+0000 7fe883fff640 1 --2- 172.21.15.139:0/4241003713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af566e0c0 0x561af5668270 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe874009930 tx=0x7fe874005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b51b7fc36c6c8c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.560+0000 7fe8817fa640 1 -- 172.21.15.139:0/4241003713 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874035040 con 0x561af566e0c0 2024-09-16T01:46:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.560+0000 7fe8817fa640 1 -- 172.21.15.139:0/4241003713 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe87403d720 con 0x561af566e0c0 2024-09-16T01:46:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.560+0000 7fe8817fa640 1 -- 172.21.15.139:0/4241003713 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874004920 con 0x561af566e0c0 2024-09-16T01:46:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.561+0000 7fe894c09740 1 -- 172.21.15.139:0/4241003713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af566e0c0 msgr2=0x561af5668270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.561+0000 7fe894c09740 1 --2- 172.21.15.139:0/4241003713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af566e0c0 0x561af5668270 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe874009930 tx=0x7fe874005b70 comp rx=0 tx=0).stop 2024-09-16T01:46:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.561+0000 7fe894c09740 1 -- 172.21.15.139:0/4241003713 shutdown_connections 2024-09-16T01:46:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.561+0000 7fe894c09740 1 --2- 172.21.15.139:0/4241003713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af566e0c0 0x561af5668270 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.561+0000 7fe894c09740 1 --2- 172.21.15.139:0/4241003713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af565f2d0 0x561af5685f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.561+0000 7fe894c09740 1 --2- 172.21.15.139:0/4241003713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 0x561af5685a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.561+0000 7fe894c09740 1 -- 172.21.15.139:0/4241003713 >> 172.21.15.139:0/4241003713 conn(0x561af565b060 msgr2=0x561af566d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.561+0000 7fe894c09740 1 -- 172.21.15.139:0/4241003713 shutdown_connections 2024-09-16T01:46:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.561+0000 7fe894c09740 1 -- 172.21.15.139:0/4241003713 wait complete. 2024-09-16T01:46:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.561+0000 7fe894c09740 1 -- start start 2024-09-16T01:46:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.561+0000 7fe894c09740 1 Processor -- start 2024-09-16T01:46:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.561+0000 7fe894c09740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 0x561af5672000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.561+0000 7fe894c09740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565f2d0 0x561af5672540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe894c09740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 0x561af5672a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe894c09740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c1ae2a0 con 0x561af565f2d0 2024-09-16T01:46:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe894c09740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c1adbd0 con 0x561af565e9b0 2024-09-16T01:46:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe894c09740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c1aad50 con 0x561af566e0c0 2024-09-16T01:46:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe894c09740 10 client.0 _ll_register_callbacks cb 0x7fe87c00be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:46:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe88b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565f2d0 0x561af5672540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe883fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 0x561af5672000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe883fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 0x561af5672000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:46020/0 (socket says 172.21.15.139:46020) 2024-09-16T01:46:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe88b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565f2d0 0x561af5672540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55860/0 (socket says 172.21.15.139:55860) 2024-09-16T01:46:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe883fff640 1 -- 172.21.15.139:0/1412222605 learned_addr learned my addr 172.21.15.139:0/1412222605 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:46:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe883fff640 1 --2- 172.21.15.139:0/1412222605 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 0x561af5672a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe88b7fe640 1 -- 172.21.15.139:0/1412222605 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 msgr2=0x561af5672a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe88b7fe640 1 --2- 172.21.15.139:0/1412222605 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 0x561af5672a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe88b7fe640 1 -- 172.21.15.139:0/1412222605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 msgr2=0x561af5672000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe88b7fe640 1 --2- 172.21.15.139:0/1412222605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 0x561af5672000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe88b7fe640 1 -- 172.21.15.139:0/1412222605 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af56e7b90 con 0x561af565f2d0 2024-09-16T01:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe883fff640 1 --2- 172.21.15.139:0/1412222605 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 0x561af5672a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe88b7fe640 1 --2- 172.21.15.139:0/1412222605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565f2d0 0x561af5672540 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00bbc0 tx=0x7fe87c005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b74ca57d34e3550 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe86affd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe86affd640 1 -- 172.21.15.139:0/1412222605 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c03a060 con 0x561af565f2d0 2024-09-16T01:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe86affd640 1 -- 172.21.15.139:0/1412222605 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7fe87c01c8c0 con 0x561af565f2d0 2024-09-16T01:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe894c09740 1 -- 172.21.15.139:0/1412222605 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe87403d720 con 0x561af565f2d0 2024-09-16T01:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe86affd640 1 -- 172.21.15.139:0/1412222605 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe87c041050 con 0x561af565f2d0 2024-09-16T01:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe894c09740 10 client.16239 fetch_fsmap learned FSMap version 36 2024-09-16T01:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.562+0000 7fe894c09740 1 -- 172.21.15.139:0/1412222605 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561af56e8e80 con 0x561af565f2d0 2024-09-16T01:46:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.563+0000 7fe86affd640 1 -- 172.21.15.139:0/1412222605 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe87403d720 con 0x561af565f2d0 2024-09-16T01:46:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.563+0000 7fe86affd640 1 -- 172.21.15.139:0/1412222605 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 36) ==== 8414+0+0 (secure 0 0 0) 0x7fe87c0035f0 con 0x561af565f2d0 2024-09-16T01:46:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.563+0000 7fe894c09740 10 client.16239 fetch_fsmap finished waiting for FSMap version 36 2024-09-16T01:46:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.563+0000 7fe894c09740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:46:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.563+0000 7fe894c09740 20 client.16239 populate_metadata read hostname 'smithi139' 2024-09-16T01:46:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.563+0000 7fe894c09740 1 --2- 172.21.15.139:0/1412222605 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561af56e9630 0x561af56eba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:46:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.563+0000 7fe894c09740 4 client.16239 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:46:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.563+0000 7fe894c09740 1 -- 172.21.15.139:0/1412222605 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561af55cad70 con 0x561af56e9630 2024-09-16T01:46:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.563+0000 7fe883fff640 1 --2- 172.21.15.139:0/1412222605 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561af56e9630 0x561af56eba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:46:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.563+0000 7fe86affd640 10 client.16239 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:53.563+0000 7fe883fff640 1 --2- 172.21.15.139:0/1412222605 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561af56e9630 0x561af56eba50 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=594e60ccec4b67 server_cookie=5269693dc354810e in_seq=0 out_seq=0 2024-09-16T01:46:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:53 smithi164 ceph-mon[35121]: pgmap v197: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.2 MiB/s wr, 530 op/s 2024-09-16T01:46:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:53 smithi139 ceph-mon[31175]: pgmap v197: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.2 MiB/s wr, 530 op/s 2024-09-16T01:46:53.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:53 smithi176 ceph-mon[35324]: pgmap v197: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.2 MiB/s wr, 530 op/s 2024-09-16T01:46:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:55 smithi164 ceph-mon[35121]: pgmap v198: 129 pgs: 129 active+clean; 1.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 582 op/s 2024-09-16T01:46:55.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:55 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:46:55.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:55 smithi139 ceph-mon[31175]: pgmap v198: 129 pgs: 129 active+clean; 1.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 582 op/s 2024-09-16T01:46:55.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:55 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:46:55.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:55 smithi176 ceph-mon[35324]: pgmap v198: 129 pgs: 129 active+clean; 1.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 582 op/s 2024-09-16T01:46:55.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:55 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:46:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.803+0000 7fe86affd640 1 -- 172.21.15.139:0/1412222605 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561af55cad70 con 0x561af56e9630 2024-09-16T01:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.803+0000 7fe86affd640 10 client.16239 handle_command_reply: tid=0 2024-09-16T01:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.803+0000 7fe894c09740 1 client.16239 shutdown 2024-09-16T01:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.804+0000 7fe894c09740 1 -- 172.21.15.139:0/1412222605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565f2d0 msgr2=0x561af5672540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.804+0000 7fe894c09740 1 --2- 172.21.15.139:0/1412222605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565f2d0 0x561af5672540 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00bbc0 tx=0x7fe87c005fc0 comp rx=0 tx=0).stop 2024-09-16T01:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.804+0000 7fe894c09740 1 -- 172.21.15.139:0/1412222605 shutdown_connections 2024-09-16T01:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.804+0000 7fe894c09740 1 --2- 172.21.15.139:0/1412222605 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561af56e9630 0x561af56eba50 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.804+0000 7fe86affd640 0 client.16239 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.804+0000 7fe894c09740 1 --2- 172.21.15.139:0/1412222605 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561af566e0c0 0x561af5672a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.804+0000 7fe894c09740 1 --2- 172.21.15.139:0/1412222605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561af565f2d0 0x561af5672540 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.804+0000 7fe894c09740 1 --2- 172.21.15.139:0/1412222605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561af565e9b0 0x561af5672000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.804+0000 7fe894c09740 1 -- 172.21.15.139:0/1412222605 >> 172.21.15.139:0/1412222605 conn(0x561af565b060 msgr2=0x561af56599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.805+0000 7fe894c09740 1 -- 172.21.15.139:0/1412222605 shutdown_connections 2024-09-16T01:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.805+0000 7fe894c09740 1 -- 172.21.15.139:0/1412222605 wait complete. 2024-09-16T01:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.805+0000 7fe894c09740 20 client.16239 trim_cache size 0 max 16384 2024-09-16T01:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:46:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.806+0000 7fe891d91640 1 -- 172.21.15.139:0/1898530280 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe8780a3460 msgr2=0x7fe8780a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.806+0000 7fe891d91640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe8780a3460 0x7fe8780a5920 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe884099030 tx=0x7fe88400efd0 comp rx=0 tx=0).stop 2024-09-16T01:46:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.806+0000 7fe891d91640 1 -- 172.21.15.139:0/1898530280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88c1069c0 msgr2=0x7fe88c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:46:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.806+0000 7fe891d91640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88c1069c0 0x7fe88c19cc20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe88400eb10 tx=0x7fe88400c2d0 comp rx=0 tx=0).stop 2024-09-16T01:46:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.806+0000 7fe891d91640 1 -- 172.21.15.139:0/1898530280 shutdown_connections 2024-09-16T01:46:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.806+0000 7fe891d91640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe8780a3460 0x7fe8780a5920 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.806+0000 7fe891d91640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88c1069c0 0x7fe88c19cc20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.806+0000 7fe891d91640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe88c101350 0x7fe88c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.806+0000 7fe891d91640 1 --2- 172.21.15.139:0/1898530280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe88c1009b0 0x7fe88c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:46:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.806+0000 7fe891d91640 1 -- 172.21.15.139:0/1898530280 >> 172.21.15.139:0/1898530280 conn(0x7fe88c077990 msgr2=0x7fe88c069ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:46:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.807+0000 7fe891d91640 1 -- 172.21.15.139:0/1898530280 shutdown_connections 2024-09-16T01:46:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:46:56.807+0000 7fe891d91640 1 -- 172.21.15.139:0/1898530280 wait complete. 2024-09-16T01:46:57.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:57 smithi164 ceph-mon[35121]: pgmap v199: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 955 op/s 2024-09-16T01:46:57.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:57 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:46:57.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:57 smithi139 ceph-mon[31175]: pgmap v199: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 955 op/s 2024-09-16T01:46:57.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:57 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:46:57.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:57 smithi176 ceph-mon[35324]: pgmap v199: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 955 op/s 2024-09-16T01:46:57.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:57 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:46:59.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:46:59 smithi164 ceph-mon[35121]: pgmap v200: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-09-16T01:46:59.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:46:59 smithi139 ceph-mon[31175]: pgmap v200: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-09-16T01:46:59.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:46:59 smithi176 ceph-mon[35324]: pgmap v200: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-09-16T01:47:00.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:00 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:47:00.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:00 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:47:00.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:00 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:47:01.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:01 smithi164 ceph-mon[35121]: pgmap v201: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.54k op/s 2024-09-16T01:47:01.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:01 smithi139 ceph-mon[31175]: pgmap v201: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.54k op/s 2024-09-16T01:47:01.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:01 smithi176 ceph-mon[35324]: pgmap v201: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.54k op/s 2024-09-16T01:47:03.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:03 smithi164 ceph-mon[35121]: pgmap v202: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 1.76k op/s 2024-09-16T01:47:03.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:03 smithi139 ceph-mon[31175]: pgmap v202: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 1.76k op/s 2024-09-16T01:47:03.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:03 smithi176 ceph-mon[35324]: pgmap v202: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 1.76k op/s 2024-09-16T01:47:05.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:05 smithi164 ceph-mon[35121]: pgmap v203: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.8 MiB/s wr, 1.55k op/s 2024-09-16T01:47:05.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:05 smithi139 ceph-mon[31175]: pgmap v203: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.8 MiB/s wr, 1.55k op/s 2024-09-16T01:47:05.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:05 smithi176 ceph-mon[35324]: pgmap v203: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.8 MiB/s wr, 1.55k op/s 2024-09-16T01:47:07.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:07 smithi164 ceph-mon[35121]: pgmap v204: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 1.41k op/s 2024-09-16T01:47:07.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:07 smithi139 ceph-mon[31175]: pgmap v204: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 1.41k op/s 2024-09-16T01:47:07.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:07 smithi176 ceph-mon[35324]: pgmap v204: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 1.41k op/s 2024-09-16T01:47:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:09 smithi164 ceph-mon[35121]: pgmap v205: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.2 MiB/s wr, 961 op/s 2024-09-16T01:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.894+0000 7f9fc8db7640 1 -- 172.21.15.139:0/3466530041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fc41069c0 msgr2=0x7f9fc4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.894+0000 7f9fc8db7640 1 --2- 172.21.15.139:0/3466530041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fc41069c0 0x7f9fc4106da0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009930 tx=0x7f9fb402a420 comp rx=0 tx=0).stop 2024-09-16T01:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.894+0000 7f9fc8db7640 1 -- 172.21.15.139:0/3466530041 shutdown_connections 2024-09-16T01:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.894+0000 7f9fc8db7640 1 --2- 172.21.15.139:0/3466530041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc4101350 0x7f9fc410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.894+0000 7f9fc8db7640 1 --2- 172.21.15.139:0/3466530041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fc41009b0 0x7f9fc4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.894+0000 7f9fc8db7640 1 --2- 172.21.15.139:0/3466530041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fc41069c0 0x7f9fc4106da0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.894+0000 7f9fc8db7640 1 -- 172.21.15.139:0/3466530041 >> 172.21.15.139:0/3466530041 conn(0x7f9fc4077990 msgr2=0x7f9fc40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.894+0000 7f9fc8db7640 1 -- 172.21.15.139:0/3466530041 shutdown_connections 2024-09-16T01:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.894+0000 7f9fc8db7640 1 -- 172.21.15.139:0/3466530041 wait complete. 2024-09-16T01:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.894+0000 7f9fc8db7640 1 Processor -- start 2024-09-16T01:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc8db7640 1 -- start start 2024-09-16T01:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc8db7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc41009b0 0x7f9fc4197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc8db7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fc4101350 0x7f9fc4197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc8db7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fc41069c0 0x7f9fc419bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc8db7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc4110e00 con 0x7f9fc4101350 2024-09-16T01:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc8db7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc4110c80 con 0x7f9fc41009b0 2024-09-16T01:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc8db7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc4110f80 con 0x7f9fc41069c0 2024-09-16T01:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc1d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fc4101350 0x7f9fc4197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc1d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fc4101350 0x7f9fc4197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46814/0 (socket says 172.21.15.139:46814) 2024-09-16T01:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc1d9b640 1 -- 172.21.15.139:0/2088456889 learned_addr learned my addr 172.21.15.139:0/2088456889 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc2d9d640 1 --2- 172.21.15.139:0/2088456889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fc41069c0 0x7f9fc419bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc259c640 1 --2- 172.21.15.139:0/2088456889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc41009b0 0x7f9fc4197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc1d9b640 1 -- 172.21.15.139:0/2088456889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fc41069c0 msgr2=0x7f9fc419bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc1d9b640 1 --2- 172.21.15.139:0/2088456889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fc41069c0 0x7f9fc419bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc1d9b640 1 -- 172.21.15.139:0/2088456889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc41009b0 msgr2=0x7f9fc4197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc1d9b640 1 --2- 172.21.15.139:0/2088456889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc41009b0 0x7f9fc4197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc1d9b640 1 -- 172.21.15.139:0/2088456889 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc419c240 con 0x7f9fc4101350 2024-09-16T01:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fc1d9b640 1 --2- 172.21.15.139:0/2088456889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fc4101350 0x7f9fc4197790 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800d950 tx=0x7f9fb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.895+0000 7f9fb37fe640 1 -- 172.21.15.139:0/2088456889 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8017070 con 0x7f9fc4101350 2024-09-16T01:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.896+0000 7f9fb37fe640 1 -- 172.21.15.139:0/2088456889 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb8009c30 con 0x7f9fc4101350 2024-09-16T01:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.896+0000 7f9fb37fe640 1 -- 172.21.15.139:0/2088456889 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb800f6b0 con 0x7f9fc4101350 2024-09-16T01:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.896+0000 7f9fc8db7640 1 -- 172.21.15.139:0/2088456889 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc419c530 con 0x7f9fc4101350 2024-09-16T01:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.896+0000 7f9fc8db7640 1 -- 172.21.15.139:0/2088456889 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc41a3dc0 con 0x7f9fc4101350 2024-09-16T01:47:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.898+0000 7f9fb37fe640 1 -- 172.21.15.139:0/2088456889 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9fb80109d0 con 0x7f9fc4101350 2024-09-16T01:47:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.898+0000 7f9fcbc2c740 1 Processor -- start 2024-09-16T01:47:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.898+0000 7f9fcbc2c740 1 -- start start 2024-09-16T01:47:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.898+0000 7f9fcbc2c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35115af0 0x557b35115ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.898+0000 7f9fcbc2c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35116410 0x557b35120770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fcbc2c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 0x557b351230a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fcbc2c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557b3507a2f0 con 0x557b35116410 2024-09-16T01:47:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fcbc2c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557b35022270 con 0x557b35115af0 2024-09-16T01:47:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fcbc2c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557b350762b0 con 0x557b35120cb0 2024-09-16T01:47:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fb37fe640 1 --2- 172.21.15.139:0/2088456889 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9fac0a3460 0x7f9fac0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35116410 0x557b35120770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35116410 0x557b35120770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46826/0 (socket says 172.21.15.139:46826) 2024-09-16T01:47:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc259c640 1 -- 172.21.15.139:0/1923459673 learned_addr learned my addr 172.21.15.139:0/1923459673 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc2d9d640 1 --2- 172.21.15.139:0/2088456889 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9fac0a3460 0x7f9fac0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc1d9b640 1 --2- 172.21.15.139:0/1923459673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 0x557b351230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fb37fe640 1 -- 172.21.15.139:0/2088456889 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f9fb80cfdd0 con 0x7f9fc4101350 2024-09-16T01:47:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc2d9d640 1 --2- 172.21.15.139:0/1923459673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35115af0 0x557b35115ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc259c640 1 -- 172.21.15.139:0/1923459673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 msgr2=0x557b351230a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc259c640 1 --2- 172.21.15.139:0/1923459673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 0x557b351230a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc259c640 1 -- 172.21.15.139:0/1923459673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35115af0 msgr2=0x557b35115ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc259c640 1 --2- 172.21.15.139:0/1923459673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35115af0 0x557b35115ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc259c640 1 -- 172.21.15.139:0/1923459673 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b35021340 con 0x557b35116410 2024-09-16T01:47:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc2d9d640 1 --2- 172.21.15.139:0/1923459673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35115af0 0x557b35115ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:47:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc1d9b640 1 --2- 172.21.15.139:0/1923459673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 0x557b351230a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:47:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc2d9d640 1 --2- 172.21.15.139:0/2088456889 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9fac0a3460 0x7f9fac0a5920 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc005fd0 tx=0x7f9fbc009d50 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.899+0000 7f9fc259c640 1 --2- 172.21.15.139:0/1923459673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35116410 0x557b35120770 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9fb402a940 tx=0x7f9fb4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6833c44a1c6a4e85 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.900+0000 7f9fb0ff9640 1 -- 172.21.15.139:0/1923459673 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb403f070 con 0x557b35116410 2024-09-16T01:47:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.900+0000 7f9fb0ff9640 1 -- 172.21.15.139:0/1923459673 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb4004030 con 0x557b35116410 2024-09-16T01:47:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.900+0000 7f9fb0ff9640 1 -- 172.21.15.139:0/1923459673 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4035070 con 0x557b35116410 2024-09-16T01:47:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.900+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1923459673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35116410 msgr2=0x557b35120770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.900+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/1923459673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35116410 0x557b35120770 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9fb402a940 tx=0x7f9fb4005e80 comp rx=0 tx=0).stop 2024-09-16T01:47:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.900+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1923459673 shutdown_connections 2024-09-16T01:47:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.900+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/1923459673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 0x557b351230a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.900+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/1923459673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35116410 0x557b35120770 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.900+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/1923459673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35115af0 0x557b35115ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.900+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1923459673 >> 172.21.15.139:0/1923459673 conn(0x557b35112060 msgr2=0x557b35112440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1923459673 shutdown_connections 2024-09-16T01:47:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1923459673 wait complete. 2024-09-16T01:47:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fcbc2c740 1 -- start start 2024-09-16T01:47:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fcbc2c740 1 Processor -- start 2024-09-16T01:47:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fcbc2c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35115af0 0x557b3511db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fcbc2c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b3511e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fcbc2c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 0x557b3511e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fcbc2c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557b35123880 con 0x557b35115af0 2024-09-16T01:47:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fcbc2c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557b35003290 con 0x557b35116410 2024-09-16T01:47:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fcbc2c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557b35123a00 con 0x557b35120cb0 2024-09-16T01:47:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fcbc2c740 10 client.0 _ll_register_callbacks cb 0x557b35034140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:47:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fc259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35115af0 0x557b3511db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fc259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35115af0 0x557b3511db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46834/0 (socket says 172.21.15.139:46834) 2024-09-16T01:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fc259c640 1 -- 172.21.15.139:0/4145184615 learned_addr learned my addr 172.21.15.139:0/4145184615 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fc1d9b640 1 --2- 172.21.15.139:0/4145184615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b3511e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.901+0000 7f9fc2d9d640 1 --2- 172.21.15.139:0/4145184615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 0x557b3511e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fc259c640 1 -- 172.21.15.139:0/4145184615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 msgr2=0x557b3511e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fc259c640 1 --2- 172.21.15.139:0/4145184615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 0x557b3511e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fc259c640 1 -- 172.21.15.139:0/4145184615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 msgr2=0x557b3511e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fc259c640 1 --2- 172.21.15.139:0/4145184615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b3511e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fc259c640 1 -- 172.21.15.139:0/4145184615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b34d8d210 con 0x557b35115af0 2024-09-16T01:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fc1d9b640 1 --2- 172.21.15.139:0/4145184615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b3511e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fc259c640 1 --2- 172.21.15.139:0/4145184615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35115af0 0x557b3511db30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4032fe0 tx=0x7f9fb4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36683c5f11652b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fa9ffb640 1 -- 172.21.15.139:0/4145184615 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4035070 con 0x557b35115af0 2024-09-16T01:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fa9ffb640 1 -- 172.21.15.139:0/4145184615 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f9fb40315b0 con 0x557b35115af0 2024-09-16T01:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/4145184615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb4004030 con 0x557b35115af0 2024-09-16T01:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fa9ffb640 1 -- 172.21.15.139:0/4145184615 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9fb4055050 con 0x557b35115af0 2024-09-16T01:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fcbc2c740 10 client.16251 fetch_fsmap learned FSMap version 39 2024-09-16T01:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/4145184615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557b34e59620 con 0x557b35115af0 2024-09-16T01:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.902+0000 7f9fa9ffb640 1 -- 172.21.15.139:0/4145184615 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb4004030 con 0x557b35115af0 2024-09-16T01:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.903+0000 7f9fa9ffb640 1 -- 172.21.15.139:0/4145184615 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 39) ==== 8422+0+0 (secure 0 0 0) 0x7f9fb40046c0 con 0x557b35115af0 2024-09-16T01:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.903+0000 7f9fcbc2c740 10 client.16251 fetch_fsmap finished waiting for FSMap version 39 2024-09-16T01:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.903+0000 7f9fcbc2c740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.903+0000 7f9fcbc2c740 20 client.16251 populate_metadata read hostname 'smithi139' 2024-09-16T01:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.903+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/4145184615 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b35130a10 0x557b35132e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.903+0000 7f9fcbc2c740 4 client.16251 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.903+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/4145184615 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b35087a20 con 0x557b35130a10 2024-09-16T01:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.903+0000 7f9fc1d9b640 1 --2- 172.21.15.139:0/4145184615 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b35130a10 0x557b35132e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.903+0000 7f9fa9ffb640 10 client.16251 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.903+0000 7f9fc1d9b640 1 --2- 172.21.15.139:0/4145184615 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b35130a10 0x557b35132e00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1cccd22922f6ba server_cookie=4ec34eacb5b91fc in_seq=0 out_seq=0 2024-09-16T01:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fa9ffb640 1 -- 172.21.15.139:0/4145184615 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b35087a20 con 0x557b35130a10 2024-09-16T01:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fa9ffb640 10 client.16251 handle_command_reply: tid=0 2024-09-16T01:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fcbc2c740 1 client.16251 shutdown 2024-09-16T01:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/4145184615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35115af0 msgr2=0x557b3511db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/4145184615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35115af0 0x557b3511db30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4032fe0 tx=0x7f9fb4005bd0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/4145184615 shutdown_connections 2024-09-16T01:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/4145184615 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b35130a10 0x557b35132e00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/4145184615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 0x557b3511e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/4145184615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b3511e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fa9ffb640 0 client.16251 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/4145184615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35115af0 0x557b3511db30 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/4145184615 >> 172.21.15.139:0/4145184615 conn(0x557b35112060 msgr2=0x557b35117dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/4145184615 shutdown_connections 2024-09-16T01:47:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/4145184615 wait complete. 2024-09-16T01:47:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.904+0000 7f9fcbc2c740 20 client.16251 trim_cache size 0 max 16384 2024-09-16T01:47:09.922 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.918+0000 7f9fcbc2c740 1 Processor -- start 2024-09-16T01:47:09.922 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.918+0000 7f9fcbc2c740 1 -- start start 2024-09-16T01:47:09.922 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.918+0000 7f9fcbc2c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35115af0 0x557b3511f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.918+0000 7f9fcbc2c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b3511e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fcbc2c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 0x557b35138430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fcbc2c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc41acd60 con 0x557b35115af0 2024-09-16T01:47:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fcbc2c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc41aea50 con 0x557b35116410 2024-09-16T01:47:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fcbc2c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc41ad4f0 con 0x557b35120cb0 2024-09-16T01:47:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fc259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35115af0 0x557b3511f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fc259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35115af0 0x557b3511f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46846/0 (socket says 172.21.15.139:46846) 2024-09-16T01:47:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fc259c640 1 -- 172.21.15.139:0/1138439467 learned_addr learned my addr 172.21.15.139:0/1138439467 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fc1d9b640 1 --2- 172.21.15.139:0/1138439467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b3511e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fc2d9d640 1 --2- 172.21.15.139:0/1138439467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 0x557b35138430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fc259c640 1 -- 172.21.15.139:0/1138439467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 msgr2=0x557b35138430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fc259c640 1 --2- 172.21.15.139:0/1138439467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 0x557b35138430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fc259c640 1 -- 172.21.15.139:0/1138439467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 msgr2=0x557b3511e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fc259c640 1 --2- 172.21.15.139:0/1138439467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b3511e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fc259c640 1 -- 172.21.15.139:0/1138439467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b35111e90 con 0x557b35115af0 2024-09-16T01:47:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fc259c640 1 --2- 172.21.15.139:0/1138439467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35115af0 0x557b3511f390 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4004a90 tx=0x7f9fb4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5fefe0697bfcb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fabfff640 1 -- 172.21.15.139:0/1138439467 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4035020 con 0x557b35115af0 2024-09-16T01:47:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fabfff640 1 -- 172.21.15.139:0/1138439467 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb404c070 con 0x557b35115af0 2024-09-16T01:47:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.919+0000 7f9fabfff640 1 -- 172.21.15.139:0/1138439467 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb403f040 con 0x557b35115af0 2024-09-16T01:47:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.920+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1138439467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35115af0 msgr2=0x557b3511f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.920+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/1138439467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35115af0 0x557b3511f390 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4004a90 tx=0x7f9fb4005e80 comp rx=0 tx=0).stop 2024-09-16T01:47:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.920+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1138439467 shutdown_connections 2024-09-16T01:47:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.920+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/1138439467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35120cb0 0x557b35138430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.920+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/1138439467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b3511e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.920+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/1138439467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35115af0 0x557b3511f390 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.920+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1138439467 >> 172.21.15.139:0/1138439467 conn(0x557b35112060 msgr2=0x557b3511cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.920+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1138439467 shutdown_connections 2024-09-16T01:47:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.920+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1138439467 wait complete. 2024-09-16T01:47:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.920+0000 7f9fcbc2c740 1 -- start start 2024-09-16T01:47:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.920+0000 7f9fcbc2c740 1 Processor -- start 2024-09-16T01:47:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fcbc2c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35115af0 0x557b351292d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fcbc2c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b351245f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fcbc2c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35120cb0 0x557b35124b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fcbc2c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc41ae350 con 0x557b35120cb0 2024-09-16T01:47:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fcbc2c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc41adc80 con 0x557b35116410 2024-09-16T01:47:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fcbc2c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc41aad40 con 0x557b35115af0 2024-09-16T01:47:09.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:09 smithi139 ceph-mon[31175]: pgmap v205: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.2 MiB/s wr, 961 op/s 2024-09-16T01:47:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fcbc2c740 10 client.0 _ll_register_callbacks cb 0x7f9fbc014690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:47:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fc1d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b351245f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fc1d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b351245f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:43182/0 (socket says 172.21.15.139:43182) 2024-09-16T01:47:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fc1d9b640 1 -- 172.21.15.139:0/1564323661 learned_addr learned my addr 172.21.15.139:0/1564323661 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fc2d9d640 1 --2- 172.21.15.139:0/1564323661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35120cb0 0x557b35124b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fc259c640 1 --2- 172.21.15.139:0/1564323661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35115af0 0x557b351292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fc1d9b640 1 -- 172.21.15.139:0/1564323661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35115af0 msgr2=0x557b351292d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fc1d9b640 1 --2- 172.21.15.139:0/1564323661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35115af0 0x557b351292d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fc1d9b640 1 -- 172.21.15.139:0/1564323661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35120cb0 msgr2=0x557b35124b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fc1d9b640 1 --2- 172.21.15.139:0/1564323661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35120cb0 0x557b35124b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fc1d9b640 1 -- 172.21.15.139:0/1564323661 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b35125220 con 0x557b35116410 2024-09-16T01:47:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fc1d9b640 1 --2- 172.21.15.139:0/1564323661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b351245f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8098160 tx=0x7f9fb801e770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18d2509ccc0bff7f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:47:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.921+0000 7f9fa9ffb640 1 -- 172.21.15.139:0/1564323661 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb80b9060 con 0x557b35116410 2024-09-16T01:47:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.922+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1564323661 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb404c070 con 0x557b35116410 2024-09-16T01:47:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.922+0000 7f9fa9ffb640 1 -- 172.21.15.139:0/1564323661 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9466+0+0 (secure 0 0 0) 0x7f9fb8098370 con 0x557b35116410 2024-09-16T01:47:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.922+0000 7f9fa9ffb640 1 -- 172.21.15.139:0/1564323661 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9fb8020040 con 0x557b35116410 2024-09-16T01:47:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.922+0000 7f9fcbc2c740 10 client.25736 fetch_fsmap learned FSMap version 39 2024-09-16T01:47:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.922+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1564323661 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557b3519b2f0 con 0x557b35116410 2024-09-16T01:47:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.922+0000 7f9fa9ffb640 1 -- 172.21.15.139:0/1564323661 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb404c070 con 0x557b35116410 2024-09-16T01:47:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.922+0000 7f9fa9ffb640 1 -- 172.21.15.139:0/1564323661 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 39) ==== 8422+0+0 (secure 0 0 0) 0x7f9fb809b150 con 0x557b35116410 2024-09-16T01:47:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.922+0000 7f9fcbc2c740 10 client.25736 fetch_fsmap finished waiting for FSMap version 39 2024-09-16T01:47:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.922+0000 7f9fcbc2c740 10 client.25736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:47:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.922+0000 7f9fcbc2c740 20 client.25736 populate_metadata read hostname 'smithi139' 2024-09-16T01:47:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.922+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/1564323661 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b3519ba50 0x557b3519de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.922+0000 7f9fcbc2c740 4 client.25736 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.922+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1564323661 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b35081d70 con 0x557b3519ba50 2024-09-16T01:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.922+0000 7f9fc259c640 1 --2- 172.21.15.139:0/1564323661 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b3519ba50 0x557b3519de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.923+0000 7f9fc259c640 1 --2- 172.21.15.139:0/1564323661 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b3519ba50 0x557b3519de70 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15c4af81f0dea84 server_cookie=577bf739b959851a in_seq=0 out_seq=0 2024-09-16T01:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.923+0000 7f9fa9ffb640 10 client.25736 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:09.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:09 smithi176 ceph-mon[35324]: pgmap v205: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.2 MiB/s wr, 961 op/s 2024-09-16T01:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fa9ffb640 1 -- 172.21.15.139:0/1564323661 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b35081d70 con 0x557b3519ba50 2024-09-16T01:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fa9ffb640 10 client.25736 handle_command_reply: tid=0 2024-09-16T01:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fcbc2c740 1 client.25736 shutdown 2024-09-16T01:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1564323661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 msgr2=0x557b351245f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/1564323661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b351245f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8098160 tx=0x7f9fb801e770 comp rx=0 tx=0).stop 2024-09-16T01:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1564323661 shutdown_connections 2024-09-16T01:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/1564323661 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b3519ba50 0x557b3519de70 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/1564323661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b35120cb0 0x557b35124b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/1564323661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b35116410 0x557b351245f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fa9ffb640 0 client.25736 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fcbc2c740 1 --2- 172.21.15.139:0/1564323661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b35115af0 0x557b351292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1564323661 >> 172.21.15.139:0/1564323661 conn(0x557b35112060 msgr2=0x557b3512bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1564323661 shutdown_connections 2024-09-16T01:47:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fcbc2c740 1 -- 172.21.15.139:0/1564323661 wait complete. 2024-09-16T01:47:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.958+0000 7f9fcbc2c740 20 client.25736 trim_cache size 0 max 16384 2024-09-16T01:47:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:47:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:47:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:47:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:47:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.959+0000 7f9fc8db7640 1 -- 172.21.15.139:0/2088456889 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9fac0a3460 msgr2=0x7f9fac0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.959+0000 7f9fc8db7640 1 --2- 172.21.15.139:0/2088456889 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9fac0a3460 0x7f9fac0a5920 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc005fd0 tx=0x7f9fbc009d50 comp rx=0 tx=0).stop 2024-09-16T01:47:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.959+0000 7f9fc8db7640 1 -- 172.21.15.139:0/2088456889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fc4101350 msgr2=0x7f9fc4197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.959+0000 7f9fc8db7640 1 --2- 172.21.15.139:0/2088456889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fc4101350 0x7f9fc4197790 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800d950 tx=0x7f9fb800de10 comp rx=0 tx=0).stop 2024-09-16T01:47:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.959+0000 7f9fc8db7640 1 -- 172.21.15.139:0/2088456889 shutdown_connections 2024-09-16T01:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.959+0000 7f9fc8db7640 1 --2- 172.21.15.139:0/2088456889 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9fac0a3460 0x7f9fac0a5920 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.959+0000 7f9fc8db7640 1 --2- 172.21.15.139:0/2088456889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fc41069c0 0x7f9fc419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.959+0000 7f9fc8db7640 1 --2- 172.21.15.139:0/2088456889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fc4101350 0x7f9fc4197790 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.959+0000 7f9fc8db7640 1 --2- 172.21.15.139:0/2088456889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc41009b0 0x7f9fc4197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.959+0000 7f9fc8db7640 1 -- 172.21.15.139:0/2088456889 >> 172.21.15.139:0/2088456889 conn(0x7f9fc4077990 msgr2=0x7f9fc40ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.959+0000 7f9fc8db7640 1 -- 172.21.15.139:0/2088456889 shutdown_connections 2024-09-16T01:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:09.959+0000 7f9fc8db7640 1 -- 172.21.15.139:0/2088456889 wait complete. 2024-09-16T01:47:11.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:11 smithi164 ceph-mon[35121]: pgmap v206: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 423 KiB/s wr, 526 op/s 2024-09-16T01:47:11.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:11 smithi139 ceph-mon[31175]: pgmap v206: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 423 KiB/s wr, 526 op/s 2024-09-16T01:47:11.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:11 smithi176 ceph-mon[35324]: pgmap v206: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 423 KiB/s wr, 526 op/s 2024-09-16T01:47:13.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:13 smithi164 ceph-mon[35121]: pgmap v207: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 225 KiB/s wr, 302 op/s 2024-09-16T01:47:13.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:13 smithi139 ceph-mon[31175]: pgmap v207: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 225 KiB/s wr, 302 op/s 2024-09-16T01:47:13.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:13 smithi176 ceph-mon[35324]: pgmap v207: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 225 KiB/s wr, 302 op/s 2024-09-16T01:47:15.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:15 smithi164 ceph-mon[35121]: pgmap v208: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 KiB/s wr, 36 op/s 2024-09-16T01:47:15.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:15 smithi139 ceph-mon[31175]: pgmap v208: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 KiB/s wr, 36 op/s 2024-09-16T01:47:15.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:15 smithi176 ceph-mon[35324]: pgmap v208: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 KiB/s wr, 36 op/s 2024-09-16T01:47:17.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:17 smithi164 ceph-mon[35121]: pgmap v209: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 77 KiB/s wr, 27 op/s 2024-09-16T01:47:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:17 smithi139 ceph-mon[31175]: pgmap v209: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 77 KiB/s wr, 27 op/s 2024-09-16T01:47:17.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:17 smithi176 ceph-mon[35324]: pgmap v209: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 77 KiB/s wr, 27 op/s 2024-09-16T01:47:19.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:19 smithi164 ceph-mon[35121]: pgmap v210: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 62 KiB/s wr, 24 op/s 2024-09-16T01:47:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:19 smithi139 ceph-mon[31175]: pgmap v210: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 62 KiB/s wr, 24 op/s 2024-09-16T01:47:19.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:19 smithi176 ceph-mon[35324]: pgmap v210: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 62 KiB/s wr, 24 op/s 2024-09-16T01:47:21.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:21 smithi164 ceph-mon[35121]: pgmap v211: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 104 KiB/s wr, 28 op/s 2024-09-16T01:47:21.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:21 smithi139 ceph-mon[31175]: pgmap v211: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 104 KiB/s wr, 28 op/s 2024-09-16T01:47:21.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:21 smithi176 ceph-mon[35324]: pgmap v211: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 104 KiB/s wr, 28 op/s 2024-09-16T01:47:22.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:22 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:47:22.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:22 smithi139 ceph-mon[31175]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T01:47:22.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:22 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:47:22.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:22 smithi176 ceph-mon[35324]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T01:47:23.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:22 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:47:23.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:22 smithi164 ceph-mon[35121]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T01:47:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.111+0000 7fdf208d2640 1 -- 172.21.15.139:0/312344461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c11b140 msgr2=0x7fdf1c11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.111+0000 7fdf208d2640 1 --2- 172.21.15.139:0/312344461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c11b140 0x7fdf1c11d530 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdf1401acb0 tx=0x7fdf1403b920 comp rx=0 tx=0).stop 2024-09-16T01:47:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.112+0000 7fdf208d2640 1 -- 172.21.15.139:0/312344461 shutdown_connections 2024-09-16T01:47:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.112+0000 7fdf208d2640 1 --2- 172.21.15.139:0/312344461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c11b140 0x7fdf1c11d530 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.112+0000 7fdf208d2640 1 --2- 172.21.15.139:0/312344461 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf1c1095d0 0x7fdf1c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.112+0000 7fdf208d2640 1 --2- 172.21.15.139:0/312344461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c1091f0 0x7fdf1c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.112+0000 7fdf208d2640 1 -- 172.21.15.139:0/312344461 >> 172.21.15.139:0/312344461 conn(0x7fdf1c06ce00 msgr2=0x7fdf1c06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.116+0000 7fdf208d2640 1 -- 172.21.15.139:0/312344461 shutdown_connections 2024-09-16T01:47:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.116+0000 7fdf208d2640 1 -- 172.21.15.139:0/312344461 wait complete. 2024-09-16T01:47:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.117+0000 7fdf208d2640 1 Processor -- start 2024-09-16T01:47:23.122 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.119+0000 7fdf208d2640 1 -- start start 2024-09-16T01:47:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.119+0000 7fdf208d2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c1091f0 0x7fdf1c11a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.119+0000 7fdf208d2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf1c1095d0 0x7fdf1c116800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.119+0000 7fdf208d2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c11b140 0x7fdf1c116d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.119+0000 7fdf208d2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c113310 con 0x7fdf1c1091f0 2024-09-16T01:47:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.119+0000 7fdf208d2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c113190 con 0x7fdf1c11b140 2024-09-16T01:47:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.119+0000 7fdf208d2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c113490 con 0x7fdf1c1095d0 2024-09-16T01:47:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.120+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c1091f0 0x7fdf1c11a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.120+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c1091f0 0x7fdf1c11a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38306/0 (socket says 172.21.15.139:38306) 2024-09-16T01:47:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.120+0000 7fdf1a59c640 1 -- 172.21.15.139:0/1242380177 learned_addr learned my addr 172.21.15.139:0/1242380177 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.120+0000 7fdf1a59c640 1 -- 172.21.15.139:0/1242380177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf1c1095d0 msgr2=0x7fdf1c116800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:47:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.120+0000 7fdf1a59c640 1 --2- 172.21.15.139:0/1242380177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf1c1095d0 0x7fdf1c116800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.120+0000 7fdf1a59c640 1 -- 172.21.15.139:0/1242380177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c11b140 msgr2=0x7fdf1c116d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:47:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.120+0000 7fdf1a59c640 1 --2- 172.21.15.139:0/1242380177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c11b140 0x7fdf1c116d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.120+0000 7fdf1a59c640 1 -- 172.21.15.139:0/1242380177 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c117300 con 0x7fdf1c1091f0 2024-09-16T01:47:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.121+0000 7fdf1a59c640 1 --2- 172.21.15.139:0/1242380177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c1091f0 0x7fdf1c11a770 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000b480 tx=0x7fdf1000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.123+0000 7fdf0b7fe640 1 -- 172.21.15.139:0/1242380177 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10014070 con 0x7fdf1c1091f0 2024-09-16T01:47:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.123+0000 7fdf208d2640 1 -- 172.21.15.139:0/1242380177 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c1175f0 con 0x7fdf1c1091f0 2024-09-16T01:47:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.123+0000 7fdf208d2640 1 -- 172.21.15.139:0/1242380177 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c1ab440 con 0x7fdf1c1091f0 2024-09-16T01:47:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.123+0000 7fdf0b7fe640 1 -- 172.21.15.139:0/1242380177 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf10010d50 con 0x7fdf1c1091f0 2024-09-16T01:47:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.123+0000 7fdf0b7fe640 1 -- 172.21.15.139:0/1242380177 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1000d360 con 0x7fdf1c1091f0 2024-09-16T01:47:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.128+0000 7fdf23663740 1 Processor -- start 2024-09-16T01:47:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.131+0000 7fdf23663740 1 -- start start 2024-09-16T01:47:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.131+0000 7fdf23663740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb01af0 0x55936eb01ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.131+0000 7fdf23663740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb02410 0x55936eb0c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.131+0000 7fdf23663740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb0ccb0 0x55936eb0f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.131+0000 7fdf23663740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55936ea662f0 con 0x55936eb01af0 2024-09-16T01:47:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.131+0000 7fdf23663740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55936ea0e270 con 0x55936eb02410 2024-09-16T01:47:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.131+0000 7fdf23663740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55936ea622b0 con 0x55936eb0ccb0 2024-09-16T01:47:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.132+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb01af0 0x55936eb01ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.132+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb01af0 0x55936eb01ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38320/0 (socket says 172.21.15.139:38320) 2024-09-16T01:47:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.132+0000 7fdf1ad9d640 1 -- 172.21.15.139:0/292718871 learned_addr learned my addr 172.21.15.139:0/292718871 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.133+0000 7fdf19d9b640 1 --2- 172.21.15.139:0/292718871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb0ccb0 0x55936eb0f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.137+0000 7fdf0b7fe640 1 -- 172.21.15.139:0/1242380177 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fdf10017070 con 0x7fdf1c1091f0 2024-09-16T01:47:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.139+0000 7fdf0b7fe640 1 --2- 172.21.15.139:0/1242380177 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fdf040a36d0 0x7fdf040a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.139+0000 7fdf0b7fe640 1 -- 172.21.15.139:0/1242380177 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9642+0+0 (secure 0 0 0) 0x7fdf100cf720 con 0x7fdf1c1091f0 2024-09-16T01:47:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.139+0000 7fdf1a59c640 1 --2- 172.21.15.139:0/292718871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb02410 0x55936eb0c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.140+0000 7fdf1ad9d640 1 --2- 172.21.15.139:0/1242380177 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fdf040a36d0 0x7fdf040a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.140+0000 7fdf1a59c640 1 -- 172.21.15.139:0/292718871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb0ccb0 msgr2=0x55936eb0f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.140+0000 7fdf1a59c640 1 --2- 172.21.15.139:0/292718871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb0ccb0 0x55936eb0f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.140+0000 7fdf1a59c640 1 -- 172.21.15.139:0/292718871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb01af0 msgr2=0x55936eb01ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.140+0000 7fdf1a59c640 1 --2- 172.21.15.139:0/292718871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb01af0 0x55936eb01ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.140+0000 7fdf1a59c640 1 -- 172.21.15.139:0/292718871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55936ea0d340 con 0x55936eb02410 2024-09-16T01:47:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.141+0000 7fdf1a59c640 1 --2- 172.21.15.139:0/292718871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb02410 0x55936eb0c770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdf100ccac0 tx=0x7fdf1009b2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50547daff4bebddf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.141+0000 7fdf1ad9d640 1 --2- 172.21.15.139:0/1242380177 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fdf040a36d0 0x7fdf040a5b90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdf140408b0 tx=0x7fdf1401a5d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.143+0000 7fdf08ff9640 1 -- 172.21.15.139:0/292718871 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1009fa90 con 0x55936eb02410 2024-09-16T01:47:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.143+0000 7fdf08ff9640 1 -- 172.21.15.139:0/292718871 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf100a42c0 con 0x55936eb02410 2024-09-16T01:47:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.143+0000 7fdf08ff9640 1 -- 172.21.15.139:0/292718871 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1000f3c0 con 0x55936eb02410 2024-09-16T01:47:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.145+0000 7fdf23663740 1 -- 172.21.15.139:0/292718871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb02410 msgr2=0x55936eb0c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.145+0000 7fdf23663740 1 --2- 172.21.15.139:0/292718871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb02410 0x55936eb0c770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdf100ccac0 tx=0x7fdf1009b2e0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.146+0000 7fdf23663740 1 -- 172.21.15.139:0/292718871 shutdown_connections 2024-09-16T01:47:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.146+0000 7fdf23663740 1 --2- 172.21.15.139:0/292718871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb0ccb0 0x55936eb0f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.146+0000 7fdf23663740 1 --2- 172.21.15.139:0/292718871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb02410 0x55936eb0c770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.146+0000 7fdf23663740 1 --2- 172.21.15.139:0/292718871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb01af0 0x55936eb01ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.146+0000 7fdf23663740 1 -- 172.21.15.139:0/292718871 >> 172.21.15.139:0/292718871 conn(0x55936eafe060 msgr2=0x55936eafe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.150+0000 7fdf23663740 1 -- 172.21.15.139:0/292718871 shutdown_connections 2024-09-16T01:47:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.151+0000 7fdf23663740 1 -- 172.21.15.139:0/292718871 wait complete. 2024-09-16T01:47:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.151+0000 7fdf23663740 1 -- start start 2024-09-16T01:47:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.151+0000 7fdf23663740 1 Processor -- start 2024-09-16T01:47:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.154+0000 7fdf23663740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb01af0 0x55936eb09c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.154+0000 7fdf23663740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 0x55936eb0a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.154+0000 7fdf23663740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb0ccb0 0x55936eb0a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.154+0000 7fdf23663740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55936eb0f880 con 0x55936eb02410 2024-09-16T01:47:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.154+0000 7fdf23663740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55936e9ef290 con 0x55936eb01af0 2024-09-16T01:47:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.154+0000 7fdf23663740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55936eb0fa00 con 0x55936eb0ccb0 2024-09-16T01:47:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.154+0000 7fdf23663740 10 client.0 _ll_register_callbacks cb 0x55936ea20140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:47:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.155+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb0ccb0 0x55936eb0a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.155+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb0ccb0 0x55936eb0a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:38876/0 (socket says 172.21.15.139:38876) 2024-09-16T01:47:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.155+0000 7fdf1ad9d640 1 -- 172.21.15.139:0/73172230 learned_addr learned my addr 172.21.15.139:0/73172230 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.155+0000 7fdf19d9b640 1 --2- 172.21.15.139:0/73172230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 0x55936eb0a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.155+0000 7fdf1ad9d640 1 -- 172.21.15.139:0/73172230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb01af0 msgr2=0x55936eb09c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.155+0000 7fdf1ad9d640 1 --2- 172.21.15.139:0/73172230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb01af0 0x55936eb09c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.155+0000 7fdf1ad9d640 1 -- 172.21.15.139:0/73172230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 msgr2=0x55936eb0a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.155+0000 7fdf1ad9d640 1 --2- 172.21.15.139:0/73172230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 0x55936eb0a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.155+0000 7fdf1ad9d640 1 -- 172.21.15.139:0/73172230 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55936e779210 con 0x55936eb0ccb0 2024-09-16T01:47:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.156+0000 7fdf1ad9d640 1 --2- 172.21.15.139:0/73172230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb0ccb0 0x55936eb0a6e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdf1401ada0 tx=0x7fdf1403b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c91104a5d462a2c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.157+0000 7fdf01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:47:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.157+0000 7fdf01ffb640 1 -- 172.21.15.139:0/73172230 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14060020 con 0x55936eb0ccb0 2024-09-16T01:47:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.157+0000 7fdf23663740 1 -- 172.21.15.139:0/73172230 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf100a42c0 con 0x55936eb0ccb0 2024-09-16T01:47:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.157+0000 7fdf01ffb640 1 -- 172.21.15.139:0/73172230 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9642+0+0 (secure 0 0 0) 0x7fdf14019040 con 0x55936eb0ccb0 2024-09-16T01:47:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.159+0000 7fdf01ffb640 1 -- 172.21.15.139:0/73172230 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdf1406b050 con 0x55936eb0ccb0 2024-09-16T01:47:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.164+0000 7fdf23663740 10 client.25726 fetch_fsmap learned FSMap version 40 2024-09-16T01:47:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.164+0000 7fdf23663740 1 -- 172.21.15.139:0/73172230 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55936e845620 con 0x55936eb0ccb0 2024-09-16T01:47:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.165+0000 7fdf01ffb640 1 -- 172.21.15.139:0/73172230 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf100a42c0 con 0x55936eb0ccb0 2024-09-16T01:47:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.165+0000 7fdf01ffb640 1 -- 172.21.15.139:0/73172230 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 40) ==== 8430+0+0 (secure 0 0 0) 0x7fdf1405c460 con 0x55936eb0ccb0 2024-09-16T01:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.166+0000 7fdf23663740 10 client.25726 fetch_fsmap finished waiting for FSMap version 40 2024-09-16T01:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.166+0000 7fdf23663740 10 client.25726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.166+0000 7fdf23663740 20 client.25726 populate_metadata read hostname 'smithi139' 2024-09-16T01:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.166+0000 7fdf23663740 1 --2- 172.21.15.139:0/73172230 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55936eb1ce00 0x55936eb1f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.166+0000 7fdf23663740 4 client.25726 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.166+0000 7fdf23663740 1 -- 172.21.15.139:0/73172230 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55936ea73a20 con 0x55936eb1ce00 2024-09-16T01:47:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.168+0000 7fdf1a59c640 1 --2- 172.21.15.139:0/73172230 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55936eb1ce00 0x55936eb1f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.181+0000 7fdf1a59c640 1 --2- 172.21.15.139:0/73172230 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55936eb1ce00 0x55936eb1f290 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3302c76c2adaa0d7 server_cookie=f02f64a0ae350066 in_seq=0 out_seq=0 2024-09-16T01:47:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.181+0000 7fdf01ffb640 10 client.25726 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.183+0000 7fdf01ffb640 1 -- 172.21.15.139:0/73172230 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55936ea73a20 con 0x55936eb1ce00 2024-09-16T01:47:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.183+0000 7fdf01ffb640 10 client.25726 handle_command_reply: tid=0 2024-09-16T01:47:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.186+0000 7fdf23663740 1 client.25726 shutdown 2024-09-16T01:47:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.187+0000 7fdf23663740 1 -- 172.21.15.139:0/73172230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb0ccb0 msgr2=0x55936eb0a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.187+0000 7fdf23663740 1 --2- 172.21.15.139:0/73172230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb0ccb0 0x55936eb0a6e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdf1401ada0 tx=0x7fdf1403b920 comp rx=0 tx=0).stop 2024-09-16T01:47:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.189+0000 7fdf23663740 1 -- 172.21.15.139:0/73172230 shutdown_connections 2024-09-16T01:47:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.189+0000 7fdf23663740 1 --2- 172.21.15.139:0/73172230 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55936eb1ce00 0x55936eb1f290 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.189+0000 7fdf23663740 1 --2- 172.21.15.139:0/73172230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb0ccb0 0x55936eb0a6e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.189+0000 7fdf23663740 1 --2- 172.21.15.139:0/73172230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 0x55936eb0a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.189+0000 7fdf23663740 1 --2- 172.21.15.139:0/73172230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb01af0 0x55936eb09c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.189+0000 7fdf23663740 1 -- 172.21.15.139:0/73172230 >> 172.21.15.139:0/73172230 conn(0x55936eafe060 msgr2=0x55936eb03dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.190+0000 7fdf01ffb640 0 client.25726 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.193+0000 7fdf23663740 1 -- 172.21.15.139:0/73172230 shutdown_connections 2024-09-16T01:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.193+0000 7fdf23663740 1 -- 172.21.15.139:0/73172230 wait complete. 2024-09-16T01:47:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.194+0000 7fdf23663740 20 client.25726 trim_cache size 0 max 16384 2024-09-16T01:47:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.224+0000 7fdf23663740 1 Processor -- start 2024-09-16T01:47:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.224+0000 7fdf23663740 1 -- start start 2024-09-16T01:47:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.224+0000 7fdf23663740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eafe060 0x55936eb0a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.224+0000 7fdf23663740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb01af0 0x55936eb0bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.224+0000 7fdf23663740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 0x55936eb24ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.224+0000 7fdf23663740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc0102d0 con 0x55936eb02410 2024-09-16T01:47:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.224+0000 7fdf23663740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc012020 con 0x55936eb01af0 2024-09-16T01:47:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.224+0000 7fdf23663740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc010a60 con 0x55936eafe060 2024-09-16T01:47:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.225+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eafe060 0x55936eb0a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.225+0000 7fdf19d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb01af0 0x55936eb0bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.226+0000 7fdf19d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb01af0 0x55936eb0bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:60500/0 (socket says 172.21.15.139:60500) 2024-09-16T01:47:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.226+0000 7fdf19d9b640 1 -- 172.21.15.139:0/3965123616 learned_addr learned my addr 172.21.15.139:0/3965123616 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.226+0000 7fdf1ad9d640 1 --2- 172.21.15.139:0/3965123616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 0x55936eb24ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.227+0000 7fdf19d9b640 1 -- 172.21.15.139:0/3965123616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eafe060 msgr2=0x55936eb0a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.227+0000 7fdf19d9b640 1 --2- 172.21.15.139:0/3965123616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eafe060 0x55936eb0a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.227+0000 7fdf19d9b640 1 -- 172.21.15.139:0/3965123616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 msgr2=0x55936eb24ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.227+0000 7fdf19d9b640 1 --2- 172.21.15.139:0/3965123616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 0x55936eb24ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.228+0000 7fdf19d9b640 1 -- 172.21.15.139:0/3965123616 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55936eb1b8e0 con 0x55936eb01af0 2024-09-16T01:47:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.228+0000 7fdf19d9b640 1 --2- 172.21.15.139:0/3965123616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb01af0 0x55936eb0bbd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c118050 tx=0x7fdf0c00bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ac86d176dfebcf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.228+0000 7fdf1ad9d640 1 --2- 172.21.15.139:0/3965123616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 0x55936eb24ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:47:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.230+0000 7fdf03fff640 1 -- 172.21.15.139:0/3965123616 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c011ce0 con 0x55936eb01af0 2024-09-16T01:47:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.230+0000 7fdf03fff640 1 -- 172.21.15.139:0/3965123616 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c011e80 con 0x55936eb01af0 2024-09-16T01:47:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.230+0000 7fdf03fff640 1 -- 172.21.15.139:0/3965123616 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c01a930 con 0x55936eb01af0 2024-09-16T01:47:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.231+0000 7fdf23663740 1 -- 172.21.15.139:0/3965123616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb01af0 msgr2=0x55936eb0bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.231+0000 7fdf23663740 1 --2- 172.21.15.139:0/3965123616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb01af0 0x55936eb0bbd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c118050 tx=0x7fdf0c00bec0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.234+0000 7fdf23663740 1 -- 172.21.15.139:0/3965123616 shutdown_connections 2024-09-16T01:47:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.234+0000 7fdf23663740 1 --2- 172.21.15.139:0/3965123616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 0x55936eb24ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.234+0000 7fdf23663740 1 --2- 172.21.15.139:0/3965123616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eb01af0 0x55936eb0bbd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.234+0000 7fdf23663740 1 --2- 172.21.15.139:0/3965123616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eafe060 0x55936eb0a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.234+0000 7fdf23663740 1 -- 172.21.15.139:0/3965123616 >> 172.21.15.139:0/3965123616 conn(0x55936eb05610 msgr2=0x55936eb0b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.236+0000 7fdf23663740 1 -- 172.21.15.139:0/3965123616 shutdown_connections 2024-09-16T01:47:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.239+0000 7fdf23663740 1 -- 172.21.15.139:0/3965123616 wait complete. 2024-09-16T01:47:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.239+0000 7fdf23663740 1 -- start start 2024-09-16T01:47:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.239+0000 7fdf23663740 1 Processor -- start 2024-09-16T01:47:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.240+0000 7fdf23663740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eafe060 0x55936eb15380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.240+0000 7fdf23663740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb01af0 0x55936eb10740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.240+0000 7fdf23663740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 0x55936eb10c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.240+0000 7fdf23663740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc0118c0 con 0x55936eb02410 2024-09-16T01:47:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.240+0000 7fdf23663740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc0111f0 con 0x55936eafe060 2024-09-16T01:47:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.240+0000 7fdf23663740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc00e2d0 con 0x55936eb01af0 2024-09-16T01:47:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.240+0000 7fdf23663740 10 client.0 _ll_register_callbacks cb 0x7fdf1401a370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:47:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.241+0000 7fdf19d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb01af0 0x55936eb10740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.241+0000 7fdf19d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb01af0 0x55936eb10740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:38904/0 (socket says 172.21.15.139:38904) 2024-09-16T01:47:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.241+0000 7fdf19d9b640 1 -- 172.21.15.139:0/1883736742 learned_addr learned my addr 172.21.15.139:0/1883736742 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.242+0000 7fdf1a59c640 1 --2- 172.21.15.139:0/1883736742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eafe060 0x55936eb15380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.243+0000 7fdf1ad9d640 1 --2- 172.21.15.139:0/1883736742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 0x55936eb10c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.244+0000 7fdf19d9b640 1 -- 172.21.15.139:0/1883736742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eafe060 msgr2=0x55936eb15380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.244+0000 7fdf19d9b640 1 --2- 172.21.15.139:0/1883736742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eafe060 0x55936eb15380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.244+0000 7fdf19d9b640 1 -- 172.21.15.139:0/1883736742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 msgr2=0x55936eb10c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.244+0000 7fdf19d9b640 1 --2- 172.21.15.139:0/1883736742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 0x55936eb10c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.244+0000 7fdf19d9b640 1 -- 172.21.15.139:0/1883736742 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55936eb0aa50 con 0x55936eb01af0 2024-09-16T01:47:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.246+0000 7fdf19d9b640 1 --2- 172.21.15.139:0/1883736742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb01af0 0x55936eb10740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c01a410 tx=0x7fdf0c011ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=812885cf92322c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.246+0000 7fdf01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:47:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.247+0000 7fdf01ffb640 1 -- 172.21.15.139:0/1883736742 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c029060 con 0x55936eb01af0 2024-09-16T01:47:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.247+0000 7fdf01ffb640 1 -- 172.21.15.139:0/1883736742 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9642+0+0 (secure 0 0 0) 0x7fdf0c004240 con 0x55936eb01af0 2024-09-16T01:47:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.248+0000 7fdf23663740 1 -- 172.21.15.139:0/1883736742 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55936ea6dd70 con 0x55936eb01af0 2024-09-16T01:47:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.251+0000 7fdf01ffb640 1 -- 172.21.15.139:0/1883736742 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdf0c004440 con 0x55936eb01af0 2024-09-16T01:47:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.253+0000 7fdf23663740 10 client.25732 fetch_fsmap learned FSMap version 40 2024-09-16T01:47:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.253+0000 7fdf23663740 1 -- 172.21.15.139:0/1883736742 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55936eb87880 con 0x55936eb01af0 2024-09-16T01:47:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.254+0000 7fdf01ffb640 1 -- 172.21.15.139:0/1883736742 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55936ea6dd70 con 0x55936eb01af0 2024-09-16T01:47:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.254+0000 7fdf01ffb640 1 -- 172.21.15.139:0/1883736742 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 40) ==== 8430+0+0 (secure 0 0 0) 0x7fdf0c012450 con 0x55936eb01af0 2024-09-16T01:47:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.254+0000 7fdf23663740 10 client.25732 fetch_fsmap finished waiting for FSMap version 40 2024-09-16T01:47:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.254+0000 7fdf23663740 10 client.25732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:47:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.254+0000 7fdf23663740 20 client.25732 populate_metadata read hostname 'smithi139' 2024-09-16T01:47:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.254+0000 7fdf23663740 1 --2- 172.21.15.139:0/1883736742 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55936eb0ccb0 0x55936eb89fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.254+0000 7fdf23663740 4 client.25732 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:47:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.254+0000 7fdf23663740 1 -- 172.21.15.139:0/1883736742 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55936e9ea900 con 0x55936eb0ccb0 2024-09-16T01:47:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.258+0000 7fdf1a59c640 1 --2- 172.21.15.139:0/1883736742 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55936eb0ccb0 0x55936eb89fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.258+0000 7fdf1a59c640 1 --2- 172.21.15.139:0/1883736742 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55936eb0ccb0 0x55936eb89fa0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb7f614a61c4c58 server_cookie=74c03e3e1f6d77ec in_seq=0 out_seq=0 2024-09-16T01:47:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.258+0000 7fdf01ffb640 10 client.25732 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.323+0000 7fdf01ffb640 1 -- 172.21.15.139:0/1883736742 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55936e9ea900 con 0x55936eb0ccb0 2024-09-16T01:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.323+0000 7fdf01ffb640 10 client.25732 handle_command_reply: tid=0 2024-09-16T01:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.323+0000 7fdf23663740 1 client.25732 shutdown 2024-09-16T01:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.324+0000 7fdf23663740 1 -- 172.21.15.139:0/1883736742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb01af0 msgr2=0x55936eb10740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.324+0000 7fdf23663740 1 --2- 172.21.15.139:0/1883736742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb01af0 0x55936eb10740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c01a410 tx=0x7fdf0c011ae0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.324+0000 7fdf23663740 1 -- 172.21.15.139:0/1883736742 shutdown_connections 2024-09-16T01:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.324+0000 7fdf23663740 1 --2- 172.21.15.139:0/1883736742 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55936eb0ccb0 0x55936eb89fa0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.324+0000 7fdf23663740 1 --2- 172.21.15.139:0/1883736742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55936eb02410 0x55936eb10c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.324+0000 7fdf23663740 1 --2- 172.21.15.139:0/1883736742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55936eb01af0 0x55936eb10740 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.324+0000 7fdf23663740 1 --2- 172.21.15.139:0/1883736742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55936eafe060 0x55936eb15380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.324+0000 7fdf23663740 1 -- 172.21.15.139:0/1883736742 >> 172.21.15.139:0/1883736742 conn(0x55936eb05610 msgr2=0x55936eb03d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.324+0000 7fdf01ffb640 0 client.25732 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.324+0000 7fdf23663740 1 -- 172.21.15.139:0/1883736742 shutdown_connections 2024-09-16T01:47:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.324+0000 7fdf23663740 1 -- 172.21.15.139:0/1883736742 wait complete. 2024-09-16T01:47:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.324+0000 7fdf23663740 20 client.25732 trim_cache size 0 max 16384 2024-09-16T01:47:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:47:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:47:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:47:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:47:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.325+0000 7fdf208d2640 1 -- 172.21.15.139:0/1242380177 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fdf040a36d0 msgr2=0x7fdf040a5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.325+0000 7fdf208d2640 1 --2- 172.21.15.139:0/1242380177 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fdf040a36d0 0x7fdf040a5b90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdf140408b0 tx=0x7fdf1401a5d0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.325+0000 7fdf208d2640 1 -- 172.21.15.139:0/1242380177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c1091f0 msgr2=0x7fdf1c11a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.325+0000 7fdf208d2640 1 --2- 172.21.15.139:0/1242380177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c1091f0 0x7fdf1c11a770 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000b480 tx=0x7fdf1000b940 comp rx=0 tx=0).stop 2024-09-16T01:47:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.325+0000 7fdf208d2640 1 -- 172.21.15.139:0/1242380177 shutdown_connections 2024-09-16T01:47:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.325+0000 7fdf208d2640 1 --2- 172.21.15.139:0/1242380177 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fdf040a36d0 0x7fdf040a5b90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.325+0000 7fdf208d2640 1 --2- 172.21.15.139:0/1242380177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c11b140 0x7fdf1c116d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.326+0000 7fdf208d2640 1 --2- 172.21.15.139:0/1242380177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf1c1095d0 0x7fdf1c116800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.326+0000 7fdf208d2640 1 --2- 172.21.15.139:0/1242380177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c1091f0 0x7fdf1c11a770 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.326+0000 7fdf208d2640 1 -- 172.21.15.139:0/1242380177 >> 172.21.15.139:0/1242380177 conn(0x7fdf1c06ce00 msgr2=0x7fdf1c0708b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.326+0000 7fdf208d2640 1 -- 172.21.15.139:0/1242380177 shutdown_connections 2024-09-16T01:47:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:23.326+0000 7fdf208d2640 1 -- 172.21.15.139:0/1242380177 wait complete. 2024-09-16T01:47:23.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:23 smithi139 ceph-mon[31175]: pgmap v213: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 140 KiB/s wr, 26 op/s 2024-09-16T01:47:23.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:23 smithi176 ceph-mon[35324]: pgmap v213: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 140 KiB/s wr, 26 op/s 2024-09-16T01:47:24.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:23 smithi164 ceph-mon[35121]: pgmap v213: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 140 KiB/s wr, 26 op/s 2024-09-16T01:47:25.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:25 smithi139 ceph-mon[31175]: pgmap v214: 129 pgs: 8 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 97 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 319 KiB/s wr, 30 op/s 2024-09-16T01:47:25.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:25 smithi176 ceph-mon[35324]: pgmap v214: 129 pgs: 8 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 97 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 319 KiB/s wr, 30 op/s 2024-09-16T01:47:26.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:25 smithi164 ceph-mon[35121]: pgmap v214: 129 pgs: 8 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 97 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 319 KiB/s wr, 30 op/s 2024-09-16T01:47:26.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:26 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:47:26.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:26 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:47:27.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:26 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:47:27.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:27 smithi139 ceph-mon[31175]: pgmap v215: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 312 KiB/s wr, 29 op/s 2024-09-16T01:47:27.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:27 smithi176 ceph-mon[35324]: pgmap v215: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 312 KiB/s wr, 29 op/s 2024-09-16T01:47:28.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:27 smithi164 ceph-mon[35121]: pgmap v215: 129 pgs: 18 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 66 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 312 KiB/s wr, 29 op/s 2024-09-16T01:47:29.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:29 smithi139 ceph-mon[31175]: pgmap v216: 129 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 81 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 566 KiB/s wr, 39 op/s 2024-09-16T01:47:29.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:29 smithi176 ceph-mon[35324]: pgmap v216: 129 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 81 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 566 KiB/s wr, 39 op/s 2024-09-16T01:47:30.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:29 smithi164 ceph-mon[35121]: pgmap v216: 129 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 81 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 566 KiB/s wr, 39 op/s 2024-09-16T01:47:31.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:31 smithi139 ceph-mon[31175]: pgmap v217: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 518 KiB/s wr, 36 op/s 2024-09-16T01:47:31.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:31 smithi176 ceph-mon[35324]: pgmap v217: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 518 KiB/s wr, 36 op/s 2024-09-16T01:47:32.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:31 smithi164 ceph-mon[35121]: pgmap v217: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 518 KiB/s wr, 36 op/s 2024-09-16T01:47:33.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:33 smithi139 ceph-mon[31175]: pgmap v218: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 610 KiB/s wr, 36 op/s 2024-09-16T01:47:33.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:33 smithi176 ceph-mon[35324]: pgmap v218: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 610 KiB/s wr, 36 op/s 2024-09-16T01:47:34.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:33 smithi164 ceph-mon[35121]: pgmap v218: 129 pgs: 3 active+clean+snaptrim, 126 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 610 KiB/s wr, 36 op/s 2024-09-16T01:47:35.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:35 smithi139 ceph-mon[31175]: pgmap v219: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 557 KiB/s wr, 35 op/s 2024-09-16T01:47:35.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:35 smithi176 ceph-mon[35324]: pgmap v219: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 557 KiB/s wr, 35 op/s 2024-09-16T01:47:36.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:35 smithi164 ceph-mon[35121]: pgmap v219: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 557 KiB/s wr, 35 op/s 2024-09-16T01:47:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.428+0000 7f6dc3eac640 1 -- 172.21.15.139:0/2685432928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc105ec0 msgr2=0x7f6dbc106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.428+0000 7f6dc3eac640 1 --2- 172.21.15.139:0/2685432928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc105ec0 0x7f6dbc106370 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009cd0 tx=0x7f6db402a970 comp rx=0 tx=0).stop 2024-09-16T01:47:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.428+0000 7f6dc3eac640 1 -- 172.21.15.139:0/2685432928 shutdown_connections 2024-09-16T01:47:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.428+0000 7f6dc3eac640 1 --2- 172.21.15.139:0/2685432928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc105ec0 0x7f6dbc106370 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.428+0000 7f6dc3eac640 1 --2- 172.21.15.139:0/2685432928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dbc1013b0 0x7f6dbc105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.428+0000 7f6dc3eac640 1 --2- 172.21.15.139:0/2685432928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dbc100a90 0x7f6dbc100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.428+0000 7f6dc3eac640 1 -- 172.21.15.139:0/2685432928 >> 172.21.15.139:0/2685432928 conn(0x7f6dbc0fc6f0 msgr2=0x7f6dbc0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.428+0000 7f6dc3eac640 1 -- 172.21.15.139:0/2685432928 shutdown_connections 2024-09-16T01:47:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.428+0000 7f6dc3eac640 1 -- 172.21.15.139:0/2685432928 wait complete. 2024-09-16T01:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.429+0000 7f6dc3eac640 1 Processor -- start 2024-09-16T01:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.429+0000 7f6dc3eac640 1 -- start start 2024-09-16T01:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.429+0000 7f6dc3eac640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dbc100a90 0x7f6dbc1a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.429+0000 7f6dc3eac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc1013b0 0x7f6dbc1a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.429+0000 7f6dc3eac640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dbc105ec0 0x7f6dbc19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.429+0000 7f6dc3eac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc1184d0 con 0x7f6dbc1013b0 2024-09-16T01:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.429+0000 7f6dc3eac640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc118350 con 0x7f6dbc105ec0 2024-09-16T01:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.429+0000 7f6dc3eac640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc118650 con 0x7f6dbc100a90 2024-09-16T01:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.429+0000 7f6dc1447640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc1013b0 0x7f6dbc1a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.429+0000 7f6dc1447640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc1013b0 0x7f6dbc1a4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33168/0 (socket says 172.21.15.139:33168) 2024-09-16T01:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.429+0000 7f6dc1447640 1 -- 172.21.15.139:0/512256988 learned_addr learned my addr 172.21.15.139:0/512256988 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.429+0000 7f6dc1c48640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dbc100a90 0x7f6dbc1a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.430+0000 7f6dc2449640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dbc105ec0 0x7f6dbc19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.430+0000 7f6dc1447640 1 -- 172.21.15.139:0/512256988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dbc100a90 msgr2=0x7f6dbc1a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.430+0000 7f6dc1447640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dbc100a90 0x7f6dbc1a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.430+0000 7f6dc1447640 1 -- 172.21.15.139:0/512256988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dbc105ec0 msgr2=0x7f6dbc19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.430+0000 7f6dc1447640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dbc105ec0 0x7f6dbc19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.430+0000 7f6dc1447640 1 -- 172.21.15.139:0/512256988 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc19ef90 con 0x7f6dbc1013b0 2024-09-16T01:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.430+0000 7f6dc2449640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dbc105ec0 0x7f6dbc19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.430+0000 7f6dc1447640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc1013b0 0x7f6dbc1a4b90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6db000d950 tx=0x7f6db000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.430+0000 7f6daeffd640 1 -- 172.21.15.139:0/512256988 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0017070 con 0x7f6dbc1013b0 2024-09-16T01:47:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.430+0000 7f6daeffd640 1 -- 172.21.15.139:0/512256988 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db0009c30 con 0x7f6dbc1013b0 2024-09-16T01:47:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.430+0000 7f6daeffd640 1 -- 172.21.15.139:0/512256988 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db000f6b0 con 0x7f6dbc1013b0 2024-09-16T01:47:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.430+0000 7f6dc3eac640 1 -- 172.21.15.139:0/512256988 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dbc19f280 con 0x7f6dbc1013b0 2024-09-16T01:47:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.430+0000 7f6dc3eac640 1 -- 172.21.15.139:0/512256988 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dbc1ab4a0 con 0x7f6dbc1013b0 2024-09-16T01:47:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.433+0000 7f6daeffd640 1 -- 172.21.15.139:0/512256988 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6db00109d0 con 0x7f6dbc1013b0 2024-09-16T01:47:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.433+0000 7f6dc6c4b740 1 Processor -- start 2024-09-16T01:47:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.433+0000 7f6dc6c4b740 1 -- start start 2024-09-16T01:47:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.433+0000 7f6dc6c4b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f5547ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.433+0000 7f6dc6c4b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f5552770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.433+0000 7f6dc6c4b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 0x5560f55550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.433+0000 7f6dc6c4b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5560f54ac2f0 con 0x5560f5547af0 2024-09-16T01:47:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.433+0000 7f6dc6c4b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5560f5454270 con 0x5560f5552cb0 2024-09-16T01:47:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.433+0000 7f6dc6c4b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5560f54a82b0 con 0x5560f5548410 2024-09-16T01:47:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.433+0000 7f6dc2449640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f5547ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.433+0000 7f6dc2449640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f5547ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33178/0 (socket says 172.21.15.139:33178) 2024-09-16T01:47:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6dc2449640 1 -- 172.21.15.139:0/611339646 learned_addr learned my addr 172.21.15.139:0/611339646 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6daeffd640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6da80a3460 0x7f6da80a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6dc1447640 1 --2- 172.21.15.139:0/611339646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 0x5560f55550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6dc1c48640 1 --2- 172.21.15.139:0/611339646 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f5552770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6dc2449640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6da80a3460 0x7f6da80a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6dc2449640 1 -- 172.21.15.139:0/611339646 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 msgr2=0x5560f5552770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6dc2449640 1 --2- 172.21.15.139:0/611339646 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f5552770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6dc2449640 1 -- 172.21.15.139:0/611339646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 msgr2=0x5560f55550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6dc2449640 1 --2- 172.21.15.139:0/611339646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 0x5560f55550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6dc2449640 1 -- 172.21.15.139:0/611339646 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560f5453340 con 0x5560f5547af0 2024-09-16T01:47:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6daeffd640 1 -- 172.21.15.139:0/512256988 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9554+0+0 (secure 0 0 0) 0x7f6db00cfcc0 con 0x7f6dbc1013b0 2024-09-16T01:47:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6dc2449640 1 --2- 172.21.15.139:0/611339646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f5547ed0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6db4004af0 tx=0x7f6db402fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198991b0c7520c03 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6da7fff640 1 -- 172.21.15.139:0/611339646 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4043050 con 0x5560f5547af0 2024-09-16T01:47:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6da7fff640 1 -- 172.21.15.139:0/611339646 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db4044040 con 0x5560f5547af0 2024-09-16T01:47:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6da7fff640 1 -- 172.21.15.139:0/611339646 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db404c020 con 0x5560f5547af0 2024-09-16T01:47:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.434+0000 7f6dc2449640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6da80a3460 0x7f6da80a5920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6db404d000 tx=0x7f6db404b000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.435+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/611339646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 msgr2=0x5560f5547ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.435+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/611339646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f5547ed0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6db4004af0 tx=0x7f6db402fd60 comp rx=0 tx=0).stop 2024-09-16T01:47:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.435+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/611339646 shutdown_connections 2024-09-16T01:47:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.435+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/611339646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 0x5560f55550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.435+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/611339646 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f5552770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.435+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/611339646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f5547ed0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.435+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/611339646 >> 172.21.15.139:0/611339646 conn(0x5560f5544060 msgr2=0x5560f5544440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.435+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/611339646 shutdown_connections 2024-09-16T01:47:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.435+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/611339646 wait complete. 2024-09-16T01:47:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.436+0000 7f6dc6c4b740 1 -- start start 2024-09-16T01:47:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.436+0000 7f6dc6c4b740 1 Processor -- start 2024-09-16T01:47:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.436+0000 7f6dc6c4b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f554fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.436+0000 7f6dc6c4b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f55500d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.436+0000 7f6dc6c4b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 0x5560f5550610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.436+0000 7f6dc6c4b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5560f5555880 con 0x5560f5547af0 2024-09-16T01:47:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.436+0000 7f6dc6c4b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5560f5435290 con 0x5560f5552cb0 2024-09-16T01:47:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.436+0000 7f6dc6c4b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5560f5555a00 con 0x5560f5548410 2024-09-16T01:47:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.436+0000 7f6dc6c4b740 10 client.0 _ll_register_callbacks cb 0x5560f5466140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:47:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.436+0000 7f6dc1c48640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f554fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.436+0000 7f6dc1c48640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f554fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33194/0 (socket says 172.21.15.139:33194) 2024-09-16T01:47:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.436+0000 7f6dc1c48640 1 -- 172.21.15.139:0/1353718584 learned_addr learned my addr 172.21.15.139:0/1353718584 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.436+0000 7f6dc1447640 1 --2- 172.21.15.139:0/1353718584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f55500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6dc2449640 1 --2- 172.21.15.139:0/1353718584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 0x5560f5550610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6dc1c48640 1 -- 172.21.15.139:0/1353718584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 msgr2=0x5560f55500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6dc1c48640 1 --2- 172.21.15.139:0/1353718584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f55500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6dc1c48640 1 -- 172.21.15.139:0/1353718584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 msgr2=0x5560f5550610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6dc1c48640 1 --2- 172.21.15.139:0/1353718584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 0x5560f5550610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6dc1c48640 1 -- 172.21.15.139:0/1353718584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560f51bf210 con 0x5560f5547af0 2024-09-16T01:47:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6dc1c48640 1 --2- 172.21.15.139:0/1353718584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f554fb90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6db8012770 tx=0x7f6db8012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ca79e4b2e7c43a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6da57fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:47:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6da57fa640 1 -- 172.21.15.139:0/1353718584 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db800f920 con 0x5560f5547af0 2024-09-16T01:47:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6da57fa640 1 -- 172.21.15.139:0/1353718584 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9554+0+0 (secure 0 0 0) 0x7f6db800fac0 con 0x5560f5547af0 2024-09-16T01:47:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/1353718584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db4044040 con 0x5560f5547af0 2024-09-16T01:47:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6da57fa640 1 -- 172.21.15.139:0/1353718584 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6db800fcc0 con 0x5560f5547af0 2024-09-16T01:47:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6dc6c4b740 10 client.16284 fetch_fsmap learned FSMap version 41 2024-09-16T01:47:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/1353718584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5560f528b620 con 0x5560f5547af0 2024-09-16T01:47:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.437+0000 7f6da57fa640 1 -- 172.21.15.139:0/1353718584 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db4044040 con 0x5560f5547af0 2024-09-16T01:47:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.438+0000 7f6da57fa640 1 -- 172.21.15.139:0/1353718584 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 41) ==== 8434+0+0 (secure 0 0 0) 0x7f6db801c420 con 0x5560f5547af0 2024-09-16T01:47:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.438+0000 7f6dc6c4b740 10 client.16284 fetch_fsmap finished waiting for FSMap version 41 2024-09-16T01:47:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.438+0000 7f6dc6c4b740 10 client.16284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:47:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.438+0000 7f6dc6c4b740 20 client.16284 populate_metadata read hostname 'smithi139' 2024-09-16T01:47:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.438+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/1353718584 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5560f5562d10 0x5560f5565140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.438+0000 7f6dc6c4b740 4 client.16284 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:47:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.438+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/1353718584 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560f54b9a20 con 0x5560f5562d10 2024-09-16T01:47:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.438+0000 7f6dc1447640 1 --2- 172.21.15.139:0/1353718584 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5560f5562d10 0x5560f5565140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.438+0000 7f6dc1447640 1 --2- 172.21.15.139:0/1353718584 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5560f5562d10 0x5560f5565140 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7fc9e74741f3baf server_cookie=691b51db5ccab002 in_seq=0 out_seq=0 2024-09-16T01:47:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.438+0000 7f6da57fa640 10 client.16284 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.439+0000 7f6da57fa640 1 -- 172.21.15.139:0/1353718584 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560f54b9a20 con 0x5560f5562d10 2024-09-16T01:47:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.439+0000 7f6da57fa640 10 client.16284 handle_command_reply: tid=0 2024-09-16T01:47:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.439+0000 7f6dc6c4b740 1 client.16284 shutdown 2024-09-16T01:47:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.439+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/1353718584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 msgr2=0x5560f554fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.439+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/1353718584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f554fb90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6db8012770 tx=0x7f6db8012c30 comp rx=0 tx=0).stop 2024-09-16T01:47:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.439+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/1353718584 shutdown_connections 2024-09-16T01:47:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.439+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/1353718584 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5560f5562d10 0x5560f5565140 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.439+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/1353718584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 0x5560f5550610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.439+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/1353718584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f55500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.439+0000 7f6da57fa640 0 client.16284 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.439+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/1353718584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f554fb90 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.439+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/1353718584 >> 172.21.15.139:0/1353718584 conn(0x5560f5544060 msgr2=0x5560f55431c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.440+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/1353718584 shutdown_connections 2024-09-16T01:47:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.440+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/1353718584 wait complete. 2024-09-16T01:47:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.440+0000 7f6dc6c4b740 20 client.16284 trim_cache size 0 max 16384 2024-09-16T01:47:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.454+0000 7f6dc6c4b740 1 Processor -- start 2024-09-16T01:47:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.454+0000 7f6dc6c4b740 1 -- start start 2024-09-16T01:47:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.454+0000 7f6dc6c4b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5547af0 0x5560f554a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.454+0000 7f6dc6c4b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f554ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.454+0000 7f6dc6c4b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5552cb0 0x5560f5550460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.454+0000 7f6dc6c4b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc1b5650 con 0x5560f5552cb0 2024-09-16T01:47:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.454+0000 7f6dc6c4b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc1b73d0 con 0x5560f5547af0 2024-09-16T01:47:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.454+0000 7f6dc6c4b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc1b5de0 con 0x5560f5548410 2024-09-16T01:47:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.454+0000 7f6dc2449640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5552cb0 0x5560f5550460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.454+0000 7f6dc2449640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5552cb0 0x5560f5550460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33210/0 (socket says 172.21.15.139:33210) 2024-09-16T01:47:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.454+0000 7f6dc2449640 1 -- 172.21.15.139:0/576966430 learned_addr learned my addr 172.21.15.139:0/576966430 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.455+0000 7f6dc1c48640 1 --2- 172.21.15.139:0/576966430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5547af0 0x5560f554a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.455+0000 7f6dc1447640 1 --2- 172.21.15.139:0/576966430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f554ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.455+0000 7f6dc2449640 1 -- 172.21.15.139:0/576966430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 msgr2=0x5560f554ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.455+0000 7f6dc2449640 1 --2- 172.21.15.139:0/576966430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f554ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.455+0000 7f6dc2449640 1 -- 172.21.15.139:0/576966430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5547af0 msgr2=0x5560f554a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.455+0000 7f6dc2449640 1 --2- 172.21.15.139:0/576966430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5547af0 0x5560f554a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.455+0000 7f6dc2449640 1 -- 172.21.15.139:0/576966430 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560f5545110 con 0x5560f5552cb0 2024-09-16T01:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.455+0000 7f6dc2449640 1 --2- 172.21.15.139:0/576966430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5552cb0 0x5560f5550460 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009600 tx=0x7f6db4004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34b6f9e6def246b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.455+0000 7f6da77fe640 1 -- 172.21.15.139:0/576966430 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db404c040 con 0x5560f5552cb0 2024-09-16T01:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.455+0000 7f6da77fe640 1 -- 172.21.15.139:0/576966430 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db402f750 con 0x5560f5552cb0 2024-09-16T01:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.455+0000 7f6da77fe640 1 -- 172.21.15.139:0/576966430 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4034070 con 0x5560f5552cb0 2024-09-16T01:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/576966430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5552cb0 msgr2=0x5560f5550460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/576966430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5552cb0 0x5560f5550460 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009600 tx=0x7f6db4004a80 comp rx=0 tx=0).stop 2024-09-16T01:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/576966430 shutdown_connections 2024-09-16T01:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/576966430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5552cb0 0x5560f5550460 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/576966430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f554ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/576966430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5547af0 0x5560f554a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/576966430 >> 172.21.15.139:0/576966430 conn(0x5560f5544060 msgr2=0x5560f5551c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/576966430 shutdown_connections 2024-09-16T01:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/576966430 wait complete. 2024-09-16T01:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 -- start start 2024-09-16T01:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 Processor -- start 2024-09-16T01:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f5556430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f5556970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 0x5560f55c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc1b6c40 con 0x5560f5547af0 2024-09-16T01:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc1b6570 con 0x5560f5552cb0 2024-09-16T01:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc1b3650 con 0x5560f5548410 2024-09-16T01:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.456+0000 7f6dc6c4b740 10 client.0 _ll_register_callbacks cb 0x7f6db8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6dc1c48640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f5556430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6dc1c48640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f5556430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33214/0 (socket says 172.21.15.139:33214) 2024-09-16T01:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6dc1c48640 1 -- 172.21.15.139:0/3130804561 learned_addr learned my addr 172.21.15.139:0/3130804561 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6dc2449640 1 --2- 172.21.15.139:0/3130804561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 0x5560f55c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6dc1c48640 1 -- 172.21.15.139:0/3130804561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 msgr2=0x5560f5556970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6dc1c48640 1 --2- 172.21.15.139:0/3130804561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f5556970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6dc1c48640 1 -- 172.21.15.139:0/3130804561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 msgr2=0x5560f55c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6dc1c48640 1 --2- 172.21.15.139:0/3130804561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 0x5560f55c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6dc1c48640 1 -- 172.21.15.139:0/3130804561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560f553f2d0 con 0x5560f5547af0 2024-09-16T01:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6dc1c48640 1 --2- 172.21.15.139:0/3130804561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f5556430 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6db801c890 tx=0x7f6db801c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b2e03e3c77e212 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6da57fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6da57fa640 1 -- 172.21.15.139:0/3130804561 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db800fe60 con 0x5560f5547af0 2024-09-16T01:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6da57fa640 1 -- 172.21.15.139:0/3130804561 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9554+0+0 (secure 0 0 0) 0x7f6db801cd10 con 0x5560f5547af0 2024-09-16T01:47:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/3130804561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db402f750 con 0x5560f5547af0 2024-09-16T01:47:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6da57fa640 1 -- 172.21.15.139:0/3130804561 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6db80050c0 con 0x5560f5547af0 2024-09-16T01:47:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6dc6c4b740 10 client.16290 fetch_fsmap learned FSMap version 41 2024-09-16T01:47:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/3130804561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5560f55cd200 con 0x5560f5547af0 2024-09-16T01:47:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.457+0000 7f6da57fa640 1 -- 172.21.15.139:0/3130804561 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db402f750 con 0x5560f5547af0 2024-09-16T01:47:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.458+0000 7f6da57fa640 1 -- 172.21.15.139:0/3130804561 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 41) ==== 8434+0+0 (secure 0 0 0) 0x7f6db8003a40 con 0x5560f5547af0 2024-09-16T01:47:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.458+0000 7f6dc6c4b740 10 client.16290 fetch_fsmap finished waiting for FSMap version 41 2024-09-16T01:47:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.458+0000 7f6dc6c4b740 10 client.16290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:47:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.458+0000 7f6dc6c4b740 20 client.16290 populate_metadata read hostname 'smithi139' 2024-09-16T01:47:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.458+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/3130804561 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5560f55cdac0 0x5560f55cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.458+0000 7f6dc6c4b740 4 client.16290 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:47:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.458+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/3130804561 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560f54b3d70 con 0x5560f55cdac0 2024-09-16T01:47:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.458+0000 7f6dc1447640 1 --2- 172.21.15.139:0/3130804561 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5560f55cdac0 0x5560f55cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.458+0000 7f6da57fa640 10 client.16290 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.458+0000 7f6dc1447640 1 --2- 172.21.15.139:0/3130804561 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5560f55cdac0 0x5560f55cff00 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4339e4a00ef6c04 server_cookie=846dd92a7de4dfd in_seq=0 out_seq=0 2024-09-16T01:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6da57fa640 1 -- 172.21.15.139:0/3130804561 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5560f54b3d70 con 0x5560f55cdac0 2024-09-16T01:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6da57fa640 10 client.16290 handle_command_reply: tid=0 2024-09-16T01:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6dc6c4b740 1 client.16290 shutdown 2024-09-16T01:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/3130804561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 msgr2=0x5560f5556430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/3130804561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f5556430 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6db801c890 tx=0x7f6db801c6b0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/3130804561 shutdown_connections 2024-09-16T01:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/3130804561 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5560f55cdac0 0x5560f55cff00 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/3130804561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f5552cb0 0x5560f55c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/3130804561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560f5548410 0x5560f5556970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6da57fa640 0 client.16290 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6dc6c4b740 1 --2- 172.21.15.139:0/3130804561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5560f5547af0 0x5560f5556430 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/3130804561 >> 172.21.15.139:0/3130804561 conn(0x5560f5544060 msgr2=0x5560f55429a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/3130804561 shutdown_connections 2024-09-16T01:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6dc6c4b740 1 -- 172.21.15.139:0/3130804561 wait complete. 2024-09-16T01:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.476+0000 7f6dc6c4b740 20 client.16290 trim_cache size 0 max 16384 2024-09-16T01:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.477+0000 7f6dc3eac640 1 -- 172.21.15.139:0/512256988 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6da80a3460 msgr2=0x7f6da80a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.477+0000 7f6dc3eac640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6da80a3460 0x7f6da80a5920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6db404d000 tx=0x7f6db404b000 comp rx=0 tx=0).stop 2024-09-16T01:47:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.477+0000 7f6dc3eac640 1 -- 172.21.15.139:0/512256988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc1013b0 msgr2=0x7f6dbc1a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.477+0000 7f6dc3eac640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc1013b0 0x7f6dbc1a4b90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6db000d950 tx=0x7f6db000de10 comp rx=0 tx=0).stop 2024-09-16T01:47:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.477+0000 7f6dc3eac640 1 -- 172.21.15.139:0/512256988 shutdown_connections 2024-09-16T01:47:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.477+0000 7f6dc3eac640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6da80a3460 0x7f6da80a5920 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.477+0000 7f6dc3eac640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dbc105ec0 0x7f6dbc19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.477+0000 7f6dc3eac640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc1013b0 0x7f6dbc1a4b90 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.477+0000 7f6dc3eac640 1 --2- 172.21.15.139:0/512256988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dbc100a90 0x7f6dbc1a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.477+0000 7f6dc3eac640 1 -- 172.21.15.139:0/512256988 >> 172.21.15.139:0/512256988 conn(0x7f6dbc0fc6f0 msgr2=0x7f6dbc103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.477+0000 7f6dc3eac640 1 -- 172.21.15.139:0/512256988 shutdown_connections 2024-09-16T01:47:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:36.477+0000 7f6dc3eac640 1 -- 172.21.15.139:0/512256988 wait complete. 2024-09-16T01:47:37.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:36 smithi139 ceph-mon[31175]: pgmap v220: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 641 KiB/s wr, 33 op/s 2024-09-16T01:47:37.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:36 smithi139 ceph-mon[31175]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T01:47:37.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:36 smithi176 ceph-mon[35324]: pgmap v220: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 641 KiB/s wr, 33 op/s 2024-09-16T01:47:37.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:36 smithi176 ceph-mon[35324]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T01:47:37.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:36 smithi164 ceph-mon[35121]: pgmap v220: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 641 KiB/s wr, 33 op/s 2024-09-16T01:47:37.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:36 smithi164 ceph-mon[35121]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T01:47:39.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:38 smithi139 ceph-mon[31175]: pgmap v222: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 504 KiB/s wr, 25 op/s 2024-09-16T01:47:39.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:38 smithi176 ceph-mon[35324]: pgmap v222: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 504 KiB/s wr, 25 op/s 2024-09-16T01:47:39.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:38 smithi164 ceph-mon[35121]: pgmap v222: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 504 KiB/s wr, 25 op/s 2024-09-16T01:47:41.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:40 smithi139 ceph-mon[31175]: pgmap v223: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 507 KiB/s wr, 27 op/s 2024-09-16T01:47:41.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:40 smithi176 ceph-mon[35324]: pgmap v223: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 507 KiB/s wr, 27 op/s 2024-09-16T01:47:41.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:40 smithi164 ceph-mon[35121]: pgmap v223: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 507 KiB/s wr, 27 op/s 2024-09-16T01:47:42.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:41 smithi139 ceph-mon[31175]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T01:47:42.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:41 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:47:42.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:41 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:47:42.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:41 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:47:42.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:41 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:47:42.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:41 smithi176 ceph-mon[35324]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T01:47:42.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:41 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:47:42.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:41 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:47:42.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:41 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:47:42.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:41 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:47:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:41 smithi164 ceph-mon[35121]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T01:47:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:41 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:47:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:41 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:47:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:41 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:47:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:41 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:47:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:43 smithi139 ceph-mon[31175]: pgmap v225: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 424 KiB/s wr, 28 op/s 2024-09-16T01:47:43.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:43 smithi176 ceph-mon[35324]: pgmap v225: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 424 KiB/s wr, 28 op/s 2024-09-16T01:47:44.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:43 smithi164 ceph-mon[35121]: pgmap v225: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 424 KiB/s wr, 28 op/s 2024-09-16T01:47:45.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:45 smithi139 ceph-mon[31175]: pgmap v226: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-09-16T01:47:45.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:45 smithi176 ceph-mon[35324]: pgmap v226: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-09-16T01:47:46.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:45 smithi164 ceph-mon[35121]: pgmap v226: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-09-16T01:47:47.846 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:47 smithi164 ceph-mon[35121]: pgmap v227: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 87 KiB/s wr, 25 op/s 2024-09-16T01:47:47.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:47 smithi139 ceph-mon[31175]: pgmap v227: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 87 KiB/s wr, 25 op/s 2024-09-16T01:47:47.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:47 smithi176 ceph-mon[35324]: pgmap v227: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 87 KiB/s wr, 25 op/s 2024-09-16T01:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.575+0000 7fc322f6d640 1 -- 172.21.15.139:0/2906319602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc31c101ae0 msgr2=0x7fc31c101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.575+0000 7fc322f6d640 1 --2- 172.21.15.139:0/2906319602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc31c101ae0 0x7fc31c101f40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc308009930 tx=0x7fc30802a420 comp rx=0 tx=0).stop 2024-09-16T01:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.575+0000 7fc322f6d640 1 -- 172.21.15.139:0/2906319602 shutdown_connections 2024-09-16T01:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.575+0000 7fc322f6d640 1 --2- 172.21.15.139:0/2906319602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c102480 0x7fc31c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.575+0000 7fc322f6d640 1 --2- 172.21.15.139:0/2906319602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc31c101ae0 0x7fc31c101f40 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.575+0000 7fc322f6d640 1 --2- 172.21.15.139:0/2906319602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c107b80 0x7fc31c107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.575+0000 7fc322f6d640 1 -- 172.21.15.139:0/2906319602 >> 172.21.15.139:0/2906319602 conn(0x7fc31c0fd7e0 msgr2=0x7fc31c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.575+0000 7fc322f6d640 1 -- 172.21.15.139:0/2906319602 shutdown_connections 2024-09-16T01:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.575+0000 7fc322f6d640 1 -- 172.21.15.139:0/2906319602 wait complete. 2024-09-16T01:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.576+0000 7fc322f6d640 1 Processor -- start 2024-09-16T01:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.576+0000 7fc322f6d640 1 -- start start 2024-09-16T01:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.576+0000 7fc322f6d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c101ae0 0x7fc31c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.576+0000 7fc322f6d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c102480 0x7fc31c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.576+0000 7fc322f6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc31c107b80 0x7fc31c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.576+0000 7fc322f6d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c111f30 con 0x7fc31c107b80 2024-09-16T01:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.576+0000 7fc322f6d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c111db0 con 0x7fc31c101ae0 2024-09-16T01:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.576+0000 7fc322f6d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c1120b0 con 0x7fc31c102480 2024-09-16T01:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.576+0000 7fc32150a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc31c107b80 0x7fc31c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.576+0000 7fc32150a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc31c107b80 0x7fc31c19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55488/0 (socket says 172.21.15.139:55488) 2024-09-16T01:47:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.576+0000 7fc32150a640 1 -- 172.21.15.139:0/4159398592 learned_addr learned my addr 172.21.15.139:0/4159398592 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.576+0000 7fc320d09640 1 --2- 172.21.15.139:0/4159398592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c101ae0 0x7fc31c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.576+0000 7fc31bfff640 1 --2- 172.21.15.139:0/4159398592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c102480 0x7fc31c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.577+0000 7fc32150a640 1 -- 172.21.15.139:0/4159398592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c102480 msgr2=0x7fc31c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.577+0000 7fc32150a640 1 --2- 172.21.15.139:0/4159398592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c102480 0x7fc31c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.577+0000 7fc32150a640 1 -- 172.21.15.139:0/4159398592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c101ae0 msgr2=0x7fc31c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.577+0000 7fc32150a640 1 --2- 172.21.15.139:0/4159398592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c101ae0 0x7fc31c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.577+0000 7fc32150a640 1 -- 172.21.15.139:0/4159398592 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31c19d370 con 0x7fc31c107b80 2024-09-16T01:47:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.577+0000 7fc32150a640 1 --2- 172.21.15.139:0/4159398592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc31c107b80 0x7fc31c19cc40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc31400eb10 tx=0x7fc31400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.577+0000 7fc319ffb640 1 -- 172.21.15.139:0/4159398592 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314004b20 con 0x7fc31c107b80 2024-09-16T01:47:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.577+0000 7fc319ffb640 1 -- 172.21.15.139:0/4159398592 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc314004cc0 con 0x7fc31c107b80 2024-09-16T01:47:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.577+0000 7fc322f6d640 1 -- 172.21.15.139:0/4159398592 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31c19d660 con 0x7fc31c107b80 2024-09-16T01:47:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.577+0000 7fc319ffb640 1 -- 172.21.15.139:0/4159398592 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314010760 con 0x7fc31c107b80 2024-09-16T01:47:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.577+0000 7fc322f6d640 1 -- 172.21.15.139:0/4159398592 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc31c1a4ef0 con 0x7fc31c107b80 2024-09-16T01:47:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.579+0000 7fc319ffb640 1 -- 172.21.15.139:0/4159398592 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc314005020 con 0x7fc31c107b80 2024-09-16T01:47:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.580+0000 7fc325dec740 1 Processor -- start 2024-09-16T01:47:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.580+0000 7fc325dec740 1 -- start start 2024-09-16T01:47:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.580+0000 7fc325dec740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae30af0 0x5594dae30ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.580+0000 7fc325dec740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae3b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.580+0000 7fc325dec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae3bcb0 0x5594dae3e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.580+0000 7fc325dec740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5594dad952f0 con 0x5594dae31410 2024-09-16T01:47:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.580+0000 7fc325dec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5594dad3d270 con 0x5594dae3bcb0 2024-09-16T01:47:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.580+0000 7fc319ffb640 1 --2- 172.21.15.139:0/4159398592 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc30c0a3460 0x7fc30c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.580+0000 7fc325dec740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5594dad912b0 con 0x5594dae30af0 2024-09-16T01:47:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc320d09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae3b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc320d09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae3b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55496/0 (socket says 172.21.15.139:55496) 2024-09-16T01:47:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc320d09640 1 -- 172.21.15.139:0/1923348519 learned_addr learned my addr 172.21.15.139:0/1923348519 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc32150a640 1 --2- 172.21.15.139:0/4159398592 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc30c0a3460 0x7fc30c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc31bfff640 1 --2- 172.21.15.139:0/1923348519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae3bcb0 0x5594dae3e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc31bfff640 1 --2- 172.21.15.139:0/1923348519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae30af0 0x5594dae30ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc319ffb640 1 -- 172.21.15.139:0/4159398592 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9510+0+0 (secure 0 0 0) 0x7fc3140d08b0 con 0x7fc31c107b80 2024-09-16T01:47:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc320d09640 1 -- 172.21.15.139:0/1923348519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae30af0 msgr2=0x5594dae30ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc320d09640 1 --2- 172.21.15.139:0/1923348519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae30af0 0x5594dae30ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc320d09640 1 -- 172.21.15.139:0/1923348519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae3bcb0 msgr2=0x5594dae3e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc320d09640 1 --2- 172.21.15.139:0/1923348519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae3bcb0 0x5594dae3e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc320d09640 1 -- 172.21.15.139:0/1923348519 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594dad3c340 con 0x5594dae31410 2024-09-16T01:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc32150a640 1 --2- 172.21.15.139:0/4159398592 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc30c0a3460 0x7fc30c0a5920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc3140a4e90 tx=0x7fc314004e70 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc320d09640 1 --2- 172.21.15.139:0/1923348519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae3b770 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc31000bc90 tx=0x7fc3100107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcb971bdf2d645c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc306ffd640 1 -- 172.21.15.139:0/1923348519 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31001c480 con 0x5594dae31410 2024-09-16T01:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc306ffd640 1 -- 172.21.15.139:0/1923348519 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc31001cb00 con 0x5594dae31410 2024-09-16T01:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.581+0000 7fc306ffd640 1 -- 172.21.15.139:0/1923348519 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310011bb0 con 0x5594dae31410 2024-09-16T01:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.582+0000 7fc325dec740 1 -- 172.21.15.139:0/1923348519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 msgr2=0x5594dae3b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.582+0000 7fc325dec740 1 --2- 172.21.15.139:0/1923348519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae3b770 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc31000bc90 tx=0x7fc3100107a0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.582+0000 7fc325dec740 1 -- 172.21.15.139:0/1923348519 shutdown_connections 2024-09-16T01:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.582+0000 7fc325dec740 1 --2- 172.21.15.139:0/1923348519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae3bcb0 0x5594dae3e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.582+0000 7fc325dec740 1 --2- 172.21.15.139:0/1923348519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae3b770 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.582+0000 7fc325dec740 1 --2- 172.21.15.139:0/1923348519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae30af0 0x5594dae30ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.582+0000 7fc325dec740 1 -- 172.21.15.139:0/1923348519 >> 172.21.15.139:0/1923348519 conn(0x5594dae2d060 msgr2=0x5594dae2d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.582+0000 7fc325dec740 1 -- 172.21.15.139:0/1923348519 shutdown_connections 2024-09-16T01:47:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.582+0000 7fc325dec740 1 -- 172.21.15.139:0/1923348519 wait complete. 2024-09-16T01:47:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.582+0000 7fc325dec740 1 -- start start 2024-09-16T01:47:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc325dec740 1 Processor -- start 2024-09-16T01:47:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc325dec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 0x5594dae38c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc325dec740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae39150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc325dec740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae3bcb0 0x5594dae39690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc325dec740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5594dae3e880 con 0x5594dae31410 2024-09-16T01:47:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc325dec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5594dad1e290 con 0x5594dae30af0 2024-09-16T01:47:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc325dec740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5594dae3ea00 con 0x5594dae3bcb0 2024-09-16T01:47:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc325dec740 10 client.0 _ll_register_callbacks cb 0x5594dad4f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc320d09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae39150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc320d09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae39150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55498/0 (socket says 172.21.15.139:55498) 2024-09-16T01:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc320d09640 1 -- 172.21.15.139:0/125101996 learned_addr learned my addr 172.21.15.139:0/125101996 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc31bfff640 1 --2- 172.21.15.139:0/125101996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 0x5594dae38c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc320d09640 1 -- 172.21.15.139:0/125101996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae3bcb0 msgr2=0x5594dae39690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc320d09640 1 --2- 172.21.15.139:0/125101996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae3bcb0 0x5594dae39690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc320d09640 1 -- 172.21.15.139:0/125101996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 msgr2=0x5594dae38c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc320d09640 1 --2- 172.21.15.139:0/125101996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 0x5594dae38c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.583+0000 7fc320d09640 1 -- 172.21.15.139:0/125101996 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594daaa8210 con 0x5594dae31410 2024-09-16T01:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc320d09640 1 --2- 172.21.15.139:0/125101996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae39150 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc3100117d0 tx=0x7fc3100058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89641d0171ca7f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc2fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc2fbfff640 1 -- 172.21.15.139:0/125101996 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31001c890 con 0x5594dae31410 2024-09-16T01:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc2fbfff640 1 -- 172.21.15.139:0/125101996 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9510+0+0 (secure 0 0 0) 0x7fc310011930 con 0x5594dae31410 2024-09-16T01:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc325dec740 1 -- 172.21.15.139:0/125101996 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc31001cb00 con 0x5594dae31410 2024-09-16T01:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc2fbfff640 1 -- 172.21.15.139:0/125101996 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc310032050 con 0x5594dae31410 2024-09-16T01:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc325dec740 10 client.16302 fetch_fsmap learned FSMap version 41 2024-09-16T01:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc325dec740 1 -- 172.21.15.139:0/125101996 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5594dab74620 con 0x5594dae31410 2024-09-16T01:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc2fbfff640 1 -- 172.21.15.139:0/125101996 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc31001cb00 con 0x5594dae31410 2024-09-16T01:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc2fbfff640 1 -- 172.21.15.139:0/125101996 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 41) ==== 8434+0+0 (secure 0 0 0) 0x7fc3100047a0 con 0x5594dae31410 2024-09-16T01:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc325dec740 10 client.16302 fetch_fsmap finished waiting for FSMap version 41 2024-09-16T01:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc325dec740 10 client.16302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc325dec740 20 client.16302 populate_metadata read hostname 'smithi139' 2024-09-16T01:47:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc325dec740 1 --2- 172.21.15.139:0/125101996 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5594dae4bc10 0x5594dae4e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc325dec740 4 client.16302 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:47:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.584+0000 7fc325dec740 1 -- 172.21.15.139:0/125101996 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594dada2a20 con 0x5594dae4bc10 2024-09-16T01:47:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.585+0000 7fc31bfff640 1 --2- 172.21.15.139:0/125101996 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5594dae4bc10 0x5594dae4e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.585+0000 7fc31bfff640 1 --2- 172.21.15.139:0/125101996 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5594dae4bc10 0x5594dae4e000 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43f2d8b3e917caae server_cookie=2b469c462461adec in_seq=0 out_seq=0 2024-09-16T01:47:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.585+0000 7fc2fbfff640 10 client.16302 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc2fbfff640 1 -- 172.21.15.139:0/125101996 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594dada2a20 con 0x5594dae4bc10 2024-09-16T01:47:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc2fbfff640 10 client.16302 handle_command_reply: tid=0 2024-09-16T01:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc325dec740 1 client.16302 shutdown 2024-09-16T01:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc325dec740 1 -- 172.21.15.139:0/125101996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 msgr2=0x5594dae39150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc325dec740 1 --2- 172.21.15.139:0/125101996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae39150 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc3100117d0 tx=0x7fc3100058e0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc325dec740 1 -- 172.21.15.139:0/125101996 shutdown_connections 2024-09-16T01:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc325dec740 1 --2- 172.21.15.139:0/125101996 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5594dae4bc10 0x5594dae4e000 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc325dec740 1 --2- 172.21.15.139:0/125101996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae3bcb0 0x5594dae39690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc325dec740 1 --2- 172.21.15.139:0/125101996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae39150 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc2fbfff640 0 client.16302 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc325dec740 1 --2- 172.21.15.139:0/125101996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 0x5594dae38c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc325dec740 1 -- 172.21.15.139:0/125101996 >> 172.21.15.139:0/125101996 conn(0x5594dae2d060 msgr2=0x5594dae2c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc325dec740 1 -- 172.21.15.139:0/125101996 shutdown_connections 2024-09-16T01:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc325dec740 1 -- 172.21.15.139:0/125101996 wait complete. 2024-09-16T01:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.586+0000 7fc325dec740 20 client.16302 trim_cache size 0 max 16384 2024-09-16T01:47:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.600+0000 7fc325dec740 1 Processor -- start 2024-09-16T01:47:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.600+0000 7fc325dec740 1 -- start start 2024-09-16T01:47:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc325dec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 0x5594dae394c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc325dec740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae31410 0x5594dae53830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc325dec740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae3bcb0 0x5594dae53d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc325dec740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c1af1a0 con 0x5594dae3bcb0 2024-09-16T01:47:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc325dec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c1b0f20 con 0x5594dae30af0 2024-09-16T01:47:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc325dec740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c1af930 con 0x5594dae31410 2024-09-16T01:47:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc31bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae3bcb0 0x5594dae53d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc31bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae3bcb0 0x5594dae53d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55506/0 (socket says 172.21.15.139:55506) 2024-09-16T01:47:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc31bfff640 1 -- 172.21.15.139:0/3137881264 learned_addr learned my addr 172.21.15.139:0/3137881264 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc320d09640 1 --2- 172.21.15.139:0/3137881264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae31410 0x5594dae53830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc31bfff640 1 --2- 172.21.15.139:0/3137881264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 0x5594dae394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc31bfff640 1 -- 172.21.15.139:0/3137881264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae31410 msgr2=0x5594dae53830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc31bfff640 1 --2- 172.21.15.139:0/3137881264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae31410 0x5594dae53830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc31bfff640 1 -- 172.21.15.139:0/3137881264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 msgr2=0x5594dae394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc31bfff640 1 --2- 172.21.15.139:0/3137881264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 0x5594dae394c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc31bfff640 1 -- 172.21.15.139:0/3137881264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594dae272f0 con 0x5594dae3bcb0 2024-09-16T01:47:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc31bfff640 1 --2- 172.21.15.139:0/3137881264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae3bcb0 0x5594dae53d70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc3080092a0 tx=0x7fc308005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df659fb2941c3862 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc3067fc640 1 -- 172.21.15.139:0/3137881264 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308035040 con 0x5594dae3bcb0 2024-09-16T01:47:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc3067fc640 1 -- 172.21.15.139:0/3137881264 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3080053d0 con 0x5594dae3bcb0 2024-09-16T01:47:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.601+0000 7fc3067fc640 1 -- 172.21.15.139:0/3137881264 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308004d20 con 0x5594dae3bcb0 2024-09-16T01:47:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.602+0000 7fc325dec740 1 -- 172.21.15.139:0/3137881264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae3bcb0 msgr2=0x5594dae53d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.602+0000 7fc325dec740 1 --2- 172.21.15.139:0/3137881264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae3bcb0 0x5594dae53d70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc3080092a0 tx=0x7fc308005750 comp rx=0 tx=0).stop 2024-09-16T01:47:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.602+0000 7fc325dec740 1 -- 172.21.15.139:0/3137881264 shutdown_connections 2024-09-16T01:47:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.602+0000 7fc325dec740 1 --2- 172.21.15.139:0/3137881264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae3bcb0 0x5594dae53d70 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.602+0000 7fc325dec740 1 --2- 172.21.15.139:0/3137881264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae31410 0x5594dae53830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.602+0000 7fc325dec740 1 --2- 172.21.15.139:0/3137881264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 0x5594dae394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.602+0000 7fc325dec740 1 -- 172.21.15.139:0/3137881264 >> 172.21.15.139:0/3137881264 conn(0x5594dae2d060 msgr2=0x5594dae3ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.602+0000 7fc325dec740 1 -- 172.21.15.139:0/3137881264 shutdown_connections 2024-09-16T01:47:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.602+0000 7fc325dec740 1 -- 172.21.15.139:0/3137881264 wait complete. 2024-09-16T01:47:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.602+0000 7fc325dec740 1 -- start start 2024-09-16T01:47:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.602+0000 7fc325dec740 1 Processor -- start 2024-09-16T01:47:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc325dec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 0x5594dae35cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc325dec740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae361f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc325dec740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae3bcb0 0x5594dae36730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc325dec740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c1b0790 con 0x5594dae31410 2024-09-16T01:47:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc325dec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c1b00c0 con 0x5594dae30af0 2024-09-16T01:47:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc325dec740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c1ad1a0 con 0x5594dae3bcb0 2024-09-16T01:47:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc325dec740 10 client.0 _ll_register_callbacks cb 0x7fc3140093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:47:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc320d09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc320d09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae361f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55522/0 (socket says 172.21.15.139:55522) 2024-09-16T01:47:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc320d09640 1 -- 172.21.15.139:0/2015252123 learned_addr learned my addr 172.21.15.139:0/2015252123 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:47:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc31bfff640 1 --2- 172.21.15.139:0/2015252123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 0x5594dae35cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc31bfff640 1 --2- 172.21.15.139:0/2015252123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae3bcb0 0x5594dae36730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc320d09640 1 -- 172.21.15.139:0/2015252123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae3bcb0 msgr2=0x5594dae36730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc320d09640 1 --2- 172.21.15.139:0/2015252123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae3bcb0 0x5594dae36730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc320d09640 1 -- 172.21.15.139:0/2015252123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 msgr2=0x5594dae35cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:47:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc320d09640 1 --2- 172.21.15.139:0/2015252123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 0x5594dae35cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc320d09640 1 -- 172.21.15.139:0/2015252123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594daeb9c90 con 0x5594dae31410 2024-09-16T01:47:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc31bfff640 1 --2- 172.21.15.139:0/2015252123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae3bcb0 0x5594dae36730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc320d09640 1 --2- 172.21.15.139:0/2015252123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae361f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc310004d60 tx=0x7fc310010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f202fc697c11684 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc2fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc2fbfff640 1 -- 172.21.15.139:0/2015252123 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310011a90 con 0x5594dae31410 2024-09-16T01:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc2fbfff640 1 -- 172.21.15.139:0/2015252123 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9510+0+0 (secure 0 0 0) 0x7fc310011c30 con 0x5594dae31410 2024-09-16T01:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.603+0000 7fc325dec740 1 -- 172.21.15.139:0/2015252123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3080053d0 con 0x5594dae31410 2024-09-16T01:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.604+0000 7fc2fbfff640 1 -- 172.21.15.139:0/2015252123 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc310011e50 con 0x5594dae31410 2024-09-16T01:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.604+0000 7fc325dec740 10 client.16308 fetch_fsmap learned FSMap version 41 2024-09-16T01:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.604+0000 7fc325dec740 1 -- 172.21.15.139:0/2015252123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5594daebaf80 con 0x5594dae31410 2024-09-16T01:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.604+0000 7fc2fbfff640 1 -- 172.21.15.139:0/2015252123 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3080053d0 con 0x5594dae31410 2024-09-16T01:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.604+0000 7fc2fbfff640 1 -- 172.21.15.139:0/2015252123 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 41) ==== 8434+0+0 (secure 0 0 0) 0x7fc310003890 con 0x5594dae31410 2024-09-16T01:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.604+0000 7fc325dec740 10 client.16308 fetch_fsmap finished waiting for FSMap version 41 2024-09-16T01:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.604+0000 7fc325dec740 10 client.16308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.604+0000 7fc325dec740 20 client.16308 populate_metadata read hostname 'smithi139' 2024-09-16T01:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.604+0000 7fc325dec740 1 --2- 172.21.15.139:0/2015252123 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5594daebb750 0x5594daebdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:47:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.604+0000 7fc325dec740 4 client.16308 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:47:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.604+0000 7fc325dec740 1 -- 172.21.15.139:0/2015252123 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5594dad9cd70 con 0x5594daebb750 2024-09-16T01:47:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.604+0000 7fc31bfff640 1 --2- 172.21.15.139:0/2015252123 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5594daebb750 0x5594daebdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:47:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.604+0000 7fc2fbfff640 10 client.16308 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.604+0000 7fc31bfff640 1 --2- 172.21.15.139:0/2015252123 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5594daebb750 0x5594daebdb70 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b1f51ec3e0bf46 server_cookie=574e3ac544b16dbe in_seq=0 out_seq=0 2024-09-16T01:47:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.618+0000 7fc2fbfff640 1 -- 172.21.15.139:0/2015252123 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5594dad9cd70 con 0x5594daebb750 2024-09-16T01:47:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.618+0000 7fc2fbfff640 10 client.16308 handle_command_reply: tid=0 2024-09-16T01:47:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.618+0000 7fc325dec740 1 client.16308 shutdown 2024-09-16T01:47:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.618+0000 7fc325dec740 1 -- 172.21.15.139:0/2015252123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 msgr2=0x5594dae361f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.618+0000 7fc325dec740 1 --2- 172.21.15.139:0/2015252123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae361f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc310004d60 tx=0x7fc310010d50 comp rx=0 tx=0).stop 2024-09-16T01:47:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.618+0000 7fc325dec740 1 -- 172.21.15.139:0/2015252123 shutdown_connections 2024-09-16T01:47:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.618+0000 7fc325dec740 1 --2- 172.21.15.139:0/2015252123 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5594daebb750 0x5594daebdb70 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.618+0000 7fc325dec740 1 --2- 172.21.15.139:0/2015252123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594dae3bcb0 0x5594dae36730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.618+0000 7fc325dec740 1 --2- 172.21.15.139:0/2015252123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5594dae31410 0x5594dae361f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.618+0000 7fc325dec740 1 --2- 172.21.15.139:0/2015252123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594dae30af0 0x5594dae35cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.618+0000 7fc2fbfff640 0 client.16308 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:47:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.618+0000 7fc325dec740 1 -- 172.21.15.139:0/2015252123 >> 172.21.15.139:0/2015252123 conn(0x5594dae2d060 msgr2=0x5594dae2b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.618+0000 7fc325dec740 1 -- 172.21.15.139:0/2015252123 shutdown_connections 2024-09-16T01:47:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.618+0000 7fc325dec740 1 -- 172.21.15.139:0/2015252123 wait complete. 2024-09-16T01:47:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.619+0000 7fc325dec740 20 client.16308 trim_cache size 0 max 16384 2024-09-16T01:47:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:47:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:47:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:47:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:47:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.619+0000 7fc322f6d640 1 -- 172.21.15.139:0/4159398592 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc30c0a3460 msgr2=0x7fc30c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.619+0000 7fc322f6d640 1 --2- 172.21.15.139:0/4159398592 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc30c0a3460 0x7fc30c0a5920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc3140a4e90 tx=0x7fc314004e70 comp rx=0 tx=0).stop 2024-09-16T01:47:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.619+0000 7fc322f6d640 1 -- 172.21.15.139:0/4159398592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc31c107b80 msgr2=0x7fc31c19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:47:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.619+0000 7fc322f6d640 1 --2- 172.21.15.139:0/4159398592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc31c107b80 0x7fc31c19cc40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc31400eb10 tx=0x7fc31400c2d0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.619+0000 7fc322f6d640 1 -- 172.21.15.139:0/4159398592 shutdown_connections 2024-09-16T01:47:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.619+0000 7fc322f6d640 1 --2- 172.21.15.139:0/4159398592 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc30c0a3460 0x7fc30c0a5920 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.619+0000 7fc322f6d640 1 --2- 172.21.15.139:0/4159398592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc31c107b80 0x7fc31c19cc40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.619+0000 7fc322f6d640 1 --2- 172.21.15.139:0/4159398592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c102480 0x7fc31c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.619+0000 7fc322f6d640 1 --2- 172.21.15.139:0/4159398592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c101ae0 0x7fc31c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:47:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.619+0000 7fc322f6d640 1 -- 172.21.15.139:0/4159398592 >> 172.21.15.139:0/4159398592 conn(0x7fc31c0fd7e0 msgr2=0x7fc31c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:47:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.619+0000 7fc322f6d640 1 -- 172.21.15.139:0/4159398592 shutdown_connections 2024-09-16T01:47:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:47:49.619+0000 7fc322f6d640 1 -- 172.21.15.139:0/4159398592 wait complete. 2024-09-16T01:47:49.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:49 smithi139 ceph-mon[31175]: pgmap v228: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-16T01:47:49.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:49 smithi176 ceph-mon[35324]: pgmap v228: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-16T01:47:50.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:49 smithi164 ceph-mon[35121]: pgmap v228: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-16T01:47:51.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:51 smithi139 ceph-mon[31175]: pgmap v229: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-09-16T01:47:51.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:51 smithi176 ceph-mon[35324]: pgmap v229: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-09-16T01:47:52.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:51 smithi164 ceph-mon[35121]: pgmap v229: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-09-16T01:47:53.615 INFO:tasks.workunit.client.0.smithi139.stdout: 2024-09-16T01:47:53.616 INFO:tasks.workunit.client.0.smithi139.stdout:Final score for writes: 510 2024-09-16T01:47:53.616 INFO:tasks.workunit.client.0.smithi139.stdout:Final score for reads : 530487 2024-09-16T01:47:53.616 INFO:tasks.workunit.client.0.smithi139.stdout: 2024-09-16T01:47:53.616 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T01:47:53.616 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T01:47:53.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:53 smithi139 ceph-mon[31175]: pgmap v230: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-09-16T01:47:53.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:53 smithi176 ceph-mon[35324]: pgmap v230: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-09-16T01:47:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:53 smithi164 ceph-mon[35121]: pgmap v230: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-09-16T01:47:55.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:55 smithi139 ceph-mon[31175]: pgmap v231: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 75 KiB/s wr, 19 op/s 2024-09-16T01:47:55.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:55 smithi176 ceph-mon[35324]: pgmap v231: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 75 KiB/s wr, 19 op/s 2024-09-16T01:47:56.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:55 smithi164 ceph-mon[35121]: pgmap v231: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 75 KiB/s wr, 19 op/s 2024-09-16T01:47:57.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:57 smithi139 ceph-mon[31175]: pgmap v232: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 23 op/s 2024-09-16T01:47:57.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:57 smithi176 ceph-mon[35324]: pgmap v232: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 23 op/s 2024-09-16T01:47:58.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:57 smithi164 ceph-mon[35121]: pgmap v232: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 23 op/s 2024-09-16T01:47:58.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:58 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:47:58.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:58 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:47:59.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:58 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:47:59.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:47:59 smithi139 ceph-mon[31175]: pgmap v233: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 43 op/s 2024-09-16T01:47:59.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:47:59 smithi176 ceph-mon[35324]: pgmap v233: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 43 op/s 2024-09-16T01:48:00.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:47:59 smithi164 ceph-mon[35121]: pgmap v233: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 43 op/s 2024-09-16T01:48:01.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:01 smithi139 ceph-mon[31175]: pgmap v234: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 964 KiB/s wr, 72 op/s 2024-09-16T01:48:01.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:01 smithi176 ceph-mon[35324]: pgmap v234: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 964 KiB/s wr, 72 op/s 2024-09-16T01:48:02.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:01 smithi164 ceph-mon[35121]: pgmap v234: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 964 KiB/s wr, 72 op/s 2024-09-16T01:48:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.758+0000 7fe6cff18640 1 -- 172.21.15.139:0/322111334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6c8077040 msgr2=0x7fe6c8112440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.758+0000 7fe6cff18640 1 --2- 172.21.15.139:0/322111334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6c8077040 0x7fe6c8112440 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009900 tx=0x7fe6b802a3f0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.759+0000 7fe6cff18640 1 -- 172.21.15.139:0/322111334 shutdown_connections 2024-09-16T01:48:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.759+0000 7fe6cff18640 1 --2- 172.21.15.139:0/322111334 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6c8112980 0x7fe6c8114d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.759+0000 7fe6cff18640 1 --2- 172.21.15.139:0/322111334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6c8077040 0x7fe6c8112440 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.759+0000 7fe6cff18640 1 --2- 172.21.15.139:0/322111334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6c8076720 0x7fe6c8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.759+0000 7fe6cff18640 1 -- 172.21.15.139:0/322111334 >> 172.21.15.139:0/322111334 conn(0x7fe6c80ff980 msgr2=0x7fe6c8101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.760+0000 7fe6cff18640 1 -- 172.21.15.139:0/322111334 shutdown_connections 2024-09-16T01:48:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.761+0000 7fe6cff18640 1 -- 172.21.15.139:0/322111334 wait complete. 2024-09-16T01:48:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.761+0000 7fe6cff18640 1 Processor -- start 2024-09-16T01:48:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.762+0000 7fe6cff18640 1 -- start start 2024-09-16T01:48:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.762+0000 7fe6cff18640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6c8076720 0x7fe6c8112070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.762+0000 7fe6cff18640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6c8077040 0x7fe6c810c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.762+0000 7fe6cff18640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6c8112980 0x7fe6c810c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.762+0000 7fe6cff18640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c8104640 con 0x7fe6c8076720 2024-09-16T01:48:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.762+0000 7fe6cff18640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81044c0 con 0x7fe6c8077040 2024-09-16T01:48:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.762+0000 7fe6cff18640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81047c0 con 0x7fe6c8112980 2024-09-16T01:48:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.762+0000 7fe6cdcb4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6c8076720 0x7fe6c8112070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.762+0000 7fe6cdcb4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6c8076720 0x7fe6c8112070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55516/0 (socket says 172.21.15.139:55516) 2024-09-16T01:48:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.762+0000 7fe6cdcb4640 1 -- 172.21.15.139:0/1827815850 learned_addr learned my addr 172.21.15.139:0/1827815850 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.762+0000 7fe6cd4b3640 1 --2- 172.21.15.139:0/1827815850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6c8077040 0x7fe6c810c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.762+0000 7fe6ce4b5640 1 --2- 172.21.15.139:0/1827815850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6c8112980 0x7fe6c810c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.763+0000 7fe6cdcb4640 1 -- 172.21.15.139:0/1827815850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6c8112980 msgr2=0x7fe6c810c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.763+0000 7fe6cdcb4640 1 --2- 172.21.15.139:0/1827815850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6c8112980 0x7fe6c810c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.763+0000 7fe6cdcb4640 1 -- 172.21.15.139:0/1827815850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6c8077040 msgr2=0x7fe6c810c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.763+0000 7fe6cdcb4640 1 --2- 172.21.15.139:0/1827815850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6c8077040 0x7fe6c810c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.763+0000 7fe6cdcb4640 1 -- 172.21.15.139:0/1827815850 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c810cd20 con 0x7fe6c8076720 2024-09-16T01:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.763+0000 7fe6cdcb4640 1 --2- 172.21.15.139:0/1827815850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6c8076720 0x7fe6c8112070 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000ba50 tx=0x7fe6c000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.764+0000 7fe6beffd640 1 -- 172.21.15.139:0/1827815850 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0019070 con 0x7fe6c8076720 2024-09-16T01:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.764+0000 7fe6beffd640 1 -- 172.21.15.139:0/1827815850 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c000cbe0 con 0x7fe6c8076720 2024-09-16T01:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.764+0000 7fe6beffd640 1 -- 172.21.15.139:0/1827815850 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0007730 con 0x7fe6c8076720 2024-09-16T01:48:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.764+0000 7fe6cff18640 1 -- 172.21.15.139:0/1827815850 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c806bc10 con 0x7fe6c8076720 2024-09-16T01:48:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.765+0000 7fe6cff18640 1 -- 172.21.15.139:0/1827815850 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c806c0e0 con 0x7fe6c8076720 2024-09-16T01:48:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.771+0000 7fe6d2cb2740 1 Processor -- start 2024-09-16T01:48:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.771+0000 7fe6d2cb2740 1 -- start start 2024-09-16T01:48:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.772+0000 7fe6beffd640 1 -- 172.21.15.139:0/1827815850 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe6c00078d0 con 0x7fe6c8076720 2024-09-16T01:48:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.773+0000 7fe6beffd640 1 --2- 172.21.15.139:0/1827815850 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe6b40a34e0 0x7fe6b40a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.773+0000 7fe6d2cb2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9c8af0 0x55d76c9c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.773+0000 7fe6d2cb2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9c9410 0x55d76c9d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.773+0000 7fe6beffd640 1 -- 172.21.15.139:0/1827815850 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9510+0+0 (secure 0 0 0) 0x7fe6c00cf3f0 con 0x7fe6c8076720 2024-09-16T01:48:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.773+0000 7fe6d2cb2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.773+0000 7fe6d2cb2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d76c92d2f0 con 0x55d76c9d3cb0 2024-09-16T01:48:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.773+0000 7fe6d2cb2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d76c8d5270 con 0x55d76c9c9410 2024-09-16T01:48:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.773+0000 7fe6d2cb2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d76c9292b0 con 0x55d76c9c8af0 2024-09-16T01:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.773+0000 7fe6ce4b5640 1 --2- 172.21.15.139:0/1827815850 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe6b40a34e0 0x7fe6b40a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.773+0000 7fe6ce4b5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.773+0000 7fe6ce4b5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55524/0 (socket says 172.21.15.139:55524) 2024-09-16T01:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.773+0000 7fe6ce4b5640 1 -- 172.21.15.139:0/1293153860 learned_addr learned my addr 172.21.15.139:0/1293153860 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.774+0000 7fe6ce4b5640 1 -- 172.21.15.139:0/1293153860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9c8af0 msgr2=0x55d76c9c8ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.774+0000 7fe6ce4b5640 1 --2- 172.21.15.139:0/1293153860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9c8af0 0x55d76c9c8ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.774+0000 7fe6ce4b5640 1 -- 172.21.15.139:0/1293153860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9c9410 msgr2=0x55d76c9d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.774+0000 7fe6ce4b5640 1 --2- 172.21.15.139:0/1293153860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9c9410 0x55d76c9d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.774+0000 7fe6ce4b5640 1 -- 172.21.15.139:0/1293153860 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d76c8d4340 con 0x55d76c9d3cb0 2024-09-16T01:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.775+0000 7fe6ce4b5640 1 --2- 172.21.15.139:0/1293153860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d60a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4011a20 tx=0x7fe6c4011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb28c447763c2056 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.775+0000 7fe6ce4b5640 1 --2- 172.21.15.139:0/1827815850 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe6b40a34e0 0x7fe6b40a59a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400f850 tx=0x7fe6c4018000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.775+0000 7fe6b3fff640 1 -- 172.21.15.139:0/1293153860 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4023070 con 0x55d76c9d3cb0 2024-09-16T01:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.775+0000 7fe6b3fff640 1 -- 172.21.15.139:0/1293153860 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c400ed60 con 0x55d76c9d3cb0 2024-09-16T01:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.775+0000 7fe6b3fff640 1 -- 172.21.15.139:0/1293153860 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c401fcd0 con 0x55d76c9d3cb0 2024-09-16T01:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.776+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/1293153860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d3cb0 msgr2=0x55d76c9d60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.776+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/1293153860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d60a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4011a20 tx=0x7fe6c4011ee0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.776+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/1293153860 shutdown_connections 2024-09-16T01:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.776+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/1293153860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d60a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.776+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/1293153860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9c9410 0x55d76c9d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.776+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/1293153860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9c8af0 0x55d76c9c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.776+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/1293153860 >> 172.21.15.139:0/1293153860 conn(0x55d76c9c5060 msgr2=0x55d76c9c5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.777+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/1293153860 shutdown_connections 2024-09-16T01:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.778+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/1293153860 wait complete. 2024-09-16T01:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.779+0000 7fe6d2cb2740 1 -- start start 2024-09-16T01:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.779+0000 7fe6d2cb2740 1 Processor -- start 2024-09-16T01:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.779+0000 7fe6d2cb2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.779+0000 7fe6d2cb2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d12d0 0x55d76c9d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.780+0000 7fe6d2cb2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9dde40 0x55d76c9e0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.780+0000 7fe6d2cb2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d76c9d6880 con 0x55d76c9d12d0 2024-09-16T01:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.780+0000 7fe6d2cb2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d76c8b6290 con 0x55d76c9d3cb0 2024-09-16T01:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.780+0000 7fe6d2cb2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d76c9d6a00 con 0x55d76c9dde40 2024-09-16T01:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.780+0000 7fe6d2cb2740 10 client.0 _ll_register_callbacks cb 0x55d76c8e7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.780+0000 7fe6cd4b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d12d0 0x55d76c9d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.780+0000 7fe6cd4b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d12d0 0x55d76c9d1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55532/0 (socket says 172.21.15.139:55532) 2024-09-16T01:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.780+0000 7fe6cd4b3640 1 -- 172.21.15.139:0/2598845421 learned_addr learned my addr 172.21.15.139:0/2598845421 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.781+0000 7fe6cdcb4640 1 --2- 172.21.15.139:0/2598845421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.781+0000 7fe6cd4b3640 1 -- 172.21.15.139:0/2598845421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9dde40 msgr2=0x55d76c9e0230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:48:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.781+0000 7fe6cd4b3640 1 --2- 172.21.15.139:0/2598845421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9dde40 0x55d76c9e0230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.781+0000 7fe6cd4b3640 1 -- 172.21.15.139:0/2598845421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9d3cb0 msgr2=0x55d76c9d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.781+0000 7fe6cd4b3640 1 --2- 172.21.15.139:0/2598845421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.781+0000 7fe6cd4b3640 1 -- 172.21.15.139:0/2598845421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d76c640210 con 0x55d76c9d12d0 2024-09-16T01:48:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.782+0000 7fe6cd4b3640 1 --2- 172.21.15.139:0/2598845421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d12d0 0x55d76c9d1750 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009230 tx=0x7fe6b80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929378915555b074 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.782+0000 7fe6b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:48:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.782+0000 7fe6b17fa640 1 -- 172.21.15.139:0/2598845421 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8040070 con 0x55d76c9d12d0 2024-09-16T01:48:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.782+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/2598845421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c400ed60 con 0x55d76c9d12d0 2024-09-16T01:48:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.783+0000 7fe6b17fa640 1 -- 172.21.15.139:0/2598845421 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9510+0+0 (secure 0 0 0) 0x7fe6b8005450 con 0x55d76c9d12d0 2024-09-16T01:48:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.783+0000 7fe6b17fa640 1 -- 172.21.15.139:0/2598845421 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe6b804c050 con 0x55d76c9d12d0 2024-09-16T01:48:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.783+0000 7fe6d2cb2740 10 client.16320 fetch_fsmap learned FSMap version 43 2024-09-16T01:48:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.783+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/2598845421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d76c70c620 con 0x55d76c9d12d0 2024-09-16T01:48:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.784+0000 7fe6b17fa640 1 -- 172.21.15.139:0/2598845421 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c400ed60 con 0x55d76c9d12d0 2024-09-16T01:48:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.784+0000 7fe6b17fa640 1 -- 172.21.15.139:0/2598845421 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8442+0+0 (secure 0 0 0) 0x7fe6b8003970 con 0x55d76c9d12d0 2024-09-16T01:48:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.784+0000 7fe6d2cb2740 10 client.16320 fetch_fsmap finished waiting for FSMap version 43 2024-09-16T01:48:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.784+0000 7fe6d2cb2740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:48:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.784+0000 7fe6d2cb2740 20 client.16320 populate_metadata read hostname 'smithi139' 2024-09-16T01:48:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.784+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/2598845421 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d76c9e4c40 0x55d76c9e7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.784+0000 7fe6d2cb2740 4 client.16320 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:48:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.784+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/2598845421 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d76c93aa20 con 0x55d76c9e4c40 2024-09-16T01:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.784+0000 7fe6cdcb4640 1 --2- 172.21.15.139:0/2598845421 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d76c9e4c40 0x55d76c9e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.785+0000 7fe6b17fa640 10 client.16320 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.785+0000 7fe6cdcb4640 1 --2- 172.21.15.139:0/2598845421 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d76c9e4c40 0x55d76c9e7080 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1edae3f6f1b431 server_cookie=676250ebb1d598dd in_seq=0 out_seq=0 2024-09-16T01:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.787+0000 7fe6b17fa640 1 -- 172.21.15.139:0/2598845421 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d76c93aa20 con 0x55d76c9e4c40 2024-09-16T01:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.787+0000 7fe6b17fa640 10 client.16320 handle_command_reply: tid=0 2024-09-16T01:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.787+0000 7fe6d2cb2740 1 client.16320 shutdown 2024-09-16T01:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.787+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/2598845421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d12d0 msgr2=0x55d76c9d1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.787+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/2598845421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d12d0 0x55d76c9d1750 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009230 tx=0x7fe6b80043b0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.788+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/2598845421 shutdown_connections 2024-09-16T01:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.788+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/2598845421 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d76c9e4c40 0x55d76c9e7080 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.788+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/2598845421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9dde40 0x55d76c9e0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.788+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/2598845421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d12d0 0x55d76c9d1750 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.788+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/2598845421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.788+0000 7fe6b17fa640 0 client.16320 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.788+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/2598845421 >> 172.21.15.139:0/2598845421 conn(0x55d76c9c5060 msgr2=0x55d76c9cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.788+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/2598845421 shutdown_connections 2024-09-16T01:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.788+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/2598845421 wait complete. 2024-09-16T01:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.788+0000 7fe6d2cb2740 20 client.16320 trim_cache size 0 max 16384 2024-09-16T01:48:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.808+0000 7fe6d2cb2740 1 Processor -- start 2024-09-16T01:48:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.808+0000 7fe6d2cb2740 1 -- start start 2024-09-16T01:48:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.809+0000 7fe6d2cb2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d12d0 0x55d76c9d2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.809+0000 7fe6d2cb2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9d3cb0 0x55d76c9ec990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.809+0000 7fe6d2cb2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9dde40 0x55d76c9eced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.809+0000 7fe6d2cb2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac0100d0 con 0x55d76c9d12d0 2024-09-16T01:48:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.809+0000 7fe6d2cb2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac011eb0 con 0x55d76c9dde40 2024-09-16T01:48:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.809+0000 7fe6d2cb2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac010860 con 0x55d76c9d3cb0 2024-09-16T01:48:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.809+0000 7fe6cd4b3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9d3cb0 0x55d76c9ec990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.809+0000 7fe6cd4b3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9d3cb0 0x55d76c9ec990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:46432/0 (socket says 172.21.15.139:46432) 2024-09-16T01:48:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.809+0000 7fe6cd4b3640 1 -- 172.21.15.139:0/292401028 learned_addr learned my addr 172.21.15.139:0/292401028 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.809+0000 7fe6cdcb4640 1 --2- 172.21.15.139:0/292401028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d12d0 0x55d76c9d2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.810+0000 7fe6ce4b5640 1 --2- 172.21.15.139:0/292401028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9dde40 0x55d76c9eced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.810+0000 7fe6cd4b3640 1 -- 172.21.15.139:0/292401028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9dde40 msgr2=0x55d76c9eced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.810+0000 7fe6cd4b3640 1 --2- 172.21.15.139:0/292401028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9dde40 0x55d76c9eced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.810+0000 7fe6cd4b3640 1 -- 172.21.15.139:0/292401028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d12d0 msgr2=0x55d76c9d2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.810+0000 7fe6cd4b3640 1 --2- 172.21.15.139:0/292401028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d12d0 0x55d76c9d2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.810+0000 7fe6cd4b3640 1 -- 172.21.15.139:0/292401028 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d76c9c4e90 con 0x55d76c9d3cb0 2024-09-16T01:48:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.810+0000 7fe6cdcb4640 1 --2- 172.21.15.139:0/292401028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d12d0 0x55d76c9d2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:48:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.810+0000 7fe6cd4b3640 1 --2- 172.21.15.139:0/292401028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9d3cb0 0x55d76c9ec990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8031550 tx=0x7fe6b8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=577b3237d4508c85 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.810+0000 7fe6b37fe640 1 -- 172.21.15.139:0/292401028 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8040070 con 0x55d76c9d3cb0 2024-09-16T01:48:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.810+0000 7fe6b37fe640 1 -- 172.21.15.139:0/292401028 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b8055070 con 0x55d76c9d3cb0 2024-09-16T01:48:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.810+0000 7fe6b37fe640 1 -- 172.21.15.139:0/292401028 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8035040 con 0x55d76c9d3cb0 2024-09-16T01:48:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.811+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/292401028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9d3cb0 msgr2=0x55d76c9ec990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.811+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/292401028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9d3cb0 0x55d76c9ec990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8031550 tx=0x7fe6b8009260 comp rx=0 tx=0).stop 2024-09-16T01:48:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.811+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/292401028 shutdown_connections 2024-09-16T01:48:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.811+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/292401028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9dde40 0x55d76c9eced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.811+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/292401028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9d3cb0 0x55d76c9ec990 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.811+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/292401028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d12d0 0x55d76c9d2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.811+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/292401028 >> 172.21.15.139:0/292401028 conn(0x55d76c9c5060 msgr2=0x55d76c9d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.811+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/292401028 shutdown_connections 2024-09-16T01:48:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.812+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/292401028 wait complete. 2024-09-16T01:48:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.812+0000 7fe6d2cb2740 1 -- start start 2024-09-16T01:48:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.812+0000 7fe6d2cb2740 1 Processor -- start 2024-09-16T01:48:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6d2cb2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9d12d0 0x55d76c9d9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6d2cb2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6d2cb2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9dde40 0x55d76ca4e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6d2cb2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac011720 con 0x55d76c9d3cb0 2024-09-16T01:48:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6d2cb2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac010ff0 con 0x55d76c9dde40 2024-09-16T01:48:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6d2cb2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac00e0d0 con 0x55d76c9d12d0 2024-09-16T01:48:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6cd4b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6d2cb2740 10 client.0 _ll_register_callbacks cb 0x7fe6b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:48:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6cd4b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55564/0 (socket says 172.21.15.139:55564) 2024-09-16T01:48:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6cd4b3640 1 -- 172.21.15.139:0/1800673348 learned_addr learned my addr 172.21.15.139:0/1800673348 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6cdcb4640 1 --2- 172.21.15.139:0/1800673348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9d12d0 0x55d76c9d9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6ce4b5640 1 --2- 172.21.15.139:0/1800673348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9dde40 0x55d76ca4e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6cd4b3640 1 -- 172.21.15.139:0/1800673348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9d12d0 msgr2=0x55d76c9d9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6cd4b3640 1 --2- 172.21.15.139:0/1800673348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9d12d0 0x55d76c9d9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6cd4b3640 1 -- 172.21.15.139:0/1800673348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9dde40 msgr2=0x55d76ca4e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6cd4b3640 1 --2- 172.21.15.139:0/1800673348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9dde40 0x55d76ca4e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.813+0000 7fe6cd4b3640 1 -- 172.21.15.139:0/1800673348 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d76c9e3280 con 0x55d76c9d3cb0 2024-09-16T01:48:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.814+0000 7fe6cd4b3640 1 --2- 172.21.15.139:0/1800673348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d9a90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80040e0 tx=0x7fe6b802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98843929bfb311e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.814+0000 7fe6b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:48:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.814+0000 7fe6b17fa640 1 -- 172.21.15.139:0/1800673348 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b805e060 con 0x55d76c9d3cb0 2024-09-16T01:48:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.814+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/1800673348 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6b8055070 con 0x55d76c9d3cb0 2024-09-16T01:48:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.814+0000 7fe6b17fa640 1 -- 172.21.15.139:0/1800673348 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9510+0+0 (secure 0 0 0) 0x7fe6b8003a00 con 0x55d76c9d3cb0 2024-09-16T01:48:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.814+0000 7fe6b17fa640 1 -- 172.21.15.139:0/1800673348 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe6b8065050 con 0x55d76c9d3cb0 2024-09-16T01:48:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.814+0000 7fe6d2cb2740 10 client.16326 fetch_fsmap learned FSMap version 43 2024-09-16T01:48:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.814+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/1800673348 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d76ca50060 con 0x55d76c9d3cb0 2024-09-16T01:48:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.815+0000 7fe6b17fa640 1 -- 172.21.15.139:0/1800673348 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b8055070 con 0x55d76c9d3cb0 2024-09-16T01:48:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.815+0000 7fe6b17fa640 1 -- 172.21.15.139:0/1800673348 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) ==== 8442+0+0 (secure 0 0 0) 0x7fe6b80315b0 con 0x55d76c9d3cb0 2024-09-16T01:48:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.815+0000 7fe6d2cb2740 10 client.16326 fetch_fsmap finished waiting for FSMap version 43 2024-09-16T01:48:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.815+0000 7fe6d2cb2740 10 client.16326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:48:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.815+0000 7fe6d2cb2740 20 client.16326 populate_metadata read hostname 'smithi139' 2024-09-16T01:48:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.815+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/1800673348 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d76ca507c0 0x55d76ca52be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.815+0000 7fe6d2cb2740 4 client.16326 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:48:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.815+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/1800673348 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d76c934d70 con 0x55d76ca507c0 2024-09-16T01:48:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.815+0000 7fe6cdcb4640 1 --2- 172.21.15.139:0/1800673348 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d76ca507c0 0x55d76ca52be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.816+0000 7fe6cdcb4640 1 --2- 172.21.15.139:0/1800673348 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d76ca507c0 0x55d76ca52be0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf231b763739373 server_cookie=d8f6989567600ac1 in_seq=0 out_seq=0 2024-09-16T01:48:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.816+0000 7fe6b17fa640 10 client.16326 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.897+0000 7fe6b17fa640 1 -- 172.21.15.139:0/1800673348 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d76c934d70 con 0x55d76ca507c0 2024-09-16T01:48:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.897+0000 7fe6b17fa640 10 client.16326 handle_command_reply: tid=0 2024-09-16T01:48:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.897+0000 7fe6d2cb2740 1 client.16326 shutdown 2024-09-16T01:48:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.897+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/1800673348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d3cb0 msgr2=0x55d76c9d9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.897+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/1800673348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d9a90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80040e0 tx=0x7fe6b802a3f0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.897+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/1800673348 shutdown_connections 2024-09-16T01:48:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.897+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/1800673348 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d76ca507c0 0x55d76ca52be0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.897+0000 7fe6b17fa640 0 client.16326 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.897+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/1800673348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d76c9dde40 0x55d76ca4e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.897+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/1800673348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d76c9d3cb0 0x55d76c9d9a90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.897+0000 7fe6d2cb2740 1 --2- 172.21.15.139:0/1800673348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d76c9d12d0 0x55d76c9d9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.897+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/1800673348 >> 172.21.15.139:0/1800673348 conn(0x55d76c9c5060 msgr2=0x55d76c9cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.897+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/1800673348 shutdown_connections 2024-09-16T01:48:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.898+0000 7fe6d2cb2740 1 -- 172.21.15.139:0/1800673348 wait complete. 2024-09-16T01:48:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.898+0000 7fe6d2cb2740 20 client.16326 trim_cache size 0 max 16384 2024-09-16T01:48:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:48:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:48:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:48:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:48:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.899+0000 7fe6cff18640 1 -- 172.21.15.139:0/1827815850 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe6b40a34e0 msgr2=0x7fe6b40a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.899+0000 7fe6cff18640 1 --2- 172.21.15.139:0/1827815850 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe6b40a34e0 0x7fe6b40a59a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400f850 tx=0x7fe6c4018000 comp rx=0 tx=0).stop 2024-09-16T01:48:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.899+0000 7fe6cff18640 1 -- 172.21.15.139:0/1827815850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6c8076720 msgr2=0x7fe6c8112070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.899+0000 7fe6cff18640 1 --2- 172.21.15.139:0/1827815850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6c8076720 0x7fe6c8112070 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000ba50 tx=0x7fe6c000bf10 comp rx=0 tx=0).stop 2024-09-16T01:48:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.899+0000 7fe6cff18640 1 -- 172.21.15.139:0/1827815850 shutdown_connections 2024-09-16T01:48:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.899+0000 7fe6cff18640 1 --2- 172.21.15.139:0/1827815850 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe6b40a34e0 0x7fe6b40a59a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.899+0000 7fe6cff18640 1 --2- 172.21.15.139:0/1827815850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6c8112980 0x7fe6c810c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.899+0000 7fe6cff18640 1 --2- 172.21.15.139:0/1827815850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6c8077040 0x7fe6c810c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.899+0000 7fe6cff18640 1 --2- 172.21.15.139:0/1827815850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6c8076720 0x7fe6c8112070 secure :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000ba50 tx=0x7fe6c000bf10 comp rx=0 tx=0).stop 2024-09-16T01:48:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.899+0000 7fe6cff18640 1 -- 172.21.15.139:0/1827815850 >> 172.21.15.139:0/1827815850 conn(0x7fe6c80ff980 msgr2=0x7fe6c8101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.899+0000 7fe6cff18640 1 -- 172.21.15.139:0/1827815850 shutdown_connections 2024-09-16T01:48:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:02.899+0000 7fe6cff18640 1 -- 172.21.15.139:0/1827815850 wait complete. 2024-09-16T01:48:02.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:02 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:02.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:02 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:03.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:02 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:03.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:03 smithi139 ceph-mon[31175]: pgmap v235: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-16T01:48:03.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:03 smithi176 ceph-mon[35324]: pgmap v235: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-16T01:48:04.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:03 smithi164 ceph-mon[35121]: pgmap v235: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-16T01:48:04.659 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.7 2024-09-16T01:48:04.660 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:48:05.035 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:48:05.721 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:05 smithi139 ceph-mon[31175]: pgmap v236: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-16T01:48:05.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.717+0000 7f777baea640 1 -- 172.21.15.139:0/3521519235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f777410d530 msgr2=0x7f77741150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:05.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.717+0000 7f777baea640 1 --2- 172.21.15.139:0/3521519235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f777410d530 0x7f77741150b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f776c009be0 tx=0x7f776c02a710 comp rx=0 tx=0).stop 2024-09-16T01:48:05.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.718+0000 7f777baea640 1 -- 172.21.15.139:0/3521519235 shutdown_connections 2024-09-16T01:48:05.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.718+0000 7f777baea640 1 --2- 172.21.15.139:0/3521519235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f777410d530 0x7f77741150b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:05.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.718+0000 7f777baea640 1 --2- 172.21.15.139:0/3521519235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7774108e00 0x7f777410ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:05.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.718+0000 7f777baea640 1 --2- 172.21.15.139:0/3521519235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7774108450 0x7f7774108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:05.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.718+0000 7f777baea640 1 -- 172.21.15.139:0/3521519235 >> 172.21.15.139:0/3521519235 conn(0x7f7774077ba0 msgr2=0x7f7774079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:05.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.718+0000 7f777baea640 1 -- 172.21.15.139:0/3521519235 shutdown_connections 2024-09-16T01:48:05.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.718+0000 7f777baea640 1 -- 172.21.15.139:0/3521519235 wait complete. 2024-09-16T01:48:05.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.719+0000 7f777baea640 1 Processor -- start 2024-09-16T01:48:05.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.719+0000 7f777baea640 1 -- start start 2024-09-16T01:48:05.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.719+0000 7f777baea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7774108450 0x7f7774082890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:05.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.719+0000 7f777baea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7774108e00 0x7f777407c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:05.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.719+0000 7f777baea640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f777410d530 0x7f777407ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:05.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.720+0000 7f777baea640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f777407b490 con 0x7f7774108450 2024-09-16T01:48:05.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.720+0000 7f777baea640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f777407b310 con 0x7f7774108e00 2024-09-16T01:48:05.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.720+0000 7f7779085640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7774108e00 0x7f777407c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:05.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.720+0000 7f777baea640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f777407b610 con 0x7f777410d530 2024-09-16T01:48:05.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.720+0000 7f777a087640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f777410d530 0x7f777407ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:05.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.720+0000 7f777a087640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f777410d530 0x7f777407ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:46456/0 (socket says 172.21.15.139:46456) 2024-09-16T01:48:05.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.720+0000 7f777a087640 1 -- 172.21.15.139:0/3866522277 learned_addr learned my addr 172.21.15.139:0/3866522277 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:05.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.720+0000 7f7779886640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7774108450 0x7f7774082890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:05.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.722+0000 7f777a087640 1 -- 172.21.15.139:0/3866522277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7774108e00 msgr2=0x7f777407c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:05.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.722+0000 7f777a087640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7774108e00 0x7f777407c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:05.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.722+0000 7f777a087640 1 -- 172.21.15.139:0/3866522277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7774108450 msgr2=0x7f7774082890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:05.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.722+0000 7f777a087640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7774108450 0x7f7774082890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:05.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.722+0000 7f777a087640 1 -- 172.21.15.139:0/3866522277 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777407d490 con 0x7f777410d530 2024-09-16T01:48:05.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.722+0000 7f777a087640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f777410d530 0x7f777407ce50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f776c034040 tx=0x7f776c003e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:05.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.723+0000 7f7779085640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7774108e00 0x7f777407c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:48:05.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.723+0000 7f7779886640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7774108450 0x7f7774082890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:48:05.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.723+0000 7f776affd640 1 -- 172.21.15.139:0/3866522277 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c040020 con 0x7f777410d530 2024-09-16T01:48:05.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.723+0000 7f776affd640 1 -- 172.21.15.139:0/3866522277 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f776c031920 con 0x7f777410d530 2024-09-16T01:48:05.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.723+0000 7f776affd640 1 -- 172.21.15.139:0/3866522277 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c036040 con 0x7f777410d530 2024-09-16T01:48:05.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.723+0000 7f777baea640 1 -- 172.21.15.139:0/3866522277 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777407d720 con 0x7f777410d530 2024-09-16T01:48:05.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.723+0000 7f777baea640 1 -- 172.21.15.139:0/3866522277 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7774071ed0 con 0x7f777410d530 2024-09-16T01:48:05.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.728+0000 7f776affd640 1 -- 172.21.15.139:0/3866522277 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f776c043020 con 0x7f777410d530 2024-09-16T01:48:05.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.729+0000 7f776affd640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f77600a35c0 0x7f77600a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:05.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.727+0000 7f777baea640 1 -- 172.21.15.139:0/3866522277 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77741098e0 con 0x7f777410d530 2024-09-16T01:48:05.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.729+0000 7f7779886640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f77600a35c0 0x7f77600a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:05.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.729+0000 7f776affd640 1 -- 172.21.15.139:0/3866522277 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9642+0+0 (secure 0 0 0) 0x7f776c0f0d50 con 0x7f777410d530 2024-09-16T01:48:05.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.729+0000 7f7779886640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f77600a35c0 0x7f77600a5a80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7770009b40 tx=0x7f7770005e50 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:05.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:05.739+0000 7f776affd640 1 -- 172.21.15.139:0/3866522277 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f776c04a020 con 0x7f777410d530 2024-09-16T01:48:05.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:05 smithi176 ceph-mon[35324]: pgmap v236: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-16T01:48:06.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:05 smithi164 ceph-mon[35121]: pgmap v236: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-16T01:48:06.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:06.143+0000 7f777baea640 1 -- 172.21.15.139:0/3866522277 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77740722b0 con 0x7f777410d530 2024-09-16T01:48:06.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:06.146+0000 7f776affd640 1 -- 172.21.15.139:0/3866522277 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+12026 (secure 0 0 0) 0x7f776c0bd770 con 0x7f777410d530 2024-09-16T01:48:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:48:06.149 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":43,"btime":"2024-09-16T01:48:01:593149+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:48:01.591871+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:48:06.149 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 43 2024-09-16T01:48:06.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:06.153+0000 7f777baea640 1 -- 172.21.15.139:0/3866522277 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f77600a35c0 msgr2=0x7f77600a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:06.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:06.153+0000 7f777baea640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f77600a35c0 0x7f77600a5a80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7770009b40 tx=0x7f7770005e50 comp rx=0 tx=0).stop 2024-09-16T01:48:06.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:06.153+0000 7f777baea640 1 -- 172.21.15.139:0/3866522277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f777410d530 msgr2=0x7f777407ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:06.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:06.153+0000 7f777baea640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f777410d530 0x7f777407ce50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f776c034040 tx=0x7f776c003e60 comp rx=0 tx=0).stop 2024-09-16T01:48:06.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:06.153+0000 7f777baea640 1 -- 172.21.15.139:0/3866522277 shutdown_connections 2024-09-16T01:48:06.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:06.153+0000 7f777baea640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f77600a35c0 0x7f77600a5a80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:06.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:06.153+0000 7f777baea640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f777410d530 0x7f777407ce50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:06.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:06.153+0000 7f777baea640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7774108e00 0x7f777407c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:06.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:06.153+0000 7f777baea640 1 --2- 172.21.15.139:0/3866522277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7774108450 0x7f7774082890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:06.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:06.154+0000 7f777baea640 1 -- 172.21.15.139:0/3866522277 >> 172.21.15.139:0/3866522277 conn(0x7f7774077ba0 msgr2=0x7f7774078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:06.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:06.154+0000 7f777baea640 1 -- 172.21.15.139:0/3866522277 shutdown_connections 2024-09-16T01:48:06.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:06.155+0000 7f777baea640 1 -- 172.21.15.139:0/3866522277 wait complete. 2024-09-16T01:48:06.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:06 smithi139 ceph-mon[31175]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T01:48:06.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:06 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3866522277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:48:06.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:06 smithi176 ceph-mon[35324]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T01:48:06.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:06 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3866522277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:48:07.009 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 143.7 --await-for 45.0' 2024-09-16T01:48:07.010 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 143.7 --await-for 45.0 2024-09-16T01:48:07.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:06 smithi164 ceph-mon[35121]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T01:48:07.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:06 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3866522277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:48:07.488 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:48:07.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:07 smithi139 ceph-mon[31175]: pgmap v238: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 375 op/s 2024-09-16T01:48:07.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:07 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:07.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:07 smithi176 ceph-mon[35324]: pgmap v238: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 375 op/s 2024-09-16T01:48:07.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:07 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:08.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:07 smithi164 ceph-mon[35121]: pgmap v238: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 375 op/s 2024-09-16T01:48:08.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:07 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:09.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.527+0000 7fcb6348a640 1 -- 172.21.15.139:0/1252209314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb5c101270 msgr2=0x7fcb5c10d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:09.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.527+0000 7fcb6348a640 1 --2- 172.21.15.139:0/1252209314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb5c101270 0x7fcb5c10d7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009ca0 tx=0x7fcb5402a8b0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.527+0000 7fcb6348a640 1 -- 172.21.15.139:0/1252209314 shutdown_connections 2024-09-16T01:48:09.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.527+0000 7fcb6348a640 1 --2- 172.21.15.139:0/1252209314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb5c101270 0x7fcb5c10d7a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.527+0000 7fcb6348a640 1 --2- 172.21.15.139:0/1252209314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb5c1008d0 0x7fcb5c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.527+0000 7fcb6348a640 1 --2- 172.21.15.139:0/1252209314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb5c1068d0 0x7fcb5c106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.527+0000 7fcb6348a640 1 -- 172.21.15.139:0/1252209314 >> 172.21.15.139:0/1252209314 conn(0x7fcb5c0fc610 msgr2=0x7fcb5c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:09.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.529+0000 7fcb6348a640 1 -- 172.21.15.139:0/1252209314 shutdown_connections 2024-09-16T01:48:09.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.529+0000 7fcb6348a640 1 -- 172.21.15.139:0/1252209314 wait complete. 2024-09-16T01:48:09.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.529+0000 7fcb6348a640 1 Processor -- start 2024-09-16T01:48:09.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.530+0000 7fcb6348a640 1 -- start start 2024-09-16T01:48:09.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.530+0000 7fcb6348a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb5c1008d0 0x7fcb5c19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.530+0000 7fcb6348a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb5c101270 0x7fcb5c1961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.530+0000 7fcb6348a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb5c1068d0 0x7fcb5c196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.530+0000 7fcb6348a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c110cf0 con 0x7fcb5c1068d0 2024-09-16T01:48:09.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.530+0000 7fcb6348a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c110b70 con 0x7fcb5c1008d0 2024-09-16T01:48:09.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.530+0000 7fcb6348a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c110e70 con 0x7fcb5c101270 2024-09-16T01:48:09.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.530+0000 7fcb62c89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb5c1068d0 0x7fcb5c196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.530+0000 7fcb61c87640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb5c101270 0x7fcb5c1961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.530+0000 7fcb62c89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb5c1068d0 0x7fcb5c196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55604/0 (socket says 172.21.15.139:55604) 2024-09-16T01:48:09.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.530+0000 7fcb62c89640 1 -- 172.21.15.139:0/3970553362 learned_addr learned my addr 172.21.15.139:0/3970553362 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:09.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.530+0000 7fcb62488640 1 --2- 172.21.15.139:0/3970553362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb5c1008d0 0x7fcb5c19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.531+0000 7fcb61c87640 1 -- 172.21.15.139:0/3970553362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb5c1008d0 msgr2=0x7fcb5c19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:09.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.531+0000 7fcb61c87640 1 --2- 172.21.15.139:0/3970553362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb5c1008d0 0x7fcb5c19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.531+0000 7fcb61c87640 1 -- 172.21.15.139:0/3970553362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb5c1068d0 msgr2=0x7fcb5c196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:09.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.531+0000 7fcb61c87640 1 --2- 172.21.15.139:0/3970553362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb5c1068d0 0x7fcb5c196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.531+0000 7fcb61c87640 1 -- 172.21.15.139:0/3970553362 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c196c40 con 0x7fcb5c101270 2024-09-16T01:48:09.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.531+0000 7fcb61c87640 1 --2- 172.21.15.139:0/3970553362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb5c101270 0x7fcb5c1961c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800b3f0 tx=0x7fcb5800b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:09.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.533+0000 7fcb537fe640 1 -- 172.21.15.139:0/3970553362 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58015020 con 0x7fcb5c101270 2024-09-16T01:48:09.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.533+0000 7fcb6348a640 1 -- 172.21.15.139:0/3970553362 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5c196f30 con 0x7fcb5c101270 2024-09-16T01:48:09.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.533+0000 7fcb6348a640 1 -- 172.21.15.139:0/3970553362 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5c19eb40 con 0x7fcb5c101270 2024-09-16T01:48:09.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.533+0000 7fcb537fe640 1 -- 172.21.15.139:0/3970553362 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb58011d10 con 0x7fcb5c101270 2024-09-16T01:48:09.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.535+0000 7fcb537fe640 1 -- 172.21.15.139:0/3970553362 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58010e40 con 0x7fcb5c101270 2024-09-16T01:48:09.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.539+0000 7fcb66ac5740 1 Processor -- start 2024-09-16T01:48:09.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.540+0000 7fcb66ac5740 1 -- start start 2024-09-16T01:48:09.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.540+0000 7fcb66ac5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.540+0000 7fcb66ac5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e1220 0x55c4cb9eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.540+0000 7fcb66ac5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9eded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.540+0000 7fcb66ac5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c4cb9462a0 con 0x55c4cb9ebae0 2024-09-16T01:48:09.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.540+0000 7fcb66ac5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c4cb8edb10 con 0x55c4cb9e0900 2024-09-16T01:48:09.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.540+0000 7fcb66ac5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c4cb942260 con 0x55c4cb9e1220 2024-09-16T01:48:09.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.541+0000 7fcb62488640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e1220 0x55c4cb9eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.541+0000 7fcb62c89640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.541+0000 7fcb62c89640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:36664/0 (socket says 172.21.15.139:36664) 2024-09-16T01:48:09.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.541+0000 7fcb62c89640 1 -- 172.21.15.139:0/3197662839 learned_addr learned my addr 172.21.15.139:0/3197662839 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:09.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.541+0000 7fcb61c87640 1 --2- 172.21.15.139:0/3197662839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9eded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.542+0000 7fcb537fe640 1 -- 172.21.15.139:0/3970553362 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fcb58018020 con 0x7fcb5c101270 2024-09-16T01:48:09.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.542+0000 7fcb61c87640 1 -- 172.21.15.139:0/3197662839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e1220 msgr2=0x55c4cb9eb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:09.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.542+0000 7fcb61c87640 1 --2- 172.21.15.139:0/3197662839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e1220 0x55c4cb9eb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.542+0000 7fcb61c87640 1 -- 172.21.15.139:0/3197662839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9e0900 msgr2=0x55c4cb9e0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:09.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.542+0000 7fcb61c87640 1 --2- 172.21.15.139:0/3197662839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.542+0000 7fcb61c87640 1 -- 172.21.15.139:0/3197662839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4cb6b7b90 con 0x55c4cb9ebae0 2024-09-16T01:48:09.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.545+0000 7fcb61c87640 1 --2- 172.21.15.139:0/3197662839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9eded0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcb580cc9a0 tx=0x7fcb580d72a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e376b5e77f69c6e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:09.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.546+0000 7fcb537fe640 1 --2- 172.21.15.139:0/3970553362 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcb480a35c0 0x7fcb480a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.546+0000 7fcb537fe640 1 -- 172.21.15.139:0/3970553362 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9642+0+0 (secure 0 0 0) 0x7fcb580cf600 con 0x7fcb5c101270 2024-09-16T01:48:09.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.548+0000 7fcb62c89640 1 --2- 172.21.15.139:0/3970553362 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcb480a35c0 0x7fcb480a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.549+0000 7fcb62c89640 1 --2- 172.21.15.139:0/3970553362 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcb480a35c0 0x7fcb480a5a80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcb54030840 tx=0x7fcb54005fa0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:09.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.555+0000 7fcb50ff9640 1 -- 172.21.15.139:0/3197662839 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb580de070 con 0x55c4cb9ebae0 2024-09-16T01:48:09.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.555+0000 7fcb50ff9640 1 -- 172.21.15.139:0/3197662839 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb5809bc30 con 0x55c4cb9ebae0 2024-09-16T01:48:09.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.556+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3197662839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9ebae0 msgr2=0x55c4cb9eded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:09.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.556+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3197662839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9eded0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcb580cc9a0 tx=0x7fcb580d72a0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.557+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3197662839 shutdown_connections 2024-09-16T01:48:09.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.557+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3197662839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9eded0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.557+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3197662839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e1220 0x55c4cb9eb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.557+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3197662839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.557+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3197662839 >> 172.21.15.139:0/3197662839 conn(0x55c4cb9dce90 msgr2=0x55c4cb9dd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:09.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.565+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3197662839 shutdown_connections 2024-09-16T01:48:09.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.565+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3197662839 wait complete. 2024-09-16T01:48:09.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.565+0000 7fcb66ac5740 1 -- start start 2024-09-16T01:48:09.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.565+0000 7fcb66ac5740 1 Processor -- start 2024-09-16T01:48:09.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.566+0000 7fcb66ac5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.566+0000 7fcb66ac5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e1220 0x55c4cb9e8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.566+0000 7fcb66ac5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9e9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.566+0000 7fcb66ac5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c4cb9ee700 con 0x55c4cb9ebae0 2024-09-16T01:48:09.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.566+0000 7fcb66ac5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c4cb8ce6e0 con 0x55c4cb9e0900 2024-09-16T01:48:09.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.566+0000 7fcb66ac5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c4cb9ee880 con 0x55c4cb9e1220 2024-09-16T01:48:09.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.566+0000 7fcb66ac5740 10 client.0 _ll_register_callbacks cb 0x55c4cb8f2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:48:09.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.566+0000 7fcb62488640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.566+0000 7fcb62488640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:36676/0 (socket says 172.21.15.139:36676) 2024-09-16T01:48:09.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.566+0000 7fcb62488640 1 -- 172.21.15.139:0/3150065803 learned_addr learned my addr 172.21.15.139:0/3150065803 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:09.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.566+0000 7fcb61c87640 1 --2- 172.21.15.139:0/3150065803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e1220 0x55c4cb9e8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.567+0000 7fcb62c89640 1 --2- 172.21.15.139:0/3150065803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.567+0000 7fcb62488640 1 -- 172.21.15.139:0/3150065803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e1220 msgr2=0x55c4cb9e8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.567+0000 7fcb62488640 1 --2- 172.21.15.139:0/3150065803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e1220 0x55c4cb9e8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.567+0000 7fcb62488640 1 -- 172.21.15.139:0/3150065803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9ebae0 msgr2=0x55c4cb9e9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.567+0000 7fcb62488640 1 --2- 172.21.15.139:0/3150065803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9e9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.567+0000 7fcb62488640 1 -- 172.21.15.139:0/3150065803 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4cba08c60 con 0x55c4cb9e0900 2024-09-16T01:48:09.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.569+0000 7fcb62488640 1 --2- 172.21.15.139:0/3150065803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e88d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c012770 tx=0x7fcb4c012c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86e247c0c822950b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:09.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.569+0000 7fcb45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:48:09.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.569+0000 7fcb45ffb640 1 -- 172.21.15.139:0/3150065803 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c010d60 con 0x55c4cb9e0900 2024-09-16T01:48:09.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.570+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3150065803 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb5809bc30 con 0x55c4cb9e0900 2024-09-16T01:48:09.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.570+0000 7fcb45ffb640 1 -- 172.21.15.139:0/3150065803 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9642+0+0 (secure 0 0 0) 0x7fcb4c005020 con 0x55c4cb9e0900 2024-09-16T01:48:09.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.570+0000 7fcb45ffb640 1 -- 172.21.15.139:0/3150065803 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcb4c025050 con 0x55c4cb9e0900 2024-09-16T01:48:09.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.570+0000 7fcb66ac5740 10 client.25808 fetch_fsmap learned FSMap version 44 2024-09-16T01:48:09.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.570+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3150065803 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c4cba0b620 con 0x55c4cb9e0900 2024-09-16T01:48:09.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.570+0000 7fcb45ffb640 1 -- 172.21.15.139:0/3150065803 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb5809bc30 con 0x55c4cb9e0900 2024-09-16T01:48:09.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.574+0000 7fcb45ffb640 1 -- 172.21.15.139:0/3150065803 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 44) ==== 8446+0+0 (secure 0 0 0) 0x7fcb4c005220 con 0x55c4cb9e0900 2024-09-16T01:48:09.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.574+0000 7fcb66ac5740 10 client.25808 fetch_fsmap finished waiting for FSMap version 44 2024-09-16T01:48:09.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.574+0000 7fcb66ac5740 10 client.25808 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:48:09.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.574+0000 7fcb66ac5740 20 client.25808 populate_metadata read hostname 'smithi139' 2024-09-16T01:48:09.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.574+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3150065803 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c4cba0cf50 0x55c4cba0f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.574+0000 7fcb66ac5740 4 client.25808 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:48:09.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.574+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3150065803 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4cb9539d0 con 0x55c4cba0cf50 2024-09-16T01:48:09.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.575+0000 7fcb61c87640 1 --2- 172.21.15.139:0/3150065803 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c4cba0cf50 0x55c4cba0f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.578+0000 7fcb61c87640 1 --2- 172.21.15.139:0/3150065803 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c4cba0cf50 0x55c4cba0f3f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad06194922b668c5 server_cookie=e779b674318fd91a in_seq=0 out_seq=0 2024-09-16T01:48:09.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.578+0000 7fcb45ffb640 10 client.25808 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:09.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.581+0000 7fcb45ffb640 1 -- 172.21.15.139:0/3150065803 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4cb9539d0 con 0x55c4cba0cf50 2024-09-16T01:48:09.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.581+0000 7fcb45ffb640 10 client.25808 handle_command_reply: tid=0 2024-09-16T01:48:09.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.581+0000 7fcb66ac5740 1 client.25808 shutdown 2024-09-16T01:48:09.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.585+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3150065803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9e0900 msgr2=0x55c4cb9e88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:09.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.585+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3150065803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e88d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c012770 tx=0x7fcb4c012c30 comp rx=0 tx=0).stop 2024-09-16T01:48:09.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.585+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3150065803 shutdown_connections 2024-09-16T01:48:09.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.585+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3150065803 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c4cba0cf50 0x55c4cba0f3f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.585+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3150065803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.585+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3150065803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e1220 0x55c4cb9e8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.585+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3150065803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e88d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.585+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3150065803 >> 172.21.15.139:0/3150065803 conn(0x55c4cb9dce90 msgr2=0x55c4cb9dc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:09.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.585+0000 7fcb45ffb640 0 client.25808 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:09.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.586+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3150065803 shutdown_connections 2024-09-16T01:48:09.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.586+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3150065803 wait complete. 2024-09-16T01:48:09.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.588+0000 7fcb66ac5740 20 client.25808 trim_cache size 0 max 16384 2024-09-16T01:48:09.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.622+0000 7fcb66ac5740 1 Processor -- start 2024-09-16T01:48:09.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.622+0000 7fcb66ac5740 1 -- start start 2024-09-16T01:48:09.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.622+0000 7fcb66ac5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.622+0000 7fcb66ac5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e1220 0x55c4cba14cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.622+0000 7fcb66ac5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9ebae0 0x55c4cba15200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.622+0000 7fcb66ac5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcb400101c0 con 0x55c4cb9e0900 2024-09-16T01:48:09.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.622+0000 7fcb66ac5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb40011fd0 con 0x55c4cb9ebae0 2024-09-16T01:48:09.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.622+0000 7fcb66ac5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb40010950 con 0x55c4cb9e1220 2024-09-16T01:48:09.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.622+0000 7fcb62488640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.622+0000 7fcb62c89640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9ebae0 0x55c4cba15200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.622+0000 7fcb62488640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55638/0 (socket says 172.21.15.139:55638) 2024-09-16T01:48:09.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.622+0000 7fcb62488640 1 -- 172.21.15.139:0/1550770555 learned_addr learned my addr 172.21.15.139:0/1550770555 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:09.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.623+0000 7fcb62488640 1 -- 172.21.15.139:0/1550770555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e1220 msgr2=0x55c4cba14cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:48:09.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.623+0000 7fcb62488640 1 --2- 172.21.15.139:0/1550770555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e1220 0x55c4cba14cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.623+0000 7fcb62488640 1 -- 172.21.15.139:0/1550770555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9ebae0 msgr2=0x55c4cba15200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:09.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.623+0000 7fcb62488640 1 --2- 172.21.15.139:0/1550770555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9ebae0 0x55c4cba15200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.623+0000 7fcb62488640 1 -- 172.21.15.139:0/1550770555 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4cb9db480 con 0x55c4cb9e0900 2024-09-16T01:48:09.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.623+0000 7fcb62c89640 1 --2- 172.21.15.139:0/1550770555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9ebae0 0x55c4cba15200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:48:09.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.623+0000 7fcb62488640 1 --2- 172.21.15.139:0/1550770555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e81b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c005840 tx=0x7fcb4c013e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e05b1f7f7b8b051 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:09.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.624+0000 7fcb47fff640 1 -- 172.21.15.139:0/1550770555 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c010de0 con 0x55c4cb9e0900 2024-09-16T01:48:09.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.624+0000 7fcb47fff640 1 -- 172.21.15.139:0/1550770555 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb4c0035f0 con 0x55c4cb9e0900 2024-09-16T01:48:09.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.624+0000 7fcb47fff640 1 -- 172.21.15.139:0/1550770555 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c01a670 con 0x55c4cb9e0900 2024-09-16T01:48:09.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.625+0000 7fcb66ac5740 1 -- 172.21.15.139:0/1550770555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9e0900 msgr2=0x55c4cb9e81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:09.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.625+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/1550770555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e81b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c005840 tx=0x7fcb4c013e90 comp rx=0 tx=0).stop 2024-09-16T01:48:09.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.626+0000 7fcb66ac5740 1 -- 172.21.15.139:0/1550770555 shutdown_connections 2024-09-16T01:48:09.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.626+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/1550770555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9ebae0 0x55c4cba15200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.626+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/1550770555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e1220 0x55c4cba14cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.626+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/1550770555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9e0900 0x55c4cb9e81b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.626+0000 7fcb66ac5740 1 -- 172.21.15.139:0/1550770555 >> 172.21.15.139:0/1550770555 conn(0x55c4cb9dce90 msgr2=0x55c4cb9ea130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:09.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.631+0000 7fcb66ac5740 1 -- 172.21.15.139:0/1550770555 shutdown_connections 2024-09-16T01:48:09.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.633+0000 7fcb66ac5740 1 -- 172.21.15.139:0/1550770555 wait complete. 2024-09-16T01:48:09.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.633+0000 7fcb66ac5740 1 -- start start 2024-09-16T01:48:09.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.633+0000 7fcb66ac5740 1 Processor -- start 2024-09-16T01:48:09.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.634+0000 7fcb66ac5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9e0900 0x55c4cb9f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.634+0000 7fcb66ac5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9e5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.634+0000 7fcb66ac5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e5be0 0x55c4cb9e6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.634+0000 7fcb66ac5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcb40011840 con 0x55c4cb9e0900 2024-09-16T01:48:09.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.634+0000 7fcb66ac5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb400110e0 con 0x55c4cb9ebae0 2024-09-16T01:48:09.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.634+0000 7fcb66ac5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4000e1c0 con 0x55c4cb9e5be0 2024-09-16T01:48:09.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.634+0000 7fcb66ac5740 10 client.0 _ll_register_callbacks cb 0x7fcb4c00aed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:48:09.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.637+0000 7fcb61c87640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.638+0000 7fcb61c87640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9e5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:36704/0 (socket says 172.21.15.139:36704) 2024-09-16T01:48:09.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.638+0000 7fcb61c87640 1 -- 172.21.15.139:0/3376076753 learned_addr learned my addr 172.21.15.139:0/3376076753 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:09.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.638+0000 7fcb62488640 1 --2- 172.21.15.139:0/3376076753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9e0900 0x55c4cb9f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.640+0000 7fcb61c87640 1 -- 172.21.15.139:0/3376076753 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e5be0 msgr2=0x55c4cb9e6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:09.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.640+0000 7fcb61c87640 1 --2- 172.21.15.139:0/3376076753 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e5be0 0x55c4cb9e6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.640+0000 7fcb61c87640 1 -- 172.21.15.139:0/3376076753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9e0900 msgr2=0x55c4cb9f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:09.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.640+0000 7fcb61c87640 1 --2- 172.21.15.139:0/3376076753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9e0900 0x55c4cb9f43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:09.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.640+0000 7fcb61c87640 1 -- 172.21.15.139:0/3376076753 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4cba76e60 con 0x55c4cb9ebae0 2024-09-16T01:48:09.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.641+0000 7fcb45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:48:09.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.642+0000 7fcb61c87640 1 --2- 172.21.15.139:0/3376076753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9e5650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcb58097f00 tx=0x7fcb580047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87ffefb2c7a5f824 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:09.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.642+0000 7fcb45ffb640 1 -- 172.21.15.139:0/3376076753 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb580de040 con 0x55c4cb9ebae0 2024-09-16T01:48:09.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.642+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3376076753 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb4c0035f0 con 0x55c4cb9ebae0 2024-09-16T01:48:09.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.643+0000 7fcb45ffb640 1 -- 172.21.15.139:0/3376076753 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9642+0+0 (secure 0 0 0) 0x7fcb580d5650 con 0x55c4cb9ebae0 2024-09-16T01:48:09.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.643+0000 7fcb45ffb640 1 -- 172.21.15.139:0/3376076753 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcb580de040 con 0x55c4cb9ebae0 2024-09-16T01:48:09.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.643+0000 7fcb66ac5740 10 client.25814 fetch_fsmap learned FSMap version 44 2024-09-16T01:48:09.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.643+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3376076753 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c4cba78140 con 0x55c4cb9ebae0 2024-09-16T01:48:09.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.644+0000 7fcb45ffb640 1 -- 172.21.15.139:0/3376076753 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb4c0035f0 con 0x55c4cb9ebae0 2024-09-16T01:48:09.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.644+0000 7fcb45ffb640 1 -- 172.21.15.139:0/3376076753 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 44) ==== 8446+0+0 (secure 0 0 0) 0x7fcb58097ba0 con 0x55c4cb9ebae0 2024-09-16T01:48:09.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.644+0000 7fcb66ac5740 10 client.25814 fetch_fsmap finished waiting for FSMap version 44 2024-09-16T01:48:09.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.644+0000 7fcb66ac5740 10 client.25814 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:48:09.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.644+0000 7fcb66ac5740 20 client.25814 populate_metadata read hostname 'smithi139' 2024-09-16T01:48:09.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.644+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3376076753 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c4cba784d0 0x55c4cba7a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:09.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.644+0000 7fcb66ac5740 4 client.25814 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.7, "await_for": 45.0}] 2024-09-16T01:48:09.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.644+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3376076753 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.7, "await_for": 45.0}) -- 0x55c4cb94dd20 con 0x55c4cba784d0 2024-09-16T01:48:09.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.645+0000 7fcb62488640 1 --2- 172.21.15.139:0/3376076753 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c4cba784d0 0x55c4cba7a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:09.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.647+0000 7fcb62488640 1 --2- 172.21.15.139:0/3376076753 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c4cba784d0 0x55c4cba7a8f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f35bdba25b8db1 server_cookie=bbd3959190b5c00f in_seq=0 out_seq=0 2024-09-16T01:48:09.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:09.647+0000 7fcb45ffb640 10 client.25814 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:09 smithi139 ceph-mon[31175]: pgmap v239: 129 pgs: 10 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 82 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 400 op/s 2024-09-16T01:48:09.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:09 smithi176 ceph-mon[35324]: pgmap v239: 129 pgs: 10 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 82 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 400 op/s 2024-09-16T01:48:10.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:09 smithi164 ceph-mon[35121]: pgmap v239: 129 pgs: 10 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 82 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 400 op/s 2024-09-16T01:48:11.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:11 smithi139 ceph-mon[31175]: pgmap v240: 129 pgs: 13 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 76 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 379 op/s 2024-09-16T01:48:11.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:11 smithi176 ceph-mon[35324]: pgmap v240: 129 pgs: 13 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 76 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 379 op/s 2024-09-16T01:48:12.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:11 smithi164 ceph-mon[35121]: pgmap v240: 129 pgs: 13 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 76 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 379 op/s 2024-09-16T01:48:13.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:13 smithi139 ceph-mon[31175]: pgmap v241: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 351 op/s 2024-09-16T01:48:13.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:13 smithi176 ceph-mon[35324]: pgmap v241: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 351 op/s 2024-09-16T01:48:14.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:13 smithi164 ceph-mon[35121]: pgmap v241: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 351 op/s 2024-09-16T01:48:15.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:15 smithi139 ceph-mon[31175]: pgmap v242: 129 pgs: 18 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 65 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 300 op/s 2024-09-16T01:48:15.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:15 smithi176 ceph-mon[35324]: pgmap v242: 129 pgs: 18 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 65 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 300 op/s 2024-09-16T01:48:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.039+0000 7fbda0766640 1 -- 172.21.15.139:0/86672344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd9811b140 msgr2=0x7fbd9811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.039+0000 7fbda0766640 1 --2- 172.21.15.139:0/86672344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd9811b140 0x7fbd9811d530 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009c10 tx=0x7fbd8c02a720 comp rx=0 tx=0).stop 2024-09-16T01:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.039+0000 7fbda0766640 1 -- 172.21.15.139:0/86672344 shutdown_connections 2024-09-16T01:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.039+0000 7fbda0766640 1 --2- 172.21.15.139:0/86672344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd9811b140 0x7fbd9811d530 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.039+0000 7fbda0766640 1 --2- 172.21.15.139:0/86672344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd981095d0 0x7fbd98109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.039+0000 7fbda0766640 1 --2- 172.21.15.139:0/86672344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd981091f0 0x7fbd98110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.039+0000 7fbda0766640 1 -- 172.21.15.139:0/86672344 >> 172.21.15.139:0/86672344 conn(0x7fbd9806ce00 msgr2=0x7fbd9806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.040+0000 7fbda0766640 1 -- 172.21.15.139:0/86672344 shutdown_connections 2024-09-16T01:48:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.040+0000 7fbda0766640 1 -- 172.21.15.139:0/86672344 wait complete. 2024-09-16T01:48:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.041+0000 7fbda0766640 1 Processor -- start 2024-09-16T01:48:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.041+0000 7fbda0766640 1 -- start start 2024-09-16T01:48:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.041+0000 7fbda0766640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd981091f0 0x7fbd9811a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.041+0000 7fbda0766640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd981095d0 0x7fbd98116880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.041+0000 7fbda0766640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd9811b140 0x7fbd98116dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.041+0000 7fbda0766640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd98113310 con 0x7fbd981095d0 2024-09-16T01:48:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.041+0000 7fbda0766640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbd98113190 con 0x7fbd9811b140 2024-09-16T01:48:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.041+0000 7fbda0766640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbd98113490 con 0x7fbd981091f0 2024-09-16T01:48:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.041+0000 7fbd9ed03640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd9811b140 0x7fbd98116dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.041+0000 7fbd9ed03640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd9811b140 0x7fbd98116dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:35780/0 (socket says 172.21.15.139:35780) 2024-09-16T01:48:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.041+0000 7fbd9ed03640 1 -- 172.21.15.139:0/1793243874 learned_addr learned my addr 172.21.15.139:0/1793243874 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.041+0000 7fbd9e502640 1 --2- 172.21.15.139:0/1793243874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd981091f0 0x7fbd9811a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.042+0000 7fbd9ed03640 1 -- 172.21.15.139:0/1793243874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd981091f0 msgr2=0x7fbd9811a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.042+0000 7fbd9ed03640 1 --2- 172.21.15.139:0/1793243874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd981091f0 0x7fbd9811a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.042+0000 7fbd9ed03640 1 -- 172.21.15.139:0/1793243874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd981095d0 msgr2=0x7fbd98116880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.042+0000 7fbd9ed03640 1 --2- 172.21.15.139:0/1793243874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd981095d0 0x7fbd98116880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.042+0000 7fbd9ed03640 1 -- 172.21.15.139:0/1793243874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd981174a0 con 0x7fbd9811b140 2024-09-16T01:48:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.042+0000 7fbd9ed03640 1 --2- 172.21.15.139:0/1793243874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd9811b140 0x7fbd98116dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009b30 tx=0x7fbd8c003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.043+0000 7fbd977fe640 1 -- 172.21.15.139:0/1793243874 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c041020 con 0x7fbd9811b140 2024-09-16T01:48:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.043+0000 7fbd977fe640 1 -- 172.21.15.139:0/1793243874 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd8c030880 con 0x7fbd9811b140 2024-09-16T01:48:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.043+0000 7fbda0766640 1 -- 172.21.15.139:0/1793243874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd981ab070 con 0x7fbd9811b140 2024-09-16T01:48:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.043+0000 7fbda0766640 1 -- 172.21.15.139:0/1793243874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd981ab420 con 0x7fbd9811b140 2024-09-16T01:48:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.043+0000 7fbd977fe640 1 -- 172.21.15.139:0/1793243874 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c036040 con 0x7fbd9811b140 2024-09-16T01:48:16.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:15 smithi164 ceph-mon[35121]: pgmap v242: 129 pgs: 18 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 65 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 300 op/s 2024-09-16T01:48:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.047+0000 7fbda35dc740 1 Processor -- start 2024-09-16T01:48:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.047+0000 7fbd977fe640 1 -- 172.21.15.139:0/1793243874 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fbd8c044020 con 0x7fbd9811b140 2024-09-16T01:48:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.048+0000 7fbda35dc740 1 -- start start 2024-09-16T01:48:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.048+0000 7fbda35dc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d73eaf0 0x55750d73eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.048+0000 7fbda35dc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d749770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.048+0000 7fbda35dc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d749cb0 0x55750d74c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.048+0000 7fbda35dc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55750d6a32f0 con 0x55750d73f410 2024-09-16T01:48:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.048+0000 7fbda35dc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55750d64b270 con 0x55750d73eaf0 2024-09-16T01:48:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.048+0000 7fbda35dc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55750d69f2b0 con 0x55750d749cb0 2024-09-16T01:48:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.048+0000 7fbd9e502640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d749770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.048+0000 7fbd9dd01640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d749cb0 0x55750d74c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.048+0000 7fbd9e502640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d749770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48906/0 (socket says 172.21.15.139:48906) 2024-09-16T01:48:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.048+0000 7fbd9dd01640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d749cb0 0x55750d74c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:47568/0 (socket says 172.21.15.139:47568) 2024-09-16T01:48:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.048+0000 7fbd9e502640 1 -- 172.21.15.139:0/1225570998 learned_addr learned my addr 172.21.15.139:0/1225570998 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.048+0000 7fbd977fe640 1 --2- 172.21.15.139:0/1793243874 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbd880a34b0 0x7fbd880a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.049+0000 7fbd9ed03640 1 --2- 172.21.15.139:0/1793243874 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbd880a34b0 0x7fbd880a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.049+0000 7fbd9dd01640 1 -- 172.21.15.139:0/1225570998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d73eaf0 msgr2=0x55750d73eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.049+0000 7fbd9dd01640 1 --2- 172.21.15.139:0/1225570998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d73eaf0 0x55750d73eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.049+0000 7fbd9dd01640 1 -- 172.21.15.139:0/1225570998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 msgr2=0x55750d749770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.049+0000 7fbd9dd01640 1 --2- 172.21.15.139:0/1225570998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d749770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.049+0000 7fbd9dd01640 1 -- 172.21.15.139:0/1225570998 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55750d64a340 con 0x55750d749cb0 2024-09-16T01:48:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.049+0000 7fbd9e502640 1 --2- 172.21.15.139:0/1225570998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d749770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:48:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.049+0000 7fbd977fe640 1 -- 172.21.15.139:0/1793243874 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9642+0+0 (secure 0 0 0) 0x7fbd8c0f0e50 con 0x7fbd9811b140 2024-09-16T01:48:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.049+0000 7fbd9dd01640 1 --2- 172.21.15.139:0/1225570998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d749cb0 0x55750d74c0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbd981180d0 tx=0x7fbd84001bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57f5fc254d23742a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.049+0000 7fbd94ff9640 1 -- 172.21.15.139:0/1225570998 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84004b80 con 0x55750d749cb0 2024-09-16T01:48:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.049+0000 7fbd94ff9640 1 -- 172.21.15.139:0/1225570998 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd840075b0 con 0x55750d749cb0 2024-09-16T01:48:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.050+0000 7fbd94ff9640 1 -- 172.21.15.139:0/1225570998 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8400eb80 con 0x55750d749cb0 2024-09-16T01:48:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.050+0000 7fbd9ed03640 1 --2- 172.21.15.139:0/1793243874 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbd880a34b0 0x7fbd880a5970 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0ee1f0 tx=0x7fbd8c090c60 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.050+0000 7fbda35dc740 1 -- 172.21.15.139:0/1225570998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d749cb0 msgr2=0x55750d74c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.050+0000 7fbda35dc740 1 --2- 172.21.15.139:0/1225570998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d749cb0 0x55750d74c0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbd981180d0 tx=0x7fbd84001bf0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.050+0000 7fbda35dc740 1 -- 172.21.15.139:0/1225570998 shutdown_connections 2024-09-16T01:48:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.050+0000 7fbda35dc740 1 --2- 172.21.15.139:0/1225570998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d749cb0 0x55750d74c0a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.050+0000 7fbda35dc740 1 --2- 172.21.15.139:0/1225570998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d749770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.050+0000 7fbda35dc740 1 --2- 172.21.15.139:0/1225570998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d73eaf0 0x55750d73eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.050+0000 7fbda35dc740 1 -- 172.21.15.139:0/1225570998 >> 172.21.15.139:0/1225570998 conn(0x55750d73b060 msgr2=0x55750d73b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.051+0000 7fbda35dc740 1 -- 172.21.15.139:0/1225570998 shutdown_connections 2024-09-16T01:48:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.051+0000 7fbda35dc740 1 -- 172.21.15.139:0/1225570998 wait complete. 2024-09-16T01:48:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.051+0000 7fbda35dc740 1 -- start start 2024-09-16T01:48:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.051+0000 7fbda35dc740 1 Processor -- start 2024-09-16T01:48:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.051+0000 7fbda35dc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d73eaf0 0x55750d746bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.052+0000 7fbda35dc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d747110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.052+0000 7fbda35dc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d749cb0 0x55750d747650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.052+0000 7fbd9dd01640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d73eaf0 0x55750d746bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.052+0000 7fbd9dd01640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d73eaf0 0x55750d746bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:47576/0 (socket says 172.21.15.139:47576) 2024-09-16T01:48:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.052+0000 7fbd9dd01640 1 -- 172.21.15.139:0/19970786 learned_addr learned my addr 172.21.15.139:0/19970786 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.052+0000 7fbd9e502640 1 --2- 172.21.15.139:0/19970786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d747110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.052+0000 7fbda35dc740 1 -- 172.21.15.139:0/19970786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55750d74c8d0 con 0x55750d73f410 2024-09-16T01:48:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.052+0000 7fbda35dc740 1 -- 172.21.15.139:0/19970786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55750d62c290 con 0x55750d749cb0 2024-09-16T01:48:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.052+0000 7fbda35dc740 1 -- 172.21.15.139:0/19970786 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55750d74ca50 con 0x55750d73eaf0 2024-09-16T01:48:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.052+0000 7fbd9dd01640 1 --2- 172.21.15.139:0/19970786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d749cb0 0x55750d747650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.052+0000 7fbda35dc740 10 client.0 _ll_register_callbacks cb 0x55750d65d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:48:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.053+0000 7fbd9e502640 1 -- 172.21.15.139:0/19970786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d73eaf0 msgr2=0x55750d746bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.053+0000 7fbd9e502640 1 --2- 172.21.15.139:0/19970786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d73eaf0 0x55750d746bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.053+0000 7fbd9e502640 1 -- 172.21.15.139:0/19970786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d749cb0 msgr2=0x55750d747650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.053+0000 7fbd9e502640 1 --2- 172.21.15.139:0/19970786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d749cb0 0x55750d747650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.053+0000 7fbd9e502640 1 -- 172.21.15.139:0/19970786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55750d3b6210 con 0x55750d73f410 2024-09-16T01:48:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.053+0000 7fbd9e502640 1 --2- 172.21.15.139:0/19970786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d747110 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbd900226a0 tx=0x7fbd90022b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bdc39019f0a255 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.053+0000 7fbd81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:48:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.054+0000 7fbd81ffb640 1 -- 172.21.15.139:0/19970786 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90030060 con 0x55750d73f410 2024-09-16T01:48:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.054+0000 7fbda35dc740 1 -- 172.21.15.139:0/19970786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd840075b0 con 0x55750d73f410 2024-09-16T01:48:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.054+0000 7fbd81ffb640 1 -- 172.21.15.139:0/19970786 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9642+0+0 (secure 0 0 0) 0x7fbd90019040 con 0x55750d73f410 2024-09-16T01:48:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.054+0000 7fbd81ffb640 1 -- 172.21.15.139:0/19970786 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbd90004260 con 0x55750d73f410 2024-09-16T01:48:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.054+0000 7fbda35dc740 10 client.16350 fetch_fsmap learned FSMap version 44 2024-09-16T01:48:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.054+0000 7fbda35dc740 1 -- 172.21.15.139:0/19970786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55750d482620 con 0x55750d73f410 2024-09-16T01:48:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.055+0000 7fbd81ffb640 1 -- 172.21.15.139:0/19970786 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd90020e10 con 0x55750d73f410 2024-09-16T01:48:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.055+0000 7fbd81ffb640 1 -- 172.21.15.139:0/19970786 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 44) ==== 8446+0+0 (secure 0 0 0) 0x7fbd9002a370 con 0x55750d73f410 2024-09-16T01:48:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.055+0000 7fbda35dc740 10 client.16350 fetch_fsmap finished waiting for FSMap version 44 2024-09-16T01:48:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.055+0000 7fbda35dc740 10 client.16350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:48:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.055+0000 7fbda35dc740 20 client.16350 populate_metadata read hostname 'smithi139' 2024-09-16T01:48:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.055+0000 7fbda35dc740 1 --2- 172.21.15.139:0/19970786 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55750d759f00 0x55750d75c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.055+0000 7fbda35dc740 4 client.16350 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:48:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.055+0000 7fbda35dc740 1 -- 172.21.15.139:0/19970786 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbd840075b0 con 0x55750d759f00 2024-09-16T01:48:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.056+0000 7fbd9dd01640 1 --2- 172.21.15.139:0/19970786 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55750d759f00 0x55750d75c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.060+0000 7fbd9dd01640 1 --2- 172.21.15.139:0/19970786 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55750d759f00 0x55750d75c320 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67a07d0c94633da7 server_cookie=1b00ca7935d710f9 in_seq=0 out_seq=0 2024-09-16T01:48:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.060+0000 7fbd81ffb640 10 client.16350 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.063+0000 7fbd81ffb640 1 -- 172.21.15.139:0/19970786 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fbd840075b0 con 0x55750d759f00 2024-09-16T01:48:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.063+0000 7fbd81ffb640 10 client.16350 handle_command_reply: tid=0 2024-09-16T01:48:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.064+0000 7fbda35dc740 1 client.16350 shutdown 2024-09-16T01:48:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.064+0000 7fbda35dc740 1 -- 172.21.15.139:0/19970786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 msgr2=0x55750d747110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.064+0000 7fbda35dc740 1 --2- 172.21.15.139:0/19970786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d747110 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbd900226a0 tx=0x7fbd90022b60 comp rx=0 tx=0).stop 2024-09-16T01:48:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.065+0000 7fbda35dc740 1 -- 172.21.15.139:0/19970786 shutdown_connections 2024-09-16T01:48:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.065+0000 7fbda35dc740 1 --2- 172.21.15.139:0/19970786 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55750d759f00 0x55750d75c320 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.065+0000 7fbda35dc740 1 --2- 172.21.15.139:0/19970786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d749cb0 0x55750d747650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.065+0000 7fbda35dc740 1 --2- 172.21.15.139:0/19970786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d747110 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.065+0000 7fbda35dc740 1 --2- 172.21.15.139:0/19970786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d73eaf0 0x55750d746bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.065+0000 7fbda35dc740 1 -- 172.21.15.139:0/19970786 >> 172.21.15.139:0/19970786 conn(0x55750d73b060 msgr2=0x55750d740dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.065+0000 7fbd81ffb640 0 client.16350 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.066+0000 7fbda35dc740 1 -- 172.21.15.139:0/19970786 shutdown_connections 2024-09-16T01:48:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.066+0000 7fbda35dc740 1 -- 172.21.15.139:0/19970786 wait complete. 2024-09-16T01:48:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.066+0000 7fbda35dc740 20 client.16350 trim_cache size 0 max 16384 2024-09-16T01:48:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.089+0000 7fbda35dc740 1 Processor -- start 2024-09-16T01:48:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.089+0000 7fbda35dc740 1 -- start start 2024-09-16T01:48:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.090+0000 7fbda35dc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d73eaf0 0x55750d7415b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.090+0000 7fbda35dc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d747480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.090+0000 7fbda35dc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d749cb0 0x55750d748b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.090+0000 7fbda35dc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c010230 con 0x55750d73f410 2024-09-16T01:48:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.090+0000 7fbda35dc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c012040 con 0x55750d73eaf0 2024-09-16T01:48:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.090+0000 7fbda35dc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c0109c0 con 0x55750d749cb0 2024-09-16T01:48:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.090+0000 7fbd9dd01640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d73eaf0 0x55750d7415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.090+0000 7fbd9dd01640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d73eaf0 0x55750d7415b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:35804/0 (socket says 172.21.15.139:35804) 2024-09-16T01:48:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.090+0000 7fbd9dd01640 1 -- 172.21.15.139:0/1409578486 learned_addr learned my addr 172.21.15.139:0/1409578486 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.090+0000 7fbd9e502640 1 --2- 172.21.15.139:0/1409578486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d747480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.090+0000 7fbd9dd01640 1 --2- 172.21.15.139:0/1409578486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d749cb0 0x55750d748b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.091+0000 7fbd9dd01640 1 -- 172.21.15.139:0/1409578486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d749cb0 msgr2=0x55750d748b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.091+0000 7fbd9dd01640 1 --2- 172.21.15.139:0/1409578486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d749cb0 0x55750d748b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.091+0000 7fbd9dd01640 1 -- 172.21.15.139:0/1409578486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 msgr2=0x55750d747480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.091+0000 7fbd9dd01640 1 --2- 172.21.15.139:0/1409578486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d747480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.091+0000 7fbd9dd01640 1 -- 172.21.15.139:0/1409578486 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55750d73bc10 con 0x55750d73eaf0 2024-09-16T01:48:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.091+0000 7fbd9e502640 1 --2- 172.21.15.139:0/1409578486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d747480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:48:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.091+0000 7fbd9dd01640 1 --2- 172.21.15.139:0/1409578486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d73eaf0 0x55750d7415b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400e750 tx=0x7fbd84001bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9792e3e27b152dfe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.091+0000 7fbd83fff640 1 -- 172.21.15.139:0/1409578486 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84007180 con 0x55750d73eaf0 2024-09-16T01:48:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.091+0000 7fbd83fff640 1 -- 172.21.15.139:0/1409578486 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd84004500 con 0x55750d73eaf0 2024-09-16T01:48:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.091+0000 7fbd83fff640 1 -- 172.21.15.139:0/1409578486 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8400ecd0 con 0x55750d73eaf0 2024-09-16T01:48:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.092+0000 7fbda35dc740 1 -- 172.21.15.139:0/1409578486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d73eaf0 msgr2=0x55750d7415b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.092+0000 7fbda35dc740 1 --2- 172.21.15.139:0/1409578486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d73eaf0 0x55750d7415b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400e750 tx=0x7fbd84001bf0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.092+0000 7fbda35dc740 1 -- 172.21.15.139:0/1409578486 shutdown_connections 2024-09-16T01:48:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.092+0000 7fbda35dc740 1 --2- 172.21.15.139:0/1409578486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d749cb0 0x55750d748b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.092+0000 7fbda35dc740 1 --2- 172.21.15.139:0/1409578486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d747480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.092+0000 7fbda35dc740 1 --2- 172.21.15.139:0/1409578486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d73eaf0 0x55750d7415b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.092+0000 7fbda35dc740 1 -- 172.21.15.139:0/1409578486 >> 172.21.15.139:0/1409578486 conn(0x55750d73b060 msgr2=0x55750d748430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.092+0000 7fbda35dc740 1 -- 172.21.15.139:0/1409578486 shutdown_connections 2024-09-16T01:48:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.092+0000 7fbda35dc740 1 -- 172.21.15.139:0/1409578486 wait complete. 2024-09-16T01:48:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.092+0000 7fbda35dc740 1 -- start start 2024-09-16T01:48:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.092+0000 7fbda35dc740 1 Processor -- start 2024-09-16T01:48:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbda35dc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d73eaf0 0x55750d7522b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbda35dc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d743690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbda35dc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d749cb0 0x55750d743bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbda35dc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c0118b0 con 0x55750d73f410 2024-09-16T01:48:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbda35dc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c011150 con 0x55750d749cb0 2024-09-16T01:48:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbda35dc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c00e230 con 0x55750d73eaf0 2024-09-16T01:48:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbda35dc740 10 client.0 _ll_register_callbacks cb 0x7fbd9001a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:48:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbd9e502640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d743690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbd9dd01640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d73eaf0 0x55750d7522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbd9dd01640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d73eaf0 0x55750d7522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:47586/0 (socket says 172.21.15.139:47586) 2024-09-16T01:48:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbd9e502640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d743690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48942/0 (socket says 172.21.15.139:48942) 2024-09-16T01:48:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbd9dd01640 1 -- 172.21.15.139:0/3734768915 learned_addr learned my addr 172.21.15.139:0/3734768915 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbd9dd01640 1 --2- 172.21.15.139:0/3734768915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d749cb0 0x55750d743bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbd9e502640 1 -- 172.21.15.139:0/3734768915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d73eaf0 msgr2=0x55750d7522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbd9e502640 1 --2- 172.21.15.139:0/3734768915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d73eaf0 0x55750d7522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbd9e502640 1 -- 172.21.15.139:0/3734768915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d749cb0 msgr2=0x55750d743bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbd9e502640 1 --2- 172.21.15.139:0/3734768915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d749cb0 0x55750d743bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbd9e502640 1 -- 172.21.15.139:0/3734768915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55750d7441c0 con 0x55750d73f410 2024-09-16T01:48:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbd9e502640 1 --2- 172.21.15.139:0/3734768915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d743690 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbd9002a7c0 tx=0x7fbd9002abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49853933ecc9770b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.093+0000 7fbd81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:48:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.094+0000 7fbd81ffb640 1 -- 172.21.15.139:0/3734768915 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9003a080 con 0x55750d73f410 2024-09-16T01:48:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.094+0000 7fbd81ffb640 1 -- 172.21.15.139:0/3734768915 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9642+0+0 (secure 0 0 0) 0x7fbd90028040 con 0x55750d73f410 2024-09-16T01:48:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.094+0000 7fbda35dc740 1 -- 172.21.15.139:0/3734768915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd84004500 con 0x55750d73f410 2024-09-16T01:48:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.094+0000 7fbd81ffb640 1 -- 172.21.15.139:0/3734768915 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbd90020b40 con 0x55750d73f410 2024-09-16T01:48:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.094+0000 7fbda35dc740 10 client.16356 fetch_fsmap learned FSMap version 44 2024-09-16T01:48:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.094+0000 7fbda35dc740 1 -- 172.21.15.139:0/3734768915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55750d7c49d0 con 0x55750d73f410 2024-09-16T01:48:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.094+0000 7fbd81ffb640 1 -- 172.21.15.139:0/3734768915 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd84004500 con 0x55750d73f410 2024-09-16T01:48:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.094+0000 7fbd81ffb640 1 -- 172.21.15.139:0/3734768915 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 44) ==== 8446+0+0 (secure 0 0 0) 0x7fbd90003660 con 0x55750d73f410 2024-09-16T01:48:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.094+0000 7fbda35dc740 10 client.16356 fetch_fsmap finished waiting for FSMap version 44 2024-09-16T01:48:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.094+0000 7fbda35dc740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:48:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.094+0000 7fbda35dc740 20 client.16356 populate_metadata read hostname 'smithi139' 2024-09-16T01:48:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.094+0000 7fbda35dc740 1 --2- 172.21.15.139:0/3734768915 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55750d7c51a0 0x55750d7c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.094+0000 7fbda35dc740 4 client.16356 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:48:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.094+0000 7fbda35dc740 1 -- 172.21.15.139:0/3734768915 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55750d6b0a20 con 0x55750d7c51a0 2024-09-16T01:48:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.095+0000 7fbd9dd01640 1 --2- 172.21.15.139:0/3734768915 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55750d7c51a0 0x55750d7c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.095+0000 7fbd9dd01640 1 --2- 172.21.15.139:0/3734768915 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55750d7c51a0 0x55750d7c75c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=619189222d3fb627 server_cookie=7ee0f085481ba17b in_seq=0 out_seq=0 2024-09-16T01:48:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.095+0000 7fbd81ffb640 10 client.16356 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.159+0000 7fbd81ffb640 1 -- 172.21.15.139:0/3734768915 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55750d6b0a20 con 0x55750d7c51a0 2024-09-16T01:48:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.159+0000 7fbd81ffb640 10 client.16356 handle_command_reply: tid=0 2024-09-16T01:48:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.159+0000 7fbda35dc740 1 client.16356 shutdown 2024-09-16T01:48:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.160+0000 7fbda35dc740 1 -- 172.21.15.139:0/3734768915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 msgr2=0x55750d743690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.160+0000 7fbda35dc740 1 --2- 172.21.15.139:0/3734768915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d743690 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbd9002a7c0 tx=0x7fbd9002abe0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.160+0000 7fbda35dc740 1 -- 172.21.15.139:0/3734768915 shutdown_connections 2024-09-16T01:48:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.160+0000 7fbda35dc740 1 --2- 172.21.15.139:0/3734768915 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55750d7c51a0 0x55750d7c75c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.160+0000 7fbda35dc740 1 --2- 172.21.15.139:0/3734768915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55750d749cb0 0x55750d743bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.160+0000 7fbda35dc740 1 --2- 172.21.15.139:0/3734768915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55750d73f410 0x55750d743690 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.160+0000 7fbd81ffb640 0 client.16356 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.160+0000 7fbda35dc740 1 --2- 172.21.15.139:0/3734768915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55750d73eaf0 0x55750d7522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.160+0000 7fbda35dc740 1 -- 172.21.15.139:0/3734768915 >> 172.21.15.139:0/3734768915 conn(0x55750d73b060 msgr2=0x55750d748c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.160+0000 7fbda35dc740 1 -- 172.21.15.139:0/3734768915 shutdown_connections 2024-09-16T01:48:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.160+0000 7fbda35dc740 1 -- 172.21.15.139:0/3734768915 wait complete. 2024-09-16T01:48:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.160+0000 7fbda35dc740 20 client.16356 trim_cache size 0 max 16384 2024-09-16T01:48:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:48:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:48:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:48:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.161+0000 7fbda0766640 1 -- 172.21.15.139:0/1793243874 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbd880a34b0 msgr2=0x7fbd880a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.161+0000 7fbda0766640 1 --2- 172.21.15.139:0/1793243874 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbd880a34b0 0x7fbd880a5970 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0ee1f0 tx=0x7fbd8c090c60 comp rx=0 tx=0).stop 2024-09-16T01:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.161+0000 7fbda0766640 1 -- 172.21.15.139:0/1793243874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd9811b140 msgr2=0x7fbd98116dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.161+0000 7fbda0766640 1 --2- 172.21.15.139:0/1793243874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd9811b140 0x7fbd98116dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009b30 tx=0x7fbd8c003b80 comp rx=0 tx=0).stop 2024-09-16T01:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.162+0000 7fbda0766640 1 -- 172.21.15.139:0/1793243874 shutdown_connections 2024-09-16T01:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.162+0000 7fbda0766640 1 --2- 172.21.15.139:0/1793243874 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbd880a34b0 0x7fbd880a5970 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.162+0000 7fbda0766640 1 --2- 172.21.15.139:0/1793243874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd9811b140 0x7fbd98116dc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.162+0000 7fbda0766640 1 --2- 172.21.15.139:0/1793243874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd981095d0 0x7fbd98116880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.162+0000 7fbda0766640 1 --2- 172.21.15.139:0/1793243874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd981091f0 0x7fbd9811a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.162+0000 7fbda0766640 1 -- 172.21.15.139:0/1793243874 >> 172.21.15.139:0/1793243874 conn(0x7fbd9806ce00 msgr2=0x7fbd98070850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.162+0000 7fbda0766640 1 -- 172.21.15.139:0/1793243874 shutdown_connections 2024-09-16T01:48:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:16.162+0000 7fbda0766640 1 -- 172.21.15.139:0/1793243874 wait complete. 2024-09-16T01:48:17.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:17 smithi176 ceph-mon[35324]: pgmap v243: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 656 KiB/s wr, 205 op/s 2024-09-16T01:48:18.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:17 smithi164 ceph-mon[35121]: pgmap v243: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 656 KiB/s wr, 205 op/s 2024-09-16T01:48:18.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:17 smithi139 ceph-mon[31175]: pgmap v243: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 656 KiB/s wr, 205 op/s 2024-09-16T01:48:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:19 smithi139 ceph-mon[31175]: pgmap v244: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 572 KiB/s wr, 159 op/s 2024-09-16T01:48:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:19 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:19.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:19 smithi176 ceph-mon[35324]: pgmap v244: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 572 KiB/s wr, 159 op/s 2024-09-16T01:48:19.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:19 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:20.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:19 smithi164 ceph-mon[35121]: pgmap v244: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 572 KiB/s wr, 159 op/s 2024-09-16T01:48:20.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:19 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:20.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb45ffb640 1 -- 172.21.15.139:0/3376076753 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c4cb94dd20 con 0x55c4cba784d0 2024-09-16T01:48:20.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb45ffb640 10 client.25814 handle_command_reply: tid=0 2024-09-16T01:48:20.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb66ac5740 1 client.25814 shutdown 2024-09-16T01:48:20.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3376076753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9ebae0 msgr2=0x55c4cb9e5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:20.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3376076753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9e5650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcb58097f00 tx=0x7fcb580047d0 comp rx=0 tx=0).stop 2024-09-16T01:48:20.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3376076753 shutdown_connections 2024-09-16T01:48:20.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3376076753 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c4cba784d0 0x55c4cba7a8f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:20.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3376076753 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c4cb9e5be0 0x55c4cb9e6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:20.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3376076753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4cb9ebae0 0x55c4cb9e5650 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:20.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb45ffb640 0 client.25814 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:20.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb66ac5740 1 --2- 172.21.15.139:0/3376076753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4cb9e0900 0x55c4cb9f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:20.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3376076753 >> 172.21.15.139:0/3376076753 conn(0x55c4cb9dce90 msgr2=0x55c4cb9db7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:20.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3376076753 shutdown_connections 2024-09-16T01:48:20.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb66ac5740 1 -- 172.21.15.139:0/3376076753 wait complete. 2024-09-16T01:48:20.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.343+0000 7fcb66ac5740 20 client.25814 trim_cache size 0 max 16384 2024-09-16T01:48:20.346 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 45, 2024-09-16T01:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 26, 2024-09-16T01:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout: "70ffc2a7": { 2024-09-16T01:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 26, 2024-09-16T01:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:48:20.347 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 143.7, 2024-09-16T01:48:20.348 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:48:20.348 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:48:20.348 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:48:20.348 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:48:20.348 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:48:20.348 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:48:20.348 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:48:20.348 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:48:20.348 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:48:20.348 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:48:20.348 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:48:20.348 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:48:20.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.344+0000 7fcb6348a640 1 -- 172.21.15.139:0/3970553362 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcb480a35c0 msgr2=0x7fcb480a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:20.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.344+0000 7fcb6348a640 1 --2- 172.21.15.139:0/3970553362 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcb480a35c0 0x7fcb480a5a80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcb54030840 tx=0x7fcb54005fa0 comp rx=0 tx=0).stop 2024-09-16T01:48:20.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.344+0000 7fcb6348a640 1 -- 172.21.15.139:0/3970553362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb5c101270 msgr2=0x7fcb5c1961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:20.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.344+0000 7fcb6348a640 1 --2- 172.21.15.139:0/3970553362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb5c101270 0x7fcb5c1961c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800b3f0 tx=0x7fcb5800b8b0 comp rx=0 tx=0).stop 2024-09-16T01:48:20.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.344+0000 7fcb6348a640 1 -- 172.21.15.139:0/3970553362 shutdown_connections 2024-09-16T01:48:20.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.344+0000 7fcb6348a640 1 --2- 172.21.15.139:0/3970553362 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcb480a35c0 0x7fcb480a5a80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:20.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.344+0000 7fcb6348a640 1 --2- 172.21.15.139:0/3970553362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb5c1068d0 0x7fcb5c196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:20.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.344+0000 7fcb6348a640 1 --2- 172.21.15.139:0/3970553362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb5c101270 0x7fcb5c1961c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:20.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.344+0000 7fcb6348a640 1 --2- 172.21.15.139:0/3970553362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb5c1008d0 0x7fcb5c19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:20.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.344+0000 7fcb6348a640 1 -- 172.21.15.139:0/3970553362 >> 172.21.15.139:0/3970553362 conn(0x7fcb5c0fc610 msgr2=0x7fcb5c1023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:20.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.344+0000 7fcb6348a640 1 -- 172.21.15.139:0/3970553362 shutdown_connections 2024-09-16T01:48:20.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:20.344+0000 7fcb6348a640 1 -- 172.21.15.139:0/3970553362 wait complete. 2024-09-16T01:48:21.210 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '70ffc2a7', quiesce took 16.6 seconds. Will release after: 7.099999999999998 2024-09-16T01:48:22.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:21 smithi164 ceph-mon[35121]: pgmap v245: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 563 KiB/s wr, 118 op/s 2024-09-16T01:48:22.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:21 smithi139 ceph-mon[31175]: pgmap v245: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 563 KiB/s wr, 118 op/s 2024-09-16T01:48:22.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:21 smithi176 ceph-mon[35324]: pgmap v245: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 563 KiB/s wr, 118 op/s 2024-09-16T01:48:24.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:23 smithi164 ceph-mon[35121]: pgmap v246: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 517 KiB/s wr, 111 op/s 2024-09-16T01:48:24.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:23 smithi139 ceph-mon[31175]: pgmap v246: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 517 KiB/s wr, 111 op/s 2024-09-16T01:48:24.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:23 smithi176 ceph-mon[35324]: pgmap v246: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 517 KiB/s wr, 111 op/s 2024-09-16T01:48:26.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:25 smithi164 ceph-mon[35121]: pgmap v247: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 100 KiB/s wr, 61 op/s 2024-09-16T01:48:26.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:25 smithi139 ceph-mon[31175]: pgmap v247: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 100 KiB/s wr, 61 op/s 2024-09-16T01:48:26.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:25 smithi176 ceph-mon[35324]: pgmap v247: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 100 KiB/s wr, 61 op/s 2024-09-16T01:48:27.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:26 smithi176 ceph-mon[35324]: pgmap v248: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 45 op/s 2024-09-16T01:48:27.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:26 smithi176 ceph-mon[35324]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T01:48:27.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:26 smithi164 ceph-mon[35121]: pgmap v248: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 45 op/s 2024-09-16T01:48:27.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:26 smithi164 ceph-mon[35121]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T01:48:27.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:26 smithi139 ceph-mon[31175]: pgmap v248: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 45 op/s 2024-09-16T01:48:27.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:26 smithi139 ceph-mon[31175]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T01:48:28.310 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:48:28.575 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:48:28.977 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:28 smithi139 ceph-mon[31175]: pgmap v250: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 24 op/s 2024-09-16T01:48:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.054+0000 7f2de0ed3640 1 -- 172.21.15.139:0/2630908235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ddc10c170 msgr2=0x7f2ddc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.054+0000 7f2de0ed3640 1 --2- 172.21.15.139:0/2630908235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ddc10c170 0x7f2ddc10e610 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009900 tx=0x7f2dd402a480 comp rx=0 tx=0).stop 2024-09-16T01:48:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.054+0000 7f2de0ed3640 1 -- 172.21.15.139:0/2630908235 shutdown_connections 2024-09-16T01:48:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.054+0000 7f2de0ed3640 1 --2- 172.21.15.139:0/2630908235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ddc10c170 0x7f2ddc10e610 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.054+0000 7f2de0ed3640 1 --2- 172.21.15.139:0/2630908235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ddc103660 0x7f2ddc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.054+0000 7f2de0ed3640 1 --2- 172.21.15.139:0/2630908235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ddc102d40 0x7f2ddc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.054+0000 7f2de0ed3640 1 -- 172.21.15.139:0/2630908235 >> 172.21.15.139:0/2630908235 conn(0x7f2ddc0fc7a0 msgr2=0x7f2ddc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.055+0000 7f2de0ed3640 1 -- 172.21.15.139:0/2630908235 shutdown_connections 2024-09-16T01:48:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.055+0000 7f2de0ed3640 1 -- 172.21.15.139:0/2630908235 wait complete. 2024-09-16T01:48:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.055+0000 7f2de0ed3640 1 Processor -- start 2024-09-16T01:48:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.055+0000 7f2de0ed3640 1 -- start start 2024-09-16T01:48:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.055+0000 7f2de0ed3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ddc102d40 0x7f2ddc198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.055+0000 7f2de0ed3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ddc103660 0x7f2ddc198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.055+0000 7f2de0ed3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ddc10c170 0x7f2ddc19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.055+0000 7f2de0ed3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc111f30 con 0x7f2ddc103660 2024-09-16T01:48:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.055+0000 7f2de0ed3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc111db0 con 0x7f2ddc102d40 2024-09-16T01:48:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.055+0000 7f2de0ed3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc1120b0 con 0x7f2ddc10c170 2024-09-16T01:48:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2dd9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ddc103660 0x7f2ddc198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2dd9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ddc103660 0x7f2ddc198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45804/0 (socket says 172.21.15.139:45804) 2024-09-16T01:48:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2dd9d9b640 1 -- 172.21.15.139:0/512521375 learned_addr learned my addr 172.21.15.139:0/512521375 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2ddad9d640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ddc10c170 0x7f2ddc19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2dda59c640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ddc102d40 0x7f2ddc198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2dd9d9b640 1 -- 172.21.15.139:0/512521375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ddc10c170 msgr2=0x7f2ddc19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2dd9d9b640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ddc10c170 0x7f2ddc19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2dd9d9b640 1 -- 172.21.15.139:0/512521375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ddc102d40 msgr2=0x7f2ddc198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2dd9d9b640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ddc102d40 0x7f2ddc198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2dd9d9b640 1 -- 172.21.15.139:0/512521375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ddc19d3f0 con 0x7f2ddc103660 2024-09-16T01:48:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2ddad9d640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ddc10c170 0x7f2ddc19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:48:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2dd9d9b640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ddc103660 0x7f2ddc198980 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00ea00 tx=0x7f2dcc00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2dcb7fe640 1 -- 172.21.15.139:0/512521375 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00cd80 con 0x7f2ddc103660 2024-09-16T01:48:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2dcb7fe640 1 -- 172.21.15.139:0/512521375 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dcc011950 con 0x7f2ddc103660 2024-09-16T01:48:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2de0ed3640 1 -- 172.21.15.139:0/512521375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ddc19d6e0 con 0x7f2ddc103660 2024-09-16T01:48:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2dcb7fe640 1 -- 172.21.15.139:0/512521375 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc01aa30 con 0x7f2ddc103660 2024-09-16T01:48:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.056+0000 7f2de0ed3640 1 -- 172.21.15.139:0/512521375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ddc1a4fc0 con 0x7f2ddc103660 2024-09-16T01:48:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.057+0000 7f2de0ed3640 1 -- 172.21.15.139:0/512521375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ddc103120 con 0x7f2ddc103660 2024-09-16T01:48:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.058+0000 7f2dcb7fe640 1 -- 172.21.15.139:0/512521375 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f2dcc011b00 con 0x7f2ddc103660 2024-09-16T01:48:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.059+0000 7f2dcb7fe640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2dc40a33b0 0x7f2dc40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.059+0000 7f2dda59c640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2dc40a33b0 0x7f2dc40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.059+0000 7f2dcb7fe640 1 -- 172.21.15.139:0/512521375 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9554+0+0 (secure 0 0 0) 0x7f2dcc0d0640 con 0x7f2ddc103660 2024-09-16T01:48:29.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.059+0000 7f2dda59c640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2dc40a33b0 0x7f2dc40a5870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009b60 tx=0x7f2dd0005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:29.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.063+0000 7f2dcb7fe640 1 -- 172.21.15.139:0/512521375 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2dcc016030 con 0x7f2ddc103660 2024-09-16T01:48:29.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:28 smithi176 ceph-mon[35324]: pgmap v250: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 24 op/s 2024-09-16T01:48:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.257+0000 7fbf6c598640 1 -- 172.21.15.139:0/3022357879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf64103640 msgr2=0x7fbf6410bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.257+0000 7fbf6c598640 1 --2- 172.21.15.139:0/3022357879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf64103640 0x7fbf6410bb20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009900 tx=0x7fbf5402a480 comp rx=0 tx=0).stop 2024-09-16T01:48:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.257+0000 7fbf6c598640 1 -- 172.21.15.139:0/3022357879 shutdown_connections 2024-09-16T01:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.257+0000 7fbf6c598640 1 --2- 172.21.15.139:0/3022357879 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf6410c0f0 0x7fbf6410e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.257+0000 7fbf6c598640 1 --2- 172.21.15.139:0/3022357879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf64103640 0x7fbf6410bb20 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.257+0000 7fbf6c598640 1 --2- 172.21.15.139:0/3022357879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf64102d20 0x7fbf64103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.257+0000 7fbf6c598640 1 -- 172.21.15.139:0/3022357879 >> 172.21.15.139:0/3022357879 conn(0x7fbf64077990 msgr2=0x7fbf640fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.257+0000 7fbf6c598640 1 -- 172.21.15.139:0/3022357879 shutdown_connections 2024-09-16T01:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.257+0000 7fbf6c598640 1 -- 172.21.15.139:0/3022357879 wait complete. 2024-09-16T01:48:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.258+0000 7fbf6c598640 1 Processor -- start 2024-09-16T01:48:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.258+0000 7fbf6c598640 1 -- start start 2024-09-16T01:48:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.258+0000 7fbf6c598640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf64102d20 0x7fbf64198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.258+0000 7fbf6c598640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf64103640 0x7fbf641988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.258+0000 7fbf6c598640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf6410c0f0 0x7fbf6419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.258+0000 7fbf6c598640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64111ec0 con 0x7fbf64102d20 2024-09-16T01:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.258+0000 7fbf6c598640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64111d40 con 0x7fbf64103640 2024-09-16T01:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.258+0000 7fbf6c598640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64112040 con 0x7fbf6410c0f0 2024-09-16T01:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.258+0000 7fbf6a334640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf64102d20 0x7fbf64198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.258+0000 7fbf6a334640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf64102d20 0x7fbf64198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45808/0 (socket says 172.21.15.139:45808) 2024-09-16T01:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.258+0000 7fbf6a334640 1 -- 172.21.15.139:0/1371884948 learned_addr learned my addr 172.21.15.139:0/1371884948 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.258+0000 7fbf69b33640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf64103640 0x7fbf641988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.259+0000 7fbf6a334640 1 -- 172.21.15.139:0/1371884948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf6410c0f0 msgr2=0x7fbf6419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.259+0000 7fbf6ab35640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf6410c0f0 0x7fbf6419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.259+0000 7fbf6a334640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf6410c0f0 0x7fbf6419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.259+0000 7fbf6a334640 1 -- 172.21.15.139:0/1371884948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf64103640 msgr2=0x7fbf641988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.259+0000 7fbf6a334640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf64103640 0x7fbf641988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.259+0000 7fbf6a334640 1 -- 172.21.15.139:0/1371884948 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6419d360 con 0x7fbf64102d20 2024-09-16T01:48:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.259+0000 7fbf69b33640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf64103640 0x7fbf641988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:48:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.259+0000 7fbf6a334640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf64102d20 0x7fbf64198370 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000d950 tx=0x7fbf6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.259+0000 7fbf5b7fe640 1 -- 172.21.15.139:0/1371884948 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60017070 con 0x7fbf64102d20 2024-09-16T01:48:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.259+0000 7fbf5b7fe640 1 -- 172.21.15.139:0/1371884948 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf60009c30 con 0x7fbf64102d20 2024-09-16T01:48:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.259+0000 7fbf6c598640 1 -- 172.21.15.139:0/1371884948 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6419d650 con 0x7fbf64102d20 2024-09-16T01:48:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.259+0000 7fbf5b7fe640 1 -- 172.21.15.139:0/1371884948 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6000f6b0 con 0x7fbf64102d20 2024-09-16T01:48:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.259+0000 7fbf6c598640 1 -- 172.21.15.139:0/1371884948 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf641a4ee0 con 0x7fbf64102d20 2024-09-16T01:48:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.261+0000 7fbf5b7fe640 1 -- 172.21.15.139:0/1371884948 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fbf600109d0 con 0x7fbf64102d20 2024-09-16T01:48:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.262+0000 7fbf6f413740 1 Processor -- start 2024-09-16T01:48:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.262+0000 7fbf6f413740 1 -- start start 2024-09-16T01:48:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.262+0000 7fbf6f413740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b19b0 0x55d78e8b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.262+0000 7fbf6f413740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b22d0 0x55d78e8c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.262+0000 7fbf6f413740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 0x55d78e8c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.262+0000 7fbf6f413740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d78e8162f0 con 0x55d78e8b19b0 2024-09-16T01:48:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.262+0000 7fbf6f413740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d78e7be270 con 0x55d78e8c10c0 2024-09-16T01:48:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.262+0000 7fbf6f413740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d78e8122b0 con 0x55d78e8b22d0 2024-09-16T01:48:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.262+0000 7fbf6ab35640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b19b0 0x55d78e8b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.262+0000 7fbf6ab35640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b19b0 0x55d78e8b1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45816/0 (socket says 172.21.15.139:45816) 2024-09-16T01:48:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.262+0000 7fbf6ab35640 1 -- 172.21.15.139:0/4205350846 learned_addr learned my addr 172.21.15.139:0/4205350846 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.262+0000 7fbf5b7fe640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbf500a3460 0x7fbf500a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf69b33640 1 --2- 172.21.15.139:0/4205350846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 0x55d78e8c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf6a334640 1 --2- 172.21.15.139:0/4205350846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b22d0 0x55d78e8c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf6ab35640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbf500a3460 0x7fbf500a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf6ab35640 1 -- 172.21.15.139:0/4205350846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b22d0 msgr2=0x55d78e8c0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf6ab35640 1 --2- 172.21.15.139:0/4205350846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b22d0 0x55d78e8c0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf5b7fe640 1 -- 172.21.15.139:0/1371884948 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9554+0+0 (secure 0 0 0) 0x7fbf600cfe90 con 0x7fbf64102d20 2024-09-16T01:48:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf6ab35640 1 -- 172.21.15.139:0/4205350846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 msgr2=0x55d78e8c34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf6ab35640 1 --2- 172.21.15.139:0/4205350846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 0x55d78e8c34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf6ab35640 1 -- 172.21.15.139:0/4205350846 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d78e7bd340 con 0x55d78e8b19b0 2024-09-16T01:48:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf69b33640 1 --2- 172.21.15.139:0/4205350846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 0x55d78e8c34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:48:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf6ab35640 1 --2- 172.21.15.139:0/4205350846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b19b0 0x55d78e8b1d90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0094d0 tx=0x7fbf5c013dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0802daa3c6e8716 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf58bfe640 1 -- 172.21.15.139:0/4205350846 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c013250 con 0x55d78e8b19b0 2024-09-16T01:48:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf58bfe640 1 -- 172.21.15.139:0/4205350846 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf5c0133f0 con 0x55d78e8b19b0 2024-09-16T01:48:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf58bfe640 1 -- 172.21.15.139:0/4205350846 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c01eab0 con 0x55d78e8b19b0 2024-09-16T01:48:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.263+0000 7fbf6ab35640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbf500a3460 0x7fbf500a5920 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c022000 tx=0x7fbf5c020000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.264+0000 7fbf6f413740 1 -- 172.21.15.139:0/4205350846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b19b0 msgr2=0x55d78e8b1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.264+0000 7fbf6f413740 1 --2- 172.21.15.139:0/4205350846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b19b0 0x55d78e8b1d90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0094d0 tx=0x7fbf5c013dc0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.264+0000 7fbf6f413740 1 -- 172.21.15.139:0/4205350846 shutdown_connections 2024-09-16T01:48:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.264+0000 7fbf6f413740 1 --2- 172.21.15.139:0/4205350846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 0x55d78e8c34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.264+0000 7fbf6f413740 1 --2- 172.21.15.139:0/4205350846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b22d0 0x55d78e8c0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.264+0000 7fbf6f413740 1 --2- 172.21.15.139:0/4205350846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b19b0 0x55d78e8b1d90 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.264+0000 7fbf6f413740 1 -- 172.21.15.139:0/4205350846 >> 172.21.15.139:0/4205350846 conn(0x55d78e8ae060 msgr2=0x55d78e8ae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.264+0000 7fbf6f413740 1 -- 172.21.15.139:0/4205350846 shutdown_connections 2024-09-16T01:48:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.264+0000 7fbf6f413740 1 -- 172.21.15.139:0/4205350846 wait complete. 2024-09-16T01:48:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.264+0000 7fbf6f413740 1 -- start start 2024-09-16T01:48:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf6f413740 1 Processor -- start 2024-09-16T01:48:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf6f413740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b19b0 0x55d78e8be040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf6f413740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b22d0 0x55d78e8be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf6f413740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 0x55d78e8beac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf6f413740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d78e8c3c90 con 0x55d78e8b22d0 2024-09-16T01:48:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf6f413740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d78e79f290 con 0x55d78e8c10c0 2024-09-16T01:48:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf6f413740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d78e8c3e10 con 0x55d78e8b19b0 2024-09-16T01:48:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf6f413740 10 client.0 _ll_register_callbacks cb 0x55d78e7d0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:48:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf69b33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b22d0 0x55d78e8be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf69b33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b22d0 0x55d78e8be580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45820/0 (socket says 172.21.15.139:45820) 2024-09-16T01:48:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf69b33640 1 -- 172.21.15.139:0/1434883287 learned_addr learned my addr 172.21.15.139:0/1434883287 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf6a334640 1 --2- 172.21.15.139:0/1434883287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b19b0 0x55d78e8be040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf6ab35640 1 --2- 172.21.15.139:0/1434883287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 0x55d78e8beac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf69b33640 1 -- 172.21.15.139:0/1434883287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b19b0 msgr2=0x55d78e8be040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf69b33640 1 --2- 172.21.15.139:0/1434883287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b19b0 0x55d78e8be040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf69b33640 1 -- 172.21.15.139:0/1434883287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 msgr2=0x55d78e8beac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf69b33640 1 --2- 172.21.15.139:0/1434883287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 0x55d78e8beac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf69b33640 1 -- 172.21.15.139:0/1434883287 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d78e529210 con 0x55d78e8b22d0 2024-09-16T01:48:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf6ab35640 1 --2- 172.21.15.139:0/1434883287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 0x55d78e8beac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:48:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf69b33640 1 --2- 172.21.15.139:0/1434883287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b22d0 0x55d78e8be580 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009900 tx=0x7fbf54031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884a239d55a9f9b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.265+0000 7fbf4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:48:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf4dffb640 1 -- 172.21.15.139:0/1434883287 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54035040 con 0x55d78e8b22d0 2024-09-16T01:48:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf4dffb640 1 -- 172.21.15.139:0/1434883287 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9554+0+0 (secure 0 0 0) 0x7fbf54004240 con 0x55d78e8b22d0 2024-09-16T01:48:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf6f413740 1 -- 172.21.15.139:0/1434883287 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf5c0133f0 con 0x55d78e8b22d0 2024-09-16T01:48:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf4dffb640 1 -- 172.21.15.139:0/1434883287 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbf54004440 con 0x55d78e8b22d0 2024-09-16T01:48:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf6f413740 10 client.16374 fetch_fsmap learned FSMap version 45 2024-09-16T01:48:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf6f413740 1 -- 172.21.15.139:0/1434883287 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d78e5f5620 con 0x55d78e8b22d0 2024-09-16T01:48:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf4dffb640 1 -- 172.21.15.139:0/1434883287 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf5c0133f0 con 0x55d78e8b22d0 2024-09-16T01:48:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf4dffb640 1 -- 172.21.15.139:0/1434883287 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7fbf54004bb0 con 0x55d78e8b22d0 2024-09-16T01:48:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf6f413740 10 client.16374 fetch_fsmap finished waiting for FSMap version 45 2024-09-16T01:48:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf6f413740 10 client.16374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:48:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf6f413740 20 client.16374 populate_metadata read hostname 'smithi139' 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf6f413740 1 --2- 172.21.15.139:0/1434883287 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d78e8d1230 0x55d78e8d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf6f413740 4 client.16374 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf6f413740 1 -- 172.21.15.139:0/1434883287 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d78e823a20 con 0x55d78e8d1230 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.266+0000 7fbf6a334640 1 --2- 172.21.15.139:0/1434883287 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d78e8d1230 0x55d78e8d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.267+0000 7fbf6a334640 1 --2- 172.21.15.139:0/1434883287 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d78e8d1230 0x55d78e8d3670 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2296c9c928d8d5e1 server_cookie=3f5e089a6f965075 in_seq=0 out_seq=0 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.267+0000 7fbf4dffb640 10 client.16374 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf4dffb640 1 -- 172.21.15.139:0/1434883287 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d78e823a20 con 0x55d78e8d1230 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf4dffb640 10 client.16374 handle_command_reply: tid=0 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf6f413740 1 client.16374 shutdown 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf6f413740 1 -- 172.21.15.139:0/1434883287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b22d0 msgr2=0x55d78e8be580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf6f413740 1 --2- 172.21.15.139:0/1434883287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b22d0 0x55d78e8be580 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009900 tx=0x7fbf54031860 comp rx=0 tx=0).stop 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf6f413740 1 -- 172.21.15.139:0/1434883287 shutdown_connections 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf6f413740 1 --2- 172.21.15.139:0/1434883287 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d78e8d1230 0x55d78e8d3670 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf6f413740 1 --2- 172.21.15.139:0/1434883287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 0x55d78e8beac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf6f413740 1 --2- 172.21.15.139:0/1434883287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b22d0 0x55d78e8be580 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf4dffb640 0 client.16374 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf6f413740 1 --2- 172.21.15.139:0/1434883287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b19b0 0x55d78e8be040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf6f413740 1 -- 172.21.15.139:0/1434883287 >> 172.21.15.139:0/1434883287 conn(0x55d78e8ae060 msgr2=0x55d78e8b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf6f413740 1 -- 172.21.15.139:0/1434883287 shutdown_connections 2024-09-16T01:48:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf6f413740 1 -- 172.21.15.139:0/1434883287 wait complete. 2024-09-16T01:48:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.268+0000 7fbf6f413740 20 client.16374 trim_cache size 0 max 16384 2024-09-16T01:48:29.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.273+0000 7f2de0ed3640 1 -- 172.21.15.139:0/512521375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ddc1071e0 con 0x7f2ddc103660 2024-09-16T01:48:29.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.274+0000 7f2dcb7fe640 1 -- 172.21.15.139:0/512521375 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+12026 (secure 0 0 0) 0x7f2dcc09cc10 con 0x7f2ddc103660 2024-09-16T01:48:29.277 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:48:29.277 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":45,"btime":"2024-09-16T01:48:18:676586+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:48:18.356529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:48:29.278 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 45 2024-09-16T01:48:29.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.277+0000 7f2de0ed3640 1 -- 172.21.15.139:0/512521375 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2dc40a33b0 msgr2=0x7f2dc40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:29.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.277+0000 7f2de0ed3640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2dc40a33b0 0x7f2dc40a5870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009b60 tx=0x7f2dd0005f90 comp rx=0 tx=0).stop 2024-09-16T01:48:29.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.277+0000 7f2de0ed3640 1 -- 172.21.15.139:0/512521375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ddc103660 msgr2=0x7f2ddc198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:29.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.277+0000 7f2de0ed3640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ddc103660 0x7f2ddc198980 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00ea00 tx=0x7f2dcc00eec0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.277+0000 7f2de0ed3640 1 -- 172.21.15.139:0/512521375 shutdown_connections 2024-09-16T01:48:29.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.277+0000 7f2de0ed3640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2dc40a33b0 0x7f2dc40a5870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.277+0000 7f2de0ed3640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ddc10c170 0x7f2ddc19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.277+0000 7f2de0ed3640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ddc103660 0x7f2ddc198980 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.277+0000 7f2de0ed3640 1 --2- 172.21.15.139:0/512521375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ddc102d40 0x7f2ddc198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.277+0000 7f2de0ed3640 1 -- 172.21.15.139:0/512521375 >> 172.21.15.139:0/512521375 conn(0x7f2ddc0fc7a0 msgr2=0x7f2ddc0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:29.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.277+0000 7f2de0ed3640 1 -- 172.21.15.139:0/512521375 shutdown_connections 2024-09-16T01:48:29.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:29.277+0000 7f2de0ed3640 1 -- 172.21.15.139:0/512521375 wait complete. 2024-09-16T01:48:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.283+0000 7fbf6f413740 1 Processor -- start 2024-09-16T01:48:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.283+0000 7fbf6f413740 1 -- start start 2024-09-16T01:48:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6f413740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8b19b0 0x55d78e8bc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6f413740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b22d0 0x55d78e8be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6f413740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8c10c0 0x55d78e8d8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6f413740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbf641acfd0 con 0x55d78e8c10c0 2024-09-16T01:48:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6f413740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbf641aedb0 con 0x55d78e8b19b0 2024-09-16T01:48:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6f413740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbf641ad760 con 0x55d78e8b22d0 2024-09-16T01:48:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6a334640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8b19b0 0x55d78e8bc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6ab35640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8c10c0 0x55d78e8d8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6a334640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8b19b0 0x55d78e8bc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:48178/0 (socket says 172.21.15.139:48178) 2024-09-16T01:48:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6ab35640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8c10c0 0x55d78e8d8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45834/0 (socket says 172.21.15.139:45834) 2024-09-16T01:48:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6a334640 1 -- 172.21.15.139:0/784135233 learned_addr learned my addr 172.21.15.139:0/784135233 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf69b33640 1 --2- 172.21.15.139:0/784135233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b22d0 0x55d78e8be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6a334640 1 -- 172.21.15.139:0/784135233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b22d0 msgr2=0x55d78e8be8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6a334640 1 --2- 172.21.15.139:0/784135233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b22d0 0x55d78e8be8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6a334640 1 -- 172.21.15.139:0/784135233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8c10c0 msgr2=0x55d78e8d8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6a334640 1 --2- 172.21.15.139:0/784135233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8c10c0 0x55d78e8d8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6a334640 1 -- 172.21.15.139:0/784135233 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d78e8add30 con 0x55d78e8b19b0 2024-09-16T01:48:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.284+0000 7fbf6a334640 1 --2- 172.21.15.139:0/784135233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8b19b0 0x55d78e8bc910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbf6009bc90 tx=0x7fbf6000bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13e18477a5f9c716 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.285+0000 7fbf4ffff640 1 -- 172.21.15.139:0/784135233 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60020040 con 0x55d78e8b19b0 2024-09-16T01:48:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.285+0000 7fbf4ffff640 1 -- 172.21.15.139:0/784135233 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf6009a000 con 0x55d78e8b19b0 2024-09-16T01:48:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.285+0000 7fbf4ffff640 1 -- 172.21.15.139:0/784135233 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6009b3b0 con 0x55d78e8b19b0 2024-09-16T01:48:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.285+0000 7fbf6f413740 1 -- 172.21.15.139:0/784135233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8b19b0 msgr2=0x55d78e8bc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.285+0000 7fbf6f413740 1 --2- 172.21.15.139:0/784135233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8b19b0 0x55d78e8bc910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbf6009bc90 tx=0x7fbf6000bf60 comp rx=0 tx=0).stop 2024-09-16T01:48:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.285+0000 7fbf6f413740 1 -- 172.21.15.139:0/784135233 shutdown_connections 2024-09-16T01:48:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.285+0000 7fbf6f413740 1 --2- 172.21.15.139:0/784135233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8c10c0 0x55d78e8d8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.285+0000 7fbf6f413740 1 --2- 172.21.15.139:0/784135233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b22d0 0x55d78e8be8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.285+0000 7fbf6f413740 1 --2- 172.21.15.139:0/784135233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8b19b0 0x55d78e8bc910 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.285+0000 7fbf6f413740 1 -- 172.21.15.139:0/784135233 >> 172.21.15.139:0/784135233 conn(0x55d78e8ae060 msgr2=0x55d78e8c00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf6f413740 1 -- 172.21.15.139:0/784135233 shutdown_connections 2024-09-16T01:48:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf6f413740 1 -- 172.21.15.139:0/784135233 wait complete. 2024-09-16T01:48:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf6f413740 1 -- start start 2024-09-16T01:48:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf6f413740 1 Processor -- start 2024-09-16T01:48:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf6f413740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b19b0 0x55d78e8c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf6f413740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b22d0 0x55d78e8c9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf6f413740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 0x55d78e8c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf6f413740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbf641ae620 con 0x55d78e8b22d0 2024-09-16T01:48:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf6f413740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbf641adec0 con 0x55d78e8c10c0 2024-09-16T01:48:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf6f413740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbf641aafd0 con 0x55d78e8b19b0 2024-09-16T01:48:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf6f413740 10 client.0 _ll_register_callbacks cb 0x7fbf54009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:48:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf69b33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b22d0 0x55d78e8c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf69b33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b22d0 0x55d78e8c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45838/0 (socket says 172.21.15.139:45838) 2024-09-16T01:48:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf69b33640 1 -- 172.21.15.139:0/2153179699 learned_addr learned my addr 172.21.15.139:0/2153179699 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf6a334640 1 --2- 172.21.15.139:0/2153179699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b19b0 0x55d78e8c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.286+0000 7fbf6ab35640 1 --2- 172.21.15.139:0/2153179699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 0x55d78e8c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf69b33640 1 -- 172.21.15.139:0/2153179699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b19b0 msgr2=0x55d78e8c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf69b33640 1 --2- 172.21.15.139:0/2153179699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b19b0 0x55d78e8c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf69b33640 1 -- 172.21.15.139:0/2153179699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 msgr2=0x55d78e8c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf69b33640 1 --2- 172.21.15.139:0/2153179699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 0x55d78e8c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf69b33640 1 -- 172.21.15.139:0/2153179699 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d78e8c65e0 con 0x55d78e8b22d0 2024-09-16T01:48:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf69b33640 1 --2- 172.21.15.139:0/2153179699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b22d0 0x55d78e8c9610 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009850 tx=0x7fbf54005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae82cf1561f5f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:48:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf4dffb640 1 -- 172.21.15.139:0/2153179699 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54035070 con 0x55d78e8b22d0 2024-09-16T01:48:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf6f413740 1 -- 172.21.15.139:0/2153179699 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf6009a000 con 0x55d78e8b22d0 2024-09-16T01:48:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf4dffb640 1 -- 172.21.15.139:0/2153179699 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9554+0+0 (secure 0 0 0) 0x7fbf540048c0 con 0x55d78e8b22d0 2024-09-16T01:48:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf4dffb640 1 -- 172.21.15.139:0/2153179699 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbf5405f050 con 0x55d78e8b22d0 2024-09-16T01:48:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf6f413740 10 client.16380 fetch_fsmap learned FSMap version 45 2024-09-16T01:48:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf6f413740 1 -- 172.21.15.139:0/2153179699 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d78e93c610 con 0x55d78e8b22d0 2024-09-16T01:48:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf4dffb640 1 -- 172.21.15.139:0/2153179699 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf6009a000 con 0x55d78e8b22d0 2024-09-16T01:48:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf4dffb640 1 -- 172.21.15.139:0/2153179699 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7fbf54004ac0 con 0x55d78e8b22d0 2024-09-16T01:48:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.287+0000 7fbf6f413740 10 client.16380 fetch_fsmap finished waiting for FSMap version 45 2024-09-16T01:48:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.288+0000 7fbf6f413740 10 client.16380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:48:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.288+0000 7fbf6f413740 20 client.16380 populate_metadata read hostname 'smithi139' 2024-09-16T01:48:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.288+0000 7fbf6f413740 1 --2- 172.21.15.139:0/2153179699 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d78e93ce40 0x55d78e93f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.288+0000 7fbf6f413740 4 client.16380 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:48:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.288+0000 7fbf6f413740 1 -- 172.21.15.139:0/2153179699 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d78e81dd70 con 0x55d78e93ce40 2024-09-16T01:48:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.288+0000 7fbf6a334640 1 --2- 172.21.15.139:0/2153179699 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d78e93ce40 0x55d78e93f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.288+0000 7fbf4dffb640 10 client.16380 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.288+0000 7fbf6a334640 1 --2- 172.21.15.139:0/2153179699 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d78e93ce40 0x55d78e93f260 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52281344c1733bd1 server_cookie=cb87ba40498df7c2 in_seq=0 out_seq=0 2024-09-16T01:48:29.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:28 smithi164 ceph-mon[35121]: pgmap v250: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 24 op/s 2024-09-16T01:48:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.303+0000 7fbf4dffb640 1 -- 172.21.15.139:0/2153179699 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d78e81dd70 con 0x55d78e93ce40 2024-09-16T01:48:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.303+0000 7fbf4dffb640 10 client.16380 handle_command_reply: tid=0 2024-09-16T01:48:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.303+0000 7fbf6f413740 1 client.16380 shutdown 2024-09-16T01:48:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.303+0000 7fbf6f413740 1 -- 172.21.15.139:0/2153179699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b22d0 msgr2=0x55d78e8c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.303+0000 7fbf6f413740 1 --2- 172.21.15.139:0/2153179699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b22d0 0x55d78e8c9610 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009850 tx=0x7fbf54005970 comp rx=0 tx=0).stop 2024-09-16T01:48:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.303+0000 7fbf6f413740 1 -- 172.21.15.139:0/2153179699 shutdown_connections 2024-09-16T01:48:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.303+0000 7fbf6f413740 1 --2- 172.21.15.139:0/2153179699 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d78e93ce40 0x55d78e93f260 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.303+0000 7fbf6f413740 1 --2- 172.21.15.139:0/2153179699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d78e8c10c0 0x55d78e8c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.303+0000 7fbf6f413740 1 --2- 172.21.15.139:0/2153179699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d78e8b22d0 0x55d78e8c9610 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.303+0000 7fbf4dffb640 0 client.16380 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.303+0000 7fbf6f413740 1 --2- 172.21.15.139:0/2153179699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d78e8b19b0 0x55d78e8c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.303+0000 7fbf6f413740 1 -- 172.21.15.139:0/2153179699 >> 172.21.15.139:0/2153179699 conn(0x55d78e8ae060 msgr2=0x55d78e8ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6f413740 1 -- 172.21.15.139:0/2153179699 shutdown_connections 2024-09-16T01:48:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6f413740 1 -- 172.21.15.139:0/2153179699 wait complete. 2024-09-16T01:48:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6f413740 20 client.16380 trim_cache size 0 max 16384 2024-09-16T01:48:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:48:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:48:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:48:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:48:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6c598640 1 -- 172.21.15.139:0/1371884948 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbf500a3460 msgr2=0x7fbf500a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6c598640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbf500a3460 0x7fbf500a5920 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c022000 tx=0x7fbf5c020000 comp rx=0 tx=0).stop 2024-09-16T01:48:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6c598640 1 -- 172.21.15.139:0/1371884948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf64102d20 msgr2=0x7fbf64198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6c598640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf64102d20 0x7fbf64198370 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000d950 tx=0x7fbf6000de10 comp rx=0 tx=0).stop 2024-09-16T01:48:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6c598640 1 -- 172.21.15.139:0/1371884948 shutdown_connections 2024-09-16T01:48:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6c598640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbf500a3460 0x7fbf500a5920 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6c598640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf6410c0f0 0x7fbf6419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6c598640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf64103640 0x7fbf641988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6c598640 1 --2- 172.21.15.139:0/1371884948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf64102d20 0x7fbf64198370 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6c598640 1 -- 172.21.15.139:0/1371884948 >> 172.21.15.139:0/1371884948 conn(0x7fbf64077990 msgr2=0x7fbf640fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6c598640 1 -- 172.21.15.139:0/1371884948 shutdown_connections 2024-09-16T01:48:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:29.304+0000 7fbf6c598640 1 -- 172.21.15.139:0/1371884948 wait complete. 2024-09-16T01:48:29.957 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 70ffc2a7 --release --await' 2024-09-16T01:48:29.957 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 70ffc2a7 --release --await 2024-09-16T01:48:29.980 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:29 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/512521375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:48:30.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:29 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/512521375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:48:30.205 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:48:30.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:29 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/512521375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:48:30.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.690+0000 7fba2f439640 1 -- 172.21.15.139:0/788248940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba28101020 msgr2=0x7fba2810d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:30.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.690+0000 7fba2f439640 1 --2- 172.21.15.139:0/788248940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba28101020 0x7fba2810d0e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fba18009ca0 tx=0x7fba1802a8c0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.690+0000 7fba2f439640 1 -- 172.21.15.139:0/788248940 shutdown_connections 2024-09-16T01:48:30.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.690+0000 7fba2f439640 1 --2- 172.21.15.139:0/788248940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba28101020 0x7fba2810d0e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.690+0000 7fba2f439640 1 --2- 172.21.15.139:0/788248940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba28100680 0x7fba28100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.690+0000 7fba2f439640 1 --2- 172.21.15.139:0/788248940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba28105220 0x7fba28105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.690+0000 7fba2f439640 1 -- 172.21.15.139:0/788248940 >> 172.21.15.139:0/788248940 conn(0x7fba28077210 msgr2=0x7fba28077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:30.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.690+0000 7fba2f439640 1 -- 172.21.15.139:0/788248940 shutdown_connections 2024-09-16T01:48:30.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.690+0000 7fba2f439640 1 -- 172.21.15.139:0/788248940 wait complete. 2024-09-16T01:48:30.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.690+0000 7fba2f439640 1 Processor -- start 2024-09-16T01:48:30.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.690+0000 7fba2f439640 1 -- start start 2024-09-16T01:48:30.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2f439640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba28100680 0x7fba28197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2f439640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba28101020 0x7fba28197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2f439640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba28105220 0x7fba2819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2f439640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fba28110fe0 con 0x7fba28101020 2024-09-16T01:48:30.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2f439640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fba28110e60 con 0x7fba28100680 2024-09-16T01:48:30.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2f439640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fba28111160 con 0x7fba28105220 2024-09-16T01:48:30.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2c9d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba28101020 0x7fba28197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2c9d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba28101020 0x7fba28197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45854/0 (socket says 172.21.15.139:45854) 2024-09-16T01:48:30.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2c9d4640 1 -- 172.21.15.139:0/3795425599 learned_addr learned my addr 172.21.15.139:0/3795425599 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:30.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2d9d6640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba28105220 0x7fba2819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2d1d5640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba28100680 0x7fba28197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2c9d4640 1 -- 172.21.15.139:0/3795425599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba28105220 msgr2=0x7fba2819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:30.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba28105220 0x7fba2819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2c9d4640 1 -- 172.21.15.139:0/3795425599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba28100680 msgr2=0x7fba28197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:30.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba28100680 0x7fba28197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2c9d4640 1 -- 172.21.15.139:0/3795425599 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2819c2d0 con 0x7fba28101020 2024-09-16T01:48:30.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2d9d6640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba28105220 0x7fba2819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:48:30.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2d1d5640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba28100680 0x7fba28197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:48:30.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba28101020 0x7fba28197860 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00c920 tx=0x7fba1c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:30.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba267fc640 1 -- 172.21.15.139:0/3795425599 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c007c70 con 0x7fba28101020 2024-09-16T01:48:30.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba267fc640 1 -- 172.21.15.139:0/3795425599 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1c007e10 con 0x7fba28101020 2024-09-16T01:48:30.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2f439640 1 -- 172.21.15.139:0/3795425599 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2819c5c0 con 0x7fba28101020 2024-09-16T01:48:30.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba267fc640 1 -- 172.21.15.139:0/3795425599 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c018890 con 0x7fba28101020 2024-09-16T01:48:30.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.691+0000 7fba2f439640 1 -- 172.21.15.139:0/3795425599 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba281a3ea0 con 0x7fba28101020 2024-09-16T01:48:30.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.694+0000 7fba267fc640 1 -- 172.21.15.139:0/3795425599 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fba1c018a70 con 0x7fba28101020 2024-09-16T01:48:30.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.694+0000 7fba31812740 1 Processor -- start 2024-09-16T01:48:30.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.694+0000 7fba31812740 1 -- start start 2024-09-16T01:48:30.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.694+0000 7fba31812740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 0x55fdfcb71ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.694+0000 7fba31812740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb724b0 0x55fdfcb72910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.694+0000 7fba31812740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb81300 0x55fdfcb83710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.694+0000 7fba31812740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55fdfcad7620 con 0x55fdfcb81300 2024-09-16T01:48:30.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.694+0000 7fba31812740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fdfca7ee50 con 0x55fdfcb724b0 2024-09-16T01:48:30.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.694+0000 7fba31812740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fdfcad35e0 con 0x55fdfcb736f0 2024-09-16T01:48:30.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.694+0000 7fba267fc640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fba100a33b0 0x7fba100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba2c9d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb81300 0x55fdfcb83710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba2d9d6640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fba100a33b0 0x7fba100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba2c9d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb81300 0x55fdfcb83710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45856/0 (socket says 172.21.15.139:45856) 2024-09-16T01:48:30.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba2c9d4640 1 -- 172.21.15.139:0/2401856241 learned_addr learned my addr 172.21.15.139:0/2401856241 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:30.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba2d9d6640 1 --2- 172.21.15.139:0/2401856241 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 0x55fdfcb71ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba2d1d5640 1 --2- 172.21.15.139:0/2401856241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb724b0 0x55fdfcb72910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba267fc640 1 -- 172.21.15.139:0/3795425599 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9554+0+0 (secure 0 0 0) 0x7fba1c0d0330 con 0x7fba28101020 2024-09-16T01:48:30.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba2c9d4640 1 -- 172.21.15.139:0/2401856241 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 msgr2=0x55fdfcb71ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:30.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/2401856241 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 0x55fdfcb71ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba2c9d4640 1 -- 172.21.15.139:0/2401856241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb724b0 msgr2=0x55fdfcb72910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:30.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/2401856241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb724b0 0x55fdfcb72910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba2c9d4640 1 -- 172.21.15.139:0/2401856241 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdfca6fd40 con 0x55fdfcb81300 2024-09-16T01:48:30.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba2d9d6640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fba100a33b0 0x7fba100a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba1802f540 tx=0x7fba18009880 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:30.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/2401856241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb81300 0x55fdfcb83710 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0a4940 tx=0x7fba1c06f030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8444fde18a1708e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:30.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba0f7fe640 1 -- 172.21.15.139:0/2401856241 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c09c900 con 0x55fdfcb81300 2024-09-16T01:48:30.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba0f7fe640 1 -- 172.21.15.139:0/2401856241 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1c067050 con 0x55fdfcb81300 2024-09-16T01:48:30.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.695+0000 7fba0f7fe640 1 -- 172.21.15.139:0/2401856241 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c018d60 con 0x55fdfcb81300 2024-09-16T01:48:30.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.696+0000 7fba31812740 1 -- 172.21.15.139:0/2401856241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb81300 msgr2=0x55fdfcb83710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:30.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.696+0000 7fba31812740 1 --2- 172.21.15.139:0/2401856241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb81300 0x55fdfcb83710 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0a4940 tx=0x7fba1c06f030 comp rx=0 tx=0).stop 2024-09-16T01:48:30.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.696+0000 7fba31812740 1 -- 172.21.15.139:0/2401856241 shutdown_connections 2024-09-16T01:48:30.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.696+0000 7fba31812740 1 --2- 172.21.15.139:0/2401856241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb81300 0x55fdfcb83710 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.696+0000 7fba31812740 1 --2- 172.21.15.139:0/2401856241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb724b0 0x55fdfcb72910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.696+0000 7fba31812740 1 --2- 172.21.15.139:0/2401856241 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 0x55fdfcb71ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.696+0000 7fba31812740 1 -- 172.21.15.139:0/2401856241 >> 172.21.15.139:0/2401856241 conn(0x55fdfcb6e5f0 msgr2=0x55fdfcb6e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:30.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.696+0000 7fba31812740 1 -- 172.21.15.139:0/2401856241 shutdown_connections 2024-09-16T01:48:30.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.696+0000 7fba31812740 1 -- 172.21.15.139:0/2401856241 wait complete. 2024-09-16T01:48:30.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.696+0000 7fba31812740 1 -- start start 2024-09-16T01:48:30.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba31812740 1 Processor -- start 2024-09-16T01:48:30.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba31812740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb724b0 0x55fdfcb7e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba31812740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 0x55fdfcb7e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba31812740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 0x55fdfcb7ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba31812740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55fdfcb84230 con 0x55fdfcb724b0 2024-09-16T01:48:30.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba31812740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fdfca5ff10 con 0x55fdfcb81300 2024-09-16T01:48:30.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba31812740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fdfcb843b0 con 0x55fdfcb736f0 2024-09-16T01:48:30.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba31812740 10 client.0 _ll_register_callbacks cb 0x55fdfca83db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:48:30.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba2d1d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb724b0 0x55fdfcb7e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba2d1d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb724b0 0x55fdfcb7e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45866/0 (socket says 172.21.15.139:45866) 2024-09-16T01:48:30.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba2d1d5640 1 -- 172.21.15.139:0/3345176349 learned_addr learned my addr 172.21.15.139:0/3345176349 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:30.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba2d9d6640 1 --2- 172.21.15.139:0/3345176349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 0x55fdfcb7ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/3345176349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 0x55fdfcb7e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba2d1d5640 1 -- 172.21.15.139:0/3345176349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 msgr2=0x55fdfcb7e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:30.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba2d1d5640 1 --2- 172.21.15.139:0/3345176349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 0x55fdfcb7e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba2d1d5640 1 -- 172.21.15.139:0/3345176349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 msgr2=0x55fdfcb7ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:30.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba2d1d5640 1 --2- 172.21.15.139:0/3345176349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 0x55fdfcb7ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba2d1d5640 1 -- 172.21.15.139:0/3345176349 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdfc7e6580 con 0x55fdfcb724b0 2024-09-16T01:48:30.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.697+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/3345176349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 0x55fdfcb7e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:48:30.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba2d1d5640 1 --2- 172.21.15.139:0/3345176349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb724b0 0x55fdfcb7e2a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fba20010750 tx=0x7fba20010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e60ad3020ce755 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:30.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:48:30.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba0cff9640 1 -- 172.21.15.139:0/3345176349 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2001d060 con 0x55fdfcb724b0 2024-09-16T01:48:30.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba0cff9640 1 -- 172.21.15.139:0/3345176349 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9554+0+0 (secure 0 0 0) 0x7fba2000eda0 con 0x55fdfcb724b0 2024-09-16T01:48:30.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba31812740 1 -- 172.21.15.139:0/3345176349 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba1c067050 con 0x55fdfcb724b0 2024-09-16T01:48:30.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba0cff9640 1 -- 172.21.15.139:0/3345176349 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fba20024050 con 0x55fdfcb724b0 2024-09-16T01:48:30.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba31812740 10 client.16392 fetch_fsmap learned FSMap version 45 2024-09-16T01:48:30.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba31812740 1 -- 172.21.15.139:0/3345176349 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55fdfc8b0960 con 0x55fdfcb724b0 2024-09-16T01:48:30.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba0cff9640 1 -- 172.21.15.139:0/3345176349 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1c067050 con 0x55fdfcb724b0 2024-09-16T01:48:30.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba0cff9640 1 -- 172.21.15.139:0/3345176349 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7fba200164f0 con 0x55fdfcb724b0 2024-09-16T01:48:30.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba31812740 10 client.16392 fetch_fsmap finished waiting for FSMap version 45 2024-09-16T01:48:30.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba31812740 10 client.16392 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:48:30.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba31812740 20 client.16392 populate_metadata read hostname 'smithi139' 2024-09-16T01:48:30.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba31812740 1 --2- 172.21.15.139:0/3345176349 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fdfcb913b0 0x55fdfcb93850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba31812740 4 client.16392 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:48:30.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.698+0000 7fba31812740 1 -- 172.21.15.139:0/3345176349 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdfcae4b90 con 0x55fdfcb913b0 2024-09-16T01:48:30.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.699+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/3345176349 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fdfcb913b0 0x55fdfcb93850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.699+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/3345176349 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fdfcb913b0 0x55fdfcb93850 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22b59911f5d31c1b server_cookie=c8b812b936cd72da in_seq=0 out_seq=0 2024-09-16T01:48:30.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.699+0000 7fba0cff9640 10 client.16392 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:30.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba0cff9640 1 -- 172.21.15.139:0/3345176349 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fdfcae4b90 con 0x55fdfcb913b0 2024-09-16T01:48:30.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba0cff9640 10 client.16392 handle_command_reply: tid=0 2024-09-16T01:48:30.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba31812740 1 client.16392 shutdown 2024-09-16T01:48:30.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba31812740 1 -- 172.21.15.139:0/3345176349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb724b0 msgr2=0x55fdfcb7e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:30.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba31812740 1 --2- 172.21.15.139:0/3345176349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb724b0 0x55fdfcb7e2a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fba20010750 tx=0x7fba20010c10 comp rx=0 tx=0).stop 2024-09-16T01:48:30.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba31812740 1 -- 172.21.15.139:0/3345176349 shutdown_connections 2024-09-16T01:48:30.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba31812740 1 --2- 172.21.15.139:0/3345176349 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fdfcb913b0 0x55fdfcb93850 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba31812740 1 --2- 172.21.15.139:0/3345176349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 0x55fdfcb7ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba31812740 1 --2- 172.21.15.139:0/3345176349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 0x55fdfcb7e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba31812740 1 --2- 172.21.15.139:0/3345176349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb724b0 0x55fdfcb7e2a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba31812740 1 -- 172.21.15.139:0/3345176349 >> 172.21.15.139:0/3345176349 conn(0x55fdfcb6e5f0 msgr2=0x55fdfcb74350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:30.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba0cff9640 0 client.16392 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:30.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba31812740 1 -- 172.21.15.139:0/3345176349 shutdown_connections 2024-09-16T01:48:30.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba31812740 1 -- 172.21.15.139:0/3345176349 wait complete. 2024-09-16T01:48:30.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.700+0000 7fba31812740 20 client.16392 trim_cache size 0 max 16384 2024-09-16T01:48:30.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.717+0000 7fba31812740 1 Processor -- start 2024-09-16T01:48:30.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.717+0000 7fba31812740 1 -- start start 2024-09-16T01:48:30.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.717+0000 7fba31812740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb724b0 0x55fdfcb835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.717+0000 7fba31812740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb736f0 0x55fdfcb6cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.717+0000 7fba31812740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 0x55fdfcb6d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba31812740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fba281acf20 con 0x55fdfcb736f0 2024-09-16T01:48:30.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba31812740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fba281aed00 con 0x55fdfcb81300 2024-09-16T01:48:30.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba31812740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fba281ad6b0 con 0x55fdfcb724b0 2024-09-16T01:48:30.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba2c9d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb736f0 0x55fdfcb6cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba2c9d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb736f0 0x55fdfcb6cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45874/0 (socket says 172.21.15.139:45874) 2024-09-16T01:48:30.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba2c9d4640 1 -- 172.21.15.139:0/2128135094 learned_addr learned my addr 172.21.15.139:0/2128135094 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:30.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba2d9d6640 1 --2- 172.21.15.139:0/2128135094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 0x55fdfcb6d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba2d1d5640 1 --2- 172.21.15.139:0/2128135094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb724b0 0x55fdfcb835b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba2c9d4640 1 -- 172.21.15.139:0/2128135094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb724b0 msgr2=0x55fdfcb835b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:30.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/2128135094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb724b0 0x55fdfcb835b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba2c9d4640 1 -- 172.21.15.139:0/2128135094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 msgr2=0x55fdfcb6d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:30.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/2128135094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 0x55fdfcb6d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba2c9d4640 1 -- 172.21.15.139:0/2128135094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdfcb6f2c0 con 0x55fdfcb736f0 2024-09-16T01:48:30.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba2d1d5640 1 --2- 172.21.15.139:0/2128135094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb724b0 0x55fdfcb835b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:48:30.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/2128135094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb736f0 0x55fdfcb6cf30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0cdc70 tx=0x7fba1c0c6fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33679a83e004ee2f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:30.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba0effd640 1 -- 172.21.15.139:0/2128135094 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0b9020 con 0x55fdfcb736f0 2024-09-16T01:48:30.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba0effd640 1 -- 172.21.15.139:0/2128135094 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1c00de20 con 0x55fdfcb736f0 2024-09-16T01:48:30.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.718+0000 7fba0effd640 1 -- 172.21.15.139:0/2128135094 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c09a590 con 0x55fdfcb736f0 2024-09-16T01:48:30.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.719+0000 7fba31812740 1 -- 172.21.15.139:0/2128135094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb736f0 msgr2=0x55fdfcb6cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:30.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.719+0000 7fba31812740 1 --2- 172.21.15.139:0/2128135094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb736f0 0x55fdfcb6cf30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0cdc70 tx=0x7fba1c0c6fd0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.719+0000 7fba31812740 1 -- 172.21.15.139:0/2128135094 shutdown_connections 2024-09-16T01:48:30.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.719+0000 7fba31812740 1 --2- 172.21.15.139:0/2128135094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 0x55fdfcb6d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.719+0000 7fba31812740 1 --2- 172.21.15.139:0/2128135094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb736f0 0x55fdfcb6cf30 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.719+0000 7fba31812740 1 --2- 172.21.15.139:0/2128135094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb724b0 0x55fdfcb835b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.719+0000 7fba31812740 1 -- 172.21.15.139:0/2128135094 >> 172.21.15.139:0/2128135094 conn(0x55fdfcb6e5f0 msgr2=0x55fdfcb82ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:30.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.719+0000 7fba31812740 1 -- 172.21.15.139:0/2128135094 shutdown_connections 2024-09-16T01:48:30.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.719+0000 7fba31812740 1 -- 172.21.15.139:0/2128135094 wait complete. 2024-09-16T01:48:30.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.719+0000 7fba31812740 1 -- start start 2024-09-16T01:48:30.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba31812740 1 Processor -- start 2024-09-16T01:48:30.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba31812740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb724b0 0x55fdfcb89820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba31812740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 0x55fdfcb89d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba31812740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 0x55fdfcb98110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba31812740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fba281ae570 con 0x55fdfcb724b0 2024-09-16T01:48:30.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba31812740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fba281ade10 con 0x55fdfcb81300 2024-09-16T01:48:30.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba31812740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fba281aaf20 con 0x55fdfcb736f0 2024-09-16T01:48:30.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba31812740 10 client.0 _ll_register_callbacks cb 0x7fba2000bef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:48:30.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba2d1d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb724b0 0x55fdfcb89820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba2d1d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb724b0 0x55fdfcb89820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45890/0 (socket says 172.21.15.139:45890) 2024-09-16T01:48:30.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba2d1d5640 1 -- 172.21.15.139:0/1713590626 learned_addr learned my addr 172.21.15.139:0/1713590626 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:30.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/1713590626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 0x55fdfcb89d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba2d9d6640 1 --2- 172.21.15.139:0/1713590626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 0x55fdfcb98110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba2d1d5640 1 -- 172.21.15.139:0/1713590626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 msgr2=0x55fdfcb89d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:30.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.720+0000 7fba2d1d5640 1 --2- 172.21.15.139:0/1713590626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 0x55fdfcb89d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba2d1d5640 1 -- 172.21.15.139:0/1713590626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 msgr2=0x55fdfcb98110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:30.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba2d1d5640 1 --2- 172.21.15.139:0/1713590626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 0x55fdfcb98110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:30.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba2d1d5640 1 -- 172.21.15.139:0/1713590626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdfcb98800 con 0x55fdfcb724b0 2024-09-16T01:48:30.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba2d1d5640 1 --2- 172.21.15.139:0/1713590626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb724b0 0x55fdfcb89820 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fba20010720 tx=0x7fba20016d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda745dbab1d3972 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:30.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:48:30.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba0cff9640 1 -- 172.21.15.139:0/1713590626 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20010e60 con 0x55fdfcb724b0 2024-09-16T01:48:30.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba0cff9640 1 -- 172.21.15.139:0/1713590626 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9554+0+0 (secure 0 0 0) 0x7fba20016330 con 0x55fdfcb724b0 2024-09-16T01:48:30.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba31812740 1 -- 172.21.15.139:0/1713590626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba1c00de20 con 0x55fdfcb724b0 2024-09-16T01:48:30.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba0cff9640 1 -- 172.21.15.139:0/1713590626 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fba2001d020 con 0x55fdfcb724b0 2024-09-16T01:48:30.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba31812740 10 client.16398 fetch_fsmap learned FSMap version 45 2024-09-16T01:48:30.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba31812740 1 -- 172.21.15.139:0/1713590626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55fdfcb99ba0 con 0x55fdfcb724b0 2024-09-16T01:48:30.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba0cff9640 1 -- 172.21.15.139:0/1713590626 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1c00de20 con 0x55fdfcb724b0 2024-09-16T01:48:30.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba0cff9640 1 -- 172.21.15.139:0/1713590626 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7fba2002e020 con 0x55fdfcb724b0 2024-09-16T01:48:30.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba31812740 10 client.16398 fetch_fsmap finished waiting for FSMap version 45 2024-09-16T01:48:30.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba31812740 10 client.16398 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:48:30.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba31812740 20 client.16398 populate_metadata read hostname 'smithi139' 2024-09-16T01:48:30.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba31812740 1 --2- 172.21.15.139:0/1713590626 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fdfcbfd570 0x55fdfcbff990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:30.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba31812740 4 client.16398 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "70ffc2a7", "release": true, "await": true}] 2024-09-16T01:48:30.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.721+0000 7fba31812740 1 -- 172.21.15.139:0/1713590626 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "70ffc2a7", "release": true, "await": true}) -- 0x55fdfcadf0a0 con 0x55fdfcbfd570 2024-09-16T01:48:30.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.722+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/1713590626 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fdfcbfd570 0x55fdfcbff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:30.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.722+0000 7fba2c9d4640 1 --2- 172.21.15.139:0/1713590626 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fdfcbfd570 0x55fdfcbff990 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e5646694b2e819 server_cookie=4e1b9f51b68418c6 in_seq=0 out_seq=0 2024-09-16T01:48:30.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:30.722+0000 7fba0cff9640 10 client.16398 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:31.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:30 smithi164 ceph-mon[35121]: pgmap v251: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 21 op/s 2024-09-16T01:48:31.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:30 smithi139 ceph-mon[31175]: pgmap v251: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 21 op/s 2024-09-16T01:48:31.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:30 smithi176 ceph-mon[35324]: pgmap v251: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 21 op/s 2024-09-16T01:48:31.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.667+0000 7fba0cff9640 1 -- 172.21.15.139:0/1713590626 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fdfcadf0a0 con 0x55fdfcbfd570 2024-09-16T01:48:31.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.667+0000 7fba0cff9640 10 client.16398 handle_command_reply: tid=0 2024-09-16T01:48:31.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.668+0000 7fba31812740 1 client.16398 shutdown 2024-09-16T01:48:31.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.668+0000 7fba31812740 1 -- 172.21.15.139:0/1713590626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb724b0 msgr2=0x55fdfcb89820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:31.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.668+0000 7fba31812740 1 --2- 172.21.15.139:0/1713590626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb724b0 0x55fdfcb89820 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fba20010720 tx=0x7fba20016d60 comp rx=0 tx=0).stop 2024-09-16T01:48:31.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.668+0000 7fba31812740 1 -- 172.21.15.139:0/1713590626 shutdown_connections 2024-09-16T01:48:31.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.668+0000 7fba31812740 1 --2- 172.21.15.139:0/1713590626 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55fdfcbfd570 0x55fdfcbff990 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:31.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.668+0000 7fba31812740 1 --2- 172.21.15.139:0/1713590626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fdfcb81300 0x55fdfcb98110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:31.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.668+0000 7fba31812740 1 --2- 172.21.15.139:0/1713590626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fdfcb736f0 0x55fdfcb89d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:31.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.668+0000 7fba31812740 1 --2- 172.21.15.139:0/1713590626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fdfcb724b0 0x55fdfcb89820 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:31.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.668+0000 7fba31812740 1 -- 172.21.15.139:0/1713590626 >> 172.21.15.139:0/1713590626 conn(0x55fdfcb6e5f0 msgr2=0x55fdfcb7fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:31.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.668+0000 7fba0cff9640 0 client.16398 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:31.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.668+0000 7fba31812740 1 -- 172.21.15.139:0/1713590626 shutdown_connections 2024-09-16T01:48:31.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.668+0000 7fba31812740 1 -- 172.21.15.139:0/1713590626 wait complete. 2024-09-16T01:48:31.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.668+0000 7fba31812740 20 client.16398 trim_cache size 0 max 16384 2024-09-16T01:48:31.671 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:48:31.671 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 45, 2024-09-16T01:48:31.671 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:48:31.671 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 28, 2024-09-16T01:48:31.671 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:48:31.671 INFO:teuthology.orchestra.run.smithi139.stdout: "70ffc2a7": { 2024-09-16T01:48:31.671 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 28, 2024-09-16T01:48:31.671 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:48:31.671 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:48:31.671 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:48:31.671 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:48:31.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:48:31.672 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:48:31.672 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 143.7, 2024-09-16T01:48:31.672 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:48:31.672 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:48:31.672 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:48:31.672 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:48:31.672 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:48:31.672 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:48:31.672 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:48:31.672 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:48:31.672 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:48:31.672 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:48:31.672 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:48:31.672 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:48:31.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.669+0000 7fba2f439640 1 -- 172.21.15.139:0/3795425599 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fba100a33b0 msgr2=0x7fba100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:31.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.669+0000 7fba2f439640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fba100a33b0 0x7fba100a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba1802f540 tx=0x7fba18009880 comp rx=0 tx=0).stop 2024-09-16T01:48:31.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.669+0000 7fba2f439640 1 -- 172.21.15.139:0/3795425599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba28101020 msgr2=0x7fba28197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:31.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.669+0000 7fba2f439640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba28101020 0x7fba28197860 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00c920 tx=0x7fba1c00cde0 comp rx=0 tx=0).stop 2024-09-16T01:48:31.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.669+0000 7fba2f439640 1 -- 172.21.15.139:0/3795425599 shutdown_connections 2024-09-16T01:48:31.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.669+0000 7fba2f439640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fba100a33b0 0x7fba100a5870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:31.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.669+0000 7fba2f439640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba28105220 0x7fba2819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:31.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.669+0000 7fba2f439640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba28101020 0x7fba28197860 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:31.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.669+0000 7fba2f439640 1 --2- 172.21.15.139:0/3795425599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba28100680 0x7fba28197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:31.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.669+0000 7fba2f439640 1 -- 172.21.15.139:0/3795425599 >> 172.21.15.139:0/3795425599 conn(0x7fba28077210 msgr2=0x7fba280ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:31.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.670+0000 7fba2f439640 1 -- 172.21.15.139:0/3795425599 shutdown_connections 2024-09-16T01:48:31.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:48:31.670+0000 7fba2f439640 1 -- 172.21.15.139:0/3795425599 wait complete. 2024-09-16T01:48:32.654 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '70ffc2a7', total seconds elapsed: 28.0 2024-09-16T01:48:33.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:32 smithi164 ceph-mon[35121]: pgmap v252: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-16T01:48:33.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:32 smithi139 ceph-mon[31175]: pgmap v252: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-16T01:48:33.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:32 smithi176 ceph-mon[35324]: pgmap v252: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-16T01:48:35.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:34 smithi164 ceph-mon[35121]: pgmap v253: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 184 KiB/s wr, 15 op/s 2024-09-16T01:48:35.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:34 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:34 smithi139 ceph-mon[31175]: pgmap v253: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 184 KiB/s wr, 15 op/s 2024-09-16T01:48:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:34 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:35.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:34 smithi176 ceph-mon[35324]: pgmap v253: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 184 KiB/s wr, 15 op/s 2024-09-16T01:48:35.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:34 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:37.225 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:36 smithi139 ceph-mon[31175]: pgmap v254: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 566 KiB/s wr, 46 op/s 2024-09-16T01:48:37.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:36 smithi164 ceph-mon[35121]: pgmap v254: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 566 KiB/s wr, 46 op/s 2024-09-16T01:48:37.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:36 smithi176 ceph-mon[35324]: pgmap v254: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 566 KiB/s wr, 46 op/s 2024-09-16T01:48:39.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:38 smithi139 ceph-mon[31175]: pgmap v255: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 813 KiB/s wr, 95 op/s 2024-09-16T01:48:39.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:38 smithi164 ceph-mon[35121]: pgmap v255: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 813 KiB/s wr, 95 op/s 2024-09-16T01:48:39.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:38 smithi176 ceph-mon[35324]: pgmap v255: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 813 KiB/s wr, 95 op/s 2024-09-16T01:48:40.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:39 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:40.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:39 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:40.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:39 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:41.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:40 smithi164 ceph-mon[35121]: pgmap v256: 129 pgs: 129 active+clean; 2.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1010 KiB/s wr, 114 op/s 2024-09-16T01:48:41.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:40 smithi139 ceph-mon[31175]: pgmap v256: 129 pgs: 129 active+clean; 2.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1010 KiB/s wr, 114 op/s 2024-09-16T01:48:41.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:40 smithi176 ceph-mon[35324]: pgmap v256: 129 pgs: 129 active+clean; 2.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1010 KiB/s wr, 114 op/s 2024-09-16T01:48:42.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:41 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:48:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.396+0000 7f1c10d4b640 1 -- 172.21.15.139:0/3009628049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c0c101350 msgr2=0x7f1c0c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.396+0000 7f1c10d4b640 1 --2- 172.21.15.139:0/3009628049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c0c101350 0x7f1c0c10d880 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009c10 tx=0x7f1c0402a6e0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.396+0000 7f1c10d4b640 1 -- 172.21.15.139:0/3009628049 shutdown_connections 2024-09-16T01:48:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.396+0000 7f1c10d4b640 1 --2- 172.21.15.139:0/3009628049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c0c101350 0x7f1c0c10d880 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.396+0000 7f1c10d4b640 1 --2- 172.21.15.139:0/3009628049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c1009b0 0x7f1c0c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.396+0000 7f1c10d4b640 1 --2- 172.21.15.139:0/3009628049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c0c1069c0 0x7f1c0c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.396+0000 7f1c10d4b640 1 -- 172.21.15.139:0/3009628049 >> 172.21.15.139:0/3009628049 conn(0x7f1c0c077990 msgr2=0x7f1c0c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.396+0000 7f1c10d4b640 1 -- 172.21.15.139:0/3009628049 shutdown_connections 2024-09-16T01:48:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.396+0000 7f1c10d4b640 1 -- 172.21.15.139:0/3009628049 wait complete. 2024-09-16T01:48:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.397+0000 7f1c10d4b640 1 Processor -- start 2024-09-16T01:48:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.397+0000 7f1c10d4b640 1 -- start start 2024-09-16T01:48:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.397+0000 7f1c10d4b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c1009b0 0x7f1c0c1028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.397+0000 7f1c10d4b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c0c101350 0x7f1c0c102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.397+0000 7f1c10d4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c0c1069c0 0x7f1c0c103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.397+0000 7f1c10d4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c110db0 con 0x7f1c0c1069c0 2024-09-16T01:48:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.397+0000 7f1c10d4b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c110c30 con 0x7f1c0c1009b0 2024-09-16T01:48:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.397+0000 7f1c10d4b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c110f30 con 0x7f1c0c101350 2024-09-16T01:48:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.397+0000 7f1c0ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c0c1069c0 0x7f1c0c103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.397+0000 7f1c0a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c1009b0 0x7f1c0c1028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.397+0000 7f1c0a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c1009b0 0x7f1c0c1028f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:60818/0 (socket says 172.21.15.139:60818) 2024-09-16T01:48:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.397+0000 7f1c0ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c0c1069c0 0x7f1c0c103370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38738/0 (socket says 172.21.15.139:38738) 2024-09-16T01:48:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.397+0000 7f1c0a59c640 1 -- 172.21.15.139:0/2974347077 learned_addr learned my addr 172.21.15.139:0/2974347077 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.397+0000 7f1c09d9b640 1 --2- 172.21.15.139:0/2974347077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c0c101350 0x7f1c0c102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.398+0000 7f1c0a59c640 1 -- 172.21.15.139:0/2974347077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c0c101350 msgr2=0x7f1c0c102e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.398+0000 7f1c0a59c640 1 --2- 172.21.15.139:0/2974347077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c0c101350 0x7f1c0c102e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.398+0000 7f1c0a59c640 1 -- 172.21.15.139:0/2974347077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c0c1069c0 msgr2=0x7f1c0c103370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.398+0000 7f1c0a59c640 1 --2- 172.21.15.139:0/2974347077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c0c1069c0 0x7f1c0c103370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.398+0000 7f1c0a59c640 1 -- 172.21.15.139:0/2974347077 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0c1a6810 con 0x7f1c0c1009b0 2024-09-16T01:48:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.398+0000 7f1c0ad9d640 1 --2- 172.21.15.139:0/2974347077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c0c1069c0 0x7f1c0c103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:48:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.398+0000 7f1c0a59c640 1 --2- 172.21.15.139:0/2974347077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c1009b0 0x7f1c0c1028f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00e9f0 tx=0x7f1bfc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.398+0000 7f1bfb7fe640 1 -- 172.21.15.139:0/2974347077 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00ccc0 con 0x7f1c0c1009b0 2024-09-16T01:48:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.398+0000 7f1bfb7fe640 1 -- 172.21.15.139:0/2974347077 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bfc010910 con 0x7f1c0c1009b0 2024-09-16T01:48:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.398+0000 7f1c10d4b640 1 -- 172.21.15.139:0/2974347077 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0c1a6b00 con 0x7f1c0c1009b0 2024-09-16T01:48:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.398+0000 7f1bfb7fe640 1 -- 172.21.15.139:0/2974347077 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0188a0 con 0x7f1c0c1009b0 2024-09-16T01:48:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.398+0000 7f1c10d4b640 1 -- 172.21.15.139:0/2974347077 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0c1a7050 con 0x7f1c0c1009b0 2024-09-16T01:48:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1bfb7fe640 1 -- 172.21.15.139:0/2974347077 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1bfc010ac0 con 0x7f1c0c1009b0 2024-09-16T01:48:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1c13bc7740 1 Processor -- start 2024-09-16T01:48:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1c13bc7740 1 -- start start 2024-09-16T01:48:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1c13bc7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02101faf0 0x55d02101fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1c13bc7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d02102a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1c13bc7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 0x55d02102d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1c13bc7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d020f842f0 con 0x55d021020410 2024-09-16T01:48:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1c13bc7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d020f2c270 con 0x55d02102acb0 2024-09-16T01:48:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1c13bc7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d020f802b0 con 0x55d02101faf0 2024-09-16T01:48:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1c0a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d02102a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1c0ad9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02101faf0 0x55d02101fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1c09d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 0x55d02102d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1c0a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d02102a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38748/0 (socket says 172.21.15.139:38748) 2024-09-16T01:48:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1c0a59c640 1 -- 172.21.15.139:0/2108571917 learned_addr learned my addr 172.21.15.139:0/2108571917 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1c0ad9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02101faf0 0x55d02101fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:32896/0 (socket says 172.21.15.139:32896) 2024-09-16T01:48:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.401+0000 7f1bfb7fe640 1 --2- 172.21.15.139:0/2974347077 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1bf40a3460 0x7f1bf40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.402+0000 7f1c0ad9d640 1 --2- 172.21.15.139:0/2974347077 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1bf40a3460 0x7f1bf40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.402+0000 7f1c09d9b640 1 -- 172.21.15.139:0/2108571917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02101faf0 msgr2=0x55d02101fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.402+0000 7f1c09d9b640 1 --2- 172.21.15.139:0/2108571917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02101faf0 0x55d02101fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.402+0000 7f1c09d9b640 1 -- 172.21.15.139:0/2108571917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 msgr2=0x55d02102a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.402+0000 7f1c09d9b640 1 --2- 172.21.15.139:0/2108571917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d02102a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.402+0000 7f1c09d9b640 1 -- 172.21.15.139:0/2108571917 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d020f2b340 con 0x55d02102acb0 2024-09-16T01:48:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.402+0000 7f1c0a59c640 1 --2- 172.21.15.139:0/2108571917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d02102a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:48:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.402+0000 7f1bfb7fe640 1 -- 172.21.15.139:0/2974347077 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9554+0+0 (secure 0 0 0) 0x7f1bfc0d0480 con 0x7f1c0c1009b0 2024-09-16T01:48:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.402+0000 7f1c09d9b640 1 --2- 172.21.15.139:0/2108571917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 0x55d02102d0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1c00005fd0 tx=0x7f1c00011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b3a2ea7c4b2e926 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.402+0000 7f1c0ad9d640 1 --2- 172.21.15.139:0/2974347077 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1bf40a3460 0x7f1bf40a5920 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1c04004af0 tx=0x7f1c0402a6e0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.402+0000 7f1bf8ff9640 1 -- 172.21.15.139:0/2108571917 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0001e070 con 0x55d02102acb0 2024-09-16T01:48:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.402+0000 7f1bf8ff9640 1 -- 172.21.15.139:0/2108571917 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c000035f0 con 0x55d02102acb0 2024-09-16T01:48:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.402+0000 7f1bf8ff9640 1 -- 172.21.15.139:0/2108571917 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00019630 con 0x55d02102acb0 2024-09-16T01:48:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.403+0000 7f1c13bc7740 1 -- 172.21.15.139:0/2108571917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 msgr2=0x55d02102d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.403+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/2108571917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 0x55d02102d0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1c00005fd0 tx=0x7f1c00011810 comp rx=0 tx=0).stop 2024-09-16T01:48:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.403+0000 7f1c13bc7740 1 -- 172.21.15.139:0/2108571917 shutdown_connections 2024-09-16T01:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.403+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/2108571917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 0x55d02102d0a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.403+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/2108571917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d02102a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.403+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/2108571917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02101faf0 0x55d02101fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.403+0000 7f1c13bc7740 1 -- 172.21.15.139:0/2108571917 >> 172.21.15.139:0/2108571917 conn(0x55d02101c060 msgr2=0x55d02101c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.403+0000 7f1c13bc7740 1 -- 172.21.15.139:0/2108571917 shutdown_connections 2024-09-16T01:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.403+0000 7f1c13bc7740 1 -- 172.21.15.139:0/2108571917 wait complete. 2024-09-16T01:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.403+0000 7f1c13bc7740 1 -- start start 2024-09-16T01:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.403+0000 7f1c13bc7740 1 Processor -- start 2024-09-16T01:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c13bc7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02101faf0 0x55d021027b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c13bc7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d021020410 0x55d021028050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c13bc7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 0x55d021028590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c13bc7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d02102d880 con 0x55d02101faf0 2024-09-16T01:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c13bc7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d020f0d290 con 0x55d02102acb0 2024-09-16T01:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c13bc7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d02102da00 con 0x55d021020410 2024-09-16T01:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c13bc7740 10 client.0 _ll_register_callbacks cb 0x55d020f3e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c0a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02101faf0 0x55d021027b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c0a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02101faf0 0x55d021027b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38754/0 (socket says 172.21.15.139:38754) 2024-09-16T01:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c0a59c640 1 -- 172.21.15.139:0/820562151 learned_addr learned my addr 172.21.15.139:0/820562151 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c0ad9d640 1 --2- 172.21.15.139:0/820562151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 0x55d021028590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c09d9b640 1 --2- 172.21.15.139:0/820562151 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d021020410 0x55d021028050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c0a59c640 1 -- 172.21.15.139:0/820562151 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d021020410 msgr2=0x55d021028050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c0a59c640 1 --2- 172.21.15.139:0/820562151 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d021020410 0x55d021028050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c0a59c640 1 -- 172.21.15.139:0/820562151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 msgr2=0x55d021028590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c0a59c640 1 --2- 172.21.15.139:0/820562151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 0x55d021028590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c0a59c640 1 -- 172.21.15.139:0/820562151 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d020c97210 con 0x55d02101faf0 2024-09-16T01:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c09d9b640 1 --2- 172.21.15.139:0/820562151 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d021020410 0x55d021028050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c0a59c640 1 --2- 172.21.15.139:0/820562151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02101faf0 0x55d021027b10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc098d20 tx=0x7f1bfc009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a9184f6a525f69 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1bf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1bf1ffb640 1 -- 172.21.15.139:0/820562151 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0d3e20 con 0x55d02101faf0 2024-09-16T01:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1bf1ffb640 1 -- 172.21.15.139:0/820562151 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9554+0+0 (secure 0 0 0) 0x7f1bfc0a4b30 con 0x55d02101faf0 2024-09-16T01:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c13bc7740 1 -- 172.21.15.139:0/820562151 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c000035f0 con 0x55d02101faf0 2024-09-16T01:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1bf1ffb640 1 -- 172.21.15.139:0/820562151 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f1bfc09ca50 con 0x55d02101faf0 2024-09-16T01:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c13bc7740 10 client.16410 fetch_fsmap learned FSMap version 48 2024-09-16T01:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.404+0000 7f1c13bc7740 1 -- 172.21.15.139:0/820562151 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d020d63620 con 0x55d02101faf0 2024-09-16T01:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.405+0000 7f1bf1ffb640 1 -- 172.21.15.139:0/820562151 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c000035f0 con 0x55d02101faf0 2024-09-16T01:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.405+0000 7f1bf1ffb640 1 -- 172.21.15.139:0/820562151 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 48) ==== 8438+0+0 (secure 0 0 0) 0x7f1bfc09b7a0 con 0x55d02101faf0 2024-09-16T01:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.405+0000 7f1c13bc7740 10 client.16410 fetch_fsmap finished waiting for FSMap version 48 2024-09-16T01:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.405+0000 7f1c13bc7740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.405+0000 7f1c13bc7740 20 client.16410 populate_metadata read hostname 'smithi139' 2024-09-16T01:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.405+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/820562151 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d02103abf0 0x55d02103d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.405+0000 7f1c13bc7740 4 client.16410 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.405+0000 7f1c13bc7740 1 -- 172.21.15.139:0/820562151 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d020f91a20 con 0x55d02103abf0 2024-09-16T01:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.405+0000 7f1c09d9b640 1 --2- 172.21.15.139:0/820562151 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d02103abf0 0x55d02103d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.405+0000 7f1c09d9b640 1 --2- 172.21.15.139:0/820562151 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d02103abf0 0x55d02103d040 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68374711695a32b4 server_cookie=7f841c15f6912b67 in_seq=0 out_seq=0 2024-09-16T01:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.405+0000 7f1bf1ffb640 10 client.16410 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1bf1ffb640 1 -- 172.21.15.139:0/820562151 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d020f91a20 con 0x55d02103abf0 2024-09-16T01:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1bf1ffb640 10 client.16410 handle_command_reply: tid=0 2024-09-16T01:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1c13bc7740 1 client.16410 shutdown 2024-09-16T01:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1c13bc7740 1 -- 172.21.15.139:0/820562151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02101faf0 msgr2=0x55d021027b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/820562151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02101faf0 0x55d021027b10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc098d20 tx=0x7f1bfc009700 comp rx=0 tx=0).stop 2024-09-16T01:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1c13bc7740 1 -- 172.21.15.139:0/820562151 shutdown_connections 2024-09-16T01:48:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/820562151 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d02103abf0 0x55d02103d040 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/820562151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 0x55d021028590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/820562151 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d021020410 0x55d021028050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/820562151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d02101faf0 0x55d021027b10 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1bf1ffb640 0 client.16410 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1c13bc7740 1 -- 172.21.15.139:0/820562151 >> 172.21.15.139:0/820562151 conn(0x55d02101c060 msgr2=0x55d021021dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1c13bc7740 1 -- 172.21.15.139:0/820562151 shutdown_connections 2024-09-16T01:48:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1c13bc7740 1 -- 172.21.15.139:0/820562151 wait complete. 2024-09-16T01:48:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.406+0000 7f1c13bc7740 20 client.16410 trim_cache size 0 max 16384 2024-09-16T01:48:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.420+0000 7f1c13bc7740 1 Processor -- start 2024-09-16T01:48:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.420+0000 7f1c13bc7740 1 -- start start 2024-09-16T01:48:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.420+0000 7f1c13bc7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02101faf0 0x55d0210283c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.420+0000 7f1c13bc7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d021029a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.420+0000 7f1c13bc7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 0x55d021042950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.420+0000 7f1c13bc7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1b1300 con 0x55d021020410 2024-09-16T01:48:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.420+0000 7f1c13bc7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1b30e0 con 0x55d02102acb0 2024-09-16T01:48:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.420+0000 7f1c13bc7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1b1a90 con 0x55d02101faf0 2024-09-16T01:48:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1c09d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d021029a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1c0a59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02101faf0 0x55d0210283c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1c09d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d021029a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38766/0 (socket says 172.21.15.139:38766) 2024-09-16T01:48:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1c09d9b640 1 -- 172.21.15.139:0/2096360362 learned_addr learned my addr 172.21.15.139:0/2096360362 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1c0ad9d640 1 --2- 172.21.15.139:0/2096360362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 0x55d021042950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1c09d9b640 1 -- 172.21.15.139:0/2096360362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02101faf0 msgr2=0x55d0210283c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1c09d9b640 1 --2- 172.21.15.139:0/2096360362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02101faf0 0x55d0210283c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1c09d9b640 1 -- 172.21.15.139:0/2096360362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 msgr2=0x55d021042950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1c09d9b640 1 --2- 172.21.15.139:0/2096360362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 0x55d021042950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1c09d9b640 1 -- 172.21.15.139:0/2096360362 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d021020240 con 0x55d021020410 2024-09-16T01:48:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1c0a59c640 1 --2- 172.21.15.139:0/2096360362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02101faf0 0x55d0210283c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:48:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1c09d9b640 1 --2- 172.21.15.139:0/2096360362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d021029a80 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1c0001c040 tx=0x7f1c00009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f003be1b4b76dc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1bf3fff640 1 -- 172.21.15.139:0/2096360362 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0001e040 con 0x55d021020410 2024-09-16T01:48:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1bf3fff640 1 -- 172.21.15.139:0/2096360362 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c00003b20 con 0x55d021020410 2024-09-16T01:48:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.421+0000 7f1bf3fff640 1 -- 172.21.15.139:0/2096360362 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00019ad0 con 0x55d021020410 2024-09-16T01:48:42.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:41 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:48:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.422+0000 7f1c13bc7740 1 -- 172.21.15.139:0/2096360362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 msgr2=0x55d021029a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.422+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/2096360362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d021029a80 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1c0001c040 tx=0x7f1c00009fb0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.422+0000 7f1c13bc7740 1 -- 172.21.15.139:0/2096360362 shutdown_connections 2024-09-16T01:48:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.422+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/2096360362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02102acb0 0x55d021042950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.422+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/2096360362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d021029a80 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.422+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/2096360362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02101faf0 0x55d0210283c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.422+0000 7f1c13bc7740 1 -- 172.21.15.139:0/2096360362 >> 172.21.15.139:0/2096360362 conn(0x55d02101c060 msgr2=0x55d021029370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.422+0000 7f1c13bc7740 1 -- 172.21.15.139:0/2096360362 shutdown_connections 2024-09-16T01:48:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.422+0000 7f1c13bc7740 1 -- 172.21.15.139:0/2096360362 wait complete. 2024-09-16T01:48:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.422+0000 7f1c13bc7740 1 -- start start 2024-09-16T01:48:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.422+0000 7f1c13bc7740 1 Processor -- start 2024-09-16T01:48:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c13bc7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02101faf0 0x55d021032f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c13bc7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d021033470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c13bc7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02102acb0 0x55d02102b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c13bc7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1b2950 con 0x55d021020410 2024-09-16T01:48:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c13bc7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1b21f0 con 0x55d02101faf0 2024-09-16T01:48:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c13bc7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1af300 con 0x55d02102acb0 2024-09-16T01:48:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c13bc7740 10 client.0 _ll_register_callbacks cb 0x7f1bfc09c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:48:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c09d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d021033470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c09d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d021033470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38780/0 (socket says 172.21.15.139:38780) 2024-09-16T01:48:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c09d9b640 1 -- 172.21.15.139:0/1023639398 learned_addr learned my addr 172.21.15.139:0/1023639398 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:48:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c0ad9d640 1 --2- 172.21.15.139:0/1023639398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02102acb0 0x55d02102b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c0a59c640 1 --2- 172.21.15.139:0/1023639398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02101faf0 0x55d021032f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c09d9b640 1 -- 172.21.15.139:0/1023639398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02102acb0 msgr2=0x55d02102b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c09d9b640 1 --2- 172.21.15.139:0/1023639398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02102acb0 0x55d02102b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c09d9b640 1 -- 172.21.15.139:0/1023639398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02101faf0 msgr2=0x55d021032f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:48:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c09d9b640 1 --2- 172.21.15.139:0/1023639398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02101faf0 0x55d021032f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c09d9b640 1 -- 172.21.15.139:0/1023639398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d02102bec0 con 0x55d021020410 2024-09-16T01:48:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c0ad9d640 1 --2- 172.21.15.139:0/1023639398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02102acb0 0x55d02102b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:48:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1c09d9b640 1 --2- 172.21.15.139:0/1023639398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d021033470 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1c00005040 tx=0x7f1c00003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad4f0f5b8dbe649 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:48:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1bf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:48:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.423+0000 7f1bf1ffb640 1 -- 172.21.15.139:0/1023639398 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0001e040 con 0x55d021020410 2024-09-16T01:48:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.424+0000 7f1c13bc7740 1 -- 172.21.15.139:0/1023639398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c00003b20 con 0x55d021020410 2024-09-16T01:48:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.424+0000 7f1bf1ffb640 1 -- 172.21.15.139:0/1023639398 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9554+0+0 (secure 0 0 0) 0x7f1c00019430 con 0x55d021020410 2024-09-16T01:48:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.424+0000 7f1bf1ffb640 1 -- 172.21.15.139:0/1023639398 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f1c00005b30 con 0x55d021020410 2024-09-16T01:48:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.424+0000 7f1c13bc7740 10 client.16416 fetch_fsmap learned FSMap version 48 2024-09-16T01:48:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.424+0000 7f1c13bc7740 1 -- 172.21.15.139:0/1023639398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d021032810 con 0x55d021020410 2024-09-16T01:48:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.424+0000 7f1bf1ffb640 1 -- 172.21.15.139:0/1023639398 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c00003b20 con 0x55d021020410 2024-09-16T01:48:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.424+0000 7f1bf1ffb640 1 -- 172.21.15.139:0/1023639398 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 48) ==== 8438+0+0 (secure 0 0 0) 0x7f1c00025420 con 0x55d021020410 2024-09-16T01:48:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.424+0000 7f1c13bc7740 10 client.16416 fetch_fsmap finished waiting for FSMap version 48 2024-09-16T01:48:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.424+0000 7f1c13bc7740 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:48:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.424+0000 7f1c13bc7740 20 client.16416 populate_metadata read hostname 'smithi139' 2024-09-16T01:48:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.424+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/1023639398 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d0210a5ab0 0x55d0210a7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:48:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.424+0000 7f1c13bc7740 4 client.16416 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:48:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.424+0000 7f1c13bc7740 1 -- 172.21.15.139:0/1023639398 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d020f8bd70 con 0x55d0210a5ab0 2024-09-16T01:48:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.424+0000 7f1c0a59c640 1 --2- 172.21.15.139:0/1023639398 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d0210a5ab0 0x55d0210a7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:48:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.425+0000 7f1bf1ffb640 10 client.16416 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:42.425+0000 7f1c0a59c640 1 --2- 172.21.15.139:0/1023639398 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d0210a5ab0 0x55d0210a7ed0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92f18d7d153c3f67 server_cookie=7fb9d6e584b19bbf in_seq=0 out_seq=0 2024-09-16T01:48:42.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:41 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:48:43.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:42 smithi164 ceph-mon[35121]: pgmap v257: 129 pgs: 129 active+clean; 2.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-16T01:48:43.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:42 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:43.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:42 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:48:43.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:42 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:48:43.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:42 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:48:43.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:42 smithi139 ceph-mon[31175]: pgmap v257: 129 pgs: 129 active+clean; 2.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-16T01:48:43.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:42 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:43.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:42 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:48:43.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:42 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:48:43.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:42 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:48:43.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:42 smithi176 ceph-mon[35324]: pgmap v257: 129 pgs: 129 active+clean; 2.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-16T01:48:43.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:42 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:43.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:42 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:48:43.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:42 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:48:43.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:42 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:48:45.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:45 smithi164 ceph-mon[35121]: pgmap v258: 129 pgs: 129 active+clean; 2.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 252 op/s 2024-09-16T01:48:45.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:45 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:45.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:45 smithi139 ceph-mon[31175]: pgmap v258: 129 pgs: 129 active+clean; 2.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 252 op/s 2024-09-16T01:48:45.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:45 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:45.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:45 smithi176 ceph-mon[35324]: pgmap v258: 129 pgs: 129 active+clean; 2.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 252 op/s 2024-09-16T01:48:45.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:45 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:47.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:47 smithi164 ceph-mon[35121]: pgmap v259: 129 pgs: 129 active+clean; 2.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 785 op/s 2024-09-16T01:48:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:47 smithi139 ceph-mon[31175]: pgmap v259: 129 pgs: 129 active+clean; 2.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 785 op/s 2024-09-16T01:48:47.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:47 smithi176 ceph-mon[35324]: pgmap v259: 129 pgs: 129 active+clean; 2.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 785 op/s 2024-09-16T01:48:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.287+0000 7f1bf1ffb640 1 -- 172.21.15.139:0/1023639398 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d020f8bd70 con 0x55d0210a5ab0 2024-09-16T01:48:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.287+0000 7f1bf1ffb640 10 client.16416 handle_command_reply: tid=0 2024-09-16T01:48:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.287+0000 7f1c13bc7740 1 client.16416 shutdown 2024-09-16T01:48:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.287+0000 7f1c13bc7740 1 -- 172.21.15.139:0/1023639398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 msgr2=0x55d021033470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.287+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/1023639398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d021033470 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1c00005040 tx=0x7f1c00003730 comp rx=0 tx=0).stop 2024-09-16T01:48:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.288+0000 7f1c13bc7740 1 -- 172.21.15.139:0/1023639398 shutdown_connections 2024-09-16T01:48:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.288+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/1023639398 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d0210a5ab0 0x55d0210a7ed0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.288+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/1023639398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02102acb0 0x55d02102b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.288+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/1023639398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d021020410 0x55d021033470 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.288+0000 7f1bf1ffb640 0 client.16416 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:48:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.288+0000 7f1c13bc7740 1 --2- 172.21.15.139:0/1023639398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d02101faf0 0x55d021032f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.288+0000 7f1c13bc7740 1 -- 172.21.15.139:0/1023639398 >> 172.21.15.139:0/1023639398 conn(0x55d02101c060 msgr2=0x55d02101b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.288+0000 7f1c13bc7740 1 -- 172.21.15.139:0/1023639398 shutdown_connections 2024-09-16T01:48:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.288+0000 7f1c13bc7740 1 -- 172.21.15.139:0/1023639398 wait complete. 2024-09-16T01:48:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.288+0000 7f1c13bc7740 20 client.16416 trim_cache size 0 max 16384 2024-09-16T01:48:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:48:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:48:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:48:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:48:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.289+0000 7f1c10d4b640 1 -- 172.21.15.139:0/2974347077 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1bf40a3460 msgr2=0x7f1bf40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.289+0000 7f1c10d4b640 1 --2- 172.21.15.139:0/2974347077 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1bf40a3460 0x7f1bf40a5920 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1c04004af0 tx=0x7f1c0402a6e0 comp rx=0 tx=0).stop 2024-09-16T01:48:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.289+0000 7f1c10d4b640 1 -- 172.21.15.139:0/2974347077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c1009b0 msgr2=0x7f1c0c1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:48:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.289+0000 7f1c10d4b640 1 --2- 172.21.15.139:0/2974347077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c1009b0 0x7f1c0c1028f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00e9f0 tx=0x7f1bfc00edf0 comp rx=0 tx=0).stop 2024-09-16T01:48:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.289+0000 7f1c10d4b640 1 -- 172.21.15.139:0/2974347077 shutdown_connections 2024-09-16T01:48:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.289+0000 7f1c10d4b640 1 --2- 172.21.15.139:0/2974347077 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1bf40a3460 0x7f1bf40a5920 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.289+0000 7f1c10d4b640 1 --2- 172.21.15.139:0/2974347077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c0c1069c0 0x7f1c0c103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.289+0000 7f1c10d4b640 1 --2- 172.21.15.139:0/2974347077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c0c101350 0x7f1c0c102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.289+0000 7f1c10d4b640 1 --2- 172.21.15.139:0/2974347077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c0c1009b0 0x7f1c0c1028f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:48:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.289+0000 7f1c10d4b640 1 -- 172.21.15.139:0/2974347077 >> 172.21.15.139:0/2974347077 conn(0x7f1c0c077990 msgr2=0x7f1c0c104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:48:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.289+0000 7f1c10d4b640 1 -- 172.21.15.139:0/2974347077 shutdown_connections 2024-09-16T01:48:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:48:48.289+0000 7f1c10d4b640 1 -- 172.21.15.139:0/2974347077 wait complete. 2024-09-16T01:48:49.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:49 smithi164 ceph-mon[35121]: pgmap v260: 129 pgs: 129 active+clean; 2.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 1.41k op/s 2024-09-16T01:48:49.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:49 smithi139 ceph-mon[31175]: pgmap v260: 129 pgs: 129 active+clean; 2.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 1.41k op/s 2024-09-16T01:48:49.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:49 smithi176 ceph-mon[35324]: pgmap v260: 129 pgs: 129 active+clean; 2.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 1.41k op/s 2024-09-16T01:48:51.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:51 smithi164 ceph-mon[35121]: pgmap v261: 129 pgs: 129 active+clean; 2.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 1.48k op/s 2024-09-16T01:48:51.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:51 smithi139 ceph-mon[31175]: pgmap v261: 129 pgs: 129 active+clean; 2.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 1.48k op/s 2024-09-16T01:48:51.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:51 smithi176 ceph-mon[35324]: pgmap v261: 129 pgs: 129 active+clean; 2.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 1.48k op/s 2024-09-16T01:48:53.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:53 smithi164 ceph-mon[35121]: pgmap v262: 129 pgs: 129 active+clean; 2.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 1.92k op/s 2024-09-16T01:48:53.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:53 smithi139 ceph-mon[31175]: pgmap v262: 129 pgs: 129 active+clean; 2.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 1.92k op/s 2024-09-16T01:48:53.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:53 smithi176 ceph-mon[35324]: pgmap v262: 129 pgs: 129 active+clean; 2.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 1.92k op/s 2024-09-16T01:48:55.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:55 smithi164 ceph-mon[35121]: pgmap v263: 129 pgs: 129 active+clean; 2.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 1.93k op/s 2024-09-16T01:48:55.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:55 smithi139 ceph-mon[31175]: pgmap v263: 129 pgs: 129 active+clean; 2.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 1.93k op/s 2024-09-16T01:48:55.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:55 smithi176 ceph-mon[35324]: pgmap v263: 129 pgs: 129 active+clean; 2.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 1.93k op/s 2024-09-16T01:48:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:57 smithi139 ceph-mon[31175]: pgmap v264: 129 pgs: 129 active+clean; 2.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 1.94k op/s 2024-09-16T01:48:57.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:57 smithi176 ceph-mon[35324]: pgmap v264: 129 pgs: 129 active+clean; 2.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 1.94k op/s 2024-09-16T01:48:57.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:57 smithi164 ceph-mon[35121]: pgmap v264: 129 pgs: 129 active+clean; 2.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 1.94k op/s 2024-09-16T01:48:58.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:58 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:58.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:58 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:58.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:58 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:48:59.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:48:59 smithi139 ceph-mon[31175]: pgmap v265: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.1 MiB/s wr, 1.47k op/s 2024-09-16T01:48:59.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:48:59 smithi176 ceph-mon[35324]: pgmap v265: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.1 MiB/s wr, 1.47k op/s 2024-09-16T01:48:59.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:48:59 smithi164 ceph-mon[35121]: pgmap v265: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.1 MiB/s wr, 1.47k op/s 2024-09-16T01:49:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.383+0000 7fb160359640 1 -- 172.21.15.139:0/2853162546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb15810c110 msgr2=0x7fb15810e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.383+0000 7fb160359640 1 --2- 172.21.15.139:0/2853162546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb15810c110 0x7fb15810e590 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb150009ca0 tx=0x7fb15002a950 comp rx=0 tx=0).stop 2024-09-16T01:49:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 -- 172.21.15.139:0/2853162546 shutdown_connections 2024-09-16T01:49:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 --2- 172.21.15.139:0/2853162546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb15810c110 0x7fb15810e590 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 --2- 172.21.15.139:0/2853162546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1581035e0 0x7fb15810bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 --2- 172.21.15.139:0/2853162546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb158102cc0 0x7fb1581030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 -- 172.21.15.139:0/2853162546 >> 172.21.15.139:0/2853162546 conn(0x7fb1580fc6b0 msgr2=0x7fb1580fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 -- 172.21.15.139:0/2853162546 shutdown_connections 2024-09-16T01:49:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 -- 172.21.15.139:0/2853162546 wait complete. 2024-09-16T01:49:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 Processor -- start 2024-09-16T01:49:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 -- start start 2024-09-16T01:49:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb158102cc0 0x7fb158198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1581035e0 0x7fb158198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb15810c110 0x7fb15819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb158111ed0 con 0x7fb1581035e0 2024-09-16T01:49:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb158111d50 con 0x7fb15810c110 2024-09-16T01:49:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.384+0000 7fb160359640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb158112050 con 0x7fb158102cc0 2024-09-16T01:49:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb15d8f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1581035e0 0x7fb158198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb15d8f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1581035e0 0x7fb158198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60212/0 (socket says 172.21.15.139:60212) 2024-09-16T01:49:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb15d8f4640 1 -- 172.21.15.139:0/3389406395 learned_addr learned my addr 172.21.15.139:0/3389406395 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb15e0f5640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb158102cc0 0x7fb158198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb15e8f6640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb15810c110 0x7fb15819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb15d8f4640 1 -- 172.21.15.139:0/3389406395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb158102cc0 msgr2=0x7fb158198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb15d8f4640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb158102cc0 0x7fb158198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb15d8f4640 1 -- 172.21.15.139:0/3389406395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb15810c110 msgr2=0x7fb15819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb15d8f4640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb15810c110 0x7fb15819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb15d8f4640 1 -- 172.21.15.139:0/3389406395 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15819d2f0 con 0x7fb1581035e0 2024-09-16T01:49:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb15e8f6640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb15810c110 0x7fb15819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:49:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb15d8f4640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1581035e0 0x7fb158198840 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb14800d750 tx=0x7fb14800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb14f7fe640 1 -- 172.21.15.139:0/3389406395 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148017070 con 0x7fb1581035e0 2024-09-16T01:49:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb14f7fe640 1 -- 172.21.15.139:0/3389406395 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb148005020 con 0x7fb1581035e0 2024-09-16T01:49:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb14f7fe640 1 -- 172.21.15.139:0/3389406395 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148004d00 con 0x7fb1581035e0 2024-09-16T01:49:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb160359640 1 -- 172.21.15.139:0/3389406395 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb15819d5e0 con 0x7fb1581035e0 2024-09-16T01:49:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.385+0000 7fb160359640 1 -- 172.21.15.139:0/3389406395 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1581a4e70 con 0x7fb1581035e0 2024-09-16T01:49:01.443 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:01 smithi176 ceph-mon[35324]: pgmap v266: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 833 op/s 2024-09-16T01:49:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.388+0000 7fb14f7fe640 1 -- 172.21.15.139:0/3389406395 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb148009a30 con 0x7fb1581035e0 2024-09-16T01:49:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.388+0000 7fb1631d2740 1 Processor -- start 2024-09-16T01:49:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.388+0000 7fb1631d2740 1 -- start start 2024-09-16T01:49:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.388+0000 7fb1631d2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d3fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.388+0000 7fb1631d2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d4a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.388+0000 7fb1631d2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 0x565429d4d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.388+0000 7fb1631d2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565429ca42f0 con 0x565429d3faf0 2024-09-16T01:49:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.388+0000 7fb1631d2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565429c4c270 con 0x565429d4acb0 2024-09-16T01:49:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.388+0000 7fb1631d2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x565429ca02b0 con 0x565429d40410 2024-09-16T01:49:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb14f7fe640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb1440a34e0 0x7fb1440a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb15e0f5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d4a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb15e8f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d3fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb15e0f5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d4a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:46718/0 (socket says 172.21.15.139:46718) 2024-09-16T01:49:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb15e0f5640 1 -- 172.21.15.139:0/3470705585 learned_addr learned my addr 172.21.15.139:0/3470705585 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb15d8f4640 1 --2- 172.21.15.139:0/3470705585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 0x565429d4d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb15e8f6640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb1440a34e0 0x7fb1440a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb14f7fe640 1 -- 172.21.15.139:0/3389406395 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9554+0+0 (secure 0 0 0) 0x7fb1480cfac0 con 0x7fb1581035e0 2024-09-16T01:49:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb15e0f5640 1 -- 172.21.15.139:0/3470705585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 msgr2=0x565429d4d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb15e0f5640 1 --2- 172.21.15.139:0/3470705585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 0x565429d4d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb15e0f5640 1 -- 172.21.15.139:0/3470705585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 msgr2=0x565429d3fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb15e0f5640 1 --2- 172.21.15.139:0/3470705585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d3fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb15e0f5640 1 -- 172.21.15.139:0/3470705585 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565429c4b340 con 0x565429d40410 2024-09-16T01:49:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb15d8f4640 1 --2- 172.21.15.139:0/3470705585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 0x565429d4d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:49:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb15e8f6640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb1440a34e0 0x7fb1440a59a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb150005fc0 tx=0x7fb1500095c0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb15e0f5640 1 --2- 172.21.15.139:0/3470705585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d4a770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb15400bd10 tx=0x7fb1540107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea540ff4c07a7cac server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb14cbfe640 1 -- 172.21.15.139:0/3470705585 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154004030 con 0x565429d40410 2024-09-16T01:49:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb14cbfe640 1 -- 172.21.15.139:0/3470705585 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1540041d0 con 0x565429d40410 2024-09-16T01:49:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.389+0000 7fb14cbfe640 1 -- 172.21.15.139:0/3470705585 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154011820 con 0x565429d40410 2024-09-16T01:49:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.390+0000 7fb1631d2740 1 -- 172.21.15.139:0/3470705585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 msgr2=0x565429d4a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.390+0000 7fb1631d2740 1 --2- 172.21.15.139:0/3470705585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d4a770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb15400bd10 tx=0x7fb1540107a0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.390+0000 7fb1631d2740 1 -- 172.21.15.139:0/3470705585 shutdown_connections 2024-09-16T01:49:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.390+0000 7fb1631d2740 1 --2- 172.21.15.139:0/3470705585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 0x565429d4d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.390+0000 7fb1631d2740 1 --2- 172.21.15.139:0/3470705585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d4a770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.390+0000 7fb1631d2740 1 --2- 172.21.15.139:0/3470705585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d3fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.390+0000 7fb1631d2740 1 -- 172.21.15.139:0/3470705585 >> 172.21.15.139:0/3470705585 conn(0x565429d3c060 msgr2=0x565429d3c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.390+0000 7fb1631d2740 1 -- 172.21.15.139:0/3470705585 shutdown_connections 2024-09-16T01:49:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.390+0000 7fb1631d2740 1 -- 172.21.15.139:0/3470705585 wait complete. 2024-09-16T01:49:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb1631d2740 1 -- start start 2024-09-16T01:49:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb1631d2740 1 Processor -- start 2024-09-16T01:49:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb1631d2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d3faf0 0x565429d47b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb1631d2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d48090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb1631d2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d4acb0 0x565429d485d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb1631d2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x565429d4d880 con 0x565429d4acb0 2024-09-16T01:49:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb1631d2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565429c2d290 con 0x565429d3faf0 2024-09-16T01:49:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb1631d2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x565429d4da00 con 0x565429d40410 2024-09-16T01:49:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb1631d2740 10 client.0 _ll_register_callbacks cb 0x565429c5e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb15e8f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d4acb0 0x565429d485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb15e8f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d4acb0 0x565429d485d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60234/0 (socket says 172.21.15.139:60234) 2024-09-16T01:49:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb15e8f6640 1 -- 172.21.15.139:0/996317416 learned_addr learned my addr 172.21.15.139:0/996317416 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb15d8f4640 1 --2- 172.21.15.139:0/996317416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d48090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb15e0f5640 1 --2- 172.21.15.139:0/996317416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d3faf0 0x565429d47b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb15e8f6640 1 -- 172.21.15.139:0/996317416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 msgr2=0x565429d48090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb15e8f6640 1 --2- 172.21.15.139:0/996317416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d48090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb15e8f6640 1 -- 172.21.15.139:0/996317416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d3faf0 msgr2=0x565429d47b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb15e8f6640 1 --2- 172.21.15.139:0/996317416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d3faf0 0x565429d47b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb15e8f6640 1 -- 172.21.15.139:0/996317416 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654299b7210 con 0x565429d4acb0 2024-09-16T01:49:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.391+0000 7fb15d8f4640 1 --2- 172.21.15.139:0/996317416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d48090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:49:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb15e8f6640 1 --2- 172.21.15.139:0/996317416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d4acb0 0x565429d485d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb15002fce0 tx=0x7fb15002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcc97b63ac9b034 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb141ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:49:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb141ffb640 1 -- 172.21.15.139:0/996317416 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15004f030 con 0x565429d4acb0 2024-09-16T01:49:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb1631d2740 1 -- 172.21.15.139:0/996317416 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1540041d0 con 0x565429d4acb0 2024-09-16T01:49:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb141ffb640 1 -- 172.21.15.139:0/996317416 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9554+0+0 (secure 0 0 0) 0x7fb150003d80 con 0x565429d4acb0 2024-09-16T01:49:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb141ffb640 1 -- 172.21.15.139:0/996317416 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fb15005c050 con 0x565429d4acb0 2024-09-16T01:49:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb1631d2740 10 client.16428 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb1631d2740 1 -- 172.21.15.139:0/996317416 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x565429a83620 con 0x565429d4acb0 2024-09-16T01:49:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb141ffb640 1 -- 172.21.15.139:0/996317416 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1540041d0 con 0x565429d4acb0 2024-09-16T01:49:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb141ffb640 1 -- 172.21.15.139:0/996317416 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7fb15004b730 con 0x565429d4acb0 2024-09-16T01:49:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb1631d2740 10 client.16428 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb1631d2740 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:49:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb1631d2740 20 client.16428 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb1631d2740 1 --2- 172.21.15.139:0/996317416 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x565429d5ac00 0x565429d5d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb1631d2740 4 client.16428 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:49:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.392+0000 7fb1631d2740 1 -- 172.21.15.139:0/996317416 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565429cb1a20 con 0x565429d5ac00 2024-09-16T01:49:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.393+0000 7fb15e0f5640 1 --2- 172.21.15.139:0/996317416 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x565429d5ac00 0x565429d5d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.393+0000 7fb141ffb640 10 client.16428 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.393+0000 7fb15e0f5640 1 --2- 172.21.15.139:0/996317416 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x565429d5ac00 0x565429d5d0a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b720a15aa26e30 server_cookie=51a18a6c4288e332 in_seq=0 out_seq=0 2024-09-16T01:49:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb141ffb640 1 -- 172.21.15.139:0/996317416 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565429cb1a20 con 0x565429d5ac00 2024-09-16T01:49:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb141ffb640 10 client.16428 handle_command_reply: tid=0 2024-09-16T01:49:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb1631d2740 1 client.16428 shutdown 2024-09-16T01:49:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb1631d2740 1 -- 172.21.15.139:0/996317416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d4acb0 msgr2=0x565429d485d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb1631d2740 1 --2- 172.21.15.139:0/996317416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d4acb0 0x565429d485d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb15002fce0 tx=0x7fb15002a950 comp rx=0 tx=0).stop 2024-09-16T01:49:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb1631d2740 1 -- 172.21.15.139:0/996317416 shutdown_connections 2024-09-16T01:49:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb1631d2740 1 --2- 172.21.15.139:0/996317416 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x565429d5ac00 0x565429d5d0a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb1631d2740 1 --2- 172.21.15.139:0/996317416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d4acb0 0x565429d485d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb141ffb640 0 client.16428 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb1631d2740 1 --2- 172.21.15.139:0/996317416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d48090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb1631d2740 1 --2- 172.21.15.139:0/996317416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d3faf0 0x565429d47b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb1631d2740 1 -- 172.21.15.139:0/996317416 >> 172.21.15.139:0/996317416 conn(0x565429d3c060 msgr2=0x565429d3b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb1631d2740 1 -- 172.21.15.139:0/996317416 shutdown_connections 2024-09-16T01:49:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb1631d2740 1 -- 172.21.15.139:0/996317416 wait complete. 2024-09-16T01:49:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.394+0000 7fb1631d2740 20 client.16428 trim_cache size 0 max 16384 2024-09-16T01:49:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb1631d2740 1 Processor -- start 2024-09-16T01:49:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb1631d2740 1 -- start start 2024-09-16T01:49:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb1631d2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d42560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb1631d2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d48400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb1631d2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 0x565429d53070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb1631d2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581af0c0 con 0x565429d3faf0 2024-09-16T01:49:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb1631d2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581b0f00 con 0x565429d4acb0 2024-09-16T01:49:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb1631d2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581af850 con 0x565429d40410 2024-09-16T01:49:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb15e0f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d42560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb15e0f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d42560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60242/0 (socket says 172.21.15.139:60242) 2024-09-16T01:49:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb15e0f5640 1 -- 172.21.15.139:0/1270389093 learned_addr learned my addr 172.21.15.139:0/1270389093 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb15e8f6640 1 --2- 172.21.15.139:0/1270389093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 0x565429d53070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb15d8f4640 1 --2- 172.21.15.139:0/1270389093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d48400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb15e0f5640 1 -- 172.21.15.139:0/1270389093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 msgr2=0x565429d48400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.408+0000 7fb15e0f5640 1 --2- 172.21.15.139:0/1270389093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d48400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb15e0f5640 1 -- 172.21.15.139:0/1270389093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 msgr2=0x565429d53070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb15e0f5640 1 --2- 172.21.15.139:0/1270389093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 0x565429d53070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb15e0f5640 1 -- 172.21.15.139:0/1270389093 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565429d4d9f0 con 0x565429d3faf0 2024-09-16T01:49:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb15d8f4640 1 --2- 172.21.15.139:0/1270389093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d48400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:49:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb15e0f5640 1 --2- 172.21.15.139:0/1270389093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d42560 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb15400bd10 tx=0x7fb154011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210d487359ec7cb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb143fff640 1 -- 172.21.15.139:0/1270389093 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154005240 con 0x565429d3faf0 2024-09-16T01:49:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb143fff640 1 -- 172.21.15.139:0/1270389093 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1540035f0 con 0x565429d3faf0 2024-09-16T01:49:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb143fff640 1 -- 172.21.15.139:0/1270389093 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154011710 con 0x565429d3faf0 2024-09-16T01:49:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb1631d2740 1 -- 172.21.15.139:0/1270389093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 msgr2=0x565429d42560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb1631d2740 1 --2- 172.21.15.139:0/1270389093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d42560 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb15400bd10 tx=0x7fb154011440 comp rx=0 tx=0).stop 2024-09-16T01:49:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb1631d2740 1 -- 172.21.15.139:0/1270389093 shutdown_connections 2024-09-16T01:49:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb1631d2740 1 --2- 172.21.15.139:0/1270389093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 0x565429d53070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb1631d2740 1 --2- 172.21.15.139:0/1270389093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d48400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb1631d2740 1 --2- 172.21.15.139:0/1270389093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d42560 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb1631d2740 1 -- 172.21.15.139:0/1270389093 >> 172.21.15.139:0/1270389093 conn(0x565429d3c060 msgr2=0x565429d49bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb1631d2740 1 -- 172.21.15.139:0/1270389093 shutdown_connections 2024-09-16T01:49:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.409+0000 7fb1631d2740 1 -- 172.21.15.139:0/1270389093 wait complete. 2024-09-16T01:49:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb1631d2740 1 -- start start 2024-09-16T01:49:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb1631d2740 1 Processor -- start 2024-09-16T01:49:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb1631d2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d62500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb1631d2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d62a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb1631d2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 0x565429dc0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb1631d2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581b0770 con 0x565429d3faf0 2024-09-16T01:49:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb1631d2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581affe0 con 0x565429d4acb0 2024-09-16T01:49:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb1631d2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581ad0c0 con 0x565429d40410 2024-09-16T01:49:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb1631d2740 10 client.0 _ll_register_callbacks cb 0x7fb15002feb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb15e0f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d62500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb15e0f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d62500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60246/0 (socket says 172.21.15.139:60246) 2024-09-16T01:49:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb15e0f5640 1 -- 172.21.15.139:0/3115197437 learned_addr learned my addr 172.21.15.139:0/3115197437 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb15e8f6640 1 --2- 172.21.15.139:0/3115197437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 0x565429dc0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb15d8f4640 1 --2- 172.21.15.139:0/3115197437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d62a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb15e0f5640 1 -- 172.21.15.139:0/3115197437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 msgr2=0x565429d62a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb15e0f5640 1 --2- 172.21.15.139:0/3115197437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d62a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb15e0f5640 1 -- 172.21.15.139:0/3115197437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 msgr2=0x565429dc0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb15e0f5640 1 --2- 172.21.15.139:0/3115197437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 0x565429dc0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.410+0000 7fb15e0f5640 1 -- 172.21.15.139:0/3115197437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565429d3ccf0 con 0x565429d3faf0 2024-09-16T01:49:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb15e0f5640 1 --2- 172.21.15.139:0/3115197437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d62500 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb1540107e0 tx=0x7fb154005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc237a9d0a0d9d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb141ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:49:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb141ffb640 1 -- 172.21.15.139:0/3115197437 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1540052d0 con 0x565429d3faf0 2024-09-16T01:49:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb141ffb640 1 -- 172.21.15.139:0/3115197437 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9554+0+0 (secure 0 0 0) 0x7fb154005470 con 0x565429d3faf0 2024-09-16T01:49:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb1631d2740 1 -- 172.21.15.139:0/3115197437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1540035f0 con 0x565429d3faf0 2024-09-16T01:49:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb141ffb640 1 -- 172.21.15.139:0/3115197437 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fb154005670 con 0x565429d3faf0 2024-09-16T01:49:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb1631d2740 10 client.16434 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb1631d2740 1 -- 172.21.15.139:0/3115197437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x565429dc9630 con 0x565429d3faf0 2024-09-16T01:49:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb141ffb640 1 -- 172.21.15.139:0/3115197437 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1540035f0 con 0x565429d3faf0 2024-09-16T01:49:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb141ffb640 1 -- 172.21.15.139:0/3115197437 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7fb154011b30 con 0x565429d3faf0 2024-09-16T01:49:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb1631d2740 10 client.16434 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb1631d2740 10 client.16434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:49:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb1631d2740 20 client.16434 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb1631d2740 1 --2- 172.21.15.139:0/3115197437 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x565429dc9bd0 0x565429dcbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb1631d2740 4 client.16434 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:49:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.411+0000 7fb1631d2740 1 -- 172.21.15.139:0/3115197437 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565429cabd70 con 0x565429dc9bd0 2024-09-16T01:49:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.412+0000 7fb15d8f4640 1 --2- 172.21.15.139:0/3115197437 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x565429dc9bd0 0x565429dcbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.412+0000 7fb141ffb640 10 client.16434 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.412+0000 7fb15d8f4640 1 --2- 172.21.15.139:0/3115197437 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x565429dc9bd0 0x565429dcbff0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d24eb58111c19b2 server_cookie=9d9c60a814ac976c in_seq=0 out_seq=0 2024-09-16T01:49:01.470 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:01 smithi139 ceph-mon[31175]: pgmap v266: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 833 op/s 2024-09-16T01:49:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb141ffb640 1 -- 172.21.15.139:0/3115197437 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565429cabd70 con 0x565429dc9bd0 2024-09-16T01:49:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb141ffb640 10 client.16434 handle_command_reply: tid=0 2024-09-16T01:49:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb1631d2740 1 client.16434 shutdown 2024-09-16T01:49:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb1631d2740 1 -- 172.21.15.139:0/3115197437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 msgr2=0x565429d62500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb1631d2740 1 --2- 172.21.15.139:0/3115197437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d62500 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb1540107e0 tx=0x7fb154005c80 comp rx=0 tx=0).stop 2024-09-16T01:49:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb1631d2740 1 -- 172.21.15.139:0/3115197437 shutdown_connections 2024-09-16T01:49:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb1631d2740 1 --2- 172.21.15.139:0/3115197437 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x565429dc9bd0 0x565429dcbff0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb1631d2740 1 --2- 172.21.15.139:0/3115197437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565429d4acb0 0x565429dc0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb141ffb640 0 client.16434 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb1631d2740 1 --2- 172.21.15.139:0/3115197437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565429d40410 0x565429d62a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb1631d2740 1 --2- 172.21.15.139:0/3115197437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x565429d3faf0 0x565429d62500 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb1631d2740 1 -- 172.21.15.139:0/3115197437 >> 172.21.15.139:0/3115197437 conn(0x565429d3c060 msgr2=0x565429d46000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb1631d2740 1 -- 172.21.15.139:0/3115197437 shutdown_connections 2024-09-16T01:49:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb1631d2740 1 -- 172.21.15.139:0/3115197437 wait complete. 2024-09-16T01:49:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.435+0000 7fb1631d2740 20 client.16434 trim_cache size 0 max 16384 2024-09-16T01:49:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:49:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:49:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:49:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:49:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.436+0000 7fb160359640 1 -- 172.21.15.139:0/3389406395 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb1440a34e0 msgr2=0x7fb1440a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.436+0000 7fb160359640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb1440a34e0 0x7fb1440a59a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb150005fc0 tx=0x7fb1500095c0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.436+0000 7fb160359640 1 -- 172.21.15.139:0/3389406395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1581035e0 msgr2=0x7fb158198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.436+0000 7fb160359640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1581035e0 0x7fb158198840 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb14800d750 tx=0x7fb14800dc10 comp rx=0 tx=0).stop 2024-09-16T01:49:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.436+0000 7fb160359640 1 -- 172.21.15.139:0/3389406395 shutdown_connections 2024-09-16T01:49:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.436+0000 7fb160359640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb1440a34e0 0x7fb1440a59a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.436+0000 7fb160359640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb15810c110 0x7fb15819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.436+0000 7fb160359640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1581035e0 0x7fb158198840 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.436+0000 7fb160359640 1 --2- 172.21.15.139:0/3389406395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb158102cc0 0x7fb158198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.436+0000 7fb160359640 1 -- 172.21.15.139:0/3389406395 >> 172.21.15.139:0/3389406395 conn(0x7fb1580fc6b0 msgr2=0x7fb1580fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.436+0000 7fb160359640 1 -- 172.21.15.139:0/3389406395 shutdown_connections 2024-09-16T01:49:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:01.436+0000 7fb160359640 1 -- 172.21.15.139:0/3389406395 wait complete. 2024-09-16T01:49:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:01 smithi164 ceph-mon[35121]: pgmap v266: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 833 op/s 2024-09-16T01:49:03.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:03 smithi139 ceph-mon[31175]: pgmap v267: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 795 op/s 2024-09-16T01:49:03.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:03 smithi176 ceph-mon[35324]: pgmap v267: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 795 op/s 2024-09-16T01:49:03.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:03 smithi164 ceph-mon[35121]: pgmap v267: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 795 op/s 2024-09-16T01:49:05.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:05 smithi139 ceph-mon[31175]: pgmap v268: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 385 op/s 2024-09-16T01:49:05.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:05 smithi176 ceph-mon[35324]: pgmap v268: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 385 op/s 2024-09-16T01:49:05.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:05 smithi164 ceph-mon[35121]: pgmap v268: 129 pgs: 129 active+clean; 2.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 385 op/s 2024-09-16T01:49:07.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:07 smithi139 ceph-mon[31175]: pgmap v269: 129 pgs: 129 active+clean; 2.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 314 op/s 2024-09-16T01:49:07.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:07 smithi176 ceph-mon[35324]: pgmap v269: 129 pgs: 129 active+clean; 2.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 314 op/s 2024-09-16T01:49:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:07 smithi164 ceph-mon[35121]: pgmap v269: 129 pgs: 129 active+clean; 2.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 314 op/s 2024-09-16T01:49:09.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:09 smithi139 ceph-mon[31175]: pgmap v270: 129 pgs: 129 active+clean; 2.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-09-16T01:49:09.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:09 smithi176 ceph-mon[35324]: pgmap v270: 129 pgs: 129 active+clean; 2.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-09-16T01:49:09.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:09 smithi164 ceph-mon[35121]: pgmap v270: 129 pgs: 129 active+clean; 2.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-09-16T01:49:11.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:11 smithi139 ceph-mon[31175]: pgmap v271: 129 pgs: 129 active+clean; 2.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 241 op/s 2024-09-16T01:49:11.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:11 smithi176 ceph-mon[35324]: pgmap v271: 129 pgs: 129 active+clean; 2.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 241 op/s 2024-09-16T01:49:11.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:11 smithi164 ceph-mon[35121]: pgmap v271: 129 pgs: 129 active+clean; 2.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 241 op/s 2024-09-16T01:49:12.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:12 smithi139 ceph-mon[31175]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T01:49:12.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:12 smithi176 ceph-mon[35324]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T01:49:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:12 smithi164 ceph-mon[35121]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T01:49:13.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:13 smithi139 ceph-mon[31175]: pgmap v273: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 2.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 263 op/s 2024-09-16T01:49:13.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:13 smithi176 ceph-mon[35324]: pgmap v273: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 2.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 263 op/s 2024-09-16T01:49:13.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:13 smithi164 ceph-mon[35121]: pgmap v273: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 2.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 263 op/s 2024-09-16T01:49:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.581+0000 7f83e35f6640 1 -- 172.21.15.139:0/4010875126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83dc105fa0 msgr2=0x7f83dc106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.581+0000 7f83e35f6640 1 --2- 172.21.15.139:0/4010875126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83dc105fa0 0x7f83dc106450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009930 tx=0x7f83d402a440 comp rx=0 tx=0).stop 2024-09-16T01:49:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.587+0000 7f83e35f6640 1 -- 172.21.15.139:0/4010875126 shutdown_connections 2024-09-16T01:49:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.587+0000 7f83e35f6640 1 --2- 172.21.15.139:0/4010875126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83dc105fa0 0x7f83dc106450 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.587+0000 7f83e35f6640 1 --2- 172.21.15.139:0/4010875126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83dc1013b0 0x7f83dc105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.587+0000 7f83e35f6640 1 --2- 172.21.15.139:0/4010875126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83dc100a90 0x7f83dc100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.587+0000 7f83e35f6640 1 -- 172.21.15.139:0/4010875126 >> 172.21.15.139:0/4010875126 conn(0x7f83dc0fc6f0 msgr2=0x7f83dc0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.592+0000 7f83e35f6640 1 -- 172.21.15.139:0/4010875126 shutdown_connections 2024-09-16T01:49:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.593+0000 7f83e35f6640 1 -- 172.21.15.139:0/4010875126 wait complete. 2024-09-16T01:49:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.593+0000 7f83e35f6640 1 Processor -- start 2024-09-16T01:49:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.595+0000 7f83e35f6640 1 -- start start 2024-09-16T01:49:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.595+0000 7f83e35f6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83dc100a90 0x7f83dc0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.595+0000 7f83e35f6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83dc1013b0 0x7f83dc06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.595+0000 7f83e35f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83dc105fa0 0x7f83dc06cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.595+0000 7f83e35f6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1184b0 con 0x7f83dc105fa0 2024-09-16T01:49:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.595+0000 7f83e35f6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc118330 con 0x7f83dc1013b0 2024-09-16T01:49:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.595+0000 7f83e35f6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc118630 con 0x7f83dc100a90 2024-09-16T01:49:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.595+0000 7f83e1b93640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83dc105fa0 0x7f83dc06cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.595+0000 7f83e0b91640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83dc1013b0 0x7f83dc06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.595+0000 7f83e0b91640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83dc1013b0 0x7f83dc06ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:49524/0 (socket says 172.21.15.139:49524) 2024-09-16T01:49:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.595+0000 7f83e0b91640 1 -- 172.21.15.139:0/1188881200 learned_addr learned my addr 172.21.15.139:0/1188881200 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.596+0000 7f83e0b91640 1 -- 172.21.15.139:0/1188881200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83dc100a90 msgr2=0x7f83dc0739c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:49:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.596+0000 7f83e0b91640 1 --2- 172.21.15.139:0/1188881200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83dc100a90 0x7f83dc0739c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.596+0000 7f83e0b91640 1 -- 172.21.15.139:0/1188881200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83dc105fa0 msgr2=0x7f83dc06cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.596+0000 7f83e0b91640 1 --2- 172.21.15.139:0/1188881200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83dc105fa0 0x7f83dc06cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.596+0000 7f83e0b91640 1 -- 172.21.15.139:0/1188881200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83dc06d540 con 0x7f83dc1013b0 2024-09-16T01:49:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.598+0000 7f83e0b91640 1 --2- 172.21.15.139:0/1188881200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83dc1013b0 0x7f83dc06ca40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00b4a0 tx=0x7f83cc00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.600+0000 7f83d27fc640 1 -- 172.21.15.139:0/1188881200 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc014070 con 0x7f83dc1013b0 2024-09-16T01:49:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.600+0000 7f83e35f6640 1 -- 172.21.15.139:0/1188881200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83dc06d830 con 0x7f83dc1013b0 2024-09-16T01:49:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.600+0000 7f83e35f6640 1 -- 172.21.15.139:0/1188881200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83dc071810 con 0x7f83dc1013b0 2024-09-16T01:49:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.601+0000 7f83d27fc640 1 -- 172.21.15.139:0/1188881200 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc010af0 con 0x7f83dc1013b0 2024-09-16T01:49:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.602+0000 7f83d27fc640 1 -- 172.21.15.139:0/1188881200 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc017d10 con 0x7f83dc1013b0 2024-09-16T01:49:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.614+0000 7f83e6469740 1 Processor -- start 2024-09-16T01:49:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.616+0000 7f83e6469740 1 -- start start 2024-09-16T01:49:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.617+0000 7f83e6469740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x55796304ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.617+0000 7f83e6469740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 0x557963057770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.617+0000 7f83e6469740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x55796305a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.617+0000 7f83e6469740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557962fb12f0 con 0x55796304caf0 2024-09-16T01:49:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.617+0000 7f83e6469740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557962f59270 con 0x55796304d410 2024-09-16T01:49:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.617+0000 7f83e6469740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557962fad2b0 con 0x557963057cb0 2024-09-16T01:49:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.617+0000 7f83d27fc640 1 -- 172.21.15.139:0/1188881200 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f83cc017070 con 0x7f83dc1013b0 2024-09-16T01:49:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.617+0000 7f83e0b91640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x55796305a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.617+0000 7f83e0b91640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x55796305a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43034/0 (socket says 172.21.15.139:43034) 2024-09-16T01:49:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.617+0000 7f83e0b91640 1 -- 172.21.15.139:0/694474253 learned_addr learned my addr 172.21.15.139:0/694474253 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.617+0000 7f83e1b93640 1 --2- 172.21.15.139:0/694474253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x55796304ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.620+0000 7f83e1392640 1 --2- 172.21.15.139:0/694474253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 0x557963057770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.625+0000 7f83d27fc640 1 --2- 172.21.15.139:0/1188881200 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f83c80a3510 0x7f83c80a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.626+0000 7f83d27fc640 1 -- 172.21.15.139:0/1188881200 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9642+0+0 (secure 0 0 0) 0x7f83cc0cf2f0 con 0x7f83dc1013b0 2024-09-16T01:49:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.627+0000 7f83e0b91640 1 -- 172.21.15.139:0/694474253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 msgr2=0x557963057770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.627+0000 7f83e0b91640 1 --2- 172.21.15.139:0/694474253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 0x557963057770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.627+0000 7f83e0b91640 1 -- 172.21.15.139:0/694474253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 msgr2=0x55796304ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.627+0000 7f83e0b91640 1 --2- 172.21.15.139:0/694474253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x55796304ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.627+0000 7f83e0b91640 1 -- 172.21.15.139:0/694474253 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557962f58340 con 0x557963057cb0 2024-09-16T01:49:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.630+0000 7f83e1b93640 1 --2- 172.21.15.139:0/1188881200 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f83c80a3510 0x7f83c80a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.630+0000 7f83e0b91640 1 --2- 172.21.15.139:0/694474253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x55796305a0a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0cc5d0 tx=0x7f83cc017f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4be9700965d093b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.632+0000 7f83c77fe640 1 -- 172.21.15.139:0/694474253 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc01e070 con 0x557963057cb0 2024-09-16T01:49:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.632+0000 7f83c77fe640 1 -- 172.21.15.139:0/694474253 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc0d2e30 con 0x557963057cb0 2024-09-16T01:49:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.632+0000 7f83c77fe640 1 -- 172.21.15.139:0/694474253 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0b9e60 con 0x557963057cb0 2024-09-16T01:49:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.632+0000 7f83e1392640 1 --2- 172.21.15.139:0/694474253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 0x557963057770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:49:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.633+0000 7f83e6469740 1 -- 172.21.15.139:0/694474253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 msgr2=0x55796305a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.633+0000 7f83e6469740 1 --2- 172.21.15.139:0/694474253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x55796305a0a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0cc5d0 tx=0x7f83cc017f90 comp rx=0 tx=0).stop 2024-09-16T01:49:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.639+0000 7f83e6469740 1 -- 172.21.15.139:0/694474253 shutdown_connections 2024-09-16T01:49:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.639+0000 7f83e6469740 1 --2- 172.21.15.139:0/694474253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x55796305a0a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.639+0000 7f83e6469740 1 --2- 172.21.15.139:0/694474253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 0x557963057770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.639+0000 7f83e6469740 1 --2- 172.21.15.139:0/694474253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x55796304ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.639+0000 7f83e6469740 1 -- 172.21.15.139:0/694474253 >> 172.21.15.139:0/694474253 conn(0x557963049060 msgr2=0x557963049440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.641+0000 7f83e1b93640 1 --2- 172.21.15.139:0/1188881200 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f83c80a3510 0x7f83c80a59d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f83d4031550 tx=0x7f83d402a440 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.644+0000 7f83e6469740 1 -- 172.21.15.139:0/694474253 shutdown_connections 2024-09-16T01:49:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.646+0000 7f83e6469740 1 -- 172.21.15.139:0/694474253 wait complete. 2024-09-16T01:49:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.646+0000 7f83e6469740 1 -- start start 2024-09-16T01:49:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.647+0000 7f83e6469740 1 Processor -- start 2024-09-16T01:49:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.648+0000 7f83e6469740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304caf0 0x557963054c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.648+0000 7f83e6469740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304d410 0x557963055160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.648+0000 7f83e6469740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x5579630556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.649+0000 7f83e6469740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55796305a880 con 0x55796304d410 2024-09-16T01:49:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.649+0000 7f83e6469740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557962f3a290 con 0x55796304caf0 2024-09-16T01:49:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.649+0000 7f83e6469740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55796305aa00 con 0x557963057cb0 2024-09-16T01:49:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.649+0000 7f83e6469740 10 client.0 _ll_register_callbacks cb 0x557962f6b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.652+0000 7f83e1b93640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x5579630556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.653+0000 7f83e1392640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304caf0 0x557963054c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.653+0000 7f83e1392640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304caf0 0x557963054c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:49550/0 (socket says 172.21.15.139:49550) 2024-09-16T01:49:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.653+0000 7f83e1392640 1 -- 172.21.15.139:0/296710952 learned_addr learned my addr 172.21.15.139:0/296710952 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.654+0000 7f83e0b91640 1 --2- 172.21.15.139:0/296710952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304d410 0x557963055160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.656+0000 7f83e0b91640 1 -- 172.21.15.139:0/296710952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 msgr2=0x5579630556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.656+0000 7f83e0b91640 1 --2- 172.21.15.139:0/296710952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x5579630556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.656+0000 7f83e0b91640 1 -- 172.21.15.139:0/296710952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304caf0 msgr2=0x557963054c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.656+0000 7f83e0b91640 1 --2- 172.21.15.139:0/296710952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304caf0 0x557963054c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.656+0000 7f83e0b91640 1 -- 172.21.15.139:0/296710952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557962cc4210 con 0x55796304d410 2024-09-16T01:49:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.659+0000 7f83e0b91640 1 --2- 172.21.15.139:0/296710952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304d410 0x557963055160 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f83cc097bf0 tx=0x7f83cc0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aad752d591b7ada server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.659+0000 7f83c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:49:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.660+0000 7f83c4ff9640 1 -- 172.21.15.139:0/296710952 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc01e050 con 0x55796304d410 2024-09-16T01:49:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.660+0000 7f83e6469740 1 -- 172.21.15.139:0/296710952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83cc0d2e30 con 0x55796304d410 2024-09-16T01:49:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.660+0000 7f83c4ff9640 1 -- 172.21.15.139:0/296710952 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9642+0+0 (secure 0 0 0) 0x7f83cc097890 con 0x55796304d410 2024-09-16T01:49:14.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.661+0000 7f83c4ff9640 1 -- 172.21.15.139:0/296710952 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f83cc0b9e60 con 0x55796304d410 2024-09-16T01:49:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.665+0000 7f83e6469740 10 client.16443 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.665+0000 7f83e6469740 1 -- 172.21.15.139:0/296710952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x557962d90620 con 0x55796304d410 2024-09-16T01:49:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.668+0000 7f83c4ff9640 1 -- 172.21.15.139:0/296710952 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc0d2e30 con 0x55796304d410 2024-09-16T01:49:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.668+0000 7f83c4ff9640 1 -- 172.21.15.139:0/296710952 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7f83cc017360 con 0x55796304d410 2024-09-16T01:49:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.669+0000 7f83e6469740 10 client.16443 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.669+0000 7f83e6469740 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:49:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.669+0000 7f83e6469740 20 client.16443 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.669+0000 7f83e6469740 1 --2- 172.21.15.139:0/296710952 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557963067b00 0x557963069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.669+0000 7f83e6469740 4 client.16443 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:49:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.669+0000 7f83e6469740 1 -- 172.21.15.139:0/296710952 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557962fbea20 con 0x557963067b00 2024-09-16T01:49:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.673+0000 7f83e1392640 1 --2- 172.21.15.139:0/296710952 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557963067b00 0x557963069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.694+0000 7f83e1392640 1 --2- 172.21.15.139:0/296710952 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557963067b00 0x557963069ef0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a8a37bd867bcace server_cookie=fc8cf4ada1f7997d in_seq=0 out_seq=0 2024-09-16T01:49:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.694+0000 7f83c4ff9640 10 client.16443 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.697+0000 7f83c4ff9640 1 -- 172.21.15.139:0/296710952 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557962fbea20 con 0x557963067b00 2024-09-16T01:49:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.697+0000 7f83c4ff9640 10 client.16443 handle_command_reply: tid=0 2024-09-16T01:49:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.698+0000 7f83e6469740 1 client.16443 shutdown 2024-09-16T01:49:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.702+0000 7f83e6469740 1 -- 172.21.15.139:0/296710952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304d410 msgr2=0x557963055160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.702+0000 7f83e6469740 1 --2- 172.21.15.139:0/296710952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304d410 0x557963055160 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f83cc097bf0 tx=0x7f83cc0047d0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.703+0000 7f83e6469740 1 -- 172.21.15.139:0/296710952 shutdown_connections 2024-09-16T01:49:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.703+0000 7f83e6469740 1 --2- 172.21.15.139:0/296710952 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557963067b00 0x557963069ef0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.703+0000 7f83e6469740 1 --2- 172.21.15.139:0/296710952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x5579630556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.703+0000 7f83e6469740 1 --2- 172.21.15.139:0/296710952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304d410 0x557963055160 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.703+0000 7f83e6469740 1 --2- 172.21.15.139:0/296710952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304caf0 0x557963054c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.703+0000 7f83e6469740 1 -- 172.21.15.139:0/296710952 >> 172.21.15.139:0/296710952 conn(0x557963049060 msgr2=0x55796304edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.706+0000 7f83c4ff9640 0 client.16443 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.706+0000 7f83e6469740 1 -- 172.21.15.139:0/296710952 shutdown_connections 2024-09-16T01:49:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.706+0000 7f83e6469740 1 -- 172.21.15.139:0/296710952 wait complete. 2024-09-16T01:49:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.707+0000 7f83e6469740 20 client.16443 trim_cache size 0 max 16384 2024-09-16T01:49:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.736+0000 7f83e6469740 1 Processor -- start 2024-09-16T01:49:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.736+0000 7f83e6469740 1 -- start start 2024-09-16T01:49:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.736+0000 7f83e6469740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x5579630554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.736+0000 7f83e6469740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 0x557963056b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.736+0000 7f83e6469740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x55796306f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.736+0000 7f83e6469740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f83c00101d0 con 0x55796304caf0 2024-09-16T01:49:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.736+0000 7f83e6469740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83c0012010 con 0x55796304d410 2024-09-16T01:49:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.736+0000 7f83e6469740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f83c0010960 con 0x557963057cb0 2024-09-16T01:49:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.737+0000 7f83e1392640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x5579630554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.737+0000 7f83e0b91640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 0x557963056b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.737+0000 7f83e1392640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x5579630554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39696/0 (socket says 172.21.15.139:39696) 2024-09-16T01:49:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.737+0000 7f83e1392640 1 -- 172.21.15.139:0/371721897 learned_addr learned my addr 172.21.15.139:0/371721897 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.737+0000 7f83e1b93640 1 --2- 172.21.15.139:0/371721897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x55796306f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.738+0000 7f83e0b91640 1 -- 172.21.15.139:0/371721897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 msgr2=0x55796306f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.738+0000 7f83e0b91640 1 --2- 172.21.15.139:0/371721897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x55796306f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.738+0000 7f83e0b91640 1 -- 172.21.15.139:0/371721897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 msgr2=0x5579630554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.738+0000 7f83e0b91640 1 --2- 172.21.15.139:0/371721897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x5579630554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.738+0000 7f83e0b91640 1 -- 172.21.15.139:0/371721897 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557963064d80 con 0x55796304d410 2024-09-16T01:49:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.741+0000 7f83e0b91640 1 --2- 172.21.15.139:0/371721897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 0x557963056b90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x5579630561d0 tx=0x7f83cc0bb1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ec4f17f8c846b58 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.742+0000 7f83c6ffd640 1 -- 172.21.15.139:0/371721897 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc01e070 con 0x55796304d410 2024-09-16T01:49:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.742+0000 7f83c6ffd640 1 -- 172.21.15.139:0/371721897 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc09f5d0 con 0x55796304d410 2024-09-16T01:49:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.742+0000 7f83c6ffd640 1 -- 172.21.15.139:0/371721897 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc09aff0 con 0x55796304d410 2024-09-16T01:49:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.743+0000 7f83e6469740 1 -- 172.21.15.139:0/371721897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 msgr2=0x557963056b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.744+0000 7f83e6469740 1 --2- 172.21.15.139:0/371721897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 0x557963056b90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x5579630561d0 tx=0x7f83cc0bb1d0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.747+0000 7f83e6469740 1 -- 172.21.15.139:0/371721897 shutdown_connections 2024-09-16T01:49:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.747+0000 7f83e6469740 1 --2- 172.21.15.139:0/371721897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x55796306f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.748+0000 7f83e6469740 1 --2- 172.21.15.139:0/371721897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 0x557963056b90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.748+0000 7f83e6469740 1 --2- 172.21.15.139:0/371721897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x5579630554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.748+0000 7f83e6469740 1 -- 172.21.15.139:0/371721897 >> 172.21.15.139:0/371721897 conn(0x557963049060 msgr2=0x557963056480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.751+0000 7f83e6469740 1 -- 172.21.15.139:0/371721897 shutdown_connections 2024-09-16T01:49:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.751+0000 7f83e6469740 1 -- 172.21.15.139:0/371721897 wait complete. 2024-09-16T01:49:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.751+0000 7f83e6469740 1 -- start start 2024-09-16T01:49:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.752+0000 7f83e6469740 1 Processor -- start 2024-09-16T01:49:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.752+0000 7f83e6469740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x55796305ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.753+0000 7f83e6469740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 0x557963060450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.753+0000 7f83e6469740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x557963051430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.753+0000 7f83e6469740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f83c0011880 con 0x55796304caf0 2024-09-16T01:49:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.753+0000 7f83e6469740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83c00110f0 con 0x55796304d410 2024-09-16T01:49:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.753+0000 7f83e6469740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f83c000e1d0 con 0x557963057cb0 2024-09-16T01:49:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.753+0000 7f83e6469740 10 client.0 _ll_register_callbacks cb 0x7f83d4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.754+0000 7f83e1392640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x55796305ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.754+0000 7f83e1392640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x55796305ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39700/0 (socket says 172.21.15.139:39700) 2024-09-16T01:49:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.754+0000 7f83e1392640 1 -- 172.21.15.139:0/463860816 learned_addr learned my addr 172.21.15.139:0/463860816 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.754+0000 7f83e1b93640 1 --2- 172.21.15.139:0/463860816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x557963051430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.754+0000 7f83e1392640 1 -- 172.21.15.139:0/463860816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 msgr2=0x557963051430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.754+0000 7f83e1392640 1 --2- 172.21.15.139:0/463860816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x557963051430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.754+0000 7f83e1392640 1 -- 172.21.15.139:0/463860816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 msgr2=0x557963060450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.754+0000 7f83e0b91640 1 --2- 172.21.15.139:0/463860816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 0x557963060450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.754+0000 7f83e1392640 1 --2- 172.21.15.139:0/463860816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 0x557963060450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.755+0000 7f83e1392640 1 -- 172.21.15.139:0/463860816 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557963051c60 con 0x55796304caf0 2024-09-16T01:49:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.755+0000 7f83e1b93640 1 --2- 172.21.15.139:0/463860816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x557963051430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:49:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.755+0000 7f83e1392640 1 --2- 172.21.15.139:0/463860816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x55796305ff10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f83d8005e10 tx=0x7f83d80189a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4686d219e07d16cf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.755+0000 7f83c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:49:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.756+0000 7f83c4ff9640 1 -- 172.21.15.139:0/463860816 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8018cd0 con 0x55796304caf0 2024-09-16T01:49:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.756+0000 7f83c4ff9640 1 -- 172.21.15.139:0/463860816 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9642+0+0 (secure 0 0 0) 0x7f83d8011970 con 0x55796304caf0 2024-09-16T01:49:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.757+0000 7f83e6469740 1 -- 172.21.15.139:0/463860816 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83cc09f5d0 con 0x55796304caf0 2024-09-16T01:49:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.760+0000 7f83c4ff9640 1 -- 172.21.15.139:0/463860816 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f83d8011b70 con 0x55796304caf0 2024-09-16T01:49:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.764+0000 7f83e6469740 10 client.16449 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.764+0000 7f83e6469740 1 -- 172.21.15.139:0/463860816 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5579630d1ff0 con 0x55796304caf0 2024-09-16T01:49:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.772+0000 7f83c4ff9640 1 -- 172.21.15.139:0/463860816 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d8011d10 con 0x55796304caf0 2024-09-16T01:49:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.772+0000 7f83c4ff9640 1 -- 172.21.15.139:0/463860816 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7f83d801f070 con 0x55796304caf0 2024-09-16T01:49:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.773+0000 7f83e6469740 10 client.16449 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.773+0000 7f83e6469740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:49:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.773+0000 7f83e6469740 20 client.16449 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.773+0000 7f83e6469740 1 --2- 172.21.15.139:0/463860816 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5579630d27b0 0x5579630d4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.773+0000 7f83e6469740 4 client.16449 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:49:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.773+0000 7f83e6469740 1 -- 172.21.15.139:0/463860816 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f83cc09f5d0 con 0x5579630d27b0 2024-09-16T01:49:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.774+0000 7f83e0b91640 1 --2- 172.21.15.139:0/463860816 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5579630d27b0 0x5579630d4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.776+0000 7f83e0b91640 1 --2- 172.21.15.139:0/463860816 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5579630d27b0 0x5579630d4bd0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ba4f83928ea3df server_cookie=4308a43ccc8bba58 in_seq=0 out_seq=0 2024-09-16T01:49:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.776+0000 7f83c4ff9640 10 client.16449 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.865+0000 7f83c4ff9640 1 -- 172.21.15.139:0/463860816 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f83cc09f5d0 con 0x5579630d27b0 2024-09-16T01:49:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.865+0000 7f83c4ff9640 10 client.16449 handle_command_reply: tid=0 2024-09-16T01:49:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.865+0000 7f83e6469740 1 client.16449 shutdown 2024-09-16T01:49:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.865+0000 7f83e6469740 1 -- 172.21.15.139:0/463860816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 msgr2=0x55796305ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.865+0000 7f83e6469740 1 --2- 172.21.15.139:0/463860816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x55796305ff10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f83d8005e10 tx=0x7f83d80189a0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.865+0000 7f83e6469740 1 -- 172.21.15.139:0/463860816 shutdown_connections 2024-09-16T01:49:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.865+0000 7f83e6469740 1 --2- 172.21.15.139:0/463860816 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5579630d27b0 0x5579630d4bd0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.865+0000 7f83e6469740 1 --2- 172.21.15.139:0/463860816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557963057cb0 0x557963051430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.865+0000 7f83e6469740 1 --2- 172.21.15.139:0/463860816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55796304d410 0x557963060450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.865+0000 7f83e6469740 1 --2- 172.21.15.139:0/463860816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55796304caf0 0x55796305ff10 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.865+0000 7f83e6469740 1 -- 172.21.15.139:0/463860816 >> 172.21.15.139:0/463860816 conn(0x557963049060 msgr2=0x5579630481c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.866+0000 7f83c4ff9640 0 client.16449 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.866+0000 7f83e6469740 1 -- 172.21.15.139:0/463860816 shutdown_connections 2024-09-16T01:49:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.866+0000 7f83e6469740 1 -- 172.21.15.139:0/463860816 wait complete. 2024-09-16T01:49:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.866+0000 7f83e6469740 20 client.16449 trim_cache size 0 max 16384 2024-09-16T01:49:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:49:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:49:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:49:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:49:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.870+0000 7f83e35f6640 1 -- 172.21.15.139:0/1188881200 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f83c80a3510 msgr2=0x7f83c80a59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.870+0000 7f83e35f6640 1 --2- 172.21.15.139:0/1188881200 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f83c80a3510 0x7f83c80a59d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f83d4031550 tx=0x7f83d402a440 comp rx=0 tx=0).stop 2024-09-16T01:49:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.870+0000 7f83e35f6640 1 -- 172.21.15.139:0/1188881200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83dc1013b0 msgr2=0x7f83dc06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.870+0000 7f83e35f6640 1 --2- 172.21.15.139:0/1188881200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83dc1013b0 0x7f83dc06ca40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00b4a0 tx=0x7f83cc00b960 comp rx=0 tx=0).stop 2024-09-16T01:49:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.870+0000 7f83e35f6640 1 -- 172.21.15.139:0/1188881200 shutdown_connections 2024-09-16T01:49:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.870+0000 7f83e35f6640 1 --2- 172.21.15.139:0/1188881200 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f83c80a3510 0x7f83c80a59d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.870+0000 7f83e35f6640 1 --2- 172.21.15.139:0/1188881200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83dc105fa0 0x7f83dc06cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.870+0000 7f83e35f6640 1 --2- 172.21.15.139:0/1188881200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83dc1013b0 0x7f83dc06ca40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.870+0000 7f83e35f6640 1 --2- 172.21.15.139:0/1188881200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83dc100a90 0x7f83dc0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.870+0000 7f83e35f6640 1 -- 172.21.15.139:0/1188881200 >> 172.21.15.139:0/1188881200 conn(0x7f83dc0fc6f0 msgr2=0x7f83dc103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.870+0000 7f83e35f6640 1 -- 172.21.15.139:0/1188881200 shutdown_connections 2024-09-16T01:49:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:14.870+0000 7f83e35f6640 1 -- 172.21.15.139:0/1188881200 wait complete. 2024-09-16T01:49:15.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:15 smithi139 ceph-mon[31175]: pgmap v274: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 251 op/s 2024-09-16T01:49:15.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:15 smithi176 ceph-mon[35324]: pgmap v274: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 251 op/s 2024-09-16T01:49:15.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:15 smithi164 ceph-mon[35121]: pgmap v274: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 251 op/s 2024-09-16T01:49:16.669 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T01:49:16.669 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.35 2024-09-16T01:49:16.670 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:49:17.148 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:49:17.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:17 smithi139 ceph-mon[31175]: pgmap v275: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 272 op/s 2024-09-16T01:49:17.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:17 smithi176 ceph-mon[35324]: pgmap v275: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 272 op/s 2024-09-16T01:49:17.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:17 smithi164 ceph-mon[35121]: pgmap v275: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 272 op/s 2024-09-16T01:49:18.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.117+0000 7fd4ef245640 1 -- 172.21.15.139:0/1526708572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e8109830 msgr2=0x7fd4e8109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:18.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.117+0000 7fd4ef245640 1 --2- 172.21.15.139:0/1526708572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e8109830 0x7fd4e8109c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009900 tx=0x7fd4d402a440 comp rx=0 tx=0).stop 2024-09-16T01:49:18.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.117+0000 7fd4ef245640 1 -- 172.21.15.139:0/1526708572 shutdown_connections 2024-09-16T01:49:18.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.117+0000 7fd4ef245640 1 --2- 172.21.15.139:0/1526708572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e811b140 0x7fd4e811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:18.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.117+0000 7fd4ef245640 1 --2- 172.21.15.139:0/1526708572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e8109830 0x7fd4e8109c90 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009900 tx=0x7fd4d402a440 comp rx=0 tx=0).stop 2024-09-16T01:49:18.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.117+0000 7fd4ef245640 1 --2- 172.21.15.139:0/1526708572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e8109450 0x7fd4e8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:18.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.117+0000 7fd4ef245640 1 -- 172.21.15.139:0/1526708572 >> 172.21.15.139:0/1526708572 conn(0x7fd4e806e140 msgr2=0x7fd4e806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:18.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.121+0000 7fd4ef245640 1 -- 172.21.15.139:0/1526708572 shutdown_connections 2024-09-16T01:49:18.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.121+0000 7fd4ef245640 1 -- 172.21.15.139:0/1526708572 wait complete. 2024-09-16T01:49:18.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.122+0000 7fd4ef245640 1 Processor -- start 2024-09-16T01:49:18.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.124+0000 7fd4ef245640 1 -- start start 2024-09-16T01:49:18.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.124+0000 7fd4ef245640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e8109450 0x7fd4e811a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:18.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.124+0000 7fd4ef245640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e811b140 0x7fd4e8113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:18.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.124+0000 7fd4ef245640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e8113e60 0x7fd4e8114310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:18.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.124+0000 7fd4ef245640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8120e30 con 0x7fd4e811b140 2024-09-16T01:49:18.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.124+0000 7fd4ef245640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8120cb0 con 0x7fd4e8113e60 2024-09-16T01:49:18.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.124+0000 7fd4ef245640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8120fb0 con 0x7fd4e8109450 2024-09-16T01:49:18.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.124+0000 7fd4e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e811b140 0x7fd4e8113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:18.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.124+0000 7fd4e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e811b140 0x7fd4e8113920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39712/0 (socket says 172.21.15.139:39712) 2024-09-16T01:49:18.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.124+0000 7fd4e7fff640 1 -- 172.21.15.139:0/2527938554 learned_addr learned my addr 172.21.15.139:0/2527938554 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:18.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.125+0000 7fd4ed7e2640 1 --2- 172.21.15.139:0/2527938554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e8113e60 0x7fd4e8114310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:18.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.125+0000 7fd4ecfe1640 1 --2- 172.21.15.139:0/2527938554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e8109450 0x7fd4e811a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:18.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.125+0000 7fd4e7fff640 1 -- 172.21.15.139:0/2527938554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e8109450 msgr2=0x7fd4e811a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:18.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.126+0000 7fd4e7fff640 1 --2- 172.21.15.139:0/2527938554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e8109450 0x7fd4e811a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:18.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.126+0000 7fd4e7fff640 1 -- 172.21.15.139:0/2527938554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e8113e60 msgr2=0x7fd4e8114310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:18.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.126+0000 7fd4e7fff640 1 --2- 172.21.15.139:0/2527938554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e8113e60 0x7fd4e8114310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:18.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.126+0000 7fd4e7fff640 1 -- 172.21.15.139:0/2527938554 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e8118440 con 0x7fd4e811b140 2024-09-16T01:49:18.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.129+0000 7fd4e7fff640 1 --2- 172.21.15.139:0/2527938554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e811b140 0x7fd4e8113920 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd4d402aa30 tx=0x7fd4d4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:18.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.130+0000 7fd4e5ffb640 1 -- 172.21.15.139:0/2527938554 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d403f020 con 0x7fd4e811b140 2024-09-16T01:49:18.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.131+0000 7fd4e5ffb640 1 -- 172.21.15.139:0/2527938554 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d4031c20 con 0x7fd4e811b140 2024-09-16T01:49:18.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.131+0000 7fd4ef245640 1 -- 172.21.15.139:0/2527938554 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e81186d0 con 0x7fd4e811b140 2024-09-16T01:49:18.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.131+0000 7fd4ef245640 1 -- 172.21.15.139:0/2527938554 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e8118b80 con 0x7fd4e811b140 2024-09-16T01:49:18.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.132+0000 7fd4e5ffb640 1 -- 172.21.15.139:0/2527938554 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4035070 con 0x7fd4e811b140 2024-09-16T01:49:18.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.133+0000 7fd4ef245640 1 -- 172.21.15.139:0/2527938554 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e8073b00 con 0x7fd4e811b140 2024-09-16T01:49:18.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.135+0000 7fd4e5ffb640 1 -- 172.21.15.139:0/2527938554 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd4d4042020 con 0x7fd4e811b140 2024-09-16T01:49:18.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.136+0000 7fd4e5ffb640 1 --2- 172.21.15.139:0/2527938554 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd4d80a35c0 0x7fd4d80a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:18.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.136+0000 7fd4e5ffb640 1 -- 172.21.15.139:0/2527938554 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9642+0+0 (secure 0 0 0) 0x7fd4d40f0470 con 0x7fd4e811b140 2024-09-16T01:49:18.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.137+0000 7fd4ecfe1640 1 --2- 172.21.15.139:0/2527938554 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd4d80a35c0 0x7fd4d80a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:18.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.138+0000 7fd4ecfe1640 1 --2- 172.21.15.139:0/2527938554 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd4d80a35c0 0x7fd4d80a5a80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005fd0 tx=0x7fd4dc005850 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:18.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.146+0000 7fd4e5ffb640 1 -- 172.21.15.139:0/2527938554 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4d4049060 con 0x7fd4e811b140 2024-09-16T01:49:18.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.558+0000 7fd4ef245640 1 -- 172.21.15.139:0/2527938554 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4e810d5c0 con 0x7fd4e811b140 2024-09-16T01:49:18.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.564+0000 7fd4e5ffb640 1 -- 172.21.15.139:0/2527938554 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+12028 (secure 0 0 0) 0x7fd4d40bca20 con 0x7fd4e811b140 2024-09-16T01:49:18.566 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:49:18.567 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":50,"btime":"2024-09-16T01:48:57:039963+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:48:56.596845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24454,24443,24442,24433]},"id":1}]} 2024-09-16T01:49:18.567 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 50 2024-09-16T01:49:18.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.568+0000 7fd4ef245640 1 -- 172.21.15.139:0/2527938554 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd4d80a35c0 msgr2=0x7fd4d80a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:18.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.568+0000 7fd4ef245640 1 --2- 172.21.15.139:0/2527938554 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd4d80a35c0 0x7fd4d80a5a80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005fd0 tx=0x7fd4dc005850 comp rx=0 tx=0).stop 2024-09-16T01:49:18.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.568+0000 7fd4ef245640 1 -- 172.21.15.139:0/2527938554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e811b140 msgr2=0x7fd4e8113920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:18.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.568+0000 7fd4ef245640 1 --2- 172.21.15.139:0/2527938554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e811b140 0x7fd4e8113920 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd4d402aa30 tx=0x7fd4d4003b90 comp rx=0 tx=0).stop 2024-09-16T01:49:18.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.568+0000 7fd4ef245640 1 -- 172.21.15.139:0/2527938554 shutdown_connections 2024-09-16T01:49:18.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.568+0000 7fd4ef245640 1 --2- 172.21.15.139:0/2527938554 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd4d80a35c0 0x7fd4d80a5a80 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005fd0 tx=0x7fd4dc005850 comp rx=0 tx=0).stop 2024-09-16T01:49:18.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.568+0000 7fd4ef245640 1 --2- 172.21.15.139:0/2527938554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e8113e60 0x7fd4e8114310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:18.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.568+0000 7fd4ef245640 1 --2- 172.21.15.139:0/2527938554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e811b140 0x7fd4e8113920 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:18.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.568+0000 7fd4ef245640 1 --2- 172.21.15.139:0/2527938554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e8109450 0x7fd4e811a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:18.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.568+0000 7fd4ef245640 1 -- 172.21.15.139:0/2527938554 >> 172.21.15.139:0/2527938554 conn(0x7fd4e806e140 msgr2=0x7fd4e8072440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:18.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.569+0000 7fd4ef245640 1 -- 172.21.15.139:0/2527938554 shutdown_connections 2024-09-16T01:49:18.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:18.569+0000 7fd4ef245640 1 -- 172.21.15.139:0/2527938554 wait complete. 2024-09-16T01:49:19.198 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:19 smithi139 ceph-mon[31175]: pgmap v276: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-09-16T01:49:19.198 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:19 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2527938554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:49:19.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:19 smithi176 ceph-mon[35324]: pgmap v276: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-09-16T01:49:19.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:19 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2527938554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:49:19.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:19 smithi164 ceph-mon[35121]: pgmap v276: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-09-16T01:49:19.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:19 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2527938554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:49:19.572 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0' 2024-09-16T01:49:19.572 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0 2024-09-16T01:49:20.058 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:49:20.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.733+0000 7ff95660d640 1 -- 172.21.15.139:0/960169246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9501079c0 msgr2=0x7ff950107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:20.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.733+0000 7ff95660d640 1 --2- 172.21.15.139:0/960169246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9501079c0 0x7ff950107da0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff948009930 tx=0x7ff94802a480 comp rx=0 tx=0).stop 2024-09-16T01:49:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.734+0000 7ff95660d640 1 -- 172.21.15.139:0/960169246 shutdown_connections 2024-09-16T01:49:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.734+0000 7ff95660d640 1 --2- 172.21.15.139:0/960169246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff950102360 0x7ff95010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.734+0000 7ff95660d640 1 --2- 172.21.15.139:0/960169246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9501019c0 0x7ff950101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.734+0000 7ff95660d640 1 --2- 172.21.15.139:0/960169246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9501079c0 0x7ff950107da0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.734+0000 7ff95660d640 1 -- 172.21.15.139:0/960169246 >> 172.21.15.139:0/960169246 conn(0x7ff9500fd740 msgr2=0x7ff9500ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.737+0000 7ff95660d640 1 -- 172.21.15.139:0/960169246 shutdown_connections 2024-09-16T01:49:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.737+0000 7ff95660d640 1 -- 172.21.15.139:0/960169246 wait complete. 2024-09-16T01:49:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.738+0000 7ff95660d640 1 Processor -- start 2024-09-16T01:49:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.738+0000 7ff95660d640 1 -- start start 2024-09-16T01:49:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.738+0000 7ff95660d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9501019c0 0x7ff95006bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.738+0000 7ff95660d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff950102360 0x7ff95006c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.738+0000 7ff95660d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9501079c0 0x7ff950071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.738+0000 7ff95660d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff9500748f0 con 0x7ff9501079c0 2024-09-16T01:49:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.738+0000 7ff95660d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff950074770 con 0x7ff9501019c0 2024-09-16T01:49:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.738+0000 7ff95660d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff950074a70 con 0x7ff950102360 2024-09-16T01:49:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.739+0000 7ff954e0a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff950102360 0x7ff95006c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.739+0000 7ff954e0a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff950102360 0x7ff95006c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43118/0 (socket says 172.21.15.139:43118) 2024-09-16T01:49:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.739+0000 7ff954e0a640 1 -- 172.21.15.139:0/2044318933 learned_addr learned my addr 172.21.15.139:0/2044318933 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.739+0000 7ff95560b640 1 --2- 172.21.15.139:0/2044318933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9501019c0 0x7ff95006bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.739+0000 7ff955e0c640 1 --2- 172.21.15.139:0/2044318933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9501079c0 0x7ff950071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.742+0000 7ff95560b640 1 -- 172.21.15.139:0/2044318933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff950102360 msgr2=0x7ff95006c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.742+0000 7ff95560b640 1 --2- 172.21.15.139:0/2044318933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff950102360 0x7ff95006c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.742+0000 7ff95560b640 1 -- 172.21.15.139:0/2044318933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9501079c0 msgr2=0x7ff950071210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.742+0000 7ff95560b640 1 --2- 172.21.15.139:0/2044318933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9501079c0 0x7ff950071210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.742+0000 7ff95560b640 1 -- 172.21.15.139:0/2044318933 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9500717d0 con 0x7ff9501019c0 2024-09-16T01:49:20.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.743+0000 7ff95560b640 1 --2- 172.21.15.139:0/2044318933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9501019c0 0x7ff95006bde0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff94802aa70 tx=0x7ff9480053e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:20.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.747+0000 7ff9467fc640 1 -- 172.21.15.139:0/2044318933 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948040030 con 0x7ff9501019c0 2024-09-16T01:49:20.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.747+0000 7ff9467fc640 1 -- 172.21.15.139:0/2044318933 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff94803cb30 con 0x7ff9501019c0 2024-09-16T01:49:20.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.747+0000 7ff9467fc640 1 -- 172.21.15.139:0/2044318933 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948031af0 con 0x7ff9501019c0 2024-09-16T01:49:20.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.748+0000 7ff95660d640 1 -- 172.21.15.139:0/2044318933 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff950071a60 con 0x7ff9501019c0 2024-09-16T01:49:20.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.748+0000 7ff95660d640 1 -- 172.21.15.139:0/2044318933 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff950071ec0 con 0x7ff9501019c0 2024-09-16T01:49:20.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.752+0000 7ff9467fc640 1 -- 172.21.15.139:0/2044318933 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff94804d060 con 0x7ff9501019c0 2024-09-16T01:49:20.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.753+0000 7ff9467fc640 1 --2- 172.21.15.139:0/2044318933 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff93c0a35c0 0x7ff93c0a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.753+0000 7ff9467fc640 1 -- 172.21.15.139:0/2044318933 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9642+0+0 (secure 0 0 0) 0x7ff94804b070 con 0x7ff9501019c0 2024-09-16T01:49:20.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.753+0000 7ff954e0a640 1 --2- 172.21.15.139:0/2044318933 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff93c0a35c0 0x7ff93c0a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.755+0000 7ff959c48740 1 Processor -- start 2024-09-16T01:49:20.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.756+0000 7ff959c48740 1 -- start start 2024-09-16T01:49:20.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.756+0000 7ff959c48740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 0x557f696ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.756+0000 7ff959c48740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69712ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.756+0000 7ff959c48740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f69713410 0x557f69715800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.756+0000 7ff959c48740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557f69664160 con 0x557f69700260 2024-09-16T01:49:20.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.756+0000 7ff959c48740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557f6960b720 con 0x557f696ff940 2024-09-16T01:49:20.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.756+0000 7ff959c48740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557f69660120 con 0x557f69713410 2024-09-16T01:49:20.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.758+0000 7ff954e0a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69712ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.758+0000 7ff954e0a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69712ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39726/0 (socket says 172.21.15.139:39726) 2024-09-16T01:49:20.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.758+0000 7ff954e0a640 1 -- 172.21.15.139:0/1574415488 learned_addr learned my addr 172.21.15.139:0/1574415488 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:20.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.760+0000 7ff955e0c640 1 --2- 172.21.15.139:0/1574415488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f69713410 0x557f69715800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.760+0000 7ff954e0a640 1 --2- 172.21.15.139:0/2044318933 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff93c0a35c0 0x7ff93c0a5a80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff95010ed60 tx=0x7ff94002b1a0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:20.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.760+0000 7ff95560b640 1 --2- 172.21.15.139:0/1574415488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 0x557f696ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.762+0000 7ff954e0a640 1 -- 172.21.15.139:0/1574415488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f69713410 msgr2=0x557f69715800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:20.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.762+0000 7ff954e0a640 1 --2- 172.21.15.139:0/1574415488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f69713410 0x557f69715800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.762+0000 7ff954e0a640 1 -- 172.21.15.139:0/1574415488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 msgr2=0x557f696ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:20.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.762+0000 7ff954e0a640 1 --2- 172.21.15.139:0/1574415488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 0x557f696ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.762+0000 7ff954e0a640 1 -- 172.21.15.139:0/1574415488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f693d5ba0 con 0x557f69700260 2024-09-16T01:49:20.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.762+0000 7ff954e0a640 1 --2- 172.21.15.139:0/1574415488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69712ed0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff940030a50 tx=0x7ff940033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8332967f4fa4cfd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:20.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.763+0000 7ff93b7fe640 1 -- 172.21.15.139:0/1574415488 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940044070 con 0x557f69700260 2024-09-16T01:49:20.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.763+0000 7ff93b7fe640 1 -- 172.21.15.139:0/1574415488 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff94003a070 con 0x557f69700260 2024-09-16T01:49:20.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.763+0000 7ff93b7fe640 1 -- 172.21.15.139:0/1574415488 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9400363f0 con 0x557f69700260 2024-09-16T01:49:20.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.766+0000 7ff959c48740 1 -- 172.21.15.139:0/1574415488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 msgr2=0x557f69712ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:20.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.766+0000 7ff959c48740 1 --2- 172.21.15.139:0/1574415488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69712ed0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff940030a50 tx=0x7ff940033000 comp rx=0 tx=0).stop 2024-09-16T01:49:20.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.767+0000 7ff959c48740 1 -- 172.21.15.139:0/1574415488 shutdown_connections 2024-09-16T01:49:20.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.767+0000 7ff959c48740 1 --2- 172.21.15.139:0/1574415488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f69713410 0x557f69715800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.767+0000 7ff959c48740 1 --2- 172.21.15.139:0/1574415488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69712ed0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.767+0000 7ff959c48740 1 --2- 172.21.15.139:0/1574415488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 0x557f696ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.767+0000 7ff959c48740 1 -- 172.21.15.139:0/1574415488 >> 172.21.15.139:0/1574415488 conn(0x557f696fbfb0 msgr2=0x557f696fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:20.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.767+0000 7ff959c48740 1 -- 172.21.15.139:0/1574415488 shutdown_connections 2024-09-16T01:49:20.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.768+0000 7ff959c48740 1 -- 172.21.15.139:0/1574415488 wait complete. 2024-09-16T01:49:20.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.768+0000 7ff959c48740 1 -- start start 2024-09-16T01:49:20.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.768+0000 7ff959c48740 1 Processor -- start 2024-09-16T01:49:20.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.769+0000 7ff959c48740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 0x557f69710450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.769+0000 7ff959c48740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69710990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.769+0000 7ff959c48740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f69713410 0x557f69710ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.769+0000 7ff959c48740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557f69715f80 con 0x557f69700260 2024-09-16T01:49:20.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.769+0000 7ff959c48740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557f695ec900 con 0x557f696ff940 2024-09-16T01:49:20.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.769+0000 7ff959c48740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557f69716100 con 0x557f69713410 2024-09-16T01:49:20.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.769+0000 7ff959c48740 10 client.0 _ll_register_callbacks cb 0x557f695ea780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:20.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.769+0000 7ff955e0c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f69713410 0x557f69710ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.769+0000 7ff955e0c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f69713410 0x557f69710ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43130/0 (socket says 172.21.15.139:43130) 2024-09-16T01:49:20.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.769+0000 7ff955e0c640 1 -- 172.21.15.139:0/3042675262 learned_addr learned my addr 172.21.15.139:0/3042675262 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:20.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.769+0000 7ff95560b640 1 --2- 172.21.15.139:0/3042675262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 0x557f69710450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.770+0000 7ff954e0a640 1 --2- 172.21.15.139:0/3042675262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69710990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.770+0000 7ff95560b640 1 -- 172.21.15.139:0/3042675262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f69713410 msgr2=0x557f69710ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:20.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.770+0000 7ff95560b640 1 --2- 172.21.15.139:0/3042675262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f69713410 0x557f69710ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.770+0000 7ff95560b640 1 -- 172.21.15.139:0/3042675262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 msgr2=0x557f69710990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:20.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.770+0000 7ff95560b640 1 --2- 172.21.15.139:0/3042675262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69710990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.770+0000 7ff95560b640 1 -- 172.21.15.139:0/3042675262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f6962cc50 con 0x557f696ff940 2024-09-16T01:49:20.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.770+0000 7ff95560b640 1 --2- 172.21.15.139:0/3042675262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 0x557f69710450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff9480ed520 tx=0x7ff9480bbd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a5b72deea39bb56 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:20.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.771+0000 7ff938ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:49:20.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.771+0000 7ff938ff9640 1 -- 172.21.15.139:0/3042675262 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94804a040 con 0x557f696ff940 2024-09-16T01:49:20.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.771+0000 7ff938ff9640 1 -- 172.21.15.139:0/3042675262 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9642+0+0 (secure 0 0 0) 0x7ff9480c0080 con 0x557f696ff940 2024-09-16T01:49:20.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.771+0000 7ff959c48740 1 -- 172.21.15.139:0/3042675262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff94003a070 con 0x557f696ff940 2024-09-16T01:49:20.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.771+0000 7ff938ff9640 1 -- 172.21.15.139:0/3042675262 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff948031440 con 0x557f696ff940 2024-09-16T01:49:20.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.772+0000 7ff959c48740 10 client.25916 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:20.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.772+0000 7ff959c48740 1 -- 172.21.15.139:0/3042675262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x557f69704f10 con 0x557f696ff940 2024-09-16T01:49:20.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.774+0000 7ff938ff9640 1 -- 172.21.15.139:0/3042675262 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff94003a070 con 0x557f696ff940 2024-09-16T01:49:20.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.774+0000 7ff938ff9640 1 -- 172.21.15.139:0/3042675262 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7ff9480ed7a0 con 0x557f696ff940 2024-09-16T01:49:20.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.775+0000 7ff959c48740 10 client.25916 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:20.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.775+0000 7ff959c48740 10 client.25916 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:49:20.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.775+0000 7ff959c48740 20 client.25916 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:20.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.775+0000 7ff959c48740 1 --2- 172.21.15.139:0/3042675262 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557f69706680 0x557f69708ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.775+0000 7ff959c48740 4 client.25916 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:49:20.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.775+0000 7ff959c48740 1 -- 172.21.15.139:0/3042675262 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f69671890 con 0x557f69706680 2024-09-16T01:49:20.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.776+0000 7ff954e0a640 1 --2- 172.21.15.139:0/3042675262 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557f69706680 0x557f69708ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.777+0000 7ff954e0a640 1 --2- 172.21.15.139:0/3042675262 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557f69706680 0x557f69708ac0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d79ce7d19e0ab23 server_cookie=6c0163066e14aa69 in_seq=0 out_seq=0 2024-09-16T01:49:20.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.777+0000 7ff938ff9640 10 client.25916 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:20.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.777+0000 7ff938ff9640 1 -- 172.21.15.139:0/3042675262 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f69671890 con 0x557f69706680 2024-09-16T01:49:20.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.777+0000 7ff938ff9640 10 client.25916 handle_command_reply: tid=0 2024-09-16T01:49:20.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.777+0000 7ff959c48740 1 client.25916 shutdown 2024-09-16T01:49:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.783+0000 7ff959c48740 1 -- 172.21.15.139:0/3042675262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 msgr2=0x557f69710450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.783+0000 7ff959c48740 1 --2- 172.21.15.139:0/3042675262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 0x557f69710450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff9480ed520 tx=0x7ff9480bbd60 comp rx=0 tx=0).stop 2024-09-16T01:49:20.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.784+0000 7ff959c48740 1 -- 172.21.15.139:0/3042675262 shutdown_connections 2024-09-16T01:49:20.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.784+0000 7ff959c48740 1 --2- 172.21.15.139:0/3042675262 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557f69706680 0x557f69708ac0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.784+0000 7ff959c48740 1 --2- 172.21.15.139:0/3042675262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f69713410 0x557f69710ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.784+0000 7ff959c48740 1 --2- 172.21.15.139:0/3042675262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69710990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.784+0000 7ff959c48740 1 --2- 172.21.15.139:0/3042675262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 0x557f69710450 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.784+0000 7ff959c48740 1 -- 172.21.15.139:0/3042675262 >> 172.21.15.139:0/3042675262 conn(0x557f696fbfb0 msgr2=0x557f69701c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:20.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.784+0000 7ff938ff9640 0 client.25916 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:20.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.784+0000 7ff959c48740 1 -- 172.21.15.139:0/3042675262 shutdown_connections 2024-09-16T01:49:20.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.784+0000 7ff959c48740 1 -- 172.21.15.139:0/3042675262 wait complete. 2024-09-16T01:49:20.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.785+0000 7ff959c48740 20 client.25916 trim_cache size 0 max 16384 2024-09-16T01:49:20.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.817+0000 7ff959c48740 1 Processor -- start 2024-09-16T01:49:20.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.819+0000 7ff959c48740 1 -- start start 2024-09-16T01:49:20.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.819+0000 7ff959c48740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f696fbfb0 0x557f696fc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.819+0000 7ff959c48740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f696ff940 0x557f696faf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.819+0000 7ff959c48740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f69700260 0x557f696fb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.820+0000 7ff95560b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f696fbfb0 0x557f696fc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.820+0000 7ff954e0a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f696ff940 0x557f696faf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.820+0000 7ff959c48740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff9501a07f0 con 0x557f696ff940 2024-09-16T01:49:20.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.820+0000 7ff959c48740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff9501a2630 con 0x557f69700260 2024-09-16T01:49:20.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.820+0000 7ff959c48740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff9501a0f80 con 0x557f696fbfb0 2024-09-16T01:49:20.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.820+0000 7ff95560b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f696fbfb0 0x557f696fc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43136/0 (socket says 172.21.15.139:43136) 2024-09-16T01:49:20.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.820+0000 7ff95560b640 1 -- 172.21.15.139:0/2362001449 learned_addr learned my addr 172.21.15.139:0/2362001449 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:20.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.820+0000 7ff955e0c640 1 --2- 172.21.15.139:0/2362001449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f69700260 0x557f696fb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.820+0000 7ff95560b640 1 -- 172.21.15.139:0/2362001449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f69700260 msgr2=0x557f696fb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:20.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.820+0000 7ff95560b640 1 --2- 172.21.15.139:0/2362001449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f69700260 0x557f696fb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.820+0000 7ff95560b640 1 -- 172.21.15.139:0/2362001449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f696ff940 msgr2=0x557f696faf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:20.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.820+0000 7ff95560b640 1 --2- 172.21.15.139:0/2362001449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f696ff940 0x557f696faf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.820+0000 7ff95560b640 1 -- 172.21.15.139:0/2362001449 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f69701a70 con 0x557f696fbfb0 2024-09-16T01:49:20.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.822+0000 7ff95560b640 1 --2- 172.21.15.139:0/2362001449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f696fbfb0 0x557f696fc390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff9480bbe60 tx=0x7ff9480bbdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=222c1813711154d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:20.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.823+0000 7ff93affd640 1 -- 172.21.15.139:0/2362001449 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94804a040 con 0x557f696fbfb0 2024-09-16T01:49:20.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.823+0000 7ff93affd640 1 -- 172.21.15.139:0/2362001449 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9480bc160 con 0x557f696fbfb0 2024-09-16T01:49:20.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.823+0000 7ff93affd640 1 -- 172.21.15.139:0/2362001449 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948031c90 con 0x557f696fbfb0 2024-09-16T01:49:20.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.824+0000 7ff959c48740 1 -- 172.21.15.139:0/2362001449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f696fbfb0 msgr2=0x557f696fc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:20.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.824+0000 7ff959c48740 1 --2- 172.21.15.139:0/2362001449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f696fbfb0 0x557f696fc390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff9480bbe60 tx=0x7ff9480bbdd0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.826+0000 7ff959c48740 1 -- 172.21.15.139:0/2362001449 shutdown_connections 2024-09-16T01:49:20.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.826+0000 7ff959c48740 1 --2- 172.21.15.139:0/2362001449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f69700260 0x557f696fb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.826+0000 7ff959c48740 1 --2- 172.21.15.139:0/2362001449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f696ff940 0x557f696faf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.826+0000 7ff959c48740 1 --2- 172.21.15.139:0/2362001449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f696fbfb0 0x557f696fc390 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.826+0000 7ff959c48740 1 -- 172.21.15.139:0/2362001449 >> 172.21.15.139:0/2362001449 conn(0x557f69701f30 msgr2=0x557f696fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:20.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.827+0000 7ff959c48740 1 -- 172.21.15.139:0/2362001449 shutdown_connections 2024-09-16T01:49:20.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.827+0000 7ff959c48740 1 -- 172.21.15.139:0/2362001449 wait complete. 2024-09-16T01:49:20.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.828+0000 7ff959c48740 1 -- start start 2024-09-16T01:49:20.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.828+0000 7ff959c48740 1 Processor -- start 2024-09-16T01:49:20.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.829+0000 7ff959c48740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f696fbfb0 0x557f6971bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.829+0000 7ff959c48740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 0x557f69718f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.829+0000 7ff959c48740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69719540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.829+0000 7ff954e0a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 0x557f69718f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.829+0000 7ff959c48740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff9501a1ea0 con 0x557f69700260 2024-09-16T01:49:20.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.829+0000 7ff95560b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f696fbfb0 0x557f6971bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.829+0000 7ff95560b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f696fbfb0 0x557f6971bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43138/0 (socket says 172.21.15.139:43138) 2024-09-16T01:49:20.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.829+0000 7ff95560b640 1 -- 172.21.15.139:0/3355873802 learned_addr learned my addr 172.21.15.139:0/3355873802 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:20.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.829+0000 7ff959c48740 1 -- 172.21.15.139:0/3355873802 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff9501a1710 con 0x557f696ff940 2024-09-16T01:49:20.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.829+0000 7ff959c48740 1 -- 172.21.15.139:0/3355873802 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff95019e850 con 0x557f696fbfb0 2024-09-16T01:49:20.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.829+0000 7ff959c48740 10 client.0 _ll_register_callbacks cb 0x7ff94803ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:20.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.830+0000 7ff955e0c640 1 --2- 172.21.15.139:0/3355873802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69719540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.830+0000 7ff955e0c640 1 -- 172.21.15.139:0/3355873802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f696fbfb0 msgr2=0x557f6971bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:20.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.830+0000 7ff955e0c640 1 --2- 172.21.15.139:0/3355873802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f696fbfb0 0x557f6971bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.830+0000 7ff955e0c640 1 -- 172.21.15.139:0/3355873802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 msgr2=0x557f69718f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:20.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.830+0000 7ff955e0c640 1 --2- 172.21.15.139:0/3355873802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 0x557f69718f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.830+0000 7ff955e0c640 1 -- 172.21.15.139:0/3355873802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f69719d70 con 0x557f69700260 2024-09-16T01:49:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.831+0000 7ff955e0c640 1 --2- 172.21.15.139:0/3355873802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69719540 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x557f696fb9e0 tx=0x7ff94c012e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbcb70abe38f2fb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.831+0000 7ff938ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:49:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.831+0000 7ff938ff9640 1 -- 172.21.15.139:0/3355873802 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c022060 con 0x557f69700260 2024-09-16T01:49:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.831+0000 7ff938ff9640 1 -- 172.21.15.139:0/3355873802 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9642+0+0 (secure 0 0 0) 0x7ff94c011e20 con 0x557f69700260 2024-09-16T01:49:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.832+0000 7ff959c48740 1 -- 172.21.15.139:0/3355873802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9480bc160 con 0x557f69700260 2024-09-16T01:49:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.832+0000 7ff938ff9640 1 -- 172.21.15.139:0/3355873802 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff94c01d3f0 con 0x557f69700260 2024-09-16T01:49:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.832+0000 7ff959c48740 10 client.16473 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:20.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.832+0000 7ff959c48740 1 -- 172.21.15.139:0/3355873802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x557f69785650 con 0x557f69700260 2024-09-16T01:49:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.833+0000 7ff938ff9640 1 -- 172.21.15.139:0/3355873802 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff94c01da70 con 0x557f69700260 2024-09-16T01:49:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.833+0000 7ff938ff9640 1 -- 172.21.15.139:0/3355873802 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7ff94c01c070 con 0x557f69700260 2024-09-16T01:49:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.833+0000 7ff959c48740 10 client.16473 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:20.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.833+0000 7ff959c48740 10 client.16473 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:49:20.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.833+0000 7ff959c48740 20 client.16473 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:20.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.833+0000 7ff959c48740 1 --2- 172.21.15.139:0/3355873802 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557f69713410 0x557f69787e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:20.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.833+0000 7ff959c48740 4 client.16473 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}] 2024-09-16T01:49:20.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.833+0000 7ff959c48740 1 -- 172.21.15.139:0/3355873802 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}) -- 0x7ff9480bc160 con 0x557f69713410 2024-09-16T01:49:20.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.835+0000 7ff95560b640 1 --2- 172.21.15.139:0/3355873802 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557f69713410 0x557f69787e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:20.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.840+0000 7ff938ff9640 10 client.16473 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:20.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:20.840+0000 7ff95560b640 1 --2- 172.21.15.139:0/3355873802 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557f69713410 0x557f69787e70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86e75159dd80a64a server_cookie=b8c204b730820e40 in_seq=0 out_seq=0 2024-09-16T01:49:21.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:21 smithi139 ceph-mon[31175]: pgmap v277: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-09-16T01:49:21.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:21 smithi176 ceph-mon[35324]: pgmap v277: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-09-16T01:49:21.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:21 smithi164 ceph-mon[35121]: pgmap v277: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-09-16T01:49:23.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:23 smithi139 ceph-mon[31175]: pgmap v278: 129 pgs: 14 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 70 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-09-16T01:49:23.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:23 smithi176 ceph-mon[35324]: pgmap v278: 129 pgs: 14 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 70 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-09-16T01:49:23.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:23 smithi164 ceph-mon[35121]: pgmap v278: 129 pgs: 14 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 70 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-09-16T01:49:25.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:25 smithi139 ceph-mon[31175]: pgmap v279: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2024-09-16T01:49:25.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:25 smithi176 ceph-mon[35324]: pgmap v279: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2024-09-16T01:49:25.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:25 smithi164 ceph-mon[35121]: pgmap v279: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 2.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2024-09-16T01:49:27.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.088+0000 7ff938ff9640 1 -- 172.21.15.139:0/3355873802 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7ff9480bc160 con 0x557f69713410 2024-09-16T01:49:27.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.088+0000 7ff938ff9640 10 client.16473 handle_command_reply: tid=0 2024-09-16T01:49:27.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.088+0000 7ff959c48740 1 client.16473 shutdown 2024-09-16T01:49:27.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.088+0000 7ff959c48740 1 -- 172.21.15.139:0/3355873802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 msgr2=0x557f69719540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:27.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.088+0000 7ff959c48740 1 --2- 172.21.15.139:0/3355873802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69719540 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x557f696fb9e0 tx=0x7ff94c012e70 comp rx=0 tx=0).stop 2024-09-16T01:49:27.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.088+0000 7ff959c48740 1 -- 172.21.15.139:0/3355873802 shutdown_connections 2024-09-16T01:49:27.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.088+0000 7ff959c48740 1 --2- 172.21.15.139:0/3355873802 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557f69713410 0x557f69787e70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.088+0000 7ff959c48740 1 --2- 172.21.15.139:0/3355873802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557f69700260 0x557f69719540 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.088+0000 7ff959c48740 1 --2- 172.21.15.139:0/3355873802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f696ff940 0x557f69718f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.088+0000 7ff959c48740 1 --2- 172.21.15.139:0/3355873802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f696fbfb0 0x557f6971bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.088+0000 7ff938ff9640 0 client.16473 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:27.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.088+0000 7ff959c48740 1 -- 172.21.15.139:0/3355873802 >> 172.21.15.139:0/3355873802 conn(0x557f69701f30 msgr2=0x557f696fb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:27.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.089+0000 7ff959c48740 1 -- 172.21.15.139:0/3355873802 shutdown_connections 2024-09-16T01:49:27.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.089+0000 7ff959c48740 1 -- 172.21.15.139:0/3355873802 wait complete. 2024-09-16T01:49:27.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.089+0000 7ff959c48740 20 client.16473 trim_cache size 0 max 16384 2024-09-16T01:49:27.091 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:49:27.091 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 50, 2024-09-16T01:49:27.092 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:49:27.092 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 30, 2024-09-16T01:49:27.092 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:49:27.092 INFO:teuthology.orchestra.run.smithi139.stdout: "7f2d60e9": { 2024-09-16T01:49:27.092 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 30, 2024-09-16T01:49:27.092 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:49:27.092 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:49:27.092 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:49:27.092 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:49:27.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:49:27.092 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:49:27.092 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 142.3, 2024-09-16T01:49:27.092 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:49:27.092 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:49:27.093 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:49:27.093 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:49:27.093 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:49:27.093 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:49:27.093 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:27.093 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:27.093 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:27.093 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:27.093 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:27.093 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:49:27.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.089+0000 7ff95660d640 1 -- 172.21.15.139:0/2044318933 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff93c0a35c0 msgr2=0x7ff93c0a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:27.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.089+0000 7ff95660d640 1 --2- 172.21.15.139:0/2044318933 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff93c0a35c0 0x7ff93c0a5a80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff95010ed60 tx=0x7ff94002b1a0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.089+0000 7ff95660d640 1 -- 172.21.15.139:0/2044318933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9501019c0 msgr2=0x7ff95006bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:27.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.089+0000 7ff95660d640 1 --2- 172.21.15.139:0/2044318933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9501019c0 0x7ff95006bde0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff94802aa70 tx=0x7ff9480053e0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.090+0000 7ff95660d640 1 -- 172.21.15.139:0/2044318933 shutdown_connections 2024-09-16T01:49:27.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.090+0000 7ff95660d640 1 --2- 172.21.15.139:0/2044318933 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff93c0a35c0 0x7ff93c0a5a80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.090+0000 7ff95660d640 1 --2- 172.21.15.139:0/2044318933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9501079c0 0x7ff950071210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.090+0000 7ff95660d640 1 --2- 172.21.15.139:0/2044318933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff950102360 0x7ff95006c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.090+0000 7ff95660d640 1 --2- 172.21.15.139:0/2044318933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9501019c0 0x7ff95006bde0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.090+0000 7ff95660d640 1 -- 172.21.15.139:0/2044318933 >> 172.21.15.139:0/2044318933 conn(0x7ff9500fd740 msgr2=0x7ff950075290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:27.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.090+0000 7ff95660d640 1 -- 172.21.15.139:0/2044318933 shutdown_connections 2024-09-16T01:49:27.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:27.090+0000 7ff95660d640 1 -- 172.21.15.139:0/2044318933 wait complete. 2024-09-16T01:49:27.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:27 smithi139 ceph-mon[31175]: pgmap v280: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-09-16T01:49:27.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:27 smithi176 ceph-mon[35324]: pgmap v280: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-09-16T01:49:27.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:27 smithi164 ceph-mon[35121]: pgmap v280: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-09-16T01:49:27.754 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7f2d60e9', quiesce took 11.1 seconds. Will release after: 11.250000000000002 2024-09-16T01:49:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.970+0000 7f836074b640 1 -- 172.21.15.139:0/3040803861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8358100850 msgr2=0x7f8358100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.970+0000 7f836074b640 1 --2- 172.21.15.139:0/3040803861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8358100850 0x7f8358100c30 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8354009900 tx=0x7f835402a410 comp rx=0 tx=0).stop 2024-09-16T01:49:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.970+0000 7f836074b640 1 -- 172.21.15.139:0/3040803861 shutdown_connections 2024-09-16T01:49:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.970+0000 7f836074b640 1 --2- 172.21.15.139:0/3040803861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8358105540 0x7f835810d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.970+0000 7f836074b640 1 --2- 172.21.15.139:0/3040803861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8358101200 0x7f8358104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.970+0000 7f836074b640 1 --2- 172.21.15.139:0/3040803861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8358100850 0x7f8358100c30 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.970+0000 7f836074b640 1 -- 172.21.15.139:0/3040803861 >> 172.21.15.139:0/3040803861 conn(0x7f8358075db0 msgr2=0x7f83580761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.970+0000 7f836074b640 1 -- 172.21.15.139:0/3040803861 shutdown_connections 2024-09-16T01:49:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.970+0000 7f836074b640 1 -- 172.21.15.139:0/3040803861 wait complete. 2024-09-16T01:49:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.970+0000 7f836074b640 1 Processor -- start 2024-09-16T01:49:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.970+0000 7f836074b640 1 -- start start 2024-09-16T01:49:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f836074b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8358100850 0x7f8358198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f836074b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8358105540 0x7f8358198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f836074b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f835819cc00 0x7f835819d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f836074b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8358111f30 con 0x7f8358100850 2024-09-16T01:49:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f836074b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8358111db0 con 0x7f8358105540 2024-09-16T01:49:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f836074b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f83581120b0 con 0x7f835819cc00 2024-09-16T01:49:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f835e4e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8358100850 0x7f8358198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f835e4e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8358100850 0x7f8358198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56130/0 (socket says 172.21.15.139:56130) 2024-09-16T01:49:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f835e4e7640 1 -- 172.21.15.139:0/1790075328 learned_addr learned my addr 172.21.15.139:0/1790075328 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f835dce6640 1 --2- 172.21.15.139:0/1790075328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8358105540 0x7f8358198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f835ece8640 1 --2- 172.21.15.139:0/1790075328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f835819cc00 0x7f835819d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f835e4e7640 1 -- 172.21.15.139:0/1790075328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f835819cc00 msgr2=0x7f835819d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f835e4e7640 1 --2- 172.21.15.139:0/1790075328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f835819cc00 0x7f835819d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f835e4e7640 1 -- 172.21.15.139:0/1790075328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8358105540 msgr2=0x7f8358198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f835e4e7640 1 --2- 172.21.15.139:0/1790075328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8358105540 0x7f8358198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f835e4e7640 1 -- 172.21.15.139:0/1790075328 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835819d6f0 con 0x7f8358100850 2024-09-16T01:49:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f835e4e7640 1 --2- 172.21.15.139:0/1790075328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8358100850 0x7f8358198340 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8354006010 tx=0x7f83540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f834f7fe640 1 -- 172.21.15.139:0/1790075328 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835403f030 con 0x7f8358100850 2024-09-16T01:49:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f836074b640 1 -- 172.21.15.139:0/1790075328 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835819d980 con 0x7f8358100850 2024-09-16T01:49:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f834f7fe640 1 -- 172.21.15.139:0/1790075328 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8354003ce0 con 0x7f8358100850 2024-09-16T01:49:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f834f7fe640 1 -- 172.21.15.139:0/1790075328 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354035070 con 0x7f8358100850 2024-09-16T01:49:27.976 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.971+0000 7f836074b640 1 -- 172.21.15.139:0/1790075328 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83581a5140 con 0x7f8358100850 2024-09-16T01:49:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.974+0000 7f834f7fe640 1 -- 172.21.15.139:0/1790075328 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8354042030 con 0x7f8358100850 2024-09-16T01:49:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.974+0000 7f83635c4740 1 Processor -- start 2024-09-16T01:49:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.974+0000 7f83635c4740 1 -- start start 2024-09-16T01:49:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.974+0000 7f83635c4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d643af0 0x56411d643ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.974+0000 7f83635c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d644410 0x56411d64e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.974+0000 7f83635c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d64ecb0 0x56411d6510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.974+0000 7f83635c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56411d5a82f0 con 0x56411d64ecb0 2024-09-16T01:49:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.974+0000 7f83635c4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56411d550270 con 0x56411d643af0 2024-09-16T01:49:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.974+0000 7f83635c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56411d5a42b0 con 0x56411d644410 2024-09-16T01:49:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.974+0000 7f834f7fe640 1 --2- 172.21.15.139:0/1790075328 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f83440a3460 0x7f83440a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f835dce6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d64ecb0 0x56411d6510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f835dce6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d64ecb0 0x56411d6510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56144/0 (socket says 172.21.15.139:56144) 2024-09-16T01:49:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f835dce6640 1 -- 172.21.15.139:0/2189968850 learned_addr learned my addr 172.21.15.139:0/2189968850 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f835e4e7640 1 --2- 172.21.15.139:0/2189968850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d644410 0x56411d64e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f835ece8640 1 --2- 172.21.15.139:0/1790075328 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f83440a3460 0x7f83440a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f835ece8640 1 --2- 172.21.15.139:0/2189968850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d643af0 0x56411d643ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f834f7fe640 1 -- 172.21.15.139:0/1790075328 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9642+0+0 (secure 0 0 0) 0x7f83540f0190 con 0x7f8358100850 2024-09-16T01:49:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f835dce6640 1 -- 172.21.15.139:0/2189968850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d644410 msgr2=0x56411d64e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f835dce6640 1 --2- 172.21.15.139:0/2189968850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d644410 0x56411d64e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f835dce6640 1 -- 172.21.15.139:0/2189968850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d643af0 msgr2=0x56411d643ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f835dce6640 1 --2- 172.21.15.139:0/2189968850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d643af0 0x56411d643ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f835dce6640 1 -- 172.21.15.139:0/2189968850 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56411d54f340 con 0x56411d64ecb0 2024-09-16T01:49:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f835ece8640 1 --2- 172.21.15.139:0/2189968850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d643af0 0x56411d643ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:49:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f835ece8640 1 --2- 172.21.15.139:0/1790075328 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f83440a3460 0x7f83440a5920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8350009fd0 tx=0x7f8350009ad0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f835dce6640 1 --2- 172.21.15.139:0/2189968850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d64ecb0 0x56411d6510a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f834800ac30 tx=0x7f834800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6fa75cd8ecb26f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f834cff9640 1 -- 172.21.15.139:0/2189968850 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834801d070 con 0x56411d64ecb0 2024-09-16T01:49:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f834cff9640 1 -- 172.21.15.139:0/2189968850 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8348018b20 con 0x56411d64ecb0 2024-09-16T01:49:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.975+0000 7f834cff9640 1 -- 172.21.15.139:0/2189968850 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348010ba0 con 0x56411d64ecb0 2024-09-16T01:49:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.976+0000 7f83635c4740 1 -- 172.21.15.139:0/2189968850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d64ecb0 msgr2=0x56411d6510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.976+0000 7f83635c4740 1 --2- 172.21.15.139:0/2189968850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d64ecb0 0x56411d6510a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f834800ac30 tx=0x7f834800f790 comp rx=0 tx=0).stop 2024-09-16T01:49:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.976+0000 7f83635c4740 1 -- 172.21.15.139:0/2189968850 shutdown_connections 2024-09-16T01:49:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.976+0000 7f83635c4740 1 --2- 172.21.15.139:0/2189968850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d64ecb0 0x56411d6510a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.976+0000 7f83635c4740 1 --2- 172.21.15.139:0/2189968850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d644410 0x56411d64e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.976+0000 7f83635c4740 1 --2- 172.21.15.139:0/2189968850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d643af0 0x56411d643ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.976+0000 7f83635c4740 1 -- 172.21.15.139:0/2189968850 >> 172.21.15.139:0/2189968850 conn(0x56411d640060 msgr2=0x56411d640440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.976+0000 7f83635c4740 1 -- 172.21.15.139:0/2189968850 shutdown_connections 2024-09-16T01:49:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.976+0000 7f83635c4740 1 -- 172.21.15.139:0/2189968850 wait complete. 2024-09-16T01:49:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f83635c4740 1 -- start start 2024-09-16T01:49:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f83635c4740 1 Processor -- start 2024-09-16T01:49:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f83635c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d643af0 0x56411d64bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f83635c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d644410 0x56411d64c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f83635c4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d64ecb0 0x56411d64c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f83635c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56411d651880 con 0x56411d643af0 2024-09-16T01:49:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f83635c4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56411d531290 con 0x56411d64ecb0 2024-09-16T01:49:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f83635c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56411d651a00 con 0x56411d644410 2024-09-16T01:49:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f83635c4740 10 client.0 _ll_register_callbacks cb 0x56411d562140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f835e4e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d643af0 0x56411d64bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f835e4e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d643af0 0x56411d64bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56156/0 (socket says 172.21.15.139:56156) 2024-09-16T01:49:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f835e4e7640 1 -- 172.21.15.139:0/2651112433 learned_addr learned my addr 172.21.15.139:0/2651112433 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f835dce6640 1 --2- 172.21.15.139:0/2651112433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d644410 0x56411d64c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f835e4e7640 1 -- 172.21.15.139:0/2651112433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d644410 msgr2=0x56411d64c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f835ece8640 1 --2- 172.21.15.139:0/2651112433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d64ecb0 0x56411d64c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f835e4e7640 1 --2- 172.21.15.139:0/2651112433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d644410 0x56411d64c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f835e4e7640 1 -- 172.21.15.139:0/2651112433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d64ecb0 msgr2=0x56411d64c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f835e4e7640 1 --2- 172.21.15.139:0/2651112433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d64ecb0 0x56411d64c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f835e4e7640 1 -- 172.21.15.139:0/2651112433 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56411d2bb210 con 0x56411d643af0 2024-09-16T01:49:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f835e4e7640 1 --2- 172.21.15.139:0/2651112433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d643af0 0x56411d64bbc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8354000c00 tx=0x7f8354042d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f412e56e390590c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.977+0000 7f8341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:49:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f8341ffb640 1 -- 172.21.15.139:0/2651112433 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354049030 con 0x56411d643af0 2024-09-16T01:49:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f8341ffb640 1 -- 172.21.15.139:0/2651112433 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9642+0+0 (secure 0 0 0) 0x7f83540b8680 con 0x56411d643af0 2024-09-16T01:49:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f83635c4740 1 -- 172.21.15.139:0/2651112433 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8348018b20 con 0x56411d643af0 2024-09-16T01:49:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f8341ffb640 1 -- 172.21.15.139:0/2651112433 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f83540bb970 con 0x56411d643af0 2024-09-16T01:49:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f83635c4740 10 client.16485 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f83635c4740 1 -- 172.21.15.139:0/2651112433 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56411d387620 con 0x56411d643af0 2024-09-16T01:49:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f8341ffb640 1 -- 172.21.15.139:0/2651112433 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8348018b20 con 0x56411d643af0 2024-09-16T01:49:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f8341ffb640 1 -- 172.21.15.139:0/2651112433 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7f835403bc40 con 0x56411d643af0 2024-09-16T01:49:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f83635c4740 10 client.16485 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f83635c4740 10 client.16485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:49:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f83635c4740 20 client.16485 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f83635c4740 1 --2- 172.21.15.139:0/2651112433 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56411d65ec40 0x56411d6610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f83635c4740 4 client.16485 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:49:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f83635c4740 1 -- 172.21.15.139:0/2651112433 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56411d5b5a20 con 0x56411d65ec40 2024-09-16T01:49:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.978+0000 7f835dce6640 1 --2- 172.21.15.139:0/2651112433 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56411d65ec40 0x56411d6610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.979+0000 7f8341ffb640 10 client.16485 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.979+0000 7f835dce6640 1 --2- 172.21.15.139:0/2651112433 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56411d65ec40 0x56411d6610e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83912f93d3528da2 server_cookie=f5471f564b9c203c in_seq=0 out_seq=0 2024-09-16T01:49:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.979+0000 7f8341ffb640 1 -- 172.21.15.139:0/2651112433 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56411d5b5a20 con 0x56411d65ec40 2024-09-16T01:49:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.979+0000 7f8341ffb640 10 client.16485 handle_command_reply: tid=0 2024-09-16T01:49:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.980+0000 7f83635c4740 1 client.16485 shutdown 2024-09-16T01:49:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.980+0000 7f83635c4740 1 -- 172.21.15.139:0/2651112433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d643af0 msgr2=0x56411d64bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.980+0000 7f83635c4740 1 --2- 172.21.15.139:0/2651112433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d643af0 0x56411d64bbc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8354000c00 tx=0x7f8354042d70 comp rx=0 tx=0).stop 2024-09-16T01:49:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.980+0000 7f83635c4740 1 -- 172.21.15.139:0/2651112433 shutdown_connections 2024-09-16T01:49:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.980+0000 7f83635c4740 1 --2- 172.21.15.139:0/2651112433 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56411d65ec40 0x56411d6610e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.980+0000 7f83635c4740 1 --2- 172.21.15.139:0/2651112433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d64ecb0 0x56411d64c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.980+0000 7f83635c4740 1 --2- 172.21.15.139:0/2651112433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d644410 0x56411d64c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.980+0000 7f8341ffb640 0 client.16485 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:27.990 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.980+0000 7f83635c4740 1 --2- 172.21.15.139:0/2651112433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d643af0 0x56411d64bbc0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:27.990 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.980+0000 7f83635c4740 1 -- 172.21.15.139:0/2651112433 >> 172.21.15.139:0/2651112433 conn(0x56411d640060 msgr2=0x56411d645d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:27.990 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.980+0000 7f83635c4740 1 -- 172.21.15.139:0/2651112433 shutdown_connections 2024-09-16T01:49:27.990 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.980+0000 7f83635c4740 1 -- 172.21.15.139:0/2651112433 wait complete. 2024-09-16T01:49:27.990 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.980+0000 7f83635c4740 20 client.16485 trim_cache size 0 max 16384 2024-09-16T01:49:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f83635c4740 1 Processor -- start 2024-09-16T01:49:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f83635c4740 1 -- start start 2024-09-16T01:49:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f83635c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d643af0 0x56411d64c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f83635c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d644410 0x56411d64c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f83635c4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d64ecb0 0x56411d6669f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f83635c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f83581af140 con 0x56411d644410 2024-09-16T01:49:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f83635c4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83581b0f80 con 0x56411d64ecb0 2024-09-16T01:49:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f83635c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f83581af8d0 con 0x56411d643af0 2024-09-16T01:49:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f835dce6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d644410 0x56411d64c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f835dce6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d644410 0x56411d64c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56164/0 (socket says 172.21.15.139:56164) 2024-09-16T01:49:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f835dce6640 1 -- 172.21.15.139:0/1767349373 learned_addr learned my addr 172.21.15.139:0/1767349373 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f835dce6640 1 -- 172.21.15.139:0/1767349373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d643af0 msgr2=0x56411d64c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f835dce6640 1 --2- 172.21.15.139:0/1767349373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d643af0 0x56411d64c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f835dce6640 1 -- 172.21.15.139:0/1767349373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d64ecb0 msgr2=0x56411d6669f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f835dce6640 1 --2- 172.21.15.139:0/1767349373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d64ecb0 0x56411d6669f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.994+0000 7f835dce6640 1 -- 172.21.15.139:0/1767349373 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56411d643910 con 0x56411d644410 2024-09-16T01:49:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.995+0000 7f835dce6640 1 --2- 172.21.15.139:0/1767349373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d644410 0x56411d64c630 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8348000c00 tx=0x7f834800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4c920ce8e0940a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.995+0000 7f8343fff640 1 -- 172.21.15.139:0/1767349373 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348032070 con 0x56411d644410 2024-09-16T01:49:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.995+0000 7f8343fff640 1 -- 172.21.15.139:0/1767349373 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f834801e040 con 0x56411d644410 2024-09-16T01:49:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.995+0000 7f8343fff640 1 -- 172.21.15.139:0/1767349373 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834801d040 con 0x56411d644410 2024-09-16T01:49:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.995+0000 7f83635c4740 1 -- 172.21.15.139:0/1767349373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d644410 msgr2=0x56411d64c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.995+0000 7f83635c4740 1 --2- 172.21.15.139:0/1767349373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d644410 0x56411d64c630 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8348000c00 tx=0x7f834800a650 comp rx=0 tx=0).stop 2024-09-16T01:49:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.995+0000 7f83635c4740 1 -- 172.21.15.139:0/1767349373 shutdown_connections 2024-09-16T01:49:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.995+0000 7f83635c4740 1 --2- 172.21.15.139:0/1767349373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d64ecb0 0x56411d6669f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.995+0000 7f83635c4740 1 --2- 172.21.15.139:0/1767349373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d644410 0x56411d64c630 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.995+0000 7f83635c4740 1 --2- 172.21.15.139:0/1767349373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d643af0 0x56411d64c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.995+0000 7f83635c4740 1 -- 172.21.15.139:0/1767349373 >> 172.21.15.139:0/1767349373 conn(0x56411d640060 msgr2=0x56411d64dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:28.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.995+0000 7f83635c4740 1 -- 172.21.15.139:0/1767349373 shutdown_connections 2024-09-16T01:49:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.995+0000 7f83635c4740 1 -- 172.21.15.139:0/1767349373 wait complete. 2024-09-16T01:49:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f83635c4740 1 -- start start 2024-09-16T01:49:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f83635c4740 1 Processor -- start 2024-09-16T01:49:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f83635c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d643af0 0x56411d6529a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f83635c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d644410 0x56411d652ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f83635c4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d64ecb0 0x56411d653420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f83635c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f83581b07f0 con 0x56411d644410 2024-09-16T01:49:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f83635c4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83581b0060 con 0x56411d64ecb0 2024-09-16T01:49:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f83635c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f83581ad140 con 0x56411d643af0 2024-09-16T01:49:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f83635c4740 10 client.0 _ll_register_callbacks cb 0x7f8354005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f835dce6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d644410 0x56411d652ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f835dce6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d644410 0x56411d652ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56176/0 (socket says 172.21.15.139:56176) 2024-09-16T01:49:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f835dce6640 1 -- 172.21.15.139:0/1420022793 learned_addr learned my addr 172.21.15.139:0/1420022793 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f835e4e7640 1 --2- 172.21.15.139:0/1420022793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d643af0 0x56411d6529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f835ece8640 1 --2- 172.21.15.139:0/1420022793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d64ecb0 0x56411d653420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f835dce6640 1 -- 172.21.15.139:0/1420022793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d643af0 msgr2=0x56411d6529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f835dce6640 1 --2- 172.21.15.139:0/1420022793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d643af0 0x56411d6529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f835dce6640 1 -- 172.21.15.139:0/1420022793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d64ecb0 msgr2=0x56411d653420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f835dce6640 1 --2- 172.21.15.139:0/1420022793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d64ecb0 0x56411d653420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f835dce6640 1 -- 172.21.15.139:0/1420022793 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56411d6c89c0 con 0x56411d644410 2024-09-16T01:49:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f835e4e7640 1 --2- 172.21.15.139:0/1420022793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d643af0 0x56411d6529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:49:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f835dce6640 1 --2- 172.21.15.139:0/1420022793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d644410 0x56411d652ee0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f834800ab50 tx=0x7f8348018c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d6c055748013bd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.996+0000 7f8341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:49:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f8341ffb640 1 -- 172.21.15.139:0/1420022793 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834801d070 con 0x56411d644410 2024-09-16T01:49:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f8341ffb640 1 -- 172.21.15.139:0/1420022793 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9642+0+0 (secure 0 0 0) 0x7f8348005240 con 0x56411d644410 2024-09-16T01:49:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f83635c4740 1 -- 172.21.15.139:0/1420022793 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f834801e040 con 0x56411d644410 2024-09-16T01:49:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f8341ffb640 1 -- 172.21.15.139:0/1420022793 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8348032070 con 0x56411d644410 2024-09-16T01:49:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f83635c4740 10 client.16491 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f83635c4740 1 -- 172.21.15.139:0/1420022793 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56411d6c9c80 con 0x56411d644410 2024-09-16T01:49:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f8341ffb640 1 -- 172.21.15.139:0/1420022793 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f834801e040 con 0x56411d644410 2024-09-16T01:49:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f8341ffb640 1 -- 172.21.15.139:0/1420022793 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7f83480106f0 con 0x56411d644410 2024-09-16T01:49:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f83635c4740 10 client.16491 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f83635c4740 10 client.16491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:49:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f83635c4740 20 client.16491 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f83635c4740 1 --2- 172.21.15.139:0/1420022793 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56411d6ca1d0 0x56411d6cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f83635c4740 4 client.16491 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:49:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f83635c4740 1 -- 172.21.15.139:0/1420022793 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56411d5afd70 con 0x56411d6ca1d0 2024-09-16T01:49:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.997+0000 7f835e4e7640 1 --2- 172.21.15.139:0/1420022793 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56411d6ca1d0 0x56411d6cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.998+0000 7f8341ffb640 10 client.16491 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:27.998+0000 7f835e4e7640 1 --2- 172.21.15.139:0/1420022793 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56411d6ca1d0 0x56411d6cc5f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d46c01b2ac1a3f7a server_cookie=9674b363b58a6cc8 in_seq=0 out_seq=0 2024-09-16T01:49:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f8341ffb640 1 -- 172.21.15.139:0/1420022793 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56411d5afd70 con 0x56411d6ca1d0 2024-09-16T01:49:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f8341ffb640 10 client.16491 handle_command_reply: tid=0 2024-09-16T01:49:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f83635c4740 1 client.16491 shutdown 2024-09-16T01:49:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f83635c4740 1 -- 172.21.15.139:0/1420022793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d644410 msgr2=0x56411d652ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f83635c4740 1 --2- 172.21.15.139:0/1420022793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d644410 0x56411d652ee0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f834800ab50 tx=0x7f8348018c00 comp rx=0 tx=0).stop 2024-09-16T01:49:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f83635c4740 1 -- 172.21.15.139:0/1420022793 shutdown_connections 2024-09-16T01:49:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f83635c4740 1 --2- 172.21.15.139:0/1420022793 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56411d6ca1d0 0x56411d6cc5f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f83635c4740 1 --2- 172.21.15.139:0/1420022793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56411d64ecb0 0x56411d653420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f8341ffb640 0 client.16491 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f83635c4740 1 --2- 172.21.15.139:0/1420022793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56411d644410 0x56411d652ee0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f83635c4740 1 --2- 172.21.15.139:0/1420022793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56411d643af0 0x56411d6529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f83635c4740 1 -- 172.21.15.139:0/1420022793 >> 172.21.15.139:0/1420022793 conn(0x56411d640060 msgr2=0x56411d63e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f83635c4740 1 -- 172.21.15.139:0/1420022793 shutdown_connections 2024-09-16T01:49:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f83635c4740 1 -- 172.21.15.139:0/1420022793 wait complete. 2024-09-16T01:49:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.270+0000 7f83635c4740 20 client.16491 trim_cache size 0 max 16384 2024-09-16T01:49:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:49:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:49:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:49:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:49:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.271+0000 7f836074b640 1 -- 172.21.15.139:0/1790075328 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f83440a3460 msgr2=0x7f83440a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.271+0000 7f836074b640 1 --2- 172.21.15.139:0/1790075328 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f83440a3460 0x7f83440a5920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8350009fd0 tx=0x7f8350009ad0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.271+0000 7f836074b640 1 -- 172.21.15.139:0/1790075328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8358100850 msgr2=0x7f8358198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.271+0000 7f836074b640 1 --2- 172.21.15.139:0/1790075328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8358100850 0x7f8358198340 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8354006010 tx=0x7f83540042f0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.271+0000 7f836074b640 1 -- 172.21.15.139:0/1790075328 shutdown_connections 2024-09-16T01:49:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.271+0000 7f836074b640 1 --2- 172.21.15.139:0/1790075328 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f83440a3460 0x7f83440a5920 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.271+0000 7f836074b640 1 --2- 172.21.15.139:0/1790075328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f835819cc00 0x7f835819d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.271+0000 7f836074b640 1 --2- 172.21.15.139:0/1790075328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8358105540 0x7f8358198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.271+0000 7f836074b640 1 --2- 172.21.15.139:0/1790075328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8358100850 0x7f8358198340 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.271+0000 7f836074b640 1 -- 172.21.15.139:0/1790075328 >> 172.21.15.139:0/1790075328 conn(0x7f8358075db0 msgr2=0x7f83580fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.271+0000 7f836074b640 1 -- 172.21.15.139:0/1790075328 shutdown_connections 2024-09-16T01:49:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:28.271+0000 7f836074b640 1 -- 172.21.15.139:0/1790075328 wait complete. 2024-09-16T01:49:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:29 smithi139 ceph-mon[31175]: pgmap v281: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 604 KiB/s wr, 145 op/s 2024-09-16T01:49:29.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:29 smithi176 ceph-mon[35324]: pgmap v281: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 604 KiB/s wr, 145 op/s 2024-09-16T01:49:29.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:29 smithi164 ceph-mon[35121]: pgmap v281: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 604 KiB/s wr, 145 op/s 2024-09-16T01:49:31.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:31 smithi139 ceph-mon[31175]: pgmap v282: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 478 KiB/s wr, 131 op/s 2024-09-16T01:49:31.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:31 smithi139 ceph-mon[31175]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T01:49:31.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:31 smithi176 ceph-mon[35324]: pgmap v282: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 478 KiB/s wr, 131 op/s 2024-09-16T01:49:31.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:31 smithi176 ceph-mon[35324]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T01:49:31.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:31 smithi164 ceph-mon[35121]: pgmap v282: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 478 KiB/s wr, 131 op/s 2024-09-16T01:49:31.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:31 smithi164 ceph-mon[35121]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T01:49:33.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:33 smithi139 ceph-mon[31175]: pgmap v284: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 374 KiB/s wr, 145 op/s 2024-09-16T01:49:33.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:33 smithi176 ceph-mon[35324]: pgmap v284: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 374 KiB/s wr, 145 op/s 2024-09-16T01:49:33.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:33 smithi164 ceph-mon[35121]: pgmap v284: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 374 KiB/s wr, 145 op/s 2024-09-16T01:49:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:35 smithi139 ceph-mon[31175]: pgmap v285: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 149 KiB/s wr, 134 op/s 2024-09-16T01:49:35.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:35 smithi176 ceph-mon[35324]: pgmap v285: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 149 KiB/s wr, 134 op/s 2024-09-16T01:49:35.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:35 smithi164 ceph-mon[35121]: pgmap v285: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 149 KiB/s wr, 134 op/s 2024-09-16T01:49:37.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:37 smithi139 ceph-mon[31175]: pgmap v286: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 92 KiB/s wr, 105 op/s 2024-09-16T01:49:37.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:37 smithi176 ceph-mon[35324]: pgmap v286: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 92 KiB/s wr, 105 op/s 2024-09-16T01:49:37.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:37 smithi164 ceph-mon[35121]: pgmap v286: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 92 KiB/s wr, 105 op/s 2024-09-16T01:49:39.006 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:49:39.267 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:49:39.388 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:39 smithi139 ceph-mon[31175]: pgmap v287: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 79 KiB/s wr, 91 op/s 2024-09-16T01:49:39.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:39 smithi176 ceph-mon[35324]: pgmap v287: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 79 KiB/s wr, 91 op/s 2024-09-16T01:49:39.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:39 smithi164 ceph-mon[35121]: pgmap v287: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 79 KiB/s wr, 91 op/s 2024-09-16T01:49:39.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.766+0000 7ff86c5f7640 1 -- 172.21.15.139:0/3059745205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff864100e20 msgr2=0x7ff864101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:39.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.766+0000 7ff86c5f7640 1 --2- 172.21.15.139:0/3059745205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff864100e20 0x7ff864101200 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff860009900 tx=0x7ff86002a480 comp rx=0 tx=0).stop 2024-09-16T01:49:39.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.766+0000 7ff86c5f7640 1 -- 172.21.15.139:0/3059745205 shutdown_connections 2024-09-16T01:49:39.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.766+0000 7ff86c5f7640 1 --2- 172.21.15.139:0/3059745205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff864102ef0 0x7ff86410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:39.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.766+0000 7ff86c5f7640 1 --2- 172.21.15.139:0/3059745205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff864102570 0x7ff8641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:39.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.766+0000 7ff86c5f7640 1 --2- 172.21.15.139:0/3059745205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff864100e20 0x7ff864101200 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:39.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.766+0000 7ff86c5f7640 1 -- 172.21.15.139:0/3059745205 >> 172.21.15.139:0/3059745205 conn(0x7ff8640fc7d0 msgr2=0x7ff8640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:39.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.766+0000 7ff86c5f7640 1 -- 172.21.15.139:0/3059745205 shutdown_connections 2024-09-16T01:49:39.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.766+0000 7ff86c5f7640 1 -- 172.21.15.139:0/3059745205 wait complete. 2024-09-16T01:49:39.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86c5f7640 1 Processor -- start 2024-09-16T01:49:39.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86c5f7640 1 -- start start 2024-09-16T01:49:39.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86c5f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff864100e20 0x7ff864199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:39.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86c5f7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff864102570 0x7ff8641997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:39.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86c5f7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff864102ef0 0x7ff8641a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:39.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86c5f7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff864110f50 con 0x7ff864100e20 2024-09-16T01:49:39.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86c5f7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff864110dd0 con 0x7ff864102570 2024-09-16T01:49:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86c5f7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff8641110d0 con 0x7ff864102ef0 2024-09-16T01:49:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86a393640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff864100e20 0x7ff864199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86a393640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff864100e20 0x7ff864199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42166/0 (socket says 172.21.15.139:42166) 2024-09-16T01:49:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86a393640 1 -- 172.21.15.139:0/2988157207 learned_addr learned my addr 172.21.15.139:0/2988157207 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff869b92640 1 --2- 172.21.15.139:0/2988157207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff864102570 0x7ff8641997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86ab94640 1 --2- 172.21.15.139:0/2988157207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff864102ef0 0x7ff8641a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86a393640 1 -- 172.21.15.139:0/2988157207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff864102ef0 msgr2=0x7ff8641a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86a393640 1 --2- 172.21.15.139:0/2988157207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff864102ef0 0x7ff8641a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86a393640 1 -- 172.21.15.139:0/2988157207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff864102570 msgr2=0x7ff8641997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86a393640 1 --2- 172.21.15.139:0/2988157207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff864102570 0x7ff8641997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.767+0000 7ff86a393640 1 -- 172.21.15.139:0/2988157207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8641a4820 con 0x7ff864100e20 2024-09-16T01:49:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.768+0000 7ff86a393640 1 --2- 172.21.15.139:0/2988157207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff864100e20 0x7ff864199260 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff860006010 tx=0x7ff860003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.768+0000 7ff85b7fe640 1 -- 172.21.15.139:0/2988157207 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86003f070 con 0x7ff864100e20 2024-09-16T01:49:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.768+0000 7ff86c5f7640 1 -- 172.21.15.139:0/2988157207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8641a49f0 con 0x7ff864100e20 2024-09-16T01:49:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.768+0000 7ff85b7fe640 1 -- 172.21.15.139:0/2988157207 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff860031e30 con 0x7ff864100e20 2024-09-16T01:49:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.768+0000 7ff85b7fe640 1 -- 172.21.15.139:0/2988157207 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860035030 con 0x7ff864100e20 2024-09-16T01:49:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.768+0000 7ff86c5f7640 1 -- 172.21.15.139:0/2988157207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8641a4ea0 con 0x7ff864100e20 2024-09-16T01:49:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.769+0000 7ff86c5f7640 1 -- 172.21.15.139:0/2988157207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff864101200 con 0x7ff864100e20 2024-09-16T01:49:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.770+0000 7ff85b7fe640 1 -- 172.21.15.139:0/2988157207 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff860042070 con 0x7ff864100e20 2024-09-16T01:49:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.771+0000 7ff85b7fe640 1 --2- 172.21.15.139:0/2988157207 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff8500a33b0 0x7ff8500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.771+0000 7ff869b92640 1 --2- 172.21.15.139:0/2988157207 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff8500a33b0 0x7ff8500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.771+0000 7ff85b7fe640 1 -- 172.21.15.139:0/2988157207 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7ff8600f11b0 con 0x7ff864100e20 2024-09-16T01:49:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.771+0000 7ff869b92640 1 --2- 172.21.15.139:0/2988157207 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff8500a33b0 0x7ff8500a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff86406aac0 tx=0x7ff85400a2d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.774+0000 7ff85b7fe640 1 -- 172.21.15.139:0/2988157207 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8600bd780 con 0x7ff864100e20 2024-09-16T01:49:39.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.977+0000 7ff86c5f7640 1 -- 172.21.15.139:0/2988157207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8641a5130 con 0x7ff864100e20 2024-09-16T01:49:39.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.978+0000 7ff85b7fe640 1 -- 172.21.15.139:0/2988157207 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+12028 (secure 0 0 0) 0x7ff8600bd080 con 0x7ff864100e20 2024-09-16T01:49:39.980 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:49:39.980 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":50,"btime":"2024-09-16T01:48:57:039963+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:48:56.596845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24454,24443,24442,24433]},"id":1}]} 2024-09-16T01:49:39.980 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 50 2024-09-16T01:49:39.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.980+0000 7ff86c5f7640 1 -- 172.21.15.139:0/2988157207 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff8500a33b0 msgr2=0x7ff8500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:39.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.980+0000 7ff86c5f7640 1 --2- 172.21.15.139:0/2988157207 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff8500a33b0 0x7ff8500a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff86406aac0 tx=0x7ff85400a2d0 comp rx=0 tx=0).stop 2024-09-16T01:49:39.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.980+0000 7ff86c5f7640 1 -- 172.21.15.139:0/2988157207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff864100e20 msgr2=0x7ff864199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:39.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.980+0000 7ff86c5f7640 1 --2- 172.21.15.139:0/2988157207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff864100e20 0x7ff864199260 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff860006010 tx=0x7ff860003c80 comp rx=0 tx=0).stop 2024-09-16T01:49:39.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.980+0000 7ff86c5f7640 1 -- 172.21.15.139:0/2988157207 shutdown_connections 2024-09-16T01:49:39.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.980+0000 7ff86c5f7640 1 --2- 172.21.15.139:0/2988157207 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ff8500a33b0 0x7ff8500a5870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:39.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.980+0000 7ff86c5f7640 1 --2- 172.21.15.139:0/2988157207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff864102ef0 0x7ff8641a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:39.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.980+0000 7ff86c5f7640 1 --2- 172.21.15.139:0/2988157207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff864102570 0x7ff8641997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:39.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.980+0000 7ff86c5f7640 1 --2- 172.21.15.139:0/2988157207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff864100e20 0x7ff864199260 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:39.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.980+0000 7ff86c5f7640 1 -- 172.21.15.139:0/2988157207 >> 172.21.15.139:0/2988157207 conn(0x7ff8640fc7d0 msgr2=0x7ff8641067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:39.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.980+0000 7ff86c5f7640 1 -- 172.21.15.139:0/2988157207 shutdown_connections 2024-09-16T01:49:39.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:39.980+0000 7ff86c5f7640 1 -- 172.21.15.139:0/2988157207 wait complete. 2024-09-16T01:49:40.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:40 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2988157207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:49:40.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:40 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2988157207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:49:40.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:40 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2988157207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:49:40.831 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 7f2d60e9 --release --await' 2024-09-16T01:49:40.831 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 7f2d60e9 --release --await 2024-09-16T01:49:41.086 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:49:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.358+0000 7fdedf2ba640 1 -- 172.21.15.139:0/4046576483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded8102050 msgr2=0x7fded810d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.358+0000 7fdedf2ba640 1 --2- 172.21.15.139:0/4046576483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded8102050 0x7fded810d7d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fded0009ca0 tx=0x7fded002a950 comp rx=0 tx=0).stop 2024-09-16T01:49:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.358+0000 7fdedf2ba640 1 -- 172.21.15.139:0/4046576483 shutdown_connections 2024-09-16T01:49:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.358+0000 7fdedf2ba640 1 --2- 172.21.15.139:0/4046576483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded8102050 0x7fded810d7d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.358+0000 7fdedf2ba640 1 --2- 172.21.15.139:0/4046576483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fded81016d0 0x7fded8101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.358+0000 7fdedf2ba640 1 --2- 172.21.15.139:0/4046576483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded8105af0 0x7fded8105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.358+0000 7fdedf2ba640 1 -- 172.21.15.139:0/4046576483 >> 172.21.15.139:0/4046576483 conn(0x7fded8075ec0 msgr2=0x7fded80762d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.358+0000 7fdedf2ba640 1 -- 172.21.15.139:0/4046576483 shutdown_connections 2024-09-16T01:49:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.358+0000 7fdedf2ba640 1 -- 172.21.15.139:0/4046576483 wait complete. 2024-09-16T01:49:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedf2ba640 1 Processor -- start 2024-09-16T01:49:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedf2ba640 1 -- start start 2024-09-16T01:49:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedf2ba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fded81016d0 0x7fded8197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedf2ba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded8102050 0x7fded8197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedf2ba640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded8105af0 0x7fded819ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedf2ba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fded8110e30 con 0x7fded8102050 2024-09-16T01:49:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedf2ba640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fded8110cb0 con 0x7fded81016d0 2024-09-16T01:49:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedf2ba640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fded8110fb0 con 0x7fded8105af0 2024-09-16T01:49:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedc855640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded8102050 0x7fded8197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedc855640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded8102050 0x7fded8197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42182/0 (socket says 172.21.15.139:42182) 2024-09-16T01:49:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedc855640 1 -- 172.21.15.139:0/4106849571 learned_addr learned my addr 172.21.15.139:0/4106849571 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedd056640 1 --2- 172.21.15.139:0/4106849571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fded81016d0 0x7fded8197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedd857640 1 --2- 172.21.15.139:0/4106849571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded8105af0 0x7fded819ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedc855640 1 -- 172.21.15.139:0/4106849571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded8105af0 msgr2=0x7fded819ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedc855640 1 --2- 172.21.15.139:0/4106849571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded8105af0 0x7fded819ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedc855640 1 -- 172.21.15.139:0/4106849571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fded81016d0 msgr2=0x7fded8197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.359+0000 7fdedc855640 1 --2- 172.21.15.139:0/4106849571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fded81016d0 0x7fded8197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.360+0000 7fdedc855640 1 -- 172.21.15.139:0/4106849571 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded819c130 con 0x7fded8102050 2024-09-16T01:49:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.360+0000 7fdedc855640 1 --2- 172.21.15.139:0/4106849571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded8102050 0x7fded8197680 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdec800d950 tx=0x7fdec800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.360+0000 7fdece7fc640 1 -- 172.21.15.139:0/4106849571 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8017070 con 0x7fded8102050 2024-09-16T01:49:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.360+0000 7fdedf2ba640 1 -- 172.21.15.139:0/4106849571 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded819c420 con 0x7fded8102050 2024-09-16T01:49:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.360+0000 7fdece7fc640 1 -- 172.21.15.139:0/4106849571 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdec8009c30 con 0x7fded8102050 2024-09-16T01:49:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.360+0000 7fdedf2ba640 1 -- 172.21.15.139:0/4106849571 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded81005c0 con 0x7fded8102050 2024-09-16T01:49:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.361+0000 7fdece7fc640 1 -- 172.21.15.139:0/4106849571 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec800f6b0 con 0x7fded8102050 2024-09-16T01:49:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.362+0000 7fdece7fc640 1 -- 172.21.15.139:0/4106849571 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fdec800f850 con 0x7fded8102050 2024-09-16T01:49:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.363+0000 7fdee2057740 1 Processor -- start 2024-09-16T01:49:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.363+0000 7fdee2057740 1 -- start start 2024-09-16T01:49:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.363+0000 7fdee2057740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb567af0 0x557ebb567ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.363+0000 7fdee2057740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb568410 0x557ebb572770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.363+0000 7fdee2057740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb572cb0 0x557ebb5750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.363+0000 7fdee2057740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557ebb4cc2f0 con 0x557ebb567af0 2024-09-16T01:49:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.363+0000 7fdee2057740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557ebb474270 con 0x557ebb572cb0 2024-09-16T01:49:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.363+0000 7fdee2057740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557ebb4c82b0 con 0x557ebb568410 2024-09-16T01:49:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.363+0000 7fdedd857640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb567af0 0x557ebb567ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.363+0000 7fdece7fc640 1 --2- 172.21.15.139:0/4106849571 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fdec40a34e0 0x7fdec40a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.363+0000 7fdedd857640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb567af0 0x557ebb567ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42194/0 (socket says 172.21.15.139:42194) 2024-09-16T01:49:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.363+0000 7fdedd857640 1 -- 172.21.15.139:0/3595812258 learned_addr learned my addr 172.21.15.139:0/3595812258 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.364+0000 7fdedc855640 1 --2- 172.21.15.139:0/3595812258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb572cb0 0x557ebb5750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.364+0000 7fdedd056640 1 --2- 172.21.15.139:0/3595812258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb568410 0x557ebb572770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.364+0000 7fdedd857640 1 --2- 172.21.15.139:0/4106849571 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fdec40a34e0 0x7fdec40a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.364+0000 7fdece7fc640 1 -- 172.21.15.139:0/4106849571 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7fdec80cfe50 con 0x7fded8102050 2024-09-16T01:49:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.364+0000 7fdedd857640 1 -- 172.21.15.139:0/3595812258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb568410 msgr2=0x557ebb572770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.364+0000 7fdedd857640 1 --2- 172.21.15.139:0/3595812258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb568410 0x557ebb572770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.364+0000 7fdedd857640 1 -- 172.21.15.139:0/3595812258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb572cb0 msgr2=0x557ebb5750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.364+0000 7fdedd857640 1 --2- 172.21.15.139:0/3595812258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb572cb0 0x557ebb5750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.364+0000 7fdedd857640 1 -- 172.21.15.139:0/3595812258 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ebb473340 con 0x557ebb567af0 2024-09-16T01:49:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.364+0000 7fdedd857640 1 --2- 172.21.15.139:0/3595812258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb567af0 0x557ebb567ed0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fded0009430 tx=0x7fded0003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1d32126f115902 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.364+0000 7fdec37fe640 1 -- 172.21.15.139:0/3595812258 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0043050 con 0x557ebb567af0 2024-09-16T01:49:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.364+0000 7fdedd857640 1 --2- 172.21.15.139:0/4106849571 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fdec40a34e0 0x7fdec40a59a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fded004e000 tx=0x7fded004c000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.364+0000 7fdec37fe640 1 -- 172.21.15.139:0/3595812258 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded0044040 con 0x557ebb567af0 2024-09-16T01:49:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.364+0000 7fdec37fe640 1 -- 172.21.15.139:0/3595812258 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded004d020 con 0x557ebb567af0 2024-09-16T01:49:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.365+0000 7fdee2057740 1 -- 172.21.15.139:0/3595812258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb567af0 msgr2=0x557ebb567ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.365+0000 7fdee2057740 1 --2- 172.21.15.139:0/3595812258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb567af0 0x557ebb567ed0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fded0009430 tx=0x7fded0003b20 comp rx=0 tx=0).stop 2024-09-16T01:49:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.365+0000 7fdee2057740 1 -- 172.21.15.139:0/3595812258 shutdown_connections 2024-09-16T01:49:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.365+0000 7fdee2057740 1 --2- 172.21.15.139:0/3595812258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb572cb0 0x557ebb5750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.365+0000 7fdee2057740 1 --2- 172.21.15.139:0/3595812258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb568410 0x557ebb572770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.365+0000 7fdee2057740 1 --2- 172.21.15.139:0/3595812258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb567af0 0x557ebb567ed0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.365+0000 7fdee2057740 1 -- 172.21.15.139:0/3595812258 >> 172.21.15.139:0/3595812258 conn(0x557ebb564060 msgr2=0x557ebb564440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.365+0000 7fdee2057740 1 -- 172.21.15.139:0/3595812258 shutdown_connections 2024-09-16T01:49:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.365+0000 7fdee2057740 1 -- 172.21.15.139:0/3595812258 wait complete. 2024-09-16T01:49:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.366+0000 7fdee2057740 1 -- start start 2024-09-16T01:49:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.366+0000 7fdee2057740 1 Processor -- start 2024-09-16T01:49:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.366+0000 7fdee2057740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb567af0 0x557ebb56fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.366+0000 7fdee2057740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb568410 0x557ebb5700b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.366+0000 7fdee2057740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb572cb0 0x557ebb5705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.366+0000 7fdee2057740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557ebb575880 con 0x557ebb567af0 2024-09-16T01:49:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.366+0000 7fdee2057740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557ebb455290 con 0x557ebb568410 2024-09-16T01:49:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.366+0000 7fdee2057740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557ebb575a00 con 0x557ebb572cb0 2024-09-16T01:49:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.366+0000 7fdee2057740 10 client.0 _ll_register_callbacks cb 0x557ebb486140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.366+0000 7fdedd056640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb567af0 0x557ebb56fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.366+0000 7fdedd056640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb567af0 0x557ebb56fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42204/0 (socket says 172.21.15.139:42204) 2024-09-16T01:49:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.366+0000 7fdedd056640 1 -- 172.21.15.139:0/3729145424 learned_addr learned my addr 172.21.15.139:0/3729145424 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.366+0000 7fdedd857640 1 --2- 172.21.15.139:0/3729145424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb572cb0 0x557ebb5705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.366+0000 7fdedc855640 1 --2- 172.21.15.139:0/3729145424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb568410 0x557ebb5700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdedd056640 1 -- 172.21.15.139:0/3729145424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb572cb0 msgr2=0x557ebb5705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdedd056640 1 --2- 172.21.15.139:0/3729145424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb572cb0 0x557ebb5705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdedd056640 1 -- 172.21.15.139:0/3729145424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb568410 msgr2=0x557ebb5700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdedd056640 1 --2- 172.21.15.139:0/3729145424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb568410 0x557ebb5700b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdedd056640 1 -- 172.21.15.139:0/3729145424 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ebb1df210 con 0x557ebb567af0 2024-09-16T01:49:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdedc855640 1 --2- 172.21.15.139:0/3729145424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb568410 0x557ebb5700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:49:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdedd056640 1 --2- 172.21.15.139:0/3729145424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb567af0 0x557ebb56fb70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fded400ba40 tx=0x7fded400bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b04a72bf0e4968e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdec0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:49:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdec0ff9640 1 -- 172.21.15.139:0/3729145424 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4003e60 con 0x557ebb567af0 2024-09-16T01:49:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdec0ff9640 1 -- 172.21.15.139:0/3729145424 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7fded4011440 con 0x557ebb567af0 2024-09-16T01:49:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdee2057740 1 -- 172.21.15.139:0/3729145424 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded0044040 con 0x557ebb567af0 2024-09-16T01:49:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdec0ff9640 1 -- 172.21.15.139:0/3729145424 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fded4011640 con 0x557ebb567af0 2024-09-16T01:49:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdee2057740 10 client.16509 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdee2057740 1 -- 172.21.15.139:0/3729145424 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x557ebb2ab620 con 0x557ebb567af0 2024-09-16T01:49:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdec0ff9640 1 -- 172.21.15.139:0/3729145424 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded0044040 con 0x557ebb567af0 2024-09-16T01:49:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdec0ff9640 1 -- 172.21.15.139:0/3729145424 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7fded401b9b0 con 0x557ebb567af0 2024-09-16T01:49:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.367+0000 7fdee2057740 10 client.16509 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.368+0000 7fdee2057740 10 client.16509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:49:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.368+0000 7fdee2057740 20 client.16509 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.368+0000 7fdee2057740 1 --2- 172.21.15.139:0/3729145424 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557ebb582d80 0x557ebb585200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.368+0000 7fdee2057740 4 client.16509 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:49:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.368+0000 7fdee2057740 1 -- 172.21.15.139:0/3729145424 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ebb4d9a20 con 0x557ebb582d80 2024-09-16T01:49:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.368+0000 7fdedc855640 1 --2- 172.21.15.139:0/3729145424 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557ebb582d80 0x557ebb585200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.368+0000 7fdec0ff9640 10 client.16509 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.368+0000 7fdedc855640 1 --2- 172.21.15.139:0/3729145424 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557ebb582d80 0x557ebb585200 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b338c259da4ab6 server_cookie=7221894b26f4cdfe in_seq=0 out_seq=0 2024-09-16T01:49:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.369+0000 7fdec0ff9640 1 -- 172.21.15.139:0/3729145424 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ebb4d9a20 con 0x557ebb582d80 2024-09-16T01:49:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.369+0000 7fdec0ff9640 10 client.16509 handle_command_reply: tid=0 2024-09-16T01:49:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.369+0000 7fdee2057740 1 client.16509 shutdown 2024-09-16T01:49:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.369+0000 7fdee2057740 1 -- 172.21.15.139:0/3729145424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb567af0 msgr2=0x557ebb56fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.369+0000 7fdee2057740 1 --2- 172.21.15.139:0/3729145424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb567af0 0x557ebb56fb70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fded400ba40 tx=0x7fded400bf00 comp rx=0 tx=0).stop 2024-09-16T01:49:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.369+0000 7fdee2057740 1 -- 172.21.15.139:0/3729145424 shutdown_connections 2024-09-16T01:49:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.369+0000 7fdee2057740 1 --2- 172.21.15.139:0/3729145424 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557ebb582d80 0x557ebb585200 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.369+0000 7fdee2057740 1 --2- 172.21.15.139:0/3729145424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb572cb0 0x557ebb5705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.369+0000 7fdee2057740 1 --2- 172.21.15.139:0/3729145424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb568410 0x557ebb5700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.369+0000 7fdec0ff9640 0 client.16509 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.369+0000 7fdee2057740 1 --2- 172.21.15.139:0/3729145424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb567af0 0x557ebb56fb70 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.369+0000 7fdee2057740 1 -- 172.21.15.139:0/3729145424 >> 172.21.15.139:0/3729145424 conn(0x557ebb564060 msgr2=0x557ebb5631c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.386+0000 7fdee2057740 1 -- 172.21.15.139:0/3729145424 shutdown_connections 2024-09-16T01:49:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.386+0000 7fdee2057740 1 -- 172.21.15.139:0/3729145424 wait complete. 2024-09-16T01:49:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.386+0000 7fdee2057740 20 client.16509 trim_cache size 0 max 16384 2024-09-16T01:49:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.400+0000 7fdee2057740 1 Processor -- start 2024-09-16T01:49:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.400+0000 7fdee2057740 1 -- start start 2024-09-16T01:49:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.400+0000 7fdee2057740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb567af0 0x557ebb5713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.400+0000 7fdee2057740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb568410 0x557ebb570420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.400+0000 7fdee2057740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb572cb0 0x557ebb58aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.400+0000 7fdee2057740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc010090 con 0x557ebb572cb0 2024-09-16T01:49:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.400+0000 7fdee2057740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc011ed0 con 0x557ebb568410 2024-09-16T01:49:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.400+0000 7fdee2057740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc010820 con 0x557ebb567af0 2024-09-16T01:49:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.400+0000 7fdedd857640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb572cb0 0x557ebb58aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.400+0000 7fdedd857640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb572cb0 0x557ebb58aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42212/0 (socket says 172.21.15.139:42212) 2024-09-16T01:49:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.400+0000 7fdedd857640 1 -- 172.21.15.139:0/696679214 learned_addr learned my addr 172.21.15.139:0/696679214 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.401+0000 7fdedc855640 1 --2- 172.21.15.139:0/696679214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb568410 0x557ebb570420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.401+0000 7fdedd857640 1 -- 172.21.15.139:0/696679214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb567af0 msgr2=0x557ebb5713d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:49:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.401+0000 7fdedd857640 1 --2- 172.21.15.139:0/696679214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb567af0 0x557ebb5713d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.401+0000 7fdedd857640 1 -- 172.21.15.139:0/696679214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb568410 msgr2=0x557ebb570420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.401+0000 7fdedd857640 1 --2- 172.21.15.139:0/696679214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb568410 0x557ebb570420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.401+0000 7fdedd857640 1 -- 172.21.15.139:0/696679214 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ebb563e90 con 0x557ebb572cb0 2024-09-16T01:49:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.401+0000 7fdedc855640 1 --2- 172.21.15.139:0/696679214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb568410 0x557ebb570420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:49:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.401+0000 7fdedd857640 1 --2- 172.21.15.139:0/696679214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb572cb0 0x557ebb58aad0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fded0009890 tx=0x7fded003e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de7b10bd4940a73 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.401+0000 7fdec2ffd640 1 -- 172.21.15.139:0/696679214 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded005a070 con 0x557ebb572cb0 2024-09-16T01:49:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.401+0000 7fdec2ffd640 1 -- 172.21.15.139:0/696679214 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded0049dc0 con 0x557ebb572cb0 2024-09-16T01:49:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.401+0000 7fdec2ffd640 1 -- 172.21.15.139:0/696679214 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded004d030 con 0x557ebb572cb0 2024-09-16T01:49:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 -- 172.21.15.139:0/696679214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb572cb0 msgr2=0x557ebb58aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 --2- 172.21.15.139:0/696679214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb572cb0 0x557ebb58aad0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fded0009890 tx=0x7fded003e040 comp rx=0 tx=0).stop 2024-09-16T01:49:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 -- 172.21.15.139:0/696679214 shutdown_connections 2024-09-16T01:49:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 --2- 172.21.15.139:0/696679214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb572cb0 0x557ebb58aad0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 --2- 172.21.15.139:0/696679214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb568410 0x557ebb570420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 --2- 172.21.15.139:0/696679214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb567af0 0x557ebb5713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 -- 172.21.15.139:0/696679214 >> 172.21.15.139:0/696679214 conn(0x557ebb564060 msgr2=0x557ebb5631c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 -- 172.21.15.139:0/696679214 shutdown_connections 2024-09-16T01:49:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 -- 172.21.15.139:0/696679214 wait complete. 2024-09-16T01:49:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 -- start start 2024-09-16T01:49:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 Processor -- start 2024-09-16T01:49:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb567af0 0x557ebb576dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb568410 0x557ebb577310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb572cb0 0x557ebb5ecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.402+0000 7fdee2057740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc011740 con 0x557ebb568410 2024-09-16T01:49:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdee2057740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc010fb0 con 0x557ebb567af0 2024-09-16T01:49:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdee2057740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc00e090 con 0x557ebb572cb0 2024-09-16T01:49:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdee2057740 10 client.0 _ll_register_callbacks cb 0x7fded40042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdedc855640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb568410 0x557ebb577310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdedc855640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb568410 0x557ebb577310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42214/0 (socket says 172.21.15.139:42214) 2024-09-16T01:49:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdedc855640 1 -- 172.21.15.139:0/1444170328 learned_addr learned my addr 172.21.15.139:0/1444170328 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdedd056640 1 --2- 172.21.15.139:0/1444170328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb567af0 0x557ebb576dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdedd857640 1 --2- 172.21.15.139:0/1444170328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb572cb0 0x557ebb5ecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdedc855640 1 -- 172.21.15.139:0/1444170328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb572cb0 msgr2=0x557ebb5ecaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdedc855640 1 --2- 172.21.15.139:0/1444170328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb572cb0 0x557ebb5ecaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdedc855640 1 -- 172.21.15.139:0/1444170328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb567af0 msgr2=0x557ebb576dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdedc855640 1 --2- 172.21.15.139:0/1444170328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb567af0 0x557ebb576dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdedc855640 1 -- 172.21.15.139:0/1444170328 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ebb577bf0 con 0x557ebb568410 2024-09-16T01:49:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdedc855640 1 --2- 172.21.15.139:0/1444170328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb568410 0x557ebb577310 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fdec80cd790 tx=0x7fdec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9854ed5c7e055a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdec0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:49:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdec0ff9640 1 -- 172.21.15.139:0/1444170328 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec80b8070 con 0x557ebb568410 2024-09-16T01:49:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdec0ff9640 1 -- 172.21.15.139:0/1444170328 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7fdec8098440 con 0x557ebb568410 2024-09-16T01:49:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdee2057740 1 -- 172.21.15.139:0/1444170328 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded0049dc0 con 0x557ebb568410 2024-09-16T01:49:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdec0ff9640 1 -- 172.21.15.139:0/1444170328 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdec8020040 con 0x557ebb568410 2024-09-16T01:49:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdee2057740 10 client.16515 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.403+0000 7fdee2057740 1 -- 172.21.15.139:0/1444170328 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x557ebb5ee0d0 con 0x557ebb568410 2024-09-16T01:49:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.404+0000 7fdec0ff9640 1 -- 172.21.15.139:0/1444170328 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded0049dc0 con 0x557ebb568410 2024-09-16T01:49:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.404+0000 7fdec0ff9640 1 -- 172.21.15.139:0/1444170328 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7fdec80b4050 con 0x557ebb568410 2024-09-16T01:49:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.404+0000 7fdee2057740 10 client.16515 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.404+0000 7fdee2057740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:49:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.404+0000 7fdee2057740 20 client.16515 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.404+0000 7fdee2057740 1 --2- 172.21.15.139:0/1444170328 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557ebb5ee870 0x557ebb5f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.404+0000 7fdee2057740 4 client.16515 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:49:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.404+0000 7fdee2057740 1 -- 172.21.15.139:0/1444170328 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ebb4d3d70 con 0x557ebb5ee870 2024-09-16T01:49:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.404+0000 7fdedd056640 1 --2- 172.21.15.139:0/1444170328 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557ebb5ee870 0x557ebb5f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.404+0000 7fdedd056640 1 --2- 172.21.15.139:0/1444170328 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557ebb5ee870 0x557ebb5f0c90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e00efe687089c25 server_cookie=88d1c53e64945774 in_seq=0 out_seq=0 2024-09-16T01:49:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.404+0000 7fdec0ff9640 10 client.16515 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:41.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:41 smithi139 ceph-mon[31175]: pgmap v288: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 KiB/s wr, 80 op/s 2024-09-16T01:49:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.432+0000 7fdec0ff9640 1 -- 172.21.15.139:0/1444170328 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ebb4d3d70 con 0x557ebb5ee870 2024-09-16T01:49:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.432+0000 7fdec0ff9640 10 client.16515 handle_command_reply: tid=0 2024-09-16T01:49:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.432+0000 7fdee2057740 1 client.16515 shutdown 2024-09-16T01:49:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.432+0000 7fdee2057740 1 -- 172.21.15.139:0/1444170328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb568410 msgr2=0x557ebb577310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.432+0000 7fdee2057740 1 --2- 172.21.15.139:0/1444170328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb568410 0x557ebb577310 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fdec80cd790 tx=0x7fdec80046c0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdee2057740 1 -- 172.21.15.139:0/1444170328 shutdown_connections 2024-09-16T01:49:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdee2057740 1 --2- 172.21.15.139:0/1444170328 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557ebb5ee870 0x557ebb5f0c90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdee2057740 1 --2- 172.21.15.139:0/1444170328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ebb572cb0 0x557ebb5ecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdee2057740 1 --2- 172.21.15.139:0/1444170328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557ebb568410 0x557ebb577310 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdec0ff9640 0 client.16515 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdee2057740 1 --2- 172.21.15.139:0/1444170328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ebb567af0 0x557ebb576dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdee2057740 1 -- 172.21.15.139:0/1444170328 >> 172.21.15.139:0/1444170328 conn(0x557ebb564060 msgr2=0x557ebb57db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdee2057740 1 -- 172.21.15.139:0/1444170328 shutdown_connections 2024-09-16T01:49:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdee2057740 1 -- 172.21.15.139:0/1444170328 wait complete. 2024-09-16T01:49:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdee2057740 20 client.16515 trim_cache size 0 max 16384 2024-09-16T01:49:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:49:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:49:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:49:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:49:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdedf2ba640 1 -- 172.21.15.139:0/4106849571 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fdec40a34e0 msgr2=0x7fdec40a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdedf2ba640 1 --2- 172.21.15.139:0/4106849571 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fdec40a34e0 0x7fdec40a59a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fded004e000 tx=0x7fded004c000 comp rx=0 tx=0).stop 2024-09-16T01:49:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdedf2ba640 1 -- 172.21.15.139:0/4106849571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded8102050 msgr2=0x7fded8197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdedf2ba640 1 --2- 172.21.15.139:0/4106849571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded8102050 0x7fded8197680 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdec800d950 tx=0x7fdec800de10 comp rx=0 tx=0).stop 2024-09-16T01:49:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdedf2ba640 1 -- 172.21.15.139:0/4106849571 shutdown_connections 2024-09-16T01:49:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdedf2ba640 1 --2- 172.21.15.139:0/4106849571 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fdec40a34e0 0x7fdec40a59a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdedf2ba640 1 --2- 172.21.15.139:0/4106849571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded8105af0 0x7fded819ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdedf2ba640 1 --2- 172.21.15.139:0/4106849571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fded8102050 0x7fded8197680 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdedf2ba640 1 --2- 172.21.15.139:0/4106849571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fded81016d0 0x7fded8197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdedf2ba640 1 -- 172.21.15.139:0/4106849571 >> 172.21.15.139:0/4106849571 conn(0x7fded8075ec0 msgr2=0x7fded80ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdedf2ba640 1 -- 172.21.15.139:0/4106849571 shutdown_connections 2024-09-16T01:49:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:41.433+0000 7fdedf2ba640 1 -- 172.21.15.139:0/4106849571 wait complete. 2024-09-16T01:49:41.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:41 smithi176 ceph-mon[35324]: pgmap v288: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 KiB/s wr, 80 op/s 2024-09-16T01:49:41.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:41 smithi164 ceph-mon[35121]: pgmap v288: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 KiB/s wr, 80 op/s 2024-09-16T01:49:41.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.586+0000 7fae5430b640 1 -- 172.21.15.139:0/1223719928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c101de0 msgr2=0x7fae4c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.586+0000 7fae5430b640 1 --2- 172.21.15.139:0/1223719928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c101de0 0x7fae4c1021c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fae48009900 tx=0x7fae4802a410 comp rx=0 tx=0).stop 2024-09-16T01:49:41.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.586+0000 7fae5430b640 1 -- 172.21.15.139:0/1223719928 shutdown_connections 2024-09-16T01:49:41.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.586+0000 7fae5430b640 1 --2- 172.21.15.139:0/1223719928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae4c106f10 0x7fae4c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.586+0000 7fae5430b640 1 --2- 172.21.15.139:0/1223719928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae4c102790 0x7fae4c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.586+0000 7fae5430b640 1 --2- 172.21.15.139:0/1223719928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c101de0 0x7fae4c1021c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.586+0000 7fae5430b640 1 -- 172.21.15.139:0/1223719928 >> 172.21.15.139:0/1223719928 conn(0x7fae4c0fd8c0 msgr2=0x7fae4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:41.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.586+0000 7fae5430b640 1 -- 172.21.15.139:0/1223719928 shutdown_connections 2024-09-16T01:49:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.586+0000 7fae5430b640 1 -- 172.21.15.139:0/1223719928 wait complete. 2024-09-16T01:49:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.587+0000 7fae5430b640 1 Processor -- start 2024-09-16T01:49:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.587+0000 7fae5430b640 1 -- start start 2024-09-16T01:49:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.587+0000 7fae5430b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae4c101de0 0x7fae4c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.587+0000 7fae5430b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c102790 0x7fae4c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.587+0000 7fae5430b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae4c106f10 0x7fae4c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.587+0000 7fae5430b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c111f90 con 0x7fae4c102790 2024-09-16T01:49:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.587+0000 7fae5430b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c111e10 con 0x7fae4c101de0 2024-09-16T01:49:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.587+0000 7fae5430b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c112110 con 0x7fae4c106f10 2024-09-16T01:49:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.587+0000 7fae518a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c102790 0x7fae4c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.587+0000 7fae518a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c102790 0x7fae4c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42218/0 (socket says 172.21.15.139:42218) 2024-09-16T01:49:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.587+0000 7fae518a6640 1 -- 172.21.15.139:0/4183378256 learned_addr learned my addr 172.21.15.139:0/4183378256 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:41.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.587+0000 7fae520a7640 1 --2- 172.21.15.139:0/4183378256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae4c101de0 0x7fae4c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.587+0000 7fae528a8640 1 --2- 172.21.15.139:0/4183378256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae4c106f10 0x7fae4c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.588+0000 7fae518a6640 1 -- 172.21.15.139:0/4183378256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae4c106f10 msgr2=0x7fae4c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.588+0000 7fae518a6640 1 --2- 172.21.15.139:0/4183378256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae4c106f10 0x7fae4c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.588+0000 7fae518a6640 1 -- 172.21.15.139:0/4183378256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae4c101de0 msgr2=0x7fae4c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.588+0000 7fae518a6640 1 --2- 172.21.15.139:0/4183378256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae4c101de0 0x7fae4c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.588+0000 7fae518a6640 1 -- 172.21.15.139:0/4183378256 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c1a69b0 con 0x7fae4c102790 2024-09-16T01:49:41.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.588+0000 7fae518a6640 1 --2- 172.21.15.139:0/4183378256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c102790 0x7fae4c074a30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00b4a0 tx=0x7fae3c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:41.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.588+0000 7fae437fe640 1 -- 172.21.15.139:0/4183378256 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c014070 con 0x7fae4c102790 2024-09-16T01:49:41.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.588+0000 7fae5430b640 1 -- 172.21.15.139:0/4183378256 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4c1a6ca0 con 0x7fae4c102790 2024-09-16T01:49:41.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.588+0000 7fae437fe640 1 -- 172.21.15.139:0/4183378256 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae3c010af0 con 0x7fae4c102790 2024-09-16T01:49:41.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.588+0000 7fae437fe640 1 -- 172.21.15.139:0/4183378256 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c017d10 con 0x7fae4c102790 2024-09-16T01:49:41.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.588+0000 7fae5430b640 1 -- 172.21.15.139:0/4183378256 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae4c1a7190 con 0x7fae4c102790 2024-09-16T01:49:41.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.590+0000 7fae437fe640 1 -- 172.21.15.139:0/4183378256 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fae3c017070 con 0x7fae4c102790 2024-09-16T01:49:41.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae566e4740 1 Processor -- start 2024-09-16T01:49:41.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae566e4740 1 -- start start 2024-09-16T01:49:41.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae437fe640 1 --2- 172.21.15.139:0/4183378256 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fae380a33b0 0x7fae380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae566e4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e6d060 0x55c871e6d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae566e4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 0x55c871e77d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae566e4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e78240 0x55c871e7a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae566e4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c871dd2620 con 0x55c871e78240 2024-09-16T01:49:41.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae566e4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c871d79e50 con 0x55c871e6d980 2024-09-16T01:49:41.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae566e4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c871dce5e0 con 0x55c871e6d060 2024-09-16T01:49:41.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae528a8640 1 --2- 172.21.15.139:0/4183378256 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fae380a33b0 0x7fae380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae520a7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e6d060 0x55c871e6d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae437fe640 1 -- 172.21.15.139:0/4183378256 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7fae3c0cf2e0 con 0x7fae4c102790 2024-09-16T01:49:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae518a6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 0x55c871e77d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae520a7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e6d060 0x55c871e6d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:47402/0 (socket says 172.21.15.139:47402) 2024-09-16T01:49:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae520a7640 1 -- 172.21.15.139:0/3830543503 learned_addr learned my addr 172.21.15.139:0/3830543503 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae528a8640 1 --2- 172.21.15.139:0/3830543503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e78240 0x55c871e7a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae518a6640 1 -- 172.21.15.139:0/3830543503 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e6d060 msgr2=0x55c871e6d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae518a6640 1 --2- 172.21.15.139:0/3830543503 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e6d060 0x55c871e6d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae518a6640 1 -- 172.21.15.139:0/3830543503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e78240 msgr2=0x55c871e7a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae518a6640 1 --2- 172.21.15.139:0/3830543503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e78240 0x55c871e7a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.591+0000 7fae518a6640 1 -- 172.21.15.139:0/3830543503 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c871d6ad40 con 0x55c871e6d980 2024-09-16T01:49:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.592+0000 7fae528a8640 1 --2- 172.21.15.139:0/4183378256 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fae380a33b0 0x7fae380a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fae440098b0 tx=0x7fae44009e20 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.592+0000 7fae518a6640 1 --2- 172.21.15.139:0/3830543503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 0x55c871e77d00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0cc730 tx=0x7fae3c01ea20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d90048595f1e2573 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:41.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.592+0000 7fae40dea640 1 -- 172.21.15.139:0/3830543503 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c0ae1c0 con 0x55c871e6d980 2024-09-16T01:49:41.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.592+0000 7fae40dea640 1 -- 172.21.15.139:0/3830543503 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae3c0adbd0 con 0x55c871e6d980 2024-09-16T01:49:41.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.592+0000 7fae40dea640 1 -- 172.21.15.139:0/3830543503 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c013170 con 0x55c871e6d980 2024-09-16T01:49:41.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.593+0000 7fae566e4740 1 -- 172.21.15.139:0/3830543503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 msgr2=0x55c871e77d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.593+0000 7fae566e4740 1 --2- 172.21.15.139:0/3830543503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 0x55c871e77d00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0cc730 tx=0x7fae3c01ea20 comp rx=0 tx=0).stop 2024-09-16T01:49:41.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.593+0000 7fae566e4740 1 -- 172.21.15.139:0/3830543503 shutdown_connections 2024-09-16T01:49:41.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.593+0000 7fae566e4740 1 --2- 172.21.15.139:0/3830543503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e78240 0x55c871e7a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.593+0000 7fae566e4740 1 --2- 172.21.15.139:0/3830543503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 0x55c871e77d00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.593+0000 7fae566e4740 1 --2- 172.21.15.139:0/3830543503 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e6d060 0x55c871e6d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.593+0000 7fae566e4740 1 -- 172.21.15.139:0/3830543503 >> 172.21.15.139:0/3830543503 conn(0x55c871e695f0 msgr2=0x55c871e699d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:41.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.593+0000 7fae566e4740 1 -- 172.21.15.139:0/3830543503 shutdown_connections 2024-09-16T01:49:41.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.593+0000 7fae566e4740 1 -- 172.21.15.139:0/3830543503 wait complete. 2024-09-16T01:49:41.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.593+0000 7fae566e4740 1 -- start start 2024-09-16T01:49:41.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.593+0000 7fae566e4740 1 Processor -- start 2024-09-16T01:49:41.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae566e4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e6d060 0x55c871e75140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae566e4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 0x55c871e75680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae566e4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e78240 0x55c871e75bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae566e4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c871e7ae10 con 0x55c871e6d060 2024-09-16T01:49:41.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae566e4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c871d5af10 con 0x55c871e6d980 2024-09-16T01:49:41.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae566e4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c871e7af90 con 0x55c871e78240 2024-09-16T01:49:41.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae566e4740 10 client.0 _ll_register_callbacks cb 0x55c871d7edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:41.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae520a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e6d060 0x55c871e75140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae520a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e6d060 0x55c871e75140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42244/0 (socket says 172.21.15.139:42244) 2024-09-16T01:49:41.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae520a7640 1 -- 172.21.15.139:0/3220345568 learned_addr learned my addr 172.21.15.139:0/3220345568 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:41.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae528a8640 1 --2- 172.21.15.139:0/3220345568 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e78240 0x55c871e75bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae518a6640 1 --2- 172.21.15.139:0/3220345568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 0x55c871e75680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae520a7640 1 -- 172.21.15.139:0/3220345568 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e78240 msgr2=0x55c871e75bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae520a7640 1 --2- 172.21.15.139:0/3220345568 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e78240 0x55c871e75bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae520a7640 1 -- 172.21.15.139:0/3220345568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 msgr2=0x55c871e75680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae520a7640 1 --2- 172.21.15.139:0/3220345568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 0x55c871e75680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae520a7640 1 -- 172.21.15.139:0/3220345568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c871ae1580 con 0x55c871e6d060 2024-09-16T01:49:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae518a6640 1 --2- 172.21.15.139:0/3220345568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 0x55c871e75680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:49:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae520a7640 1 --2- 172.21.15.139:0/3220345568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e6d060 0x55c871e75140 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fae48009230 tx=0x7fae480319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e529a4788d7ec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:49:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae35ffb640 1 -- 172.21.15.139:0/3220345568 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48035040 con 0x55c871e6d060 2024-09-16T01:49:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae35ffb640 1 -- 172.21.15.139:0/3220345568 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7fae48005030 con 0x55c871e6d060 2024-09-16T01:49:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.594+0000 7fae566e4740 1 -- 172.21.15.139:0/3220345568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae3c0adbd0 con 0x55c871e6d060 2024-09-16T01:49:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.595+0000 7fae35ffb640 1 -- 172.21.15.139:0/3220345568 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fae48005230 con 0x55c871e6d060 2024-09-16T01:49:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.595+0000 7fae566e4740 10 client.16527 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.595+0000 7fae566e4740 1 -- 172.21.15.139:0/3220345568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55c871bab960 con 0x55c871e6d060 2024-09-16T01:49:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.595+0000 7fae35ffb640 1 -- 172.21.15.139:0/3220345568 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae3c0adbd0 con 0x55c871e6d060 2024-09-16T01:49:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.595+0000 7fae35ffb640 1 -- 172.21.15.139:0/3220345568 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7fae48003680 con 0x55c871e6d060 2024-09-16T01:49:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.595+0000 7fae566e4740 10 client.16527 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.595+0000 7fae566e4740 10 client.16527 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:49:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.595+0000 7fae566e4740 20 client.16527 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.595+0000 7fae566e4740 1 --2- 172.21.15.139:0/3220345568 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c871e87e60 0x55c871e8a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.595+0000 7fae566e4740 4 client.16527 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:49:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.595+0000 7fae566e4740 1 -- 172.21.15.139:0/3220345568 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c871ddfb90 con 0x55c871e87e60 2024-09-16T01:49:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.595+0000 7fae518a6640 1 --2- 172.21.15.139:0/3220345568 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c871e87e60 0x55c871e8a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.596+0000 7fae35ffb640 10 client.16527 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.596+0000 7fae518a6640 1 --2- 172.21.15.139:0/3220345568 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c871e87e60 0x55c871e8a2e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b161efcce407fb66 server_cookie=d3e8358794873985 in_seq=0 out_seq=0 2024-09-16T01:49:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.596+0000 7fae35ffb640 1 -- 172.21.15.139:0/3220345568 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c871ddfb90 con 0x55c871e87e60 2024-09-16T01:49:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.596+0000 7fae35ffb640 10 client.16527 handle_command_reply: tid=0 2024-09-16T01:49:41.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.596+0000 7fae566e4740 1 client.16527 shutdown 2024-09-16T01:49:41.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.597+0000 7fae566e4740 1 -- 172.21.15.139:0/3220345568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e6d060 msgr2=0x55c871e75140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.597+0000 7fae566e4740 1 --2- 172.21.15.139:0/3220345568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e6d060 0x55c871e75140 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fae48009230 tx=0x7fae480319b0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.597+0000 7fae566e4740 1 -- 172.21.15.139:0/3220345568 shutdown_connections 2024-09-16T01:49:41.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.597+0000 7fae566e4740 1 --2- 172.21.15.139:0/3220345568 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c871e87e60 0x55c871e8a2e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.597+0000 7fae566e4740 1 --2- 172.21.15.139:0/3220345568 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e78240 0x55c871e75bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.597+0000 7fae566e4740 1 --2- 172.21.15.139:0/3220345568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 0x55c871e75680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.597+0000 7fae566e4740 1 --2- 172.21.15.139:0/3220345568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e6d060 0x55c871e75140 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.597+0000 7fae566e4740 1 -- 172.21.15.139:0/3220345568 >> 172.21.15.139:0/3220345568 conn(0x55c871e695f0 msgr2=0x55c871e6f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:41.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.597+0000 7fae35ffb640 0 client.16527 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:41.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.597+0000 7fae566e4740 1 -- 172.21.15.139:0/3220345568 shutdown_connections 2024-09-16T01:49:41.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.597+0000 7fae566e4740 1 -- 172.21.15.139:0/3220345568 wait complete. 2024-09-16T01:49:41.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.597+0000 7fae566e4740 20 client.16527 trim_cache size 0 max 16384 2024-09-16T01:49:41.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae566e4740 1 Processor -- start 2024-09-16T01:49:41.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae566e4740 1 -- start start 2024-09-16T01:49:41.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae566e4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d060 0x55c871e706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae566e4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e6d980 0x55c871e6f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae566e4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e78240 0x55c871e8fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae566e4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c1b13d0 con 0x55c871e78240 2024-09-16T01:49:41.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae566e4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c1b3210 con 0x55c871e6d060 2024-09-16T01:49:41.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae566e4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c1b1b60 con 0x55c871e6d980 2024-09-16T01:49:41.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae528a8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e78240 0x55c871e8fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae528a8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e78240 0x55c871e8fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42254/0 (socket says 172.21.15.139:42254) 2024-09-16T01:49:41.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae528a8640 1 -- 172.21.15.139:0/2049462526 learned_addr learned my addr 172.21.15.139:0/2049462526 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:41.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae520a7640 1 --2- 172.21.15.139:0/2049462526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d060 0x55c871e706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae518a6640 1 --2- 172.21.15.139:0/2049462526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e6d980 0x55c871e6f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae528a8640 1 -- 172.21.15.139:0/2049462526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e6d980 msgr2=0x55c871e6f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae528a8640 1 --2- 172.21.15.139:0/2049462526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e6d980 0x55c871e6f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae528a8640 1 -- 172.21.15.139:0/2049462526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d060 msgr2=0x55c871e706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae528a8640 1 --2- 172.21.15.139:0/2049462526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d060 0x55c871e706b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.614+0000 7fae528a8640 1 -- 172.21.15.139:0/2049462526 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c871e67bb0 con 0x55c871e78240 2024-09-16T01:49:41.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.615+0000 7fae528a8640 1 --2- 172.21.15.139:0/2049462526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e78240 0x55c871e8fbb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fae44009a80 tx=0x7fae44004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac0247c0e376387 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:41.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.615+0000 7fae37fff640 1 -- 172.21.15.139:0/2049462526 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44004030 con 0x55c871e78240 2024-09-16T01:49:41.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.615+0000 7fae37fff640 1 -- 172.21.15.139:0/2049462526 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae440046b0 con 0x55c871e78240 2024-09-16T01:49:41.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.615+0000 7fae37fff640 1 -- 172.21.15.139:0/2049462526 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44029b60 con 0x55c871e78240 2024-09-16T01:49:41.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.615+0000 7fae566e4740 1 -- 172.21.15.139:0/2049462526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e78240 msgr2=0x55c871e8fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:41.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.615+0000 7fae566e4740 1 --2- 172.21.15.139:0/2049462526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e78240 0x55c871e8fbb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fae44009a80 tx=0x7fae44004ab0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 -- 172.21.15.139:0/2049462526 shutdown_connections 2024-09-16T01:49:41.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 --2- 172.21.15.139:0/2049462526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e78240 0x55c871e8fbb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 --2- 172.21.15.139:0/2049462526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e6d980 0x55c871e6f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 --2- 172.21.15.139:0/2049462526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d060 0x55c871e706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 -- 172.21.15.139:0/2049462526 >> 172.21.15.139:0/2049462526 conn(0x55c871e695f0 msgr2=0x55c871e6eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:41.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 -- 172.21.15.139:0/2049462526 shutdown_connections 2024-09-16T01:49:41.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 -- 172.21.15.139:0/2049462526 wait complete. 2024-09-16T01:49:41.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 -- start start 2024-09-16T01:49:41.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 Processor -- start 2024-09-16T01:49:41.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e6d060 0x55c871e7c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 0x55c871e7c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e78240 0x55c871e7cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c1b2a80 con 0x55c871e6d060 2024-09-16T01:49:41.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c1b22f0 con 0x55c871e6d980 2024-09-16T01:49:41.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c1af3d0 con 0x55c871e78240 2024-09-16T01:49:41.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae520a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e6d060 0x55c871e7c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae566e4740 10 client.0 _ll_register_callbacks cb 0x7fae48005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:41.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae520a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e6d060 0x55c871e7c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42270/0 (socket says 172.21.15.139:42270) 2024-09-16T01:49:41.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.616+0000 7fae520a7640 1 -- 172.21.15.139:0/4189304294 learned_addr learned my addr 172.21.15.139:0/4189304294 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:41.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae518a6640 1 --2- 172.21.15.139:0/4189304294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 0x55c871e7c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae528a8640 1 --2- 172.21.15.139:0/4189304294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e78240 0x55c871e7cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae520a7640 1 -- 172.21.15.139:0/4189304294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e78240 msgr2=0x55c871e7cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae520a7640 1 --2- 172.21.15.139:0/4189304294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e78240 0x55c871e7cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae520a7640 1 -- 172.21.15.139:0/4189304294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 msgr2=0x55c871e7c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:41.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae520a7640 1 --2- 172.21.15.139:0/4189304294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 0x55c871e7c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:41.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae520a7640 1 -- 172.21.15.139:0/4189304294 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c871ef1dd0 con 0x55c871e6d060 2024-09-16T01:49:41.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae520a7640 1 --2- 172.21.15.139:0/4189304294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e6d060 0x55c871e7c190 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fae4802a450 tx=0x7fae48003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973f5c5f2497f8cb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:41.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:49:41.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae35ffb640 1 -- 172.21.15.139:0/4189304294 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48057020 con 0x55c871e6d060 2024-09-16T01:49:41.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae35ffb640 1 -- 172.21.15.139:0/4189304294 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7fae480053d0 con 0x55c871e6d060 2024-09-16T01:49:41.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae566e4740 1 -- 172.21.15.139:0/4189304294 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae440046b0 con 0x55c871e6d060 2024-09-16T01:49:41.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae35ffb640 1 -- 172.21.15.139:0/4189304294 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fae48035070 con 0x55c871e6d060 2024-09-16T01:49:41.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae566e4740 10 client.16533 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:41.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae566e4740 1 -- 172.21.15.139:0/4189304294 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55c871ef30f0 con 0x55c871e6d060 2024-09-16T01:49:41.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.617+0000 7fae35ffb640 1 -- 172.21.15.139:0/4189304294 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae440046b0 con 0x55c871e6d060 2024-09-16T01:49:41.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.618+0000 7fae35ffb640 1 -- 172.21.15.139:0/4189304294 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7fae480047e0 con 0x55c871e6d060 2024-09-16T01:49:41.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.618+0000 7fae566e4740 10 client.16533 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:41.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.618+0000 7fae566e4740 10 client.16533 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:49:41.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.618+0000 7fae566e4740 20 client.16533 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:41.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.618+0000 7fae566e4740 1 --2- 172.21.15.139:0/4189304294 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c871ef38c0 0x55c871ef5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:41.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.618+0000 7fae566e4740 4 client.16533 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7f2d60e9", "release": true, "await": true}] 2024-09-16T01:49:41.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.618+0000 7fae566e4740 1 -- 172.21.15.139:0/4189304294 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7f2d60e9", "release": true, "await": true}) -- 0x55c871dda0a0 con 0x55c871ef38c0 2024-09-16T01:49:41.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.618+0000 7fae518a6640 1 --2- 172.21.15.139:0/4189304294 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c871ef38c0 0x55c871ef5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:41.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.618+0000 7fae35ffb640 10 client.16533 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:41.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:41.618+0000 7fae518a6640 1 --2- 172.21.15.139:0/4189304294 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c871ef38c0 0x55c871ef5ce0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d93b19060900e53 server_cookie=3b5d866584cf8a08 in_seq=0 out_seq=0 2024-09-16T01:49:42.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae35ffb640 1 -- 172.21.15.139:0/4189304294 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c871dda0a0 con 0x55c871ef38c0 2024-09-16T01:49:42.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae35ffb640 10 client.16533 handle_command_reply: tid=0 2024-09-16T01:49:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae566e4740 1 client.16533 shutdown 2024-09-16T01:49:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae566e4740 1 -- 172.21.15.139:0/4189304294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e6d060 msgr2=0x55c871e7c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae566e4740 1 --2- 172.21.15.139:0/4189304294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e6d060 0x55c871e7c190 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fae4802a450 tx=0x7fae48003ef0 comp rx=0 tx=0).stop 2024-09-16T01:49:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae566e4740 1 -- 172.21.15.139:0/4189304294 shutdown_connections 2024-09-16T01:49:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae566e4740 1 --2- 172.21.15.139:0/4189304294 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c871ef38c0 0x55c871ef5ce0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae566e4740 1 --2- 172.21.15.139:0/4189304294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c871e78240 0x55c871e7cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae566e4740 1 --2- 172.21.15.139:0/4189304294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c871e6d980 0x55c871e7c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae566e4740 1 --2- 172.21.15.139:0/4189304294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c871e6d060 0x55c871e7c190 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae566e4740 1 -- 172.21.15.139:0/4189304294 >> 172.21.15.139:0/4189304294 conn(0x55c871e695f0 msgr2=0x55c871e6f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae35ffb640 0 client.16533 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:42.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae566e4740 1 -- 172.21.15.139:0/4189304294 shutdown_connections 2024-09-16T01:49:42.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae566e4740 1 -- 172.21.15.139:0/4189304294 wait complete. 2024-09-16T01:49:42.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.523+0000 7fae566e4740 20 client.16533 trim_cache size 0 max 16384 2024-09-16T01:49:42.527 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:49:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 50, 2024-09-16T01:49:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:49:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 32, 2024-09-16T01:49:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:49:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "7f2d60e9": { 2024-09-16T01:49:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 32, 2024-09-16T01:49:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:49:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:49:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:49:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:49:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:49:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:49:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 142.3, 2024-09-16T01:49:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:49:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:49:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:49:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:49:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:49:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:49:42.529 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:42.529 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:42.529 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:42.529 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:42.529 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:42.529 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:49:42.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.524+0000 7fae5430b640 1 -- 172.21.15.139:0/4183378256 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fae380a33b0 msgr2=0x7fae380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.524+0000 7fae5430b640 1 --2- 172.21.15.139:0/4183378256 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fae380a33b0 0x7fae380a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fae440098b0 tx=0x7fae44009e20 comp rx=0 tx=0).stop 2024-09-16T01:49:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.524+0000 7fae5430b640 1 -- 172.21.15.139:0/4183378256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c102790 msgr2=0x7fae4c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.524+0000 7fae5430b640 1 --2- 172.21.15.139:0/4183378256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c102790 0x7fae4c074a30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00b4a0 tx=0x7fae3c00b960 comp rx=0 tx=0).stop 2024-09-16T01:49:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.524+0000 7fae5430b640 1 -- 172.21.15.139:0/4183378256 shutdown_connections 2024-09-16T01:49:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.524+0000 7fae5430b640 1 --2- 172.21.15.139:0/4183378256 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fae380a33b0 0x7fae380a5870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.524+0000 7fae5430b640 1 --2- 172.21.15.139:0/4183378256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae4c106f10 0x7fae4c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.524+0000 7fae5430b640 1 --2- 172.21.15.139:0/4183378256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c102790 0x7fae4c074a30 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.524+0000 7fae5430b640 1 --2- 172.21.15.139:0/4183378256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae4c101de0 0x7fae4c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.524+0000 7fae5430b640 1 -- 172.21.15.139:0/4183378256 >> 172.21.15.139:0/4183378256 conn(0x7fae4c0fd8c0 msgr2=0x7fae4c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:42.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.524+0000 7fae5430b640 1 -- 172.21.15.139:0/4183378256 shutdown_connections 2024-09-16T01:49:42.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:42.524+0000 7fae5430b640 1 -- 172.21.15.139:0/4183378256 wait complete. 2024-09-16T01:49:43.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:43 smithi139 ceph-mon[31175]: pgmap v289: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 43 op/s 2024-09-16T01:49:43.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:43 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:49:43.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:43 smithi176 ceph-mon[35324]: pgmap v289: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 43 op/s 2024-09-16T01:49:43.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:43 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:49:43.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:43 smithi164 ceph-mon[35121]: pgmap v289: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 43 op/s 2024-09-16T01:49:43.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:43 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:49:43.601 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7f2d60e9', total seconds elapsed: 26.9 2024-09-16T01:49:43.601 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.35 2024-09-16T01:49:43.601 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:49:43.935 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:49:44.308 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:44 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:49:44.308 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:44 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:49:44.308 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:44 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:49:44.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:44 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:49:44.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:44 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:49:44.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:44 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:49:44.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:44 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:49:44.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:44 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:49:44.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:44 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:49:44.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.581+0000 7f1ada5c3640 1 -- 172.21.15.139:0/351562462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad4102570 msgr2=0x7f1ad41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:44.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.581+0000 7f1ada5c3640 1 --2- 172.21.15.139:0/351562462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad4102570 0x7f1ad41029b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800dc20 tx=0x7f1ac802e390 comp rx=0 tx=0).stop 2024-09-16T01:49:44.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.581+0000 7f1ada5c3640 1 -- 172.21.15.139:0/351562462 shutdown_connections 2024-09-16T01:49:44.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.581+0000 7f1ada5c3640 1 --2- 172.21.15.139:0/351562462 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ad4102ef0 0x7f1ad410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:44.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.581+0000 7f1ada5c3640 1 --2- 172.21.15.139:0/351562462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad4102570 0x7f1ad41029b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:44.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.581+0000 7f1ada5c3640 1 --2- 172.21.15.139:0/351562462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ad4100e20 0x7f1ad4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:44.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.581+0000 7f1ada5c3640 1 -- 172.21.15.139:0/351562462 >> 172.21.15.139:0/351562462 conn(0x7f1ad40fc7d0 msgr2=0x7f1ad40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:44.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.582+0000 7f1ada5c3640 1 -- 172.21.15.139:0/351562462 shutdown_connections 2024-09-16T01:49:44.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.582+0000 7f1ada5c3640 1 -- 172.21.15.139:0/351562462 wait complete. 2024-09-16T01:49:44.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.583+0000 7f1ada5c3640 1 Processor -- start 2024-09-16T01:49:44.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.583+0000 7f1ada5c3640 1 -- start start 2024-09-16T01:49:44.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.584+0000 7f1ada5c3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad4100e20 0x7f1ad4196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:44.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.584+0000 7f1ada5c3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ad4102570 0x7f1ad41968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:44.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.584+0000 7f1ada5c3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ad4102ef0 0x7f1ad419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:44.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.584+0000 7f1ada5c3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad410fe80 con 0x7f1ad4102ef0 2024-09-16T01:49:44.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.584+0000 7f1ada5c3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad410fd00 con 0x7f1ad4100e20 2024-09-16T01:49:44.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.584+0000 7f1ada5c3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad4110000 con 0x7f1ad4102570 2024-09-16T01:49:44.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.584+0000 7f1ad8b60640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ad4102ef0 0x7f1ad419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:44.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.584+0000 7f1ad8b60640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ad4102ef0 0x7f1ad419ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39066/0 (socket says 172.21.15.139:39066) 2024-09-16T01:49:44.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.584+0000 7f1ad8b60640 1 -- 172.21.15.139:0/1750491033 learned_addr learned my addr 172.21.15.139:0/1750491033 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:44.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.584+0000 7f1ad3fff640 1 --2- 172.21.15.139:0/1750491033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad4100e20 0x7f1ad4196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:44.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.586+0000 7f1ad8b60640 1 -- 172.21.15.139:0/1750491033 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ad4102570 msgr2=0x7f1ad41968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:44.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.586+0000 7f1ad8b60640 1 --2- 172.21.15.139:0/1750491033 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ad4102570 0x7f1ad41968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:44.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.586+0000 7f1ad8b60640 1 -- 172.21.15.139:0/1750491033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad4100e20 msgr2=0x7f1ad4196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:44.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.586+0000 7f1ad8b60640 1 --2- 172.21.15.139:0/1750491033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad4100e20 0x7f1ad4196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:44.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.586+0000 7f1ad8b60640 1 -- 172.21.15.139:0/1750491033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad419b330 con 0x7f1ad4102ef0 2024-09-16T01:49:44.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.586+0000 7f1ad8b60640 1 --2- 172.21.15.139:0/1750491033 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ad4102ef0 0x7f1ad419ac50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00eaa0 tx=0x7f1acc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:44.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.586+0000 7f1ad17fa640 1 -- 172.21.15.139:0/1750491033 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc004d20 con 0x7f1ad4102ef0 2024-09-16T01:49:44.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.586+0000 7f1ad17fa640 1 -- 172.21.15.139:0/1750491033 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1acc005020 con 0x7f1ad4102ef0 2024-09-16T01:49:44.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.586+0000 7f1ad17fa640 1 -- 172.21.15.139:0/1750491033 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc0116b0 con 0x7f1ad4102ef0 2024-09-16T01:49:44.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.586+0000 7f1ada5c3640 1 -- 172.21.15.139:0/1750491033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad419b620 con 0x7f1ad4102ef0 2024-09-16T01:49:44.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.587+0000 7f1ada5c3640 1 -- 172.21.15.139:0/1750491033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad41a2f00 con 0x7f1ad4102ef0 2024-09-16T01:49:44.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.589+0000 7f1abaffd640 1 -- 172.21.15.139:0/1750491033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad4101200 con 0x7f1ad4102ef0 2024-09-16T01:49:44.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.592+0000 7f1ad17fa640 1 -- 172.21.15.139:0/1750491033 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1acc0051d0 con 0x7f1ad4102ef0 2024-09-16T01:49:44.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.592+0000 7f1ad17fa640 1 --2- 172.21.15.139:0/1750491033 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1abc0a34b0 0x7f1abc0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:44.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.593+0000 7f1ad17fa640 1 -- 172.21.15.139:0/1750491033 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7f1acc0d0390 con 0x7f1ad4102ef0 2024-09-16T01:49:44.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.594+0000 7f1ad3fff640 1 --2- 172.21.15.139:0/1750491033 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1abc0a34b0 0x7f1abc0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:44.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.595+0000 7f1ad3fff640 1 --2- 172.21.15.139:0/1750491033 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1abc0a34b0 0x7f1abc0a5970 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4005fd0 tx=0x7f1ac400f040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:44.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.601+0000 7f1ad17fa640 1 -- 172.21.15.139:0/1750491033 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1acc016030 con 0x7f1ad4102ef0 2024-09-16T01:49:44.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.995+0000 7f1abaffd640 1 -- 172.21.15.139:0/1750491033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ad4062df0 con 0x7f1ad4102ef0 2024-09-16T01:49:44.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.996+0000 7f1ad17fa640 1 -- 172.21.15.139:0/1750491033 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+12028 (secure 0 0 0) 0x7f1acc09c900 con 0x7f1ad4102ef0 2024-09-16T01:49:44.998 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:49:44.998 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":50,"btime":"2024-09-16T01:48:57:039963+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:48:56.596845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24454,24443,24442,24433]},"id":1}]} 2024-09-16T01:49:44.999 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 50 2024-09-16T01:49:45.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.998+0000 7f1abaffd640 1 -- 172.21.15.139:0/1750491033 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1abc0a34b0 msgr2=0x7f1abc0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:45.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.998+0000 7f1abaffd640 1 --2- 172.21.15.139:0/1750491033 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1abc0a34b0 0x7f1abc0a5970 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4005fd0 tx=0x7f1ac400f040 comp rx=0 tx=0).stop 2024-09-16T01:49:45.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.998+0000 7f1abaffd640 1 -- 172.21.15.139:0/1750491033 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ad4102ef0 msgr2=0x7f1ad419ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:45.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.998+0000 7f1abaffd640 1 --2- 172.21.15.139:0/1750491033 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ad4102ef0 0x7f1ad419ac50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00eaa0 tx=0x7f1acc00eea0 comp rx=0 tx=0).stop 2024-09-16T01:49:45.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.999+0000 7f1abaffd640 1 -- 172.21.15.139:0/1750491033 shutdown_connections 2024-09-16T01:49:45.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.999+0000 7f1abaffd640 1 --2- 172.21.15.139:0/1750491033 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1abc0a34b0 0x7f1abc0a5970 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:45.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.999+0000 7f1abaffd640 1 --2- 172.21.15.139:0/1750491033 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ad4102ef0 0x7f1ad419ac50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:45.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.999+0000 7f1abaffd640 1 --2- 172.21.15.139:0/1750491033 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ad4102570 0x7f1ad41968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:45.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.999+0000 7f1abaffd640 1 --2- 172.21.15.139:0/1750491033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad4100e20 0x7f1ad4196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:45.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.999+0000 7f1abaffd640 1 -- 172.21.15.139:0/1750491033 >> 172.21.15.139:0/1750491033 conn(0x7f1ad40fc7d0 msgr2=0x7f1ad41048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:45.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.999+0000 7f1abaffd640 1 -- 172.21.15.139:0/1750491033 shutdown_connections 2024-09-16T01:49:45.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:44.999+0000 7f1abaffd640 1 -- 172.21.15.139:0/1750491033 wait complete. 2024-09-16T01:49:45.377 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:45 smithi139 ceph-mon[31175]: pgmap v290: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 26 op/s 2024-09-16T01:49:45.377 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:45 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1750491033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:49:45.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:45 smithi164 ceph-mon[35121]: pgmap v290: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 26 op/s 2024-09-16T01:49:45.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:45 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1750491033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:49:45.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:45 smithi176 ceph-mon[35324]: pgmap v290: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 26 op/s 2024-09-16T01:49:45.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:45 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1750491033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:49:45.729 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0' 2024-09-16T01:49:45.729 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0 2024-09-16T01:49:46.177 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:49:46.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.707+0000 7f9f573eb640 1 -- 172.21.15.139:0/850821010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f50102ef0 msgr2=0x7f9f5010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:46.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.707+0000 7f9f573eb640 1 --2- 172.21.15.139:0/850821010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f50102ef0 0x7f9f5010da40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009ca0 tx=0x7f9f4802a950 comp rx=0 tx=0).stop 2024-09-16T01:49:46.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.707+0000 7f9f573eb640 1 -- 172.21.15.139:0/850821010 shutdown_connections 2024-09-16T01:49:46.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.707+0000 7f9f573eb640 1 --2- 172.21.15.139:0/850821010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f50102ef0 0x7f9f5010da40 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.707+0000 7f9f573eb640 1 --2- 172.21.15.139:0/850821010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f50102570 0x7f9f501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.707+0000 7f9f573eb640 1 --2- 172.21.15.139:0/850821010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f50100e20 0x7f9f50101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.707+0000 7f9f573eb640 1 -- 172.21.15.139:0/850821010 >> 172.21.15.139:0/850821010 conn(0x7f9f500fc7d0 msgr2=0x7f9f500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:46.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.707+0000 7f9f573eb640 1 -- 172.21.15.139:0/850821010 shutdown_connections 2024-09-16T01:49:46.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.707+0000 7f9f573eb640 1 -- 172.21.15.139:0/850821010 wait complete. 2024-09-16T01:49:46.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f573eb640 1 Processor -- start 2024-09-16T01:49:46.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f573eb640 1 -- start start 2024-09-16T01:49:46.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f573eb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f50100e20 0x7f9f501972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f573eb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f50102570 0x7f9f50197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f573eb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f50102ef0 0x7f9f5019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f573eb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50110f80 con 0x7f9f50102ef0 2024-09-16T01:49:46.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f573eb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50110e00 con 0x7f9f50102570 2024-09-16T01:49:46.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f573eb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50111100 con 0x7f9f50100e20 2024-09-16T01:49:46.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f55988640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f50102ef0 0x7f9f5019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f55187640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f50100e20 0x7f9f501972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f55988640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f50102ef0 0x7f9f5019bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39080/0 (socket says 172.21.15.139:39080) 2024-09-16T01:49:46.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f55187640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f50100e20 0x7f9f501972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:53792/0 (socket says 172.21.15.139:53792) 2024-09-16T01:49:46.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f55988640 1 -- 172.21.15.139:0/2318627452 learned_addr learned my addr 172.21.15.139:0/2318627452 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:46.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f54986640 1 --2- 172.21.15.139:0/2318627452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f50102570 0x7f9f50197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f55988640 1 -- 172.21.15.139:0/2318627452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f50100e20 msgr2=0x7f9f501972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:46.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f55988640 1 --2- 172.21.15.139:0/2318627452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f50100e20 0x7f9f501972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f55988640 1 -- 172.21.15.139:0/2318627452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f50102570 msgr2=0x7f9f50197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:46.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f55988640 1 --2- 172.21.15.139:0/2318627452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f50102570 0x7f9f50197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f55988640 1 -- 172.21.15.139:0/2318627452 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5019c290 con 0x7f9f50102ef0 2024-09-16T01:49:46.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.708+0000 7f9f55187640 1 --2- 172.21.15.139:0/2318627452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f50100e20 0x7f9f501972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:49:46.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.709+0000 7f9f55988640 1 --2- 172.21.15.139:0/2318627452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f50102ef0 0x7f9f5019bbb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9f48004af0 tx=0x7f9f4802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:46.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.709+0000 7f9f427fc640 1 -- 172.21.15.139:0/2318627452 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f48034040 con 0x7f9f50102ef0 2024-09-16T01:49:46.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.709+0000 7f9f573eb640 1 -- 172.21.15.139:0/2318627452 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5019c520 con 0x7f9f50102ef0 2024-09-16T01:49:46.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.709+0000 7f9f427fc640 1 -- 172.21.15.139:0/2318627452 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f48030920 con 0x7f9f50102ef0 2024-09-16T01:49:46.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.709+0000 7f9f427fc640 1 -- 172.21.15.139:0/2318627452 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f48004740 con 0x7f9f50102ef0 2024-09-16T01:49:46.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.709+0000 7f9f573eb640 1 -- 172.21.15.139:0/2318627452 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f501a3e60 con 0x7f9f50102ef0 2024-09-16T01:49:46.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.711+0000 7f9f427fc640 1 -- 172.21.15.139:0/2318627452 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9f48037020 con 0x7f9f50102ef0 2024-09-16T01:49:46.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f597c4740 1 Processor -- start 2024-09-16T01:49:46.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f597c4740 1 -- start start 2024-09-16T01:49:46.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f597c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136a0850 0x5563136a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f597c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a1170 0x5563136ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f597c4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136aba30 0x5563136ade20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f597c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556313605160 con 0x5563136a0850 2024-09-16T01:49:46.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f597c4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5563135ac720 con 0x5563136aba30 2024-09-16T01:49:46.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f597c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556313601120 con 0x5563136a1170 2024-09-16T01:49:46.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f427fc640 1 --2- 172.21.15.139:0/2318627452 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9f3c0a33b0 0x7f9f3c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f55187640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a1170 0x5563136ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f54986640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136a0850 0x5563136a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f55187640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a1170 0x5563136ab4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:53802/0 (socket says 172.21.15.139:53802) 2024-09-16T01:49:46.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f55187640 1 -- 172.21.15.139:0/683446451 learned_addr learned my addr 172.21.15.139:0/683446451 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:46.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f55988640 1 --2- 172.21.15.139:0/2318627452 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9f3c0a33b0 0x7f9f3c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f54986640 1 --2- 172.21.15.139:0/683446451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136aba30 0x5563136ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f55187640 1 -- 172.21.15.139:0/683446451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136aba30 msgr2=0x5563136ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:46.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.712+0000 7f9f55187640 1 --2- 172.21.15.139:0/683446451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136aba30 0x5563136ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.713+0000 7f9f55187640 1 -- 172.21.15.139:0/683446451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136a0850 msgr2=0x5563136a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:46.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.713+0000 7f9f55187640 1 --2- 172.21.15.139:0/683446451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136a0850 0x5563136a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.713+0000 7f9f55187640 1 -- 172.21.15.139:0/683446451 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556313376ba0 con 0x5563136a1170 2024-09-16T01:49:46.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.713+0000 7f9f427fc640 1 -- 172.21.15.139:0/2318627452 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7f9f480f0e00 con 0x7f9f50102ef0 2024-09-16T01:49:46.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.713+0000 7f9f54986640 1 --2- 172.21.15.139:0/683446451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136aba30 0x5563136ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:49:46.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.713+0000 7f9f55187640 1 --2- 172.21.15.139:0/683446451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a1170 0x5563136ab4f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0115c0 tx=0x7f9f4c011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c91980c0ae4517e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:46.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.713+0000 7f9f55988640 1 --2- 172.21.15.139:0/2318627452 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9f3c0a33b0 0x7f9f3c0a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9f48006010 tx=0x7f9f480b2210 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:46.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.713+0000 7f9f3b7fe640 1 -- 172.21.15.139:0/683446451 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c00f640 con 0x5563136a1170 2024-09-16T01:49:46.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.713+0000 7f9f3b7fe640 1 -- 172.21.15.139:0/683446451 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c00fcc0 con 0x5563136a1170 2024-09-16T01:49:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.713+0000 7f9f3b7fe640 1 -- 172.21.15.139:0/683446451 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c019da0 con 0x5563136a1170 2024-09-16T01:49:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.714+0000 7f9f597c4740 1 -- 172.21.15.139:0/683446451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a1170 msgr2=0x5563136ab4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.714+0000 7f9f597c4740 1 --2- 172.21.15.139:0/683446451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a1170 0x5563136ab4f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0115c0 tx=0x7f9f4c011a80 comp rx=0 tx=0).stop 2024-09-16T01:49:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.714+0000 7f9f597c4740 1 -- 172.21.15.139:0/683446451 shutdown_connections 2024-09-16T01:49:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.714+0000 7f9f597c4740 1 --2- 172.21.15.139:0/683446451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136aba30 0x5563136ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.714+0000 7f9f597c4740 1 --2- 172.21.15.139:0/683446451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a1170 0x5563136ab4f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.714+0000 7f9f597c4740 1 --2- 172.21.15.139:0/683446451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136a0850 0x5563136a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.714+0000 7f9f597c4740 1 -- 172.21.15.139:0/683446451 >> 172.21.15.139:0/683446451 conn(0x55631369cde0 msgr2=0x55631369d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.714+0000 7f9f597c4740 1 -- 172.21.15.139:0/683446451 shutdown_connections 2024-09-16T01:49:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.714+0000 7f9f597c4740 1 -- 172.21.15.139:0/683446451 wait complete. 2024-09-16T01:49:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.714+0000 7f9f597c4740 1 -- start start 2024-09-16T01:49:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.714+0000 7f9f597c4740 1 Processor -- start 2024-09-16T01:49:46.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f597c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136a0850 0x5563136a8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f597c4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136a8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f597c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136aba30 0x5563136a9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f597c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5563136ae600 con 0x5563136a0850 2024-09-16T01:49:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f597c4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55631358d900 con 0x5563136a1170 2024-09-16T01:49:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f597c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5563136ae780 con 0x5563136aba30 2024-09-16T01:49:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f597c4740 10 client.0 _ll_register_callbacks cb 0x55631358b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f54986640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136a0850 0x5563136a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f54986640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136a0850 0x5563136a8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39096/0 (socket says 172.21.15.139:39096) 2024-09-16T01:49:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f54986640 1 -- 172.21.15.139:0/1072127878 learned_addr learned my addr 172.21.15.139:0/1072127878 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f54986640 1 --2- 172.21.15.139:0/1072127878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136aba30 0x5563136a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f55187640 1 --2- 172.21.15.139:0/1072127878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136a8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f54986640 1 -- 172.21.15.139:0/1072127878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136aba30 msgr2=0x5563136a9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:46.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f54986640 1 --2- 172.21.15.139:0/1072127878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136aba30 0x5563136a9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f54986640 1 -- 172.21.15.139:0/1072127878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 msgr2=0x5563136a8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f54986640 1 --2- 172.21.15.139:0/1072127878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136a8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f54986640 1 -- 172.21.15.139:0/1072127878 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563135cdc50 con 0x5563136a0850 2024-09-16T01:49:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f55187640 1 --2- 172.21.15.139:0/1072127878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:49:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f54986640 1 --2- 172.21.15.139:0/1072127878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136a0850 0x5563136a8910 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400ff20 tx=0x7f9f4400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3ad02a7f884a3f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:49:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f38ff9640 1 -- 172.21.15.139:0/1072127878 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44005020 con 0x5563136a0850 2024-09-16T01:49:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f38ff9640 1 -- 172.21.15.139:0/1072127878 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7f9f440051c0 con 0x5563136a0850 2024-09-16T01:49:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f597c4740 1 -- 172.21.15.139:0/1072127878 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4c00fcc0 con 0x5563136a0850 2024-09-16T01:49:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f38ff9640 1 -- 172.21.15.139:0/1072127878 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9f44029050 con 0x5563136a0850 2024-09-16T01:49:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f597c4740 10 client.16551 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.715+0000 7f9f597c4740 1 -- 172.21.15.139:0/1072127878 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5563136ba270 con 0x5563136a0850 2024-09-16T01:49:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.716+0000 7f9f38ff9640 1 -- 172.21.15.139:0/1072127878 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c00fcc0 con 0x5563136a0850 2024-09-16T01:49:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.716+0000 7f9f38ff9640 1 -- 172.21.15.139:0/1072127878 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7f9f440116e0 con 0x5563136a0850 2024-09-16T01:49:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.716+0000 7f9f597c4740 10 client.16551 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.716+0000 7f9f597c4740 10 client.16551 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:49:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.716+0000 7f9f597c4740 20 client.16551 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.716+0000 7f9f597c4740 1 --2- 172.21.15.139:0/1072127878 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5563136bb770 0x5563136bdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.716+0000 7f9f597c4740 4 client.16551 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:49:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.716+0000 7f9f597c4740 1 -- 172.21.15.139:0/1072127878 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556313612890 con 0x5563136bb770 2024-09-16T01:49:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.716+0000 7f9f55187640 1 --2- 172.21.15.139:0/1072127878 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5563136bb770 0x5563136bdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.716+0000 7f9f55187640 1 --2- 172.21.15.139:0/1072127878 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5563136bb770 0x5563136bdbf0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce450bfd751b93d9 server_cookie=98b49f8b91dce901 in_seq=0 out_seq=0 2024-09-16T01:49:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.716+0000 7f9f38ff9640 10 client.16551 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.717+0000 7f9f38ff9640 1 -- 172.21.15.139:0/1072127878 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556313612890 con 0x5563136bb770 2024-09-16T01:49:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.717+0000 7f9f38ff9640 10 client.16551 handle_command_reply: tid=0 2024-09-16T01:49:46.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.717+0000 7f9f597c4740 1 client.16551 shutdown 2024-09-16T01:49:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.717+0000 7f9f597c4740 1 -- 172.21.15.139:0/1072127878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136a0850 msgr2=0x5563136a8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.717+0000 7f9f597c4740 1 --2- 172.21.15.139:0/1072127878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136a0850 0x5563136a8910 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400ff20 tx=0x7f9f4400a2d0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.717+0000 7f9f597c4740 1 -- 172.21.15.139:0/1072127878 shutdown_connections 2024-09-16T01:49:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.717+0000 7f9f597c4740 1 --2- 172.21.15.139:0/1072127878 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5563136bb770 0x5563136bdbf0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.717+0000 7f9f597c4740 1 --2- 172.21.15.139:0/1072127878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136aba30 0x5563136a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.717+0000 7f9f597c4740 1 --2- 172.21.15.139:0/1072127878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.717+0000 7f9f597c4740 1 --2- 172.21.15.139:0/1072127878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136a0850 0x5563136a8910 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.717+0000 7f9f38ff9640 0 client.16551 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.717+0000 7f9f597c4740 1 -- 172.21.15.139:0/1072127878 >> 172.21.15.139:0/1072127878 conn(0x55631369cde0 msgr2=0x5563136a2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.717+0000 7f9f597c4740 1 -- 172.21.15.139:0/1072127878 shutdown_connections 2024-09-16T01:49:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.717+0000 7f9f597c4740 1 -- 172.21.15.139:0/1072127878 wait complete. 2024-09-16T01:49:46.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.718+0000 7f9f597c4740 20 client.16551 trim_cache size 0 max 16384 2024-09-16T01:49:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.735+0000 7f9f597c4740 1 Processor -- start 2024-09-16T01:49:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.735+0000 7f9f597c4740 1 -- start start 2024-09-16T01:49:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.735+0000 7f9f597c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a0850 0x5563136a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.735+0000 7f9f597c4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136c3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.735+0000 7f9f597c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136aba30 0x5563136c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.735+0000 7f9f597c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501acd50 con 0x5563136aba30 2024-09-16T01:49:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.735+0000 7f9f597c4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501aeb90 con 0x5563136a1170 2024-09-16T01:49:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.735+0000 7f9f597c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501ad4e0 con 0x5563136a0850 2024-09-16T01:49:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.735+0000 7f9f54986640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136aba30 0x5563136c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.735+0000 7f9f54986640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136aba30 0x5563136c3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39106/0 (socket says 172.21.15.139:39106) 2024-09-16T01:49:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.735+0000 7f9f54986640 1 -- 172.21.15.139:0/2115830899 learned_addr learned my addr 172.21.15.139:0/2115830899 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.735+0000 7f9f54986640 1 --2- 172.21.15.139:0/2115830899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a0850 0x5563136a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.735+0000 7f9f55187640 1 --2- 172.21.15.139:0/2115830899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.736+0000 7f9f54986640 1 -- 172.21.15.139:0/2115830899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a0850 msgr2=0x5563136a29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.736+0000 7f9f54986640 1 --2- 172.21.15.139:0/2115830899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a0850 0x5563136a29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.736+0000 7f9f54986640 1 -- 172.21.15.139:0/2115830899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 msgr2=0x5563136c3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.736+0000 7f9f54986640 1 --2- 172.21.15.139:0/2115830899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136c3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.736+0000 7f9f54986640 1 -- 172.21.15.139:0/2115830899 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563136a78c0 con 0x5563136aba30 2024-09-16T01:49:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.736+0000 7f9f54986640 1 --2- 172.21.15.139:0/2115830899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136aba30 0x5563136c3a40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400fcd0 tx=0x7f9f44005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f68b7dcaa5b9ec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.736+0000 7f9f3affd640 1 -- 172.21.15.139:0/2115830899 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400ad20 con 0x5563136aba30 2024-09-16T01:49:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.736+0000 7f9f3affd640 1 -- 172.21.15.139:0/2115830899 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f44005500 con 0x5563136aba30 2024-09-16T01:49:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.736+0000 7f9f3affd640 1 -- 172.21.15.139:0/2115830899 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f440048e0 con 0x5563136aba30 2024-09-16T01:49:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 -- 172.21.15.139:0/2115830899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136aba30 msgr2=0x5563136c3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 --2- 172.21.15.139:0/2115830899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136aba30 0x5563136c3a40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400fcd0 tx=0x7f9f44005b10 comp rx=0 tx=0).stop 2024-09-16T01:49:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 -- 172.21.15.139:0/2115830899 shutdown_connections 2024-09-16T01:49:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 --2- 172.21.15.139:0/2115830899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136aba30 0x5563136c3a40 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 --2- 172.21.15.139:0/2115830899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 --2- 172.21.15.139:0/2115830899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a0850 0x5563136a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 -- 172.21.15.139:0/2115830899 >> 172.21.15.139:0/2115830899 conn(0x55631369cde0 msgr2=0x5563136a22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 -- 172.21.15.139:0/2115830899 shutdown_connections 2024-09-16T01:49:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 -- 172.21.15.139:0/2115830899 wait complete. 2024-09-16T01:49:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 -- start start 2024-09-16T01:49:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 Processor -- start 2024-09-16T01:49:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a0850 0x5563136af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136aba30 0x5563136b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501ae400 con 0x5563136aba30 2024-09-16T01:49:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501adc70 con 0x5563136a1170 2024-09-16T01:49:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.737+0000 7f9f597c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501aad50 con 0x5563136a0850 2024-09-16T01:49:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f597c4740 10 client.0 _ll_register_callbacks cb 0x7f9f4400ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f55187640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f54986640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136aba30 0x5563136b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f55187640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:48412/0 (socket says 172.21.15.139:48412) 2024-09-16T01:49:46.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f54986640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136aba30 0x5563136b0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39120/0 (socket says 172.21.15.139:39120) 2024-09-16T01:49:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f55187640 1 -- 172.21.15.139:0/1273562146 learned_addr learned my addr 172.21.15.139:0/1273562146 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f54986640 1 --2- 172.21.15.139:0/1273562146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a0850 0x5563136af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f55187640 1 -- 172.21.15.139:0/1273562146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a0850 msgr2=0x5563136af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f55187640 1 --2- 172.21.15.139:0/1273562146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a0850 0x5563136af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f55187640 1 -- 172.21.15.139:0/1273562146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136aba30 msgr2=0x5563136b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f55187640 1 --2- 172.21.15.139:0/1273562146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136aba30 0x5563136b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f55187640 1 -- 172.21.15.139:0/1273562146 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556313725550 con 0x5563136a1170 2024-09-16T01:49:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f54986640 1 --2- 172.21.15.139:0/1273562146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a0850 0x5563136af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:49:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f55187640 1 --2- 172.21.15.139:0/1273562146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136afd30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0058c0 tx=0x7f9f4c019940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79ea685bb12b03e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:49:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f38ff9640 1 -- 172.21.15.139:0/1273562146 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0199b0 con 0x5563136a1170 2024-09-16T01:49:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f597c4740 1 -- 172.21.15.139:0/1273562146 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f44005500 con 0x5563136a1170 2024-09-16T01:49:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f38ff9640 1 -- 172.21.15.139:0/1273562146 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7f9f4c016040 con 0x5563136a1170 2024-09-16T01:49:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f38ff9640 1 -- 172.21.15.139:0/1273562146 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9f4c03f050 con 0x5563136a1170 2024-09-16T01:49:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f597c4740 10 client.25985 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.738+0000 7f9f597c4740 1 -- 172.21.15.139:0/1273562146 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5563137267f0 con 0x5563136a1170 2024-09-16T01:49:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.739+0000 7f9f38ff9640 1 -- 172.21.15.139:0/1273562146 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f44005500 con 0x5563136a1170 2024-09-16T01:49:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.739+0000 7f9f38ff9640 1 -- 172.21.15.139:0/1273562146 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7f9f4c026a30 con 0x5563136a1170 2024-09-16T01:49:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.739+0000 7f9f597c4740 10 client.25985 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.739+0000 7f9f597c4740 10 client.25985 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:49:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.739+0000 7f9f597c4740 20 client.25985 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.739+0000 7f9f597c4740 1 --2- 172.21.15.139:0/1273562146 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556313726f70 0x556313729390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.739+0000 7f9f597c4740 4 client.25985 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}] 2024-09-16T01:49:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.739+0000 7f9f597c4740 1 -- 172.21.15.139:0/1273562146 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}) -- 0x556313698410 con 0x556313726f70 2024-09-16T01:49:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.739+0000 7f9f54986640 1 --2- 172.21.15.139:0/1273562146 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556313726f70 0x556313729390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.739+0000 7f9f38ff9640 10 client.25985 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:46.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:46.740+0000 7f9f54986640 1 --2- 172.21.15.139:0/1273562146 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556313726f70 0x556313729390 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c673dad6267c1fe9 server_cookie=6eeaec5146d7be14 in_seq=0 out_seq=0 2024-09-16T01:49:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:47 smithi164 ceph-mon[35121]: pgmap v291: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 453 KiB/s wr, 37 op/s 2024-09-16T01:49:47.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:47 smithi139 ceph-mon[31175]: pgmap v291: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 453 KiB/s wr, 37 op/s 2024-09-16T01:49:47.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:47 smithi176 ceph-mon[35324]: pgmap v291: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 453 KiB/s wr, 37 op/s 2024-09-16T01:49:49.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:49 smithi164 ceph-mon[35121]: pgmap v292: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 622 KiB/s wr, 87 op/s 2024-09-16T01:49:49.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:49 smithi139 ceph-mon[31175]: pgmap v292: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 622 KiB/s wr, 87 op/s 2024-09-16T01:49:49.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:49 smithi176 ceph-mon[35324]: pgmap v292: 129 pgs: 129 active+clean; 2.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 622 KiB/s wr, 87 op/s 2024-09-16T01:49:51.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:51 smithi164 ceph-mon[35121]: pgmap v293: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 622 KiB/s wr, 98 op/s 2024-09-16T01:49:51.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:51 smithi139 ceph-mon[31175]: pgmap v293: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 622 KiB/s wr, 98 op/s 2024-09-16T01:49:51.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:51 smithi176 ceph-mon[35324]: pgmap v293: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 622 KiB/s wr, 98 op/s 2024-09-16T01:49:53.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:53 smithi164 ceph-mon[35121]: pgmap v294: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 889 KiB/s wr, 142 op/s 2024-09-16T01:49:53.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:53 smithi139 ceph-mon[31175]: pgmap v294: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 889 KiB/s wr, 142 op/s 2024-09-16T01:49:53.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:53 smithi176 ceph-mon[35324]: pgmap v294: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 889 KiB/s wr, 142 op/s 2024-09-16T01:49:54.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.525+0000 7f9f38ff9640 1 -- 172.21.15.139:0/1273562146 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556313698410 con 0x556313726f70 2024-09-16T01:49:54.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.525+0000 7f9f38ff9640 10 client.25985 handle_command_reply: tid=0 2024-09-16T01:49:54.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.525+0000 7f9f597c4740 1 client.25985 shutdown 2024-09-16T01:49:54.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.525+0000 7f9f597c4740 1 -- 172.21.15.139:0/1273562146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 msgr2=0x5563136afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:54.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.525+0000 7f9f597c4740 1 --2- 172.21.15.139:0/1273562146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136afd30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0058c0 tx=0x7f9f4c019940 comp rx=0 tx=0).stop 2024-09-16T01:49:54.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.525+0000 7f9f597c4740 1 -- 172.21.15.139:0/1273562146 shutdown_connections 2024-09-16T01:49:54.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.525+0000 7f9f597c4740 1 --2- 172.21.15.139:0/1273562146 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556313726f70 0x556313729390 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.525+0000 7f9f597c4740 1 --2- 172.21.15.139:0/1273562146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5563136aba30 0x5563136b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.525+0000 7f9f597c4740 1 --2- 172.21.15.139:0/1273562146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563136a1170 0x5563136afd30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.525+0000 7f9f38ff9640 0 client.25985 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:54.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.525+0000 7f9f597c4740 1 --2- 172.21.15.139:0/1273562146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5563136a0850 0x5563136af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.525+0000 7f9f597c4740 1 -- 172.21.15.139:0/1273562146 >> 172.21.15.139:0/1273562146 conn(0x55631369cde0 msgr2=0x5563136a2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:54.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.526+0000 7f9f597c4740 1 -- 172.21.15.139:0/1273562146 shutdown_connections 2024-09-16T01:49:54.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.526+0000 7f9f597c4740 1 -- 172.21.15.139:0/1273562146 wait complete. 2024-09-16T01:49:54.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.526+0000 7f9f597c4740 20 client.25985 trim_cache size 0 max 16384 2024-09-16T01:49:54.529 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:49:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 50, 2024-09-16T01:49:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:49:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 34, 2024-09-16T01:49:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:49:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "c50bf1fe": { 2024-09-16T01:49:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 34, 2024-09-16T01:49:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:49:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:49:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:49:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:49:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:49:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:49:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 142.3, 2024-09-16T01:49:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:49:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:49:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:49:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:49:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:49:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:49:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:49:54.531 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:49:54.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.526+0000 7f9f573eb640 1 -- 172.21.15.139:0/2318627452 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9f3c0a33b0 msgr2=0x7f9f3c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:54.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.526+0000 7f9f573eb640 1 --2- 172.21.15.139:0/2318627452 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9f3c0a33b0 0x7f9f3c0a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9f48006010 tx=0x7f9f480b2210 comp rx=0 tx=0).stop 2024-09-16T01:49:54.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.526+0000 7f9f573eb640 1 -- 172.21.15.139:0/2318627452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f50102ef0 msgr2=0x7f9f5019bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:54.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.526+0000 7f9f573eb640 1 --2- 172.21.15.139:0/2318627452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f50102ef0 0x7f9f5019bbb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9f48004af0 tx=0x7f9f4802f5b0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.526+0000 7f9f573eb640 1 -- 172.21.15.139:0/2318627452 shutdown_connections 2024-09-16T01:49:54.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.526+0000 7f9f573eb640 1 --2- 172.21.15.139:0/2318627452 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9f3c0a33b0 0x7f9f3c0a5870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.526+0000 7f9f573eb640 1 --2- 172.21.15.139:0/2318627452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f50102ef0 0x7f9f5019bbb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.526+0000 7f9f573eb640 1 --2- 172.21.15.139:0/2318627452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f50102570 0x7f9f50197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.526+0000 7f9f573eb640 1 --2- 172.21.15.139:0/2318627452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f50100e20 0x7f9f501972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.526+0000 7f9f573eb640 1 -- 172.21.15.139:0/2318627452 >> 172.21.15.139:0/2318627452 conn(0x7f9f500fc7d0 msgr2=0x7f9f50106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:54.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.527+0000 7f9f573eb640 1 -- 172.21.15.139:0/2318627452 shutdown_connections 2024-09-16T01:49:54.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:49:54.527+0000 7f9f573eb640 1 -- 172.21.15.139:0/2318627452 wait complete. 2024-09-16T01:49:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.530+0000 7f35dc18d640 1 -- 172.21.15.139:0/478571927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35d4101ad0 msgr2=0x7f35d4101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.530+0000 7f35dc18d640 1 --2- 172.21.15.139:0/478571927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35d4101ad0 0x7f35d4101f30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009930 tx=0x7f35c402a420 comp rx=0 tx=0).stop 2024-09-16T01:49:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.530+0000 7f35dc18d640 1 -- 172.21.15.139:0/478571927 shutdown_connections 2024-09-16T01:49:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.530+0000 7f35dc18d640 1 --2- 172.21.15.139:0/478571927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35d4102470 0x7f35d410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.530+0000 7f35dc18d640 1 --2- 172.21.15.139:0/478571927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35d4101ad0 0x7f35d4101f30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.530+0000 7f35dc18d640 1 --2- 172.21.15.139:0/478571927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35d4107b70 0x7f35d4107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.530+0000 7f35dc18d640 1 -- 172.21.15.139:0/478571927 >> 172.21.15.139:0/478571927 conn(0x7f35d40fd7d0 msgr2=0x7f35d40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.530+0000 7f35dc18d640 1 -- 172.21.15.139:0/478571927 shutdown_connections 2024-09-16T01:49:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.530+0000 7f35dc18d640 1 -- 172.21.15.139:0/478571927 wait complete. 2024-09-16T01:49:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.531+0000 7f35dc18d640 1 Processor -- start 2024-09-16T01:49:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.531+0000 7f35dc18d640 1 -- start start 2024-09-16T01:49:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.531+0000 7f35dc18d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35d4101ad0 0x7f35d40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.531+0000 7f35dc18d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35d4102470 0x7f35d4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.531+0000 7f35dc18d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35d4107b70 0x7f35d4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.531+0000 7f35dc18d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4111f20 con 0x7f35d4101ad0 2024-09-16T01:49:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.531+0000 7f35dc18d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4111da0 con 0x7f35d4107b70 2024-09-16T01:49:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.531+0000 7f35dc18d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f35d41120a0 con 0x7f35d4102470 2024-09-16T01:49:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.531+0000 7f35d9f29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35d4101ad0 0x7f35d40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.531+0000 7f35d9f29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35d4101ad0 0x7f35d40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58644/0 (socket says 172.21.15.139:58644) 2024-09-16T01:49:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.531+0000 7f35d9f29640 1 -- 172.21.15.139:0/679262635 learned_addr learned my addr 172.21.15.139:0/679262635 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.531+0000 7f35d9728640 1 --2- 172.21.15.139:0/679262635 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35d4102470 0x7f35d4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.531+0000 7f35da72a640 1 --2- 172.21.15.139:0/679262635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35d4107b70 0x7f35d4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.532+0000 7f35d9f29640 1 -- 172.21.15.139:0/679262635 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35d4102470 msgr2=0x7f35d4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.532+0000 7f35d9f29640 1 --2- 172.21.15.139:0/679262635 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35d4102470 0x7f35d4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.532+0000 7f35d9f29640 1 -- 172.21.15.139:0/679262635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35d4107b70 msgr2=0x7f35d4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.532+0000 7f35d9f29640 1 --2- 172.21.15.139:0/679262635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35d4107b70 0x7f35d4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.532+0000 7f35d9f29640 1 -- 172.21.15.139:0/679262635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d41a6830 con 0x7f35d4101ad0 2024-09-16T01:49:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.532+0000 7f35d9f29640 1 --2- 172.21.15.139:0/679262635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35d4101ad0 0x7f35d40744f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f35d000ea60 tx=0x7f35d000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.532+0000 7f35caffd640 1 -- 172.21.15.139:0/679262635 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d000cde0 con 0x7f35d4101ad0 2024-09-16T01:49:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.532+0000 7f35dc18d640 1 -- 172.21.15.139:0/679262635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d41a6b20 con 0x7f35d4101ad0 2024-09-16T01:49:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.532+0000 7f35caffd640 1 -- 172.21.15.139:0/679262635 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d00035f0 con 0x7f35d4101ad0 2024-09-16T01:49:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.532+0000 7f35caffd640 1 -- 172.21.15.139:0/679262635 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0010670 con 0x7f35d4101ad0 2024-09-16T01:49:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.532+0000 7f35dc18d640 1 -- 172.21.15.139:0/679262635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d41a7040 con 0x7f35d4101ad0 2024-09-16T01:49:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.535+0000 7f35caffd640 1 -- 172.21.15.139:0/679262635 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f35d00037a0 con 0x7f35d4101ad0 2024-09-16T01:49:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.535+0000 7f35df005740 1 Processor -- start 2024-09-16T01:49:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.535+0000 7f35df005740 1 -- start start 2024-09-16T01:49:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35caffd640 1 --2- 172.21.15.139:0/679262635 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f35c00a3460 0x7f35c00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35df005740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93377af0 0x55cd93377ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35df005740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93378410 0x55cd93382770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35df005740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd933850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35df005740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cd932dc2f0 con 0x55cd93382cb0 2024-09-16T01:49:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35df005740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cd93284270 con 0x55cd93377af0 2024-09-16T01:49:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35df005740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55cd932d82b0 con 0x55cd93378410 2024-09-16T01:49:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35caffd640 1 -- 172.21.15.139:0/679262635 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7f35d00d0400 con 0x7f35d4101ad0 2024-09-16T01:49:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35da72a640 1 --2- 172.21.15.139:0/679262635 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f35c00a3460 0x7f35c00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35d9f29640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93377af0 0x55cd93377ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35d9f29640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93377af0 0x55cd93377ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:52130/0 (socket says 172.21.15.139:52130) 2024-09-16T01:49:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35d9f29640 1 -- 172.21.15.139:0/2805260678 learned_addr learned my addr 172.21.15.139:0/2805260678 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35d9728640 1 --2- 172.21.15.139:0/2805260678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93378410 0x55cd93382770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35da72a640 1 --2- 172.21.15.139:0/2805260678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd933850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35d9f29640 1 -- 172.21.15.139:0/2805260678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93378410 msgr2=0x55cd93382770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35d9f29640 1 --2- 172.21.15.139:0/2805260678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93378410 0x55cd93382770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35d9f29640 1 -- 172.21.15.139:0/2805260678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 msgr2=0x55cd933850a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35d9f29640 1 --2- 172.21.15.139:0/2805260678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd933850a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.536+0000 7f35d9f29640 1 -- 172.21.15.139:0/2805260678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd93283340 con 0x55cd93377af0 2024-09-16T01:49:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.537+0000 7f35d9f29640 1 --2- 172.21.15.139:0/2805260678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93377af0 0x55cd93377ed0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f35d001f000 tx=0x7f35d00985d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b718b04e8f60bd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.537+0000 7f35da72a640 1 --2- 172.21.15.139:0/679262635 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f35c00a3460 0x7f35c00a5920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55cd933753e0 tx=0x7f35cc009bf0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.537+0000 7f35bffff640 1 -- 172.21.15.139:0/2805260678 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d009c970 con 0x55cd93377af0 2024-09-16T01:49:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.537+0000 7f35bffff640 1 -- 172.21.15.139:0/2805260678 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d009c270 con 0x55cd93377af0 2024-09-16T01:49:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.537+0000 7f35bffff640 1 -- 172.21.15.139:0/2805260678 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d00cd960 con 0x55cd93377af0 2024-09-16T01:49:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.537+0000 7f35df005740 1 -- 172.21.15.139:0/2805260678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93377af0 msgr2=0x55cd93377ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.537+0000 7f35df005740 1 --2- 172.21.15.139:0/2805260678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93377af0 0x55cd93377ed0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f35d001f000 tx=0x7f35d00985d0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.538+0000 7f35df005740 1 -- 172.21.15.139:0/2805260678 shutdown_connections 2024-09-16T01:49:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.538+0000 7f35df005740 1 --2- 172.21.15.139:0/2805260678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd933850a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.538+0000 7f35df005740 1 --2- 172.21.15.139:0/2805260678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93378410 0x55cd93382770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.538+0000 7f35df005740 1 --2- 172.21.15.139:0/2805260678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93377af0 0x55cd93377ed0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.538+0000 7f35df005740 1 -- 172.21.15.139:0/2805260678 >> 172.21.15.139:0/2805260678 conn(0x55cd93374060 msgr2=0x55cd93374440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.538+0000 7f35df005740 1 -- 172.21.15.139:0/2805260678 shutdown_connections 2024-09-16T01:49:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.538+0000 7f35df005740 1 -- 172.21.15.139:0/2805260678 wait complete. 2024-09-16T01:49:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.538+0000 7f35df005740 1 -- start start 2024-09-16T01:49:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.538+0000 7f35df005740 1 Processor -- start 2024-09-16T01:49:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35df005740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93377af0 0x55cd9337fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35df005740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93378410 0x55cd93380140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35df005740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93380680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35df005740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cd93385880 con 0x55cd93382cb0 2024-09-16T01:49:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35df005740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cd93265290 con 0x55cd93377af0 2024-09-16T01:49:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35df005740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55cd93385a00 con 0x55cd93378410 2024-09-16T01:49:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35df005740 10 client.0 _ll_register_callbacks cb 0x55cd93296140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35d9f29640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93377af0 0x55cd9337fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35da72a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93380680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35d9f29640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93377af0 0x55cd9337fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:52146/0 (socket says 172.21.15.139:52146) 2024-09-16T01:49:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35d9f29640 1 -- 172.21.15.139:0/2950607089 learned_addr learned my addr 172.21.15.139:0/2950607089 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35d9728640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93378410 0x55cd93380140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35d9f29640 1 -- 172.21.15.139:0/2950607089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93378410 msgr2=0x55cd93380140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35d9f29640 1 --2- 172.21.15.139:0/2950607089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93378410 0x55cd93380140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35d9f29640 1 -- 172.21.15.139:0/2950607089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 msgr2=0x55cd93380680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35d9f29640 1 --2- 172.21.15.139:0/2950607089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93380680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35d9f29640 1 -- 172.21.15.139:0/2950607089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd92fef210 con 0x55cd93377af0 2024-09-16T01:49:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35d9f29640 1 --2- 172.21.15.139:0/2950607089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93377af0 0x55cd9337fc00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35d000d010 tx=0x7f35d0003d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fb9be9378385c65 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.539+0000 7f35bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:49:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35bd7fa640 1 -- 172.21.15.139:0/2950607089 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d00a4ee0 con 0x55cd93377af0 2024-09-16T01:49:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35bd7fa640 1 -- 172.21.15.139:0/2950607089 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7f35d00cd960 con 0x55cd93377af0 2024-09-16T01:49:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35df005740 1 -- 172.21.15.139:0/2950607089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d009c270 con 0x55cd93377af0 2024-09-16T01:49:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35bd7fa640 1 -- 172.21.15.139:0/2950607089 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f35d00cdb60 con 0x55cd93377af0 2024-09-16T01:49:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35df005740 10 client.25997 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35df005740 1 -- 172.21.15.139:0/2950607089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55cd930bb620 con 0x55cd93377af0 2024-09-16T01:49:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35bd7fa640 1 -- 172.21.15.139:0/2950607089 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d009c270 con 0x55cd93377af0 2024-09-16T01:49:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35bd7fa640 1 -- 172.21.15.139:0/2950607089 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7f35d00a0160 con 0x55cd93377af0 2024-09-16T01:49:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35df005740 10 client.25997 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35df005740 10 client.25997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:49:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35df005740 20 client.25997 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35df005740 1 --2- 172.21.15.139:0/2950607089 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cd93392b00 0x55cd93394ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35df005740 4 client.25997 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:49:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35df005740 1 -- 172.21.15.139:0/2950607089 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd932e9a20 con 0x55cd93392b00 2024-09-16T01:49:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.540+0000 7f35d9728640 1 --2- 172.21.15.139:0/2950607089 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cd93392b00 0x55cd93394ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.541+0000 7f35bd7fa640 10 client.25997 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.541+0000 7f35d9728640 1 --2- 172.21.15.139:0/2950607089 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cd93392b00 0x55cd93394ef0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8708d19c0bd1c126 server_cookie=1fc61e1810ab9ed4 in_seq=0 out_seq=0 2024-09-16T01:49:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35bd7fa640 1 -- 172.21.15.139:0/2950607089 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd932e9a20 con 0x55cd93392b00 2024-09-16T01:49:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35bd7fa640 10 client.25997 handle_command_reply: tid=0 2024-09-16T01:49:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35df005740 1 client.25997 shutdown 2024-09-16T01:49:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35df005740 1 -- 172.21.15.139:0/2950607089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93377af0 msgr2=0x55cd9337fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35df005740 1 --2- 172.21.15.139:0/2950607089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93377af0 0x55cd9337fc00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35d000d010 tx=0x7f35d0003d60 comp rx=0 tx=0).stop 2024-09-16T01:49:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35df005740 1 -- 172.21.15.139:0/2950607089 shutdown_connections 2024-09-16T01:49:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35df005740 1 --2- 172.21.15.139:0/2950607089 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cd93392b00 0x55cd93394ef0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35df005740 1 --2- 172.21.15.139:0/2950607089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93380680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35df005740 1 --2- 172.21.15.139:0/2950607089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93378410 0x55cd93380140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35df005740 1 --2- 172.21.15.139:0/2950607089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93377af0 0x55cd9337fc00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35bd7fa640 0 client.25997 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35df005740 1 -- 172.21.15.139:0/2950607089 >> 172.21.15.139:0/2950607089 conn(0x55cd93374060 msgr2=0x55cd93379dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35df005740 1 -- 172.21.15.139:0/2950607089 shutdown_connections 2024-09-16T01:49:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35df005740 1 -- 172.21.15.139:0/2950607089 wait complete. 2024-09-16T01:49:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.542+0000 7f35df005740 20 client.25997 trim_cache size 0 max 16384 2024-09-16T01:49:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.558+0000 7f35df005740 1 Processor -- start 2024-09-16T01:49:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.558+0000 7f35df005740 1 -- start start 2024-09-16T01:49:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.558+0000 7f35df005740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93377af0 0x55cd93380130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.558+0000 7f35df005740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93378410 0x55cd93380670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.558+0000 7f35df005740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93397f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.558+0000 7f35df005740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f35d41b1300 con 0x55cd93382cb0 2024-09-16T01:49:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.558+0000 7f35df005740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f35d41b3140 con 0x55cd93378410 2024-09-16T01:49:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.558+0000 7f35df005740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f35d41b1a90 con 0x55cd93377af0 2024-09-16T01:49:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.558+0000 7f35da72a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93397f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.558+0000 7f35da72a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93397f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58680/0 (socket says 172.21.15.139:58680) 2024-09-16T01:49:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.558+0000 7f35da72a640 1 -- 172.21.15.139:0/3792288322 learned_addr learned my addr 172.21.15.139:0/3792288322 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.558+0000 7f35d9728640 1 --2- 172.21.15.139:0/3792288322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93378410 0x55cd93380670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.558+0000 7f35d9f29640 1 --2- 172.21.15.139:0/3792288322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93377af0 0x55cd93380130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.559+0000 7f35da72a640 1 -- 172.21.15.139:0/3792288322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93377af0 msgr2=0x55cd93380130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.559+0000 7f35da72a640 1 --2- 172.21.15.139:0/3792288322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93377af0 0x55cd93380130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.559+0000 7f35da72a640 1 -- 172.21.15.139:0/3792288322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93378410 msgr2=0x55cd93380670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.559+0000 7f35da72a640 1 --2- 172.21.15.139:0/3792288322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93378410 0x55cd93380670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.559+0000 7f35da72a640 1 -- 172.21.15.139:0/3792288322 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd930bb620 con 0x55cd93382cb0 2024-09-16T01:49:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.559+0000 7f35d9728640 1 --2- 172.21.15.139:0/3792288322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93378410 0x55cd93380670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:49:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.559+0000 7f35da72a640 1 --2- 172.21.15.139:0/3792288322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93397f10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009ba0 tx=0x7f35cc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd66999ec05bd6b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.559+0000 7f35bf7fe640 1 -- 172.21.15.139:0/3792288322 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00fcc0 con 0x55cd93382cb0 2024-09-16T01:49:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.559+0000 7f35bf7fe640 1 -- 172.21.15.139:0/3792288322 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35cc003e50 con 0x55cd93382cb0 2024-09-16T01:49:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.559+0000 7f35bf7fe640 1 -- 172.21.15.139:0/3792288322 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc004820 con 0x55cd93382cb0 2024-09-16T01:49:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.560+0000 7f35df005740 1 -- 172.21.15.139:0/3792288322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 msgr2=0x55cd93397f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.560+0000 7f35df005740 1 --2- 172.21.15.139:0/3792288322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93397f10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009ba0 tx=0x7f35cc009500 comp rx=0 tx=0).stop 2024-09-16T01:49:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.560+0000 7f35df005740 1 -- 172.21.15.139:0/3792288322 shutdown_connections 2024-09-16T01:49:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.560+0000 7f35df005740 1 --2- 172.21.15.139:0/3792288322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93397f10 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.560+0000 7f35df005740 1 --2- 172.21.15.139:0/3792288322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93378410 0x55cd93380670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.560+0000 7f35df005740 1 --2- 172.21.15.139:0/3792288322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93377af0 0x55cd93380130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.560+0000 7f35df005740 1 -- 172.21.15.139:0/3792288322 >> 172.21.15.139:0/3792288322 conn(0x55cd93374060 msgr2=0x55cd93381c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.560+0000 7f35df005740 1 -- 172.21.15.139:0/3792288322 shutdown_connections 2024-09-16T01:49:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.560+0000 7f35df005740 1 -- 172.21.15.139:0/3792288322 wait complete. 2024-09-16T01:49:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.560+0000 7f35df005740 1 -- start start 2024-09-16T01:49:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.560+0000 7f35df005740 1 Processor -- start 2024-09-16T01:49:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35df005740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93377af0 0x55cd933984f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35df005740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93378410 0x55cd93395880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35df005740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93395dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35df005740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f35d41b29b0 con 0x55cd93382cb0 2024-09-16T01:49:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35df005740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f35d41b2220 con 0x55cd93378410 2024-09-16T01:49:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35df005740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f35d41af300 con 0x55cd93377af0 2024-09-16T01:49:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35df005740 10 client.0 _ll_register_callbacks cb 0x7f35cc0117a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:49:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35da72a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93395dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35da72a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93395dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58686/0 (socket says 172.21.15.139:58686) 2024-09-16T01:49:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35da72a640 1 -- 172.21.15.139:0/669127479 learned_addr learned my addr 172.21.15.139:0/669127479 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:49:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35d9728640 1 --2- 172.21.15.139:0/669127479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93378410 0x55cd93395880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35d9f29640 1 --2- 172.21.15.139:0/669127479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93377af0 0x55cd933984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35da72a640 1 -- 172.21.15.139:0/669127479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93377af0 msgr2=0x55cd933984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35da72a640 1 --2- 172.21.15.139:0/669127479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93377af0 0x55cd933984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35da72a640 1 -- 172.21.15.139:0/669127479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93378410 msgr2=0x55cd93395880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:49:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35da72a640 1 --2- 172.21.15.139:0/669127479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93378410 0x55cd93395880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35da72a640 1 -- 172.21.15.139:0/669127479 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd93396460 con 0x55cd93382cb0 2024-09-16T01:49:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35d9f29640 1 --2- 172.21.15.139:0/669127479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93377af0 0x55cd933984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:49:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35da72a640 1 --2- 172.21.15.139:0/669127479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93395dc0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f35cc020bf0 tx=0x7f35cc009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141562d53fc22878 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:49:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:49:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35bd7fa640 1 -- 172.21.15.139:0/669127479 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc036060 con 0x55cd93382cb0 2024-09-16T01:49:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35bd7fa640 1 -- 172.21.15.139:0/669127479 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7f35cc003b70 con 0x55cd93382cb0 2024-09-16T01:49:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.561+0000 7f35df005740 1 -- 172.21.15.139:0/669127479 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35cc003e50 con 0x55cd93382cb0 2024-09-16T01:49:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.562+0000 7f35bd7fa640 1 -- 172.21.15.139:0/669127479 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f35cc02d050 con 0x55cd93382cb0 2024-09-16T01:49:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.562+0000 7f35df005740 10 client.16575 fetch_fsmap learned FSMap version 50 2024-09-16T01:49:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.562+0000 7f35df005740 1 -- 172.21.15.139:0/669127479 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55cd933fd2c0 con 0x55cd93382cb0 2024-09-16T01:49:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.562+0000 7f35bd7fa640 1 -- 172.21.15.139:0/669127479 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35cc003e50 con 0x55cd93382cb0 2024-09-16T01:49:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.562+0000 7f35bd7fa640 1 -- 172.21.15.139:0/669127479 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) ==== 8446+0+0 (secure 0 0 0) 0x7f35cc028070 con 0x55cd93382cb0 2024-09-16T01:49:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.562+0000 7f35df005740 10 client.16575 fetch_fsmap finished waiting for FSMap version 50 2024-09-16T01:49:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.562+0000 7f35df005740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:49:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.562+0000 7f35df005740 20 client.16575 populate_metadata read hostname 'smithi139' 2024-09-16T01:49:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.562+0000 7f35df005740 1 --2- 172.21.15.139:0/669127479 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cd933fda40 0x55cd933ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:49:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.562+0000 7f35df005740 4 client.16575 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:49:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.562+0000 7f35df005740 1 -- 172.21.15.139:0/669127479 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd932e3d70 con 0x55cd933fda40 2024-09-16T01:49:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.562+0000 7f35d9f29640 1 --2- 172.21.15.139:0/669127479 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cd933fda40 0x55cd933ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:49:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.562+0000 7f35d9f29640 1 --2- 172.21.15.139:0/669127479 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cd933fda40 0x55cd933ffe60 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e3bec1c87268f0e server_cookie=7097dea642dae28 in_seq=0 out_seq=0 2024-09-16T01:49:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.562+0000 7f35bd7fa640 10 client.16575 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35bd7fa640 1 -- 172.21.15.139:0/669127479 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cd932e3d70 con 0x55cd933fda40 2024-09-16T01:49:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35bd7fa640 10 client.16575 handle_command_reply: tid=0 2024-09-16T01:49:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35df005740 1 client.16575 shutdown 2024-09-16T01:49:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35df005740 1 -- 172.21.15.139:0/669127479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 msgr2=0x55cd93395dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35df005740 1 --2- 172.21.15.139:0/669127479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93395dc0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f35cc020bf0 tx=0x7f35cc009fa0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35df005740 1 -- 172.21.15.139:0/669127479 shutdown_connections 2024-09-16T01:49:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35df005740 1 --2- 172.21.15.139:0/669127479 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cd933fda40 0x55cd933ffe60 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35df005740 1 --2- 172.21.15.139:0/669127479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cd93382cb0 0x55cd93395dc0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35df005740 1 --2- 172.21.15.139:0/669127479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd93378410 0x55cd93395880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35df005740 1 --2- 172.21.15.139:0/669127479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cd93377af0 0x55cd933984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35bd7fa640 0 client.16575 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:49:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35df005740 1 -- 172.21.15.139:0/669127479 >> 172.21.15.139:0/669127479 conn(0x55cd93374060 msgr2=0x55cd93381c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35df005740 1 -- 172.21.15.139:0/669127479 shutdown_connections 2024-09-16T01:49:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35df005740 1 -- 172.21.15.139:0/669127479 wait complete. 2024-09-16T01:49:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.576+0000 7f35df005740 20 client.16575 trim_cache size 0 max 16384 2024-09-16T01:49:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:49:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:49:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:49:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:49:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.577+0000 7f35dc18d640 1 -- 172.21.15.139:0/679262635 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f35c00a3460 msgr2=0x7f35c00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.577+0000 7f35dc18d640 1 --2- 172.21.15.139:0/679262635 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f35c00a3460 0x7f35c00a5920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55cd933753e0 tx=0x7f35cc009bf0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.577+0000 7f35dc18d640 1 -- 172.21.15.139:0/679262635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35d4101ad0 msgr2=0x7f35d40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:49:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.577+0000 7f35dc18d640 1 --2- 172.21.15.139:0/679262635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35d4101ad0 0x7f35d40744f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f35d000ea60 tx=0x7f35d000ef20 comp rx=0 tx=0).stop 2024-09-16T01:49:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.577+0000 7f35dc18d640 1 -- 172.21.15.139:0/679262635 shutdown_connections 2024-09-16T01:49:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.577+0000 7f35dc18d640 1 --2- 172.21.15.139:0/679262635 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f35c00a3460 0x7f35c00a5920 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.577+0000 7f35dc18d640 1 --2- 172.21.15.139:0/679262635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35d4107b70 0x7f35d4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.577+0000 7f35dc18d640 1 --2- 172.21.15.139:0/679262635 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35d4102470 0x7f35d4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.577+0000 7f35dc18d640 1 --2- 172.21.15.139:0/679262635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35d4101ad0 0x7f35d40744f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:49:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.577+0000 7f35dc18d640 1 -- 172.21.15.139:0/679262635 >> 172.21.15.139:0/679262635 conn(0x7f35d40fd7d0 msgr2=0x7f35d40fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:49:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.577+0000 7f35dc18d640 1 -- 172.21.15.139:0/679262635 shutdown_connections 2024-09-16T01:49:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:49:54.577+0000 7f35dc18d640 1 -- 172.21.15.139:0/679262635 wait complete. 2024-09-16T01:49:55.232 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c50bf1fe', quiesce took 11.6 seconds. Will release after: 10.750000000000002 2024-09-16T01:49:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:55 smithi164 ceph-mon[35121]: pgmap v295: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 893 KiB/s wr, 152 op/s 2024-09-16T01:49:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:55 smithi139 ceph-mon[31175]: pgmap v295: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 893 KiB/s wr, 152 op/s 2024-09-16T01:49:55.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:55 smithi176 ceph-mon[35324]: pgmap v295: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 893 KiB/s wr, 152 op/s 2024-09-16T01:49:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:57 smithi164 ceph-mon[35121]: pgmap v296: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 960 KiB/s wr, 158 op/s 2024-09-16T01:49:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:57 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:49:57.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:57 smithi139 ceph-mon[31175]: pgmap v296: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 960 KiB/s wr, 158 op/s 2024-09-16T01:49:57.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:57 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:49:57.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:57 smithi176 ceph-mon[35324]: pgmap v296: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 960 KiB/s wr, 158 op/s 2024-09-16T01:49:57.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:57 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:49:59.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:49:59 smithi164 ceph-mon[35121]: pgmap v297: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 553 KiB/s wr, 135 op/s 2024-09-16T01:49:59.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:49:59 smithi139 ceph-mon[31175]: pgmap v297: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 553 KiB/s wr, 135 op/s 2024-09-16T01:49:59.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:49:59 smithi176 ceph-mon[35324]: pgmap v297: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 553 KiB/s wr, 135 op/s 2024-09-16T01:50:00.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:00 smithi164 ceph-mon[35121]: overall HEALTH_OK 2024-09-16T01:50:00.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:00 smithi139 ceph-mon[31175]: overall HEALTH_OK 2024-09-16T01:50:00.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:00 smithi176 ceph-mon[35324]: overall HEALTH_OK 2024-09-16T01:50:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:01 smithi164 ceph-mon[35121]: pgmap v298: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 384 KiB/s wr, 86 op/s 2024-09-16T01:50:01.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:01 smithi139 ceph-mon[31175]: pgmap v298: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 384 KiB/s wr, 86 op/s 2024-09-16T01:50:01.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:01 smithi176 ceph-mon[35324]: pgmap v298: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 384 KiB/s wr, 86 op/s 2024-09-16T01:50:03.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:03 smithi164 ceph-mon[35121]: pgmap v299: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 392 KiB/s wr, 84 op/s 2024-09-16T01:50:03.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:03 smithi139 ceph-mon[31175]: pgmap v299: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 392 KiB/s wr, 84 op/s 2024-09-16T01:50:03.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:03 smithi176 ceph-mon[35324]: pgmap v299: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 392 KiB/s wr, 84 op/s 2024-09-16T01:50:05.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:05 smithi164 ceph-mon[35121]: pgmap v300: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 152 KiB/s wr, 54 op/s 2024-09-16T01:50:05.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:05 smithi139 ceph-mon[31175]: pgmap v300: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 152 KiB/s wr, 54 op/s 2024-09-16T01:50:05.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:05 smithi176 ceph-mon[35324]: pgmap v300: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 152 KiB/s wr, 54 op/s 2024-09-16T01:50:05.984 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:50:06.242 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:50:06.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.737+0000 7f56a4582640 1 -- 172.21.15.139:0/1737255746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f569c10c170 msgr2=0x7f569c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:06.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.737+0000 7f56a4582640 1 --2- 172.21.15.139:0/1737255746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f569c10c170 0x7f569c10e610 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5694009900 tx=0x7f569402a480 comp rx=0 tx=0).stop 2024-09-16T01:50:06.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.738+0000 7f56a4582640 1 -- 172.21.15.139:0/1737255746 shutdown_connections 2024-09-16T01:50:06.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.738+0000 7f56a4582640 1 --2- 172.21.15.139:0/1737255746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f569c10c170 0x7f569c10e610 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:06.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.738+0000 7f56a4582640 1 --2- 172.21.15.139:0/1737255746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f569c103660 0x7f569c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:06.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.738+0000 7f56a4582640 1 --2- 172.21.15.139:0/1737255746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f569c102d40 0x7f569c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:06.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.738+0000 7f56a4582640 1 -- 172.21.15.139:0/1737255746 >> 172.21.15.139:0/1737255746 conn(0x7f569c0fc7a0 msgr2=0x7f569c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:06.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.738+0000 7f56a4582640 1 -- 172.21.15.139:0/1737255746 shutdown_connections 2024-09-16T01:50:06.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.738+0000 7f56a4582640 1 -- 172.21.15.139:0/1737255746 wait complete. 2024-09-16T01:50:06.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.738+0000 7f56a4582640 1 Processor -- start 2024-09-16T01:50:06.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.738+0000 7f56a4582640 1 -- start start 2024-09-16T01:50:06.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a4582640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f569c102d40 0x7f569c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:06.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a4582640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f569c103660 0x7f569c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:06.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a4582640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f569c10c170 0x7f569c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:06.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a4582640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f569c111fb0 con 0x7f569c102d40 2024-09-16T01:50:06.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a4582640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f569c111e30 con 0x7f569c10c170 2024-09-16T01:50:06.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a4582640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f569c112130 con 0x7f569c103660 2024-09-16T01:50:06.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a231e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f569c102d40 0x7f569c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:06.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a231e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f569c102d40 0x7f569c198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45272/0 (socket says 172.21.15.139:45272) 2024-09-16T01:50:06.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a231e640 1 -- 172.21.15.139:0/767559998 learned_addr learned my addr 172.21.15.139:0/767559998 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:06.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a2b1f640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f569c10c170 0x7f569c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:06.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a1b1d640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f569c103660 0x7f569c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:06.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a231e640 1 -- 172.21.15.139:0/767559998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f569c103660 msgr2=0x7f569c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:06.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a231e640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f569c103660 0x7f569c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:06.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a231e640 1 -- 172.21.15.139:0/767559998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f569c10c170 msgr2=0x7f569c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:06.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a231e640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f569c10c170 0x7f569c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:06.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a231e640 1 -- 172.21.15.139:0/767559998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569c19d4c0 con 0x7f569c102d40 2024-09-16T01:50:06.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a1b1d640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f569c103660 0x7f569c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:50:06.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a2b1f640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f569c10c170 0x7f569c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:50:06.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a231e640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f569c102d40 0x7f569c198510 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f568c00eab0 tx=0x7f568c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:06.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f569b7fe640 1 -- 172.21.15.139:0/767559998 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c00ce40 con 0x7f569c102d40 2024-09-16T01:50:06.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f569b7fe640 1 -- 172.21.15.139:0/767559998 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f568c011950 con 0x7f569c102d40 2024-09-16T01:50:06.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f56a4582640 1 -- 172.21.15.139:0/767559998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569c19d7b0 con 0x7f569c102d40 2024-09-16T01:50:06.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.739+0000 7f569b7fe640 1 -- 172.21.15.139:0/767559998 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c010ae0 con 0x7f569c102d40 2024-09-16T01:50:06.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.740+0000 7f56a4582640 1 -- 172.21.15.139:0/767559998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f569c1a5090 con 0x7f569c102d40 2024-09-16T01:50:06.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.741+0000 7f56a4582640 1 -- 172.21.15.139:0/767559998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569c103120 con 0x7f569c102d40 2024-09-16T01:50:06.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.742+0000 7f569b7fe640 1 -- 172.21.15.139:0/767559998 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f568c011b00 con 0x7f569c102d40 2024-09-16T01:50:06.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.742+0000 7f569b7fe640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f56840a3430 0x7f56840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:06.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.742+0000 7f56a1b1d640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f56840a3430 0x7f56840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:06.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.742+0000 7f569b7fe640 1 -- 172.21.15.139:0/767559998 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7f568c0d0890 con 0x7f569c102d40 2024-09-16T01:50:06.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.743+0000 7f56a1b1d640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f56840a3430 0x7f56840a58f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5690006fd0 tx=0x7f5690008040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:06.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.746+0000 7f569b7fe640 1 -- 172.21.15.139:0/767559998 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f568c016030 con 0x7f569c102d40 2024-09-16T01:50:06.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.947+0000 7f56a4582640 1 -- 172.21.15.139:0/767559998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f569c107230 con 0x7f569c102d40 2024-09-16T01:50:06.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.948+0000 7f569b7fe640 1 -- 172.21.15.139:0/767559998 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+12026 (secure 0 0 0) 0x7f569c107230 con 0x7f569c102d40 2024-09-16T01:50:06.950 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:50:06.950 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":51,"btime":"2024-09-16T01:49:56:260165+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:49:55.565438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:50:06.951 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 51 2024-09-16T01:50:06.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.950+0000 7f56a4582640 1 -- 172.21.15.139:0/767559998 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f56840a3430 msgr2=0x7f56840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:06.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.950+0000 7f56a4582640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f56840a3430 0x7f56840a58f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5690006fd0 tx=0x7f5690008040 comp rx=0 tx=0).stop 2024-09-16T01:50:06.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.950+0000 7f56a4582640 1 -- 172.21.15.139:0/767559998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f569c102d40 msgr2=0x7f569c198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.950+0000 7f56a4582640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f569c102d40 0x7f569c198510 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f568c00eab0 tx=0x7f568c00ef70 comp rx=0 tx=0).stop 2024-09-16T01:50:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.950+0000 7f56a4582640 1 -- 172.21.15.139:0/767559998 shutdown_connections 2024-09-16T01:50:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.950+0000 7f56a4582640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f56840a3430 0x7f56840a58f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.950+0000 7f56a4582640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f569c10c170 0x7f569c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.950+0000 7f56a4582640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f569c103660 0x7f569c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.950+0000 7f56a4582640 1 --2- 172.21.15.139:0/767559998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f569c102d40 0x7f569c198510 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.950+0000 7f56a4582640 1 -- 172.21.15.139:0/767559998 >> 172.21.15.139:0/767559998 conn(0x7f569c0fc7a0 msgr2=0x7f569c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.951+0000 7f56a4582640 1 -- 172.21.15.139:0/767559998 shutdown_connections 2024-09-16T01:50:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:06.951+0000 7f56a4582640 1 -- 172.21.15.139:0/767559998 wait complete. 2024-09-16T01:50:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:07 smithi164 ceph-mon[35121]: pgmap v301: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 170 KiB/s wr, 49 op/s 2024-09-16T01:50:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:07 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/767559998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:50:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.668+0000 7f90316f1640 1 -- 172.21.15.139:0/1431683851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902c1035c0 msgr2=0x7f902c10baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.668+0000 7f90316f1640 1 --2- 172.21.15.139:0/1431683851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902c1035c0 0x7f902c10baa0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9020009900 tx=0x7f902002a410 comp rx=0 tx=0).stop 2024-09-16T01:50:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.668+0000 7f90316f1640 1 -- 172.21.15.139:0/1431683851 shutdown_connections 2024-09-16T01:50:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.668+0000 7f90316f1640 1 --2- 172.21.15.139:0/1431683851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f902c10c0d0 0x7f902c10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.668+0000 7f90316f1640 1 --2- 172.21.15.139:0/1431683851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902c1035c0 0x7f902c10baa0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.668+0000 7f90316f1640 1 --2- 172.21.15.139:0/1431683851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f902c102ca0 0x7f902c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.668+0000 7f90316f1640 1 -- 172.21.15.139:0/1431683851 >> 172.21.15.139:0/1431683851 conn(0x7f902c0fc6b0 msgr2=0x7f902c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.668+0000 7f90316f1640 1 -- 172.21.15.139:0/1431683851 shutdown_connections 2024-09-16T01:50:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.668+0000 7f90316f1640 1 -- 172.21.15.139:0/1431683851 wait complete. 2024-09-16T01:50:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.669+0000 7f90316f1640 1 Processor -- start 2024-09-16T01:50:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.669+0000 7f90316f1640 1 -- start start 2024-09-16T01:50:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.669+0000 7f90316f1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f902c102ca0 0x7f902c1008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.669+0000 7f90316f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f902c1035c0 0x7f902c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.669+0000 7f90316f1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902c10c0d0 0x7f902c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.669+0000 7f90316f1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f902c111ed0 con 0x7f902c10c0d0 2024-09-16T01:50:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.669+0000 7f90316f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f902c111d50 con 0x7f902c1035c0 2024-09-16T01:50:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.669+0000 7f90316f1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f902c112050 con 0x7f902c102ca0 2024-09-16T01:50:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.669+0000 7f902b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902c10c0d0 0x7f902c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.669+0000 7f902b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902c10c0d0 0x7f902c101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45292/0 (socket says 172.21.15.139:45292) 2024-09-16T01:50:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.669+0000 7f902b7fe640 1 -- 172.21.15.139:0/2758326981 learned_addr learned my addr 172.21.15.139:0/2758326981 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.669+0000 7f902affd640 1 --2- 172.21.15.139:0/2758326981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f902c102ca0 0x7f902c1008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.670+0000 7f902a7fc640 1 --2- 172.21.15.139:0/2758326981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f902c1035c0 0x7f902c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.670+0000 7f902b7fe640 1 -- 172.21.15.139:0/2758326981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f902c102ca0 msgr2=0x7f902c1008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.670+0000 7f902b7fe640 1 --2- 172.21.15.139:0/2758326981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f902c102ca0 0x7f902c1008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.670+0000 7f902b7fe640 1 -- 172.21.15.139:0/2758326981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f902c1035c0 msgr2=0x7f902c100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.670+0000 7f902b7fe640 1 --2- 172.21.15.139:0/2758326981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f902c1035c0 0x7f902c100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.670+0000 7f902b7fe640 1 -- 172.21.15.139:0/2758326981 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902c1a6880 con 0x7f902c10c0d0 2024-09-16T01:50:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.670+0000 7f902b7fe640 1 --2- 172.21.15.139:0/2758326981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902c10c0d0 0x7f902c101370 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f902400eb10 tx=0x7f902400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.670+0000 7f9013fff640 1 -- 172.21.15.139:0/2758326981 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9024004b20 con 0x7f902c10c0d0 2024-09-16T01:50:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.670+0000 7f9013fff640 1 -- 172.21.15.139:0/2758326981 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9024004cc0 con 0x7f902c10c0d0 2024-09-16T01:50:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.670+0000 7f9013fff640 1 -- 172.21.15.139:0/2758326981 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9024010760 con 0x7f902c10c0d0 2024-09-16T01:50:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.670+0000 7f90316f1640 1 -- 172.21.15.139:0/2758326981 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f902c1a6b70 con 0x7f902c10c0d0 2024-09-16T01:50:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.670+0000 7f90316f1640 1 -- 172.21.15.139:0/2758326981 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f902c1a7090 con 0x7f902c10c0d0 2024-09-16T01:50:07.677 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:07 smithi139 ceph-mon[31175]: pgmap v301: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 170 KiB/s wr, 49 op/s 2024-09-16T01:50:07.677 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:07 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/767559998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:50:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.672+0000 7f9013fff640 1 -- 172.21.15.139:0/2758326981 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9024010900 con 0x7f902c10c0d0 2024-09-16T01:50:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f9034490740 1 Processor -- start 2024-09-16T01:50:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f9034490740 1 -- start start 2024-09-16T01:50:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f9034490740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257a9af0 0x556d257a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f9034490740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257aa410 0x556d257b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f9034490740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257b4cb0 0x556d257b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f9034490740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556d2570e2f0 con 0x556d257aa410 2024-09-16T01:50:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f9034490740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556d256b6270 con 0x556d257b4cb0 2024-09-16T01:50:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f9034490740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556d2570a2b0 con 0x556d257a9af0 2024-09-16T01:50:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f9013fff640 1 --2- 172.21.15.139:0/2758326981 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90140a3460 0x7f90140a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f902affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257aa410 0x556d257b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f902affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257aa410 0x556d257b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45298/0 (socket says 172.21.15.139:45298) 2024-09-16T01:50:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f902affd640 1 -- 172.21.15.139:0/1512062788 learned_addr learned my addr 172.21.15.139:0/1512062788 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f902a7fc640 1 --2- 172.21.15.139:0/1512062788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257b4cb0 0x556d257b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f902a7fc640 1 --2- 172.21.15.139:0/1512062788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257a9af0 0x556d257a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f9013fff640 1 -- 172.21.15.139:0/2758326981 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7f90240d08d0 con 0x7f902c10c0d0 2024-09-16T01:50:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f902b7fe640 1 --2- 172.21.15.139:0/2758326981 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90140a3460 0x7f90140a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f902affd640 1 -- 172.21.15.139:0/1512062788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257a9af0 msgr2=0x556d257a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f902affd640 1 --2- 172.21.15.139:0/1512062788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257a9af0 0x556d257a9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f902affd640 1 -- 172.21.15.139:0/1512062788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257b4cb0 msgr2=0x556d257b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f902affd640 1 --2- 172.21.15.139:0/1512062788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257b4cb0 0x556d257b70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f902affd640 1 -- 172.21.15.139:0/1512062788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d256b5340 con 0x556d257aa410 2024-09-16T01:50:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.673+0000 7f902a7fc640 1 --2- 172.21.15.139:0/1512062788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257a9af0 0x556d257a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:50:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.674+0000 7f902affd640 1 --2- 172.21.15.139:0/1512062788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257aa410 0x556d257b4770 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f901c00bd10 tx=0x7f901c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260a11a6a5f1f7f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.674+0000 7f902b7fe640 1 --2- 172.21.15.139:0/2758326981 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90140a3460 0x7f90140a5920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f902409c440 tx=0x7f902400efd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.674+0000 7f90117fa640 1 -- 172.21.15.139:0/1512062788 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c004030 con 0x556d257aa410 2024-09-16T01:50:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.674+0000 7f90117fa640 1 -- 172.21.15.139:0/1512062788 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f901c0041d0 con 0x556d257aa410 2024-09-16T01:50:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.674+0000 7f90117fa640 1 -- 172.21.15.139:0/1512062788 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c011840 con 0x556d257aa410 2024-09-16T01:50:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 -- 172.21.15.139:0/1512062788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257aa410 msgr2=0x556d257b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 --2- 172.21.15.139:0/1512062788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257aa410 0x556d257b4770 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f901c00bd10 tx=0x7f901c0107a0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 -- 172.21.15.139:0/1512062788 shutdown_connections 2024-09-16T01:50:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 --2- 172.21.15.139:0/1512062788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257b4cb0 0x556d257b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 --2- 172.21.15.139:0/1512062788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257aa410 0x556d257b4770 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 --2- 172.21.15.139:0/1512062788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257a9af0 0x556d257a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 -- 172.21.15.139:0/1512062788 >> 172.21.15.139:0/1512062788 conn(0x556d257a6060 msgr2=0x556d257a6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 -- 172.21.15.139:0/1512062788 shutdown_connections 2024-09-16T01:50:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 -- 172.21.15.139:0/1512062788 wait complete. 2024-09-16T01:50:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 -- start start 2024-09-16T01:50:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 Processor -- start 2024-09-16T01:50:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257a9af0 0x556d257b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257aa410 0x556d257b2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257b4cb0 0x556d257b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556d257b7880 con 0x556d257aa410 2024-09-16T01:50:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556d25697290 con 0x556d257a9af0 2024-09-16T01:50:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556d257b7a00 con 0x556d257b4cb0 2024-09-16T01:50:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f902affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257aa410 0x556d257b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f902affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257aa410 0x556d257b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45302/0 (socket says 172.21.15.139:45302) 2024-09-16T01:50:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f902affd640 1 -- 172.21.15.139:0/1705535129 learned_addr learned my addr 172.21.15.139:0/1705535129 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f9034490740 10 client.0 _ll_register_callbacks cb 0x556d256c8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:50:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.675+0000 7f902a7fc640 1 --2- 172.21.15.139:0/1705535129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257a9af0 0x556d257b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f902a7fc640 1 --2- 172.21.15.139:0/1705535129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257b4cb0 0x556d257b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f902affd640 1 -- 172.21.15.139:0/1705535129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257b4cb0 msgr2=0x556d257b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f902affd640 1 --2- 172.21.15.139:0/1705535129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257b4cb0 0x556d257b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f902affd640 1 -- 172.21.15.139:0/1705535129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257a9af0 msgr2=0x556d257b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f902affd640 1 --2- 172.21.15.139:0/1705535129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257a9af0 0x556d257b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f902affd640 1 -- 172.21.15.139:0/1705535129 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d25421210 con 0x556d257aa410 2024-09-16T01:50:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f902affd640 1 --2- 172.21.15.139:0/1705535129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257aa410 0x556d257b2150 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f901c003d90 tx=0x7f901c003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8aa07f2433ae2be server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f900eff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:50:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f900eff5640 1 -- 172.21.15.139:0/1705535129 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c029060 con 0x556d257aa410 2024-09-16T01:50:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f900eff5640 1 -- 172.21.15.139:0/1705535129 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7f901c0035f0 con 0x556d257aa410 2024-09-16T01:50:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f9034490740 1 -- 172.21.15.139:0/1705535129 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f901c0041d0 con 0x556d257aa410 2024-09-16T01:50:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f900eff5640 1 -- 172.21.15.139:0/1705535129 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f901c030050 con 0x556d257aa410 2024-09-16T01:50:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f9034490740 10 client.16593 fetch_fsmap learned FSMap version 51 2024-09-16T01:50:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f9034490740 1 -- 172.21.15.139:0/1705535129 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x556d254ed620 con 0x556d257aa410 2024-09-16T01:50:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.676+0000 7f900eff5640 1 -- 172.21.15.139:0/1705535129 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f901c0041d0 con 0x556d257aa410 2024-09-16T01:50:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.677+0000 7f900eff5640 1 -- 172.21.15.139:0/1705535129 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 51) ==== 8442+0+0 (secure 0 0 0) 0x7f901c004750 con 0x556d257aa410 2024-09-16T01:50:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.677+0000 7f9034490740 10 client.16593 fetch_fsmap finished waiting for FSMap version 51 2024-09-16T01:50:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.677+0000 7f9034490740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:50:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.677+0000 7f9034490740 20 client.16593 populate_metadata read hostname 'smithi139' 2024-09-16T01:50:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.677+0000 7f9034490740 1 --2- 172.21.15.139:0/1705535129 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556d257c4bd0 0x556d257c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.677+0000 7f9034490740 4 client.16593 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:50:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.677+0000 7f9034490740 1 -- 172.21.15.139:0/1705535129 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d2571ba20 con 0x556d257c4bd0 2024-09-16T01:50:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.677+0000 7f902a7fc640 1 --2- 172.21.15.139:0/1705535129 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556d257c4bd0 0x556d257c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.677+0000 7f900eff5640 10 client.16593 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.677+0000 7f902a7fc640 1 --2- 172.21.15.139:0/1705535129 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556d257c4bd0 0x556d257c6fc0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=233b3718fc505faa server_cookie=a58fb4eee5230f74 in_seq=0 out_seq=0 2024-09-16T01:50:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f900eff5640 1 -- 172.21.15.139:0/1705535129 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d2571ba20 con 0x556d257c4bd0 2024-09-16T01:50:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f900eff5640 10 client.16593 handle_command_reply: tid=0 2024-09-16T01:50:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f9034490740 1 client.16593 shutdown 2024-09-16T01:50:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f9034490740 1 -- 172.21.15.139:0/1705535129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257aa410 msgr2=0x556d257b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f9034490740 1 --2- 172.21.15.139:0/1705535129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257aa410 0x556d257b2150 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f901c003d90 tx=0x7f901c003dc0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f9034490740 1 -- 172.21.15.139:0/1705535129 shutdown_connections 2024-09-16T01:50:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f9034490740 1 --2- 172.21.15.139:0/1705535129 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556d257c4bd0 0x556d257c6fc0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f9034490740 1 --2- 172.21.15.139:0/1705535129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257b4cb0 0x556d257b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f900eff5640 0 client.16593 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f9034490740 1 --2- 172.21.15.139:0/1705535129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257aa410 0x556d257b2150 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f9034490740 1 --2- 172.21.15.139:0/1705535129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257a9af0 0x556d257b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f9034490740 1 -- 172.21.15.139:0/1705535129 >> 172.21.15.139:0/1705535129 conn(0x556d257a6060 msgr2=0x556d257a51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f9034490740 1 -- 172.21.15.139:0/1705535129 shutdown_connections 2024-09-16T01:50:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f9034490740 1 -- 172.21.15.139:0/1705535129 wait complete. 2024-09-16T01:50:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.678+0000 7f9034490740 20 client.16593 trim_cache size 0 max 16384 2024-09-16T01:50:07.692 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:07 smithi176 ceph-mon[35324]: pgmap v301: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 170 KiB/s wr, 49 op/s 2024-09-16T01:50:07.693 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:07 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/767559998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:50:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.705+0000 7f9034490740 1 Processor -- start 2024-09-16T01:50:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.705+0000 7f9034490740 1 -- start start 2024-09-16T01:50:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.705+0000 7f9034490740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257a9af0 0x556d257b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.705+0000 7f9034490740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257aa410 0x556d257cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f9034490740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257b4cb0 0x556d257ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f9034490740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f902c1b1430 con 0x556d257a9af0 2024-09-16T01:50:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f9034490740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f902c1b3240 con 0x556d257b4cb0 2024-09-16T01:50:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f9034490740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f902c1b1bc0 con 0x556d257aa410 2024-09-16T01:50:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f902a7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257a9af0 0x556d257b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f902a7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257a9af0 0x556d257b24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45316/0 (socket says 172.21.15.139:45316) 2024-09-16T01:50:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f902a7fc640 1 -- 172.21.15.139:0/706334704 learned_addr learned my addr 172.21.15.139:0/706334704 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f902a7fc640 1 --2- 172.21.15.139:0/706334704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257b4cb0 0x556d257ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f902affd640 1 --2- 172.21.15.139:0/706334704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257aa410 0x556d257cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f902a7fc640 1 -- 172.21.15.139:0/706334704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257aa410 msgr2=0x556d257cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f902a7fc640 1 --2- 172.21.15.139:0/706334704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257aa410 0x556d257cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f902a7fc640 1 -- 172.21.15.139:0/706334704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257b4cb0 msgr2=0x556d257ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f902a7fc640 1 --2- 172.21.15.139:0/706334704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257b4cb0 0x556d257ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f902a7fc640 1 -- 172.21.15.139:0/706334704 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d257a9870 con 0x556d257a9af0 2024-09-16T01:50:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f902a7fc640 1 --2- 172.21.15.139:0/706334704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257a9af0 0x556d257b24c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9020000c00 tx=0x7f9020005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27dd698152fe2e7a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f9010ff9640 1 -- 172.21.15.139:0/706334704 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020050030 con 0x556d257a9af0 2024-09-16T01:50:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f9010ff9640 1 -- 172.21.15.139:0/706334704 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9020032a10 con 0x556d257a9af0 2024-09-16T01:50:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.706+0000 7f9010ff9640 1 -- 172.21.15.139:0/706334704 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020035040 con 0x556d257a9af0 2024-09-16T01:50:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.707+0000 7f9034490740 1 -- 172.21.15.139:0/706334704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257a9af0 msgr2=0x556d257b24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.707+0000 7f9034490740 1 --2- 172.21.15.139:0/706334704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257a9af0 0x556d257b24c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9020000c00 tx=0x7f9020005b50 comp rx=0 tx=0).stop 2024-09-16T01:50:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.707+0000 7f9034490740 1 -- 172.21.15.139:0/706334704 shutdown_connections 2024-09-16T01:50:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.707+0000 7f9034490740 1 --2- 172.21.15.139:0/706334704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257b4cb0 0x556d257ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.707+0000 7f9034490740 1 --2- 172.21.15.139:0/706334704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257aa410 0x556d257cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.707+0000 7f9034490740 1 --2- 172.21.15.139:0/706334704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257a9af0 0x556d257b24c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.707+0000 7f9034490740 1 -- 172.21.15.139:0/706334704 >> 172.21.15.139:0/706334704 conn(0x556d257a6060 msgr2=0x556d257b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.707+0000 7f9034490740 1 -- 172.21.15.139:0/706334704 shutdown_connections 2024-09-16T01:50:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.707+0000 7f9034490740 1 -- 172.21.15.139:0/706334704 wait complete. 2024-09-16T01:50:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f9034490740 1 -- start start 2024-09-16T01:50:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f9034490740 1 Processor -- start 2024-09-16T01:50:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f9034490740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257a9af0 0x556d257bd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f9034490740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257aa410 0x556d257ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f9034490740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257b4cb0 0x556d257aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f9034490740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f902c1b2ae0 con 0x556d257b4cb0 2024-09-16T01:50:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f9034490740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f902c1b2350 con 0x556d257a9af0 2024-09-16T01:50:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f9034490740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f902c1af430 con 0x556d257aa410 2024-09-16T01:50:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f9034490740 10 client.0 _ll_register_callbacks cb 0x7f90240093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:50:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f902a7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257b4cb0 0x556d257aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f902a7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257b4cb0 0x556d257aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45328/0 (socket says 172.21.15.139:45328) 2024-09-16T01:50:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f902a7fc640 1 -- 172.21.15.139:0/1751115762 learned_addr learned my addr 172.21.15.139:0/1751115762 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f902a7fc640 1 --2- 172.21.15.139:0/1751115762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257a9af0 0x556d257bd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f902affd640 1 --2- 172.21.15.139:0/1751115762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257aa410 0x556d257ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f902a7fc640 1 -- 172.21.15.139:0/1751115762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257aa410 msgr2=0x556d257ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f902a7fc640 1 --2- 172.21.15.139:0/1751115762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257aa410 0x556d257ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f902a7fc640 1 -- 172.21.15.139:0/1751115762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257a9af0 msgr2=0x556d257bd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f902a7fc640 1 --2- 172.21.15.139:0/1751115762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257a9af0 0x556d257bd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f902a7fc640 1 -- 172.21.15.139:0/1751115762 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d257af1b0 con 0x556d257b4cb0 2024-09-16T01:50:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f902a7fc640 1 --2- 172.21.15.139:0/1751115762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257b4cb0 0x556d257aebf0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9020005ec0 tx=0x7f90200314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a491a1231790331d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.708+0000 7f900eff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:50:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.709+0000 7f900eff5640 1 -- 172.21.15.139:0/1751115762 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020035070 con 0x556d257b4cb0 2024-09-16T01:50:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.709+0000 7f9034490740 1 -- 172.21.15.139:0/1751115762 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9020032a10 con 0x556d257b4cb0 2024-09-16T01:50:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.709+0000 7f900eff5640 1 -- 172.21.15.139:0/1751115762 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7f9020032050 con 0x556d257b4cb0 2024-09-16T01:50:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.709+0000 7f900eff5640 1 -- 172.21.15.139:0/1751115762 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f9020032250 con 0x556d257b4cb0 2024-09-16T01:50:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.709+0000 7f9034490740 10 client.16599 fetch_fsmap learned FSMap version 51 2024-09-16T01:50:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.709+0000 7f9034490740 1 -- 172.21.15.139:0/1751115762 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x556d2582f3c0 con 0x556d257b4cb0 2024-09-16T01:50:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.709+0000 7f900eff5640 1 -- 172.21.15.139:0/1751115762 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9020032a10 con 0x556d257b4cb0 2024-09-16T01:50:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.709+0000 7f900eff5640 1 -- 172.21.15.139:0/1751115762 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 51) ==== 8442+0+0 (secure 0 0 0) 0x7f902004a030 con 0x556d257b4cb0 2024-09-16T01:50:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.709+0000 7f9034490740 10 client.16599 fetch_fsmap finished waiting for FSMap version 51 2024-09-16T01:50:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.709+0000 7f9034490740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:50:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.709+0000 7f9034490740 20 client.16599 populate_metadata read hostname 'smithi139' 2024-09-16T01:50:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.709+0000 7f9034490740 1 --2- 172.21.15.139:0/1751115762 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556d2582fb20 0x556d25831f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.709+0000 7f9034490740 4 client.16599 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:50:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.709+0000 7f9034490740 1 -- 172.21.15.139:0/1751115762 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d25715d70 con 0x556d2582fb20 2024-09-16T01:50:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.710+0000 7f902affd640 1 --2- 172.21.15.139:0/1751115762 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556d2582fb20 0x556d25831f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.710+0000 7f902affd640 1 --2- 172.21.15.139:0/1751115762 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556d2582fb20 0x556d25831f40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6734ee096b1d5d5b server_cookie=ead62a44798d0852 in_seq=0 out_seq=0 2024-09-16T01:50:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.710+0000 7f900eff5640 10 client.16599 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.725+0000 7f900eff5640 1 -- 172.21.15.139:0/1751115762 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d25715d70 con 0x556d2582fb20 2024-09-16T01:50:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.725+0000 7f900eff5640 10 client.16599 handle_command_reply: tid=0 2024-09-16T01:50:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.725+0000 7f9034490740 1 client.16599 shutdown 2024-09-16T01:50:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.725+0000 7f9034490740 1 -- 172.21.15.139:0/1751115762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257b4cb0 msgr2=0x556d257aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.725+0000 7f9034490740 1 --2- 172.21.15.139:0/1751115762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257b4cb0 0x556d257aebf0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9020005ec0 tx=0x7f90200314e0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f9034490740 1 -- 172.21.15.139:0/1751115762 shutdown_connections 2024-09-16T01:50:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f9034490740 1 --2- 172.21.15.139:0/1751115762 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556d2582fb20 0x556d25831f40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f9034490740 1 --2- 172.21.15.139:0/1751115762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d257b4cb0 0x556d257aebf0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f900eff5640 0 client.16599 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f9034490740 1 --2- 172.21.15.139:0/1751115762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d257aa410 0x556d257ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f9034490740 1 --2- 172.21.15.139:0/1751115762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d257a9af0 0x556d257bd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f9034490740 1 -- 172.21.15.139:0/1751115762 >> 172.21.15.139:0/1751115762 conn(0x556d257a6060 msgr2=0x556d257b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f9034490740 1 -- 172.21.15.139:0/1751115762 shutdown_connections 2024-09-16T01:50:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f9034490740 1 -- 172.21.15.139:0/1751115762 wait complete. 2024-09-16T01:50:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f9034490740 20 client.16599 trim_cache size 0 max 16384 2024-09-16T01:50:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:50:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:50:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:50:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:50:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f90316f1640 1 -- 172.21.15.139:0/2758326981 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90140a3460 msgr2=0x7f90140a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f90316f1640 1 --2- 172.21.15.139:0/2758326981 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90140a3460 0x7f90140a5920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f902409c440 tx=0x7f902400efd0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f90316f1640 1 -- 172.21.15.139:0/2758326981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902c10c0d0 msgr2=0x7f902c101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f90316f1640 1 --2- 172.21.15.139:0/2758326981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902c10c0d0 0x7f902c101370 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f902400eb10 tx=0x7f902400c2d0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f90316f1640 1 -- 172.21.15.139:0/2758326981 shutdown_connections 2024-09-16T01:50:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f90316f1640 1 --2- 172.21.15.139:0/2758326981 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90140a3460 0x7f90140a5920 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f90316f1640 1 --2- 172.21.15.139:0/2758326981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902c10c0d0 0x7f902c101370 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f90316f1640 1 --2- 172.21.15.139:0/2758326981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f902c1035c0 0x7f902c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f90316f1640 1 --2- 172.21.15.139:0/2758326981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f902c102ca0 0x7f902c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.726+0000 7f90316f1640 1 -- 172.21.15.139:0/2758326981 >> 172.21.15.139:0/2758326981 conn(0x7f902c0fc6b0 msgr2=0x7f902c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.727+0000 7f90316f1640 1 -- 172.21.15.139:0/2758326981 shutdown_connections 2024-09-16T01:50:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:07.727+0000 7f90316f1640 1 -- 172.21.15.139:0/2758326981 wait complete. 2024-09-16T01:50:07.783 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id c50bf1fe --release --await' 2024-09-16T01:50:07.783 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id c50bf1fe --release --await 2024-09-16T01:50:08.081 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:50:08.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.588+0000 7fb4f0a9d640 1 -- 172.21.15.139:0/2878167927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4ec1008d0 msgr2=0x7fb4ec100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:08.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.588+0000 7fb4f0a9d640 1 --2- 172.21.15.139:0/2878167927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4ec1008d0 0x7fb4ec100cb0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009900 tx=0x7fb4dc02a410 comp rx=0 tx=0).stop 2024-09-16T01:50:08.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.589+0000 7fb4f0a9d640 1 -- 172.21.15.139:0/2878167927 shutdown_connections 2024-09-16T01:50:08.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.589+0000 7fb4f0a9d640 1 --2- 172.21.15.139:0/2878167927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4ec10e480 0x7fb4ec110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.589+0000 7fb4f0a9d640 1 --2- 172.21.15.139:0/2878167927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ec101280 0x7fb4ec10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.589+0000 7fb4f0a9d640 1 --2- 172.21.15.139:0/2878167927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4ec1008d0 0x7fb4ec100cb0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.589+0000 7fb4f0a9d640 1 -- 172.21.15.139:0/2878167927 >> 172.21.15.139:0/2878167927 conn(0x7fb4ec0fc7a0 msgr2=0x7fb4ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:08.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.589+0000 7fb4f0a9d640 1 -- 172.21.15.139:0/2878167927 shutdown_connections 2024-09-16T01:50:08.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.589+0000 7fb4f0a9d640 1 -- 172.21.15.139:0/2878167927 wait complete. 2024-09-16T01:50:08.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.589+0000 7fb4f0a9d640 1 Processor -- start 2024-09-16T01:50:08.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.589+0000 7fb4f0a9d640 1 -- start start 2024-09-16T01:50:08.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4f0a9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ec1008d0 0x7fb4ec19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4f0a9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4ec101280 0x7fb4ec19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4f0a9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4ec10e480 0x7fb4ec19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4f0a9d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1141f0 con 0x7fb4ec10e480 2024-09-16T01:50:08.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4f0a9d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec114070 con 0x7fb4ec101280 2024-09-16T01:50:08.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4f0a9d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec114370 con 0x7fb4ec1008d0 2024-09-16T01:50:08.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4ead9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4ec10e480 0x7fb4ec19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4ea59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ec1008d0 0x7fb4ec19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4e9d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4ec101280 0x7fb4ec19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4ea59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ec1008d0 0x7fb4ec19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:34912/0 (socket says 172.21.15.139:34912) 2024-09-16T01:50:08.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4e9d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4ec101280 0x7fb4ec19ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47358/0 (socket says 172.21.15.139:47358) 2024-09-16T01:50:08.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4ead9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4ec10e480 0x7fb4ec19f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45358/0 (socket says 172.21.15.139:45358) 2024-09-16T01:50:08.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4ea59c640 1 -- 172.21.15.139:0/3040107392 learned_addr learned my addr 172.21.15.139:0/3040107392 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:08.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4ead9d640 1 -- 172.21.15.139:0/3040107392 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ec1008d0 msgr2=0x7fb4ec19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:08.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4ead9d640 1 --2- 172.21.15.139:0/3040107392 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ec1008d0 0x7fb4ec19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4ead9d640 1 -- 172.21.15.139:0/3040107392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4ec101280 msgr2=0x7fb4ec19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:08.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4ead9d640 1 --2- 172.21.15.139:0/3040107392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4ec101280 0x7fb4ec19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4ead9d640 1 -- 172.21.15.139:0/3040107392 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ec19f700 con 0x7fb4ec10e480 2024-09-16T01:50:08.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.590+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/3040107392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4ec101280 0x7fb4ec19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:50:08.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.591+0000 7fb4ead9d640 1 --2- 172.21.15.139:0/3040107392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4ec10e480 0x7fb4ec19f020 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec102ab0 tx=0x7fb4e4009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:08.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.591+0000 7fb4db7fe640 1 -- 172.21.15.139:0/3040107392 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4019070 con 0x7fb4ec10e480 2024-09-16T01:50:08.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.591+0000 7fb4db7fe640 1 -- 172.21.15.139:0/3040107392 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4e400cdf0 con 0x7fb4ec10e480 2024-09-16T01:50:08.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.591+0000 7fb4db7fe640 1 -- 172.21.15.139:0/3040107392 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4012920 con 0x7fb4ec10e480 2024-09-16T01:50:08.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.591+0000 7fb4f0a9d640 1 -- 172.21.15.139:0/3040107392 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ec19f9f0 con 0x7fb4ec10e480 2024-09-16T01:50:08.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.591+0000 7fb4f0a9d640 1 -- 172.21.15.139:0/3040107392 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ec1a72d0 con 0x7fb4ec10e480 2024-09-16T01:50:08.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.593+0000 7fb4db7fe640 1 -- 172.21.15.139:0/3040107392 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb4e401c070 con 0x7fb4ec10e480 2024-09-16T01:50:08.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.593+0000 7fb4f2e76740 1 Processor -- start 2024-09-16T01:50:08.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4f2e76740 1 -- start start 2024-09-16T01:50:08.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4f2e76740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a2852060 0x55f9a2852440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4f2e76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852980 0x55f9a285cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4f2e76740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a285d240 0x55f9a285f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4f2e76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f9a27b7620 con 0x55f9a2852980 2024-09-16T01:50:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4f2e76740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f9a275ee50 con 0x55f9a285d240 2024-09-16T01:50:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4f2e76740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f9a27b35e0 con 0x55f9a2852060 2024-09-16T01:50:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4ea59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852980 0x55f9a285cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4ea59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852980 0x55f9a285cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45360/0 (socket says 172.21.15.139:45360) 2024-09-16T01:50:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4ea59c640 1 -- 172.21.15.139:0/1117208685 learned_addr learned my addr 172.21.15.139:0/1117208685 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/1117208685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a285d240 0x55f9a285f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4db7fe640 1 --2- 172.21.15.139:0/3040107392 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb4d40a33b0 0x7fb4d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/1117208685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a2852060 0x55f9a2852440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4ead9d640 1 --2- 172.21.15.139:0/3040107392 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb4d40a33b0 0x7fb4d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4ea59c640 1 -- 172.21.15.139:0/1117208685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a2852060 msgr2=0x55f9a2852440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4ea59c640 1 --2- 172.21.15.139:0/1117208685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a2852060 0x55f9a2852440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4ea59c640 1 -- 172.21.15.139:0/1117208685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a285d240 msgr2=0x55f9a285f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4ea59c640 1 --2- 172.21.15.139:0/1117208685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a285d240 0x55f9a285f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4ea59c640 1 -- 172.21.15.139:0/1117208685 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a274fd40 con 0x55f9a2852980 2024-09-16T01:50:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/1117208685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a285d240 0x55f9a285f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:50:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.594+0000 7fb4db7fe640 1 -- 172.21.15.139:0/3040107392 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7fb4e40cf810 con 0x7fb4ec10e480 2024-09-16T01:50:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.595+0000 7fb4ea59c640 1 --2- 172.21.15.139:0/1117208685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852980 0x55f9a285cd00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc032d60 tx=0x7fb4dc032d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4982baa1254ba78a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.595+0000 7fb4ead9d640 1 --2- 172.21.15.139:0/3040107392 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb4d40a33b0 0x7fb4d40a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4e409b5a0 tx=0x7fb4e406d620 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.595+0000 7fb4d8ff9640 1 -- 172.21.15.139:0/1117208685 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc040070 con 0x55f9a2852980 2024-09-16T01:50:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.595+0000 7fb4d8ff9640 1 -- 172.21.15.139:0/1117208685 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4dc031870 con 0x55f9a2852980 2024-09-16T01:50:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.595+0000 7fb4d8ff9640 1 -- 172.21.15.139:0/1117208685 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc035040 con 0x55f9a2852980 2024-09-16T01:50:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 -- 172.21.15.139:0/1117208685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852980 msgr2=0x55f9a285cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/1117208685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852980 0x55f9a285cd00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc032d60 tx=0x7fb4dc032d90 comp rx=0 tx=0).stop 2024-09-16T01:50:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 -- 172.21.15.139:0/1117208685 shutdown_connections 2024-09-16T01:50:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/1117208685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a285d240 0x55f9a285f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/1117208685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852980 0x55f9a285cd00 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/1117208685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a2852060 0x55f9a2852440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 -- 172.21.15.139:0/1117208685 >> 172.21.15.139:0/1117208685 conn(0x55f9a284e5f0 msgr2=0x55f9a284e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 -- 172.21.15.139:0/1117208685 shutdown_connections 2024-09-16T01:50:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 -- 172.21.15.139:0/1117208685 wait complete. 2024-09-16T01:50:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 -- start start 2024-09-16T01:50:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 Processor -- start 2024-09-16T01:50:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a2852060 0x55f9a285a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852980 0x55f9a285a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a285d240 0x55f9a285ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f9a285fe10 con 0x55f9a2852980 2024-09-16T01:50:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f9a273ff10 con 0x55f9a2852060 2024-09-16T01:50:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f9a285ff90 con 0x55f9a285d240 2024-09-16T01:50:08.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4f2e76740 10 client.0 _ll_register_callbacks cb 0x55f9a2763db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:50:08.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4ea59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852980 0x55f9a285a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4ea59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852980 0x55f9a285a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45370/0 (socket says 172.21.15.139:45370) 2024-09-16T01:50:08.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.596+0000 7fb4ea59c640 1 -- 172.21.15.139:0/1883995439 learned_addr learned my addr 172.21.15.139:0/1883995439 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:08.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/1883995439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a2852060 0x55f9a285a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4ea59c640 1 -- 172.21.15.139:0/1883995439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a285d240 msgr2=0x55f9a285ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:08.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4ea59c640 1 --2- 172.21.15.139:0/1883995439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a285d240 0x55f9a285ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4ea59c640 1 -- 172.21.15.139:0/1883995439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a2852060 msgr2=0x55f9a285a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:08.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4ea59c640 1 --2- 172.21.15.139:0/1883995439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a2852060 0x55f9a285a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4ea59c640 1 -- 172.21.15.139:0/1883995439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a24c6580 con 0x55f9a2852980 2024-09-16T01:50:08.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/1883995439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a2852060 0x55f9a285a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:50:08.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4ea59c640 1 --2- 172.21.15.139:0/1883995439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852980 0x55f9a285a6c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc02ab90 tx=0x7fb4dc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffb47bd745969cb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:08.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:50:08.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4d1ffb640 1 -- 172.21.15.139:0/1883995439 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc035070 con 0x55f9a2852980 2024-09-16T01:50:08.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4d1ffb640 1 -- 172.21.15.139:0/1883995439 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7fb4dc004020 con 0x55f9a2852980 2024-09-16T01:50:08.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4f2e76740 1 -- 172.21.15.139:0/1883995439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4dc031870 con 0x55f9a2852980 2024-09-16T01:50:08.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4d1ffb640 1 -- 172.21.15.139:0/1883995439 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fb4dc055050 con 0x55f9a2852980 2024-09-16T01:50:08.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4f2e76740 10 client.16611 fetch_fsmap learned FSMap version 51 2024-09-16T01:50:08.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4f2e76740 1 -- 172.21.15.139:0/1883995439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f9a2590960 con 0x55f9a2852980 2024-09-16T01:50:08.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4d1ffb640 1 -- 172.21.15.139:0/1883995439 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4dc031870 con 0x55f9a2852980 2024-09-16T01:50:08.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.597+0000 7fb4d1ffb640 1 -- 172.21.15.139:0/1883995439 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 51) ==== 8442+0+0 (secure 0 0 0) 0x7fb4dc0035f0 con 0x55f9a2852980 2024-09-16T01:50:08.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.598+0000 7fb4f2e76740 10 client.16611 fetch_fsmap finished waiting for FSMap version 51 2024-09-16T01:50:08.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.598+0000 7fb4f2e76740 10 client.16611 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:50:08.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.598+0000 7fb4f2e76740 20 client.16611 populate_metadata read hostname 'smithi139' 2024-09-16T01:50:08.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.598+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/1883995439 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f9a286cc90 0x55f9a286f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.598+0000 7fb4f2e76740 4 client.16611 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:50:08.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.598+0000 7fb4f2e76740 1 -- 172.21.15.139:0/1883995439 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9a27c4b90 con 0x55f9a286cc90 2024-09-16T01:50:08.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.598+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/1883995439 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f9a286cc90 0x55f9a286f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.598+0000 7fb4d1ffb640 10 client.16611 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:08.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.598+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/1883995439 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f9a286cc90 0x55f9a286f080 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1025e5488f48b2ac server_cookie=e04b2bba4d1e89f8 in_seq=0 out_seq=0 2024-09-16T01:50:08.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4d1ffb640 1 -- 172.21.15.139:0/1883995439 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9a27c4b90 con 0x55f9a286cc90 2024-09-16T01:50:08.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4d1ffb640 10 client.16611 handle_command_reply: tid=0 2024-09-16T01:50:08.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4f2e76740 1 client.16611 shutdown 2024-09-16T01:50:08.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4f2e76740 1 -- 172.21.15.139:0/1883995439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852980 msgr2=0x55f9a285a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:08.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/1883995439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852980 0x55f9a285a6c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc02ab90 tx=0x7fb4dc005780 comp rx=0 tx=0).stop 2024-09-16T01:50:08.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4f2e76740 1 -- 172.21.15.139:0/1883995439 shutdown_connections 2024-09-16T01:50:08.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/1883995439 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f9a286cc90 0x55f9a286f080 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/1883995439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a285d240 0x55f9a285ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4d1ffb640 0 client.16611 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:08.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/1883995439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852980 0x55f9a285a6c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/1883995439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a2852060 0x55f9a285a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4f2e76740 1 -- 172.21.15.139:0/1883995439 >> 172.21.15.139:0/1883995439 conn(0x55f9a284e5f0 msgr2=0x55f9a28542d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:08.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4f2e76740 1 -- 172.21.15.139:0/1883995439 shutdown_connections 2024-09-16T01:50:08.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4f2e76740 1 -- 172.21.15.139:0/1883995439 wait complete. 2024-09-16T01:50:08.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.599+0000 7fb4f2e76740 20 client.16611 trim_cache size 0 max 16384 2024-09-16T01:50:08.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4f2e76740 1 Processor -- start 2024-09-16T01:50:08.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4f2e76740 1 -- start start 2024-09-16T01:50:08.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4f2e76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852060 0x55f9a284da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4f2e76740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a2852980 0x55f9a2854a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4f2e76740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a285d240 0x55f9a2874710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4f2e76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1b1350 con 0x55f9a2852060 2024-09-16T01:50:08.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4f2e76740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1b3160 con 0x55f9a2852980 2024-09-16T01:50:08.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4f2e76740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1b1ae0 con 0x55f9a285d240 2024-09-16T01:50:08.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4e9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852060 0x55f9a284da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4e9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852060 0x55f9a284da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45380/0 (socket says 172.21.15.139:45380) 2024-09-16T01:50:08.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4e9d9b640 1 -- 172.21.15.139:0/2722198591 learned_addr learned my addr 172.21.15.139:0/2722198591 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:08.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4ea59c640 1 --2- 172.21.15.139:0/2722198591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a2852980 0x55f9a2854a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/2722198591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a285d240 0x55f9a2874710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4e9d9b640 1 -- 172.21.15.139:0/2722198591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a285d240 msgr2=0x55f9a2874710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:08.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/2722198591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a285d240 0x55f9a2874710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4e9d9b640 1 -- 172.21.15.139:0/2722198591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a2852980 msgr2=0x55f9a2854a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:08.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/2722198591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a2852980 0x55f9a2854a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.616+0000 7fb4e9d9b640 1 -- 172.21.15.139:0/2722198591 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a2859130 con 0x55f9a2852060 2024-09-16T01:50:08.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.617+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/2722198591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852060 0x55f9a284da10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0005d00 tx=0x7fb4e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821a728c24451177 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:08.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.617+0000 7fb4d3fff640 1 -- 172.21.15.139:0/2722198591 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0010bc0 con 0x55f9a2852060 2024-09-16T01:50:08.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.617+0000 7fb4d3fff640 1 -- 172.21.15.139:0/2722198591 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4e0027040 con 0x55f9a2852060 2024-09-16T01:50:08.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.617+0000 7fb4d3fff640 1 -- 172.21.15.139:0/2722198591 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0003ad0 con 0x55f9a2852060 2024-09-16T01:50:08.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.617+0000 7fb4f2e76740 1 -- 172.21.15.139:0/2722198591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852060 msgr2=0x55f9a284da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:08.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.617+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/2722198591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852060 0x55f9a284da10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0005d00 tx=0x7fb4e0004730 comp rx=0 tx=0).stop 2024-09-16T01:50:08.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.617+0000 7fb4f2e76740 1 -- 172.21.15.139:0/2722198591 shutdown_connections 2024-09-16T01:50:08.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.617+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/2722198591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a285d240 0x55f9a2874710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.617+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/2722198591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a2852980 0x55f9a2854a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.617+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/2722198591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852060 0x55f9a284da10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.617+0000 7fb4f2e76740 1 -- 172.21.15.139:0/2722198591 >> 172.21.15.139:0/2722198591 conn(0x55f9a284e5f0 msgr2=0x55f9a285b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:08.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.617+0000 7fb4f2e76740 1 -- 172.21.15.139:0/2722198591 shutdown_connections 2024-09-16T01:50:08.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.617+0000 7fb4f2e76740 1 -- 172.21.15.139:0/2722198591 wait complete. 2024-09-16T01:50:08.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.618+0000 7fb4f2e76740 1 -- start start 2024-09-16T01:50:08.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.618+0000 7fb4f2e76740 1 Processor -- start 2024-09-16T01:50:08.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.618+0000 7fb4f2e76740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852060 0x55f9a28656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.618+0000 7fb4f2e76740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a2852980 0x55f9a2861a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.618+0000 7fb4f2e76740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a285d240 0x55f9a2861f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.618+0000 7fb4f2e76740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1b2a00 con 0x55f9a2852060 2024-09-16T01:50:08.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.618+0000 7fb4f2e76740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1b2270 con 0x55f9a285d240 2024-09-16T01:50:08.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.618+0000 7fb4f2e76740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1af350 con 0x55f9a2852980 2024-09-16T01:50:08.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.618+0000 7fb4f2e76740 10 client.0 _ll_register_callbacks cb 0x7fb4dc032c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:50:08.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.618+0000 7fb4e9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852060 0x55f9a28656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.618+0000 7fb4e9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852060 0x55f9a28656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45388/0 (socket says 172.21.15.139:45388) 2024-09-16T01:50:08.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.618+0000 7fb4e9d9b640 1 -- 172.21.15.139:0/4123619232 learned_addr learned my addr 172.21.15.139:0/4123619232 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:08.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.618+0000 7fb4ea59c640 1 --2- 172.21.15.139:0/4123619232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a2852980 0x55f9a2861a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.618+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/4123619232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a285d240 0x55f9a2861f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4e9d9b640 1 -- 172.21.15.139:0/4123619232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a2852980 msgr2=0x55f9a2861a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:08.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/4123619232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a2852980 0x55f9a2861a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4e9d9b640 1 -- 172.21.15.139:0/4123619232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a285d240 msgr2=0x55f9a2861f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:08.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/4123619232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a285d240 0x55f9a2861f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:08.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4e9d9b640 1 -- 172.21.15.139:0/4123619232 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a2862520 con 0x55f9a2852060 2024-09-16T01:50:08.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4e9d9b640 1 --2- 172.21.15.139:0/4123619232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852060 0x55f9a28656e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000bb90 tx=0x7fb4e0012970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281173c8034b7c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:08.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:50:08.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4d1ffb640 1 -- 172.21.15.139:0/4123619232 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0034020 con 0x55f9a2852060 2024-09-16T01:50:08.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4d1ffb640 1 -- 172.21.15.139:0/4123619232 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9554+0+0 (secure 0 0 0) 0x7fb4e002b630 con 0x55f9a2852060 2024-09-16T01:50:08.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4f2e76740 1 -- 172.21.15.139:0/4123619232 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4e0027040 con 0x55f9a2852060 2024-09-16T01:50:08.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4d1ffb640 1 -- 172.21.15.139:0/4123619232 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fb4e003f050 con 0x55f9a2852060 2024-09-16T01:50:08.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4f2e76740 10 client.16617 fetch_fsmap learned FSMap version 51 2024-09-16T01:50:08.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4f2e76740 1 -- 172.21.15.139:0/4123619232 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f9a28d84e0 con 0x55f9a2852060 2024-09-16T01:50:08.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4d1ffb640 1 -- 172.21.15.139:0/4123619232 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4e0027040 con 0x55f9a2852060 2024-09-16T01:50:08.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.619+0000 7fb4d1ffb640 1 -- 172.21.15.139:0/4123619232 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 51) ==== 8442+0+0 (secure 0 0 0) 0x7fb4e0004100 con 0x55f9a2852060 2024-09-16T01:50:08.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.620+0000 7fb4f2e76740 10 client.16617 fetch_fsmap finished waiting for FSMap version 51 2024-09-16T01:50:08.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.620+0000 7fb4f2e76740 10 client.16617 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:50:08.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.620+0000 7fb4f2e76740 20 client.16617 populate_metadata read hostname 'smithi139' 2024-09-16T01:50:08.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.620+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/4123619232 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f9a286cc90 0x55f9a28dac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:08.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.620+0000 7fb4f2e76740 4 client.16617 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c50bf1fe", "release": true, "await": true}] 2024-09-16T01:50:08.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.620+0000 7fb4f2e76740 1 -- 172.21.15.139:0/4123619232 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c50bf1fe", "release": true, "await": true}) -- 0x55f9a27bf0a0 con 0x55f9a286cc90 2024-09-16T01:50:08.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.620+0000 7fb4ea59c640 1 --2- 172.21.15.139:0/4123619232 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f9a286cc90 0x55f9a28dac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:08.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.620+0000 7fb4ea59c640 1 --2- 172.21.15.139:0/4123619232 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f9a286cc90 0x55f9a28dac10 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23115ef5869dbabc server_cookie=607b7a7b136003c4 in_seq=0 out_seq=0 2024-09-16T01:50:08.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:08.620+0000 7fb4d1ffb640 10 client.16617 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.427+0000 7fb4d1ffb640 1 -- 172.21.15.139:0/4123619232 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f9a27bf0a0 con 0x55f9a286cc90 2024-09-16T01:50:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.427+0000 7fb4d1ffb640 10 client.16617 handle_command_reply: tid=0 2024-09-16T01:50:09.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.427+0000 7fb4f2e76740 1 client.16617 shutdown 2024-09-16T01:50:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.428+0000 7fb4f2e76740 1 -- 172.21.15.139:0/4123619232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852060 msgr2=0x55f9a28656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.428+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/4123619232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852060 0x55f9a28656e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000bb90 tx=0x7fb4e0012970 comp rx=0 tx=0).stop 2024-09-16T01:50:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.428+0000 7fb4f2e76740 1 -- 172.21.15.139:0/4123619232 shutdown_connections 2024-09-16T01:50:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.428+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/4123619232 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f9a286cc90 0x55f9a28dac10 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.428+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/4123619232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9a285d240 0x55f9a2861f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:09.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.428+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/4123619232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9a2852980 0x55f9a2861a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.428+0000 7fb4d1ffb640 0 client.16617 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.428+0000 7fb4f2e76740 1 --2- 172.21.15.139:0/4123619232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9a2852060 0x55f9a28656e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.428+0000 7fb4f2e76740 1 -- 172.21.15.139:0/4123619232 >> 172.21.15.139:0/4123619232 conn(0x55f9a284e5f0 msgr2=0x55f9a2859fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.428+0000 7fb4f2e76740 1 -- 172.21.15.139:0/4123619232 shutdown_connections 2024-09-16T01:50:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.428+0000 7fb4f2e76740 1 -- 172.21.15.139:0/4123619232 wait complete. 2024-09-16T01:50:09.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.428+0000 7fb4f2e76740 20 client.16617 trim_cache size 0 max 16384 2024-09-16T01:50:09.432 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:50:09.433 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 52, 2024-09-16T01:50:09.433 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:50:09.433 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 36, 2024-09-16T01:50:09.433 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:50:09.433 INFO:teuthology.orchestra.run.smithi139.stdout: "c50bf1fe": { 2024-09-16T01:50:09.433 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 36, 2024-09-16T01:50:09.433 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:50:09.433 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:50:09.433 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:50:09.433 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:50:09.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:50:09.433 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:50:09.433 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 142.3, 2024-09-16T01:50:09.433 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:50:09.434 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:50:09.434 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:50:09.434 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:50:09.434 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:50:09.434 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:50:09.434 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:50:09.434 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:50:09.434 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:50:09.434 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:50:09.434 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:50:09.434 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:50:09.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.429+0000 7fb4f0a9d640 1 -- 172.21.15.139:0/3040107392 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb4d40a33b0 msgr2=0x7fb4d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:09.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.429+0000 7fb4f0a9d640 1 --2- 172.21.15.139:0/3040107392 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb4d40a33b0 0x7fb4d40a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4e409b5a0 tx=0x7fb4e406d620 comp rx=0 tx=0).stop 2024-09-16T01:50:09.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.429+0000 7fb4f0a9d640 1 -- 172.21.15.139:0/3040107392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4ec10e480 msgr2=0x7fb4ec19f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:09.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.429+0000 7fb4f0a9d640 1 --2- 172.21.15.139:0/3040107392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4ec10e480 0x7fb4ec19f020 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec102ab0 tx=0x7fb4e4009ed0 comp rx=0 tx=0).stop 2024-09-16T01:50:09.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.429+0000 7fb4f0a9d640 1 -- 172.21.15.139:0/3040107392 shutdown_connections 2024-09-16T01:50:09.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.429+0000 7fb4f0a9d640 1 --2- 172.21.15.139:0/3040107392 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb4d40a33b0 0x7fb4d40a5870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:09.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.429+0000 7fb4f0a9d640 1 --2- 172.21.15.139:0/3040107392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4ec10e480 0x7fb4ec19f020 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:09.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.429+0000 7fb4f0a9d640 1 --2- 172.21.15.139:0/3040107392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4ec101280 0x7fb4ec19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:09.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.429+0000 7fb4f0a9d640 1 --2- 172.21.15.139:0/3040107392 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ec1008d0 0x7fb4ec19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:09.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.429+0000 7fb4f0a9d640 1 -- 172.21.15.139:0/3040107392 >> 172.21.15.139:0/3040107392 conn(0x7fb4ec0fc7a0 msgr2=0x7fb4ec0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:09.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.429+0000 7fb4f0a9d640 1 -- 172.21.15.139:0/3040107392 shutdown_connections 2024-09-16T01:50:09.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:50:09.429+0000 7fb4f0a9d640 1 -- 172.21.15.139:0/3040107392 wait complete. 2024-09-16T01:50:09.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:09 smithi139 ceph-mon[31175]: pgmap v302: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 82 KiB/s wr, 42 op/s 2024-09-16T01:50:09.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:09 smithi176 ceph-mon[35324]: pgmap v302: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 82 KiB/s wr, 42 op/s 2024-09-16T01:50:09.797 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:09 smithi164 ceph-mon[35121]: pgmap v302: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 82 KiB/s wr, 42 op/s 2024-09-16T01:50:10.371 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c50bf1fe', total seconds elapsed: 26.8 2024-09-16T01:50:10.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:10 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:10.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:10 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:10.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:10 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:11.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:11 smithi139 ceph-mon[31175]: pgmap v303: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 59 KiB/s wr, 39 op/s 2024-09-16T01:50:11.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:11 smithi176 ceph-mon[35324]: pgmap v303: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 59 KiB/s wr, 39 op/s 2024-09-16T01:50:11.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:11 smithi164 ceph-mon[35121]: pgmap v303: 129 pgs: 129 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 59 KiB/s wr, 39 op/s 2024-09-16T01:50:12.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:12 smithi139 ceph-mon[31175]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T01:50:12.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:12 smithi176 ceph-mon[35324]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T01:50:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:12 smithi164 ceph-mon[35121]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T01:50:13.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:13 smithi139 ceph-mon[31175]: pgmap v305: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 214 KiB/s wr, 45 op/s 2024-09-16T01:50:13.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:13 smithi176 ceph-mon[35324]: pgmap v305: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 214 KiB/s wr, 45 op/s 2024-09-16T01:50:13.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:13 smithi164 ceph-mon[35121]: pgmap v305: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 214 KiB/s wr, 45 op/s 2024-09-16T01:50:15.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:15 smithi139 ceph-mon[31175]: pgmap v306: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 640 KiB/s wr, 51 op/s 2024-09-16T01:50:15.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:15 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:15.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:15 smithi176 ceph-mon[35324]: pgmap v306: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 640 KiB/s wr, 51 op/s 2024-09-16T01:50:15.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:15 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:15.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:15 smithi164 ceph-mon[35121]: pgmap v306: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 640 KiB/s wr, 51 op/s 2024-09-16T01:50:15.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:15 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:17.596 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:17 smithi164 ceph-mon[35121]: pgmap v307: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 963 KiB/s wr, 81 op/s 2024-09-16T01:50:17.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:17 smithi139 ceph-mon[31175]: pgmap v307: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 963 KiB/s wr, 81 op/s 2024-09-16T01:50:17.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:17 smithi176 ceph-mon[35324]: pgmap v307: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 963 KiB/s wr, 81 op/s 2024-09-16T01:50:19.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:19 smithi139 ceph-mon[31175]: pgmap v308: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-09-16T01:50:19.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:19 smithi176 ceph-mon[35324]: pgmap v308: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-09-16T01:50:19.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:19 smithi164 ceph-mon[35121]: pgmap v308: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-09-16T01:50:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.873+0000 7f204759e640 1 -- 172.21.15.139:0/4183135788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2040095520 msgr2=0x7f2040099170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.873+0000 7f204759e640 1 --2- 172.21.15.139:0/4183135788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2040095520 0x7f2040099170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f203c00fb10 tx=0x7f203c02e570 comp rx=0 tx=0).stop 2024-09-16T01:50:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.873+0000 7f204759e640 1 -- 172.21.15.139:0/4183135788 shutdown_connections 2024-09-16T01:50:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.873+0000 7f204759e640 1 --2- 172.21.15.139:0/4183135788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2040099880 0x7f20400a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.873+0000 7f204759e640 1 --2- 172.21.15.139:0/4183135788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2040095520 0x7f2040099170 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.873+0000 7f204759e640 1 --2- 172.21.15.139:0/4183135788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2040094b70 0x7f2040094f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.873+0000 7f204759e640 1 -- 172.21.15.139:0/4183135788 >> 172.21.15.139:0/4183135788 conn(0x7f204000a1e0 msgr2=0x7f204000a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.874+0000 7f204759e640 1 -- 172.21.15.139:0/4183135788 shutdown_connections 2024-09-16T01:50:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.875+0000 7f204759e640 1 -- 172.21.15.139:0/4183135788 wait complete. 2024-09-16T01:50:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.875+0000 7f204759e640 1 Processor -- start 2024-09-16T01:50:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.877+0000 7f204759e640 1 -- start start 2024-09-16T01:50:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.877+0000 7f204759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2040094b70 0x7f20400a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.877+0000 7f204759e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2040099880 0x7f20400a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.877+0000 7f204759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20400a4320 0x7f20400a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.877+0000 7f204759e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f20400a2970 con 0x7f2040094b70 2024-09-16T01:50:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.877+0000 7f204759e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f20400a27f0 con 0x7f20400a4320 2024-09-16T01:50:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.877+0000 7f204759e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f20400a2af0 con 0x7f2040099880 2024-09-16T01:50:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.877+0000 7f2046d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20400a4320 0x7f20400a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.878+0000 7f2046d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20400a4320 0x7f20400a47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:36382/0 (socket says 172.21.15.139:36382) 2024-09-16T01:50:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.878+0000 7f2046d9d640 1 -- 172.21.15.139:0/1262885654 learned_addr learned my addr 172.21.15.139:0/1262885654 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.878+0000 7f2046d9d640 1 -- 172.21.15.139:0/1262885654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2040099880 msgr2=0x7f20400a3de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:50:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.878+0000 7f2046d9d640 1 --2- 172.21.15.139:0/1262885654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2040099880 0x7f20400a3de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.878+0000 7f2046d9d640 1 -- 172.21.15.139:0/1262885654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2040094b70 msgr2=0x7f20400a9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.878+0000 7f2046d9d640 1 --2- 172.21.15.139:0/1262885654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2040094b70 0x7f20400a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.878+0000 7f2046d9d640 1 -- 172.21.15.139:0/1262885654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2040158330 con 0x7f20400a4320 2024-09-16T01:50:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.879+0000 7f2046d9d640 1 --2- 172.21.15.139:0/1262885654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20400a4320 0x7f20400a47d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2048065a40 tx=0x7f204806e430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.879+0000 7f20377fe640 1 -- 172.21.15.139:0/1262885654 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204807a020 con 0x7f20400a4320 2024-09-16T01:50:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.879+0000 7f204759e640 1 -- 172.21.15.139:0/1262885654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2040158560 con 0x7f20400a4320 2024-09-16T01:50:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.879+0000 7f204759e640 1 -- 172.21.15.139:0/1262885654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2040158a30 con 0x7f20400a4320 2024-09-16T01:50:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.883+0000 7f20377fe640 1 -- 172.21.15.139:0/1262885654 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2048076420 con 0x7f20400a4320 2024-09-16T01:50:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.883+0000 7f20377fe640 1 -- 172.21.15.139:0/1262885654 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20480755e0 con 0x7f20400a4320 2024-09-16T01:50:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.890+0000 7f20377fe640 1 -- 172.21.15.139:0/1262885654 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f204807d020 con 0x7f20400a4320 2024-09-16T01:50:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.891+0000 7f204f9d7740 1 Processor -- start 2024-09-16T01:50:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.891+0000 7f204f9d7740 1 -- start start 2024-09-16T01:50:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.892+0000 7f204f9d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cb70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.892+0000 7f204f9d7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cb75e0 0x55b355cc1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.892+0000 7f204f9d7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cc1e80 0x55b355cc4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.892+0000 7f204f9d7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b355c1b2f0 con 0x55b355cb6cc0 2024-09-16T01:50:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.892+0000 7f204f9d7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b355bc3270 con 0x55b355cc1e80 2024-09-16T01:50:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.892+0000 7f204f9d7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b355c172b0 con 0x55b355cb75e0 2024-09-16T01:50:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.893+0000 7f20377fe640 1 --2- 172.21.15.139:0/1262885654 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f20300a35c0 0x7f20300a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.893+0000 7f20377fe640 1 -- 172.21.15.139:0/1262885654 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9642+0+0 (secure 0 0 0) 0x7f2048075380 con 0x7f20400a4320 2024-09-16T01:50:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.893+0000 7f204659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cb75e0 0x55b355cc1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.894+0000 7f204659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cb75e0 0x55b355cc1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:51786/0 (socket says 172.21.15.139:51786) 2024-09-16T01:50:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.894+0000 7f204659c640 1 -- 172.21.15.139:0/3905230721 learned_addr learned my addr 172.21.15.139:0/3905230721 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.894+0000 7f2046d9d640 1 --2- 172.21.15.139:0/1262885654 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f20300a35c0 0x7f20300a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.894+0000 7f2045d9b640 1 --2- 172.21.15.139:0/3905230721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cc1e80 0x55b355cc4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.894+0000 7f2045d9b640 1 --2- 172.21.15.139:0/3905230721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cb70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.899+0000 7f204659c640 1 -- 172.21.15.139:0/3905230721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cc1e80 msgr2=0x55b355cc4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.899+0000 7f204659c640 1 --2- 172.21.15.139:0/3905230721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cc1e80 0x55b355cc4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.899+0000 7f204659c640 1 -- 172.21.15.139:0/3905230721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 msgr2=0x55b355cb70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.899+0000 7f204659c640 1 --2- 172.21.15.139:0/3905230721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cb70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.899+0000 7f204659c640 1 -- 172.21.15.139:0/3905230721 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b355bc2340 con 0x55b355cb75e0 2024-09-16T01:50:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.899+0000 7f204659c640 1 --2- 172.21.15.139:0/3905230721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cb75e0 0x55b355cc1940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f203800bb00 tx=0x7f20380071c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50563a5d58cc1088 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.900+0000 7f2034ff9640 1 -- 172.21.15.139:0/3905230721 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038018030 con 0x55b355cb75e0 2024-09-16T01:50:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.900+0000 7f2034ff9640 1 -- 172.21.15.139:0/3905230721 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2038005500 con 0x55b355cb75e0 2024-09-16T01:50:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.900+0000 7f2034ff9640 1 -- 172.21.15.139:0/3905230721 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20380143f0 con 0x55b355cb75e0 2024-09-16T01:50:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.901+0000 7f204f9d7740 1 -- 172.21.15.139:0/3905230721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cb75e0 msgr2=0x55b355cc1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.901+0000 7f204f9d7740 1 --2- 172.21.15.139:0/3905230721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cb75e0 0x55b355cc1940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f203800bb00 tx=0x7f20380071c0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.901+0000 7f2046d9d640 1 --2- 172.21.15.139:0/1262885654 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f20300a35c0 0x7f20300a5a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2048050280 tx=0x7f20480504d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.906+0000 7f204f9d7740 1 -- 172.21.15.139:0/3905230721 shutdown_connections 2024-09-16T01:50:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.906+0000 7f204f9d7740 1 --2- 172.21.15.139:0/3905230721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cc1e80 0x55b355cc4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.906+0000 7f204f9d7740 1 --2- 172.21.15.139:0/3905230721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cb75e0 0x55b355cc1940 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.906+0000 7f204f9d7740 1 --2- 172.21.15.139:0/3905230721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cb70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.906+0000 7f204f9d7740 1 -- 172.21.15.139:0/3905230721 >> 172.21.15.139:0/3905230721 conn(0x55b355cb3230 msgr2=0x55b355cb3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.912+0000 7f204f9d7740 1 -- 172.21.15.139:0/3905230721 shutdown_connections 2024-09-16T01:50:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.914+0000 7f204f9d7740 1 -- 172.21.15.139:0/3905230721 wait complete. 2024-09-16T01:50:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.915+0000 7f204f9d7740 1 -- start start 2024-09-16T01:50:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.915+0000 7f204f9d7740 1 Processor -- start 2024-09-16T01:50:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.915+0000 7f204f9d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cbec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.915+0000 7f204f9d7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 0x55b355cbf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.915+0000 7f204f9d7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cc1e80 0x55b355cbf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.915+0000 7f204f9d7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b355cc4aa0 con 0x55b355cb6cc0 2024-09-16T01:50:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.915+0000 7f204f9d7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b355ba4290 con 0x55b355cb75e0 2024-09-16T01:50:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.915+0000 7f204f9d7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b355cc4c20 con 0x55b355cc1e80 2024-09-16T01:50:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.915+0000 7f204f9d7740 10 client.0 _ll_register_callbacks cb 0x55b355bd5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:50:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.916+0000 7f2045d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cbec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.916+0000 7f2045d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cbec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40480/0 (socket says 172.21.15.139:40480) 2024-09-16T01:50:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.916+0000 7f2045d9b640 1 -- 172.21.15.139:0/2106628141 learned_addr learned my addr 172.21.15.139:0/2106628141 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.916+0000 7f2045d9b640 1 --2- 172.21.15.139:0/2106628141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cc1e80 0x55b355cbf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.917+0000 7f204659c640 1 --2- 172.21.15.139:0/2106628141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 0x55b355cbf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.917+0000 7f2045d9b640 1 -- 172.21.15.139:0/2106628141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cc1e80 msgr2=0x55b355cbf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.917+0000 7f2045d9b640 1 --2- 172.21.15.139:0/2106628141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cc1e80 0x55b355cbf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.917+0000 7f2045d9b640 1 -- 172.21.15.139:0/2106628141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 msgr2=0x55b355cbf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.917+0000 7f2045d9b640 1 --2- 172.21.15.139:0/2106628141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 0x55b355cbf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.917+0000 7f2045d9b640 1 -- 172.21.15.139:0/2106628141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b35592e210 con 0x55b355cb6cc0 2024-09-16T01:50:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.917+0000 7f2045d9b640 1 --2- 172.21.15.139:0/2106628141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cbec40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f204000b770 tx=0x7f203c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a281cc4365251ca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.917+0000 7f202dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:50:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.918+0000 7f202dffb640 1 -- 172.21.15.139:0/2106628141 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c00f520 con 0x55b355cb6cc0 2024-09-16T01:50:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.918+0000 7f202dffb640 1 -- 172.21.15.139:0/2106628141 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9642+0+0 (secure 0 0 0) 0x7f203c00f6c0 con 0x55b355cb6cc0 2024-09-16T01:50:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.918+0000 7f204f9d7740 1 -- 172.21.15.139:0/2106628141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2038005500 con 0x55b355cb6cc0 2024-09-16T01:50:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.918+0000 7f202dffb640 1 -- 172.21.15.139:0/2106628141 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f203c053050 con 0x55b355cb6cc0 2024-09-16T01:50:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.918+0000 7f204f9d7740 10 client.16623 fetch_fsmap learned FSMap version 53 2024-09-16T01:50:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.918+0000 7f204f9d7740 1 -- 172.21.15.139:0/2106628141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55b3559fa620 con 0x55b355cb6cc0 2024-09-16T01:50:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.918+0000 7f202dffb640 1 -- 172.21.15.139:0/2106628141 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2038005500 con 0x55b355cb6cc0 2024-09-16T01:50:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.919+0000 7f202dffb640 1 -- 172.21.15.139:0/2106628141 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 53) ==== 8442+0+0 (secure 0 0 0) 0x7f203c041040 con 0x55b355cb6cc0 2024-09-16T01:50:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.919+0000 7f204f9d7740 10 client.16623 fetch_fsmap finished waiting for FSMap version 53 2024-09-16T01:50:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.919+0000 7f204f9d7740 10 client.16623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:50:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.919+0000 7f204f9d7740 20 client.16623 populate_metadata read hostname 'smithi139' 2024-09-16T01:50:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.919+0000 7f204f9d7740 1 --2- 172.21.15.139:0/2106628141 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b355cd1e10 0x55b355cd4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.919+0000 7f204f9d7740 4 client.16623 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:50:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.919+0000 7f204f9d7740 1 -- 172.21.15.139:0/2106628141 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b355c28a20 con 0x55b355cd1e10 2024-09-16T01:50:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.921+0000 7f204659c640 1 --2- 172.21.15.139:0/2106628141 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b355cd1e10 0x55b355cd4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.922+0000 7f202dffb640 10 client.16623 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.922+0000 7f204659c640 1 --2- 172.21.15.139:0/2106628141 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b355cd1e10 0x55b355cd4280 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24986c30136c7eb6 server_cookie=6794b50f89a4f779 in_seq=0 out_seq=0 2024-09-16T01:50:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.925+0000 7f202dffb640 1 -- 172.21.15.139:0/2106628141 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b355c28a20 con 0x55b355cd1e10 2024-09-16T01:50:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.925+0000 7f202dffb640 10 client.16623 handle_command_reply: tid=0 2024-09-16T01:50:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.926+0000 7f204f9d7740 1 client.16623 shutdown 2024-09-16T01:50:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.929+0000 7f204f9d7740 1 -- 172.21.15.139:0/2106628141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 msgr2=0x55b355cbec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.929+0000 7f204f9d7740 1 --2- 172.21.15.139:0/2106628141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cbec40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f204000b770 tx=0x7f203c00ec10 comp rx=0 tx=0).stop 2024-09-16T01:50:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.937+0000 7f204f9d7740 1 -- 172.21.15.139:0/2106628141 shutdown_connections 2024-09-16T01:50:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.937+0000 7f204f9d7740 1 --2- 172.21.15.139:0/2106628141 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b355cd1e10 0x55b355cd4280 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.937+0000 7f204f9d7740 1 --2- 172.21.15.139:0/2106628141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cc1e80 0x55b355cbf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.937+0000 7f204f9d7740 1 --2- 172.21.15.139:0/2106628141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 0x55b355cbf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.937+0000 7f204f9d7740 1 --2- 172.21.15.139:0/2106628141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cbec40 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.937+0000 7f204f9d7740 1 -- 172.21.15.139:0/2106628141 >> 172.21.15.139:0/2106628141 conn(0x55b355cb3230 msgr2=0x55b355cb8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.939+0000 7f202dffb640 0 client.16623 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.946+0000 7f204f9d7740 1 -- 172.21.15.139:0/2106628141 shutdown_connections 2024-09-16T01:50:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.947+0000 7f204f9d7740 1 -- 172.21.15.139:0/2106628141 wait complete. 2024-09-16T01:50:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.950+0000 7f204f9d7740 20 client.16623 trim_cache size 0 max 16384 2024-09-16T01:50:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.981+0000 7f204f9d7740 1 Processor -- start 2024-09-16T01:50:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.981+0000 7f204f9d7740 1 -- start start 2024-09-16T01:50:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.982+0000 7f204f9d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cbd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.982+0000 7f204f9d7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 0x55b355cbf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.982+0000 7f2045d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cbd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.982+0000 7f204f9d7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cc1e80 0x55b355cd9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.982+0000 7f204f9d7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2028010270 con 0x55b355cb6cc0 2024-09-16T01:50:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.982+0000 7f204f9d7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2028012080 con 0x55b355cb75e0 2024-09-16T01:50:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.982+0000 7f204f9d7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2028010a00 con 0x55b355cc1e80 2024-09-16T01:50:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.982+0000 7f204659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 0x55b355cbf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.982+0000 7f204659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 0x55b355cbf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:36412/0 (socket says 172.21.15.139:36412) 2024-09-16T01:50:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.982+0000 7f204659c640 1 -- 172.21.15.139:0/3536807606 learned_addr learned my addr 172.21.15.139:0/3536807606 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.983+0000 7f204659c640 1 -- 172.21.15.139:0/3536807606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cc1e80 msgr2=0x55b355cd9b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:50:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.983+0000 7f204659c640 1 --2- 172.21.15.139:0/3536807606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cc1e80 0x55b355cd9b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.983+0000 7f204659c640 1 -- 172.21.15.139:0/3536807606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 msgr2=0x55b355cbd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.983+0000 7f204659c640 1 --2- 172.21.15.139:0/3536807606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cbd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.983+0000 7f204659c640 1 -- 172.21.15.139:0/3536807606 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b355cb17b0 con 0x55b355cb75e0 2024-09-16T01:50:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.984+0000 7f204659c640 1 --2- 172.21.15.139:0/3536807606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 0x55b355cbf4f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2038004b60 tx=0x7f2038004c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=505f8da2211a2fde server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.984+0000 7f202ffff640 1 -- 172.21.15.139:0/3536807606 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038007e00 con 0x55b355cb75e0 2024-09-16T01:50:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.984+0000 7f202ffff640 1 -- 172.21.15.139:0/3536807606 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2038022070 con 0x55b355cb75e0 2024-09-16T01:50:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.984+0000 7f202ffff640 1 -- 172.21.15.139:0/3536807606 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038003e40 con 0x55b355cb75e0 2024-09-16T01:50:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.985+0000 7f204f9d7740 1 -- 172.21.15.139:0/3536807606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 msgr2=0x55b355cbf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.985+0000 7f204f9d7740 1 --2- 172.21.15.139:0/3536807606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 0x55b355cbf4f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2038004b60 tx=0x7f2038004c30 comp rx=0 tx=0).stop 2024-09-16T01:50:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.985+0000 7f204f9d7740 1 -- 172.21.15.139:0/3536807606 shutdown_connections 2024-09-16T01:50:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.985+0000 7f204f9d7740 1 --2- 172.21.15.139:0/3536807606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cc1e80 0x55b355cd9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.985+0000 7f204f9d7740 1 --2- 172.21.15.139:0/3536807606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 0x55b355cbf4f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.985+0000 7f204f9d7740 1 --2- 172.21.15.139:0/3536807606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cbd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.985+0000 7f204f9d7740 1 -- 172.21.15.139:0/3536807606 >> 172.21.15.139:0/3536807606 conn(0x55b355cb3230 msgr2=0x55b355cc0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.989+0000 7f204f9d7740 1 -- 172.21.15.139:0/3536807606 shutdown_connections 2024-09-16T01:50:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.989+0000 7f204f9d7740 1 -- 172.21.15.139:0/3536807606 wait complete. 2024-09-16T01:50:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.990+0000 7f204f9d7740 1 -- start start 2024-09-16T01:50:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.990+0000 7f204f9d7740 1 Processor -- start 2024-09-16T01:50:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.991+0000 7f204f9d7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.991+0000 7f204f9d7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 0x55b355cc2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.991+0000 7f204f9d7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cc1e80 0x55b355cc3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.991+0000 7f204f9d7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2028011920 con 0x55b355cb6cc0 2024-09-16T01:50:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.991+0000 7f2045d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.991+0000 7f204f9d7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2028011190 con 0x55b355cb75e0 2024-09-16T01:50:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.991+0000 7f204f9d7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f202800e270 con 0x55b355cc1e80 2024-09-16T01:50:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.991+0000 7f2045d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40488/0 (socket says 172.21.15.139:40488) 2024-09-16T01:50:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.991+0000 7f2045d9b640 1 -- 172.21.15.139:0/1253677294 learned_addr learned my addr 172.21.15.139:0/1253677294 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.991+0000 7f204f9d7740 10 client.0 _ll_register_callbacks cb 0x7f203c00ee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:50:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.991+0000 7f2045d9b640 1 --2- 172.21.15.139:0/1253677294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cc1e80 0x55b355cc3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.992+0000 7f204659c640 1 --2- 172.21.15.139:0/1253677294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 0x55b355cc2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.993+0000 7f2045d9b640 1 -- 172.21.15.139:0/1253677294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cc1e80 msgr2=0x55b355cc3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.993+0000 7f2045d9b640 1 --2- 172.21.15.139:0/1253677294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cc1e80 0x55b355cc3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.993+0000 7f2045d9b640 1 -- 172.21.15.139:0/1253677294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 msgr2=0x55b355cc2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.993+0000 7f2045d9b640 1 --2- 172.21.15.139:0/1253677294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 0x55b355cc2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.993+0000 7f2045d9b640 1 -- 172.21.15.139:0/1253677294 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b355cc8360 con 0x55b355cb6cc0 2024-09-16T01:50:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.993+0000 7f2045d9b640 1 --2- 172.21.15.139:0/1253677294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cca880 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f203c001eb0 tx=0x7f203c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed501c1b178db65d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.993+0000 7f202dffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:50:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.994+0000 7f202dffb640 1 -- 172.21.15.139:0/1253677294 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c0050f0 con 0x55b355cb6cc0 2024-09-16T01:50:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.994+0000 7f202dffb640 1 -- 172.21.15.139:0/1253677294 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9642+0+0 (secure 0 0 0) 0x7f203c005290 con 0x55b355cb6cc0 2024-09-16T01:50:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.995+0000 7f204f9d7740 1 -- 172.21.15.139:0/1253677294 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2038022070 con 0x55b355cb6cc0 2024-09-16T01:50:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.995+0000 7f202dffb640 1 -- 172.21.15.139:0/1253677294 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f203c00f320 con 0x55b355cb6cc0 2024-09-16T01:50:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.995+0000 7f204f9d7740 10 client.16626 fetch_fsmap learned FSMap version 53 2024-09-16T01:50:20.996 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.995+0000 7f204f9d7740 1 -- 172.21.15.139:0/1253677294 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55b355d3cd00 con 0x55b355cb6cc0 2024-09-16T01:50:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.997+0000 7f202dffb640 1 -- 172.21.15.139:0/1253677294 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2038022070 con 0x55b355cb6cc0 2024-09-16T01:50:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.997+0000 7f202dffb640 1 -- 172.21.15.139:0/1253677294 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 53) ==== 8442+0+0 (secure 0 0 0) 0x7f203c00e340 con 0x55b355cb6cc0 2024-09-16T01:50:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.997+0000 7f204f9d7740 10 client.16626 fetch_fsmap finished waiting for FSMap version 53 2024-09-16T01:50:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.997+0000 7f204f9d7740 10 client.16626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:50:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.997+0000 7f204f9d7740 20 client.16626 populate_metadata read hostname 'smithi139' 2024-09-16T01:50:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.997+0000 7f204f9d7740 1 --2- 172.21.15.139:0/1253677294 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b355d3d460 0x55b355d3f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.997+0000 7f204f9d7740 4 client.16626 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:50:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:20.997+0000 7f204f9d7740 1 -- 172.21.15.139:0/1253677294 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b355c22d70 con 0x55b355d3d460 2024-09-16T01:50:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:21.001+0000 7f204659c640 1 --2- 172.21.15.139:0/1253677294 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b355d3d460 0x55b355d3f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:21.001+0000 7f204659c640 1 --2- 172.21.15.139:0/1253677294 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b355d3d460 0x55b355d3f880 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0d5115365ae3f3 server_cookie=e950cc27c9d8855 in_seq=0 out_seq=0 2024-09-16T01:50:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:21.001+0000 7f202dffb640 10 client.16626 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:21.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:21 smithi139 ceph-mon[31175]: pgmap v309: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-09-16T01:50:21.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:21 smithi176 ceph-mon[35324]: pgmap v309: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-09-16T01:50:21.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:21 smithi164 ceph-mon[35121]: pgmap v309: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-09-16T01:50:23.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:23 smithi139 ceph-mon[31175]: pgmap v310: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-09-16T01:50:23.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:23 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:23.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:23 smithi176 ceph-mon[35324]: pgmap v310: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-09-16T01:50:23.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:23 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:23.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:23 smithi164 ceph-mon[35121]: pgmap v310: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-09-16T01:50:23.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:23 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:25.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:25 smithi139 ceph-mon[31175]: pgmap v311: 129 pgs: 16 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 73 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 327 op/s 2024-09-16T01:50:25.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:25 smithi176 ceph-mon[35324]: pgmap v311: 129 pgs: 16 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 73 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 327 op/s 2024-09-16T01:50:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:25 smithi164 ceph-mon[35121]: pgmap v311: 129 pgs: 16 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 73 active+clean; 2.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 327 op/s 2024-09-16T01:50:26.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.407+0000 7f202dffb640 1 -- 172.21.15.139:0/1253677294 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b355c22d70 con 0x55b355d3d460 2024-09-16T01:50:26.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.407+0000 7f202dffb640 10 client.16626 handle_command_reply: tid=0 2024-09-16T01:50:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.407+0000 7f204f9d7740 1 client.16626 shutdown 2024-09-16T01:50:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.407+0000 7f204f9d7740 1 -- 172.21.15.139:0/1253677294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 msgr2=0x55b355cca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.407+0000 7f204f9d7740 1 --2- 172.21.15.139:0/1253677294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cca880 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f203c001eb0 tx=0x7f203c000bc0 comp rx=0 tx=0).stop 2024-09-16T01:50:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.408+0000 7f204f9d7740 1 -- 172.21.15.139:0/1253677294 shutdown_connections 2024-09-16T01:50:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.408+0000 7f204f9d7740 1 --2- 172.21.15.139:0/1253677294 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b355d3d460 0x55b355d3f880 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.408+0000 7f204f9d7740 1 --2- 172.21.15.139:0/1253677294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b355cc1e80 0x55b355cc3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.408+0000 7f204f9d7740 1 --2- 172.21.15.139:0/1253677294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b355cb75e0 0x55b355cc2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.408+0000 7f202dffb640 0 client.16626 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.408+0000 7f204f9d7740 1 --2- 172.21.15.139:0/1253677294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b355cb6cc0 0x55b355cca880 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.408+0000 7f204f9d7740 1 -- 172.21.15.139:0/1253677294 >> 172.21.15.139:0/1253677294 conn(0x55b355cb3230 msgr2=0x55b355cb1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.408+0000 7f204f9d7740 1 -- 172.21.15.139:0/1253677294 shutdown_connections 2024-09-16T01:50:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.408+0000 7f204f9d7740 1 -- 172.21.15.139:0/1253677294 wait complete. 2024-09-16T01:50:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.408+0000 7f204f9d7740 20 client.16626 trim_cache size 0 max 16384 2024-09-16T01:50:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:50:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:50:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:50:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:50:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.409+0000 7f204759e640 1 -- 172.21.15.139:0/1262885654 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f20300a35c0 msgr2=0x7f20300a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.409+0000 7f204759e640 1 --2- 172.21.15.139:0/1262885654 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f20300a35c0 0x7f20300a5a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2048050280 tx=0x7f20480504d0 comp rx=0 tx=0).stop 2024-09-16T01:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.409+0000 7f204759e640 1 -- 172.21.15.139:0/1262885654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20400a4320 msgr2=0x7f20400a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.409+0000 7f204759e640 1 --2- 172.21.15.139:0/1262885654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20400a4320 0x7f20400a47d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2048065a40 tx=0x7f204806e430 comp rx=0 tx=0).stop 2024-09-16T01:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.409+0000 7f204759e640 1 -- 172.21.15.139:0/1262885654 shutdown_connections 2024-09-16T01:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.409+0000 7f204759e640 1 --2- 172.21.15.139:0/1262885654 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f20300a35c0 0x7f20300a5a80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.409+0000 7f204759e640 1 --2- 172.21.15.139:0/1262885654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20400a4320 0x7f20400a47d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.409+0000 7f204759e640 1 --2- 172.21.15.139:0/1262885654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2040099880 0x7f20400a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.409+0000 7f204759e640 1 --2- 172.21.15.139:0/1262885654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2040094b70 0x7f20400a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.409+0000 7f204759e640 1 -- 172.21.15.139:0/1262885654 >> 172.21.15.139:0/1262885654 conn(0x7f204000a1e0 msgr2=0x7f2040092800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.409+0000 7f204759e640 1 -- 172.21.15.139:0/1262885654 shutdown_connections 2024-09-16T01:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:26.409+0000 7f204759e640 1 -- 172.21.15.139:0/1262885654 wait complete. 2024-09-16T01:50:27.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:27 smithi139 ceph-mon[31175]: pgmap v312: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 870 op/s 2024-09-16T01:50:27.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:27 smithi176 ceph-mon[35324]: pgmap v312: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 870 op/s 2024-09-16T01:50:27.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:27 smithi164 ceph-mon[35121]: pgmap v312: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 870 op/s 2024-09-16T01:50:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:29 smithi139 ceph-mon[31175]: pgmap v313: 129 pgs: 12 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 85 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 1.31k op/s 2024-09-16T01:50:29.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:29 smithi176 ceph-mon[35324]: pgmap v313: 129 pgs: 12 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 85 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 1.31k op/s 2024-09-16T01:50:29.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:29 smithi164 ceph-mon[35121]: pgmap v313: 129 pgs: 12 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 85 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 1.31k op/s 2024-09-16T01:50:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:31 smithi139 ceph-mon[31175]: pgmap v314: 129 pgs: 10 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 90 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.1 MiB/s wr, 1.31k op/s 2024-09-16T01:50:31.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:31 smithi176 ceph-mon[35324]: pgmap v314: 129 pgs: 10 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 90 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.1 MiB/s wr, 1.31k op/s 2024-09-16T01:50:31.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:31 smithi164 ceph-mon[35121]: pgmap v314: 129 pgs: 10 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 90 active+clean; 2.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.1 MiB/s wr, 1.31k op/s 2024-09-16T01:50:33.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:33 smithi139 ceph-mon[31175]: pgmap v315: 129 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 106 active+clean; 2.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 857 KiB/s wr, 1.35k op/s 2024-09-16T01:50:33.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:33 smithi176 ceph-mon[35324]: pgmap v315: 129 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 106 active+clean; 2.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 857 KiB/s wr, 1.35k op/s 2024-09-16T01:50:33.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:33 smithi164 ceph-mon[35121]: pgmap v315: 129 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 106 active+clean; 2.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 857 KiB/s wr, 1.35k op/s 2024-09-16T01:50:35.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:35 smithi139 ceph-mon[31175]: pgmap v316: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 2.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 996 KiB/s wr, 1.34k op/s 2024-09-16T01:50:35.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:35 smithi176 ceph-mon[35324]: pgmap v316: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 2.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 996 KiB/s wr, 1.34k op/s 2024-09-16T01:50:35.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:35 smithi164 ceph-mon[35121]: pgmap v316: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 2.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 996 KiB/s wr, 1.34k op/s 2024-09-16T01:50:37.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:36 smithi164 ceph-mon[35121]: pgmap v317: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 911 KiB/s wr, 1.19k op/s 2024-09-16T01:50:37.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:36 smithi164 ceph-mon[35121]: osdmap e95: 12 total, 12 up, 12 in 2024-09-16T01:50:37.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:36 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:37.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:36 smithi139 ceph-mon[31175]: pgmap v317: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 911 KiB/s wr, 1.19k op/s 2024-09-16T01:50:37.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:36 smithi139 ceph-mon[31175]: osdmap e95: 12 total, 12 up, 12 in 2024-09-16T01:50:37.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:36 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:37.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:36 smithi176 ceph-mon[35324]: pgmap v317: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 911 KiB/s wr, 1.19k op/s 2024-09-16T01:50:37.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:36 smithi176 ceph-mon[35324]: osdmap e95: 12 total, 12 up, 12 in 2024-09-16T01:50:37.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:36 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:39.057 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:38 smithi139 ceph-mon[31175]: pgmap v319: 129 pgs: 129 active+clean; 2.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 316 op/s 2024-09-16T01:50:39.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:38 smithi164 ceph-mon[35121]: pgmap v319: 129 pgs: 129 active+clean; 2.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 316 op/s 2024-09-16T01:50:39.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:38 smithi176 ceph-mon[35324]: pgmap v319: 129 pgs: 129 active+clean; 2.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 316 op/s 2024-09-16T01:50:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.508+0000 7fae99a0a640 1 -- 172.21.15.139:0/1962603504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae941069c0 msgr2=0x7fae94106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.508+0000 7fae99a0a640 1 --2- 172.21.15.139:0/1962603504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae941069c0 0x7fae94106da0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fae84009890 tx=0x7fae8402a380 comp rx=0 tx=0).stop 2024-09-16T01:50:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.508+0000 7fae99a0a640 1 -- 172.21.15.139:0/1962603504 shutdown_connections 2024-09-16T01:50:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.508+0000 7fae99a0a640 1 --2- 172.21.15.139:0/1962603504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae94101350 0x7fae9410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.508+0000 7fae99a0a640 1 --2- 172.21.15.139:0/1962603504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae941009b0 0x7fae94100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.508+0000 7fae99a0a640 1 --2- 172.21.15.139:0/1962603504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae941069c0 0x7fae94106da0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.508+0000 7fae99a0a640 1 -- 172.21.15.139:0/1962603504 >> 172.21.15.139:0/1962603504 conn(0x7fae94077990 msgr2=0x7fae940fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.508+0000 7fae99a0a640 1 -- 172.21.15.139:0/1962603504 shutdown_connections 2024-09-16T01:50:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.508+0000 7fae99a0a640 1 -- 172.21.15.139:0/1962603504 wait complete. 2024-09-16T01:50:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.508+0000 7fae99a0a640 1 Processor -- start 2024-09-16T01:50:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.508+0000 7fae99a0a640 1 -- start start 2024-09-16T01:50:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae99a0a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae941009b0 0x7fae941991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae99a0a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae94101350 0x7fae941996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae99a0a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae941069c0 0x7fae941a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae99a0a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae94110e00 con 0x7fae94101350 2024-09-16T01:50:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae99a0a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae94110c80 con 0x7fae941009b0 2024-09-16T01:50:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae99a0a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fae94110f80 con 0x7fae941069c0 2024-09-16T01:50:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae927fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae94101350 0x7fae941996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae927fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae94101350 0x7fae941996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50832/0 (socket says 172.21.15.139:50832) 2024-09-16T01:50:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae927fc640 1 -- 172.21.15.139:0/780012220 learned_addr learned my addr 172.21.15.139:0/780012220 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae937fe640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae941069c0 0x7fae941a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae92ffd640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae941009b0 0x7fae941991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae92ffd640 1 -- 172.21.15.139:0/780012220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae941069c0 msgr2=0x7fae941a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae92ffd640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae941069c0 0x7fae941a4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae92ffd640 1 -- 172.21.15.139:0/780012220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae94101350 msgr2=0x7fae941996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae92ffd640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae94101350 0x7fae941996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae92ffd640 1 -- 172.21.15.139:0/780012220 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae941a4770 con 0x7fae941009b0 2024-09-16T01:50:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae927fc640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae94101350 0x7fae941996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:50:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae937fe640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae941069c0 0x7fae941a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:50:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.509+0000 7fae92ffd640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae941009b0 0x7fae941991b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fae840057c0 tx=0x7fae84033ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.510+0000 7fae98a08640 1 -- 172.21.15.139:0/780012220 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84037040 con 0x7fae941009b0 2024-09-16T01:50:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.510+0000 7fae99a0a640 1 -- 172.21.15.139:0/780012220 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae941a4940 con 0x7fae941009b0 2024-09-16T01:50:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.510+0000 7fae99a0a640 1 -- 172.21.15.139:0/780012220 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae941a4e00 con 0x7fae941009b0 2024-09-16T01:50:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.510+0000 7fae98a08640 1 -- 172.21.15.139:0/780012220 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae84033d90 con 0x7fae941009b0 2024-09-16T01:50:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.510+0000 7fae98a08640 1 -- 172.21.15.139:0/780012220 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84003920 con 0x7fae941009b0 2024-09-16T01:50:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.512+0000 7fae98a08640 1 -- 172.21.15.139:0/780012220 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fae84042020 con 0x7fae941009b0 2024-09-16T01:50:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.512+0000 7fae9c877740 1 Processor -- start 2024-09-16T01:50:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.512+0000 7fae9c877740 1 -- start start 2024-09-16T01:50:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.512+0000 7fae9c877740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872020faf0 0x55872020fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.512+0000 7fae9c877740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558720210410 0x55872021a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.512+0000 7fae9c877740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872021acb0 0x55872021d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.512+0000 7fae9c877740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5587201742f0 con 0x55872021acb0 2024-09-16T01:50:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.512+0000 7fae9c877740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55872011c270 con 0x558720210410 2024-09-16T01:50:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.512+0000 7fae9c877740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5587201702b0 con 0x55872020faf0 2024-09-16T01:50:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae937fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872020faf0 0x55872020fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae937fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872020faf0 0x55872020fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:49510/0 (socket says 172.21.15.139:49510) 2024-09-16T01:50:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae937fe640 1 -- 172.21.15.139:0/2423686456 learned_addr learned my addr 172.21.15.139:0/2423686456 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae927fc640 1 --2- 172.21.15.139:0/2423686456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872021acb0 0x55872021d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae92ffd640 1 --2- 172.21.15.139:0/2423686456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558720210410 0x55872021a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae98a08640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fae7c0a3460 0x7fae7c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae937fe640 1 -- 172.21.15.139:0/2423686456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558720210410 msgr2=0x55872021a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae937fe640 1 --2- 172.21.15.139:0/2423686456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558720210410 0x55872021a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae937fe640 1 -- 172.21.15.139:0/2423686456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872021acb0 msgr2=0x55872021d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae937fe640 1 --2- 172.21.15.139:0/2423686456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872021acb0 0x55872021d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae937fe640 1 -- 172.21.15.139:0/2423686456 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55872011b340 con 0x55872020faf0 2024-09-16T01:50:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae92ffd640 1 --2- 172.21.15.139:0/2423686456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558720210410 0x55872021a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:50:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae927fc640 1 --2- 172.21.15.139:0/2423686456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872021acb0 0x55872021d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:50:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae98a08640 1 -- 172.21.15.139:0/780012220 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9566+0+0 (secure 0 0 0) 0x7fae840ef710 con 0x7fae941009b0 2024-09-16T01:50:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae937fe640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fae7c0a3460 0x7fae7c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae937fe640 1 --2- 172.21.15.139:0/2423686456 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872020faf0 0x55872020fed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0118d0 tx=0x7fae8c011d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f488bdcb28a98aa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae79ffb640 1 -- 172.21.15.139:0/2423686456 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c017050 con 0x55872020faf0 2024-09-16T01:50:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae79ffb640 1 -- 172.21.15.139:0/2423686456 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae8c01bc10 con 0x55872020faf0 2024-09-16T01:50:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.513+0000 7fae79ffb640 1 -- 172.21.15.139:0/2423686456 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c01a650 con 0x55872020faf0 2024-09-16T01:50:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.514+0000 7fae937fe640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fae7c0a3460 0x7fae7c0a5920 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae8c023000 tx=0x7fae8c024000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.514+0000 7fae9c877740 1 -- 172.21.15.139:0/2423686456 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872020faf0 msgr2=0x55872020fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.514+0000 7fae9c877740 1 --2- 172.21.15.139:0/2423686456 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872020faf0 0x55872020fed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0118d0 tx=0x7fae8c011d90 comp rx=0 tx=0).stop 2024-09-16T01:50:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.514+0000 7fae9c877740 1 -- 172.21.15.139:0/2423686456 shutdown_connections 2024-09-16T01:50:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.514+0000 7fae9c877740 1 --2- 172.21.15.139:0/2423686456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872021acb0 0x55872021d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.514+0000 7fae9c877740 1 --2- 172.21.15.139:0/2423686456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558720210410 0x55872021a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.514+0000 7fae9c877740 1 --2- 172.21.15.139:0/2423686456 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872020faf0 0x55872020fed0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.514+0000 7fae9c877740 1 -- 172.21.15.139:0/2423686456 >> 172.21.15.139:0/2423686456 conn(0x55872020c060 msgr2=0x55872020c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.514+0000 7fae9c877740 1 -- 172.21.15.139:0/2423686456 shutdown_connections 2024-09-16T01:50:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.514+0000 7fae9c877740 1 -- 172.21.15.139:0/2423686456 wait complete. 2024-09-16T01:50:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.514+0000 7fae9c877740 1 -- start start 2024-09-16T01:50:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae9c877740 1 Processor -- start 2024-09-16T01:50:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae9c877740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872020faf0 0x55872021a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae9c877740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558720210410 0x558720217bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae9c877740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872021acb0 0x558720218100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae9c877740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55872021d880 con 0x55872021acb0 2024-09-16T01:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae9c877740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5587200fd290 con 0x558720210410 2024-09-16T01:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae9c877740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55872021da00 con 0x55872020faf0 2024-09-16T01:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae9c877740 10 client.0 _ll_register_callbacks cb 0x55872012e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae937fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872021acb0 0x558720218100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae92ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872020faf0 0x55872021a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae92ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872020faf0 0x55872021a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:49514/0 (socket says 172.21.15.139:49514) 2024-09-16T01:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae927fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558720210410 0x558720217bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae92ffd640 1 -- 172.21.15.139:0/1948076294 learned_addr learned my addr 172.21.15.139:0/1948076294 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae937fe640 1 --2- 172.21.15.139:0/1948076294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872021acb0 0x558720218100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50848/0 (socket says 172.21.15.139:50848) 2024-09-16T01:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae92ffd640 1 -- 172.21.15.139:0/1948076294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558720210410 msgr2=0x558720217bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae92ffd640 1 --2- 172.21.15.139:0/1948076294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558720210410 0x558720217bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae92ffd640 1 -- 172.21.15.139:0/1948076294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872021acb0 msgr2=0x558720218100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae92ffd640 1 --2- 172.21.15.139:0/1948076294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872021acb0 0x558720218100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.515+0000 7fae92ffd640 1 -- 172.21.15.139:0/1948076294 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871fe87210 con 0x55872020faf0 2024-09-16T01:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae92ffd640 1 --2- 172.21.15.139:0/1948076294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872020faf0 0x55872021a6a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fae8402a970 tx=0x7fae840308f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc0d17284ea5dd6a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae72ffd640 1 -- 172.21.15.139:0/1948076294 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae840bb720 con 0x55872020faf0 2024-09-16T01:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae9c877740 1 -- 172.21.15.139:0/1948076294 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae8c01bc10 con 0x55872020faf0 2024-09-16T01:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae72ffd640 1 -- 172.21.15.139:0/1948076294 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9566+0+0 (secure 0 0 0) 0x7fae840ecc70 con 0x55872020faf0 2024-09-16T01:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae72ffd640 1 -- 172.21.15.139:0/1948076294 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fae840ece70 con 0x55872020faf0 2024-09-16T01:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae9c877740 10 client.26044 fetch_fsmap learned FSMap version 55 2024-09-16T01:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae9c877740 1 -- 172.21.15.139:0/1948076294 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55871ff53620 con 0x55872020faf0 2024-09-16T01:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae72ffd640 1 -- 172.21.15.139:0/1948076294 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae8c01bc10 con 0x55872020faf0 2024-09-16T01:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae72ffd640 1 -- 172.21.15.139:0/1948076294 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 55) ==== 8442+0+0 (secure 0 0 0) 0x7fae8403a070 con 0x55872020faf0 2024-09-16T01:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae9c877740 10 client.26044 fetch_fsmap finished waiting for FSMap version 55 2024-09-16T01:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae9c877740 10 client.26044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae9c877740 20 client.26044 populate_metadata read hostname 'smithi139' 2024-09-16T01:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae9c877740 1 --2- 172.21.15.139:0/1948076294 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558720233400 0x558720235860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae9c877740 4 client.26044 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.516+0000 7fae9c877740 1 -- 172.21.15.139:0/1948076294 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558720181a20 con 0x558720233400 2024-09-16T01:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.517+0000 7fae927fc640 1 --2- 172.21.15.139:0/1948076294 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558720233400 0x558720235860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.517+0000 7fae72ffd640 10 client.26044 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.517+0000 7fae927fc640 1 --2- 172.21.15.139:0/1948076294 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558720233400 0x558720235860 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8135631c2b284a2c server_cookie=8b548ba209e69d4a in_seq=0 out_seq=0 2024-09-16T01:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.517+0000 7fae72ffd640 1 -- 172.21.15.139:0/1948076294 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558720181a20 con 0x558720233400 2024-09-16T01:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.517+0000 7fae72ffd640 10 client.26044 handle_command_reply: tid=0 2024-09-16T01:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.518+0000 7fae9c877740 1 client.26044 shutdown 2024-09-16T01:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.518+0000 7fae9c877740 1 -- 172.21.15.139:0/1948076294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872020faf0 msgr2=0x55872021a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.518+0000 7fae9c877740 1 --2- 172.21.15.139:0/1948076294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872020faf0 0x55872021a6a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fae8402a970 tx=0x7fae840308f0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.518+0000 7fae9c877740 1 -- 172.21.15.139:0/1948076294 shutdown_connections 2024-09-16T01:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.518+0000 7fae9c877740 1 --2- 172.21.15.139:0/1948076294 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558720233400 0x558720235860 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.518+0000 7fae9c877740 1 --2- 172.21.15.139:0/1948076294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872021acb0 0x558720218100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.518+0000 7fae9c877740 1 --2- 172.21.15.139:0/1948076294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558720210410 0x558720217bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.518+0000 7fae9c877740 1 --2- 172.21.15.139:0/1948076294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872020faf0 0x55872021a6a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.518+0000 7fae72ffd640 0 client.26044 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.518+0000 7fae9c877740 1 -- 172.21.15.139:0/1948076294 >> 172.21.15.139:0/1948076294 conn(0x55872020c060 msgr2=0x558720211d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.518+0000 7fae9c877740 1 -- 172.21.15.139:0/1948076294 shutdown_connections 2024-09-16T01:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.518+0000 7fae9c877740 1 -- 172.21.15.139:0/1948076294 wait complete. 2024-09-16T01:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.518+0000 7fae9c877740 20 client.26044 trim_cache size 0 max 16384 2024-09-16T01:50:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.533+0000 7fae9c877740 1 Processor -- start 2024-09-16T01:50:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.533+0000 7fae9c877740 1 -- start start 2024-09-16T01:50:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.533+0000 7fae9c877740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55872020faf0 0x5587202203a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.533+0000 7fae9c877740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558720210410 0x55872023b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.533+0000 7fae9c877740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872021acb0 0x55872023b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.533+0000 7fae9c877740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae941acfc0 con 0x558720210410 2024-09-16T01:50:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.533+0000 7fae9c877740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae941aeda0 con 0x55872020faf0 2024-09-16T01:50:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.533+0000 7fae9c877740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fae941ad750 con 0x55872021acb0 2024-09-16T01:50:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.533+0000 7fae927fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558720210410 0x55872023b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.533+0000 7fae927fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558720210410 0x55872023b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50850/0 (socket says 172.21.15.139:50850) 2024-09-16T01:50:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.533+0000 7fae937fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872021acb0 0x55872023b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.533+0000 7fae927fc640 1 -- 172.21.15.139:0/547059309 learned_addr learned my addr 172.21.15.139:0/547059309 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.533+0000 7fae92ffd640 1 --2- 172.21.15.139:0/547059309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55872020faf0 0x5587202203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.534+0000 7fae927fc640 1 -- 172.21.15.139:0/547059309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872021acb0 msgr2=0x55872023b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.534+0000 7fae927fc640 1 --2- 172.21.15.139:0/547059309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872021acb0 0x55872023b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.534+0000 7fae927fc640 1 -- 172.21.15.139:0/547059309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55872020faf0 msgr2=0x5587202203a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.534+0000 7fae927fc640 1 --2- 172.21.15.139:0/547059309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55872020faf0 0x5587202203a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.534+0000 7fae927fc640 1 -- 172.21.15.139:0/547059309 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55872021d760 con 0x558720210410 2024-09-16T01:50:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.534+0000 7fae927fc640 1 --2- 172.21.15.139:0/547059309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558720210410 0x55872023b170 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x558720212690 tx=0x7fae88009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8925dca004d9fe79 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.534+0000 7fae797fa640 1 -- 172.21.15.139:0/547059309 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88004230 con 0x558720210410 2024-09-16T01:50:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.534+0000 7fae797fa640 1 -- 172.21.15.139:0/547059309 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae88010dc0 con 0x558720210410 2024-09-16T01:50:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.534+0000 7fae797fa640 1 -- 172.21.15.139:0/547059309 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8801e3f0 con 0x558720210410 2024-09-16T01:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 -- 172.21.15.139:0/547059309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558720210410 msgr2=0x55872023b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 --2- 172.21.15.139:0/547059309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558720210410 0x55872023b170 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x558720212690 tx=0x7fae88009260 comp rx=0 tx=0).stop 2024-09-16T01:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 -- 172.21.15.139:0/547059309 shutdown_connections 2024-09-16T01:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 --2- 172.21.15.139:0/547059309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55872021acb0 0x55872023b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 --2- 172.21.15.139:0/547059309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558720210410 0x55872023b170 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 --2- 172.21.15.139:0/547059309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55872020faf0 0x5587202203a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 -- 172.21.15.139:0/547059309 >> 172.21.15.139:0/547059309 conn(0x55872020c060 msgr2=0x55872021fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 -- 172.21.15.139:0/547059309 shutdown_connections 2024-09-16T01:50:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 -- 172.21.15.139:0/547059309 wait complete. 2024-09-16T01:50:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 -- start start 2024-09-16T01:50:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 Processor -- start 2024-09-16T01:50:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872020faf0 0x55872022f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558720210410 0x55872022fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55872021acb0 0x5587202303e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae941ae670 con 0x55872020faf0 2024-09-16T01:50:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae941adee0 con 0x55872021acb0 2024-09-16T01:50:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fae941aafa0 con 0x558720210410 2024-09-16T01:50:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.535+0000 7fae9c877740 10 client.0 _ll_register_callbacks cb 0x7fae84005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:50:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae92ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872020faf0 0x55872022f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae937fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55872021acb0 0x5587202303e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae927fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558720210410 0x55872022fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae92ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872020faf0 0x55872022f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50854/0 (socket says 172.21.15.139:50854) 2024-09-16T01:50:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae92ffd640 1 -- 172.21.15.139:0/1987553245 learned_addr learned my addr 172.21.15.139:0/1987553245 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:50:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae92ffd640 1 -- 172.21.15.139:0/1987553245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558720210410 msgr2=0x55872022fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae92ffd640 1 --2- 172.21.15.139:0/1987553245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558720210410 0x55872022fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae92ffd640 1 -- 172.21.15.139:0/1987553245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55872021acb0 msgr2=0x5587202303e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:50:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae92ffd640 1 --2- 172.21.15.139:0/1987553245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55872021acb0 0x5587202303e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae92ffd640 1 -- 172.21.15.139:0/1987553245 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558720218470 con 0x55872020faf0 2024-09-16T01:50:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae92ffd640 1 --2- 172.21.15.139:0/1987553245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872020faf0 0x55872022f960 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fae840bb420 tx=0x7fae84005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2c387905aefeff server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:50:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:50:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae72ffd640 1 -- 172.21.15.139:0/1987553245 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae840bb4f0 con 0x55872020faf0 2024-09-16T01:50:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae72ffd640 1 -- 172.21.15.139:0/1987553245 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9566+0+0 (secure 0 0 0) 0x7fae840ecde0 con 0x55872020faf0 2024-09-16T01:50:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae9c877740 1 -- 172.21.15.139:0/1987553245 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae88010dc0 con 0x55872020faf0 2024-09-16T01:50:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae72ffd640 1 -- 172.21.15.139:0/1987553245 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fae840b79c0 con 0x55872020faf0 2024-09-16T01:50:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae9c877740 10 client.16644 fetch_fsmap learned FSMap version 55 2024-09-16T01:50:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.536+0000 7fae9c877740 1 -- 172.21.15.139:0/1987553245 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558720218470 con 0x55872020faf0 2024-09-16T01:50:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.537+0000 7fae72ffd640 1 -- 172.21.15.139:0/1987553245 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae88010dc0 con 0x55872020faf0 2024-09-16T01:50:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.537+0000 7fae72ffd640 1 -- 172.21.15.139:0/1987553245 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 55) ==== 8442+0+0 (secure 0 0 0) 0x7fae8403a070 con 0x55872020faf0 2024-09-16T01:50:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.537+0000 7fae9c877740 10 client.16644 fetch_fsmap finished waiting for FSMap version 55 2024-09-16T01:50:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.537+0000 7fae9c877740 10 client.16644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:50:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.537+0000 7fae9c877740 20 client.16644 populate_metadata read hostname 'smithi139' 2024-09-16T01:50:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.537+0000 7fae9c877740 1 --2- 172.21.15.139:0/1987553245 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55872029e930 0x5587202a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:50:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.537+0000 7fae9c877740 4 client.16644 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:50:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.537+0000 7fae9c877740 1 -- 172.21.15.139:0/1987553245 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55872017bd70 con 0x55872029e930 2024-09-16T01:50:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.537+0000 7fae927fc640 1 --2- 172.21.15.139:0/1987553245 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55872029e930 0x5587202a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:50:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.537+0000 7fae927fc640 1 --2- 172.21.15.139:0/1987553245 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55872029e930 0x5587202a0d50 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf543b2a88a33ef1 server_cookie=6baac0d6000af68b in_seq=0 out_seq=0 2024-09-16T01:50:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:39.537+0000 7fae72ffd640 10 client.16644 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:41.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:40 smithi164 ceph-mon[35121]: pgmap v320: 129 pgs: 129 active+clean; 2.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 309 op/s 2024-09-16T01:50:41.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:40 smithi164 ceph-mon[35121]: osdmap e96: 12 total, 12 up, 12 in 2024-09-16T01:50:41.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:40 smithi139 ceph-mon[31175]: pgmap v320: 129 pgs: 129 active+clean; 2.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 309 op/s 2024-09-16T01:50:41.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:40 smithi139 ceph-mon[31175]: osdmap e96: 12 total, 12 up, 12 in 2024-09-16T01:50:41.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:40 smithi176 ceph-mon[35324]: pgmap v320: 129 pgs: 129 active+clean; 2.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 309 op/s 2024-09-16T01:50:41.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:40 smithi176 ceph-mon[35324]: osdmap e96: 12 total, 12 up, 12 in 2024-09-16T01:50:43.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:42 smithi164 ceph-mon[35121]: pgmap v322: 129 pgs: 129 active+clean; 2.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 687 op/s 2024-09-16T01:50:43.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:42 smithi139 ceph-mon[31175]: pgmap v322: 129 pgs: 129 active+clean; 2.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 687 op/s 2024-09-16T01:50:43.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:42 smithi176 ceph-mon[35324]: pgmap v322: 129 pgs: 129 active+clean; 2.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 687 op/s 2024-09-16T01:50:44.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:44 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:50:44.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:44 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:50:44.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:44 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:50:45.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:45 smithi164 ceph-mon[35121]: pgmap v323: 129 pgs: 129 active+clean; 2.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 1.23k op/s 2024-09-16T01:50:45.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:45 smithi139 ceph-mon[31175]: pgmap v323: 129 pgs: 129 active+clean; 2.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 1.23k op/s 2024-09-16T01:50:45.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:45 smithi176 ceph-mon[35324]: pgmap v323: 129 pgs: 129 active+clean; 2.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 1.23k op/s 2024-09-16T01:50:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:47 smithi139 ceph-mon[31175]: pgmap v324: 129 pgs: 129 active+clean; 2.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 1.82k op/s 2024-09-16T01:50:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:47 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:47 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:47 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:47 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:47 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:47 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:47 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi164 to 176.0M 2024-09-16T01:50:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:47 smithi139 ceph-mon[31175]: Unable to set osd_memory_target on smithi164 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-16T01:50:47.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:47 smithi176 ceph-mon[35324]: pgmap v324: 129 pgs: 129 active+clean; 2.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 1.82k op/s 2024-09-16T01:50:47.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:47 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:47.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:47 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:47.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:47 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:47.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:47 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:47.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:47 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:47.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:47 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:47.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:47 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi164 to 176.0M 2024-09-16T01:50:47.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:47 smithi176 ceph-mon[35324]: Unable to set osd_memory_target on smithi164 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-16T01:50:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:47 smithi164 ceph-mon[35121]: pgmap v324: 129 pgs: 129 active+clean; 2.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 1.82k op/s 2024-09-16T01:50:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:47 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:47 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:47 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:47 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:47 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:47 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:47 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi164 to 176.0M 2024-09-16T01:50:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:47 smithi164 ceph-mon[35121]: Unable to set osd_memory_target on smithi164 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-16T01:50:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.311+0000 7fae72ffd640 1 -- 172.21.15.139:0/1987553245 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55872017bd70 con 0x55872029e930 2024-09-16T01:50:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.311+0000 7fae72ffd640 10 client.16644 handle_command_reply: tid=0 2024-09-16T01:50:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.311+0000 7fae9c877740 1 client.16644 shutdown 2024-09-16T01:50:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.311+0000 7fae9c877740 1 -- 172.21.15.139:0/1987553245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872020faf0 msgr2=0x55872022f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.311+0000 7fae9c877740 1 --2- 172.21.15.139:0/1987553245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872020faf0 0x55872022f960 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fae840bb420 tx=0x7fae84005bd0 comp rx=0 tx=0).stop 2024-09-16T01:50:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.312+0000 7fae9c877740 1 -- 172.21.15.139:0/1987553245 shutdown_connections 2024-09-16T01:50:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.312+0000 7fae9c877740 1 --2- 172.21.15.139:0/1987553245 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55872029e930 0x5587202a0d50 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.312+0000 7fae9c877740 1 --2- 172.21.15.139:0/1987553245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55872021acb0 0x5587202303e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.312+0000 7fae9c877740 1 --2- 172.21.15.139:0/1987553245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558720210410 0x55872022fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.312+0000 7fae9c877740 1 --2- 172.21.15.139:0/1987553245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55872020faf0 0x55872022f960 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.312+0000 7fae9c877740 1 -- 172.21.15.139:0/1987553245 >> 172.21.15.139:0/1987553245 conn(0x55872020c060 msgr2=0x558720217c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.312+0000 7fae72ffd640 0 client.16644 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:50:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.312+0000 7fae9c877740 1 -- 172.21.15.139:0/1987553245 shutdown_connections 2024-09-16T01:50:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.312+0000 7fae9c877740 1 -- 172.21.15.139:0/1987553245 wait complete. 2024-09-16T01:50:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.312+0000 7fae9c877740 20 client.16644 trim_cache size 0 max 16384 2024-09-16T01:50:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:50:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:50:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:50:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:50:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.313+0000 7fae99a0a640 1 -- 172.21.15.139:0/780012220 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fae7c0a3460 msgr2=0x7fae7c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.313+0000 7fae99a0a640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fae7c0a3460 0x7fae7c0a5920 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae8c023000 tx=0x7fae8c024000 comp rx=0 tx=0).stop 2024-09-16T01:50:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.313+0000 7fae99a0a640 1 -- 172.21.15.139:0/780012220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae941009b0 msgr2=0x7fae941991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:50:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.313+0000 7fae99a0a640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae941009b0 0x7fae941991b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fae840057c0 tx=0x7fae84033ae0 comp rx=0 tx=0).stop 2024-09-16T01:50:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.313+0000 7fae99a0a640 1 -- 172.21.15.139:0/780012220 shutdown_connections 2024-09-16T01:50:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.313+0000 7fae99a0a640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fae7c0a3460 0x7fae7c0a5920 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.313+0000 7fae99a0a640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae941069c0 0x7fae941a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.313+0000 7fae99a0a640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae94101350 0x7fae941996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.313+0000 7fae99a0a640 1 --2- 172.21.15.139:0/780012220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae941009b0 0x7fae941991b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:50:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.313+0000 7fae99a0a640 1 -- 172.21.15.139:0/780012220 >> 172.21.15.139:0/780012220 conn(0x7fae94077990 msgr2=0x7fae94103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:50:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.313+0000 7fae99a0a640 1 -- 172.21.15.139:0/780012220 shutdown_connections 2024-09-16T01:50:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:50:48.313+0000 7fae99a0a640 1 -- 172.21.15.139:0/780012220 wait complete. 2024-09-16T01:50:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:48 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:48 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:48 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:48 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:48 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:48.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:48 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:48.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:48 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:48.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:48 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:48.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:48 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:48.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:48 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:48.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:48 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:48.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:48 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:48.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:48 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:48.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:48 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:48.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:48 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:48.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:48 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:48.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:48 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:48.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:48 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:49.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:49 smithi164 ceph-mon[35121]: pgmap v325: 129 pgs: 129 active+clean; 2.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 2.16k op/s 2024-09-16T01:50:49.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:49 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi139 to 176.0M 2024-09-16T01:50:49.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:49 smithi164 ceph-mon[35121]: Unable to set osd_memory_target on smithi139 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-16T01:50:49.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:49 smithi139 ceph-mon[31175]: pgmap v325: 129 pgs: 129 active+clean; 2.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 2.16k op/s 2024-09-16T01:50:49.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:49 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi139 to 176.0M 2024-09-16T01:50:49.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:49 smithi139 ceph-mon[31175]: Unable to set osd_memory_target on smithi139 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-16T01:50:49.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:49 smithi176 ceph-mon[35324]: pgmap v325: 129 pgs: 129 active+clean; 2.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 2.16k op/s 2024-09-16T01:50:49.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:49 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi139 to 176.0M 2024-09-16T01:50:49.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:49 smithi176 ceph-mon[35324]: Unable to set osd_memory_target on smithi139 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-16T01:50:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:51 smithi164 ceph-mon[35121]: pgmap v326: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 2.33k op/s 2024-09-16T01:50:51.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:51 smithi139 ceph-mon[31175]: pgmap v326: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 2.33k op/s 2024-09-16T01:50:51.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:51 smithi176 ceph-mon[35324]: pgmap v326: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 2.33k op/s 2024-09-16T01:50:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:53 smithi164 ceph-mon[35121]: pgmap v327: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 2.07k op/s 2024-09-16T01:50:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:53 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:53 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:53 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:53 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:53 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:53 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:53 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:50:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:53 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:50:53.796 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:53 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:53 smithi139 ceph-mon[31175]: pgmap v327: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 2.07k op/s 2024-09-16T01:50:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:53 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:53 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:53 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:53 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:53 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:53 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:53 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:50:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:53 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:50:53.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:53 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:53.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:53 smithi176 ceph-mon[35324]: pgmap v327: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 2.07k op/s 2024-09-16T01:50:53.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:53 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:53.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:53 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:53.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:53 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:53.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:53 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:53.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:53 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:53.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:53 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T01:50:53.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:53 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:50:53.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:53 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:50:53.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:53 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:50:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:54 smithi164 ceph-mon[35121]: Adjusting osd_memory_target on smithi176 to 176.0M 2024-09-16T01:50:54.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:54 smithi164 ceph-mon[35121]: Unable to set osd_memory_target on smithi176 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-16T01:50:54.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:54 smithi139 ceph-mon[31175]: Adjusting osd_memory_target on smithi176 to 176.0M 2024-09-16T01:50:54.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:54 smithi139 ceph-mon[31175]: Unable to set osd_memory_target on smithi176 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-16T01:50:54.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:54 smithi176 ceph-mon[35324]: Adjusting osd_memory_target on smithi176 to 176.0M 2024-09-16T01:50:54.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:54 smithi176 ceph-mon[35324]: Unable to set osd_memory_target on smithi176 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-16T01:50:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:55 smithi164 ceph-mon[35121]: pgmap v328: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 1.94k op/s 2024-09-16T01:50:55.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:55 smithi139 ceph-mon[31175]: pgmap v328: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 1.94k op/s 2024-09-16T01:50:55.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:55 smithi176 ceph-mon[35324]: pgmap v328: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 1.94k op/s 2024-09-16T01:50:57.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:57 smithi164 ceph-mon[35121]: pgmap v329: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 994 KiB/s wr, 1.61k op/s 2024-09-16T01:50:57.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:57 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:57.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:57 smithi139 ceph-mon[31175]: pgmap v329: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 994 KiB/s wr, 1.61k op/s 2024-09-16T01:50:57.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:57 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:57.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:57 smithi176 ceph-mon[35324]: pgmap v329: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 994 KiB/s wr, 1.61k op/s 2024-09-16T01:50:57.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:57 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:50:59.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:50:59 smithi164 ceph-mon[35121]: pgmap v330: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 1.01k op/s 2024-09-16T01:50:59.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:50:59 smithi139 ceph-mon[31175]: pgmap v330: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 1.01k op/s 2024-09-16T01:50:59.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:50:59 smithi176 ceph-mon[35324]: pgmap v330: 129 pgs: 129 active+clean; 2.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 1.01k op/s 2024-09-16T01:51:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.403+0000 7fccb39e4640 1 -- 172.21.15.139:0/368613770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccac1035c0 msgr2=0x7fccac10baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.403+0000 7fccb39e4640 1 --2- 172.21.15.139:0/368613770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccac1035c0 0x7fccac10baa0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009900 tx=0x7fcc9c02a480 comp rx=0 tx=0).stop 2024-09-16T01:51:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.403+0000 7fccb39e4640 1 -- 172.21.15.139:0/368613770 shutdown_connections 2024-09-16T01:51:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.403+0000 7fccb39e4640 1 --2- 172.21.15.139:0/368613770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fccac10c0c0 0x7fccac10e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.403+0000 7fccb39e4640 1 --2- 172.21.15.139:0/368613770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccac1035c0 0x7fccac10baa0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.403+0000 7fccb39e4640 1 --2- 172.21.15.139:0/368613770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccac102ca0 0x7fccac103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.403+0000 7fccb39e4640 1 -- 172.21.15.139:0/368613770 >> 172.21.15.139:0/368613770 conn(0x7fccac0fc6f0 msgr2=0x7fccac0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.403+0000 7fccb39e4640 1 -- 172.21.15.139:0/368613770 shutdown_connections 2024-09-16T01:51:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.403+0000 7fccb39e4640 1 -- 172.21.15.139:0/368613770 wait complete. 2024-09-16T01:51:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.403+0000 7fccb39e4640 1 Processor -- start 2024-09-16T01:51:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.403+0000 7fccb39e4640 1 -- start start 2024-09-16T01:51:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb39e4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fccac102ca0 0x7fccac1008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb39e4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccac1035c0 0x7fccac100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb39e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccac10c0c0 0x7fccac101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb39e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fccac111ee0 con 0x7fccac10c0c0 2024-09-16T01:51:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb39e4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fccac111d60 con 0x7fccac1035c0 2024-09-16T01:51:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb39e4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fccac112060 con 0x7fccac102ca0 2024-09-16T01:51:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb1f81640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccac10c0c0 0x7fccac101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb0f7f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccac1035c0 0x7fccac100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb1780640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fccac102ca0 0x7fccac1008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb1f81640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccac10c0c0 0x7fccac101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47584/0 (socket says 172.21.15.139:47584) 2024-09-16T01:51:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb1f81640 1 -- 172.21.15.139:0/3534603422 learned_addr learned my addr 172.21.15.139:0/3534603422 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb1f81640 1 -- 172.21.15.139:0/3534603422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fccac102ca0 msgr2=0x7fccac1008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb1f81640 1 --2- 172.21.15.139:0/3534603422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fccac102ca0 0x7fccac1008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb1f81640 1 -- 172.21.15.139:0/3534603422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccac1035c0 msgr2=0x7fccac100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb1f81640 1 --2- 172.21.15.139:0/3534603422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccac1035c0 0x7fccac100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb1f81640 1 -- 172.21.15.139:0/3534603422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccac1a6810 con 0x7fccac10c0c0 2024-09-16T01:51:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb1f81640 1 --2- 172.21.15.139:0/3534603422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccac10c0c0 0x7fccac101370 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcca400eb10 tx=0x7fcca400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fcca27fc640 1 -- 172.21.15.139:0/3534603422 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca4004b20 con 0x7fccac10c0c0 2024-09-16T01:51:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fcca27fc640 1 -- 172.21.15.139:0/3534603422 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcca4004cc0 con 0x7fccac10c0c0 2024-09-16T01:51:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fccb39e4640 1 -- 172.21.15.139:0/3534603422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccac1a6b00 con 0x7fccac10c0c0 2024-09-16T01:51:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.404+0000 7fcca27fc640 1 -- 172.21.15.139:0/3534603422 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca4010760 con 0x7fccac10c0c0 2024-09-16T01:51:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.405+0000 7fccb39e4640 1 -- 172.21.15.139:0/3534603422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccac1a7020 con 0x7fccac10c0c0 2024-09-16T01:51:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.407+0000 7fcca27fc640 1 -- 172.21.15.139:0/3534603422 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fcca4005020 con 0x7fccac10c0c0 2024-09-16T01:51:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.407+0000 7fccb6858740 1 Processor -- start 2024-09-16T01:51:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.407+0000 7fccb6858740 1 -- start start 2024-09-16T01:51:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.407+0000 7fccb6858740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592ecaf0 0x5644592eced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.407+0000 7fccb6858740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ed410 0x5644592f7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.407+0000 7fccb6858740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592f7cb0 0x5644592fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.407+0000 7fccb6858740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5644592512f0 con 0x5644592ecaf0 2024-09-16T01:51:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.407+0000 7fccb6858740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5644591f9270 con 0x5644592f7cb0 2024-09-16T01:51:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.407+0000 7fccb6858740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56445924d2b0 con 0x5644592ed410 2024-09-16T01:51:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fcca27fc640 1 --2- 172.21.15.139:0/3534603422 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcc980a3460 0x7fcc980a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fccb0f7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592ecaf0 0x5644592eced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fccb1780640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ed410 0x5644592f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fccb0f7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592ecaf0 0x5644592eced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47598/0 (socket says 172.21.15.139:47598) 2024-09-16T01:51:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fccb0f7f640 1 -- 172.21.15.139:0/1927170910 learned_addr learned my addr 172.21.15.139:0/1927170910 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fccb1f81640 1 --2- 172.21.15.139:0/3534603422 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcc980a3460 0x7fcc980a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fccb1780640 1 -- 172.21.15.139:0/1927170910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592f7cb0 msgr2=0x5644592fa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fccb1780640 1 --2- 172.21.15.139:0/1927170910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592f7cb0 0x5644592fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fccb1780640 1 -- 172.21.15.139:0/1927170910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592ecaf0 msgr2=0x5644592eced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fccb1780640 1 --2- 172.21.15.139:0/1927170910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592ecaf0 0x5644592eced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fcca27fc640 1 -- 172.21.15.139:0/3534603422 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9538+0+0 (secure 0 0 0) 0x7fcca40d0750 con 0x7fccac10c0c0 2024-09-16T01:51:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fccb1780640 1 -- 172.21.15.139:0/1927170910 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644591f8340 con 0x5644592ed410 2024-09-16T01:51:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fccb0f7f640 1 --2- 172.21.15.139:0/1927170910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592ecaf0 0x5644592eced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:51:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fccb1f81640 1 --2- 172.21.15.139:0/3534603422 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcc980a3460 0x7fcc980a5920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcca4098a90 tx=0x7fcca406e3b0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fccb1780640 1 --2- 172.21.15.139:0/1927170910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ed410 0x5644592f7770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcca800bbf0 tx=0x7fcca80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c03301e628ba941f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fcc977fe640 1 -- 172.21.15.139:0/1927170910 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8004030 con 0x5644592ed410 2024-09-16T01:51:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fcc977fe640 1 -- 172.21.15.139:0/1927170910 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcca80041d0 con 0x5644592ed410 2024-09-16T01:51:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.408+0000 7fcc977fe640 1 -- 172.21.15.139:0/1927170910 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8011680 con 0x5644592ed410 2024-09-16T01:51:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.409+0000 7fccb6858740 1 -- 172.21.15.139:0/1927170910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ed410 msgr2=0x5644592f7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.409+0000 7fccb6858740 1 --2- 172.21.15.139:0/1927170910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ed410 0x5644592f7770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcca800bbf0 tx=0x7fcca80107a0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.409+0000 7fccb6858740 1 -- 172.21.15.139:0/1927170910 shutdown_connections 2024-09-16T01:51:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.409+0000 7fccb6858740 1 --2- 172.21.15.139:0/1927170910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592f7cb0 0x5644592fa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.409+0000 7fccb6858740 1 --2- 172.21.15.139:0/1927170910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ed410 0x5644592f7770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.409+0000 7fccb6858740 1 --2- 172.21.15.139:0/1927170910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592ecaf0 0x5644592eced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.409+0000 7fccb6858740 1 -- 172.21.15.139:0/1927170910 >> 172.21.15.139:0/1927170910 conn(0x5644592e9060 msgr2=0x5644592e9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.409+0000 7fccb6858740 1 -- 172.21.15.139:0/1927170910 shutdown_connections 2024-09-16T01:51:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.409+0000 7fccb6858740 1 -- 172.21.15.139:0/1927170910 wait complete. 2024-09-16T01:51:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.409+0000 7fccb6858740 1 -- start start 2024-09-16T01:51:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb6858740 1 Processor -- start 2024-09-16T01:51:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb6858740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592ecaf0 0x5644592f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb6858740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ed410 0x5644592f5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb6858740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592f7cb0 0x5644592f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb6858740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5644592fa8d0 con 0x5644592ecaf0 2024-09-16T01:51:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb6858740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5644591da290 con 0x5644592f7cb0 2024-09-16T01:51:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb6858740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5644592faa50 con 0x5644592ed410 2024-09-16T01:51:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb6858740 10 client.0 _ll_register_callbacks cb 0x56445920b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:51:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb0f7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592ecaf0 0x5644592f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb0f7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592ecaf0 0x5644592f4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47602/0 (socket says 172.21.15.139:47602) 2024-09-16T01:51:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb0f7f640 1 -- 172.21.15.139:0/1711481920 learned_addr learned my addr 172.21.15.139:0/1711481920 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb1780640 1 --2- 172.21.15.139:0/1711481920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ed410 0x5644592f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb0f7f640 1 --2- 172.21.15.139:0/1711481920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592f7cb0 0x5644592f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb0f7f640 1 -- 172.21.15.139:0/1711481920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ed410 msgr2=0x5644592f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb0f7f640 1 --2- 172.21.15.139:0/1711481920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ed410 0x5644592f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb0f7f640 1 -- 172.21.15.139:0/1711481920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592f7cb0 msgr2=0x5644592f55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb0f7f640 1 --2- 172.21.15.139:0/1711481920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592f7cb0 0x5644592f55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb0f7f640 1 -- 172.21.15.139:0/1711481920 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564458f64210 con 0x5644592ecaf0 2024-09-16T01:51:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fccb0f7f640 1 --2- 172.21.15.139:0/1711481920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592ecaf0 0x5644592f4b50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009230 tx=0x7fcc9c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5131688bc0fb798 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.410+0000 7fcc94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:51:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.411+0000 7fcc94ff9640 1 -- 172.21.15.139:0/1711481920 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c035040 con 0x5644592ecaf0 2024-09-16T01:51:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.411+0000 7fcc94ff9640 1 -- 172.21.15.139:0/1711481920 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9538+0+0 (secure 0 0 0) 0x7fcc9c004250 con 0x5644592ecaf0 2024-09-16T01:51:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.411+0000 7fccb6858740 1 -- 172.21.15.139:0/1711481920 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca80041d0 con 0x5644592ecaf0 2024-09-16T01:51:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.411+0000 7fcc94ff9640 1 -- 172.21.15.139:0/1711481920 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fcc9c04e050 con 0x5644592ecaf0 2024-09-16T01:51:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.411+0000 7fccb6858740 10 client.16656 fetch_fsmap learned FSMap version 56 2024-09-16T01:51:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.411+0000 7fccb6858740 1 -- 172.21.15.139:0/1711481920 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x564459030620 con 0x5644592ecaf0 2024-09-16T01:51:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.411+0000 7fcc94ff9640 1 -- 172.21.15.139:0/1711481920 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcca80041d0 con 0x5644592ecaf0 2024-09-16T01:51:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.411+0000 7fcc94ff9640 1 -- 172.21.15.139:0/1711481920 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 56) ==== 8438+0+0 (secure 0 0 0) 0x7fcc9c004b50 con 0x5644592ecaf0 2024-09-16T01:51:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.411+0000 7fccb6858740 10 client.16656 fetch_fsmap finished waiting for FSMap version 56 2024-09-16T01:51:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.411+0000 7fccb6858740 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:51:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.411+0000 7fccb6858740 20 client.16656 populate_metadata read hostname 'smithi139' 2024-09-16T01:51:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.411+0000 7fccb6858740 1 --2- 172.21.15.139:0/1711481920 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564459307d00 0x56445930a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.411+0000 7fccb6858740 4 client.16656 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:51:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.411+0000 7fccb6858740 1 -- 172.21.15.139:0/1711481920 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56445925ea20 con 0x564459307d00 2024-09-16T01:51:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.412+0000 7fccb1780640 1 --2- 172.21.15.139:0/1711481920 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564459307d00 0x56445930a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.412+0000 7fcc94ff9640 10 client.16656 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.412+0000 7fccb1780640 1 --2- 172.21.15.139:0/1711481920 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564459307d00 0x56445930a0f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11310ea912b260a server_cookie=596cfda81f4fee89 in_seq=0 out_seq=0 2024-09-16T01:51:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.412+0000 7fcc94ff9640 1 -- 172.21.15.139:0/1711481920 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56445925ea20 con 0x564459307d00 2024-09-16T01:51:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.412+0000 7fcc94ff9640 10 client.16656 handle_command_reply: tid=0 2024-09-16T01:51:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.413+0000 7fccb6858740 1 client.16656 shutdown 2024-09-16T01:51:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.413+0000 7fccb6858740 1 -- 172.21.15.139:0/1711481920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592ecaf0 msgr2=0x5644592f4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.413+0000 7fccb6858740 1 --2- 172.21.15.139:0/1711481920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592ecaf0 0x5644592f4b50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009230 tx=0x7fcc9c009560 comp rx=0 tx=0).stop 2024-09-16T01:51:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.413+0000 7fccb6858740 1 -- 172.21.15.139:0/1711481920 shutdown_connections 2024-09-16T01:51:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.413+0000 7fccb6858740 1 --2- 172.21.15.139:0/1711481920 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564459307d00 0x56445930a0f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.413+0000 7fccb6858740 1 --2- 172.21.15.139:0/1711481920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592f7cb0 0x5644592f55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.413+0000 7fccb6858740 1 --2- 172.21.15.139:0/1711481920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ed410 0x5644592f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.413+0000 7fcc94ff9640 0 client.16656 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.413+0000 7fccb6858740 1 --2- 172.21.15.139:0/1711481920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592ecaf0 0x5644592f4b50 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.413+0000 7fccb6858740 1 -- 172.21.15.139:0/1711481920 >> 172.21.15.139:0/1711481920 conn(0x5644592e9060 msgr2=0x5644592eedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.413+0000 7fccb6858740 1 -- 172.21.15.139:0/1711481920 shutdown_connections 2024-09-16T01:51:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.413+0000 7fccb6858740 1 -- 172.21.15.139:0/1711481920 wait complete. 2024-09-16T01:51:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.413+0000 7fccb6858740 20 client.16656 trim_cache size 0 max 16384 2024-09-16T01:51:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.428+0000 7fccb6858740 1 Processor -- start 2024-09-16T01:51:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.428+0000 7fccb6858740 1 -- start start 2024-09-16T01:51:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb6858740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ecaf0 0x5644592f5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb6858740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592ed410 0x56445930f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb6858740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592f7cb0 0x56445930fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb6858740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1b1320 con 0x5644592f7cb0 2024-09-16T01:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb6858740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1b30d0 con 0x5644592ed410 2024-09-16T01:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb6858740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1b1ab0 con 0x5644592ecaf0 2024-09-16T01:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb0f7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592f7cb0 0x56445930fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb0f7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592f7cb0 0x56445930fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47606/0 (socket says 172.21.15.139:47606) 2024-09-16T01:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb0f7f640 1 -- 172.21.15.139:0/651410884 learned_addr learned my addr 172.21.15.139:0/651410884 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb0f7f640 1 --2- 172.21.15.139:0/651410884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ecaf0 0x5644592f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb1780640 1 --2- 172.21.15.139:0/651410884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592ed410 0x56445930f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb0f7f640 1 -- 172.21.15.139:0/651410884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ecaf0 msgr2=0x5644592f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb0f7f640 1 --2- 172.21.15.139:0/651410884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ecaf0 0x5644592f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb0f7f640 1 -- 172.21.15.139:0/651410884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592ed410 msgr2=0x56445930f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb0f7f640 1 --2- 172.21.15.139:0/651410884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592ed410 0x56445930f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb0f7f640 1 -- 172.21.15.139:0/651410884 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644592e8e90 con 0x5644592f7cb0 2024-09-16T01:51:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fccb0f7f640 1 --2- 172.21.15.139:0/651410884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592f7cb0 0x56445930fe10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x5644592e8bd0 tx=0x7fcc9c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc529faad79a0a81 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fcc96ffd640 1 -- 172.21.15.139:0/651410884 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c035070 con 0x5644592f7cb0 2024-09-16T01:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fcc96ffd640 1 -- 172.21.15.139:0/651410884 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc9c004fa0 con 0x5644592f7cb0 2024-09-16T01:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.429+0000 7fcc96ffd640 1 -- 172.21.15.139:0/651410884 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c003d70 con 0x5644592f7cb0 2024-09-16T01:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.430+0000 7fccb6858740 1 -- 172.21.15.139:0/651410884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592f7cb0 msgr2=0x56445930fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.430+0000 7fccb6858740 1 --2- 172.21.15.139:0/651410884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592f7cb0 0x56445930fe10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x5644592e8bd0 tx=0x7fcc9c000bc0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.430+0000 7fccb6858740 1 -- 172.21.15.139:0/651410884 shutdown_connections 2024-09-16T01:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.430+0000 7fccb6858740 1 --2- 172.21.15.139:0/651410884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592f7cb0 0x56445930fe10 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.430+0000 7fccb6858740 1 --2- 172.21.15.139:0/651410884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592ed410 0x56445930f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.430+0000 7fccb6858740 1 --2- 172.21.15.139:0/651410884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ecaf0 0x5644592f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.430+0000 7fccb6858740 1 -- 172.21.15.139:0/651410884 >> 172.21.15.139:0/651410884 conn(0x5644592e9060 msgr2=0x5644592f6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.430+0000 7fccb6858740 1 -- 172.21.15.139:0/651410884 shutdown_connections 2024-09-16T01:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.430+0000 7fccb6858740 1 -- 172.21.15.139:0/651410884 wait complete. 2024-09-16T01:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.430+0000 7fccb6858740 1 -- start start 2024-09-16T01:51:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb6858740 1 Processor -- start 2024-09-16T01:51:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb6858740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ecaf0 0x564459300540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb6858740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592ed410 0x5644592f1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb6858740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592f7cb0 0x5644592f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb6858740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1b29d0 con 0x5644592f7cb0 2024-09-16T01:51:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb6858740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1b2240 con 0x5644592ed410 2024-09-16T01:51:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb6858740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1af320 con 0x5644592ecaf0 2024-09-16T01:51:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb6858740 10 client.0 _ll_register_callbacks cb 0x7fcc9c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:51:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb0f7f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ecaf0 0x564459300540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb1780640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592ed410 0x5644592f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb0f7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592f7cb0 0x5644592f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb0f7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592f7cb0 0x5644592f1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47614/0 (socket says 172.21.15.139:47614) 2024-09-16T01:51:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb0f7f640 1 -- 172.21.15.139:0/184089849 learned_addr learned my addr 172.21.15.139:0/184089849 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb1780640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592ed410 0x5644592f1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:51760/0 (socket says 172.21.15.139:51760) 2024-09-16T01:51:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb0f7f640 1 -- 172.21.15.139:0/184089849 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ecaf0 msgr2=0x564459300540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb0f7f640 1 --2- 172.21.15.139:0/184089849 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ecaf0 0x564459300540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb0f7f640 1 -- 172.21.15.139:0/184089849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592ed410 msgr2=0x5644592f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb0f7f640 1 --2- 172.21.15.139:0/184089849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592ed410 0x5644592f1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb0f7f640 1 -- 172.21.15.139:0/184089849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644592e42d0 con 0x5644592f7cb0 2024-09-16T01:51:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.431+0000 7fccb1780640 1 --2- 172.21.15.139:0/184089849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592ed410 0x5644592f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:51:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fccb0f7f640 1 --2- 172.21.15.139:0/184089849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592f7cb0 0x5644592f1fb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c004f20 tx=0x7fcc9c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cccea2a6f397ae server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fcc94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:51:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fcc94ff9640 1 -- 172.21.15.139:0/184089849 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c057070 con 0x5644592f7cb0 2024-09-16T01:51:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fcc94ff9640 1 -- 172.21.15.139:0/184089849 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9538+0+0 (secure 0 0 0) 0x7fcc9c005350 con 0x5644592f7cb0 2024-09-16T01:51:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fccb6858740 1 -- 172.21.15.139:0/184089849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc9c004fa0 con 0x5644592f7cb0 2024-09-16T01:51:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fcc94ff9640 1 -- 172.21.15.139:0/184089849 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fcc9c035040 con 0x5644592f7cb0 2024-09-16T01:51:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fccb6858740 10 client.16662 fetch_fsmap learned FSMap version 56 2024-09-16T01:51:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fccb6858740 1 -- 172.21.15.139:0/184089849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5644593728a0 con 0x5644592f7cb0 2024-09-16T01:51:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fcc94ff9640 1 -- 172.21.15.139:0/184089849 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc9c004fa0 con 0x5644592f7cb0 2024-09-16T01:51:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fcc94ff9640 1 -- 172.21.15.139:0/184089849 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 56) ==== 8438+0+0 (secure 0 0 0) 0x7fcc9c062500 con 0x5644592f7cb0 2024-09-16T01:51:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fccb6858740 10 client.16662 fetch_fsmap finished waiting for FSMap version 56 2024-09-16T01:51:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fccb6858740 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:51:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fccb6858740 20 client.16662 populate_metadata read hostname 'smithi139' 2024-09-16T01:51:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fccb6858740 1 --2- 172.21.15.139:0/184089849 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564459372e40 0x564459375260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fccb6858740 4 client.16662 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:51:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fccb6858740 1 -- 172.21.15.139:0/184089849 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564459258d70 con 0x564459372e40 2024-09-16T01:51:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.432+0000 7fccb1780640 1 --2- 172.21.15.139:0/184089849 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564459372e40 0x564459375260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.433+0000 7fcc94ff9640 10 client.16662 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.433+0000 7fccb1780640 1 --2- 172.21.15.139:0/184089849 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564459372e40 0x564459375260 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49e46c743f8807ec server_cookie=90452457159765ad in_seq=0 out_seq=0 2024-09-16T01:51:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fcc94ff9640 1 -- 172.21.15.139:0/184089849 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564459258d70 con 0x564459372e40 2024-09-16T01:51:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fcc94ff9640 10 client.16662 handle_command_reply: tid=0 2024-09-16T01:51:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fccb6858740 1 client.16662 shutdown 2024-09-16T01:51:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fccb6858740 1 -- 172.21.15.139:0/184089849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592f7cb0 msgr2=0x5644592f1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fccb6858740 1 --2- 172.21.15.139:0/184089849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592f7cb0 0x5644592f1fb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c004f20 tx=0x7fcc9c005c00 comp rx=0 tx=0).stop 2024-09-16T01:51:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fccb6858740 1 -- 172.21.15.139:0/184089849 shutdown_connections 2024-09-16T01:51:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fccb6858740 1 --2- 172.21.15.139:0/184089849 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564459372e40 0x564459375260 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fccb6858740 1 --2- 172.21.15.139:0/184089849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5644592f7cb0 0x5644592f1fb0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fcc94ff9640 0 client.16662 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fccb6858740 1 --2- 172.21.15.139:0/184089849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644592ed410 0x5644592f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fccb6858740 1 --2- 172.21.15.139:0/184089849 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5644592ecaf0 0x564459300540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fccb6858740 1 -- 172.21.15.139:0/184089849 >> 172.21.15.139:0/184089849 conn(0x5644592e9060 msgr2=0x5644592e79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fccb6858740 1 -- 172.21.15.139:0/184089849 shutdown_connections 2024-09-16T01:51:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fccb6858740 1 -- 172.21.15.139:0/184089849 wait complete. 2024-09-16T01:51:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.450+0000 7fccb6858740 20 client.16662 trim_cache size 0 max 16384 2024-09-16T01:51:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:51:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:51:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:51:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:51:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.451+0000 7fccb39e4640 1 -- 172.21.15.139:0/3534603422 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcc980a3460 msgr2=0x7fcc980a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.451+0000 7fccb39e4640 1 --2- 172.21.15.139:0/3534603422 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcc980a3460 0x7fcc980a5920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcca4098a90 tx=0x7fcca406e3b0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.451+0000 7fccb39e4640 1 -- 172.21.15.139:0/3534603422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccac10c0c0 msgr2=0x7fccac101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.451+0000 7fccb39e4640 1 --2- 172.21.15.139:0/3534603422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccac10c0c0 0x7fccac101370 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcca400eb10 tx=0x7fcca400c2d0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.451+0000 7fccb39e4640 1 -- 172.21.15.139:0/3534603422 shutdown_connections 2024-09-16T01:51:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.451+0000 7fccb39e4640 1 --2- 172.21.15.139:0/3534603422 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcc980a3460 0x7fcc980a5920 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.451+0000 7fccb39e4640 1 --2- 172.21.15.139:0/3534603422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccac10c0c0 0x7fccac101370 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.451+0000 7fccb39e4640 1 --2- 172.21.15.139:0/3534603422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccac1035c0 0x7fccac100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.451+0000 7fccb39e4640 1 --2- 172.21.15.139:0/3534603422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fccac102ca0 0x7fccac1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.451+0000 7fccb39e4640 1 -- 172.21.15.139:0/3534603422 >> 172.21.15.139:0/3534603422 conn(0x7fccac0fc6f0 msgr2=0x7fccac0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.451+0000 7fccb39e4640 1 -- 172.21.15.139:0/3534603422 shutdown_connections 2024-09-16T01:51:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:01.451+0000 7fccb39e4640 1 -- 172.21.15.139:0/3534603422 wait complete. 2024-09-16T01:51:01.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:01 smithi164 ceph-mon[35121]: pgmap v331: 129 pgs: 129 active+clean; 2.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 648 op/s 2024-09-16T01:51:01.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:01 smithi139 ceph-mon[31175]: pgmap v331: 129 pgs: 129 active+clean; 2.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 648 op/s 2024-09-16T01:51:01.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:01 smithi176 ceph-mon[35324]: pgmap v331: 129 pgs: 129 active+clean; 2.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 648 op/s 2024-09-16T01:51:03.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:03 smithi164 ceph-mon[35121]: pgmap v332: 129 pgs: 129 active+clean; 2.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 532 op/s 2024-09-16T01:51:03.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:03 smithi139 ceph-mon[31175]: pgmap v332: 129 pgs: 129 active+clean; 2.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 532 op/s 2024-09-16T01:51:03.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:03 smithi176 ceph-mon[35324]: pgmap v332: 129 pgs: 129 active+clean; 2.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 532 op/s 2024-09-16T01:51:05.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:05 smithi164 ceph-mon[35121]: pgmap v333: 129 pgs: 129 active+clean; 2.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 438 op/s 2024-09-16T01:51:05.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:05 smithi139 ceph-mon[31175]: pgmap v333: 129 pgs: 129 active+clean; 2.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 438 op/s 2024-09-16T01:51:05.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:05 smithi176 ceph-mon[35324]: pgmap v333: 129 pgs: 129 active+clean; 2.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 438 op/s 2024-09-16T01:51:07.727 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:07 smithi139 ceph-mon[31175]: pgmap v334: 129 pgs: 129 active+clean; 2.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 415 op/s 2024-09-16T01:51:07.727 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:07 smithi139 ceph-mon[31175]: osdmap e97: 12 total, 12 up, 12 in 2024-09-16T01:51:07.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:07 smithi176 ceph-mon[35324]: pgmap v334: 129 pgs: 129 active+clean; 2.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 415 op/s 2024-09-16T01:51:07.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:07 smithi176 ceph-mon[35324]: osdmap e97: 12 total, 12 up, 12 in 2024-09-16T01:51:08.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:07 smithi164 ceph-mon[35121]: pgmap v334: 129 pgs: 129 active+clean; 2.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 415 op/s 2024-09-16T01:51:08.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:07 smithi164 ceph-mon[35121]: osdmap e97: 12 total, 12 up, 12 in 2024-09-16T01:51:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:09 smithi139 ceph-mon[31175]: pgmap v336: 129 pgs: 6 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 99 active+clean; 2.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 382 op/s 2024-09-16T01:51:09.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:09 smithi176 ceph-mon[35324]: pgmap v336: 129 pgs: 6 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 99 active+clean; 2.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 382 op/s 2024-09-16T01:51:10.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:09 smithi164 ceph-mon[35121]: pgmap v336: 129 pgs: 6 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 99 active+clean; 2.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 382 op/s 2024-09-16T01:51:11.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:11 smithi139 ceph-mon[31175]: pgmap v337: 129 pgs: 12 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 80 active+clean; 2.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 379 op/s 2024-09-16T01:51:11.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:11 smithi176 ceph-mon[35324]: pgmap v337: 129 pgs: 12 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 80 active+clean; 2.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 379 op/s 2024-09-16T01:51:12.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:11 smithi164 ceph-mon[35121]: pgmap v337: 129 pgs: 12 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 80 active+clean; 2.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 379 op/s 2024-09-16T01:51:13.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:13 smithi139 ceph-mon[31175]: pgmap v338: 129 pgs: 23 active+clean+snaptrim, 71 active+clean+snaptrim_wait, 35 active+clean; 2.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 378 op/s 2024-09-16T01:51:13.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:13 smithi176 ceph-mon[35324]: pgmap v338: 129 pgs: 23 active+clean+snaptrim, 71 active+clean+snaptrim_wait, 35 active+clean; 2.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 378 op/s 2024-09-16T01:51:14.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:13 smithi164 ceph-mon[35121]: pgmap v338: 129 pgs: 23 active+clean+snaptrim, 71 active+clean+snaptrim_wait, 35 active+clean; 2.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 378 op/s 2024-09-16T01:51:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.599+0000 7fa44269b640 1 -- 172.21.15.139:0/644834622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa43c1091f0 msgr2=0x7fa43c110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.599+0000 7fa44269b640 1 --2- 172.21.15.139:0/644834622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa43c1091f0 0x7fa43c110250 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009890 tx=0x7fa42c02a3b0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.601+0000 7fa44269b640 1 -- 172.21.15.139:0/644834622 shutdown_connections 2024-09-16T01:51:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.601+0000 7fa44269b640 1 --2- 172.21.15.139:0/644834622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa43c11b140 0x7fa43c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.601+0000 7fa44269b640 1 --2- 172.21.15.139:0/644834622 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa43c1095d0 0x7fa43c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.601+0000 7fa44269b640 1 --2- 172.21.15.139:0/644834622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa43c1091f0 0x7fa43c110250 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.601+0000 7fa44269b640 1 -- 172.21.15.139:0/644834622 >> 172.21.15.139:0/644834622 conn(0x7fa43c06ce00 msgr2=0x7fa43c06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.605+0000 7fa44269b640 1 -- 172.21.15.139:0/644834622 shutdown_connections 2024-09-16T01:51:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.606+0000 7fa44269b640 1 -- 172.21.15.139:0/644834622 wait complete. 2024-09-16T01:51:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.607+0000 7fa44269b640 1 Processor -- start 2024-09-16T01:51:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.609+0000 7fa44269b640 1 -- start start 2024-09-16T01:51:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.609+0000 7fa44269b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa43c1091f0 0x7fa43c11a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.609+0000 7fa44269b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa43c1095d0 0x7fa43c116960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.609+0000 7fa44269b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa43c11b140 0x7fa43c116ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.609+0000 7fa44269b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1132f0 con 0x7fa43c11b140 2024-09-16T01:51:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.609+0000 7fa44269b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c113170 con 0x7fa43c1095d0 2024-09-16T01:51:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.609+0000 7fa44269b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c113470 con 0x7fa43c1091f0 2024-09-16T01:51:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.609+0000 7fa440c38640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa43c11b140 0x7fa43c116ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.609+0000 7fa440c38640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa43c11b140 0x7fa43c116ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54778/0 (socket says 172.21.15.139:54778) 2024-09-16T01:51:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.609+0000 7fa440c38640 1 -- 172.21.15.139:0/3785518122 learned_addr learned my addr 172.21.15.139:0/3785518122 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.609+0000 7fa43bfff640 1 --2- 172.21.15.139:0/3785518122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa43c1091f0 0x7fa43c11a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.611+0000 7fa43bfff640 1 -- 172.21.15.139:0/3785518122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa43c1095d0 msgr2=0x7fa43c116960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:51:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.611+0000 7fa43bfff640 1 --2- 172.21.15.139:0/3785518122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa43c1095d0 0x7fa43c116960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.611+0000 7fa43bfff640 1 -- 172.21.15.139:0/3785518122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa43c11b140 msgr2=0x7fa43c116ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.611+0000 7fa43bfff640 1 --2- 172.21.15.139:0/3785518122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa43c11b140 0x7fa43c116ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.611+0000 7fa43bfff640 1 -- 172.21.15.139:0/3785518122 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c1175d0 con 0x7fa43c1091f0 2024-09-16T01:51:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.613+0000 7fa43bfff640 1 --2- 172.21.15.139:0/3785518122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa43c1091f0 0x7fa43c11a880 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa42c02a8d0 tx=0x7fa42c033ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.617+0000 7fa439ffb640 1 -- 172.21.15.139:0/3785518122 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c037040 con 0x7fa43c1091f0 2024-09-16T01:51:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.617+0000 7fa439ffb640 1 -- 172.21.15.139:0/3785518122 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa42c033d90 con 0x7fa43c1091f0 2024-09-16T01:51:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.617+0000 7fa439ffb640 1 -- 172.21.15.139:0/3785518122 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c003a50 con 0x7fa43c1091f0 2024-09-16T01:51:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.620+0000 7fa44269b640 1 -- 172.21.15.139:0/3785518122 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43c1ab070 con 0x7fa43c1091f0 2024-09-16T01:51:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.620+0000 7fa44269b640 1 -- 172.21.15.139:0/3785518122 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43c1ab420 con 0x7fa43c1091f0 2024-09-16T01:51:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.625+0000 7fa445433740 1 Processor -- start 2024-09-16T01:51:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.626+0000 7fa439ffb640 1 -- 172.21.15.139:0/3785518122 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa42c030750 con 0x7fa43c1091f0 2024-09-16T01:51:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.627+0000 7fa439ffb640 1 --2- 172.21.15.139:0/3785518122 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa41c0a3640 0x7fa41c0a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.627+0000 7fa439ffb640 1 -- 172.21.15.139:0/3785518122 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9642+0+0 (secure 0 0 0) 0x7fa42c0efc70 con 0x7fa43c1091f0 2024-09-16T01:51:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.628+0000 7fa440c38640 1 --2- 172.21.15.139:0/3785518122 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa41c0a3640 0x7fa41c0a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.628+0000 7fa445433740 1 -- start start 2024-09-16T01:51:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.629+0000 7fa445433740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25aaf0 0x55742d25aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.629+0000 7fa445433740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d25b410 0x55742d265770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.629+0000 7fa445433740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d265cb0 0x55742d2680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.629+0000 7fa445433740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55742d1bf2f0 con 0x55742d25b410 2024-09-16T01:51:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.629+0000 7fa445433740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55742d167270 con 0x55742d265cb0 2024-09-16T01:51:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.629+0000 7fa445433740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55742d1bb2b0 con 0x55742d25aaf0 2024-09-16T01:51:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.630+0000 7fa43bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25aaf0 0x55742d25aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.632+0000 7fa43bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25aaf0 0x55742d25aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:39222/0 (socket says 172.21.15.139:39222) 2024-09-16T01:51:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.632+0000 7fa43bfff640 1 -- 172.21.15.139:0/4008064663 learned_addr learned my addr 172.21.15.139:0/4008064663 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.633+0000 7fa433fff640 1 --2- 172.21.15.139:0/4008064663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d25b410 0x55742d265770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.633+0000 7fa440c38640 1 --2- 172.21.15.139:0/4008064663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d265cb0 0x55742d2680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.633+0000 7fa440c38640 1 --2- 172.21.15.139:0/3785518122 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa41c0a3640 0x7fa41c0a5b00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa43401e500 tx=0x7fa434021040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.633+0000 7fa43bfff640 1 -- 172.21.15.139:0/4008064663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d265cb0 msgr2=0x55742d2680a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.633+0000 7fa43bfff640 1 --2- 172.21.15.139:0/4008064663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d265cb0 0x55742d2680a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.633+0000 7fa43bfff640 1 -- 172.21.15.139:0/4008064663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d25b410 msgr2=0x55742d265770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.633+0000 7fa43bfff640 1 --2- 172.21.15.139:0/4008064663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d25b410 0x55742d265770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.633+0000 7fa43bfff640 1 -- 172.21.15.139:0/4008064663 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742d166340 con 0x55742d25aaf0 2024-09-16T01:51:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.640+0000 7fa43bfff640 1 --2- 172.21.15.139:0/4008064663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25aaf0 0x55742d25aed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0c42e0 tx=0x7fa42c005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dced8cf0ebcd83c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.642+0000 7fa4327fc640 1 -- 172.21.15.139:0/4008064663 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c0bc2a0 con 0x55742d25aaf0 2024-09-16T01:51:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.643+0000 7fa4327fc640 1 -- 172.21.15.139:0/4008064663 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa42c0bbba0 con 0x55742d25aaf0 2024-09-16T01:51:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.643+0000 7fa4327fc640 1 -- 172.21.15.139:0/4008064663 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c003bf0 con 0x55742d25aaf0 2024-09-16T01:51:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.644+0000 7fa445433740 1 -- 172.21.15.139:0/4008064663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25aaf0 msgr2=0x55742d25aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.644+0000 7fa445433740 1 --2- 172.21.15.139:0/4008064663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25aaf0 0x55742d25aed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0c42e0 tx=0x7fa42c005f10 comp rx=0 tx=0).stop 2024-09-16T01:51:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.651+0000 7fa445433740 1 -- 172.21.15.139:0/4008064663 shutdown_connections 2024-09-16T01:51:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.651+0000 7fa445433740 1 --2- 172.21.15.139:0/4008064663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d265cb0 0x55742d2680a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.651+0000 7fa445433740 1 --2- 172.21.15.139:0/4008064663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d25b410 0x55742d265770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.651+0000 7fa445433740 1 --2- 172.21.15.139:0/4008064663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25aaf0 0x55742d25aed0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.651+0000 7fa445433740 1 -- 172.21.15.139:0/4008064663 >> 172.21.15.139:0/4008064663 conn(0x55742d257060 msgr2=0x55742d257440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.656+0000 7fa445433740 1 -- 172.21.15.139:0/4008064663 shutdown_connections 2024-09-16T01:51:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.657+0000 7fa445433740 1 -- 172.21.15.139:0/4008064663 wait complete. 2024-09-16T01:51:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.658+0000 7fa445433740 1 -- start start 2024-09-16T01:51:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.658+0000 7fa445433740 1 Processor -- start 2024-09-16T01:51:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.658+0000 7fa445433740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d25aaf0 0x55742d262ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.659+0000 7fa445433740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d2630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.659+0000 7fa445433740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d265cb0 0x55742d263620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.659+0000 7fa445433740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55742d268640 con 0x55742d265cb0 2024-09-16T01:51:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.659+0000 7fa445433740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55742d148290 con 0x55742d25aaf0 2024-09-16T01:51:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.659+0000 7fa445433740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55742d2687c0 con 0x55742d25b410 2024-09-16T01:51:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.659+0000 7fa445433740 10 client.0 _ll_register_callbacks cb 0x55742d179140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:51:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.659+0000 7fa433fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d2630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.659+0000 7fa433fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d2630e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:39228/0 (socket says 172.21.15.139:39228) 2024-09-16T01:51:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.659+0000 7fa433fff640 1 -- 172.21.15.139:0/3634712330 learned_addr learned my addr 172.21.15.139:0/3634712330 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.659+0000 7fa440c38640 1 --2- 172.21.15.139:0/3634712330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d265cb0 0x55742d263620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.660+0000 7fa433fff640 1 -- 172.21.15.139:0/3634712330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d25aaf0 msgr2=0x55742d262ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:51:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.660+0000 7fa433fff640 1 --2- 172.21.15.139:0/3634712330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d25aaf0 0x55742d262ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.660+0000 7fa433fff640 1 -- 172.21.15.139:0/3634712330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d265cb0 msgr2=0x55742d263620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.660+0000 7fa433fff640 1 --2- 172.21.15.139:0/3634712330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d265cb0 0x55742d263620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.660+0000 7fa433fff640 1 -- 172.21.15.139:0/3634712330 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742ced2210 con 0x55742d25b410 2024-09-16T01:51:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.665+0000 7fa433fff640 1 --2- 172.21.15.139:0/3634712330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d2630e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa42400bf90 tx=0x7fa42400d5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9520de2eed6ee15 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.666+0000 7fa42b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:51:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.667+0000 7fa42b7fe640 1 -- 172.21.15.139:0/3634712330 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424014430 con 0x55742d25b410 2024-09-16T01:51:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.668+0000 7fa445433740 1 -- 172.21.15.139:0/3634712330 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa42c0bbba0 con 0x55742d25b410 2024-09-16T01:51:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.668+0000 7fa42b7fe640 1 -- 172.21.15.139:0/3634712330 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9642+0+0 (secure 0 0 0) 0x7fa4240145d0 con 0x55742d25b410 2024-09-16T01:51:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.668+0000 7fa42b7fe640 1 -- 172.21.15.139:0/3634712330 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fa424022050 con 0x55742d25b410 2024-09-16T01:51:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.672+0000 7fa445433740 10 client.26080 fetch_fsmap learned FSMap version 56 2024-09-16T01:51:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.672+0000 7fa445433740 1 -- 172.21.15.139:0/3634712330 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55742cf9e620 con 0x55742d25b410 2024-09-16T01:51:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.678+0000 7fa42b7fe640 1 -- 172.21.15.139:0/3634712330 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa42c0bbba0 con 0x55742d25b410 2024-09-16T01:51:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.678+0000 7fa42b7fe640 1 -- 172.21.15.139:0/3634712330 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 56) ==== 8438+0+0 (secure 0 0 0) 0x7fa424013070 con 0x55742d25b410 2024-09-16T01:51:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.679+0000 7fa445433740 10 client.26080 fetch_fsmap finished waiting for FSMap version 56 2024-09-16T01:51:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.679+0000 7fa445433740 10 client.26080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:51:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.679+0000 7fa445433740 20 client.26080 populate_metadata read hostname 'smithi139' 2024-09-16T01:51:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.679+0000 7fa445433740 1 --2- 172.21.15.139:0/3634712330 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55742d275a00 0x55742d277e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.679+0000 7fa445433740 4 client.26080 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:51:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.679+0000 7fa445433740 1 -- 172.21.15.139:0/3634712330 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55742d1cca20 con 0x55742d275a00 2024-09-16T01:51:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.687+0000 7fa43bfff640 1 --2- 172.21.15.139:0/3634712330 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55742d275a00 0x55742d277e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.700+0000 7fa43bfff640 1 --2- 172.21.15.139:0/3634712330 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55742d275a00 0x55742d277e70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8adaf430ae41c17e server_cookie=98bcffa1dd74330b in_seq=0 out_seq=0 2024-09-16T01:51:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.701+0000 7fa42b7fe640 10 client.26080 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.704+0000 7fa42b7fe640 1 -- 172.21.15.139:0/3634712330 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55742d1cca20 con 0x55742d275a00 2024-09-16T01:51:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.704+0000 7fa42b7fe640 10 client.26080 handle_command_reply: tid=0 2024-09-16T01:51:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.704+0000 7fa445433740 1 client.26080 shutdown 2024-09-16T01:51:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.713+0000 7fa445433740 1 -- 172.21.15.139:0/3634712330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 msgr2=0x55742d2630e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.713+0000 7fa445433740 1 --2- 172.21.15.139:0/3634712330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d2630e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa42400bf90 tx=0x7fa42400d5f0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.716+0000 7fa445433740 1 -- 172.21.15.139:0/3634712330 shutdown_connections 2024-09-16T01:51:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.716+0000 7fa445433740 1 --2- 172.21.15.139:0/3634712330 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55742d275a00 0x55742d277e70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.716+0000 7fa445433740 1 --2- 172.21.15.139:0/3634712330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d265cb0 0x55742d263620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.716+0000 7fa445433740 1 --2- 172.21.15.139:0/3634712330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d2630e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.716+0000 7fa445433740 1 --2- 172.21.15.139:0/3634712330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d25aaf0 0x55742d262ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.716+0000 7fa445433740 1 -- 172.21.15.139:0/3634712330 >> 172.21.15.139:0/3634712330 conn(0x55742d257060 msgr2=0x55742d25cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.719+0000 7fa42b7fe640 0 client.26080 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.720+0000 7fa445433740 1 -- 172.21.15.139:0/3634712330 shutdown_connections 2024-09-16T01:51:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.721+0000 7fa445433740 1 -- 172.21.15.139:0/3634712330 wait complete. 2024-09-16T01:51:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.723+0000 7fa445433740 20 client.26080 trim_cache size 0 max 16384 2024-09-16T01:51:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.752+0000 7fa445433740 1 Processor -- start 2024-09-16T01:51:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.754+0000 7fa445433740 1 -- start start 2024-09-16T01:51:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.754+0000 7fa445433740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d25aaf0 0x55742d263450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.754+0000 7fa445433740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d25ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.754+0000 7fa445433740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d265cb0 0x55742d27d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.754+0000 7fa445433740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1b59e0 con 0x55742d265cb0 2024-09-16T01:51:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.754+0000 7fa445433740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1b7790 con 0x55742d25aaf0 2024-09-16T01:51:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.754+0000 7fa445433740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1b6170 con 0x55742d25b410 2024-09-16T01:51:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.755+0000 7fa433fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d25ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.755+0000 7fa433fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d25ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:39236/0 (socket says 172.21.15.139:39236) 2024-09-16T01:51:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.755+0000 7fa433fff640 1 -- 172.21.15.139:0/372564045 learned_addr learned my addr 172.21.15.139:0/372564045 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.755+0000 7fa43bfff640 1 --2- 172.21.15.139:0/372564045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d25aaf0 0x55742d263450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.755+0000 7fa433fff640 1 -- 172.21.15.139:0/372564045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d25aaf0 msgr2=0x55742d263450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.755+0000 7fa433fff640 1 --2- 172.21.15.139:0/372564045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d25aaf0 0x55742d263450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.755+0000 7fa433fff640 1 -- 172.21.15.139:0/372564045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d265cb0 msgr2=0x55742d27d740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:51:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.755+0000 7fa433fff640 1 --2- 172.21.15.139:0/372564045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d265cb0 0x55742d27d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.755+0000 7fa433fff640 1 -- 172.21.15.139:0/372564045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742d25a920 con 0x55742d25b410 2024-09-16T01:51:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.755+0000 7fa433fff640 1 --2- 172.21.15.139:0/372564045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d25ccb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa424000c00 tx=0x7fa424006e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c7c4c5356fd2982 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.758+0000 7fa431ffb640 1 -- 172.21.15.139:0/372564045 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42400b2d0 con 0x55742d25b410 2024-09-16T01:51:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.758+0000 7fa431ffb640 1 -- 172.21.15.139:0/372564045 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa424019070 con 0x55742d25b410 2024-09-16T01:51:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.758+0000 7fa431ffb640 1 -- 172.21.15.139:0/372564045 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424012950 con 0x55742d25b410 2024-09-16T01:51:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.761+0000 7fa445433740 1 -- 172.21.15.139:0/372564045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 msgr2=0x55742d25ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.761+0000 7fa445433740 1 --2- 172.21.15.139:0/372564045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d25ccb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa424000c00 tx=0x7fa424006e10 comp rx=0 tx=0).stop 2024-09-16T01:51:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.768+0000 7fa445433740 1 -- 172.21.15.139:0/372564045 shutdown_connections 2024-09-16T01:51:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.768+0000 7fa445433740 1 --2- 172.21.15.139:0/372564045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d265cb0 0x55742d27d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.768+0000 7fa445433740 1 --2- 172.21.15.139:0/372564045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d25ccb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.768+0000 7fa445433740 1 --2- 172.21.15.139:0/372564045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d25aaf0 0x55742d263450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.768+0000 7fa445433740 1 -- 172.21.15.139:0/372564045 >> 172.21.15.139:0/372564045 conn(0x55742d257060 msgr2=0x55742d25c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.771+0000 7fa445433740 1 -- 172.21.15.139:0/372564045 shutdown_connections 2024-09-16T01:51:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.771+0000 7fa445433740 1 -- 172.21.15.139:0/372564045 wait complete. 2024-09-16T01:51:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.772+0000 7fa445433740 1 -- start start 2024-09-16T01:51:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.772+0000 7fa445433740 1 Processor -- start 2024-09-16T01:51:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.772+0000 7fa445433740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d25aaf0 0x55742d26da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.772+0000 7fa445433740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d26df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.772+0000 7fa445433740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d265cb0 0x55742d25ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.772+0000 7fa445433740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1b7090 con 0x55742d25aaf0 2024-09-16T01:51:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.772+0000 7fa445433740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1b6900 con 0x55742d265cb0 2024-09-16T01:51:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.772+0000 7fa445433740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1b39e0 con 0x55742d25b410 2024-09-16T01:51:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.772+0000 7fa445433740 10 client.0 _ll_register_callbacks cb 0x7fa42400bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:51:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.773+0000 7fa440c38640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d265cb0 0x55742d25ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.773+0000 7fa440c38640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d265cb0 0x55742d25ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:50594/0 (socket says 172.21.15.139:50594) 2024-09-16T01:51:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.773+0000 7fa440c38640 1 -- 172.21.15.139:0/2335987651 learned_addr learned my addr 172.21.15.139:0/2335987651 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.773+0000 7fa440c38640 1 -- 172.21.15.139:0/2335987651 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 msgr2=0x55742d26df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.773+0000 7fa440c38640 1 --2- 172.21.15.139:0/2335987651 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d26df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.773+0000 7fa440c38640 1 -- 172.21.15.139:0/2335987651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d25aaf0 msgr2=0x55742d26da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.773+0000 7fa440c38640 1 --2- 172.21.15.139:0/2335987651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d25aaf0 0x55742d26da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.773+0000 7fa440c38640 1 -- 172.21.15.139:0/2335987651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742d25f450 con 0x55742d265cb0 2024-09-16T01:51:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.774+0000 7fa440c38640 1 --2- 172.21.15.139:0/2335987651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d265cb0 0x55742d25ef10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa43401dea0 tx=0x7fa434004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd09622571f25e6f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.774+0000 7fa42b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:51:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.776+0000 7fa42b7fe640 1 -- 172.21.15.139:0/2335987651 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43403b030 con 0x55742d265cb0 2024-09-16T01:51:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.776+0000 7fa42b7fe640 1 -- 172.21.15.139:0/2335987651 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9642+0+0 (secure 0 0 0) 0x7fa434036650 con 0x55742d265cb0 2024-09-16T01:51:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.778+0000 7fa445433740 1 -- 172.21.15.139:0/2335987651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa424019070 con 0x55742d265cb0 2024-09-16T01:51:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.779+0000 7fa42b7fe640 1 -- 172.21.15.139:0/2335987651 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fa43402ae60 con 0x55742d265cb0 2024-09-16T01:51:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.784+0000 7fa445433740 10 client.26096 fetch_fsmap learned FSMap version 56 2024-09-16T01:51:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.784+0000 7fa445433740 1 -- 172.21.15.139:0/2335987651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55742d2dfdd0 con 0x55742d265cb0 2024-09-16T01:51:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.784+0000 7fa42b7fe640 1 -- 172.21.15.139:0/2335987651 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa424019070 con 0x55742d265cb0 2024-09-16T01:51:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.784+0000 7fa42b7fe640 1 -- 172.21.15.139:0/2335987651 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 56) ==== 8438+0+0 (secure 0 0 0) 0x7fa434037ac0 con 0x55742d265cb0 2024-09-16T01:51:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.785+0000 7fa445433740 10 client.26096 fetch_fsmap finished waiting for FSMap version 56 2024-09-16T01:51:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.785+0000 7fa445433740 10 client.26096 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:51:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.785+0000 7fa445433740 20 client.26096 populate_metadata read hostname 'smithi139' 2024-09-16T01:51:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.785+0000 7fa445433740 1 --2- 172.21.15.139:0/2335987651 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55742d2e0580 0x55742d2e2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.785+0000 7fa445433740 4 client.26096 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:51:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.785+0000 7fa445433740 1 -- 172.21.15.139:0/2335987651 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55742d1c6d70 con 0x55742d2e0580 2024-09-16T01:51:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.786+0000 7fa43bfff640 1 --2- 172.21.15.139:0/2335987651 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55742d2e0580 0x55742d2e2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.795+0000 7fa43bfff640 1 --2- 172.21.15.139:0/2335987651 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55742d2e0580 0x55742d2e2a00 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a2659f64f4c722 server_cookie=e0c65ba7fa40ed9d in_seq=0 out_seq=0 2024-09-16T01:51:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.795+0000 7fa42b7fe640 10 client.26096 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.944+0000 7fa42b7fe640 1 -- 172.21.15.139:0/2335987651 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55742d1c6d70 con 0x55742d2e0580 2024-09-16T01:51:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.944+0000 7fa42b7fe640 10 client.26096 handle_command_reply: tid=0 2024-09-16T01:51:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.944+0000 7fa445433740 1 client.26096 shutdown 2024-09-16T01:51:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.945+0000 7fa445433740 1 -- 172.21.15.139:0/2335987651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d265cb0 msgr2=0x55742d25ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.945+0000 7fa445433740 1 --2- 172.21.15.139:0/2335987651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d265cb0 0x55742d25ef10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa43401dea0 tx=0x7fa434004a40 comp rx=0 tx=0).stop 2024-09-16T01:51:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.945+0000 7fa445433740 1 -- 172.21.15.139:0/2335987651 shutdown_connections 2024-09-16T01:51:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.945+0000 7fa445433740 1 --2- 172.21.15.139:0/2335987651 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55742d2e0580 0x55742d2e2a00 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.945+0000 7fa445433740 1 --2- 172.21.15.139:0/2335987651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55742d265cb0 0x55742d25ef10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.945+0000 7fa445433740 1 --2- 172.21.15.139:0/2335987651 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742d25b410 0x55742d26df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.945+0000 7fa42b7fe640 0 client.26096 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.945+0000 7fa445433740 1 --2- 172.21.15.139:0/2335987651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55742d25aaf0 0x55742d26da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.945+0000 7fa445433740 1 -- 172.21.15.139:0/2335987651 >> 172.21.15.139:0/2335987651 conn(0x55742d257060 msgr2=0x55742d25c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.945+0000 7fa445433740 1 -- 172.21.15.139:0/2335987651 shutdown_connections 2024-09-16T01:51:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.945+0000 7fa445433740 1 -- 172.21.15.139:0/2335987651 wait complete. 2024-09-16T01:51:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.945+0000 7fa445433740 20 client.26096 trim_cache size 0 max 16384 2024-09-16T01:51:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:51:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:51:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:51:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:51:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.946+0000 7fa44269b640 1 -- 172.21.15.139:0/3785518122 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa41c0a3640 msgr2=0x7fa41c0a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.946+0000 7fa44269b640 1 --2- 172.21.15.139:0/3785518122 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa41c0a3640 0x7fa41c0a5b00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa43401e500 tx=0x7fa434021040 comp rx=0 tx=0).stop 2024-09-16T01:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.946+0000 7fa44269b640 1 -- 172.21.15.139:0/3785518122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa43c1091f0 msgr2=0x7fa43c11a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.946+0000 7fa44269b640 1 --2- 172.21.15.139:0/3785518122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa43c1091f0 0x7fa43c11a880 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa42c02a8d0 tx=0x7fa42c033ae0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.946+0000 7fa44269b640 1 -- 172.21.15.139:0/3785518122 shutdown_connections 2024-09-16T01:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.946+0000 7fa44269b640 1 --2- 172.21.15.139:0/3785518122 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa41c0a3640 0x7fa41c0a5b00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.946+0000 7fa44269b640 1 --2- 172.21.15.139:0/3785518122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa43c11b140 0x7fa43c116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.946+0000 7fa44269b640 1 --2- 172.21.15.139:0/3785518122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa43c1095d0 0x7fa43c116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.946+0000 7fa44269b640 1 --2- 172.21.15.139:0/3785518122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa43c1091f0 0x7fa43c11a880 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.946+0000 7fa44269b640 1 -- 172.21.15.139:0/3785518122 >> 172.21.15.139:0/3785518122 conn(0x7fa43c06ce00 msgr2=0x7fa43c071bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.946+0000 7fa44269b640 1 -- 172.21.15.139:0/3785518122 shutdown_connections 2024-09-16T01:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:14.946+0000 7fa44269b640 1 -- 172.21.15.139:0/3785518122 wait complete. 2024-09-16T01:51:15.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:15 smithi139 ceph-mon[31175]: pgmap v339: 129 pgs: 23 active+clean+snaptrim, 71 active+clean+snaptrim_wait, 35 active+clean; 2.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 338 op/s 2024-09-16T01:51:15.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:15 smithi176 ceph-mon[35324]: pgmap v339: 129 pgs: 23 active+clean+snaptrim, 71 active+clean+snaptrim_wait, 35 active+clean; 2.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 338 op/s 2024-09-16T01:51:16.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:15 smithi164 ceph-mon[35121]: pgmap v339: 129 pgs: 23 active+clean+snaptrim, 71 active+clean+snaptrim_wait, 35 active+clean; 2.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 338 op/s 2024-09-16T01:51:17.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:17 smithi139 ceph-mon[31175]: pgmap v340: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 256 op/s 2024-09-16T01:51:17.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:17 smithi176 ceph-mon[35324]: pgmap v340: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 256 op/s 2024-09-16T01:51:18.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:17 smithi164 ceph-mon[35121]: pgmap v340: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 256 op/s 2024-09-16T01:51:19.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:19 smithi139 ceph-mon[31175]: pgmap v341: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 614 KiB/s wr, 226 op/s 2024-09-16T01:51:19.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:19 smithi176 ceph-mon[35324]: pgmap v341: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 614 KiB/s wr, 226 op/s 2024-09-16T01:51:20.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:19 smithi164 ceph-mon[35121]: pgmap v341: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 614 KiB/s wr, 226 op/s 2024-09-16T01:51:21.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:21 smithi139 ceph-mon[31175]: pgmap v342: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 603 KiB/s wr, 174 op/s 2024-09-16T01:51:21.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:21 smithi176 ceph-mon[35324]: pgmap v342: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 603 KiB/s wr, 174 op/s 2024-09-16T01:51:22.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:21 smithi164 ceph-mon[35121]: pgmap v342: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 603 KiB/s wr, 174 op/s 2024-09-16T01:51:23.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:23 smithi139 ceph-mon[31175]: pgmap v343: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 819 KiB/s wr, 193 op/s 2024-09-16T01:51:23.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:23 smithi176 ceph-mon[35324]: pgmap v343: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 819 KiB/s wr, 193 op/s 2024-09-16T01:51:24.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:23 smithi164 ceph-mon[35121]: pgmap v343: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 819 KiB/s wr, 193 op/s 2024-09-16T01:51:24.423 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:51:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:51:24.072+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:51:25.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:25 smithi139 ceph-mon[31175]: pgmap v344: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 858 KiB/s wr, 181 op/s 2024-09-16T01:51:25.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:25 smithi176 ceph-mon[35324]: pgmap v344: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 858 KiB/s wr, 181 op/s 2024-09-16T01:51:26.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:25 smithi164 ceph-mon[35121]: pgmap v344: 129 pgs: 21 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 858 KiB/s wr, 181 op/s 2024-09-16T01:51:27.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:27 smithi139 ceph-mon[31175]: pgmap v345: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 220 op/s 2024-09-16T01:51:27.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:27 smithi176 ceph-mon[35324]: pgmap v345: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 220 op/s 2024-09-16T01:51:28.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:27 smithi164 ceph-mon[35121]: pgmap v345: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 220 op/s 2024-09-16T01:51:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.095+0000 7f7b17427640 1 -- 172.21.15.139:0/1455404931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b10100490 msgr2=0x7f7b10100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.095+0000 7f7b17427640 1 --2- 172.21.15.139:0/1455404931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b10100490 0x7f7b10100870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009930 tx=0x7f7b0c02b670 comp rx=0 tx=0).stop 2024-09-16T01:51:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.099+0000 7f7b17427640 1 -- 172.21.15.139:0/1455404931 shutdown_connections 2024-09-16T01:51:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.099+0000 7f7b17427640 1 --2- 172.21.15.139:0/1455404931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b101026f0 0x7f7b1010d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.099+0000 7f7b17427640 1 --2- 172.21.15.139:0/1455404931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b10101d70 0x7f7b101021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.099+0000 7f7b17427640 1 --2- 172.21.15.139:0/1455404931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b10100490 0x7f7b10100870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.099+0000 7f7b17427640 1 -- 172.21.15.139:0/1455404931 >> 172.21.15.139:0/1455404931 conn(0x7f7b100770f0 msgr2=0x7f7b100fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.101+0000 7f7b17427640 1 -- 172.21.15.139:0/1455404931 shutdown_connections 2024-09-16T01:51:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.105+0000 7f7b17427640 1 -- 172.21.15.139:0/1455404931 wait complete. 2024-09-16T01:51:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.105+0000 7f7b17427640 1 Processor -- start 2024-09-16T01:51:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.106+0000 7f7b17427640 1 -- start start 2024-09-16T01:51:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.107+0000 7f7b17427640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b10100490 0x7f7b10073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.107+0000 7f7b17427640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b10101d70 0x7f7b1006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.107+0000 7f7b151c3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b10100490 0x7f7b10073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.107+0000 7f7b151c3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b10100490 0x7f7b10073ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43942/0 (socket says 172.21.15.139:43942) 2024-09-16T01:51:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.107+0000 7f7b17427640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b101026f0 0x7f7b100700a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.107+0000 7f7b149c2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b10101d70 0x7f7b1006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.108+0000 7f7b149c2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b10101d70 0x7f7b1006fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44512/0 (socket says 172.21.15.139:44512) 2024-09-16T01:51:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.108+0000 7f7b151c3640 1 -- 172.21.15.139:0/598854102 learned_addr learned my addr 172.21.15.139:0/598854102 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.108+0000 7f7b17427640 1 -- 172.21.15.139:0/598854102 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1006c620 con 0x7f7b101026f0 2024-09-16T01:51:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.108+0000 7f7b17427640 1 -- 172.21.15.139:0/598854102 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1006c4a0 con 0x7f7b10101d70 2024-09-16T01:51:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.108+0000 7f7b17427640 1 -- 172.21.15.139:0/598854102 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1006c7a0 con 0x7f7b10100490 2024-09-16T01:51:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.108+0000 7f7b159c4640 1 --2- 172.21.15.139:0/598854102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b101026f0 0x7f7b100700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.109+0000 7f7b159c4640 1 -- 172.21.15.139:0/598854102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b10100490 msgr2=0x7f7b10073ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.109+0000 7f7b159c4640 1 --2- 172.21.15.139:0/598854102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b10100490 0x7f7b10073ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.109+0000 7f7b159c4640 1 -- 172.21.15.139:0/598854102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b10101d70 msgr2=0x7f7b1006fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.109+0000 7f7b159c4640 1 --2- 172.21.15.139:0/598854102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b10101d70 0x7f7b1006fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.109+0000 7f7b159c4640 1 -- 172.21.15.139:0/598854102 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b10070780 con 0x7f7b101026f0 2024-09-16T01:51:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.110+0000 7f7b159c4640 1 --2- 172.21.15.139:0/598854102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b101026f0 0x7f7b100700a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7b08004620 tx=0x7f7b0800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.111+0000 7f7b027fc640 1 -- 172.21.15.139:0/598854102 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0801f070 con 0x7f7b101026f0 2024-09-16T01:51:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.111+0000 7f7b027fc640 1 -- 172.21.15.139:0/598854102 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b080135a0 con 0x7f7b101026f0 2024-09-16T01:51:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.111+0000 7f7b027fc640 1 -- 172.21.15.139:0/598854102 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08004800 con 0x7f7b101026f0 2024-09-16T01:51:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.112+0000 7f7b17427640 1 -- 172.21.15.139:0/598854102 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b100fc470 con 0x7f7b101026f0 2024-09-16T01:51:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.112+0000 7f7b17427640 1 -- 172.21.15.139:0/598854102 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b100fc910 con 0x7f7b101026f0 2024-09-16T01:51:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.116+0000 7f7b1a1c5740 1 Processor -- start 2024-09-16T01:51:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.119+0000 7f7b1a1c5740 1 -- start start 2024-09-16T01:51:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.119+0000 7f7b1a1c5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa45af0 0x562fdfa45ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.119+0000 7f7b1a1c5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa46410 0x562fdfa50770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.119+0000 7f7b1a1c5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa50cb0 0x562fdfa530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.119+0000 7f7b1a1c5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562fdf9aa2f0 con 0x562fdfa46410 2024-09-16T01:51:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.119+0000 7f7b1a1c5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562fdf952270 con 0x562fdfa45af0 2024-09-16T01:51:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.119+0000 7f7b1a1c5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562fdf9a62b0 con 0x562fdfa50cb0 2024-09-16T01:51:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.119+0000 7f7b151c3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa46410 0x562fdfa50770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.119+0000 7f7b151c3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa46410 0x562fdfa50770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54874/0 (socket says 172.21.15.139:54874) 2024-09-16T01:51:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.119+0000 7f7b151c3640 1 -- 172.21.15.139:0/238487418 learned_addr learned my addr 172.21.15.139:0/238487418 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.119+0000 7f7b149c2640 1 --2- 172.21.15.139:0/238487418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa45af0 0x562fdfa45ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.119+0000 7f7b027fc640 1 -- 172.21.15.139:0/598854102 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7b0800f3c0 con 0x7f7b101026f0 2024-09-16T01:51:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.119+0000 7f7b149c2640 1 --2- 172.21.15.139:0/238487418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa50cb0 0x562fdfa530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.120+0000 7f7b027fc640 1 --2- 172.21.15.139:0/598854102 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f7afc0a3640 0x7f7afc0a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.121+0000 7f7b027fc640 1 -- 172.21.15.139:0/598854102 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9642+0+0 (secure 0 0 0) 0x7f7b080d8780 con 0x7f7b101026f0 2024-09-16T01:51:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.122+0000 7f7b151c3640 1 -- 172.21.15.139:0/238487418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa50cb0 msgr2=0x562fdfa530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.122+0000 7f7b151c3640 1 --2- 172.21.15.139:0/238487418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa50cb0 0x562fdfa530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.122+0000 7f7b151c3640 1 -- 172.21.15.139:0/238487418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa45af0 msgr2=0x562fdfa45ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.122+0000 7f7b151c3640 1 --2- 172.21.15.139:0/238487418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa45af0 0x562fdfa45ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.122+0000 7f7b151c3640 1 -- 172.21.15.139:0/238487418 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fdf951340 con 0x562fdfa46410 2024-09-16T01:51:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.123+0000 7f7b151c3640 1 --2- 172.21.15.139:0/238487418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa46410 0x562fdfa50770 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c02be30 tx=0x7f7b0c02be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658a308390c700e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.124+0000 7f7b159c4640 1 --2- 172.21.15.139:0/598854102 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f7afc0a3640 0x7f7afc0a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.125+0000 7f7afb7fe640 1 -- 172.21.15.139:0/238487418 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c042070 con 0x562fdfa46410 2024-09-16T01:51:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.125+0000 7f7afb7fe640 1 -- 172.21.15.139:0/238487418 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0c03e930 con 0x562fdfa46410 2024-09-16T01:51:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.125+0000 7f7afb7fe640 1 -- 172.21.15.139:0/238487418 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c032900 con 0x562fdfa46410 2024-09-16T01:51:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.127+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/238487418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa46410 msgr2=0x562fdfa50770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.127+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/238487418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa46410 0x562fdfa50770 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c02be30 tx=0x7f7b0c02be60 comp rx=0 tx=0).stop 2024-09-16T01:51:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.127+0000 7f7b159c4640 1 --2- 172.21.15.139:0/598854102 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f7afc0a3640 0x7f7afc0a5b00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7b080a0f70 tx=0x7f7b0800ef90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.127+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/238487418 shutdown_connections 2024-09-16T01:51:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.127+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/238487418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa50cb0 0x562fdfa530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.127+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/238487418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa46410 0x562fdfa50770 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.127+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/238487418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa45af0 0x562fdfa45ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.127+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/238487418 >> 172.21.15.139:0/238487418 conn(0x562fdfa42060 msgr2=0x562fdfa42440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.132+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/238487418 shutdown_connections 2024-09-16T01:51:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.133+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/238487418 wait complete. 2024-09-16T01:51:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.134+0000 7f7b1a1c5740 1 -- start start 2024-09-16T01:51:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.134+0000 7f7b1a1c5740 1 Processor -- start 2024-09-16T01:51:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.135+0000 7f7b1a1c5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa45af0 0x562fdfa4dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.135+0000 7f7b1a1c5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa46410 0x562fdfa4e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.135+0000 7f7b1a1c5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa50cb0 0x562fdfa4e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.135+0000 7f7b1a1c5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562fdfa64890 con 0x562fdfa45af0 2024-09-16T01:51:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.136+0000 7f7b1a1c5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562fdf933290 con 0x562fdfa46410 2024-09-16T01:51:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.136+0000 7f7b151c3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa46410 0x562fdfa4e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.136+0000 7f7b151c3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa46410 0x562fdfa4e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44532/0 (socket says 172.21.15.139:44532) 2024-09-16T01:51:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.136+0000 7f7b151c3640 1 -- 172.21.15.139:0/108524388 learned_addr learned my addr 172.21.15.139:0/108524388 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.136+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/108524388 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562fdfa64a10 con 0x562fdfa50cb0 2024-09-16T01:51:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.136+0000 7f7b1a1c5740 10 client.0 _ll_register_callbacks cb 0x562fdf964140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:51:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.136+0000 7f7b151c3640 1 -- 172.21.15.139:0/108524388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa50cb0 msgr2=0x562fdfa4e650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:51:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.136+0000 7f7b151c3640 1 --2- 172.21.15.139:0/108524388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa50cb0 0x562fdfa4e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.136+0000 7f7b151c3640 1 -- 172.21.15.139:0/108524388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa45af0 msgr2=0x562fdfa4dbd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:51:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.136+0000 7f7b151c3640 1 --2- 172.21.15.139:0/108524388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa45af0 0x562fdfa4dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.136+0000 7f7b151c3640 1 -- 172.21.15.139:0/108524388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fdf6bd210 con 0x562fdfa46410 2024-09-16T01:51:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.137+0000 7f7b151c3640 1 --2- 172.21.15.139:0/108524388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa46410 0x562fdfa4e110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00b010 tx=0x7f7b0c003730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2a7af4ed33192d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.140+0000 7f7af8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:51:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.141+0000 7f7af8ff9640 1 -- 172.21.15.139:0/108524388 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c042050 con 0x562fdfa46410 2024-09-16T01:51:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.141+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/108524388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b0c03e930 con 0x562fdfa46410 2024-09-16T01:51:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.141+0000 7f7af8ff9640 1 -- 172.21.15.139:0/108524388 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9642+0+0 (secure 0 0 0) 0x7f7b0c032a90 con 0x562fdfa46410 2024-09-16T01:51:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.146+0000 7f7af8ff9640 1 -- 172.21.15.139:0/108524388 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f7b0c03ee10 con 0x562fdfa46410 2024-09-16T01:51:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.146+0000 7f7b1a1c5740 10 client.26108 fetch_fsmap learned FSMap version 56 2024-09-16T01:51:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.146+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/108524388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x562fdf789620 con 0x562fdfa46410 2024-09-16T01:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.156+0000 7f7af8ff9640 1 -- 172.21.15.139:0/108524388 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0c03e930 con 0x562fdfa46410 2024-09-16T01:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.156+0000 7f7af8ff9640 1 -- 172.21.15.139:0/108524388 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 56) ==== 8438+0+0 (secure 0 0 0) 0x7f7b0c05c020 con 0x562fdfa46410 2024-09-16T01:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.156+0000 7f7b1a1c5740 10 client.26108 fetch_fsmap finished waiting for FSMap version 56 2024-09-16T01:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.156+0000 7f7b1a1c5740 10 client.26108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.156+0000 7f7b1a1c5740 20 client.26108 populate_metadata read hostname 'smithi139' 2024-09-16T01:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.156+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/108524388 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562fdfa71ec0 0x562fdfa74320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.156+0000 7f7b1a1c5740 4 client.26108 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:51:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.156+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/108524388 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fdf9b7a20 con 0x562fdfa71ec0 2024-09-16T01:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.162+0000 7f7b149c2640 1 --2- 172.21.15.139:0/108524388 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562fdfa71ec0 0x562fdfa74320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.168+0000 7f7b149c2640 1 --2- 172.21.15.139:0/108524388 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562fdfa71ec0 0x562fdfa74320 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28ecafb27253ce60 server_cookie=a03e4181ad86e57e in_seq=0 out_seq=0 2024-09-16T01:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.168+0000 7f7af8ff9640 10 client.26108 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.171+0000 7f7af8ff9640 1 -- 172.21.15.139:0/108524388 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fdf9b7a20 con 0x562fdfa71ec0 2024-09-16T01:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.171+0000 7f7af8ff9640 10 client.26108 handle_command_reply: tid=0 2024-09-16T01:51:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.171+0000 7f7b1a1c5740 1 client.26108 shutdown 2024-09-16T01:51:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.177+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/108524388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa46410 msgr2=0x562fdfa4e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.177+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/108524388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa46410 0x562fdfa4e110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00b010 tx=0x7f7b0c003730 comp rx=0 tx=0).stop 2024-09-16T01:51:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.182+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/108524388 shutdown_connections 2024-09-16T01:51:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.182+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/108524388 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562fdfa71ec0 0x562fdfa74320 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.182+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/108524388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa50cb0 0x562fdfa4e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.182+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/108524388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa46410 0x562fdfa4e110 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.182+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/108524388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa45af0 0x562fdfa4dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.182+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/108524388 >> 172.21.15.139:0/108524388 conn(0x562fdfa42060 msgr2=0x562fdfa47dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.182+0000 7f7af8ff9640 0 client.26108 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.185+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/108524388 shutdown_connections 2024-09-16T01:51:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.186+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/108524388 wait complete. 2024-09-16T01:51:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.186+0000 7f7b1a1c5740 20 client.26108 trim_cache size 0 max 16384 2024-09-16T01:51:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.214+0000 7f7b1a1c5740 1 Processor -- start 2024-09-16T01:51:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.214+0000 7f7b1a1c5740 1 -- start start 2024-09-16T01:51:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.215+0000 7f7b1a1c5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa45af0 0x562fdfa4e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.215+0000 7f7b1a1c5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa46410 0x562fdfa4fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.215+0000 7f7b1a1c5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa50cb0 0x562fdfa79bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.215+0000 7f7b1a1c5740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101b5960 con 0x562fdfa50cb0 2024-09-16T01:51:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.216+0000 7f7b149c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa50cb0 0x562fdfa79bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.216+0000 7f7b149c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa50cb0 0x562fdfa79bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54894/0 (socket says 172.21.15.139:54894) 2024-09-16T01:51:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.216+0000 7f7b149c2640 1 -- 172.21.15.139:0/989200304 learned_addr learned my addr 172.21.15.139:0/989200304 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.216+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/989200304 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101b7710 con 0x562fdfa45af0 2024-09-16T01:51:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.216+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/989200304 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101b60f0 con 0x562fdfa46410 2024-09-16T01:51:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.217+0000 7f7b151c3640 1 --2- 172.21.15.139:0/989200304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa46410 0x562fdfa4fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.217+0000 7f7b149c2640 1 --2- 172.21.15.139:0/989200304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa45af0 0x562fdfa4e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.217+0000 7f7b151c3640 1 -- 172.21.15.139:0/989200304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa45af0 msgr2=0x562fdfa4e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.217+0000 7f7b151c3640 1 --2- 172.21.15.139:0/989200304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa45af0 0x562fdfa4e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.217+0000 7f7b151c3640 1 -- 172.21.15.139:0/989200304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa50cb0 msgr2=0x562fdfa79bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.217+0000 7f7b151c3640 1 --2- 172.21.15.139:0/989200304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa50cb0 0x562fdfa79bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.217+0000 7f7b151c3640 1 -- 172.21.15.139:0/989200304 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fdfa46c90 con 0x562fdfa46410 2024-09-16T01:51:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.218+0000 7f7b151c3640 1 --2- 172.21.15.139:0/989200304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa46410 0x562fdfa4fb40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x562fdfa4f180 tx=0x7f7b0c032440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5be7533bf0772836 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.222+0000 7f7afaffd640 1 -- 172.21.15.139:0/989200304 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c062070 con 0x562fdfa46410 2024-09-16T01:51:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.222+0000 7f7afaffd640 1 -- 172.21.15.139:0/989200304 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0c032d70 con 0x562fdfa46410 2024-09-16T01:51:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.222+0000 7f7afaffd640 1 -- 172.21.15.139:0/989200304 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c042070 con 0x562fdfa46410 2024-09-16T01:51:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.224+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/989200304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa46410 msgr2=0x562fdfa4fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.225+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/989200304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa46410 0x562fdfa4fb40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x562fdfa4f180 tx=0x7f7b0c032440 comp rx=0 tx=0).stop 2024-09-16T01:51:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.226+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/989200304 shutdown_connections 2024-09-16T01:51:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.226+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/989200304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa50cb0 0x562fdfa79bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.226+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/989200304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa46410 0x562fdfa4fb40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.226+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/989200304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa45af0 0x562fdfa4e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.226+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/989200304 >> 172.21.15.139:0/989200304 conn(0x562fdfa42060 msgr2=0x562fdfa4f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.228+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/989200304 shutdown_connections 2024-09-16T01:51:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.229+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/989200304 wait complete. 2024-09-16T01:51:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.229+0000 7f7b1a1c5740 1 -- start start 2024-09-16T01:51:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.229+0000 7f7b1a1c5740 1 Processor -- start 2024-09-16T01:51:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.230+0000 7f7b1a1c5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa45af0 0x562fdfa6a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.230+0000 7f7b1a1c5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa46410 0x562fdfa64c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.230+0000 7f7b1a1c5740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa50cb0 0x562fdfa65380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.231+0000 7f7b149c2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa45af0 0x562fdfa6a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.231+0000 7f7b151c3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa46410 0x562fdfa64c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.231+0000 7f7b151c3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa46410 0x562fdfa64c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44542/0 (socket says 172.21.15.139:44542) 2024-09-16T01:51:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.231+0000 7f7b149c2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa45af0 0x562fdfa6a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43974/0 (socket says 172.21.15.139:43974) 2024-09-16T01:51:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.231+0000 7f7b151c3640 1 -- 172.21.15.139:0/2378738170 learned_addr learned my addr 172.21.15.139:0/2378738170 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.231+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/2378738170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101b7010 con 0x562fdfa50cb0 2024-09-16T01:51:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.231+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/2378738170 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101b6880 con 0x562fdfa46410 2024-09-16T01:51:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.231+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/2378738170 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101b3960 con 0x562fdfa45af0 2024-09-16T01:51:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.231+0000 7f7b1a1c5740 10 client.0 _ll_register_callbacks cb 0x7f7b0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:51:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.231+0000 7f7b151c3640 1 -- 172.21.15.139:0/2378738170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa45af0 msgr2=0x562fdfa6a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.231+0000 7f7b151c3640 1 --2- 172.21.15.139:0/2378738170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa45af0 0x562fdfa6a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.231+0000 7f7b151c3640 1 -- 172.21.15.139:0/2378738170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa50cb0 msgr2=0x562fdfa65380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.231+0000 7f7b151c3640 1 --2- 172.21.15.139:0/2378738170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa50cb0 0x562fdfa65380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.231+0000 7f7b151c3640 1 -- 172.21.15.139:0/2378738170 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fdfadabb0 con 0x562fdfa46410 2024-09-16T01:51:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.232+0000 7f7b151c3640 1 --2- 172.21.15.139:0/2378738170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa46410 0x562fdfa64c90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c060040 tx=0x7f7b0c00d230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=992f5a8e8309815d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.232+0000 7f7af8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:51:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.232+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/2378738170 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b0c032d70 con 0x562fdfa46410 2024-09-16T01:51:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.232+0000 7f7af8ff9640 1 -- 172.21.15.139:0/2378738170 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c062070 con 0x562fdfa46410 2024-09-16T01:51:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.232+0000 7f7af8ff9640 1 -- 172.21.15.139:0/2378738170 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9642+0+0 (secure 0 0 0) 0x7f7b0c051710 con 0x562fdfa46410 2024-09-16T01:51:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.233+0000 7f7af8ff9640 1 -- 172.21.15.139:0/2378738170 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f7b0c042070 con 0x562fdfa46410 2024-09-16T01:51:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.234+0000 7f7b1a1c5740 10 client.26114 fetch_fsmap learned FSMap version 56 2024-09-16T01:51:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.234+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/2378738170 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=56}) -- 0x562fdfadbb80 con 0x562fdfa46410 2024-09-16T01:51:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.236+0000 7f7af8ff9640 1 -- 172.21.15.139:0/2378738170 <== mon.2 v2:172.21.15.164:3300/0 4 ==== fsmap(e 56) ==== 8438+0+0 (secure 0 0 0) 0x7f7b0c05c030 con 0x562fdfa46410 2024-09-16T01:51:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.236+0000 7f7b1a1c5740 10 client.26114 fetch_fsmap finished waiting for FSMap version 56 2024-09-16T01:51:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.236+0000 7f7b1a1c5740 10 client.26114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:51:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.236+0000 7f7b1a1c5740 20 client.26114 populate_metadata read hostname 'smithi139' 2024-09-16T01:51:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.236+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/2378738170 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562fdfadc0c0 0x562fdfade4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.236+0000 7f7b1a1c5740 4 client.26114 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:51:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.236+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/2378738170 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562fdf9b1d70 con 0x562fdfadc0c0 2024-09-16T01:51:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.239+0000 7f7b149c2640 1 --2- 172.21.15.139:0/2378738170 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562fdfadc0c0 0x562fdfade4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.250+0000 7f7b149c2640 1 --2- 172.21.15.139:0/2378738170 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562fdfadc0c0 0x562fdfade4e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb316b22e640465 server_cookie=c84e5ead011d42f6 in_seq=0 out_seq=0 2024-09-16T01:51:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:28.250+0000 7f7af8ff9640 10 client.26114 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.203+0000 7f7af8ff9640 1 -- 172.21.15.139:0/2378738170 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562fdf9b1d70 con 0x562fdfadc0c0 2024-09-16T01:51:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.203+0000 7f7af8ff9640 10 client.26114 handle_command_reply: tid=0 2024-09-16T01:51:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.203+0000 7f7b1a1c5740 1 client.26114 shutdown 2024-09-16T01:51:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.203+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/2378738170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa46410 msgr2=0x562fdfa64c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.203+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/2378738170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa46410 0x562fdfa64c90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c060040 tx=0x7f7b0c00d230 comp rx=0 tx=0).stop 2024-09-16T01:51:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.203+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/2378738170 shutdown_connections 2024-09-16T01:51:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.203+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/2378738170 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562fdfadc0c0 0x562fdfade4e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.203+0000 7f7af8ff9640 0 client.26114 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.203+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/2378738170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562fdfa50cb0 0x562fdfa65380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.203+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/2378738170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fdfa46410 0x562fdfa64c90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.203+0000 7f7b1a1c5740 1 --2- 172.21.15.139:0/2378738170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562fdfa45af0 0x562fdfa6a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.203+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/2378738170 >> 172.21.15.139:0/2378738170 conn(0x562fdfa42060 msgr2=0x562fdfa411c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.204+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/2378738170 shutdown_connections 2024-09-16T01:51:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.204+0000 7f7b1a1c5740 1 -- 172.21.15.139:0/2378738170 wait complete. 2024-09-16T01:51:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.204+0000 7f7b1a1c5740 20 client.26114 trim_cache size 0 max 16384 2024-09-16T01:51:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:51:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:51:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:51:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:51:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.205+0000 7f7b17427640 1 -- 172.21.15.139:0/598854102 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f7afc0a3640 msgr2=0x7f7afc0a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.205+0000 7f7b17427640 1 --2- 172.21.15.139:0/598854102 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f7afc0a3640 0x7f7afc0a5b00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7b080a0f70 tx=0x7f7b0800ef90 comp rx=0 tx=0).stop 2024-09-16T01:51:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.205+0000 7f7b17427640 1 -- 172.21.15.139:0/598854102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b101026f0 msgr2=0x7f7b100700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.205+0000 7f7b17427640 1 --2- 172.21.15.139:0/598854102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b101026f0 0x7f7b100700a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7b08004620 tx=0x7f7b0800ed40 comp rx=0 tx=0).stop 2024-09-16T01:51:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.205+0000 7f7b17427640 1 -- 172.21.15.139:0/598854102 shutdown_connections 2024-09-16T01:51:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.205+0000 7f7b17427640 1 --2- 172.21.15.139:0/598854102 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f7afc0a3640 0x7f7afc0a5b00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.205+0000 7f7b17427640 1 --2- 172.21.15.139:0/598854102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b101026f0 0x7f7b100700a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.205+0000 7f7b17427640 1 --2- 172.21.15.139:0/598854102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b10101d70 0x7f7b1006fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.205+0000 7f7b17427640 1 --2- 172.21.15.139:0/598854102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b10100490 0x7f7b10073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.205+0000 7f7b17427640 1 -- 172.21.15.139:0/598854102 >> 172.21.15.139:0/598854102 conn(0x7f7b100770f0 msgr2=0x7f7b100fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.206+0000 7f7b17427640 1 -- 172.21.15.139:0/598854102 shutdown_connections 2024-09-16T01:51:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:29.206+0000 7f7b17427640 1 -- 172.21.15.139:0/598854102 wait complete. 2024-09-16T01:51:29.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:29 smithi139 ceph-mon[31175]: pgmap v346: 129 pgs: 13 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 79 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 892 KiB/s wr, 256 op/s 2024-09-16T01:51:29.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:29 smithi176 ceph-mon[35324]: pgmap v346: 129 pgs: 13 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 79 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 892 KiB/s wr, 256 op/s 2024-09-16T01:51:30.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:29 smithi164 ceph-mon[35121]: pgmap v346: 129 pgs: 13 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 79 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 892 KiB/s wr, 256 op/s 2024-09-16T01:51:31.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:31 smithi139 ceph-mon[31175]: pgmap v347: 129 pgs: 11 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 84 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 989 KiB/s wr, 257 op/s 2024-09-16T01:51:31.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:31 smithi176 ceph-mon[35324]: pgmap v347: 129 pgs: 11 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 84 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 989 KiB/s wr, 257 op/s 2024-09-16T01:51:32.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:31 smithi164 ceph-mon[35121]: pgmap v347: 129 pgs: 11 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 84 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 989 KiB/s wr, 257 op/s 2024-09-16T01:51:33.386 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T01:51:33.387 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.85 2024-09-16T01:51:33.387 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:51:33.848 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:51:33.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:33 smithi139 ceph-mon[31175]: pgmap v348: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 376 op/s 2024-09-16T01:51:33.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:33 smithi176 ceph-mon[35324]: pgmap v348: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 376 op/s 2024-09-16T01:51:34.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:33 smithi164 ceph-mon[35121]: pgmap v348: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 376 op/s 2024-09-16T01:51:34.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.476+0000 7f4df377f640 1 -- 172.21.15.139:0/1519737480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dec102570 msgr2=0x7f4dec1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:34.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.477+0000 7f4df377f640 1 --2- 172.21.15.139:0/1519737480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dec102570 0x7f4dec1029b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009900 tx=0x7f4ddc02a4d0 comp rx=0 tx=0).stop 2024-09-16T01:51:34.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.482+0000 7f4df377f640 1 -- 172.21.15.139:0/1519737480 shutdown_connections 2024-09-16T01:51:34.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.482+0000 7f4df377f640 1 --2- 172.21.15.139:0/1519737480 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dec102ef0 0x7f4dec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:34.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.482+0000 7f4df377f640 1 --2- 172.21.15.139:0/1519737480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dec102570 0x7f4dec1029b0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:34.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.482+0000 7f4df377f640 1 --2- 172.21.15.139:0/1519737480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dec100e20 0x7f4dec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:34.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.482+0000 7f4df377f640 1 -- 172.21.15.139:0/1519737480 >> 172.21.15.139:0/1519737480 conn(0x7f4dec0fc7d0 msgr2=0x7f4dec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:34.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.485+0000 7f4df377f640 1 -- 172.21.15.139:0/1519737480 shutdown_connections 2024-09-16T01:51:34.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.485+0000 7f4df377f640 1 -- 172.21.15.139:0/1519737480 wait complete. 2024-09-16T01:51:34.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.486+0000 7f4df377f640 1 Processor -- start 2024-09-16T01:51:34.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.486+0000 7f4df377f640 1 -- start start 2024-09-16T01:51:34.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.486+0000 7f4df377f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dec100e20 0x7f4dec196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:34.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.486+0000 7f4df377f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dec102570 0x7f4dec196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:34.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.486+0000 7f4df377f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dec102ef0 0x7f4dec1a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:34.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.486+0000 7f4df377f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec111090 con 0x7f4dec102ef0 2024-09-16T01:51:34.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.486+0000 7f4df377f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec110f10 con 0x7f4dec100e20 2024-09-16T01:51:34.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.486+0000 7f4df377f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec111210 con 0x7f4dec102570 2024-09-16T01:51:34.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.487+0000 7f4df0d1a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dec102570 0x7f4dec196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:34.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.487+0000 7f4df0d1a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dec102570 0x7f4dec196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:33686/0 (socket says 172.21.15.139:33686) 2024-09-16T01:51:34.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.487+0000 7f4df0d1a640 1 -- 172.21.15.139:0/2014405406 learned_addr learned my addr 172.21.15.139:0/2014405406 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:34.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.489+0000 7f4df151b640 1 --2- 172.21.15.139:0/2014405406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dec100e20 0x7f4dec196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:34.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.489+0000 7f4df0d1a640 1 -- 172.21.15.139:0/2014405406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dec100e20 msgr2=0x7f4dec196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:34.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.489+0000 7f4df0d1a640 1 --2- 172.21.15.139:0/2014405406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dec100e20 0x7f4dec196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:34.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.489+0000 7f4df0d1a640 1 -- 172.21.15.139:0/2014405406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dec102ef0 msgr2=0x7f4dec1a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:34.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.489+0000 7f4df0d1a640 1 --2- 172.21.15.139:0/2014405406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dec102ef0 0x7f4dec1a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:34.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.489+0000 7f4df0d1a640 1 -- 172.21.15.139:0/2014405406 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dec1a2830 con 0x7f4dec102570 2024-09-16T01:51:34.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.489+0000 7f4df151b640 1 --2- 172.21.15.139:0/2014405406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dec100e20 0x7f4dec196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:51:34.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.492+0000 7f4df0d1a640 1 --2- 172.21.15.139:0/2014405406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dec102570 0x7f4dec196aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc02aac0 tx=0x7f4ddc003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:34.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.498+0000 7f4de27fc640 1 -- 172.21.15.139:0/2014405406 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc03f070 con 0x7f4dec102570 2024-09-16T01:51:34.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.498+0000 7f4de27fc640 1 -- 172.21.15.139:0/2014405406 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ddc003e40 con 0x7f4dec102570 2024-09-16T01:51:34.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.498+0000 7f4de27fc640 1 -- 172.21.15.139:0/2014405406 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc035030 con 0x7f4dec102570 2024-09-16T01:51:34.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.500+0000 7f4df377f640 1 -- 172.21.15.139:0/2014405406 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dec1a2ac0 con 0x7f4dec102570 2024-09-16T01:51:34.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.500+0000 7f4df377f640 1 -- 172.21.15.139:0/2014405406 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dec1a2f20 con 0x7f4dec102570 2024-09-16T01:51:34.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.501+0000 7f4df377f640 1 -- 172.21.15.139:0/2014405406 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dec1021b0 con 0x7f4dec102570 2024-09-16T01:51:34.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.508+0000 7f4de27fc640 1 -- 172.21.15.139:0/2014405406 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4ddc042070 con 0x7f4dec102570 2024-09-16T01:51:34.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.509+0000 7f4de27fc640 1 --2- 172.21.15.139:0/2014405406 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f4dd80a35c0 0x7f4dd80a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:34.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.509+0000 7f4de27fc640 1 -- 172.21.15.139:0/2014405406 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9642+0+0 (secure 0 0 0) 0x7f4ddc0f10b0 con 0x7f4dec102570 2024-09-16T01:51:34.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.509+0000 7f4df151b640 1 --2- 172.21.15.139:0/2014405406 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f4dd80a35c0 0x7f4dd80a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:34.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.513+0000 7f4de27fc640 1 -- 172.21.15.139:0/2014405406 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ddc049070 con 0x7f4dec102570 2024-09-16T01:51:34.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.513+0000 7f4df151b640 1 --2- 172.21.15.139:0/2014405406 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f4dd80a35c0 0x7f4dd80a5a80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4de8006fd0 tx=0x7f4de8008040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:34.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.911+0000 7f4df377f640 1 -- 172.21.15.139:0/2014405406 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4dec101280 con 0x7f4dec102570 2024-09-16T01:51:34.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.913+0000 7f4de27fc640 1 -- 172.21.15.139:0/2014405406 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+12024 (secure 0 0 0) 0x7f4ddc0bd650 con 0x7f4dec102570 2024-09-16T01:51:34.915 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:51:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 56 2024-09-16T01:51:34.915 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":56,"btime":"2024-09-16T01:50:56:520164+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":56,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:50:55.569206+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24454,24443,24442,24433]},"id":1}]} 2024-09-16T01:51:34.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.918+0000 7f4dd7fff640 1 -- 172.21.15.139:0/2014405406 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f4dd80a35c0 msgr2=0x7f4dd80a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:34.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.918+0000 7f4dd7fff640 1 --2- 172.21.15.139:0/2014405406 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f4dd80a35c0 0x7f4dd80a5a80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4de8006fd0 tx=0x7f4de8008040 comp rx=0 tx=0).stop 2024-09-16T01:51:34.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.919+0000 7f4dd7fff640 1 -- 172.21.15.139:0/2014405406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dec102570 msgr2=0x7f4dec196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:34.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.919+0000 7f4dd7fff640 1 --2- 172.21.15.139:0/2014405406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dec102570 0x7f4dec196aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc02aac0 tx=0x7f4ddc003cd0 comp rx=0 tx=0).stop 2024-09-16T01:51:34.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.920+0000 7f4dd7fff640 1 -- 172.21.15.139:0/2014405406 shutdown_connections 2024-09-16T01:51:34.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.920+0000 7f4dd7fff640 1 --2- 172.21.15.139:0/2014405406 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f4dd80a35c0 0x7f4dd80a5a80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:34.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.920+0000 7f4dd7fff640 1 --2- 172.21.15.139:0/2014405406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dec102ef0 0x7f4dec1a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:34.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.920+0000 7f4dd7fff640 1 --2- 172.21.15.139:0/2014405406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dec102570 0x7f4dec196aa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:34.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.920+0000 7f4dd7fff640 1 --2- 172.21.15.139:0/2014405406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4dec100e20 0x7f4dec196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:34.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.920+0000 7f4dd7fff640 1 -- 172.21.15.139:0/2014405406 >> 172.21.15.139:0/2014405406 conn(0x7f4dec0fc7d0 msgr2=0x7f4dec06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:34.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.921+0000 7f4dd7fff640 1 -- 172.21.15.139:0/2014405406 shutdown_connections 2024-09-16T01:51:34.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:34.921+0000 7f4dd7fff640 1 -- 172.21.15.139:0/2014405406 wait complete. 2024-09-16T01:51:35.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:35 smithi139 ceph-mon[31175]: pgmap v349: 129 pgs: 9 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 93 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 392 op/s 2024-09-16T01:51:35.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:35 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2014405406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:51:35.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:35 smithi176 ceph-mon[35324]: pgmap v349: 129 pgs: 9 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 93 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 392 op/s 2024-09-16T01:51:35.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:35 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2014405406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:51:36.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:35 smithi164 ceph-mon[35121]: pgmap v349: 129 pgs: 9 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 93 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 392 op/s 2024-09-16T01:51:36.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:35 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2014405406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:51:36.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:36 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:51:36.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:36 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:51:36.999 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0' 2024-09-16T01:51:36.999 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0 2024-09-16T01:51:37.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:36 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:51:37.467 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:51:37.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:37 smithi176 ceph-mon[35324]: pgmap v350: 129 pgs: 9 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 93 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 407 op/s 2024-09-16T01:51:38.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:37 smithi164 ceph-mon[35121]: pgmap v350: 129 pgs: 9 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 93 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 407 op/s 2024-09-16T01:51:38.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:37 smithi139 ceph-mon[31175]: pgmap v350: 129 pgs: 9 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 93 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 407 op/s 2024-09-16T01:51:39.922 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:39 smithi139 ceph-mon[31175]: pgmap v351: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.2 MiB/s wr, 425 op/s 2024-09-16T01:51:39.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:39 smithi176 ceph-mon[35324]: pgmap v351: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.2 MiB/s wr, 425 op/s 2024-09-16T01:51:40.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:39 smithi164 ceph-mon[35121]: pgmap v351: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.2 MiB/s wr, 425 op/s 2024-09-16T01:51:40.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.776+0000 7f12f68c2640 1 -- 172.21.15.139:0/117690077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12e8095a00 msgr2=0x7f12e8095e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:40.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.776+0000 7f12f68c2640 1 --2- 172.21.15.139:0/117690077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12e8095a00 0x7f12e8095e60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009900 tx=0x7f12dc02a410 comp rx=0 tx=0).stop 2024-09-16T01:51:40.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.776+0000 7f12f68c2640 1 -- 172.21.15.139:0/117690077 shutdown_connections 2024-09-16T01:51:40.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.776+0000 7f12f68c2640 1 --2- 172.21.15.139:0/117690077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12e80963a0 0x7f12e80a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.776+0000 7f12f68c2640 1 --2- 172.21.15.139:0/117690077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12e8095a00 0x7f12e8095e60 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.776+0000 7f12f68c2640 1 --2- 172.21.15.139:0/117690077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12e809b640 0x7f12e809ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.776+0000 7f12f68c2640 1 -- 172.21.15.139:0/117690077 >> 172.21.15.139:0/117690077 conn(0x7f12e800b320 msgr2=0x7f12e800b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:40.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.776+0000 7f12f68c2640 1 -- 172.21.15.139:0/117690077 shutdown_connections 2024-09-16T01:51:40.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.777+0000 7f12f68c2640 1 -- 172.21.15.139:0/117690077 wait complete. 2024-09-16T01:51:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.777+0000 7f12f68c2640 1 Processor -- start 2024-09-16T01:51:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.777+0000 7f12f68c2640 1 -- start start 2024-09-16T01:51:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12f68c2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12e8095a00 0x7f12e8007670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12f68c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12e80963a0 0x7f12e8007bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12f68c2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12e809b640 0x7f12e80080f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12f68c2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f12e80a59a0 con 0x7f12e80963a0 2024-09-16T01:51:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12f68c2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12e80a5820 con 0x7f12e809b640 2024-09-16T01:51:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12f68c2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f12e80a5b20 con 0x7f12e8095a00 2024-09-16T01:51:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12effff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12e8095a00 0x7f12e8007670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12effff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12e8095a00 0x7f12e8007670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:33710/0 (socket says 172.21.15.139:33710) 2024-09-16T01:51:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12effff640 1 -- 172.21.15.139:0/857297690 learned_addr learned my addr 172.21.15.139:0/857297690 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12ef7fe640 1 --2- 172.21.15.139:0/857297690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12e80963a0 0x7f12e8007bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12f4e5f640 1 --2- 172.21.15.139:0/857297690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12e809b640 0x7f12e80080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12effff640 1 -- 172.21.15.139:0/857297690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12e809b640 msgr2=0x7f12e80080f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12effff640 1 --2- 172.21.15.139:0/857297690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12e809b640 0x7f12e80080f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12effff640 1 -- 172.21.15.139:0/857297690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12e80963a0 msgr2=0x7f12e8007bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12effff640 1 --2- 172.21.15.139:0/857297690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12e80963a0 0x7f12e8007bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12effff640 1 -- 172.21.15.139:0/857297690 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e813a290 con 0x7f12e8095a00 2024-09-16T01:51:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.778+0000 7f12effff640 1 --2- 172.21.15.139:0/857297690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12e8095a00 0x7f12e8007670 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f12e400bf40 tx=0x7f12e400dd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.779+0000 7f12ed7fa640 1 -- 172.21.15.139:0/857297690 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e401c070 con 0x7f12e8095a00 2024-09-16T01:51:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.779+0000 7f12ed7fa640 1 -- 172.21.15.139:0/857297690 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e401a070 con 0x7f12e8095a00 2024-09-16T01:51:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.779+0000 7f12f68c2640 1 -- 172.21.15.139:0/857297690 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e812bb90 con 0x7f12e8095a00 2024-09-16T01:51:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.779+0000 7f12ed7fa640 1 -- 172.21.15.139:0/857297690 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4009b10 con 0x7f12e8095a00 2024-09-16T01:51:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.779+0000 7f12f68c2640 1 -- 172.21.15.139:0/857297690 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e812c090 con 0x7f12e8095a00 2024-09-16T01:51:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.781+0000 7f12ed7fa640 1 -- 172.21.15.139:0/857297690 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f12e4009cb0 con 0x7f12e8095a00 2024-09-16T01:51:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.781+0000 7f12ed7fa640 1 --2- 172.21.15.139:0/857297690 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f12e00a3440 0x7f12e00a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.782+0000 7f12ef7fe640 1 --2- 172.21.15.139:0/857297690 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f12e00a3440 0x7f12e00a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.782+0000 7f12ed7fa640 1 -- 172.21.15.139:0/857297690 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9642+0+0 (secure 0 0 0) 0x7f12e40d7b10 con 0x7f12e8095a00 2024-09-16T01:51:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.782+0000 7f12ef7fe640 1 --2- 172.21.15.139:0/857297690 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f12e00a3440 0x7f12e00a5900 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009230 tx=0x7f12dc0058d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.782+0000 7f12f8c9b740 1 Processor -- start 2024-09-16T01:51:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.782+0000 7f12f8c9b740 1 -- start start 2024-09-16T01:51:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12f8c9b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 0x562e90026e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12f8c9b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90027340 0x562e900316c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12f8c9b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90031c00 0x562e90033ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12f8c9b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562e8ff8b160 con 0x562e90031c00 2024-09-16T01:51:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12f8c9b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562e8ff32720 con 0x562e90027340 2024-09-16T01:51:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12f8c9b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562e8ff87120 con 0x562e90026a20 2024-09-16T01:51:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12f4e5f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90031c00 0x562e90033ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12effff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 0x562e90026e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12effff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 0x562e90026e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:33726/0 (socket says 172.21.15.139:33726) 2024-09-16T01:51:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12f4e5f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90031c00 0x562e90033ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59096/0 (socket says 172.21.15.139:59096) 2024-09-16T01:51:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12effff640 1 -- 172.21.15.139:0/3350358504 learned_addr learned my addr 172.21.15.139:0/3350358504 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12ef7fe640 1 --2- 172.21.15.139:0/3350358504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90027340 0x562e900316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12effff640 1 -- 172.21.15.139:0/3350358504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90027340 msgr2=0x562e900316c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12effff640 1 --2- 172.21.15.139:0/3350358504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90027340 0x562e900316c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12effff640 1 -- 172.21.15.139:0/3350358504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90031c00 msgr2=0x562e90033ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12effff640 1 --2- 172.21.15.139:0/3350358504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90031c00 0x562e90033ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12effff640 1 -- 172.21.15.139:0/3350358504 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e8fcfcba0 con 0x562e90026a20 2024-09-16T01:51:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12f4e5f640 1 --2- 172.21.15.139:0/3350358504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90031c00 0x562e90033ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:51:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12effff640 1 --2- 172.21.15.139:0/3350358504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 0x562e90026e00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f12e4016fd0 tx=0x7f12e40a7df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d249010da4a79e97 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12da7fc640 1 -- 172.21.15.139:0/3350358504 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4025040 con 0x562e90026a20 2024-09-16T01:51:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12da7fc640 1 -- 172.21.15.139:0/3350358504 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e40a1df0 con 0x562e90026a20 2024-09-16T01:51:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.783+0000 7f12da7fc640 1 -- 172.21.15.139:0/3350358504 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e40ac480 con 0x562e90026a20 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.784+0000 7f12f8c9b740 1 -- 172.21.15.139:0/3350358504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 msgr2=0x562e90026e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.784+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/3350358504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 0x562e90026e00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f12e4016fd0 tx=0x7f12e40a7df0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.784+0000 7f12f8c9b740 1 -- 172.21.15.139:0/3350358504 shutdown_connections 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.784+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/3350358504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90031c00 0x562e90033ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.784+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/3350358504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90027340 0x562e900316c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.784+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/3350358504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 0x562e90026e00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.784+0000 7f12f8c9b740 1 -- 172.21.15.139:0/3350358504 >> 172.21.15.139:0/3350358504 conn(0x562e90022fb0 msgr2=0x562e90023390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.784+0000 7f12f8c9b740 1 -- 172.21.15.139:0/3350358504 shutdown_connections 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.784+0000 7f12f8c9b740 1 -- 172.21.15.139:0/3350358504 wait complete. 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.785+0000 7f12f8c9b740 1 -- start start 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.785+0000 7f12f8c9b740 1 Processor -- start 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12f8c9b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90026a20 0x562e9002eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12f8c9b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90027340 0x562e9002f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12f8c9b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90031c00 0x562e9002f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12f8c9b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562e900347d0 con 0x562e90027340 2024-09-16T01:51:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12f8c9b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562e8ff13900 con 0x562e90026a20 2024-09-16T01:51:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12f8c9b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562e90034950 con 0x562e90031c00 2024-09-16T01:51:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12f8c9b740 10 client.0 _ll_register_callbacks cb 0x562e8ff11780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:51:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12ef7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90027340 0x562e9002f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12ef7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90027340 0x562e9002f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59098/0 (socket says 172.21.15.139:59098) 2024-09-16T01:51:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12ef7fe640 1 -- 172.21.15.139:0/1705326150 learned_addr learned my addr 172.21.15.139:0/1705326150 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12effff640 1 --2- 172.21.15.139:0/1705326150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90026a20 0x562e9002eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12f4e5f640 1 --2- 172.21.15.139:0/1705326150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90031c00 0x562e9002f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12ef7fe640 1 -- 172.21.15.139:0/1705326150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90031c00 msgr2=0x562e9002f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12ef7fe640 1 --2- 172.21.15.139:0/1705326150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90031c00 0x562e9002f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12ef7fe640 1 -- 172.21.15.139:0/1705326150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90026a20 msgr2=0x562e9002eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12ef7fe640 1 --2- 172.21.15.139:0/1705326150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90026a20 0x562e9002eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12ef7fe640 1 -- 172.21.15.139:0/1705326150 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e8ff53c50 con 0x562e90027340 2024-09-16T01:51:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12ef7fe640 1 --2- 172.21.15.139:0/1705326150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90027340 0x562e9002f080 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f12dc033000 tx=0x7f12dc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf67e66e1f19ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.786+0000 7f12cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:51:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.787+0000 7f12cf7fe640 1 -- 172.21.15.139:0/1705326150 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc035050 con 0x562e90027340 2024-09-16T01:51:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.787+0000 7f12cf7fe640 1 -- 172.21.15.139:0/1705326150 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9642+0+0 (secure 0 0 0) 0x7f12dc031540 con 0x562e90027340 2024-09-16T01:51:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.787+0000 7f12f8c9b740 1 -- 172.21.15.139:0/1705326150 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e40a1df0 con 0x562e90027340 2024-09-16T01:51:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.787+0000 7f12cf7fe640 1 -- 172.21.15.139:0/1705326150 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f12dc031740 con 0x562e90027340 2024-09-16T01:51:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.787+0000 7f12f8c9b740 10 client.16698 fetch_fsmap learned FSMap version 57 2024-09-16T01:51:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.787+0000 7f12f8c9b740 1 -- 172.21.15.139:0/1705326150 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x562e900404d0 con 0x562e90027340 2024-09-16T01:51:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.791+0000 7f12cf7fe640 1 -- 172.21.15.139:0/1705326150 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e40a1df0 con 0x562e90027340 2024-09-16T01:51:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.791+0000 7f12cf7fe640 1 -- 172.21.15.139:0/1705326150 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 57) ==== 8434+0+0 (secure 0 0 0) 0x7f12dc03cbc0 con 0x562e90027340 2024-09-16T01:51:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.792+0000 7f12f8c9b740 10 client.16698 fetch_fsmap finished waiting for FSMap version 57 2024-09-16T01:51:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.792+0000 7f12f8c9b740 10 client.16698 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:51:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.792+0000 7f12f8c9b740 20 client.16698 populate_metadata read hostname 'smithi139' 2024-09-16T01:51:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.792+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/1705326150 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562e90041990 0x562e90043de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.792+0000 7f12f8c9b740 4 client.16698 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:51:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.792+0000 7f12f8c9b740 1 -- 172.21.15.139:0/1705326150 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e8ff98890 con 0x562e90041990 2024-09-16T01:51:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.793+0000 7f12effff640 1 --2- 172.21.15.139:0/1705326150 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562e90041990 0x562e90043de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.794+0000 7f12effff640 1 --2- 172.21.15.139:0/1705326150 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562e90041990 0x562e90043de0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e6cd2881b31263b server_cookie=d046c3d86aafcc36 in_seq=0 out_seq=0 2024-09-16T01:51:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.794+0000 7f12cf7fe640 10 client.16698 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.795+0000 7f12cf7fe640 1 -- 172.21.15.139:0/1705326150 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e8ff98890 con 0x562e90041990 2024-09-16T01:51:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.795+0000 7f12cf7fe640 10 client.16698 handle_command_reply: tid=0 2024-09-16T01:51:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.795+0000 7f12f8c9b740 1 client.16698 shutdown 2024-09-16T01:51:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.795+0000 7f12f8c9b740 1 -- 172.21.15.139:0/1705326150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90027340 msgr2=0x562e9002f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.795+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/1705326150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90027340 0x562e9002f080 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f12dc033000 tx=0x7f12dc02a410 comp rx=0 tx=0).stop 2024-09-16T01:51:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.796+0000 7f12f8c9b740 1 -- 172.21.15.139:0/1705326150 shutdown_connections 2024-09-16T01:51:40.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.796+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/1705326150 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562e90041990 0x562e90043de0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.796+0000 7f12cf7fe640 0 client.16698 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.796+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/1705326150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90031c00 0x562e9002f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.796+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/1705326150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90027340 0x562e9002f080 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.796+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/1705326150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90026a20 0x562e9002eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.796+0000 7f12f8c9b740 1 -- 172.21.15.139:0/1705326150 >> 172.21.15.139:0/1705326150 conn(0x562e90022fb0 msgr2=0x562e90028c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.796+0000 7f12f8c9b740 1 -- 172.21.15.139:0/1705326150 shutdown_connections 2024-09-16T01:51:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.796+0000 7f12f8c9b740 1 -- 172.21.15.139:0/1705326150 wait complete. 2024-09-16T01:51:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.796+0000 7f12f8c9b740 20 client.16698 trim_cache size 0 max 16384 2024-09-16T01:51:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.832+0000 7f12f8c9b740 1 Processor -- start 2024-09-16T01:51:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.836+0000 7f12f8c9b740 1 -- start start 2024-09-16T01:51:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.836+0000 7f12f8c9b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 0x562e90030ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.836+0000 7f12f8c9b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90027340 0x562e900496f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.836+0000 7f12effff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 0x562e90030ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.837+0000 7f12effff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 0x562e90030ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:33752/0 (socket says 172.21.15.139:33752) 2024-09-16T01:51:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.837+0000 7f12effff640 1 -- 172.21.15.139:0/944658846 learned_addr learned my addr 172.21.15.139:0/944658846 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.837+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/944658846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90031c00 0x562e90049c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.837+0000 7f12f8c9b740 1 -- 172.21.15.139:0/944658846 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f12e8140bf0 con 0x562e90027340 2024-09-16T01:51:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.837+0000 7f12f8c9b740 1 -- 172.21.15.139:0/944658846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12e8142970 con 0x562e90031c00 2024-09-16T01:51:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.837+0000 7f12f8c9b740 1 -- 172.21.15.139:0/944658846 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f12e8141380 con 0x562e90026a20 2024-09-16T01:51:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.838+0000 7f12ef7fe640 1 --2- 172.21.15.139:0/944658846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90027340 0x562e900496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.838+0000 7f12f4e5f640 1 --2- 172.21.15.139:0/944658846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90031c00 0x562e90049c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.839+0000 7f12f4e5f640 1 -- 172.21.15.139:0/944658846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 msgr2=0x562e90030ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.839+0000 7f12f4e5f640 1 --2- 172.21.15.139:0/944658846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 0x562e90030ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.839+0000 7f12f4e5f640 1 -- 172.21.15.139:0/944658846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90027340 msgr2=0x562e900496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.839+0000 7f12f4e5f640 1 --2- 172.21.15.139:0/944658846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90027340 0x562e900496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.839+0000 7f12f4e5f640 1 -- 172.21.15.139:0/944658846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e9002d0b0 con 0x562e90031c00 2024-09-16T01:51:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.842+0000 7f12f4e5f640 1 --2- 172.21.15.139:0/944658846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90031c00 0x562e90049c30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f12f0068040 tx=0x7f12f0050650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ceb180d494552b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.845+0000 7f12d9ffb640 1 -- 172.21.15.139:0/944658846 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0091070 con 0x562e90031c00 2024-09-16T01:51:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.845+0000 7f12d9ffb640 1 -- 172.21.15.139:0/944658846 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f0080b20 con 0x562e90031c00 2024-09-16T01:51:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.845+0000 7f12d9ffb640 1 -- 172.21.15.139:0/944658846 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0074a90 con 0x562e90031c00 2024-09-16T01:51:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.846+0000 7f12f8c9b740 1 -- 172.21.15.139:0/944658846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90031c00 msgr2=0x562e90049c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.846+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/944658846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90031c00 0x562e90049c30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f12f0068040 tx=0x7f12f0050650 comp rx=0 tx=0).stop 2024-09-16T01:51:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.852+0000 7f12f8c9b740 1 -- 172.21.15.139:0/944658846 shutdown_connections 2024-09-16T01:51:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.852+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/944658846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90031c00 0x562e90049c30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.852+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/944658846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90027340 0x562e900496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.852+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/944658846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 0x562e90030ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.852+0000 7f12f8c9b740 1 -- 172.21.15.139:0/944658846 >> 172.21.15.139:0/944658846 conn(0x562e90022fb0 msgr2=0x562e900303a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.856+0000 7f12f8c9b740 1 -- 172.21.15.139:0/944658846 shutdown_connections 2024-09-16T01:51:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.857+0000 7f12f8c9b740 1 -- 172.21.15.139:0/944658846 wait complete. 2024-09-16T01:51:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.858+0000 7f12f8c9b740 1 -- start start 2024-09-16T01:51:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.858+0000 7f12f8c9b740 1 Processor -- start 2024-09-16T01:51:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.859+0000 7f12f8c9b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 0x562e90039ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.859+0000 7f12f8c9b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90027340 0x562e9003a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.859+0000 7f12f8c9b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90031c00 0x562e90036cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.859+0000 7f12f8c9b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f12e81422a0 con 0x562e90031c00 2024-09-16T01:51:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.859+0000 7f12f8c9b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12e8141b10 con 0x562e90027340 2024-09-16T01:51:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.859+0000 7f12f8c9b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f12e813ebf0 con 0x562e90026a20 2024-09-16T01:51:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.860+0000 7f12f8c9b740 10 client.0 _ll_register_callbacks cb 0x7f12dc004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:51:40.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.860+0000 7f12f4e5f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90031c00 0x562e90036cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.860+0000 7f12f4e5f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90031c00 0x562e90036cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59112/0 (socket says 172.21.15.139:59112) 2024-09-16T01:51:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.860+0000 7f12f4e5f640 1 -- 172.21.15.139:0/2496635173 learned_addr learned my addr 172.21.15.139:0/2496635173 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.860+0000 7f12ef7fe640 1 --2- 172.21.15.139:0/2496635173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90027340 0x562e9003a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.860+0000 7f12f4e5f640 1 -- 172.21.15.139:0/2496635173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 msgr2=0x562e90039ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:51:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.860+0000 7f12f4e5f640 1 --2- 172.21.15.139:0/2496635173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 0x562e90039ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.860+0000 7f12f4e5f640 1 -- 172.21.15.139:0/2496635173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90027340 msgr2=0x562e9003a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.860+0000 7f12f4e5f640 1 --2- 172.21.15.139:0/2496635173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90027340 0x562e9003a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.860+0000 7f12f4e5f640 1 -- 172.21.15.139:0/2496635173 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e900374e0 con 0x562e90031c00 2024-09-16T01:51:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.861+0000 7f12f4e5f640 1 --2- 172.21.15.139:0/2496635173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90031c00 0x562e90036cb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f12f00518b0 tx=0x7f12f0074650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645e418c3ecb45aa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.862+0000 7f12cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:51:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.862+0000 7f12cf7fe640 1 -- 172.21.15.139:0/2496635173 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0091040 con 0x562e90031c00 2024-09-16T01:51:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.862+0000 7f12cf7fe640 1 -- 172.21.15.139:0/2496635173 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9642+0+0 (secure 0 0 0) 0x7f12f0080440 con 0x562e90031c00 2024-09-16T01:51:40.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.862+0000 7f12f8c9b740 1 -- 172.21.15.139:0/2496635173 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12f0080b20 con 0x562e90031c00 2024-09-16T01:51:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.863+0000 7f12cf7fe640 1 -- 172.21.15.139:0/2496635173 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f12f0080680 con 0x562e90031c00 2024-09-16T01:51:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.863+0000 7f12f8c9b740 10 client.16704 fetch_fsmap learned FSMap version 57 2024-09-16T01:51:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.863+0000 7f12f8c9b740 1 -- 172.21.15.139:0/2496635173 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x562e900ad310 con 0x562e90031c00 2024-09-16T01:51:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.864+0000 7f12cf7fe640 1 -- 172.21.15.139:0/2496635173 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f0080820 con 0x562e90031c00 2024-09-16T01:51:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.864+0000 7f12cf7fe640 1 -- 172.21.15.139:0/2496635173 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 57) ==== 8434+0+0 (secure 0 0 0) 0x7f12f009f030 con 0x562e90031c00 2024-09-16T01:51:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.864+0000 7f12f8c9b740 10 client.16704 fetch_fsmap finished waiting for FSMap version 57 2024-09-16T01:51:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.864+0000 7f12f8c9b740 10 client.16704 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:51:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.864+0000 7f12f8c9b740 20 client.16704 populate_metadata read hostname 'smithi139' 2024-09-16T01:51:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.864+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/2496635173 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562e90041990 0x562e900afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.864+0000 7f12f8c9b740 4 client.16704 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}] 2024-09-16T01:51:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.864+0000 7f12f8c9b740 1 -- 172.21.15.139:0/2496635173 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}) -- 0x7f12f0080b20 con 0x562e90041990 2024-09-16T01:51:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.866+0000 7f12effff640 1 --2- 172.21.15.139:0/2496635173 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562e90041990 0x562e900afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.868+0000 7f12effff640 1 --2- 172.21.15.139:0/2496635173 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562e90041990 0x562e900afab0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5ad6f82a435c0f3 server_cookie=84b4a146cac3196 in_seq=0 out_seq=0 2024-09-16T01:51:40.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:39.868+0000 7f12cf7fe640 10 client.16704 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:42.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:41 smithi164 ceph-mon[35121]: pgmap v352: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.1 MiB/s wr, 386 op/s 2024-09-16T01:51:42.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:41 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:51:42.172 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:41 smithi139 ceph-mon[31175]: pgmap v352: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.1 MiB/s wr, 386 op/s 2024-09-16T01:51:42.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:41 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:51:42.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:41 smithi176 ceph-mon[35324]: pgmap v352: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.1 MiB/s wr, 386 op/s 2024-09-16T01:51:42.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:41 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:51:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.346+0000 7f8a8bc47640 1 -- 172.21.15.139:0/1682110677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a84101350 msgr2=0x7f8a840698a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.346+0000 7f8a8bc47640 1 --2- 172.21.15.139:0/1682110677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a84101350 0x7f8a840698a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009900 tx=0x7f8a8002a440 comp rx=0 tx=0).stop 2024-09-16T01:51:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.346+0000 7f8a8bc47640 1 -- 172.21.15.139:0/1682110677 shutdown_connections 2024-09-16T01:51:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.346+0000 7f8a8bc47640 1 --2- 172.21.15.139:0/1682110677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a84101350 0x7f8a840698a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.346+0000 7f8a8bc47640 1 --2- 172.21.15.139:0/1682110677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a841009b0 0x7f8a84100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.346+0000 7f8a8bc47640 1 --2- 172.21.15.139:0/1682110677 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a841069c0 0x7f8a84106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.346+0000 7f8a8bc47640 1 -- 172.21.15.139:0/1682110677 >> 172.21.15.139:0/1682110677 conn(0x7f8a84077990 msgr2=0x7f8a840fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.348+0000 7f8a8bc47640 1 -- 172.21.15.139:0/1682110677 shutdown_connections 2024-09-16T01:51:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.349+0000 7f8a8bc47640 1 -- 172.21.15.139:0/1682110677 wait complete. 2024-09-16T01:51:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.349+0000 7f8a8bc47640 1 Processor -- start 2024-09-16T01:51:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.349+0000 7f8a8bc47640 1 -- start start 2024-09-16T01:51:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.350+0000 7f8a8bc47640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a841009b0 0x7f8a840738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.350+0000 7f8a8bc47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a84101350 0x7f8a84073df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.350+0000 7f8a8bc47640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a841069c0 0x7f8a8406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.350+0000 7f8a8bc47640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8a84114170 con 0x7f8a84101350 2024-09-16T01:51:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.350+0000 7f8a8bc47640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a84113ff0 con 0x7f8a841009b0 2024-09-16T01:51:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.350+0000 7f8a8bc47640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8a841142f0 con 0x7f8a841069c0 2024-09-16T01:51:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.350+0000 7f8a891e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a84101350 0x7f8a84073df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.350+0000 7f8a891e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a84101350 0x7f8a84073df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44062/0 (socket says 172.21.15.139:44062) 2024-09-16T01:51:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.350+0000 7f8a891e2640 1 -- 172.21.15.139:0/3602260016 learned_addr learned my addr 172.21.15.139:0/3602260016 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.351+0000 7f8a891e2640 1 -- 172.21.15.139:0/3602260016 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a841069c0 msgr2=0x7f8a8406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.351+0000 7f8a891e2640 1 --2- 172.21.15.139:0/3602260016 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a841069c0 0x7f8a8406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.351+0000 7f8a891e2640 1 -- 172.21.15.139:0/3602260016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a841009b0 msgr2=0x7f8a840738b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:51:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.351+0000 7f8a891e2640 1 --2- 172.21.15.139:0/3602260016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a841009b0 0x7f8a840738b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.351+0000 7f8a891e2640 1 -- 172.21.15.139:0/3602260016 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8406d330 con 0x7f8a84101350 2024-09-16T01:51:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.351+0000 7f8a891e2640 1 --2- 172.21.15.139:0/3602260016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a84101350 0x7f8a84073df0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800d950 tx=0x7f8a7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.353+0000 7f8a76ffd640 1 -- 172.21.15.139:0/3602260016 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78017070 con 0x7f8a84101350 2024-09-16T01:51:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.353+0000 7f8a76ffd640 1 -- 172.21.15.139:0/3602260016 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a78009c30 con 0x7f8a84101350 2024-09-16T01:51:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.353+0000 7f8a76ffd640 1 -- 172.21.15.139:0/3602260016 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800f690 con 0x7f8a84101350 2024-09-16T01:51:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.354+0000 7f8a8bc47640 1 -- 172.21.15.139:0/3602260016 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8406d620 con 0x7f8a84101350 2024-09-16T01:51:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.354+0000 7f8a8bc47640 1 -- 172.21.15.139:0/3602260016 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a840715b0 con 0x7f8a84101350 2024-09-16T01:51:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.360+0000 7f8a8e9e0740 1 Processor -- start 2024-09-16T01:51:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.361+0000 7f8a8e9e0740 1 -- start start 2024-09-16T01:51:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.361+0000 7f8a8e9e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e50af0 0x55d0f0e50ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.361+0000 7f8a8e9e0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e51410 0x55d0f0e5b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.361+0000 7f8a8e9e0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e5e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.361+0000 7f8a8e9e0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0db52f0 con 0x55d0f0e5bcb0 2024-09-16T01:51:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.361+0000 7f8a8e9e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0d5d270 con 0x55d0f0e50af0 2024-09-16T01:51:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.361+0000 7f8a8e9e0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0db12b0 con 0x55d0f0e51410 2024-09-16T01:51:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.362+0000 7f8a76ffd640 1 -- 172.21.15.139:0/3602260016 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8a780109d0 con 0x7f8a84101350 2024-09-16T01:51:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.363+0000 7f8a76ffd640 1 --2- 172.21.15.139:0/3602260016 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f8a700a3640 0x7f8a700a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.363+0000 7f8a8a1e4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e50af0 0x55d0f0e50ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.363+0000 7f8a8a1e4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e50af0 0x55d0f0e50ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:36674/0 (socket says 172.21.15.139:36674) 2024-09-16T01:51:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.363+0000 7f8a8a1e4640 1 -- 172.21.15.139:0/3528052045 learned_addr learned my addr 172.21.15.139:0/3528052045 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.363+0000 7f8a76ffd640 1 -- 172.21.15.139:0/3602260016 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9642+0+0 (secure 0 0 0) 0x7f8a780cfb70 con 0x7f8a84101350 2024-09-16T01:51:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.364+0000 7f8a8a1e4640 1 --2- 172.21.15.139:0/3602260016 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f8a700a3640 0x7f8a700a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.365+0000 7f8a891e2640 1 --2- 172.21.15.139:0/3528052045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e5e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.365+0000 7f8a8a1e4640 1 -- 172.21.15.139:0/3528052045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e51410 msgr2=0x55d0f0e5b770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:51:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.365+0000 7f8a8a1e4640 1 --2- 172.21.15.139:0/3528052045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e51410 0x55d0f0e5b770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.365+0000 7f8a8a1e4640 1 -- 172.21.15.139:0/3528052045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e5bcb0 msgr2=0x55d0f0e5e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.365+0000 7f8a8a1e4640 1 --2- 172.21.15.139:0/3528052045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e5e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.365+0000 7f8a8a1e4640 1 -- 172.21.15.139:0/3528052045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f0d5c340 con 0x55d0f0e50af0 2024-09-16T01:51:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.365+0000 7f8a891e2640 1 --2- 172.21.15.139:0/3528052045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e5e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:51:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.365+0000 7f8a8a1e4640 1 --2- 172.21.15.139:0/3528052045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e50af0 0x55d0f0e50ed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a80005ab0 tx=0x7f8a80004a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6902da30507b55d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.366+0000 7f8a8a1e4640 1 --2- 172.21.15.139:0/3602260016 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f8a700a3640 0x7f8a700a5b00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a8002aa30 tx=0x7f8a80046000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.366+0000 7f8a6ffff640 1 -- 172.21.15.139:0/3528052045 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80005650 con 0x55d0f0e50af0 2024-09-16T01:51:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.367+0000 7f8a6ffff640 1 -- 172.21.15.139:0/3528052045 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a80040940 con 0x55d0f0e50af0 2024-09-16T01:51:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.367+0000 7f8a6ffff640 1 -- 172.21.15.139:0/3528052045 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a800316a0 con 0x55d0f0e50af0 2024-09-16T01:51:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.368+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/3528052045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e50af0 msgr2=0x55d0f0e50ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.368+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/3528052045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e50af0 0x55d0f0e50ed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a80005ab0 tx=0x7f8a80004a70 comp rx=0 tx=0).stop 2024-09-16T01:51:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.368+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/3528052045 shutdown_connections 2024-09-16T01:51:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.368+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/3528052045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e5e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.368+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/3528052045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e51410 0x55d0f0e5b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.368+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/3528052045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e50af0 0x55d0f0e50ed0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.368+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/3528052045 >> 172.21.15.139:0/3528052045 conn(0x55d0f0e4d060 msgr2=0x55d0f0e4d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.370+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/3528052045 shutdown_connections 2024-09-16T01:51:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.370+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/3528052045 wait complete. 2024-09-16T01:51:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.371+0000 7f8a8e9e0740 1 -- start start 2024-09-16T01:51:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.371+0000 7f8a8e9e0740 1 Processor -- start 2024-09-16T01:51:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a8e9e0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e50af0 0x55d0f0e58c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a8e9e0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e51410 0x55d0f0e59180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a8e9e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e596c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a8e9e0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0e5e820 con 0x55d0f0e50af0 2024-09-16T01:51:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a8e9e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0d3e290 con 0x55d0f0e5bcb0 2024-09-16T01:51:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a8e9e0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d0f0e5e9a0 con 0x55d0f0e51410 2024-09-16T01:51:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a8e9e0740 10 client.0 _ll_register_callbacks cb 0x55d0f0d6f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:51:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a899e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e50af0 0x55d0f0e58c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a899e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e50af0 0x55d0f0e58c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44080/0 (socket says 172.21.15.139:44080) 2024-09-16T01:51:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a899e3640 1 -- 172.21.15.139:0/2574545626 learned_addr learned my addr 172.21.15.139:0/2574545626 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a8a1e4640 1 --2- 172.21.15.139:0/2574545626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a899e3640 1 -- 172.21.15.139:0/2574545626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e51410 msgr2=0x55d0f0e59180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:51:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a899e3640 1 --2- 172.21.15.139:0/2574545626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e51410 0x55d0f0e59180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a899e3640 1 -- 172.21.15.139:0/2574545626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e5bcb0 msgr2=0x55d0f0e596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a899e3640 1 --2- 172.21.15.139:0/2574545626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.372+0000 7f8a899e3640 1 -- 172.21.15.139:0/2574545626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f0ac8210 con 0x55d0f0e50af0 2024-09-16T01:51:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.373+0000 7f8a899e3640 1 --2- 172.21.15.139:0/2574545626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e50af0 0x55d0f0e58c40 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00e620 tx=0x7f8a7c00eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd63ee1e81bfd37c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.373+0000 7f8a6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:51:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.373+0000 7f8a6d7fa640 1 -- 172.21.15.139:0/2574545626 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c004d20 con 0x55d0f0e50af0 2024-09-16T01:51:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.373+0000 7f8a6d7fa640 1 -- 172.21.15.139:0/2574545626 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9642+0+0 (secure 0 0 0) 0x7f8a7c0106a0 con 0x55d0f0e50af0 2024-09-16T01:51:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.373+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/2574545626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a80040940 con 0x55d0f0e50af0 2024-09-16T01:51:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.373+0000 7f8a6d7fa640 1 -- 172.21.15.139:0/2574545626 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8a7c022050 con 0x55d0f0e50af0 2024-09-16T01:51:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.373+0000 7f8a8e9e0740 10 client.16716 fetch_fsmap learned FSMap version 58 2024-09-16T01:51:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.373+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/2574545626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55d0f0b94620 con 0x55d0f0e50af0 2024-09-16T01:51:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.373+0000 7f8a6d7fa640 1 -- 172.21.15.139:0/2574545626 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a80040940 con 0x55d0f0e50af0 2024-09-16T01:51:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.374+0000 7f8a6d7fa640 1 -- 172.21.15.139:0/2574545626 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 58) ==== 8438+0+0 (secure 0 0 0) 0x7f8a7c00f040 con 0x55d0f0e50af0 2024-09-16T01:51:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.374+0000 7f8a8e9e0740 10 client.16716 fetch_fsmap finished waiting for FSMap version 58 2024-09-16T01:51:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.374+0000 7f8a8e9e0740 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:51:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.374+0000 7f8a8e9e0740 20 client.16716 populate_metadata read hostname 'smithi139' 2024-09-16T01:51:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.374+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/2574545626 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d0f0e74460 0x55d0f0e76870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.374+0000 7f8a8e9e0740 4 client.16716 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:51:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.374+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/2574545626 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0f0dc2a20 con 0x55d0f0e74460 2024-09-16T01:51:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.377+0000 7f8a891e2640 1 --2- 172.21.15.139:0/2574545626 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d0f0e74460 0x55d0f0e76870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.384+0000 7f8a891e2640 1 --2- 172.21.15.139:0/2574545626 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d0f0e74460 0x55d0f0e76870 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9246c5a2ff99ebb4 server_cookie=b8df462e3176dfc4 in_seq=0 out_seq=0 2024-09-16T01:51:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.384+0000 7f8a6d7fa640 10 client.16716 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.385+0000 7f8a6d7fa640 1 -- 172.21.15.139:0/2574545626 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0f0dc2a20 con 0x55d0f0e74460 2024-09-16T01:51:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.385+0000 7f8a6d7fa640 10 client.16716 handle_command_reply: tid=0 2024-09-16T01:51:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.385+0000 7f8a8e9e0740 1 client.16716 shutdown 2024-09-16T01:51:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.385+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/2574545626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e50af0 msgr2=0x55d0f0e58c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.385+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/2574545626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e50af0 0x55d0f0e58c40 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00e620 tx=0x7f8a7c00eae0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.387+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/2574545626 shutdown_connections 2024-09-16T01:51:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.387+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/2574545626 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d0f0e74460 0x55d0f0e76870 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.387+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/2574545626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.387+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/2574545626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e51410 0x55d0f0e59180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.387+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/2574545626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e50af0 0x55d0f0e58c40 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.387+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/2574545626 >> 172.21.15.139:0/2574545626 conn(0x55d0f0e4d060 msgr2=0x55d0f0e52dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.387+0000 7f8a6d7fa640 0 client.16716 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.387+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/2574545626 shutdown_connections 2024-09-16T01:51:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.387+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/2574545626 wait complete. 2024-09-16T01:51:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.387+0000 7f8a8e9e0740 20 client.16716 trim_cache size 0 max 16384 2024-09-16T01:51:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.415+0000 7f8a8e9e0740 1 Processor -- start 2024-09-16T01:51:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a8e9e0740 1 -- start start 2024-09-16T01:51:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a8e9e0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e50af0 0x55d0f0e52cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a8e9e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e51410 0x55d0f0e7c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a8e9e0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e7c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a8e9e0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8a841a4890 con 0x55d0f0e50af0 2024-09-16T01:51:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a8e9e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a841a6640 con 0x55d0f0e51410 2024-09-16T01:51:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a8e9e0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8a841a5020 con 0x55d0f0e5bcb0 2024-09-16T01:51:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a899e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e50af0 0x55d0f0e52cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a899e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e50af0 0x55d0f0e52cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44086/0 (socket says 172.21.15.139:44086) 2024-09-16T01:51:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a899e3640 1 -- 172.21.15.139:0/2565600739 learned_addr learned my addr 172.21.15.139:0/2565600739 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a899e3640 1 -- 172.21.15.139:0/2565600739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e5bcb0 msgr2=0x55d0f0e7c6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:51:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a899e3640 1 --2- 172.21.15.139:0/2565600739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e7c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a899e3640 1 -- 172.21.15.139:0/2565600739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e51410 msgr2=0x55d0f0e7c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a899e3640 1 --2- 172.21.15.139:0/2565600739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e51410 0x55d0f0e7c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.416+0000 7f8a899e3640 1 -- 172.21.15.139:0/2565600739 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f0e51240 con 0x55d0f0e50af0 2024-09-16T01:51:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.417+0000 7f8a899e3640 1 --2- 172.21.15.139:0/2565600739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e50af0 0x55d0f0e52cb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009430 tx=0x7f8a7c005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2aa228b34dee83a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.417+0000 7f8a6f7fe640 1 -- 172.21.15.139:0/2565600739 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c009cc0 con 0x55d0f0e50af0 2024-09-16T01:51:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.417+0000 7f8a6f7fe640 1 -- 172.21.15.139:0/2565600739 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a7c01f040 con 0x55d0f0e50af0 2024-09-16T01:51:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.417+0000 7f8a6f7fe640 1 -- 172.21.15.139:0/2565600739 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c011650 con 0x55d0f0e50af0 2024-09-16T01:51:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.418+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/2565600739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e50af0 msgr2=0x55d0f0e52cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.418+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/2565600739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e50af0 0x55d0f0e52cb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009430 tx=0x7f8a7c005a90 comp rx=0 tx=0).stop 2024-09-16T01:51:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.418+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/2565600739 shutdown_connections 2024-09-16T01:51:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.418+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/2565600739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e7c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.418+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/2565600739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e51410 0x55d0f0e7c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.418+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/2565600739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e50af0 0x55d0f0e52cb0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.418+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/2565600739 >> 172.21.15.139:0/2565600739 conn(0x55d0f0e4d060 msgr2=0x55d0f0e525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.418+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/2565600739 shutdown_connections 2024-09-16T01:51:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.420+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/2565600739 wait complete. 2024-09-16T01:51:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.420+0000 7f8a8e9e0740 1 -- start start 2024-09-16T01:51:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.420+0000 7f8a8e9e0740 1 Processor -- start 2024-09-16T01:51:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.420+0000 7f8a8e9e0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e50af0 0x55d0f0e6cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.420+0000 7f8a8e9e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e51410 0x55d0f0e559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.420+0000 7f8a8e9e0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e55f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.420+0000 7f8a8e9e0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8a841a5f40 con 0x55d0f0e5bcb0 2024-09-16T01:51:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.420+0000 7f8a8e9e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a841a57b0 con 0x55d0f0e51410 2024-09-16T01:51:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.420+0000 7f8a8e9e0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8a841a28f0 con 0x55d0f0e50af0 2024-09-16T01:51:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.420+0000 7f8a8e9e0740 10 client.0 _ll_register_callbacks cb 0x7f8a7c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:51:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.420+0000 7f8a8a1e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e55f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.420+0000 7f8a891e2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e51410 0x55d0f0e559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.421+0000 7f8a891e2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e51410 0x55d0f0e559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:36702/0 (socket says 172.21.15.139:36702) 2024-09-16T01:51:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.421+0000 7f8a8a1e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e55f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44090/0 (socket says 172.21.15.139:44090) 2024-09-16T01:51:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.421+0000 7f8a891e2640 1 -- 172.21.15.139:0/1464219900 learned_addr learned my addr 172.21.15.139:0/1464219900 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.421+0000 7f8a899e3640 1 --2- 172.21.15.139:0/1464219900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e50af0 0x55d0f0e6cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.421+0000 7f8a891e2640 1 -- 172.21.15.139:0/1464219900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e50af0 msgr2=0x55d0f0e6cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.421+0000 7f8a891e2640 1 --2- 172.21.15.139:0/1464219900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e50af0 0x55d0f0e6cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.421+0000 7f8a891e2640 1 -- 172.21.15.139:0/1464219900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e5bcb0 msgr2=0x55d0f0e55f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.421+0000 7f8a891e2640 1 --2- 172.21.15.139:0/1464219900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e55f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.421+0000 7f8a891e2640 1 -- 172.21.15.139:0/1464219900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f0e56750 con 0x55d0f0e51410 2024-09-16T01:51:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.421+0000 7f8a891e2640 1 --2- 172.21.15.139:0/1464219900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e51410 0x55d0f0e559e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8a780cfe50 tx=0x7f8a7809c180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd731dadeb7a11f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.421+0000 7f8a6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:51:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.422+0000 7f8a6d7fa640 1 -- 172.21.15.139:0/1464219900 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a780a4680 con 0x55d0f0e51410 2024-09-16T01:51:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.422+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/1464219900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a7c01f040 con 0x55d0f0e51410 2024-09-16T01:51:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.422+0000 7f8a6d7fa640 1 -- 172.21.15.139:0/1464219900 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9642+0+0 (secure 0 0 0) 0x7f8a780cd080 con 0x55d0f0e51410 2024-09-16T01:51:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.422+0000 7f8a6d7fa640 1 -- 172.21.15.139:0/1464219900 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8a780cd280 con 0x55d0f0e51410 2024-09-16T01:51:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.422+0000 7f8a8e9e0740 10 client.26144 fetch_fsmap learned FSMap version 58 2024-09-16T01:51:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.422+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/1464219900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55d0f0edf220 con 0x55d0f0e51410 2024-09-16T01:51:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.422+0000 7f8a6d7fa640 1 -- 172.21.15.139:0/1464219900 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a7c01f040 con 0x55d0f0e51410 2024-09-16T01:51:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.423+0000 7f8a6d7fa640 1 -- 172.21.15.139:0/1464219900 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 58) ==== 8438+0+0 (secure 0 0 0) 0x7f8a7809aa60 con 0x55d0f0e51410 2024-09-16T01:51:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.423+0000 7f8a8e9e0740 10 client.26144 fetch_fsmap finished waiting for FSMap version 58 2024-09-16T01:51:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.423+0000 7f8a8e9e0740 10 client.26144 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:51:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.423+0000 7f8a8e9e0740 20 client.26144 populate_metadata read hostname 'smithi139' 2024-09-16T01:51:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.423+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/1464219900 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d0f0edf970 0x55d0f0ee1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.423+0000 7f8a8e9e0740 4 client.26144 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:51:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.423+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/1464219900 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0f0dbcd70 con 0x55d0f0edf970 2024-09-16T01:51:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.423+0000 7f8a899e3640 1 --2- 172.21.15.139:0/1464219900 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d0f0edf970 0x55d0f0ee1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.428+0000 7f8a899e3640 1 --2- 172.21.15.139:0/1464219900 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d0f0edf970 0x55d0f0ee1d90 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88df442e506f973c server_cookie=6236ac4e5dfb6588 in_seq=0 out_seq=0 2024-09-16T01:51:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:42.428+0000 7f8a6d7fa640 10 client.26144 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:44.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:43 smithi164 ceph-mon[35121]: pgmap v353: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 463 op/s 2024-09-16T01:51:44.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:43 smithi139 ceph-mon[31175]: pgmap v353: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 463 op/s 2024-09-16T01:51:44.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:43 smithi176 ceph-mon[35324]: pgmap v353: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 463 op/s 2024-09-16T01:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a6d7fa640 1 -- 172.21.15.139:0/1464219900 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0f0dbcd70 con 0x55d0f0edf970 2024-09-16T01:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a6d7fa640 10 client.26144 handle_command_reply: tid=0 2024-09-16T01:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a8e9e0740 1 client.26144 shutdown 2024-09-16T01:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/1464219900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e51410 msgr2=0x55d0f0e559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/1464219900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e51410 0x55d0f0e559e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8a780cfe50 tx=0x7f8a7809c180 comp rx=0 tx=0).stop 2024-09-16T01:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/1464219900 shutdown_connections 2024-09-16T01:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/1464219900 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55d0f0edf970 0x55d0f0ee1d90 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/1464219900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d0f0e5bcb0 0x55d0f0e55f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/1464219900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0f0e51410 0x55d0f0e559e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a6d7fa640 0 client.26144 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a8e9e0740 1 --2- 172.21.15.139:0/1464219900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d0f0e50af0 0x55d0f0e6cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/1464219900 >> 172.21.15.139:0/1464219900 conn(0x55d0f0e4d060 msgr2=0x55d0f0e570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/1464219900 shutdown_connections 2024-09-16T01:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a8e9e0740 1 -- 172.21.15.139:0/1464219900 wait complete. 2024-09-16T01:51:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.009+0000 7f8a8e9e0740 20 client.26144 trim_cache size 0 max 16384 2024-09-16T01:51:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:51:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:51:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:51:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:51:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.010+0000 7f8a8bc47640 1 -- 172.21.15.139:0/3602260016 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f8a700a3640 msgr2=0x7f8a700a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.010+0000 7f8a8bc47640 1 --2- 172.21.15.139:0/3602260016 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f8a700a3640 0x7f8a700a5b00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a8002aa30 tx=0x7f8a80046000 comp rx=0 tx=0).stop 2024-09-16T01:51:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.010+0000 7f8a8bc47640 1 -- 172.21.15.139:0/3602260016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a84101350 msgr2=0x7f8a84073df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.010+0000 7f8a8bc47640 1 --2- 172.21.15.139:0/3602260016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a84101350 0x7f8a84073df0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800d950 tx=0x7f8a7800de10 comp rx=0 tx=0).stop 2024-09-16T01:51:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.010+0000 7f8a8bc47640 1 -- 172.21.15.139:0/3602260016 shutdown_connections 2024-09-16T01:51:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.010+0000 7f8a8bc47640 1 --2- 172.21.15.139:0/3602260016 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f8a700a3640 0x7f8a700a5b00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.010+0000 7f8a8bc47640 1 --2- 172.21.15.139:0/3602260016 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a841069c0 0x7f8a8406ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.010+0000 7f8a8bc47640 1 --2- 172.21.15.139:0/3602260016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a84101350 0x7f8a84073df0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.010+0000 7f8a8bc47640 1 --2- 172.21.15.139:0/3602260016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a841009b0 0x7f8a840738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.010+0000 7f8a8bc47640 1 -- 172.21.15.139:0/3602260016 >> 172.21.15.139:0/3602260016 conn(0x7f8a84077990 msgr2=0x7f8a84107830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.010+0000 7f8a8bc47640 1 -- 172.21.15.139:0/3602260016 shutdown_connections 2024-09-16T01:51:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:45.010+0000 7f8a8bc47640 1 -- 172.21.15.139:0/3602260016 wait complete. 2024-09-16T01:51:46.046 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:45 smithi164 ceph-mon[35121]: pgmap v354: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-09-16T01:51:46.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:45 smithi139 ceph-mon[31175]: pgmap v354: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-09-16T01:51:46.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:45 smithi176 ceph-mon[35324]: pgmap v354: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-09-16T01:51:47.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:46 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:51:47.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:46 smithi164 ceph-mon[35121]: osdmap e98: 12 total, 12 up, 12 in 2024-09-16T01:51:47.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:46 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:51:47.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:46 smithi139 ceph-mon[31175]: osdmap e98: 12 total, 12 up, 12 in 2024-09-16T01:51:47.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:46 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:51:47.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:46 smithi176 ceph-mon[35324]: osdmap e98: 12 total, 12 up, 12 in 2024-09-16T01:51:48.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:47 smithi164 ceph-mon[35121]: pgmap v355: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 441 op/s 2024-09-16T01:51:48.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:47 smithi139 ceph-mon[31175]: pgmap v355: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 441 op/s 2024-09-16T01:51:48.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:47 smithi176 ceph-mon[35324]: pgmap v355: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 441 op/s 2024-09-16T01:51:49.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.129+0000 7f12cf7fe640 1 -- 172.21.15.139:0/2496635173 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f12f0080b20 con 0x562e90041990 2024-09-16T01:51:49.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.129+0000 7f12cf7fe640 10 client.16704 handle_command_reply: tid=0 2024-09-16T01:51:49.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.129+0000 7f12f8c9b740 1 client.16704 shutdown 2024-09-16T01:51:49.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.129+0000 7f12f8c9b740 1 -- 172.21.15.139:0/2496635173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90031c00 msgr2=0x562e90036cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:49.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.129+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/2496635173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90031c00 0x562e90036cb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f12f00518b0 tx=0x7f12f0074650 comp rx=0 tx=0).stop 2024-09-16T01:51:49.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.129+0000 7f12f8c9b740 1 -- 172.21.15.139:0/2496635173 shutdown_connections 2024-09-16T01:51:49.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.129+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/2496635173 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x562e90041990 0x562e900afab0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:49.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.129+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/2496635173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562e90031c00 0x562e90036cb0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:49.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.129+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/2496635173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e90027340 0x562e9003a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:49.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.129+0000 7f12cf7fe640 0 client.16704 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:49.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.129+0000 7f12f8c9b740 1 --2- 172.21.15.139:0/2496635173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e90026a20 0x562e90039ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:49.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.129+0000 7f12f8c9b740 1 -- 172.21.15.139:0/2496635173 >> 172.21.15.139:0/2496635173 conn(0x562e90022fb0 msgr2=0x562e90022050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:49.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.130+0000 7f12f8c9b740 1 -- 172.21.15.139:0/2496635173 shutdown_connections 2024-09-16T01:51:49.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.130+0000 7f12f8c9b740 1 -- 172.21.15.139:0/2496635173 wait complete. 2024-09-16T01:51:49.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.130+0000 7f12f8c9b740 20 client.16704 trim_cache size 0 max 16384 2024-09-16T01:51:49.133 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:51:49.133 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 59, 2024-09-16T01:51:49.133 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:51:49.133 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 38, 2024-09-16T01:51:49.133 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:51:49.134 INFO:teuthology.orchestra.run.smithi139.stdout: "ad0e7e37": { 2024-09-16T01:51:49.134 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 38, 2024-09-16T01:51:49.134 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:51:49.134 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:51:49.134 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:51:49.134 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:51:49.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:51:49.134 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:51:49.135 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 141.8, 2024-09-16T01:51:49.135 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:51:49.135 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:51:49.135 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:51:49.135 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:51:49.135 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:51:49.135 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:51:49.135 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:51:49.136 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:51:49.136 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:51:49.136 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:51:49.136 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:51:49.136 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:51:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.131+0000 7f12f68c2640 1 -- 172.21.15.139:0/857297690 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f12e00a3440 msgr2=0x7f12e00a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.131+0000 7f12f68c2640 1 --2- 172.21.15.139:0/857297690 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f12e00a3440 0x7f12e00a5900 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009230 tx=0x7f12dc0058d0 comp rx=0 tx=0).stop 2024-09-16T01:51:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.131+0000 7f12f68c2640 1 -- 172.21.15.139:0/857297690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12e8095a00 msgr2=0x7f12e8007670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.131+0000 7f12f68c2640 1 --2- 172.21.15.139:0/857297690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12e8095a00 0x7f12e8007670 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f12e400bf40 tx=0x7f12e400dd60 comp rx=0 tx=0).stop 2024-09-16T01:51:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.131+0000 7f12f68c2640 1 -- 172.21.15.139:0/857297690 shutdown_connections 2024-09-16T01:51:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.131+0000 7f12f68c2640 1 --2- 172.21.15.139:0/857297690 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f12e00a3440 0x7f12e00a5900 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.131+0000 7f12f68c2640 1 --2- 172.21.15.139:0/857297690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12e809b640 0x7f12e80080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.131+0000 7f12f68c2640 1 --2- 172.21.15.139:0/857297690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12e80963a0 0x7f12e8007bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.131+0000 7f12f68c2640 1 --2- 172.21.15.139:0/857297690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12e8095a00 0x7f12e8007670 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.131+0000 7f12f68c2640 1 -- 172.21.15.139:0/857297690 >> 172.21.15.139:0/857297690 conn(0x7f12e800b320 msgr2=0x7f12e8094700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.131+0000 7f12f68c2640 1 -- 172.21.15.139:0/857297690 shutdown_connections 2024-09-16T01:51:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:49.131+0000 7f12f68c2640 1 -- 172.21.15.139:0/857297690 wait complete. 2024-09-16T01:51:49.818 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:49 smithi139 ceph-mon[31175]: pgmap v357: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 454 op/s 2024-09-16T01:51:49.850 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ad0e7e37', quiesce took 16.5 seconds. Will release after: 5.350000000000001 2024-09-16T01:51:49.988 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:49 smithi176 ceph-mon[35324]: pgmap v357: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 454 op/s 2024-09-16T01:51:50.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:49 smithi164 ceph-mon[35121]: pgmap v357: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 454 op/s 2024-09-16T01:51:52.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:51 smithi164 ceph-mon[35121]: pgmap v358: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 430 op/s 2024-09-16T01:51:52.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:51 smithi164 ceph-mon[35121]: osdmap e99: 12 total, 12 up, 12 in 2024-09-16T01:51:52.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:51 smithi139 ceph-mon[31175]: pgmap v358: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 430 op/s 2024-09-16T01:51:52.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:51 smithi139 ceph-mon[31175]: osdmap e99: 12 total, 12 up, 12 in 2024-09-16T01:51:52.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:51 smithi176 ceph-mon[35324]: pgmap v358: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 430 op/s 2024-09-16T01:51:52.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:51 smithi176 ceph-mon[35324]: osdmap e99: 12 total, 12 up, 12 in 2024-09-16T01:51:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:53 smithi164 ceph-mon[35121]: pgmap v360: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.0 MiB/s wr, 252 op/s 2024-09-16T01:51:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:53 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:51:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:53 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:53 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:53 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:54.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:53 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:54.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:53 smithi139 ceph-mon[31175]: pgmap v360: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.0 MiB/s wr, 252 op/s 2024-09-16T01:51:54.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:53 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:51:54.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:53 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:54.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:53 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:54.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:53 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:54.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:53 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:54.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:53 smithi176 ceph-mon[35324]: pgmap v360: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.0 MiB/s wr, 252 op/s 2024-09-16T01:51:54.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:53 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:51:54.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:53 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:54.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:53 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:54.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:53 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:54.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:53 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:55.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:54 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:51:55.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:54 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:51:55.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:54 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:55.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:54 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:51:55.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:54 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:51:55.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:54 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:55.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:54 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:51:55.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:54 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:51:55.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:54 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:51:55.201 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:51:55.449 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:51:55.864 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:55 smithi139 ceph-mon[31175]: pgmap v361: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 558 KiB/s wr, 112 op/s 2024-09-16T01:51:55.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.943+0000 7f30cc74f640 1 -- 172.21.15.139:0/731618519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c4101de0 msgr2=0x7f30c41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:55.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.943+0000 7f30cc74f640 1 --2- 172.21.15.139:0/731618519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c4101de0 0x7f30c41021c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009900 tx=0x7f30b802a410 comp rx=0 tx=0).stop 2024-09-16T01:51:55.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.943+0000 7f30cc74f640 1 -- 172.21.15.139:0/731618519 shutdown_connections 2024-09-16T01:51:55.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.943+0000 7f30cc74f640 1 --2- 172.21.15.139:0/731618519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30c4106f10 0x7f30c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:55.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.943+0000 7f30cc74f640 1 --2- 172.21.15.139:0/731618519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30c4102790 0x7f30c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:55.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.943+0000 7f30cc74f640 1 --2- 172.21.15.139:0/731618519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c4101de0 0x7f30c41021c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:55.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.943+0000 7f30cc74f640 1 -- 172.21.15.139:0/731618519 >> 172.21.15.139:0/731618519 conn(0x7f30c40fd8c0 msgr2=0x7f30c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:55.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.943+0000 7f30cc74f640 1 -- 172.21.15.139:0/731618519 shutdown_connections 2024-09-16T01:51:55.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.943+0000 7f30cc74f640 1 -- 172.21.15.139:0/731618519 wait complete. 2024-09-16T01:51:55.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.944+0000 7f30cc74f640 1 Processor -- start 2024-09-16T01:51:55.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.944+0000 7f30cc74f640 1 -- start start 2024-09-16T01:51:55.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.944+0000 7f30cc74f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c4101de0 0x7f30c4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:55.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.944+0000 7f30cc74f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30c4102790 0x7f30c4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:55.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.944+0000 7f30cc74f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30c4106f10 0x7f30c419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:55.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.944+0000 7f30cc74f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f30c4111f30 con 0x7f30c4101de0 2024-09-16T01:51:55.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.944+0000 7f30cc74f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f30c4111db0 con 0x7f30c4106f10 2024-09-16T01:51:55.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.944+0000 7f30cc74f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f30c41120b0 con 0x7f30c4102790 2024-09-16T01:51:55.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.944+0000 7f30ca4eb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c4101de0 0x7f30c4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:55.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.944+0000 7f30ca4eb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c4101de0 0x7f30c4198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51780/0 (socket says 172.21.15.139:51780) 2024-09-16T01:51:55.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.944+0000 7f30ca4eb640 1 -- 172.21.15.139:0/465563331 learned_addr learned my addr 172.21.15.139:0/465563331 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:55.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.944+0000 7f30cacec640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30c4106f10 0x7f30c419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:55.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.945+0000 7f30c9cea640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30c4102790 0x7f30c4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:55.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.945+0000 7f30ca4eb640 1 -- 172.21.15.139:0/465563331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30c4102790 msgr2=0x7f30c4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:55.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.945+0000 7f30ca4eb640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30c4102790 0x7f30c4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:55.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.945+0000 7f30ca4eb640 1 -- 172.21.15.139:0/465563331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30c4106f10 msgr2=0x7f30c419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:55.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.945+0000 7f30ca4eb640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30c4106f10 0x7f30c419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:55.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.945+0000 7f30ca4eb640 1 -- 172.21.15.139:0/465563331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c419d4c0 con 0x7f30c4101de0 2024-09-16T01:51:55.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.945+0000 7f30c9cea640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30c4102790 0x7f30c4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:51:55.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.945+0000 7f30ca4eb640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c4101de0 0x7f30c4198510 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f30b8006010 tx=0x7f30b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:55.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.945+0000 7f30b77fe640 1 -- 172.21.15.139:0/465563331 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b803f030 con 0x7f30c4101de0 2024-09-16T01:51:55.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.945+0000 7f30b77fe640 1 -- 172.21.15.139:0/465563331 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30b8003ce0 con 0x7f30c4101de0 2024-09-16T01:51:55.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.945+0000 7f30cc74f640 1 -- 172.21.15.139:0/465563331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c419d750 con 0x7f30c4101de0 2024-09-16T01:51:55.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.945+0000 7f30b77fe640 1 -- 172.21.15.139:0/465563331 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8035070 con 0x7f30c4101de0 2024-09-16T01:51:55.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.945+0000 7f30cc74f640 1 -- 172.21.15.139:0/465563331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c41a5090 con 0x7f30c4101de0 2024-09-16T01:51:55.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.946+0000 7f30cc74f640 1 -- 172.21.15.139:0/465563331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c41032c0 con 0x7f30c4101de0 2024-09-16T01:51:55.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.947+0000 7f30b77fe640 1 -- 172.21.15.139:0/465563331 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f30b8042030 con 0x7f30c4101de0 2024-09-16T01:51:55.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.948+0000 7f30b77fe640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f30b00a33b0 0x7f30b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:55.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.948+0000 7f30c9cea640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f30b00a33b0 0x7f30b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:55.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.948+0000 7f30b77fe640 1 -- 172.21.15.139:0/465563331 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9538+0+0 (secure 0 0 0) 0x7f30b80f0700 con 0x7f30c4101de0 2024-09-16T01:51:55.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.948+0000 7f30c9cea640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f30b00a33b0 0x7f30b00a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f30c41087b0 tx=0x7f30c0005e50 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:55.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:55.952+0000 7f30b77fe640 1 -- 172.21.15.139:0/465563331 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30b80bcda0 con 0x7f30c4101de0 2024-09-16T01:51:56.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:55 smithi164 ceph-mon[35121]: pgmap v361: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 558 KiB/s wr, 112 op/s 2024-09-16T01:51:56.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:56.160+0000 7f30cc74f640 1 -- 172.21.15.139:0/465563331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30c410a0f0 con 0x7f30c4101de0 2024-09-16T01:51:56.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:56.161+0000 7f30b77fe640 1 -- 172.21.15.139:0/465563331 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+12030 (secure 0 0 0) 0x7f30b80bc6a0 con 0x7f30c4101de0 2024-09-16T01:51:56.162 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:51:56.162 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":59,"btime":"2024-09-16T01:51:45:706357+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":59,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:51:45.576176+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:51:56.162 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 59 2024-09-16T01:51:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:56.163+0000 7f30cc74f640 1 -- 172.21.15.139:0/465563331 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f30b00a33b0 msgr2=0x7f30b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:56.163+0000 7f30cc74f640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f30b00a33b0 0x7f30b00a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f30c41087b0 tx=0x7f30c0005e50 comp rx=0 tx=0).stop 2024-09-16T01:51:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:56.163+0000 7f30cc74f640 1 -- 172.21.15.139:0/465563331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c4101de0 msgr2=0x7f30c4198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:56.163+0000 7f30cc74f640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c4101de0 0x7f30c4198510 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f30b8006010 tx=0x7f30b80042f0 comp rx=0 tx=0).stop 2024-09-16T01:51:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:56.163+0000 7f30cc74f640 1 -- 172.21.15.139:0/465563331 shutdown_connections 2024-09-16T01:51:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:56.163+0000 7f30cc74f640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f30b00a33b0 0x7f30b00a5870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:56.163+0000 7f30cc74f640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30c4106f10 0x7f30c419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:56.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:56.163+0000 7f30cc74f640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30c4102790 0x7f30c4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:56.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:56.163+0000 7f30cc74f640 1 --2- 172.21.15.139:0/465563331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c4101de0 0x7f30c4198510 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:56.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:56.163+0000 7f30cc74f640 1 -- 172.21.15.139:0/465563331 >> 172.21.15.139:0/465563331 conn(0x7f30c40fd8c0 msgr2=0x7f30c40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:56.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:56.163+0000 7f30cc74f640 1 -- 172.21.15.139:0/465563331 shutdown_connections 2024-09-16T01:51:56.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:56.163+0000 7f30cc74f640 1 -- 172.21.15.139:0/465563331 wait complete. 2024-09-16T01:51:56.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:55 smithi176 ceph-mon[35324]: pgmap v361: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 558 KiB/s wr, 112 op/s 2024-09-16T01:51:56.900 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id ad0e7e37 --release --await' 2024-09-16T01:51:56.901 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id ad0e7e37 --release --await 2024-09-16T01:51:57.149 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:51:57.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:56 smithi139 ceph-mon[31175]: pgmap v362: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 449 KiB/s wr, 93 op/s 2024-09-16T01:51:57.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:56 smithi139 ceph-mon[31175]: osdmap e100: 12 total, 12 up, 12 in 2024-09-16T01:51:57.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:56 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/465563331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:51:57.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:56 smithi164 ceph-mon[35121]: pgmap v362: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 449 KiB/s wr, 93 op/s 2024-09-16T01:51:57.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:56 smithi164 ceph-mon[35121]: osdmap e100: 12 total, 12 up, 12 in 2024-09-16T01:51:57.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:56 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/465563331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:51:57.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:56 smithi176 ceph-mon[35324]: pgmap v362: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 449 KiB/s wr, 93 op/s 2024-09-16T01:51:57.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:56 smithi176 ceph-mon[35324]: osdmap e100: 12 total, 12 up, 12 in 2024-09-16T01:51:57.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:56 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/465563331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:51:57.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.643+0000 7fce98062640 1 -- 172.21.15.139:0/2941590058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce90112640 msgr2=0x7fce90112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:57.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.643+0000 7fce98062640 1 --2- 172.21.15.139:0/2941590058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce90112640 0x7fce90112a20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009900 tx=0x7fce8c02a410 comp rx=0 tx=0).stop 2024-09-16T01:51:57.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.643+0000 7fce98062640 1 -- 172.21.15.139:0/2941590058 shutdown_connections 2024-09-16T01:51:57.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.643+0000 7fce98062640 1 --2- 172.21.15.139:0/2941590058 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce9010cf30 0x7fce90119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.643+0000 7fce98062640 1 --2- 172.21.15.139:0/2941590058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce9010c590 0x7fce9010c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.643+0000 7fce98062640 1 --2- 172.21.15.139:0/2941590058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce90112640 0x7fce90112a20 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.643+0000 7fce98062640 1 -- 172.21.15.139:0/2941590058 >> 172.21.15.139:0/2941590058 conn(0x7fce9006cdc0 msgr2=0x7fce9006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:57.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.643+0000 7fce98062640 1 -- 172.21.15.139:0/2941590058 shutdown_connections 2024-09-16T01:51:57.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.643+0000 7fce98062640 1 -- 172.21.15.139:0/2941590058 wait complete. 2024-09-16T01:51:57.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.643+0000 7fce98062640 1 Processor -- start 2024-09-16T01:51:57.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.643+0000 7fce98062640 1 -- start start 2024-09-16T01:51:57.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce98062640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce9010c590 0x7fce901a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce98062640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce9010cf30 0x7fce901a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce98062640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce90112640 0x7fce901a7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce98062640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fce9011c960 con 0x7fce90112640 2024-09-16T01:51:57.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce98062640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce9011c7e0 con 0x7fce9010c590 2024-09-16T01:51:57.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce98062640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fce9011cae0 con 0x7fce9010cf30 2024-09-16T01:51:57.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce965ff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce90112640 0x7fce901a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce965ff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce90112640 0x7fce901a7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51796/0 (socket says 172.21.15.139:51796) 2024-09-16T01:51:57.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce965ff640 1 -- 172.21.15.139:0/2639553620 learned_addr learned my addr 172.21.15.139:0/2639553620 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:57.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce95dfe640 1 --2- 172.21.15.139:0/2639553620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce9010c590 0x7fce901a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce955fd640 1 --2- 172.21.15.139:0/2639553620 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce9010cf30 0x7fce901a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce965ff640 1 -- 172.21.15.139:0/2639553620 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce9010cf30 msgr2=0x7fce901a33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:57.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce965ff640 1 --2- 172.21.15.139:0/2639553620 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce9010cf30 0x7fce901a33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce965ff640 1 -- 172.21.15.139:0/2639553620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce9010c590 msgr2=0x7fce901a2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:57.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce965ff640 1 --2- 172.21.15.139:0/2639553620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce9010c590 0x7fce901a2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce965ff640 1 -- 172.21.15.139:0/2639553620 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce901a7e50 con 0x7fce90112640 2024-09-16T01:51:57.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce965ff640 1 --2- 172.21.15.139:0/2639553620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce90112640 0x7fce901a7770 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fce8800eaa0 tx=0x7fce8800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:57.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce86ffd640 1 -- 172.21.15.139:0/2639553620 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8800cdb0 con 0x7fce90112640 2024-09-16T01:51:57.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce86ffd640 1 -- 172.21.15.139:0/2639553620 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce88005020 con 0x7fce90112640 2024-09-16T01:51:57.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.644+0000 7fce98062640 1 -- 172.21.15.139:0/2639553620 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce901a8140 con 0x7fce90112640 2024-09-16T01:51:57.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.645+0000 7fce86ffd640 1 -- 172.21.15.139:0/2639553620 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce880116b0 con 0x7fce90112640 2024-09-16T01:51:57.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.645+0000 7fce98062640 1 -- 172.21.15.139:0/2639553620 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce901afa20 con 0x7fce90112640 2024-09-16T01:51:57.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.647+0000 7fce86ffd640 1 -- 172.21.15.139:0/2639553620 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fce88005560 con 0x7fce90112640 2024-09-16T01:51:57.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.647+0000 7fce9a43b740 1 Processor -- start 2024-09-16T01:51:57.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce9a43b740 1 -- start start 2024-09-16T01:51:57.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce9a43b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437060 0x560f75437440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce9a43b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75437980 0x560f75441d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce9a43b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75442240 0x560f75444630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce9a43b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560f7539c620 con 0x560f75437980 2024-09-16T01:51:57.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce9a43b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f75343e50 con 0x560f75442240 2024-09-16T01:51:57.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce9a43b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560f753985e0 con 0x560f75437060 2024-09-16T01:51:57.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce95dfe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75437980 0x560f75441d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce95dfe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75437980 0x560f75441d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51800/0 (socket says 172.21.15.139:51800) 2024-09-16T01:51:57.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce95dfe640 1 -- 172.21.15.139:0/113806487 learned_addr learned my addr 172.21.15.139:0/113806487 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:57.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce955fd640 1 --2- 172.21.15.139:0/113806487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437060 0x560f75437440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce955fd640 1 --2- 172.21.15.139:0/113806487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75442240 0x560f75444630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce86ffd640 1 --2- 172.21.15.139:0/2639553620 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fce7c0a33b0 0x7fce7c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce95dfe640 1 -- 172.21.15.139:0/113806487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437060 msgr2=0x560f75437440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:57.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce95dfe640 1 --2- 172.21.15.139:0/113806487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437060 0x560f75437440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce95dfe640 1 -- 172.21.15.139:0/113806487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75442240 msgr2=0x560f75444630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:57.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce95dfe640 1 --2- 172.21.15.139:0/113806487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75442240 0x560f75444630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce95dfe640 1 -- 172.21.15.139:0/113806487 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f75334d40 con 0x560f75437980 2024-09-16T01:51:57.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.648+0000 7fce955fd640 1 --2- 172.21.15.139:0/113806487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437060 0x560f75437440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:51:57.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.649+0000 7fce95dfe640 1 --2- 172.21.15.139:0/113806487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75437980 0x560f75441d00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fce8c02a930 tx=0x7fce8c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64819c946098ae1e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:57.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.649+0000 7fce965ff640 1 --2- 172.21.15.139:0/2639553620 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fce7c0a33b0 0x7fce7c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.649+0000 7fce86ffd640 1 -- 172.21.15.139:0/2639553620 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9510+0+0 (secure 0 0 0) 0x7fce880d0240 con 0x7fce90112640 2024-09-16T01:51:57.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.649+0000 7fce7bfff640 1 -- 172.21.15.139:0/113806487 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c035030 con 0x560f75437980 2024-09-16T01:51:57.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.649+0000 7fce7bfff640 1 -- 172.21.15.139:0/113806487 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce8c040070 con 0x560f75437980 2024-09-16T01:51:57.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.649+0000 7fce7bfff640 1 -- 172.21.15.139:0/113806487 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c0319d0 con 0x560f75437980 2024-09-16T01:51:57.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.650+0000 7fce9a43b740 1 -- 172.21.15.139:0/113806487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75437980 msgr2=0x560f75441d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:57.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.650+0000 7fce965ff640 1 --2- 172.21.15.139:0/2639553620 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fce7c0a33b0 0x7fce7c0a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fce88098bc0 tx=0x7fce88091680 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:57.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.650+0000 7fce9a43b740 1 --2- 172.21.15.139:0/113806487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75437980 0x560f75441d00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fce8c02a930 tx=0x7fce8c0040c0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.650+0000 7fce9a43b740 1 -- 172.21.15.139:0/113806487 shutdown_connections 2024-09-16T01:51:57.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.650+0000 7fce9a43b740 1 --2- 172.21.15.139:0/113806487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75442240 0x560f75444630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.650+0000 7fce9a43b740 1 --2- 172.21.15.139:0/113806487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75437980 0x560f75441d00 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.650+0000 7fce9a43b740 1 --2- 172.21.15.139:0/113806487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437060 0x560f75437440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.650+0000 7fce9a43b740 1 -- 172.21.15.139:0/113806487 >> 172.21.15.139:0/113806487 conn(0x560f754335f0 msgr2=0x560f754339d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:57.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.650+0000 7fce9a43b740 1 -- 172.21.15.139:0/113806487 shutdown_connections 2024-09-16T01:51:57.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.650+0000 7fce9a43b740 1 -- 172.21.15.139:0/113806487 wait complete. 2024-09-16T01:51:57.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.650+0000 7fce9a43b740 1 -- start start 2024-09-16T01:51:57.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.650+0000 7fce9a43b740 1 Processor -- start 2024-09-16T01:51:57.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce9a43b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437060 0x560f7543f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce9a43b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437980 0x560f7543f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce9a43b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75442240 0x560f7543fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce9a43b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560f75444e10 con 0x560f75442240 2024-09-16T01:51:57.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce9a43b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f75324f10 con 0x560f75437060 2024-09-16T01:51:57.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce9a43b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560f75444f90 con 0x560f75437980 2024-09-16T01:51:57.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce9a43b740 10 client.0 _ll_register_callbacks cb 0x560f75348db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:51:57.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce955fd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75442240 0x560f7543fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce955fd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75442240 0x560f7543fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51816/0 (socket says 172.21.15.139:51816) 2024-09-16T01:51:57.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce955fd640 1 -- 172.21.15.139:0/3744970818 learned_addr learned my addr 172.21.15.139:0/3744970818 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:57.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce95dfe640 1 --2- 172.21.15.139:0/3744970818 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437980 0x560f7543f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce955fd640 1 -- 172.21.15.139:0/3744970818 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437980 msgr2=0x560f7543f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:57.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce955fd640 1 --2- 172.21.15.139:0/3744970818 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437980 0x560f7543f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce955fd640 1 -- 172.21.15.139:0/3744970818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437060 msgr2=0x560f7543f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:57.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce955fd640 1 --2- 172.21.15.139:0/3744970818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437060 0x560f7543f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce955fd640 1 -- 172.21.15.139:0/3744970818 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f750ab580 con 0x560f75442240 2024-09-16T01:51:57.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce955fd640 1 --2- 172.21.15.139:0/3744970818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75442240 0x560f7543fc00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fce80004570 tx=0x7fce8000bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c09b080502a0b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:57.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce797fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:51:57.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce797fa640 1 -- 172.21.15.139:0/3744970818 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80018030 con 0x560f75442240 2024-09-16T01:51:57.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce797fa640 1 -- 172.21.15.139:0/3744970818 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9510+0+0 (secure 0 0 0) 0x7fce80012450 con 0x560f75442240 2024-09-16T01:51:57.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.651+0000 7fce9a43b740 1 -- 172.21.15.139:0/3744970818 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce8c040070 con 0x560f75442240 2024-09-16T01:51:57.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.652+0000 7fce797fa640 1 -- 172.21.15.139:0/3744970818 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fce80012650 con 0x560f75442240 2024-09-16T01:51:57.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.652+0000 7fce9a43b740 10 client.16740 fetch_fsmap learned FSMap version 59 2024-09-16T01:51:57.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.652+0000 7fce9a43b740 1 -- 172.21.15.139:0/3744970818 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x560f75175960 con 0x560f75442240 2024-09-16T01:51:57.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.652+0000 7fce797fa640 1 -- 172.21.15.139:0/3744970818 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce8c040070 con 0x560f75442240 2024-09-16T01:51:57.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.652+0000 7fce797fa640 1 -- 172.21.15.139:0/3744970818 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 59) ==== 8450+0+0 (secure 0 0 0) 0x7fce8001d9b0 con 0x560f75442240 2024-09-16T01:51:57.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.652+0000 7fce9a43b740 10 client.16740 fetch_fsmap finished waiting for FSMap version 59 2024-09-16T01:51:57.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.652+0000 7fce9a43b740 10 client.16740 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.652+0000 7fce9a43b740 20 client.16740 populate_metadata read hostname 'smithi139' 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.652+0000 7fce9a43b740 1 --2- 172.21.15.139:0/3744970818 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560f75451e60 0x560f75454290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.652+0000 7fce9a43b740 4 client.16740 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.652+0000 7fce9a43b740 1 -- 172.21.15.139:0/3744970818 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f753a9b90 con 0x560f75451e60 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.652+0000 7fce95dfe640 1 --2- 172.21.15.139:0/3744970818 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560f75451e60 0x560f75454290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.653+0000 7fce95dfe640 1 --2- 172.21.15.139:0/3744970818 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560f75451e60 0x560f75454290 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c233aaf29382a57 server_cookie=c6acbda1d2fbd176 in_seq=0 out_seq=0 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.653+0000 7fce797fa640 10 client.16740 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.653+0000 7fce797fa640 1 -- 172.21.15.139:0/3744970818 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f753a9b90 con 0x560f75451e60 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.653+0000 7fce797fa640 10 client.16740 handle_command_reply: tid=0 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.653+0000 7fce9a43b740 1 client.16740 shutdown 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.654+0000 7fce9a43b740 1 -- 172.21.15.139:0/3744970818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75442240 msgr2=0x560f7543fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.654+0000 7fce9a43b740 1 --2- 172.21.15.139:0/3744970818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75442240 0x560f7543fc00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fce80004570 tx=0x7fce8000bfb0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.654+0000 7fce9a43b740 1 -- 172.21.15.139:0/3744970818 shutdown_connections 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.654+0000 7fce9a43b740 1 --2- 172.21.15.139:0/3744970818 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560f75451e60 0x560f75454290 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.654+0000 7fce9a43b740 1 --2- 172.21.15.139:0/3744970818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75442240 0x560f7543fc00 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.654+0000 7fce9a43b740 1 --2- 172.21.15.139:0/3744970818 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437980 0x560f7543f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.654+0000 7fce9a43b740 1 --2- 172.21.15.139:0/3744970818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437060 0x560f7543f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.654+0000 7fce797fa640 0 client.16740 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.654+0000 7fce9a43b740 1 -- 172.21.15.139:0/3744970818 >> 172.21.15.139:0/3744970818 conn(0x560f754335f0 msgr2=0x560f754392d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.654+0000 7fce9a43b740 1 -- 172.21.15.139:0/3744970818 shutdown_connections 2024-09-16T01:51:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.654+0000 7fce9a43b740 1 -- 172.21.15.139:0/3744970818 wait complete. 2024-09-16T01:51:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.654+0000 7fce9a43b740 20 client.16740 trim_cache size 0 max 16384 2024-09-16T01:51:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.670+0000 7fce9a43b740 1 Processor -- start 2024-09-16T01:51:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.670+0000 7fce9a43b740 1 -- start start 2024-09-16T01:51:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.670+0000 7fce9a43b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437060 0x560f75432a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.670+0000 7fce9a43b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437980 0x560f75439ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.670+0000 7fce9a43b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75442240 0x560f75459ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.670+0000 7fce9a43b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fce901b9b80 con 0x560f75442240 2024-09-16T01:51:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.670+0000 7fce9a43b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce901bb9c0 con 0x560f75437060 2024-09-16T01:51:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.670+0000 7fce9a43b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fce901ba310 con 0x560f75437980 2024-09-16T01:51:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.670+0000 7fce955fd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75442240 0x560f75459ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.670+0000 7fce955fd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75442240 0x560f75459ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51828/0 (socket says 172.21.15.139:51828) 2024-09-16T01:51:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.670+0000 7fce955fd640 1 -- 172.21.15.139:0/1027835656 learned_addr learned my addr 172.21.15.139:0/1027835656 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.670+0000 7fce955fd640 1 --2- 172.21.15.139:0/1027835656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437060 0x560f75432a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.670+0000 7fce95dfe640 1 --2- 172.21.15.139:0/1027835656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437980 0x560f75439ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.671+0000 7fce955fd640 1 -- 172.21.15.139:0/1027835656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437980 msgr2=0x560f75439ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.671+0000 7fce955fd640 1 --2- 172.21.15.139:0/1027835656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437980 0x560f75439ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.671+0000 7fce955fd640 1 -- 172.21.15.139:0/1027835656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437060 msgr2=0x560f75432a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:57.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.671+0000 7fce955fd640 1 --2- 172.21.15.139:0/1027835656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437060 0x560f75432a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.671+0000 7fce955fd640 1 -- 172.21.15.139:0/1027835656 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f7543e130 con 0x560f75442240 2024-09-16T01:51:57.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.671+0000 7fce955fd640 1 --2- 172.21.15.139:0/1027835656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75442240 0x560f75459ba0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fce80004310 tx=0x7fce800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3608437cd8ed33db server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:57.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.671+0000 7fce7b7fe640 1 -- 172.21.15.139:0/1027835656 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80018070 con 0x560f75442240 2024-09-16T01:51:57.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.671+0000 7fce7b7fe640 1 -- 172.21.15.139:0/1027835656 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce800127f0 con 0x560f75442240 2024-09-16T01:51:57.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.671+0000 7fce7b7fe640 1 -- 172.21.15.139:0/1027835656 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8001d800 con 0x560f75442240 2024-09-16T01:51:57.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 -- 172.21.15.139:0/1027835656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75442240 msgr2=0x560f75459ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:57.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 --2- 172.21.15.139:0/1027835656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75442240 0x560f75459ba0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fce80004310 tx=0x7fce800046c0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 -- 172.21.15.139:0/1027835656 shutdown_connections 2024-09-16T01:51:57.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 --2- 172.21.15.139:0/1027835656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75442240 0x560f75459ba0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 --2- 172.21.15.139:0/1027835656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75437980 0x560f75439ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 --2- 172.21.15.139:0/1027835656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437060 0x560f75432a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 -- 172.21.15.139:0/1027835656 >> 172.21.15.139:0/1027835656 conn(0x560f754335f0 msgr2=0x560f754409e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:57.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 -- 172.21.15.139:0/1027835656 shutdown_connections 2024-09-16T01:51:57.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 -- 172.21.15.139:0/1027835656 wait complete. 2024-09-16T01:51:57.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 -- start start 2024-09-16T01:51:57.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 Processor -- start 2024-09-16T01:51:57.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75437060 0x560f7544a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437980 0x560f75446bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75442240 0x560f75447100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fce901bb230 con 0x560f75437060 2024-09-16T01:51:57.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce901baaa0 con 0x560f75437980 2024-09-16T01:51:57.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.672+0000 7fce9a43b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fce901b7b80 con 0x560f75442240 2024-09-16T01:51:57.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce9a43b740 10 client.0 _ll_register_callbacks cb 0x7fce800043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:51:57.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce955fd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75437060 0x560f7544a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce955fd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75437060 0x560f7544a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51832/0 (socket says 172.21.15.139:51832) 2024-09-16T01:51:57.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce955fd640 1 -- 172.21.15.139:0/3230272170 learned_addr learned my addr 172.21.15.139:0/3230272170 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:57.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce955fd640 1 --2- 172.21.15.139:0/3230272170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75442240 0x560f75447100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce95dfe640 1 --2- 172.21.15.139:0/3230272170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437980 0x560f75446bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce955fd640 1 -- 172.21.15.139:0/3230272170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75442240 msgr2=0x560f75447100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:57.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce955fd640 1 --2- 172.21.15.139:0/3230272170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75442240 0x560f75447100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce955fd640 1 -- 172.21.15.139:0/3230272170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437980 msgr2=0x560f75446bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:57.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce955fd640 1 --2- 172.21.15.139:0/3230272170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437980 0x560f75446bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:57.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce955fd640 1 -- 172.21.15.139:0/3230272170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f754477f0 con 0x560f75437060 2024-09-16T01:51:57.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce95dfe640 1 --2- 172.21.15.139:0/3230272170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437980 0x560f75446bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:51:57.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce955fd640 1 --2- 172.21.15.139:0/3230272170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75437060 0x560f7544a7e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fce8000c010 tx=0x7fce80005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b6b9ee45f83c73 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:57.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce797fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:51:57.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce797fa640 1 -- 172.21.15.139:0/3230272170 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80018070 con 0x560f75437060 2024-09-16T01:51:57.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce797fa640 1 -- 172.21.15.139:0/3230272170 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9510+0+0 (secure 0 0 0) 0x7fce80005260 con 0x560f75437060 2024-09-16T01:51:57.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce9a43b740 1 -- 172.21.15.139:0/3230272170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce800127f0 con 0x560f75437060 2024-09-16T01:51:57.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce797fa640 1 -- 172.21.15.139:0/3230272170 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fce80045050 con 0x560f75437060 2024-09-16T01:51:57.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce9a43b740 10 client.16746 fetch_fsmap learned FSMap version 59 2024-09-16T01:51:57.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.673+0000 7fce9a43b740 1 -- 172.21.15.139:0/3230272170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x560f754c6270 con 0x560f75437060 2024-09-16T01:51:57.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.674+0000 7fce797fa640 1 -- 172.21.15.139:0/3230272170 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce800127f0 con 0x560f75437060 2024-09-16T01:51:57.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.674+0000 7fce797fa640 1 -- 172.21.15.139:0/3230272170 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 59) ==== 8450+0+0 (secure 0 0 0) 0x7fce80038070 con 0x560f75437060 2024-09-16T01:51:57.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.674+0000 7fce9a43b740 10 client.16746 fetch_fsmap finished waiting for FSMap version 59 2024-09-16T01:51:57.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.674+0000 7fce9a43b740 10 client.16746 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:51:57.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.674+0000 7fce9a43b740 20 client.16746 populate_metadata read hostname 'smithi139' 2024-09-16T01:51:57.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.674+0000 7fce9a43b740 1 --2- 172.21.15.139:0/3230272170 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560f754c68d0 0x560f754c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:57.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.674+0000 7fce9a43b740 4 client.16746 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ad0e7e37", "release": true, "await": true}] 2024-09-16T01:51:57.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.674+0000 7fce9a43b740 1 -- 172.21.15.139:0/3230272170 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ad0e7e37", "release": true, "await": true}) -- 0x560f753a40a0 con 0x560f754c68d0 2024-09-16T01:51:57.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.674+0000 7fce95dfe640 1 --2- 172.21.15.139:0/3230272170 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560f754c68d0 0x560f754c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:57.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.674+0000 7fce797fa640 10 client.16746 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:57.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:57.674+0000 7fce95dfe640 1 --2- 172.21.15.139:0/3230272170 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560f754c68d0 0x560f754c8cf0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19d6a5a4cecfacda server_cookie=9001c3bf07afb178 in_seq=0 out_seq=0 2024-09-16T01:51:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.100+0000 7fd815b91640 1 -- 172.21.15.139:0/1399923356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8101009d0 msgr2=0x7fd810100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.100+0000 7fd815b91640 1 --2- 172.21.15.139:0/1399923356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8101009d0 0x7fd810100e30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd804009930 tx=0x7fd80402a420 comp rx=0 tx=0).stop 2024-09-16T01:51:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.100+0000 7fd815b91640 1 -- 172.21.15.139:0/1399923356 shutdown_connections 2024-09-16T01:51:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.100+0000 7fd815b91640 1 --2- 172.21.15.139:0/1399923356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd810101370 0x7fd81010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.100+0000 7fd815b91640 1 --2- 172.21.15.139:0/1399923356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8101009d0 0x7fd810100e30 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.100+0000 7fd815b91640 1 --2- 172.21.15.139:0/1399923356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8101069e0 0x7fd810106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.100+0000 7fd815b91640 1 -- 172.21.15.139:0/1399923356 >> 172.21.15.139:0/1399923356 conn(0x7fd8100fc6f0 msgr2=0x7fd8100feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.100+0000 7fd815b91640 1 -- 172.21.15.139:0/1399923356 shutdown_connections 2024-09-16T01:51:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.100+0000 7fd815b91640 1 -- 172.21.15.139:0/1399923356 wait complete. 2024-09-16T01:51:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.100+0000 7fd815b91640 1 Processor -- start 2024-09-16T01:51:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.100+0000 7fd815b91640 1 -- start start 2024-09-16T01:51:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd815b91640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8101009d0 0x7fd8100fcc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd815b91640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd810101370 0x7fd8100fd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd815b91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8101069e0 0x7fd8100fd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd815b91640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd810110e20 con 0x7fd8101069e0 2024-09-16T01:51:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd815b91640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd810110ca0 con 0x7fd8101009d0 2024-09-16T01:51:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd815b91640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd810110fa0 con 0x7fd810101370 2024-09-16T01:51:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8101069e0 0x7fd8100fd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8101009d0 0x7fd8100fcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd810101370 0x7fd8100fd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8101009d0 0x7fd8100fcc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:50890/0 (socket says 172.21.15.139:50890) 2024-09-16T01:51:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80f7fe640 1 -- 172.21.15.139:0/4085972006 learned_addr learned my addr 172.21.15.139:0/4085972006 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80ffff640 1 -- 172.21.15.139:0/4085972006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd810101370 msgr2=0x7fd8100fd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80ffff640 1 --2- 172.21.15.139:0/4085972006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd810101370 0x7fd8100fd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80ffff640 1 -- 172.21.15.139:0/4085972006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8101009d0 msgr2=0x7fd8100fcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80ffff640 1 --2- 172.21.15.139:0/4085972006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8101009d0 0x7fd8100fcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80ffff640 1 -- 172.21.15.139:0/4085972006 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8101a68d0 con 0x7fd8101069e0 2024-09-16T01:51:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80effd640 1 --2- 172.21.15.139:0/4085972006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd810101370 0x7fd8100fd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:51:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80ffff640 1 --2- 172.21.15.139:0/4085972006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8101069e0 0x7fd8100fd6f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd808009fd0 tx=0x7fd80800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80cff9640 1 -- 172.21.15.139:0/4085972006 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80800ce20 con 0x7fd8101069e0 2024-09-16T01:51:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80cff9640 1 -- 172.21.15.139:0/4085972006 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd808005020 con 0x7fd8101069e0 2024-09-16T01:51:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd80cff9640 1 -- 172.21.15.139:0/4085972006 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808010720 con 0x7fd8101069e0 2024-09-16T01:51:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.101+0000 7fd815b91640 1 -- 172.21.15.139:0/4085972006 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8101a6bc0 con 0x7fd8101069e0 2024-09-16T01:51:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.102+0000 7fd815b91640 1 -- 172.21.15.139:0/4085972006 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8101a7130 con 0x7fd8101069e0 2024-09-16T01:51:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.104+0000 7fd80cff9640 1 -- 172.21.15.139:0/4085972006 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd808005540 con 0x7fd8101069e0 2024-09-16T01:51:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.104+0000 7fd818a0d740 1 Processor -- start 2024-09-16T01:51:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.104+0000 7fd818a0d740 1 -- start start 2024-09-16T01:51:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.104+0000 7fd818a0d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b07af0 0x55e294b07ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.104+0000 7fd818a0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b12770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.104+0000 7fd818a0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 0x55e294b150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.104+0000 7fd818a0d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e294a6c2f0 con 0x55e294b08410 2024-09-16T01:51:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.104+0000 7fd818a0d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e294a14270 con 0x55e294b12cb0 2024-09-16T01:51:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.104+0000 7fd818a0d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e294a682b0 con 0x55e294b07af0 2024-09-16T01:51:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.104+0000 7fd80f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b12770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.104+0000 7fd80effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b07af0 0x55e294b07ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.104+0000 7fd80effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b07af0 0x55e294b07ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:60838/0 (socket says 172.21.15.139:60838) 2024-09-16T01:51:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.104+0000 7fd80effd640 1 -- 172.21.15.139:0/1710830238 learned_addr learned my addr 172.21.15.139:0/1710830238 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.105+0000 7fd80effd640 1 --2- 172.21.15.139:0/1710830238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 0x55e294b150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.105+0000 7fd80cff9640 1 --2- 172.21.15.139:0/4085972006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd7f80a34e0 0x7fd7f80a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.105+0000 7fd80effd640 1 -- 172.21.15.139:0/1710830238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 msgr2=0x55e294b150a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.105+0000 7fd80effd640 1 --2- 172.21.15.139:0/1710830238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 0x55e294b150a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.105+0000 7fd80effd640 1 -- 172.21.15.139:0/1710830238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 msgr2=0x55e294b12770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.105+0000 7fd80effd640 1 --2- 172.21.15.139:0/1710830238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b12770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.105+0000 7fd80effd640 1 -- 172.21.15.139:0/1710830238 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e294a13340 con 0x55e294b07af0 2024-09-16T01:51:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.105+0000 7fd80f7fe640 1 --2- 172.21.15.139:0/1710830238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b12770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:51:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.105+0000 7fd80ffff640 1 --2- 172.21.15.139:0/4085972006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd7f80a34e0 0x7fd7f80a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.105+0000 7fd80cff9640 1 -- 172.21.15.139:0/4085972006 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9510+0+0 (secure 0 0 0) 0x7fd8080d06f0 con 0x7fd8101069e0 2024-09-16T01:51:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.105+0000 7fd80effd640 1 --2- 172.21.15.139:0/1710830238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b07af0 0x55e294b07ed0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd804006010 tx=0x7fd804031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85e2a4a3adc596a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.105+0000 7fd7f5ffb640 1 -- 172.21.15.139:0/1710830238 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804035070 con 0x55e294b07af0 2024-09-16T01:51:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.105+0000 7fd7f5ffb640 1 -- 172.21.15.139:0/1710830238 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd804005500 con 0x55e294b07af0 2024-09-16T01:51:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.105+0000 7fd7f5ffb640 1 -- 172.21.15.139:0/1710830238 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804004bc0 con 0x55e294b07af0 2024-09-16T01:51:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.106+0000 7fd818a0d740 1 -- 172.21.15.139:0/1710830238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b07af0 msgr2=0x55e294b07ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.106+0000 7fd80ffff640 1 --2- 172.21.15.139:0/4085972006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd7f80a34e0 0x7fd7f80a59a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd808098f60 tx=0x7fd8080054b0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.106+0000 7fd818a0d740 1 --2- 172.21.15.139:0/1710830238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b07af0 0x55e294b07ed0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd804006010 tx=0x7fd804031400 comp rx=0 tx=0).stop 2024-09-16T01:51:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.106+0000 7fd818a0d740 1 -- 172.21.15.139:0/1710830238 shutdown_connections 2024-09-16T01:51:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.106+0000 7fd818a0d740 1 --2- 172.21.15.139:0/1710830238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 0x55e294b150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.106+0000 7fd818a0d740 1 --2- 172.21.15.139:0/1710830238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b12770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.106+0000 7fd818a0d740 1 --2- 172.21.15.139:0/1710830238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b07af0 0x55e294b07ed0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.106+0000 7fd818a0d740 1 -- 172.21.15.139:0/1710830238 >> 172.21.15.139:0/1710830238 conn(0x55e294b04060 msgr2=0x55e294b04440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.106+0000 7fd818a0d740 1 -- 172.21.15.139:0/1710830238 shutdown_connections 2024-09-16T01:51:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.106+0000 7fd818a0d740 1 -- 172.21.15.139:0/1710830238 wait complete. 2024-09-16T01:51:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.106+0000 7fd818a0d740 1 -- start start 2024-09-16T01:51:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.106+0000 7fd818a0d740 1 Processor -- start 2024-09-16T01:51:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd818a0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b07af0 0x55e294b0fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd818a0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd818a0d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b12cb0 0x55e294b105e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd818a0d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e294b15880 con 0x55e294b08410 2024-09-16T01:51:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd818a0d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e2949f5290 con 0x55e294b07af0 2024-09-16T01:51:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd818a0d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e294b15a00 con 0x55e294b12cb0 2024-09-16T01:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd818a0d740 10 client.0 _ll_register_callbacks cb 0x55e294a26140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd80f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd80effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b07af0 0x55e294b0fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd80f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b100a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51866/0 (socket says 172.21.15.139:51866) 2024-09-16T01:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd80f7fe640 1 -- 172.21.15.139:0/3112541452 learned_addr learned my addr 172.21.15.139:0/3112541452 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd80effd640 1 --2- 172.21.15.139:0/3112541452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b12cb0 0x55e294b105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd80f7fe640 1 -- 172.21.15.139:0/3112541452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b12cb0 msgr2=0x55e294b105e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd80f7fe640 1 --2- 172.21.15.139:0/3112541452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b12cb0 0x55e294b105e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd80f7fe640 1 -- 172.21.15.139:0/3112541452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b07af0 msgr2=0x55e294b0fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd80f7fe640 1 --2- 172.21.15.139:0/3112541452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b07af0 0x55e294b0fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd80f7fe640 1 -- 172.21.15.139:0/3112541452 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e29477f210 con 0x55e294b08410 2024-09-16T01:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd80f7fe640 1 --2- 172.21.15.139:0/3112541452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b100a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd8000127c0 tx=0x7fd800012c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59e4820feac8406 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd7eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd7eeffd640 1 -- 172.21.15.139:0/3112541452 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800004030 con 0x55e294b08410 2024-09-16T01:51:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd7eeffd640 1 -- 172.21.15.139:0/3112541452 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9510+0+0 (secure 0 0 0) 0x7fd8000041d0 con 0x55e294b08410 2024-09-16T01:51:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.107+0000 7fd818a0d740 1 -- 172.21.15.139:0/3112541452 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd804005500 con 0x55e294b08410 2024-09-16T01:51:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.108+0000 7fd7eeffd640 1 -- 172.21.15.139:0/3112541452 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fd8000043d0 con 0x55e294b08410 2024-09-16T01:51:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.108+0000 7fd818a0d740 10 client.16758 fetch_fsmap learned FSMap version 59 2024-09-16T01:51:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.108+0000 7fd818a0d740 1 -- 172.21.15.139:0/3112541452 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55e29484b620 con 0x55e294b08410 2024-09-16T01:51:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.108+0000 7fd7eeffd640 1 -- 172.21.15.139:0/3112541452 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd804005500 con 0x55e294b08410 2024-09-16T01:51:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.108+0000 7fd7eeffd640 1 -- 172.21.15.139:0/3112541452 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 59) ==== 8450+0+0 (secure 0 0 0) 0x7fd800003c70 con 0x55e294b08410 2024-09-16T01:51:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.108+0000 7fd818a0d740 10 client.16758 fetch_fsmap finished waiting for FSMap version 59 2024-09-16T01:51:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.108+0000 7fd818a0d740 10 client.16758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:51:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.108+0000 7fd818a0d740 20 client.16758 populate_metadata read hostname 'smithi139' 2024-09-16T01:51:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.108+0000 7fd818a0d740 1 --2- 172.21.15.139:0/3112541452 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e294b22cd0 0x55e294b25150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.108+0000 7fd818a0d740 4 client.16758 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:51:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.108+0000 7fd818a0d740 1 -- 172.21.15.139:0/3112541452 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e294a79a20 con 0x55e294b22cd0 2024-09-16T01:51:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.108+0000 7fd80effd640 1 --2- 172.21.15.139:0/3112541452 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e294b22cd0 0x55e294b25150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.109+0000 7fd7eeffd640 10 client.16758 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.109+0000 7fd80effd640 1 --2- 172.21.15.139:0/3112541452 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e294b22cd0 0x55e294b25150 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66036506b9af3843 server_cookie=4ddf15ab2a941ae7 in_seq=0 out_seq=0 2024-09-16T01:51:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.359+0000 7fd7eeffd640 1 -- 172.21.15.139:0/3112541452 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e294a79a20 con 0x55e294b22cd0 2024-09-16T01:51:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.359+0000 7fd7eeffd640 10 client.16758 handle_command_reply: tid=0 2024-09-16T01:51:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.359+0000 7fd818a0d740 1 client.16758 shutdown 2024-09-16T01:51:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.359+0000 7fd818a0d740 1 -- 172.21.15.139:0/3112541452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 msgr2=0x55e294b100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.359+0000 7fd818a0d740 1 --2- 172.21.15.139:0/3112541452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b100a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd8000127c0 tx=0x7fd800012c80 comp rx=0 tx=0).stop 2024-09-16T01:51:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.359+0000 7fd818a0d740 1 -- 172.21.15.139:0/3112541452 shutdown_connections 2024-09-16T01:51:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.359+0000 7fd818a0d740 1 --2- 172.21.15.139:0/3112541452 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e294b22cd0 0x55e294b25150 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.359+0000 7fd818a0d740 1 --2- 172.21.15.139:0/3112541452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b12cb0 0x55e294b105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.359+0000 7fd818a0d740 1 --2- 172.21.15.139:0/3112541452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b100a0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.359+0000 7fd818a0d740 1 --2- 172.21.15.139:0/3112541452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b07af0 0x55e294b0fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.359+0000 7fd7eeffd640 0 client.16758 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.359+0000 7fd818a0d740 1 -- 172.21.15.139:0/3112541452 >> 172.21.15.139:0/3112541452 conn(0x55e294b04060 msgr2=0x55e294b09dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.359+0000 7fd818a0d740 1 -- 172.21.15.139:0/3112541452 shutdown_connections 2024-09-16T01:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.359+0000 7fd818a0d740 1 -- 172.21.15.139:0/3112541452 wait complete. 2024-09-16T01:51:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.360+0000 7fd818a0d740 20 client.16758 trim_cache size 0 max 16384 2024-09-16T01:51:58.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce797fa640 1 -- 172.21.15.139:0/3230272170 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560f753a40a0 con 0x560f754c68d0 2024-09-16T01:51:58.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce797fa640 10 client.16746 handle_command_reply: tid=0 2024-09-16T01:51:58.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce9a43b740 1 client.16746 shutdown 2024-09-16T01:51:58.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce9a43b740 1 -- 172.21.15.139:0/3230272170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75437060 msgr2=0x560f7544a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:58.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce9a43b740 1 --2- 172.21.15.139:0/3230272170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75437060 0x560f7544a7e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fce8000c010 tx=0x7fce80005970 comp rx=0 tx=0).stop 2024-09-16T01:51:58.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce9a43b740 1 -- 172.21.15.139:0/3230272170 shutdown_connections 2024-09-16T01:51:58.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce9a43b740 1 --2- 172.21.15.139:0/3230272170 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560f754c68d0 0x560f754c8cf0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce9a43b740 1 --2- 172.21.15.139:0/3230272170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560f75442240 0x560f75447100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce797fa640 0 client.16746 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:58.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce9a43b740 1 --2- 172.21.15.139:0/3230272170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f75437980 0x560f75446bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce9a43b740 1 --2- 172.21.15.139:0/3230272170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560f75437060 0x560f7544a7e0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce9a43b740 1 -- 172.21.15.139:0/3230272170 >> 172.21.15.139:0/3230272170 conn(0x560f754335f0 msgr2=0x560f7543f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:58.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce9a43b740 1 -- 172.21.15.139:0/3230272170 shutdown_connections 2024-09-16T01:51:58.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce9a43b740 1 -- 172.21.15.139:0/3230272170 wait complete. 2024-09-16T01:51:58.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.361+0000 7fce9a43b740 20 client.16746 trim_cache size 0 max 16384 2024-09-16T01:51:58.364 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:51:58.364 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 59, 2024-09-16T01:51:58.364 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:51:58.364 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 40, 2024-09-16T01:51:58.364 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:51:58.365 INFO:teuthology.orchestra.run.smithi139.stdout: "ad0e7e37": { 2024-09-16T01:51:58.365 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 40, 2024-09-16T01:51:58.365 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:51:58.365 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:51:58.365 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:51:58.365 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:51:58.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:51:58.365 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:51:58.365 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 141.8, 2024-09-16T01:51:58.365 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:51:58.365 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:51:58.365 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:51:58.365 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:51:58.365 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:51:58.366 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:51:58.366 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:51:58.366 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:51:58.366 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:51:58.366 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:51:58.366 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:51:58.366 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:51:58.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.362+0000 7fce98062640 1 -- 172.21.15.139:0/2639553620 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fce7c0a33b0 msgr2=0x7fce7c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.362+0000 7fce98062640 1 --2- 172.21.15.139:0/2639553620 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fce7c0a33b0 0x7fce7c0a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fce88098bc0 tx=0x7fce88091680 comp rx=0 tx=0).stop 2024-09-16T01:51:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.362+0000 7fce98062640 1 -- 172.21.15.139:0/2639553620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce90112640 msgr2=0x7fce901a7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.362+0000 7fce98062640 1 --2- 172.21.15.139:0/2639553620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce90112640 0x7fce901a7770 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fce8800eaa0 tx=0x7fce8800ef60 comp rx=0 tx=0).stop 2024-09-16T01:51:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.362+0000 7fce98062640 1 -- 172.21.15.139:0/2639553620 shutdown_connections 2024-09-16T01:51:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.362+0000 7fce98062640 1 --2- 172.21.15.139:0/2639553620 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fce7c0a33b0 0x7fce7c0a5870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.362+0000 7fce98062640 1 --2- 172.21.15.139:0/2639553620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce90112640 0x7fce901a7770 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.362+0000 7fce98062640 1 --2- 172.21.15.139:0/2639553620 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce9010cf30 0x7fce901a33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.362+0000 7fce98062640 1 --2- 172.21.15.139:0/2639553620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce9010c590 0x7fce901a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.362+0000 7fce98062640 1 -- 172.21.15.139:0/2639553620 >> 172.21.15.139:0/2639553620 conn(0x7fce9006cdc0 msgr2=0x7fce9010af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.362+0000 7fce98062640 1 -- 172.21.15.139:0/2639553620 shutdown_connections 2024-09-16T01:51:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:58.362+0000 7fce98062640 1 -- 172.21.15.139:0/2639553620 wait complete. 2024-09-16T01:51:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.375+0000 7fd818a0d740 1 Processor -- start 2024-09-16T01:51:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.375+0000 7fd818a0d740 1 -- start start 2024-09-16T01:51:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.375+0000 7fd818a0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b07af0 0x55e294b0a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.375+0000 7fd818a0d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b08410 0x55e294b10410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.375+0000 7fd818a0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 0x55e294b2aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.375+0000 7fd818a0d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd8101b12a0 con 0x55e294b07af0 2024-09-16T01:51:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.375+0000 7fd818a0d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd8101b30e0 con 0x55e294b12cb0 2024-09-16T01:51:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.375+0000 7fd818a0d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd8101b1a30 con 0x55e294b08410 2024-09-16T01:51:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.375+0000 7fd80effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b07af0 0x55e294b0a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.375+0000 7fd80f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b08410 0x55e294b10410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.375+0000 7fd80effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b07af0 0x55e294b0a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51874/0 (socket says 172.21.15.139:51874) 2024-09-16T01:51:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.375+0000 7fd80effd640 1 -- 172.21.15.139:0/1372272059 learned_addr learned my addr 172.21.15.139:0/1372272059 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.376+0000 7fd80effd640 1 --2- 172.21.15.139:0/1372272059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 0x55e294b2aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.376+0000 7fd80f7fe640 1 -- 172.21.15.139:0/1372272059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 msgr2=0x55e294b2aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.376+0000 7fd80f7fe640 1 --2- 172.21.15.139:0/1372272059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 0x55e294b2aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.376+0000 7fd80f7fe640 1 -- 172.21.15.139:0/1372272059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b07af0 msgr2=0x55e294b0a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.376+0000 7fd80f7fe640 1 --2- 172.21.15.139:0/1372272059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b07af0 0x55e294b0a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.376+0000 7fd80f7fe640 1 -- 172.21.15.139:0/1372272059 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e294b05110 con 0x55e294b08410 2024-09-16T01:51:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.376+0000 7fd80effd640 1 --2- 172.21.15.139:0/1372272059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b07af0 0x55e294b0a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:51:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.376+0000 7fd80f7fe640 1 --2- 172.21.15.139:0/1372272059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b08410 0x55e294b10410 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55e294b11110 tx=0x7fd800009860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91ba38846ac284d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.376+0000 7fd7f57fa640 1 -- 172.21.15.139:0/1372272059 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800003fe0 con 0x55e294b08410 2024-09-16T01:51:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.376+0000 7fd7f57fa640 1 -- 172.21.15.139:0/1372272059 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd800004180 con 0x55e294b08410 2024-09-16T01:51:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.376+0000 7fd7f57fa640 1 -- 172.21.15.139:0/1372272059 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800010760 con 0x55e294b08410 2024-09-16T01:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.377+0000 7fd818a0d740 1 -- 172.21.15.139:0/1372272059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b08410 msgr2=0x55e294b10410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.377+0000 7fd818a0d740 1 --2- 172.21.15.139:0/1372272059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b08410 0x55e294b10410 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55e294b11110 tx=0x7fd800009860 comp rx=0 tx=0).stop 2024-09-16T01:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.377+0000 7fd818a0d740 1 -- 172.21.15.139:0/1372272059 shutdown_connections 2024-09-16T01:51:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.377+0000 7fd818a0d740 1 --2- 172.21.15.139:0/1372272059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 0x55e294b2aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.377+0000 7fd818a0d740 1 --2- 172.21.15.139:0/1372272059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b08410 0x55e294b10410 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.377+0000 7fd818a0d740 1 --2- 172.21.15.139:0/1372272059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b07af0 0x55e294b0a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.377+0000 7fd818a0d740 1 -- 172.21.15.139:0/1372272059 >> 172.21.15.139:0/1372272059 conn(0x55e294b04060 msgr2=0x55e294b11be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.377+0000 7fd818a0d740 1 -- 172.21.15.139:0/1372272059 shutdown_connections 2024-09-16T01:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.377+0000 7fd818a0d740 1 -- 172.21.15.139:0/1372272059 wait complete. 2024-09-16T01:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.377+0000 7fd818a0d740 1 -- start start 2024-09-16T01:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd818a0d740 1 Processor -- start 2024-09-16T01:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd818a0d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b07af0 0x55e294b1b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd818a0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b165d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd818a0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 0x55e294b16b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd818a0d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd8101b2950 con 0x55e294b08410 2024-09-16T01:51:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd818a0d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd8101b21c0 con 0x55e294b12cb0 2024-09-16T01:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd818a0d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd8101af2a0 con 0x55e294b07af0 2024-09-16T01:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd818a0d740 10 client.0 _ll_register_callbacks cb 0x7fd800009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd80effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b07af0 0x55e294b1b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd80f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd80effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b07af0 0x55e294b1b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:60850/0 (socket says 172.21.15.139:60850) 2024-09-16T01:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd80effd640 1 -- 172.21.15.139:0/4274994392 learned_addr learned my addr 172.21.15.139:0/4274994392 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd80effd640 1 --2- 172.21.15.139:0/4274994392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 0x55e294b16b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd80effd640 1 -- 172.21.15.139:0/4274994392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 msgr2=0x55e294b16b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd80effd640 1 --2- 172.21.15.139:0/4274994392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 0x55e294b16b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd80effd640 1 -- 172.21.15.139:0/4274994392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 msgr2=0x55e294b165d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd80effd640 1 --2- 172.21.15.139:0/4274994392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b165d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd80effd640 1 -- 172.21.15.139:0/4274994392 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e294b171b0 con 0x55e294b07af0 2024-09-16T01:51:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.378+0000 7fd80f7fe640 1 --2- 172.21.15.139:0/4274994392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:51:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd80effd640 1 --2- 172.21.15.139:0/4274994392 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b07af0 0x55e294b1b2b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd80402a940 tx=0x7fd804041040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2276b700b6211e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd7eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:51:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd7eeffd640 1 -- 172.21.15.139:0/4274994392 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804035040 con 0x55e294b07af0 2024-09-16T01:51:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd7eeffd640 1 -- 172.21.15.139:0/4274994392 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9510+0+0 (secure 0 0 0) 0x7fd8040043f0 con 0x55e294b07af0 2024-09-16T01:51:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd818a0d740 1 -- 172.21.15.139:0/4274994392 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd800004180 con 0x55e294b07af0 2024-09-16T01:51:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd7eeffd640 1 -- 172.21.15.139:0/4274994392 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fd804005020 con 0x55e294b07af0 2024-09-16T01:51:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd818a0d740 10 client.26161 fetch_fsmap learned FSMap version 59 2024-09-16T01:51:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd818a0d740 1 -- 172.21.15.139:0/4274994392 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55e294b8d8d0 con 0x55e294b07af0 2024-09-16T01:51:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd7eeffd640 1 -- 172.21.15.139:0/4274994392 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd800004180 con 0x55e294b07af0 2024-09-16T01:51:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd7eeffd640 1 -- 172.21.15.139:0/4274994392 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 59) ==== 8450+0+0 (secure 0 0 0) 0x7fd8040035f0 con 0x55e294b07af0 2024-09-16T01:51:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd818a0d740 10 client.26161 fetch_fsmap finished waiting for FSMap version 59 2024-09-16T01:51:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd818a0d740 10 client.26161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:51:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd818a0d740 20 client.26161 populate_metadata read hostname 'smithi139' 2024-09-16T01:51:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd818a0d740 1 --2- 172.21.15.139:0/4274994392 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e294b8dcb0 0x55e294b900d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd818a0d740 4 client.26161 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:51:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.379+0000 7fd818a0d740 1 -- 172.21.15.139:0/4274994392 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e294a73d70 con 0x55e294b8dcb0 2024-09-16T01:51:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.380+0000 7fd80f7fe640 1 --2- 172.21.15.139:0/4274994392 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e294b8dcb0 0x55e294b900d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.380+0000 7fd7eeffd640 10 client.26161 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.380+0000 7fd80f7fe640 1 --2- 172.21.15.139:0/4274994392 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e294b8dcb0 0x55e294b900d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4756bf73030c8004 server_cookie=7e2ba9fc509d2227 in_seq=0 out_seq=0 2024-09-16T01:51:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd7eeffd640 1 -- 172.21.15.139:0/4274994392 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e294a73d70 con 0x55e294b8dcb0 2024-09-16T01:51:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd7eeffd640 10 client.26161 handle_command_reply: tid=0 2024-09-16T01:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd818a0d740 1 client.26161 shutdown 2024-09-16T01:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd818a0d740 1 -- 172.21.15.139:0/4274994392 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b07af0 msgr2=0x55e294b1b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd818a0d740 1 --2- 172.21.15.139:0/4274994392 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b07af0 0x55e294b1b2b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd80402a940 tx=0x7fd804041040 comp rx=0 tx=0).stop 2024-09-16T01:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd818a0d740 1 -- 172.21.15.139:0/4274994392 shutdown_connections 2024-09-16T01:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd818a0d740 1 --2- 172.21.15.139:0/4274994392 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e294b8dcb0 0x55e294b900d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd818a0d740 1 --2- 172.21.15.139:0/4274994392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e294b12cb0 0x55e294b16b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd818a0d740 1 --2- 172.21.15.139:0/4274994392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e294b08410 0x55e294b165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd7eeffd640 0 client.26161 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd818a0d740 1 --2- 172.21.15.139:0/4274994392 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e294b07af0 0x55e294b1b2b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd818a0d740 1 -- 172.21.15.139:0/4274994392 >> 172.21.15.139:0/4274994392 conn(0x55e294b04060 msgr2=0x55e294b029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd818a0d740 1 -- 172.21.15.139:0/4274994392 shutdown_connections 2024-09-16T01:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd818a0d740 1 -- 172.21.15.139:0/4274994392 wait complete. 2024-09-16T01:51:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.412+0000 7fd818a0d740 20 client.26161 trim_cache size 0 max 16384 2024-09-16T01:51:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:51:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:51:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:51:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:51:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.413+0000 7fd815b91640 1 -- 172.21.15.139:0/4085972006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd7f80a34e0 msgr2=0x7fd7f80a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.413+0000 7fd815b91640 1 --2- 172.21.15.139:0/4085972006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd7f80a34e0 0x7fd7f80a59a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd808098f60 tx=0x7fd8080054b0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.413+0000 7fd815b91640 1 -- 172.21.15.139:0/4085972006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8101069e0 msgr2=0x7fd8100fd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.413+0000 7fd815b91640 1 --2- 172.21.15.139:0/4085972006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8101069e0 0x7fd8100fd6f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd808009fd0 tx=0x7fd80800ef10 comp rx=0 tx=0).stop 2024-09-16T01:51:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.413+0000 7fd815b91640 1 -- 172.21.15.139:0/4085972006 shutdown_connections 2024-09-16T01:51:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.413+0000 7fd815b91640 1 --2- 172.21.15.139:0/4085972006 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd7f80a34e0 0x7fd7f80a59a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.413+0000 7fd815b91640 1 --2- 172.21.15.139:0/4085972006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8101069e0 0x7fd8100fd6f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.413+0000 7fd815b91640 1 --2- 172.21.15.139:0/4085972006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd810101370 0x7fd8100fd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.413+0000 7fd815b91640 1 --2- 172.21.15.139:0/4085972006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8101009d0 0x7fd8100fcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.413+0000 7fd815b91640 1 -- 172.21.15.139:0/4085972006 >> 172.21.15.139:0/4085972006 conn(0x7fd8100fc6f0 msgr2=0x7fd81006a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.413+0000 7fd815b91640 1 -- 172.21.15.139:0/4085972006 shutdown_connections 2024-09-16T01:51:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:51:58.413+0000 7fd815b91640 1 -- 172.21.15.139:0/4085972006 wait complete. 2024-09-16T01:51:59.063 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:51:58 smithi139 ceph-mon[31175]: pgmap v364: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 14 op/s 2024-09-16T01:51:59.090 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ad0e7e37', total seconds elapsed: 25.7 2024-09-16T01:51:59.090 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.85 2024-09-16T01:51:59.091 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:51:59.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:51:58 smithi164 ceph-mon[35121]: pgmap v364: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 14 op/s 2024-09-16T01:51:59.346 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:51:59.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:51:58 smithi176 ceph-mon[35324]: pgmap v364: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 14 op/s 2024-09-16T01:51:59.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.835+0000 7fb0cdda0640 1 -- 172.21.15.139:0/234254815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0c8100e20 msgr2=0x7fb0c8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:59.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.835+0000 7fb0cdda0640 1 --2- 172.21.15.139:0/234254815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0c8100e20 0x7fb0c8101200 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009900 tx=0x7fb0b802a480 comp rx=0 tx=0).stop 2024-09-16T01:51:59.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.835+0000 7fb0cdda0640 1 -- 172.21.15.139:0/234254815 shutdown_connections 2024-09-16T01:51:59.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.835+0000 7fb0cdda0640 1 --2- 172.21.15.139:0/234254815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0c8102ef0 0x7fb0c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:59.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.835+0000 7fb0cdda0640 1 --2- 172.21.15.139:0/234254815 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0c8102570 0x7fb0c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:59.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.835+0000 7fb0cdda0640 1 --2- 172.21.15.139:0/234254815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0c8100e20 0x7fb0c8101200 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:59.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.835+0000 7fb0cdda0640 1 -- 172.21.15.139:0/234254815 >> 172.21.15.139:0/234254815 conn(0x7fb0c80fc7d0 msgr2=0x7fb0c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:51:59.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.835+0000 7fb0cdda0640 1 -- 172.21.15.139:0/234254815 shutdown_connections 2024-09-16T01:51:59.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.835+0000 7fb0cdda0640 1 -- 172.21.15.139:0/234254815 wait complete. 2024-09-16T01:51:59.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.836+0000 7fb0cdda0640 1 Processor -- start 2024-09-16T01:51:59.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.836+0000 7fb0cdda0640 1 -- start start 2024-09-16T01:51:59.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.836+0000 7fb0cdda0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0c8100e20 0x7fb0c81993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:59.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.836+0000 7fb0cdda0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0c8102570 0x7fb0c8199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:59.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.836+0000 7fb0cdda0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0c8102ef0 0x7fb0c81a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:59.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.836+0000 7fb0cdda0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8110f80 con 0x7fb0c8100e20 2024-09-16T01:51:59.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.836+0000 7fb0cdda0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8110e00 con 0x7fb0c8102ef0 2024-09-16T01:51:59.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.836+0000 7fb0cdda0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8111100 con 0x7fb0c8102570 2024-09-16T01:51:59.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.836+0000 7fb0c77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0c8100e20 0x7fb0c81993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:59.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.836+0000 7fb0c77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0c8100e20 0x7fb0c81993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51906/0 (socket says 172.21.15.139:51906) 2024-09-16T01:51:59.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.836+0000 7fb0c77fe640 1 -- 172.21.15.139:0/447122380 learned_addr learned my addr 172.21.15.139:0/447122380 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:51:59.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.836+0000 7fb0c7fff640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0c8102ef0 0x7fb0c81a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:59.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.836+0000 7fb0c6ffd640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0c8102570 0x7fb0c8199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:59.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.837+0000 7fb0c7fff640 1 -- 172.21.15.139:0/447122380 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0c8102570 msgr2=0x7fb0c8199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:59.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.837+0000 7fb0c7fff640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0c8102570 0x7fb0c8199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:59.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.837+0000 7fb0c7fff640 1 -- 172.21.15.139:0/447122380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0c8100e20 msgr2=0x7fb0c81993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:51:59.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.837+0000 7fb0c7fff640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0c8100e20 0x7fb0c81993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:51:59.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.837+0000 7fb0c7fff640 1 -- 172.21.15.139:0/447122380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c81a4990 con 0x7fb0c8102ef0 2024-09-16T01:51:59.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.837+0000 7fb0c6ffd640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0c8102570 0x7fb0c8199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:51:59.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.837+0000 7fb0c77fe640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0c8100e20 0x7fb0c81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:51:59.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.837+0000 7fb0c7fff640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0c8102ef0 0x7fb0c81a4450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009fd0 tx=0x7fb0c000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:59.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.837+0000 7fb0c4ff9640 1 -- 172.21.15.139:0/447122380 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000ce20 con 0x7fb0c8102ef0 2024-09-16T01:51:59.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.837+0000 7fb0cdda0640 1 -- 172.21.15.139:0/447122380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c81a4b60 con 0x7fb0c8102ef0 2024-09-16T01:51:59.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.837+0000 7fb0c4ff9640 1 -- 172.21.15.139:0/447122380 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0c0005020 con 0x7fb0c8102ef0 2024-09-16T01:51:59.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.837+0000 7fb0c4ff9640 1 -- 172.21.15.139:0/447122380 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0010720 con 0x7fb0c8102ef0 2024-09-16T01:51:59.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.837+0000 7fb0cdda0640 1 -- 172.21.15.139:0/447122380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c81a5070 con 0x7fb0c8102ef0 2024-09-16T01:51:59.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.838+0000 7fb0cdda0640 1 -- 172.21.15.139:0/447122380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c8101200 con 0x7fb0c8102ef0 2024-09-16T01:51:59.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.839+0000 7fb0c4ff9640 1 -- 172.21.15.139:0/447122380 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb0c0005540 con 0x7fb0c8102ef0 2024-09-16T01:51:59.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.840+0000 7fb0c4ff9640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0b00a33b0 0x7fb0b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:51:59.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.840+0000 7fb0c77fe640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0b00a33b0 0x7fb0b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:51:59.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.840+0000 7fb0c4ff9640 1 -- 172.21.15.139:0/447122380 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9510+0+0 (secure 0 0 0) 0x7fb0c00d07a0 con 0x7fb0c8102ef0 2024-09-16T01:51:59.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.840+0000 7fb0c77fe640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0b00a33b0 0x7fb0b00a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80096c0 tx=0x7fb0b8005a20 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:51:59.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:51:59.844+0000 7fb0c4ff9640 1 -- 172.21.15.139:0/447122380 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0c009cd40 con 0x7fb0c8102ef0 2024-09-16T01:52:00.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:00.067+0000 7fb0cdda0640 1 -- 172.21.15.139:0/447122380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0c8109090 con 0x7fb0c8102ef0 2024-09-16T01:52:00.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:00.068+0000 7fb0c4ff9640 1 -- 172.21.15.139:0/447122380 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+12030 (secure 0 0 0) 0x7fb0c0098020 con 0x7fb0c8102ef0 2024-09-16T01:52:00.069 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 59 2024-09-16T01:52:00.069 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:52:00.069 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":59,"btime":"2024-09-16T01:51:45:706357+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":59,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:51:45.576176+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:52:00.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:00.071+0000 7fb0cdda0640 1 -- 172.21.15.139:0/447122380 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0b00a33b0 msgr2=0x7fb0b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:00.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:00.071+0000 7fb0cdda0640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0b00a33b0 0x7fb0b00a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80096c0 tx=0x7fb0b8005a20 comp rx=0 tx=0).stop 2024-09-16T01:52:00.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:00.071+0000 7fb0cdda0640 1 -- 172.21.15.139:0/447122380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0c8102ef0 msgr2=0x7fb0c81a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:00.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:00.071+0000 7fb0cdda0640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0c8102ef0 0x7fb0c81a4450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009fd0 tx=0x7fb0c000ef10 comp rx=0 tx=0).stop 2024-09-16T01:52:00.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:00.071+0000 7fb0cdda0640 1 -- 172.21.15.139:0/447122380 shutdown_connections 2024-09-16T01:52:00.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:00.071+0000 7fb0cdda0640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0b00a33b0 0x7fb0b00a5870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:00.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:00.071+0000 7fb0cdda0640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0c8102ef0 0x7fb0c81a4450 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:00.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:00.071+0000 7fb0cdda0640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0c8102570 0x7fb0c8199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:00.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:00.071+0000 7fb0cdda0640 1 --2- 172.21.15.139:0/447122380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0c8100e20 0x7fb0c81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:00.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:00.071+0000 7fb0cdda0640 1 -- 172.21.15.139:0/447122380 >> 172.21.15.139:0/447122380 conn(0x7fb0c80fc7d0 msgr2=0x7fb0c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:00.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:00.071+0000 7fb0cdda0640 1 -- 172.21.15.139:0/447122380 shutdown_connections 2024-09-16T01:52:00.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:00.071+0000 7fb0cdda0640 1 -- 172.21.15.139:0/447122380 wait complete. 2024-09-16T01:52:00.749 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0' 2024-09-16T01:52:00.749 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0 2024-09-16T01:52:01.254 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:52:01.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:01 smithi164 ceph-mon[35121]: pgmap v365: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2024-09-16T01:52:01.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:01 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/447122380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:52:01.406 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:01 smithi139 ceph-mon[31175]: pgmap v365: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2024-09-16T01:52:01.406 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:01 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/447122380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:52:01.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:01 smithi176 ceph-mon[35324]: pgmap v365: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2024-09-16T01:52:01.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:01 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/447122380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:52:01.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.916+0000 7fa8e1965640 1 -- 172.21.15.139:0/1433474376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8dc10e890 msgr2=0x7fa8dc10ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:01.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.916+0000 7fa8e1965640 1 --2- 172.21.15.139:0/1433474376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8dc10e890 0x7fa8dc10ed40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009be0 tx=0x7fa8cc02a710 comp rx=0 tx=0).stop 2024-09-16T01:52:01.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.916+0000 7fa8e1965640 1 -- 172.21.15.139:0/1433474376 shutdown_connections 2024-09-16T01:52:01.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.916+0000 7fa8e1965640 1 --2- 172.21.15.139:0/1433474376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8dc10e890 0x7fa8dc10ed40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.916+0000 7fa8e1965640 1 --2- 172.21.15.139:0/1433474376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8dc109b10 0x7fa8dc10e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.916+0000 7fa8e1965640 1 --2- 172.21.15.139:0/1433474376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8dc1091f0 0x7fa8dc1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.916+0000 7fa8e1965640 1 -- 172.21.15.139:0/1433474376 >> 172.21.15.139:0/1433474376 conn(0x7fa8dc06e2d0 msgr2=0x7fa8dc06e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:01.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.916+0000 7fa8e1965640 1 -- 172.21.15.139:0/1433474376 shutdown_connections 2024-09-16T01:52:01.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.916+0000 7fa8e1965640 1 -- 172.21.15.139:0/1433474376 wait complete. 2024-09-16T01:52:01.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.917+0000 7fa8e1965640 1 Processor -- start 2024-09-16T01:52:01.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.917+0000 7fa8e1965640 1 -- start start 2024-09-16T01:52:01.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8e1965640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8dc1091f0 0x7fa8dc119dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:01.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8e1965640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8dc10e890 0x7fa8dc11a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:01.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8e1965640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8dc11a850 0x7fa8dc11e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:01.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8e1965640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc112580 con 0x7fa8dc1091f0 2024-09-16T01:52:01.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8e1965640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc112400 con 0x7fa8dc10e890 2024-09-16T01:52:01.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8e1965640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc118810 con 0x7fa8dc11a850 2024-09-16T01:52:01.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8daffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8dc1091f0 0x7fa8dc119dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:01.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8daffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8dc1091f0 0x7fa8dc119dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51916/0 (socket says 172.21.15.139:51916) 2024-09-16T01:52:01.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8daffd640 1 -- 172.21.15.139:0/3010180170 learned_addr learned my addr 172.21.15.139:0/3010180170 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:01.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8da7fc640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8dc10e890 0x7fa8dc11a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:01.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8db7fe640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8dc11a850 0x7fa8dc11e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:01.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8daffd640 1 -- 172.21.15.139:0/3010180170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8dc11a850 msgr2=0x7fa8dc11e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:01.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8daffd640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8dc11a850 0x7fa8dc11e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8daffd640 1 -- 172.21.15.139:0/3010180170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8dc10e890 msgr2=0x7fa8dc11a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:01.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.918+0000 7fa8daffd640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8dc10e890 0x7fa8dc11a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.919+0000 7fa8daffd640 1 -- 172.21.15.139:0/3010180170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8dc11eeb0 con 0x7fa8dc1091f0 2024-09-16T01:52:01.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.919+0000 7fa8da7fc640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8dc10e890 0x7fa8dc11a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:52:01.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.919+0000 7fa8daffd640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8dc1091f0 0x7fa8dc119dd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400e9c0 tx=0x7fa8d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:01.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.920+0000 7fa8e0963640 1 -- 172.21.15.139:0/3010180170 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d400ccc0 con 0x7fa8dc1091f0 2024-09-16T01:52:01.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.920+0000 7fa8e0963640 1 -- 172.21.15.139:0/3010180170 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d4005500 con 0x7fa8dc1091f0 2024-09-16T01:52:01.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.920+0000 7fa8e0963640 1 -- 172.21.15.139:0/3010180170 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4010690 con 0x7fa8dc1091f0 2024-09-16T01:52:01.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.920+0000 7fa8e1965640 1 -- 172.21.15.139:0/3010180170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8dc11f1a0 con 0x7fa8dc1091f0 2024-09-16T01:52:01.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.920+0000 7fa8e1965640 1 -- 172.21.15.139:0/3010180170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8dc11f630 con 0x7fa8dc1091f0 2024-09-16T01:52:01.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.925+0000 7fa8e0963640 1 -- 172.21.15.139:0/3010180170 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa8d40046c0 con 0x7fa8dc1091f0 2024-09-16T01:52:01.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.925+0000 7fa8e3d3e740 1 Processor -- start 2024-09-16T01:52:01.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.925+0000 7fa8e3d3e740 1 -- start start 2024-09-16T01:52:01.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.925+0000 7fa8e3d3e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea614c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:01.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.925+0000 7fa8e3d3e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea615170 0x564aea61f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:01.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.925+0000 7fa8e3d3e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 0x564aea621e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:01.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.925+0000 7fa8e3d3e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564aea579160 con 0x564aea614850 2024-09-16T01:52:01.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.925+0000 7fa8e3d3e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564aea520720 con 0x564aea615170 2024-09-16T01:52:01.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.925+0000 7fa8e3d3e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564aea575120 con 0x564aea61fa30 2024-09-16T01:52:01.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.926+0000 7fa8db7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea614c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:01.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.926+0000 7fa8db7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea614c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51918/0 (socket says 172.21.15.139:51918) 2024-09-16T01:52:01.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.926+0000 7fa8db7fe640 1 -- 172.21.15.139:0/3918027510 learned_addr learned my addr 172.21.15.139:0/3918027510 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:01.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.927+0000 7fa8daffd640 1 --2- 172.21.15.139:0/3918027510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea615170 0x564aea61f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:01.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.927+0000 7fa8e0963640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa8c80a35e0 0x7fa8c80a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:01.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.927+0000 7fa8e0963640 1 -- 172.21.15.139:0/3010180170 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7fa8d40d02e0 con 0x7fa8dc1091f0 2024-09-16T01:52:01.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.928+0000 7fa8db7fe640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa8c80a35e0 0x7fa8c80a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:01.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.928+0000 7fa8db7fe640 1 -- 172.21.15.139:0/3918027510 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 msgr2=0x564aea621e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:01.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.928+0000 7fa8db7fe640 1 --2- 172.21.15.139:0/3918027510 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 0x564aea621e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.928+0000 7fa8db7fe640 1 -- 172.21.15.139:0/3918027510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea615170 msgr2=0x564aea61f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:01.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.928+0000 7fa8db7fe640 1 --2- 172.21.15.139:0/3918027510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea615170 0x564aea61f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.928+0000 7fa8db7fe640 1 -- 172.21.15.139:0/3918027510 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aea2eaba0 con 0x564aea614850 2024-09-16T01:52:01.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.929+0000 7fa8db7fe640 1 --2- 172.21.15.139:0/3918027510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea614c30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009be0 tx=0x7fa8cc0304a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85aa59e4cf403e33 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:01.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.929+0000 7fa8db7fe640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa8c80a35e0 0x7fa8c80a5aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc068000 tx=0x7fa8cc06a000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:01.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.929+0000 7fa8d1ffb640 1 -- 172.21.15.139:0/3918027510 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc036040 con 0x564aea614850 2024-09-16T01:52:01.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.929+0000 7fa8d1ffb640 1 -- 172.21.15.139:0/3918027510 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8cc004520 con 0x564aea614850 2024-09-16T01:52:01.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.929+0000 7fa8d1ffb640 1 -- 172.21.15.139:0/3918027510 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc05fa80 con 0x564aea614850 2024-09-16T01:52:01.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.931+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/3918027510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 msgr2=0x564aea614c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:01.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.931+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/3918027510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea614c30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009be0 tx=0x7fa8cc0304a0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.931+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/3918027510 shutdown_connections 2024-09-16T01:52:01.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.931+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/3918027510 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 0x564aea621e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.931+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/3918027510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea615170 0x564aea61f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.931+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/3918027510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea614c30 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.931+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/3918027510 >> 172.21.15.139:0/3918027510 conn(0x564aea610de0 msgr2=0x564aea6111c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:01.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.934+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/3918027510 shutdown_connections 2024-09-16T01:52:01.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.934+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/3918027510 wait complete. 2024-09-16T01:52:01.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.935+0000 7fa8e3d3e740 1 -- start start 2024-09-16T01:52:01.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.935+0000 7fa8e3d3e740 1 Processor -- start 2024-09-16T01:52:01.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.935+0000 7fa8e3d3e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea614850 0x564aea61ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:01.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.935+0000 7fa8e3d3e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea615170 0x564aea61cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:01.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.935+0000 7fa8e3d3e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 0x564aea61d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:01.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.936+0000 7fa8e3d3e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564aea622600 con 0x564aea615170 2024-09-16T01:52:01.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.936+0000 7fa8e3d3e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564aea501900 con 0x564aea614850 2024-09-16T01:52:01.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.936+0000 7fa8e3d3e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564aea622780 con 0x564aea61fa30 2024-09-16T01:52:01.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.936+0000 7fa8e3d3e740 10 client.0 _ll_register_callbacks cb 0x564aea4ff780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:52:01.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.936+0000 7fa8daffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea614850 0x564aea61ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:01.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.936+0000 7fa8db7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 0x564aea61d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:01.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.936+0000 7fa8daffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea614850 0x564aea61ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:50972/0 (socket says 172.21.15.139:50972) 2024-09-16T01:52:01.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.936+0000 7fa8daffd640 1 -- 172.21.15.139:0/3808168229 learned_addr learned my addr 172.21.15.139:0/3808168229 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:01.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.937+0000 7fa8daffd640 1 -- 172.21.15.139:0/3808168229 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 msgr2=0x564aea61d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:01.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.937+0000 7fa8daffd640 1 --2- 172.21.15.139:0/3808168229 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 0x564aea61d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.937+0000 7fa8daffd640 1 -- 172.21.15.139:0/3808168229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea615170 msgr2=0x564aea61cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:01.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.937+0000 7fa8daffd640 1 --2- 172.21.15.139:0/3808168229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea615170 0x564aea61cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.937+0000 7fa8daffd640 1 -- 172.21.15.139:0/3808168229 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aea541c50 con 0x564aea614850 2024-09-16T01:52:01.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.940+0000 7fa8daffd640 1 --2- 172.21.15.139:0/3808168229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea614850 0x564aea61ca00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa8d409be00 tx=0x7fa8d409bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55517ce95c7af61c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:01.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.942+0000 7fa8baffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:52:01.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.943+0000 7fa8baffd640 1 -- 172.21.15.139:0/3808168229 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40056b0 con 0x564aea614850 2024-09-16T01:52:01.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.943+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/3808168229 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8cc004520 con 0x564aea614850 2024-09-16T01:52:01.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.943+0000 7fa8baffd640 1 -- 172.21.15.139:0/3808168229 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7fa8d4098770 con 0x564aea614850 2024-09-16T01:52:01.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.944+0000 7fa8baffd640 1 -- 172.21.15.139:0/3808168229 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa8d409c800 con 0x564aea614850 2024-09-16T01:52:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.944+0000 7fa8e3d3e740 10 client.26201 fetch_fsmap learned FSMap version 59 2024-09-16T01:52:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.944+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/3808168229 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x564aea62e3b0 con 0x564aea614850 2024-09-16T01:52:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.948+0000 7fa8baffd640 1 -- 172.21.15.139:0/3808168229 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d409c100 con 0x564aea614850 2024-09-16T01:52:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.948+0000 7fa8baffd640 1 -- 172.21.15.139:0/3808168229 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 59) ==== 8450+0+0 (secure 0 0 0) 0x7fa8d40a4970 con 0x564aea614850 2024-09-16T01:52:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.948+0000 7fa8e3d3e740 10 client.26201 fetch_fsmap finished waiting for FSMap version 59 2024-09-16T01:52:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.948+0000 7fa8e3d3e740 10 client.26201 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:52:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.948+0000 7fa8e3d3e740 20 client.26201 populate_metadata read hostname 'smithi139' 2024-09-16T01:52:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.948+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/3808168229 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564aea62fdb0 0x564aea632250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.948+0000 7fa8e3d3e740 4 client.26201 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:52:01.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.948+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/3808168229 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa8cc004520 con 0x564aea62fdb0 2024-09-16T01:52:01.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.950+0000 7fa8da7fc640 1 --2- 172.21.15.139:0/3808168229 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564aea62fdb0 0x564aea632250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:01.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.952+0000 7fa8da7fc640 1 --2- 172.21.15.139:0/3808168229 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564aea62fdb0 0x564aea632250 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26de67ea0b7772cf server_cookie=dd9397429314763 in_seq=0 out_seq=0 2024-09-16T01:52:01.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.952+0000 7fa8baffd640 10 client.26201 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:01.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.965+0000 7fa8baffd640 1 -- 172.21.15.139:0/3808168229 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa8cc004520 con 0x564aea62fdb0 2024-09-16T01:52:01.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.965+0000 7fa8baffd640 10 client.26201 handle_command_reply: tid=0 2024-09-16T01:52:01.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.966+0000 7fa8e3d3e740 1 client.26201 shutdown 2024-09-16T01:52:01.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.966+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/3808168229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea614850 msgr2=0x564aea61ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:01.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.966+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/3808168229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea614850 0x564aea61ca00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa8d409be00 tx=0x7fa8d409bf90 comp rx=0 tx=0).stop 2024-09-16T01:52:01.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.966+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/3808168229 shutdown_connections 2024-09-16T01:52:01.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.966+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/3808168229 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564aea62fdb0 0x564aea632250 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.966+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/3808168229 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 0x564aea61d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.966+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/3808168229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea615170 0x564aea61cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.966+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/3808168229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea614850 0x564aea61ca00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:01.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.966+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/3808168229 >> 172.21.15.139:0/3808168229 conn(0x564aea610de0 msgr2=0x564aea616b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:01.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.966+0000 7fa8baffd640 0 client.26201 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:01.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.966+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/3808168229 shutdown_connections 2024-09-16T01:52:01.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.966+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/3808168229 wait complete. 2024-09-16T01:52:01.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:01.966+0000 7fa8e3d3e740 20 client.26201 trim_cache size 0 max 16384 2024-09-16T01:52:02.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.000+0000 7fa8e3d3e740 1 Processor -- start 2024-09-16T01:52:02.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.001+0000 7fa8e3d3e740 1 -- start start 2024-09-16T01:52:02.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.001+0000 7fa8e3d3e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea61b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:02.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.001+0000 7fa8e3d3e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea615170 0x564aea617300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:02.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.001+0000 7fa8e3d3e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea61fa30 0x564aea637b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:02.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.002+0000 7fa8e3d3e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc07e650 con 0x564aea614850 2024-09-16T01:52:02.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.002+0000 7fa8e3d3e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc080490 con 0x564aea61fa30 2024-09-16T01:52:02.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.002+0000 7fa8e3d3e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc07ede0 con 0x564aea615170 2024-09-16T01:52:02.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.002+0000 7fa8daffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea61b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:02.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.002+0000 7fa8daffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea61b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51942/0 (socket says 172.21.15.139:51942) 2024-09-16T01:52:02.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.002+0000 7fa8daffd640 1 -- 172.21.15.139:0/2293425719 learned_addr learned my addr 172.21.15.139:0/2293425719 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:02.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.002+0000 7fa8daffd640 1 -- 172.21.15.139:0/2293425719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea615170 msgr2=0x564aea617300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:52:02.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.002+0000 7fa8daffd640 1 --2- 172.21.15.139:0/2293425719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea615170 0x564aea617300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:02.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.002+0000 7fa8daffd640 1 -- 172.21.15.139:0/2293425719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea61fa30 msgr2=0x564aea637b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:52:02.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.002+0000 7fa8daffd640 1 --2- 172.21.15.139:0/2293425719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea61fa30 0x564aea637b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:02.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.002+0000 7fa8daffd640 1 -- 172.21.15.139:0/2293425719 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aea6118a0 con 0x564aea614850 2024-09-16T01:52:02.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.002+0000 7fa8daffd640 1 --2- 172.21.15.139:0/2293425719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea61b2d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4000c00 tx=0x7fa8d409c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d323838237fe6f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:02.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.011+0000 7fa8d17fa640 1 -- 172.21.15.139:0/2293425719 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40056b0 con 0x564aea614850 2024-09-16T01:52:02.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.011+0000 7fa8d17fa640 1 -- 172.21.15.139:0/2293425719 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d409c100 con 0x564aea614850 2024-09-16T01:52:02.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.012+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/2293425719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 msgr2=0x564aea61b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:02.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.012+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/2293425719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea61b2d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4000c00 tx=0x7fa8d409c000 comp rx=0 tx=0).stop 2024-09-16T01:52:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.015+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/2293425719 shutdown_connections 2024-09-16T01:52:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.015+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/2293425719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea61fa30 0x564aea637b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.015+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/2293425719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea615170 0x564aea617300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.015+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/2293425719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea61b2d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:02.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.015+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/2293425719 >> 172.21.15.139:0/2293425719 conn(0x564aea610de0 msgr2=0x564aea61d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:02.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.016+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/2293425719 shutdown_connections 2024-09-16T01:52:02.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.019+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/2293425719 wait complete. 2024-09-16T01:52:02.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.020+0000 7fa8e3d3e740 1 -- start start 2024-09-16T01:52:02.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.020+0000 7fa8e3d3e740 1 Processor -- start 2024-09-16T01:52:02.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.021+0000 7fa8e3d3e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea619960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:02.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.021+0000 7fa8e3d3e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea615170 0x564aea619ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:02.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.021+0000 7fa8e3d3e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 0x564aea61a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:02.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.021+0000 7fa8e3d3e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc07fd00 con 0x564aea614850 2024-09-16T01:52:02.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.021+0000 7fa8e3d3e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc07f570 con 0x564aea615170 2024-09-16T01:52:02.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.021+0000 7fa8e3d3e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc1095d0 con 0x564aea61fa30 2024-09-16T01:52:02.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.021+0000 7fa8e3d3e740 10 client.0 _ll_register_callbacks cb 0x7fa8d40d3840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:52:02.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.023+0000 7fa8da7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea615170 0x564aea619ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:02.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.023+0000 7fa8da7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea615170 0x564aea619ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:50992/0 (socket says 172.21.15.139:50992) 2024-09-16T01:52:02.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.023+0000 7fa8da7fc640 1 -- 172.21.15.139:0/1625422272 learned_addr learned my addr 172.21.15.139:0/1625422272 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:02.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.023+0000 7fa8db7fe640 1 --2- 172.21.15.139:0/1625422272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 0x564aea61a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:02.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.023+0000 7fa8daffd640 1 --2- 172.21.15.139:0/1625422272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea619960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.023+0000 7fa8db7fe640 1 -- 172.21.15.139:0/1625422272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea615170 msgr2=0x564aea619ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.023+0000 7fa8db7fe640 1 --2- 172.21.15.139:0/1625422272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea615170 0x564aea619ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.023+0000 7fa8db7fe640 1 -- 172.21.15.139:0/1625422272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 msgr2=0x564aea619960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.023+0000 7fa8db7fe640 1 --2- 172.21.15.139:0/1625422272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea619960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.023+0000 7fa8db7fe640 1 -- 172.21.15.139:0/1625422272 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aea61b9b0 con 0x564aea61fa30 2024-09-16T01:52:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.024+0000 7fa8db7fe640 1 --2- 172.21.15.139:0/1625422272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 0x564aea61a3e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x564aea615850 tx=0x7fa8cc05f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=641b86de85fa86bf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.025+0000 7fa8baffd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:52:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.026+0000 7fa8baffd640 1 -- 172.21.15.139:0/1625422272 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc079060 con 0x564aea61fa30 2024-09-16T01:52:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.026+0000 7fa8baffd640 1 -- 172.21.15.139:0/1625422272 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7fa8cc05f9b0 con 0x564aea61fa30 2024-09-16T01:52:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.026+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/1625422272 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564aea586890 con 0x564aea61fa30 2024-09-16T01:52:02.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.028+0000 7fa8baffd640 1 -- 172.21.15.139:0/1625422272 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa8cc036040 con 0x564aea61fa30 2024-09-16T01:52:02.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.028+0000 7fa8e3d3e740 10 client.26176 fetch_fsmap learned FSMap version 59 2024-09-16T01:52:02.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.028+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/1625422272 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x564aea69ac10 con 0x564aea61fa30 2024-09-16T01:52:02.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.029+0000 7fa8baffd640 1 -- 172.21.15.139:0/1625422272 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564aea586890 con 0x564aea61fa30 2024-09-16T01:52:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.030+0000 7fa8baffd640 1 -- 172.21.15.139:0/1625422272 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 59) ==== 8450+0+0 (secure 0 0 0) 0x7fa8cc05d040 con 0x564aea61fa30 2024-09-16T01:52:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.030+0000 7fa8e3d3e740 10 client.26176 fetch_fsmap finished waiting for FSMap version 59 2024-09-16T01:52:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.030+0000 7fa8e3d3e740 10 client.26176 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:52:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.030+0000 7fa8e3d3e740 20 client.26176 populate_metadata read hostname 'smithi139' 2024-09-16T01:52:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.030+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/1625422272 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564aea69af50 0x564aea69d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.030+0000 7fa8e3d3e740 4 client.26176 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}] 2024-09-16T01:52:02.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.030+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/1625422272 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}) -- 0x564aea580be0 con 0x564aea69af50 2024-09-16T01:52:02.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.033+0000 7fa8daffd640 1 --2- 172.21.15.139:0/1625422272 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564aea69af50 0x564aea69d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:02.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.038+0000 7fa8daffd640 1 --2- 172.21.15.139:0/1625422272 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564aea69af50 0x564aea69d340 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ecbe4cc9d062009 server_cookie=7c7a44180c31bedd in_seq=0 out_seq=0 2024-09-16T01:52:02.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:02.038+0000 7fa8baffd640 10 client.26176 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:02.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:02 smithi164 ceph-mon[35121]: osdmap e101: 12 total, 12 up, 12 in 2024-09-16T01:52:02.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:02 smithi139 ceph-mon[31175]: osdmap e101: 12 total, 12 up, 12 in 2024-09-16T01:52:02.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:02 smithi176 ceph-mon[35324]: osdmap e101: 12 total, 12 up, 12 in 2024-09-16T01:52:03.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:03 smithi164 ceph-mon[35121]: pgmap v367: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 408 KiB/s wr, 42 op/s 2024-09-16T01:52:03.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:03 smithi139 ceph-mon[31175]: pgmap v367: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 408 KiB/s wr, 42 op/s 2024-09-16T01:52:03.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:03 smithi176 ceph-mon[35324]: pgmap v367: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 408 KiB/s wr, 42 op/s 2024-09-16T01:52:05.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:05 smithi164 ceph-mon[35121]: pgmap v368: 129 pgs: 12 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 73 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 742 KiB/s wr, 82 op/s 2024-09-16T01:52:05.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:05 smithi139 ceph-mon[31175]: pgmap v368: 129 pgs: 12 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 73 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 742 KiB/s wr, 82 op/s 2024-09-16T01:52:05.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:05 smithi176 ceph-mon[35324]: pgmap v368: 129 pgs: 12 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 73 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 742 KiB/s wr, 82 op/s 2024-09-16T01:52:07.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:07 smithi164 ceph-mon[35121]: pgmap v369: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 623 KiB/s wr, 132 op/s 2024-09-16T01:52:07.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:07 smithi139 ceph-mon[31175]: pgmap v369: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 623 KiB/s wr, 132 op/s 2024-09-16T01:52:07.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:07 smithi176 ceph-mon[35324]: pgmap v369: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 623 KiB/s wr, 132 op/s 2024-09-16T01:52:09.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:09 smithi139 ceph-mon[31175]: pgmap v370: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 743 KiB/s wr, 171 op/s 2024-09-16T01:52:09.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:09 smithi176 ceph-mon[35324]: pgmap v370: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 743 KiB/s wr, 171 op/s 2024-09-16T01:52:09.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:09 smithi164 ceph-mon[35121]: pgmap v370: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 743 KiB/s wr, 171 op/s 2024-09-16T01:52:11.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:11 smithi139 ceph-mon[31175]: pgmap v371: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 745 KiB/s wr, 191 op/s 2024-09-16T01:52:11.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:11 smithi176 ceph-mon[35324]: pgmap v371: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 745 KiB/s wr, 191 op/s 2024-09-16T01:52:11.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:11 smithi164 ceph-mon[35121]: pgmap v371: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 745 KiB/s wr, 191 op/s 2024-09-16T01:52:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.559+0000 7f1e1a118640 1 -- 172.21.15.139:0/1124481249 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e141009d0 msgr2=0x7f1e14100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.559+0000 7f1e1a118640 1 --2- 172.21.15.139:0/1124481249 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e141009d0 0x7f1e14100e30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009860 tx=0x7f1e0802a380 comp rx=0 tx=0).stop 2024-09-16T01:52:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.562+0000 7f1e1a118640 1 -- 172.21.15.139:0/1124481249 shutdown_connections 2024-09-16T01:52:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.562+0000 7f1e1a118640 1 --2- 172.21.15.139:0/1124481249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e14101370 0x7f1e1406a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.562+0000 7f1e1a118640 1 --2- 172.21.15.139:0/1124481249 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e141009d0 0x7f1e14100e30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.562+0000 7f1e1a118640 1 --2- 172.21.15.139:0/1124481249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e141069e0 0x7f1e14106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.562+0000 7f1e1a118640 1 -- 172.21.15.139:0/1124481249 >> 172.21.15.139:0/1124481249 conn(0x7f1e140fc6f0 msgr2=0x7f1e140feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.563+0000 7f1e1a118640 1 -- 172.21.15.139:0/1124481249 shutdown_connections 2024-09-16T01:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.563+0000 7f1e1a118640 1 -- 172.21.15.139:0/1124481249 wait complete. 2024-09-16T01:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.563+0000 7f1e1a118640 1 Processor -- start 2024-09-16T01:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.564+0000 7f1e1a118640 1 -- start start 2024-09-16T01:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.564+0000 7f1e1a118640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e141009d0 0x7f1e1406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.564+0000 7f1e1a118640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e14101370 0x7f1e1406d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.564+0000 7f1e1a118640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e141069e0 0x7f1e14071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.564+0000 7f1e1a118640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1406bee0 con 0x7f1e141069e0 2024-09-16T01:52:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.564+0000 7f1e1a118640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1406bd60 con 0x7f1e14101370 2024-09-16T01:52:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.564+0000 7f1e1a118640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1406c060 con 0x7f1e141009d0 2024-09-16T01:52:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.565+0000 7f1e13fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e141069e0 0x7f1e14071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.565+0000 7f1e13fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e141069e0 0x7f1e14071ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44174/0 (socket says 172.21.15.139:44174) 2024-09-16T01:52:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.565+0000 7f1e137fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e141009d0 0x7f1e1406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.565+0000 7f1e137fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e141009d0 0x7f1e1406d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:33354/0 (socket says 172.21.15.139:33354) 2024-09-16T01:52:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.565+0000 7f1e13fff640 1 -- 172.21.15.139:0/303353199 learned_addr learned my addr 172.21.15.139:0/303353199 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.566+0000 7f1e13fff640 1 -- 172.21.15.139:0/303353199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e141009d0 msgr2=0x7f1e1406d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.566+0000 7f1e13fff640 1 --2- 172.21.15.139:0/303353199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e141009d0 0x7f1e1406d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.566+0000 7f1e13fff640 1 -- 172.21.15.139:0/303353199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e14101370 msgr2=0x7f1e1406d980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:52:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.566+0000 7f1e13fff640 1 --2- 172.21.15.139:0/303353199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e14101370 0x7f1e1406d980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.566+0000 7f1e13fff640 1 -- 172.21.15.139:0/303353199 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e14072400 con 0x7f1e141069e0 2024-09-16T01:52:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.567+0000 7f1e13fff640 1 --2- 172.21.15.139:0/303353199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e141069e0 0x7f1e14071ec0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00b970 tx=0x7f1e0c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.571+0000 7f1e10ff9640 1 -- 172.21.15.139:0/303353199 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c019070 con 0x7f1e141069e0 2024-09-16T01:52:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.571+0000 7f1e1a118640 1 -- 172.21.15.139:0/303353199 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e140726f0 con 0x7f1e141069e0 2024-09-16T01:52:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.571+0000 7f1e1a118640 1 -- 172.21.15.139:0/303353199 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e14072c10 con 0x7f1e141069e0 2024-09-16T01:52:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.571+0000 7f1e10ff9640 1 -- 172.21.15.139:0/303353199 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0c014440 con 0x7f1e141069e0 2024-09-16T01:52:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.571+0000 7f1e10ff9640 1 -- 172.21.15.139:0/303353199 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0075b0 con 0x7f1e141069e0 2024-09-16T01:52:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.578+0000 7f1e1ceb1740 1 Processor -- start 2024-09-16T01:52:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.578+0000 7f1e1ceb1740 1 -- start start 2024-09-16T01:52:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.578+0000 7f1e1ceb1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e4168aaf0 0x564e4168aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.579+0000 7f1e1ceb1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168b410 0x564e41695770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.579+0000 7f1e1ceb1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e41695cb0 0x564e416980a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.579+0000 7f1e12ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e4168aaf0 0x564e4168aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.579+0000 7f1e12ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e4168aaf0 0x564e4168aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:33362/0 (socket says 172.21.15.139:33362) 2024-09-16T01:52:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.579+0000 7f1e12ffd640 1 -- 172.21.15.139:0/4136120027 learned_addr learned my addr 172.21.15.139:0/4136120027 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.579+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/4136120027 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564e415ef2f0 con 0x564e41695cb0 2024-09-16T01:52:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.579+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/4136120027 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564e41597270 con 0x564e4168b410 2024-09-16T01:52:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.579+0000 7f1e137fe640 1 --2- 172.21.15.139:0/4136120027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168b410 0x564e41695770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.579+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/4136120027 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564e415eb2b0 con 0x564e4168aaf0 2024-09-16T01:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.583+0000 7f1e12ffd640 1 --2- 172.21.15.139:0/4136120027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e41695cb0 0x564e416980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.584+0000 7f1e10ff9640 1 -- 172.21.15.139:0/303353199 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1e0c00caa0 con 0x7f1e141069e0 2024-09-16T01:52:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.585+0000 7f1e10ff9640 1 --2- 172.21.15.139:0/303353199 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1dfc0a3610 0x7f1dfc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.589+0000 7f1e13fff640 1 --2- 172.21.15.139:0/303353199 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1dfc0a3610 0x7f1dfc0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.589+0000 7f1e10ff9640 1 -- 172.21.15.139:0/303353199 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f1e0c099480 con 0x7f1e141069e0 2024-09-16T01:52:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.589+0000 7f1e137fe640 1 -- 172.21.15.139:0/4136120027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e4168aaf0 msgr2=0x564e4168aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.589+0000 7f1e137fe640 1 --2- 172.21.15.139:0/4136120027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e4168aaf0 0x564e4168aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.589+0000 7f1e137fe640 1 -- 172.21.15.139:0/4136120027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e41695cb0 msgr2=0x564e416980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.589+0000 7f1e137fe640 1 --2- 172.21.15.139:0/4136120027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e41695cb0 0x564e416980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.589+0000 7f1e137fe640 1 -- 172.21.15.139:0/4136120027 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e41596340 con 0x564e4168b410 2024-09-16T01:52:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.590+0000 7f1e12ffd640 1 --2- 172.21.15.139:0/4136120027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e41695cb0 0x564e416980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:52:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.591+0000 7f1e137fe640 1 --2- 172.21.15.139:0/4136120027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168b410 0x564e41695770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1e04017580 tx=0x7f1e04011a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28f647d1ddf1e605 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.592+0000 7f1df9ffb640 1 -- 172.21.15.139:0/4136120027 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04021020 con 0x564e4168b410 2024-09-16T01:52:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.593+0000 7f1df9ffb640 1 -- 172.21.15.139:0/4136120027 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e04011110 con 0x564e4168b410 2024-09-16T01:52:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.593+0000 7f1df9ffb640 1 -- 172.21.15.139:0/4136120027 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04003850 con 0x564e4168b410 2024-09-16T01:52:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.594+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/4136120027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168b410 msgr2=0x564e41695770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.594+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/4136120027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168b410 0x564e41695770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1e04017580 tx=0x7f1e04011a70 comp rx=0 tx=0).stop 2024-09-16T01:52:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.594+0000 7f1e13fff640 1 --2- 172.21.15.139:0/303353199 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1dfc0a3610 0x7f1dfc0a5ad0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00b940 tx=0x7f1e0c09b1b0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.602+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/4136120027 shutdown_connections 2024-09-16T01:52:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.602+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/4136120027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e41695cb0 0x564e416980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.602+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/4136120027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168b410 0x564e41695770 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.602+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/4136120027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e4168aaf0 0x564e4168aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.602+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/4136120027 >> 172.21.15.139:0/4136120027 conn(0x564e41687060 msgr2=0x564e41687440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.608+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/4136120027 shutdown_connections 2024-09-16T01:52:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.610+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/4136120027 wait complete. 2024-09-16T01:52:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.610+0000 7f1e1ceb1740 1 -- start start 2024-09-16T01:52:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.611+0000 7f1e1ceb1740 1 Processor -- start 2024-09-16T01:52:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.612+0000 7f1e1ceb1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168aaf0 0x564e41692ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.612+0000 7f1e1ceb1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168b410 0x564e416930e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.612+0000 7f1e1ceb1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e41695cb0 0x564e41693620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.612+0000 7f1e1ceb1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564e41698880 con 0x564e4168b410 2024-09-16T01:52:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.612+0000 7f1e1ceb1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564e41578290 con 0x564e4168aaf0 2024-09-16T01:52:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.612+0000 7f1e1ceb1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564e41698a00 con 0x564e41695cb0 2024-09-16T01:52:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.612+0000 7f1e137fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168b410 0x564e416930e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.612+0000 7f1e137fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168b410 0x564e416930e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44200/0 (socket says 172.21.15.139:44200) 2024-09-16T01:52:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.612+0000 7f1e137fe640 1 -- 172.21.15.139:0/654090671 learned_addr learned my addr 172.21.15.139:0/654090671 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.612+0000 7f1e1ceb1740 10 client.0 _ll_register_callbacks cb 0x564e415a9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:52:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.615+0000 7f1e12ffd640 1 --2- 172.21.15.139:0/654090671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168aaf0 0x564e41692ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.615+0000 7f1e12ffd640 1 --2- 172.21.15.139:0/654090671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e41695cb0 0x564e41693620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.617+0000 7f1e137fe640 1 -- 172.21.15.139:0/654090671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e41695cb0 msgr2=0x564e41693620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.617+0000 7f1e137fe640 1 --2- 172.21.15.139:0/654090671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e41695cb0 0x564e41693620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.617+0000 7f1e137fe640 1 -- 172.21.15.139:0/654090671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168aaf0 msgr2=0x564e41692ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.617+0000 7f1e137fe640 1 --2- 172.21.15.139:0/654090671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168aaf0 0x564e41692ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.617+0000 7f1e137fe640 1 -- 172.21.15.139:0/654090671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e41302210 con 0x564e4168b410 2024-09-16T01:52:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.617+0000 7f1e137fe640 1 --2- 172.21.15.139:0/654090671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168b410 0x564e416930e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1e04005c80 tx=0x7f1e04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f726a8bc3dea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.617+0000 7f1df2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:52:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.617+0000 7f1df2ffd640 1 -- 172.21.15.139:0/654090671 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04021070 con 0x564e4168b410 2024-09-16T01:52:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.617+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/654090671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e04011110 con 0x564e4168b410 2024-09-16T01:52:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.618+0000 7f1df2ffd640 1 -- 172.21.15.139:0/654090671 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f1e04005230 con 0x564e4168b410 2024-09-16T01:52:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.618+0000 7f1df2ffd640 1 -- 172.21.15.139:0/654090671 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f1e04037050 con 0x564e4168b410 2024-09-16T01:52:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.620+0000 7f1e1ceb1740 10 client.16794 fetch_fsmap learned FSMap version 59 2024-09-16T01:52:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.620+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/654090671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x564e413ce620 con 0x564e4168b410 2024-09-16T01:52:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.622+0000 7f1df2ffd640 1 -- 172.21.15.139:0/654090671 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0402a030 con 0x564e4168b410 2024-09-16T01:52:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.622+0000 7f1df2ffd640 1 -- 172.21.15.139:0/654090671 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 59) ==== 8450+0+0 (secure 0 0 0) 0x7f1e040035f0 con 0x564e4168b410 2024-09-16T01:52:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.622+0000 7f1e1ceb1740 10 client.16794 fetch_fsmap finished waiting for FSMap version 59 2024-09-16T01:52:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.622+0000 7f1e1ceb1740 10 client.16794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:52:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.622+0000 7f1e1ceb1740 20 client.16794 populate_metadata read hostname 'smithi139' 2024-09-16T01:52:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.622+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/654090671 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564e416a6030 0x564e416a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.622+0000 7f1e1ceb1740 4 client.16794 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:52:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.622+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/654090671 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1e04011110 con 0x564e416a6030 2024-09-16T01:52:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.624+0000 7f1e12ffd640 1 --2- 172.21.15.139:0/654090671 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564e416a6030 0x564e416a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:11.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.631+0000 7f1e12ffd640 1 --2- 172.21.15.139:0/654090671 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564e416a6030 0x564e416a84d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d887b3dc0aae0 server_cookie=26655a6e76935a20 in_seq=0 out_seq=0 2024-09-16T01:52:11.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.631+0000 7f1df2ffd640 10 client.16794 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.646+0000 7f1df2ffd640 1 -- 172.21.15.139:0/654090671 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f1e04011110 con 0x564e416a6030 2024-09-16T01:52:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.646+0000 7f1df2ffd640 10 client.16794 handle_command_reply: tid=0 2024-09-16T01:52:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.647+0000 7f1e1ceb1740 1 client.16794 shutdown 2024-09-16T01:52:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.651+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/654090671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168b410 msgr2=0x564e416930e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.651+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/654090671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168b410 0x564e416930e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1e04005c80 tx=0x7f1e04004270 comp rx=0 tx=0).stop 2024-09-16T01:52:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.658+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/654090671 shutdown_connections 2024-09-16T01:52:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.658+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/654090671 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564e416a6030 0x564e416a84d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.658+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/654090671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e41695cb0 0x564e41693620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.658+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/654090671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168b410 0x564e416930e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.658+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/654090671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168aaf0 0x564e41692ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.658+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/654090671 >> 172.21.15.139:0/654090671 conn(0x564e41687060 msgr2=0x564e4168cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.658+0000 7f1df2ffd640 0 client.16794 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.662+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/654090671 shutdown_connections 2024-09-16T01:52:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.664+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/654090671 wait complete. 2024-09-16T01:52:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.666+0000 7f1e1ceb1740 20 client.16794 trim_cache size 0 max 16384 2024-09-16T01:52:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.696+0000 7f1e1ceb1740 1 Processor -- start 2024-09-16T01:52:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.697+0000 7f1e1ceb1740 1 -- start start 2024-09-16T01:52:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.698+0000 7f1e1ceb1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168aaf0 0x564e41693490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.698+0000 7f1e1ceb1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e4168b410 0x564e4168cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.698+0000 7f1e1ceb1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e41695cb0 0x564e416adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.698+0000 7f1e1ceb1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00010280 con 0x564e4168aaf0 2024-09-16T01:52:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.698+0000 7f1e1ceb1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e000120c0 con 0x564e41695cb0 2024-09-16T01:52:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.698+0000 7f1e1ceb1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00010a10 con 0x564e4168b410 2024-09-16T01:52:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.698+0000 7f1e137fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e4168b410 0x564e4168cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.698+0000 7f1e137fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e4168b410 0x564e4168cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:33384/0 (socket says 172.21.15.139:33384) 2024-09-16T01:52:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.698+0000 7f1e137fe640 1 -- 172.21.15.139:0/822166882 learned_addr learned my addr 172.21.15.139:0/822166882 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.699+0000 7f1e12ffd640 1 --2- 172.21.15.139:0/822166882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e41695cb0 0x564e416adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.699+0000 7f1e137fe640 1 -- 172.21.15.139:0/822166882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e41695cb0 msgr2=0x564e416adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.699+0000 7f1e137fe640 1 --2- 172.21.15.139:0/822166882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e41695cb0 0x564e416adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.699+0000 7f1e137fe640 1 -- 172.21.15.139:0/822166882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168aaf0 msgr2=0x564e41693490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.699+0000 7f1e137fe640 1 --2- 172.21.15.139:0/822166882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168aaf0 0x564e41693490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.699+0000 7f1e137fe640 1 -- 172.21.15.139:0/822166882 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e416a2640 con 0x564e4168b410 2024-09-16T01:52:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.700+0000 7f1e137fe640 1 --2- 172.21.15.139:0/822166882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e4168b410 0x564e4168cc60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1e04011ea0 tx=0x7f1e04010fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=734b71ab6efb5c95 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.701+0000 7f1df97fa640 1 -- 172.21.15.139:0/822166882 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04021070 con 0x564e4168b410 2024-09-16T01:52:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.704+0000 7f1df97fa640 1 -- 172.21.15.139:0/822166882 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0402a070 con 0x564e4168b410 2024-09-16T01:52:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.704+0000 7f1df97fa640 1 -- 172.21.15.139:0/822166882 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04005380 con 0x564e4168b410 2024-09-16T01:52:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.705+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/822166882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e4168b410 msgr2=0x564e4168cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.705+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/822166882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e4168b410 0x564e4168cc60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1e04011ea0 tx=0x7f1e04010fc0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.711+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/822166882 shutdown_connections 2024-09-16T01:52:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.711+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/822166882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e41695cb0 0x564e416adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.711+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/822166882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e4168b410 0x564e4168cc60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.711+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/822166882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168aaf0 0x564e41693490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.711+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/822166882 >> 172.21.15.139:0/822166882 conn(0x564e41687060 msgr2=0x564e4168c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.714+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/822166882 shutdown_connections 2024-09-16T01:52:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.716+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/822166882 wait complete. 2024-09-16T01:52:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.716+0000 7f1e1ceb1740 1 -- start start 2024-09-16T01:52:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.716+0000 7f1e1ceb1740 1 Processor -- start 2024-09-16T01:52:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.717+0000 7f1e1ceb1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168aaf0 0x564e4169dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.717+0000 7f1e1ceb1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168b410 0x564e4169e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.717+0000 7f1e1ceb1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e41695cb0 0x564e4170bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.717+0000 7f1e1ceb1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00011930 con 0x564e4168aaf0 2024-09-16T01:52:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.717+0000 7f1e137fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168b410 0x564e4169e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.717+0000 7f1e137fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168b410 0x564e4169e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:51670/0 (socket says 172.21.15.139:51670) 2024-09-16T01:52:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.717+0000 7f1e137fe640 1 -- 172.21.15.139:0/559387751 learned_addr learned my addr 172.21.15.139:0/559387751 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.717+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/559387751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e000111a0 con 0x564e4168b410 2024-09-16T01:52:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.717+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/559387751 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0000e280 con 0x564e41695cb0 2024-09-16T01:52:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.718+0000 7f1e1ceb1740 10 client.0 _ll_register_callbacks cb 0x7f1e04017480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.721+0000 7f1e137fe640 1 -- 172.21.15.139:0/559387751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e41695cb0 msgr2=0x564e4170bd50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.721+0000 7f1e137fe640 1 --2- 172.21.15.139:0/559387751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e41695cb0 0x564e4170bd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.721+0000 7f1e137fe640 1 -- 172.21.15.139:0/559387751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168aaf0 msgr2=0x564e4169dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.721+0000 7f1e137fe640 1 --2- 172.21.15.139:0/559387751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168aaf0 0x564e4169dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.721+0000 7f1e137fe640 1 -- 172.21.15.139:0/559387751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e4170c390 con 0x564e4168b410 2024-09-16T01:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.721+0000 7f1e137fe640 1 --2- 172.21.15.139:0/559387751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168b410 0x564e4169e260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1e04005ce0 tx=0x7f1e04011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd11a96170cb4def server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.721+0000 7f1df2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.722+0000 7f1df2ffd640 1 -- 172.21.15.139:0/559387751 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04021070 con 0x564e4168b410 2024-09-16T01:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.722+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/559387751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e0402a070 con 0x564e4168b410 2024-09-16T01:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.722+0000 7f1df2ffd640 1 -- 172.21.15.139:0/559387751 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f1e040114d0 con 0x564e4168b410 2024-09-16T01:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.723+0000 7f1df2ffd640 1 -- 172.21.15.139:0/559387751 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f1e04044050 con 0x564e4168b410 2024-09-16T01:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.723+0000 7f1e1ceb1740 10 client.26213 fetch_fsmap learned FSMap version 59 2024-09-16T01:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.723+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/559387751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x564e4168fd40 con 0x564e4168b410 2024-09-16T01:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.724+0000 7f1df2ffd640 1 -- 172.21.15.139:0/559387751 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0402a070 con 0x564e4168b410 2024-09-16T01:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.724+0000 7f1df2ffd640 1 -- 172.21.15.139:0/559387751 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 59) ==== 8450+0+0 (secure 0 0 0) 0x7f1e0403b670 con 0x564e4168b410 2024-09-16T01:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.724+0000 7f1e1ceb1740 10 client.26213 fetch_fsmap finished waiting for FSMap version 59 2024-09-16T01:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.724+0000 7f1e1ceb1740 10 client.26213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.724+0000 7f1e1ceb1740 20 client.26213 populate_metadata read hostname 'smithi139' 2024-09-16T01:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.724+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/559387751 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564e41721e90 0x564e417242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.724+0000 7f1e1ceb1740 4 client.26213 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.724+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/559387751 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e415fca20 con 0x564e41721e90 2024-09-16T01:52:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.731+0000 7f1e12ffd640 1 --2- 172.21.15.139:0/559387751 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564e41721e90 0x564e417242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.747+0000 7f1df2ffd640 10 client.26213 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:11.747+0000 7f1e12ffd640 1 --2- 172.21.15.139:0/559387751 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564e41721e90 0x564e417242b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d764d9845e791b3b server_cookie=f7642f10470edc2b in_seq=0 out_seq=0 2024-09-16T01:52:13.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:13 smithi139 ceph-mon[31175]: pgmap v372: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 624 KiB/s wr, 196 op/s 2024-09-16T01:52:13.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:13 smithi176 ceph-mon[35324]: pgmap v372: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 624 KiB/s wr, 196 op/s 2024-09-16T01:52:13.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:13 smithi164 ceph-mon[35121]: pgmap v372: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 624 KiB/s wr, 196 op/s 2024-09-16T01:52:14.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.537+0000 7fa8baffd640 1 -- 172.21.15.139:0/1625422272 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564aea580be0 con 0x564aea69af50 2024-09-16T01:52:14.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.537+0000 7fa8baffd640 10 client.26176 handle_command_reply: tid=0 2024-09-16T01:52:14.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.537+0000 7fa8e3d3e740 1 client.26176 shutdown 2024-09-16T01:52:14.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.537+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/1625422272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 msgr2=0x564aea61a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:14.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.537+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/1625422272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 0x564aea61a3e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x564aea615850 tx=0x7fa8cc05f6a0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.538+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/1625422272 shutdown_connections 2024-09-16T01:52:14.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.538+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/1625422272 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564aea69af50 0x564aea69d340 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.538+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/1625422272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aea61fa30 0x564aea61a3e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.538+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/1625422272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564aea615170 0x564aea619ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.538+0000 7fa8e3d3e740 1 --2- 172.21.15.139:0/1625422272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564aea614850 0x564aea619960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.538+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/1625422272 >> 172.21.15.139:0/1625422272 conn(0x564aea610de0 msgr2=0x564aea60f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:14.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.538+0000 7fa8baffd640 0 client.26176 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:14.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.538+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/1625422272 shutdown_connections 2024-09-16T01:52:14.543 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:52:14.543 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 60, 2024-09-16T01:52:14.543 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:52:14.543 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 42, 2024-09-16T01:52:14.543 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:52:14.543 INFO:teuthology.orchestra.run.smithi139.stdout: "d1c43de6": { 2024-09-16T01:52:14.544 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 42, 2024-09-16T01:52:14.544 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:52:14.544 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:52:14.544 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:52:14.544 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:52:14.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:52:14.545 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:52:14.545 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 141.8, 2024-09-16T01:52:14.545 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:52:14.545 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:52:14.545 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:52:14.546 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:52:14.546 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:52:14.546 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:52:14.546 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:52:14.546 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:52:14.546 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:52:14.547 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:52:14.547 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:52:14.547 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:52:14.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.538+0000 7fa8e3d3e740 1 -- 172.21.15.139:0/1625422272 wait complete. 2024-09-16T01:52:14.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.538+0000 7fa8e3d3e740 20 client.26176 trim_cache size 0 max 16384 2024-09-16T01:52:14.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.539+0000 7fa8e1965640 1 -- 172.21.15.139:0/3010180170 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa8c80a35e0 msgr2=0x7fa8c80a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:14.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.539+0000 7fa8e1965640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa8c80a35e0 0x7fa8c80a5aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc068000 tx=0x7fa8cc06a000 comp rx=0 tx=0).stop 2024-09-16T01:52:14.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.539+0000 7fa8e1965640 1 -- 172.21.15.139:0/3010180170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8dc1091f0 msgr2=0x7fa8dc119dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:14.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.540+0000 7fa8e1965640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8dc1091f0 0x7fa8dc119dd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400e9c0 tx=0x7fa8d400ee80 comp rx=0 tx=0).stop 2024-09-16T01:52:14.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.540+0000 7fa8e1965640 1 -- 172.21.15.139:0/3010180170 shutdown_connections 2024-09-16T01:52:14.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.540+0000 7fa8e1965640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa8c80a35e0 0x7fa8c80a5aa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.540+0000 7fa8e1965640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8dc11a850 0x7fa8dc11e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.540+0000 7fa8e1965640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8dc10e890 0x7fa8dc11a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.540+0000 7fa8e1965640 1 --2- 172.21.15.139:0/3010180170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8dc1091f0 0x7fa8dc119dd0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.540+0000 7fa8e1965640 1 -- 172.21.15.139:0/3010180170 >> 172.21.15.139:0/3010180170 conn(0x7fa8dc06e2d0 msgr2=0x7fa8dc112700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:14.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.540+0000 7fa8e1965640 1 -- 172.21.15.139:0/3010180170 shutdown_connections 2024-09-16T01:52:14.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:14.540+0000 7fa8e1965640 1 -- 172.21.15.139:0/3010180170 wait complete. 2024-09-16T01:52:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.566+0000 7f1df2ffd640 1 -- 172.21.15.139:0/559387751 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e415fca20 con 0x564e41721e90 2024-09-16T01:52:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.566+0000 7f1df2ffd640 10 client.26213 handle_command_reply: tid=0 2024-09-16T01:52:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.566+0000 7f1e1ceb1740 1 client.26213 shutdown 2024-09-16T01:52:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.566+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/559387751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168b410 msgr2=0x564e4169e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.567+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/559387751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168b410 0x564e4169e260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1e04005ce0 tx=0x7f1e04011ca0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.567+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/559387751 shutdown_connections 2024-09-16T01:52:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.567+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/559387751 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564e41721e90 0x564e417242b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.567+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/559387751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e41695cb0 0x564e4170bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.567+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/559387751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e4168b410 0x564e4169e260 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.567+0000 7f1e1ceb1740 1 --2- 172.21.15.139:0/559387751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564e4168aaf0 0x564e4169dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.567+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/559387751 >> 172.21.15.139:0/559387751 conn(0x564e41687060 msgr2=0x564e4168cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.567+0000 7f1df2ffd640 0 client.26213 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.567+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/559387751 shutdown_connections 2024-09-16T01:52:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.567+0000 7f1e1ceb1740 1 -- 172.21.15.139:0/559387751 wait complete. 2024-09-16T01:52:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.567+0000 7f1e1ceb1740 20 client.26213 trim_cache size 0 max 16384 2024-09-16T01:52:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:52:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:52:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:52:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:52:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.568+0000 7f1e1a118640 1 -- 172.21.15.139:0/303353199 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1dfc0a3610 msgr2=0x7f1dfc0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.568+0000 7f1e1a118640 1 --2- 172.21.15.139:0/303353199 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1dfc0a3610 0x7f1dfc0a5ad0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00b940 tx=0x7f1e0c09b1b0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.568+0000 7f1e1a118640 1 -- 172.21.15.139:0/303353199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e141069e0 msgr2=0x7f1e14071ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.568+0000 7f1e1a118640 1 --2- 172.21.15.139:0/303353199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e141069e0 0x7f1e14071ec0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00b970 tx=0x7f1e0c00bd70 comp rx=0 tx=0).stop 2024-09-16T01:52:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.568+0000 7f1e1a118640 1 -- 172.21.15.139:0/303353199 shutdown_connections 2024-09-16T01:52:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.568+0000 7f1e1a118640 1 --2- 172.21.15.139:0/303353199 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1dfc0a3610 0x7f1dfc0a5ad0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.568+0000 7f1e1a118640 1 --2- 172.21.15.139:0/303353199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e141069e0 0x7f1e14071ec0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.568+0000 7f1e1a118640 1 --2- 172.21.15.139:0/303353199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e14101370 0x7f1e1406d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.568+0000 7f1e1a118640 1 --2- 172.21.15.139:0/303353199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e141009d0 0x7f1e1406d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.568+0000 7f1e1a118640 1 -- 172.21.15.139:0/303353199 >> 172.21.15.139:0/303353199 conn(0x7f1e140fc6f0 msgr2=0x7f1e140fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.568+0000 7f1e1a118640 1 -- 172.21.15.139:0/303353199 shutdown_connections 2024-09-16T01:52:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:14.568+0000 7f1e1a118640 1 -- 172.21.15.139:0/303353199 wait complete. 2024-09-16T01:52:15.304 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:15 smithi139 ceph-mon[31175]: pgmap v373: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 376 KiB/s wr, 171 op/s 2024-09-16T01:52:15.304 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:15 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:15.356 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd1c43de6', quiesce took 16.3 seconds. Will release after: 5.550000000000001 2024-09-16T01:52:15.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:15 smithi176 ceph-mon[35324]: pgmap v373: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 376 KiB/s wr, 171 op/s 2024-09-16T01:52:15.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:15 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:15.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:15 smithi164 ceph-mon[35121]: pgmap v373: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 376 KiB/s wr, 171 op/s 2024-09-16T01:52:15.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:15 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:17.346 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:17 smithi164 ceph-mon[35121]: pgmap v374: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 174 KiB/s wr, 153 op/s 2024-09-16T01:52:17.425 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:17 smithi139 ceph-mon[31175]: pgmap v374: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 174 KiB/s wr, 153 op/s 2024-09-16T01:52:17.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:17 smithi176 ceph-mon[35324]: pgmap v374: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 174 KiB/s wr, 153 op/s 2024-09-16T01:52:19.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:19 smithi139 ceph-mon[31175]: pgmap v375: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 195 KiB/s wr, 110 op/s 2024-09-16T01:52:19.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:19 smithi176 ceph-mon[35324]: pgmap v375: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 195 KiB/s wr, 110 op/s 2024-09-16T01:52:19.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:19 smithi164 ceph-mon[35121]: pgmap v375: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 195 KiB/s wr, 110 op/s 2024-09-16T01:52:20.907 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:52:21.418 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:52:21.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:21 smithi139 ceph-mon[31175]: pgmap v376: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 92 KiB/s wr, 76 op/s 2024-09-16T01:52:21.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:21 smithi176 ceph-mon[35324]: pgmap v376: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 92 KiB/s wr, 76 op/s 2024-09-16T01:52:21.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:21 smithi164 ceph-mon[35121]: pgmap v376: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 92 KiB/s wr, 76 op/s 2024-09-16T01:52:22.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.097+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/1723362705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e4406ea30 msgr2=0x7f7e4406ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:22.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.097+0000 7f7e4b8b3640 1 --2- 172.21.15.139:0/1723362705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e4406ea30 0x7f7e4406ee90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7e3801a910 tx=0x7f7e3803b450 comp rx=0 tx=0).stop 2024-09-16T01:52:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.101+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/1723362705 shutdown_connections 2024-09-16T01:52:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.101+0000 7f7e4b8b3640 1 --2- 172.21.15.139:0/1723362705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4411b140 0x7f7e4411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.101+0000 7f7e4b8b3640 1 --2- 172.21.15.139:0/1723362705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e4406ea30 0x7f7e4406ee90 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.101+0000 7f7e4b8b3640 1 --2- 172.21.15.139:0/1723362705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e44110220 0x7f7e4406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:22.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.101+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/1723362705 >> 172.21.15.139:0/1723362705 conn(0x7f7e44073210 msgr2=0x7f7e44073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.102+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/1723362705 shutdown_connections 2024-09-16T01:52:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.102+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/1723362705 wait complete. 2024-09-16T01:52:22.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.103+0000 7f7e4b8b3640 1 Processor -- start 2024-09-16T01:52:22.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4b8b3640 1 -- start start 2024-09-16T01:52:22.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4b8b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e4406ea30 0x7f7e4411a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:22.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4b8b3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e44110220 0x7f7e44116910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:22.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4b8b3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4411b140 0x7f7e44116e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:22.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4b8b3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7e44113340 con 0x7f7e4406ea30 2024-09-16T01:52:22.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4b8b3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441131c0 con 0x7f7e4411b140 2024-09-16T01:52:22.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4b8b3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7e441134c0 con 0x7f7e44110220 2024-09-16T01:52:22.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4964f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e4406ea30 0x7f7e4411a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:22.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4964f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e4406ea30 0x7f7e4411a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51552/0 (socket says 172.21.15.139:51552) 2024-09-16T01:52:22.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4964f640 1 -- 172.21.15.139:0/841715866 learned_addr learned my addr 172.21.15.139:0/841715866 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:22.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e48e4e640 1 --2- 172.21.15.139:0/841715866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e44110220 0x7f7e44116910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:22.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e49e50640 1 --2- 172.21.15.139:0/841715866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4411b140 0x7f7e44116e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:22.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4964f640 1 -- 172.21.15.139:0/841715866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e44110220 msgr2=0x7f7e44116910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:22.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4964f640 1 --2- 172.21.15.139:0/841715866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e44110220 0x7f7e44116910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:22.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4964f640 1 -- 172.21.15.139:0/841715866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4411b140 msgr2=0x7f7e44116e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:22.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4964f640 1 --2- 172.21.15.139:0/841715866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4411b140 0x7f7e44116e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:22.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.104+0000 7f7e4964f640 1 -- 172.21.15.139:0/841715866 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e44117580 con 0x7f7e4406ea30 2024-09-16T01:52:22.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.105+0000 7f7e4964f640 1 --2- 172.21.15.139:0/841715866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e4406ea30 0x7f7e4411a870 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000ebc0 tx=0x7f7e4000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:22.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.109+0000 7f7e367fc640 1 -- 172.21.15.139:0/841715866 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40010630 con 0x7f7e4406ea30 2024-09-16T01:52:22.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.109+0000 7f7e367fc640 1 -- 172.21.15.139:0/841715866 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e40010cb0 con 0x7f7e4406ea30 2024-09-16T01:52:22.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.109+0000 7f7e367fc640 1 -- 172.21.15.139:0/841715866 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40018bd0 con 0x7f7e4406ea30 2024-09-16T01:52:22.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.112+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/841715866 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e441cd0b0 con 0x7f7e4406ea30 2024-09-16T01:52:22.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.112+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/841715866 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e441cd4b0 con 0x7f7e4406ea30 2024-09-16T01:52:22.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.113+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/841715866 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e44109740 con 0x7f7e4406ea30 2024-09-16T01:52:22.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.118+0000 7f7e367fc640 1 -- 172.21.15.139:0/841715866 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7e400107d0 con 0x7f7e4406ea30 2024-09-16T01:52:22.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.118+0000 7f7e367fc640 1 --2- 172.21.15.139:0/841715866 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f7e300a35c0 0x7f7e300a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.119+0000 7f7e367fc640 1 -- 172.21.15.139:0/841715866 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f7e400d0810 con 0x7f7e4406ea30 2024-09-16T01:52:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.119+0000 7f7e48e4e640 1 --2- 172.21.15.139:0/841715866 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f7e300a35c0 0x7f7e300a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.120+0000 7f7e48e4e640 1 --2- 172.21.15.139:0/841715866 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f7e300a35c0 0x7f7e300a5a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7e38006010 tx=0x7f7e38005b50 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:22.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.127+0000 7f7e367fc640 1 -- 172.21.15.139:0/841715866 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e4009cd30 con 0x7f7e4406ea30 2024-09-16T01:52:22.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.522+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/841715866 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e4406ee90 con 0x7f7e4406ea30 2024-09-16T01:52:22.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.524+0000 7f7e367fc640 1 -- 172.21.15.139:0/841715866 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+12026 (secure 0 0 0) 0x7f7e4009c630 con 0x7f7e4406ea30 2024-09-16T01:52:22.526 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:52:22.527 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":60,"btime":"2024-09-16T01:52:14:062472+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:52:13.360579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/3662238682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3662238682},{"type":"v1","addr":"172.21.15.164:6835","nonce":3662238682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24454,24443,24442,24433]},"id":1}]} 2024-09-16T01:52:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 60 2024-09-16T01:52:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.529+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/841715866 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f7e300a35c0 msgr2=0x7f7e300a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.529+0000 7f7e4b8b3640 1 --2- 172.21.15.139:0/841715866 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f7e300a35c0 0x7f7e300a5a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7e38006010 tx=0x7f7e38005b50 comp rx=0 tx=0).stop 2024-09-16T01:52:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.529+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/841715866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e4406ea30 msgr2=0x7f7e4411a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.529+0000 7f7e4b8b3640 1 --2- 172.21.15.139:0/841715866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e4406ea30 0x7f7e4411a870 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000ebc0 tx=0x7f7e4000c6a0 comp rx=0 tx=0).stop 2024-09-16T01:52:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.529+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/841715866 shutdown_connections 2024-09-16T01:52:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.529+0000 7f7e4b8b3640 1 --2- 172.21.15.139:0/841715866 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f7e300a35c0 0x7f7e300a5a80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.529+0000 7f7e4b8b3640 1 --2- 172.21.15.139:0/841715866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e4411b140 0x7f7e44116e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.529+0000 7f7e4b8b3640 1 --2- 172.21.15.139:0/841715866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e44110220 0x7f7e44116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.529+0000 7f7e4b8b3640 1 --2- 172.21.15.139:0/841715866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e4406ea30 0x7f7e4411a870 secure :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000ebc0 tx=0x7f7e4000c6a0 comp rx=0 tx=0).stop 2024-09-16T01:52:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.529+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/841715866 >> 172.21.15.139:0/841715866 conn(0x7f7e44073210 msgr2=0x7f7e4411b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.530+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/841715866 shutdown_connections 2024-09-16T01:52:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:22.531+0000 7f7e4b8b3640 1 -- 172.21.15.139:0/841715866 wait complete. 2024-09-16T01:52:23.273 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id d1c43de6 --release --await' 2024-09-16T01:52:23.273 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id d1c43de6 --release --await 2024-09-16T01:52:23.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:23 smithi139 ceph-mon[31175]: pgmap v377: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 95 KiB/s wr, 61 op/s 2024-09-16T01:52:23.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:23 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/841715866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:52:23.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:23 smithi176 ceph-mon[35324]: pgmap v377: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 95 KiB/s wr, 61 op/s 2024-09-16T01:52:23.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:23 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/841715866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:52:23.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:23 smithi164 ceph-mon[35121]: pgmap v377: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 95 KiB/s wr, 61 op/s 2024-09-16T01:52:23.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:23 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/841715866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:52:23.820 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:52:24.173 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:52:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:52:24.105+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:52:24.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.489+0000 7f2d6c66c640 1 -- 172.21.15.139:0/1577381237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d64100480 msgr2=0x7f2d64100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:24.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.489+0000 7f2d6c66c640 1 --2- 172.21.15.139:0/1577381237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d64100480 0x7f2d64100860 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000aa10 tx=0x7f2d6002b760 comp rx=0 tx=0).stop 2024-09-16T01:52:24.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.497+0000 7f2d6c66c640 1 -- 172.21.15.139:0/1577381237 shutdown_connections 2024-09-16T01:52:24.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.497+0000 7f2d6c66c640 1 --2- 172.21.15.139:0/1577381237 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d641055b0 0x7f2d6410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.497+0000 7f2d6c66c640 1 --2- 172.21.15.139:0/1577381237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d64100e30 0x7f2d64104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.497+0000 7f2d6c66c640 1 --2- 172.21.15.139:0/1577381237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d64100480 0x7f2d64100860 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.497+0000 7f2d6c66c640 1 -- 172.21.15.139:0/1577381237 >> 172.21.15.139:0/1577381237 conn(0x7f2d64077210 msgr2=0x7f2d640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:24.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.497+0000 7f2d6c66c640 1 -- 172.21.15.139:0/1577381237 shutdown_connections 2024-09-16T01:52:24.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.500+0000 7f2d6c66c640 1 -- 172.21.15.139:0/1577381237 wait complete. 2024-09-16T01:52:24.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.501+0000 7f2d6c66c640 1 Processor -- start 2024-09-16T01:52:24.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.501+0000 7f2d6c66c640 1 -- start start 2024-09-16T01:52:24.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.501+0000 7f2d6c66c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d64100480 0x7f2d640739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.501+0000 7f2d6c66c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d64100e30 0x7f2d64073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.501+0000 7f2d6c66c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d641055b0 0x7f2d6406cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.501+0000 7f2d6c66c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6410fea0 con 0x7f2d641055b0 2024-09-16T01:52:24.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.501+0000 7f2d6c66c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6410fd20 con 0x7f2d64100e30 2024-09-16T01:52:24.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.501+0000 7f2d6c66c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2d64110020 con 0x7f2d64100480 2024-09-16T01:52:24.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.503+0000 7f2d6a408640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d64100480 0x7f2d640739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:24.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.503+0000 7f2d6a408640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d64100480 0x7f2d640739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:38636/0 (socket says 172.21.15.139:38636) 2024-09-16T01:52:24.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.503+0000 7f2d6a408640 1 -- 172.21.15.139:0/2639924332 learned_addr learned my addr 172.21.15.139:0/2639924332 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:24.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.503+0000 7f2d69c07640 1 --2- 172.21.15.139:0/2639924332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d64100e30 0x7f2d64073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:24.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.504+0000 7f2d6ac09640 1 --2- 172.21.15.139:0/2639924332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d641055b0 0x7f2d6406cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:24.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.504+0000 7f2d6a408640 1 -- 172.21.15.139:0/2639924332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d64100e30 msgr2=0x7f2d64073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:24.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.504+0000 7f2d6a408640 1 --2- 172.21.15.139:0/2639924332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d64100e30 0x7f2d64073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.504+0000 7f2d6a408640 1 -- 172.21.15.139:0/2639924332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d641055b0 msgr2=0x7f2d6406cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:24.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.504+0000 7f2d6a408640 1 --2- 172.21.15.139:0/2639924332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d641055b0 0x7f2d6406cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.504+0000 7f2d6a408640 1 -- 172.21.15.139:0/2639924332 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6406d4d0 con 0x7f2d64100480 2024-09-16T01:52:24.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.506+0000 7f2d6a408640 1 --2- 172.21.15.139:0/2639924332 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d64100480 0x7f2d640739a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d60006010 tx=0x7f2d60003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:24.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.509+0000 7f2d5b7fe640 1 -- 172.21.15.139:0/2639924332 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6003f070 con 0x7f2d64100480 2024-09-16T01:52:24.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.509+0000 7f2d6c66c640 1 -- 172.21.15.139:0/2639924332 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6406d760 con 0x7f2d64100480 2024-09-16T01:52:24.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.509+0000 7f2d6c66c640 1 -- 172.21.15.139:0/2639924332 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d640716f0 con 0x7f2d64100480 2024-09-16T01:52:24.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.509+0000 7f2d5b7fe640 1 -- 172.21.15.139:0/2639924332 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d60003e30 con 0x7f2d64100480 2024-09-16T01:52:24.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.509+0000 7f2d5b7fe640 1 -- 172.21.15.139:0/2639924332 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60036040 con 0x7f2d64100480 2024-09-16T01:52:24.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.517+0000 7f2d5b7fe640 1 -- 172.21.15.139:0/2639924332 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f2d60042070 con 0x7f2d64100480 2024-09-16T01:52:24.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.518+0000 7f2d6ea45740 1 Processor -- start 2024-09-16T01:52:24.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.519+0000 7f2d5b7fe640 1 --2- 172.21.15.139:0/2639924332 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2d500a35c0 0x7f2d500a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.519+0000 7f2d6ac09640 1 --2- 172.21.15.139:0/2639924332 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2d500a35c0 0x7f2d500a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:24.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.519+0000 7f2d5b7fe640 1 -- 172.21.15.139:0/2639924332 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f2d600f0900 con 0x7f2d64100480 2024-09-16T01:52:24.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.520+0000 7f2d6ea45740 1 -- start start 2024-09-16T01:52:24.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.520+0000 7f2d6ea45740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6df9060 0x5616d6df9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.520+0000 7f2d6ea45740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6df9980 0x5616d6e03d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.520+0000 7f2d6ea45740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6e04240 0x5616d6e06630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.520+0000 7f2d6ea45740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5616d6d5e620 con 0x5616d6df9980 2024-09-16T01:52:24.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.520+0000 7f2d6ea45740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5616d6d05e50 con 0x5616d6df9060 2024-09-16T01:52:24.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.520+0000 7f2d6ea45740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5616d6d5a5e0 con 0x5616d6e04240 2024-09-16T01:52:24.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.521+0000 7f2d6a408640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6df9060 0x5616d6df9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:24.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.521+0000 7f2d6a408640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6df9060 0x5616d6df9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:42422/0 (socket says 172.21.15.139:42422) 2024-09-16T01:52:24.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.521+0000 7f2d6a408640 1 -- 172.21.15.139:0/2567965450 learned_addr learned my addr 172.21.15.139:0/2567965450 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:24.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.522+0000 7f2d6a408640 1 -- 172.21.15.139:0/2567965450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6e04240 msgr2=0x5616d6e06630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:52:24.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.523+0000 7f2d6a408640 1 --2- 172.21.15.139:0/2567965450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6e04240 0x5616d6e06630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.523+0000 7f2d6a408640 1 -- 172.21.15.139:0/2567965450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6df9980 msgr2=0x5616d6e03d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:24.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.523+0000 7f2d6a408640 1 --2- 172.21.15.139:0/2567965450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6df9980 0x5616d6e03d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.523+0000 7f2d6a408640 1 -- 172.21.15.139:0/2567965450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616d6cf6d40 con 0x5616d6df9060 2024-09-16T01:52:24.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.527+0000 7f2d6ac09640 1 --2- 172.21.15.139:0/2639924332 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2d500a35c0 0x7f2d500a5a80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x5616d6df6970 tx=0x7f2d5c009450 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:24.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.528+0000 7f2d6a408640 1 --2- 172.21.15.139:0/2567965450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6df9060 0x5616d6df9440 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2d600c4e60 tx=0x7f2d6000a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=242700416867fe2b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:24.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.536+0000 7f2d58ff9640 1 -- 172.21.15.139:0/2567965450 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60049040 con 0x5616d6df9060 2024-09-16T01:52:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.536+0000 7f2d58ff9640 1 -- 172.21.15.139:0/2567965450 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d600c0ad0 con 0x5616d6df9060 2024-09-16T01:52:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.536+0000 7f2d58ff9640 1 -- 172.21.15.139:0/2567965450 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d600c5160 con 0x5616d6df9060 2024-09-16T01:52:24.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.539+0000 7f2d6ea45740 1 -- 172.21.15.139:0/2567965450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6df9060 msgr2=0x5616d6df9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:24.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.539+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/2567965450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6df9060 0x5616d6df9440 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2d600c4e60 tx=0x7f2d6000a640 comp rx=0 tx=0).stop 2024-09-16T01:52:24.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.542+0000 7f2d6ea45740 1 -- 172.21.15.139:0/2567965450 shutdown_connections 2024-09-16T01:52:24.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.542+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/2567965450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6e04240 0x5616d6e06630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.542+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/2567965450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6df9980 0x5616d6e03d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.542+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/2567965450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6df9060 0x5616d6df9440 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.542+0000 7f2d6ea45740 1 -- 172.21.15.139:0/2567965450 >> 172.21.15.139:0/2567965450 conn(0x5616d6df55f0 msgr2=0x5616d6df59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:24.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.547+0000 7f2d6ea45740 1 -- 172.21.15.139:0/2567965450 shutdown_connections 2024-09-16T01:52:24.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.550+0000 7f2d6ea45740 1 -- 172.21.15.139:0/2567965450 wait complete. 2024-09-16T01:52:24.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.551+0000 7f2d6ea45740 1 -- start start 2024-09-16T01:52:24.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.551+0000 7f2d6ea45740 1 Processor -- start 2024-09-16T01:52:24.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.553+0000 7f2d6ea45740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6df9060 0x5616d6e01180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.553+0000 7f2d6ea45740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9980 0x5616d6e016c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.553+0000 7f2d6ea45740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6e04240 0x5616d6e01c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.553+0000 7f2d6ea45740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5616d6e06bd0 con 0x5616d6df9060 2024-09-16T01:52:24.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.553+0000 7f2d6ea45740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5616d6ce6f10 con 0x5616d6e04240 2024-09-16T01:52:24.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.553+0000 7f2d6ea45740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5616d6e06d50 con 0x5616d6df9980 2024-09-16T01:52:24.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.553+0000 7f2d6ea45740 10 client.0 _ll_register_callbacks cb 0x5616d6d0adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:52:24.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.553+0000 7f2d69c07640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9980 0x5616d6e016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:24.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.553+0000 7f2d6ac09640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6e04240 0x5616d6e01c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:24.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.553+0000 7f2d69c07640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9980 0x5616d6e016c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:38642/0 (socket says 172.21.15.139:38642) 2024-09-16T01:52:24.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.553+0000 7f2d69c07640 1 -- 172.21.15.139:0/3601137384 learned_addr learned my addr 172.21.15.139:0/3601137384 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:24.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.554+0000 7f2d6a408640 1 --2- 172.21.15.139:0/3601137384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6df9060 0x5616d6e01180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:24.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.558+0000 7f2d69c07640 1 -- 172.21.15.139:0/3601137384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6e04240 msgr2=0x5616d6e01c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:24.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.558+0000 7f2d69c07640 1 --2- 172.21.15.139:0/3601137384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6e04240 0x5616d6e01c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.558+0000 7f2d69c07640 1 -- 172.21.15.139:0/3601137384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6df9060 msgr2=0x5616d6e01180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:24.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.558+0000 7f2d69c07640 1 --2- 172.21.15.139:0/3601137384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6df9060 0x5616d6e01180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.558+0000 7f2d69c07640 1 -- 172.21.15.139:0/3601137384 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616d6a6d580 con 0x5616d6df9980 2024-09-16T01:52:24.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.558+0000 7f2d6a408640 1 --2- 172.21.15.139:0/3601137384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6df9060 0x5616d6e01180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:52:24.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.558+0000 7f2d69c07640 1 --2- 172.21.15.139:0/3601137384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9980 0x5616d6e016c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400bfc0 tx=0x7f2d5400d5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4776392427a8cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:24.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.558+0000 7f2d4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:52:24.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.559+0000 7f2d4dffb640 1 -- 172.21.15.139:0/3601137384 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54012420 con 0x5616d6df9980 2024-09-16T01:52:24.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.560+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3601137384 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d600c0ad0 con 0x5616d6df9980 2024-09-16T01:52:24.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.560+0000 7f2d4dffb640 1 -- 172.21.15.139:0/3601137384 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f2d540125c0 con 0x5616d6df9980 2024-09-16T01:52:24.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.562+0000 7f2d4dffb640 1 -- 172.21.15.139:0/3601137384 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f2d54021050 con 0x5616d6df9980 2024-09-16T01:52:24.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.565+0000 7f2d6ea45740 10 client.26203 fetch_fsmap learned FSMap version 61 2024-09-16T01:52:24.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.565+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3601137384 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5616d6b37960 con 0x5616d6df9980 2024-09-16T01:52:24.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.567+0000 7f2d4dffb640 1 -- 172.21.15.139:0/3601137384 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d600c0ad0 con 0x5616d6df9980 2024-09-16T01:52:24.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.567+0000 7f2d4dffb640 1 -- 172.21.15.139:0/3601137384 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 61) ==== 8438+0+0 (secure 0 0 0) 0x7f2d5401c030 con 0x5616d6df9980 2024-09-16T01:52:24.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.569+0000 7f2d6ea45740 10 client.26203 fetch_fsmap finished waiting for FSMap version 61 2024-09-16T01:52:24.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.569+0000 7f2d6ea45740 10 client.26203 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:52:24.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.569+0000 7f2d6ea45740 20 client.26203 populate_metadata read hostname 'smithi139' 2024-09-16T01:52:24.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.569+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/3601137384 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5616d6e13cb0 0x5616d6e160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.569+0000 7f2d6ea45740 4 client.26203 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:52:24.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.569+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3601137384 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616d6d6bb90 con 0x5616d6e13cb0 2024-09-16T01:52:24.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.570+0000 7f2d6a408640 1 --2- 172.21.15.139:0/3601137384 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5616d6e13cb0 0x5616d6e160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:24.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.573+0000 7f2d6a408640 1 --2- 172.21.15.139:0/3601137384 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5616d6e13cb0 0x5616d6e160a0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d5d5ddda407d713 server_cookie=1d8bf33627e99137 in_seq=0 out_seq=0 2024-09-16T01:52:24.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.573+0000 7f2d4dffb640 10 client.26203 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:24.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.575+0000 7f2d4dffb640 1 -- 172.21.15.139:0/3601137384 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616d6d6bb90 con 0x5616d6e13cb0 2024-09-16T01:52:24.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.575+0000 7f2d4dffb640 10 client.26203 handle_command_reply: tid=0 2024-09-16T01:52:24.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.576+0000 7f2d6ea45740 1 client.26203 shutdown 2024-09-16T01:52:24.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.577+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3601137384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9980 msgr2=0x5616d6e016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:24.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.577+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/3601137384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9980 0x5616d6e016c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400bfc0 tx=0x7f2d5400d5f0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.577+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3601137384 shutdown_connections 2024-09-16T01:52:24.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.577+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/3601137384 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5616d6e13cb0 0x5616d6e160a0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.577+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/3601137384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6e04240 0x5616d6e01c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.577+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/3601137384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9980 0x5616d6e016c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.577+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/3601137384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6df9060 0x5616d6e01180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.577+0000 7f2d4dffb640 0 client.26203 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:24.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.577+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3601137384 >> 172.21.15.139:0/3601137384 conn(0x5616d6df55f0 msgr2=0x5616d6dfb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:24.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.583+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3601137384 shutdown_connections 2024-09-16T01:52:24.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.583+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3601137384 wait complete. 2024-09-16T01:52:24.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.585+0000 7f2d6ea45740 20 client.26203 trim_cache size 0 max 16384 2024-09-16T01:52:24.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.619+0000 7f2d6ea45740 1 Processor -- start 2024-09-16T01:52:24.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.619+0000 7f2d6ea45740 1 -- start start 2024-09-16T01:52:24.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6ea45740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9060 0x5616d6dfb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6ea45740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6df9980 0x5616d6e1b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6ea45740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6e04240 0x5616d6e1be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6ea45740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d480102d0 con 0x5616d6df9980 2024-09-16T01:52:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6ea45740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d48012080 con 0x5616d6e04240 2024-09-16T01:52:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6ea45740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2d48010a60 con 0x5616d6df9060 2024-09-16T01:52:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6ac09640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6e04240 0x5616d6e1be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6ac09640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6e04240 0x5616d6e1be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:42452/0 (socket says 172.21.15.139:42452) 2024-09-16T01:52:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6ac09640 1 -- 172.21.15.139:0/560614600 learned_addr learned my addr 172.21.15.139:0/560614600 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6a408640 1 --2- 172.21.15.139:0/560614600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9060 0x5616d6dfb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6ac09640 1 -- 172.21.15.139:0/560614600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9060 msgr2=0x5616d6dfb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6ac09640 1 --2- 172.21.15.139:0/560614600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9060 0x5616d6dfb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6ac09640 1 -- 172.21.15.139:0/560614600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6df9980 msgr2=0x5616d6e1b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:24.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6ac09640 1 --2- 172.21.15.139:0/560614600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6df9980 0x5616d6e1b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.620+0000 7f2d6ac09640 1 -- 172.21.15.139:0/560614600 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616d6e0fe30 con 0x5616d6e04240 2024-09-16T01:52:24.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.623+0000 7f2d6ac09640 1 --2- 172.21.15.139:0/560614600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6e04240 0x5616d6e1be90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c017810 tx=0x7f2d5c015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92ec27cb50a30593 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:24.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.624+0000 7f2d4ffff640 1 -- 172.21.15.139:0/560614600 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c023410 con 0x5616d6e04240 2024-09-16T01:52:24.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.624+0000 7f2d4ffff640 1 -- 172.21.15.139:0/560614600 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d5c028070 con 0x5616d6e04240 2024-09-16T01:52:24.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.624+0000 7f2d4ffff640 1 -- 172.21.15.139:0/560614600 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c021920 con 0x5616d6e04240 2024-09-16T01:52:24.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.625+0000 7f2d6ea45740 1 -- 172.21.15.139:0/560614600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6e04240 msgr2=0x5616d6e1be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:24.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.625+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/560614600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6e04240 0x5616d6e1be90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c017810 tx=0x7f2d5c015000 comp rx=0 tx=0).stop 2024-09-16T01:52:24.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.627+0000 7f2d6ea45740 1 -- 172.21.15.139:0/560614600 shutdown_connections 2024-09-16T01:52:24.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.627+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/560614600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6e04240 0x5616d6e1be90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.627+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/560614600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6df9980 0x5616d6e1b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.627+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/560614600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9060 0x5616d6dfb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.627+0000 7f2d6ea45740 1 -- 172.21.15.139:0/560614600 >> 172.21.15.139:0/560614600 conn(0x5616d6df55f0 msgr2=0x5616d6dfab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:24.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.631+0000 7f2d6ea45740 1 -- 172.21.15.139:0/560614600 shutdown_connections 2024-09-16T01:52:24.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.631+0000 7f2d6ea45740 1 -- 172.21.15.139:0/560614600 wait complete. 2024-09-16T01:52:24.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.631+0000 7f2d6ea45740 1 -- start start 2024-09-16T01:52:24.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.631+0000 7f2d6ea45740 1 Processor -- start 2024-09-16T01:52:24.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.632+0000 7f2d6ea45740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6df9060 0x5616d6e0c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.632+0000 7f2d6ea45740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9980 0x5616d6e08830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.632+0000 7f2d6a408640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6df9060 0x5616d6e0c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:24.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.632+0000 7f2d6a408640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6df9060 0x5616d6e0c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:42458/0 (socket says 172.21.15.139:42458) 2024-09-16T01:52:24.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.632+0000 7f2d6ea45740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6e04240 0x5616d6e08d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.632+0000 7f2d6a408640 1 -- 172.21.15.139:0/3660674399 learned_addr learned my addr 172.21.15.139:0/3660674399 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:24.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.633+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3660674399 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d480118f0 con 0x5616d6e04240 2024-09-16T01:52:24.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.633+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3660674399 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d48011160 con 0x5616d6df9060 2024-09-16T01:52:24.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.633+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3660674399 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4800e2d0 con 0x5616d6df9980 2024-09-16T01:52:24.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.633+0000 7f2d6ea45740 10 client.0 _ll_register_callbacks cb 0x7f2d5400bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:52:24.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.634+0000 7f2d69c07640 1 --2- 172.21.15.139:0/3660674399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9980 0x5616d6e08830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:24.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.636+0000 7f2d6a408640 1 -- 172.21.15.139:0/3660674399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9980 msgr2=0x5616d6e08830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:24.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.636+0000 7f2d6a408640 1 --2- 172.21.15.139:0/3660674399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9980 0x5616d6e08830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.636+0000 7f2d6a408640 1 -- 172.21.15.139:0/3660674399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6e04240 msgr2=0x5616d6e08d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:24.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.636+0000 7f2d6a408640 1 --2- 172.21.15.139:0/3660674399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6e04240 0x5616d6e08d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:24.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.636+0000 7f2d6a408640 1 -- 172.21.15.139:0/3660674399 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616d6e09360 con 0x5616d6df9060 2024-09-16T01:52:24.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.636+0000 7f2d6a408640 1 --2- 172.21.15.139:0/3660674399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6df9060 0x5616d6e0c4f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2d60032a80 tx=0x7f2d6000a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59de451a379debf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:24.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.636+0000 7f2d4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:52:24.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.636+0000 7f2d4dffb640 1 -- 172.21.15.139:0/3660674399 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60049040 con 0x5616d6df9060 2024-09-16T01:52:24.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.637+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3660674399 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d5c028070 con 0x5616d6df9060 2024-09-16T01:52:24.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.637+0000 7f2d4dffb640 1 -- 172.21.15.139:0/3660674399 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f2d600baa20 con 0x5616d6df9060 2024-09-16T01:52:24.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.638+0000 7f2d4dffb640 1 -- 172.21.15.139:0/3660674399 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f2d600a2050 con 0x5616d6df9060 2024-09-16T01:52:24.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.639+0000 7f2d6ea45740 10 client.26234 fetch_fsmap learned FSMap version 61 2024-09-16T01:52:24.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.639+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3660674399 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5616d6e7f750 con 0x5616d6df9060 2024-09-16T01:52:24.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.640+0000 7f2d4dffb640 1 -- 172.21.15.139:0/3660674399 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d5c028070 con 0x5616d6df9060 2024-09-16T01:52:24.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.640+0000 7f2d4dffb640 1 -- 172.21.15.139:0/3660674399 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 61) ==== 8438+0+0 (secure 0 0 0) 0x7f2d6000b040 con 0x5616d6df9060 2024-09-16T01:52:24.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.640+0000 7f2d6ea45740 10 client.26234 fetch_fsmap finished waiting for FSMap version 61 2024-09-16T01:52:24.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.640+0000 7f2d6ea45740 10 client.26234 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:52:24.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.640+0000 7f2d6ea45740 20 client.26234 populate_metadata read hostname 'smithi139' 2024-09-16T01:52:24.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.640+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/3660674399 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5616d6e7fed0 0x5616d6e82310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:24.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.640+0000 7f2d6ea45740 4 client.26234 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d1c43de6", "release": true, "await": true}] 2024-09-16T01:52:24.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.640+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3660674399 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d1c43de6", "release": true, "await": true}) -- 0x5616d6d660a0 con 0x5616d6e7fed0 2024-09-16T01:52:24.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.641+0000 7f2d69c07640 1 --2- 172.21.15.139:0/3660674399 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5616d6e7fed0 0x5616d6e82310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:24.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.653+0000 7f2d69c07640 1 --2- 172.21.15.139:0/3660674399 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5616d6e7fed0 0x5616d6e82310 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a7de35813bb22db server_cookie=25ae7a032cdf77d1 in_seq=0 out_seq=0 2024-09-16T01:52:24.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:24.653+0000 7f2d4dffb640 10 client.26234 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:25.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:25 smithi139 ceph-mon[31175]: pgmap v378: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 77 KiB/s wr, 35 op/s 2024-09-16T01:52:25.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:25 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:25.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:25 smithi176 ceph-mon[35324]: pgmap v378: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 77 KiB/s wr, 35 op/s 2024-09-16T01:52:25.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:25 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:25.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:25 smithi164 ceph-mon[35121]: pgmap v378: 129 pgs: 21 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 50 active+clean; 2.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 77 KiB/s wr, 35 op/s 2024-09-16T01:52:25.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:25 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:25.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.914+0000 7f2d4dffb640 1 -- 172.21.15.139:0/3660674399 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5616d6d660a0 con 0x5616d6e7fed0 2024-09-16T01:52:25.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.914+0000 7f2d4dffb640 10 client.26234 handle_command_reply: tid=0 2024-09-16T01:52:25.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.915+0000 7f2d6ea45740 1 client.26234 shutdown 2024-09-16T01:52:25.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.915+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3660674399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6df9060 msgr2=0x5616d6e0c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:25.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.915+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/3660674399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6df9060 0x5616d6e0c4f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2d60032a80 tx=0x7f2d6000a370 comp rx=0 tx=0).stop 2024-09-16T01:52:25.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.915+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3660674399 shutdown_connections 2024-09-16T01:52:25.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.915+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/3660674399 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5616d6e7fed0 0x5616d6e82310 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:25.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.915+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/3660674399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616d6e04240 0x5616d6e08d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:25.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.915+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/3660674399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5616d6df9980 0x5616d6e08830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:25.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.915+0000 7f2d6ea45740 1 --2- 172.21.15.139:0/3660674399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d6df9060 0x5616d6e0c4f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:25.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.915+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3660674399 >> 172.21.15.139:0/3660674399 conn(0x5616d6df55f0 msgr2=0x5616d6e0f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:25.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.915+0000 7f2d4dffb640 0 client.26234 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:25.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.915+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3660674399 shutdown_connections 2024-09-16T01:52:25.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.915+0000 7f2d6ea45740 1 -- 172.21.15.139:0/3660674399 wait complete. 2024-09-16T01:52:25.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.916+0000 7f2d6ea45740 20 client.26234 trim_cache size 0 max 16384 2024-09-16T01:52:25.920 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:52:25.920 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 61, 2024-09-16T01:52:25.921 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:52:25.921 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 44, 2024-09-16T01:52:25.921 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:52:25.921 INFO:teuthology.orchestra.run.smithi139.stdout: "d1c43de6": { 2024-09-16T01:52:25.921 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 44, 2024-09-16T01:52:25.921 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:52:25.921 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:52:25.921 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:52:25.921 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:52:25.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:52:25.922 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:52:25.922 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 141.8, 2024-09-16T01:52:25.922 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:52:25.922 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:52:25.922 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:52:25.922 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:52:25.922 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:52:25.922 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:52:25.922 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:52:25.922 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:52:25.922 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:52:25.922 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:52:25.923 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:52:25.923 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:52:25.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.917+0000 7f2d6c66c640 1 -- 172.21.15.139:0/2639924332 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2d500a35c0 msgr2=0x7f2d500a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:25.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.917+0000 7f2d6c66c640 1 --2- 172.21.15.139:0/2639924332 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2d500a35c0 0x7f2d500a5a80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x5616d6df6970 tx=0x7f2d5c009450 comp rx=0 tx=0).stop 2024-09-16T01:52:25.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.917+0000 7f2d6c66c640 1 -- 172.21.15.139:0/2639924332 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d64100480 msgr2=0x7f2d640739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:25.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.917+0000 7f2d6c66c640 1 --2- 172.21.15.139:0/2639924332 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d64100480 0x7f2d640739a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d60006010 tx=0x7f2d60003c00 comp rx=0 tx=0).stop 2024-09-16T01:52:25.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.917+0000 7f2d6c66c640 1 -- 172.21.15.139:0/2639924332 shutdown_connections 2024-09-16T01:52:25.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.917+0000 7f2d6c66c640 1 --2- 172.21.15.139:0/2639924332 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2d500a35c0 0x7f2d500a5a80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:25.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.917+0000 7f2d6c66c640 1 --2- 172.21.15.139:0/2639924332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d641055b0 0x7f2d6406cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:25.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.917+0000 7f2d6c66c640 1 --2- 172.21.15.139:0/2639924332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d64100e30 0x7f2d64073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:25.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.917+0000 7f2d6c66c640 1 --2- 172.21.15.139:0/2639924332 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d64100480 0x7f2d640739a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:25.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.917+0000 7f2d6c66c640 1 -- 172.21.15.139:0/2639924332 >> 172.21.15.139:0/2639924332 conn(0x7f2d64077210 msgr2=0x7f2d641036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:25.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.919+0000 7f2d6c66c640 1 -- 172.21.15.139:0/2639924332 shutdown_connections 2024-09-16T01:52:25.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:52:25.919+0000 7f2d6c66c640 1 -- 172.21.15.139:0/2639924332 wait complete. 2024-09-16T01:52:27.018 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd1c43de6', total seconds elapsed: 27.9 2024-09-16T01:52:27.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:27 smithi139 ceph-mon[31175]: pgmap v379: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 68 KiB/s wr, 29 op/s 2024-09-16T01:52:27.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:27 smithi176 ceph-mon[35324]: pgmap v379: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 68 KiB/s wr, 29 op/s 2024-09-16T01:52:27.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:27 smithi164 ceph-mon[35121]: pgmap v379: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 68 KiB/s wr, 29 op/s 2024-09-16T01:52:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.718+0000 7f3571aa6640 1 -- 172.21.15.139:0/3670651158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f356c1091f0 msgr2=0x7f356c109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.718+0000 7f3571aa6640 1 --2- 172.21.15.139:0/3670651158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f356c1091f0 0x7f356c109630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3554008570 tx=0x7f3554029f80 comp rx=0 tx=0).stop 2024-09-16T01:52:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.719+0000 7f3571aa6640 1 -- 172.21.15.139:0/3670651158 shutdown_connections 2024-09-16T01:52:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.719+0000 7f3571aa6640 1 --2- 172.21.15.139:0/3670651158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f356c109b70 0x7f356c07b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.719+0000 7f3571aa6640 1 --2- 172.21.15.139:0/3670651158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f356c1091f0 0x7f356c109630 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.719+0000 7f3571aa6640 1 --2- 172.21.15.139:0/3670651158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f356c10d610 0x7f356c10d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.719+0000 7f3571aa6640 1 -- 172.21.15.139:0/3670651158 >> 172.21.15.139:0/3670651158 conn(0x7f356c06ce00 msgr2=0x7f356c06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.722+0000 7f3571aa6640 1 -- 172.21.15.139:0/3670651158 shutdown_connections 2024-09-16T01:52:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.726+0000 7f3571aa6640 1 -- 172.21.15.139:0/3670651158 wait complete. 2024-09-16T01:52:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.726+0000 7f3571aa6640 1 Processor -- start 2024-09-16T01:52:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.727+0000 7f3571aa6640 1 -- start start 2024-09-16T01:52:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.727+0000 7f3571aa6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f356c1091f0 0x7f356c07ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.727+0000 7f3571aa6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f356c109b70 0x7f356c075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.727+0000 7f3571aa6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f356c10d610 0x7f356c076110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.727+0000 7f3571aa6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f356c07e690 con 0x7f356c1091f0 2024-09-16T01:52:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.727+0000 7f3571aa6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f356c07e510 con 0x7f356c109b70 2024-09-16T01:52:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.727+0000 7f3571aa6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f356c07e810 con 0x7f356c10d610 2024-09-16T01:52:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.732+0000 7f356affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f356c109b70 0x7f356c075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:27.735 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.733+0000 7f356bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f356c10d610 0x7f356c076110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:27.735 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.733+0000 7f356affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f356c109b70 0x7f356c075bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:42472/0 (socket says 172.21.15.139:42472) 2024-09-16T01:52:27.735 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.733+0000 7f356affd640 1 -- 172.21.15.139:0/4208920444 learned_addr learned my addr 172.21.15.139:0/4208920444 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.734+0000 7f356affd640 1 -- 172.21.15.139:0/4208920444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f356c10d610 msgr2=0x7f356c076110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.735+0000 7f356affd640 1 --2- 172.21.15.139:0/4208920444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f356c10d610 0x7f356c076110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.735+0000 7f356affd640 1 -- 172.21.15.139:0/4208920444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f356c1091f0 msgr2=0x7f356c07ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.735+0000 7f356affd640 1 --2- 172.21.15.139:0/4208920444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f356c1091f0 0x7f356c07ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.735+0000 7f356affd640 1 -- 172.21.15.139:0/4208920444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356c076650 con 0x7f356c109b70 2024-09-16T01:52:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.743+0000 7f356affd640 1 --2- 172.21.15.139:0/4208920444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f356c109b70 0x7f356c075bd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3554004cd0 tx=0x7f3554004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.744+0000 7f3568ff9640 1 -- 172.21.15.139:0/4208920444 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554005130 con 0x7f356c109b70 2024-09-16T01:52:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.744+0000 7f3568ff9640 1 -- 172.21.15.139:0/4208920444 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f355400a5b0 con 0x7f356c109b70 2024-09-16T01:52:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.744+0000 7f3568ff9640 1 -- 172.21.15.139:0/4208920444 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554032af0 con 0x7f356c109b70 2024-09-16T01:52:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.746+0000 7f3571aa6640 1 -- 172.21.15.139:0/4208920444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356c0768e0 con 0x7f356c109b70 2024-09-16T01:52:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.746+0000 7f3571aa6640 1 -- 172.21.15.139:0/4208920444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f356c1cd190 con 0x7f356c109b70 2024-09-16T01:52:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.753+0000 7f3568ff9640 1 -- 172.21.15.139:0/4208920444 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3554032c90 con 0x7f356c109b70 2024-09-16T01:52:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.754+0000 7f3568ff9640 1 --2- 172.21.15.139:0/4208920444 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f35580a3640 0x7f35580a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.754+0000 7f3568ff9640 1 -- 172.21.15.139:0/4208920444 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f355400a1f0 con 0x7f356c109b70 2024-09-16T01:52:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.755+0000 7f3574839740 1 Processor -- start 2024-09-16T01:52:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.756+0000 7f356b7fe640 1 --2- 172.21.15.139:0/4208920444 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f35580a3640 0x7f35580a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.759+0000 7f3574839740 1 -- start start 2024-09-16T01:52:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.759+0000 7f3574839740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212daaf0 0x55e1212daed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.759+0000 7f3574839740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212db410 0x55e1212e5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.759+0000 7f3574839740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.759+0000 7f3574839740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e12123f2f0 con 0x55e1212db410 2024-09-16T01:52:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.759+0000 7f3574839740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e1211e7270 con 0x55e1212e5cb0 2024-09-16T01:52:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.759+0000 7f3574839740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e12123b2b0 con 0x55e1212daaf0 2024-09-16T01:52:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.759+0000 7f356affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212db410 0x55e1212e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.759+0000 7f356affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212db410 0x55e1212e5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49924/0 (socket says 172.21.15.139:49924) 2024-09-16T01:52:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.759+0000 7f356affd640 1 -- 172.21.15.139:0/3912606609 learned_addr learned my addr 172.21.15.139:0/3912606609 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.760+0000 7f356affd640 1 -- 172.21.15.139:0/3912606609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212daaf0 msgr2=0x55e1212daed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:52:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.760+0000 7f356affd640 1 --2- 172.21.15.139:0/3912606609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212daaf0 0x55e1212daed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.760+0000 7f356affd640 1 -- 172.21.15.139:0/3912606609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 msgr2=0x55e1212e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.760+0000 7f356affd640 1 --2- 172.21.15.139:0/3912606609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.760+0000 7f356affd640 1 -- 172.21.15.139:0/3912606609 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1211e6340 con 0x55e1212db410 2024-09-16T01:52:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.760+0000 7f356affd640 1 --2- 172.21.15.139:0/3912606609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212db410 0x55e1212e5770 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f35540bc8b0 tx=0x7f35540bc7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b667561d9af7216 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.761+0000 7f3561ffb640 1 -- 172.21.15.139:0/3912606609 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35540d4020 con 0x55e1212db410 2024-09-16T01:52:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.761+0000 7f3561ffb640 1 -- 172.21.15.139:0/3912606609 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35540bc920 con 0x55e1212db410 2024-09-16T01:52:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.761+0000 7f3561ffb640 1 -- 172.21.15.139:0/3912606609 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35540c5100 con 0x55e1212db410 2024-09-16T01:52:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.763+0000 7f3574839740 1 -- 172.21.15.139:0/3912606609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212db410 msgr2=0x55e1212e5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.763+0000 7f3574839740 1 --2- 172.21.15.139:0/3912606609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212db410 0x55e1212e5770 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f35540bc8b0 tx=0x7f35540bc7b0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.764+0000 7f356b7fe640 1 --2- 172.21.15.139:0/4208920444 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f35580a3640 0x7f35580a5b00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3564011b00 tx=0x7f3564011a30 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.766+0000 7f3574839740 1 -- 172.21.15.139:0/3912606609 shutdown_connections 2024-09-16T01:52:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.766+0000 7f3574839740 1 --2- 172.21.15.139:0/3912606609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.766+0000 7f3574839740 1 --2- 172.21.15.139:0/3912606609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212db410 0x55e1212e5770 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.766+0000 7f3574839740 1 --2- 172.21.15.139:0/3912606609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212daaf0 0x55e1212daed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.766+0000 7f3574839740 1 -- 172.21.15.139:0/3912606609 >> 172.21.15.139:0/3912606609 conn(0x55e1212d7060 msgr2=0x55e1212d7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.770+0000 7f3574839740 1 -- 172.21.15.139:0/3912606609 shutdown_connections 2024-09-16T01:52:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.772+0000 7f3574839740 1 -- 172.21.15.139:0/3912606609 wait complete. 2024-09-16T01:52:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.772+0000 7f3574839740 1 -- start start 2024-09-16T01:52:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.773+0000 7f3574839740 1 Processor -- start 2024-09-16T01:52:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.773+0000 7f3574839740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212daaf0 0x55e1212e2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.773+0000 7f3574839740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212db410 0x55e1212e3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.773+0000 7f3574839740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.773+0000 7f3574839740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e12123f2f0 con 0x55e1212daaf0 2024-09-16T01:52:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.773+0000 7f3574839740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e1211c8290 con 0x55e1212e5cb0 2024-09-16T01:52:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.773+0000 7f3574839740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e1212e8640 con 0x55e1212db410 2024-09-16T01:52:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.773+0000 7f3574839740 10 client.0 _ll_register_callbacks cb 0x55e1211f9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:52:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.773+0000 7f356bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.774+0000 7f356bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:42478/0 (socket says 172.21.15.139:42478) 2024-09-16T01:52:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.774+0000 7f356bfff640 1 -- 172.21.15.139:0/844906200 learned_addr learned my addr 172.21.15.139:0/844906200 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.774+0000 7f356bfff640 1 -- 172.21.15.139:0/844906200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212db410 msgr2=0x55e1212e3130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:52:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.774+0000 7f356bfff640 1 --2- 172.21.15.139:0/844906200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212db410 0x55e1212e3130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.774+0000 7f356bfff640 1 -- 172.21.15.139:0/844906200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212daaf0 msgr2=0x55e1212e2bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:52:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.774+0000 7f356bfff640 1 --2- 172.21.15.139:0/844906200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212daaf0 0x55e1212e2bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.774+0000 7f356bfff640 1 -- 172.21.15.139:0/844906200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e120f52210 con 0x55e1212e5cb0 2024-09-16T01:52:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.774+0000 7f356bfff640 1 --2- 172.21.15.139:0/844906200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e3670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f355c009620 tx=0x7f355c009ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e734d5c9deafa80a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.776+0000 7f354affd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:52:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.780+0000 7f354affd640 1 -- 172.21.15.139:0/844906200 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c016470 con 0x55e1212e5cb0 2024-09-16T01:52:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.780+0000 7f354affd640 1 -- 172.21.15.139:0/844906200 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f355c016610 con 0x55e1212e5cb0 2024-09-16T01:52:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.780+0000 7f3574839740 1 -- 172.21.15.139:0/844906200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35540bc920 con 0x55e1212e5cb0 2024-09-16T01:52:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.782+0000 7f354affd640 1 -- 172.21.15.139:0/844906200 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f355c016810 con 0x55e1212e5cb0 2024-09-16T01:52:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.785+0000 7f3574839740 10 client.26243 fetch_fsmap learned FSMap version 61 2024-09-16T01:52:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.785+0000 7f3574839740 1 -- 172.21.15.139:0/844906200 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e12101e620 con 0x55e1212e5cb0 2024-09-16T01:52:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.789+0000 7f354affd640 1 -- 172.21.15.139:0/844906200 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35540bc920 con 0x55e1212e5cb0 2024-09-16T01:52:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.789+0000 7f354affd640 1 -- 172.21.15.139:0/844906200 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 61) ==== 8438+0+0 (secure 0 0 0) 0x7f355c00aca0 con 0x55e1212e5cb0 2024-09-16T01:52:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.792+0000 7f3574839740 10 client.26243 fetch_fsmap finished waiting for FSMap version 61 2024-09-16T01:52:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.792+0000 7f3574839740 10 client.26243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:52:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.792+0000 7f3574839740 20 client.26243 populate_metadata read hostname 'smithi139' 2024-09-16T01:52:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.792+0000 7f3574839740 1 --2- 172.21.15.139:0/844906200 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e1212f5980 0x55e1212f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.792+0000 7f3574839740 4 client.26243 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:52:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.792+0000 7f3574839740 1 -- 172.21.15.139:0/844906200 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e12124ca20 con 0x55e1212f5980 2024-09-16T01:52:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.794+0000 7f356b7fe640 1 --2- 172.21.15.139:0/844906200 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e1212f5980 0x55e1212f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.798+0000 7f356b7fe640 1 --2- 172.21.15.139:0/844906200 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e1212f5980 0x55e1212f7e00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab3ab20c882ce96 server_cookie=350b3bb414f390 in_seq=0 out_seq=0 2024-09-16T01:52:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.799+0000 7f354affd640 10 client.26243 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.803+0000 7f354affd640 1 -- 172.21.15.139:0/844906200 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e12124ca20 con 0x55e1212f5980 2024-09-16T01:52:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.803+0000 7f354affd640 10 client.26243 handle_command_reply: tid=0 2024-09-16T01:52:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.803+0000 7f3574839740 1 client.26243 shutdown 2024-09-16T01:52:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.811+0000 7f3574839740 1 -- 172.21.15.139:0/844906200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 msgr2=0x55e1212e3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.811+0000 7f3574839740 1 --2- 172.21.15.139:0/844906200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e3670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f355c009620 tx=0x7f355c009ae0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.812+0000 7f3574839740 1 -- 172.21.15.139:0/844906200 shutdown_connections 2024-09-16T01:52:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.812+0000 7f3574839740 1 --2- 172.21.15.139:0/844906200 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e1212f5980 0x55e1212f7e00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.812+0000 7f3574839740 1 --2- 172.21.15.139:0/844906200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e3670 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.812+0000 7f3574839740 1 --2- 172.21.15.139:0/844906200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212db410 0x55e1212e3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.812+0000 7f3574839740 1 --2- 172.21.15.139:0/844906200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212daaf0 0x55e1212e2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.812+0000 7f3574839740 1 -- 172.21.15.139:0/844906200 >> 172.21.15.139:0/844906200 conn(0x55e1212d7060 msgr2=0x55e1212dcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.813+0000 7f354affd640 0 client.26243 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.818+0000 7f3574839740 1 -- 172.21.15.139:0/844906200 shutdown_connections 2024-09-16T01:52:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.820+0000 7f3574839740 1 -- 172.21.15.139:0/844906200 wait complete. 2024-09-16T01:52:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.820+0000 7f3574839740 20 client.26243 trim_cache size 0 max 16384 2024-09-16T01:52:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.850+0000 7f3574839740 1 Processor -- start 2024-09-16T01:52:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.850+0000 7f3574839740 1 -- start start 2024-09-16T01:52:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.850+0000 7f3574839740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212daaf0 0x55e1212e34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.851+0000 7f3574839740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212db410 0x55e1212dcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.851+0000 7f3574839740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212e5cb0 0x55e1212fd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.851+0000 7f3574839740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f354c0102d0 con 0x55e1212daaf0 2024-09-16T01:52:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.851+0000 7f3574839740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f354c012080 con 0x55e1212db410 2024-09-16T01:52:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.851+0000 7f3574839740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f354c010a60 con 0x55e1212e5cb0 2024-09-16T01:52:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.851+0000 7f356b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212daaf0 0x55e1212e34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.851+0000 7f356b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212daaf0 0x55e1212e34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49926/0 (socket says 172.21.15.139:49926) 2024-09-16T01:52:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.851+0000 7f356b7fe640 1 -- 172.21.15.139:0/3549171993 learned_addr learned my addr 172.21.15.139:0/3549171993 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.852+0000 7f356bfff640 1 --2- 172.21.15.139:0/3549171993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212e5cb0 0x55e1212fd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.852+0000 7f356affd640 1 --2- 172.21.15.139:0/3549171993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212db410 0x55e1212dcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.854+0000 7f356bfff640 1 -- 172.21.15.139:0/3549171993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212db410 msgr2=0x55e1212dcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.854+0000 7f356bfff640 1 --2- 172.21.15.139:0/3549171993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212db410 0x55e1212dcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.854+0000 7f356bfff640 1 -- 172.21.15.139:0/3549171993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212daaf0 msgr2=0x55e1212e34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.854+0000 7f356bfff640 1 --2- 172.21.15.139:0/3549171993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212daaf0 0x55e1212e34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.854+0000 7f356bfff640 1 -- 172.21.15.139:0/3549171993 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1212f8480 con 0x55e1212e5cb0 2024-09-16T01:52:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.854+0000 7f356bfff640 1 --2- 172.21.15.139:0/3549171993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212e5cb0 0x55e1212fd710 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f355c007460 tx=0x7f355c0059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78d3402386e478f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.857+0000 7f35617fa640 1 -- 172.21.15.139:0/3549171993 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c0076d0 con 0x55e1212e5cb0 2024-09-16T01:52:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.857+0000 7f35617fa640 1 -- 172.21.15.139:0/3549171993 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f355c01b070 con 0x55e1212e5cb0 2024-09-16T01:52:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.858+0000 7f3574839740 1 -- 172.21.15.139:0/3549171993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212e5cb0 msgr2=0x55e1212fd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.858+0000 7f3574839740 1 --2- 172.21.15.139:0/3549171993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212e5cb0 0x55e1212fd710 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f355c007460 tx=0x7f355c0059b0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.858+0000 7f3574839740 1 -- 172.21.15.139:0/3549171993 shutdown_connections 2024-09-16T01:52:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.858+0000 7f3574839740 1 --2- 172.21.15.139:0/3549171993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212e5cb0 0x55e1212fd710 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.858+0000 7f3574839740 1 --2- 172.21.15.139:0/3549171993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212db410 0x55e1212dcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.858+0000 7f3574839740 1 --2- 172.21.15.139:0/3549171993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212daaf0 0x55e1212e34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.858+0000 7f3574839740 1 -- 172.21.15.139:0/3549171993 >> 172.21.15.139:0/3549171993 conn(0x55e1212d7060 msgr2=0x55e1212dc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.862+0000 7f3574839740 1 -- 172.21.15.139:0/3549171993 shutdown_connections 2024-09-16T01:52:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.862+0000 7f3574839740 1 -- 172.21.15.139:0/3549171993 wait complete. 2024-09-16T01:52:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.863+0000 7f3574839740 1 -- start start 2024-09-16T01:52:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.863+0000 7f3574839740 1 Processor -- start 2024-09-16T01:52:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.864+0000 7f3574839740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212daaf0 0x55e1212ee110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.864+0000 7f3574839740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212db410 0x55e1212e9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.864+0000 7f3574839740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.864+0000 7f3574839740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f354c0118f0 con 0x55e1212daaf0 2024-09-16T01:52:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.864+0000 7f3574839740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f354c011160 con 0x55e1212e5cb0 2024-09-16T01:52:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.864+0000 7f3574839740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f354c00e2d0 con 0x55e1212db410 2024-09-16T01:52:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.864+0000 7f3574839740 10 client.0 _ll_register_callbacks cb 0x7f355c00b550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:52:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.864+0000 7f356bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.864+0000 7f356bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:42502/0 (socket says 172.21.15.139:42502) 2024-09-16T01:52:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.864+0000 7f356bfff640 1 -- 172.21.15.139:0/2639449277 learned_addr learned my addr 172.21.15.139:0/2639449277 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.865+0000 7f356bfff640 1 -- 172.21.15.139:0/2639449277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212db410 msgr2=0x55e1212e9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.865+0000 7f356bfff640 1 --2- 172.21.15.139:0/2639449277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212db410 0x55e1212e9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.865+0000 7f356bfff640 1 -- 172.21.15.139:0/2639449277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212daaf0 msgr2=0x55e1212ee110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.865+0000 7f356bfff640 1 --2- 172.21.15.139:0/2639449277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212daaf0 0x55e1212ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.865+0000 7f356bfff640 1 -- 172.21.15.139:0/2639449277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1212ea270 con 0x55e1212e5cb0 2024-09-16T01:52:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.867+0000 7f356bfff640 1 --2- 172.21.15.139:0/2639449277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e9a40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f355c007650 tx=0x7f355c005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6df1e2d28f008b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.868+0000 7f354affd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:52:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.868+0000 7f354affd640 1 -- 172.21.15.139:0/2639449277 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c016bb0 con 0x55e1212e5cb0 2024-09-16T01:52:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.868+0000 7f354affd640 1 -- 172.21.15.139:0/2639449277 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f355c0078e0 con 0x55e1212e5cb0 2024-09-16T01:52:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.869+0000 7f3574839740 1 -- 172.21.15.139:0/2639449277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f355c01b070 con 0x55e1212e5cb0 2024-09-16T01:52:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.874+0000 7f354affd640 1 -- 172.21.15.139:0/2639449277 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f355c007b00 con 0x55e1212e5cb0 2024-09-16T01:52:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.878+0000 7f3574839740 10 client.26249 fetch_fsmap learned FSMap version 61 2024-09-16T01:52:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.878+0000 7f3574839740 1 -- 172.21.15.139:0/2639449277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e121360820 con 0x55e1212e5cb0 2024-09-16T01:52:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.880+0000 7f354affd640 1 -- 172.21.15.139:0/2639449277 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f355c01b070 con 0x55e1212e5cb0 2024-09-16T01:52:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.880+0000 7f354affd640 1 -- 172.21.15.139:0/2639449277 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 61) ==== 8438+0+0 (secure 0 0 0) 0x7f355c0204e0 con 0x55e1212e5cb0 2024-09-16T01:52:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.881+0000 7f3574839740 10 client.26249 fetch_fsmap finished waiting for FSMap version 61 2024-09-16T01:52:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.881+0000 7f3574839740 10 client.26249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:52:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.881+0000 7f3574839740 20 client.26249 populate_metadata read hostname 'smithi139' 2024-09-16T01:52:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.881+0000 7f3574839740 1 --2- 172.21.15.139:0/2639449277 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e121360b20 0x55e121362fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.881+0000 7f3574839740 4 client.26249 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:52:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.881+0000 7f3574839740 1 -- 172.21.15.139:0/2639449277 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e121246d70 con 0x55e121360b20 2024-09-16T01:52:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.883+0000 7f356b7fe640 1 --2- 172.21.15.139:0/2639449277 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e121360b20 0x55e121362fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.891+0000 7f356b7fe640 1 --2- 172.21.15.139:0/2639449277 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e121360b20 0x55e121362fa0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=251ed1708015c565 server_cookie=988eea61bb424e5c in_seq=0 out_seq=0 2024-09-16T01:52:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:27.892+0000 7f354affd640 10 client.26249 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.406+0000 7f354affd640 1 -- 172.21.15.139:0/2639449277 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e121246d70 con 0x55e121360b20 2024-09-16T01:52:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.407+0000 7f354affd640 10 client.26249 handle_command_reply: tid=0 2024-09-16T01:52:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.407+0000 7f3574839740 1 client.26249 shutdown 2024-09-16T01:52:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.407+0000 7f3574839740 1 -- 172.21.15.139:0/2639449277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 msgr2=0x55e1212e9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.407+0000 7f3574839740 1 --2- 172.21.15.139:0/2639449277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e9a40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f355c007650 tx=0x7f355c005780 comp rx=0 tx=0).stop 2024-09-16T01:52:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.407+0000 7f3574839740 1 -- 172.21.15.139:0/2639449277 shutdown_connections 2024-09-16T01:52:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.407+0000 7f3574839740 1 --2- 172.21.15.139:0/2639449277 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e121360b20 0x55e121362fa0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.407+0000 7f3574839740 1 --2- 172.21.15.139:0/2639449277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1212e5cb0 0x55e1212e9a40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.407+0000 7f3574839740 1 --2- 172.21.15.139:0/2639449277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1212db410 0x55e1212e9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.407+0000 7f3574839740 1 --2- 172.21.15.139:0/2639449277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1212daaf0 0x55e1212ee110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.407+0000 7f3574839740 1 -- 172.21.15.139:0/2639449277 >> 172.21.15.139:0/2639449277 conn(0x55e1212d7060 msgr2=0x55e1212d61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.407+0000 7f354affd640 0 client.26249 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.407+0000 7f3574839740 1 -- 172.21.15.139:0/2639449277 shutdown_connections 2024-09-16T01:52:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.408+0000 7f3574839740 1 -- 172.21.15.139:0/2639449277 wait complete. 2024-09-16T01:52:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.408+0000 7f3574839740 20 client.26249 trim_cache size 0 max 16384 2024-09-16T01:52:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:52:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:52:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:52:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:52:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.409+0000 7f3571aa6640 1 -- 172.21.15.139:0/4208920444 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f35580a3640 msgr2=0x7f35580a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.409+0000 7f3571aa6640 1 --2- 172.21.15.139:0/4208920444 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f35580a3640 0x7f35580a5b00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3564011b00 tx=0x7f3564011a30 comp rx=0 tx=0).stop 2024-09-16T01:52:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.409+0000 7f3571aa6640 1 -- 172.21.15.139:0/4208920444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f356c109b70 msgr2=0x7f356c075bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.409+0000 7f3571aa6640 1 --2- 172.21.15.139:0/4208920444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f356c109b70 0x7f356c075bd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3554004cd0 tx=0x7f3554004d00 comp rx=0 tx=0).stop 2024-09-16T01:52:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.409+0000 7f3571aa6640 1 -- 172.21.15.139:0/4208920444 shutdown_connections 2024-09-16T01:52:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.409+0000 7f3571aa6640 1 --2- 172.21.15.139:0/4208920444 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f35580a3640 0x7f35580a5b00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.409+0000 7f3571aa6640 1 --2- 172.21.15.139:0/4208920444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f356c10d610 0x7f356c076110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.409+0000 7f3571aa6640 1 --2- 172.21.15.139:0/4208920444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f356c109b70 0x7f356c075bd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.409+0000 7f3571aa6640 1 --2- 172.21.15.139:0/4208920444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f356c1091f0 0x7f356c07ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.409+0000 7f3571aa6640 1 -- 172.21.15.139:0/4208920444 >> 172.21.15.139:0/4208920444 conn(0x7f356c06ce00 msgr2=0x7f356c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.409+0000 7f3571aa6640 1 -- 172.21.15.139:0/4208920444 shutdown_connections 2024-09-16T01:52:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:29.409+0000 7f3571aa6640 1 -- 172.21.15.139:0/4208920444 wait complete. 2024-09-16T01:52:29.426 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:29 smithi139 ceph-mon[31175]: pgmap v380: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 90 KiB/s wr, 21 op/s 2024-09-16T01:52:29.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:29 smithi176 ceph-mon[35324]: pgmap v380: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 90 KiB/s wr, 21 op/s 2024-09-16T01:52:29.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:29 smithi164 ceph-mon[35121]: pgmap v380: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 90 KiB/s wr, 21 op/s 2024-09-16T01:52:31.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:31 smithi139 ceph-mon[31175]: pgmap v381: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-09-16T01:52:31.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:31 smithi176 ceph-mon[35324]: pgmap v381: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-09-16T01:52:31.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:31 smithi164 ceph-mon[35121]: pgmap v381: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-09-16T01:52:33.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:33 smithi139 ceph-mon[31175]: pgmap v382: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 409 KiB/s wr, 59 op/s 2024-09-16T01:52:33.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:33 smithi176 ceph-mon[35324]: pgmap v382: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 409 KiB/s wr, 59 op/s 2024-09-16T01:52:33.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:33 smithi164 ceph-mon[35121]: pgmap v382: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 409 KiB/s wr, 59 op/s 2024-09-16T01:52:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:35 smithi139 ceph-mon[31175]: pgmap v383: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 419 KiB/s wr, 89 op/s 2024-09-16T01:52:35.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:35 smithi176 ceph-mon[35324]: pgmap v383: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 419 KiB/s wr, 89 op/s 2024-09-16T01:52:35.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:35 smithi164 ceph-mon[35121]: pgmap v383: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 419 KiB/s wr, 89 op/s 2024-09-16T01:52:37.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:37 smithi139 ceph-mon[31175]: pgmap v384: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 762 KiB/s wr, 119 op/s 2024-09-16T01:52:37.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:37 smithi176 ceph-mon[35324]: pgmap v384: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 762 KiB/s wr, 119 op/s 2024-09-16T01:52:37.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:37 smithi164 ceph-mon[35121]: pgmap v384: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 762 KiB/s wr, 119 op/s 2024-09-16T01:52:39.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:39 smithi139 ceph-mon[31175]: pgmap v385: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 780 KiB/s wr, 145 op/s 2024-09-16T01:52:39.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:39 smithi176 ceph-mon[35324]: pgmap v385: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 780 KiB/s wr, 145 op/s 2024-09-16T01:52:39.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:39 smithi164 ceph-mon[35121]: pgmap v385: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 780 KiB/s wr, 145 op/s 2024-09-16T01:52:40.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:40 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:40.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:40 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:40.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:40 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:41.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:41 smithi176 ceph-mon[35324]: pgmap v386: 129 pgs: 12 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 83 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-16T01:52:41.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:41 smithi164 ceph-mon[35121]: pgmap v386: 129 pgs: 12 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 83 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-16T01:52:41.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:41 smithi139 ceph-mon[31175]: pgmap v386: 129 pgs: 12 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 83 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-16T01:52:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.554+0000 7f5a50fa6640 1 -- 172.21.15.139:0/4169900101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4c1069e0 msgr2=0x7f5a4c106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.554+0000 7f5a50fa6640 1 --2- 172.21.15.139:0/4169900101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4c1069e0 0x7f5a4c106dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5a4001a910 tx=0x7f5a4003b450 comp rx=0 tx=0).stop 2024-09-16T01:52:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.560+0000 7f5a50fa6640 1 -- 172.21.15.139:0/4169900101 shutdown_connections 2024-09-16T01:52:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.560+0000 7f5a50fa6640 1 --2- 172.21.15.139:0/4169900101 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a4c101370 0x7f5a4c10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.560+0000 7f5a50fa6640 1 --2- 172.21.15.139:0/4169900101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a4c1009d0 0x7f5a4c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.560+0000 7f5a50fa6640 1 --2- 172.21.15.139:0/4169900101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4c1069e0 0x7f5a4c106dc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.560+0000 7f5a50fa6640 1 -- 172.21.15.139:0/4169900101 >> 172.21.15.139:0/4169900101 conn(0x7f5a4c0fc6b0 msgr2=0x7f5a4c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.565+0000 7f5a50fa6640 1 -- 172.21.15.139:0/4169900101 shutdown_connections 2024-09-16T01:52:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.566+0000 7f5a50fa6640 1 -- 172.21.15.139:0/4169900101 wait complete. 2024-09-16T01:52:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.567+0000 7f5a50fa6640 1 Processor -- start 2024-09-16T01:52:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.568+0000 7f5a50fa6640 1 -- start start 2024-09-16T01:52:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.568+0000 7f5a50fa6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4c1009d0 0x7f5a4c073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.568+0000 7f5a50fa6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a4c101370 0x7f5a4c06eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.568+0000 7f5a50fa6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a4c1069e0 0x7f5a4c06f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.568+0000 7f5a50fa6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c06d540 con 0x7f5a4c1069e0 2024-09-16T01:52:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.568+0000 7f5a50fa6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c06d3c0 con 0x7f5a4c1009d0 2024-09-16T01:52:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.568+0000 7f5a50fa6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c06d6c0 con 0x7f5a4c101370 2024-09-16T01:52:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.570+0000 7f5a4a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4c1009d0 0x7f5a4c073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.570+0000 7f5a4a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4c1009d0 0x7f5a4c073ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:42982/0 (socket says 172.21.15.139:42982) 2024-09-16T01:52:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.570+0000 7f5a4a59c640 1 -- 172.21.15.139:0/627846954 learned_addr learned my addr 172.21.15.139:0/627846954 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.570+0000 7f5a49d9b640 1 --2- 172.21.15.139:0/627846954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a4c101370 0x7f5a4c06eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.572+0000 7f5a4a59c640 1 -- 172.21.15.139:0/627846954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a4c101370 msgr2=0x7f5a4c06eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.572+0000 7f5a4a59c640 1 --2- 172.21.15.139:0/627846954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a4c101370 0x7f5a4c06eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.572+0000 7f5a4a59c640 1 -- 172.21.15.139:0/627846954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a4c1069e0 msgr2=0x7f5a4c06f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.572+0000 7f5a4a59c640 1 --2- 172.21.15.139:0/627846954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a4c1069e0 0x7f5a4c06f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.572+0000 7f5a4a59c640 1 -- 172.21.15.139:0/627846954 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c06f710 con 0x7f5a4c1009d0 2024-09-16T01:52:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.573+0000 7f5a49d9b640 1 --2- 172.21.15.139:0/627846954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a4c101370 0x7f5a4c06eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:52:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.576+0000 7f5a4a59c640 1 --2- 172.21.15.139:0/627846954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4c1009d0 0x7f5a4c073ad0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5a40005ab0 tx=0x7f5a40004bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.579+0000 7f5a3b7fe640 1 -- 172.21.15.139:0/627846954 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4004f020 con 0x7f5a4c1009d0 2024-09-16T01:52:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.579+0000 7f5a50fa6640 1 -- 172.21.15.139:0/627846954 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4c10e490 con 0x7f5a4c1009d0 2024-09-16T01:52:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.579+0000 7f5a50fa6640 1 -- 172.21.15.139:0/627846954 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4c10e980 con 0x7f5a4c1009d0 2024-09-16T01:52:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.579+0000 7f5a3b7fe640 1 -- 172.21.15.139:0/627846954 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a40004260 con 0x7f5a4c1009d0 2024-09-16T01:52:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.580+0000 7f5a3b7fe640 1 -- 172.21.15.139:0/627846954 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40046070 con 0x7f5a4c1009d0 2024-09-16T01:52:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.583+0000 7f5a53e22740 1 Processor -- start 2024-09-16T01:52:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.584+0000 7f5a53e22740 1 -- start start 2024-09-16T01:52:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.585+0000 7f5a53e22740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab3af0 0x556895ab3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.585+0000 7f5a53e22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895ab4410 0x556895abe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.585+0000 7f5a53e22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895abecb0 0x556895ac10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.585+0000 7f5a53e22740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556895a182f0 con 0x556895ab4410 2024-09-16T01:52:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.585+0000 7f5a53e22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5568959c0270 con 0x556895abecb0 2024-09-16T01:52:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.585+0000 7f5a53e22740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556895a142b0 con 0x556895ab3af0 2024-09-16T01:52:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.585+0000 7f5a49d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895abecb0 0x556895ac10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.585+0000 7f5a49d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895abecb0 0x556895ac10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:42988/0 (socket says 172.21.15.139:42988) 2024-09-16T01:52:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.585+0000 7f5a49d9b640 1 -- 172.21.15.139:0/2056517756 learned_addr learned my addr 172.21.15.139:0/2056517756 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.585+0000 7f5a49d9b640 1 -- 172.21.15.139:0/2056517756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab3af0 msgr2=0x556895ab3ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:52:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.585+0000 7f5a49d9b640 1 --2- 172.21.15.139:0/2056517756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab3af0 0x556895ab3ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.585+0000 7f5a49d9b640 1 -- 172.21.15.139:0/2056517756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895ab4410 msgr2=0x556895abe770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:52:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.585+0000 7f5a49d9b640 1 --2- 172.21.15.139:0/2056517756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895ab4410 0x556895abe770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.585+0000 7f5a49d9b640 1 -- 172.21.15.139:0/2056517756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568959bf340 con 0x556895abecb0 2024-09-16T01:52:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.589+0000 7f5a49d9b640 1 --2- 172.21.15.139:0/2056517756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895abecb0 0x556895ac10a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009de0 tx=0x7f5a3c00e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3a29740e069655d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.591+0000 7f5a3b7fe640 1 -- 172.21.15.139:0/627846954 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f5a40052020 con 0x7f5a4c1009d0 2024-09-16T01:52:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.592+0000 7f5a3b7fe640 1 --2- 172.21.15.139:0/627846954 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5a340a3640 0x7f5a340a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.592+0000 7f5a3b7fe640 1 -- 172.21.15.139:0/627846954 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f5a401014f0 con 0x7f5a4c1009d0 2024-09-16T01:52:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.593+0000 7f5a38ff9640 1 -- 172.21.15.139:0/2056517756 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c017070 con 0x556895abecb0 2024-09-16T01:52:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.593+0000 7f5a38ff9640 1 -- 172.21.15.139:0/2056517756 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c013950 con 0x556895abecb0 2024-09-16T01:52:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.593+0000 7f5a4ad9d640 1 --2- 172.21.15.139:0/627846954 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5a340a3640 0x7f5a340a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.593+0000 7f5a38ff9640 1 -- 172.21.15.139:0/2056517756 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c012ab0 con 0x556895abecb0 2024-09-16T01:52:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.594+0000 7f5a53e22740 1 -- 172.21.15.139:0/2056517756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895abecb0 msgr2=0x556895ac10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.594+0000 7f5a53e22740 1 --2- 172.21.15.139:0/2056517756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895abecb0 0x556895ac10a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009de0 tx=0x7f5a3c00e5a0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.594+0000 7f5a53e22740 1 -- 172.21.15.139:0/2056517756 shutdown_connections 2024-09-16T01:52:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.594+0000 7f5a53e22740 1 --2- 172.21.15.139:0/2056517756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895abecb0 0x556895ac10a0 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009de0 tx=0x7f5a3c00e5a0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.594+0000 7f5a53e22740 1 --2- 172.21.15.139:0/2056517756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895ab4410 0x556895abe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.594+0000 7f5a53e22740 1 --2- 172.21.15.139:0/2056517756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab3af0 0x556895ab3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.594+0000 7f5a53e22740 1 -- 172.21.15.139:0/2056517756 >> 172.21.15.139:0/2056517756 conn(0x556895ab0060 msgr2=0x556895ab0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.594+0000 7f5a4ad9d640 1 --2- 172.21.15.139:0/627846954 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5a340a3640 0x7f5a340a5b00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400aa90 tx=0x7f5a44009250 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.599+0000 7f5a53e22740 1 -- 172.21.15.139:0/2056517756 shutdown_connections 2024-09-16T01:52:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.599+0000 7f5a53e22740 1 -- 172.21.15.139:0/2056517756 wait complete. 2024-09-16T01:52:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.599+0000 7f5a53e22740 1 -- start start 2024-09-16T01:52:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.599+0000 7f5a53e22740 1 Processor -- start 2024-09-16T01:52:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.600+0000 7f5a53e22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895abbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.600+0000 7f5a53e22740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab4410 0x556895abc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.600+0000 7f5a53e22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895abc880 0x556895acae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.600+0000 7f5a53e22740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556895ac18d0 con 0x556895abc880 2024-09-16T01:52:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.600+0000 7f5a53e22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5568959a1290 con 0x556895ab3af0 2024-09-16T01:52:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.600+0000 7f5a53e22740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556895ac1a50 con 0x556895ab4410 2024-09-16T01:52:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.600+0000 7f5a53e22740 10 client.0 _ll_register_callbacks cb 0x5568959d2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:52:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.600+0000 7f5a49d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab4410 0x556895abc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.600+0000 7f5a49d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab4410 0x556895abc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:34534/0 (socket says 172.21.15.139:34534) 2024-09-16T01:52:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.600+0000 7f5a49d9b640 1 -- 172.21.15.139:0/666854203 learned_addr learned my addr 172.21.15.139:0/666854203 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.601+0000 7f5a4ad9d640 1 --2- 172.21.15.139:0/666854203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895abc880 0x556895acae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.601+0000 7f5a4a59c640 1 --2- 172.21.15.139:0/666854203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895abbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.602+0000 7f5a49d9b640 1 -- 172.21.15.139:0/666854203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 msgr2=0x556895abbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.602+0000 7f5a49d9b640 1 --2- 172.21.15.139:0/666854203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895abbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.602+0000 7f5a49d9b640 1 -- 172.21.15.139:0/666854203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895abc880 msgr2=0x556895acae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.602+0000 7f5a49d9b640 1 --2- 172.21.15.139:0/666854203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895abc880 0x556895acae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.602+0000 7f5a49d9b640 1 -- 172.21.15.139:0/666854203 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55689572b210 con 0x556895ab4410 2024-09-16T01:52:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.603+0000 7f5a49d9b640 1 --2- 172.21.15.139:0/666854203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab4410 0x556895abc340 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c004700 tx=0x7f5a3c0125f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edd2b11b0f8bc8e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.607+0000 7f5a31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:52:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.607+0000 7f5a31ffb640 1 -- 172.21.15.139:0/666854203 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c022070 con 0x556895ab4410 2024-09-16T01:52:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.607+0000 7f5a31ffb640 1 -- 172.21.15.139:0/666854203 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f5a3c012c40 con 0x556895ab4410 2024-09-16T01:52:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.607+0000 7f5a53e22740 1 -- 172.21.15.139:0/666854203 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a3c013950 con 0x556895ab4410 2024-09-16T01:52:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.607+0000 7f5a31ffb640 1 -- 172.21.15.139:0/666854203 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f5a3c017040 con 0x556895ab4410 2024-09-16T01:52:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.610+0000 7f5a53e22740 10 client.26224 fetch_fsmap learned FSMap version 62 2024-09-16T01:52:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.610+0000 7f5a53e22740 1 -- 172.21.15.139:0/666854203 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x5568957f7620 con 0x556895ab4410 2024-09-16T01:52:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.613+0000 7f5a31ffb640 1 -- 172.21.15.139:0/666854203 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c013950 con 0x556895ab4410 2024-09-16T01:52:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.613+0000 7f5a31ffb640 1 -- 172.21.15.139:0/666854203 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 62) ==== 8434+0+0 (secure 0 0 0) 0x7f5a3c030c80 con 0x556895ab4410 2024-09-16T01:52:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.614+0000 7f5a53e22740 10 client.26224 fetch_fsmap finished waiting for FSMap version 62 2024-09-16T01:52:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.614+0000 7f5a53e22740 10 client.26224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:52:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.614+0000 7f5a53e22740 20 client.26224 populate_metadata read hostname 'smithi139' 2024-09-16T01:52:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.614+0000 7f5a53e22740 1 --2- 172.21.15.139:0/666854203 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556895acf2d0 0x556895ad16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.614+0000 7f5a53e22740 4 client.26224 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:52:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.614+0000 7f5a53e22740 1 -- 172.21.15.139:0/666854203 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556895a25a20 con 0x556895acf2d0 2024-09-16T01:52:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.616+0000 7f5a4a59c640 1 --2- 172.21.15.139:0/666854203 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556895acf2d0 0x556895ad16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.622+0000 7f5a4a59c640 1 --2- 172.21.15.139:0/666854203 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556895acf2d0 0x556895ad16c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c991d73de77fdf server_cookie=a8757c2750809242 in_seq=0 out_seq=0 2024-09-16T01:52:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.622+0000 7f5a31ffb640 10 client.26224 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.626+0000 7f5a31ffb640 1 -- 172.21.15.139:0/666854203 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556895a25a20 con 0x556895acf2d0 2024-09-16T01:52:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.626+0000 7f5a31ffb640 10 client.26224 handle_command_reply: tid=0 2024-09-16T01:52:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.627+0000 7f5a53e22740 1 client.26224 shutdown 2024-09-16T01:52:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.627+0000 7f5a53e22740 1 -- 172.21.15.139:0/666854203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab4410 msgr2=0x556895abc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.627+0000 7f5a53e22740 1 --2- 172.21.15.139:0/666854203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab4410 0x556895abc340 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c004700 tx=0x7f5a3c0125f0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.628+0000 7f5a53e22740 1 -- 172.21.15.139:0/666854203 shutdown_connections 2024-09-16T01:52:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.628+0000 7f5a53e22740 1 --2- 172.21.15.139:0/666854203 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556895acf2d0 0x556895ad16c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.628+0000 7f5a53e22740 1 --2- 172.21.15.139:0/666854203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895abc880 0x556895acae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.628+0000 7f5a53e22740 1 --2- 172.21.15.139:0/666854203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab4410 0x556895abc340 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.628+0000 7f5a53e22740 1 --2- 172.21.15.139:0/666854203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895abbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.628+0000 7f5a53e22740 1 -- 172.21.15.139:0/666854203 >> 172.21.15.139:0/666854203 conn(0x556895ab0060 msgr2=0x556895ab5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.628+0000 7f5a31ffb640 0 client.26224 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.630+0000 7f5a53e22740 1 -- 172.21.15.139:0/666854203 shutdown_connections 2024-09-16T01:52:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.631+0000 7f5a53e22740 1 -- 172.21.15.139:0/666854203 wait complete. 2024-09-16T01:52:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.631+0000 7f5a53e22740 20 client.26224 trim_cache size 0 max 16384 2024-09-16T01:52:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.661+0000 7f5a53e22740 1 Processor -- start 2024-09-16T01:52:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.661+0000 7f5a53e22740 1 -- start start 2024-09-16T01:52:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.662+0000 7f5a53e22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895acac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.662+0000 7f5a53e22740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab4410 0x556895abdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.662+0000 7f5a4a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895acac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.662+0000 7f5a4a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895acac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:43006/0 (socket says 172.21.15.139:43006) 2024-09-16T01:52:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.662+0000 7f5a49d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab4410 0x556895abdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.662+0000 7f5a49d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab4410 0x556895abdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:34546/0 (socket says 172.21.15.139:34546) 2024-09-16T01:52:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.662+0000 7f5a53e22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895abc880 0x556895ad6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.662+0000 7f5a4a59c640 1 -- 172.21.15.139:0/1348083886 learned_addr learned my addr 172.21.15.139:0/1348083886 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.663+0000 7f5a53e22740 1 -- 172.21.15.139:0/1348083886 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c010300 con 0x556895abc880 2024-09-16T01:52:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.663+0000 7f5a4ad9d640 1 --2- 172.21.15.139:0/1348083886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895abc880 0x556895ad6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.663+0000 7f5a53e22740 1 -- 172.21.15.139:0/1348083886 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c012080 con 0x556895ab3af0 2024-09-16T01:52:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.663+0000 7f5a53e22740 1 -- 172.21.15.139:0/1348083886 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c010a90 con 0x556895ab4410 2024-09-16T01:52:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.665+0000 7f5a4a59c640 1 -- 172.21.15.139:0/1348083886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab4410 msgr2=0x556895abdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.665+0000 7f5a4a59c640 1 --2- 172.21.15.139:0/1348083886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab4410 0x556895abdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.665+0000 7f5a4a59c640 1 -- 172.21.15.139:0/1348083886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895abc880 msgr2=0x556895ad6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.665+0000 7f5a4a59c640 1 --2- 172.21.15.139:0/1348083886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895abc880 0x556895ad6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.665+0000 7f5a4a59c640 1 -- 172.21.15.139:0/1348083886 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556895abc6b0 con 0x556895ab3af0 2024-09-16T01:52:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.665+0000 7f5a4a59c640 1 --2- 172.21.15.139:0/1348083886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895acac50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5a40005ec0 tx=0x7f5a40100040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6908cfcddf152a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.668+0000 7f5a33fff640 1 -- 172.21.15.139:0/1348083886 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a400d16c0 con 0x556895ab3af0 2024-09-16T01:52:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.668+0000 7f5a33fff640 1 -- 172.21.15.139:0/1348083886 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a400d5f80 con 0x556895ab3af0 2024-09-16T01:52:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.669+0000 7f5a33fff640 1 -- 172.21.15.139:0/1348083886 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4005cdb0 con 0x556895ab3af0 2024-09-16T01:52:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.670+0000 7f5a53e22740 1 -- 172.21.15.139:0/1348083886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 msgr2=0x556895acac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.670+0000 7f5a53e22740 1 --2- 172.21.15.139:0/1348083886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895acac50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5a40005ec0 tx=0x7f5a40100040 comp rx=0 tx=0).stop 2024-09-16T01:52:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.670+0000 7f5a53e22740 1 -- 172.21.15.139:0/1348083886 shutdown_connections 2024-09-16T01:52:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.670+0000 7f5a53e22740 1 --2- 172.21.15.139:0/1348083886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895abc880 0x556895ad6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.670+0000 7f5a53e22740 1 --2- 172.21.15.139:0/1348083886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895ab4410 0x556895abdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.670+0000 7f5a53e22740 1 --2- 172.21.15.139:0/1348083886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895acac50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.670+0000 7f5a53e22740 1 -- 172.21.15.139:0/1348083886 >> 172.21.15.139:0/1348083886 conn(0x556895ab0060 msgr2=0x556895abd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.674+0000 7f5a53e22740 1 -- 172.21.15.139:0/1348083886 shutdown_connections 2024-09-16T01:52:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.674+0000 7f5a53e22740 1 -- 172.21.15.139:0/1348083886 wait complete. 2024-09-16T01:52:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.674+0000 7f5a53e22740 1 -- start start 2024-09-16T01:52:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.674+0000 7f5a53e22740 1 Processor -- start 2024-09-16T01:52:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.675+0000 7f5a53e22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895ac7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.675+0000 7f5a53e22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895ab4410 0x556895ac2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.675+0000 7f5a4a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895ac7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.675+0000 7f5a4a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895ac7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:43014/0 (socket says 172.21.15.139:43014) 2024-09-16T01:52:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.675+0000 7f5a4a59c640 1 -- 172.21.15.139:0/2460543726 learned_addr learned my addr 172.21.15.139:0/2460543726 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.675+0000 7f5a53e22740 1 --2- 172.21.15.139:0/2460543726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895abc880 0x556895ac2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.675+0000 7f5a53e22740 1 -- 172.21.15.139:0/2460543726 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c0118f0 con 0x556895ab4410 2024-09-16T01:52:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.675+0000 7f5a53e22740 1 -- 172.21.15.139:0/2460543726 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c011160 con 0x556895ab3af0 2024-09-16T01:52:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.675+0000 7f5a4ad9d640 1 --2- 172.21.15.139:0/2460543726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895abc880 0x556895ac2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.676+0000 7f5a53e22740 1 -- 172.21.15.139:0/2460543726 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c00e300 con 0x556895abc880 2024-09-16T01:52:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.676+0000 7f5a53e22740 10 client.0 _ll_register_callbacks cb 0x7f5a3c009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:52:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.677+0000 7f5a4a59c640 1 -- 172.21.15.139:0/2460543726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895abc880 msgr2=0x556895ac2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.677+0000 7f5a4a59c640 1 --2- 172.21.15.139:0/2460543726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895abc880 0x556895ac2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.677+0000 7f5a4a59c640 1 -- 172.21.15.139:0/2460543726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895ab4410 msgr2=0x556895ac2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.677+0000 7f5a4a59c640 1 --2- 172.21.15.139:0/2460543726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895ab4410 0x556895ac2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.677+0000 7f5a4a59c640 1 -- 172.21.15.139:0/2460543726 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556895ac3730 con 0x556895ab3af0 2024-09-16T01:52:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.677+0000 7f5a4a59c640 1 --2- 172.21.15.139:0/2460543726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895ac7610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5a4001aa30 tx=0x7f5a4001a690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcbd426767b2f735 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.677+0000 7f5a31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:52:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.678+0000 7f5a31ffb640 1 -- 172.21.15.139:0/2460543726 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a400eb060 con 0x556895ab3af0 2024-09-16T01:52:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.678+0000 7f5a53e22740 1 -- 172.21.15.139:0/2460543726 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a400d5f80 con 0x556895ab3af0 2024-09-16T01:52:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.678+0000 7f5a31ffb640 1 -- 172.21.15.139:0/2460543726 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f5a400e6270 con 0x556895ab3af0 2024-09-16T01:52:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.678+0000 7f5a31ffb640 1 -- 172.21.15.139:0/2460543726 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f5a400dd050 con 0x556895ab3af0 2024-09-16T01:52:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.679+0000 7f5a53e22740 10 client.26267 fetch_fsmap learned FSMap version 62 2024-09-16T01:52:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.679+0000 7f5a53e22740 1 -- 172.21.15.139:0/2460543726 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x556895b39e30 con 0x556895ab3af0 2024-09-16T01:52:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.684+0000 7f5a31ffb640 1 -- 172.21.15.139:0/2460543726 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a400d5f80 con 0x556895ab3af0 2024-09-16T01:52:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.684+0000 7f5a31ffb640 1 -- 172.21.15.139:0/2460543726 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 62) ==== 8434+0+0 (secure 0 0 0) 0x7f5a400c9980 con 0x556895ab3af0 2024-09-16T01:52:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.685+0000 7f5a53e22740 10 client.26267 fetch_fsmap finished waiting for FSMap version 62 2024-09-16T01:52:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.685+0000 7f5a53e22740 10 client.26267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:52:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.685+0000 7f5a53e22740 20 client.26267 populate_metadata read hostname 'smithi139' 2024-09-16T01:52:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.685+0000 7f5a53e22740 1 --2- 172.21.15.139:0/2460543726 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556895b3a540 0x556895b3c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.685+0000 7f5a53e22740 4 client.26267 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:52:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.685+0000 7f5a53e22740 1 -- 172.21.15.139:0/2460543726 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556895a1fd70 con 0x556895b3a540 2024-09-16T01:52:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.685+0000 7f5a49d9b640 1 --2- 172.21.15.139:0/2460543726 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556895b3a540 0x556895b3c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.686+0000 7f5a31ffb640 10 client.26267 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.686+0000 7f5a49d9b640 1 --2- 172.21.15.139:0/2460543726 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556895b3a540 0x556895b3c960 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e045e17b1698d6bd server_cookie=1d09fd06815dd6d1 in_seq=0 out_seq=0 2024-09-16T01:52:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.793+0000 7f5a31ffb640 1 -- 172.21.15.139:0/2460543726 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556895a1fd70 con 0x556895b3a540 2024-09-16T01:52:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.793+0000 7f5a31ffb640 10 client.26267 handle_command_reply: tid=0 2024-09-16T01:52:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.793+0000 7f5a53e22740 1 client.26267 shutdown 2024-09-16T01:52:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.794+0000 7f5a53e22740 1 -- 172.21.15.139:0/2460543726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 msgr2=0x556895ac7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.794+0000 7f5a53e22740 1 --2- 172.21.15.139:0/2460543726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895ac7610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5a4001aa30 tx=0x7f5a4001a690 comp rx=0 tx=0).stop 2024-09-16T01:52:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.794+0000 7f5a53e22740 1 -- 172.21.15.139:0/2460543726 shutdown_connections 2024-09-16T01:52:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.794+0000 7f5a53e22740 1 --2- 172.21.15.139:0/2460543726 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x556895b3a540 0x556895b3c960 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.794+0000 7f5a53e22740 1 --2- 172.21.15.139:0/2460543726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556895abc880 0x556895ac2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.794+0000 7f5a53e22740 1 --2- 172.21.15.139:0/2460543726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556895ab4410 0x556895ac2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.794+0000 7f5a53e22740 1 --2- 172.21.15.139:0/2460543726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556895ab3af0 0x556895ac7610 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.794+0000 7f5a53e22740 1 -- 172.21.15.139:0/2460543726 >> 172.21.15.139:0/2460543726 conn(0x556895ab0060 msgr2=0x556895abbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.794+0000 7f5a31ffb640 0 client.26267 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.794+0000 7f5a53e22740 1 -- 172.21.15.139:0/2460543726 shutdown_connections 2024-09-16T01:52:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.794+0000 7f5a53e22740 1 -- 172.21.15.139:0/2460543726 wait complete. 2024-09-16T01:52:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.794+0000 7f5a53e22740 20 client.26267 trim_cache size 0 max 16384 2024-09-16T01:52:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:52:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:52:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:52:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:52:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.795+0000 7f5a50fa6640 1 -- 172.21.15.139:0/627846954 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5a340a3640 msgr2=0x7f5a340a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.795+0000 7f5a50fa6640 1 --2- 172.21.15.139:0/627846954 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5a340a3640 0x7f5a340a5b00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400aa90 tx=0x7f5a44009250 comp rx=0 tx=0).stop 2024-09-16T01:52:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.795+0000 7f5a50fa6640 1 -- 172.21.15.139:0/627846954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4c1009d0 msgr2=0x7f5a4c073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.795+0000 7f5a50fa6640 1 --2- 172.21.15.139:0/627846954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4c1009d0 0x7f5a4c073ad0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5a40005ab0 tx=0x7f5a40004bf0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.795+0000 7f5a50fa6640 1 -- 172.21.15.139:0/627846954 shutdown_connections 2024-09-16T01:52:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.795+0000 7f5a50fa6640 1 --2- 172.21.15.139:0/627846954 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5a340a3640 0x7f5a340a5b00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.795+0000 7f5a50fa6640 1 --2- 172.21.15.139:0/627846954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a4c1069e0 0x7f5a4c06f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.795+0000 7f5a50fa6640 1 --2- 172.21.15.139:0/627846954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a4c101370 0x7f5a4c06eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.795+0000 7f5a50fa6640 1 --2- 172.21.15.139:0/627846954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4c1009d0 0x7f5a4c073ad0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.795+0000 7f5a50fa6640 1 -- 172.21.15.139:0/627846954 >> 172.21.15.139:0/627846954 conn(0x7f5a4c0fc6b0 msgr2=0x7f5a4c0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.795+0000 7f5a50fa6640 1 -- 172.21.15.139:0/627846954 shutdown_connections 2024-09-16T01:52:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:42.795+0000 7f5a50fa6640 1 -- 172.21.15.139:0/627846954 wait complete. 2024-09-16T01:52:43.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:43 smithi176 ceph-mon[35324]: pgmap v387: 129 pgs: 11 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 84 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-09-16T01:52:43.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:43 smithi164 ceph-mon[35121]: pgmap v387: 129 pgs: 11 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 84 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-09-16T01:52:43.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:43 smithi139 ceph-mon[31175]: pgmap v387: 129 pgs: 11 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 84 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-09-16T01:52:45.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:45 smithi164 ceph-mon[35121]: pgmap v388: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 270 op/s 2024-09-16T01:52:45.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:45 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:45.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:45 smithi139 ceph-mon[31175]: pgmap v388: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 270 op/s 2024-09-16T01:52:45.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:45 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:45.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:45 smithi176 ceph-mon[35324]: pgmap v388: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 270 op/s 2024-09-16T01:52:45.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:45 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:47 smithi164 ceph-mon[35121]: pgmap v389: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-09-16T01:52:47.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:47 smithi139 ceph-mon[31175]: pgmap v389: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-09-16T01:52:47.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:47 smithi176 ceph-mon[35324]: pgmap v389: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-09-16T01:52:49.488 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:49 smithi176 ceph-mon[35324]: pgmap v390: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 285 op/s 2024-09-16T01:52:49.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:49 smithi164 ceph-mon[35121]: pgmap v390: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 285 op/s 2024-09-16T01:52:49.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:49 smithi139 ceph-mon[31175]: pgmap v390: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 285 op/s 2024-09-16T01:52:51.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:51 smithi164 ceph-mon[35121]: pgmap v391: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-09-16T01:52:51.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:51 smithi139 ceph-mon[31175]: pgmap v391: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-09-16T01:52:51.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:51 smithi176 ceph-mon[35324]: pgmap v391: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-09-16T01:52:53.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:53 smithi164 ceph-mon[35121]: pgmap v392: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 277 op/s 2024-09-16T01:52:53.672 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:53 smithi139 ceph-mon[31175]: pgmap v392: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 277 op/s 2024-09-16T01:52:53.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:53 smithi176 ceph-mon[35324]: pgmap v392: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 277 op/s 2024-09-16T01:52:54.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:54 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:52:54.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:54 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:52:54.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:54 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:52:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:55 smithi164 ceph-mon[35121]: pgmap v393: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.2 MiB/s wr, 229 op/s 2024-09-16T01:52:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:55 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:52:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:52:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:52:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:52:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:52:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:55 smithi139 ceph-mon[31175]: pgmap v393: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.2 MiB/s wr, 229 op/s 2024-09-16T01:52:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:55 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:52:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:52:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:52:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:52:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:52:55.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:55 smithi176 ceph-mon[35324]: pgmap v393: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.2 MiB/s wr, 229 op/s 2024-09-16T01:52:55.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:55 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:52:55.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:52:55.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:52:55.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:52:55.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:52:55.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:52:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.931+0000 7f1bbb217640 1 -- 172.21.15.139:0/2676080730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1bb41069e0 msgr2=0x7f1bb4106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.931+0000 7f1bbb217640 1 --2- 172.21.15.139:0/2676080730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1bb41069e0 0x7f1bb4106dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009890 tx=0x7f1ba802a390 comp rx=0 tx=0).stop 2024-09-16T01:52:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.932+0000 7f1bbb217640 1 -- 172.21.15.139:0/2676080730 shutdown_connections 2024-09-16T01:52:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.932+0000 7f1bbb217640 1 --2- 172.21.15.139:0/2676080730 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb4101370 0x7f1bb410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.932+0000 7f1bbb217640 1 --2- 172.21.15.139:0/2676080730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb41009d0 0x7f1bb4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.932+0000 7f1bbb217640 1 --2- 172.21.15.139:0/2676080730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1bb41069e0 0x7f1bb4106dc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.932+0000 7f1bbb217640 1 -- 172.21.15.139:0/2676080730 >> 172.21.15.139:0/2676080730 conn(0x7f1bb40fc6b0 msgr2=0x7f1bb40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.933+0000 7f1bbb217640 1 -- 172.21.15.139:0/2676080730 shutdown_connections 2024-09-16T01:52:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.933+0000 7f1bbb217640 1 -- 172.21.15.139:0/2676080730 wait complete. 2024-09-16T01:52:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.934+0000 7f1bbb217640 1 Processor -- start 2024-09-16T01:52:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.935+0000 7f1bbb217640 1 -- start start 2024-09-16T01:52:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.935+0000 7f1bbb217640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb41009d0 0x7f1bb4116200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.936+0000 7f1bbb217640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb4101370 0x7f1bb4116740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.936+0000 7f1bbb217640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1bb41069e0 0x7f1bb4110480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.936+0000 7f1bbb217640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb410eea0 con 0x7f1bb41009d0 2024-09-16T01:52:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.936+0000 7f1bbb217640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb410ed20 con 0x7f1bb4101370 2024-09-16T01:52:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.936+0000 7f1bbb217640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb410f020 con 0x7f1bb41069e0 2024-09-16T01:52:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.936+0000 7f1bb8fb3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb41009d0 0x7f1bb4116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.936+0000 7f1bb8fb3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb41009d0 0x7f1bb4116200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35614/0 (socket says 172.21.15.139:35614) 2024-09-16T01:52:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.936+0000 7f1bb8fb3640 1 -- 172.21.15.139:0/3651493326 learned_addr learned my addr 172.21.15.139:0/3651493326 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.936+0000 7f1bb8fb3640 1 -- 172.21.15.139:0/3651493326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1bb41069e0 msgr2=0x7f1bb4110480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.936+0000 7f1bb8fb3640 1 --2- 172.21.15.139:0/3651493326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1bb41069e0 0x7f1bb4110480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.936+0000 7f1bb8fb3640 1 -- 172.21.15.139:0/3651493326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb4101370 msgr2=0x7f1bb4116740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:52:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.936+0000 7f1bb8fb3640 1 --2- 172.21.15.139:0/3651493326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb4101370 0x7f1bb4116740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.936+0000 7f1bb8fb3640 1 -- 172.21.15.139:0/3651493326 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb4110d40 con 0x7f1bb41009d0 2024-09-16T01:52:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.937+0000 7f1bb8fb3640 1 --2- 172.21.15.139:0/3651493326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb41009d0 0x7f1bb4116200 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1ba802a980 tx=0x7f1ba8033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.939+0000 7f1bb1ffb640 1 -- 172.21.15.139:0/3651493326 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8037040 con 0x7f1bb41009d0 2024-09-16T01:52:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.939+0000 7f1bbb217640 1 -- 172.21.15.139:0/3651493326 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb4110fd0 con 0x7f1bb41009d0 2024-09-16T01:52:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.939+0000 7f1bbb217640 1 -- 172.21.15.139:0/3651493326 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb4071e20 con 0x7f1bb41009d0 2024-09-16T01:52:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.939+0000 7f1bb1ffb640 1 -- 172.21.15.139:0/3651493326 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba8040070 con 0x7f1bb41009d0 2024-09-16T01:52:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.940+0000 7f1bb1ffb640 1 -- 172.21.15.139:0/3651493326 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80046c0 con 0x7f1bb41009d0 2024-09-16T01:52:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.944+0000 7f1bb1ffb640 1 -- 172.21.15.139:0/3651493326 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1ba8004240 con 0x7f1bb41009d0 2024-09-16T01:52:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.945+0000 7f1bb1ffb640 1 --2- 172.21.15.139:0/3651493326 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1ba40a3390 0x7f1ba40a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.946+0000 7f1bb1ffb640 1 -- 172.21.15.139:0/3651493326 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f1ba80f0130 con 0x7f1bb41009d0 2024-09-16T01:52:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.948+0000 7f1bbe089740 1 Processor -- start 2024-09-16T01:52:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.949+0000 7f1bbe089740 1 -- start start 2024-09-16T01:52:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.949+0000 7f1bbe089740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec0baf0 0x55883ec0bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.949+0000 7f1bbe089740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec16770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.949+0000 7f1bbe089740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.949+0000 7f1bbe089740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55883eb702f0 con 0x55883ec0baf0 2024-09-16T01:52:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.949+0000 7f1bbe089740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55883eb18270 con 0x55883ec16cb0 2024-09-16T01:52:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.949+0000 7f1bbe089740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55883eb6c2b0 con 0x55883ec0c410 2024-09-16T01:52:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.950+0000 7f1bb97b4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.950+0000 7f1bb97b4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec190a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:32822/0 (socket says 172.21.15.139:32822) 2024-09-16T01:52:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.950+0000 7f1bb97b4640 1 -- 172.21.15.139:0/1377685110 learned_addr learned my addr 172.21.15.139:0/1377685110 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.950+0000 7f1bb3fff640 1 --2- 172.21.15.139:0/3651493326 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1ba40a3390 0x7f1ba40a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.950+0000 7f1bb97b4640 1 -- 172.21.15.139:0/1377685110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 msgr2=0x55883ec16770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:52:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.950+0000 7f1bb97b4640 1 --2- 172.21.15.139:0/1377685110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec16770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.950+0000 7f1bb97b4640 1 -- 172.21.15.139:0/1377685110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec0baf0 msgr2=0x55883ec0bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.950+0000 7f1bb97b4640 1 --2- 172.21.15.139:0/1377685110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec0baf0 0x55883ec0bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.950+0000 7f1bb97b4640 1 -- 172.21.15.139:0/1377685110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883eb17340 con 0x55883ec16cb0 2024-09-16T01:52:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.951+0000 7f1bb97b4640 1 --2- 172.21.15.139:0/1377685110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec190a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00e470 tx=0x7f1bac00e870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ae13879e5031b44 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.951+0000 7f1b9effd640 1 -- 172.21.15.139:0/1377685110 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac004d20 con 0x55883ec16cb0 2024-09-16T01:52:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.951+0000 7f1b9effd640 1 -- 172.21.15.139:0/1377685110 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bac003e50 con 0x55883ec16cb0 2024-09-16T01:52:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.951+0000 7f1b9effd640 1 -- 172.21.15.139:0/1377685110 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac00c8a0 con 0x55883ec16cb0 2024-09-16T01:52:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.952+0000 7f1bbe089740 1 -- 172.21.15.139:0/1377685110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 msgr2=0x55883ec190a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.952+0000 7f1bbe089740 1 --2- 172.21.15.139:0/1377685110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec190a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00e470 tx=0x7f1bac00e870 comp rx=0 tx=0).stop 2024-09-16T01:52:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.952+0000 7f1bb3fff640 1 --2- 172.21.15.139:0/3651493326 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1ba40a3390 0x7f1ba40a5850 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55883ec08ec0 tx=0x7f1ba0006bd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.953+0000 7f1bbe089740 1 -- 172.21.15.139:0/1377685110 shutdown_connections 2024-09-16T01:52:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.953+0000 7f1bbe089740 1 --2- 172.21.15.139:0/1377685110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec190a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.953+0000 7f1bbe089740 1 --2- 172.21.15.139:0/1377685110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec16770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.953+0000 7f1bbe089740 1 --2- 172.21.15.139:0/1377685110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec0baf0 0x55883ec0bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.953+0000 7f1bbe089740 1 -- 172.21.15.139:0/1377685110 >> 172.21.15.139:0/1377685110 conn(0x55883ec08060 msgr2=0x55883ec08440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.953+0000 7f1bbe089740 1 -- 172.21.15.139:0/1377685110 shutdown_connections 2024-09-16T01:52:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.953+0000 7f1bbe089740 1 -- 172.21.15.139:0/1377685110 wait complete. 2024-09-16T01:52:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.954+0000 7f1bbe089740 1 -- start start 2024-09-16T01:52:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.954+0000 7f1bbe089740 1 Processor -- start 2024-09-16T01:52:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bbe089740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec0baf0 0x55883ec13bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bbe089740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec14130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bbe089740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec16cb0 0x55883ec14670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bbe089740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55883ec19880 con 0x55883ec16cb0 2024-09-16T01:52:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bbe089740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55883eaf9290 con 0x55883ec0baf0 2024-09-16T01:52:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bbe089740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55883ec19a00 con 0x55883ec0c410 2024-09-16T01:52:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bbe089740 10 client.0 _ll_register_callbacks cb 0x55883eb2a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:52:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bb3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec14130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bb97b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec16cb0 0x55883ec14670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bb3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec14130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:45236/0 (socket says 172.21.15.139:45236) 2024-09-16T01:52:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bb3fff640 1 -- 172.21.15.139:0/1817577093 learned_addr learned my addr 172.21.15.139:0/1817577093 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bb3fff640 1 -- 172.21.15.139:0/1817577093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec0baf0 msgr2=0x55883ec13bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:52:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bb3fff640 1 --2- 172.21.15.139:0/1817577093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec0baf0 0x55883ec13bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bb3fff640 1 -- 172.21.15.139:0/1817577093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec16cb0 msgr2=0x55883ec14670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.955+0000 7f1bb3fff640 1 --2- 172.21.15.139:0/1817577093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec16cb0 0x55883ec14670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.956+0000 7f1bb3fff640 1 -- 172.21.15.139:0/1817577093 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883e883210 con 0x55883ec0c410 2024-09-16T01:52:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.956+0000 7f1bb3fff640 1 --2- 172.21.15.139:0/1817577093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec14130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0013f50 tx=0x7f1ba0004700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c72c061b99dd333d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.956+0000 7f1b93fff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:52:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.956+0000 7f1b93fff640 1 -- 172.21.15.139:0/1817577093 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba001d5f0 con 0x55883ec0c410 2024-09-16T01:52:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.957+0000 7f1b93fff640 1 -- 172.21.15.139:0/1817577093 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9642+0+0 (secure 0 0 0) 0x7f1ba001d790 con 0x55883ec0c410 2024-09-16T01:52:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.957+0000 7f1bbe089740 1 -- 172.21.15.139:0/1817577093 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bac003e50 con 0x55883ec0c410 2024-09-16T01:52:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.958+0000 7f1b93fff640 1 -- 172.21.15.139:0/1817577093 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f1ba001d990 con 0x55883ec0c410 2024-09-16T01:52:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.959+0000 7f1bbe089740 10 client.26236 fetch_fsmap learned FSMap version 64 2024-09-16T01:52:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.959+0000 7f1bbe089740 1 -- 172.21.15.139:0/1817577093 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55883e94f620 con 0x55883ec0c410 2024-09-16T01:52:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.961+0000 7f1b93fff640 1 -- 172.21.15.139:0/1817577093 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bac003e50 con 0x55883ec0c410 2024-09-16T01:52:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.961+0000 7f1b93fff640 1 -- 172.21.15.139:0/1817577093 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 64) ==== 8442+0+0 (secure 0 0 0) 0x7f1ba0012070 con 0x55883ec0c410 2024-09-16T01:52:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.962+0000 7f1bbe089740 10 client.26236 fetch_fsmap finished waiting for FSMap version 64 2024-09-16T01:52:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.962+0000 7f1bbe089740 10 client.26236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:52:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.962+0000 7f1bbe089740 20 client.26236 populate_metadata read hostname 'smithi139' 2024-09-16T01:52:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.962+0000 7f1bbe089740 1 --2- 172.21.15.139:0/1817577093 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55883ec26eb0 0x55883ec29310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.962+0000 7f1bbe089740 4 client.26236 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:52:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.962+0000 7f1bbe089740 1 -- 172.21.15.139:0/1817577093 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55883eb7da20 con 0x55883ec26eb0 2024-09-16T01:52:55.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.963+0000 7f1bb8fb3640 1 --2- 172.21.15.139:0/1817577093 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55883ec26eb0 0x55883ec29310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.964+0000 7f1bb8fb3640 1 --2- 172.21.15.139:0/1817577093 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55883ec26eb0 0x55883ec29310 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32b5a1991a32b88a server_cookie=696a582f63df809b in_seq=0 out_seq=0 2024-09-16T01:52:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.964+0000 7f1b93fff640 10 client.26236 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.965+0000 7f1b93fff640 1 -- 172.21.15.139:0/1817577093 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55883eb7da20 con 0x55883ec26eb0 2024-09-16T01:52:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.965+0000 7f1b93fff640 10 client.26236 handle_command_reply: tid=0 2024-09-16T01:52:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.965+0000 7f1bbe089740 1 client.26236 shutdown 2024-09-16T01:52:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.966+0000 7f1bbe089740 1 -- 172.21.15.139:0/1817577093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 msgr2=0x55883ec14130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.966+0000 7f1bbe089740 1 --2- 172.21.15.139:0/1817577093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec14130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0013f50 tx=0x7f1ba0004700 comp rx=0 tx=0).stop 2024-09-16T01:52:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.966+0000 7f1bbe089740 1 -- 172.21.15.139:0/1817577093 shutdown_connections 2024-09-16T01:52:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.966+0000 7f1bbe089740 1 --2- 172.21.15.139:0/1817577093 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55883ec26eb0 0x55883ec29310 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.966+0000 7f1bbe089740 1 --2- 172.21.15.139:0/1817577093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec16cb0 0x55883ec14670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.966+0000 7f1bbe089740 1 --2- 172.21.15.139:0/1817577093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec14130 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.966+0000 7f1bbe089740 1 --2- 172.21.15.139:0/1817577093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec0baf0 0x55883ec13bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.966+0000 7f1bbe089740 1 -- 172.21.15.139:0/1817577093 >> 172.21.15.139:0/1817577093 conn(0x55883ec08060 msgr2=0x55883ec0ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.966+0000 7f1b93fff640 0 client.26236 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.966+0000 7f1bbe089740 1 -- 172.21.15.139:0/1817577093 shutdown_connections 2024-09-16T01:52:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.966+0000 7f1bbe089740 1 -- 172.21.15.139:0/1817577093 wait complete. 2024-09-16T01:52:55.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.968+0000 7f1bbe089740 20 client.26236 trim_cache size 0 max 16384 2024-09-16T01:52:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.993+0000 7f1bbe089740 1 Processor -- start 2024-09-16T01:52:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.993+0000 7f1bbe089740 1 -- start start 2024-09-16T01:52:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.994+0000 7f1bbe089740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0baf0 0x55883ec0efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.994+0000 7f1bbe089740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec0c410 0x55883ec144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.994+0000 7f1bbe089740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec2eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.994+0000 7f1bbe089740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b980102d0 con 0x55883ec0c410 2024-09-16T01:52:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.994+0000 7f1bbe089740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b980120b0 con 0x55883ec16cb0 2024-09-16T01:52:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.994+0000 7f1bbe089740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1b98010a00 con 0x55883ec0baf0 2024-09-16T01:52:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.994+0000 7f1bb8fb3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0baf0 0x55883ec0efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.994+0000 7f1bb3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec0c410 0x55883ec144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.994+0000 7f1bb8fb3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0baf0 0x55883ec0efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:45238/0 (socket says 172.21.15.139:45238) 2024-09-16T01:52:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.994+0000 7f1bb8fb3640 1 -- 172.21.15.139:0/1366866985 learned_addr learned my addr 172.21.15.139:0/1366866985 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.994+0000 7f1bb97b4640 1 --2- 172.21.15.139:0/1366866985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec2eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.995+0000 7f1bb97b4640 1 -- 172.21.15.139:0/1366866985 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0baf0 msgr2=0x55883ec0efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.995+0000 7f1bb97b4640 1 --2- 172.21.15.139:0/1366866985 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0baf0 0x55883ec0efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.995+0000 7f1bb97b4640 1 -- 172.21.15.139:0/1366866985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec0c410 msgr2=0x55883ec144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.995+0000 7f1bb97b4640 1 --2- 172.21.15.139:0/1366866985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec0c410 0x55883ec144c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.995+0000 7f1bb97b4640 1 -- 172.21.15.139:0/1366866985 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883ec14f70 con 0x55883ec16cb0 2024-09-16T01:52:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.996+0000 7f1bb97b4640 1 --2- 172.21.15.139:0/1366866985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec2eb80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1bac006010 tx=0x7f1bac009d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5fdc9ba1999d958 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.996+0000 7f1b9e7fc640 1 -- 172.21.15.139:0/1366866985 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac024020 con 0x55883ec16cb0 2024-09-16T01:52:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.996+0000 7f1b9e7fc640 1 -- 172.21.15.139:0/1366866985 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bac01b040 con 0x55883ec16cb0 2024-09-16T01:52:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.996+0000 7f1b9e7fc640 1 -- 172.21.15.139:0/1366866985 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac01a040 con 0x55883ec16cb0 2024-09-16T01:52:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.997+0000 7f1bbe089740 1 -- 172.21.15.139:0/1366866985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 msgr2=0x55883ec2eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.997+0000 7f1bbe089740 1 --2- 172.21.15.139:0/1366866985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec2eb80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1bac006010 tx=0x7f1bac009d50 comp rx=0 tx=0).stop 2024-09-16T01:52:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.998+0000 7f1bbe089740 1 -- 172.21.15.139:0/1366866985 shutdown_connections 2024-09-16T01:52:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.998+0000 7f1bbe089740 1 --2- 172.21.15.139:0/1366866985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec2eb80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.998+0000 7f1bbe089740 1 --2- 172.21.15.139:0/1366866985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec0c410 0x55883ec144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.998+0000 7f1bbe089740 1 --2- 172.21.15.139:0/1366866985 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0baf0 0x55883ec0efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.998+0000 7f1bbe089740 1 -- 172.21.15.139:0/1366866985 >> 172.21.15.139:0/1366866985 conn(0x55883ec08060 msgr2=0x55883ec15c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.998+0000 7f1bbe089740 1 -- 172.21.15.139:0/1366866985 shutdown_connections 2024-09-16T01:52:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.998+0000 7f1bbe089740 1 -- 172.21.15.139:0/1366866985 wait complete. 2024-09-16T01:52:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.999+0000 7f1bbe089740 1 -- start start 2024-09-16T01:52:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:55.999+0000 7f1bbe089740 1 Processor -- start 2024-09-16T01:52:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.000+0000 7f1bbe089740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec0baf0 0x55883ec10e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.000+0000 7f1bbe089740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec11380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.000+0000 7f1bbe089740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec118c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.000+0000 7f1bbe089740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b98011920 con 0x55883ec0baf0 2024-09-16T01:52:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.000+0000 7f1bb3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec11380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.000+0000 7f1bb3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec11380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:45240/0 (socket says 172.21.15.139:45240) 2024-09-16T01:52:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.000+0000 7f1bb3fff640 1 -- 172.21.15.139:0/187836115 learned_addr learned my addr 172.21.15.139:0/187836115 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:52:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.000+0000 7f1bb97b4640 1 --2- 172.21.15.139:0/187836115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.000+0000 7f1bbe089740 1 -- 172.21.15.139:0/187836115 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b98011190 con 0x55883ec16cb0 2024-09-16T01:52:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.001+0000 7f1bbe089740 1 -- 172.21.15.139:0/187836115 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9800e2d0 con 0x55883ec0c410 2024-09-16T01:52:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.001+0000 7f1bb8fb3640 1 --2- 172.21.15.139:0/187836115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec0baf0 0x55883ec10e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.001+0000 7f1bbe089740 10 client.0 _ll_register_callbacks cb 0x7f1ba00093d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:52:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.001+0000 7f1bb3fff640 1 -- 172.21.15.139:0/187836115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 msgr2=0x55883ec118c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.001+0000 7f1bb3fff640 1 --2- 172.21.15.139:0/187836115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec118c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.001+0000 7f1bb3fff640 1 -- 172.21.15.139:0/187836115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec0baf0 msgr2=0x55883ec10e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:52:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.001+0000 7f1bb3fff640 1 --2- 172.21.15.139:0/187836115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec0baf0 0x55883ec10e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.001+0000 7f1bb3fff640 1 -- 172.21.15.139:0/187836115 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883ec90da0 con 0x55883ec0c410 2024-09-16T01:52:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.003+0000 7f1bb3fff640 1 --2- 172.21.15.139:0/187836115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec11380 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55883ec13480 tx=0x7f1ba0006d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b173f2a1a0af8a65 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:52:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.003+0000 7f1b93fff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:52:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.009+0000 7f1b93fff640 1 -- 172.21.15.139:0/187836115 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0036070 con 0x55883ec0c410 2024-09-16T01:52:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.009+0000 7f1bbe089740 1 -- 172.21.15.139:0/187836115 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bac01b040 con 0x55883ec0c410 2024-09-16T01:52:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.009+0000 7f1b93fff640 1 -- 172.21.15.139:0/187836115 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9566+0+0 (secure 0 0 0) 0x7f1ba0027650 con 0x55883ec0c410 2024-09-16T01:52:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.012+0000 7f1b93fff640 1 -- 172.21.15.139:0/187836115 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f1ba0042050 con 0x55883ec0c410 2024-09-16T01:52:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.014+0000 7f1bbe089740 10 client.26242 fetch_fsmap learned FSMap version 64 2024-09-16T01:52:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.014+0000 7f1bbe089740 1 -- 172.21.15.139:0/187836115 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55883ec92040 con 0x55883ec0c410 2024-09-16T01:52:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.016+0000 7f1b93fff640 1 -- 172.21.15.139:0/187836115 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bac01b040 con 0x55883ec0c410 2024-09-16T01:52:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.017+0000 7f1b93fff640 1 -- 172.21.15.139:0/187836115 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 64) ==== 8442+0+0 (secure 0 0 0) 0x7f1ba0030770 con 0x55883ec0c410 2024-09-16T01:52:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.017+0000 7f1bbe089740 10 client.26242 fetch_fsmap finished waiting for FSMap version 64 2024-09-16T01:52:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.017+0000 7f1bbe089740 10 client.26242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:52:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.017+0000 7f1bbe089740 20 client.26242 populate_metadata read hostname 'smithi139' 2024-09-16T01:52:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.017+0000 7f1bbe089740 1 --2- 172.21.15.139:0/187836115 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55883ec92850 0x55883ec94c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:52:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.017+0000 7f1bbe089740 4 client.26242 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:52:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.017+0000 7f1bbe089740 1 -- 172.21.15.139:0/187836115 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55883eb77d70 con 0x55883ec92850 2024-09-16T01:52:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.021+0000 7f1bb8fb3640 1 --2- 172.21.15.139:0/187836115 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55883ec92850 0x55883ec94c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:52:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.023+0000 7f1bb8fb3640 1 --2- 172.21.15.139:0/187836115 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55883ec92850 0x55883ec94c70 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1d4f166bf17ee51 server_cookie=eec7f80e59c0e6c2 in_seq=0 out_seq=0 2024-09-16T01:52:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.024+0000 7f1b93fff640 10 client.26242 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.266+0000 7f1b93fff640 1 -- 172.21.15.139:0/187836115 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55883eb77d70 con 0x55883ec92850 2024-09-16T01:52:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.266+0000 7f1b93fff640 10 client.26242 handle_command_reply: tid=0 2024-09-16T01:52:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.266+0000 7f1bbe089740 1 client.26242 shutdown 2024-09-16T01:52:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.267+0000 7f1bbe089740 1 -- 172.21.15.139:0/187836115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 msgr2=0x55883ec11380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.267+0000 7f1bbe089740 1 --2- 172.21.15.139:0/187836115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec11380 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55883ec13480 tx=0x7f1ba0006d80 comp rx=0 tx=0).stop 2024-09-16T01:52:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.267+0000 7f1bbe089740 1 -- 172.21.15.139:0/187836115 shutdown_connections 2024-09-16T01:52:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.267+0000 7f1bbe089740 1 --2- 172.21.15.139:0/187836115 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55883ec92850 0x55883ec94c70 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.267+0000 7f1bbe089740 1 --2- 172.21.15.139:0/187836115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55883ec16cb0 0x55883ec118c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.267+0000 7f1b93fff640 0 client.26242 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:52:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.267+0000 7f1bbe089740 1 --2- 172.21.15.139:0/187836115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55883ec0c410 0x55883ec11380 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.267+0000 7f1bbe089740 1 --2- 172.21.15.139:0/187836115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55883ec0baf0 0x55883ec10e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.267+0000 7f1bbe089740 1 -- 172.21.15.139:0/187836115 >> 172.21.15.139:0/187836115 conn(0x55883ec08060 msgr2=0x55883ec069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.267+0000 7f1bbe089740 1 -- 172.21.15.139:0/187836115 shutdown_connections 2024-09-16T01:52:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.267+0000 7f1bbe089740 1 -- 172.21.15.139:0/187836115 wait complete. 2024-09-16T01:52:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.268+0000 7f1bbe089740 20 client.26242 trim_cache size 0 max 16384 2024-09-16T01:52:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:52:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:52:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:52:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:52:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.269+0000 7f1bbb217640 1 -- 172.21.15.139:0/3651493326 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1ba40a3390 msgr2=0x7f1ba40a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.269+0000 7f1bbb217640 1 --2- 172.21.15.139:0/3651493326 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1ba40a3390 0x7f1ba40a5850 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55883ec08ec0 tx=0x7f1ba0006bd0 comp rx=0 tx=0).stop 2024-09-16T01:52:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.269+0000 7f1bbb217640 1 -- 172.21.15.139:0/3651493326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb41009d0 msgr2=0x7f1bb4116200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:52:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.269+0000 7f1bbb217640 1 --2- 172.21.15.139:0/3651493326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb41009d0 0x7f1bb4116200 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1ba802a980 tx=0x7f1ba8033d90 comp rx=0 tx=0).stop 2024-09-16T01:52:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.269+0000 7f1bbb217640 1 -- 172.21.15.139:0/3651493326 shutdown_connections 2024-09-16T01:52:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.269+0000 7f1bbb217640 1 --2- 172.21.15.139:0/3651493326 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1ba40a3390 0x7f1ba40a5850 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.269+0000 7f1bbb217640 1 --2- 172.21.15.139:0/3651493326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1bb41069e0 0x7f1bb4110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.269+0000 7f1bbb217640 1 --2- 172.21.15.139:0/3651493326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bb4101370 0x7f1bb4116740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.269+0000 7f1bbb217640 1 --2- 172.21.15.139:0/3651493326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb41009d0 0x7f1bb4116200 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:52:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.269+0000 7f1bbb217640 1 -- 172.21.15.139:0/3651493326 >> 172.21.15.139:0/3651493326 conn(0x7f1bb40fc6b0 msgr2=0x7f1bb40fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:52:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.269+0000 7f1bbb217640 1 -- 172.21.15.139:0/3651493326 shutdown_connections 2024-09-16T01:52:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:52:56.269+0000 7f1bbb217640 1 -- 172.21.15.139:0/3651493326 wait complete. 2024-09-16T01:52:57.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:57 smithi164 ceph-mon[35121]: pgmap v394: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-09-16T01:52:57.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:57 smithi164 ceph-mon[35121]: osdmap e102: 12 total, 12 up, 12 in 2024-09-16T01:52:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:57 smithi139 ceph-mon[31175]: pgmap v394: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-09-16T01:52:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:57 smithi139 ceph-mon[31175]: osdmap e102: 12 total, 12 up, 12 in 2024-09-16T01:52:57.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:57 smithi176 ceph-mon[35324]: pgmap v394: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-09-16T01:52:57.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:57 smithi176 ceph-mon[35324]: osdmap e102: 12 total, 12 up, 12 in 2024-09-16T01:52:59.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:52:59 smithi164 ceph-mon[35121]: pgmap v396: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-09-16T01:52:59.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:52:59 smithi139 ceph-mon[31175]: pgmap v396: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-09-16T01:52:59.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:52:59 smithi176 ceph-mon[35324]: pgmap v396: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-09-16T01:53:01.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:01 smithi164 ceph-mon[35121]: pgmap v397: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-09-16T01:53:01.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:01 smithi139 ceph-mon[31175]: pgmap v397: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-09-16T01:53:01.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:01 smithi176 ceph-mon[35324]: pgmap v397: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-09-16T01:53:03.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:03 smithi164 ceph-mon[35121]: pgmap v398: 129 pgs: 129 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-09-16T01:53:03.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:03 smithi139 ceph-mon[31175]: pgmap v398: 129 pgs: 129 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-09-16T01:53:03.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:03 smithi176 ceph-mon[35324]: pgmap v398: 129 pgs: 129 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-09-16T01:53:05.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:05 smithi164 ceph-mon[35121]: pgmap v399: 129 pgs: 129 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 324 op/s 2024-09-16T01:53:05.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:05 smithi139 ceph-mon[31175]: pgmap v399: 129 pgs: 129 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 324 op/s 2024-09-16T01:53:05.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:05 smithi176 ceph-mon[35324]: pgmap v399: 129 pgs: 129 active+clean; 2.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 324 op/s 2024-09-16T01:53:07.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:07 smithi164 ceph-mon[35121]: pgmap v400: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 344 op/s 2024-09-16T01:53:07.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:07 smithi164 ceph-mon[35121]: osdmap e103: 12 total, 12 up, 12 in 2024-09-16T01:53:07.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:07 smithi139 ceph-mon[31175]: pgmap v400: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 344 op/s 2024-09-16T01:53:07.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:07 smithi139 ceph-mon[31175]: osdmap e103: 12 total, 12 up, 12 in 2024-09-16T01:53:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:07 smithi176 ceph-mon[35324]: pgmap v400: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 344 op/s 2024-09-16T01:53:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:07 smithi176 ceph-mon[35324]: osdmap e103: 12 total, 12 up, 12 in 2024-09-16T01:53:08.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:08 smithi164 ceph-mon[35121]: osdmap e104: 12 total, 12 up, 12 in 2024-09-16T01:53:08.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:08 smithi139 ceph-mon[31175]: osdmap e104: 12 total, 12 up, 12 in 2024-09-16T01:53:08.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:08 smithi176 ceph-mon[35324]: osdmap e104: 12 total, 12 up, 12 in 2024-09-16T01:53:09.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:09 smithi164 ceph-mon[35121]: pgmap v403: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 316 op/s 2024-09-16T01:53:09.422 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:09 smithi139 ceph-mon[31175]: pgmap v403: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 316 op/s 2024-09-16T01:53:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.425+0000 7f263bc5c640 1 -- 172.21.15.139:0/3393698422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2634074590 msgr2=0x7f2634074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.425+0000 7f263bc5c640 1 --2- 172.21.15.139:0/3393698422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2634074590 0x7f2634074970 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2620009860 tx=0x7f262002a3f0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.427+0000 7f263bc5c640 1 -- 172.21.15.139:0/3393698422 shutdown_connections 2024-09-16T01:53:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.427+0000 7f263bc5c640 1 --2- 172.21.15.139:0/3393698422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2634112ae0 0x7f2634114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.427+0000 7f263bc5c640 1 --2- 172.21.15.139:0/3393698422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2634074eb0 0x7f26341125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.427+0000 7f263bc5c640 1 --2- 172.21.15.139:0/3393698422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2634074590 0x7f2634074970 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.427+0000 7f263bc5c640 1 -- 172.21.15.139:0/3393698422 >> 172.21.15.139:0/3393698422 conn(0x7f26340fd7d0 msgr2=0x7f26340ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.433+0000 7f263bc5c640 1 -- 172.21.15.139:0/3393698422 shutdown_connections 2024-09-16T01:53:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.434+0000 7f263bc5c640 1 -- 172.21.15.139:0/3393698422 wait complete. 2024-09-16T01:53:09.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:09 smithi176 ceph-mon[35324]: pgmap v403: 129 pgs: 129 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 316 op/s 2024-09-16T01:53:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.435+0000 7f263bc5c640 1 Processor -- start 2024-09-16T01:53:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.439+0000 7f263bc5c640 1 -- start start 2024-09-16T01:53:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.439+0000 7f263bc5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2634074590 0x7f2634070fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.439+0000 7f263bc5c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2634074eb0 0x7f2634071500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.439+0000 7f263bc5c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2634112ae0 0x7f2634071a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.439+0000 7f263bc5c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2634106cd0 con 0x7f2634074590 2024-09-16T01:53:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.439+0000 7f263bc5c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2634106b50 con 0x7f2634112ae0 2024-09-16T01:53:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.439+0000 7f263bc5c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2634106e50 con 0x7f2634074eb0 2024-09-16T01:53:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.440+0000 7f263a1f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2634112ae0 0x7f2634071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.440+0000 7f263a1f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2634112ae0 0x7f2634071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37102/0 (socket says 172.21.15.139:37102) 2024-09-16T01:53:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.440+0000 7f263a1f9640 1 -- 172.21.15.139:0/2789256096 learned_addr learned my addr 172.21.15.139:0/2789256096 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.440+0000 7f26391f7640 1 --2- 172.21.15.139:0/2789256096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2634074eb0 0x7f2634071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.440+0000 7f26399f8640 1 --2- 172.21.15.139:0/2789256096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2634074590 0x7f2634070fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.440+0000 7f263a1f9640 1 -- 172.21.15.139:0/2789256096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2634074eb0 msgr2=0x7f2634071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.440+0000 7f263a1f9640 1 --2- 172.21.15.139:0/2789256096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2634074eb0 0x7f2634071500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.440+0000 7f263a1f9640 1 -- 172.21.15.139:0/2789256096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2634074590 msgr2=0x7f2634070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.440+0000 7f263a1f9640 1 --2- 172.21.15.139:0/2789256096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2634074590 0x7f2634070fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.440+0000 7f263a1f9640 1 -- 172.21.15.139:0/2789256096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26341104f0 con 0x7f2634112ae0 2024-09-16T01:53:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.441+0000 7f263a1f9640 1 --2- 172.21.15.139:0/2789256096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2634112ae0 0x7f2634071a40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c970 tx=0x7f262c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.441+0000 7f2632ffd640 1 -- 172.21.15.139:0/2789256096 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c007bb0 con 0x7f2634112ae0 2024-09-16T01:53:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.441+0000 7f2632ffd640 1 -- 172.21.15.139:0/2789256096 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262c007d50 con 0x7f2634112ae0 2024-09-16T01:53:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.442+0000 7f2632ffd640 1 -- 172.21.15.139:0/2789256096 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c00f450 con 0x7f2634112ae0 2024-09-16T01:53:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.442+0000 7f263bc5c640 1 -- 172.21.15.139:0/2789256096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26341107e0 con 0x7f2634112ae0 2024-09-16T01:53:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.443+0000 7f263bc5c640 1 -- 172.21.15.139:0/2789256096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2634110cb0 con 0x7f2634112ae0 2024-09-16T01:53:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.447+0000 7f263e9f3740 1 Processor -- start 2024-09-16T01:53:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.457+0000 7f263e9f3740 1 -- start start 2024-09-16T01:53:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.458+0000 7f263e9f3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddb8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.458+0000 7f263e9f3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddc3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.458+0000 7f263e9f3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109ddc60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.458+0000 7f263e9f3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56109dd1d2f0 con 0x56109ddc3cb0 2024-09-16T01:53:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.458+0000 7f263e9f3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56109dcc5270 con 0x56109ddb9410 2024-09-16T01:53:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.458+0000 7f263e9f3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56109dd192b0 con 0x56109ddb8af0 2024-09-16T01:53:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.458+0000 7f2632ffd640 1 -- 172.21.15.139:0/2789256096 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f262c038030 con 0x7f2634112ae0 2024-09-16T01:53:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.459+0000 7f2632ffd640 1 --2- 172.21.15.139:0/2789256096 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f26240a33d0 0x7f26240a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.459+0000 7f2632ffd640 1 -- 172.21.15.139:0/2789256096 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f262c0f1f10 con 0x7f2634112ae0 2024-09-16T01:53:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.459+0000 7f26399f8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddc3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.459+0000 7f26399f8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddc3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37118/0 (socket says 172.21.15.139:37118) 2024-09-16T01:53:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.459+0000 7f26399f8640 1 -- 172.21.15.139:0/2103995683 learned_addr learned my addr 172.21.15.139:0/2103995683 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.460+0000 7f26391f7640 1 --2- 172.21.15.139:0/2103995683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddb8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.460+0000 7f26399f8640 1 -- 172.21.15.139:0/2103995683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 msgr2=0x56109ddb8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.460+0000 7f263a1f9640 1 --2- 172.21.15.139:0/2789256096 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f26240a33d0 0x7f26240a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.460+0000 7f26399f8640 1 --2- 172.21.15.139:0/2103995683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddb8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.461+0000 7f26399f8640 1 -- 172.21.15.139:0/2103995683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 msgr2=0x56109ddc60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.461+0000 7f26399f8640 1 --2- 172.21.15.139:0/2103995683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109ddc60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.461+0000 7f26399f8640 1 -- 172.21.15.139:0/2103995683 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56109dcc4340 con 0x56109ddb9410 2024-09-16T01:53:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.461+0000 7f26391f7640 1 --2- 172.21.15.139:0/2103995683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddb8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:53:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.461+0000 7f26399f8640 1 --2- 172.21.15.139:0/2103995683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddc3770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f262002f3c0 tx=0x7f262002f3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26d91a65d3443cfc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.461+0000 7f263a1f9640 1 --2- 172.21.15.139:0/2789256096 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f26240a33d0 0x7f26240a5890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c940 tx=0x7f262c0f2160 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.462+0000 7f261ffff640 1 -- 172.21.15.139:0/2103995683 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620030d80 con 0x56109ddb9410 2024-09-16T01:53:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.462+0000 7f261ffff640 1 -- 172.21.15.139:0/2103995683 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2620041070 con 0x56109ddb9410 2024-09-16T01:53:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.462+0000 7f261ffff640 1 -- 172.21.15.139:0/2103995683 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620004920 con 0x56109ddb9410 2024-09-16T01:53:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.463+0000 7f263e9f3740 1 -- 172.21.15.139:0/2103995683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 msgr2=0x56109ddc3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.463+0000 7f263e9f3740 1 --2- 172.21.15.139:0/2103995683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddc3770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f262002f3c0 tx=0x7f262002f3f0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.467+0000 7f263e9f3740 1 -- 172.21.15.139:0/2103995683 shutdown_connections 2024-09-16T01:53:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.467+0000 7f263e9f3740 1 --2- 172.21.15.139:0/2103995683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109ddc60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.467+0000 7f263e9f3740 1 --2- 172.21.15.139:0/2103995683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddc3770 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.467+0000 7f263e9f3740 1 --2- 172.21.15.139:0/2103995683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddb8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.467+0000 7f263e9f3740 1 -- 172.21.15.139:0/2103995683 >> 172.21.15.139:0/2103995683 conn(0x56109ddb5060 msgr2=0x56109ddb5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.470+0000 7f263e9f3740 1 -- 172.21.15.139:0/2103995683 shutdown_connections 2024-09-16T01:53:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.473+0000 7f263e9f3740 1 -- 172.21.15.139:0/2103995683 wait complete. 2024-09-16T01:53:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.473+0000 7f263e9f3740 1 -- start start 2024-09-16T01:53:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.473+0000 7f263e9f3740 1 Processor -- start 2024-09-16T01:53:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.475+0000 7f263e9f3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddc0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.475+0000 7f263e9f3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddc11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.475+0000 7f263e9f3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109ddc16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.475+0000 7f263e9f3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56109ddc6880 con 0x56109ddc3cb0 2024-09-16T01:53:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.475+0000 7f263e9f3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56109dca6290 con 0x56109ddb9410 2024-09-16T01:53:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.475+0000 7f263e9f3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56109ddc6a00 con 0x56109ddb8af0 2024-09-16T01:53:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.475+0000 7f26391f7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddc0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.476+0000 7f26391f7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddc0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:51170/0 (socket says 172.21.15.139:51170) 2024-09-16T01:53:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.476+0000 7f26391f7640 1 -- 172.21.15.139:0/1419350841 learned_addr learned my addr 172.21.15.139:0/1419350841 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.476+0000 7f26391f7640 1 --2- 172.21.15.139:0/1419350841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109ddc16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.476+0000 7f263e9f3740 10 client.0 _ll_register_callbacks cb 0x56109dcd7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:53:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.477+0000 7f26399f8640 1 --2- 172.21.15.139:0/1419350841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddc11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.479+0000 7f26391f7640 1 -- 172.21.15.139:0/1419350841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 msgr2=0x56109ddc0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.479+0000 7f26391f7640 1 --2- 172.21.15.139:0/1419350841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddc0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.479+0000 7f26391f7640 1 -- 172.21.15.139:0/1419350841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 msgr2=0x56109ddc11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.479+0000 7f26391f7640 1 --2- 172.21.15.139:0/1419350841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddc11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.479+0000 7f26391f7640 1 -- 172.21.15.139:0/1419350841 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56109da30210 con 0x56109ddc3cb0 2024-09-16T01:53:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.480+0000 7f26391f7640 1 --2- 172.21.15.139:0/1419350841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109ddc16f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2628005fd0 tx=0x7f2628004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1750558cd932c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.481+0000 7f261d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:53:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.481+0000 7f261d7fa640 1 -- 172.21.15.139:0/1419350841 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628003970 con 0x56109ddc3cb0 2024-09-16T01:53:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.481+0000 7f261d7fa640 1 -- 172.21.15.139:0/1419350841 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f2628003b10 con 0x56109ddc3cb0 2024-09-16T01:53:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.482+0000 7f263e9f3740 1 -- 172.21.15.139:0/1419350841 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2620041070 con 0x56109ddc3cb0 2024-09-16T01:53:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.482+0000 7f261d7fa640 1 -- 172.21.15.139:0/1419350841 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f26280106a0 con 0x56109ddc3cb0 2024-09-16T01:53:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.483+0000 7f263e9f3740 10 client.16845 fetch_fsmap learned FSMap version 64 2024-09-16T01:53:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.483+0000 7f263e9f3740 1 -- 172.21.15.139:0/1419350841 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56109dafc620 con 0x56109ddc3cb0 2024-09-16T01:53:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.483+0000 7f261d7fa640 1 -- 172.21.15.139:0/1419350841 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2620041070 con 0x56109ddc3cb0 2024-09-16T01:53:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.484+0000 7f261d7fa640 1 -- 172.21.15.139:0/1419350841 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 64) ==== 8442+0+0 (secure 0 0 0) 0x7f262801c030 con 0x56109ddc3cb0 2024-09-16T01:53:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.485+0000 7f263e9f3740 10 client.16845 fetch_fsmap finished waiting for FSMap version 64 2024-09-16T01:53:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.485+0000 7f263e9f3740 10 client.16845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:53:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.485+0000 7f263e9f3740 20 client.16845 populate_metadata read hostname 'smithi139' 2024-09-16T01:53:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.485+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1419350841 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56109ddd4110 0x56109ddd6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.485+0000 7f263e9f3740 4 client.16845 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:53:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.485+0000 7f263e9f3740 1 -- 172.21.15.139:0/1419350841 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56109dd2aa20 con 0x56109ddd4110 2024-09-16T01:53:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.486+0000 7f26399f8640 1 --2- 172.21.15.139:0/1419350841 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56109ddd4110 0x56109ddd6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.496+0000 7f26399f8640 1 --2- 172.21.15.139:0/1419350841 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56109ddd4110 0x56109ddd6550 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3864518722b8a4d0 server_cookie=4ce757581c929ceb in_seq=0 out_seq=0 2024-09-16T01:53:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.496+0000 7f261d7fa640 10 client.16845 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.500+0000 7f261d7fa640 1 -- 172.21.15.139:0/1419350841 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56109dd2aa20 con 0x56109ddd4110 2024-09-16T01:53:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.500+0000 7f261d7fa640 10 client.16845 handle_command_reply: tid=0 2024-09-16T01:53:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.500+0000 7f263e9f3740 1 client.16845 shutdown 2024-09-16T01:53:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.509+0000 7f263e9f3740 1 -- 172.21.15.139:0/1419350841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 msgr2=0x56109ddc16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.509+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1419350841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109ddc16f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2628005fd0 tx=0x7f2628004a40 comp rx=0 tx=0).stop 2024-09-16T01:53:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.515+0000 7f263e9f3740 1 -- 172.21.15.139:0/1419350841 shutdown_connections 2024-09-16T01:53:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.515+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1419350841 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56109ddd4110 0x56109ddd6550 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.515+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1419350841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109ddc16f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.515+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1419350841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddc11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.515+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1419350841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddc0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.515+0000 7f263e9f3740 1 -- 172.21.15.139:0/1419350841 >> 172.21.15.139:0/1419350841 conn(0x56109ddb5060 msgr2=0x56109ddbad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.515+0000 7f261d7fa640 0 client.16845 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.516+0000 7f263e9f3740 1 -- 172.21.15.139:0/1419350841 shutdown_connections 2024-09-16T01:53:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.519+0000 7f263e9f3740 1 -- 172.21.15.139:0/1419350841 wait complete. 2024-09-16T01:53:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.523+0000 7f263e9f3740 20 client.16845 trim_cache size 0 max 16384 2024-09-16T01:53:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.559+0000 7f263e9f3740 1 Processor -- start 2024-09-16T01:53:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.560+0000 7f263e9f3740 1 -- start start 2024-09-16T01:53:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.560+0000 7f263e9f3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddc1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.560+0000 7f263e9f3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddb40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.560+0000 7f263e9f3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109dddbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.560+0000 7f263e9f3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f263411f840 con 0x56109ddc3cb0 2024-09-16T01:53:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.560+0000 7f263e9f3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2634121620 con 0x56109ddb9410 2024-09-16T01:53:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.560+0000 7f263e9f3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f263411ff70 con 0x56109ddb8af0 2024-09-16T01:53:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.562+0000 7f26399f8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddb40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.562+0000 7f26399f8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddb40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37142/0 (socket says 172.21.15.139:37142) 2024-09-16T01:53:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.562+0000 7f26399f8640 1 -- 172.21.15.139:0/1889586623 learned_addr learned my addr 172.21.15.139:0/1889586623 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.562+0000 7f26391f7640 1 --2- 172.21.15.139:0/1889586623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddc1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.563+0000 7f26391f7640 1 --2- 172.21.15.139:0/1889586623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109dddbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.564+0000 7f26399f8640 1 -- 172.21.15.139:0/1889586623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 msgr2=0x56109ddc1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.564+0000 7f26399f8640 1 --2- 172.21.15.139:0/1889586623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddc1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.564+0000 7f26399f8640 1 -- 172.21.15.139:0/1889586623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 msgr2=0x56109dddbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.564+0000 7f26399f8640 1 --2- 172.21.15.139:0/1889586623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109dddbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.564+0000 7f26399f8640 1 -- 172.21.15.139:0/1889586623 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56109ddc6410 con 0x56109ddb9410 2024-09-16T01:53:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.565+0000 7f26399f8640 1 --2- 172.21.15.139:0/1889586623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddb40b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f26200054c0 tx=0x7f2620005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f59e68d90f8b774 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.566+0000 7f261f7fe640 1 -- 172.21.15.139:0/1889586623 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26200305a0 con 0x56109ddb9410 2024-09-16T01:53:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.566+0000 7f261f7fe640 1 -- 172.21.15.139:0/1889586623 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2620041070 con 0x56109ddb9410 2024-09-16T01:53:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.566+0000 7f261f7fe640 1 -- 172.21.15.139:0/1889586623 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620003ab0 con 0x56109ddb9410 2024-09-16T01:53:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.567+0000 7f263e9f3740 1 -- 172.21.15.139:0/1889586623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 msgr2=0x56109ddb40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.567+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1889586623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddb40b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f26200054c0 tx=0x7f2620005b20 comp rx=0 tx=0).stop 2024-09-16T01:53:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.569+0000 7f263e9f3740 1 -- 172.21.15.139:0/1889586623 shutdown_connections 2024-09-16T01:53:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.569+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1889586623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109dddbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.569+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1889586623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddb40b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.569+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1889586623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddc1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.569+0000 7f263e9f3740 1 -- 172.21.15.139:0/1889586623 >> 172.21.15.139:0/1889586623 conn(0x56109ddb5060 msgr2=0x56109ddb39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.574+0000 7f263e9f3740 1 -- 172.21.15.139:0/1889586623 shutdown_connections 2024-09-16T01:53:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.575+0000 7f263e9f3740 1 -- 172.21.15.139:0/1889586623 wait complete. 2024-09-16T01:53:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.576+0000 7f263e9f3740 1 -- start start 2024-09-16T01:53:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.576+0000 7f263e9f3740 1 Processor -- start 2024-09-16T01:53:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.576+0000 7f263e9f3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddcbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.576+0000 7f263e9f3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddcc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.576+0000 7f263e9f3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109ddc8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.576+0000 7f263e9f3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2634120e90 con 0x56109ddc3cb0 2024-09-16T01:53:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.576+0000 7f263e9f3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2634120700 con 0x56109ddb9410 2024-09-16T01:53:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.576+0000 7f263e9f3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f263411d8d0 con 0x56109ddb8af0 2024-09-16T01:53:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.576+0000 7f263e9f3740 10 client.0 _ll_register_callbacks cb 0x7f26280104d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:53:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.578+0000 7f26399f8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddcc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.578+0000 7f26399f8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddcc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37148/0 (socket says 172.21.15.139:37148) 2024-09-16T01:53:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.578+0000 7f26399f8640 1 -- 172.21.15.139:0/1665762783 learned_addr learned my addr 172.21.15.139:0/1665762783 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.580+0000 7f26391f7640 1 --2- 172.21.15.139:0/1665762783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddcbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.580+0000 7f26391f7640 1 --2- 172.21.15.139:0/1665762783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109ddc8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.580+0000 7f26399f8640 1 -- 172.21.15.139:0/1665762783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 msgr2=0x56109ddcbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.580+0000 7f26399f8640 1 --2- 172.21.15.139:0/1665762783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddcbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.580+0000 7f26399f8640 1 -- 172.21.15.139:0/1665762783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 msgr2=0x56109ddc8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.580+0000 7f26399f8640 1 --2- 172.21.15.139:0/1665762783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109ddc8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.580+0000 7f26399f8640 1 -- 172.21.15.139:0/1665762783 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56109ddcc7c0 con 0x56109ddb9410 2024-09-16T01:53:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.582+0000 7f26399f8640 1 --2- 172.21.15.139:0/1665762783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddcc1d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f262002a910 tx=0x7f262002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d386204ac76ff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.582+0000 7f261d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:53:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.586+0000 7f261d7fa640 1 -- 172.21.15.139:0/1665762783 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262005f060 con 0x56109ddb9410 2024-09-16T01:53:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.586+0000 7f263e9f3740 1 -- 172.21.15.139:0/1665762783 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2620041070 con 0x56109ddb9410 2024-09-16T01:53:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.586+0000 7f261d7fa640 1 -- 172.21.15.139:0/1665762783 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f26200305a0 con 0x56109ddb9410 2024-09-16T01:53:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.588+0000 7f261d7fa640 1 -- 172.21.15.139:0/1665762783 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f26200307a0 con 0x56109ddb9410 2024-09-16T01:53:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.592+0000 7f263e9f3740 10 client.26291 fetch_fsmap learned FSMap version 64 2024-09-16T01:53:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.592+0000 7f263e9f3740 1 -- 172.21.15.139:0/1665762783 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56109de3f360 con 0x56109ddb9410 2024-09-16T01:53:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.593+0000 7f261d7fa640 1 -- 172.21.15.139:0/1665762783 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2620041070 con 0x56109ddb9410 2024-09-16T01:53:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.593+0000 7f261d7fa640 1 -- 172.21.15.139:0/1665762783 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 64) ==== 8442+0+0 (secure 0 0 0) 0x7f2620059070 con 0x56109ddb9410 2024-09-16T01:53:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.594+0000 7f263e9f3740 10 client.26291 fetch_fsmap finished waiting for FSMap version 64 2024-09-16T01:53:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.594+0000 7f263e9f3740 10 client.26291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:53:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.594+0000 7f263e9f3740 20 client.26291 populate_metadata read hostname 'smithi139' 2024-09-16T01:53:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.594+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1665762783 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56109de3f590 0x56109de419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.594+0000 7f263e9f3740 4 client.26291 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:53:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.594+0000 7f263e9f3740 1 -- 172.21.15.139:0/1665762783 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56109dd24d70 con 0x56109de3f590 2024-09-16T01:53:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.595+0000 7f26391f7640 1 --2- 172.21.15.139:0/1665762783 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56109de3f590 0x56109de419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.596+0000 7f261d7fa640 10 client.26291 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.596+0000 7f26391f7640 1 --2- 172.21.15.139:0/1665762783 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56109de3f590 0x56109de419b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21c73d29b7e49071 server_cookie=a05cee35e8c8432 in_seq=0 out_seq=0 2024-09-16T01:53:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.740+0000 7f261d7fa640 1 -- 172.21.15.139:0/1665762783 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56109dd24d70 con 0x56109de3f590 2024-09-16T01:53:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.740+0000 7f261d7fa640 10 client.26291 handle_command_reply: tid=0 2024-09-16T01:53:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.740+0000 7f263e9f3740 1 client.26291 shutdown 2024-09-16T01:53:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.740+0000 7f263e9f3740 1 -- 172.21.15.139:0/1665762783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 msgr2=0x56109ddcc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.740+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1665762783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddcc1d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f262002a910 tx=0x7f262002a3f0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.740+0000 7f263e9f3740 1 -- 172.21.15.139:0/1665762783 shutdown_connections 2024-09-16T01:53:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.740+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1665762783 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56109de3f590 0x56109de419b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.740+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1665762783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56109ddc3cb0 0x56109ddc8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.740+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1665762783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56109ddb9410 0x56109ddcc1d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.740+0000 7f263e9f3740 1 --2- 172.21.15.139:0/1665762783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56109ddb8af0 0x56109ddcbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.740+0000 7f263e9f3740 1 -- 172.21.15.139:0/1665762783 >> 172.21.15.139:0/1665762783 conn(0x56109ddb5060 msgr2=0x56109ddb41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.741+0000 7f261d7fa640 0 client.26291 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.741+0000 7f263e9f3740 1 -- 172.21.15.139:0/1665762783 shutdown_connections 2024-09-16T01:53:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.741+0000 7f263e9f3740 1 -- 172.21.15.139:0/1665762783 wait complete. 2024-09-16T01:53:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.741+0000 7f263e9f3740 20 client.26291 trim_cache size 0 max 16384 2024-09-16T01:53:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:53:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:53:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:53:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:53:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.741+0000 7f263bc5c640 1 -- 172.21.15.139:0/2789256096 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f26240a33d0 msgr2=0x7f26240a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.741+0000 7f263bc5c640 1 --2- 172.21.15.139:0/2789256096 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f26240a33d0 0x7f26240a5890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c940 tx=0x7f262c0f2160 comp rx=0 tx=0).stop 2024-09-16T01:53:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.742+0000 7f263bc5c640 1 -- 172.21.15.139:0/2789256096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2634112ae0 msgr2=0x7f2634071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.742+0000 7f263bc5c640 1 --2- 172.21.15.139:0/2789256096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2634112ae0 0x7f2634071a40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c970 tx=0x7f262c00ce30 comp rx=0 tx=0).stop 2024-09-16T01:53:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.742+0000 7f263bc5c640 1 -- 172.21.15.139:0/2789256096 shutdown_connections 2024-09-16T01:53:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.742+0000 7f263bc5c640 1 --2- 172.21.15.139:0/2789256096 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f26240a33d0 0x7f26240a5890 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.742+0000 7f263bc5c640 1 --2- 172.21.15.139:0/2789256096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2634112ae0 0x7f2634071a40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.742+0000 7f263bc5c640 1 --2- 172.21.15.139:0/2789256096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2634074eb0 0x7f2634071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.742+0000 7f263bc5c640 1 --2- 172.21.15.139:0/2789256096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2634074590 0x7f2634070fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.742+0000 7f263bc5c640 1 -- 172.21.15.139:0/2789256096 >> 172.21.15.139:0/2789256096 conn(0x7f26340fd7d0 msgr2=0x7f26340feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.742+0000 7f263bc5c640 1 -- 172.21.15.139:0/2789256096 shutdown_connections 2024-09-16T01:53:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:09.742+0000 7f263bc5c640 1 -- 172.21.15.139:0/2789256096 wait complete. 2024-09-16T01:53:11.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:11 smithi139 ceph-mon[31175]: pgmap v404: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 275 op/s 2024-09-16T01:53:11.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:11 smithi176 ceph-mon[35324]: pgmap v404: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 275 op/s 2024-09-16T01:53:11.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:11 smithi164 ceph-mon[35121]: pgmap v404: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 275 op/s 2024-09-16T01:53:13.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:13 smithi139 ceph-mon[31175]: pgmap v405: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 317 op/s 2024-09-16T01:53:13.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:13 smithi176 ceph-mon[35324]: pgmap v405: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 317 op/s 2024-09-16T01:53:13.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:13 smithi164 ceph-mon[35121]: pgmap v405: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 317 op/s 2024-09-16T01:53:15.335 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:15 smithi164 ceph-mon[35121]: pgmap v406: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-09-16T01:53:15.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:15 smithi139 ceph-mon[31175]: pgmap v406: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-09-16T01:53:15.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:15 smithi176 ceph-mon[35324]: pgmap v406: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:15 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e[64034]: *** Caught signal (Segmentation fault) ** 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:15 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e[64034]: in thread 7f95c7185640 thread_name:MR_Finisher 2024-09-16T01:53:15.795 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:15 smithi164 podman[70622]: 2024-09-16 01:53:15.523980786 +0000 UTC m=+0.040022857 container died d9cc4c9246da86a26f00a2258ce9ee4a44803380e559083b7c8ca84cf643c881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-09-16T01:53:17.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:17 smithi139 ceph-mon[31175]: pgmap v407: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 250 op/s 2024-09-16T01:53:17.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:17 smithi176 ceph-mon[35324]: pgmap v407: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 250 op/s 2024-09-16T01:53:17.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:17 smithi164 ceph-mon[35121]: pgmap v407: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 250 op/s 2024-09-16T01:53:19.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:19 smithi139 ceph-mon[31175]: pgmap v408: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-09-16T01:53:19.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:19 smithi176 ceph-mon[35324]: pgmap v408: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-09-16T01:53:19.534 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:19 smithi164 ceph-mon[35121]: pgmap v408: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-09-16T01:53:21.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:21 smithi139 ceph-mon[31175]: pgmap v409: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-09-16T01:53:21.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:21 smithi176 ceph-mon[35324]: pgmap v409: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-09-16T01:53:21.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:21 smithi164 ceph-mon[35121]: pgmap v409: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-09-16T01:53:22.795 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:22 smithi164 podman[70622]: 2024-09-16 01:53:22.312266842 +0000 UTC m=+6.828308904 container remove d9cc4c9246da86a26f00a2258ce9ee4a44803380e559083b7c8ca84cf643c881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True) 2024-09-16T01:53:22.796 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:22 smithi164 systemd[1]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.e.service: Main process exited, code=exited, status=139/n/a 2024-09-16T01:53:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.893+0000 7f57c72d9640 1 -- 172.21.15.139:0/1538727886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57c01026f0 msgr2=0x7f57c010d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.893+0000 7f57c72d9640 1 --2- 172.21.15.139:0/1538727886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57c01026f0 0x7f57c010d110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f57b800ade0 tx=0x7f57b802b810 comp rx=0 tx=0).stop 2024-09-16T01:53:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.895+0000 7f57c72d9640 1 -- 172.21.15.139:0/1538727886 shutdown_connections 2024-09-16T01:53:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.895+0000 7f57c72d9640 1 --2- 172.21.15.139:0/1538727886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57c01026f0 0x7f57c010d110 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.895+0000 7f57c72d9640 1 --2- 172.21.15.139:0/1538727886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57c0101d70 0x7f57c01021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.895+0000 7f57c72d9640 1 --2- 172.21.15.139:0/1538727886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57c0100490 0x7f57c0100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.895+0000 7f57c72d9640 1 -- 172.21.15.139:0/1538727886 >> 172.21.15.139:0/1538727886 conn(0x7f57c00770f0 msgr2=0x7f57c00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.897+0000 7f57c72d9640 1 -- 172.21.15.139:0/1538727886 shutdown_connections 2024-09-16T01:53:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.899+0000 7f57c72d9640 1 -- 172.21.15.139:0/1538727886 wait complete. 2024-09-16T01:53:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.900+0000 7f57c72d9640 1 Processor -- start 2024-09-16T01:53:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.904+0000 7f57c72d9640 1 -- start start 2024-09-16T01:53:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.904+0000 7f57c72d9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57c0100490 0x7f57c01161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.904+0000 7f57c72d9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57c0101d70 0x7f57c0116700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.904+0000 7f57c72d9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57c01026f0 0x7f57c01112d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.904+0000 7f57c72d9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f57c010fb50 con 0x7f57c0101d70 2024-09-16T01:53:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.904+0000 7f57c72d9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57c010f9d0 con 0x7f57c0100490 2024-09-16T01:53:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.904+0000 7f57c72d9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f57c010fcd0 con 0x7f57c01026f0 2024-09-16T01:53:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.906+0000 7f57c5876640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57c01026f0 0x7f57c01112d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.906+0000 7f57c5876640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57c01026f0 0x7f57c01112d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:58696/0 (socket says 172.21.15.139:58696) 2024-09-16T01:53:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.906+0000 7f57c5876640 1 -- 172.21.15.139:0/1058790727 learned_addr learned my addr 172.21.15.139:0/1058790727 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.906+0000 7f57c4874640 1 --2- 172.21.15.139:0/1058790727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57c0101d70 0x7f57c0116700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.906+0000 7f57c5876640 1 -- 172.21.15.139:0/1058790727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57c0100490 msgr2=0x7f57c01161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.906+0000 7f57c5876640 1 --2- 172.21.15.139:0/1058790727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57c0100490 0x7f57c01161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.906+0000 7f57c5876640 1 -- 172.21.15.139:0/1058790727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57c0101d70 msgr2=0x7f57c0116700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.906+0000 7f57c5876640 1 --2- 172.21.15.139:0/1058790727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57c0101d70 0x7f57c0116700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.906+0000 7f57c5876640 1 -- 172.21.15.139:0/1058790727 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c0111b90 con 0x7f57c01026f0 2024-09-16T01:53:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.911+0000 7f57c5876640 1 --2- 172.21.15.139:0/1058790727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57c01026f0 0x7f57c01112d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f57b802be00 tx=0x7f57b8003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.912+0000 7f57b67fc640 1 -- 172.21.15.139:0/1058790727 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b8039050 con 0x7f57c01026f0 2024-09-16T01:53:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.912+0000 7f57b67fc640 1 -- 172.21.15.139:0/1058790727 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57b8045070 con 0x7f57c01026f0 2024-09-16T01:53:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.912+0000 7f57b67fc640 1 -- 172.21.15.139:0/1058790727 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b8040450 con 0x7f57c01026f0 2024-09-16T01:53:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.913+0000 7f57c72d9640 1 -- 172.21.15.139:0/1058790727 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c0111e20 con 0x7f57c01026f0 2024-09-16T01:53:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.913+0000 7f57c72d9640 1 -- 172.21.15.139:0/1058790727 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c011bed0 con 0x7f57c01026f0 2024-09-16T01:53:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.917+0000 7f57ca06e740 1 Processor -- start 2024-09-16T01:53:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.917+0000 7f57ca06e740 1 -- start start 2024-09-16T01:53:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.919+0000 7f57b67fc640 1 -- 172.21.15.139:0/1058790727 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f57b8032070 con 0x7f57c01026f0 2024-09-16T01:53:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.919+0000 7f57ca06e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad829f9af0 0x55ad829f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.920+0000 7f57ca06e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829fa410 0x55ad82a04770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.920+0000 7f57ca06e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad82a04cb0 0x55ad82a070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.920+0000 7f57ca06e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ad8295e2f0 con 0x55ad829f9af0 2024-09-16T01:53:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.920+0000 7f57ca06e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ad82906270 con 0x55ad82a04cb0 2024-09-16T01:53:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.920+0000 7f57ca06e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ad8295a2b0 con 0x55ad829fa410 2024-09-16T01:53:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.920+0000 7f57b67fc640 1 --2- 172.21.15.139:0/1058790727 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f57ac0a3600 0x7f57ac0a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.921+0000 7f57b67fc640 1 -- 172.21.15.139:0/1058790727 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f57b80f0170 con 0x7f57c01026f0 2024-09-16T01:53:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.925+0000 7f57c4874640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad82a04cb0 0x55ad82a070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.925+0000 7f57c4874640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad82a04cb0 0x55ad82a070a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:46226/0 (socket says 172.21.15.139:46226) 2024-09-16T01:53:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.925+0000 7f57c4874640 1 -- 172.21.15.139:0/563623293 learned_addr learned my addr 172.21.15.139:0/563623293 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.925+0000 7f57c4874640 1 --2- 172.21.15.139:0/563623293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad829f9af0 0x55ad829f9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.925+0000 7f57c5075640 1 --2- 172.21.15.139:0/563623293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829fa410 0x55ad82a04770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.925+0000 7f57c4874640 1 -- 172.21.15.139:0/563623293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829fa410 msgr2=0x55ad82a04770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.925+0000 7f57c4874640 1 --2- 172.21.15.139:0/563623293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829fa410 0x55ad82a04770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.925+0000 7f57c4874640 1 -- 172.21.15.139:0/563623293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad829f9af0 msgr2=0x55ad829f9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.925+0000 7f57c4874640 1 --2- 172.21.15.139:0/563623293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad829f9af0 0x55ad829f9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.926+0000 7f57c4874640 1 -- 172.21.15.139:0/563623293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad82905340 con 0x55ad82a04cb0 2024-09-16T01:53:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.930+0000 7f57c4874640 1 --2- 172.21.15.139:0/563623293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad82a04cb0 0x55ad82a070a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009fc0 tx=0x7f57bc0097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=668eafcd293669b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.931+0000 7f57c5876640 1 --2- 172.21.15.139:0/1058790727 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f57ac0a3600 0x7f57ac0a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.931+0000 7f57ab7fe640 1 -- 172.21.15.139:0/563623293 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc004030 con 0x55ad82a04cb0 2024-09-16T01:53:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.931+0000 7f57ab7fe640 1 -- 172.21.15.139:0/563623293 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57bc005500 con 0x55ad82a04cb0 2024-09-16T01:53:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.932+0000 7f57ca06e740 1 -- 172.21.15.139:0/563623293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad82a04cb0 msgr2=0x55ad82a070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.932+0000 7f57ca06e740 1 --2- 172.21.15.139:0/563623293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad82a04cb0 0x55ad82a070a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009fc0 tx=0x7f57bc0097a0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.936+0000 7f57ca06e740 1 -- 172.21.15.139:0/563623293 shutdown_connections 2024-09-16T01:53:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.936+0000 7f57ca06e740 1 --2- 172.21.15.139:0/563623293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad82a04cb0 0x55ad82a070a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.936+0000 7f57ca06e740 1 --2- 172.21.15.139:0/563623293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829fa410 0x55ad82a04770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.936+0000 7f57ca06e740 1 --2- 172.21.15.139:0/563623293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad829f9af0 0x55ad829f9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.936+0000 7f57ca06e740 1 -- 172.21.15.139:0/563623293 >> 172.21.15.139:0/563623293 conn(0x55ad829f6060 msgr2=0x55ad829f6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.937+0000 7f57c5876640 1 --2- 172.21.15.139:0/1058790727 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f57ac0a3600 0x7f57ac0a5ac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f57b80eda60 tx=0x7f57b802b810 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.939+0000 7f57ca06e740 1 -- 172.21.15.139:0/563623293 shutdown_connections 2024-09-16T01:53:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.940+0000 7f57ca06e740 1 -- 172.21.15.139:0/563623293 wait complete. 2024-09-16T01:53:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.941+0000 7f57ca06e740 1 -- start start 2024-09-16T01:53:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.941+0000 7f57ca06e740 1 Processor -- start 2024-09-16T01:53:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.941+0000 7f57ca06e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829f9af0 0x55ad82a01ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.941+0000 7f57ca06e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829fa410 0x55ad82a02000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.941+0000 7f57ca06e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a02540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.942+0000 7f57ca06e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ad82a078d0 con 0x55ad82a04cb0 2024-09-16T01:53:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.942+0000 7f57ca06e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ad828e7290 con 0x55ad829fa410 2024-09-16T01:53:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.942+0000 7f57ca06e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ad82a07a50 con 0x55ad829f9af0 2024-09-16T01:53:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.942+0000 7f57ca06e740 10 client.0 _ll_register_callbacks cb 0x55ad82918140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:53:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.942+0000 7f57c5075640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829fa410 0x55ad82a02000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.942+0000 7f57c5075640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829fa410 0x55ad82a02000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:46236/0 (socket says 172.21.15.139:46236) 2024-09-16T01:53:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.942+0000 7f57c5075640 1 -- 172.21.15.139:0/2894458769 learned_addr learned my addr 172.21.15.139:0/2894458769 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.942+0000 7f57c4874640 1 --2- 172.21.15.139:0/2894458769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829f9af0 0x55ad82a01ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.942+0000 7f57c4874640 1 --2- 172.21.15.139:0/2894458769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a02540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.944+0000 7f57c4874640 1 -- 172.21.15.139:0/2894458769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829fa410 msgr2=0x55ad82a02000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.944+0000 7f57c4874640 1 --2- 172.21.15.139:0/2894458769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829fa410 0x55ad82a02000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.944+0000 7f57c4874640 1 -- 172.21.15.139:0/2894458769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 msgr2=0x55ad82a02540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.945+0000 7f57c4874640 1 --2- 172.21.15.139:0/2894458769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a02540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.945+0000 7f57c4874640 1 -- 172.21.15.139:0/2894458769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad82671210 con 0x55ad829f9af0 2024-09-16T01:53:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.949+0000 7f57c4874640 1 --2- 172.21.15.139:0/2894458769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829f9af0 0x55ad82a01ac0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f57bc001eb0 tx=0x7f57bc0049e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4881c18adb8d2281 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.949+0000 7f57a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:53:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.950+0000 7f57a8ff9640 1 -- 172.21.15.139:0/2894458769 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc027040 con 0x55ad829f9af0 2024-09-16T01:53:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.950+0000 7f57ca06e740 1 -- 172.21.15.139:0/2894458769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57bc005500 con 0x55ad829f9af0 2024-09-16T01:53:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.950+0000 7f57a8ff9640 1 -- 172.21.15.139:0/2894458769 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f57bc01e760 con 0x55ad829f9af0 2024-09-16T01:53:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.950+0000 7f57a8ff9640 1 -- 172.21.15.139:0/2894458769 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f57bc01e960 con 0x55ad829f9af0 2024-09-16T01:53:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.953+0000 7f57ca06e740 10 client.26266 fetch_fsmap learned FSMap version 64 2024-09-16T01:53:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.953+0000 7f57ca06e740 1 -- 172.21.15.139:0/2894458769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55ad8273d620 con 0x55ad829f9af0 2024-09-16T01:53:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.953+0000 7f57a8ff9640 1 -- 172.21.15.139:0/2894458769 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57bc005500 con 0x55ad829f9af0 2024-09-16T01:53:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.957+0000 7f57a8ff9640 1 -- 172.21.15.139:0/2894458769 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 64) ==== 8442+0+0 (secure 0 0 0) 0x7f57bc02cc10 con 0x55ad829f9af0 2024-09-16T01:53:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.957+0000 7f57ca06e740 10 client.26266 fetch_fsmap finished waiting for FSMap version 64 2024-09-16T01:53:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.957+0000 7f57ca06e740 10 client.26266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:53:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.957+0000 7f57ca06e740 20 client.26266 populate_metadata read hostname 'smithi139' 2024-09-16T01:53:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.957+0000 7f57ca06e740 1 --2- 172.21.15.139:0/2894458769 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ad82a25c00 0x55ad82a28020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.957+0000 7f57ca06e740 4 client.26266 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:53:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.957+0000 7f57ca06e740 1 -- 172.21.15.139:0/2894458769 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad8296ba20 con 0x55ad82a25c00 2024-09-16T01:53:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.959+0000 7f57c5075640 1 --2- 172.21.15.139:0/2894458769 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ad82a25c00 0x55ad82a28020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.962+0000 7f57c5075640 1 --2- 172.21.15.139:0/2894458769 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ad82a25c00 0x55ad82a28020 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4e6c1217f8b52a5 server_cookie=acd306437137187a in_seq=0 out_seq=0 2024-09-16T01:53:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.963+0000 7f57a8ff9640 10 client.26266 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.964+0000 7f57a8ff9640 1 -- 172.21.15.139:0/2894458769 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad8296ba20 con 0x55ad82a25c00 2024-09-16T01:53:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.964+0000 7f57a8ff9640 10 client.26266 handle_command_reply: tid=0 2024-09-16T01:53:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.966+0000 7f57ca06e740 1 client.26266 shutdown 2024-09-16T01:53:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.972+0000 7f57ca06e740 1 -- 172.21.15.139:0/2894458769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829f9af0 msgr2=0x55ad82a01ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.972+0000 7f57ca06e740 1 --2- 172.21.15.139:0/2894458769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829f9af0 0x55ad82a01ac0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f57bc001eb0 tx=0x7f57bc0049e0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.976+0000 7f57ca06e740 1 -- 172.21.15.139:0/2894458769 shutdown_connections 2024-09-16T01:53:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.976+0000 7f57ca06e740 1 --2- 172.21.15.139:0/2894458769 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ad82a25c00 0x55ad82a28020 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.976+0000 7f57ca06e740 1 --2- 172.21.15.139:0/2894458769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a02540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.976+0000 7f57ca06e740 1 --2- 172.21.15.139:0/2894458769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829fa410 0x55ad82a02000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.976+0000 7f57ca06e740 1 --2- 172.21.15.139:0/2894458769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829f9af0 0x55ad82a01ac0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.976+0000 7f57ca06e740 1 -- 172.21.15.139:0/2894458769 >> 172.21.15.139:0/2894458769 conn(0x55ad829f6060 msgr2=0x55ad829fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.977+0000 7f57a8ff9640 0 client.26266 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.984+0000 7f57ca06e740 1 -- 172.21.15.139:0/2894458769 shutdown_connections 2024-09-16T01:53:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.984+0000 7f57ca06e740 1 -- 172.21.15.139:0/2894458769 wait complete. 2024-09-16T01:53:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:22.985+0000 7f57ca06e740 20 client.26266 trim_cache size 0 max 16384 2024-09-16T01:53:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.020+0000 7f57ca06e740 1 Processor -- start 2024-09-16T01:53:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.020+0000 7f57ca06e740 1 -- start start 2024-09-16T01:53:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.020+0000 7f57ca06e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829f9af0 0x55ad82a03320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.020+0000 7f57ca06e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829fa410 0x55ad82a02370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.020+0000 7f57ca06e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a2d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.020+0000 7f57ca06e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f57a40102c0 con 0x55ad82a04cb0 2024-09-16T01:53:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.020+0000 7f57ca06e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57a40120a0 con 0x55ad829f9af0 2024-09-16T01:53:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.020+0000 7f57ca06e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f57a40109f0 con 0x55ad829fa410 2024-09-16T01:53:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.023+0000 7f57c4874640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a2d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.023+0000 7f57c4874640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a2d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39758/0 (socket says 172.21.15.139:39758) 2024-09-16T01:53:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.023+0000 7f57c4874640 1 -- 172.21.15.139:0/935715926 learned_addr learned my addr 172.21.15.139:0/935715926 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.023+0000 7f57c4874640 1 --2- 172.21.15.139:0/935715926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829f9af0 0x55ad82a03320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.025+0000 7f57c4874640 1 -- 172.21.15.139:0/935715926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829fa410 msgr2=0x55ad82a02370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:53:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.025+0000 7f57c4874640 1 --2- 172.21.15.139:0/935715926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829fa410 0x55ad82a02370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.025+0000 7f57c4874640 1 -- 172.21.15.139:0/935715926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 msgr2=0x55ad82a2d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.025+0000 7f57c4874640 1 --2- 172.21.15.139:0/935715926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a2d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.025+0000 7f57c4874640 1 -- 172.21.15.139:0/935715926 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad829fa240 con 0x55ad829f9af0 2024-09-16T01:53:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.029+0000 7f57c4874640 1 --2- 172.21.15.139:0/935715926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829f9af0 0x55ad82a03320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00f7c0 tx=0x7f57bc00f660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b5c1f1b8c8d74c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.034+0000 7f57aaffd640 1 -- 172.21.15.139:0/935715926 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc0097a0 con 0x55ad829f9af0 2024-09-16T01:53:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.034+0000 7f57aaffd640 1 -- 172.21.15.139:0/935715926 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57bc023070 con 0x55ad829f9af0 2024-09-16T01:53:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.034+0000 7f57aaffd640 1 -- 172.21.15.139:0/935715926 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00f8e0 con 0x55ad829f9af0 2024-09-16T01:53:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.036+0000 7f57ca06e740 1 -- 172.21.15.139:0/935715926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829f9af0 msgr2=0x55ad82a03320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.036+0000 7f57ca06e740 1 --2- 172.21.15.139:0/935715926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829f9af0 0x55ad82a03320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00f7c0 tx=0x7f57bc00f660 comp rx=0 tx=0).stop 2024-09-16T01:53:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.037+0000 7f57ca06e740 1 -- 172.21.15.139:0/935715926 shutdown_connections 2024-09-16T01:53:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.037+0000 7f57ca06e740 1 --2- 172.21.15.139:0/935715926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a2d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.037+0000 7f57ca06e740 1 --2- 172.21.15.139:0/935715926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad829fa410 0x55ad82a02370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.037+0000 7f57ca06e740 1 --2- 172.21.15.139:0/935715926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829f9af0 0x55ad82a03320 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.037+0000 7f57ca06e740 1 -- 172.21.15.139:0/935715926 >> 172.21.15.139:0/935715926 conn(0x55ad829f6060 msgr2=0x55ad82a00a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.041+0000 7f57ca06e740 1 -- 172.21.15.139:0/935715926 shutdown_connections 2024-09-16T01:53:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.041+0000 7f57ca06e740 1 -- 172.21.15.139:0/935715926 wait complete. 2024-09-16T01:53:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.041+0000 7f57ca06e740 1 -- start start 2024-09-16T01:53:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.041+0000 7f57ca06e740 1 Processor -- start 2024-09-16T01:53:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.043+0000 7f57ca06e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829fa410 0x55ad82a0d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.043+0000 7f57ca06e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a05a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.043+0000 7f57ca06e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad82a05fc0 0x55ad82a06440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.043+0000 7f57ca06e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4011910 con 0x55ad82a04cb0 2024-09-16T01:53:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.043+0000 7f57ca06e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4011180 con 0x55ad829fa410 2024-09-16T01:53:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.043+0000 7f57ca06e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f57a400e2c0 con 0x55ad82a05fc0 2024-09-16T01:53:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.043+0000 7f57ca06e740 10 client.0 _ll_register_callbacks cb 0x7f57bc009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:53:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.044+0000 7f57c5075640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a05a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.044+0000 7f57c5075640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a05a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39768/0 (socket says 172.21.15.139:39768) 2024-09-16T01:53:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.044+0000 7f57c5075640 1 -- 172.21.15.139:0/509046449 learned_addr learned my addr 172.21.15.139:0/509046449 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.044+0000 7f57c5075640 1 -- 172.21.15.139:0/509046449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad82a05fc0 msgr2=0x55ad82a06440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:53:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.044+0000 7f57c5075640 1 --2- 172.21.15.139:0/509046449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad82a05fc0 0x55ad82a06440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.044+0000 7f57c5075640 1 -- 172.21.15.139:0/509046449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829fa410 msgr2=0x55ad82a0d540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:53:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.044+0000 7f57c5075640 1 --2- 172.21.15.139:0/509046449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829fa410 0x55ad82a0d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.045+0000 7f57c5075640 1 -- 172.21.15.139:0/509046449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad82a0b180 con 0x55ad82a04cb0 2024-09-16T01:53:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.045+0000 7f57c5075640 1 --2- 172.21.15.139:0/509046449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a05a80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f57b0006de0 tx=0x7f57b000b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc5b4b0bc9d8fcd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.045+0000 7f57a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:53:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.045+0000 7f57a8ff9640 1 -- 172.21.15.139:0/509046449 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0015ae0 con 0x55ad82a04cb0 2024-09-16T01:53:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.046+0000 7f57a8ff9640 1 -- 172.21.15.139:0/509046449 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f57b0015c80 con 0x55ad82a04cb0 2024-09-16T01:53:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.050+0000 7f57ca06e740 1 -- 172.21.15.139:0/509046449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57bc023070 con 0x55ad82a04cb0 2024-09-16T01:53:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.052+0000 7f57a8ff9640 1 -- 172.21.15.139:0/509046449 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f57b0023430 con 0x55ad82a04cb0 2024-09-16T01:53:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.057+0000 7f57ca06e740 10 client.16866 fetch_fsmap learned FSMap version 64 2024-09-16T01:53:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.057+0000 7f57ca06e740 1 -- 172.21.15.139:0/509046449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55ad82a90d00 con 0x55ad82a04cb0 2024-09-16T01:53:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.060+0000 7f57a8ff9640 1 -- 172.21.15.139:0/509046449 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57bc023070 con 0x55ad82a04cb0 2024-09-16T01:53:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.060+0000 7f57a8ff9640 1 -- 172.21.15.139:0/509046449 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 64) ==== 8442+0+0 (secure 0 0 0) 0x7f57b002d690 con 0x55ad82a04cb0 2024-09-16T01:53:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.060+0000 7f57ca06e740 10 client.16866 fetch_fsmap finished waiting for FSMap version 64 2024-09-16T01:53:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.060+0000 7f57ca06e740 10 client.16866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:53:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.060+0000 7f57ca06e740 20 client.16866 populate_metadata read hostname 'smithi139' 2024-09-16T01:53:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.060+0000 7f57ca06e740 1 --2- 172.21.15.139:0/509046449 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ad82a91520 0x55ad82a93940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.060+0000 7f57ca06e740 4 client.16866 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:53:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.060+0000 7f57ca06e740 1 -- 172.21.15.139:0/509046449 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad82965d70 con 0x55ad82a91520 2024-09-16T01:53:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.061+0000 7f57c4874640 1 --2- 172.21.15.139:0/509046449 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ad82a91520 0x55ad82a93940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.066+0000 7f57c4874640 1 --2- 172.21.15.139:0/509046449 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ad82a91520 0x55ad82a93940 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=291ce26a2a365e1c server_cookie=56cb2063084b653e in_seq=0 out_seq=0 2024-09-16T01:53:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.066+0000 7f57a8ff9640 10 client.16866 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.186+0000 7f57a8ff9640 1 -- 172.21.15.139:0/509046449 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad82965d70 con 0x55ad82a91520 2024-09-16T01:53:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.186+0000 7f57a8ff9640 10 client.16866 handle_command_reply: tid=0 2024-09-16T01:53:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.186+0000 7f57ca06e740 1 client.16866 shutdown 2024-09-16T01:53:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.187+0000 7f57ca06e740 1 -- 172.21.15.139:0/509046449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 msgr2=0x55ad82a05a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.187+0000 7f57ca06e740 1 --2- 172.21.15.139:0/509046449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a05a80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f57b0006de0 tx=0x7f57b000b9b0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.187+0000 7f57ca06e740 1 -- 172.21.15.139:0/509046449 shutdown_connections 2024-09-16T01:53:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.187+0000 7f57ca06e740 1 --2- 172.21.15.139:0/509046449 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ad82a91520 0x55ad82a93940 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.187+0000 7f57ca06e740 1 --2- 172.21.15.139:0/509046449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad82a05fc0 0x55ad82a06440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.187+0000 7f57a8ff9640 0 client.16866 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.187+0000 7f57ca06e740 1 --2- 172.21.15.139:0/509046449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ad82a04cb0 0x55ad82a05a80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.187+0000 7f57ca06e740 1 --2- 172.21.15.139:0/509046449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad829fa410 0x55ad82a0d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.187+0000 7f57ca06e740 1 -- 172.21.15.139:0/509046449 >> 172.21.15.139:0/509046449 conn(0x55ad829f6060 msgr2=0x55ad82a00240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.187+0000 7f57ca06e740 1 -- 172.21.15.139:0/509046449 shutdown_connections 2024-09-16T01:53:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.187+0000 7f57ca06e740 1 -- 172.21.15.139:0/509046449 wait complete. 2024-09-16T01:53:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.187+0000 7f57ca06e740 20 client.16866 trim_cache size 0 max 16384 2024-09-16T01:53:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:53:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:53:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:53:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:53:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.188+0000 7f57c72d9640 1 -- 172.21.15.139:0/1058790727 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f57ac0a3600 msgr2=0x7f57ac0a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.188+0000 7f57c72d9640 1 --2- 172.21.15.139:0/1058790727 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f57ac0a3600 0x7f57ac0a5ac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f57b80eda60 tx=0x7f57b802b810 comp rx=0 tx=0).stop 2024-09-16T01:53:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.188+0000 7f57c72d9640 1 -- 172.21.15.139:0/1058790727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57c01026f0 msgr2=0x7f57c01112d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.188+0000 7f57c72d9640 1 --2- 172.21.15.139:0/1058790727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57c01026f0 0x7f57c01112d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f57b802be00 tx=0x7f57b8003f10 comp rx=0 tx=0).stop 2024-09-16T01:53:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.188+0000 7f57c72d9640 1 -- 172.21.15.139:0/1058790727 shutdown_connections 2024-09-16T01:53:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.188+0000 7f57c72d9640 1 --2- 172.21.15.139:0/1058790727 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f57ac0a3600 0x7f57ac0a5ac0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.188+0000 7f57c72d9640 1 --2- 172.21.15.139:0/1058790727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57c01026f0 0x7f57c01112d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.188+0000 7f57c72d9640 1 --2- 172.21.15.139:0/1058790727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57c0101d70 0x7f57c0116700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.188+0000 7f57c72d9640 1 --2- 172.21.15.139:0/1058790727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57c0100490 0x7f57c01161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.188+0000 7f57c72d9640 1 -- 172.21.15.139:0/1058790727 >> 172.21.15.139:0/1058790727 conn(0x7f57c00770f0 msgr2=0x7f57c0103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.188+0000 7f57c72d9640 1 -- 172.21.15.139:0/1058790727 shutdown_connections 2024-09-16T01:53:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:23.188+0000 7f57c72d9640 1 -- 172.21.15.139:0/1058790727 wait complete. 2024-09-16T01:53:23.295 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:22 smithi164 systemd[1]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-09-16T01:53:23.295 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:22 smithi164 systemd[1]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.e.service: Consumed 29.693s CPU time. 2024-09-16T01:53:23.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:23 smithi164 ceph-mon[35121]: pgmap v410: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-09-16T01:53:23.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:23 smithi139 ceph-mon[31175]: pgmap v410: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-09-16T01:53:23.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:23 smithi176 ceph-mon[35324]: pgmap v410: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-09-16T01:53:24.423 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:53:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:53:24.144+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:53:24.423 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:53:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:53:24.144+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:53:25.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:25 smithi139 ceph-mon[31175]: pgmap v411: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-16T01:53:25.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:25 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:53:25.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:25 smithi176 ceph-mon[35324]: pgmap v411: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-16T01:53:25.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:25 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:53:25.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:25 smithi164 ceph-mon[35121]: pgmap v411: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-16T01:53:25.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:25 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:53:27.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:27 smithi139 ceph-mon[31175]: pgmap v412: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-09-16T01:53:27.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:27 smithi176 ceph-mon[35324]: pgmap v412: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-09-16T01:53:27.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:27 smithi164 ceph-mon[35121]: pgmap v412: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-09-16T01:53:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:29 smithi139 ceph-mon[31175]: pgmap v413: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-16T01:53:29.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:29 smithi176 ceph-mon[35324]: pgmap v413: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-16T01:53:29.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:29 smithi164 ceph-mon[35121]: pgmap v413: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-16T01:53:31.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:31 smithi139 ceph-mon[31175]: pgmap v414: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-09-16T01:53:31.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:31 smithi139 ceph-mon[31175]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2024-09-16T01:53:31.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:31 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-09-16T01:53:31.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:31 smithi176 ceph-mon[35324]: pgmap v414: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-09-16T01:53:31.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:31 smithi176 ceph-mon[35324]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2024-09-16T01:53:31.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:31 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-09-16T01:53:31.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:31 smithi164 ceph-mon[35121]: pgmap v414: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-09-16T01:53:31.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:31 smithi164 ceph-mon[35121]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2024-09-16T01:53:31.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:31 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-09-16T01:53:33.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:33 smithi139 ceph-mon[31175]: pgmap v415: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 988 KiB/s wr, 145 op/s 2024-09-16T01:53:33.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:33 smithi176 ceph-mon[35324]: pgmap v415: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 988 KiB/s wr, 145 op/s 2024-09-16T01:53:33.479 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:33 smithi164 ceph-mon[35121]: pgmap v415: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 988 KiB/s wr, 145 op/s 2024-09-16T01:53:33.479 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:33 smithi164 systemd[1]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 1. 2024-09-16T01:53:33.480 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:33 smithi164 systemd[1]: Stopped Ceph mds.e for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:53:33.480 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:33 smithi164 systemd[1]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.e.service: Consumed 29.693s CPU time. 2024-09-16T01:53:33.480 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:33 smithi164 systemd[1]: Starting Ceph mds.e for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T01:53:33.736 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:33 smithi164 podman[70728]: 2024-09-16 01:53:33.387101717 +0000 UTC m=+0.049496462 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T01:53:33.736 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:33 smithi164 podman[70728]: 2024-09-16 01:53:33.521064109 +0000 UTC m=+0.183458836 container create 6f8b4444de136d03541dcaef3d4044b6c9a11cdfcce97d4c22540b477d5f5998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T01:53:34.104 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:33 smithi164 podman[70728]: 2024-09-16 01:53:33.849657062 +0000 UTC m=+0.512051809 container init 6f8b4444de136d03541dcaef3d4044b6c9a11cdfcce97d4c22540b477d5f5998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T01:53:34.104 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:33 smithi164 podman[70728]: 2024-09-16 01:53:33.865958504 +0000 UTC m=+0.528353231 container start 6f8b4444de136d03541dcaef3d4044b6c9a11cdfcce97d4c22540b477d5f5998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-16T01:53:34.104 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:34 smithi164 bash[70728]: 6f8b4444de136d03541dcaef3d4044b6c9a11cdfcce97d4c22540b477d5f5998 2024-09-16T01:53:34.104 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:34 smithi164 systemd[1]: Started Ceph mds.e for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T01:53:34.545 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:34 smithi164 ceph-mds[70748]: -- 172.21.15.164:0/858841368 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560cfb2e3860 con 0x560cfb2c1000 2024-09-16T01:53:34.545 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 01:53:34 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e[70744]: starting mds.e at 2024-09-16T01:53:35.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:35 smithi164 ceph-mon[35121]: pgmap v416: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 572 KiB/s wr, 136 op/s 2024-09-16T01:53:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:35 smithi139 ceph-mon[31175]: pgmap v416: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 572 KiB/s wr, 136 op/s 2024-09-16T01:53:35.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:35 smithi176 ceph-mon[35324]: pgmap v416: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 572 KiB/s wr, 136 op/s 2024-09-16T01:53:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.342+0000 7f59fb4b7640 1 -- 172.21.15.139:0/626234055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f4101390 msgr2=0x7f59f4105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.342+0000 7f59fb4b7640 1 --2- 172.21.15.139:0/626234055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f4101390 0x7f59f4105850 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009860 tx=0x7f59e402a410 comp rx=0 tx=0).stop 2024-09-16T01:53:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.346+0000 7f59fb4b7640 1 -- 172.21.15.139:0/626234055 shutdown_connections 2024-09-16T01:53:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.346+0000 7f59fb4b7640 1 --2- 172.21.15.139:0/626234055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f4105ea0 0x7f59f4106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.346+0000 7f59fb4b7640 1 --2- 172.21.15.139:0/626234055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f4101390 0x7f59f4105850 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.346+0000 7f59fb4b7640 1 --2- 172.21.15.139:0/626234055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59f4100a70 0x7f59f4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.346+0000 7f59fb4b7640 1 -- 172.21.15.139:0/626234055 >> 172.21.15.139:0/626234055 conn(0x7f59f4077990 msgr2=0x7f59f40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.348+0000 7f59fb4b7640 1 -- 172.21.15.139:0/626234055 shutdown_connections 2024-09-16T01:53:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.352+0000 7f59fb4b7640 1 -- 172.21.15.139:0/626234055 wait complete. 2024-09-16T01:53:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.352+0000 7f59fb4b7640 1 Processor -- start 2024-09-16T01:53:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.354+0000 7f59fb4b7640 1 -- start start 2024-09-16T01:53:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.354+0000 7f59fb4b7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f4100a70 0x7f59f4114d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.354+0000 7f59fb4b7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59f4101390 0x7f59f410edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.354+0000 7f59fb4b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f4105ea0 0x7f59f410f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.354+0000 7f59fb4b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f59f4107870 con 0x7f59f4105ea0 2024-09-16T01:53:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.354+0000 7f59fb4b7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f59f41076f0 con 0x7f59f4101390 2024-09-16T01:53:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.354+0000 7f59fb4b7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f59f41079f0 con 0x7f59f4100a70 2024-09-16T01:53:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.355+0000 7f59f9253640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f4100a70 0x7f59f4114d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.355+0000 7f59f8a52640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59f4101390 0x7f59f410edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.355+0000 7f59f8a52640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59f4101390 0x7f59f410edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:38462/0 (socket says 172.21.15.139:38462) 2024-09-16T01:53:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.355+0000 7f59f8a52640 1 -- 172.21.15.139:0/305311700 learned_addr learned my addr 172.21.15.139:0/305311700 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.357+0000 7f59f9a54640 1 --2- 172.21.15.139:0/305311700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f4105ea0 0x7f59f410f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.359+0000 7f59f9253640 1 -- 172.21.15.139:0/305311700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59f4101390 msgr2=0x7f59f410edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.359+0000 7f59f9253640 1 --2- 172.21.15.139:0/305311700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59f4101390 0x7f59f410edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.359+0000 7f59f9253640 1 -- 172.21.15.139:0/305311700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f4105ea0 msgr2=0x7f59f410f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.359+0000 7f59f9253640 1 --2- 172.21.15.139:0/305311700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f4105ea0 0x7f59f410f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.359+0000 7f59f9253640 1 -- 172.21.15.139:0/305311700 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f410f9d0 con 0x7f59f4100a70 2024-09-16T01:53:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.361+0000 7f59f9253640 1 --2- 172.21.15.139:0/305311700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f4100a70 0x7f59f4114d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f59f000c970 tx=0x7f59f000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.366+0000 7f59ea7fc640 1 -- 172.21.15.139:0/305311700 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0007bb0 con 0x7f59f4100a70 2024-09-16T01:53:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.366+0000 7f59fb4b7640 1 -- 172.21.15.139:0/305311700 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f41b3860 con 0x7f59f4100a70 2024-09-16T01:53:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.366+0000 7f59fb4b7640 1 -- 172.21.15.139:0/305311700 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f41b3c60 con 0x7f59f4100a70 2024-09-16T01:53:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.367+0000 7f59ea7fc640 1 -- 172.21.15.139:0/305311700 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59f0007d50 con 0x7f59f4100a70 2024-09-16T01:53:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.367+0000 7f59ea7fc640 1 -- 172.21.15.139:0/305311700 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f000f450 con 0x7f59f4100a70 2024-09-16T01:53:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.372+0000 7f59fe251740 1 Processor -- start 2024-09-16T01:53:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.373+0000 7f59ea7fc640 1 -- 172.21.15.139:0/305311700 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f59f000f5f0 con 0x7f59f4100a70 2024-09-16T01:53:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.373+0000 7f59ea7fc640 1 --2- 172.21.15.139:0/305311700 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f59e00a3690 0x7f59e00a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.374+0000 7f59ea7fc640 1 -- 172.21.15.139:0/305311700 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f59f00d0c80 con 0x7f59f4100a70 2024-09-16T01:53:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.380+0000 7f59fe251740 1 -- start start 2024-09-16T01:53:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.380+0000 7f59fe251740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x55727014ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.380+0000 7f59fe251740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55727014e410 0x557270158770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.381+0000 7f59fe251740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557270158cb0 0x55727015b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.381+0000 7f59f9a54640 1 --2- 172.21.15.139:0/305311700 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f59e00a3690 0x7f59e00a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.381+0000 7f59fe251740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5572700b22f0 con 0x55727014daf0 2024-09-16T01:53:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.381+0000 7f59fe251740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55727005a270 con 0x55727014e410 2024-09-16T01:53:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.381+0000 7f59fe251740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5572700ae2b0 con 0x557270158cb0 2024-09-16T01:53:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.382+0000 7f59f8a52640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55727014e410 0x557270158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.382+0000 7f59f9253640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x55727014ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.382+0000 7f59f9253640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x55727014ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55338/0 (socket says 172.21.15.139:55338) 2024-09-16T01:53:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.382+0000 7f59f9253640 1 -- 172.21.15.139:0/3853719690 learned_addr learned my addr 172.21.15.139:0/3853719690 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.383+0000 7f59f9a54640 1 --2- 172.21.15.139:0/3853719690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557270158cb0 0x55727015b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.384+0000 7f59f9253640 1 -- 172.21.15.139:0/3853719690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557270158cb0 msgr2=0x55727015b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.384+0000 7f59f9253640 1 --2- 172.21.15.139:0/3853719690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557270158cb0 0x55727015b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.384+0000 7f59f9253640 1 -- 172.21.15.139:0/3853719690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55727014e410 msgr2=0x557270158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.384+0000 7f59f9253640 1 --2- 172.21.15.139:0/3853719690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55727014e410 0x557270158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.384+0000 7f59f9253640 1 -- 172.21.15.139:0/3853719690 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557270059340 con 0x55727014daf0 2024-09-16T01:53:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.390+0000 7f59f9253640 1 --2- 172.21.15.139:0/3853719690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x55727014ded0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f59f0007590 tx=0x7f59f009cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7386a193c99b2f87 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.391+0000 7f59df7fe640 1 -- 172.21.15.139:0/3853719690 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f001d040 con 0x55727014daf0 2024-09-16T01:53:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.391+0000 7f59df7fe640 1 -- 172.21.15.139:0/3853719690 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59f00991c0 con 0x55727014daf0 2024-09-16T01:53:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.392+0000 7f59fe251740 1 -- 172.21.15.139:0/3853719690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 msgr2=0x55727014ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.392+0000 7f59fe251740 1 --2- 172.21.15.139:0/3853719690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x55727014ded0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f59f0007590 tx=0x7f59f009cc90 comp rx=0 tx=0).stop 2024-09-16T01:53:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.392+0000 7f59f9a54640 1 --2- 172.21.15.139:0/305311700 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f59e00a3690 0x7f59e00a5b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f59ec005fd0 tx=0x7f59ec009a40 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.394+0000 7f59fe251740 1 -- 172.21.15.139:0/3853719690 shutdown_connections 2024-09-16T01:53:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.394+0000 7f59fe251740 1 --2- 172.21.15.139:0/3853719690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557270158cb0 0x55727015b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.394+0000 7f59fe251740 1 --2- 172.21.15.139:0/3853719690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55727014e410 0x557270158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.394+0000 7f59fe251740 1 --2- 172.21.15.139:0/3853719690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x55727014ded0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.394+0000 7f59fe251740 1 -- 172.21.15.139:0/3853719690 >> 172.21.15.139:0/3853719690 conn(0x55727014a060 msgr2=0x55727014a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.399+0000 7f59fe251740 1 -- 172.21.15.139:0/3853719690 shutdown_connections 2024-09-16T01:53:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.402+0000 7f59fe251740 1 -- 172.21.15.139:0/3853719690 wait complete. 2024-09-16T01:53:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.402+0000 7f59fe251740 1 -- start start 2024-09-16T01:53:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.402+0000 7f59fe251740 1 Processor -- start 2024-09-16T01:53:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.404+0000 7f59fe251740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.404+0000 7f59fe251740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55727014e410 0x5572701560f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.404+0000 7f59fe251740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557270158cb0 0x557270156630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.404+0000 7f59fe251740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55727015b8d0 con 0x55727014daf0 2024-09-16T01:53:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.404+0000 7f59fe251740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55727003b290 con 0x557270158cb0 2024-09-16T01:53:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.404+0000 7f59fe251740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55727015ba50 con 0x55727014e410 2024-09-16T01:53:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.404+0000 7f59fe251740 10 client.0 _ll_register_callbacks cb 0x55727006c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:53:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.405+0000 7f59f9a54640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557270158cb0 0x557270156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.405+0000 7f59f9a54640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557270158cb0 0x557270156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:38470/0 (socket says 172.21.15.139:38470) 2024-09-16T01:53:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.405+0000 7f59f9a54640 1 -- 172.21.15.139:0/4127696399 learned_addr learned my addr 172.21.15.139:0/4127696399 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.405+0000 7f59f9253640 1 --2- 172.21.15.139:0/4127696399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.405+0000 7f59f9a54640 1 -- 172.21.15.139:0/4127696399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55727014e410 msgr2=0x5572701560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.405+0000 7f59f9a54640 1 --2- 172.21.15.139:0/4127696399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55727014e410 0x5572701560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.405+0000 7f59f9a54640 1 -- 172.21.15.139:0/4127696399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 msgr2=0x557270155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.405+0000 7f59f9a54640 1 --2- 172.21.15.139:0/4127696399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.405+0000 7f59f9a54640 1 -- 172.21.15.139:0/4127696399 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55726fdc5210 con 0x557270158cb0 2024-09-16T01:53:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.405+0000 7f59f9253640 1 --2- 172.21.15.139:0/4127696399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:53:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.406+0000 7f59f9a54640 1 --2- 172.21.15.139:0/4127696399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557270158cb0 0x557270156630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f59ec01d430 tx=0x7f59ec004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8935e9b4405b5f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.406+0000 7f59dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:53:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.408+0000 7f59dcff9640 1 -- 172.21.15.139:0/4127696399 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec003d80 con 0x557270158cb0 2024-09-16T01:53:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.408+0000 7f59dcff9640 1 -- 172.21.15.139:0/4127696399 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f59ec004050 con 0x557270158cb0 2024-09-16T01:53:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.410+0000 7f59fe251740 1 -- 172.21.15.139:0/4127696399 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5572700bfa20 con 0x557270158cb0 2024-09-16T01:53:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.410+0000 7f59dcff9640 1 -- 172.21.15.139:0/4127696399 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f59ec02f050 con 0x557270158cb0 2024-09-16T01:53:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.414+0000 7f59fe251740 10 client.26318 fetch_fsmap learned FSMap version 67 2024-09-16T01:53:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.414+0000 7f59fe251740 1 -- 172.21.15.139:0/4127696399 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55726fe91620 con 0x557270158cb0 2024-09-16T01:53:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.416+0000 7f59dcff9640 1 -- 172.21.15.139:0/4127696399 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5572700bfa20 con 0x557270158cb0 2024-09-16T01:53:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.417+0000 7f59dcff9640 1 -- 172.21.15.139:0/4127696399 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 67) ==== 8438+0+0 (secure 0 0 0) 0x7f59ec020990 con 0x557270158cb0 2024-09-16T01:53:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.417+0000 7f59fe251740 10 client.26318 fetch_fsmap finished waiting for FSMap version 67 2024-09-16T01:53:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.417+0000 7f59fe251740 10 client.26318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:53:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.417+0000 7f59fe251740 20 client.26318 populate_metadata read hostname 'smithi139' 2024-09-16T01:53:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.417+0000 7f59fe251740 1 --2- 172.21.15.139:0/4127696399 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55727017a2e0 0x55727017c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.417+0000 7f59fe251740 4 client.26318 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:53:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.417+0000 7f59fe251740 1 -- 172.21.15.139:0/4127696399 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572700b9d70 con 0x55727017a2e0 2024-09-16T01:53:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.419+0000 7f59f9253640 1 --2- 172.21.15.139:0/4127696399 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55727017a2e0 0x55727017c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:36 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.164:6834/2728173014,v1:172.21.15.164:6835/2728173014] up:boot 2024-09-16T01:53:36.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:36 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:53:36.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:36 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T01:53:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.423+0000 7f59f9253640 1 --2- 172.21.15.139:0/4127696399 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55727017a2e0 0x55727017c7a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b90563c73db9991 server_cookie=d5ba357f1136df6e in_seq=0 out_seq=0 2024-09-16T01:53:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.423+0000 7f59dcff9640 10 client.26318 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.424+0000 7f59dcff9640 1 -- 172.21.15.139:0/4127696399 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572700b9d70 con 0x55727017a2e0 2024-09-16T01:53:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.424+0000 7f59dcff9640 10 client.26318 handle_command_reply: tid=0 2024-09-16T01:53:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.424+0000 7f59fe251740 1 client.26318 shutdown 2024-09-16T01:53:36.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:36 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.164:6834/2728173014,v1:172.21.15.164:6835/2728173014] up:boot 2024-09-16T01:53:36.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:36 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:53:36.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:36 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T01:53:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.436+0000 7f59fe251740 1 -- 172.21.15.139:0/4127696399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557270158cb0 msgr2=0x557270156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.436+0000 7f59fe251740 1 --2- 172.21.15.139:0/4127696399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557270158cb0 0x557270156630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f59ec01d430 tx=0x7f59ec004ab0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.440+0000 7f59fe251740 1 -- 172.21.15.139:0/4127696399 shutdown_connections 2024-09-16T01:53:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.440+0000 7f59fe251740 1 --2- 172.21.15.139:0/4127696399 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55727017a2e0 0x55727017c7a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.440+0000 7f59fe251740 1 --2- 172.21.15.139:0/4127696399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557270158cb0 0x557270156630 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.440+0000 7f59fe251740 1 --2- 172.21.15.139:0/4127696399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55727014e410 0x5572701560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.440+0000 7f59fe251740 1 --2- 172.21.15.139:0/4127696399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.440+0000 7f59fe251740 1 -- 172.21.15.139:0/4127696399 >> 172.21.15.139:0/4127696399 conn(0x55727014a060 msgr2=0x55727014fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.440+0000 7f59dcff9640 0 client.26318 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.444+0000 7f59fe251740 1 -- 172.21.15.139:0/4127696399 shutdown_connections 2024-09-16T01:53:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.444+0000 7f59fe251740 1 -- 172.21.15.139:0/4127696399 wait complete. 2024-09-16T01:53:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.446+0000 7f59fe251740 20 client.26318 trim_cache size 0 max 16384 2024-09-16T01:53:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.479+0000 7f59fe251740 1 Processor -- start 2024-09-16T01:53:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.480+0000 7f59fe251740 1 -- start start 2024-09-16T01:53:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.480+0000 7f59fe251740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270150560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.480+0000 7f59fe251740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55727014e410 0x557270156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.480+0000 7f59fe251740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557270158cb0 0x5572701820b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.480+0000 7f59fe251740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f59d80100f0 con 0x55727014daf0 2024-09-16T01:53:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.480+0000 7f59fe251740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f59d8011ea0 con 0x55727014e410 2024-09-16T01:53:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.480+0000 7f59fe251740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f59d80107f0 con 0x557270158cb0 2024-09-16T01:53:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.482+0000 7f59f9253640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.482+0000 7f59f9253640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270150560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55352/0 (socket says 172.21.15.139:55352) 2024-09-16T01:53:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.482+0000 7f59f9253640 1 -- 172.21.15.139:0/2481618358 learned_addr learned my addr 172.21.15.139:0/2481618358 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.485+0000 7f59f8a52640 1 --2- 172.21.15.139:0/2481618358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55727014e410 0x557270156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.485+0000 7f59f9a54640 1 --2- 172.21.15.139:0/2481618358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557270158cb0 0x5572701820b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.485+0000 7f59f9253640 1 -- 172.21.15.139:0/2481618358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557270158cb0 msgr2=0x5572701820b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.485+0000 7f59f9253640 1 --2- 172.21.15.139:0/2481618358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557270158cb0 0x5572701820b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.485+0000 7f59f9253640 1 -- 172.21.15.139:0/2481618358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55727014e410 msgr2=0x557270156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.485+0000 7f59f9253640 1 --2- 172.21.15.139:0/2481618358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55727014e410 0x557270156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.485+0000 7f59f9253640 1 -- 172.21.15.139:0/2481618358 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727014b110 con 0x55727014daf0 2024-09-16T01:53:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.487+0000 7f59f9253640 1 --2- 172.21.15.139:0/2481618358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270150560 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f59f0004b60 tx=0x7f59f00a5890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333c5cb2b2e2daf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.489+0000 7f59deffd640 1 -- 172.21.15.139:0/2481618358 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f001d040 con 0x55727014daf0 2024-09-16T01:53:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.490+0000 7f59deffd640 1 -- 172.21.15.139:0/2481618358 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59f00ce190 con 0x55727014daf0 2024-09-16T01:53:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.490+0000 7f59deffd640 1 -- 172.21.15.139:0/2481618358 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f009bc70 con 0x55727014daf0 2024-09-16T01:53:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.492+0000 7f59fe251740 1 -- 172.21.15.139:0/2481618358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 msgr2=0x557270150560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.492+0000 7f59fe251740 1 --2- 172.21.15.139:0/2481618358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270150560 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f59f0004b60 tx=0x7f59f00a5890 comp rx=0 tx=0).stop 2024-09-16T01:53:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.493+0000 7f59fe251740 1 -- 172.21.15.139:0/2481618358 shutdown_connections 2024-09-16T01:53:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.493+0000 7f59fe251740 1 --2- 172.21.15.139:0/2481618358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557270158cb0 0x5572701820b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.493+0000 7f59fe251740 1 --2- 172.21.15.139:0/2481618358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55727014e410 0x557270156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.493+0000 7f59fe251740 1 --2- 172.21.15.139:0/2481618358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270150560 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.493+0000 7f59fe251740 1 -- 172.21.15.139:0/2481618358 >> 172.21.15.139:0/2481618358 conn(0x55727014a060 msgr2=0x557270157c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.499+0000 7f59fe251740 1 -- 172.21.15.139:0/2481618358 shutdown_connections 2024-09-16T01:53:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.501+0000 7f59fe251740 1 -- 172.21.15.139:0/2481618358 wait complete. 2024-09-16T01:53:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.501+0000 7f59fe251740 1 -- start start 2024-09-16T01:53:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.501+0000 7f59fe251740 1 Processor -- start 2024-09-16T01:53:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.504+0000 7f59fe251740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.504+0000 7f59fe251740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55727014e410 0x5572701612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.504+0000 7f59fe251740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557270158cb0 0x5572701521d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.504+0000 7f59fe251740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f59d8011710 con 0x55727014daf0 2024-09-16T01:53:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.504+0000 7f59fe251740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f59d8010f80 con 0x557270158cb0 2024-09-16T01:53:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.504+0000 7f59fe251740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f59d800e180 con 0x55727014e410 2024-09-16T01:53:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.504+0000 7f59fe251740 10 client.0 _ll_register_callbacks cb 0x7f59ec009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:53:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.505+0000 7f59f9253640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.505+0000 7f59f9253640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55358/0 (socket says 172.21.15.139:55358) 2024-09-16T01:53:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.505+0000 7f59f9253640 1 -- 172.21.15.139:0/1395951924 learned_addr learned my addr 172.21.15.139:0/1395951924 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.505+0000 7f59f9253640 1 -- 172.21.15.139:0/1395951924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55727014e410 msgr2=0x5572701612a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:53:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.505+0000 7f59f9253640 1 --2- 172.21.15.139:0/1395951924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55727014e410 0x5572701612a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.505+0000 7f59f9253640 1 -- 172.21.15.139:0/1395951924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557270158cb0 msgr2=0x5572701521d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:53:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.505+0000 7f59f9253640 1 --2- 172.21.15.139:0/1395951924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557270158cb0 0x5572701521d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.505+0000 7f59f9253640 1 -- 172.21.15.139:0/1395951924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727014d920 con 0x55727014daf0 2024-09-16T01:53:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.505+0000 7f59f9253640 1 --2- 172.21.15.139:0/1395951924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270160d60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f59f0098fb0 tx=0x7f59f009afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78639e45c6fc50e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.508+0000 7f59dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:53:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.508+0000 7f59dcff9640 1 -- 172.21.15.139:0/1395951924 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f00aa060 con 0x55727014daf0 2024-09-16T01:53:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.508+0000 7f59fe251740 1 -- 172.21.15.139:0/1395951924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59f00ce190 con 0x55727014daf0 2024-09-16T01:53:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.509+0000 7f59dcff9640 1 -- 172.21.15.139:0/1395951924 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f59f00bb310 con 0x55727014daf0 2024-09-16T01:53:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.514+0000 7f59dcff9640 1 -- 172.21.15.139:0/1395951924 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f59f00b9070 con 0x55727014daf0 2024-09-16T01:53:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.515+0000 7f59fe251740 10 client.16887 fetch_fsmap learned FSMap version 67 2024-09-16T01:53:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.515+0000 7f59fe251740 1 -- 172.21.15.139:0/1395951924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x5572701e46c0 con 0x55727014daf0 2024-09-16T01:53:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.519+0000 7f59dcff9640 1 -- 172.21.15.139:0/1395951924 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59f00ce190 con 0x55727014daf0 2024-09-16T01:53:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.519+0000 7f59dcff9640 1 -- 172.21.15.139:0/1395951924 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 67) ==== 8438+0+0 (secure 0 0 0) 0x7f59f00a1fe0 con 0x55727014daf0 2024-09-16T01:53:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.520+0000 7f59fe251740 10 client.16887 fetch_fsmap finished waiting for FSMap version 67 2024-09-16T01:53:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.520+0000 7f59fe251740 10 client.16887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:53:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.520+0000 7f59fe251740 20 client.16887 populate_metadata read hostname 'smithi139' 2024-09-16T01:53:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.520+0000 7f59fe251740 1 --2- 172.21.15.139:0/1395951924 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5572701e48f0 0x5572701e6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.520+0000 7f59fe251740 4 client.16887 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:53:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.520+0000 7f59fe251740 1 -- 172.21.15.139:0/1395951924 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557270036900 con 0x5572701e48f0 2024-09-16T01:53:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.529+0000 7f59f8a52640 1 --2- 172.21.15.139:0/1395951924 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5572701e48f0 0x5572701e6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.542+0000 7f59f8a52640 1 --2- 172.21.15.139:0/1395951924 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5572701e48f0 0x5572701e6ce0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21c3da48fb511a09 server_cookie=e4f27921c216e529 in_seq=0 out_seq=0 2024-09-16T01:53:36.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:36 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.164:6834/2728173014,v1:172.21.15.164:6835/2728173014] up:boot 2024-09-16T01:53:36.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:36 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:53:36.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:36 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T01:53:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.542+0000 7f59dcff9640 10 client.16887 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.710+0000 7f59dcff9640 1 -- 172.21.15.139:0/1395951924 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557270036900 con 0x5572701e48f0 2024-09-16T01:53:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.710+0000 7f59dcff9640 10 client.16887 handle_command_reply: tid=0 2024-09-16T01:53:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.711+0000 7f59fe251740 1 client.16887 shutdown 2024-09-16T01:53:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.711+0000 7f59fe251740 1 -- 172.21.15.139:0/1395951924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 msgr2=0x557270160d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.711+0000 7f59fe251740 1 --2- 172.21.15.139:0/1395951924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270160d60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f59f0098fb0 tx=0x7f59f009afb0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.711+0000 7f59fe251740 1 -- 172.21.15.139:0/1395951924 shutdown_connections 2024-09-16T01:53:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.711+0000 7f59fe251740 1 --2- 172.21.15.139:0/1395951924 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5572701e48f0 0x5572701e6ce0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.711+0000 7f59fe251740 1 --2- 172.21.15.139:0/1395951924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557270158cb0 0x5572701521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.711+0000 7f59fe251740 1 --2- 172.21.15.139:0/1395951924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55727014e410 0x5572701612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.711+0000 7f59fe251740 1 --2- 172.21.15.139:0/1395951924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55727014daf0 0x557270160d60 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.711+0000 7f59fe251740 1 -- 172.21.15.139:0/1395951924 >> 172.21.15.139:0/1395951924 conn(0x55727014a060 msgr2=0x557270157410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.711+0000 7f59dcff9640 0 client.16887 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.711+0000 7f59fe251740 1 -- 172.21.15.139:0/1395951924 shutdown_connections 2024-09-16T01:53:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.711+0000 7f59fe251740 1 -- 172.21.15.139:0/1395951924 wait complete. 2024-09-16T01:53:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.711+0000 7f59fe251740 20 client.16887 trim_cache size 0 max 16384 2024-09-16T01:53:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:53:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:53:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:53:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:53:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.712+0000 7f59fb4b7640 1 -- 172.21.15.139:0/305311700 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f59e00a3690 msgr2=0x7f59e00a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.712+0000 7f59fb4b7640 1 --2- 172.21.15.139:0/305311700 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f59e00a3690 0x7f59e00a5b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f59ec005fd0 tx=0x7f59ec009a40 comp rx=0 tx=0).stop 2024-09-16T01:53:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.712+0000 7f59fb4b7640 1 -- 172.21.15.139:0/305311700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f4100a70 msgr2=0x7f59f4114d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.713+0000 7f59fb4b7640 1 --2- 172.21.15.139:0/305311700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f4100a70 0x7f59f4114d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f59f000c970 tx=0x7f59f000ce30 comp rx=0 tx=0).stop 2024-09-16T01:53:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.713+0000 7f59fb4b7640 1 -- 172.21.15.139:0/305311700 shutdown_connections 2024-09-16T01:53:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.713+0000 7f59fb4b7640 1 --2- 172.21.15.139:0/305311700 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f59e00a3690 0x7f59e00a5b50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.713+0000 7f59fb4b7640 1 --2- 172.21.15.139:0/305311700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f4105ea0 0x7f59f410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.713+0000 7f59fb4b7640 1 --2- 172.21.15.139:0/305311700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59f4101390 0x7f59f410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.713+0000 7f59fb4b7640 1 --2- 172.21.15.139:0/305311700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f4100a70 0x7f59f4114d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.713+0000 7f59fb4b7640 1 -- 172.21.15.139:0/305311700 >> 172.21.15.139:0/305311700 conn(0x7f59f4077990 msgr2=0x7f59f4103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.713+0000 7f59fb4b7640 1 -- 172.21.15.139:0/305311700 shutdown_connections 2024-09-16T01:53:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:36.713+0000 7f59fb4b7640 1 -- 172.21.15.139:0/305311700 wait complete. 2024-09-16T01:53:37.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:37 smithi139 ceph-mon[31175]: pgmap v417: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 968 KiB/s wr, 155 op/s 2024-09-16T01:53:37.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:37 smithi176 ceph-mon[35324]: pgmap v417: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 968 KiB/s wr, 155 op/s 2024-09-16T01:53:37.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:37 smithi164 ceph-mon[35121]: pgmap v417: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 968 KiB/s wr, 155 op/s 2024-09-16T01:53:39.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:39 smithi139 ceph-mon[31175]: pgmap v418: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 966 KiB/s wr, 172 op/s 2024-09-16T01:53:39.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:39 smithi176 ceph-mon[35324]: pgmap v418: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 966 KiB/s wr, 172 op/s 2024-09-16T01:53:39.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:39 smithi164 ceph-mon[35121]: pgmap v418: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 966 KiB/s wr, 172 op/s 2024-09-16T01:53:40.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:40 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:53:40.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:40 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:53:40.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:40 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:53:41.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:41 smithi139 ceph-mon[31175]: pgmap v419: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 999 KiB/s wr, 184 op/s 2024-09-16T01:53:41.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:41 smithi176 ceph-mon[35324]: pgmap v419: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 999 KiB/s wr, 184 op/s 2024-09-16T01:53:41.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:41 smithi164 ceph-mon[35121]: pgmap v419: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 999 KiB/s wr, 184 op/s 2024-09-16T01:53:43.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:43 smithi139 ceph-mon[31175]: pgmap v420: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-09-16T01:53:43.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:43 smithi176 ceph-mon[35324]: pgmap v420: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-09-16T01:53:43.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:43 smithi164 ceph-mon[35121]: pgmap v420: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-09-16T01:53:45.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:45 smithi139 ceph-mon[31175]: pgmap v421: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-09-16T01:53:45.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:45 smithi176 ceph-mon[35324]: pgmap v421: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-09-16T01:53:45.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:45 smithi164 ceph-mon[35121]: pgmap v421: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-09-16T01:53:47.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:47 smithi176 ceph-mon[35324]: pgmap v422: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 225 op/s 2024-09-16T01:53:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:47 smithi164 ceph-mon[35121]: pgmap v422: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 225 op/s 2024-09-16T01:53:47.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:47 smithi139 ceph-mon[31175]: pgmap v422: 129 pgs: 20 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 52 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 225 op/s 2024-09-16T01:53:48.176 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.4 2024-09-16T01:53:48.177 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:53:48.714 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:53:49.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.357+0000 7fa5b072f640 1 -- 172.21.15.139:0/1874892663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a8106b70 msgr2=0x7fa5a8106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:49.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.357+0000 7fa5b072f640 1 --2- 172.21.15.139:0/1874892663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a8106b70 0x7fa5a8106f50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa598009860 tx=0x7fa59802a380 comp rx=0 tx=0).stop 2024-09-16T01:53:49.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.359+0000 7fa5b072f640 1 -- 172.21.15.139:0/1874892663 shutdown_connections 2024-09-16T01:53:49.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.359+0000 7fa5b072f640 1 --2- 172.21.15.139:0/1874892663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a8100a50 0x7fa5a81057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.359+0000 7fa5b072f640 1 --2- 172.21.15.139:0/1874892663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5a8100090 0x7fa5a8100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.359+0000 7fa5b072f640 1 --2- 172.21.15.139:0/1874892663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a8106b70 0x7fa5a8106f50 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.359+0000 7fa5b072f640 1 -- 172.21.15.139:0/1874892663 >> 172.21.15.139:0/1874892663 conn(0x7fa5a8077210 msgr2=0x7fa5a80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:49.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.360+0000 7fa5b072f640 1 -- 172.21.15.139:0/1874892663 shutdown_connections 2024-09-16T01:53:49.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.360+0000 7fa5b072f640 1 -- 172.21.15.139:0/1874892663 wait complete. 2024-09-16T01:53:49.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.361+0000 7fa5b072f640 1 Processor -- start 2024-09-16T01:53:49.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.361+0000 7fa5b072f640 1 -- start start 2024-09-16T01:53:49.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.361+0000 7fa5b072f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a8100090 0x7fa5a8196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.362+0000 7fa5b072f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a8100a50 0x7fa5a8196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.362+0000 7fa5ae4cb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a8100090 0x7fa5a8196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:49.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.362+0000 7fa5b072f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5a8106b70 0x7fa5a819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.362+0000 7fa5ae4cb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a8100090 0x7fa5a8196230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54970/0 (socket says 172.21.15.139:54970) 2024-09-16T01:53:49.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.362+0000 7fa5ae4cb640 1 -- 172.21.15.139:0/4191058154 learned_addr learned my addr 172.21.15.139:0/4191058154 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:49.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.362+0000 7fa5b072f640 1 -- 172.21.15.139:0/4191058154 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a8109dd0 con 0x7fa5a8100090 2024-09-16T01:53:49.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.362+0000 7fa5b072f640 1 -- 172.21.15.139:0/4191058154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a8109c50 con 0x7fa5a8100a50 2024-09-16T01:53:49.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.362+0000 7fa5b072f640 1 -- 172.21.15.139:0/4191058154 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a8109f50 con 0x7fa5a8106b70 2024-09-16T01:53:49.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.362+0000 7fa5adcca640 1 --2- 172.21.15.139:0/4191058154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a8100a50 0x7fa5a8196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:49.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.362+0000 7fa5aeccc640 1 --2- 172.21.15.139:0/4191058154 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5a8106b70 0x7fa5a819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:49.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.363+0000 7fa5ae4cb640 1 -- 172.21.15.139:0/4191058154 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5a8106b70 msgr2=0x7fa5a819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:49.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.363+0000 7fa5ae4cb640 1 --2- 172.21.15.139:0/4191058154 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5a8106b70 0x7fa5a819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.363+0000 7fa5ae4cb640 1 -- 172.21.15.139:0/4191058154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a8100a50 msgr2=0x7fa5a8196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:49.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.363+0000 7fa5ae4cb640 1 --2- 172.21.15.139:0/4191058154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a8100a50 0x7fa5a8196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.363+0000 7fa5ae4cb640 1 -- 172.21.15.139:0/4191058154 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a819b140 con 0x7fa5a8100090 2024-09-16T01:53:49.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.363+0000 7fa5ae4cb640 1 --2- 172.21.15.139:0/4191058154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a8100090 0x7fa5a8196230 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa59802a8f0 tx=0x7fa598033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:49.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.364+0000 7fa59f7fe640 1 -- 172.21.15.139:0/4191058154 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598037040 con 0x7fa5a8100090 2024-09-16T01:53:49.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.364+0000 7fa5b072f640 1 -- 172.21.15.139:0/4191058154 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a819b3d0 con 0x7fa5a8100090 2024-09-16T01:53:49.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.364+0000 7fa5b072f640 1 -- 172.21.15.139:0/4191058154 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a81a2d10 con 0x7fa5a8100090 2024-09-16T01:53:49.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.366+0000 7fa59f7fe640 1 -- 172.21.15.139:0/4191058154 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa598005070 con 0x7fa5a8100090 2024-09-16T01:53:49.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.366+0000 7fa59f7fe640 1 -- 172.21.15.139:0/4191058154 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598004bf0 con 0x7fa5a8100090 2024-09-16T01:53:49.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.370+0000 7fa59f7fe640 1 -- 172.21.15.139:0/4191058154 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa598005220 con 0x7fa5a8100090 2024-09-16T01:53:49.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.370+0000 7fa5b072f640 1 -- 172.21.15.139:0/4191058154 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa58c005290 con 0x7fa5a8100090 2024-09-16T01:53:49.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.371+0000 7fa59f7fe640 1 --2- 172.21.15.139:0/4191058154 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa5940a35c0 0x7fa5940a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.375+0000 7fa59f7fe640 1 -- 172.21.15.139:0/4191058154 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7fa5980ef8a0 con 0x7fa5a8100090 2024-09-16T01:53:49.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.376+0000 7fa5adcca640 1 --2- 172.21.15.139:0/4191058154 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa5940a35c0 0x7fa5940a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:49.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.377+0000 7fa5adcca640 1 --2- 172.21.15.139:0/4191058154 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa5940a35c0 0x7fa5940a5a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4005fd0 tx=0x7fa5a4005850 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:49.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.385+0000 7fa59f7fe640 1 -- 172.21.15.139:0/4191058154 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa598049060 con 0x7fa5a8100090 2024-09-16T01:53:49.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:49 smithi164 ceph-mon[35121]: pgmap v423: 129 pgs: 19 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 53 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 937 KiB/s wr, 221 op/s 2024-09-16T01:53:49.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:49 smithi139 ceph-mon[31175]: pgmap v423: 129 pgs: 19 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 53 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 937 KiB/s wr, 221 op/s 2024-09-16T01:53:49.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:49 smithi176 ceph-mon[35324]: pgmap v423: 129 pgs: 19 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 53 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 937 KiB/s wr, 221 op/s 2024-09-16T01:53:49.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.777+0000 7fa5b072f640 1 -- 172.21.15.139:0/4191058154 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa58c005850 con 0x7fa5a8100090 2024-09-16T01:53:49.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.778+0000 7fa59f7fe640 1 -- 172.21.15.139:0/4191058154 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+12025 (secure 0 0 0) 0x7fa5980bbf60 con 0x7fa5a8100090 2024-09-16T01:53:49.783 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:53:49.783 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":69,"btime":"2024-09-16T01:53:49:176082+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":69,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:53:49.046798+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:53:49.784 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 69 2024-09-16T01:53:49.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.783+0000 7fa59d7fa640 1 -- 172.21.15.139:0/4191058154 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa5940a35c0 msgr2=0x7fa5940a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:49.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.783+0000 7fa59d7fa640 1 --2- 172.21.15.139:0/4191058154 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa5940a35c0 0x7fa5940a5a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4005fd0 tx=0x7fa5a4005850 comp rx=0 tx=0).stop 2024-09-16T01:53:49.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.784+0000 7fa59d7fa640 1 -- 172.21.15.139:0/4191058154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a8100090 msgr2=0x7fa5a8196230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:49.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.784+0000 7fa59d7fa640 1 --2- 172.21.15.139:0/4191058154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a8100090 0x7fa5a8196230 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa59802a8f0 tx=0x7fa598033d10 comp rx=0 tx=0).stop 2024-09-16T01:53:49.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.784+0000 7fa59d7fa640 1 -- 172.21.15.139:0/4191058154 shutdown_connections 2024-09-16T01:53:49.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.784+0000 7fa59d7fa640 1 --2- 172.21.15.139:0/4191058154 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa5940a35c0 0x7fa5940a5a80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.784+0000 7fa59d7fa640 1 --2- 172.21.15.139:0/4191058154 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5a8106b70 0x7fa5a819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.784+0000 7fa59d7fa640 1 --2- 172.21.15.139:0/4191058154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a8100a50 0x7fa5a8196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.784+0000 7fa59d7fa640 1 --2- 172.21.15.139:0/4191058154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a8100090 0x7fa5a8196230 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.784+0000 7fa59d7fa640 1 -- 172.21.15.139:0/4191058154 >> 172.21.15.139:0/4191058154 conn(0x7fa5a8077210 msgr2=0x7fa5a81037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:49.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.786+0000 7fa59d7fa640 1 -- 172.21.15.139:0/4191058154 shutdown_connections 2024-09-16T01:53:49.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:49.786+0000 7fa59d7fa640 1 -- 172.21.15.139:0/4191058154 wait complete. 2024-09-16T01:53:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.871+0000 7f0e5ec2e640 1 -- 172.21.15.139:0/789573061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e58107b80 msgr2=0x7f0e58107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.871+0000 7f0e5ec2e640 1 --2- 172.21.15.139:0/789573061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e58107b80 0x7f0e58107f60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009ca0 tx=0x7f0e4802a8d0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.872+0000 7f0e5ec2e640 1 -- 172.21.15.139:0/789573061 shutdown_connections 2024-09-16T01:53:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.872+0000 7f0e5ec2e640 1 --2- 172.21.15.139:0/789573061 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e58102480 0x7f0e5810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.872+0000 7f0e5ec2e640 1 --2- 172.21.15.139:0/789573061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e58101ae0 0x7f0e58101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.872+0000 7f0e5ec2e640 1 --2- 172.21.15.139:0/789573061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e58107b80 0x7f0e58107f60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.872+0000 7f0e5ec2e640 1 -- 172.21.15.139:0/789573061 >> 172.21.15.139:0/789573061 conn(0x7f0e580fd7e0 msgr2=0x7f0e580ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.873+0000 7f0e5ec2e640 1 -- 172.21.15.139:0/789573061 shutdown_connections 2024-09-16T01:53:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.875+0000 7f0e5ec2e640 1 -- 172.21.15.139:0/789573061 wait complete. 2024-09-16T01:53:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.875+0000 7f0e5ec2e640 1 Processor -- start 2024-09-16T01:53:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.878+0000 7f0e5ec2e640 1 -- start start 2024-09-16T01:53:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.878+0000 7f0e5ec2e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e58101ae0 0x7f0e58075c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.878+0000 7f0e5ec2e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e58102480 0x7f0e5806cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.878+0000 7f0e5ec2e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e5806d180 0x7f0e5806d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.878+0000 7f0e5ec2e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0e58111fe0 con 0x7f0e58101ae0 2024-09-16T01:53:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.878+0000 7f0e5ec2e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e58111e60 con 0x7f0e5806d180 2024-09-16T01:53:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.878+0000 7f0e5ec2e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0e58112160 con 0x7f0e58102480 2024-09-16T01:53:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.879+0000 7f0e57fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e58102480 0x7f0e5806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.879+0000 7f0e57fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e58102480 0x7f0e5806cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:49372/0 (socket says 172.21.15.139:49372) 2024-09-16T01:53:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.879+0000 7f0e57fff640 1 -- 172.21.15.139:0/854308307 learned_addr learned my addr 172.21.15.139:0/854308307 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.879+0000 7f0e57fff640 1 -- 172.21.15.139:0/854308307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e5806d180 msgr2=0x7f0e5806d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.879+0000 7f0e57fff640 1 --2- 172.21.15.139:0/854308307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e5806d180 0x7f0e5806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.879+0000 7f0e57fff640 1 -- 172.21.15.139:0/854308307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e58101ae0 msgr2=0x7f0e58075c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.879+0000 7f0e57fff640 1 --2- 172.21.15.139:0/854308307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e58101ae0 0x7f0e58075c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.879+0000 7f0e57fff640 1 -- 172.21.15.139:0/854308307 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e580716d0 con 0x7f0e58102480 2024-09-16T01:53:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.879+0000 7f0e57fff640 1 --2- 172.21.15.139:0/854308307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e58102480 0x7f0e5806cc40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000b4d0 tx=0x7f0e4000b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.880+0000 7f0e55ffb640 1 -- 172.21.15.139:0/854308307 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40014070 con 0x7f0e58102480 2024-09-16T01:53:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.880+0000 7f0e55ffb640 1 -- 172.21.15.139:0/854308307 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e40010d50 con 0x7f0e58102480 2024-09-16T01:53:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.880+0000 7f0e55ffb640 1 -- 172.21.15.139:0/854308307 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40017360 con 0x7f0e58102480 2024-09-16T01:53:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.880+0000 7f0e5ec2e640 1 -- 172.21.15.139:0/854308307 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e580719c0 con 0x7f0e58102480 2024-09-16T01:53:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.881+0000 7f0e5ec2e640 1 -- 172.21.15.139:0/854308307 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e58071ec0 con 0x7f0e58102480 2024-09-16T01:53:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.885+0000 7f0e619ca740 1 Processor -- start 2024-09-16T01:53:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.886+0000 7f0e619ca740 1 -- start start 2024-09-16T01:53:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.886+0000 7f0e619ca740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b756faf0 0x5625b756fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.886+0000 7f0e619ca740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b7570410 0x5625b757a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.886+0000 7f0e619ca740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b757acb0 0x5625b757d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.886+0000 7f0e619ca740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5625b74d42f0 con 0x5625b7570410 2024-09-16T01:53:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.886+0000 7f0e619ca740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5625b747c270 con 0x5625b756faf0 2024-09-16T01:53:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.886+0000 7f0e619ca740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5625b74d02b0 con 0x5625b757acb0 2024-09-16T01:53:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.889+0000 7f0e55ffb640 1 -- 172.21.15.139:0/854308307 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0e40017070 con 0x7f0e58102480 2024-09-16T01:53:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.890+0000 7f0e5d1cb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b756faf0 0x5625b756fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.890+0000 7f0e5d1cb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b756faf0 0x5625b756fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:36086/0 (socket says 172.21.15.139:36086) 2024-09-16T01:53:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.890+0000 7f0e5d1cb640 1 -- 172.21.15.139:0/3967766763 learned_addr learned my addr 172.21.15.139:0/3967766763 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.890+0000 7f0e57fff640 1 --2- 172.21.15.139:0/3967766763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b757acb0 0x5625b757d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.891+0000 7f0e5d1cb640 1 -- 172.21.15.139:0/3967766763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b757acb0 msgr2=0x5625b757d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.891+0000 7f0e5d1cb640 1 --2- 172.21.15.139:0/3967766763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b757acb0 0x5625b757d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.891+0000 7f0e5d1cb640 1 -- 172.21.15.139:0/3967766763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b7570410 msgr2=0x5625b757a770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:53:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.891+0000 7f0e5d1cb640 1 --2- 172.21.15.139:0/3967766763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b7570410 0x5625b757a770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.891+0000 7f0e5d1cb640 1 -- 172.21.15.139:0/3967766763 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b747b340 con 0x5625b756faf0 2024-09-16T01:53:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.891+0000 7f0e57fff640 1 --2- 172.21.15.139:0/3967766763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b757acb0 0x5625b757d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:53:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.891+0000 7f0e5d1cb640 1 --2- 172.21.15.139:0/3967766763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b756faf0 0x5625b756fed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009790 tx=0x7f0e50009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a129dd3d0252dd23 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.891+0000 7f0e4effd640 1 -- 172.21.15.139:0/3967766763 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50015020 con 0x5625b756faf0 2024-09-16T01:53:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.891+0000 7f0e4effd640 1 -- 172.21.15.139:0/3967766763 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e50011420 con 0x5625b756faf0 2024-09-16T01:53:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.892+0000 7f0e619ca740 1 -- 172.21.15.139:0/3967766763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b756faf0 msgr2=0x5625b756fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.892+0000 7f0e619ca740 1 --2- 172.21.15.139:0/3967766763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b756faf0 0x5625b756fed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009790 tx=0x7f0e50009c50 comp rx=0 tx=0).stop 2024-09-16T01:53:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.893+0000 7f0e55ffb640 1 --2- 172.21.15.139:0/854308307 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e440a35b0 0x7f0e440a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.893+0000 7f0e55ffb640 1 -- 172.21.15.139:0/854308307 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f0e400cf7e0 con 0x7f0e58102480 2024-09-16T01:53:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.895+0000 7f0e5d1cb640 1 --2- 172.21.15.139:0/854308307 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e440a35b0 0x7f0e440a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.898+0000 7f0e5d1cb640 1 --2- 172.21.15.139:0/854308307 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e440a35b0 0x7f0e440a5a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0e50011d40 tx=0x7f0e5001d040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.900+0000 7f0e619ca740 1 -- 172.21.15.139:0/3967766763 shutdown_connections 2024-09-16T01:53:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.900+0000 7f0e619ca740 1 --2- 172.21.15.139:0/3967766763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b757acb0 0x5625b757d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.900+0000 7f0e619ca740 1 --2- 172.21.15.139:0/3967766763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b7570410 0x5625b757a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.900+0000 7f0e619ca740 1 --2- 172.21.15.139:0/3967766763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b756faf0 0x5625b756fed0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.900+0000 7f0e619ca740 1 -- 172.21.15.139:0/3967766763 >> 172.21.15.139:0/3967766763 conn(0x5625b756c060 msgr2=0x5625b756c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.902+0000 7f0e619ca740 1 -- 172.21.15.139:0/3967766763 shutdown_connections 2024-09-16T01:53:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.909+0000 7f0e619ca740 1 -- 172.21.15.139:0/3967766763 wait complete. 2024-09-16T01:53:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.909+0000 7f0e619ca740 1 -- start start 2024-09-16T01:53:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.909+0000 7f0e619ca740 1 Processor -- start 2024-09-16T01:53:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.914+0000 7f0e619ca740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b756faf0 0x5625b7577a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.914+0000 7f0e619ca740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b7570410 0x5625b7577fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.915+0000 7f0e619ca740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b757acb0 0x5625b7578510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.915+0000 7f0e619ca740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5625b757d8d0 con 0x5625b7570410 2024-09-16T01:53:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.915+0000 7f0e619ca740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5625b745d290 con 0x5625b757acb0 2024-09-16T01:53:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.915+0000 7f0e619ca740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5625b757da50 con 0x5625b756faf0 2024-09-16T01:53:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.915+0000 7f0e619ca740 10 client.0 _ll_register_callbacks cb 0x5625b748e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:53:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.916+0000 7f0e5c9ca640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b756faf0 0x5625b7577a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.916+0000 7f0e5c9ca640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b756faf0 0x5625b7577a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:49396/0 (socket says 172.21.15.139:49396) 2024-09-16T01:53:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.916+0000 7f0e5c9ca640 1 -- 172.21.15.139:0/1344600326 learned_addr learned my addr 172.21.15.139:0/1344600326 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.917+0000 7f0e5d1cb640 1 --2- 172.21.15.139:0/1344600326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b757acb0 0x5625b7578510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.917+0000 7f0e57fff640 1 --2- 172.21.15.139:0/1344600326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b7570410 0x5625b7577fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.920+0000 7f0e5d1cb640 1 -- 172.21.15.139:0/1344600326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b756faf0 msgr2=0x5625b7577a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.920+0000 7f0e5d1cb640 1 --2- 172.21.15.139:0/1344600326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b756faf0 0x5625b7577a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.920+0000 7f0e5d1cb640 1 -- 172.21.15.139:0/1344600326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b7570410 msgr2=0x5625b7577fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.920+0000 7f0e5d1cb640 1 --2- 172.21.15.139:0/1344600326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b7570410 0x5625b7577fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.920+0000 7f0e5d1cb640 1 -- 172.21.15.139:0/1344600326 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b71e7210 con 0x5625b757acb0 2024-09-16T01:53:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.924+0000 7f0e5d1cb640 1 --2- 172.21.15.139:0/1344600326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b757acb0 0x5625b7578510 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0e50000c00 tx=0x7f0e50009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ede519e90090ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.925+0000 7f0e37fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:53:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.926+0000 7f0e37fff640 1 -- 172.21.15.139:0/1344600326 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50015040 con 0x5625b757acb0 2024-09-16T01:53:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.926+0000 7f0e37fff640 1 -- 172.21.15.139:0/1344600326 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f0e5000b8d0 con 0x5625b757acb0 2024-09-16T01:53:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.927+0000 7f0e619ca740 1 -- 172.21.15.139:0/1344600326 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e50011420 con 0x5625b757acb0 2024-09-16T01:53:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.929+0000 7f0e37fff640 1 -- 172.21.15.139:0/1344600326 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f0e50010ea0 con 0x5625b757acb0 2024-09-16T01:53:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.931+0000 7f0e619ca740 10 client.26333 fetch_fsmap learned FSMap version 69 2024-09-16T01:53:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.931+0000 7f0e619ca740 1 -- 172.21.15.139:0/1344600326 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5625b72b3620 con 0x5625b757acb0 2024-09-16T01:53:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.936+0000 7f0e37fff640 1 -- 172.21.15.139:0/1344600326 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e50011420 con 0x5625b757acb0 2024-09-16T01:53:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.936+0000 7f0e37fff640 1 -- 172.21.15.139:0/1344600326 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7f0e5003a740 con 0x5625b757acb0 2024-09-16T01:53:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.936+0000 7f0e619ca740 10 client.26333 fetch_fsmap finished waiting for FSMap version 69 2024-09-16T01:53:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.936+0000 7f0e619ca740 10 client.26333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:53:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.937+0000 7f0e619ca740 20 client.26333 populate_metadata read hostname 'smithi139' 2024-09-16T01:53:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.937+0000 7f0e619ca740 1 --2- 172.21.15.139:0/1344600326 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5625b758ab00 0x5625b758cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.937+0000 7f0e619ca740 4 client.26333 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:53:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.937+0000 7f0e619ca740 1 -- 172.21.15.139:0/1344600326 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625b74e1a20 con 0x5625b758ab00 2024-09-16T01:53:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.938+0000 7f0e5c9ca640 1 --2- 172.21.15.139:0/1344600326 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5625b758ab00 0x5625b758cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.946+0000 7f0e5c9ca640 1 --2- 172.21.15.139:0/1344600326 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5625b758ab00 0x5625b758cef0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d3c76d2e448b1a2 server_cookie=22ce48bf9edbacdb in_seq=0 out_seq=0 2024-09-16T01:53:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.948+0000 7f0e37fff640 10 client.26333 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:49.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.952+0000 7f0e37fff640 1 -- 172.21.15.139:0/1344600326 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625b74e1a20 con 0x5625b758ab00 2024-09-16T01:53:49.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.952+0000 7f0e37fff640 10 client.26333 handle_command_reply: tid=0 2024-09-16T01:53:49.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.953+0000 7f0e619ca740 1 client.26333 shutdown 2024-09-16T01:53:49.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.957+0000 7f0e619ca740 1 -- 172.21.15.139:0/1344600326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b757acb0 msgr2=0x5625b7578510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:49.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.957+0000 7f0e619ca740 1 --2- 172.21.15.139:0/1344600326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b757acb0 0x5625b7578510 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0e50000c00 tx=0x7f0e50009c50 comp rx=0 tx=0).stop 2024-09-16T01:53:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.961+0000 7f0e619ca740 1 -- 172.21.15.139:0/1344600326 shutdown_connections 2024-09-16T01:53:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.961+0000 7f0e619ca740 1 --2- 172.21.15.139:0/1344600326 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5625b758ab00 0x5625b758cef0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.961+0000 7f0e619ca740 1 --2- 172.21.15.139:0/1344600326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b757acb0 0x5625b7578510 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.961+0000 7f0e619ca740 1 --2- 172.21.15.139:0/1344600326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b7570410 0x5625b7577fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.961+0000 7f0e619ca740 1 --2- 172.21.15.139:0/1344600326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b756faf0 0x5625b7577a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.961+0000 7f0e619ca740 1 -- 172.21.15.139:0/1344600326 >> 172.21.15.139:0/1344600326 conn(0x5625b756c060 msgr2=0x5625b7571dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.964+0000 7f0e37fff640 0 client.26333 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.965+0000 7f0e619ca740 1 -- 172.21.15.139:0/1344600326 shutdown_connections 2024-09-16T01:53:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.967+0000 7f0e619ca740 1 -- 172.21.15.139:0/1344600326 wait complete. 2024-09-16T01:53:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.967+0000 7f0e619ca740 20 client.26333 trim_cache size 0 max 16384 2024-09-16T01:53:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.999+0000 7f0e619ca740 1 Processor -- start 2024-09-16T01:53:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:49.999+0000 7f0e619ca740 1 -- start start 2024-09-16T01:53:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.000+0000 7f0e619ca740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b756faf0 0x5625b7578340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.000+0000 7f0e619ca740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b7570410 0x5625b75926e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.000+0000 7f0e5c9ca640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b756faf0 0x5625b7578340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.000+0000 7f0e5c9ca640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b756faf0 0x5625b7578340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:49400/0 (socket says 172.21.15.139:49400) 2024-09-16T01:53:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.000+0000 7f0e5c9ca640 1 -- 172.21.15.139:0/4101519196 learned_addr learned my addr 172.21.15.139:0/4101519196 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.000+0000 7f0e619ca740 1 --2- 172.21.15.139:0/4101519196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b757acb0 0x5625b7592c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.000+0000 7f0e619ca740 1 -- 172.21.15.139:0/4101519196 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0e581c70b0 con 0x5625b7570410 2024-09-16T01:53:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.000+0000 7f0e619ca740 1 -- 172.21.15.139:0/4101519196 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e581c8e60 con 0x5625b757acb0 2024-09-16T01:53:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.000+0000 7f0e5d1cb640 1 --2- 172.21.15.139:0/4101519196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b757acb0 0x5625b7592c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.001+0000 7f0e619ca740 1 -- 172.21.15.139:0/4101519196 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0e581c77b0 con 0x5625b756faf0 2024-09-16T01:53:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.001+0000 7f0e57fff640 1 --2- 172.21.15.139:0/4101519196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b7570410 0x5625b75926e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.001+0000 7f0e5c9ca640 1 -- 172.21.15.139:0/4101519196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b757acb0 msgr2=0x5625b7592c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.001+0000 7f0e5c9ca640 1 --2- 172.21.15.139:0/4101519196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b757acb0 0x5625b7592c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.001+0000 7f0e5c9ca640 1 -- 172.21.15.139:0/4101519196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b7570410 msgr2=0x5625b75926e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.001+0000 7f0e5c9ca640 1 --2- 172.21.15.139:0/4101519196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b7570410 0x5625b75926e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.001+0000 7f0e5c9ca640 1 -- 172.21.15.139:0/4101519196 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b756be90 con 0x5625b756faf0 2024-09-16T01:53:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.002+0000 7f0e5c9ca640 1 --2- 172.21.15.139:0/4101519196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b756faf0 0x5625b7578340 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009ca0 tx=0x7f0e480098e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45c8bb0663bb6fe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.004+0000 7f0e5d1cb640 1 --2- 172.21.15.139:0/4101519196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b757acb0 0x5625b7592c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:53:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.004+0000 7f0e4e7fc640 1 -- 172.21.15.139:0/4101519196 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48031b00 con 0x5625b756faf0 2024-09-16T01:53:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.004+0000 7f0e4e7fc640 1 -- 172.21.15.139:0/4101519196 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e48036040 con 0x5625b756faf0 2024-09-16T01:53:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.004+0000 7f0e4e7fc640 1 -- 172.21.15.139:0/4101519196 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48004040 con 0x5625b756faf0 2024-09-16T01:53:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.006+0000 7f0e619ca740 1 -- 172.21.15.139:0/4101519196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b756faf0 msgr2=0x5625b7578340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.006+0000 7f0e619ca740 1 --2- 172.21.15.139:0/4101519196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b756faf0 0x5625b7578340 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009ca0 tx=0x7f0e480098e0 comp rx=0 tx=0).stop 2024-09-16T01:53:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.008+0000 7f0e619ca740 1 -- 172.21.15.139:0/4101519196 shutdown_connections 2024-09-16T01:53:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.008+0000 7f0e619ca740 1 --2- 172.21.15.139:0/4101519196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b757acb0 0x5625b7592c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.008+0000 7f0e619ca740 1 --2- 172.21.15.139:0/4101519196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b7570410 0x5625b75926e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.008+0000 7f0e619ca740 1 --2- 172.21.15.139:0/4101519196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b756faf0 0x5625b7578340 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.008+0000 7f0e619ca740 1 -- 172.21.15.139:0/4101519196 >> 172.21.15.139:0/4101519196 conn(0x5625b756c060 msgr2=0x5625b7579b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.010+0000 7f0e619ca740 1 -- 172.21.15.139:0/4101519196 shutdown_connections 2024-09-16T01:53:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.010+0000 7f0e619ca740 1 -- 172.21.15.139:0/4101519196 wait complete. 2024-09-16T01:53:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.011+0000 7f0e619ca740 1 -- start start 2024-09-16T01:53:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.011+0000 7f0e619ca740 1 Processor -- start 2024-09-16T01:53:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.012+0000 7f0e619ca740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b756faf0 0x5625b757b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.012+0000 7f0e619ca740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b7570410 0x5625b757b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.012+0000 7f0e5c9ca640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b756faf0 0x5625b757b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.012+0000 7f0e5c9ca640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b756faf0 0x5625b757b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:36098/0 (socket says 172.21.15.139:36098) 2024-09-16T01:53:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.012+0000 7f0e619ca740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b757acb0 0x5625b7581d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.012+0000 7f0e5c9ca640 1 -- 172.21.15.139:0/3302510014 learned_addr learned my addr 172.21.15.139:0/3302510014 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.013+0000 7f0e619ca740 1 -- 172.21.15.139:0/3302510014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0e581c86d0 con 0x5625b757acb0 2024-09-16T01:53:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.013+0000 7f0e619ca740 1 -- 172.21.15.139:0/3302510014 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e581c7f40 con 0x5625b756faf0 2024-09-16T01:53:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.013+0000 7f0e619ca740 1 -- 172.21.15.139:0/3302510014 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5806e170 con 0x5625b7570410 2024-09-16T01:53:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.013+0000 7f0e619ca740 10 client.0 _ll_register_callbacks cb 0x7f0e500095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:53:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.013+0000 7f0e5c9ca640 1 -- 172.21.15.139:0/3302510014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b7570410 msgr2=0x5625b757b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.013+0000 7f0e5c9ca640 1 --2- 172.21.15.139:0/3302510014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b7570410 0x5625b757b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.013+0000 7f0e5c9ca640 1 -- 172.21.15.139:0/3302510014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b757acb0 msgr2=0x5625b7581d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.013+0000 7f0e5c9ca640 1 --2- 172.21.15.139:0/3302510014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b757acb0 0x5625b7581d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.013+0000 7f0e5c9ca640 1 -- 172.21.15.139:0/3302510014 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b757c280 con 0x5625b756faf0 2024-09-16T01:53:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.016+0000 7f0e5c9ca640 1 --2- 172.21.15.139:0/3302510014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b756faf0 0x5625b757b4b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0e48003ba0 tx=0x7f0e48003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7944962726ab628e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.017+0000 7f0e37fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:53:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.021+0000 7f0e37fff640 1 -- 172.21.15.139:0/3302510014 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48031d10 con 0x5625b756faf0 2024-09-16T01:53:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.021+0000 7f0e37fff640 1 -- 172.21.15.139:0/3302510014 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f0e48031460 con 0x5625b756faf0 2024-09-16T01:53:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.021+0000 7f0e619ca740 1 -- 172.21.15.139:0/3302510014 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e48036040 con 0x5625b756faf0 2024-09-16T01:53:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.023+0000 7f0e37fff640 1 -- 172.21.15.139:0/3302510014 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f0e480316a0 con 0x5625b756faf0 2024-09-16T01:53:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.023+0000 7f0e619ca740 10 client.26336 fetch_fsmap learned FSMap version 69 2024-09-16T01:53:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.023+0000 7f0e619ca740 1 -- 172.21.15.139:0/3302510014 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5625b7583300 con 0x5625b756faf0 2024-09-16T01:53:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.024+0000 7f0e37fff640 1 -- 172.21.15.139:0/3302510014 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e48031840 con 0x5625b756faf0 2024-09-16T01:53:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.024+0000 7f0e37fff640 1 -- 172.21.15.139:0/3302510014 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7f0e48044070 con 0x5625b756faf0 2024-09-16T01:53:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.024+0000 7f0e619ca740 10 client.26336 fetch_fsmap finished waiting for FSMap version 69 2024-09-16T01:53:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.024+0000 7f0e619ca740 10 client.26336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:53:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.024+0000 7f0e619ca740 20 client.26336 populate_metadata read hostname 'smithi139' 2024-09-16T01:53:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.024+0000 7f0e619ca740 1 --2- 172.21.15.139:0/3302510014 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5625b75f56b0 0x5625b75f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.024+0000 7f0e619ca740 4 client.26336 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:53:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.024+0000 7f0e619ca740 1 -- 172.21.15.139:0/3302510014 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0e48036040 con 0x5625b75f56b0 2024-09-16T01:53:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.025+0000 7f0e57fff640 1 --2- 172.21.15.139:0/3302510014 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5625b75f56b0 0x5625b75f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.026+0000 7f0e57fff640 1 --2- 172.21.15.139:0/3302510014 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5625b75f56b0 0x5625b75f7aa0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d08c20697d1ac167 server_cookie=a5a529a4660fac81 in_seq=0 out_seq=0 2024-09-16T01:53:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:50.026+0000 7f0e37fff640 10 client.26336 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:50.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:50 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:53:50.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:50 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/4191058154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:53:50.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:50 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:53:50.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:50 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/4191058154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:53:50.624 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 131.4 --await-for 45.0' 2024-09-16T01:53:50.624 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 131.4 --await-for 45.0 2024-09-16T01:53:50.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:50 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:53:50.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:50 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/4191058154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:53:51.161 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:53:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.198+0000 7f0e37fff640 1 -- 172.21.15.139:0/3302510014 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f0e48036040 con 0x5625b75f56b0 2024-09-16T01:53:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.198+0000 7f0e37fff640 10 client.26336 handle_command_reply: tid=0 2024-09-16T01:53:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.200+0000 7f0e619ca740 1 client.26336 shutdown 2024-09-16T01:53:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.200+0000 7f0e619ca740 1 -- 172.21.15.139:0/3302510014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b756faf0 msgr2=0x5625b757b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.200+0000 7f0e619ca740 1 --2- 172.21.15.139:0/3302510014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b756faf0 0x5625b757b4b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0e48003ba0 tx=0x7f0e48003bd0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.200+0000 7f0e619ca740 1 -- 172.21.15.139:0/3302510014 shutdown_connections 2024-09-16T01:53:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.200+0000 7f0e619ca740 1 --2- 172.21.15.139:0/3302510014 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5625b75f56b0 0x5625b75f7aa0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.200+0000 7f0e619ca740 1 --2- 172.21.15.139:0/3302510014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5625b757acb0 0x5625b7581d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.200+0000 7f0e619ca740 1 --2- 172.21.15.139:0/3302510014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625b7570410 0x5625b757b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.200+0000 7f0e619ca740 1 --2- 172.21.15.139:0/3302510014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625b756faf0 0x5625b757b4b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.200+0000 7f0e619ca740 1 -- 172.21.15.139:0/3302510014 >> 172.21.15.139:0/3302510014 conn(0x5625b756c060 msgr2=0x5625b75714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.200+0000 7f0e37fff640 0 client.26336 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.201+0000 7f0e619ca740 1 -- 172.21.15.139:0/3302510014 shutdown_connections 2024-09-16T01:53:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.201+0000 7f0e619ca740 1 -- 172.21.15.139:0/3302510014 wait complete. 2024-09-16T01:53:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.202+0000 7f0e619ca740 20 client.26336 trim_cache size 0 max 16384 2024-09-16T01:53:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:53:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:53:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:53:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:53:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.210+0000 7f0e5ec2e640 1 -- 172.21.15.139:0/854308307 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e440a35b0 msgr2=0x7f0e440a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.210+0000 7f0e5ec2e640 1 --2- 172.21.15.139:0/854308307 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e440a35b0 0x7f0e440a5a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0e50011d40 tx=0x7f0e5001d040 comp rx=0 tx=0).stop 2024-09-16T01:53:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.210+0000 7f0e5ec2e640 1 -- 172.21.15.139:0/854308307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e58102480 msgr2=0x7f0e5806cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.210+0000 7f0e5ec2e640 1 --2- 172.21.15.139:0/854308307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e58102480 0x7f0e5806cc40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000b4d0 tx=0x7f0e4000b990 comp rx=0 tx=0).stop 2024-09-16T01:53:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.213+0000 7f0e5ec2e640 1 -- 172.21.15.139:0/854308307 shutdown_connections 2024-09-16T01:53:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.213+0000 7f0e5ec2e640 1 --2- 172.21.15.139:0/854308307 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e440a35b0 0x7f0e440a5a70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.213+0000 7f0e5ec2e640 1 --2- 172.21.15.139:0/854308307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e5806d180 0x7f0e5806d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.213+0000 7f0e5ec2e640 1 --2- 172.21.15.139:0/854308307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e58102480 0x7f0e5806cc40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.213+0000 7f0e5ec2e640 1 --2- 172.21.15.139:0/854308307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e58101ae0 0x7f0e58075c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.213+0000 7f0e5ec2e640 1 -- 172.21.15.139:0/854308307 >> 172.21.15.139:0/854308307 conn(0x7f0e580fd7e0 msgr2=0x7f0e580fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.218+0000 7f0e5ec2e640 1 -- 172.21.15.139:0/854308307 shutdown_connections 2024-09-16T01:53:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:53:51.218+0000 7f0e5ec2e640 1 -- 172.21.15.139:0/854308307 wait complete. 2024-09-16T01:53:51.490 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:51 smithi139 ceph-mon[31175]: pgmap v424: 129 pgs: 19 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 53 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 219 op/s 2024-09-16T01:53:51.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:51 smithi164 ceph-mon[35121]: pgmap v424: 129 pgs: 19 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 53 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 219 op/s 2024-09-16T01:53:51.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:51 smithi176 ceph-mon[35324]: pgmap v424: 129 pgs: 19 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 53 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 219 op/s 2024-09-16T01:53:51.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.843+0000 7fa76b7fe640 1 -- 172.21.15.139:0/1055780277 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764034020 con 0x7fa76c074590 2024-09-16T01:53:51.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.843+0000 7fa771d20640 1 -- 172.21.15.139:0/1055780277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa76c074590 msgr2=0x7fa76c074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:51.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.843+0000 7fa771d20640 1 --2- 172.21.15.139:0/1055780277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa76c074590 0x7fa76c074970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa764009ca0 tx=0x7fa76402a940 comp rx=0 tx=0).stop 2024-09-16T01:53:51.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.843+0000 7fa771d20640 1 -- 172.21.15.139:0/1055780277 shutdown_connections 2024-09-16T01:53:51.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.843+0000 7fa771d20640 1 --2- 172.21.15.139:0/1055780277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa76c1129b0 0x7fa76c114da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.843+0000 7fa771d20640 1 --2- 172.21.15.139:0/1055780277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa76c074eb0 0x7fa76c112470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.843+0000 7fa771d20640 1 --2- 172.21.15.139:0/1055780277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa76c074590 0x7fa76c074970 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.843+0000 7fa771d20640 1 -- 172.21.15.139:0/1055780277 >> 172.21.15.139:0/1055780277 conn(0x7fa76c0fd680 msgr2=0x7fa76c0ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:51.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.844+0000 7fa771d20640 1 -- 172.21.15.139:0/1055780277 shutdown_connections 2024-09-16T01:53:51.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.844+0000 7fa771d20640 1 -- 172.21.15.139:0/1055780277 wait complete. 2024-09-16T01:53:51.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.845+0000 7fa771d20640 1 Processor -- start 2024-09-16T01:53:51.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.845+0000 7fa771d20640 1 -- start start 2024-09-16T01:53:51.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.845+0000 7fa771d20640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa76c074eb0 0x7fa76c073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.845+0000 7fa771d20640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa76c1129b0 0x7fa76c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.845+0000 7fa771d20640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa76c06d1a0 0x7fa76c06d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.845+0000 7fa771d20640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c118430 con 0x7fa76c074eb0 2024-09-16T01:53:51.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.845+0000 7fa771d20640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1182b0 con 0x7fa76c1129b0 2024-09-16T01:53:51.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.845+0000 7fa771d20640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1185b0 con 0x7fa76c06d1a0 2024-09-16T01:53:51.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.845+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa76c1129b0 0x7fa76c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.845+0000 7fa77151f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa76c06d1a0 0x7fa76c06d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.845+0000 7fa77151f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa76c06d1a0 0x7fa76c06d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:49418/0 (socket says 172.21.15.139:49418) 2024-09-16T01:53:51.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.845+0000 7fa77151f640 1 -- 172.21.15.139:0/1795542221 learned_addr learned my addr 172.21.15.139:0/1795542221 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:51.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.846+0000 7fa76bfff640 1 -- 172.21.15.139:0/1795542221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa76c06d1a0 msgr2=0x7fa76c06d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:51.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.846+0000 7fa76bfff640 1 --2- 172.21.15.139:0/1795542221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa76c06d1a0 0x7fa76c06d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.846+0000 7fa76bfff640 1 -- 172.21.15.139:0/1795542221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa76c074eb0 msgr2=0x7fa76c073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:51.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.846+0000 7fa76bfff640 1 --2- 172.21.15.139:0/1795542221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa76c074eb0 0x7fa76c073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.846+0000 7fa76bfff640 1 -- 172.21.15.139:0/1795542221 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c071800 con 0x7fa76c1129b0 2024-09-16T01:53:51.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.846+0000 7fa76bfff640 1 --2- 172.21.15.139:0/1795542221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa76c1129b0 0x7fa76c06cc60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa76000b3f0 tx=0x7fa76000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:51.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.846+0000 7fa769ffb640 1 -- 172.21.15.139:0/1795542221 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760015020 con 0x7fa76c1129b0 2024-09-16T01:53:51.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.847+0000 7fa771d20640 1 -- 172.21.15.139:0/1795542221 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76c071af0 con 0x7fa76c1129b0 2024-09-16T01:53:51.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.847+0000 7fa771d20640 1 -- 172.21.15.139:0/1795542221 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76c072080 con 0x7fa76c1129b0 2024-09-16T01:53:51.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.848+0000 7fa769ffb640 1 -- 172.21.15.139:0/1795542221 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa760011d10 con 0x7fa76c1129b0 2024-09-16T01:53:51.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.848+0000 7fa769ffb640 1 -- 172.21.15.139:0/1795542221 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76000f2e0 con 0x7fa76c1129b0 2024-09-16T01:53:51.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.853+0000 7fa77535b740 1 Processor -- start 2024-09-16T01:53:51.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.853+0000 7fa77535b740 1 -- start start 2024-09-16T01:53:51.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.854+0000 7fa77535b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68326900 0x561a68326ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.854+0000 7fa77535b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68327220 0x561a683315a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.854+0000 7fa77535b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68331ae0 0x561a68333ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.854+0000 7fa77535b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561a6828c2a0 con 0x561a68331ae0 2024-09-16T01:53:51.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.854+0000 7fa77535b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561a68233b10 con 0x561a68327220 2024-09-16T01:53:51.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.854+0000 7fa77535b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561a68288260 con 0x561a68326900 2024-09-16T01:53:51.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.854+0000 7fa770d1e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68327220 0x561a683315a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.854+0000 7fa769ffb640 1 -- 172.21.15.139:0/1795542221 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa760018020 con 0x7fa76c1129b0 2024-09-16T01:53:51.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.854+0000 7fa77151f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68326900 0x561a68326ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.854+0000 7fa770d1e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68327220 0x561a683315a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:36110/0 (socket says 172.21.15.139:36110) 2024-09-16T01:53:51.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.854+0000 7fa770d1e640 1 -- 172.21.15.139:0/2074794088 learned_addr learned my addr 172.21.15.139:0/2074794088 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:51.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.855+0000 7fa770d1e640 1 -- 172.21.15.139:0/2074794088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68326900 msgr2=0x561a68326ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:51.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.855+0000 7fa770d1e640 1 --2- 172.21.15.139:0/2074794088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68326900 0x561a68326ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.855+0000 7fa770d1e640 1 -- 172.21.15.139:0/2074794088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68331ae0 msgr2=0x561a68333ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:53:51.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.855+0000 7fa770d1e640 1 --2- 172.21.15.139:0/2074794088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68331ae0 0x561a68333ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.855+0000 7fa770d1e640 1 -- 172.21.15.139:0/2074794088 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a67ffdb90 con 0x561a68327220 2024-09-16T01:53:51.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.855+0000 7fa769ffb640 1 --2- 172.21.15.139:0/1795542221 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa7540a3430 0x7fa7540a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.855+0000 7fa770d1e640 1 --2- 172.21.15.139:0/2074794088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68327220 0x561a683315a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa76402fab0 tx=0x7fa76402fae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f10f2f9273f9fa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:51.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.856+0000 7fa769ffb640 1 -- 172.21.15.139:0/1795542221 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7fa7600997b0 con 0x7fa76c1129b0 2024-09-16T01:53:51.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.856+0000 7fa77151f640 1 --2- 172.21.15.139:0/1795542221 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa7540a3430 0x7fa7540a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.857+0000 7fa77151f640 1 --2- 172.21.15.139:0/1795542221 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa7540a3430 0x7fa7540a58f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0058c0 tx=0x7fa75c005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:51.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.860+0000 7fa752ffd640 1 -- 172.21.15.139:0/2074794088 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764034040 con 0x561a68327220 2024-09-16T01:53:51.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.860+0000 7fa752ffd640 1 -- 172.21.15.139:0/2074794088 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa76403e030 con 0x561a68327220 2024-09-16T01:53:51.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.861+0000 7fa752ffd640 1 -- 172.21.15.139:0/2074794088 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7640308a0 con 0x561a68327220 2024-09-16T01:53:51.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.861+0000 7fa77535b740 1 -- 172.21.15.139:0/2074794088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68327220 msgr2=0x561a683315a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:51.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.861+0000 7fa77535b740 1 --2- 172.21.15.139:0/2074794088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68327220 0x561a683315a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa76402fab0 tx=0x7fa76402fae0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.862+0000 7fa77535b740 1 -- 172.21.15.139:0/2074794088 shutdown_connections 2024-09-16T01:53:51.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.862+0000 7fa77535b740 1 --2- 172.21.15.139:0/2074794088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68331ae0 0x561a68333ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.862+0000 7fa77535b740 1 --2- 172.21.15.139:0/2074794088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68327220 0x561a683315a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.862+0000 7fa77535b740 1 --2- 172.21.15.139:0/2074794088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68326900 0x561a68326ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.862+0000 7fa77535b740 1 -- 172.21.15.139:0/2074794088 >> 172.21.15.139:0/2074794088 conn(0x561a68322e90 msgr2=0x561a68323270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:51.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.862+0000 7fa77535b740 1 -- 172.21.15.139:0/2074794088 shutdown_connections 2024-09-16T01:53:51.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.862+0000 7fa77535b740 1 -- 172.21.15.139:0/2074794088 wait complete. 2024-09-16T01:53:51.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.863+0000 7fa77535b740 1 -- start start 2024-09-16T01:53:51.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.863+0000 7fa77535b740 1 Processor -- start 2024-09-16T01:53:51.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.864+0000 7fa77535b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68326900 0x561a6832ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.864+0000 7fa77535b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68327220 0x561a6832ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.864+0000 7fa77535b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68331ae0 0x561a6832f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.864+0000 7fa77535b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561a683346b0 con 0x561a68331ae0 2024-09-16T01:53:51.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.864+0000 7fa77151f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68331ae0 0x561a6832f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.864+0000 7fa77151f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68331ae0 0x561a6832f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55050/0 (socket says 172.21.15.139:55050) 2024-09-16T01:53:51.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.864+0000 7fa77151f640 1 -- 172.21.15.139:0/4156283081 learned_addr learned my addr 172.21.15.139:0/4156283081 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:51.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.864+0000 7fa77535b740 1 -- 172.21.15.139:0/4156283081 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561a682146e0 con 0x561a68326900 2024-09-16T01:53:51.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.864+0000 7fa77535b740 1 -- 172.21.15.139:0/4156283081 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561a68334830 con 0x561a68327220 2024-09-16T01:53:51.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.865+0000 7fa77535b740 10 client.0 _ll_register_callbacks cb 0x561a68238a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:53:51.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.865+0000 7fa76bfff640 1 --2- 172.21.15.139:0/4156283081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68327220 0x561a6832ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.866+0000 7fa77151f640 1 -- 172.21.15.139:0/4156283081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68327220 msgr2=0x561a6832ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:51.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.866+0000 7fa77151f640 1 --2- 172.21.15.139:0/4156283081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68327220 0x561a6832ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.866+0000 7fa77151f640 1 -- 172.21.15.139:0/4156283081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68326900 msgr2=0x561a6832ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:51.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.866+0000 7fa77151f640 1 --2- 172.21.15.139:0/4156283081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68326900 0x561a6832ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.866+0000 7fa77151f640 1 -- 172.21.15.139:0/4156283081 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a6833dc80 con 0x561a68331ae0 2024-09-16T01:53:51.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.870+0000 7fa77151f640 1 --2- 172.21.15.139:0/4156283081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68331ae0 0x561a6832f480 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa75c01c8d0 tx=0x7fa75c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e885db87b8a9f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:51.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.870+0000 7fa74bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:53:51.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.871+0000 7fa74bfff640 1 -- 172.21.15.139:0/4156283081 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c02c060 con 0x561a68331ae0 2024-09-16T01:53:51.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.871+0000 7fa74bfff640 1 -- 172.21.15.139:0/4156283081 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7fa75c016030 con 0x561a68331ae0 2024-09-16T01:53:51.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.871+0000 7fa77535b740 1 -- 172.21.15.139:0/4156283081 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa76403e030 con 0x561a68331ae0 2024-09-16T01:53:51.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.872+0000 7fa74bfff640 1 -- 172.21.15.139:0/4156283081 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fa75c027420 con 0x561a68331ae0 2024-09-16T01:53:51.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.872+0000 7fa77535b740 10 client.16902 fetch_fsmap learned FSMap version 69 2024-09-16T01:53:51.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.872+0000 7fa77535b740 1 -- 172.21.15.139:0/4156283081 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x561a68340670 con 0x561a68331ae0 2024-09-16T01:53:51.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.873+0000 7fa74bfff640 1 -- 172.21.15.139:0/4156283081 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa76403e030 con 0x561a68331ae0 2024-09-16T01:53:51.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.873+0000 7fa74bfff640 1 -- 172.21.15.139:0/4156283081 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7fa75c00f7c0 con 0x561a68331ae0 2024-09-16T01:53:51.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.873+0000 7fa77535b740 10 client.16902 fetch_fsmap finished waiting for FSMap version 69 2024-09-16T01:53:51.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.873+0000 7fa77535b740 10 client.16902 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:53:51.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.873+0000 7fa77535b740 20 client.16902 populate_metadata read hostname 'smithi139' 2024-09-16T01:53:51.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.873+0000 7fa77535b740 1 --2- 172.21.15.139:0/4156283081 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a68341bd0 0x561a68344060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.873+0000 7fa77535b740 4 client.16902 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:53:51.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.873+0000 7fa77535b740 1 -- 172.21.15.139:0/4156283081 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a682999d0 con 0x561a68341bd0 2024-09-16T01:53:51.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.877+0000 7fa770d1e640 1 --2- 172.21.15.139:0/4156283081 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a68341bd0 0x561a68344060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.880+0000 7fa770d1e640 1 --2- 172.21.15.139:0/4156283081 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a68341bd0 0x561a68344060 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5771b6ce7af97b server_cookie=253ec07660a0ad9c in_seq=0 out_seq=0 2024-09-16T01:53:51.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.880+0000 7fa74bfff640 10 client.16902 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:51.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.883+0000 7fa74bfff640 1 -- 172.21.15.139:0/4156283081 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a682999d0 con 0x561a68341bd0 2024-09-16T01:53:51.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.883+0000 7fa74bfff640 10 client.16902 handle_command_reply: tid=0 2024-09-16T01:53:51.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.883+0000 7fa77535b740 1 client.16902 shutdown 2024-09-16T01:53:51.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.886+0000 7fa77535b740 1 -- 172.21.15.139:0/4156283081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68331ae0 msgr2=0x561a6832f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:51.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.886+0000 7fa77535b740 1 --2- 172.21.15.139:0/4156283081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68331ae0 0x561a6832f480 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa75c01c8d0 tx=0x7fa75c004730 comp rx=0 tx=0).stop 2024-09-16T01:53:51.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.887+0000 7fa77535b740 1 -- 172.21.15.139:0/4156283081 shutdown_connections 2024-09-16T01:53:51.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.887+0000 7fa77535b740 1 --2- 172.21.15.139:0/4156283081 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a68341bd0 0x561a68344060 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.887+0000 7fa77535b740 1 --2- 172.21.15.139:0/4156283081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68331ae0 0x561a6832f480 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.887+0000 7fa77535b740 1 --2- 172.21.15.139:0/4156283081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68327220 0x561a6832ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.887+0000 7fa77535b740 1 --2- 172.21.15.139:0/4156283081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68326900 0x561a6832ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.887+0000 7fa77535b740 1 -- 172.21.15.139:0/4156283081 >> 172.21.15.139:0/4156283081 conn(0x561a68322e90 msgr2=0x561a68328bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:51.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.887+0000 7fa74bfff640 0 client.16902 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:51.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.887+0000 7fa77535b740 1 -- 172.21.15.139:0/4156283081 shutdown_connections 2024-09-16T01:53:51.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.887+0000 7fa77535b740 1 -- 172.21.15.139:0/4156283081 wait complete. 2024-09-16T01:53:51.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.888+0000 7fa77535b740 20 client.16902 trim_cache size 0 max 16384 2024-09-16T01:53:51.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.920+0000 7fa77535b740 1 Processor -- start 2024-09-16T01:53:51.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.921+0000 7fa77535b740 1 -- start start 2024-09-16T01:53:51.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.921+0000 7fa77535b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68326900 0x561a68330970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.921+0000 7fa77535b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68327220 0x561a68349970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.921+0000 7fa77535b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68331ae0 0x561a68349eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.921+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68327220 0x561a68349970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.921+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68327220 0x561a68349970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55060/0 (socket says 172.21.15.139:55060) 2024-09-16T01:53:51.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.922+0000 7fa76bfff640 1 -- 172.21.15.139:0/3238250322 learned_addr learned my addr 172.21.15.139:0/3238250322 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:51.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.921+0000 7fa770d1e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68326900 0x561a68330970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.922+0000 7fa77151f640 1 --2- 172.21.15.139:0/3238250322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68331ae0 0x561a68349eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.922+0000 7fa77535b740 1 -- 172.21.15.139:0/3238250322 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa758010310 con 0x561a68327220 2024-09-16T01:53:51.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.922+0000 7fa77535b740 1 -- 172.21.15.139:0/3238250322 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580120c0 con 0x561a68331ae0 2024-09-16T01:53:51.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.922+0000 7fa77535b740 1 -- 172.21.15.139:0/3238250322 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa758010a10 con 0x561a68326900 2024-09-16T01:53:51.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.924+0000 7fa770d1e640 1 -- 172.21.15.139:0/3238250322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68331ae0 msgr2=0x561a68349eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:51.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.924+0000 7fa770d1e640 1 --2- 172.21.15.139:0/3238250322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68331ae0 0x561a68349eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.924+0000 7fa770d1e640 1 -- 172.21.15.139:0/3238250322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68327220 msgr2=0x561a68349970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:51.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.924+0000 7fa770d1e640 1 --2- 172.21.15.139:0/3238250322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68327220 0x561a68349970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.924+0000 7fa770d1e640 1 -- 172.21.15.139:0/3238250322 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a6833e350 con 0x561a68326900 2024-09-16T01:53:51.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.924+0000 7fa77151f640 1 --2- 172.21.15.139:0/3238250322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68331ae0 0x561a68349eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:53:51.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.924+0000 7fa770d1e640 1 --2- 172.21.15.139:0/3238250322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68326900 0x561a68330970 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa764009ca0 tx=0x7fa76402fe30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e7a0995c22feede server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:51.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.925+0000 7fa7527fc640 1 -- 172.21.15.139:0/3238250322 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764034070 con 0x561a68326900 2024-09-16T01:53:51.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.925+0000 7fa7527fc640 1 -- 172.21.15.139:0/3238250322 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa76403e040 con 0x561a68326900 2024-09-16T01:53:51.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.925+0000 7fa7527fc640 1 -- 172.21.15.139:0/3238250322 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76404b3f0 con 0x561a68326900 2024-09-16T01:53:51.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.926+0000 7fa77535b740 1 -- 172.21.15.139:0/3238250322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68326900 msgr2=0x561a68330970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:51.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.926+0000 7fa77535b740 1 --2- 172.21.15.139:0/3238250322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68326900 0x561a68330970 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa764009ca0 tx=0x7fa76402fe30 comp rx=0 tx=0).stop 2024-09-16T01:53:51.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.926+0000 7fa77535b740 1 -- 172.21.15.139:0/3238250322 shutdown_connections 2024-09-16T01:53:51.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.926+0000 7fa77535b740 1 --2- 172.21.15.139:0/3238250322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68331ae0 0x561a68349eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.926+0000 7fa77535b740 1 --2- 172.21.15.139:0/3238250322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68327220 0x561a68349970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.926+0000 7fa77535b740 1 --2- 172.21.15.139:0/3238250322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68326900 0x561a68330970 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.926+0000 7fa77535b740 1 -- 172.21.15.139:0/3238250322 >> 172.21.15.139:0/3238250322 conn(0x561a68322e90 msgr2=0x561a68330260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:51.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.929+0000 7fa77535b740 1 -- 172.21.15.139:0/3238250322 shutdown_connections 2024-09-16T01:53:51.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.929+0000 7fa77535b740 1 -- 172.21.15.139:0/3238250322 wait complete. 2024-09-16T01:53:51.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.930+0000 7fa77535b740 1 -- start start 2024-09-16T01:53:51.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.930+0000 7fa77535b740 1 Processor -- start 2024-09-16T01:53:51.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.931+0000 7fa77535b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68326900 0x561a68336190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.931+0000 7fa77535b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68327220 0x561a683366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.931+0000 7fa77535b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68331ae0 0x561a683a5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.931+0000 7fa77535b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa758011930 con 0x561a68326900 2024-09-16T01:53:51.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.931+0000 7fa77535b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580111a0 con 0x561a68327220 2024-09-16T01:53:51.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.931+0000 7fa77535b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa75800e310 con 0x561a68331ae0 2024-09-16T01:53:51.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.931+0000 7fa77535b740 10 client.0 _ll_register_callbacks cb 0x7fa75c01c580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:53:51.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.931+0000 7fa77151f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68331ae0 0x561a683a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.931+0000 7fa77151f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68331ae0 0x561a683a5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:49444/0 (socket says 172.21.15.139:49444) 2024-09-16T01:53:51.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.931+0000 7fa77151f640 1 -- 172.21.15.139:0/3386005386 learned_addr learned my addr 172.21.15.139:0/3386005386 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:53:51.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.931+0000 7fa770d1e640 1 --2- 172.21.15.139:0/3386005386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68326900 0x561a68336190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.932+0000 7fa76bfff640 1 --2- 172.21.15.139:0/3386005386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68327220 0x561a683366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.932+0000 7fa770d1e640 1 -- 172.21.15.139:0/3386005386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68331ae0 msgr2=0x561a683a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:51.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.932+0000 7fa770d1e640 1 --2- 172.21.15.139:0/3386005386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68331ae0 0x561a683a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.932+0000 7fa770d1e640 1 -- 172.21.15.139:0/3386005386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68327220 msgr2=0x561a683366d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:53:51.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.932+0000 7fa770d1e640 1 --2- 172.21.15.139:0/3386005386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68327220 0x561a683366d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:51.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.932+0000 7fa770d1e640 1 -- 172.21.15.139:0/3386005386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a683a5fc0 con 0x561a68326900 2024-09-16T01:53:51.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.934+0000 7fa770d1e640 1 --2- 172.21.15.139:0/3386005386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68326900 0x561a68336190 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa76403d040 tx=0x7fa764005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c2b008325db5a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:53:51.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.934+0000 7fa74bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:53:51.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.934+0000 7fa74bfff640 1 -- 172.21.15.139:0/3386005386 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764034040 con 0x561a68326900 2024-09-16T01:53:51.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.935+0000 7fa74bfff640 1 -- 172.21.15.139:0/3386005386 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7fa76402f390 con 0x561a68326900 2024-09-16T01:53:51.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.935+0000 7fa77535b740 1 -- 172.21.15.139:0/3386005386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa76403e040 con 0x561a68326900 2024-09-16T01:53:51.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.935+0000 7fa74bfff640 1 -- 172.21.15.139:0/3386005386 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fa76402f5b0 con 0x561a68326900 2024-09-16T01:53:51.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.935+0000 7fa77535b740 10 client.16908 fetch_fsmap learned FSMap version 69 2024-09-16T01:53:51.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.935+0000 7fa77535b740 1 -- 172.21.15.139:0/3386005386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x561a683ad280 con 0x561a68326900 2024-09-16T01:53:51.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.936+0000 7fa74bfff640 1 -- 172.21.15.139:0/3386005386 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa76403e040 con 0x561a68326900 2024-09-16T01:53:51.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.936+0000 7fa74bfff640 1 -- 172.21.15.139:0/3386005386 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7fa76402f070 con 0x561a68326900 2024-09-16T01:53:51.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.936+0000 7fa77535b740 10 client.16908 fetch_fsmap finished waiting for FSMap version 69 2024-09-16T01:53:51.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.936+0000 7fa77535b740 10 client.16908 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:53:51.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.936+0000 7fa77535b740 20 client.16908 populate_metadata read hostname 'smithi139' 2024-09-16T01:53:51.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.936+0000 7fa77535b740 1 --2- 172.21.15.139:0/3386005386 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a683ad5c0 0x561a683af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:53:51.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.937+0000 7fa77535b740 4 client.16908 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.4, "await_for": 45.0}] 2024-09-16T01:53:51.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.937+0000 7fa77535b740 1 -- 172.21.15.139:0/3386005386 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.4, "await_for": 45.0}) -- 0x561a68293d20 con 0x561a683ad5c0 2024-09-16T01:53:51.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.938+0000 7fa76bfff640 1 --2- 172.21.15.139:0/3386005386 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a683ad5c0 0x561a683af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:53:51.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.944+0000 7fa76bfff640 1 --2- 172.21.15.139:0/3386005386 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a683ad5c0 0x561a683af9b0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7086003f18de0ae0 server_cookie=8cb69b369e381d22 in_seq=0 out_seq=0 2024-09-16T01:53:51.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:51.945+0000 7fa74bfff640 10 client.16908 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:53.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:53 smithi164 ceph-mon[35121]: pgmap v425: 129 pgs: 17 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 62 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 297 op/s 2024-09-16T01:53:53.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:53 smithi139 ceph-mon[31175]: pgmap v425: 129 pgs: 17 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 62 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 297 op/s 2024-09-16T01:53:53.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:53 smithi176 ceph-mon[35324]: pgmap v425: 129 pgs: 17 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 62 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 297 op/s 2024-09-16T01:53:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:55 smithi164 ceph-mon[35121]: pgmap v426: 129 pgs: 17 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 62 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 321 op/s 2024-09-16T01:53:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:55 smithi139 ceph-mon[31175]: pgmap v426: 129 pgs: 17 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 62 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 321 op/s 2024-09-16T01:53:55.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:55 smithi176 ceph-mon[35324]: pgmap v426: 129 pgs: 17 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 62 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 321 op/s 2024-09-16T01:53:56.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:56 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:53:56.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:56 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:53:56.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:56 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:53:56.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:56 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:53:56.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:56 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:53:56.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:56 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:53:56.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:56 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:53:56.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:56 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:53:56.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:56 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:53:56.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:56 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:53:56.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:56 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:53:56.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:56 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:53:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:57 smithi164 ceph-mon[35121]: pgmap v427: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 329 op/s 2024-09-16T01:53:57.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:57 smithi139 ceph-mon[31175]: pgmap v427: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 329 op/s 2024-09-16T01:53:57.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:57 smithi176 ceph-mon[35324]: pgmap v427: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 329 op/s 2024-09-16T01:53:57.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.917+0000 7fa74bfff640 1 -- 172.21.15.139:0/3386005386 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561a68293d20 con 0x561a683ad5c0 2024-09-16T01:53:57.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.918+0000 7fa74bfff640 10 client.16908 handle_command_reply: tid=0 2024-09-16T01:53:57.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.918+0000 7fa77535b740 1 client.16908 shutdown 2024-09-16T01:53:57.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.918+0000 7fa77535b740 1 -- 172.21.15.139:0/3386005386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68326900 msgr2=0x561a68336190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:57.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.918+0000 7fa77535b740 1 --2- 172.21.15.139:0/3386005386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68326900 0x561a68336190 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa76403d040 tx=0x7fa764005bd0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.918+0000 7fa77535b740 1 -- 172.21.15.139:0/3386005386 shutdown_connections 2024-09-16T01:53:57.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.918+0000 7fa77535b740 1 --2- 172.21.15.139:0/3386005386 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a683ad5c0 0x561a683af9b0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.918+0000 7fa77535b740 1 --2- 172.21.15.139:0/3386005386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a68331ae0 0x561a683a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.918+0000 7fa77535b740 1 --2- 172.21.15.139:0/3386005386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a68327220 0x561a683366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.918+0000 7fa77535b740 1 --2- 172.21.15.139:0/3386005386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a68326900 0x561a68336190 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.919+0000 7fa77535b740 1 -- 172.21.15.139:0/3386005386 >> 172.21.15.139:0/3386005386 conn(0x561a68322e90 msgr2=0x561a68328b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:57.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.919+0000 7fa74bfff640 0 client.16908 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:53:57.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.919+0000 7fa77535b740 1 -- 172.21.15.139:0/3386005386 shutdown_connections 2024-09-16T01:53:57.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.919+0000 7fa77535b740 1 -- 172.21.15.139:0/3386005386 wait complete. 2024-09-16T01:53:57.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.919+0000 7fa77535b740 20 client.16908 trim_cache size 0 max 16384 2024-09-16T01:53:57.922 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:53:57.922 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 69, 2024-09-16T01:53:57.922 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:53:57.922 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 46, 2024-09-16T01:53:57.923 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:53:57.923 INFO:teuthology.orchestra.run.smithi139.stdout: "34558dc0": { 2024-09-16T01:53:57.923 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 46, 2024-09-16T01:53:57.923 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:53:57.923 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:53:57.923 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:53:57.923 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:53:57.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:53:57.923 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:53:57.923 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 131.4, 2024-09-16T01:53:57.923 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:53:57.923 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:53:57.923 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:53:57.923 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:53:57.924 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:53:57.924 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:53:57.924 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:53:57.924 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:53:57.924 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:53:57.924 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:53:57.924 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:53:57.924 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:53:57.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.920+0000 7fa771d20640 1 -- 172.21.15.139:0/1795542221 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa7540a3430 msgr2=0x7fa7540a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:57.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.920+0000 7fa771d20640 1 --2- 172.21.15.139:0/1795542221 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa7540a3430 0x7fa7540a58f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0058c0 tx=0x7fa75c005f90 comp rx=0 tx=0).stop 2024-09-16T01:53:57.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.921+0000 7fa771d20640 1 -- 172.21.15.139:0/1795542221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa76c1129b0 msgr2=0x7fa76c06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:53:57.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.921+0000 7fa771d20640 1 --2- 172.21.15.139:0/1795542221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa76c1129b0 0x7fa76c06cc60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa76000b3f0 tx=0x7fa76000b8b0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.922+0000 7fa771d20640 1 -- 172.21.15.139:0/1795542221 shutdown_connections 2024-09-16T01:53:57.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.922+0000 7fa771d20640 1 --2- 172.21.15.139:0/1795542221 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa7540a3430 0x7fa7540a58f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.922+0000 7fa771d20640 1 --2- 172.21.15.139:0/1795542221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa76c06d1a0 0x7fa76c06d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.922+0000 7fa771d20640 1 --2- 172.21.15.139:0/1795542221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa76c1129b0 0x7fa76c06cc60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.922+0000 7fa771d20640 1 --2- 172.21.15.139:0/1795542221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa76c074eb0 0x7fa76c073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:53:57.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.922+0000 7fa771d20640 1 -- 172.21.15.139:0/1795542221 >> 172.21.15.139:0/1795542221 conn(0x7fa76c0fd680 msgr2=0x7fa76c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:53:57.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.924+0000 7fa771d20640 1 -- 172.21.15.139:0/1795542221 shutdown_connections 2024-09-16T01:53:57.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:53:57.924+0000 7fa771d20640 1 -- 172.21.15.139:0/1795542221 wait complete. 2024-09-16T01:53:58.668 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '34558dc0', quiesce took 10.5 seconds. Will release after: 0.9000000000000004 2024-09-16T01:53:59.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:53:59 smithi164 ceph-mon[35121]: pgmap v428: 129 pgs: 14 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 65 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 318 op/s 2024-09-16T01:53:59.569 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:53:59.608 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:53:59 smithi139 ceph-mon[31175]: pgmap v428: 129 pgs: 14 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 65 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 318 op/s 2024-09-16T01:53:59.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:53:59 smithi176 ceph-mon[35324]: pgmap v428: 129 pgs: 14 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 65 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 318 op/s 2024-09-16T01:53:59.988 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:54:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:01 smithi164 ceph-mon[35121]: pgmap v429: 129 pgs: 13 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 74 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 317 op/s 2024-09-16T01:54:01.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:01 smithi139 ceph-mon[31175]: pgmap v429: 129 pgs: 13 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 74 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 317 op/s 2024-09-16T01:54:01.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:01 smithi176 ceph-mon[35324]: pgmap v429: 129 pgs: 13 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 74 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 317 op/s 2024-09-16T01:54:03.517 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:03 smithi139 ceph-mon[31175]: pgmap v430: 129 pgs: 13 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 74 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 936 KiB/s wr, 333 op/s 2024-09-16T01:54:03.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:03 smithi164 ceph-mon[35121]: pgmap v430: 129 pgs: 13 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 74 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 936 KiB/s wr, 333 op/s 2024-09-16T01:54:03.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.547+0000 7f9b1e083640 1 -- 172.21.15.139:0/1873986687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b18108b50 msgr2=0x7f9b18115070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:03.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.547+0000 7f9b1e083640 1 --2- 172.21.15.139:0/1873986687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b18108b50 0x7f9b18115070 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009930 tx=0x7f9b0c02a480 comp rx=0 tx=0).stop 2024-09-16T01:54:03.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.547+0000 7f9b1e083640 1 -- 172.21.15.139:0/1873986687 shutdown_connections 2024-09-16T01:54:03.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.547+0000 7f9b1e083640 1 --2- 172.21.15.139:0/1873986687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b18108b50 0x7f9b18115070 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.547+0000 7f9b1e083640 1 --2- 172.21.15.139:0/1873986687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b181081f0 0x7f9b18108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.547+0000 7f9b1e083640 1 --2- 172.21.15.139:0/1873986687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b1810e230 0x7f9b1810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.547+0000 7f9b1e083640 1 -- 172.21.15.139:0/1873986687 >> 172.21.15.139:0/1873986687 conn(0x7f9b18077bb0 msgr2=0x7f9b18079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:03.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.552+0000 7f9b1e083640 1 -- 172.21.15.139:0/1873986687 shutdown_connections 2024-09-16T01:54:03.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.552+0000 7f9b1e083640 1 -- 172.21.15.139:0/1873986687 wait complete. 2024-09-16T01:54:03.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.553+0000 7f9b1e083640 1 Processor -- start 2024-09-16T01:54:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.558+0000 7f9b1e083640 1 -- start start 2024-09-16T01:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.559+0000 7f9b1e083640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b181081f0 0x7f9b18082560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.559+0000 7f9b1e083640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b18108b50 0x7f9b18082aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.559+0000 7f9b1e083640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b1810e230 0x7f9b1807c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.559+0000 7f9b1e083640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1807b030 con 0x7f9b181081f0 2024-09-16T01:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.559+0000 7f9b1e083640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1807aeb0 con 0x7f9b1810e230 2024-09-16T01:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.559+0000 7f9b1e083640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1807b1b0 con 0x7f9b18108b50 2024-09-16T01:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.559+0000 7f9b177fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b181081f0 0x7f9b18082560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.559+0000 7f9b177fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b181081f0 0x7f9b18082560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35134/0 (socket says 172.21.15.139:35134) 2024-09-16T01:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.559+0000 7f9b177fe640 1 -- 172.21.15.139:0/4098745795 learned_addr learned my addr 172.21.15.139:0/4098745795 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.559+0000 7f9b16ffd640 1 --2- 172.21.15.139:0/4098745795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b18108b50 0x7f9b18082aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.560+0000 7f9b177fe640 1 -- 172.21.15.139:0/4098745795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b18108b50 msgr2=0x7f9b18082aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.560+0000 7f9b177fe640 1 --2- 172.21.15.139:0/4098745795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b18108b50 0x7f9b18082aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.560+0000 7f9b177fe640 1 -- 172.21.15.139:0/4098745795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b1810e230 msgr2=0x7f9b1807c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.560+0000 7f9b177fe640 1 --2- 172.21.15.139:0/4098745795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b1810e230 0x7f9b1807c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.560+0000 7f9b177fe640 1 -- 172.21.15.139:0/4098745795 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1807cea0 con 0x7f9b181081f0 2024-09-16T01:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.561+0000 7f9b16ffd640 1 --2- 172.21.15.139:0/4098745795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b18108b50 0x7f9b18082aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:54:03.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.566+0000 7f9b177fe640 1 --2- 172.21.15.139:0/4098745795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b181081f0 0x7f9b18082560 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800c890 tx=0x7f9b0800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.570+0000 7f9b14ff9640 1 -- 172.21.15.139:0/4098745795 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08004220 con 0x7f9b181081f0 2024-09-16T01:54:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.570+0000 7f9b14ff9640 1 -- 172.21.15.139:0/4098745795 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b080043c0 con 0x7f9b181081f0 2024-09-16T01:54:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.570+0000 7f9b14ff9640 1 -- 172.21.15.139:0/4098745795 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b080049e0 con 0x7f9b181081f0 2024-09-16T01:54:03.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.571+0000 7f9b1e083640 1 -- 172.21.15.139:0/4098745795 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1807d190 con 0x7f9b181081f0 2024-09-16T01:54:03.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.571+0000 7f9b1e083640 1 -- 172.21.15.139:0/4098745795 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b18071e10 con 0x7f9b181081f0 2024-09-16T01:54:03.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.573+0000 7f9b1e083640 1 -- 172.21.15.139:0/4098745795 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b1807d3f0 con 0x7f9b181081f0 2024-09-16T01:54:03.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.579+0000 7f9b14ff9640 1 -- 172.21.15.139:0/4098745795 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9b080079b0 con 0x7f9b181081f0 2024-09-16T01:54:03.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.580+0000 7f9b14ff9640 1 --2- 172.21.15.139:0/4098745795 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9b000a35c0 0x7f9b000a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:03.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.580+0000 7f9b14ff9640 1 -- 172.21.15.139:0/4098745795 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f9b080d0690 con 0x7f9b181081f0 2024-09-16T01:54:03.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.584+0000 7f9b16ffd640 1 --2- 172.21.15.139:0/4098745795 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9b000a35c0 0x7f9b000a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:03.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.585+0000 7f9b14ff9640 1 -- 172.21.15.139:0/4098745795 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b0801d020 con 0x7f9b181081f0 2024-09-16T01:54:03.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.590+0000 7f9b16ffd640 1 --2- 172.21.15.139:0/4098745795 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9b000a35c0 0x7f9b000a5a80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9b1807dda0 tx=0x7f9b1000a430 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:03.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:03 smithi176 ceph-mon[35324]: pgmap v430: 129 pgs: 13 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 74 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 936 KiB/s wr, 333 op/s 2024-09-16T01:54:03.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.978+0000 7f9b1e083640 1 -- 172.21.15.139:0/4098745795 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b18108610 con 0x7f9b181081f0 2024-09-16T01:54:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.979+0000 7f9b14ff9640 1 -- 172.21.15.139:0/4098745795 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+12025 (secure 0 0 0) 0x7f9b08014030 con 0x7f9b181081f0 2024-09-16T01:54:03.984 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:54:03.985 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 69 2024-09-16T01:54:03.985 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":69,"btime":"2024-09-16T01:53:49:176082+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":69,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:53:49.046798+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:54:03.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.983+0000 7f9b1e083640 1 -- 172.21.15.139:0/4098745795 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9b000a35c0 msgr2=0x7f9b000a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:03.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.983+0000 7f9b1e083640 1 --2- 172.21.15.139:0/4098745795 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9b000a35c0 0x7f9b000a5a80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9b1807dda0 tx=0x7f9b1000a430 comp rx=0 tx=0).stop 2024-09-16T01:54:03.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.983+0000 7f9b1e083640 1 -- 172.21.15.139:0/4098745795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b181081f0 msgr2=0x7f9b18082560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:03.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.983+0000 7f9b1e083640 1 --2- 172.21.15.139:0/4098745795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b181081f0 0x7f9b18082560 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800c890 tx=0x7f9b0800cd50 comp rx=0 tx=0).stop 2024-09-16T01:54:03.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.984+0000 7f9b1e083640 1 -- 172.21.15.139:0/4098745795 shutdown_connections 2024-09-16T01:54:03.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.984+0000 7f9b1e083640 1 --2- 172.21.15.139:0/4098745795 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9b000a35c0 0x7f9b000a5a80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.984+0000 7f9b1e083640 1 --2- 172.21.15.139:0/4098745795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b1810e230 0x7f9b1807c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.984+0000 7f9b1e083640 1 --2- 172.21.15.139:0/4098745795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b18108b50 0x7f9b18082aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.984+0000 7f9b1e083640 1 --2- 172.21.15.139:0/4098745795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b181081f0 0x7f9b18082560 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:03.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.984+0000 7f9b1e083640 1 -- 172.21.15.139:0/4098745795 >> 172.21.15.139:0/4098745795 conn(0x7f9b18077bb0 msgr2=0x7f9b180792a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:03.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.985+0000 7f9b1e083640 1 -- 172.21.15.139:0/4098745795 shutdown_connections 2024-09-16T01:54:03.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:03.987+0000 7f9b1e083640 1 -- 172.21.15.139:0/4098745795 wait complete. 2024-09-16T01:54:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.498+0000 7f1a35e77640 1 -- 172.21.15.139:0/1681887736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a301026f0 msgr2=0x7f1a3010d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.498+0000 7f1a35e77640 1 --2- 172.21.15.139:0/1681887736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a301026f0 0x7f1a3010d110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800acc0 tx=0x7f1a2802a6c0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.502+0000 7f1a35e77640 1 -- 172.21.15.139:0/1681887736 shutdown_connections 2024-09-16T01:54:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.503+0000 7f1a35e77640 1 --2- 172.21.15.139:0/1681887736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a301026f0 0x7f1a3010d110 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.503+0000 7f1a35e77640 1 --2- 172.21.15.139:0/1681887736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a30101d70 0x7f1a301021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.503+0000 7f1a35e77640 1 --2- 172.21.15.139:0/1681887736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a30100490 0x7f1a30100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.503+0000 7f1a35e77640 1 -- 172.21.15.139:0/1681887736 >> 172.21.15.139:0/1681887736 conn(0x7f1a300770f0 msgr2=0x7f1a300fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.506+0000 7f1a35e77640 1 -- 172.21.15.139:0/1681887736 shutdown_connections 2024-09-16T01:54:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.506+0000 7f1a35e77640 1 -- 172.21.15.139:0/1681887736 wait complete. 2024-09-16T01:54:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.507+0000 7f1a35e77640 1 Processor -- start 2024-09-16T01:54:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.507+0000 7f1a35e77640 1 -- start start 2024-09-16T01:54:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.507+0000 7f1a35e77640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a30100490 0x7f1a30073ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.507+0000 7f1a35e77640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a30101d70 0x7f1a3006db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.507+0000 7f1a35e77640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a301026f0 0x7f1a3006e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.507+0000 7f1a35e77640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3006c640 con 0x7f1a30100490 2024-09-16T01:54:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.507+0000 7f1a35e77640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3006c4c0 con 0x7f1a30101d70 2024-09-16T01:54:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.507+0000 7f1a35e77640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3006c7c0 con 0x7f1a301026f0 2024-09-16T01:54:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.509+0000 7f1a2effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a30101d70 0x7f1a3006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.509+0000 7f1a2effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a30101d70 0x7f1a3006db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:60256/0 (socket says 172.21.15.139:60256) 2024-09-16T01:54:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.509+0000 7f1a2effd640 1 -- 172.21.15.139:0/1180094601 learned_addr learned my addr 172.21.15.139:0/1180094601 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.509+0000 7f1a2f7fe640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a30100490 0x7f1a30073ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.509+0000 7f1a2ffff640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a301026f0 0x7f1a3006e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.509+0000 7f1a2effd640 1 -- 172.21.15.139:0/1180094601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a301026f0 msgr2=0x7f1a3006e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.509+0000 7f1a2effd640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a301026f0 0x7f1a3006e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.509+0000 7f1a2effd640 1 -- 172.21.15.139:0/1180094601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a30100490 msgr2=0x7f1a30073ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.509+0000 7f1a2effd640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a30100490 0x7f1a30073ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.509+0000 7f1a2effd640 1 -- 172.21.15.139:0/1180094601 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3006e6f0 con 0x7f1a30101d70 2024-09-16T01:54:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.510+0000 7f1a2ffff640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a301026f0 0x7f1a3006e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:54:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.510+0000 7f1a2effd640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a30101d70 0x7f1a3006db30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400b4a0 tx=0x7f1a2400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.511+0000 7f1a2cff9640 1 -- 172.21.15.139:0/1180094601 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24014070 con 0x7f1a30101d70 2024-09-16T01:54:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.511+0000 7f1a2cff9640 1 -- 172.21.15.139:0/1180094601 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a24010af0 con 0x7f1a30101d70 2024-09-16T01:54:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.511+0000 7f1a2cff9640 1 -- 172.21.15.139:0/1180094601 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24017cc0 con 0x7f1a30101d70 2024-09-16T01:54:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.511+0000 7f1a35e77640 1 -- 172.21.15.139:0/1180094601 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a301a50e0 con 0x7f1a30101d70 2024-09-16T01:54:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.511+0000 7f1a35e77640 1 -- 172.21.15.139:0/1180094601 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a301a55e0 con 0x7f1a30101d70 2024-09-16T01:54:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.519+0000 7f1a2cff9640 1 -- 172.21.15.139:0/1180094601 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1a24017070 con 0x7f1a30101d70 2024-09-16T01:54:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.520+0000 7f1a2cff9640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1a180a3690 0x7f1a180a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.520+0000 7f1a2cff9640 1 -- 172.21.15.139:0/1180094601 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f1a240cf380 con 0x7f1a30101d70 2024-09-16T01:54:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.523+0000 7f1a2f7fe640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1a180a3690 0x7f1a180a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.524+0000 7f1a2f7fe640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1a180a3690 0x7f1a180a5b50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000b3c0 tx=0x7f1a2000d040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:04.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:04 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/4098745795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:54:04.546 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.545+0000 7f1a38c13740 1 Processor -- start 2024-09-16T01:54:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.546+0000 7f1a38c13740 1 -- start start 2024-09-16T01:54:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.546+0000 7f1a38c13740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed38af0 0x55f32ed38ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.546+0000 7f1a38c13740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed39410 0x55f32ed43770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.547+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed38af0 0x55f32ed38ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.547+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed38af0 0x55f32ed38ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:37320/0 (socket says 172.21.15.139:37320) 2024-09-16T01:54:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.547+0000 7f1a38c13740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed43cb0 0x55f32ed460a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.547+0000 7f1a2f7fe640 1 -- 172.21.15.139:0/1781657917 learned_addr learned my addr 172.21.15.139:0/1781657917 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.547+0000 7f1a38c13740 1 -- 172.21.15.139:0/1781657917 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f32ec9d2f0 con 0x55f32ed39410 2024-09-16T01:54:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.547+0000 7f1a38c13740 1 -- 172.21.15.139:0/1781657917 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f32ec45270 con 0x55f32ed43cb0 2024-09-16T01:54:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.547+0000 7f1a38c13740 1 -- 172.21.15.139:0/1781657917 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f32ec992b0 con 0x55f32ed38af0 2024-09-16T01:54:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.547+0000 7f1a2effd640 1 --2- 172.21.15.139:0/1781657917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed39410 0x55f32ed43770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.547+0000 7f1a2ffff640 1 --2- 172.21.15.139:0/1781657917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed43cb0 0x55f32ed460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.547+0000 7f1a2f7fe640 1 -- 172.21.15.139:0/1781657917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed43cb0 msgr2=0x55f32ed460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.547+0000 7f1a2f7fe640 1 --2- 172.21.15.139:0/1781657917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed43cb0 0x55f32ed460a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.547+0000 7f1a2f7fe640 1 -- 172.21.15.139:0/1781657917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed39410 msgr2=0x55f32ed43770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.547+0000 7f1a2f7fe640 1 --2- 172.21.15.139:0/1781657917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed39410 0x55f32ed43770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.547+0000 7f1a2f7fe640 1 -- 172.21.15.139:0/1781657917 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f32ec44340 con 0x55f32ed38af0 2024-09-16T01:54:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.548+0000 7f1a2ffff640 1 --2- 172.21.15.139:0/1781657917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed43cb0 0x55f32ed460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:54:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.548+0000 7f1a2f7fe640 1 --2- 172.21.15.139:0/1781657917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed38af0 0x55f32ed38ed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000bf00 tx=0x7f1a20013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a95f876ad7eb5f35 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.548+0000 7f1a15ffb640 1 -- 172.21.15.139:0/1781657917 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20027070 con 0x55f32ed38af0 2024-09-16T01:54:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.549+0000 7f1a15ffb640 1 -- 172.21.15.139:0/1781657917 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a2000adc0 con 0x55f32ed38af0 2024-09-16T01:54:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.550+0000 7f1a38c13740 1 -- 172.21.15.139:0/1781657917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed38af0 msgr2=0x55f32ed38ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.550+0000 7f1a38c13740 1 --2- 172.21.15.139:0/1781657917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed38af0 0x55f32ed38ed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000bf00 tx=0x7f1a20013000 comp rx=0 tx=0).stop 2024-09-16T01:54:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.550+0000 7f1a38c13740 1 -- 172.21.15.139:0/1781657917 shutdown_connections 2024-09-16T01:54:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.551+0000 7f1a38c13740 1 --2- 172.21.15.139:0/1781657917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed43cb0 0x55f32ed460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.551+0000 7f1a38c13740 1 --2- 172.21.15.139:0/1781657917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed39410 0x55f32ed43770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.551+0000 7f1a38c13740 1 --2- 172.21.15.139:0/1781657917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed38af0 0x55f32ed38ed0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.551+0000 7f1a38c13740 1 -- 172.21.15.139:0/1781657917 >> 172.21.15.139:0/1781657917 conn(0x55f32ed35060 msgr2=0x55f32ed35440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.551+0000 7f1a38c13740 1 -- 172.21.15.139:0/1781657917 shutdown_connections 2024-09-16T01:54:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.552+0000 7f1a38c13740 1 -- 172.21.15.139:0/1781657917 wait complete. 2024-09-16T01:54:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.553+0000 7f1a38c13740 1 -- start start 2024-09-16T01:54:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.553+0000 7f1a38c13740 1 Processor -- start 2024-09-16T01:54:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.554+0000 7f1a38c13740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed38af0 0x55f32ed40bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.554+0000 7f1a38c13740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed39410 0x55f32ed410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.554+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed38af0 0x55f32ed40bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.554+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed38af0 0x55f32ed40bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:37336/0 (socket says 172.21.15.139:37336) 2024-09-16T01:54:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.554+0000 7f1a2f7fe640 1 -- 172.21.15.139:0/2419955331 learned_addr learned my addr 172.21.15.139:0/2419955331 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.554+0000 7f1a38c13740 1 --2- 172.21.15.139:0/2419955331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed43cb0 0x55f32ed41630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.554+0000 7f1a38c13740 1 -- 172.21.15.139:0/2419955331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f32ed468d0 con 0x55f32ed39410 2024-09-16T01:54:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.555+0000 7f1a38c13740 1 -- 172.21.15.139:0/2419955331 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f32ec26290 con 0x55f32ed43cb0 2024-09-16T01:54:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.555+0000 7f1a38c13740 1 -- 172.21.15.139:0/2419955331 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f32ed46a50 con 0x55f32ed38af0 2024-09-16T01:54:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.555+0000 7f1a2ffff640 1 --2- 172.21.15.139:0/2419955331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed43cb0 0x55f32ed41630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.555+0000 7f1a38c13740 10 client.0 _ll_register_callbacks cb 0x55f32ec57140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:54:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.555+0000 7f1a2f7fe640 1 -- 172.21.15.139:0/2419955331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed43cb0 msgr2=0x55f32ed41630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.555+0000 7f1a2f7fe640 1 --2- 172.21.15.139:0/2419955331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed43cb0 0x55f32ed41630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.555+0000 7f1a2f7fe640 1 -- 172.21.15.139:0/2419955331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed39410 msgr2=0x55f32ed410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.555+0000 7f1a2f7fe640 1 --2- 172.21.15.139:0/2419955331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed39410 0x55f32ed410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.555+0000 7f1a2f7fe640 1 -- 172.21.15.139:0/2419955331 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f32e9b0210 con 0x55f32ed38af0 2024-09-16T01:54:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.555+0000 7f1a2f7fe640 1 --2- 172.21.15.139:0/2419955331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed38af0 0x55f32ed40bb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1a20005610 tx=0x7f1a20005640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3a2dee785284efe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.556+0000 7f1a38c13740 1 -- 172.21.15.139:0/2419955331 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a2000adc0 con 0x55f32ed38af0 2024-09-16T01:54:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.556+0000 7f1a0effd640 10 client.26323 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:54:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.556+0000 7f1a0effd640 1 -- 172.21.15.139:0/2419955331 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20030020 con 0x55f32ed38af0 2024-09-16T01:54:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.556+0000 7f1a0effd640 1 -- 172.21.15.139:0/2419955331 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f1a20004860 con 0x55f32ed38af0 2024-09-16T01:54:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.556+0000 7f1a0effd640 1 -- 172.21.15.139:0/2419955331 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f1a20027040 con 0x55f32ed38af0 2024-09-16T01:54:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.557+0000 7f1a38c13740 10 client.26323 fetch_fsmap learned FSMap version 69 2024-09-16T01:54:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.557+0000 7f1a38c13740 1 -- 172.21.15.139:0/2419955331 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({fsmap=69}) -- 0x55f32ea7c620 con 0x55f32ed38af0 2024-09-16T01:54:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.559+0000 7f1a0effd640 1 -- 172.21.15.139:0/2419955331 <== mon.1 v2:172.21.15.176:3300/0 4 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7f1a20004a60 con 0x55f32ed38af0 2024-09-16T01:54:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.560+0000 7f1a38c13740 10 client.26323 fetch_fsmap finished waiting for FSMap version 69 2024-09-16T01:54:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.560+0000 7f1a38c13740 10 client.26323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:54:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.560+0000 7f1a38c13740 20 client.26323 populate_metadata read hostname 'smithi139' 2024-09-16T01:54:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.560+0000 7f1a38c13740 1 --2- 172.21.15.139:0/2419955331 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f32ed53900 0x55f32ed55cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.560+0000 7f1a38c13740 4 client.26323 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:54:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.560+0000 7f1a38c13740 1 -- 172.21.15.139:0/2419955331 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1a2000adc0 con 0x55f32ed53900 2024-09-16T01:54:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.560+0000 7f1a2effd640 1 --2- 172.21.15.139:0/2419955331 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f32ed53900 0x55f32ed55cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.566+0000 7f1a0effd640 10 client.26323 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.566+0000 7f1a2effd640 1 --2- 172.21.15.139:0/2419955331 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f32ed53900 0x55f32ed55cf0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9199716b9b67afd server_cookie=17dd6bab341a3c35 in_seq=0 out_seq=0 2024-09-16T01:54:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.567+0000 7f1a0effd640 1 -- 172.21.15.139:0/2419955331 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f1a2000adc0 con 0x55f32ed53900 2024-09-16T01:54:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.567+0000 7f1a0effd640 10 client.26323 handle_command_reply: tid=0 2024-09-16T01:54:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.567+0000 7f1a38c13740 1 client.26323 shutdown 2024-09-16T01:54:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.568+0000 7f1a38c13740 1 -- 172.21.15.139:0/2419955331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed38af0 msgr2=0x55f32ed40bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.568+0000 7f1a38c13740 1 --2- 172.21.15.139:0/2419955331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed38af0 0x55f32ed40bb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1a20005610 tx=0x7f1a20005640 comp rx=0 tx=0).stop 2024-09-16T01:54:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.568+0000 7f1a38c13740 1 -- 172.21.15.139:0/2419955331 shutdown_connections 2024-09-16T01:54:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.568+0000 7f1a38c13740 1 --2- 172.21.15.139:0/2419955331 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f32ed53900 0x55f32ed55cf0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.568+0000 7f1a38c13740 1 --2- 172.21.15.139:0/2419955331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed43cb0 0x55f32ed41630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.568+0000 7f1a38c13740 1 --2- 172.21.15.139:0/2419955331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed39410 0x55f32ed410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.568+0000 7f1a38c13740 1 --2- 172.21.15.139:0/2419955331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed38af0 0x55f32ed40bb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.568+0000 7f1a38c13740 1 -- 172.21.15.139:0/2419955331 >> 172.21.15.139:0/2419955331 conn(0x55f32ed35060 msgr2=0x55f32ed3adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.569+0000 7f1a0effd640 0 client.26323 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.569+0000 7f1a38c13740 1 -- 172.21.15.139:0/2419955331 shutdown_connections 2024-09-16T01:54:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.569+0000 7f1a38c13740 1 -- 172.21.15.139:0/2419955331 wait complete. 2024-09-16T01:54:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.570+0000 7f1a38c13740 20 client.26323 trim_cache size 0 max 16384 2024-09-16T01:54:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.597+0000 7f1a38c13740 1 Processor -- start 2024-09-16T01:54:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.597+0000 7f1a38c13740 1 -- start start 2024-09-16T01:54:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.598+0000 7f1a38c13740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed38af0 0x55f32ed3f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.598+0000 7f1a38c13740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed39410 0x55f32ed410e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.598+0000 7f1a38c13740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed43cb0 0x55f32ed41620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.598+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed38af0 0x55f32ed3f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.598+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed38af0 0x55f32ed3f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:60274/0 (socket says 172.21.15.139:60274) 2024-09-16T01:54:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.598+0000 7f1a2f7fe640 1 -- 172.21.15.139:0/2767260379 learned_addr learned my addr 172.21.15.139:0/2767260379 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.598+0000 7f1a38c13740 1 -- 172.21.15.139:0/2767260379 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301c6930 con 0x55f32ed39410 2024-09-16T01:54:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.598+0000 7f1a38c13740 1 -- 172.21.15.139:0/2767260379 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301c86e0 con 0x55f32ed38af0 2024-09-16T01:54:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.598+0000 7f1a2ffff640 1 --2- 172.21.15.139:0/2767260379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed43cb0 0x55f32ed41620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.598+0000 7f1a38c13740 1 -- 172.21.15.139:0/2767260379 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301c7030 con 0x55f32ed43cb0 2024-09-16T01:54:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.599+0000 7f1a2effd640 1 --2- 172.21.15.139:0/2767260379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed39410 0x55f32ed410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.599+0000 7f1a2f7fe640 1 -- 172.21.15.139:0/2767260379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed43cb0 msgr2=0x55f32ed41620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.599+0000 7f1a2f7fe640 1 --2- 172.21.15.139:0/2767260379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed43cb0 0x55f32ed41620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.599+0000 7f1a2f7fe640 1 -- 172.21.15.139:0/2767260379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed39410 msgr2=0x55f32ed410e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.599+0000 7f1a2f7fe640 1 --2- 172.21.15.139:0/2767260379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed39410 0x55f32ed410e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.599+0000 7f1a2f7fe640 1 -- 172.21.15.139:0/2767260379 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f32ed3fb40 con 0x55f32ed38af0 2024-09-16T01:54:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.600+0000 7f1a2f7fe640 1 --2- 172.21.15.139:0/2767260379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed38af0 0x55f32ed3f460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a20005820 tx=0x7f1a2002c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ef583e3f577237f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.601+0000 7f1a157fa640 1 -- 172.21.15.139:0/2767260379 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20004660 con 0x55f32ed38af0 2024-09-16T01:54:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.601+0000 7f1a157fa640 1 -- 172.21.15.139:0/2767260379 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a20022400 con 0x55f32ed38af0 2024-09-16T01:54:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.602+0000 7f1a157fa640 1 -- 172.21.15.139:0/2767260379 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2003e690 con 0x55f32ed38af0 2024-09-16T01:54:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.603+0000 7f1a38c13740 1 -- 172.21.15.139:0/2767260379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed38af0 msgr2=0x55f32ed3f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.603+0000 7f1a38c13740 1 --2- 172.21.15.139:0/2767260379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed38af0 0x55f32ed3f460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a20005820 tx=0x7f1a2002c000 comp rx=0 tx=0).stop 2024-09-16T01:54:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.604+0000 7f1a38c13740 1 -- 172.21.15.139:0/2767260379 shutdown_connections 2024-09-16T01:54:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.604+0000 7f1a38c13740 1 --2- 172.21.15.139:0/2767260379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed43cb0 0x55f32ed41620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.604+0000 7f1a38c13740 1 --2- 172.21.15.139:0/2767260379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed39410 0x55f32ed410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.605+0000 7f1a38c13740 1 --2- 172.21.15.139:0/2767260379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed38af0 0x55f32ed3f460 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.605+0000 7f1a38c13740 1 -- 172.21.15.139:0/2767260379 >> 172.21.15.139:0/2767260379 conn(0x55f32ed35060 msgr2=0x55f32ed42c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.606+0000 7f1a38c13740 1 -- 172.21.15.139:0/2767260379 shutdown_connections 2024-09-16T01:54:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.606+0000 7f1a38c13740 1 -- 172.21.15.139:0/2767260379 wait complete. 2024-09-16T01:54:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.607+0000 7f1a38c13740 1 -- start start 2024-09-16T01:54:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.607+0000 7f1a38c13740 1 Processor -- start 2024-09-16T01:54:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.608+0000 7f1a38c13740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed38af0 0x55f32ed4c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.608+0000 7f1a38c13740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed39410 0x55f32ed3d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.608+0000 7f1a38c13740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed43cb0 0x55f32ed3df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.608+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed38af0 0x55f32ed4c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.608+0000 7f1a38c13740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301c7f50 con 0x55f32ed43cb0 2024-09-16T01:54:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.608+0000 7f1a38c13740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301c77c0 con 0x55f32ed38af0 2024-09-16T01:54:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.609+0000 7f1a38c13740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30100870 con 0x55f32ed39410 2024-09-16T01:54:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.609+0000 7f1a2effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed39410 0x55f32ed3d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.609+0000 7f1a38c13740 10 client.0 _ll_register_callbacks cb 0x7f1a20025070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:54:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.609+0000 7f1a2effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed39410 0x55f32ed3d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:37356/0 (socket says 172.21.15.139:37356) 2024-09-16T01:54:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.609+0000 7f1a2effd640 1 -- 172.21.15.139:0/1489001283 learned_addr learned my addr 172.21.15.139:0/1489001283 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.609+0000 7f1a2effd640 1 -- 172.21.15.139:0/1489001283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed38af0 msgr2=0x55f32ed4c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.609+0000 7f1a2effd640 1 --2- 172.21.15.139:0/1489001283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed38af0 0x55f32ed4c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.609+0000 7f1a2effd640 1 -- 172.21.15.139:0/1489001283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed43cb0 msgr2=0x55f32ed3df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.609+0000 7f1a2effd640 1 --2- 172.21.15.139:0/1489001283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed43cb0 0x55f32ed3df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.609+0000 7f1a2effd640 1 -- 172.21.15.139:0/1489001283 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f32ed50650 con 0x55f32ed39410 2024-09-16T01:54:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.610+0000 7f1a2effd640 1 --2- 172.21.15.139:0/1489001283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed39410 0x55f32ed3d9b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a24000c00 tx=0x7f1a24017760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=957431cf2915b7ae server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.612+0000 7f1a0effd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:54:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.612+0000 7f1a0effd640 1 -- 172.21.15.139:0/1489001283 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a240b4060 con 0x55f32ed39410 2024-09-16T01:54:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.612+0000 7f1a0effd640 1 -- 172.21.15.139:0/1489001283 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f1a240b4200 con 0x55f32ed39410 2024-09-16T01:54:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.612+0000 7f1a38c13740 1 -- 172.21.15.139:0/1489001283 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a20022400 con 0x55f32ed39410 2024-09-16T01:54:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.613+0000 7f1a0effd640 1 -- 172.21.15.139:0/1489001283 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f1a240a5020 con 0x55f32ed39410 2024-09-16T01:54:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.613+0000 7f1a38c13740 10 client.26329 fetch_fsmap learned FSMap version 69 2024-09-16T01:54:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.613+0000 7f1a38c13740 1 -- 172.21.15.139:0/1489001283 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55f32edbe490 con 0x55f32ed39410 2024-09-16T01:54:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.614+0000 7f1a0effd640 1 -- 172.21.15.139:0/1489001283 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a240a3f20 con 0x55f32ed39410 2024-09-16T01:54:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.615+0000 7f1a0effd640 1 -- 172.21.15.139:0/1489001283 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7f1a240cc890 con 0x55f32ed39410 2024-09-16T01:54:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.615+0000 7f1a38c13740 10 client.26329 fetch_fsmap finished waiting for FSMap version 69 2024-09-16T01:54:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.615+0000 7f1a38c13740 10 client.26329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:54:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.615+0000 7f1a38c13740 20 client.26329 populate_metadata read hostname 'smithi139' 2024-09-16T01:54:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.615+0000 7f1a38c13740 1 --2- 172.21.15.139:0/1489001283 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f32edbeba0 0x55f32edc0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.615+0000 7f1a38c13740 4 client.26329 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:54:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.615+0000 7f1a38c13740 1 -- 172.21.15.139:0/1489001283 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1a20022400 con 0x55f32edbeba0 2024-09-16T01:54:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.616+0000 7f1a2f7fe640 1 --2- 172.21.15.139:0/1489001283 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f32edbeba0 0x55f32edc0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.628+0000 7f1a2f7fe640 1 --2- 172.21.15.139:0/1489001283 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f32edbeba0 0x55f32edc0fc0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ced6fd47d03fe9 server_cookie=c9042450b9cb0f73 in_seq=0 out_seq=0 2024-09-16T01:54:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.629+0000 7f1a0effd640 10 client.26329 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:04.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:04 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/4098745795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:54:04.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:04 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/4098745795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:54:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.712+0000 7f1a0effd640 1 -- 172.21.15.139:0/1489001283 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1a20022400 con 0x55f32edbeba0 2024-09-16T01:54:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.713+0000 7f1a0effd640 10 client.26329 handle_command_reply: tid=0 2024-09-16T01:54:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.713+0000 7f1a38c13740 1 client.26329 shutdown 2024-09-16T01:54:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.713+0000 7f1a38c13740 1 -- 172.21.15.139:0/1489001283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed39410 msgr2=0x55f32ed3d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.713+0000 7f1a38c13740 1 --2- 172.21.15.139:0/1489001283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed39410 0x55f32ed3d9b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a24000c00 tx=0x7f1a24017760 comp rx=0 tx=0).stop 2024-09-16T01:54:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.713+0000 7f1a38c13740 1 -- 172.21.15.139:0/1489001283 shutdown_connections 2024-09-16T01:54:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.713+0000 7f1a38c13740 1 --2- 172.21.15.139:0/1489001283 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55f32edbeba0 0x55f32edc0fc0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.713+0000 7f1a38c13740 1 --2- 172.21.15.139:0/1489001283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f32ed43cb0 0x55f32ed3df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.713+0000 7f1a0effd640 0 client.26329 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.713+0000 7f1a38c13740 1 --2- 172.21.15.139:0/1489001283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32ed39410 0x55f32ed3d9b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.713+0000 7f1a38c13740 1 --2- 172.21.15.139:0/1489001283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f32ed38af0 0x55f32ed4c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.713+0000 7f1a38c13740 1 -- 172.21.15.139:0/1489001283 >> 172.21.15.139:0/1489001283 conn(0x55f32ed35060 msgr2=0x55f32ed3a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.713+0000 7f1a38c13740 1 -- 172.21.15.139:0/1489001283 shutdown_connections 2024-09-16T01:54:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.714+0000 7f1a38c13740 1 -- 172.21.15.139:0/1489001283 wait complete. 2024-09-16T01:54:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.714+0000 7f1a38c13740 20 client.26329 trim_cache size 0 max 16384 2024-09-16T01:54:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:54:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:54:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:54:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:54:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.714+0000 7f1a35e77640 1 -- 172.21.15.139:0/1180094601 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1a180a3690 msgr2=0x7f1a180a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.714+0000 7f1a35e77640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1a180a3690 0x7f1a180a5b50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000b3c0 tx=0x7f1a2000d040 comp rx=0 tx=0).stop 2024-09-16T01:54:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.714+0000 7f1a35e77640 1 -- 172.21.15.139:0/1180094601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a30101d70 msgr2=0x7f1a3006db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.714+0000 7f1a35e77640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a30101d70 0x7f1a3006db30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400b4a0 tx=0x7f1a2400b960 comp rx=0 tx=0).stop 2024-09-16T01:54:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.714+0000 7f1a35e77640 1 -- 172.21.15.139:0/1180094601 shutdown_connections 2024-09-16T01:54:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.714+0000 7f1a35e77640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1a180a3690 0x7f1a180a5b50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.714+0000 7f1a35e77640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a301026f0 0x7f1a3006e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.714+0000 7f1a35e77640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a30101d70 0x7f1a3006db30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.714+0000 7f1a35e77640 1 --2- 172.21.15.139:0/1180094601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a30100490 0x7f1a30073ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.714+0000 7f1a35e77640 1 -- 172.21.15.139:0/1180094601 >> 172.21.15.139:0/1180094601 conn(0x7f1a300770f0 msgr2=0x7f1a30104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.715+0000 7f1a35e77640 1 -- 172.21.15.139:0/1180094601 shutdown_connections 2024-09-16T01:54:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:04.715+0000 7f1a35e77640 1 -- 172.21.15.139:0/1180094601 wait complete. 2024-09-16T01:54:05.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:05 smithi164 ceph-mon[35121]: pgmap v431: 129 pgs: 10 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 90 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 412 KiB/s wr, 268 op/s 2024-09-16T01:54:05.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:05 smithi139 ceph-mon[31175]: pgmap v431: 129 pgs: 10 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 90 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 412 KiB/s wr, 268 op/s 2024-09-16T01:54:05.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:05 smithi176 ceph-mon[35324]: pgmap v431: 129 pgs: 10 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 90 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 412 KiB/s wr, 268 op/s 2024-09-16T01:54:05.754 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 34558dc0 --release --await' 2024-09-16T01:54:05.754 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 34558dc0 --release --await 2024-09-16T01:54:06.250 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:54:06.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.888+0000 7f73e2cb6640 1 -- 172.21.15.139:0/2287314802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73dc109830 msgr2=0x7f73dc109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:06.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.888+0000 7f73e2cb6640 1 --2- 172.21.15.139:0/2287314802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73dc109830 0x7f73dc109c90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f73d4009ca0 tx=0x7f73d402a8f0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.888+0000 7f73e2cb6640 1 -- 172.21.15.139:0/2287314802 shutdown_connections 2024-09-16T01:54:06.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.888+0000 7f73e2cb6640 1 --2- 172.21.15.139:0/2287314802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73dc11b140 0x7f73dc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.888+0000 7f73e2cb6640 1 --2- 172.21.15.139:0/2287314802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73dc109830 0x7f73dc109c90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.888+0000 7f73e2cb6640 1 --2- 172.21.15.139:0/2287314802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73dc109450 0x7f73dc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.888+0000 7f73e2cb6640 1 -- 172.21.15.139:0/2287314802 >> 172.21.15.139:0/2287314802 conn(0x7f73dc06e140 msgr2=0x7f73dc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:06.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.888+0000 7f73e2cb6640 1 -- 172.21.15.139:0/2287314802 shutdown_connections 2024-09-16T01:54:06.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.888+0000 7f73e2cb6640 1 -- 172.21.15.139:0/2287314802 wait complete. 2024-09-16T01:54:06.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.889+0000 7f73e2cb6640 1 Processor -- start 2024-09-16T01:54:06.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.889+0000 7f73e2cb6640 1 -- start start 2024-09-16T01:54:06.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.889+0000 7f73e2cb6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73dc109450 0x7f73dc1a7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.889+0000 7f73e2cb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73dc109830 0x7f73dc1a7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.889+0000 7f73e2cb6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73dc11b140 0x7f73dc1abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.889+0000 7f73e2cb6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc120df0 con 0x7f73dc109830 2024-09-16T01:54:06.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.889+0000 7f73e2cb6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc120c70 con 0x7f73dc109450 2024-09-16T01:54:06.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.889+0000 7f73e2cb6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc120f70 con 0x7f73dc11b140 2024-09-16T01:54:06.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.890+0000 7f73e1253640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73dc11b140 0x7f73dc1abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.890+0000 7f73e1253640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73dc11b140 0x7f73dc1abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:37374/0 (socket says 172.21.15.139:37374) 2024-09-16T01:54:06.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.890+0000 7f73e1253640 1 -- 172.21.15.139:0/1368894937 learned_addr learned my addr 172.21.15.139:0/1368894937 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:06.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.890+0000 7f73e0a52640 1 --2- 172.21.15.139:0/1368894937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73dc109450 0x7f73dc1a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.890+0000 7f73dbfff640 1 --2- 172.21.15.139:0/1368894937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73dc109830 0x7f73dc1a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.890+0000 7f73e1253640 1 -- 172.21.15.139:0/1368894937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73dc109450 msgr2=0x7f73dc1a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:06.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.890+0000 7f73e1253640 1 --2- 172.21.15.139:0/1368894937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73dc109450 0x7f73dc1a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.890+0000 7f73e1253640 1 -- 172.21.15.139:0/1368894937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73dc109830 msgr2=0x7f73dc1a7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:06.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.890+0000 7f73e1253640 1 --2- 172.21.15.139:0/1368894937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73dc109830 0x7f73dc1a7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.890+0000 7f73e1253640 1 -- 172.21.15.139:0/1368894937 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc1ac1f0 con 0x7f73dc11b140 2024-09-16T01:54:06.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.890+0000 7f73e1253640 1 --2- 172.21.15.139:0/1368894937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73dc11b140 0x7f73dc1abaf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00d720 tx=0x7f73cc00dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:06.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.891+0000 7f73d9ffb640 1 -- 172.21.15.139:0/1368894937 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc017030 con 0x7f73dc11b140 2024-09-16T01:54:06.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.891+0000 7f73d9ffb640 1 -- 172.21.15.139:0/1368894937 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73cc00de30 con 0x7f73dc11b140 2024-09-16T01:54:06.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.891+0000 7f73d9ffb640 1 -- 172.21.15.139:0/1368894937 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc004bc0 con 0x7f73dc11b140 2024-09-16T01:54:06.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.893+0000 7f73e2cb6640 1 -- 172.21.15.139:0/1368894937 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73dc1ac4e0 con 0x7f73dc11b140 2024-09-16T01:54:06.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.893+0000 7f73e2cb6640 1 -- 172.21.15.139:0/1368894937 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73dc110920 con 0x7f73dc11b140 2024-09-16T01:54:06.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.899+0000 7f73e508f740 1 Processor -- start 2024-09-16T01:54:06.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.899+0000 7f73d9ffb640 1 -- 172.21.15.139:0/1368894937 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f73cc01a070 con 0x7f73dc11b140 2024-09-16T01:54:06.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.899+0000 7f73e508f740 1 -- start start 2024-09-16T01:54:06.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.900+0000 7f73e508f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7060 0x561d1c0a7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.900+0000 7f73e508f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7980 0x561d1c0b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.900+0000 7f73e508f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0b2240 0x561d1c0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.900+0000 7f73e508f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561d1c00c620 con 0x561d1c0a7980 2024-09-16T01:54:06.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.900+0000 7f73e508f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561d1bfb3e50 con 0x561d1c0a7060 2024-09-16T01:54:06.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.900+0000 7f73e508f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561d1c0085e0 con 0x561d1c0b2240 2024-09-16T01:54:06.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.901+0000 7f73dbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7060 0x561d1c0a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.901+0000 7f73dbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7060 0x561d1c0a7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:60304/0 (socket says 172.21.15.139:60304) 2024-09-16T01:54:06.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.901+0000 7f73dbfff640 1 -- 172.21.15.139:0/4286463888 learned_addr learned my addr 172.21.15.139:0/4286463888 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:06.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.901+0000 7f73e0a52640 1 --2- 172.21.15.139:0/4286463888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7980 0x561d1c0b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.901+0000 7f73dbfff640 1 --2- 172.21.15.139:0/4286463888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0b2240 0x561d1c0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.901+0000 7f73e0a52640 1 -- 172.21.15.139:0/4286463888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0b2240 msgr2=0x561d1c0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:06.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.901+0000 7f73e0a52640 1 --2- 172.21.15.139:0/4286463888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0b2240 0x561d1c0b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.901+0000 7f73e0a52640 1 -- 172.21.15.139:0/4286463888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7060 msgr2=0x561d1c0a7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:06.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.901+0000 7f73e0a52640 1 --2- 172.21.15.139:0/4286463888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7060 0x561d1c0a7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.901+0000 7f73e0a52640 1 -- 172.21.15.139:0/4286463888 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d1bfa4d40 con 0x561d1c0a7980 2024-09-16T01:54:06.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.901+0000 7f73dbfff640 1 --2- 172.21.15.139:0/4286463888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0b2240 0x561d1c0b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:54:06.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.901+0000 7f73d9ffb640 1 --2- 172.21.15.139:0/1368894937 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f73c40a35c0 0x7f73c40a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.902+0000 7f73e0a52640 1 --2- 172.21.15.139:0/4286463888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7980 0x561d1c0b1d00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f73d000bbe0 tx=0x7f73d000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454c69b2a22e32c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:06.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.902+0000 7f73d9ffb640 1 -- 172.21.15.139:0/1368894937 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f73cc0cf1b0 con 0x7f73dc11b140 2024-09-16T01:54:06.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.902+0000 7f73c2ffd640 1 -- 172.21.15.139:0/4286463888 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0015070 con 0x561d1c0a7980 2024-09-16T01:54:06.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.902+0000 7f73c2ffd640 1 -- 172.21.15.139:0/4286463888 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d00035f0 con 0x561d1c0a7980 2024-09-16T01:54:06.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.902+0000 7f73c2ffd640 1 -- 172.21.15.139:0/4286463888 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d00116a0 con 0x561d1c0a7980 2024-09-16T01:54:06.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.903+0000 7f73e508f740 1 -- 172.21.15.139:0/4286463888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7980 msgr2=0x561d1c0b1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:06.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.903+0000 7f73e508f740 1 --2- 172.21.15.139:0/4286463888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7980 0x561d1c0b1d00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f73d000bbe0 tx=0x7f73d000f730 comp rx=0 tx=0).stop 2024-09-16T01:54:06.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.904+0000 7f73e1253640 1 --2- 172.21.15.139:0/1368894937 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f73c40a35c0 0x7f73c40a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.904+0000 7f73e508f740 1 -- 172.21.15.139:0/4286463888 shutdown_connections 2024-09-16T01:54:06.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.904+0000 7f73e508f740 1 --2- 172.21.15.139:0/4286463888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0b2240 0x561d1c0b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.904+0000 7f73e508f740 1 --2- 172.21.15.139:0/4286463888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7980 0x561d1c0b1d00 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.904+0000 7f73e508f740 1 --2- 172.21.15.139:0/4286463888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7060 0x561d1c0a7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.904+0000 7f73e508f740 1 -- 172.21.15.139:0/4286463888 >> 172.21.15.139:0/4286463888 conn(0x561d1c0a35f0 msgr2=0x561d1c0a39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.904+0000 7f73e508f740 1 -- 172.21.15.139:0/4286463888 shutdown_connections 2024-09-16T01:54:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.904+0000 7f73e508f740 1 -- 172.21.15.139:0/4286463888 wait complete. 2024-09-16T01:54:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.905+0000 7f73e508f740 1 -- start start 2024-09-16T01:54:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.905+0000 7f73e508f740 1 Processor -- start 2024-09-16T01:54:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.905+0000 7f73e1253640 1 --2- 172.21.15.139:0/1368894937 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f73c40a35c0 0x7f73c40a5a80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009610 tx=0x7f73cc005cf0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.905+0000 7f73e508f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7060 0x561d1c0af220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.905+0000 7f73e508f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7980 0x561d1c0af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.906+0000 7f73e508f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0b2240 0x561d1c0afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.906+0000 7f73e508f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561d1c0b4e10 con 0x561d1c0a7060 2024-09-16T01:54:06.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.906+0000 7f73e508f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561d1bf94f10 con 0x561d1c0a7980 2024-09-16T01:54:06.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.906+0000 7f73e508f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561d1c0b4f90 con 0x561d1c0b2240 2024-09-16T01:54:06.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.906+0000 7f73e508f740 10 client.0 _ll_register_callbacks cb 0x561d1bfb8db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:54:06.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.906+0000 7f73dbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7060 0x561d1c0af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.906+0000 7f73dbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7060 0x561d1c0af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35232/0 (socket says 172.21.15.139:35232) 2024-09-16T01:54:06.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.906+0000 7f73dbfff640 1 -- 172.21.15.139:0/4038080080 learned_addr learned my addr 172.21.15.139:0/4038080080 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:06.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.906+0000 7f73e0a52640 1 --2- 172.21.15.139:0/4038080080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7980 0x561d1c0af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.906+0000 7f73dbfff640 1 --2- 172.21.15.139:0/4038080080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0b2240 0x561d1c0afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.906+0000 7f73e0a52640 1 -- 172.21.15.139:0/4038080080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0b2240 msgr2=0x561d1c0afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.907+0000 7f73e0a52640 1 --2- 172.21.15.139:0/4038080080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0b2240 0x561d1c0afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.907+0000 7f73e0a52640 1 -- 172.21.15.139:0/4038080080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7060 msgr2=0x561d1c0af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.907+0000 7f73e0a52640 1 --2- 172.21.15.139:0/4038080080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7060 0x561d1c0af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.907+0000 7f73e0a52640 1 -- 172.21.15.139:0/4038080080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d1bd1b580 con 0x561d1c0a7980 2024-09-16T01:54:06.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.911+0000 7f73e0a52640 1 --2- 172.21.15.139:0/4038080080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7980 0x561d1c0af760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f73d000fc30 tx=0x7f73d0003e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b92101f96791520 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:06.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.911+0000 7f73bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:54:06.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.911+0000 7f73bbfff640 1 -- 172.21.15.139:0/4038080080 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0015070 con 0x561d1c0a7980 2024-09-16T01:54:06.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.911+0000 7f73bbfff640 1 -- 172.21.15.139:0/4038080080 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f73d000bcd0 con 0x561d1c0a7980 2024-09-16T01:54:06.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.911+0000 7f73e508f740 1 -- 172.21.15.139:0/4038080080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d00035f0 con 0x561d1c0a7980 2024-09-16T01:54:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.913+0000 7f73bbfff640 1 -- 172.21.15.139:0/4038080080 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f73d00039b0 con 0x561d1c0a7980 2024-09-16T01:54:06.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.913+0000 7f73e508f740 10 client.26375 fetch_fsmap learned FSMap version 69 2024-09-16T01:54:06.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.913+0000 7f73e508f740 1 -- 172.21.15.139:0/4038080080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x561d1bde5960 con 0x561d1c0a7980 2024-09-16T01:54:06.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.913+0000 7f73bbfff640 1 -- 172.21.15.139:0/4038080080 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d00035f0 con 0x561d1c0a7980 2024-09-16T01:54:06.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.913+0000 7f73bbfff640 1 -- 172.21.15.139:0/4038080080 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7f73d002e9d0 con 0x561d1c0a7980 2024-09-16T01:54:06.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.914+0000 7f73e508f740 10 client.26375 fetch_fsmap finished waiting for FSMap version 69 2024-09-16T01:54:06.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.914+0000 7f73e508f740 10 client.26375 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:54:06.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.914+0000 7f73e508f740 20 client.26375 populate_metadata read hostname 'smithi139' 2024-09-16T01:54:06.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.914+0000 7f73e508f740 1 --2- 172.21.15.139:0/4038080080 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d1c0c1de0 0x561d1c0c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.914+0000 7f73e508f740 4 client.26375 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:54:06.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.914+0000 7f73e508f740 1 -- 172.21.15.139:0/4038080080 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d1c019b90 con 0x561d1c0c1de0 2024-09-16T01:54:06.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.915+0000 7f73dbfff640 1 --2- 172.21.15.139:0/4038080080 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d1c0c1de0 0x561d1c0c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.916+0000 7f73dbfff640 1 --2- 172.21.15.139:0/4038080080 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d1c0c1de0 0x561d1c0c41d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90be49f6b6dce4c4 server_cookie=5f8958fc663e5715 in_seq=0 out_seq=0 2024-09-16T01:54:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.916+0000 7f73bbfff640 10 client.26375 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.917+0000 7f73bbfff640 1 -- 172.21.15.139:0/4038080080 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d1c019b90 con 0x561d1c0c1de0 2024-09-16T01:54:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.917+0000 7f73bbfff640 10 client.26375 handle_command_reply: tid=0 2024-09-16T01:54:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.917+0000 7f73e508f740 1 client.26375 shutdown 2024-09-16T01:54:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.917+0000 7f73e508f740 1 -- 172.21.15.139:0/4038080080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7980 msgr2=0x561d1c0af760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:06.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.917+0000 7f73e508f740 1 --2- 172.21.15.139:0/4038080080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7980 0x561d1c0af760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f73d000fc30 tx=0x7f73d0003e60 comp rx=0 tx=0).stop 2024-09-16T01:54:06.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.917+0000 7f73e508f740 1 -- 172.21.15.139:0/4038080080 shutdown_connections 2024-09-16T01:54:06.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.917+0000 7f73e508f740 1 --2- 172.21.15.139:0/4038080080 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d1c0c1de0 0x561d1c0c41d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.917+0000 7f73e508f740 1 --2- 172.21.15.139:0/4038080080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0b2240 0x561d1c0afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.917+0000 7f73e508f740 1 --2- 172.21.15.139:0/4038080080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7980 0x561d1c0af760 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.917+0000 7f73e508f740 1 --2- 172.21.15.139:0/4038080080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7060 0x561d1c0af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.917+0000 7f73e508f740 1 -- 172.21.15.139:0/4038080080 >> 172.21.15.139:0/4038080080 conn(0x561d1c0a35f0 msgr2=0x561d1c0a2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:06.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.917+0000 7f73bbfff640 0 client.26375 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:06.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.917+0000 7f73e508f740 1 -- 172.21.15.139:0/4038080080 shutdown_connections 2024-09-16T01:54:06.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.918+0000 7f73e508f740 1 -- 172.21.15.139:0/4038080080 wait complete. 2024-09-16T01:54:06.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.918+0000 7f73e508f740 20 client.26375 trim_cache size 0 max 16384 2024-09-16T01:54:06.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.948+0000 7f73e508f740 1 Processor -- start 2024-09-16T01:54:06.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.949+0000 7f73e508f740 1 -- start start 2024-09-16T01:54:06.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.949+0000 7f73e508f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7060 0x561d1c0a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.949+0000 7f73e508f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0a7980 0x561d1c0c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.949+0000 7f73e508f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0b2240 0x561d1c0c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.950+0000 7f73dbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7060 0x561d1c0a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.950+0000 7f73dbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7060 0x561d1c0a9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:60312/0 (socket says 172.21.15.139:60312) 2024-09-16T01:54:06.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.950+0000 7f73dbfff640 1 -- 172.21.15.139:0/3454245266 learned_addr learned my addr 172.21.15.139:0/3454245266 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:06.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.950+0000 7f73e508f740 1 -- 172.21.15.139:0/3454245266 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1c6a20 con 0x561d1c0b2240 2024-09-16T01:54:06.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.950+0000 7f73e508f740 1 -- 172.21.15.139:0/3454245266 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1c87d0 con 0x561d1c0a7060 2024-09-16T01:54:06.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.950+0000 7f73e508f740 1 -- 172.21.15.139:0/3454245266 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1c7120 con 0x561d1c0a7980 2024-09-16T01:54:06.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.950+0000 7f73dbfff640 1 --2- 172.21.15.139:0/3454245266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0b2240 0x561d1c0c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.950+0000 7f73e0a52640 1 --2- 172.21.15.139:0/3454245266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0a7980 0x561d1c0c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.951+0000 7f73dbfff640 1 -- 172.21.15.139:0/3454245266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0a7980 msgr2=0x561d1c0c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:06.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.951+0000 7f73dbfff640 1 --2- 172.21.15.139:0/3454245266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0a7980 0x561d1c0c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.951+0000 7f73dbfff640 1 -- 172.21.15.139:0/3454245266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0b2240 msgr2=0x561d1c0c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:06.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.951+0000 7f73dbfff640 1 --2- 172.21.15.139:0/3454245266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0b2240 0x561d1c0c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.951+0000 7f73dbfff640 1 -- 172.21.15.139:0/3454245266 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d1c0b1b90 con 0x561d1c0a7060 2024-09-16T01:54:06.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.951+0000 7f73dbfff640 1 --2- 172.21.15.139:0/3454245266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7060 0x561d1c0a9b10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f73d402aee0 tx=0x7f73d402adf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93360fcbb7b85784 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:06.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.952+0000 7f73c27fc640 1 -- 172.21.15.139:0/3454245266 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4031640 con 0x561d1c0a7060 2024-09-16T01:54:06.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.952+0000 7f73c27fc640 1 -- 172.21.15.139:0/3454245266 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d4051040 con 0x561d1c0a7060 2024-09-16T01:54:06.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.952+0000 7f73c27fc640 1 -- 172.21.15.139:0/3454245266 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4030850 con 0x561d1c0a7060 2024-09-16T01:54:06.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.953+0000 7f73e508f740 1 -- 172.21.15.139:0/3454245266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7060 msgr2=0x561d1c0a9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:06.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.953+0000 7f73e508f740 1 --2- 172.21.15.139:0/3454245266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7060 0x561d1c0a9b10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f73d402aee0 tx=0x7f73d402adf0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.954+0000 7f73e508f740 1 -- 172.21.15.139:0/3454245266 shutdown_connections 2024-09-16T01:54:06.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.954+0000 7f73e508f740 1 --2- 172.21.15.139:0/3454245266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0b2240 0x561d1c0c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.954+0000 7f73e508f740 1 --2- 172.21.15.139:0/3454245266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0a7980 0x561d1c0c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.954+0000 7f73e508f740 1 --2- 172.21.15.139:0/3454245266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0a7060 0x561d1c0a9b10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.954+0000 7f73e508f740 1 -- 172.21.15.139:0/3454245266 >> 172.21.15.139:0/3454245266 conn(0x561d1c0a35f0 msgr2=0x561d1c0adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:06.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.956+0000 7f73e508f740 1 -- 172.21.15.139:0/3454245266 shutdown_connections 2024-09-16T01:54:06.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.956+0000 7f73e508f740 1 -- 172.21.15.139:0/3454245266 wait complete. 2024-09-16T01:54:06.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.956+0000 7f73e508f740 1 -- start start 2024-09-16T01:54:06.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.956+0000 7f73e508f740 1 Processor -- start 2024-09-16T01:54:06.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.962+0000 7f73e508f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7060 0x561d1c0b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.962+0000 7f73e508f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0a7980 0x561d1c0b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.962+0000 7f73e508f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0b2240 0x561d1c125970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.962+0000 7f73e508f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1c8040 con 0x561d1c0a7060 2024-09-16T01:54:06.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.962+0000 7f73e508f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1c78b0 con 0x561d1c0b2240 2024-09-16T01:54:06.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.962+0000 7f73e508f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc118da0 con 0x561d1c0a7980 2024-09-16T01:54:06.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.962+0000 7f73e508f740 10 client.0 _ll_register_callbacks cb 0x7f73d000bf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:54:06.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.962+0000 7f73dbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0b2240 0x561d1c125970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.962+0000 7f73e0a52640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0a7980 0x561d1c0b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.963+0000 7f73e0a52640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0a7980 0x561d1c0b6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:37406/0 (socket says 172.21.15.139:37406) 2024-09-16T01:54:06.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.963+0000 7f73e0a52640 1 -- 172.21.15.139:0/2387477148 learned_addr learned my addr 172.21.15.139:0/2387477148 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:06.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.963+0000 7f73dbfff640 1 --2- 172.21.15.139:0/2387477148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0b2240 0x561d1c125970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:60316/0 (socket says 172.21.15.139:60316) 2024-09-16T01:54:06.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.963+0000 7f73e0a52640 1 -- 172.21.15.139:0/2387477148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0b2240 msgr2=0x561d1c125970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:06.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.963+0000 7f73e0a52640 1 --2- 172.21.15.139:0/2387477148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0b2240 0x561d1c125970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.963+0000 7f73e0a52640 1 -- 172.21.15.139:0/2387477148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7060 msgr2=0x561d1c0b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:06.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.963+0000 7f73e0a52640 1 --2- 172.21.15.139:0/2387477148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7060 0x561d1c0b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:06.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.963+0000 7f73e0a52640 1 -- 172.21.15.139:0/2387477148 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d1c125f30 con 0x561d1c0a7980 2024-09-16T01:54:06.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.964+0000 7f73e0a52640 1 --2- 172.21.15.139:0/2387477148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0a7980 0x561d1c0b6d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f73d0003cb0 tx=0x7f73d00045f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51f7f6cb14f8925b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:06.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.964+0000 7f73bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:54:06.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.964+0000 7f73bbfff640 1 -- 172.21.15.139:0/2387477148 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0015040 con 0x561d1c0a7980 2024-09-16T01:54:06.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.964+0000 7f73bbfff640 1 -- 172.21.15.139:0/2387477148 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7f73d000b4d0 con 0x561d1c0a7980 2024-09-16T01:54:06.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.964+0000 7f73e508f740 1 -- 172.21.15.139:0/2387477148 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d4051040 con 0x561d1c0a7980 2024-09-16T01:54:06.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.966+0000 7f73bbfff640 1 -- 172.21.15.139:0/2387477148 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f73d000fc60 con 0x561d1c0a7980 2024-09-16T01:54:06.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.967+0000 7f73e508f740 10 client.26347 fetch_fsmap learned FSMap version 69 2024-09-16T01:54:06.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.967+0000 7f73e508f740 1 -- 172.21.15.139:0/2387477148 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x561d1c12d320 con 0x561d1c0a7980 2024-09-16T01:54:06.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.969+0000 7f73bbfff640 1 -- 172.21.15.139:0/2387477148 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d4051040 con 0x561d1c0a7980 2024-09-16T01:54:06.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.969+0000 7f73bbfff640 1 -- 172.21.15.139:0/2387477148 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 69) ==== 8438+0+0 (secure 0 0 0) 0x7f73d00117e0 con 0x561d1c0a7980 2024-09-16T01:54:06.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.971+0000 7f73e508f740 10 client.26347 fetch_fsmap finished waiting for FSMap version 69 2024-09-16T01:54:06.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.971+0000 7f73e508f740 10 client.26347 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:54:06.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.971+0000 7f73e508f740 20 client.26347 populate_metadata read hostname 'smithi139' 2024-09-16T01:54:06.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.971+0000 7f73e508f740 1 --2- 172.21.15.139:0/2387477148 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d1c12dac0 0x561d1c12fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:06.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.971+0000 7f73e508f740 4 client.26347 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "34558dc0", "release": true, "await": true}] 2024-09-16T01:54:06.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.971+0000 7f73e508f740 1 -- 172.21.15.139:0/2387477148 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "34558dc0", "release": true, "await": true}) -- 0x561d1c0140a0 con 0x561d1c12dac0 2024-09-16T01:54:06.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.973+0000 7f73dbfff640 1 --2- 172.21.15.139:0/2387477148 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d1c12dac0 0x561d1c12fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:06.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.976+0000 7f73dbfff640 1 --2- 172.21.15.139:0/2387477148 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d1c12dac0 0x561d1c12fee0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68031ad1a7a0a7fe server_cookie=d95507d2d598ab69 in_seq=0 out_seq=0 2024-09-16T01:54:06.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:06.978+0000 7f73bbfff640 10 client.26347 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:07 smithi164 ceph-mon[35121]: pgmap v432: 129 pgs: 10 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 90 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 416 KiB/s wr, 217 op/s 2024-09-16T01:54:07.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:07 smithi139 ceph-mon[31175]: pgmap v432: 129 pgs: 10 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 90 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 416 KiB/s wr, 217 op/s 2024-09-16T01:54:07.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:07 smithi176 ceph-mon[35324]: pgmap v432: 129 pgs: 10 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 90 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 416 KiB/s wr, 217 op/s 2024-09-16T01:54:08.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.061+0000 7f73bbfff640 1 -- 172.21.15.139:0/2387477148 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561d1c0140a0 con 0x561d1c12dac0 2024-09-16T01:54:08.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.061+0000 7f73bbfff640 10 client.26347 handle_command_reply: tid=0 2024-09-16T01:54:08.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.061+0000 7f73e508f740 1 client.26347 shutdown 2024-09-16T01:54:08.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.062+0000 7f73e508f740 1 -- 172.21.15.139:0/2387477148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0a7980 msgr2=0x561d1c0b6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:08.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.062+0000 7f73e508f740 1 --2- 172.21.15.139:0/2387477148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0a7980 0x561d1c0b6d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f73d0003cb0 tx=0x7f73d00045f0 comp rx=0 tx=0).stop 2024-09-16T01:54:08.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.062+0000 7f73e508f740 1 -- 172.21.15.139:0/2387477148 shutdown_connections 2024-09-16T01:54:08.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.062+0000 7f73e508f740 1 --2- 172.21.15.139:0/2387477148 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561d1c12dac0 0x561d1c12fee0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:08.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.062+0000 7f73e508f740 1 --2- 172.21.15.139:0/2387477148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d1c0b2240 0x561d1c125970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:08.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.062+0000 7f73e508f740 1 --2- 172.21.15.139:0/2387477148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d1c0a7980 0x561d1c0b6d80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:08.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.062+0000 7f73e508f740 1 --2- 172.21.15.139:0/2387477148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561d1c0a7060 0x561d1c0b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:08.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.062+0000 7f73e508f740 1 -- 172.21.15.139:0/2387477148 >> 172.21.15.139:0/2387477148 conn(0x561d1c0a35f0 msgr2=0x561d1c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:08.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.062+0000 7f73bbfff640 0 client.26347 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:08.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.062+0000 7f73e508f740 1 -- 172.21.15.139:0/2387477148 shutdown_connections 2024-09-16T01:54:08.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.062+0000 7f73e508f740 1 -- 172.21.15.139:0/2387477148 wait complete. 2024-09-16T01:54:08.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.063+0000 7f73e508f740 20 client.26347 trim_cache size 0 max 16384 2024-09-16T01:54:08.067 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:54:08.067 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 69, 2024-09-16T01:54:08.067 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:54:08.067 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 48, 2024-09-16T01:54:08.068 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:54:08.068 INFO:teuthology.orchestra.run.smithi139.stdout: "34558dc0": { 2024-09-16T01:54:08.068 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 48, 2024-09-16T01:54:08.068 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:54:08.068 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:54:08.068 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:54:08.068 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:54:08.068 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:54:08.068 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:54:08.068 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 131.4, 2024-09-16T01:54:08.068 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:54:08.068 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:54:08.068 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:54:08.069 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:54:08.069 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:54:08.069 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:54:08.069 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:54:08.069 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:54:08.069 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:54:08.069 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:54:08.069 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:54:08.069 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:54:08.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.064+0000 7f73e2cb6640 1 -- 172.21.15.139:0/1368894937 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f73c40a35c0 msgr2=0x7f73c40a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:08.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.064+0000 7f73e2cb6640 1 --2- 172.21.15.139:0/1368894937 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f73c40a35c0 0x7f73c40a5a80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009610 tx=0x7f73cc005cf0 comp rx=0 tx=0).stop 2024-09-16T01:54:08.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.064+0000 7f73e2cb6640 1 -- 172.21.15.139:0/1368894937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73dc11b140 msgr2=0x7f73dc1abaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:08.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.064+0000 7f73e2cb6640 1 --2- 172.21.15.139:0/1368894937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73dc11b140 0x7f73dc1abaf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00d720 tx=0x7f73cc00dbe0 comp rx=0 tx=0).stop 2024-09-16T01:54:08.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.064+0000 7f73e2cb6640 1 -- 172.21.15.139:0/1368894937 shutdown_connections 2024-09-16T01:54:08.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.064+0000 7f73e2cb6640 1 --2- 172.21.15.139:0/1368894937 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f73c40a35c0 0x7f73c40a5a80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:08.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.064+0000 7f73e2cb6640 1 --2- 172.21.15.139:0/1368894937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73dc11b140 0x7f73dc1abaf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:08.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.064+0000 7f73e2cb6640 1 --2- 172.21.15.139:0/1368894937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73dc109830 0x7f73dc1a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:08.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.064+0000 7f73e2cb6640 1 --2- 172.21.15.139:0/1368894937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73dc109450 0x7f73dc1a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:08.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.064+0000 7f73e2cb6640 1 -- 172.21.15.139:0/1368894937 >> 172.21.15.139:0/1368894937 conn(0x7f73dc06e140 msgr2=0x7f73dc11c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:08.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.065+0000 7f73e2cb6640 1 -- 172.21.15.139:0/1368894937 shutdown_connections 2024-09-16T01:54:08.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:08.065+0000 7f73e2cb6640 1 -- 172.21.15.139:0/1368894937 wait complete. 2024-09-16T01:54:08.833 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '34558dc0', total seconds elapsed: 20.7 2024-09-16T01:54:09.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:09 smithi139 ceph-mon[31175]: pgmap v433: 129 pgs: 9 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 99 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 55 KiB/s wr, 223 op/s 2024-09-16T01:54:09.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:09 smithi139 ceph-mon[31175]: osdmap e105: 12 total, 12 up, 12 in 2024-09-16T01:54:09.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:09 smithi176 ceph-mon[35324]: pgmap v433: 129 pgs: 9 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 99 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 55 KiB/s wr, 223 op/s 2024-09-16T01:54:09.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:09 smithi176 ceph-mon[35324]: osdmap e105: 12 total, 12 up, 12 in 2024-09-16T01:54:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:09 smithi164 ceph-mon[35121]: pgmap v433: 129 pgs: 9 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 99 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 55 KiB/s wr, 223 op/s 2024-09-16T01:54:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:09 smithi164 ceph-mon[35121]: osdmap e105: 12 total, 12 up, 12 in 2024-09-16T01:54:11.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:11 smithi139 ceph-mon[31175]: pgmap v435: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 476 KiB/s wr, 246 op/s 2024-09-16T01:54:11.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:11 smithi176 ceph-mon[35324]: pgmap v435: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 476 KiB/s wr, 246 op/s 2024-09-16T01:54:11.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:11 smithi164 ceph-mon[35121]: pgmap v435: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 476 KiB/s wr, 246 op/s 2024-09-16T01:54:13.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:13 smithi139 ceph-mon[31175]: pgmap v436: 129 pgs: 22 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 50 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 651 KiB/s wr, 240 op/s 2024-09-16T01:54:13.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:13 smithi176 ceph-mon[35324]: pgmap v436: 129 pgs: 22 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 50 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 651 KiB/s wr, 240 op/s 2024-09-16T01:54:13.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:13 smithi164 ceph-mon[35121]: pgmap v436: 129 pgs: 22 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 50 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 651 KiB/s wr, 240 op/s 2024-09-16T01:54:15.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:15 smithi139 ceph-mon[31175]: pgmap v437: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 233 op/s 2024-09-16T01:54:15.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:15 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:54:15.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:15 smithi176 ceph-mon[35324]: pgmap v437: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 233 op/s 2024-09-16T01:54:15.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:15 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:54:15.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:15 smithi164 ceph-mon[35121]: pgmap v437: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 233 op/s 2024-09-16T01:54:15.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:15 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:54:17.596 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:17 smithi164 ceph-mon[35121]: pgmap v438: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 225 op/s 2024-09-16T01:54:17.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:17 smithi139 ceph-mon[31175]: pgmap v438: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 225 op/s 2024-09-16T01:54:17.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:17 smithi176 ceph-mon[35324]: pgmap v438: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 225 op/s 2024-09-16T01:54:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.036+0000 7f791b1fc640 1 -- 172.21.15.139:0/1158282666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f790c09ce00 msgr2=0x7f790c0a8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.036+0000 7f791b1fc640 1 --2- 172.21.15.139:0/1158282666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f790c09ce00 0x7f790c0a8a70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7908009900 tx=0x7f790802a440 comp rx=0 tx=0).stop 2024-09-16T01:54:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.041+0000 7f791b1fc640 1 -- 172.21.15.139:0/1158282666 shutdown_connections 2024-09-16T01:54:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.041+0000 7f791b1fc640 1 --2- 172.21.15.139:0/1158282666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f790c09ce00 0x7f790c0a8a70 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.041+0000 7f791b1fc640 1 --2- 172.21.15.139:0/1158282666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790c09c460 0x7f790c09c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.041+0000 7f791b1fc640 1 --2- 172.21.15.139:0/1158282666 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f790c0a1db0 0x7f790c0a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.041+0000 7f791b1fc640 1 -- 172.21.15.139:0/1158282666 >> 172.21.15.139:0/1158282666 conn(0x7f790c009ff0 msgr2=0x7f790c00a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.046+0000 7f791b1fc640 1 -- 172.21.15.139:0/1158282666 shutdown_connections 2024-09-16T01:54:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.050+0000 7f791b1fc640 1 -- 172.21.15.139:0/1158282666 wait complete. 2024-09-16T01:54:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.051+0000 7f791b1fc640 1 Processor -- start 2024-09-16T01:54:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.051+0000 7f791b1fc640 1 -- start start 2024-09-16T01:54:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.051+0000 7f791b1fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f790c09c460 0x7f790c014c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.051+0000 7f791b1fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f790c09ce00 0x7f790c00dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.051+0000 7f791b1fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790c0a1db0 0x7f790c00e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.051+0000 7f791b1fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f790c0ac170 con 0x7f790c09ce00 2024-09-16T01:54:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.051+0000 7f791b1fc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f790c0abff0 con 0x7f790c0a1db0 2024-09-16T01:54:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.051+0000 7f791b1fc640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f790c0ac2f0 con 0x7f790c09c460 2024-09-16T01:54:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.052+0000 7f791a9fb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790c0a1db0 0x7f790c00e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.052+0000 7f79199f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f790c09ce00 0x7f790c00dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.052+0000 7f79199f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f790c09ce00 0x7f790c00dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34574/0 (socket says 172.21.15.139:34574) 2024-09-16T01:54:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.052+0000 7f79199f9640 1 -- 172.21.15.139:0/529961802 learned_addr learned my addr 172.21.15.139:0/529961802 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.052+0000 7f79199f9640 1 -- 172.21.15.139:0/529961802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f790c09c460 msgr2=0x7f790c014c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:54:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.052+0000 7f79199f9640 1 --2- 172.21.15.139:0/529961802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f790c09c460 0x7f790c014c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.052+0000 7f79199f9640 1 -- 172.21.15.139:0/529961802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790c0a1db0 msgr2=0x7f790c00e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.052+0000 7f79199f9640 1 --2- 172.21.15.139:0/529961802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790c0a1db0 0x7f790c00e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.052+0000 7f79199f9640 1 -- 172.21.15.139:0/529961802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790c00e790 con 0x7f790c09ce00 2024-09-16T01:54:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.053+0000 7f79199f9640 1 --2- 172.21.15.139:0/529961802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f790c09ce00 0x7f790c00dce0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7914065a40 tx=0x7f79140506d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.055+0000 7f79077fe640 1 -- 172.21.15.139:0/529961802 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791407bdb0 con 0x7f790c09ce00 2024-09-16T01:54:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.055+0000 7f79077fe640 1 -- 172.21.15.139:0/529961802 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7914080900 con 0x7f790c09ce00 2024-09-16T01:54:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.055+0000 7f79077fe640 1 -- 172.21.15.139:0/529961802 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791407f680 con 0x7f790c09ce00 2024-09-16T01:54:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.055+0000 7f791b1fc640 1 -- 172.21.15.139:0/529961802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790c00ea80 con 0x7f790c09ce00 2024-09-16T01:54:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.055+0000 7f791b1fc640 1 -- 172.21.15.139:0/529961802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790c0129d0 con 0x7f790c09ce00 2024-09-16T01:54:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.064+0000 7f79077fe640 1 -- 172.21.15.139:0/529961802 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7914087030 con 0x7f790c09ce00 2024-09-16T01:54:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.065+0000 7f79077fe640 1 --2- 172.21.15.139:0/529961802 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f79000a35c0 0x7f79000a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.065+0000 7f79077fe640 1 -- 172.21.15.139:0/529961802 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9706+0+0 (secure 0 0 0) 0x7f791413c790 con 0x7f790c09ce00 2024-09-16T01:54:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.070+0000 7f791a1fa640 1 --2- 172.21.15.139:0/529961802 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f79000a35c0 0x7f79000a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.091+0000 7f791f1fb740 1 Processor -- start 2024-09-16T01:54:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.091+0000 7f791f1fb740 1 -- start start 2024-09-16T01:54:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.092+0000 7f791f1fb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649eacc0 0x558e649eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.092+0000 7f791f1fb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649eb5e0 0x558e649f5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.092+0000 7f791f1fb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649f5e80 0x558e649f8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.092+0000 7f791f1fb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558e6494f2f0 con 0x558e649eb5e0 2024-09-16T01:54:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.092+0000 7f791f1fb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558e648f7270 con 0x558e649eacc0 2024-09-16T01:54:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.092+0000 7f791f1fb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558e6494b2b0 con 0x558e649f5e80 2024-09-16T01:54:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.094+0000 7f791a1fa640 1 --2- 172.21.15.139:0/529961802 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f79000a35c0 0x7f79000a5a80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7910009670 tx=0x7f7910006bd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.095+0000 7f791a9fb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649f5e80 0x558e649f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.095+0000 7f791a9fb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649f5e80 0x558e649f8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:50726/0 (socket says 172.21.15.139:50726) 2024-09-16T01:54:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.095+0000 7f791a9fb640 1 -- 172.21.15.139:0/3062574752 learned_addr learned my addr 172.21.15.139:0/3062574752 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.095+0000 7f79199f9640 1 --2- 172.21.15.139:0/3062574752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649eb5e0 0x558e649f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.096+0000 7f791a9fb640 1 -- 172.21.15.139:0/3062574752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649eacc0 msgr2=0x558e649eb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.096+0000 7f791a9fb640 1 --2- 172.21.15.139:0/3062574752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649eacc0 0x558e649eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.096+0000 7f791a9fb640 1 -- 172.21.15.139:0/3062574752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649eb5e0 msgr2=0x558e649f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.096+0000 7f791a9fb640 1 --2- 172.21.15.139:0/3062574752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649eb5e0 0x558e649f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.096+0000 7f791a9fb640 1 -- 172.21.15.139:0/3062574752 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e648f6340 con 0x558e649f5e80 2024-09-16T01:54:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.096+0000 7f791a9fb640 1 --2- 172.21.15.139:0/3062574752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649f5e80 0x558e649f8270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7908006010 tx=0x7f7908004180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9110872363f09b1b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.098+0000 7f7904bfe640 1 -- 172.21.15.139:0/3062574752 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790803f070 con 0x558e649f5e80 2024-09-16T01:54:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.098+0000 7f7904bfe640 1 -- 172.21.15.139:0/3062574752 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7908005520 con 0x558e649f5e80 2024-09-16T01:54:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.099+0000 7f791f1fb740 1 -- 172.21.15.139:0/3062574752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649f5e80 msgr2=0x558e649f8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.099+0000 7f791f1fb740 1 --2- 172.21.15.139:0/3062574752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649f5e80 0x558e649f8270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7908006010 tx=0x7f7908004180 comp rx=0 tx=0).stop 2024-09-16T01:54:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.101+0000 7f791f1fb740 1 -- 172.21.15.139:0/3062574752 shutdown_connections 2024-09-16T01:54:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.101+0000 7f791f1fb740 1 --2- 172.21.15.139:0/3062574752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649f5e80 0x558e649f8270 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.101+0000 7f791f1fb740 1 --2- 172.21.15.139:0/3062574752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649eb5e0 0x558e649f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.101+0000 7f791f1fb740 1 --2- 172.21.15.139:0/3062574752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649eacc0 0x558e649eb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.101+0000 7f791f1fb740 1 -- 172.21.15.139:0/3062574752 >> 172.21.15.139:0/3062574752 conn(0x558e649e7230 msgr2=0x558e649e7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.106+0000 7f791f1fb740 1 -- 172.21.15.139:0/3062574752 shutdown_connections 2024-09-16T01:54:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.107+0000 7f791f1fb740 1 -- 172.21.15.139:0/3062574752 wait complete. 2024-09-16T01:54:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.107+0000 7f791f1fb740 1 -- start start 2024-09-16T01:54:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.107+0000 7f791f1fb740 1 Processor -- start 2024-09-16T01:54:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.109+0000 7f791f1fb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649eacc0 0x558e649f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.109+0000 7f791f1fb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eb5e0 0x558e649f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.109+0000 7f791f1fb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649f5e80 0x558e649f3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.109+0000 7f791f1fb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558e649f8aa0 con 0x558e649f5e80 2024-09-16T01:54:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.109+0000 7f791f1fb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558e648d8290 con 0x558e649eacc0 2024-09-16T01:54:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.109+0000 7f791f1fb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558e649f8c20 con 0x558e649eb5e0 2024-09-16T01:54:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.109+0000 7f791f1fb740 10 client.0 _ll_register_callbacks cb 0x558e64909140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:54:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.112+0000 7f79199f9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eb5e0 0x558e649f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.112+0000 7f79199f9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eb5e0 0x558e649f31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:50730/0 (socket says 172.21.15.139:50730) 2024-09-16T01:54:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.112+0000 7f79199f9640 1 -- 172.21.15.139:0/4170620963 learned_addr learned my addr 172.21.15.139:0/4170620963 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.112+0000 7f791a9fb640 1 --2- 172.21.15.139:0/4170620963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649f5e80 0x558e649f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.113+0000 7f791a1fa640 1 --2- 172.21.15.139:0/4170620963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649eacc0 0x558e649f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.114+0000 7f791a1fa640 1 -- 172.21.15.139:0/4170620963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eb5e0 msgr2=0x558e649f31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.114+0000 7f791a1fa640 1 --2- 172.21.15.139:0/4170620963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eb5e0 0x558e649f31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.114+0000 7f791a1fa640 1 -- 172.21.15.139:0/4170620963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649f5e80 msgr2=0x558e649f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.114+0000 7f791a1fa640 1 --2- 172.21.15.139:0/4170620963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649f5e80 0x558e649f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.114+0000 7f791a1fa640 1 -- 172.21.15.139:0/4170620963 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e64662210 con 0x558e649eacc0 2024-09-16T01:54:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.118+0000 7f791a1fa640 1 --2- 172.21.15.139:0/4170620963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649eacc0 0x558e649f2cb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7910009f00 tx=0x7f791000b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4805107de8e88ee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.119+0000 7f78fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:54:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.120+0000 7f78fdffb640 1 -- 172.21.15.139:0/4170620963 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791001d5f0 con 0x558e649eacc0 2024-09-16T01:54:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.120+0000 7f791f1fb740 1 -- 172.21.15.139:0/4170620963 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7908005520 con 0x558e649eacc0 2024-09-16T01:54:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.120+0000 7f78fdffb640 1 -- 172.21.15.139:0/4170620963 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9706+0+0 (secure 0 0 0) 0x7f791001d790 con 0x558e649eacc0 2024-09-16T01:54:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.124+0000 7f78fdffb640 1 -- 172.21.15.139:0/4170620963 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f791002b050 con 0x558e649eacc0 2024-09-16T01:54:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.127+0000 7f791f1fb740 10 client.26384 fetch_fsmap learned FSMap version 70 2024-09-16T01:54:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.127+0000 7f791f1fb740 1 -- 172.21.15.139:0/4170620963 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x558e6472e620 con 0x558e649eacc0 2024-09-16T01:54:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.131+0000 7f78fdffb640 1 -- 172.21.15.139:0/4170620963 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7908005520 con 0x558e649eacc0 2024-09-16T01:54:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.131+0000 7f78fdffb640 1 -- 172.21.15.139:0/4170620963 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 70) ==== 8442+0+0 (secure 0 0 0) 0x7f7910012070 con 0x558e649eacc0 2024-09-16T01:54:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.131+0000 7f791f1fb740 10 client.26384 fetch_fsmap finished waiting for FSMap version 70 2024-09-16T01:54:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.131+0000 7f791f1fb740 10 client.26384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:54:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.131+0000 7f791f1fb740 20 client.26384 populate_metadata read hostname 'smithi139' 2024-09-16T01:54:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.131+0000 7f791f1fb740 1 --2- 172.21.15.139:0/4170620963 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e64a173b0 0x558e64a19830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.131+0000 7f791f1fb740 4 client.26384 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:54:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.131+0000 7f791f1fb740 1 -- 172.21.15.139:0/4170620963 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e6495ca20 con 0x558e64a173b0 2024-09-16T01:54:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.136+0000 7f79199f9640 1 --2- 172.21.15.139:0/4170620963 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e64a173b0 0x558e64a19830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.141+0000 7f79199f9640 1 --2- 172.21.15.139:0/4170620963 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e64a173b0 0x558e64a19830 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3f75389f5407945 server_cookie=19f869061f8d0bac in_seq=0 out_seq=0 2024-09-16T01:54:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.141+0000 7f78fdffb640 10 client.26384 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.143+0000 7f78fdffb640 1 -- 172.21.15.139:0/4170620963 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e6495ca20 con 0x558e64a173b0 2024-09-16T01:54:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.143+0000 7f78fdffb640 10 client.26384 handle_command_reply: tid=0 2024-09-16T01:54:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.143+0000 7f791f1fb740 1 client.26384 shutdown 2024-09-16T01:54:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.148+0000 7f791f1fb740 1 -- 172.21.15.139:0/4170620963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649eacc0 msgr2=0x558e649f2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.148+0000 7f791f1fb740 1 --2- 172.21.15.139:0/4170620963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649eacc0 0x558e649f2cb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7910009f00 tx=0x7f791000b7b0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.149+0000 7f791f1fb740 1 -- 172.21.15.139:0/4170620963 shutdown_connections 2024-09-16T01:54:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.149+0000 7f791f1fb740 1 --2- 172.21.15.139:0/4170620963 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e64a173b0 0x558e64a19830 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.149+0000 7f791f1fb740 1 --2- 172.21.15.139:0/4170620963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649f5e80 0x558e649f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.149+0000 7f791f1fb740 1 --2- 172.21.15.139:0/4170620963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eb5e0 0x558e649f31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.149+0000 7f791f1fb740 1 --2- 172.21.15.139:0/4170620963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649eacc0 0x558e649f2cb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.149+0000 7f791f1fb740 1 -- 172.21.15.139:0/4170620963 >> 172.21.15.139:0/4170620963 conn(0x558e649e7230 msgr2=0x558e649ecf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.149+0000 7f78fdffb640 0 client.26384 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.152+0000 7f791f1fb740 1 -- 172.21.15.139:0/4170620963 shutdown_connections 2024-09-16T01:54:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.154+0000 7f791f1fb740 1 -- 172.21.15.139:0/4170620963 wait complete. 2024-09-16T01:54:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.157+0000 7f791f1fb740 20 client.26384 trim_cache size 0 max 16384 2024-09-16T01:54:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.188+0000 7f791f1fb740 1 Processor -- start 2024-09-16T01:54:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.188+0000 7f791f1fb740 1 -- start start 2024-09-16T01:54:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.188+0000 7f791f1fb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eacc0 0x558e649e6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.188+0000 7f791f1fb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649eb5e0 0x558e649f3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.188+0000 7f791f1fb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649f5e80 0x558e64a1f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.188+0000 7f791f1fb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8010340 con 0x558e649f5e80 2024-09-16T01:54:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.188+0000 7f791f1fb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8012120 con 0x558e649eb5e0 2024-09-16T01:54:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.188+0000 7f791f1fb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8010a70 con 0x558e649eacc0 2024-09-16T01:54:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.189+0000 7f791a1fa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eacc0 0x558e649e6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.189+0000 7f791a1fa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eacc0 0x558e649e6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:50740/0 (socket says 172.21.15.139:50740) 2024-09-16T01:54:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.189+0000 7f791a1fa640 1 -- 172.21.15.139:0/2650335824 learned_addr learned my addr 172.21.15.139:0/2650335824 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.190+0000 7f791a1fa640 1 -- 172.21.15.139:0/2650335824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649eb5e0 msgr2=0x558e649f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.190+0000 7f791a1fa640 1 --2- 172.21.15.139:0/2650335824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649eb5e0 0x558e649f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.190+0000 7f791a1fa640 1 -- 172.21.15.139:0/2650335824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649f5e80 msgr2=0x558e64a1f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.190+0000 7f791a1fa640 1 --2- 172.21.15.139:0/2650335824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649f5e80 0x558e64a1f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.190+0000 7f791a1fa640 1 -- 172.21.15.139:0/2650335824 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e649eaa40 con 0x558e649eacc0 2024-09-16T01:54:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.190+0000 7f791a1fa640 1 --2- 172.21.15.139:0/2650335824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eacc0 0x558e649e6170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7910000c00 tx=0x7f7910013b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c6daf1b78526e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.193+0000 7f78fffff640 1 -- 172.21.15.139:0/2650335824 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910034070 con 0x558e649eacc0 2024-09-16T01:54:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.193+0000 7f78fffff640 1 -- 172.21.15.139:0/2650335824 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7910022070 con 0x558e649eacc0 2024-09-16T01:54:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.194+0000 7f791f1fb740 1 -- 172.21.15.139:0/2650335824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eacc0 msgr2=0x558e649e6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.194+0000 7f791f1fb740 1 --2- 172.21.15.139:0/2650335824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eacc0 0x558e649e6170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7910000c00 tx=0x7f7910013b90 comp rx=0 tx=0).stop 2024-09-16T01:54:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.194+0000 7f791f1fb740 1 -- 172.21.15.139:0/2650335824 shutdown_connections 2024-09-16T01:54:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.194+0000 7f791f1fb740 1 --2- 172.21.15.139:0/2650335824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649f5e80 0x558e64a1f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.194+0000 7f791f1fb740 1 --2- 172.21.15.139:0/2650335824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649eb5e0 0x558e649f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.194+0000 7f791f1fb740 1 --2- 172.21.15.139:0/2650335824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eacc0 0x558e649e6170 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.194+0000 7f791f1fb740 1 -- 172.21.15.139:0/2650335824 >> 172.21.15.139:0/2650335824 conn(0x558e649e7230 msgr2=0x558e649e5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.198+0000 7f791f1fb740 1 -- 172.21.15.139:0/2650335824 shutdown_connections 2024-09-16T01:54:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.198+0000 7f791f1fb740 1 -- 172.21.15.139:0/2650335824 wait complete. 2024-09-16T01:54:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.198+0000 7f791f1fb740 1 -- start start 2024-09-16T01:54:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.198+0000 7f791f1fb740 1 Processor -- start 2024-09-16T01:54:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.199+0000 7f791f1fb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eacc0 0x558e649fdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.199+0000 7f791f1fb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649eb5e0 0x558e649fe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.199+0000 7f791f1fb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649f5e80 0x558e649ef0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.199+0000 7f791f1fb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8011990 con 0x558e649eb5e0 2024-09-16T01:54:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.199+0000 7f791f1fb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8011200 con 0x558e649f5e80 2024-09-16T01:54:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.199+0000 7f791f1fb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f78f800e340 con 0x558e649eacc0 2024-09-16T01:54:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.199+0000 7f791f1fb740 10 client.0 _ll_register_callbacks cb 0x7f7910009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:54:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.199+0000 7f79199f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649eb5e0 0x558e649fe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.199+0000 7f79199f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649eb5e0 0x558e649fe4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34614/0 (socket says 172.21.15.139:34614) 2024-09-16T01:54:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.199+0000 7f79199f9640 1 -- 172.21.15.139:0/1899404290 learned_addr learned my addr 172.21.15.139:0/1899404290 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.199+0000 7f791a1fa640 1 --2- 172.21.15.139:0/1899404290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eacc0 0x558e649fdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.200+0000 7f791a9fb640 1 --2- 172.21.15.139:0/1899404290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649f5e80 0x558e649ef0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.200+0000 7f791a1fa640 1 -- 172.21.15.139:0/1899404290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649f5e80 msgr2=0x558e649ef0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.200+0000 7f791a1fa640 1 --2- 172.21.15.139:0/1899404290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649f5e80 0x558e649ef0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.200+0000 7f791a1fa640 1 -- 172.21.15.139:0/1899404290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649eb5e0 msgr2=0x558e649fe4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.200+0000 7f791a1fa640 1 --2- 172.21.15.139:0/1899404290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649eb5e0 0x558e649fe4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.200+0000 7f791a1fa640 1 -- 172.21.15.139:0/1899404290 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e649ef620 con 0x558e649eacc0 2024-09-16T01:54:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.202+0000 7f791a1fa640 1 --2- 172.21.15.139:0/1899404290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eacc0 0x558e649fdf60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7910026de0 tx=0x7f7910003d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cf4fddafcb93421 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.202+0000 7f78fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:54:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.202+0000 7f78fdffb640 1 -- 172.21.15.139:0/1899404290 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910034070 con 0x558e649eacc0 2024-09-16T01:54:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.202+0000 7f791f1fb740 1 -- 172.21.15.139:0/1899404290 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558e64956d70 con 0x558e649eacc0 2024-09-16T01:54:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.202+0000 7f78fdffb640 1 -- 172.21.15.139:0/1899404290 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9706+0+0 (secure 0 0 0) 0x7f7910004450 con 0x558e649eacc0 2024-09-16T01:54:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.208+0000 7f78fdffb640 1 -- 172.21.15.139:0/1899404290 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f7910026e60 con 0x558e649eacc0 2024-09-16T01:54:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.209+0000 7f791f1fb740 10 client.26359 fetch_fsmap learned FSMap version 70 2024-09-16T01:54:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.209+0000 7f791f1fb740 1 -- 172.21.15.139:0/1899404290 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x558e64a818c0 con 0x558e649eacc0 2024-09-16T01:54:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.215+0000 7f78fdffb640 1 -- 172.21.15.139:0/1899404290 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558e64956d70 con 0x558e649eacc0 2024-09-16T01:54:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.215+0000 7f78fdffb640 1 -- 172.21.15.139:0/1899404290 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 70) ==== 8442+0+0 (secure 0 0 0) 0x7f7910012070 con 0x558e649eacc0 2024-09-16T01:54:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.215+0000 7f791f1fb740 10 client.26359 fetch_fsmap finished waiting for FSMap version 70 2024-09-16T01:54:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.215+0000 7f791f1fb740 10 client.26359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:54:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.215+0000 7f791f1fb740 20 client.26359 populate_metadata read hostname 'smithi139' 2024-09-16T01:54:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.215+0000 7f791f1fb740 1 --2- 172.21.15.139:0/1899404290 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e64a81e70 0x558e64a84290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.215+0000 7f791f1fb740 4 client.26359 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:54:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.215+0000 7f791f1fb740 1 -- 172.21.15.139:0/1899404290 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e648d3900 con 0x558e64a81e70 2024-09-16T01:54:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.219+0000 7f79199f9640 1 --2- 172.21.15.139:0/1899404290 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e64a81e70 0x558e64a84290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.223+0000 7f78fdffb640 10 client.26359 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.223+0000 7f79199f9640 1 --2- 172.21.15.139:0/1899404290 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e64a81e70 0x558e64a84290 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=979b150bae9ba086 server_cookie=76664ad091dea40 in_seq=0 out_seq=0 2024-09-16T01:54:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.375+0000 7f78fdffb640 1 -- 172.21.15.139:0/1899404290 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e648d3900 con 0x558e64a81e70 2024-09-16T01:54:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.375+0000 7f78fdffb640 10 client.26359 handle_command_reply: tid=0 2024-09-16T01:54:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.375+0000 7f791f1fb740 1 client.26359 shutdown 2024-09-16T01:54:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.375+0000 7f791f1fb740 1 -- 172.21.15.139:0/1899404290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eacc0 msgr2=0x558e649fdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.375+0000 7f791f1fb740 1 --2- 172.21.15.139:0/1899404290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eacc0 0x558e649fdf60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7910026de0 tx=0x7f7910003d00 comp rx=0 tx=0).stop 2024-09-16T01:54:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.375+0000 7f791f1fb740 1 -- 172.21.15.139:0/1899404290 shutdown_connections 2024-09-16T01:54:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.375+0000 7f791f1fb740 1 --2- 172.21.15.139:0/1899404290 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x558e64a81e70 0x558e64a84290 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.375+0000 7f791f1fb740 1 --2- 172.21.15.139:0/1899404290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e649f5e80 0x558e649ef0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.375+0000 7f791f1fb740 1 --2- 172.21.15.139:0/1899404290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e649eb5e0 0x558e649fe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.375+0000 7f791f1fb740 1 --2- 172.21.15.139:0/1899404290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558e649eacc0 0x558e649fdf60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.375+0000 7f791f1fb740 1 -- 172.21.15.139:0/1899404290 >> 172.21.15.139:0/1899404290 conn(0x558e649e7230 msgr2=0x558e649e6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.376+0000 7f78fdffb640 0 client.26359 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.376+0000 7f791f1fb740 1 -- 172.21.15.139:0/1899404290 shutdown_connections 2024-09-16T01:54:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.376+0000 7f791f1fb740 1 -- 172.21.15.139:0/1899404290 wait complete. 2024-09-16T01:54:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.376+0000 7f791f1fb740 20 client.26359 trim_cache size 0 max 16384 2024-09-16T01:54:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:54:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:54:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:54:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:54:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.377+0000 7f791b1fc640 1 -- 172.21.15.139:0/529961802 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f79000a35c0 msgr2=0x7f79000a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.377+0000 7f791b1fc640 1 --2- 172.21.15.139:0/529961802 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f79000a35c0 0x7f79000a5a80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7910009670 tx=0x7f7910006bd0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.377+0000 7f791b1fc640 1 -- 172.21.15.139:0/529961802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f790c09ce00 msgr2=0x7f790c00dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.377+0000 7f791b1fc640 1 --2- 172.21.15.139:0/529961802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f790c09ce00 0x7f790c00dce0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7914065a40 tx=0x7f79140506d0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.377+0000 7f791b1fc640 1 -- 172.21.15.139:0/529961802 shutdown_connections 2024-09-16T01:54:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.377+0000 7f791b1fc640 1 --2- 172.21.15.139:0/529961802 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f79000a35c0 0x7f79000a5a80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.377+0000 7f791b1fc640 1 --2- 172.21.15.139:0/529961802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f790c0a1db0 0x7f790c00e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.377+0000 7f791b1fc640 1 --2- 172.21.15.139:0/529961802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f790c09ce00 0x7f790c00dce0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.377+0000 7f791b1fc640 1 --2- 172.21.15.139:0/529961802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f790c09c460 0x7f790c014c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.377+0000 7f791b1fc640 1 -- 172.21.15.139:0/529961802 >> 172.21.15.139:0/529961802 conn(0x7f790c009ff0 msgr2=0x7f790c09b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.377+0000 7f791b1fc640 1 -- 172.21.15.139:0/529961802 shutdown_connections 2024-09-16T01:54:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:18.377+0000 7f791b1fc640 1 -- 172.21.15.139:0/529961802 wait complete. 2024-09-16T01:54:19.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:19 smithi139 ceph-mon[31175]: pgmap v439: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 210 op/s 2024-09-16T01:54:19.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:19 smithi176 ceph-mon[35324]: pgmap v439: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 210 op/s 2024-09-16T01:54:19.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:19 smithi164 ceph-mon[35121]: pgmap v439: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 210 op/s 2024-09-16T01:54:20.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:20 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:54:20.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:20 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:54:20.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:20 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:54:21.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:21 smithi139 ceph-mon[31175]: pgmap v440: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-09-16T01:54:21.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:21 smithi176 ceph-mon[35324]: pgmap v440: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-09-16T01:54:21.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:21 smithi164 ceph-mon[35121]: pgmap v440: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-09-16T01:54:23.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:23 smithi139 ceph-mon[31175]: pgmap v441: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2024-09-16T01:54:23.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:23 smithi176 ceph-mon[35324]: pgmap v441: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2024-09-16T01:54:23.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:23 smithi164 ceph-mon[35121]: pgmap v441: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2024-09-16T01:54:24.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:54:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:54:24.181+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:54:24.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:54:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:54:24.181+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:54:25.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:25 smithi139 ceph-mon[31175]: pgmap v442: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-09-16T01:54:25.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:25 smithi176 ceph-mon[35324]: pgmap v442: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-09-16T01:54:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:25 smithi164 ceph-mon[35121]: pgmap v442: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-09-16T01:54:27.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:27 smithi139 ceph-mon[31175]: pgmap v443: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-09-16T01:54:27.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:27 smithi176 ceph-mon[35324]: pgmap v443: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-09-16T01:54:27.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:27 smithi164 ceph-mon[35121]: pgmap v443: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-09-16T01:54:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:29 smithi139 ceph-mon[31175]: pgmap v444: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-09-16T01:54:29.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:29 smithi176 ceph-mon[35324]: pgmap v444: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-09-16T01:54:29.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:29 smithi164 ceph-mon[35121]: pgmap v444: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-09-16T01:54:30.005 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.5 2024-09-16T01:54:30.006 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:54:31.477 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:54:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:31 smithi139 ceph-mon[31175]: pgmap v445: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 197 op/s 2024-09-16T01:54:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:31 smithi139 ceph-mon[31175]: osdmap e106: 12 total, 12 up, 12 in 2024-09-16T01:54:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.669+0000 7f68c9c4a640 1 -- 172.21.15.139:0/3122014259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68c4101180 msgr2=0x7f68c410d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.669+0000 7f68c9c4a640 1 --2- 172.21.15.139:0/3122014259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68c4101180 0x7f68c410d6b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009cd0 tx=0x7f68bc02a930 comp rx=0 tx=0).stop 2024-09-16T01:54:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.669+0000 7f68c9c4a640 1 -- 172.21.15.139:0/3122014259 shutdown_connections 2024-09-16T01:54:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.669+0000 7f68c9c4a640 1 --2- 172.21.15.139:0/3122014259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68c4101180 0x7f68c410d6b0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.669+0000 7f68c9c4a640 1 --2- 172.21.15.139:0/3122014259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68c41007e0 0x7f68c4100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.669+0000 7f68c9c4a640 1 --2- 172.21.15.139:0/3122014259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c41067f0 0x7f68c4106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.669+0000 7f68c9c4a640 1 -- 172.21.15.139:0/3122014259 >> 172.21.15.139:0/3122014259 conn(0x7f68c40fc4e0 msgr2=0x7f68c40fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.672+0000 7f68c9c4a640 1 -- 172.21.15.139:0/3122014259 shutdown_connections 2024-09-16T01:54:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.673+0000 7f68c9c4a640 1 -- 172.21.15.139:0/3122014259 wait complete. 2024-09-16T01:54:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.674+0000 7f68c9c4a640 1 Processor -- start 2024-09-16T01:54:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.674+0000 7f68c9c4a640 1 -- start start 2024-09-16T01:54:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.674+0000 7f68c9c4a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68c41007e0 0x7f68c4115f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.674+0000 7f68c9c4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68c4101180 0x7f68c4116460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.674+0000 7f68c9c4a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c41067f0 0x7f68c4111030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.674+0000 7f68c9c4a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f68c410fb30 con 0x7f68c4101180 2024-09-16T01:54:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.674+0000 7f68c9c4a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f68c410f9b0 con 0x7f68c41007e0 2024-09-16T01:54:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.674+0000 7f68c9c4a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f68c410fcb0 con 0x7f68c41067f0 2024-09-16T01:54:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.675+0000 7f68c9449640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c41067f0 0x7f68c4111030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.675+0000 7f68c9449640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c41067f0 0x7f68c4111030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:58568/0 (socket says 172.21.15.139:58568) 2024-09-16T01:54:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.675+0000 7f68c9449640 1 -- 172.21.15.139:0/3870553558 learned_addr learned my addr 172.21.15.139:0/3870553558 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.675+0000 7f68c9449640 1 -- 172.21.15.139:0/3870553558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68c41007e0 msgr2=0x7f68c4115f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.675+0000 7f68c9449640 1 --2- 172.21.15.139:0/3870553558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68c41007e0 0x7f68c4115f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.675+0000 7f68c9449640 1 -- 172.21.15.139:0/3870553558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68c4101180 msgr2=0x7f68c4116460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.675+0000 7f68c9449640 1 --2- 172.21.15.139:0/3870553558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68c4101180 0x7f68c4116460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.675+0000 7f68c9449640 1 -- 172.21.15.139:0/3870553558 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c41118f0 con 0x7f68c41067f0 2024-09-16T01:54:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.679+0000 7f68c9449640 1 --2- 172.21.15.139:0/3870553558 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c41067f0 0x7f68c4111030 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68bc02af20 tx=0x7f68bc031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.681+0000 7f68c1ffb640 1 -- 172.21.15.139:0/3870553558 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc031ba0 con 0x7f68c41067f0 2024-09-16T01:54:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.681+0000 7f68c9c4a640 1 -- 172.21.15.139:0/3870553558 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c4111b80 con 0x7f68c41067f0 2024-09-16T01:54:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.681+0000 7f68c9c4a640 1 -- 172.21.15.139:0/3870553558 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c411de20 con 0x7f68c41067f0 2024-09-16T01:54:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.681+0000 7f68c1ffb640 1 -- 172.21.15.139:0/3870553558 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68bc040070 con 0x7f68c41067f0 2024-09-16T01:54:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.681+0000 7f68c1ffb640 1 -- 172.21.15.139:0/3870553558 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0305f0 con 0x7f68c41067f0 2024-09-16T01:54:31.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:31 smithi176 ceph-mon[35324]: pgmap v445: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 197 op/s 2024-09-16T01:54:31.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:31 smithi176 ceph-mon[35324]: osdmap e106: 12 total, 12 up, 12 in 2024-09-16T01:54:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.687+0000 7f68cdc4b740 1 Processor -- start 2024-09-16T01:54:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.688+0000 7f68cdc4b740 1 -- start start 2024-09-16T01:54:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.688+0000 7f68cdc4b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1abcc0 0x561aba1ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.688+0000 7f68cdc4b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1ac5e0 0x561aba1b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.688+0000 7f68cdc4b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b6e80 0x561aba1b9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.688+0000 7f68cdc4b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561aba1102f0 con 0x561aba1abcc0 2024-09-16T01:54:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.688+0000 7f68cdc4b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561aba0b8270 con 0x561aba1ac5e0 2024-09-16T01:54:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.688+0000 7f68cdc4b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561aba10c2b0 con 0x561aba1b6e80 2024-09-16T01:54:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.689+0000 7f68c8c48640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1ac5e0 0x561aba1b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.689+0000 7f68c8c48640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1ac5e0 0x561aba1b6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39072/0 (socket says 172.21.15.139:39072) 2024-09-16T01:54:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.689+0000 7f68c8c48640 1 -- 172.21.15.139:0/3994721397 learned_addr learned my addr 172.21.15.139:0/3994721397 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.691+0000 7f68c3fff640 1 --2- 172.21.15.139:0/3994721397 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b6e80 0x561aba1b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.691+0000 7f68c8c48640 1 -- 172.21.15.139:0/3994721397 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b6e80 msgr2=0x561aba1b9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.691+0000 7f68c8c48640 1 --2- 172.21.15.139:0/3994721397 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b6e80 0x561aba1b9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.691+0000 7f68c8c48640 1 -- 172.21.15.139:0/3994721397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1abcc0 msgr2=0x561aba1ac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.691+0000 7f68c8c48640 1 --2- 172.21.15.139:0/3994721397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1abcc0 0x561aba1ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.691+0000 7f68c8c48640 1 -- 172.21.15.139:0/3994721397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aba0b7340 con 0x561aba1ac5e0 2024-09-16T01:54:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.691+0000 7f68c1ffb640 1 -- 172.21.15.139:0/3870553558 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f68bc02f640 con 0x7f68c41067f0 2024-09-16T01:54:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.692+0000 7f68c1ffb640 1 --2- 172.21.15.139:0/3870553558 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f68b40a35c0 0x7f68b40a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.692+0000 7f68c1ffb640 1 -- 172.21.15.139:0/3870553558 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f68bc0f1a90 con 0x7f68c41067f0 2024-09-16T01:54:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.697+0000 7f68c3fff640 1 --2- 172.21.15.139:0/3870553558 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f68b40a35c0 0x7f68b40a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.698+0000 7f68c8c48640 1 --2- 172.21.15.139:0/3994721397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1ac5e0 0x561aba1b6940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f68b800ba80 tx=0x7f68b800bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6db975cb9ee4e81e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.699+0000 7f68aeffd640 1 -- 172.21.15.139:0/3994721397 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8016070 con 0x561aba1ac5e0 2024-09-16T01:54:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.703+0000 7f68c3fff640 1 --2- 172.21.15.139:0/3870553558 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f68b40a35c0 0x7f68b40a5a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x561aba1a95b0 tx=0x7f68b000a5a0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.704+0000 7f68aeffd640 1 -- 172.21.15.139:0/3994721397 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b8005500 con 0x561aba1ac5e0 2024-09-16T01:54:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.704+0000 7f68aeffd640 1 -- 172.21.15.139:0/3994721397 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8010460 con 0x561aba1ac5e0 2024-09-16T01:54:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.706+0000 7f68cdc4b740 1 -- 172.21.15.139:0/3994721397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1ac5e0 msgr2=0x561aba1b6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.706+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/3994721397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1ac5e0 0x561aba1b6940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f68b800ba80 tx=0x7f68b800bf40 comp rx=0 tx=0).stop 2024-09-16T01:54:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.706+0000 7f68cdc4b740 1 -- 172.21.15.139:0/3994721397 shutdown_connections 2024-09-16T01:54:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.706+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/3994721397 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b6e80 0x561aba1b9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.706+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/3994721397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1ac5e0 0x561aba1b6940 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f68b800ba80 tx=0x7f68b800bf40 comp rx=0 tx=0).stop 2024-09-16T01:54:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.706+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/3994721397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1abcc0 0x561aba1ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.706+0000 7f68cdc4b740 1 -- 172.21.15.139:0/3994721397 >> 172.21.15.139:0/3994721397 conn(0x561aba1a8230 msgr2=0x561aba1a8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.714+0000 7f68cdc4b740 1 -- 172.21.15.139:0/3994721397 shutdown_connections 2024-09-16T01:54:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.715+0000 7f68cdc4b740 1 -- 172.21.15.139:0/3994721397 wait complete. 2024-09-16T01:54:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.715+0000 7f68cdc4b740 1 -- start start 2024-09-16T01:54:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.715+0000 7f68cdc4b740 1 Processor -- start 2024-09-16T01:54:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.718+0000 7f68cdc4b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1abcc0 0x561aba1b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.718+0000 7f68cdc4b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1b6e80 0x561aba1b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.718+0000 7f68cdc4b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b49b0 0x561aba1c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.718+0000 7f68cdc4b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561aba1b9a50 con 0x561aba1abcc0 2024-09-16T01:54:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.718+0000 7f68cdc4b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561aba099290 con 0x561aba1b6e80 2024-09-16T01:54:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.718+0000 7f68cdc4b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561aba1b9bd0 con 0x561aba1b49b0 2024-09-16T01:54:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.718+0000 7f68cdc4b740 10 client.0 _ll_register_callbacks cb 0x561aba0ca140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:54:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.720+0000 7f68c8c48640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1abcc0 0x561aba1b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.720+0000 7f68c8c48640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1abcc0 0x561aba1b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47250/0 (socket says 172.21.15.139:47250) 2024-09-16T01:54:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.720+0000 7f68c8c48640 1 -- 172.21.15.139:0/1674827536 learned_addr learned my addr 172.21.15.139:0/1674827536 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.721+0000 7f68c9449640 1 --2- 172.21.15.139:0/1674827536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b49b0 0x561aba1c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.723+0000 7f68c3fff640 1 --2- 172.21.15.139:0/1674827536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1b6e80 0x561aba1b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.724+0000 7f68c9449640 1 -- 172.21.15.139:0/1674827536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1b6e80 msgr2=0x561aba1b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.724+0000 7f68c9449640 1 --2- 172.21.15.139:0/1674827536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1b6e80 0x561aba1b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.724+0000 7f68c9449640 1 -- 172.21.15.139:0/1674827536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1abcc0 msgr2=0x561aba1b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.724+0000 7f68c9449640 1 --2- 172.21.15.139:0/1674827536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1abcc0 0x561aba1b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.724+0000 7f68c9449640 1 -- 172.21.15.139:0/1674827536 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab9e23210 con 0x561aba1b49b0 2024-09-16T01:54:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.727+0000 7f68c9449640 1 --2- 172.21.15.139:0/1674827536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b49b0 0x561aba1c2f90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0eee10 tx=0x7f68bc0bd3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf05786304328865 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.728+0000 7f68a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:54:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.731+0000 7f68a3fff640 1 -- 172.21.15.139:0/1674827536 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0d5070 con 0x561aba1b49b0 2024-09-16T01:54:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.731+0000 7f68a3fff640 1 -- 172.21.15.139:0/1674827536 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f68bc0eefb0 con 0x561aba1b49b0 2024-09-16T01:54:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.731+0000 7f68cdc4b740 1 -- 172.21.15.139:0/1674827536 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b8005500 con 0x561aba1b49b0 2024-09-16T01:54:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.738+0000 7f68a3fff640 1 -- 172.21.15.139:0/1674827536 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f68bc0ef1b0 con 0x561aba1b49b0 2024-09-16T01:54:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.739+0000 7f68cdc4b740 10 client.26368 fetch_fsmap learned FSMap version 71 2024-09-16T01:54:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.739+0000 7f68cdc4b740 1 -- 172.21.15.139:0/1674827536 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x561ab9eef620 con 0x561aba1b49b0 2024-09-16T01:54:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.741+0000 7f68a3fff640 1 -- 172.21.15.139:0/1674827536 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b8005500 con 0x561aba1b49b0 2024-09-16T01:54:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.741+0000 7f68a3fff640 1 -- 172.21.15.139:0/1674827536 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f68bc04d070 con 0x561aba1b49b0 2024-09-16T01:54:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.743+0000 7f68cdc4b740 10 client.26368 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:54:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.743+0000 7f68cdc4b740 10 client.26368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:54:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.743+0000 7f68cdc4b740 20 client.26368 populate_metadata read hostname 'smithi139' 2024-09-16T01:54:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.743+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/1674827536 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561aba1d8490 0x561aba1da910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.743+0000 7f68cdc4b740 4 client.26368 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:54:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.743+0000 7f68cdc4b740 1 -- 172.21.15.139:0/1674827536 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561aba11da20 con 0x561aba1d8490 2024-09-16T01:54:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.744+0000 7f68c8c48640 1 --2- 172.21.15.139:0/1674827536 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561aba1d8490 0x561aba1da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.755+0000 7f68c8c48640 1 --2- 172.21.15.139:0/1674827536 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561aba1d8490 0x561aba1da910 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82ea11e41ed329bc server_cookie=df0dd2ee8cf2bb60 in_seq=0 out_seq=0 2024-09-16T01:54:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.755+0000 7f68a3fff640 10 client.26368 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.760+0000 7f68a3fff640 1 -- 172.21.15.139:0/1674827536 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561aba11da20 con 0x561aba1d8490 2024-09-16T01:54:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.760+0000 7f68a3fff640 10 client.26368 handle_command_reply: tid=0 2024-09-16T01:54:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.760+0000 7f68cdc4b740 1 client.26368 shutdown 2024-09-16T01:54:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.765+0000 7f68cdc4b740 1 -- 172.21.15.139:0/1674827536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b49b0 msgr2=0x561aba1c2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.765+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/1674827536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b49b0 0x561aba1c2f90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0eee10 tx=0x7f68bc0bd3f0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.765+0000 7f68cdc4b740 1 -- 172.21.15.139:0/1674827536 shutdown_connections 2024-09-16T01:54:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.765+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/1674827536 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561aba1d8490 0x561aba1da910 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.765+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/1674827536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b49b0 0x561aba1c2f90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.765+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/1674827536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1b6e80 0x561aba1b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.765+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/1674827536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1abcc0 0x561aba1b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.765+0000 7f68cdc4b740 1 -- 172.21.15.139:0/1674827536 >> 172.21.15.139:0/1674827536 conn(0x561aba1a8230 msgr2=0x561aba1adf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.765+0000 7f68a3fff640 0 client.26368 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.766+0000 7f68cdc4b740 1 -- 172.21.15.139:0/1674827536 shutdown_connections 2024-09-16T01:54:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.767+0000 7f68cdc4b740 1 -- 172.21.15.139:0/1674827536 wait complete. 2024-09-16T01:54:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.771+0000 7f68cdc4b740 20 client.26368 trim_cache size 0 max 16384 2024-09-16T01:54:31.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:31 smithi164 ceph-mon[35121]: pgmap v445: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 197 op/s 2024-09-16T01:54:31.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:31 smithi164 ceph-mon[35121]: osdmap e106: 12 total, 12 up, 12 in 2024-09-16T01:54:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.803+0000 7f68cdc4b740 1 Processor -- start 2024-09-16T01:54:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.808+0000 7f68cdc4b740 1 -- start start 2024-09-16T01:54:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.808+0000 7f68cdc4b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1abcc0 0x561aba1c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.808+0000 7f68cdc4b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1b49b0 0x561aba1e01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.808+0000 7f68cdc4b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1b6e80 0x561aba1e0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.808+0000 7f68cdc4b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f68a80101d0 con 0x561aba1b49b0 2024-09-16T01:54:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.808+0000 7f68cdc4b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8012010 con 0x561aba1b6e80 2024-09-16T01:54:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.808+0000 7f68cdc4b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8010960 con 0x561aba1abcc0 2024-09-16T01:54:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.809+0000 7f68c3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1b49b0 0x561aba1e01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.809+0000 7f68c3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1b49b0 0x561aba1e01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47262/0 (socket says 172.21.15.139:47262) 2024-09-16T01:54:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.809+0000 7f68c3fff640 1 -- 172.21.15.139:0/916463736 learned_addr learned my addr 172.21.15.139:0/916463736 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.814+0000 7f68c9449640 1 --2- 172.21.15.139:0/916463736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1b6e80 0x561aba1e0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.814+0000 7f68c8c48640 1 --2- 172.21.15.139:0/916463736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1abcc0 0x561aba1c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.815+0000 7f68c3fff640 1 -- 172.21.15.139:0/916463736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1abcc0 msgr2=0x561aba1c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.815+0000 7f68c3fff640 1 --2- 172.21.15.139:0/916463736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1abcc0 0x561aba1c2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.815+0000 7f68c3fff640 1 -- 172.21.15.139:0/916463736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1b6e80 msgr2=0x561aba1e0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.815+0000 7f68c3fff640 1 --2- 172.21.15.139:0/916463736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1b6e80 0x561aba1e0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.815+0000 7f68c3fff640 1 -- 172.21.15.139:0/916463736 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aba1ac3d0 con 0x561aba1b49b0 2024-09-16T01:54:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.815+0000 7f68c9449640 1 --2- 172.21.15.139:0/916463736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1b6e80 0x561aba1e0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:54:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.822+0000 7f68c3fff640 1 --2- 172.21.15.139:0/916463736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1b49b0 0x561aba1e01e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f68c41127f0 tx=0x7f68b0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c022472f16e65f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.827+0000 7f68ae7fc640 1 -- 172.21.15.139:0/916463736 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b001e430 con 0x561aba1b49b0 2024-09-16T01:54:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.827+0000 7f68ae7fc640 1 -- 172.21.15.139:0/916463736 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b001eab0 con 0x561aba1b49b0 2024-09-16T01:54:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.827+0000 7f68ae7fc640 1 -- 172.21.15.139:0/916463736 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b001db70 con 0x561aba1b49b0 2024-09-16T01:54:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.828+0000 7f68cdc4b740 1 -- 172.21.15.139:0/916463736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1b49b0 msgr2=0x561aba1e01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.828+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/916463736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1b49b0 0x561aba1e01e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f68c41127f0 tx=0x7f68b0011000 comp rx=0 tx=0).stop 2024-09-16T01:54:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.830+0000 7f68cdc4b740 1 -- 172.21.15.139:0/916463736 shutdown_connections 2024-09-16T01:54:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.830+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/916463736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1b6e80 0x561aba1e0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.830+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/916463736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1b49b0 0x561aba1e01e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.830+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/916463736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1abcc0 0x561aba1c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.830+0000 7f68cdc4b740 1 -- 172.21.15.139:0/916463736 >> 172.21.15.139:0/916463736 conn(0x561aba1a8230 msgr2=0x561aba1af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.832+0000 7f68cdc4b740 1 -- 172.21.15.139:0/916463736 shutdown_connections 2024-09-16T01:54:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.834+0000 7f68cdc4b740 1 -- 172.21.15.139:0/916463736 wait complete. 2024-09-16T01:54:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.835+0000 7f68cdc4b740 1 -- start start 2024-09-16T01:54:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.835+0000 7f68cdc4b740 1 Processor -- start 2024-09-16T01:54:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.835+0000 7f68cdc4b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1abcc0 0x561aba1bf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.835+0000 7f68cdc4b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b49b0 0x561aba1bf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.835+0000 7f68cdc4b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1b6e80 0x561aba1b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.836+0000 7f68cdc4b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8011880 con 0x561aba1b6e80 2024-09-16T01:54:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.836+0000 7f68cdc4b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f68a80110f0 con 0x561aba1abcc0 2024-09-16T01:54:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.836+0000 7f68cdc4b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f68a800e1b0 con 0x561aba1b49b0 2024-09-16T01:54:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.836+0000 7f68cdc4b740 10 client.0 _ll_register_callbacks cb 0x7f68bc009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:54:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.836+0000 7f68c3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b49b0 0x561aba1bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.836+0000 7f68c3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b49b0 0x561aba1bf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:58616/0 (socket says 172.21.15.139:58616) 2024-09-16T01:54:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.836+0000 7f68c3fff640 1 -- 172.21.15.139:0/448148331 learned_addr learned my addr 172.21.15.139:0/448148331 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.836+0000 7f68c9449640 1 --2- 172.21.15.139:0/448148331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1b6e80 0x561aba1b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.836+0000 7f68c8c48640 1 --2- 172.21.15.139:0/448148331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1abcc0 0x561aba1bf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.838+0000 7f68c9449640 1 -- 172.21.15.139:0/448148331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b49b0 msgr2=0x561aba1bf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.838+0000 7f68c9449640 1 --2- 172.21.15.139:0/448148331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b49b0 0x561aba1bf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.838+0000 7f68c9449640 1 -- 172.21.15.139:0/448148331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1abcc0 msgr2=0x561aba1bf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.838+0000 7f68c9449640 1 --2- 172.21.15.139:0/448148331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1abcc0 0x561aba1bf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.838+0000 7f68c9449640 1 -- 172.21.15.139:0/448148331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aba1b1000 con 0x561aba1b6e80 2024-09-16T01:54:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.838+0000 7f68c9449640 1 --2- 172.21.15.139:0/448148331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1b6e80 0x561aba1b0800 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f68bc005c40 tx=0x7f68bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ca0cf53f0ab8ef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.838+0000 7f68a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:54:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.838+0000 7f68a3fff640 1 -- 172.21.15.139:0/448148331 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0ca020 con 0x561aba1b6e80 2024-09-16T01:54:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.839+0000 7f68a3fff640 1 -- 172.21.15.139:0/448148331 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f68bc0dc350 con 0x561aba1b6e80 2024-09-16T01:54:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.839+0000 7f68cdc4b740 1 -- 172.21.15.139:0/448148331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b001eab0 con 0x561aba1b6e80 2024-09-16T01:54:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.839+0000 7f68a3fff640 1 -- 172.21.15.139:0/448148331 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f68bc0bd9d0 con 0x561aba1b6e80 2024-09-16T01:54:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.839+0000 7f68cdc4b740 10 client.16962 fetch_fsmap learned FSMap version 71 2024-09-16T01:54:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.839+0000 7f68cdc4b740 1 -- 172.21.15.139:0/448148331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x561aba242cc0 con 0x561aba1b6e80 2024-09-16T01:54:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.840+0000 7f68a3fff640 1 -- 172.21.15.139:0/448148331 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b001eab0 con 0x561aba1b6e80 2024-09-16T01:54:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.840+0000 7f68a3fff640 1 -- 172.21.15.139:0/448148331 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f68bc04d040 con 0x561aba1b6e80 2024-09-16T01:54:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.840+0000 7f68cdc4b740 10 client.16962 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:54:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.840+0000 7f68cdc4b740 10 client.16962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:54:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.840+0000 7f68cdc4b740 20 client.16962 populate_metadata read hostname 'smithi139' 2024-09-16T01:54:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.840+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/448148331 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561aba2433d0 0x561aba2457f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.840+0000 7f68cdc4b740 4 client.16962 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:54:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.840+0000 7f68cdc4b740 1 -- 172.21.15.139:0/448148331 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561aba117d70 con 0x561aba2433d0 2024-09-16T01:54:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.841+0000 7f68c8c48640 1 --2- 172.21.15.139:0/448148331 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561aba2433d0 0x561aba2457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.851+0000 7f68c8c48640 1 --2- 172.21.15.139:0/448148331 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561aba2433d0 0x561aba2457f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40d3ec332f21a8d6 server_cookie=516a3201e04ad4b0 in_seq=0 out_seq=0 2024-09-16T01:54:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.851+0000 7f68a3fff640 10 client.16962 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.959+0000 7f68a3fff640 1 -- 172.21.15.139:0/448148331 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561aba117d70 con 0x561aba2433d0 2024-09-16T01:54:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.959+0000 7f68a3fff640 10 client.16962 handle_command_reply: tid=0 2024-09-16T01:54:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.959+0000 7f68cdc4b740 1 client.16962 shutdown 2024-09-16T01:54:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.960+0000 7f68cdc4b740 1 -- 172.21.15.139:0/448148331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1b6e80 msgr2=0x561aba1b0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.960+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/448148331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1b6e80 0x561aba1b0800 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f68bc005c40 tx=0x7f68bc009c00 comp rx=0 tx=0).stop 2024-09-16T01:54:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.960+0000 7f68cdc4b740 1 -- 172.21.15.139:0/448148331 shutdown_connections 2024-09-16T01:54:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.960+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/448148331 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561aba2433d0 0x561aba2457f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.960+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/448148331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561aba1b6e80 0x561aba1b0800 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.960+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/448148331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561aba1b49b0 0x561aba1bf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.960+0000 7f68a3fff640 0 client.16962 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.960+0000 7f68cdc4b740 1 --2- 172.21.15.139:0/448148331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aba1abcc0 0x561aba1bf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.960+0000 7f68cdc4b740 1 -- 172.21.15.139:0/448148331 >> 172.21.15.139:0/448148331 conn(0x561aba1a8230 msgr2=0x561aba1ad720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.960+0000 7f68cdc4b740 1 -- 172.21.15.139:0/448148331 shutdown_connections 2024-09-16T01:54:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.960+0000 7f68cdc4b740 1 -- 172.21.15.139:0/448148331 wait complete. 2024-09-16T01:54:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.960+0000 7f68cdc4b740 20 client.16962 trim_cache size 0 max 16384 2024-09-16T01:54:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:54:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:54:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:54:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:54:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.961+0000 7f68c9c4a640 1 -- 172.21.15.139:0/3870553558 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f68b40a35c0 msgr2=0x7f68b40a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.961+0000 7f68c9c4a640 1 --2- 172.21.15.139:0/3870553558 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f68b40a35c0 0x7f68b40a5a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x561aba1a95b0 tx=0x7f68b000a5a0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.961+0000 7f68c9c4a640 1 -- 172.21.15.139:0/3870553558 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c41067f0 msgr2=0x7f68c4111030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.961+0000 7f68c9c4a640 1 --2- 172.21.15.139:0/3870553558 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c41067f0 0x7f68c4111030 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68bc02af20 tx=0x7f68bc031a20 comp rx=0 tx=0).stop 2024-09-16T01:54:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.961+0000 7f68c9c4a640 1 -- 172.21.15.139:0/3870553558 shutdown_connections 2024-09-16T01:54:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.961+0000 7f68c9c4a640 1 --2- 172.21.15.139:0/3870553558 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f68b40a35c0 0x7f68b40a5a80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.961+0000 7f68c9c4a640 1 --2- 172.21.15.139:0/3870553558 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c41067f0 0x7f68c4111030 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.961+0000 7f68c9c4a640 1 --2- 172.21.15.139:0/3870553558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68c4101180 0x7f68c4116460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.961+0000 7f68c9c4a640 1 --2- 172.21.15.139:0/3870553558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68c41007e0 0x7f68c4115f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.961+0000 7f68c9c4a640 1 -- 172.21.15.139:0/3870553558 >> 172.21.15.139:0/3870553558 conn(0x7f68c40fc4e0 msgr2=0x7f68c40fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.961+0000 7f68c9c4a640 1 -- 172.21.15.139:0/3870553558 shutdown_connections 2024-09-16T01:54:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:31.961+0000 7f68c9c4a640 1 -- 172.21.15.139:0/3870553558 wait complete. 2024-09-16T01:54:32.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.473+0000 7f17a1c66640 1 -- 172.21.15.139:0/2651824896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f179c10d530 msgr2=0x7f179c1150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:32.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.473+0000 7f17a1c66640 1 --2- 172.21.15.139:0/2651824896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f179c10d530 0x7f179c1150b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1790009860 tx=0x7f179002a380 comp rx=0 tx=0).stop 2024-09-16T01:54:32.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.474+0000 7f17a1c66640 1 -- 172.21.15.139:0/2651824896 shutdown_connections 2024-09-16T01:54:32.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.474+0000 7f17a1c66640 1 --2- 172.21.15.139:0/2651824896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f179c10d530 0x7f179c1150b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:32.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.474+0000 7f17a1c66640 1 --2- 172.21.15.139:0/2651824896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f179c108e00 0x7f179c10ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:32.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.474+0000 7f17a1c66640 1 --2- 172.21.15.139:0/2651824896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f179c108450 0x7f179c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:32.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.474+0000 7f17a1c66640 1 -- 172.21.15.139:0/2651824896 >> 172.21.15.139:0/2651824896 conn(0x7f179c077ba0 msgr2=0x7f179c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:32.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.474+0000 7f17a1c66640 1 -- 172.21.15.139:0/2651824896 shutdown_connections 2024-09-16T01:54:32.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.474+0000 7f17a1c66640 1 -- 172.21.15.139:0/2651824896 wait complete. 2024-09-16T01:54:32.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.475+0000 7f17a1c66640 1 Processor -- start 2024-09-16T01:54:32.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.478+0000 7f17a1c66640 1 -- start start 2024-09-16T01:54:32.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.479+0000 7f17a1c66640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f179c108450 0x7f179c073b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:32.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.479+0000 7f17a1c66640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f179c108e00 0x7f179c06db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:32.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.479+0000 7f17a1c66640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f179c10d530 0x7f179c06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:32.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.479+0000 7f17a1c66640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f179c06c660 con 0x7f179c108e00 2024-09-16T01:54:32.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.479+0000 7f17a1c66640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f179c06c4e0 con 0x7f179c10d530 2024-09-16T01:54:32.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.479+0000 7f17a1c66640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f179c06c7e0 con 0x7f179c108450 2024-09-16T01:54:32.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.479+0000 7f179bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f179c10d530 0x7f179c06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:32.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.479+0000 7f179bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f179c10d530 0x7f179c06e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:46202/0 (socket says 172.21.15.139:46202) 2024-09-16T01:54:32.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.479+0000 7f179bfff640 1 -- 172.21.15.139:0/2986800278 learned_addr learned my addr 172.21.15.139:0/2986800278 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:32.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.480+0000 7f179bfff640 1 -- 172.21.15.139:0/2986800278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f179c108450 msgr2=0x7f179c073b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:54:32.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.480+0000 7f179bfff640 1 --2- 172.21.15.139:0/2986800278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f179c108450 0x7f179c073b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:32.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.480+0000 7f179bfff640 1 -- 172.21.15.139:0/2986800278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f179c108e00 msgr2=0x7f179c06db90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:54:32.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.480+0000 7f179bfff640 1 --2- 172.21.15.139:0/2986800278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f179c108e00 0x7f179c06db90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:32.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.480+0000 7f179bfff640 1 -- 172.21.15.139:0/2986800278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c06e7d0 con 0x7f179c10d530 2024-09-16T01:54:32.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.483+0000 7f179bfff640 1 --2- 172.21.15.139:0/2986800278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f179c10d530 0x7f179c06e0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1790009830 tx=0x7f1790030340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:32.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.485+0000 7f1798ff9640 1 -- 172.21.15.139:0/2986800278 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790040020 con 0x7f179c10d530 2024-09-16T01:54:32.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.485+0000 7f1798ff9640 1 -- 172.21.15.139:0/2986800278 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1790005520 con 0x7f179c10d530 2024-09-16T01:54:32.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.486+0000 7f1798ff9640 1 -- 172.21.15.139:0/2986800278 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790037070 con 0x7f179c10d530 2024-09-16T01:54:32.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.486+0000 7f17a1c66640 1 -- 172.21.15.139:0/2986800278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c07aa40 con 0x7f179c10d530 2024-09-16T01:54:32.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.486+0000 7f17a1c66640 1 -- 172.21.15.139:0/2986800278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c07aea0 con 0x7f179c10d530 2024-09-16T01:54:32.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.490+0000 7f1798ff9640 1 -- 172.21.15.139:0/2986800278 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1790043020 con 0x7f179c10d530 2024-09-16T01:54:32.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.491+0000 7f17a1c66640 1 -- 172.21.15.139:0/2986800278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179c06e9a0 con 0x7f179c10d530 2024-09-16T01:54:32.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.491+0000 7f1798ff9640 1 --2- 172.21.15.139:0/2986800278 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17840a3460 0x7f17840a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:32.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.492+0000 7f1798ff9640 1 -- 172.21.15.139:0/2986800278 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f17900ef990 con 0x7f179c10d530 2024-09-16T01:54:32.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.496+0000 7f179b7fe640 1 --2- 172.21.15.139:0/2986800278 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17840a3460 0x7f17840a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:32.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.501+0000 7f1798ff9640 1 -- 172.21.15.139:0/2986800278 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f179004a020 con 0x7f179c10d530 2024-09-16T01:54:32.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.508+0000 7f179b7fe640 1 --2- 172.21.15.139:0/2986800278 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17840a3460 0x7f17840a5920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f178c005fd0 tx=0x7f178c009260 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:32.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.910+0000 7f17a1c66640 1 -- 172.21.15.139:0/2986800278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f179c108830 con 0x7f179c10d530 2024-09-16T01:54:32.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.914+0000 7f1798ff9640 1 -- 172.21.15.139:0/2986800278 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+12031 (secure 0 0 0) 0x7f17900bbfd0 con 0x7f179c10d530 2024-09-16T01:54:32.916 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:54:32.916 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":71,"btime":"2024-09-16T01:54:19:332763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:54:19.051272+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:54:32.916 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 71 2024-09-16T01:54:32.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.920+0000 7f17a1c66640 1 -- 172.21.15.139:0/2986800278 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17840a3460 msgr2=0x7f17840a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:32.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.920+0000 7f17a1c66640 1 --2- 172.21.15.139:0/2986800278 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17840a3460 0x7f17840a5920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f178c005fd0 tx=0x7f178c009260 comp rx=0 tx=0).stop 2024-09-16T01:54:32.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.920+0000 7f17a1c66640 1 -- 172.21.15.139:0/2986800278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f179c10d530 msgr2=0x7f179c06e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:32.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.920+0000 7f17a1c66640 1 --2- 172.21.15.139:0/2986800278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f179c10d530 0x7f179c06e0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1790009830 tx=0x7f1790030340 comp rx=0 tx=0).stop 2024-09-16T01:54:32.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.920+0000 7f17a1c66640 1 -- 172.21.15.139:0/2986800278 shutdown_connections 2024-09-16T01:54:32.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.920+0000 7f17a1c66640 1 --2- 172.21.15.139:0/2986800278 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17840a3460 0x7f17840a5920 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:32.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.920+0000 7f17a1c66640 1 --2- 172.21.15.139:0/2986800278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f179c10d530 0x7f179c06e0d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:32.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.920+0000 7f17a1c66640 1 --2- 172.21.15.139:0/2986800278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f179c108e00 0x7f179c06db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:32.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.920+0000 7f17a1c66640 1 --2- 172.21.15.139:0/2986800278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f179c108450 0x7f179c073b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:32.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.920+0000 7f17a1c66640 1 -- 172.21.15.139:0/2986800278 >> 172.21.15.139:0/2986800278 conn(0x7f179c077ba0 msgr2=0x7f179c078660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:32.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.920+0000 7f17a1c66640 1 -- 172.21.15.139:0/2986800278 shutdown_connections 2024-09-16T01:54:32.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:32.920+0000 7f17a1c66640 1 -- 172.21.15.139:0/2986800278 wait complete. 2024-09-16T01:54:33.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:33 smithi139 ceph-mon[31175]: pgmap v447: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1024 KiB/s wr, 199 op/s 2024-09-16T01:54:33.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:33 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2986800278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:54:33.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:33 smithi176 ceph-mon[35324]: pgmap v447: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1024 KiB/s wr, 199 op/s 2024-09-16T01:54:33.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:33 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2986800278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:54:33.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:33 smithi164 ceph-mon[35121]: pgmap v447: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1024 KiB/s wr, 199 op/s 2024-09-16T01:54:33.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:33 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2986800278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:54:33.865 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 146.5 --await-for 45.0' 2024-09-16T01:54:33.865 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 146.5 --await-for 45.0 2024-09-16T01:54:34.371 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:54:35.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.109+0000 7f307252a640 1 -- 172.21.15.139:0/2158730012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f306c101270 msgr2=0x7f306c10d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:35.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.109+0000 7f307252a640 1 --2- 172.21.15.139:0/2158730012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f306c101270 0x7f306c10d7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f306401ace0 tx=0x7f306403b970 comp rx=0 tx=0).stop 2024-09-16T01:54:35.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.109+0000 7f307252a640 1 -- 172.21.15.139:0/2158730012 shutdown_connections 2024-09-16T01:54:35.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.109+0000 7f307252a640 1 --2- 172.21.15.139:0/2158730012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f306c101270 0x7f306c10d7a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.109+0000 7f307252a640 1 --2- 172.21.15.139:0/2158730012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306c1008d0 0x7f306c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.109+0000 7f307252a640 1 --2- 172.21.15.139:0/2158730012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f306c1068d0 0x7f306c106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.109+0000 7f307252a640 1 -- 172.21.15.139:0/2158730012 >> 172.21.15.139:0/2158730012 conn(0x7f306c0fc610 msgr2=0x7f306c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:35.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.110+0000 7f307252a640 1 -- 172.21.15.139:0/2158730012 shutdown_connections 2024-09-16T01:54:35.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.112+0000 7f307252a640 1 -- 172.21.15.139:0/2158730012 wait complete. 2024-09-16T01:54:35.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.113+0000 7f307252a640 1 Processor -- start 2024-09-16T01:54:35.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.116+0000 7f307252a640 1 -- start start 2024-09-16T01:54:35.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.116+0000 7f307252a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f306c1008d0 0x7f306c10c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.116+0000 7f307252a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306c1068d0 0x7f306c10cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.116+0000 7f307252a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f306c1963a0 0x7f306c196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.116+0000 7f307252a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f306c06e660 con 0x7f306c1068d0 2024-09-16T01:54:35.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.116+0000 7f307252a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f306c06e4e0 con 0x7f306c1008d0 2024-09-16T01:54:35.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.116+0000 7f307252a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f306c06e7e0 con 0x7f306c1963a0 2024-09-16T01:54:35.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.116+0000 7f3070d27640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306c1068d0 0x7f306c10cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:35.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.116+0000 7f3070d27640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306c1068d0 0x7f306c10cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53754/0 (socket says 172.21.15.139:53754) 2024-09-16T01:54:35.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.116+0000 7f3070d27640 1 -- 172.21.15.139:0/3588173379 learned_addr learned my addr 172.21.15.139:0/3588173379 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:35.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.117+0000 7f3071528640 1 --2- 172.21.15.139:0/3588173379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f306c1008d0 0x7f306c10c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:35.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.119+0000 7f3070d27640 1 -- 172.21.15.139:0/3588173379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f306c1963a0 msgr2=0x7f306c196850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:54:35.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.119+0000 7f3070d27640 1 --2- 172.21.15.139:0/3588173379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f306c1963a0 0x7f306c196850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.119+0000 7f3070d27640 1 -- 172.21.15.139:0/3588173379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f306c1008d0 msgr2=0x7f306c10c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:35.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.119+0000 7f3070d27640 1 --2- 172.21.15.139:0/3588173379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f306c1008d0 0x7f306c10c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.119+0000 7f3070d27640 1 -- 172.21.15.139:0/3588173379 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306c196ef0 con 0x7f306c1068d0 2024-09-16T01:54:35.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.127+0000 7f3070d27640 1 --2- 172.21.15.139:0/3588173379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306c1068d0 0x7f306c10cc50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f305c00b4a0 tx=0x7f305c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:35.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.131+0000 7f30627fc640 1 -- 172.21.15.139:0/3588173379 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c014070 con 0x7f306c1068d0 2024-09-16T01:54:35.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.132+0000 7f307252a640 1 -- 172.21.15.139:0/3588173379 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306c1971e0 con 0x7f306c1068d0 2024-09-16T01:54:35.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.132+0000 7f307252a640 1 -- 172.21.15.139:0/3588173379 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f306c06f190 con 0x7f306c1068d0 2024-09-16T01:54:35.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.132+0000 7f30627fc640 1 -- 172.21.15.139:0/3588173379 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305c010af0 con 0x7f306c1068d0 2024-09-16T01:54:35.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.133+0000 7f30627fc640 1 -- 172.21.15.139:0/3588173379 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c017c90 con 0x7f306c1068d0 2024-09-16T01:54:35.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.136+0000 7f3075b65740 1 Processor -- start 2024-09-16T01:54:35.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.136+0000 7f3075b65740 1 -- start start 2024-09-16T01:54:35.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.136+0000 7f3075b65740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.136+0000 7f3075b65740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0ba220 0x55a93d0c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.136+0000 7f3075b65740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0c4ae0 0x55a93d0c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.136+0000 7f3075b65740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a93d01f2a0 con 0x55a93d0ba220 2024-09-16T01:54:35.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.136+0000 7f3075b65740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a93cfc6b10 con 0x55a93d0c4ae0 2024-09-16T01:54:35.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.136+0000 7f3075b65740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a93d01b260 con 0x55a93d0b9900 2024-09-16T01:54:35.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.137+0000 7f30627fc640 1 -- 172.21.15.139:0/3588173379 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f305c017070 con 0x7f306c1068d0 2024-09-16T01:54:35.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.137+0000 7f3071d29640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:35.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.138+0000 7f3071d29640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0b9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:51668/0 (socket says 172.21.15.139:51668) 2024-09-16T01:54:35.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.138+0000 7f3071d29640 1 -- 172.21.15.139:0/865371890 learned_addr learned my addr 172.21.15.139:0/865371890 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:35.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.138+0000 7f30627fc640 1 --2- 172.21.15.139:0/3588173379 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f30580a3420 0x7f30580a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.138+0000 7f3070d27640 1 --2- 172.21.15.139:0/865371890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0c4ae0 0x55a93d0c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:35.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.139+0000 7f30627fc640 1 -- 172.21.15.139:0/3588173379 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f305c0995a0 con 0x7f306c1068d0 2024-09-16T01:54:35.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.140+0000 7f3071528640 1 --2- 172.21.15.139:0/865371890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0ba220 0x55a93d0c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:35.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.140+0000 7f3071d29640 1 -- 172.21.15.139:0/865371890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0c4ae0 msgr2=0x55a93d0c6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:35.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.140+0000 7f3071d29640 1 --2- 172.21.15.139:0/865371890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0c4ae0 0x55a93d0c6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.140+0000 7f3071d29640 1 -- 172.21.15.139:0/865371890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0ba220 msgr2=0x55a93d0c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:35.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.140+0000 7f3071d29640 1 --2- 172.21.15.139:0/865371890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0ba220 0x55a93d0c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.140+0000 7f3071d29640 1 -- 172.21.15.139:0/865371890 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a93cd90b90 con 0x55a93d0b9900 2024-09-16T01:54:35.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.140+0000 7f3071528640 1 --2- 172.21.15.139:0/865371890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0ba220 0x55a93d0c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:54:35.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.142+0000 7f3071d29640 1 --2- 172.21.15.139:0/865371890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0b9ce0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f306401a340 tx=0x7f3064003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd3e2f300622032e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:35.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.142+0000 7f3071d29640 1 --2- 172.21.15.139:0/3588173379 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f30580a3420 0x7f30580a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:35.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.144+0000 7f30577fe640 1 -- 172.21.15.139:0/865371890 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306405b070 con 0x55a93d0b9900 2024-09-16T01:54:35.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.145+0000 7f30577fe640 1 -- 172.21.15.139:0/865371890 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3064003c00 con 0x55a93d0b9900 2024-09-16T01:54:35.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.145+0000 7f3075b65740 1 -- 172.21.15.139:0/865371890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 msgr2=0x55a93d0b9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:35.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.145+0000 7f3075b65740 1 --2- 172.21.15.139:0/865371890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0b9ce0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f306401a340 tx=0x7f3064003970 comp rx=0 tx=0).stop 2024-09-16T01:54:35.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.151+0000 7f3075b65740 1 -- 172.21.15.139:0/865371890 shutdown_connections 2024-09-16T01:54:35.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.151+0000 7f3075b65740 1 --2- 172.21.15.139:0/865371890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0c4ae0 0x55a93d0c6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.151+0000 7f3075b65740 1 --2- 172.21.15.139:0/865371890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0ba220 0x55a93d0c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.151+0000 7f3075b65740 1 --2- 172.21.15.139:0/865371890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0b9ce0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.151+0000 7f3075b65740 1 -- 172.21.15.139:0/865371890 >> 172.21.15.139:0/865371890 conn(0x55a93d0b5e90 msgr2=0x55a93d0b6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:35.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.151+0000 7f3071d29640 1 --2- 172.21.15.139:0/3588173379 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f30580a3420 0x7f30580a58e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f306401a340 tx=0x7f3064003970 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:35.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.151+0000 7f3075b65740 1 -- 172.21.15.139:0/865371890 shutdown_connections 2024-09-16T01:54:35.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.152+0000 7f3075b65740 1 -- 172.21.15.139:0/865371890 wait complete. 2024-09-16T01:54:35.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.153+0000 7f3075b65740 1 -- start start 2024-09-16T01:54:35.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.153+0000 7f3075b65740 1 Processor -- start 2024-09-16T01:54:35.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.153+0000 7f3075b65740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0c18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.153+0000 7f3075b65740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0ba220 0x55a93d0c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.153+0000 7f3075b65740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0c4ae0 0x55a93d0c2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.153+0000 7f3075b65740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a93d0c7700 con 0x55a93d0c4ae0 2024-09-16T01:54:35.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.153+0000 7f3075b65740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a93cfa76e0 con 0x55a93d0ba220 2024-09-16T01:54:35.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.153+0000 7f3075b65740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a93d0c7880 con 0x55a93d0b9900 2024-09-16T01:54:35.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.154+0000 7f3075b65740 10 client.0 _ll_register_callbacks cb 0x55a93cfcba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:54:35.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.154+0000 7f3071d29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0c4ae0 0x55a93d0c2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:35.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.154+0000 7f3071d29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0c4ae0 0x55a93d0c2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53768/0 (socket says 172.21.15.139:53768) 2024-09-16T01:54:35.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.154+0000 7f3071d29640 1 -- 172.21.15.139:0/1015196264 learned_addr learned my addr 172.21.15.139:0/1015196264 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:35.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.154+0000 7f3071d29640 1 -- 172.21.15.139:0/1015196264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 msgr2=0x55a93d0c18f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:54:35.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.154+0000 7f3071d29640 1 --2- 172.21.15.139:0/1015196264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0c18f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.154+0000 7f3071d29640 1 -- 172.21.15.139:0/1015196264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0ba220 msgr2=0x55a93d0c1e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:54:35.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.154+0000 7f3071d29640 1 --2- 172.21.15.139:0/1015196264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0ba220 0x55a93d0c1e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.154+0000 7f3071d29640 1 -- 172.21.15.139:0/1015196264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a93d0d0c20 con 0x55a93d0c4ae0 2024-09-16T01:54:35.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.155+0000 7f3071d29640 1 --2- 172.21.15.139:0/1015196264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0c4ae0 0x55a93d0c2370 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f306403b9b0 tx=0x7f306401a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ee5532968061d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:35.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.160+0000 7f3054ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:54:35.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.160+0000 7f3054ff9640 1 -- 172.21.15.139:0/1015196264 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064010020 con 0x55a93d0c4ae0 2024-09-16T01:54:35.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.160+0000 7f3075b65740 1 -- 172.21.15.139:0/1015196264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3064003c00 con 0x55a93d0c4ae0 2024-09-16T01:54:35.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.160+0000 7f3054ff9640 1 -- 172.21.15.139:0/1015196264 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f3064041a40 con 0x55a93d0c4ae0 2024-09-16T01:54:35.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.164+0000 7f3054ff9640 1 -- 172.21.15.139:0/1015196264 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f306405b070 con 0x55a93d0c4ae0 2024-09-16T01:54:35.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.167+0000 7f3075b65740 10 client.16971 fetch_fsmap learned FSMap version 71 2024-09-16T01:54:35.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.167+0000 7f3075b65740 1 -- 172.21.15.139:0/1015196264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55a93d0c7700 con 0x55a93d0c4ae0 2024-09-16T01:54:35.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.168+0000 7f3054ff9640 1 -- 172.21.15.139:0/1015196264 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3064003c00 con 0x55a93d0c4ae0 2024-09-16T01:54:35.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.168+0000 7f3054ff9640 1 -- 172.21.15.139:0/1015196264 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f3064040b30 con 0x55a93d0c4ae0 2024-09-16T01:54:35.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.168+0000 7f3075b65740 10 client.16971 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:54:35.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.168+0000 7f3075b65740 10 client.16971 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:54:35.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.168+0000 7f3075b65740 20 client.16971 populate_metadata read hostname 'smithi139' 2024-09-16T01:54:35.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.168+0000 7f3075b65740 1 --2- 172.21.15.139:0/1015196264 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a93d0f6650 0x55a93d0f8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.168+0000 7f3075b65740 4 client.16971 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:54:35.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.168+0000 7f3075b65740 1 -- 172.21.15.139:0/1015196264 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a93d02c9d0 con 0x55a93d0f6650 2024-09-16T01:54:35.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.171+0000 7f3071528640 1 --2- 172.21.15.139:0/1015196264 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a93d0f6650 0x55a93d0f8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:35.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.177+0000 7f3071528640 1 --2- 172.21.15.139:0/1015196264 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a93d0f6650 0x55a93d0f8a40 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c55daf5bfdd41f88 server_cookie=5aff8ea913e11bca in_seq=0 out_seq=0 2024-09-16T01:54:35.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.177+0000 7f3054ff9640 10 client.16971 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:35.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.182+0000 7f3054ff9640 1 -- 172.21.15.139:0/1015196264 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a93d02c9d0 con 0x55a93d0f6650 2024-09-16T01:54:35.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.182+0000 7f3054ff9640 10 client.16971 handle_command_reply: tid=0 2024-09-16T01:54:35.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.182+0000 7f3075b65740 1 client.16971 shutdown 2024-09-16T01:54:35.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.189+0000 7f3075b65740 1 -- 172.21.15.139:0/1015196264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0c4ae0 msgr2=0x55a93d0c2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:35.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.189+0000 7f3075b65740 1 --2- 172.21.15.139:0/1015196264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0c4ae0 0x55a93d0c2370 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f306403b9b0 tx=0x7f306401a6e0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.189+0000 7f3075b65740 1 -- 172.21.15.139:0/1015196264 shutdown_connections 2024-09-16T01:54:35.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.189+0000 7f3075b65740 1 --2- 172.21.15.139:0/1015196264 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a93d0f6650 0x55a93d0f8a40 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.189+0000 7f3075b65740 1 --2- 172.21.15.139:0/1015196264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0c4ae0 0x55a93d0c2370 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.189+0000 7f3075b65740 1 --2- 172.21.15.139:0/1015196264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0ba220 0x55a93d0c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.189+0000 7f3075b65740 1 --2- 172.21.15.139:0/1015196264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0c18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.189+0000 7f3075b65740 1 -- 172.21.15.139:0/1015196264 >> 172.21.15.139:0/1015196264 conn(0x55a93d0b5e90 msgr2=0x55a93d0bbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:35.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.189+0000 7f3054ff9640 0 client.16971 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:35.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.190+0000 7f3075b65740 1 -- 172.21.15.139:0/1015196264 shutdown_connections 2024-09-16T01:54:35.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.190+0000 7f3075b65740 1 -- 172.21.15.139:0/1015196264 wait complete. 2024-09-16T01:54:35.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.190+0000 7f3075b65740 20 client.16971 trim_cache size 0 max 16384 2024-09-16T01:54:35.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.226+0000 7f3075b65740 1 Processor -- start 2024-09-16T01:54:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.226+0000 7f3075b65740 1 -- start start 2024-09-16T01:54:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.226+0000 7f3075b65740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0bc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.226+0000 7f3075b65740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0ba220 0x55a93d0c0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.227+0000 7f3075b65740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0c4ae0 0x55a93d0fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.227+0000 7f3071528640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0bc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.227+0000 7f3071528640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0bc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:51684/0 (socket says 172.21.15.139:51684) 2024-09-16T01:54:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.227+0000 7f3071528640 1 -- 172.21.15.139:0/2948253248 learned_addr learned my addr 172.21.15.139:0/2948253248 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.227+0000 7f3075b65740 1 -- 172.21.15.139:0/2948253248 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3050010260 con 0x55a93d0ba220 2024-09-16T01:54:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.227+0000 7f3075b65740 1 -- 172.21.15.139:0/2948253248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f30500120a0 con 0x55a93d0c4ae0 2024-09-16T01:54:35.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.227+0000 7f3075b65740 1 -- 172.21.15.139:0/2948253248 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f30500109f0 con 0x55a93d0b9900 2024-09-16T01:54:35.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.227+0000 7f3071528640 1 -- 172.21.15.139:0/2948253248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0c4ae0 msgr2=0x55a93d0fe2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:54:35.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.227+0000 7f3071528640 1 --2- 172.21.15.139:0/2948253248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0c4ae0 0x55a93d0fe2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.227+0000 7f3071528640 1 -- 172.21.15.139:0/2948253248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0ba220 msgr2=0x55a93d0c0830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:54:35.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.227+0000 7f3071528640 1 --2- 172.21.15.139:0/2948253248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0ba220 0x55a93d0c0830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.228+0000 7f3071528640 1 -- 172.21.15.139:0/2948253248 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a93d0f8db0 con 0x55a93d0b9900 2024-09-16T01:54:35.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.229+0000 7f3071528640 1 --2- 172.21.15.139:0/2948253248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0bc2d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3068004720 tx=0x7f30680090e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc06addb04e8c47c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:35.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.230+0000 7f3056ffd640 1 -- 172.21.15.139:0/2948253248 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306800f040 con 0x55a93d0b9900 2024-09-16T01:54:35.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.230+0000 7f3056ffd640 1 -- 172.21.15.139:0/2948253248 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3068023070 con 0x55a93d0b9900 2024-09-16T01:54:35.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.232+0000 7f3075b65740 1 -- 172.21.15.139:0/2948253248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 msgr2=0x55a93d0bc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:35.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.232+0000 7f3075b65740 1 --2- 172.21.15.139:0/2948253248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0bc2d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3068004720 tx=0x7f30680090e0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.232+0000 7f3075b65740 1 -- 172.21.15.139:0/2948253248 shutdown_connections 2024-09-16T01:54:35.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.232+0000 7f3075b65740 1 --2- 172.21.15.139:0/2948253248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0c4ae0 0x55a93d0fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.232+0000 7f3075b65740 1 --2- 172.21.15.139:0/2948253248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0ba220 0x55a93d0c0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.232+0000 7f3075b65740 1 --2- 172.21.15.139:0/2948253248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0b9900 0x55a93d0bc2d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.232+0000 7f3075b65740 1 -- 172.21.15.139:0/2948253248 >> 172.21.15.139:0/2948253248 conn(0x55a93d0b5e90 msgr2=0x55a93d0c3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:35.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.233+0000 7f3075b65740 1 -- 172.21.15.139:0/2948253248 shutdown_connections 2024-09-16T01:54:35.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.233+0000 7f3075b65740 1 -- 172.21.15.139:0/2948253248 wait complete. 2024-09-16T01:54:35.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.234+0000 7f3075b65740 1 -- start start 2024-09-16T01:54:35.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.234+0000 7f3075b65740 1 Processor -- start 2024-09-16T01:54:35.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.234+0000 7f3075b65740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0ba220 0x55a93d0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.234+0000 7f3075b65740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0c4ae0 0x55a93d0cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.234+0000 7f3075b65740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0cb310 0x55a93d0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.234+0000 7f3075b65740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3050011910 con 0x55a93d0c4ae0 2024-09-16T01:54:35.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.234+0000 7f3075b65740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3050011180 con 0x55a93d0cb310 2024-09-16T01:54:35.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.234+0000 7f3075b65740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f305000e240 con 0x55a93d0ba220 2024-09-16T01:54:35.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.235+0000 7f3075b65740 10 client.0 _ll_register_callbacks cb 0x7f3064005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:54:35.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.237+0000 7f3071d29640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0cb310 0x55a93d0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.237+0000 7f3071528640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0ba220 0x55a93d0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.237+0000 7f3070d27640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0c4ae0 0x55a93d0cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.237+0000 7f3070d27640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0c4ae0 0x55a93d0cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53778/0 (socket says 172.21.15.139:53778) 2024-09-16T01:54:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.237+0000 7f3070d27640 1 -- 172.21.15.139:0/3938401844 learned_addr learned my addr 172.21.15.139:0/3938401844 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:35.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.239+0000 7f3071d29640 1 -- 172.21.15.139:0/3938401844 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0ba220 msgr2=0x55a93d0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:35.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.239+0000 7f3071d29640 1 --2- 172.21.15.139:0/3938401844 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0ba220 0x55a93d0cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.239+0000 7f3071d29640 1 -- 172.21.15.139:0/3938401844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0c4ae0 msgr2=0x55a93d0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:35.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.239+0000 7f3071d29640 1 --2- 172.21.15.139:0/3938401844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0c4ae0 0x55a93d0cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:35.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.239+0000 7f3071d29640 1 -- 172.21.15.139:0/3938401844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a93d0be650 con 0x55a93d0cb310 2024-09-16T01:54:35.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.240+0000 7f3071d29640 1 --2- 172.21.15.139:0/3938401844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0cb310 0x55a93d0cb790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f306401aa70 tx=0x7f3064005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50319a09902abc1a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:35.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.240+0000 7f3054ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:54:35.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.241+0000 7f3054ff9640 1 -- 172.21.15.139:0/3938401844 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064010070 con 0x55a93d0cb310 2024-09-16T01:54:35.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.241+0000 7f3054ff9640 1 -- 172.21.15.139:0/3938401844 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f3064041840 con 0x55a93d0cb310 2024-09-16T01:54:35.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.242+0000 7f3075b65740 1 -- 172.21.15.139:0/3938401844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a93d026d20 con 0x55a93d0cb310 2024-09-16T01:54:35.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.243+0000 7f3054ff9640 1 -- 172.21.15.139:0/3938401844 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f3064041a40 con 0x55a93d0cb310 2024-09-16T01:54:35.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.244+0000 7f3075b65740 10 client.26417 fetch_fsmap learned FSMap version 71 2024-09-16T01:54:35.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.244+0000 7f3075b65740 1 -- 172.21.15.139:0/3938401844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55a93d1726d0 con 0x55a93d0cb310 2024-09-16T01:54:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.245+0000 7f3054ff9640 1 -- 172.21.15.139:0/3938401844 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a93d026d20 con 0x55a93d0cb310 2024-09-16T01:54:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.246+0000 7f3054ff9640 1 -- 172.21.15.139:0/3938401844 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f30640409c0 con 0x55a93d0cb310 2024-09-16T01:54:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.247+0000 7f3075b65740 10 client.26417 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:54:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.247+0000 7f3075b65740 10 client.26417 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:54:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.247+0000 7f3075b65740 20 client.26417 populate_metadata read hostname 'smithi139' 2024-09-16T01:54:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.247+0000 7f3075b65740 1 --2- 172.21.15.139:0/3938401844 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a93d172f20 0x55a93d175340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.247+0000 7f3075b65740 4 client.26417 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.5, "await_for": 45.0}] 2024-09-16T01:54:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.247+0000 7f3075b65740 1 -- 172.21.15.139:0/3938401844 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.5, "await_for": 45.0}) -- 0x55a93cfa2ec0 con 0x55a93d172f20 2024-09-16T01:54:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.247+0000 7f3071528640 1 --2- 172.21.15.139:0/3938401844 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a93d172f20 0x55a93d175340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:35.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.248+0000 7f3071528640 1 --2- 172.21.15.139:0/3938401844 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a93d172f20 0x55a93d175340 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afdc9feebba00805 server_cookie=777a1230009994ac in_seq=0 out_seq=0 2024-09-16T01:54:35.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:35.248+0000 7f3054ff9640 10 client.26417 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:35.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:35 smithi139 ceph-mon[31175]: pgmap v448: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 936 KiB/s wr, 176 op/s 2024-09-16T01:54:35.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:35 smithi176 ceph-mon[35324]: pgmap v448: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 936 KiB/s wr, 176 op/s 2024-09-16T01:54:35.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:35 smithi164 ceph-mon[35121]: pgmap v448: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 936 KiB/s wr, 176 op/s 2024-09-16T01:54:37.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:37 smithi139 ceph-mon[31175]: pgmap v449: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-09-16T01:54:37.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:37 smithi176 ceph-mon[35324]: pgmap v449: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-09-16T01:54:37.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:37 smithi164 ceph-mon[35121]: pgmap v449: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-09-16T01:54:38.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.576+0000 7f3054ff9640 1 -- 172.21.15.139:0/3938401844 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a93cfa2ec0 con 0x55a93d172f20 2024-09-16T01:54:38.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.577+0000 7f3054ff9640 10 client.26417 handle_command_reply: tid=0 2024-09-16T01:54:38.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.577+0000 7f3075b65740 1 client.26417 shutdown 2024-09-16T01:54:38.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.577+0000 7f3075b65740 1 -- 172.21.15.139:0/3938401844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0cb310 msgr2=0x55a93d0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:38.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.577+0000 7f3075b65740 1 --2- 172.21.15.139:0/3938401844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0cb310 0x55a93d0cb790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f306401aa70 tx=0x7f3064005fa0 comp rx=0 tx=0).stop 2024-09-16T01:54:38.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.577+0000 7f3075b65740 1 -- 172.21.15.139:0/3938401844 shutdown_connections 2024-09-16T01:54:38.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.577+0000 7f3075b65740 1 --2- 172.21.15.139:0/3938401844 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55a93d172f20 0x55a93d175340 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:38.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.577+0000 7f3075b65740 1 --2- 172.21.15.139:0/3938401844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a93d0cb310 0x55a93d0cb790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:38.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.577+0000 7f3075b65740 1 --2- 172.21.15.139:0/3938401844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a93d0c4ae0 0x55a93d0cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:38.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.577+0000 7f3075b65740 1 --2- 172.21.15.139:0/3938401844 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93d0ba220 0x55a93d0cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:38.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.577+0000 7f3075b65740 1 -- 172.21.15.139:0/3938401844 >> 172.21.15.139:0/3938401844 conn(0x55a93d0b5e90 msgr2=0x55a93d0b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:38.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.577+0000 7f3054ff9640 0 client.26417 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:38.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.577+0000 7f3075b65740 1 -- 172.21.15.139:0/3938401844 shutdown_connections 2024-09-16T01:54:38.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.577+0000 7f3075b65740 1 -- 172.21.15.139:0/3938401844 wait complete. 2024-09-16T01:54:38.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.578+0000 7f3075b65740 20 client.26417 trim_cache size 0 max 16384 2024-09-16T01:54:38.581 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:54:38.581 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 71, 2024-09-16T01:54:38.581 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:54:38.581 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 50, 2024-09-16T01:54:38.581 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:54:38.581 INFO:teuthology.orchestra.run.smithi139.stdout: "724cf2d2": { 2024-09-16T01:54:38.581 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 50, 2024-09-16T01:54:38.581 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:54:38.581 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:54:38.582 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:54:38.582 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:54:38.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:54:38.582 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:54:38.582 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 146.5, 2024-09-16T01:54:38.582 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:54:38.582 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:54:38.582 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:54:38.582 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:54:38.582 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:54:38.582 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:54:38.582 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:54:38.582 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:54:38.582 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:54:38.583 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:54:38.583 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:54:38.583 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:54:38.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.580+0000 7f307252a640 1 -- 172.21.15.139:0/3588173379 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f30580a3420 msgr2=0x7f30580a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:38.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.580+0000 7f307252a640 1 --2- 172.21.15.139:0/3588173379 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f30580a3420 0x7f30580a58e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f306401a340 tx=0x7f3064003970 comp rx=0 tx=0).stop 2024-09-16T01:54:38.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.580+0000 7f307252a640 1 -- 172.21.15.139:0/3588173379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306c1068d0 msgr2=0x7f306c10cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:38.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.580+0000 7f307252a640 1 --2- 172.21.15.139:0/3588173379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306c1068d0 0x7f306c10cc50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f305c00b4a0 tx=0x7f305c00b960 comp rx=0 tx=0).stop 2024-09-16T01:54:38.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.581+0000 7f307252a640 1 -- 172.21.15.139:0/3588173379 shutdown_connections 2024-09-16T01:54:38.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.581+0000 7f307252a640 1 --2- 172.21.15.139:0/3588173379 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f30580a3420 0x7f30580a58e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:38.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.581+0000 7f307252a640 1 --2- 172.21.15.139:0/3588173379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f306c1963a0 0x7f306c196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:38.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.581+0000 7f307252a640 1 --2- 172.21.15.139:0/3588173379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306c1068d0 0x7f306c10cc50 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:38.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.581+0000 7f307252a640 1 --2- 172.21.15.139:0/3588173379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f306c1008d0 0x7f306c10c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:38.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.581+0000 7f307252a640 1 -- 172.21.15.139:0/3588173379 >> 172.21.15.139:0/3588173379 conn(0x7f306c0fc610 msgr2=0x7f306c06a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:38.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.582+0000 7f307252a640 1 -- 172.21.15.139:0/3588173379 shutdown_connections 2024-09-16T01:54:38.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:54:38.584+0000 7f307252a640 1 -- 172.21.15.139:0/3588173379 wait complete. 2024-09-16T01:54:39.568 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '724cf2d2', quiesce took 9.6 seconds. Will release after: 16.9 2024-09-16T01:54:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:39 smithi139 ceph-mon[31175]: pgmap v450: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 792 KiB/s wr, 179 op/s 2024-09-16T01:54:39.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:39 smithi176 ceph-mon[35324]: pgmap v450: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 792 KiB/s wr, 179 op/s 2024-09-16T01:54:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:39 smithi164 ceph-mon[35121]: pgmap v450: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 792 KiB/s wr, 179 op/s 2024-09-16T01:54:41.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:41 smithi139 ceph-mon[31175]: pgmap v451: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 985 KiB/s wr, 179 op/s 2024-09-16T01:54:41.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:41 smithi176 ceph-mon[35324]: pgmap v451: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 985 KiB/s wr, 179 op/s 2024-09-16T01:54:41.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:41 smithi164 ceph-mon[35121]: pgmap v451: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 985 KiB/s wr, 179 op/s 2024-09-16T01:54:43.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:43 smithi176 ceph-mon[35324]: pgmap v452: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 830 KiB/s wr, 174 op/s 2024-09-16T01:54:43.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:43 smithi164 ceph-mon[35121]: pgmap v452: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 830 KiB/s wr, 174 op/s 2024-09-16T01:54:43.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:43 smithi139 ceph-mon[31175]: pgmap v452: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 830 KiB/s wr, 174 op/s 2024-09-16T01:54:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.282+0000 7f0eb2a1e640 1 -- 172.21.15.139:0/3929228999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0eac1013b0 msgr2=0x7f0eac105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.282+0000 7f0eb2a1e640 1 --2- 172.21.15.139:0/3929228999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0eac1013b0 0x7f0eac105870 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009930 tx=0x7f0ea002a480 comp rx=0 tx=0).stop 2024-09-16T01:54:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.290+0000 7f0eb2a1e640 1 -- 172.21.15.139:0/3929228999 shutdown_connections 2024-09-16T01:54:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.290+0000 7f0eb2a1e640 1 --2- 172.21.15.139:0/3929228999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0eac105f80 0x7f0eac106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.290+0000 7f0eb2a1e640 1 --2- 172.21.15.139:0/3929228999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0eac1013b0 0x7f0eac105870 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.290+0000 7f0eb2a1e640 1 --2- 172.21.15.139:0/3929228999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eac100a00 0x7f0eac100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.290+0000 7f0eb2a1e640 1 -- 172.21.15.139:0/3929228999 >> 172.21.15.139:0/3929228999 conn(0x7f0eac0fc6f0 msgr2=0x7f0eac0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.297+0000 7f0eb2a1e640 1 -- 172.21.15.139:0/3929228999 shutdown_connections 2024-09-16T01:54:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.298+0000 7f0eb2a1e640 1 -- 172.21.15.139:0/3929228999 wait complete. 2024-09-16T01:54:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.299+0000 7f0eb2a1e640 1 Processor -- start 2024-09-16T01:54:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.303+0000 7f0eb2a1e640 1 -- start start 2024-09-16T01:54:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.303+0000 7f0eb2a1e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0eac100a00 0x7f0eac10a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.303+0000 7f0eb2a1e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0eac1013b0 0x7f0eac10a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.303+0000 7f0eb2a1e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eac105f80 0x7f0eac072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.303+0000 7f0eb2a1e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1184d0 con 0x7f0eac1013b0 2024-09-16T01:54:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.303+0000 7f0eb2a1e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac118350 con 0x7f0eac105f80 2024-09-16T01:54:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.303+0000 7f0eb2a1e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac118650 con 0x7f0eac100a00 2024-09-16T01:54:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.303+0000 7f0eabfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0eac100a00 0x7f0eac10a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.303+0000 7f0eabfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0eac100a00 0x7f0eac10a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:32890/0 (socket says 172.21.15.139:32890) 2024-09-16T01:54:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.303+0000 7f0eabfff640 1 -- 172.21.15.139:0/3126316041 learned_addr learned my addr 172.21.15.139:0/3126316041 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.304+0000 7f0eabfff640 1 -- 172.21.15.139:0/3126316041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eac105f80 msgr2=0x7f0eac072ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:54:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.304+0000 7f0eabfff640 1 --2- 172.21.15.139:0/3126316041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eac105f80 0x7f0eac072ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.304+0000 7f0eabfff640 1 -- 172.21.15.139:0/3126316041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0eac1013b0 msgr2=0x7f0eac10a7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:54:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.304+0000 7f0eabfff640 1 --2- 172.21.15.139:0/3126316041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0eac1013b0 0x7f0eac10a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.304+0000 7f0eabfff640 1 -- 172.21.15.139:0/3126316041 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac0732e0 con 0x7f0eac100a00 2024-09-16T01:54:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.304+0000 7f0eabfff640 1 --2- 172.21.15.139:0/3126316041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0eac100a00 0x7f0eac10a2a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00c890 tx=0x7f0e9c00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.305+0000 7f0ea97fa640 1 -- 172.21.15.139:0/3126316041 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c004220 con 0x7f0eac100a00 2024-09-16T01:54:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.305+0000 7f0ea97fa640 1 -- 172.21.15.139:0/3126316041 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c0043c0 con 0x7f0eac100a00 2024-09-16T01:54:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.305+0000 7f0ea97fa640 1 -- 172.21.15.139:0/3126316041 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0049e0 con 0x7f0eac100a00 2024-09-16T01:54:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.309+0000 7f0eb2a1e640 1 -- 172.21.15.139:0/3126316041 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eac0735d0 con 0x7f0eac100a00 2024-09-16T01:54:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.309+0000 7f0eb2a1e640 1 -- 172.21.15.139:0/3126316041 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eac0738f0 con 0x7f0eac100a00 2024-09-16T01:54:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.313+0000 7f0eb5890740 1 Processor -- start 2024-09-16T01:54:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.315+0000 7f0eb5890740 1 -- start start 2024-09-16T01:54:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.315+0000 7f0eb5890740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 0x560d9867bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.315+0000 7f0eb5890740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 0x560d98686770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.315+0000 7f0eb5890740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d98686cb0 0x560d986890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.315+0000 7f0eb5890740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560d985e02f0 con 0x560d9867c410 2024-09-16T01:54:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.315+0000 7f0eb5890740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560d98588270 con 0x560d9867baf0 2024-09-16T01:54:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.315+0000 7f0eb5890740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560d985dc2b0 con 0x560d98686cb0 2024-09-16T01:54:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.315+0000 7f0eabfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 0x560d98686770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.315+0000 7f0eabfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 0x560d98686770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56702/0 (socket says 172.21.15.139:56702) 2024-09-16T01:54:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.315+0000 7f0eabfff640 1 -- 172.21.15.139:0/345034438 learned_addr learned my addr 172.21.15.139:0/345034438 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.316+0000 7f0eb0fbb640 1 --2- 172.21.15.139:0/345034438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 0x560d9867bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.317+0000 7f0eabfff640 1 -- 172.21.15.139:0/345034438 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d98686cb0 msgr2=0x560d986890a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.317+0000 7f0eabfff640 1 --2- 172.21.15.139:0/345034438 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d98686cb0 0x560d986890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.317+0000 7f0eabfff640 1 -- 172.21.15.139:0/345034438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 msgr2=0x560d9867bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.317+0000 7f0eabfff640 1 --2- 172.21.15.139:0/345034438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 0x560d9867bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.317+0000 7f0eabfff640 1 -- 172.21.15.139:0/345034438 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d98587340 con 0x560d9867c410 2024-09-16T01:54:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.317+0000 7f0eabfff640 1 --2- 172.21.15.139:0/345034438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 0x560d98686770 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0cd070 tx=0x7f0e9c0d8640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a18b2be4e9c99fe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.318+0000 7f0ea97fa640 1 -- 172.21.15.139:0/3126316041 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0e9c004b80 con 0x7f0eac100a00 2024-09-16T01:54:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.318+0000 7f0e927fc640 1 -- 172.21.15.139:0/345034438 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0e3070 con 0x560d9867c410 2024-09-16T01:54:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.318+0000 7f0e927fc640 1 -- 172.21.15.139:0/345034438 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c0dcbf0 con 0x560d9867c410 2024-09-16T01:54:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.318+0000 7f0e927fc640 1 -- 172.21.15.139:0/345034438 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0deb90 con 0x560d9867c410 2024-09-16T01:54:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.319+0000 7f0ea97fa640 1 --2- 172.21.15.139:0/3126316041 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e940a3690 0x7f0e940a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.319+0000 7f0eb5890740 1 -- 172.21.15.139:0/345034438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 msgr2=0x560d98686770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.319+0000 7f0eb5890740 1 --2- 172.21.15.139:0/345034438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 0x560d98686770 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0cd070 tx=0x7f0e9c0d8640 comp rx=0 tx=0).stop 2024-09-16T01:54:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.319+0000 7f0ea97fa640 1 -- 172.21.15.139:0/3126316041 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f0e9c0cfe90 con 0x7f0eac100a00 2024-09-16T01:54:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.319+0000 7f0eb5890740 1 -- 172.21.15.139:0/345034438 shutdown_connections 2024-09-16T01:54:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.319+0000 7f0eb5890740 1 --2- 172.21.15.139:0/345034438 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d98686cb0 0x560d986890a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.319+0000 7f0eb5890740 1 --2- 172.21.15.139:0/345034438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 0x560d98686770 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.319+0000 7f0eb5890740 1 --2- 172.21.15.139:0/345034438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 0x560d9867bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.319+0000 7f0eb5890740 1 -- 172.21.15.139:0/345034438 >> 172.21.15.139:0/345034438 conn(0x560d98678060 msgr2=0x560d98678440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.320+0000 7f0eb5890740 1 -- 172.21.15.139:0/345034438 shutdown_connections 2024-09-16T01:54:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.323+0000 7f0eb5890740 1 -- 172.21.15.139:0/345034438 wait complete. 2024-09-16T01:54:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.323+0000 7f0eb5890740 1 -- start start 2024-09-16T01:54:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.323+0000 7f0eb5890740 1 Processor -- start 2024-09-16T01:54:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.323+0000 7f0eb0fbb640 1 --2- 172.21.15.139:0/3126316041 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e940a3690 0x7f0e940a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.324+0000 7f0eb5890740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 0x560d98683c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.324+0000 7f0eb5890740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 0x560d98684140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.324+0000 7f0eb5890740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d98686cb0 0x560d98684680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.324+0000 7f0eb5890740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560d98689880 con 0x560d9867c410 2024-09-16T01:54:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.324+0000 7f0eb5890740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560d98569290 con 0x560d9867baf0 2024-09-16T01:54:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.324+0000 7f0eb5890740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560d98689a00 con 0x560d98686cb0 2024-09-16T01:54:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.324+0000 7f0eb5890740 10 client.0 _ll_register_callbacks cb 0x560d9859a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:54:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.324+0000 7f0eb0fbb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d98686cb0 0x560d98684680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.324+0000 7f0eb0fbb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d98686cb0 0x560d98684680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:32916/0 (socket says 172.21.15.139:32916) 2024-09-16T01:54:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.324+0000 7f0eb0fbb640 1 -- 172.21.15.139:0/343039654 learned_addr learned my addr 172.21.15.139:0/343039654 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.325+0000 7f0eb0fbb640 1 -- 172.21.15.139:0/343039654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 msgr2=0x560d98683c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.325+0000 7f0eb0fbb640 1 --2- 172.21.15.139:0/343039654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 0x560d98683c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.325+0000 7f0eb0fbb640 1 -- 172.21.15.139:0/343039654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 msgr2=0x560d98684140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.325+0000 7f0eb0fbb640 1 --2- 172.21.15.139:0/343039654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 0x560d98684140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.325+0000 7f0eb0fbb640 1 -- 172.21.15.139:0/343039654 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d982f3210 con 0x560d98686cb0 2024-09-16T01:54:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.325+0000 7f0eb0fbb640 1 --2- 172.21.15.139:0/343039654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d98686cb0 0x560d98684680 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400b570 tx=0x7f0ea4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d30955a0c7a4452 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.326+0000 7f0e8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:54:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.330+0000 7f0eb0fbb640 1 --2- 172.21.15.139:0/3126316041 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e940a3690 0x7f0e940a5b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40128e0 tx=0x7f0ea4008bd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.332+0000 7f0e8b7fe640 1 -- 172.21.15.139:0/343039654 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea400be30 con 0x560d98686cb0 2024-09-16T01:54:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.332+0000 7f0eb5890740 1 -- 172.21.15.139:0/343039654 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9c0dcbf0 con 0x560d98686cb0 2024-09-16T01:54:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.332+0000 7f0e8b7fe640 1 -- 172.21.15.139:0/343039654 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f0ea4004b20 con 0x560d98686cb0 2024-09-16T01:54:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.333+0000 7f0e8b7fe640 1 -- 172.21.15.139:0/343039654 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f0ea4004d20 con 0x560d98686cb0 2024-09-16T01:54:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.336+0000 7f0eb5890740 10 client.26392 fetch_fsmap learned FSMap version 71 2024-09-16T01:54:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.336+0000 7f0eb5890740 1 -- 172.21.15.139:0/343039654 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x560d983bf620 con 0x560d98686cb0 2024-09-16T01:54:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.341+0000 7f0e8b7fe640 1 -- 172.21.15.139:0/343039654 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c0dcbf0 con 0x560d98686cb0 2024-09-16T01:54:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.342+0000 7f0e8b7fe640 1 -- 172.21.15.139:0/343039654 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f0ea4005370 con 0x560d98686cb0 2024-09-16T01:54:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.344+0000 7f0eb5890740 10 client.26392 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:54:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.344+0000 7f0eb5890740 10 client.26392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:54:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.344+0000 7f0eb5890740 20 client.26392 populate_metadata read hostname 'smithi139' 2024-09-16T01:54:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.344+0000 7f0eb5890740 1 --2- 172.21.15.139:0/343039654 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560d986b8ea0 0x560d986bb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.344+0000 7f0eb5890740 4 client.26392 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:54:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.344+0000 7f0eb5890740 1 -- 172.21.15.139:0/343039654 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d985eda20 con 0x560d986b8ea0 2024-09-16T01:54:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.344+0000 7f0eabfff640 1 --2- 172.21.15.139:0/343039654 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560d986b8ea0 0x560d986bb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.346+0000 7f0eabfff640 1 --2- 172.21.15.139:0/343039654 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560d986b8ea0 0x560d986bb360 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19b135c61bbe7ae3 server_cookie=5713da4ed474e7c1 in_seq=0 out_seq=0 2024-09-16T01:54:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.346+0000 7f0e8b7fe640 10 client.26392 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.349+0000 7f0e8b7fe640 1 -- 172.21.15.139:0/343039654 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d985eda20 con 0x560d986b8ea0 2024-09-16T01:54:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.349+0000 7f0e8b7fe640 10 client.26392 handle_command_reply: tid=0 2024-09-16T01:54:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.350+0000 7f0eb5890740 1 client.26392 shutdown 2024-09-16T01:54:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.353+0000 7f0eb5890740 1 -- 172.21.15.139:0/343039654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d98686cb0 msgr2=0x560d98684680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.353+0000 7f0eb5890740 1 --2- 172.21.15.139:0/343039654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d98686cb0 0x560d98684680 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400b570 tx=0x7f0ea4004a40 comp rx=0 tx=0).stop 2024-09-16T01:54:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.354+0000 7f0eb5890740 1 -- 172.21.15.139:0/343039654 shutdown_connections 2024-09-16T01:54:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.354+0000 7f0eb5890740 1 --2- 172.21.15.139:0/343039654 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560d986b8ea0 0x560d986bb360 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.354+0000 7f0eb5890740 1 --2- 172.21.15.139:0/343039654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d98686cb0 0x560d98684680 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.354+0000 7f0eb5890740 1 --2- 172.21.15.139:0/343039654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 0x560d98684140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.354+0000 7f0eb5890740 1 --2- 172.21.15.139:0/343039654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 0x560d98683c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.354+0000 7f0eb5890740 1 -- 172.21.15.139:0/343039654 >> 172.21.15.139:0/343039654 conn(0x560d98678060 msgr2=0x560d9867ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.354+0000 7f0e8b7fe640 0 client.26392 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.358+0000 7f0eb5890740 1 -- 172.21.15.139:0/343039654 shutdown_connections 2024-09-16T01:54:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.358+0000 7f0eb5890740 1 -- 172.21.15.139:0/343039654 wait complete. 2024-09-16T01:54:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.359+0000 7f0eb5890740 20 client.26392 trim_cache size 0 max 16384 2024-09-16T01:54:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.399+0000 7f0eb5890740 1 Processor -- start 2024-09-16T01:54:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.399+0000 7f0eb5890740 1 -- start start 2024-09-16T01:54:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.399+0000 7f0eb5890740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d9867baf0 0x560d986844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.399+0000 7f0eb5890740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 0x560d986c0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.399+0000 7f0eb5890740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d98686cb0 0x560d986c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.399+0000 7f0eb5890740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98010090 con 0x560d9867c410 2024-09-16T01:54:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.400+0000 7f0eb0fbb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d98686cb0 0x560d986c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.400+0000 7f0eb0fbb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d98686cb0 0x560d986c11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:58794/0 (socket says 172.21.15.139:58794) 2024-09-16T01:54:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.400+0000 7f0eb0fbb640 1 -- 172.21.15.139:0/3190934828 learned_addr learned my addr 172.21.15.139:0/3190934828 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.402+0000 7f0eabfff640 1 --2- 172.21.15.139:0/3190934828 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d9867baf0 0x560d986844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.404+0000 7f0eb5890740 1 -- 172.21.15.139:0/3190934828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98011ed0 con 0x560d98686cb0 2024-09-16T01:54:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.404+0000 7f0eab7fe640 1 --2- 172.21.15.139:0/3190934828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 0x560d986c0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.404+0000 7f0eb5890740 1 -- 172.21.15.139:0/3190934828 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98010820 con 0x560d9867baf0 2024-09-16T01:54:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.408+0000 7f0eb0fbb640 1 -- 172.21.15.139:0/3190934828 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d9867baf0 msgr2=0x560d986844b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.408+0000 7f0eb0fbb640 1 --2- 172.21.15.139:0/3190934828 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d9867baf0 0x560d986844b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.408+0000 7f0eb0fbb640 1 -- 172.21.15.139:0/3190934828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 msgr2=0x560d986c0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.408+0000 7f0eb0fbb640 1 --2- 172.21.15.139:0/3190934828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 0x560d986c0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.408+0000 7f0eb0fbb640 1 -- 172.21.15.139:0/3190934828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d9867b920 con 0x560d98686cb0 2024-09-16T01:54:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.410+0000 7f0eb0fbb640 1 --2- 172.21.15.139:0/3190934828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d98686cb0 0x560d986c11b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4008260 tx=0x7f0ea40086d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4bd37909223c7da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.412+0000 7f0e91ffb640 1 -- 172.21.15.139:0/3190934828 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4033020 con 0x560d98686cb0 2024-09-16T01:54:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.412+0000 7f0e91ffb640 1 -- 172.21.15.139:0/3190934828 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea4005a20 con 0x560d98686cb0 2024-09-16T01:54:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.412+0000 7f0e91ffb640 1 -- 172.21.15.139:0/3190934828 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea401e870 con 0x560d98686cb0 2024-09-16T01:54:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.414+0000 7f0eb5890740 1 -- 172.21.15.139:0/3190934828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d98686cb0 msgr2=0x560d986c11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.414+0000 7f0eb5890740 1 --2- 172.21.15.139:0/3190934828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d98686cb0 0x560d986c11b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4008260 tx=0x7f0ea40086d0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.417+0000 7f0eb5890740 1 -- 172.21.15.139:0/3190934828 shutdown_connections 2024-09-16T01:54:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.417+0000 7f0eb5890740 1 --2- 172.21.15.139:0/3190934828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d98686cb0 0x560d986c11b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.417+0000 7f0eb5890740 1 --2- 172.21.15.139:0/3190934828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9867c410 0x560d986c0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.417+0000 7f0eb5890740 1 --2- 172.21.15.139:0/3190934828 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d9867baf0 0x560d986844b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.417+0000 7f0eb5890740 1 -- 172.21.15.139:0/3190934828 >> 172.21.15.139:0/3190934828 conn(0x560d98678060 msgr2=0x560d98685c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.417+0000 7f0eb5890740 1 -- 172.21.15.139:0/3190934828 shutdown_connections 2024-09-16T01:54:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.417+0000 7f0eb5890740 1 -- 172.21.15.139:0/3190934828 wait complete. 2024-09-16T01:54:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.418+0000 7f0eb5890740 1 -- start start 2024-09-16T01:54:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.418+0000 7f0eb5890740 1 Processor -- start 2024-09-16T01:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.419+0000 7f0eb5890740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 0x560d9868f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.419+0000 7f0eb5890740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d9867c410 0x560d9868a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.419+0000 7f0eb5890740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d98686cb0 0x560d9868ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.419+0000 7f0eabfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 0x560d9868f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.419+0000 7f0eb5890740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98011740 con 0x560d98686cb0 2024-09-16T01:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.419+0000 7f0eb5890740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98010fb0 con 0x560d9867baf0 2024-09-16T01:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.419+0000 7f0eb5890740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9800e120 con 0x560d9867c410 2024-09-16T01:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.419+0000 7f0eabfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 0x560d9868f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:58798/0 (socket says 172.21.15.139:58798) 2024-09-16T01:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.419+0000 7f0eb5890740 10 client.0 _ll_register_callbacks cb 0x7f0ea4008a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.419+0000 7f0eabfff640 1 -- 172.21.15.139:0/1025495695 learned_addr learned my addr 172.21.15.139:0/1025495695 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.419+0000 7f0eab7fe640 1 --2- 172.21.15.139:0/1025495695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d9867c410 0x560d9868a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.421+0000 7f0eab7fe640 1 -- 172.21.15.139:0/1025495695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 msgr2=0x560d9868f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.421+0000 7f0eab7fe640 1 --2- 172.21.15.139:0/1025495695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 0x560d9868f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.421+0000 7f0eab7fe640 1 -- 172.21.15.139:0/1025495695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d98686cb0 msgr2=0x560d9868ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.421+0000 7f0eab7fe640 1 --2- 172.21.15.139:0/1025495695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d98686cb0 0x560d9868ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.421+0000 7f0eab7fe640 1 -- 172.21.15.139:0/1025495695 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d9868b450 con 0x560d9867c410 2024-09-16T01:54:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.424+0000 7f0eab7fe640 1 --2- 172.21.15.139:0/1025495695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d9867c410 0x560d9868a6e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x560d9867e690 tx=0x7f0ea0004340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa5a41400d7e6f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.425+0000 7f0e8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:54:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.425+0000 7f0e8b7fe640 1 -- 172.21.15.139:0/1025495695 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0035040 con 0x560d9867c410 2024-09-16T01:54:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.426+0000 7f0e8b7fe640 1 -- 172.21.15.139:0/1025495695 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f0ea0005220 con 0x560d9867c410 2024-09-16T01:54:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.426+0000 7f0eb5890740 1 -- 172.21.15.139:0/1025495695 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea4005a20 con 0x560d9867c410 2024-09-16T01:54:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.431+0000 7f0e8b7fe640 1 -- 172.21.15.139:0/1025495695 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f0ea0049050 con 0x560d9867c410 2024-09-16T01:54:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.431+0000 7f0eb5890740 10 client.26398 fetch_fsmap learned FSMap version 71 2024-09-16T01:54:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.431+0000 7f0eb5890740 1 -- 172.21.15.139:0/1025495695 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x560d98723bc0 con 0x560d9867c410 2024-09-16T01:54:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.438+0000 7f0e8b7fe640 1 -- 172.21.15.139:0/1025495695 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea4005a20 con 0x560d9867c410 2024-09-16T01:54:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.438+0000 7f0e8b7fe640 1 -- 172.21.15.139:0/1025495695 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f0ea00048b0 con 0x560d9867c410 2024-09-16T01:54:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.439+0000 7f0eb5890740 10 client.26398 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:54:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.439+0000 7f0eb5890740 10 client.26398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:54:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.439+0000 7f0eb5890740 20 client.26398 populate_metadata read hostname 'smithi139' 2024-09-16T01:54:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.439+0000 7f0eb5890740 1 --2- 172.21.15.139:0/1025495695 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560d98724400 0x560d98726820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.439+0000 7f0eb5890740 4 client.26398 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:54:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.439+0000 7f0eb5890740 1 -- 172.21.15.139:0/1025495695 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d985e7d70 con 0x560d98724400 2024-09-16T01:54:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.447+0000 7f0eabfff640 1 --2- 172.21.15.139:0/1025495695 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560d98724400 0x560d98726820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.448+0000 7f0eabfff640 1 --2- 172.21.15.139:0/1025495695 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560d98724400 0x560d98726820 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff2b6397f68569d0 server_cookie=5835525bc89043ac in_seq=0 out_seq=0 2024-09-16T01:54:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.448+0000 7f0e8b7fe640 10 client.26398 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.559+0000 7f0e8b7fe640 1 -- 172.21.15.139:0/1025495695 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d985e7d70 con 0x560d98724400 2024-09-16T01:54:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.559+0000 7f0e8b7fe640 10 client.26398 handle_command_reply: tid=0 2024-09-16T01:54:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.559+0000 7f0eb5890740 1 client.26398 shutdown 2024-09-16T01:54:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.559+0000 7f0eb5890740 1 -- 172.21.15.139:0/1025495695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d9867c410 msgr2=0x560d9868a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.559+0000 7f0eb5890740 1 --2- 172.21.15.139:0/1025495695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d9867c410 0x560d9868a6e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x560d9867e690 tx=0x7f0ea0004340 comp rx=0 tx=0).stop 2024-09-16T01:54:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.559+0000 7f0eb5890740 1 -- 172.21.15.139:0/1025495695 shutdown_connections 2024-09-16T01:54:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.559+0000 7f0eb5890740 1 --2- 172.21.15.139:0/1025495695 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x560d98724400 0x560d98726820 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.559+0000 7f0eb5890740 1 --2- 172.21.15.139:0/1025495695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d98686cb0 0x560d9868ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.559+0000 7f0eb5890740 1 --2- 172.21.15.139:0/1025495695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560d9867c410 0x560d9868a6e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.559+0000 7f0e8b7fe640 0 client.26398 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.559+0000 7f0eb5890740 1 --2- 172.21.15.139:0/1025495695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d9867baf0 0x560d9868f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.559+0000 7f0eb5890740 1 -- 172.21.15.139:0/1025495695 >> 172.21.15.139:0/1025495695 conn(0x560d98678060 msgr2=0x560d98685c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.560+0000 7f0eb5890740 1 -- 172.21.15.139:0/1025495695 shutdown_connections 2024-09-16T01:54:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.560+0000 7f0eb5890740 1 -- 172.21.15.139:0/1025495695 wait complete. 2024-09-16T01:54:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.560+0000 7f0eb5890740 20 client.26398 trim_cache size 0 max 16384 2024-09-16T01:54:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:54:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:54:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:54:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:54:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.561+0000 7f0eb2a1e640 1 -- 172.21.15.139:0/3126316041 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e940a3690 msgr2=0x7f0e940a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.561+0000 7f0eb2a1e640 1 --2- 172.21.15.139:0/3126316041 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e940a3690 0x7f0e940a5b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40128e0 tx=0x7f0ea4008bd0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.561+0000 7f0eb2a1e640 1 -- 172.21.15.139:0/3126316041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0eac100a00 msgr2=0x7f0eac10a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.561+0000 7f0eb2a1e640 1 --2- 172.21.15.139:0/3126316041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0eac100a00 0x7f0eac10a2a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00c890 tx=0x7f0e9c00cd50 comp rx=0 tx=0).stop 2024-09-16T01:54:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.561+0000 7f0eb2a1e640 1 -- 172.21.15.139:0/3126316041 shutdown_connections 2024-09-16T01:54:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.561+0000 7f0eb2a1e640 1 --2- 172.21.15.139:0/3126316041 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e940a3690 0x7f0e940a5b50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.561+0000 7f0eb2a1e640 1 --2- 172.21.15.139:0/3126316041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eac105f80 0x7f0eac072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.561+0000 7f0eb2a1e640 1 --2- 172.21.15.139:0/3126316041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0eac1013b0 0x7f0eac10a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.561+0000 7f0eb2a1e640 1 --2- 172.21.15.139:0/3126316041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0eac100a00 0x7f0eac10a2a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.561+0000 7f0eb2a1e640 1 -- 172.21.15.139:0/3126316041 >> 172.21.15.139:0/3126316041 conn(0x7f0eac0fc6f0 msgr2=0x7f0eac103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.561+0000 7f0eb2a1e640 1 -- 172.21.15.139:0/3126316041 shutdown_connections 2024-09-16T01:54:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:45.562+0000 7f0eb2a1e640 1 -- 172.21.15.139:0/3126316041 wait complete. 2024-09-16T01:54:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:45 smithi164 ceph-mon[35121]: pgmap v453: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 757 KiB/s wr, 167 op/s 2024-09-16T01:54:45.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:45 smithi139 ceph-mon[31175]: pgmap v453: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 757 KiB/s wr, 167 op/s 2024-09-16T01:54:45.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:45 smithi176 ceph-mon[35324]: pgmap v453: 129 pgs: 22 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 43 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 757 KiB/s wr, 167 op/s 2024-09-16T01:54:47.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:47 smithi164 ceph-mon[35121]: pgmap v454: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 487 KiB/s wr, 171 op/s 2024-09-16T01:54:47.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:47 smithi139 ceph-mon[31175]: pgmap v454: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 487 KiB/s wr, 171 op/s 2024-09-16T01:54:47.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:47 smithi176 ceph-mon[35324]: pgmap v454: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 487 KiB/s wr, 171 op/s 2024-09-16T01:54:49.738 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:49 smithi176 ceph-mon[35324]: pgmap v455: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 177 KiB/s wr, 171 op/s 2024-09-16T01:54:49.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:49 smithi164 ceph-mon[35121]: pgmap v455: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 177 KiB/s wr, 171 op/s 2024-09-16T01:54:49.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:49 smithi139 ceph-mon[31175]: pgmap v455: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 177 KiB/s wr, 171 op/s 2024-09-16T01:54:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:51 smithi164 ceph-mon[35121]: pgmap v456: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 182 KiB/s wr, 157 op/s 2024-09-16T01:54:51.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:51 smithi139 ceph-mon[31175]: pgmap v456: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 182 KiB/s wr, 157 op/s 2024-09-16T01:54:51.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:51 smithi176 ceph-mon[35324]: pgmap v456: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 182 KiB/s wr, 157 op/s 2024-09-16T01:54:53.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:53 smithi164 ceph-mon[35121]: pgmap v457: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 KiB/s wr, 157 op/s 2024-09-16T01:54:53.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:53 smithi139 ceph-mon[31175]: pgmap v457: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 KiB/s wr, 157 op/s 2024-09-16T01:54:53.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:53 smithi176 ceph-mon[35324]: pgmap v457: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 KiB/s wr, 157 op/s 2024-09-16T01:54:55.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:55 smithi164 ceph-mon[35121]: pgmap v458: 129 pgs: 19 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 53 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 155 op/s 2024-09-16T01:54:55.925 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:55 smithi139 ceph-mon[31175]: pgmap v458: 129 pgs: 19 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 53 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 155 op/s 2024-09-16T01:54:55.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:55 smithi176 ceph-mon[35324]: pgmap v458: 129 pgs: 19 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 53 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 155 op/s 2024-09-16T01:54:56.469 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:54:56.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:56 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:56.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:56 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:56.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:56 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:54:57.250 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:54:57.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:57 smithi139 ceph-mon[31175]: pgmap v459: 129 pgs: 19 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 53 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 155 op/s 2024-09-16T01:54:57.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:57 smithi164 ceph-mon[35121]: pgmap v459: 129 pgs: 19 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 53 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 155 op/s 2024-09-16T01:54:57.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:57 smithi176 ceph-mon[35324]: pgmap v459: 129 pgs: 19 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 53 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 155 op/s 2024-09-16T01:54:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.702+0000 7f081edf1640 1 -- 172.21.15.139:0/2755771274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f081811b140 msgr2=0x7f081811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.702+0000 7f081edf1640 1 --2- 172.21.15.139:0/2755771274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f081811b140 0x7f081811d530 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0810009be0 tx=0x7f081002a6d0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.703+0000 7f081edf1640 1 -- 172.21.15.139:0/2755771274 shutdown_connections 2024-09-16T01:54:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.703+0000 7f081edf1640 1 --2- 172.21.15.139:0/2755771274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f081811b140 0x7f081811d530 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.703+0000 7f081edf1640 1 --2- 172.21.15.139:0/2755771274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08181095d0 0x7f0818109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.703+0000 7f081edf1640 1 --2- 172.21.15.139:0/2755771274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f08181091f0 0x7f0818110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.703+0000 7f081edf1640 1 -- 172.21.15.139:0/2755771274 >> 172.21.15.139:0/2755771274 conn(0x7f081806cbf0 msgr2=0x7f081806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.703+0000 7f081edf1640 1 -- 172.21.15.139:0/2755771274 shutdown_connections 2024-09-16T01:54:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.703+0000 7f081edf1640 1 -- 172.21.15.139:0/2755771274 wait complete. 2024-09-16T01:54:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.704+0000 7f081edf1640 1 Processor -- start 2024-09-16T01:54:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.704+0000 7f081edf1640 1 -- start start 2024-09-16T01:54:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.704+0000 7f081edf1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08181091f0 0x7f08181a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.705+0000 7f081edf1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f08181095d0 0x7f08181a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.705+0000 7f081edf1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f081811b140 0x7f08181ab850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.705+0000 7f081edf1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0818120ac0 con 0x7f08181091f0 2024-09-16T01:54:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.705+0000 7f081d5ee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f08181095d0 0x7f08181a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.705+0000 7f081e5f0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f081811b140 0x7f08181ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.705+0000 7f081edf1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0818120940 con 0x7f081811b140 2024-09-16T01:54:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.705+0000 7f081edf1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0818120c40 con 0x7f08181095d0 2024-09-16T01:54:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.705+0000 7f081ddef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08181091f0 0x7f08181a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.705+0000 7f081ddef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08181091f0 0x7f08181a6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51094/0 (socket says 172.21.15.139:51094) 2024-09-16T01:54:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.705+0000 7f081ddef640 1 -- 172.21.15.139:0/2308391701 learned_addr learned my addr 172.21.15.139:0/2308391701 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.706+0000 7f081ddef640 1 -- 172.21.15.139:0/2308391701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f08181095d0 msgr2=0x7f08181a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.706+0000 7f081ddef640 1 --2- 172.21.15.139:0/2308391701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f08181095d0 0x7f08181a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.706+0000 7f081ddef640 1 -- 172.21.15.139:0/2308391701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f081811b140 msgr2=0x7f08181ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.706+0000 7f081ddef640 1 --2- 172.21.15.139:0/2308391701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f081811b140 0x7f08181ab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.706+0000 7f081ddef640 1 -- 172.21.15.139:0/2308391701 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08181abf80 con 0x7f08181091f0 2024-09-16T01:54:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.706+0000 7f081ddef640 1 --2- 172.21.15.139:0/2308391701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08181091f0 0x7f08181a6f90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f081400e9c0 tx=0x7f081400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.706+0000 7f080effd640 1 -- 172.21.15.139:0/2308391701 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081400ccc0 con 0x7f08181091f0 2024-09-16T01:54:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.706+0000 7f080effd640 1 -- 172.21.15.139:0/2308391701 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0814005500 con 0x7f08181091f0 2024-09-16T01:54:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.706+0000 7f080effd640 1 -- 172.21.15.139:0/2308391701 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814010430 con 0x7f08181091f0 2024-09-16T01:54:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.706+0000 7f081edf1640 1 -- 172.21.15.139:0/2308391701 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08181ac270 con 0x7f08181091f0 2024-09-16T01:54:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.706+0000 7f081edf1640 1 -- 172.21.15.139:0/2308391701 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08181b3b00 con 0x7f08181091f0 2024-09-16T01:54:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.710+0000 7f0822df0740 1 Processor -- start 2024-09-16T01:54:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.711+0000 7f0822df0740 1 -- start start 2024-09-16T01:54:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.711+0000 7f0822df0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 0x55bda3f430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.711+0000 7f0822df0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f435e0 0x55bda3f4d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.711+0000 7f0822df0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f4de80 0x55bda3f50270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.711+0000 7f0822df0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bda3ea72f0 con 0x55bda3f435e0 2024-09-16T01:54:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.711+0000 7f0822df0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bda3e4f270 con 0x55bda3f42cc0 2024-09-16T01:54:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.711+0000 7f0822df0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bda3ea32b0 con 0x55bda3f4de80 2024-09-16T01:54:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.712+0000 7f080effd640 1 -- 172.21.15.139:0/2308391701 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f08140105d0 con 0x7f08181091f0 2024-09-16T01:54:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.712+0000 7f081d5ee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f4de80 0x55bda3f50270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.712+0000 7f081d5ee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f4de80 0x55bda3f50270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:57880/0 (socket says 172.21.15.139:57880) 2024-09-16T01:54:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.712+0000 7f081d5ee640 1 -- 172.21.15.139:0/76728246 learned_addr learned my addr 172.21.15.139:0/76728246 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.712+0000 7f081e5f0640 1 --2- 172.21.15.139:0/76728246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 0x55bda3f430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.712+0000 7f081ddef640 1 --2- 172.21.15.139:0/76728246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f435e0 0x55bda3f4d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.713+0000 7f081ddef640 1 -- 172.21.15.139:0/76728246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f4de80 msgr2=0x55bda3f50270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.713+0000 7f081ddef640 1 --2- 172.21.15.139:0/76728246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f4de80 0x55bda3f50270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.713+0000 7f081ddef640 1 -- 172.21.15.139:0/76728246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 msgr2=0x55bda3f430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.713+0000 7f081ddef640 1 --2- 172.21.15.139:0/76728246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 0x55bda3f430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.713+0000 7f081ddef640 1 -- 172.21.15.139:0/76728246 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bda3e4e340 con 0x55bda3f435e0 2024-09-16T01:54:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.713+0000 7f081ddef640 1 --2- 172.21.15.139:0/76728246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f435e0 0x55bda3f4d940 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f08140cdb60 tx=0x7f08140d6b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e421ff80d79658c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.714+0000 7f080effd640 1 --2- 172.21.15.139:0/2308391701 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f08040a3430 0x7f08040a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.714+0000 7f080effd640 1 -- 172.21.15.139:0/2308391701 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f08140d0860 con 0x7f08181091f0 2024-09-16T01:54:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.714+0000 7f0803fff640 1 -- 172.21.15.139:0/76728246 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814101070 con 0x55bda3f435e0 2024-09-16T01:54:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.714+0000 7f0803fff640 1 -- 172.21.15.139:0/76728246 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08140fd450 con 0x55bda3f435e0 2024-09-16T01:54:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.714+0000 7f0803fff640 1 -- 172.21.15.139:0/76728246 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08140fc5f0 con 0x55bda3f435e0 2024-09-16T01:54:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.715+0000 7f0822df0740 1 -- 172.21.15.139:0/76728246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f435e0 msgr2=0x55bda3f4d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.715+0000 7f0822df0740 1 --2- 172.21.15.139:0/76728246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f435e0 0x55bda3f4d940 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f08140cdb60 tx=0x7f08140d6b40 comp rx=0 tx=0).stop 2024-09-16T01:54:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.716+0000 7f081e5f0640 1 --2- 172.21.15.139:0/2308391701 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f08040a3430 0x7f08040a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.716+0000 7f0822df0740 1 -- 172.21.15.139:0/76728246 shutdown_connections 2024-09-16T01:54:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.716+0000 7f0822df0740 1 --2- 172.21.15.139:0/76728246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f4de80 0x55bda3f50270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.716+0000 7f0822df0740 1 --2- 172.21.15.139:0/76728246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f435e0 0x55bda3f4d940 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.716+0000 7f0822df0740 1 --2- 172.21.15.139:0/76728246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 0x55bda3f430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.716+0000 7f0822df0740 1 -- 172.21.15.139:0/76728246 >> 172.21.15.139:0/76728246 conn(0x55bda3f3f230 msgr2=0x55bda3f3f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.718+0000 7f0822df0740 1 -- 172.21.15.139:0/76728246 shutdown_connections 2024-09-16T01:54:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.718+0000 7f0822df0740 1 -- 172.21.15.139:0/76728246 wait complete. 2024-09-16T01:54:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.719+0000 7f0822df0740 1 -- start start 2024-09-16T01:54:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.719+0000 7f0822df0740 1 Processor -- start 2024-09-16T01:54:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.719+0000 7f081e5f0640 1 --2- 172.21.15.139:0/2308391701 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f08040a3430 0x7f08040a58f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0810004af0 tx=0x7f0810005fd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.719+0000 7f0822df0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f42cc0 0x55bda3f4ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.719+0000 7f0822df0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f435e0 0x55bda3f4b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.719+0000 7f0822df0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f4de80 0x55bda3f4b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.719+0000 7f0822df0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bda3f509f0 con 0x55bda3f42cc0 2024-09-16T01:54:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.719+0000 7f0822df0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bda3e30290 con 0x55bda3f4de80 2024-09-16T01:54:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.719+0000 7f0822df0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bda3f50b70 con 0x55bda3f435e0 2024-09-16T01:54:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.719+0000 7f0822df0740 10 client.0 _ll_register_callbacks cb 0x55bda3e61140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:54:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.720+0000 7f081ddef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f42cc0 0x55bda3f4ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.720+0000 7f081e5f0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f4de80 0x55bda3f4b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.720+0000 7f081e5f0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f4de80 0x55bda3f4b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39926/0 (socket says 172.21.15.139:39926) 2024-09-16T01:54:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.720+0000 7f081e5f0640 1 -- 172.21.15.139:0/2758838846 learned_addr learned my addr 172.21.15.139:0/2758838846 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.720+0000 7f081ddef640 1 --2- 172.21.15.139:0/2758838846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f42cc0 0x55bda3f4ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51118/0 (socket says 172.21.15.139:51118) 2024-09-16T01:54:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.720+0000 7f081ddef640 1 -- 172.21.15.139:0/2758838846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f435e0 msgr2=0x55bda3f4b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.720+0000 7f081ddef640 1 --2- 172.21.15.139:0/2758838846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f435e0 0x55bda3f4b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.720+0000 7f081ddef640 1 -- 172.21.15.139:0/2758838846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f4de80 msgr2=0x55bda3f4b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.720+0000 7f081ddef640 1 --2- 172.21.15.139:0/2758838846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f4de80 0x55bda3f4b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.720+0000 7f081ddef640 1 -- 172.21.15.139:0/2758838846 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bda3bba210 con 0x55bda3f42cc0 2024-09-16T01:54:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.721+0000 7f081ddef640 1 --2- 172.21.15.139:0/2758838846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f42cc0 0x55bda3f4ad30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f081400c6a0 tx=0x7f08140d0a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853f6417748d4984 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.721+0000 7f08017fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:54:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.721+0000 7f08017fa640 1 -- 172.21.15.139:0/2758838846 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814101070 con 0x55bda3f42cc0 2024-09-16T01:54:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.721+0000 7f0822df0740 1 -- 172.21.15.139:0/2758838846 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08140fd450 con 0x55bda3f42cc0 2024-09-16T01:54:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.721+0000 7f08017fa640 1 -- 172.21.15.139:0/2758838846 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f0814098f20 con 0x55bda3f42cc0 2024-09-16T01:54:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.722+0000 7f08017fa640 1 -- 172.21.15.139:0/2758838846 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f0814098b50 con 0x55bda3f42cc0 2024-09-16T01:54:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.722+0000 7f0822df0740 10 client.16995 fetch_fsmap learned FSMap version 71 2024-09-16T01:54:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.722+0000 7f0822df0740 1 -- 172.21.15.139:0/2758838846 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55bda3c86620 con 0x55bda3f42cc0 2024-09-16T01:54:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.722+0000 7f08017fa640 1 -- 172.21.15.139:0/2758838846 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08140fd450 con 0x55bda3f42cc0 2024-09-16T01:54:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.723+0000 7f08017fa640 1 -- 172.21.15.139:0/2758838846 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f08140fd600 con 0x55bda3f42cc0 2024-09-16T01:54:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.723+0000 7f0822df0740 10 client.16995 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:54:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.723+0000 7f0822df0740 10 client.16995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:54:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.723+0000 7f0822df0740 20 client.16995 populate_metadata read hostname 'smithi139' 2024-09-16T01:54:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.723+0000 7f0822df0740 1 --2- 172.21.15.139:0/2758838846 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55bda3f5dbf0 0x55bda3f60050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.723+0000 7f0822df0740 4 client.16995 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:54:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.723+0000 7f0822df0740 1 -- 172.21.15.139:0/2758838846 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bda3eb4a20 con 0x55bda3f5dbf0 2024-09-16T01:54:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.724+0000 7f081d5ee640 1 --2- 172.21.15.139:0/2758838846 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55bda3f5dbf0 0x55bda3f60050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.726+0000 7f081d5ee640 1 --2- 172.21.15.139:0/2758838846 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55bda3f5dbf0 0x55bda3f60050 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=961e3108d7d42d21 server_cookie=bc628ac8618385e2 in_seq=0 out_seq=0 2024-09-16T01:54:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.726+0000 7f08017fa640 10 client.16995 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.727+0000 7f08017fa640 1 -- 172.21.15.139:0/2758838846 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bda3eb4a20 con 0x55bda3f5dbf0 2024-09-16T01:54:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.727+0000 7f08017fa640 10 client.16995 handle_command_reply: tid=0 2024-09-16T01:54:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.727+0000 7f0822df0740 1 client.16995 shutdown 2024-09-16T01:54:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.727+0000 7f0822df0740 1 -- 172.21.15.139:0/2758838846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f42cc0 msgr2=0x55bda3f4ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.727+0000 7f0822df0740 1 --2- 172.21.15.139:0/2758838846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f42cc0 0x55bda3f4ad30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f081400c6a0 tx=0x7f08140d0a60 comp rx=0 tx=0).stop 2024-09-16T01:54:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.728+0000 7f0822df0740 1 -- 172.21.15.139:0/2758838846 shutdown_connections 2024-09-16T01:54:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.728+0000 7f0822df0740 1 --2- 172.21.15.139:0/2758838846 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55bda3f5dbf0 0x55bda3f60050 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.728+0000 7f0822df0740 1 --2- 172.21.15.139:0/2758838846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f4de80 0x55bda3f4b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.728+0000 7f0822df0740 1 --2- 172.21.15.139:0/2758838846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f435e0 0x55bda3f4b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.728+0000 7f0822df0740 1 --2- 172.21.15.139:0/2758838846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f42cc0 0x55bda3f4ad30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.728+0000 7f0822df0740 1 -- 172.21.15.139:0/2758838846 >> 172.21.15.139:0/2758838846 conn(0x55bda3f3f230 msgr2=0x55bda3f44f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.728+0000 7f08017fa640 0 client.16995 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.728+0000 7f0822df0740 1 -- 172.21.15.139:0/2758838846 shutdown_connections 2024-09-16T01:54:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.728+0000 7f0822df0740 1 -- 172.21.15.139:0/2758838846 wait complete. 2024-09-16T01:54:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.729+0000 7f0822df0740 20 client.16995 trim_cache size 0 max 16384 2024-09-16T01:54:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.757+0000 7f0822df0740 1 Processor -- start 2024-09-16T01:54:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.757+0000 7f0822df0740 1 -- start start 2024-09-16T01:54:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f0822df0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 0x55bda3f4b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f0822df0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f435e0 0x55bda3f4c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f0822df0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f4de80 0x55bda3f4cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f0822df0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc0101c0 con 0x55bda3f435e0 2024-09-16T01:54:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f0822df0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc012000 con 0x55bda3f42cc0 2024-09-16T01:54:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f0822df0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc010950 con 0x55bda3f4de80 2024-09-16T01:54:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f081d5ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f435e0 0x55bda3f4c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f081d5ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f435e0 0x55bda3f4c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51132/0 (socket says 172.21.15.139:51132) 2024-09-16T01:54:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f081d5ee640 1 -- 172.21.15.139:0/200966616 learned_addr learned my addr 172.21.15.139:0/200966616 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f081ddef640 1 --2- 172.21.15.139:0/200966616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 0x55bda3f4b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f081d5ee640 1 -- 172.21.15.139:0/200966616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f4de80 msgr2=0x55bda3f4cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f081d5ee640 1 --2- 172.21.15.139:0/200966616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f4de80 0x55bda3f4cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f081d5ee640 1 -- 172.21.15.139:0/200966616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 msgr2=0x55bda3f4b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f081d5ee640 1 --2- 172.21.15.139:0/200966616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 0x55bda3f4b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.758+0000 7f081d5ee640 1 -- 172.21.15.139:0/200966616 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bda3f50ba0 con 0x55bda3f435e0 2024-09-16T01:54:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.759+0000 7f081d5ee640 1 --2- 172.21.15.139:0/200966616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f435e0 0x55bda3f4c590 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0808005f30 tx=0x7f0808014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62849b7c55e71324 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.759+0000 7f08037fe640 1 -- 172.21.15.139:0/200966616 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080801d430 con 0x55bda3f435e0 2024-09-16T01:54:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.759+0000 7f08037fe640 1 -- 172.21.15.139:0/200966616 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f080801dab0 con 0x55bda3f435e0 2024-09-16T01:54:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.759+0000 7f08037fe640 1 -- 172.21.15.139:0/200966616 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808012af0 con 0x55bda3f435e0 2024-09-16T01:54:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.760+0000 7f0822df0740 1 -- 172.21.15.139:0/200966616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f435e0 msgr2=0x55bda3f4c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.760+0000 7f0822df0740 1 --2- 172.21.15.139:0/200966616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f435e0 0x55bda3f4c590 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0808005f30 tx=0x7f0808014840 comp rx=0 tx=0).stop 2024-09-16T01:54:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.760+0000 7f0822df0740 1 -- 172.21.15.139:0/200966616 shutdown_connections 2024-09-16T01:54:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.760+0000 7f0822df0740 1 --2- 172.21.15.139:0/200966616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f4de80 0x55bda3f4cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.760+0000 7f0822df0740 1 --2- 172.21.15.139:0/200966616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f435e0 0x55bda3f4c590 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.760+0000 7f0822df0740 1 --2- 172.21.15.139:0/200966616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 0x55bda3f4b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.760+0000 7f0822df0740 1 -- 172.21.15.139:0/200966616 >> 172.21.15.139:0/200966616 conn(0x55bda3f3f230 msgr2=0x55bda3f5fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.765+0000 7f0822df0740 1 -- 172.21.15.139:0/200966616 shutdown_connections 2024-09-16T01:54:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.765+0000 7f0822df0740 1 -- 172.21.15.139:0/200966616 wait complete. 2024-09-16T01:54:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.766+0000 7f0822df0740 1 -- start start 2024-09-16T01:54:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.766+0000 7f0822df0740 1 Processor -- start 2024-09-16T01:54:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.767+0000 7f0822df0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 0x55bda3f562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.767+0000 7f0822df0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f435e0 0x55bda3f50e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.767+0000 7f0822df0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f4de80 0x55bda3f51340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.767+0000 7f081ddef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 0x55bda3f562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.767+0000 7f081ddef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 0x55bda3f562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39952/0 (socket says 172.21.15.139:39952) 2024-09-16T01:54:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.767+0000 7f081d5ee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f435e0 0x55bda3f50e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.767+0000 7f081d5ee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f435e0 0x55bda3f50e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:57906/0 (socket says 172.21.15.139:57906) 2024-09-16T01:54:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.768+0000 7f081ddef640 1 -- 172.21.15.139:0/2264845810 learned_addr learned my addr 172.21.15.139:0/2264845810 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:54:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.768+0000 7f0822df0740 1 -- 172.21.15.139:0/2264845810 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc011870 con 0x55bda3f4de80 2024-09-16T01:54:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.768+0000 7f0822df0740 1 -- 172.21.15.139:0/2264845810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc0110e0 con 0x55bda3f42cc0 2024-09-16T01:54:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.768+0000 7f0822df0740 1 -- 172.21.15.139:0/2264845810 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc00e1a0 con 0x55bda3f435e0 2024-09-16T01:54:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.768+0000 7f0822df0740 10 client.0 _ll_register_callbacks cb 0x7f08080095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:54:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.768+0000 7f081ddef640 1 -- 172.21.15.139:0/2264845810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f435e0 msgr2=0x55bda3f50e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.768+0000 7f081ddef640 1 --2- 172.21.15.139:0/2264845810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f435e0 0x55bda3f50e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.768+0000 7f081ddef640 1 -- 172.21.15.139:0/2264845810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f4de80 msgr2=0x55bda3f51340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:54:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.768+0000 7f081ddef640 1 --2- 172.21.15.139:0/2264845810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f4de80 0x55bda3f51340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.768+0000 7f081ddef640 1 -- 172.21.15.139:0/2264845810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bda3f45ab0 con 0x55bda3f42cc0 2024-09-16T01:54:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.769+0000 7f081ddef640 1 --2- 172.21.15.139:0/2264845810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 0x55bda3f562e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f08140a4f70 tx=0x7f08140d6fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ee1d8387eeed1c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:54:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.769+0000 7f08017fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:54:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.769+0000 7f08017fa640 1 -- 172.21.15.139:0/2264845810 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814101070 con 0x55bda3f42cc0 2024-09-16T01:54:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.769+0000 7f0822df0740 1 -- 172.21.15.139:0/2264845810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f080801dab0 con 0x55bda3f42cc0 2024-09-16T01:54:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.769+0000 7f08017fa640 1 -- 172.21.15.139:0/2264845810 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f08140048c0 con 0x55bda3f42cc0 2024-09-16T01:54:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.770+0000 7f08017fa640 1 -- 172.21.15.139:0/2264845810 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f081409c210 con 0x55bda3f42cc0 2024-09-16T01:54:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.771+0000 7f0822df0740 10 client.26441 fetch_fsmap learned FSMap version 71 2024-09-16T01:54:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.771+0000 7f0822df0740 1 -- 172.21.15.139:0/2264845810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55bda3f45ab0 con 0x55bda3f42cc0 2024-09-16T01:54:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.772+0000 7f08017fa640 1 -- 172.21.15.139:0/2264845810 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f080801dab0 con 0x55bda3f42cc0 2024-09-16T01:54:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.772+0000 7f08017fa640 1 -- 172.21.15.139:0/2264845810 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f08140a0b80 con 0x55bda3f42cc0 2024-09-16T01:54:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.773+0000 7f0822df0740 10 client.26441 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:54:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.773+0000 7f0822df0740 10 client.26441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:54:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.773+0000 7f0822df0740 20 client.26441 populate_metadata read hostname 'smithi139' 2024-09-16T01:54:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.773+0000 7f0822df0740 1 --2- 172.21.15.139:0/2264845810 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55bda3fc87b0 0x55bda3fcabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:54:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.773+0000 7f0822df0740 4 client.26441 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.773+0000 7f0822df0740 1 -- 172.21.15.139:0/2264845810 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bda3eaed70 con 0x55bda3fc87b0 2024-09-16T01:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.774+0000 7f081d5ee640 1 --2- 172.21.15.139:0/2264845810 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55bda3fc87b0 0x55bda3fcabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.775+0000 7f08017fa640 10 client.26441 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:58.775+0000 7f081d5ee640 1 --2- 172.21.15.139:0/2264845810 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55bda3fc87b0 0x55bda3fcabd0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80dc6b248a8cf5c6 server_cookie=5b50c8b63c10cc85 in_seq=0 out_seq=0 2024-09-16T01:54:58.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:58 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:58.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:58 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:58.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:58 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:54:58.831 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:58 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:58.831 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:58 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:58.831 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:58 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:54:58.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:58 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:54:58.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:58 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:54:58.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:58 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:54:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.055+0000 7f08017fa640 1 -- 172.21.15.139:0/2264845810 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bda3eaed70 con 0x55bda3fc87b0 2024-09-16T01:54:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.056+0000 7f08017fa640 10 client.26441 handle_command_reply: tid=0 2024-09-16T01:54:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.056+0000 7f0822df0740 1 client.26441 shutdown 2024-09-16T01:54:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.056+0000 7f0822df0740 1 -- 172.21.15.139:0/2264845810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 msgr2=0x55bda3f562e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.056+0000 7f0822df0740 1 --2- 172.21.15.139:0/2264845810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 0x55bda3f562e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f08140a4f70 tx=0x7f08140d6fa0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.056+0000 7f0822df0740 1 -- 172.21.15.139:0/2264845810 shutdown_connections 2024-09-16T01:54:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.056+0000 7f0822df0740 1 --2- 172.21.15.139:0/2264845810 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55bda3fc87b0 0x55bda3fcabd0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.056+0000 7f0822df0740 1 --2- 172.21.15.139:0/2264845810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bda3f4de80 0x55bda3f51340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.056+0000 7f0822df0740 1 --2- 172.21.15.139:0/2264845810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bda3f435e0 0x55bda3f50e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.056+0000 7f0822df0740 1 --2- 172.21.15.139:0/2264845810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bda3f42cc0 0x55bda3f562e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.056+0000 7f0822df0740 1 -- 172.21.15.139:0/2264845810 >> 172.21.15.139:0/2264845810 conn(0x55bda3f3f230 msgr2=0x55bda3f3e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.056+0000 7f08017fa640 0 client.26441 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:54:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.056+0000 7f0822df0740 1 -- 172.21.15.139:0/2264845810 shutdown_connections 2024-09-16T01:54:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.056+0000 7f0822df0740 1 -- 172.21.15.139:0/2264845810 wait complete. 2024-09-16T01:54:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.056+0000 7f0822df0740 20 client.26441 trim_cache size 0 max 16384 2024-09-16T01:54:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:54:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:54:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:54:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:54:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.057+0000 7f081edf1640 1 -- 172.21.15.139:0/2308391701 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f08040a3430 msgr2=0x7f08040a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.057+0000 7f081edf1640 1 --2- 172.21.15.139:0/2308391701 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f08040a3430 0x7f08040a58f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0810004af0 tx=0x7f0810005fd0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.057+0000 7f081edf1640 1 -- 172.21.15.139:0/2308391701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08181091f0 msgr2=0x7f08181a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:54:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.057+0000 7f081edf1640 1 --2- 172.21.15.139:0/2308391701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08181091f0 0x7f08181a6f90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f081400e9c0 tx=0x7f081400ee80 comp rx=0 tx=0).stop 2024-09-16T01:54:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.057+0000 7f081edf1640 1 -- 172.21.15.139:0/2308391701 shutdown_connections 2024-09-16T01:54:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.057+0000 7f081edf1640 1 --2- 172.21.15.139:0/2308391701 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f08040a3430 0x7f08040a58f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.058+0000 7f081edf1640 1 --2- 172.21.15.139:0/2308391701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f081811b140 0x7f08181ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.058+0000 7f081edf1640 1 --2- 172.21.15.139:0/2308391701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f08181095d0 0x7f08181a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.058+0000 7f081edf1640 1 --2- 172.21.15.139:0/2308391701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08181091f0 0x7f08181a6f90 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:54:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.058+0000 7f081edf1640 1 -- 172.21.15.139:0/2308391701 >> 172.21.15.139:0/2308391701 conn(0x7f081806cbf0 msgr2=0x7f081811b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:54:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.058+0000 7f081edf1640 1 -- 172.21.15.139:0/2308391701 shutdown_connections 2024-09-16T01:54:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:54:59.058+0000 7f081edf1640 1 -- 172.21.15.139:0/2308391701 wait complete. 2024-09-16T01:54:59.821 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:54:59 smithi139 ceph-mon[31175]: pgmap v460: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 139 op/s 2024-09-16T01:54:59.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:54:59 smithi176 ceph-mon[35324]: pgmap v460: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 139 op/s 2024-09-16T01:55:00.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:54:59 smithi164 ceph-mon[35121]: pgmap v460: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 139 op/s 2024-09-16T01:55:00.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.114+0000 7fe4fa3a3640 1 -- 172.21.15.139:0/3147695002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f411b140 msgr2=0x7fe4f411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:00.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.115+0000 7fe4fa3a3640 1 --2- 172.21.15.139:0/3147695002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f411b140 0x7fe4f411d530 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009ca0 tx=0x7fe4e402a8f0 comp rx=0 tx=0).stop 2024-09-16T01:55:00.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.116+0000 7fe4fa3a3640 1 -- 172.21.15.139:0/3147695002 shutdown_connections 2024-09-16T01:55:00.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.116+0000 7fe4fa3a3640 1 --2- 172.21.15.139:0/3147695002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f411b140 0x7fe4f411d530 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:00.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.116+0000 7fe4fa3a3640 1 --2- 172.21.15.139:0/3147695002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4f4109830 0x7fe4f4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:00.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.116+0000 7fe4fa3a3640 1 --2- 172.21.15.139:0/3147695002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f4109450 0x7fe4f4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:00.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.116+0000 7fe4fa3a3640 1 -- 172.21.15.139:0/3147695002 >> 172.21.15.139:0/3147695002 conn(0x7fe4f406e140 msgr2=0x7fe4f406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:00.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.122+0000 7fe4fa3a3640 1 -- 172.21.15.139:0/3147695002 shutdown_connections 2024-09-16T01:55:00.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.124+0000 7fe4fa3a3640 1 -- 172.21.15.139:0/3147695002 wait complete. 2024-09-16T01:55:00.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.126+0000 7fe4fa3a3640 1 Processor -- start 2024-09-16T01:55:00.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.126+0000 7fe4fa3a3640 1 -- start start 2024-09-16T01:55:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.127+0000 7fe4fa3a3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f4109450 0x7fe4f41a7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.127+0000 7fe4fa3a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f4109830 0x7fe4f41a7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.127+0000 7fe4fa3a3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4f411b140 0x7fe4f41abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.127+0000 7fe4fa3a3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4120da0 con 0x7fe4f4109830 2024-09-16T01:55:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.127+0000 7fe4fa3a3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4120c20 con 0x7fe4f4109450 2024-09-16T01:55:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.127+0000 7fe4fa3a3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4120f20 con 0x7fe4f411b140 2024-09-16T01:55:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.128+0000 7fe4f37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f4109830 0x7fe4f41a7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.128+0000 7fe4f37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f4109830 0x7fe4f41a7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51178/0 (socket says 172.21.15.139:51178) 2024-09-16T01:55:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.128+0000 7fe4f37fe640 1 -- 172.21.15.139:0/2390284989 learned_addr learned my addr 172.21.15.139:0/2390284989 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:00.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.129+0000 7fe4f3fff640 1 --2- 172.21.15.139:0/2390284989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f4109450 0x7fe4f41a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:00.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.129+0000 7fe4f8940640 1 --2- 172.21.15.139:0/2390284989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4f411b140 0x7fe4f41abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:00.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.130+0000 7fe4f37fe640 1 -- 172.21.15.139:0/2390284989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4f411b140 msgr2=0x7fe4f41abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:00.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.130+0000 7fe4f37fe640 1 --2- 172.21.15.139:0/2390284989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4f411b140 0x7fe4f41abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:00.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.130+0000 7fe4f37fe640 1 -- 172.21.15.139:0/2390284989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f4109450 msgr2=0x7fe4f41a7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:00.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.130+0000 7fe4f37fe640 1 --2- 172.21.15.139:0/2390284989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f4109450 0x7fe4f41a7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:00.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.130+0000 7fe4f37fe640 1 -- 172.21.15.139:0/2390284989 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f41ac2b0 con 0x7fe4f4109830 2024-09-16T01:55:00.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.131+0000 7fe4f37fe640 1 --2- 172.21.15.139:0/2390284989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f4109830 0x7fe4f41a7840 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00e9f0 tx=0x7fe4dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:00.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.134+0000 7fe4f17fa640 1 -- 172.21.15.139:0/2390284989 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc00cd80 con 0x7fe4f4109830 2024-09-16T01:55:00.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.134+0000 7fe4f17fa640 1 -- 172.21.15.139:0/2390284989 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc010910 con 0x7fe4f4109830 2024-09-16T01:55:00.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.135+0000 7fe4f17fa640 1 -- 172.21.15.139:0/2390284989 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc0188f0 con 0x7fe4f4109830 2024-09-16T01:55:00.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.135+0000 7fe4fa3a3640 1 -- 172.21.15.139:0/2390284989 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f41ac5a0 con 0x7fe4f4109830 2024-09-16T01:55:00.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.135+0000 7fe4fa3a3640 1 -- 172.21.15.139:0/2390284989 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f41b3e80 con 0x7fe4f4109830 2024-09-16T01:55:00.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.138+0000 7fe4fa3a3640 1 -- 172.21.15.139:0/2390284989 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f4073b00 con 0x7fe4f4109830 2024-09-16T01:55:00.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.141+0000 7fe4f17fa640 1 -- 172.21.15.139:0/2390284989 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe4dc010430 con 0x7fe4f4109830 2024-09-16T01:55:00.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.142+0000 7fe4f17fa640 1 --2- 172.21.15.139:0/2390284989 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe4e00a34a0 0x7fe4e00a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:00.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.143+0000 7fe4f17fa640 1 -- 172.21.15.139:0/2390284989 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7fe4dc0d0490 con 0x7fe4f4109830 2024-09-16T01:55:00.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.143+0000 7fe4f3fff640 1 --2- 172.21.15.139:0/2390284989 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe4e00a34a0 0x7fe4e00a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:00.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.143+0000 7fe4f3fff640 1 --2- 172.21.15.139:0/2390284989 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe4e00a34a0 0x7fe4e00a5960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009b60 tx=0x7fe4ec005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:00.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.151+0000 7fe4f17fa640 1 -- 172.21.15.139:0/2390284989 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4dc09c9c0 con 0x7fe4f4109830 2024-09-16T01:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.563+0000 7fe4fa3a3640 1 -- 172.21.15.139:0/2390284989 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4f410b3a0 con 0x7fe4f4109830 2024-09-16T01:55:00.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.569+0000 7fe4f17fa640 1 -- 172.21.15.139:0/2390284989 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+12031 (secure 0 0 0) 0x7fe4dc09c2c0 con 0x7fe4f4109830 2024-09-16T01:55:00.571 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:55:00.571 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":71,"btime":"2024-09-16T01:54:19:332763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:54:19.051272+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:55:00.571 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 71 2024-09-16T01:55:00.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.577+0000 7fe4eaffd640 1 -- 172.21.15.139:0/2390284989 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe4e00a34a0 msgr2=0x7fe4e00a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:00.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.577+0000 7fe4eaffd640 1 --2- 172.21.15.139:0/2390284989 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe4e00a34a0 0x7fe4e00a5960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009b60 tx=0x7fe4ec005f90 comp rx=0 tx=0).stop 2024-09-16T01:55:00.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.577+0000 7fe4eaffd640 1 -- 172.21.15.139:0/2390284989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f4109830 msgr2=0x7fe4f41a7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:00.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.577+0000 7fe4eaffd640 1 --2- 172.21.15.139:0/2390284989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f4109830 0x7fe4f41a7840 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00e9f0 tx=0x7fe4dc00eeb0 comp rx=0 tx=0).stop 2024-09-16T01:55:00.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.580+0000 7fe4eaffd640 1 -- 172.21.15.139:0/2390284989 shutdown_connections 2024-09-16T01:55:00.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.580+0000 7fe4eaffd640 1 --2- 172.21.15.139:0/2390284989 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe4e00a34a0 0x7fe4e00a5960 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:00.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.580+0000 7fe4eaffd640 1 --2- 172.21.15.139:0/2390284989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4f411b140 0x7fe4f41abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:00.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.580+0000 7fe4eaffd640 1 --2- 172.21.15.139:0/2390284989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4f4109830 0x7fe4f41a7840 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:00.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.580+0000 7fe4eaffd640 1 --2- 172.21.15.139:0/2390284989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f4109450 0x7fe4f41a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:00.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.580+0000 7fe4eaffd640 1 -- 172.21.15.139:0/2390284989 >> 172.21.15.139:0/2390284989 conn(0x7fe4f406e140 msgr2=0x7fe4f411bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:00.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.583+0000 7fe4eaffd640 1 -- 172.21.15.139:0/2390284989 shutdown_connections 2024-09-16T01:55:00.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:00.584+0000 7fe4eaffd640 1 -- 172.21.15.139:0/2390284989 wait complete. 2024-09-16T01:55:00.840 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:00 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2390284989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:55:00.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:00 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2390284989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:55:01.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:00 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2390284989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:55:01.430 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 724cf2d2 --release --await' 2024-09-16T01:55:01.430 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 724cf2d2 --release --await 2024-09-16T01:55:01.911 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:55:01.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:01 smithi139 ceph-mon[31175]: pgmap v461: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 105 op/s 2024-09-16T01:55:01.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:01 smithi176 ceph-mon[35324]: pgmap v461: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 105 op/s 2024-09-16T01:55:02.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:01 smithi164 ceph-mon[35121]: pgmap v461: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 105 op/s 2024-09-16T01:55:02.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.551+0000 7f3870162640 1 -- 172.21.15.139:0/4046382359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3868106a90 msgr2=0x7f3868106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:02.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.551+0000 7f3870162640 1 --2- 172.21.15.139:0/4046382359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3868106a90 0x7f3868106e70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3864009900 tx=0x7f386402a440 comp rx=0 tx=0).stop 2024-09-16T01:55:02.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.554+0000 7f3870162640 1 -- 172.21.15.139:0/4046382359 shutdown_connections 2024-09-16T01:55:02.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.554+0000 7f3870162640 1 --2- 172.21.15.139:0/4046382359 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3868101430 0x7f386810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.555+0000 7f3870162640 1 --2- 172.21.15.139:0/4046382359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3868100a90 0x7f3868100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.555+0000 7f3870162640 1 --2- 172.21.15.139:0/4046382359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3868106a90 0x7f3868106e70 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.555+0000 7f3870162640 1 -- 172.21.15.139:0/4046382359 >> 172.21.15.139:0/4046382359 conn(0x7f38680fc7d0 msgr2=0x7f38680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:02.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.560+0000 7f3870162640 1 -- 172.21.15.139:0/4046382359 shutdown_connections 2024-09-16T01:55:02.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.562+0000 7f3870162640 1 -- 172.21.15.139:0/4046382359 wait complete. 2024-09-16T01:55:02.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.563+0000 7f3870162640 1 Processor -- start 2024-09-16T01:55:02.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.563+0000 7f3870162640 1 -- start start 2024-09-16T01:55:02.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.563+0000 7f3870162640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3868100a90 0x7f38681a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.563+0000 7f3870162640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3868101430 0x7f38681a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.563+0000 7f3870162640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3868106a90 0x7f386810ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.563+0000 7f3870162640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f38681120d0 con 0x7f3868100a90 2024-09-16T01:55:02.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.563+0000 7f3870162640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3868111f50 con 0x7f3868101430 2024-09-16T01:55:02.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.563+0000 7f3870162640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3868112250 con 0x7f3868106a90 2024-09-16T01:55:02.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.563+0000 7f386e6ff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3868106a90 0x7f386810ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.563+0000 7f386e6ff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3868106a90 0x7f386810ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:53714/0 (socket says 172.21.15.139:53714) 2024-09-16T01:55:02.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.563+0000 7f386e6ff640 1 -- 172.21.15.139:0/159069103 learned_addr learned my addr 172.21.15.139:0/159069103 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:02.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.564+0000 7f386d6fd640 1 --2- 172.21.15.139:0/159069103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3868101430 0x7f38681a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.564+0000 7f386e6ff640 1 -- 172.21.15.139:0/159069103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3868101430 msgr2=0x7f38681a3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:02.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.564+0000 7f386e6ff640 1 --2- 172.21.15.139:0/159069103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3868101430 0x7f38681a3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.564+0000 7f386e6ff640 1 -- 172.21.15.139:0/159069103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3868100a90 msgr2=0x7f38681a3770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:55:02.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.564+0000 7f386e6ff640 1 --2- 172.21.15.139:0/159069103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3868100a90 0x7f38681a3770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.564+0000 7f386e6ff640 1 -- 172.21.15.139:0/159069103 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386810d1a0 con 0x7f3868106a90 2024-09-16T01:55:02.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.564+0000 7f386e6ff640 1 --2- 172.21.15.139:0/159069103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3868106a90 0x7f386810ca70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f386000b990 tx=0x7f386000be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:02.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.566+0000 7f385effd640 1 -- 172.21.15.139:0/159069103 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860019070 con 0x7f3868106a90 2024-09-16T01:55:02.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.566+0000 7f385effd640 1 -- 172.21.15.139:0/159069103 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f386000cba0 con 0x7f3868106a90 2024-09-16T01:55:02.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.566+0000 7f385effd640 1 -- 172.21.15.139:0/159069103 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38600076a0 con 0x7f3868106a90 2024-09-16T01:55:02.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.566+0000 7f3870162640 1 -- 172.21.15.139:0/159069103 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386810d490 con 0x7f3868106a90 2024-09-16T01:55:02.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.567+0000 7f3870162640 1 -- 172.21.15.139:0/159069103 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f386806dc20 con 0x7f3868106a90 2024-09-16T01:55:02.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.570+0000 7f387253b740 1 Processor -- start 2024-09-16T01:55:02.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.571+0000 7f387253b740 1 -- start start 2024-09-16T01:55:02.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.571+0000 7f387253b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30060 0x55711aa30440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.571+0000 7f387253b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa30980 0x55711aa3ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.571+0000 7f387253b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa3b240 0x55711aa3d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.571+0000 7f387253b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55711a995620 con 0x55711aa30980 2024-09-16T01:55:02.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.571+0000 7f387253b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55711a93ce50 con 0x55711aa3b240 2024-09-16T01:55:02.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.571+0000 7f387253b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55711a9915e0 con 0x55711aa30060 2024-09-16T01:55:02.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.573+0000 7f386d6fd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30060 0x55711aa30440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.573+0000 7f386d6fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa3b240 0x55711aa3d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.573+0000 7f386d6fd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30060 0x55711aa30440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:53728/0 (socket says 172.21.15.139:53728) 2024-09-16T01:55:02.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.573+0000 7f386d6fd640 1 -- 172.21.15.139:0/3752859310 learned_addr learned my addr 172.21.15.139:0/3752859310 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:02.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.574+0000 7f385effd640 1 -- 172.21.15.139:0/159069103 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f386001c070 con 0x7f3868106a90 2024-09-16T01:55:02.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.575+0000 7f386d6fd640 1 -- 172.21.15.139:0/3752859310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30060 msgr2=0x55711aa30440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:02.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.575+0000 7f386d6fd640 1 --2- 172.21.15.139:0/3752859310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30060 0x55711aa30440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.575+0000 7f385effd640 1 --2- 172.21.15.139:0/159069103 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f38540a35c0 0x7f38540a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.575+0000 7f386d6fd640 1 -- 172.21.15.139:0/3752859310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa30980 msgr2=0x55711aa3ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:02.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.575+0000 7f386d6fd640 1 --2- 172.21.15.139:0/3752859310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa30980 0x55711aa3ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.575+0000 7f386d6fd640 1 -- 172.21.15.139:0/3752859310 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55711a92dd40 con 0x55711aa3b240 2024-09-16T01:55:02.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.575+0000 7f385effd640 1 -- 172.21.15.139:0/159069103 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f38600cf5e0 con 0x7f3868106a90 2024-09-16T01:55:02.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.576+0000 7f386d6fd640 1 --2- 172.21.15.139:0/3752859310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa3b240 0x55711aa3d630 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f385800beb0 tx=0x7f38580046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=588cb8cf1b40ba43 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:02.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.576+0000 7f3853fff640 1 -- 172.21.15.139:0/3752859310 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385800f040 con 0x55711aa3b240 2024-09-16T01:55:02.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.577+0000 7f3853fff640 1 -- 172.21.15.139:0/3752859310 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f385800bce0 con 0x55711aa3b240 2024-09-16T01:55:02.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.577+0000 7f3853fff640 1 -- 172.21.15.139:0/3752859310 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858013750 con 0x55711aa3b240 2024-09-16T01:55:02.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.578+0000 7f387253b740 1 -- 172.21.15.139:0/3752859310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa3b240 msgr2=0x55711aa3d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:02.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.578+0000 7f387253b740 1 --2- 172.21.15.139:0/3752859310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa3b240 0x55711aa3d630 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f385800beb0 tx=0x7f38580046e0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.581+0000 7f386e6ff640 1 --2- 172.21.15.139:0/159069103 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f38540a35c0 0x7f38540a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.581+0000 7f387253b740 1 -- 172.21.15.139:0/3752859310 shutdown_connections 2024-09-16T01:55:02.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.581+0000 7f387253b740 1 --2- 172.21.15.139:0/3752859310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa3b240 0x55711aa3d630 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.581+0000 7f387253b740 1 --2- 172.21.15.139:0/3752859310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa30980 0x55711aa3ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.581+0000 7f387253b740 1 --2- 172.21.15.139:0/3752859310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30060 0x55711aa30440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.581+0000 7f387253b740 1 -- 172.21.15.139:0/3752859310 >> 172.21.15.139:0/3752859310 conn(0x55711aa2c5f0 msgr2=0x55711aa2c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:02.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.581+0000 7f387253b740 1 -- 172.21.15.139:0/3752859310 shutdown_connections 2024-09-16T01:55:02.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.581+0000 7f387253b740 1 -- 172.21.15.139:0/3752859310 wait complete. 2024-09-16T01:55:02.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.582+0000 7f387253b740 1 -- start start 2024-09-16T01:55:02.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.582+0000 7f387253b740 1 Processor -- start 2024-09-16T01:55:02.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.585+0000 7f387253b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30060 0x55711aa381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.585+0000 7f387253b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa30980 0x55711aa386f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.585+0000 7f387253b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa3b240 0x55711aa38c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.585+0000 7f387253b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55711aa3de10 con 0x55711aa30980 2024-09-16T01:55:02.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.585+0000 7f387253b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55711a91df10 con 0x55711aa3b240 2024-09-16T01:55:02.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.585+0000 7f387253b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55711aa3df90 con 0x55711aa30060 2024-09-16T01:55:02.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.585+0000 7f387253b740 10 client.0 _ll_register_callbacks cb 0x55711a941db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:55:02.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.585+0000 7f386defe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa30980 0x55711aa386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.586+0000 7f386d6fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa3b240 0x55711aa38c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.586+0000 7f386d6fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa3b240 0x55711aa38c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:55408/0 (socket says 172.21.15.139:55408) 2024-09-16T01:55:02.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.586+0000 7f386d6fd640 1 -- 172.21.15.139:0/3418864229 learned_addr learned my addr 172.21.15.139:0/3418864229 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:02.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.586+0000 7f386defe640 1 --2- 172.21.15.139:0/3418864229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa30980 0x55711aa386f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54282/0 (socket says 172.21.15.139:54282) 2024-09-16T01:55:02.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.586+0000 7f386d6fd640 1 --2- 172.21.15.139:0/3418864229 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30060 0x55711aa381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.586+0000 7f386defe640 1 -- 172.21.15.139:0/3418864229 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30060 msgr2=0x55711aa381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:02.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.586+0000 7f386defe640 1 --2- 172.21.15.139:0/3418864229 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30060 0x55711aa381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.586+0000 7f386defe640 1 -- 172.21.15.139:0/3418864229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa3b240 msgr2=0x55711aa38c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:02.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.586+0000 7f386defe640 1 --2- 172.21.15.139:0/3418864229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa3b240 0x55711aa38c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.586+0000 7f386defe640 1 -- 172.21.15.139:0/3418864229 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55711a6a4580 con 0x55711aa30980 2024-09-16T01:55:02.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.587+0000 7f386defe640 1 --2- 172.21.15.139:0/3418864229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa30980 0x55711aa386f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f386402a480 tx=0x7f3864032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7b9dcb077b400b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:02.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.587+0000 7f38517fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:55:02.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.587+0000 7f38517fa640 1 -- 172.21.15.139:0/3418864229 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386403f070 con 0x55711aa30980 2024-09-16T01:55:02.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.587+0000 7f387253b740 1 -- 172.21.15.139:0/3418864229 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f385800bce0 con 0x55711aa30980 2024-09-16T01:55:02.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.587+0000 7f38517fa640 1 -- 172.21.15.139:0/3418864229 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f3864004030 con 0x55711aa30980 2024-09-16T01:55:02.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.587+0000 7f386e6ff640 1 --2- 172.21.15.139:0/159069103 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f38540a35c0 0x7f38540a5a80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3860009500 tx=0x7f3860009470 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:02.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.592+0000 7f38517fa640 1 -- 172.21.15.139:0/3418864229 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f3864035070 con 0x55711aa30980 2024-09-16T01:55:02.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.592+0000 7f387253b740 10 client.17010 fetch_fsmap learned FSMap version 71 2024-09-16T01:55:02.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.592+0000 7f387253b740 1 -- 172.21.15.139:0/3418864229 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55711a76e960 con 0x55711aa30980 2024-09-16T01:55:02.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.593+0000 7f38517fa640 1 -- 172.21.15.139:0/3418864229 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f386404b070 con 0x55711aa30980 2024-09-16T01:55:02.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.593+0000 7f38517fa640 1 -- 172.21.15.139:0/3418864229 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f3864004860 con 0x55711aa30980 2024-09-16T01:55:02.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.594+0000 7f387253b740 10 client.17010 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:55:02.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.594+0000 7f387253b740 10 client.17010 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:55:02.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.594+0000 7f387253b740 20 client.17010 populate_metadata read hostname 'smithi139' 2024-09-16T01:55:02.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.594+0000 7f387253b740 1 --2- 172.21.15.139:0/3418864229 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55711aa5c450 0x55711aa5e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.594+0000 7f387253b740 4 client.17010 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:55:02.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.594+0000 7f387253b740 1 -- 172.21.15.139:0/3418864229 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55711a9a2b90 con 0x55711aa5c450 2024-09-16T01:55:02.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.599+0000 7f386d6fd640 1 --2- 172.21.15.139:0/3418864229 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55711aa5c450 0x55711aa5e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.600+0000 7f386d6fd640 1 --2- 172.21.15.139:0/3418864229 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55711aa5c450 0x55711aa5e890 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=634c85b5dc8c3510 server_cookie=9f3939b8cf16c958 in_seq=0 out_seq=0 2024-09-16T01:55:02.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.601+0000 7f38517fa640 10 client.17010 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:02.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.602+0000 7f38517fa640 1 -- 172.21.15.139:0/3418864229 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55711a9a2b90 con 0x55711aa5c450 2024-09-16T01:55:02.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.602+0000 7f38517fa640 10 client.17010 handle_command_reply: tid=0 2024-09-16T01:55:02.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.602+0000 7f387253b740 1 client.17010 shutdown 2024-09-16T01:55:02.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.602+0000 7f387253b740 1 -- 172.21.15.139:0/3418864229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa30980 msgr2=0x55711aa386f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:02.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.602+0000 7f387253b740 1 --2- 172.21.15.139:0/3418864229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa30980 0x55711aa386f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f386402a480 tx=0x7f3864032f90 comp rx=0 tx=0).stop 2024-09-16T01:55:02.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.602+0000 7f387253b740 1 -- 172.21.15.139:0/3418864229 shutdown_connections 2024-09-16T01:55:02.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.602+0000 7f387253b740 1 --2- 172.21.15.139:0/3418864229 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55711aa5c450 0x55711aa5e890 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.602+0000 7f387253b740 1 --2- 172.21.15.139:0/3418864229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa3b240 0x55711aa38c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.602+0000 7f387253b740 1 --2- 172.21.15.139:0/3418864229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa30980 0x55711aa386f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.602+0000 7f387253b740 1 --2- 172.21.15.139:0/3418864229 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30060 0x55711aa381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.602+0000 7f387253b740 1 -- 172.21.15.139:0/3418864229 >> 172.21.15.139:0/3418864229 conn(0x55711aa2c5f0 msgr2=0x55711aa32320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:02.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.603+0000 7f38517fa640 0 client.17010 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:02.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.603+0000 7f387253b740 1 -- 172.21.15.139:0/3418864229 shutdown_connections 2024-09-16T01:55:02.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.603+0000 7f387253b740 1 -- 172.21.15.139:0/3418864229 wait complete. 2024-09-16T01:55:02.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.604+0000 7f387253b740 20 client.17010 trim_cache size 0 max 16384 2024-09-16T01:55:02.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.636+0000 7f387253b740 1 Processor -- start 2024-09-16T01:55:02.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.636+0000 7f387253b740 1 -- start start 2024-09-16T01:55:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.636+0000 7f387253b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa30060 0x55711aa3a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f387253b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30980 0x55711aa64160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f386d6fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa30060 0x55711aa3a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f386d6fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa30060 0x55711aa3a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:55414/0 (socket says 172.21.15.139:55414) 2024-09-16T01:55:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f387253b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa3b240 0x55711aa646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f386d6fd640 1 -- 172.21.15.139:0/1682356339 learned_addr learned my addr 172.21.15.139:0/1682356339 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f387253b740 1 -- 172.21.15.139:0/1682356339 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f384c010100 con 0x55711aa3b240 2024-09-16T01:55:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f387253b740 1 -- 172.21.15.139:0/1682356339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f384c011f40 con 0x55711aa30060 2024-09-16T01:55:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f386defe640 1 --2- 172.21.15.139:0/1682356339 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30980 0x55711aa64160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f387253b740 1 -- 172.21.15.139:0/1682356339 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f384c010890 con 0x55711aa30980 2024-09-16T01:55:02.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f386d6fd640 1 --2- 172.21.15.139:0/1682356339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa3b240 0x55711aa646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f386d6fd640 1 -- 172.21.15.139:0/1682356339 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30980 msgr2=0x55711aa64160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f386d6fd640 1 --2- 172.21.15.139:0/1682356339 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30980 0x55711aa64160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f386d6fd640 1 -- 172.21.15.139:0/1682356339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa3b240 msgr2=0x55711aa646a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f386d6fd640 1 --2- 172.21.15.139:0/1682356339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa3b240 0x55711aa646a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.637+0000 7f386d6fd640 1 -- 172.21.15.139:0/1682356339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55711aa2fe90 con 0x55711aa30060 2024-09-16T01:55:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.638+0000 7f386d6fd640 1 --2- 172.21.15.139:0/1682356339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa30060 0x55711aa3a120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f38580046b0 tx=0x7f38580043f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55b5364d812cb96a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.638+0000 7f38537fe640 1 -- 172.21.15.139:0/1682356339 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858034020 con 0x55711aa30060 2024-09-16T01:55:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.638+0000 7f38537fe640 1 -- 172.21.15.139:0/1682356339 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f385802b030 con 0x55711aa30060 2024-09-16T01:55:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.638+0000 7f38537fe640 1 -- 172.21.15.139:0/1682356339 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38580139b0 con 0x55711aa30060 2024-09-16T01:55:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.639+0000 7f387253b740 1 -- 172.21.15.139:0/1682356339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa30060 msgr2=0x55711aa3a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.639+0000 7f387253b740 1 --2- 172.21.15.139:0/1682356339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa30060 0x55711aa3a120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f38580046b0 tx=0x7f38580043f0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.642+0000 7f387253b740 1 -- 172.21.15.139:0/1682356339 shutdown_connections 2024-09-16T01:55:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.642+0000 7f387253b740 1 --2- 172.21.15.139:0/1682356339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa3b240 0x55711aa646a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.642+0000 7f387253b740 1 --2- 172.21.15.139:0/1682356339 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30980 0x55711aa64160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.642+0000 7f387253b740 1 --2- 172.21.15.139:0/1682356339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa30060 0x55711aa3a120 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.642+0000 7f387253b740 1 -- 172.21.15.139:0/1682356339 >> 172.21.15.139:0/1682356339 conn(0x55711aa2c5f0 msgr2=0x55711aa39a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:02.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.642+0000 7f387253b740 1 -- 172.21.15.139:0/1682356339 shutdown_connections 2024-09-16T01:55:02.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.643+0000 7f387253b740 1 -- 172.21.15.139:0/1682356339 wait complete. 2024-09-16T01:55:02.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.643+0000 7f387253b740 1 -- start start 2024-09-16T01:55:02.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.644+0000 7f387253b740 1 Processor -- start 2024-09-16T01:55:02.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.645+0000 7f387253b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa30060 0x55711aa35380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.645+0000 7f387253b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30980 0x55711aa358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.645+0000 7f386d6fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa30060 0x55711aa35380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.645+0000 7f386d6fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa30060 0x55711aa35380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:55428/0 (socket says 172.21.15.139:55428) 2024-09-16T01:55:02.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.645+0000 7f386d6fd640 1 -- 172.21.15.139:0/2497800670 learned_addr learned my addr 172.21.15.139:0/2497800670 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:02.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.645+0000 7f387253b740 1 --2- 172.21.15.139:0/2497800670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa3b240 0x55711aa35e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.645+0000 7f387253b740 1 -- 172.21.15.139:0/2497800670 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f384c0117b0 con 0x55711aa3b240 2024-09-16T01:55:02.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.646+0000 7f387253b740 1 -- 172.21.15.139:0/2497800670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f384c011020 con 0x55711aa30060 2024-09-16T01:55:02.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.646+0000 7f387253b740 1 -- 172.21.15.139:0/2497800670 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f384c00e0e0 con 0x55711aa30980 2024-09-16T01:55:02.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.646+0000 7f387253b740 10 client.0 _ll_register_callbacks cb 0x7f3864032c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:55:02.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.647+0000 7f386defe640 1 --2- 172.21.15.139:0/2497800670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30980 0x55711aa358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.649+0000 7f386d6fd640 1 --2- 172.21.15.139:0/2497800670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa3b240 0x55711aa35e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.649+0000 7f386d6fd640 1 -- 172.21.15.139:0/2497800670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30980 msgr2=0x55711aa358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:02.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.649+0000 7f386d6fd640 1 --2- 172.21.15.139:0/2497800670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30980 0x55711aa358c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.649+0000 7f386d6fd640 1 -- 172.21.15.139:0/2497800670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa3b240 msgr2=0x55711aa35e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:02.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.649+0000 7f386d6fd640 1 --2- 172.21.15.139:0/2497800670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa3b240 0x55711aa35e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:02.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.649+0000 7f386d6fd640 1 -- 172.21.15.139:0/2497800670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55711aac6290 con 0x55711aa30060 2024-09-16T01:55:02.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.654+0000 7f386d6fd640 1 --2- 172.21.15.139:0/2497800670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa30060 0x55711aa35380 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3858000ea0 tx=0x7f3858006c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10735bf9c969d6cb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:02.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.654+0000 7f38517fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:55:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.655+0000 7f38517fa640 1 -- 172.21.15.139:0/2497800670 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858034040 con 0x55711aa30060 2024-09-16T01:55:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.655+0000 7f38517fa640 1 -- 172.21.15.139:0/2497800670 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9630+0+0 (secure 0 0 0) 0x7f38580136f0 con 0x55711aa30060 2024-09-16T01:55:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.655+0000 7f387253b740 1 -- 172.21.15.139:0/2497800670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f385802b030 con 0x55711aa30060 2024-09-16T01:55:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.655+0000 7f38517fa640 1 -- 172.21.15.139:0/2497800670 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f385804b050 con 0x55711aa30060 2024-09-16T01:55:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.655+0000 7f387253b740 10 client.26459 fetch_fsmap learned FSMap version 71 2024-09-16T01:55:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.655+0000 7f387253b740 1 -- 172.21.15.139:0/2497800670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55711aac7590 con 0x55711aa30060 2024-09-16T01:55:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.656+0000 7f38517fa640 1 -- 172.21.15.139:0/2497800670 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f385802b030 con 0x55711aa30060 2024-09-16T01:55:02.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.656+0000 7f38517fa640 1 -- 172.21.15.139:0/2497800670 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f3858041030 con 0x55711aa30060 2024-09-16T01:55:02.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.656+0000 7f387253b740 10 client.26459 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:55:02.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.656+0000 7f387253b740 10 client.26459 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:55:02.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.656+0000 7f387253b740 20 client.26459 populate_metadata read hostname 'smithi139' 2024-09-16T01:55:02.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.656+0000 7f387253b740 1 --2- 172.21.15.139:0/2497800670 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55711aac7990 0x55711aac9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:02.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.656+0000 7f387253b740 4 client.26459 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "724cf2d2", "release": true, "await": true}] 2024-09-16T01:55:02.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.656+0000 7f387253b740 1 -- 172.21.15.139:0/2497800670 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "724cf2d2", "release": true, "await": true}) -- 0x55711a99d0a0 con 0x55711aac7990 2024-09-16T01:55:02.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.660+0000 7f386defe640 1 --2- 172.21.15.139:0/2497800670 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55711aac7990 0x55711aac9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:02.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.667+0000 7f38517fa640 10 client.26459 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:02.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:02.667+0000 7f386defe640 1 --2- 172.21.15.139:0/2497800670 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55711aac7990 0x55711aac9db0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d3154c1aea432dd server_cookie=b73dd00e19581732 in_seq=0 out_seq=0 2024-09-16T01:55:03.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.243+0000 7f38517fa640 1 -- 172.21.15.139:0/2497800670 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55711a99d0a0 con 0x55711aac7990 2024-09-16T01:55:03.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.243+0000 7f38517fa640 10 client.26459 handle_command_reply: tid=0 2024-09-16T01:55:03.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.243+0000 7f387253b740 1 client.26459 shutdown 2024-09-16T01:55:03.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.244+0000 7f387253b740 1 -- 172.21.15.139:0/2497800670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa30060 msgr2=0x55711aa35380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:03.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.244+0000 7f387253b740 1 --2- 172.21.15.139:0/2497800670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa30060 0x55711aa35380 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3858000ea0 tx=0x7f3858006c80 comp rx=0 tx=0).stop 2024-09-16T01:55:03.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.244+0000 7f387253b740 1 -- 172.21.15.139:0/2497800670 shutdown_connections 2024-09-16T01:55:03.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.244+0000 7f387253b740 1 --2- 172.21.15.139:0/2497800670 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55711aac7990 0x55711aac9db0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:03.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.244+0000 7f387253b740 1 --2- 172.21.15.139:0/2497800670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55711aa3b240 0x55711aa35e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:03.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.244+0000 7f387253b740 1 --2- 172.21.15.139:0/2497800670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711aa30980 0x55711aa358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:03.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.244+0000 7f387253b740 1 --2- 172.21.15.139:0/2497800670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55711aa30060 0x55711aa35380 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:03.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.244+0000 7f387253b740 1 -- 172.21.15.139:0/2497800670 >> 172.21.15.139:0/2497800670 conn(0x55711aa2c5f0 msgr2=0x55711aa32320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:03.247 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:55:03.248 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 71, 2024-09-16T01:55:03.248 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:55:03.248 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 52, 2024-09-16T01:55:03.248 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:55:03.248 INFO:teuthology.orchestra.run.smithi139.stdout: "724cf2d2": { 2024-09-16T01:55:03.248 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 52, 2024-09-16T01:55:03.248 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:55:03.248 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:55:03.248 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:55:03.248 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:55:03.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:55:03.248 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:55:03.248 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 146.5, 2024-09-16T01:55:03.249 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:55:03.249 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:55:03.249 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:55:03.249 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:55:03.249 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:55:03.249 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:55:03.249 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:55:03.249 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:55:03.249 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:55:03.249 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:55:03.249 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:55:03.249 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:55:03.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.245+0000 7f38517fa640 0 client.26459 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:03.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.245+0000 7f387253b740 1 -- 172.21.15.139:0/2497800670 shutdown_connections 2024-09-16T01:55:03.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.245+0000 7f387253b740 1 -- 172.21.15.139:0/2497800670 wait complete. 2024-09-16T01:55:03.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.245+0000 7f387253b740 20 client.26459 trim_cache size 0 max 16384 2024-09-16T01:55:03.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.246+0000 7f3870162640 1 -- 172.21.15.139:0/159069103 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f38540a35c0 msgr2=0x7f38540a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:03.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.246+0000 7f3870162640 1 --2- 172.21.15.139:0/159069103 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f38540a35c0 0x7f38540a5a80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3860009500 tx=0x7f3860009470 comp rx=0 tx=0).stop 2024-09-16T01:55:03.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.246+0000 7f3870162640 1 -- 172.21.15.139:0/159069103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3868106a90 msgr2=0x7f386810ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:03.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.246+0000 7f3870162640 1 --2- 172.21.15.139:0/159069103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3868106a90 0x7f386810ca70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f386000b990 tx=0x7f386000be50 comp rx=0 tx=0).stop 2024-09-16T01:55:03.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.246+0000 7f3870162640 1 -- 172.21.15.139:0/159069103 shutdown_connections 2024-09-16T01:55:03.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.246+0000 7f3870162640 1 --2- 172.21.15.139:0/159069103 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f38540a35c0 0x7f38540a5a80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:03.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.246+0000 7f3870162640 1 --2- 172.21.15.139:0/159069103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3868106a90 0x7f386810ca70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:03.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.246+0000 7f3870162640 1 --2- 172.21.15.139:0/159069103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3868101430 0x7f38681a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:03.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.246+0000 7f3870162640 1 --2- 172.21.15.139:0/159069103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3868100a90 0x7f38681a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:03.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.246+0000 7f3870162640 1 -- 172.21.15.139:0/159069103 >> 172.21.15.139:0/159069103 conn(0x7f38680fc7d0 msgr2=0x7f38680fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:03.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.246+0000 7f3870162640 1 -- 172.21.15.139:0/159069103 shutdown_connections 2024-09-16T01:55:03.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:03.246+0000 7f3870162640 1 -- 172.21.15.139:0/159069103 wait complete. 2024-09-16T01:55:03.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:03 smithi139 ceph-mon[31175]: pgmap v462: 129 pgs: 14 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 65 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 103 op/s 2024-09-16T01:55:03.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:03 smithi176 ceph-mon[35324]: pgmap v462: 129 pgs: 14 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 65 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 103 op/s 2024-09-16T01:55:04.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:03 smithi164 ceph-mon[35121]: pgmap v462: 129 pgs: 14 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 65 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 103 op/s 2024-09-16T01:55:05.197 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '724cf2d2', total seconds elapsed: 35.2 2024-09-16T01:55:05.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:05 smithi139 ceph-mon[31175]: pgmap v463: 129 pgs: 14 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 65 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 50 KiB/s wr, 81 op/s 2024-09-16T01:55:05.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:05 smithi176 ceph-mon[35324]: pgmap v463: 129 pgs: 14 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 65 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 50 KiB/s wr, 81 op/s 2024-09-16T01:55:06.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:05 smithi164 ceph-mon[35121]: pgmap v463: 129 pgs: 14 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 65 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 50 KiB/s wr, 81 op/s 2024-09-16T01:55:07.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:07 smithi139 ceph-mon[31175]: pgmap v464: 129 pgs: 14 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 66 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 368 KiB/s wr, 70 op/s 2024-09-16T01:55:07.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:07 smithi176 ceph-mon[35324]: pgmap v464: 129 pgs: 14 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 66 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 368 KiB/s wr, 70 op/s 2024-09-16T01:55:08.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:07 smithi164 ceph-mon[35121]: pgmap v464: 129 pgs: 14 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 66 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 368 KiB/s wr, 70 op/s 2024-09-16T01:55:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:09 smithi139 ceph-mon[31175]: pgmap v465: 129 pgs: 14 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 66 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 459 KiB/s wr, 79 op/s 2024-09-16T01:55:09.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:09 smithi139 ceph-mon[31175]: osdmap e107: 12 total, 12 up, 12 in 2024-09-16T01:55:09.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:09 smithi176 ceph-mon[35324]: pgmap v465: 129 pgs: 14 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 66 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 459 KiB/s wr, 79 op/s 2024-09-16T01:55:09.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:09 smithi176 ceph-mon[35324]: osdmap e107: 12 total, 12 up, 12 in 2024-09-16T01:55:10.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:09 smithi164 ceph-mon[35121]: pgmap v465: 129 pgs: 14 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 66 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 459 KiB/s wr, 79 op/s 2024-09-16T01:55:10.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:09 smithi164 ceph-mon[35121]: osdmap e107: 12 total, 12 up, 12 in 2024-09-16T01:55:11.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:11 smithi139 ceph-mon[31175]: pgmap v467: 129 pgs: 14 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 66 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 574 KiB/s wr, 105 op/s 2024-09-16T01:55:11.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:11 smithi176 ceph-mon[35324]: pgmap v467: 129 pgs: 14 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 66 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 574 KiB/s wr, 105 op/s 2024-09-16T01:55:12.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:11 smithi164 ceph-mon[35121]: pgmap v467: 129 pgs: 14 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 66 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 574 KiB/s wr, 105 op/s 2024-09-16T01:55:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.227+0000 7fc452674640 1 -- 172.21.15.139:0/1488668335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44c11b140 msgr2=0x7fc44c11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.227+0000 7fc452674640 1 --2- 172.21.15.139:0/1488668335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44c11b140 0x7fc44c11d530 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009860 tx=0x7fc43c02a380 comp rx=0 tx=0).stop 2024-09-16T01:55:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.234+0000 7fc452674640 1 -- 172.21.15.139:0/1488668335 shutdown_connections 2024-09-16T01:55:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.234+0000 7fc452674640 1 --2- 172.21.15.139:0/1488668335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44c11b140 0x7fc44c11d530 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.234+0000 7fc452674640 1 --2- 172.21.15.139:0/1488668335 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc44c1095d0 0x7fc44c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.234+0000 7fc452674640 1 --2- 172.21.15.139:0/1488668335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc44c1091f0 0x7fc44c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.234+0000 7fc452674640 1 -- 172.21.15.139:0/1488668335 >> 172.21.15.139:0/1488668335 conn(0x7fc44c06ce00 msgr2=0x7fc44c06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.236+0000 7fc452674640 1 -- 172.21.15.139:0/1488668335 shutdown_connections 2024-09-16T01:55:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.238+0000 7fc452674640 1 -- 172.21.15.139:0/1488668335 wait complete. 2024-09-16T01:55:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.239+0000 7fc452674640 1 Processor -- start 2024-09-16T01:55:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.239+0000 7fc452674640 1 -- start start 2024-09-16T01:55:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.248+0000 7fc452674640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44c1091f0 0x7fc44c1a7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.248+0000 7fc452674640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc44c1095d0 0x7fc44c1a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.248+0000 7fc452674640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc44c11b140 0x7fc44c1ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.248+0000 7fc452674640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc44c120ca0 con 0x7fc44c1091f0 2024-09-16T01:55:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.248+0000 7fc452674640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc44c120b20 con 0x7fc44c11b140 2024-09-16T01:55:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.248+0000 7fc452674640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc44c120e20 con 0x7fc44c1095d0 2024-09-16T01:55:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.252+0000 7fc44bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44c1091f0 0x7fc44c1a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.252+0000 7fc44bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44c1091f0 0x7fc44c1a7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54326/0 (socket says 172.21.15.139:54326) 2024-09-16T01:55:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.252+0000 7fc44bfff640 1 -- 172.21.15.139:0/2905900655 learned_addr learned my addr 172.21.15.139:0/2905900655 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.253+0000 7fc450c11640 1 --2- 172.21.15.139:0/2905900655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc44c11b140 0x7fc44c1ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.253+0000 7fc44b7fe640 1 --2- 172.21.15.139:0/2905900655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc44c1095d0 0x7fc44c1a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.253+0000 7fc44bfff640 1 -- 172.21.15.139:0/2905900655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc44c1095d0 msgr2=0x7fc44c1a75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.253+0000 7fc44bfff640 1 --2- 172.21.15.139:0/2905900655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc44c1095d0 0x7fc44c1a75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.253+0000 7fc44bfff640 1 -- 172.21.15.139:0/2905900655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc44c11b140 msgr2=0x7fc44c1ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.253+0000 7fc44bfff640 1 --2- 172.21.15.139:0/2905900655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc44c11b140 0x7fc44c1ab950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.253+0000 7fc44bfff640 1 -- 172.21.15.139:0/2905900655 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44c1ac080 con 0x7fc44c1091f0 2024-09-16T01:55:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.254+0000 7fc44bfff640 1 --2- 172.21.15.139:0/2905900655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44c1091f0 0x7fc44c1a7090 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc44000d950 tx=0x7fc44000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.257+0000 7fc4497fa640 1 -- 172.21.15.139:0/2905900655 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440017070 con 0x7fc44c1091f0 2024-09-16T01:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.257+0000 7fc452674640 1 -- 172.21.15.139:0/2905900655 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44c1ac370 con 0x7fc44c1091f0 2024-09-16T01:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.257+0000 7fc452674640 1 -- 172.21.15.139:0/2905900655 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc44c1b3c00 con 0x7fc44c1091f0 2024-09-16T01:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.257+0000 7fc4497fa640 1 -- 172.21.15.139:0/2905900655 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc440009c30 con 0x7fc44c1091f0 2024-09-16T01:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.257+0000 7fc4497fa640 1 -- 172.21.15.139:0/2905900655 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44000f690 con 0x7fc44c1091f0 2024-09-16T01:55:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.261+0000 7fc455415740 1 Processor -- start 2024-09-16T01:55:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.262+0000 7fc455415740 1 -- start start 2024-09-16T01:55:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.262+0000 7fc455415740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b165cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.262+0000 7fc455415740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b165d2d0 0x5575b1678780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.262+0000 7fc455415740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b1678cc0 0x5575b167b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.262+0000 7fc455415740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5575b15c12f0 con 0x5575b165c9b0 2024-09-16T01:55:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.262+0000 7fc455415740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5575b1569270 con 0x5575b165d2d0 2024-09-16T01:55:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.262+0000 7fc455415740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5575b15bd2b0 con 0x5575b1678cc0 2024-09-16T01:55:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.262+0000 7fc450c11640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b165cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.262+0000 7fc450c11640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b165cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54330/0 (socket says 172.21.15.139:54330) 2024-09-16T01:55:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.262+0000 7fc450c11640 1 -- 172.21.15.139:0/3336799220 learned_addr learned my addr 172.21.15.139:0/3336799220 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.264+0000 7fc4497fa640 1 -- 172.21.15.139:0/2905900655 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc44000f830 con 0x7fc44c1091f0 2024-09-16T01:55:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.264+0000 7fc44b7fe640 1 --2- 172.21.15.139:0/3336799220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b1678cc0 0x5575b167b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.265+0000 7fc4497fa640 1 --2- 172.21.15.139:0/2905900655 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc4340a3690 0x7fc4340a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.265+0000 7fc4497fa640 1 -- 172.21.15.139:0/2905900655 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9754+0+0 (secure 0 0 0) 0x7fc4400cf8b0 con 0x7fc44c1091f0 2024-09-16T01:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.267+0000 7fc450c11640 1 -- 172.21.15.139:0/3336799220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b1678cc0 msgr2=0x5575b167b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.267+0000 7fc450c11640 1 --2- 172.21.15.139:0/3336799220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b1678cc0 0x5575b167b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.267+0000 7fc450c11640 1 -- 172.21.15.139:0/3336799220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b165d2d0 msgr2=0x5575b1678780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.267+0000 7fc450c11640 1 --2- 172.21.15.139:0/3336799220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b165d2d0 0x5575b1678780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.267+0000 7fc450c11640 1 -- 172.21.15.139:0/3336799220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575b1568340 con 0x5575b165c9b0 2024-09-16T01:55:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.271+0000 7fc450c11640 1 --2- 172.21.15.139:0/3336799220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b165cd90 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc43c035040 tx=0x7fc43c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff86d591dc474a72 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.271+0000 7fc450c11640 1 --2- 172.21.15.139:0/2905900655 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc4340a3690 0x7fc4340a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.272+0000 7fc4327fc640 1 -- 172.21.15.139:0/3336799220 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c041070 con 0x5575b165c9b0 2024-09-16T01:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.272+0000 7fc4327fc640 1 -- 172.21.15.139:0/3336799220 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc43c030780 con 0x5575b165c9b0 2024-09-16T01:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.272+0000 7fc4327fc640 1 -- 172.21.15.139:0/3336799220 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c037040 con 0x5575b165c9b0 2024-09-16T01:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.273+0000 7fc455415740 1 -- 172.21.15.139:0/3336799220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 msgr2=0x5575b165cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.273+0000 7fc455415740 1 --2- 172.21.15.139:0/3336799220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b165cd90 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc43c035040 tx=0x7fc43c009540 comp rx=0 tx=0).stop 2024-09-16T01:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.274+0000 7fc455415740 1 -- 172.21.15.139:0/3336799220 shutdown_connections 2024-09-16T01:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.274+0000 7fc455415740 1 --2- 172.21.15.139:0/3336799220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b1678cc0 0x5575b167b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.274+0000 7fc455415740 1 --2- 172.21.15.139:0/3336799220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b165d2d0 0x5575b1678780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.274+0000 7fc455415740 1 --2- 172.21.15.139:0/3336799220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b165cd90 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.274+0000 7fc455415740 1 -- 172.21.15.139:0/3336799220 >> 172.21.15.139:0/3336799220 conn(0x5575b1659060 msgr2=0x5575b1659440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.275+0000 7fc450c11640 1 --2- 172.21.15.139:0/2905900655 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc4340a3690 0x7fc4340a5b50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc43c005b60 tx=0x7fc43c003f00 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.278+0000 7fc455415740 1 -- 172.21.15.139:0/3336799220 shutdown_connections 2024-09-16T01:55:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.280+0000 7fc455415740 1 -- 172.21.15.139:0/3336799220 wait complete. 2024-09-16T01:55:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.280+0000 7fc455415740 1 -- start start 2024-09-16T01:55:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.280+0000 7fc455415740 1 Processor -- start 2024-09-16T01:55:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.281+0000 7fc455415740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b1675be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.281+0000 7fc455415740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b165d2d0 0x5575b1676120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.281+0000 7fc455415740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1676660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.281+0000 7fc455415740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5575b167b830 con 0x5575b165c9b0 2024-09-16T01:55:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.281+0000 7fc455415740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5575b154a290 con 0x5575b1678cc0 2024-09-16T01:55:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.281+0000 7fc455415740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5575b167b9b0 con 0x5575b165d2d0 2024-09-16T01:55:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.281+0000 7fc455415740 10 client.0 _ll_register_callbacks cb 0x5575b157b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:55:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.281+0000 7fc450c11640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1676660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.281+0000 7fc450c11640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1676660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:48704/0 (socket says 172.21.15.139:48704) 2024-09-16T01:55:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.281+0000 7fc450c11640 1 -- 172.21.15.139:0/2993926628 learned_addr learned my addr 172.21.15.139:0/2993926628 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.281+0000 7fc44bfff640 1 --2- 172.21.15.139:0/2993926628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b1675be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.282+0000 7fc450c11640 1 -- 172.21.15.139:0/2993926628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b165d2d0 msgr2=0x5575b1676120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:55:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.282+0000 7fc450c11640 1 --2- 172.21.15.139:0/2993926628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b165d2d0 0x5575b1676120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.282+0000 7fc450c11640 1 -- 172.21.15.139:0/2993926628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 msgr2=0x5575b1675be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.282+0000 7fc450c11640 1 --2- 172.21.15.139:0/2993926628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b1675be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.282+0000 7fc450c11640 1 -- 172.21.15.139:0/2993926628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575b12d4210 con 0x5575b1678cc0 2024-09-16T01:55:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.282+0000 7fc450c11640 1 --2- 172.21.15.139:0/2993926628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1676660 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009780 tx=0x7fc43c041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eba0f24d78a19ee server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.282+0000 7fc42b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:55:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.282+0000 7fc42b7fe640 1 -- 172.21.15.139:0/2993926628 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c037040 con 0x5575b1678cc0 2024-09-16T01:55:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.283+0000 7fc455415740 1 -- 172.21.15.139:0/2993926628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc43c030780 con 0x5575b1678cc0 2024-09-16T01:55:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.283+0000 7fc42b7fe640 1 -- 172.21.15.139:0/2993926628 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9754+0+0 (secure 0 0 0) 0x7fc43c003ac0 con 0x5575b1678cc0 2024-09-16T01:55:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.283+0000 7fc42b7fe640 1 -- 172.21.15.139:0/2993926628 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fc43c004040 con 0x5575b1678cc0 2024-09-16T01:55:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.284+0000 7fc455415740 10 client.26468 fetch_fsmap learned FSMap version 71 2024-09-16T01:55:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.284+0000 7fc455415740 1 -- 172.21.15.139:0/2993926628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5575b13a0620 con 0x5575b1678cc0 2024-09-16T01:55:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.284+0000 7fc42b7fe640 1 -- 172.21.15.139:0/2993926628 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc43c030780 con 0x5575b1678cc0 2024-09-16T01:55:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.285+0000 7fc42b7fe640 1 -- 172.21.15.139:0/2993926628 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7fc43c04f030 con 0x5575b1678cc0 2024-09-16T01:55:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.285+0000 7fc455415740 10 client.26468 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:55:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.285+0000 7fc455415740 10 client.26468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:55:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.285+0000 7fc455415740 20 client.26468 populate_metadata read hostname 'smithi139' 2024-09-16T01:55:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.285+0000 7fc455415740 1 --2- 172.21.15.139:0/2993926628 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5575b1689030 0x5575b168b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.285+0000 7fc455415740 4 client.26468 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:55:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.285+0000 7fc455415740 1 -- 172.21.15.139:0/2993926628 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575b15cea20 con 0x5575b1689030 2024-09-16T01:55:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.285+0000 7fc44bfff640 1 --2- 172.21.15.139:0/2993926628 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5575b1689030 0x5575b168b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.286+0000 7fc42b7fe640 10 client.26468 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.287+0000 7fc44bfff640 1 --2- 172.21.15.139:0/2993926628 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5575b1689030 0x5575b168b4b0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39600714b9638fe server_cookie=e284c51aeb4143a5 in_seq=0 out_seq=0 2024-09-16T01:55:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.291+0000 7fc42b7fe640 1 -- 172.21.15.139:0/2993926628 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575b15cea20 con 0x5575b1689030 2024-09-16T01:55:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.291+0000 7fc42b7fe640 10 client.26468 handle_command_reply: tid=0 2024-09-16T01:55:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.292+0000 7fc455415740 1 client.26468 shutdown 2024-09-16T01:55:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.292+0000 7fc455415740 1 -- 172.21.15.139:0/2993926628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 msgr2=0x5575b1676660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.292+0000 7fc455415740 1 --2- 172.21.15.139:0/2993926628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1676660 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009780 tx=0x7fc43c041000 comp rx=0 tx=0).stop 2024-09-16T01:55:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.298+0000 7fc455415740 1 -- 172.21.15.139:0/2993926628 shutdown_connections 2024-09-16T01:55:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.298+0000 7fc455415740 1 --2- 172.21.15.139:0/2993926628 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5575b1689030 0x5575b168b4b0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.298+0000 7fc455415740 1 --2- 172.21.15.139:0/2993926628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1676660 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.298+0000 7fc455415740 1 --2- 172.21.15.139:0/2993926628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b165d2d0 0x5575b1676120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.298+0000 7fc455415740 1 --2- 172.21.15.139:0/2993926628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b1675be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.298+0000 7fc455415740 1 -- 172.21.15.139:0/2993926628 >> 172.21.15.139:0/2993926628 conn(0x5575b1659060 msgr2=0x5575b165ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.299+0000 7fc42b7fe640 0 client.26468 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.304+0000 7fc455415740 1 -- 172.21.15.139:0/2993926628 shutdown_connections 2024-09-16T01:55:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.308+0000 7fc455415740 1 -- 172.21.15.139:0/2993926628 wait complete. 2024-09-16T01:55:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.310+0000 7fc455415740 20 client.26468 trim_cache size 0 max 16384 2024-09-16T01:55:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.338+0000 7fc455415740 1 Processor -- start 2024-09-16T01:55:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.338+0000 7fc455415740 1 -- start start 2024-09-16T01:55:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.338+0000 7fc455415740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b1676490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.339+0000 7fc455415740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b165d2d0 0x5575b16580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.339+0000 7fc455415740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1690d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.339+0000 7fc455415740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4380101f0 con 0x5575b165c9b0 2024-09-16T01:55:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.339+0000 7fc455415740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc438012030 con 0x5575b1678cc0 2024-09-16T01:55:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.339+0000 7fc455415740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc438010980 con 0x5575b165d2d0 2024-09-16T01:55:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.339+0000 7fc44bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b1676490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.339+0000 7fc44bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b1676490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36474/0 (socket says 172.21.15.139:36474) 2024-09-16T01:55:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.339+0000 7fc44bfff640 1 -- 172.21.15.139:0/4009392232 learned_addr learned my addr 172.21.15.139:0/4009392232 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.340+0000 7fc44bfff640 1 -- 172.21.15.139:0/4009392232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b165d2d0 msgr2=0x5575b16580b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:55:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.340+0000 7fc450c11640 1 --2- 172.21.15.139:0/4009392232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1690d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.340+0000 7fc44bfff640 1 --2- 172.21.15.139:0/4009392232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b165d2d0 0x5575b16580b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.340+0000 7fc44bfff640 1 -- 172.21.15.139:0/4009392232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 msgr2=0x5575b1690d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.340+0000 7fc44bfff640 1 --2- 172.21.15.139:0/4009392232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1690d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.340+0000 7fc44bfff640 1 -- 172.21.15.139:0/4009392232 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575b165a180 con 0x5575b165c9b0 2024-09-16T01:55:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.340+0000 7fc44bfff640 1 --2- 172.21.15.139:0/4009392232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b1676490 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc440098030 tx=0x7fc44000dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370300ec70580597 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.342+0000 7fc431ffb640 1 -- 172.21.15.139:0/4009392232 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440020040 con 0x5575b165c9b0 2024-09-16T01:55:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.342+0000 7fc431ffb640 1 -- 172.21.15.139:0/4009392232 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4400a42d0 con 0x5575b165c9b0 2024-09-16T01:55:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.343+0000 7fc455415740 1 -- 172.21.15.139:0/4009392232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 msgr2=0x5575b1676490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.343+0000 7fc455415740 1 --2- 172.21.15.139:0/4009392232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b1676490 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc440098030 tx=0x7fc44000dfd0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.343+0000 7fc455415740 1 -- 172.21.15.139:0/4009392232 shutdown_connections 2024-09-16T01:55:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.343+0000 7fc455415740 1 --2- 172.21.15.139:0/4009392232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1690d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.343+0000 7fc455415740 1 --2- 172.21.15.139:0/4009392232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b165d2d0 0x5575b16580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.343+0000 7fc455415740 1 --2- 172.21.15.139:0/4009392232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b165c9b0 0x5575b1676490 secure :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc440098030 tx=0x7fc44000dfd0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.343+0000 7fc455415740 1 -- 172.21.15.139:0/4009392232 >> 172.21.15.139:0/4009392232 conn(0x5575b1659060 msgr2=0x5575b16579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.346+0000 7fc455415740 1 -- 172.21.15.139:0/4009392232 shutdown_connections 2024-09-16T01:55:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.346+0000 7fc455415740 1 -- 172.21.15.139:0/4009392232 wait complete. 2024-09-16T01:55:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.346+0000 7fc455415740 1 -- start start 2024-09-16T01:55:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.347+0000 7fc455415740 1 Processor -- start 2024-09-16T01:55:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.347+0000 7fc455415740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b165d2d0 0x5575b1681040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.347+0000 7fc455415740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1681580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.347+0000 7fc455415740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b167c4c0 0x5575b167c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.347+0000 7fc455415740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4380118a0 con 0x5575b167c4c0 2024-09-16T01:55:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.347+0000 7fc455415740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc438011110 con 0x5575b1678cc0 2024-09-16T01:55:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.347+0000 7fc455415740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc43800e1f0 con 0x5575b165d2d0 2024-09-16T01:55:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.347+0000 7fc455415740 10 client.0 _ll_register_callbacks cb 0x7fc444009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:55:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.347+0000 7fc44b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1681580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.347+0000 7fc44b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1681580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:48728/0 (socket says 172.21.15.139:48728) 2024-09-16T01:55:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.347+0000 7fc44b7fe640 1 -- 172.21.15.139:0/2050771436 learned_addr learned my addr 172.21.15.139:0/2050771436 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.348+0000 7fc450c11640 1 --2- 172.21.15.139:0/2050771436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b167c4c0 0x5575b167c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.348+0000 7fc44b7fe640 1 -- 172.21.15.139:0/2050771436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b165d2d0 msgr2=0x5575b1681040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.348+0000 7fc44b7fe640 1 --2- 172.21.15.139:0/2050771436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b165d2d0 0x5575b1681040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.348+0000 7fc44b7fe640 1 -- 172.21.15.139:0/2050771436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b167c4c0 msgr2=0x5575b167c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.348+0000 7fc44b7fe640 1 --2- 172.21.15.139:0/2050771436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b167c4c0 0x5575b167c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.348+0000 7fc44b7fe640 1 -- 172.21.15.139:0/2050771436 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575b16f2d50 con 0x5575b1678cc0 2024-09-16T01:55:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.348+0000 7fc44b7fe640 1 --2- 172.21.15.139:0/2050771436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1681580 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc44c11c8e0 tx=0x7fc44400ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b66982a6b85e0193 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.348+0000 7fc42b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:55:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.349+0000 7fc42b7fe640 1 -- 172.21.15.139:0/2050771436 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44400dd20 con 0x5575b1678cc0 2024-09-16T01:55:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.349+0000 7fc42b7fe640 1 -- 172.21.15.139:0/2050771436 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9754+0+0 (secure 0 0 0) 0x7fc444018420 con 0x5575b1678cc0 2024-09-16T01:55:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.349+0000 7fc455415740 1 -- 172.21.15.139:0/2050771436 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4400a42d0 con 0x5575b1678cc0 2024-09-16T01:55:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.350+0000 7fc42b7fe640 1 -- 172.21.15.139:0/2050771436 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fc444026050 con 0x5575b1678cc0 2024-09-16T01:55:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.350+0000 7fc455415740 10 client.26471 fetch_fsmap learned FSMap version 71 2024-09-16T01:55:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.350+0000 7fc455415740 1 -- 172.21.15.139:0/2050771436 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5575b165fa00 con 0x5575b1678cc0 2024-09-16T01:55:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.354+0000 7fc42b7fe640 1 -- 172.21.15.139:0/2050771436 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4400a42d0 con 0x5575b1678cc0 2024-09-16T01:55:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.354+0000 7fc42b7fe640 1 -- 172.21.15.139:0/2050771436 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7fc44400b040 con 0x5575b1678cc0 2024-09-16T01:55:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.354+0000 7fc455415740 10 client.26471 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:55:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.354+0000 7fc455415740 10 client.26471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:55:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.354+0000 7fc455415740 20 client.26471 populate_metadata read hostname 'smithi139' 2024-09-16T01:55:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.354+0000 7fc455415740 1 --2- 172.21.15.139:0/2050771436 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5575b16602f0 0x5575b1662710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.354+0000 7fc455415740 4 client.26471 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:55:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.354+0000 7fc455415740 1 -- 172.21.15.139:0/2050771436 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575b15c8d70 con 0x5575b16602f0 2024-09-16T01:55:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.355+0000 7fc44bfff640 1 --2- 172.21.15.139:0/2050771436 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5575b16602f0 0x5575b1662710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.367+0000 7fc44bfff640 1 --2- 172.21.15.139:0/2050771436 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5575b16602f0 0x5575b1662710 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89830c944f53b760 server_cookie=e4247e861f4ee0e3 in_seq=0 out_seq=0 2024-09-16T01:55:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.367+0000 7fc42b7fe640 10 client.26471 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.440+0000 7fc42b7fe640 1 -- 172.21.15.139:0/2050771436 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5575b15c8d70 con 0x5575b16602f0 2024-09-16T01:55:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.440+0000 7fc42b7fe640 10 client.26471 handle_command_reply: tid=0 2024-09-16T01:55:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.440+0000 7fc455415740 1 client.26471 shutdown 2024-09-16T01:55:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.441+0000 7fc455415740 1 -- 172.21.15.139:0/2050771436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 msgr2=0x5575b1681580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.441+0000 7fc455415740 1 --2- 172.21.15.139:0/2050771436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1681580 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc44c11c8e0 tx=0x7fc44400ee70 comp rx=0 tx=0).stop 2024-09-16T01:55:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.441+0000 7fc455415740 1 -- 172.21.15.139:0/2050771436 shutdown_connections 2024-09-16T01:55:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.441+0000 7fc455415740 1 --2- 172.21.15.139:0/2050771436 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5575b16602f0 0x5575b1662710 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.441+0000 7fc455415740 1 --2- 172.21.15.139:0/2050771436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575b167c4c0 0x5575b167c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.441+0000 7fc455415740 1 --2- 172.21.15.139:0/2050771436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5575b1678cc0 0x5575b1681580 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.441+0000 7fc42b7fe640 0 client.26471 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.441+0000 7fc455415740 1 --2- 172.21.15.139:0/2050771436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575b165d2d0 0x5575b1681040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.441+0000 7fc455415740 1 -- 172.21.15.139:0/2050771436 >> 172.21.15.139:0/2050771436 conn(0x5575b1659060 msgr2=0x5575b16579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.441+0000 7fc455415740 1 -- 172.21.15.139:0/2050771436 shutdown_connections 2024-09-16T01:55:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.441+0000 7fc455415740 1 -- 172.21.15.139:0/2050771436 wait complete. 2024-09-16T01:55:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.441+0000 7fc455415740 20 client.26471 trim_cache size 0 max 16384 2024-09-16T01:55:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:55:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:55:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:55:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:55:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.442+0000 7fc452674640 1 -- 172.21.15.139:0/2905900655 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc4340a3690 msgr2=0x7fc4340a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.442+0000 7fc452674640 1 --2- 172.21.15.139:0/2905900655 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc4340a3690 0x7fc4340a5b50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc43c005b60 tx=0x7fc43c003f00 comp rx=0 tx=0).stop 2024-09-16T01:55:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.442+0000 7fc452674640 1 -- 172.21.15.139:0/2905900655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44c1091f0 msgr2=0x7fc44c1a7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.442+0000 7fc452674640 1 --2- 172.21.15.139:0/2905900655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44c1091f0 0x7fc44c1a7090 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc44000d950 tx=0x7fc44000de10 comp rx=0 tx=0).stop 2024-09-16T01:55:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.442+0000 7fc452674640 1 -- 172.21.15.139:0/2905900655 shutdown_connections 2024-09-16T01:55:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.442+0000 7fc452674640 1 --2- 172.21.15.139:0/2905900655 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc4340a3690 0x7fc4340a5b50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.442+0000 7fc452674640 1 --2- 172.21.15.139:0/2905900655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc44c11b140 0x7fc44c1ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.442+0000 7fc452674640 1 --2- 172.21.15.139:0/2905900655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc44c1095d0 0x7fc44c1a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.442+0000 7fc452674640 1 --2- 172.21.15.139:0/2905900655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44c1091f0 0x7fc44c1a7090 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.442+0000 7fc452674640 1 -- 172.21.15.139:0/2905900655 >> 172.21.15.139:0/2905900655 conn(0x7fc44c06ce00 msgr2=0x7fc44c11b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.442+0000 7fc452674640 1 -- 172.21.15.139:0/2905900655 shutdown_connections 2024-09-16T01:55:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:12.442+0000 7fc452674640 1 -- 172.21.15.139:0/2905900655 wait complete. 2024-09-16T01:55:13.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:13 smithi176 ceph-mon[35324]: pgmap v468: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-16T01:55:14.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:13 smithi164 ceph-mon[35121]: pgmap v468: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-16T01:55:14.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:13 smithi139 ceph-mon[31175]: pgmap v468: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-16T01:55:15.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:15 smithi176 ceph-mon[35324]: pgmap v469: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-09-16T01:55:16.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:15 smithi164 ceph-mon[35121]: pgmap v469: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-09-16T01:55:16.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:15 smithi139 ceph-mon[31175]: pgmap v469: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-09-16T01:55:17.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:17 smithi176 ceph-mon[35324]: pgmap v470: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-09-16T01:55:18.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:17 smithi164 ceph-mon[35121]: pgmap v470: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-09-16T01:55:18.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:17 smithi139 ceph-mon[31175]: pgmap v470: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-09-16T01:55:20.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:19 smithi164 ceph-mon[35121]: pgmap v471: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-09-16T01:55:20.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:19 smithi139 ceph-mon[31175]: pgmap v471: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-09-16T01:55:20.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:19 smithi176 ceph-mon[35324]: pgmap v471: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-09-16T01:55:22.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:21 smithi164 ceph-mon[35121]: pgmap v472: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 210 op/s 2024-09-16T01:55:22.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:21 smithi139 ceph-mon[31175]: pgmap v472: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 210 op/s 2024-09-16T01:55:22.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:21 smithi176 ceph-mon[35324]: pgmap v472: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 210 op/s 2024-09-16T01:55:24.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:23 smithi164 ceph-mon[35121]: pgmap v473: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2024-09-16T01:55:24.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:23 smithi139 ceph-mon[31175]: pgmap v473: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2024-09-16T01:55:24.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:23 smithi176 ceph-mon[35324]: pgmap v473: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2024-09-16T01:55:24.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:55:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:55:24.208+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:55:24.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:55:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:55:24.208+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:55:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.590+0000 7fcaed1fd640 1 -- 172.21.15.139:0/3765490936 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcae810ea10 msgr2=0x7fcae8115430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.590+0000 7fcaed1fd640 1 --2- 172.21.15.139:0/3765490936 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcae810ea10 0x7fcae8115430 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009900 tx=0x7fcae002b690 comp rx=0 tx=0).stop 2024-09-16T01:55:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.591+0000 7fcaed1fd640 1 -- 172.21.15.139:0/3765490936 shutdown_connections 2024-09-16T01:55:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.591+0000 7fcaed1fd640 1 --2- 172.21.15.139:0/3765490936 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcae810ea10 0x7fcae8115430 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.591+0000 7fcaed1fd640 1 --2- 172.21.15.139:0/3765490936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcae8109e10 0x7fcae810e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.591+0000 7fcaed1fd640 1 --2- 172.21.15.139:0/3765490936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcae8109460 0x7fcae8109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.591+0000 7fcaed1fd640 1 -- 172.21.15.139:0/3765490936 >> 172.21.15.139:0/3765490936 conn(0x7fcae806d210 msgr2=0x7fcae806d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.593+0000 7fcaed1fd640 1 -- 172.21.15.139:0/3765490936 shutdown_connections 2024-09-16T01:55:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.594+0000 7fcaed1fd640 1 -- 172.21.15.139:0/3765490936 wait complete. 2024-09-16T01:55:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.595+0000 7fcaed1fd640 1 Processor -- start 2024-09-16T01:55:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.595+0000 7fcaed1fd640 1 -- start start 2024-09-16T01:55:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcaed1fd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcae8109460 0x7fcae81a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcaed1fd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcae810ea10 0x7fcae81a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcaed1fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcae819e630 0x7fcae819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcaed1fd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcae81185b0 con 0x7fcae8109460 2024-09-16T01:55:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcaed1fd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcae8118430 con 0x7fcae819e630 2024-09-16T01:55:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcaed1fd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcae8118730 con 0x7fcae810ea10 2024-09-16T01:55:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcae659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcae810ea10 0x7fcae81a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcae659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcae810ea10 0x7fcae81a5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:48416/0 (socket says 172.21.15.139:48416) 2024-09-16T01:55:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcae659c640 1 -- 172.21.15.139:0/3547516889 learned_addr learned my addr 172.21.15.139:0/3547516889 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcae6d9d640 1 --2- 172.21.15.139:0/3547516889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcae8109460 0x7fcae81a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcae659c640 1 -- 172.21.15.139:0/3547516889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcae819e630 msgr2=0x7fcae819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcae659c640 1 --2- 172.21.15.139:0/3547516889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcae819e630 0x7fcae819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcae659c640 1 -- 172.21.15.139:0/3547516889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcae8109460 msgr2=0x7fcae81a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcae659c640 1 --2- 172.21.15.139:0/3547516889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcae8109460 0x7fcae81a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.596+0000 7fcae659c640 1 -- 172.21.15.139:0/3547516889 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae819f210 con 0x7fcae810ea10 2024-09-16T01:55:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.597+0000 7fcae659c640 1 --2- 172.21.15.139:0/3547516889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcae810ea10 0x7fcae81a5aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00e9f0 tx=0x7fcadc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.597+0000 7fcad7fff640 1 -- 172.21.15.139:0/3547516889 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc00ccc0 con 0x7fcae810ea10 2024-09-16T01:55:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.597+0000 7fcae6d9d640 1 --2- 172.21.15.139:0/3547516889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcae8109460 0x7fcae81a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:55:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.597+0000 7fcaed1fd640 1 -- 172.21.15.139:0/3547516889 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae81a3080 con 0x7fcae810ea10 2024-09-16T01:55:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.597+0000 7fcaed1fd640 1 -- 172.21.15.139:0/3547516889 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae81a3620 con 0x7fcae810ea10 2024-09-16T01:55:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.600+0000 7fcaf007c740 1 Processor -- start 2024-09-16T01:55:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.601+0000 7fcaf007c740 1 -- start start 2024-09-16T01:55:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.602+0000 7fcaf007c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.602+0000 7fcaf007c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032f1410 0x55e5032fb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.602+0000 7fcaf007c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032fbcb0 0x55e5032fe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.602+0000 7fcaf007c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e5032552f0 con 0x55e5032fbcb0 2024-09-16T01:55:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.602+0000 7fcaf007c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e5031fd270 con 0x55e5032f0af0 2024-09-16T01:55:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.602+0000 7fcaf007c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e5032512b0 con 0x55e5032f1410 2024-09-16T01:55:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.602+0000 7fcae759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.602+0000 7fcae759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:53026/0 (socket says 172.21.15.139:53026) 2024-09-16T01:55:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.602+0000 7fcae759e640 1 -- 172.21.15.139:0/825439325 learned_addr learned my addr 172.21.15.139:0/825439325 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.602+0000 7fcad7fff640 1 -- 172.21.15.139:0/3547516889 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcadc010910 con 0x7fcae810ea10 2024-09-16T01:55:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.602+0000 7fcad7fff640 1 -- 172.21.15.139:0/3547516889 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0188f0 con 0x7fcae810ea10 2024-09-16T01:55:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.604+0000 7fcae659c640 1 --2- 172.21.15.139:0/825439325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032fbcb0 0x55e5032fe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.605+0000 7fcad7fff640 1 -- 172.21.15.139:0/3547516889 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fcadc018a90 con 0x7fcae810ea10 2024-09-16T01:55:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.605+0000 7fcae759e640 1 -- 172.21.15.139:0/825439325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032f1410 msgr2=0x55e5032fb770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:55:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.605+0000 7fcae759e640 1 --2- 172.21.15.139:0/825439325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032f1410 0x55e5032fb770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.605+0000 7fcae759e640 1 -- 172.21.15.139:0/825439325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032fbcb0 msgr2=0x55e5032fe0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.605+0000 7fcae759e640 1 --2- 172.21.15.139:0/825439325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032fbcb0 0x55e5032fe0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.606+0000 7fcae759e640 1 -- 172.21.15.139:0/825439325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5031fc340 con 0x55e5032f0af0 2024-09-16T01:55:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.606+0000 7fcad7fff640 1 --2- 172.21.15.139:0/3547516889 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcad00a3560 0x7fcad00a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.607+0000 7fcad7fff640 1 -- 172.21.15.139:0/3547516889 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9754+0+0 (secure 0 0 0) 0x7fcadc0d37e0 con 0x7fcae810ea10 2024-09-16T01:55:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.607+0000 7fcae6d9d640 1 --2- 172.21.15.139:0/3547516889 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcad00a3560 0x7fcad00a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.610+0000 7fcae6d9d640 1 --2- 172.21.15.139:0/3547516889 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcad00a3560 0x7fcad00a5a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55e5032edec0 tx=0x7fcad8008040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.611+0000 7fcae759e640 1 --2- 172.21.15.139:0/825439325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f0ed0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcae0004af0 tx=0x7fcae0005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2d69d54448f20af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.614+0000 7fcad57fa640 1 -- 172.21.15.139:0/825439325 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0040070 con 0x55e5032f0af0 2024-09-16T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.614+0000 7fcad57fa640 1 -- 172.21.15.139:0/825439325 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae00328e0 con 0x55e5032f0af0 2024-09-16T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.614+0000 7fcad57fa640 1 -- 172.21.15.139:0/825439325 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0004420 con 0x55e5032f0af0 2024-09-16T01:55:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.618+0000 7fcaf007c740 1 -- 172.21.15.139:0/825439325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 msgr2=0x55e5032f0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.619+0000 7fcaf007c740 1 --2- 172.21.15.139:0/825439325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f0ed0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcae0004af0 tx=0x7fcae0005f80 comp rx=0 tx=0).stop 2024-09-16T01:55:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.620+0000 7fcaf007c740 1 -- 172.21.15.139:0/825439325 shutdown_connections 2024-09-16T01:55:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.620+0000 7fcaf007c740 1 --2- 172.21.15.139:0/825439325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032fbcb0 0x55e5032fe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.620+0000 7fcaf007c740 1 --2- 172.21.15.139:0/825439325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032f1410 0x55e5032fb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.620+0000 7fcaf007c740 1 --2- 172.21.15.139:0/825439325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f0ed0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.620+0000 7fcaf007c740 1 -- 172.21.15.139:0/825439325 >> 172.21.15.139:0/825439325 conn(0x55e5032ed060 msgr2=0x55e5032ed440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.631+0000 7fcaf007c740 1 -- 172.21.15.139:0/825439325 shutdown_connections 2024-09-16T01:55:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.631+0000 7fcaf007c740 1 -- 172.21.15.139:0/825439325 wait complete. 2024-09-16T01:55:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.632+0000 7fcaf007c740 1 -- start start 2024-09-16T01:55:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.632+0000 7fcaf007c740 1 Processor -- start 2024-09-16T01:55:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.632+0000 7fcaf007c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.632+0000 7fcaf007c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032f1410 0x55e5032f91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.632+0000 7fcaf007c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032fbcb0 0x55e5032f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.632+0000 7fcaf007c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e5032fe880 con 0x55e5032fbcb0 2024-09-16T01:55:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.632+0000 7fcaf007c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e5031de290 con 0x55e5032f0af0 2024-09-16T01:55:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.632+0000 7fcaf007c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e5032fea00 con 0x55e5032f1410 2024-09-16T01:55:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.632+0000 7fcaf007c740 10 client.0 _ll_register_callbacks cb 0x55e50320f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:55:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.633+0000 7fcae6d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.633+0000 7fcae6d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:53028/0 (socket says 172.21.15.139:53028) 2024-09-16T01:55:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.633+0000 7fcae6d9d640 1 -- 172.21.15.139:0/3308415973 learned_addr learned my addr 172.21.15.139:0/3308415973 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.633+0000 7fcae659c640 1 --2- 172.21.15.139:0/3308415973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032f1410 0x55e5032f91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.633+0000 7fcae759e640 1 --2- 172.21.15.139:0/3308415973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032fbcb0 0x55e5032f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.633+0000 7fcae6d9d640 1 -- 172.21.15.139:0/3308415973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032f1410 msgr2=0x55e5032f91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.633+0000 7fcae6d9d640 1 --2- 172.21.15.139:0/3308415973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032f1410 0x55e5032f91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.633+0000 7fcae6d9d640 1 -- 172.21.15.139:0/3308415973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032fbcb0 msgr2=0x55e5032f96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.633+0000 7fcae6d9d640 1 --2- 172.21.15.139:0/3308415973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032fbcb0 0x55e5032f96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.633+0000 7fcae6d9d640 1 -- 172.21.15.139:0/3308415973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e502f68210 con 0x55e5032f0af0 2024-09-16T01:55:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.633+0000 7fcae6d9d640 1 --2- 172.21.15.139:0/3308415973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f8c70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcad800ce60 tx=0x7fcad80047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30668f3cf8f25394 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.634+0000 7fcace7fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:55:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.637+0000 7fcace7fc640 1 -- 172.21.15.139:0/3308415973 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8023060 con 0x55e5032f0af0 2024-09-16T01:55:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.638+0000 7fcace7fc640 1 -- 172.21.15.139:0/3308415973 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9754+0+0 (secure 0 0 0) 0x7fcad801e600 con 0x55e5032f0af0 2024-09-16T01:55:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.638+0000 7fcaf007c740 1 -- 172.21.15.139:0/3308415973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae00328e0 con 0x55e5032f0af0 2024-09-16T01:55:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.643+0000 7fcace7fc640 1 -- 172.21.15.139:0/3308415973 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fcad801e800 con 0x55e5032f0af0 2024-09-16T01:55:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.646+0000 7fcaf007c740 10 client.26480 fetch_fsmap learned FSMap version 71 2024-09-16T01:55:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.646+0000 7fcaf007c740 1 -- 172.21.15.139:0/3308415973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55e503034620 con 0x55e5032f0af0 2024-09-16T01:55:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.649+0000 7fcace7fc640 1 -- 172.21.15.139:0/3308415973 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae00328e0 con 0x55e5032f0af0 2024-09-16T01:55:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.649+0000 7fcace7fc640 1 -- 172.21.15.139:0/3308415973 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7fcad8011af0 con 0x55e5032f0af0 2024-09-16T01:55:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.650+0000 7fcaf007c740 10 client.26480 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:55:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.650+0000 7fcaf007c740 10 client.26480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:55:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.650+0000 7fcaf007c740 20 client.26480 populate_metadata read hostname 'smithi139' 2024-09-16T01:55:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.650+0000 7fcaf007c740 1 --2- 172.21.15.139:0/3308415973 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e50330bdf0 0x55e50330e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.650+0000 7fcaf007c740 4 client.26480 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:55:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.650+0000 7fcaf007c740 1 -- 172.21.15.139:0/3308415973 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e503262a20 con 0x55e50330bdf0 2024-09-16T01:55:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.652+0000 7fcae659c640 1 --2- 172.21.15.139:0/3308415973 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e50330bdf0 0x55e50330e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.657+0000 7fcae659c640 1 --2- 172.21.15.139:0/3308415973 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e50330bdf0 0x55e50330e250 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=440374e06dcfb51c server_cookie=da103d63581f196e in_seq=0 out_seq=0 2024-09-16T01:55:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.657+0000 7fcace7fc640 10 client.26480 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.659+0000 7fcace7fc640 1 -- 172.21.15.139:0/3308415973 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e503262a20 con 0x55e50330bdf0 2024-09-16T01:55:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.659+0000 7fcace7fc640 10 client.26480 handle_command_reply: tid=0 2024-09-16T01:55:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.661+0000 7fcaf007c740 1 client.26480 shutdown 2024-09-16T01:55:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.667+0000 7fcaf007c740 1 -- 172.21.15.139:0/3308415973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 msgr2=0x55e5032f8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.667+0000 7fcaf007c740 1 --2- 172.21.15.139:0/3308415973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f8c70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcad800ce60 tx=0x7fcad80047d0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.670+0000 7fcaf007c740 1 -- 172.21.15.139:0/3308415973 shutdown_connections 2024-09-16T01:55:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.670+0000 7fcaf007c740 1 --2- 172.21.15.139:0/3308415973 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e50330bdf0 0x55e50330e250 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.670+0000 7fcaf007c740 1 --2- 172.21.15.139:0/3308415973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032fbcb0 0x55e5032f96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.670+0000 7fcaf007c740 1 --2- 172.21.15.139:0/3308415973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032f1410 0x55e5032f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.670+0000 7fcaf007c740 1 --2- 172.21.15.139:0/3308415973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f8c70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.670+0000 7fcaf007c740 1 -- 172.21.15.139:0/3308415973 >> 172.21.15.139:0/3308415973 conn(0x55e5032ed060 msgr2=0x55e5032f2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.670+0000 7fcace7fc640 0 client.26480 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.674+0000 7fcaf007c740 1 -- 172.21.15.139:0/3308415973 shutdown_connections 2024-09-16T01:55:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.675+0000 7fcaf007c740 1 -- 172.21.15.139:0/3308415973 wait complete. 2024-09-16T01:55:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.675+0000 7fcaf007c740 20 client.26480 trim_cache size 0 max 16384 2024-09-16T01:55:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.707+0000 7fcaf007c740 1 Processor -- start 2024-09-16T01:55:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.707+0000 7fcaf007c740 1 -- start start 2024-09-16T01:55:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.707+0000 7fcaf007c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.707+0000 7fcaf007c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032f1410 0x55e5032fabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.707+0000 7fcaf007c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032fbcb0 0x55e503313b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.708+0000 7fcaf007c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcac80102e0 con 0x55e5032f1410 2024-09-16T01:55:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.708+0000 7fcaf007c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8012120 con 0x55e5032f0af0 2024-09-16T01:55:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.708+0000 7fcaf007c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8010a70 con 0x55e5032fbcb0 2024-09-16T01:55:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.708+0000 7fcae659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032f1410 0x55e5032fabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.709+0000 7fcae659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032f1410 0x55e5032fabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43460/0 (socket says 172.21.15.139:43460) 2024-09-16T01:55:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.709+0000 7fcae659c640 1 -- 172.21.15.139:0/4079745449 learned_addr learned my addr 172.21.15.139:0/4079745449 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.710+0000 7fcae659c640 1 -- 172.21.15.139:0/4079745449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032fbcb0 msgr2=0x55e503313b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:55:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.710+0000 7fcae659c640 1 --2- 172.21.15.139:0/4079745449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032fbcb0 0x55e503313b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.710+0000 7fcae659c640 1 -- 172.21.15.139:0/4079745449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 msgr2=0x55e5032f9520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:55:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.710+0000 7fcae659c640 1 --2- 172.21.15.139:0/4079745449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f9520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.710+0000 7fcae659c640 1 -- 172.21.15.139:0/4079745449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5032f9a60 con 0x55e5032f1410 2024-09-16T01:55:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.711+0000 7fcae659c640 1 --2- 172.21.15.139:0/4079745449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032f1410 0x55e5032fabe0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00cfd0 tx=0x7fcadc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586c2a8869098641 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.717+0000 7fcad4ff9640 1 -- 172.21.15.139:0/4079745449 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc084e80 con 0x55e5032f1410 2024-09-16T01:55:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.717+0000 7fcad4ff9640 1 -- 172.21.15.139:0/4079745449 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcadc09f590 con 0x55e5032f1410 2024-09-16T01:55:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.717+0000 7fcad4ff9640 1 -- 172.21.15.139:0/4079745449 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0d0c80 con 0x55e5032f1410 2024-09-16T01:55:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.720+0000 7fcaf007c740 1 -- 172.21.15.139:0/4079745449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032f1410 msgr2=0x55e5032fabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.720+0000 7fcaf007c740 1 --2- 172.21.15.139:0/4079745449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032f1410 0x55e5032fabe0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00cfd0 tx=0x7fcadc000bc0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.726+0000 7fcaf007c740 1 -- 172.21.15.139:0/4079745449 shutdown_connections 2024-09-16T01:55:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.726+0000 7fcaf007c740 1 --2- 172.21.15.139:0/4079745449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032fbcb0 0x55e503313b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.726+0000 7fcaf007c740 1 --2- 172.21.15.139:0/4079745449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032f1410 0x55e5032fabe0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.726+0000 7fcaf007c740 1 --2- 172.21.15.139:0/4079745449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032f9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.726+0000 7fcaf007c740 1 -- 172.21.15.139:0/4079745449 >> 172.21.15.139:0/4079745449 conn(0x55e5032ed060 msgr2=0x55e5032fa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.733+0000 7fcaf007c740 1 -- 172.21.15.139:0/4079745449 shutdown_connections 2024-09-16T01:55:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.736+0000 7fcaf007c740 1 -- 172.21.15.139:0/4079745449 wait complete. 2024-09-16T01:55:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.737+0000 7fcaf007c740 1 -- start start 2024-09-16T01:55:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.737+0000 7fcaf007c740 1 Processor -- start 2024-09-16T01:55:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.740+0000 7fcaf007c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032ff960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.740+0000 7fcaf007c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032f1410 0x55e5032ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.740+0000 7fcaf007c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032fbcb0 0x55e5033003e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.740+0000 7fcaf007c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8011990 con 0x55e5032f1410 2024-09-16T01:55:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.740+0000 7fcaf007c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8011200 con 0x55e5032f0af0 2024-09-16T01:55:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.740+0000 7fcaf007c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcac800e2e0 con 0x55e5032fbcb0 2024-09-16T01:55:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.740+0000 7fcaf007c740 10 client.0 _ll_register_callbacks cb 0x7fcad800ae00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:55:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.741+0000 7fcae659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032f1410 0x55e5032ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.741+0000 7fcae659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032f1410 0x55e5032ffea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43476/0 (socket says 172.21.15.139:43476) 2024-09-16T01:55:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.741+0000 7fcae659c640 1 -- 172.21.15.139:0/426381883 learned_addr learned my addr 172.21.15.139:0/426381883 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.741+0000 7fcae659c640 1 -- 172.21.15.139:0/426381883 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032fbcb0 msgr2=0x55e5033003e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:55:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.741+0000 7fcae659c640 1 --2- 172.21.15.139:0/426381883 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032fbcb0 0x55e5033003e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.741+0000 7fcae659c640 1 -- 172.21.15.139:0/426381883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 msgr2=0x55e5032ff960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:55:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.741+0000 7fcae659c640 1 --2- 172.21.15.139:0/426381883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032ff960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.741+0000 7fcae659c640 1 -- 172.21.15.139:0/426381883 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e503375af0 con 0x55e5032f1410 2024-09-16T01:55:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.742+0000 7fcae659c640 1 --2- 172.21.15.139:0/426381883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032f1410 0x55e5032ffea0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcadc09f520 tx=0x7fcadc0a3a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695692b3c019f6ae server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.742+0000 7fcace7fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:55:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.743+0000 7fcace7fc640 1 -- 172.21.15.139:0/426381883 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0d0c80 con 0x55e5032f1410 2024-09-16T01:55:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.743+0000 7fcace7fc640 1 -- 172.21.15.139:0/426381883 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9754+0+0 (secure 0 0 0) 0x7fcadc0bf050 con 0x55e5032f1410 2024-09-16T01:55:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.750+0000 7fcaf007c740 1 -- 172.21.15.139:0/426381883 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcadc09f590 con 0x55e5032f1410 2024-09-16T01:55:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.753+0000 7fcace7fc640 1 -- 172.21.15.139:0/426381883 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fcadc0bd050 con 0x55e5032f1410 2024-09-16T01:55:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.758+0000 7fcaf007c740 10 client.17046 fetch_fsmap learned FSMap version 71 2024-09-16T01:55:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.758+0000 7fcaf007c740 1 -- 172.21.15.139:0/426381883 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55e503376d00 con 0x55e5032f1410 2024-09-16T01:55:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.759+0000 7fcace7fc640 1 -- 172.21.15.139:0/426381883 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcadc09f590 con 0x55e5032f1410 2024-09-16T01:55:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.760+0000 7fcace7fc640 1 -- 172.21.15.139:0/426381883 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7fcadc09eaf0 con 0x55e5032f1410 2024-09-16T01:55:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.760+0000 7fcaf007c740 10 client.17046 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:55:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.760+0000 7fcaf007c740 10 client.17046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:55:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.760+0000 7fcaf007c740 20 client.17046 populate_metadata read hostname 'smithi139' 2024-09-16T01:55:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.760+0000 7fcaf007c740 1 --2- 172.21.15.139:0/426381883 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e5033773a0 0x55e5033797c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.760+0000 7fcaf007c740 4 client.17046 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:55:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.760+0000 7fcaf007c740 1 -- 172.21.15.139:0/426381883 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e50325cd70 con 0x55e5033773a0 2024-09-16T01:55:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.764+0000 7fcae6d9d640 1 --2- 172.21.15.139:0/426381883 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e5033773a0 0x55e5033797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.775+0000 7fcae6d9d640 1 --2- 172.21.15.139:0/426381883 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e5033773a0 0x55e5033797c0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f91172f1ece0e192 server_cookie=fb7edb4b9ce71c14 in_seq=0 out_seq=0 2024-09-16T01:55:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.776+0000 7fcace7fc640 10 client.17046 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.965+0000 7fcace7fc640 1 -- 172.21.15.139:0/426381883 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e50325cd70 con 0x55e5033773a0 2024-09-16T01:55:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.966+0000 7fcace7fc640 10 client.17046 handle_command_reply: tid=0 2024-09-16T01:55:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.966+0000 7fcaf007c740 1 client.17046 shutdown 2024-09-16T01:55:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.966+0000 7fcaf007c740 1 -- 172.21.15.139:0/426381883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032f1410 msgr2=0x55e5032ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.966+0000 7fcaf007c740 1 --2- 172.21.15.139:0/426381883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032f1410 0x55e5032ffea0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcadc09f520 tx=0x7fcadc0a3a80 comp rx=0 tx=0).stop 2024-09-16T01:55:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.966+0000 7fcaf007c740 1 -- 172.21.15.139:0/426381883 shutdown_connections 2024-09-16T01:55:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.966+0000 7fcaf007c740 1 --2- 172.21.15.139:0/426381883 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e5033773a0 0x55e5033797c0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.966+0000 7fcaf007c740 1 --2- 172.21.15.139:0/426381883 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5032fbcb0 0x55e5033003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.966+0000 7fcaf007c740 1 --2- 172.21.15.139:0/426381883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5032f1410 0x55e5032ffea0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.966+0000 7fcaf007c740 1 --2- 172.21.15.139:0/426381883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e5032f0af0 0x55e5032ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.966+0000 7fcaf007c740 1 -- 172.21.15.139:0/426381883 >> 172.21.15.139:0/426381883 conn(0x55e5032ed060 msgr2=0x55e5032f2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.966+0000 7fcace7fc640 0 client.17046 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.966+0000 7fcaf007c740 1 -- 172.21.15.139:0/426381883 shutdown_connections 2024-09-16T01:55:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.966+0000 7fcaf007c740 1 -- 172.21.15.139:0/426381883 wait complete. 2024-09-16T01:55:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.966+0000 7fcaf007c740 20 client.17046 trim_cache size 0 max 16384 2024-09-16T01:55:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:55:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:55:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:55:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:55:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.967+0000 7fcaed1fd640 1 -- 172.21.15.139:0/3547516889 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcad00a3560 msgr2=0x7fcad00a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.967+0000 7fcaed1fd640 1 --2- 172.21.15.139:0/3547516889 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcad00a3560 0x7fcad00a5a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55e5032edec0 tx=0x7fcad8008040 comp rx=0 tx=0).stop 2024-09-16T01:55:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.967+0000 7fcaed1fd640 1 -- 172.21.15.139:0/3547516889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcae810ea10 msgr2=0x7fcae81a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.967+0000 7fcaed1fd640 1 --2- 172.21.15.139:0/3547516889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcae810ea10 0x7fcae81a5aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00e9f0 tx=0x7fcadc00eeb0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.967+0000 7fcaed1fd640 1 -- 172.21.15.139:0/3547516889 shutdown_connections 2024-09-16T01:55:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.967+0000 7fcaed1fd640 1 --2- 172.21.15.139:0/3547516889 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fcad00a3560 0x7fcad00a5a20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.967+0000 7fcaed1fd640 1 --2- 172.21.15.139:0/3547516889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcae819e630 0x7fcae819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.967+0000 7fcaed1fd640 1 --2- 172.21.15.139:0/3547516889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcae810ea10 0x7fcae81a5aa0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.967+0000 7fcaed1fd640 1 --2- 172.21.15.139:0/3547516889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcae8109460 0x7fcae81a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.967+0000 7fcaed1fd640 1 -- 172.21.15.139:0/3547516889 >> 172.21.15.139:0/3547516889 conn(0x7fcae806d210 msgr2=0x7fcae8072390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.967+0000 7fcaed1fd640 1 -- 172.21.15.139:0/3547516889 shutdown_connections 2024-09-16T01:55:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:25.967+0000 7fcaed1fd640 1 -- 172.21.15.139:0/3547516889 wait complete. 2024-09-16T01:55:26.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:25 smithi164 ceph-mon[35121]: pgmap v474: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-09-16T01:55:26.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:25 smithi139 ceph-mon[31175]: pgmap v474: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-09-16T01:55:26.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:25 smithi176 ceph-mon[35324]: pgmap v474: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-09-16T01:55:27.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:27 smithi139 ceph-mon[31175]: pgmap v475: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 236 op/s 2024-09-16T01:55:27.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:27 smithi139 ceph-mon[31175]: osdmap e108: 12 total, 12 up, 12 in 2024-09-16T01:55:27.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:27 smithi176 ceph-mon[35324]: pgmap v475: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 236 op/s 2024-09-16T01:55:27.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:27 smithi176 ceph-mon[35324]: osdmap e108: 12 total, 12 up, 12 in 2024-09-16T01:55:27.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:27 smithi164 ceph-mon[35121]: pgmap v475: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 236 op/s 2024-09-16T01:55:27.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:27 smithi164 ceph-mon[35121]: osdmap e108: 12 total, 12 up, 12 in 2024-09-16T01:55:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:29 smithi139 ceph-mon[31175]: pgmap v477: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 244 op/s 2024-09-16T01:55:29.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:29 smithi176 ceph-mon[35324]: pgmap v477: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 244 op/s 2024-09-16T01:55:29.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:29 smithi164 ceph-mon[35121]: pgmap v477: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 244 op/s 2024-09-16T01:55:31.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:31 smithi139 ceph-mon[31175]: pgmap v478: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 279 op/s 2024-09-16T01:55:31.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:31 smithi176 ceph-mon[35324]: pgmap v478: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 279 op/s 2024-09-16T01:55:31.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:31 smithi164 ceph-mon[35121]: pgmap v478: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 279 op/s 2024-09-16T01:55:33.425 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:33 smithi139 ceph-mon[31175]: pgmap v479: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 285 op/s 2024-09-16T01:55:33.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:33 smithi176 ceph-mon[35324]: pgmap v479: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 285 op/s 2024-09-16T01:55:33.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:33 smithi164 ceph-mon[35121]: pgmap v479: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 285 op/s 2024-09-16T01:55:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:35 smithi139 ceph-mon[31175]: pgmap v480: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 914 KiB/s wr, 293 op/s 2024-09-16T01:55:35.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:35 smithi176 ceph-mon[35324]: pgmap v480: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 914 KiB/s wr, 293 op/s 2024-09-16T01:55:35.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:35 smithi164 ceph-mon[35121]: pgmap v480: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 914 KiB/s wr, 293 op/s 2024-09-16T01:55:37.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:37 smithi139 ceph-mon[31175]: pgmap v481: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 298 op/s 2024-09-16T01:55:37.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:37 smithi176 ceph-mon[35324]: pgmap v481: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 298 op/s 2024-09-16T01:55:37.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:37 smithi164 ceph-mon[35121]: pgmap v481: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 298 op/s 2024-09-16T01:55:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.132+0000 7f3716805640 1 -- 172.21.15.139:0/2969354117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37080b7510 msgr2=0x7f37080b9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.132+0000 7f3716805640 1 --2- 172.21.15.139:0/2969354117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37080b7510 0x7f37080b9900 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f370001a910 tx=0x7f370003b450 comp rx=0 tx=0).stop 2024-09-16T01:55:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.137+0000 7f3716805640 1 -- 172.21.15.139:0/2969354117 shutdown_connections 2024-09-16T01:55:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.137+0000 7f3716805640 1 --2- 172.21.15.139:0/2969354117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37080b7510 0x7f37080b9900 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.137+0000 7f3716805640 1 --2- 172.21.15.139:0/2969354117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f37080a5110 0x7f37080b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.137+0000 7f3716805640 1 --2- 172.21.15.139:0/2969354117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37080a47f0 0x7f37080a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.137+0000 7f3716805640 1 -- 172.21.15.139:0/2969354117 >> 172.21.15.139:0/2969354117 conn(0x7f370801a0d0 msgr2=0x7f370801a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.144+0000 7f3716805640 1 -- 172.21.15.139:0/2969354117 shutdown_connections 2024-09-16T01:55:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.147+0000 7f3716805640 1 -- 172.21.15.139:0/2969354117 wait complete. 2024-09-16T01:55:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.150+0000 7f3716805640 1 Processor -- start 2024-09-16T01:55:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.154+0000 7f3716805640 1 -- start start 2024-09-16T01:55:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.154+0000 7f3716805640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f37080a47f0 0x7f37080b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.154+0000 7f3716805640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37080a5110 0x7f37080b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.154+0000 7f3716805640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37080b7510 0x7f37080b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.154+0000 7f3716805640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f37080af180 con 0x7f37080a5110 2024-09-16T01:55:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.154+0000 7f3716805640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37080af000 con 0x7f37080b7510 2024-09-16T01:55:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.154+0000 7f3716805640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f37080af300 con 0x7f37080a47f0 2024-09-16T01:55:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.157+0000 7f3716004640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37080b7510 0x7f37080b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.157+0000 7f3716004640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37080b7510 0x7f37080b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44416/0 (socket says 172.21.15.139:44416) 2024-09-16T01:55:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.157+0000 7f3716004640 1 -- 172.21.15.139:0/4290019796 learned_addr learned my addr 172.21.15.139:0/4290019796 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.158+0000 7f3716004640 1 -- 172.21.15.139:0/4290019796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f37080a47f0 msgr2=0x7f37080b6c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:55:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.158+0000 7f3716004640 1 --2- 172.21.15.139:0/4290019796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f37080a47f0 0x7f37080b6c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.158+0000 7f3716004640 1 -- 172.21.15.139:0/4290019796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37080a5110 msgr2=0x7f37080b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.158+0000 7f3716004640 1 --2- 172.21.15.139:0/4290019796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37080a5110 0x7f37080b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.158+0000 7f3716004640 1 -- 172.21.15.139:0/4290019796 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37080b3910 con 0x7f37080b7510 2024-09-16T01:55:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.158+0000 7f3716004640 1 --2- 172.21.15.139:0/4290019796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37080b7510 0x7f37080b31e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f37000054d0 tx=0x7f3700004b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.159+0000 7f3706ffd640 1 -- 172.21.15.139:0/4290019796 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370004f070 con 0x7f37080b7510 2024-09-16T01:55:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.159+0000 7f3706ffd640 1 -- 172.21.15.139:0/4290019796 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37000409f0 con 0x7f37080b7510 2024-09-16T01:55:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.159+0000 7f3706ffd640 1 -- 172.21.15.139:0/4290019796 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700046030 con 0x7f37080b7510 2024-09-16T01:55:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.159+0000 7f3716805640 1 -- 172.21.15.139:0/4290019796 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37081473e0 con 0x7f37080b7510 2024-09-16T01:55:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.159+0000 7f3716805640 1 -- 172.21.15.139:0/4290019796 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3708147790 con 0x7f37080b7510 2024-09-16T01:55:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.172+0000 7f371a7ff740 1 Processor -- start 2024-09-16T01:55:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.178+0000 7f3706ffd640 1 -- 172.21.15.139:0/4290019796 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3700052070 con 0x7f37080b7510 2024-09-16T01:55:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.178+0000 7f371a7ff740 1 -- start start 2024-09-16T01:55:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.179+0000 7f371a7ff740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d80dcc0 0x56256d80e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.179+0000 7f371a7ff740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80e5e0 0x56256d818940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.179+0000 7f371a7ff740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d818e80 0x56256d81b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.179+0000 7f371a7ff740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56256d7722f0 con 0x56256d80dcc0 2024-09-16T01:55:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.179+0000 7f371a7ff740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56256d71a270 con 0x56256d80e5e0 2024-09-16T01:55:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.179+0000 7f371a7ff740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56256d76e2b0 con 0x56256d818e80 2024-09-16T01:55:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.179+0000 7f3715803640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80e5e0 0x56256d818940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.179+0000 7f3715803640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80e5e0 0x56256d818940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44418/0 (socket says 172.21.15.139:44418) 2024-09-16T01:55:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.179+0000 7f3715803640 1 -- 172.21.15.139:0/242900943 learned_addr learned my addr 172.21.15.139:0/242900943 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.179+0000 7f3715803640 1 -- 172.21.15.139:0/242900943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d818e80 msgr2=0x56256d81b270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:55:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.179+0000 7f3715803640 1 --2- 172.21.15.139:0/242900943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d818e80 0x56256d81b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.179+0000 7f3715803640 1 -- 172.21.15.139:0/242900943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d80dcc0 msgr2=0x56256d80e0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:55:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.179+0000 7f3715803640 1 --2- 172.21.15.139:0/242900943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d80dcc0 0x56256d80e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.179+0000 7f3715803640 1 -- 172.21.15.139:0/242900943 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56256d719340 con 0x56256d80e5e0 2024-09-16T01:55:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.180+0000 7f3715803640 1 --2- 172.21.15.139:0/242900943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80e5e0 0x56256d818940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f371002dd80 tx=0x7f3710073720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa3528bd877486a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.180+0000 7f3706ffd640 1 --2- 172.21.15.139:0/4290019796 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f36fc0a35c0 0x7f36fc0a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.180+0000 7f3706ffd640 1 -- 172.21.15.139:0/4290019796 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9678+0+0 (secure 0 0 0) 0x7f3700101630 con 0x7f37080b7510 2024-09-16T01:55:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.181+0000 7f3715002640 1 --2- 172.21.15.139:0/4290019796 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f36fc0a35c0 0x7f36fc0a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.182+0000 7f36fbfff640 1 -- 172.21.15.139:0/242900943 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710075430 con 0x56256d80e5e0 2024-09-16T01:55:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.182+0000 7f36fbfff640 1 -- 172.21.15.139:0/242900943 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3710075ab0 con 0x56256d80e5e0 2024-09-16T01:55:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.183+0000 7f371a7ff740 1 -- 172.21.15.139:0/242900943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80e5e0 msgr2=0x56256d818940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.183+0000 7f371a7ff740 1 --2- 172.21.15.139:0/242900943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80e5e0 0x56256d818940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f371002dd80 tx=0x7f3710073720 comp rx=0 tx=0).stop 2024-09-16T01:55:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.183+0000 7f3715002640 1 --2- 172.21.15.139:0/4290019796 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f36fc0a35c0 0x7f36fc0a5a80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f370c006fd0 tx=0x7f370c006c20 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.185+0000 7f371a7ff740 1 -- 172.21.15.139:0/242900943 shutdown_connections 2024-09-16T01:55:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.185+0000 7f371a7ff740 1 --2- 172.21.15.139:0/242900943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d818e80 0x56256d81b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.185+0000 7f371a7ff740 1 --2- 172.21.15.139:0/242900943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80e5e0 0x56256d818940 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.185+0000 7f371a7ff740 1 --2- 172.21.15.139:0/242900943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d80dcc0 0x56256d80e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.185+0000 7f371a7ff740 1 -- 172.21.15.139:0/242900943 >> 172.21.15.139:0/242900943 conn(0x56256d80a230 msgr2=0x56256d80a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.186+0000 7f371a7ff740 1 -- 172.21.15.139:0/242900943 shutdown_connections 2024-09-16T01:55:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.188+0000 7f371a7ff740 1 -- 172.21.15.139:0/242900943 wait complete. 2024-09-16T01:55:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.188+0000 7f371a7ff740 1 -- start start 2024-09-16T01:55:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.188+0000 7f371a7ff740 1 Processor -- start 2024-09-16T01:55:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.189+0000 7f371a7ff740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d80dcc0 0x56256d815d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.189+0000 7f371a7ff740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80e5e0 0x56256d8162d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.189+0000 7f371a7ff740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 0x56256d816810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.189+0000 7f371a7ff740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56256d81ba50 con 0x56256d818e80 2024-09-16T01:55:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.189+0000 7f371a7ff740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56256d6fb290 con 0x56256d80e5e0 2024-09-16T01:55:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.189+0000 7f371a7ff740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56256d81bbd0 con 0x56256d80dcc0 2024-09-16T01:55:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.189+0000 7f371a7ff740 10 client.0 _ll_register_callbacks cb 0x56256d72c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:55:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.192+0000 7f3715002640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80e5e0 0x56256d8162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.192+0000 7f3715002640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80e5e0 0x56256d8162d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44430/0 (socket says 172.21.15.139:44430) 2024-09-16T01:55:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.192+0000 7f3715002640 1 -- 172.21.15.139:0/1591154772 learned_addr learned my addr 172.21.15.139:0/1591154772 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.192+0000 7f3715803640 1 --2- 172.21.15.139:0/1591154772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d80dcc0 0x56256d815d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.192+0000 7f3716004640 1 --2- 172.21.15.139:0/1591154772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 0x56256d816810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.192+0000 7f3715002640 1 -- 172.21.15.139:0/1591154772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d80dcc0 msgr2=0x56256d815d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.192+0000 7f3715002640 1 --2- 172.21.15.139:0/1591154772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d80dcc0 0x56256d815d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.192+0000 7f3715002640 1 -- 172.21.15.139:0/1591154772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 msgr2=0x56256d816810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.192+0000 7f3715002640 1 --2- 172.21.15.139:0/1591154772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 0x56256d816810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.192+0000 7f3715002640 1 -- 172.21.15.139:0/1591154772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56256d485210 con 0x56256d80e5e0 2024-09-16T01:55:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.193+0000 7f3715002640 1 --2- 172.21.15.139:0/1591154772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80e5e0 0x56256d8162d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f370c013eb0 tx=0x7f370c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9600a769b216b46 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.194+0000 7f36f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:55:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.194+0000 7f36f97fa640 1 -- 172.21.15.139:0/1591154772 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c01d5f0 con 0x56256d80e5e0 2024-09-16T01:55:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.194+0000 7f371a7ff740 1 -- 172.21.15.139:0/1591154772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3710075ab0 con 0x56256d80e5e0 2024-09-16T01:55:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.197+0000 7f36f97fa640 1 -- 172.21.15.139:0/1591154772 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9678+0+0 (secure 0 0 0) 0x7f370c01d790 con 0x56256d80e5e0 2024-09-16T01:55:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.201+0000 7f36f97fa640 1 -- 172.21.15.139:0/1591154772 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f370c02b050 con 0x56256d80e5e0 2024-09-16T01:55:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.205+0000 7f371a7ff740 10 client.26492 fetch_fsmap learned FSMap version 71 2024-09-16T01:55:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.205+0000 7f371a7ff740 1 -- 172.21.15.139:0/1591154772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x56256d551620 con 0x56256d80e5e0 2024-09-16T01:55:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.209+0000 7f36f97fa640 1 -- 172.21.15.139:0/1591154772 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3710075ab0 con 0x56256d80e5e0 2024-09-16T01:55:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.211+0000 7f36f97fa640 1 -- 172.21.15.139:0/1591154772 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f370c012070 con 0x56256d80e5e0 2024-09-16T01:55:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.211+0000 7f371a7ff740 10 client.26492 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:55:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.211+0000 7f371a7ff740 10 client.26492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:55:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.211+0000 7f371a7ff740 20 client.26492 populate_metadata read hostname 'smithi139' 2024-09-16T01:55:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.212+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1591154772 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56256d84b4c0 0x56256d84d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.212+0000 7f371a7ff740 4 client.26492 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:55:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.212+0000 7f371a7ff740 1 -- 172.21.15.139:0/1591154772 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56256d77fa20 con 0x56256d84b4c0 2024-09-16T01:55:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.212+0000 7f3715803640 1 --2- 172.21.15.139:0/1591154772 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56256d84b4c0 0x56256d84d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.213+0000 7f36f97fa640 10 client.26492 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.213+0000 7f3715803640 1 --2- 172.21.15.139:0/1591154772 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56256d84b4c0 0x56256d84d980 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ff9c5adce73494 server_cookie=7f2d318ae36c3de0 in_seq=0 out_seq=0 2024-09-16T01:55:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.218+0000 7f36f97fa640 1 -- 172.21.15.139:0/1591154772 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56256d77fa20 con 0x56256d84b4c0 2024-09-16T01:55:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.218+0000 7f36f97fa640 10 client.26492 handle_command_reply: tid=0 2024-09-16T01:55:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.218+0000 7f371a7ff740 1 client.26492 shutdown 2024-09-16T01:55:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.227+0000 7f371a7ff740 1 -- 172.21.15.139:0/1591154772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80e5e0 msgr2=0x56256d8162d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.227+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1591154772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80e5e0 0x56256d8162d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f370c013eb0 tx=0x7f370c004750 comp rx=0 tx=0).stop 2024-09-16T01:55:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.228+0000 7f371a7ff740 1 -- 172.21.15.139:0/1591154772 shutdown_connections 2024-09-16T01:55:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.228+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1591154772 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56256d84b4c0 0x56256d84d980 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.228+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1591154772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 0x56256d816810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.228+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1591154772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80e5e0 0x56256d8162d0 secure :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f370c013eb0 tx=0x7f370c004750 comp rx=0 tx=0).stop 2024-09-16T01:55:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.228+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1591154772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d80dcc0 0x56256d815d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.228+0000 7f371a7ff740 1 -- 172.21.15.139:0/1591154772 >> 172.21.15.139:0/1591154772 conn(0x56256d80a230 msgr2=0x56256d80ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.231+0000 7f36f97fa640 0 client.26492 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.236+0000 7f371a7ff740 1 -- 172.21.15.139:0/1591154772 shutdown_connections 2024-09-16T01:55:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.238+0000 7f371a7ff740 1 -- 172.21.15.139:0/1591154772 wait complete. 2024-09-16T01:55:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.239+0000 7f371a7ff740 20 client.26492 trim_cache size 0 max 16384 2024-09-16T01:55:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.277+0000 7f371a7ff740 1 Processor -- start 2024-09-16T01:55:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.281+0000 7f371a7ff740 1 -- start start 2024-09-16T01:55:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.281+0000 7f371a7ff740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80dcc0 0x56256d810730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.281+0000 7f371a7ff740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 0x56256d816640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.281+0000 7f371a7ff740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d853290 0x56256d853740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.281+0000 7f371a7ff740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4010170 con 0x56256d818e80 2024-09-16T01:55:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.281+0000 7f371a7ff740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4011fb0 con 0x56256d80dcc0 2024-09-16T01:55:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.281+0000 7f371a7ff740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4010900 con 0x56256d853290 2024-09-16T01:55:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.282+0000 7f3715002640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 0x56256d816640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:39.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.282+0000 7f3715002640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 0x56256d816640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43446/0 (socket says 172.21.15.139:43446) 2024-09-16T01:55:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.282+0000 7f3715002640 1 -- 172.21.15.139:0/1391599457 learned_addr learned my addr 172.21.15.139:0/1391599457 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.283+0000 7f3716004640 1 --2- 172.21.15.139:0/1391599457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d853290 0x56256d853740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.285+0000 7f3715803640 1 --2- 172.21.15.139:0/1391599457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80dcc0 0x56256d810730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.285+0000 7f3715002640 1 -- 172.21.15.139:0/1391599457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d853290 msgr2=0x56256d853740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.285+0000 7f3715002640 1 --2- 172.21.15.139:0/1391599457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d853290 0x56256d853740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.285+0000 7f3715002640 1 -- 172.21.15.139:0/1391599457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80dcc0 msgr2=0x56256d810730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.285+0000 7f3715002640 1 --2- 172.21.15.139:0/1391599457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80dcc0 0x56256d810730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.285+0000 7f3715002640 1 -- 172.21.15.139:0/1391599457 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56256d8142c0 con 0x56256d818e80 2024-09-16T01:55:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.285+0000 7f3715803640 1 --2- 172.21.15.139:0/1391599457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80dcc0 0x56256d810730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:55:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.285+0000 7f3715002640 1 --2- 172.21.15.139:0/1391599457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 0x56256d816640 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f370c026d70 tx=0x7f370c006a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df74a8d34a35c63 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.287+0000 7f36fb7fe640 1 -- 172.21.15.139:0/1391599457 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c012070 con 0x56256d818e80 2024-09-16T01:55:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.287+0000 7f36fb7fe640 1 -- 172.21.15.139:0/1391599457 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f370c022070 con 0x56256d818e80 2024-09-16T01:55:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.287+0000 7f36fb7fe640 1 -- 172.21.15.139:0/1391599457 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c003690 con 0x56256d818e80 2024-09-16T01:55:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.288+0000 7f371a7ff740 1 -- 172.21.15.139:0/1391599457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 msgr2=0x56256d816640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.288+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1391599457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 0x56256d816640 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f370c026d70 tx=0x7f370c006a50 comp rx=0 tx=0).stop 2024-09-16T01:55:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.289+0000 7f371a7ff740 1 -- 172.21.15.139:0/1391599457 shutdown_connections 2024-09-16T01:55:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.289+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1391599457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d853290 0x56256d853740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.289+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1391599457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 0x56256d816640 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.289+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1391599457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80dcc0 0x56256d810730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.289+0000 7f371a7ff740 1 -- 172.21.15.139:0/1391599457 >> 172.21.15.139:0/1391599457 conn(0x56256d80a230 msgr2=0x56256d811060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.291+0000 7f371a7ff740 1 -- 172.21.15.139:0/1391599457 shutdown_connections 2024-09-16T01:55:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.291+0000 7f371a7ff740 1 -- 172.21.15.139:0/1391599457 wait complete. 2024-09-16T01:55:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.291+0000 7f371a7ff740 1 -- start start 2024-09-16T01:55:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.291+0000 7f371a7ff740 1 Processor -- start 2024-09-16T01:55:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f371a7ff740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80dcc0 0x56256d8215a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f371a7ff740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 0x56256d81c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f371a7ff740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d853290 0x56256d81ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f3715803640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80dcc0 0x56256d8215a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f3715803640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80dcc0 0x56256d8215a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:44444/0 (socket says 172.21.15.139:44444) 2024-09-16T01:55:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f3715803640 1 -- 172.21.15.139:0/1634792856 learned_addr learned my addr 172.21.15.139:0/1634792856 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f371a7ff740 1 -- 172.21.15.139:0/1634792856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4011820 con 0x56256d818e80 2024-09-16T01:55:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f371a7ff740 1 -- 172.21.15.139:0/1634792856 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4011090 con 0x56256d80dcc0 2024-09-16T01:55:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f371a7ff740 1 -- 172.21.15.139:0/1634792856 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f36f400e150 con 0x56256d853290 2024-09-16T01:55:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f371a7ff740 10 client.0 _ll_register_callbacks cb 0x7f370c0092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:55:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f3716004640 1 --2- 172.21.15.139:0/1634792856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d853290 0x56256d81ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f3716004640 1 -- 172.21.15.139:0/1634792856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80dcc0 msgr2=0x56256d8215a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f3716004640 1 --2- 172.21.15.139:0/1634792856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80dcc0 0x56256d8215a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f3716004640 1 -- 172.21.15.139:0/1634792856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 msgr2=0x56256d81c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.292+0000 7f3716004640 1 --2- 172.21.15.139:0/1634792856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 0x56256d81c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.293+0000 7f3716004640 1 -- 172.21.15.139:0/1634792856 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56256d81d630 con 0x56256d853290 2024-09-16T01:55:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.293+0000 7f3716004640 1 --2- 172.21.15.139:0/1634792856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d853290 0x56256d81ce00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f37000fe8a0 tx=0x7f37000cd2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e08efae613bbf0f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.293+0000 7f36f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:55:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.293+0000 7f36f97fa640 1 -- 172.21.15.139:0/1634792856 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37000e6060 con 0x56256d853290 2024-09-16T01:55:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.293+0000 7f36f97fa640 1 -- 172.21.15.139:0/1634792856 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9678+0+0 (secure 0 0 0) 0x7f37000cd690 con 0x56256d853290 2024-09-16T01:55:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.296+0000 7f371a7ff740 1 -- 172.21.15.139:0/1634792856 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f370c022070 con 0x56256d853290 2024-09-16T01:55:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.297+0000 7f36f97fa640 1 -- 172.21.15.139:0/1634792856 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f3700059040 con 0x56256d853290 2024-09-16T01:55:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.302+0000 7f371a7ff740 10 client.26449 fetch_fsmap learned FSMap version 71 2024-09-16T01:55:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.302+0000 7f371a7ff740 1 -- 172.21.15.139:0/1634792856 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x56256d8b62b0 con 0x56256d853290 2024-09-16T01:55:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.302+0000 7f36f97fa640 1 -- 172.21.15.139:0/1634792856 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37000e6400 con 0x56256d853290 2024-09-16T01:55:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.306+0000 7f36f97fa640 1 -- 172.21.15.139:0/1634792856 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f37000d5c10 con 0x56256d853290 2024-09-16T01:55:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.306+0000 7f371a7ff740 10 client.26449 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:55:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.306+0000 7f371a7ff740 10 client.26449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:55:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.306+0000 7f371a7ff740 20 client.26449 populate_metadata read hostname 'smithi139' 2024-09-16T01:55:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.306+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1634792856 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56256d8b6aa0 0x56256d8b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.306+0000 7f371a7ff740 4 client.26449 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:55:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.306+0000 7f371a7ff740 1 -- 172.21.15.139:0/1634792856 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f370c022070 con 0x56256d8b6aa0 2024-09-16T01:55:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.308+0000 7f3715803640 1 --2- 172.21.15.139:0/1634792856 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56256d8b6aa0 0x56256d8b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.317+0000 7f3715803640 1 --2- 172.21.15.139:0/1634792856 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56256d8b6aa0 0x56256d8b8ec0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99c9f7f7ce47fefb server_cookie=6d8f67500d5657f in_seq=0 out_seq=0 2024-09-16T01:55:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.317+0000 7f36f97fa640 10 client.26449 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:39.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:39 smithi139 ceph-mon[31175]: pgmap v482: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 292 op/s 2024-09-16T01:55:39.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:39 smithi176 ceph-mon[35324]: pgmap v482: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 292 op/s 2024-09-16T01:55:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.511+0000 7f36f97fa640 1 -- 172.21.15.139:0/1634792856 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f370c022070 con 0x56256d8b6aa0 2024-09-16T01:55:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.511+0000 7f36f97fa640 10 client.26449 handle_command_reply: tid=0 2024-09-16T01:55:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.511+0000 7f371a7ff740 1 client.26449 shutdown 2024-09-16T01:55:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.511+0000 7f371a7ff740 1 -- 172.21.15.139:0/1634792856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d853290 msgr2=0x56256d81ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.511+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1634792856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d853290 0x56256d81ce00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f37000fe8a0 tx=0x7f37000cd2c0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.511+0000 7f371a7ff740 1 -- 172.21.15.139:0/1634792856 shutdown_connections 2024-09-16T01:55:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.511+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1634792856 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56256d8b6aa0 0x56256d8b8ec0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.511+0000 7f36f97fa640 0 client.26449 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.511+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1634792856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56256d853290 0x56256d81ce00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.511+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1634792856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56256d818e80 0x56256d81c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.511+0000 7f371a7ff740 1 --2- 172.21.15.139:0/1634792856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56256d80dcc0 0x56256d8215a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.511+0000 7f371a7ff740 1 -- 172.21.15.139:0/1634792856 >> 172.21.15.139:0/1634792856 conn(0x56256d80a230 msgr2=0x56256d811060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.512+0000 7f371a7ff740 1 -- 172.21.15.139:0/1634792856 shutdown_connections 2024-09-16T01:55:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.512+0000 7f371a7ff740 1 -- 172.21.15.139:0/1634792856 wait complete. 2024-09-16T01:55:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.512+0000 7f371a7ff740 20 client.26449 trim_cache size 0 max 16384 2024-09-16T01:55:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:55:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:55:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:55:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:55:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.513+0000 7f3716805640 1 -- 172.21.15.139:0/4290019796 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f36fc0a35c0 msgr2=0x7f36fc0a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.513+0000 7f3716805640 1 --2- 172.21.15.139:0/4290019796 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f36fc0a35c0 0x7f36fc0a5a80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f370c006fd0 tx=0x7f370c006c20 comp rx=0 tx=0).stop 2024-09-16T01:55:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.513+0000 7f3716805640 1 -- 172.21.15.139:0/4290019796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37080b7510 msgr2=0x7f37080b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.513+0000 7f3716805640 1 --2- 172.21.15.139:0/4290019796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37080b7510 0x7f37080b31e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f37000054d0 tx=0x7f3700004b90 comp rx=0 tx=0).stop 2024-09-16T01:55:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.513+0000 7f3716805640 1 -- 172.21.15.139:0/4290019796 shutdown_connections 2024-09-16T01:55:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.513+0000 7f3716805640 1 --2- 172.21.15.139:0/4290019796 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f36fc0a35c0 0x7f36fc0a5a80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.513+0000 7f3716805640 1 --2- 172.21.15.139:0/4290019796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37080b7510 0x7f37080b31e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.513+0000 7f3716805640 1 --2- 172.21.15.139:0/4290019796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37080a5110 0x7f37080b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.513+0000 7f3716805640 1 --2- 172.21.15.139:0/4290019796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f37080a47f0 0x7f37080b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.513+0000 7f3716805640 1 -- 172.21.15.139:0/4290019796 >> 172.21.15.139:0/4290019796 conn(0x7f370801a0d0 msgr2=0x7f37080b7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.514+0000 7f3716805640 1 -- 172.21.15.139:0/4290019796 shutdown_connections 2024-09-16T01:55:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:39.514+0000 7f3716805640 1 -- 172.21.15.139:0/4290019796 wait complete. 2024-09-16T01:55:39.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:39 smithi164 ceph-mon[35121]: pgmap v482: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 292 op/s 2024-09-16T01:55:41.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:41 smithi139 ceph-mon[31175]: pgmap v483: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 278 op/s 2024-09-16T01:55:41.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:41 smithi176 ceph-mon[35324]: pgmap v483: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 278 op/s 2024-09-16T01:55:41.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:41 smithi164 ceph-mon[35121]: pgmap v483: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 278 op/s 2024-09-16T01:55:43.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:43 smithi139 ceph-mon[31175]: pgmap v484: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 282 op/s 2024-09-16T01:55:43.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:43 smithi176 ceph-mon[35324]: pgmap v484: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 282 op/s 2024-09-16T01:55:43.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:43 smithi164 ceph-mon[35121]: pgmap v484: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 282 op/s 2024-09-16T01:55:45.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:45 smithi139 ceph-mon[31175]: pgmap v485: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 303 op/s 2024-09-16T01:55:45.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:45 smithi176 ceph-mon[35324]: pgmap v485: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 303 op/s 2024-09-16T01:55:45.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:45 smithi164 ceph-mon[35121]: pgmap v485: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 303 op/s 2024-09-16T01:55:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:47 smithi139 ceph-mon[31175]: pgmap v486: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 309 op/s 2024-09-16T01:55:47.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:47 smithi176 ceph-mon[35324]: pgmap v486: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 309 op/s 2024-09-16T01:55:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:47 smithi164 ceph-mon[35121]: pgmap v486: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 309 op/s 2024-09-16T01:55:49.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:49 smithi139 ceph-mon[31175]: pgmap v487: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 305 op/s 2024-09-16T01:55:49.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:49 smithi176 ceph-mon[35324]: pgmap v487: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 305 op/s 2024-09-16T01:55:49.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:49 smithi164 ceph-mon[35121]: pgmap v487: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 305 op/s 2024-09-16T01:55:51.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:51 smithi139 ceph-mon[31175]: pgmap v488: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 980 KiB/s wr, 283 op/s 2024-09-16T01:55:51.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:51 smithi176 ceph-mon[35324]: pgmap v488: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 980 KiB/s wr, 283 op/s 2024-09-16T01:55:51.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:51 smithi164 ceph-mon[35121]: pgmap v488: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 980 KiB/s wr, 283 op/s 2024-09-16T01:55:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.751+0000 7f0f3d2e7640 1 -- 172.21.15.139:0/801585815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f3009b690 msgr2=0x7f0f3009ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.751+0000 7f0f3d2e7640 1 --2- 172.21.15.139:0/801585815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f3009b690 0x7f0f3009ba70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c01a940 tx=0x7f0f2c03b520 comp rx=0 tx=0).stop 2024-09-16T01:55:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.754+0000 7f0f3d2e7640 1 -- 172.21.15.139:0/801585815 shutdown_connections 2024-09-16T01:55:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.754+0000 7f0f3d2e7640 1 --2- 172.21.15.139:0/801585815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f30096370 0x7f0f300a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.754+0000 7f0f3d2e7640 1 --2- 172.21.15.139:0/801585815 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f300959f0 0x7f0f30095e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.754+0000 7f0f3d2e7640 1 --2- 172.21.15.139:0/801585815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f3009b690 0x7f0f3009ba70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.754+0000 7f0f3d2e7640 1 -- 172.21.15.139:0/801585815 >> 172.21.15.139:0/801585815 conn(0x7f0f3000b350 msgr2=0x7f0f3000b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.760+0000 7f0f3d2e7640 1 -- 172.21.15.139:0/801585815 shutdown_connections 2024-09-16T01:55:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.761+0000 7f0f3d2e7640 1 -- 172.21.15.139:0/801585815 wait complete. 2024-09-16T01:55:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.761+0000 7f0f3d2e7640 1 Processor -- start 2024-09-16T01:55:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.762+0000 7f0f3d2e7640 1 -- start start 2024-09-16T01:55:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.762+0000 7f0f3d2e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f300959f0 0x7f0f300a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.762+0000 7f0f3d2e7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f30096370 0x7f0f300aa330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.762+0000 7f0f3d2e7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f3009b690 0x7f0f300afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.762+0000 7f0f3d2e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0f30008ea0 con 0x7f0f300959f0 2024-09-16T01:55:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.762+0000 7f0f3d2e7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0f30008d20 con 0x7f0f30096370 2024-09-16T01:55:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.762+0000 7f0f3d2e7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0f30009020 con 0x7f0f3009b690 2024-09-16T01:55:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.764+0000 7f0f37fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f300959f0 0x7f0f300a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.764+0000 7f0f37fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f300959f0 0x7f0f300a9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33844/0 (socket says 172.21.15.139:33844) 2024-09-16T01:55:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.764+0000 7f0f37fff640 1 -- 172.21.15.139:0/974936276 learned_addr learned my addr 172.21.15.139:0/974936276 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.765+0000 7f0f3cae6640 1 --2- 172.21.15.139:0/974936276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f3009b690 0x7f0f300afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.766+0000 7f0f37fff640 1 -- 172.21.15.139:0/974936276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f3009b690 msgr2=0x7f0f300afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.766+0000 7f0f37fff640 1 --2- 172.21.15.139:0/974936276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f3009b690 0x7f0f300afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.766+0000 7f0f37fff640 1 -- 172.21.15.139:0/974936276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f30096370 msgr2=0x7f0f300aa330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:55:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.766+0000 7f0f37fff640 1 --2- 172.21.15.139:0/974936276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f30096370 0x7f0f300aa330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.766+0000 7f0f37fff640 1 -- 172.21.15.139:0/974936276 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f300b03c0 con 0x7f0f300959f0 2024-09-16T01:55:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.769+0000 7f0f37fff640 1 --2- 172.21.15.139:0/974936276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f300959f0 0x7f0f300a9df0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c006010 tx=0x7f0f2c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.771+0000 7f0f357fa640 1 -- 172.21.15.139:0/974936276 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c050070 con 0x7f0f300959f0 2024-09-16T01:55:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.771+0000 7f0f357fa640 1 -- 172.21.15.139:0/974936276 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f2c003ca0 con 0x7f0f300959f0 2024-09-16T01:55:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.771+0000 7f0f357fa640 1 -- 172.21.15.139:0/974936276 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0409b0 con 0x7f0f300959f0 2024-09-16T01:55:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.773+0000 7f0f3d2e7640 1 -- 172.21.15.139:0/974936276 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f300b0650 con 0x7f0f300959f0 2024-09-16T01:55:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.773+0000 7f0f3d2e7640 1 -- 172.21.15.139:0/974936276 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f300a50e0 con 0x7f0f300959f0 2024-09-16T01:55:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.778+0000 7f0f413bb740 1 Processor -- start 2024-09-16T01:55:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.778+0000 7f0f413bb740 1 -- start start 2024-09-16T01:55:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.778+0000 7f0f357fa640 1 -- 172.21.15.139:0/974936276 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0f2c053070 con 0x7f0f300959f0 2024-09-16T01:55:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.778+0000 7f0f413bb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559228041cc0 0x5592280420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.778+0000 7f0f413bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592280425e0 0x55922804c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.778+0000 7f0f413bb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55922804ce80 0x55922804f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.778+0000 7f0f413bb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559227fa62f0 con 0x559228041cc0 2024-09-16T01:55:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.778+0000 7f0f413bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559227f4e270 con 0x5592280425e0 2024-09-16T01:55:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.778+0000 7f0f413bb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559227fa22b0 con 0x55922804ce80 2024-09-16T01:55:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.779+0000 7f0f357fa640 1 --2- 172.21.15.139:0/974936276 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0f280a3590 0x7f0f280a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.779+0000 7f0f37fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592280425e0 0x55922804c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.779+0000 7f0f37fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592280425e0 0x55922804c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:41672/0 (socket says 172.21.15.139:41672) 2024-09-16T01:55:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.779+0000 7f0f37fff640 1 -- 172.21.15.139:0/896432041 learned_addr learned my addr 172.21.15.139:0/896432041 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.779+0000 7f0f3cae6640 1 --2- 172.21.15.139:0/896432041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559228041cc0 0x5592280420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.780+0000 7f0f357fa640 1 -- 172.21.15.139:0/974936276 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9678+0+0 (secure 0 0 0) 0x7f0f2c101c20 con 0x7f0f300959f0 2024-09-16T01:55:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.780+0000 7f0f377fe640 1 --2- 172.21.15.139:0/896432041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55922804ce80 0x55922804f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.781+0000 7f0f377fe640 1 -- 172.21.15.139:0/896432041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592280425e0 msgr2=0x55922804c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.781+0000 7f0f377fe640 1 --2- 172.21.15.139:0/896432041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592280425e0 0x55922804c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.781+0000 7f0f377fe640 1 -- 172.21.15.139:0/896432041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559228041cc0 msgr2=0x5592280420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.781+0000 7f0f377fe640 1 --2- 172.21.15.139:0/896432041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559228041cc0 0x5592280420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.781+0000 7f0f377fe640 1 -- 172.21.15.139:0/896432041 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559227f4d340 con 0x55922804ce80 2024-09-16T01:55:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.781+0000 7f0f377fe640 1 --2- 172.21.15.139:0/896432041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55922804ce80 0x55922804f270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0f38050280 tx=0x7f0f38068f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afdf114b7c5763ea server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.781+0000 7f0f3cae6640 1 --2- 172.21.15.139:0/974936276 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0f280a3590 0x7f0f280a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.782+0000 7f0f227fc640 1 -- 172.21.15.139:0/896432041 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3807c380 con 0x55922804ce80 2024-09-16T01:55:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.782+0000 7f0f227fc640 1 -- 172.21.15.139:0/896432041 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f3807ca00 con 0x55922804ce80 2024-09-16T01:55:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.783+0000 7f0f413bb740 1 -- 172.21.15.139:0/896432041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55922804ce80 msgr2=0x55922804f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.783+0000 7f0f413bb740 1 --2- 172.21.15.139:0/896432041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55922804ce80 0x55922804f270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0f38050280 tx=0x7f0f38068f90 comp rx=0 tx=0).stop 2024-09-16T01:55:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.785+0000 7f0f3cae6640 1 --2- 172.21.15.139:0/974936276 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0f280a3590 0x7f0f280a5a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0f30093e50 tx=0x7f0f240046c0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.785+0000 7f0f413bb740 1 -- 172.21.15.139:0/896432041 shutdown_connections 2024-09-16T01:55:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.785+0000 7f0f413bb740 1 --2- 172.21.15.139:0/896432041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55922804ce80 0x55922804f270 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0f38050280 tx=0x7f0f38068f90 comp rx=0 tx=0).stop 2024-09-16T01:55:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.785+0000 7f0f413bb740 1 --2- 172.21.15.139:0/896432041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592280425e0 0x55922804c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.785+0000 7f0f413bb740 1 --2- 172.21.15.139:0/896432041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559228041cc0 0x5592280420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.785+0000 7f0f413bb740 1 -- 172.21.15.139:0/896432041 >> 172.21.15.139:0/896432041 conn(0x55922803e230 msgr2=0x55922803e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.788+0000 7f0f413bb740 1 -- 172.21.15.139:0/896432041 shutdown_connections 2024-09-16T01:55:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.792+0000 7f0f413bb740 1 -- 172.21.15.139:0/896432041 wait complete. 2024-09-16T01:55:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.792+0000 7f0f413bb740 1 -- start start 2024-09-16T01:55:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.792+0000 7f0f413bb740 1 Processor -- start 2024-09-16T01:55:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.795+0000 7f0f413bb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559228041cc0 0x559228049f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.795+0000 7f0f413bb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x55922804a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.795+0000 7f0f413bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55922804a9b0 0x559228058f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.795+0000 7f0f413bb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55922804fa50 con 0x559228041cc0 2024-09-16T01:55:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.795+0000 7f0f413bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559227f2f290 con 0x55922804a9b0 2024-09-16T01:55:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.795+0000 7f0f413bb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55922804fbd0 con 0x5592280425e0 2024-09-16T01:55:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.795+0000 7f0f413bb740 10 client.0 _ll_register_callbacks cb 0x559227f60140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:55:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.797+0000 7f0f37fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559228041cc0 0x559228049f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.797+0000 7f0f37fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559228041cc0 0x559228049f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33870/0 (socket says 172.21.15.139:33870) 2024-09-16T01:55:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.797+0000 7f0f37fff640 1 -- 172.21.15.139:0/1261177915 learned_addr learned my addr 172.21.15.139:0/1261177915 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.798+0000 7f0f3cae6640 1 --2- 172.21.15.139:0/1261177915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55922804a9b0 0x559228058f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.798+0000 7f0f377fe640 1 --2- 172.21.15.139:0/1261177915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x55922804a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.804+0000 7f0f37fff640 1 -- 172.21.15.139:0/1261177915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 msgr2=0x55922804a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.804+0000 7f0f37fff640 1 --2- 172.21.15.139:0/1261177915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x55922804a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.804+0000 7f0f37fff640 1 -- 172.21.15.139:0/1261177915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55922804a9b0 msgr2=0x559228058f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.804+0000 7f0f37fff640 1 --2- 172.21.15.139:0/1261177915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55922804a9b0 0x559228058f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.804+0000 7f0f37fff640 1 -- 172.21.15.139:0/1261177915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559227cb9210 con 0x559228041cc0 2024-09-16T01:55:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.810+0000 7f0f37fff640 1 --2- 172.21.15.139:0/1261177915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559228041cc0 0x559228049f30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c005c20 tx=0x7f0f2c0cd7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9ebf6cdfd98530 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.811+0000 7f0f177fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:55:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.817+0000 7f0f177fe640 1 -- 172.21.15.139:0/1261177915 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0e6060 con 0x559228041cc0 2024-09-16T01:55:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.817+0000 7f0f413bb740 1 -- 172.21.15.139:0/1261177915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559227fb3a20 con 0x559228041cc0 2024-09-16T01:55:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.817+0000 7f0f177fe640 1 -- 172.21.15.139:0/1261177915 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9678+0+0 (secure 0 0 0) 0x7f0f2c0d6200 con 0x559228041cc0 2024-09-16T01:55:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.825+0000 7f0f177fe640 1 -- 172.21.15.139:0/1261177915 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f0f2c0eb050 con 0x559228041cc0 2024-09-16T01:55:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.826+0000 7f0f413bb740 10 client.17067 fetch_fsmap learned FSMap version 71 2024-09-16T01:55:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.826+0000 7f0f413bb740 1 -- 172.21.15.139:0/1261177915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x559227d85620 con 0x559228041cc0 2024-09-16T01:55:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.833+0000 7f0f177fe640 1 -- 172.21.15.139:0/1261177915 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559227fb3a20 con 0x559228041cc0 2024-09-16T01:55:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.833+0000 7f0f177fe640 1 -- 172.21.15.139:0/1261177915 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f0f2c05d060 con 0x559228041cc0 2024-09-16T01:55:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.833+0000 7f0f413bb740 10 client.17067 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:55:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.833+0000 7f0f413bb740 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:55:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.833+0000 7f0f413bb740 20 client.17067 populate_metadata read hostname 'smithi139' 2024-09-16T01:55:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.833+0000 7f0f413bb740 1 --2- 172.21.15.139:0/1261177915 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55922805d4b0 0x55922805f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.833+0000 7f0f413bb740 4 client.17067 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:55:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.833+0000 7f0f413bb740 1 -- 172.21.15.139:0/1261177915 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559227fadd70 con 0x55922805d4b0 2024-09-16T01:55:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.838+0000 7f0f377fe640 1 --2- 172.21.15.139:0/1261177915 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55922805d4b0 0x55922805f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.840+0000 7f0f377fe640 1 --2- 172.21.15.139:0/1261177915 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55922805d4b0 0x55922805f930 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b626ff249bed7ef server_cookie=f87e91f199489556 in_seq=0 out_seq=0 2024-09-16T01:55:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.841+0000 7f0f177fe640 10 client.17067 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.841+0000 7f0f177fe640 1 -- 172.21.15.139:0/1261177915 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559227fadd70 con 0x55922805d4b0 2024-09-16T01:55:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.841+0000 7f0f177fe640 10 client.17067 handle_command_reply: tid=0 2024-09-16T01:55:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.841+0000 7f0f413bb740 1 client.17067 shutdown 2024-09-16T01:55:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.844+0000 7f0f413bb740 1 -- 172.21.15.139:0/1261177915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559228041cc0 msgr2=0x559228049f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.844+0000 7f0f413bb740 1 --2- 172.21.15.139:0/1261177915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559228041cc0 0x559228049f30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c005c20 tx=0x7f0f2c0cd7f0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.844+0000 7f0f413bb740 1 -- 172.21.15.139:0/1261177915 shutdown_connections 2024-09-16T01:55:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.844+0000 7f0f413bb740 1 --2- 172.21.15.139:0/1261177915 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55922805d4b0 0x55922805f930 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.844+0000 7f0f413bb740 1 --2- 172.21.15.139:0/1261177915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55922804a9b0 0x559228058f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.844+0000 7f0f413bb740 1 --2- 172.21.15.139:0/1261177915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x55922804a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.844+0000 7f0f413bb740 1 --2- 172.21.15.139:0/1261177915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559228041cc0 0x559228049f30 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.844+0000 7f0f413bb740 1 -- 172.21.15.139:0/1261177915 >> 172.21.15.139:0/1261177915 conn(0x55922803e230 msgr2=0x559228043f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.844+0000 7f0f177fe640 0 client.17067 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.845+0000 7f0f413bb740 1 -- 172.21.15.139:0/1261177915 shutdown_connections 2024-09-16T01:55:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.845+0000 7f0f413bb740 1 -- 172.21.15.139:0/1261177915 wait complete. 2024-09-16T01:55:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.845+0000 7f0f413bb740 20 client.17067 trim_cache size 0 max 16384 2024-09-16T01:55:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.875+0000 7f0f413bb740 1 Processor -- start 2024-09-16T01:55:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.875+0000 7f0f413bb740 1 -- start start 2024-09-16T01:55:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.875+0000 7f0f413bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559228041cc0 0x559228058cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.875+0000 7f0f413bb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x55922803d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.876+0000 7f0f37fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559228041cc0 0x559228058cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.876+0000 7f0f37fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559228041cc0 0x559228058cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:41694/0 (socket says 172.21.15.139:41694) 2024-09-16T01:55:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.876+0000 7f0f413bb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55922804a9b0 0x559228065240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.876+0000 7f0f377fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x55922803d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.876+0000 7f0f377fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x55922803d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:59140/0 (socket says 172.21.15.139:59140) 2024-09-16T01:55:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.876+0000 7f0f37fff640 1 -- 172.21.15.139:0/2579231489 learned_addr learned my addr 172.21.15.139:0/2579231489 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.876+0000 7f0f413bb740 1 -- 172.21.15.139:0/2579231489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0f30140eb0 con 0x55922804a9b0 2024-09-16T01:55:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.876+0000 7f0f3cae6640 1 --2- 172.21.15.139:0/2579231489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55922804a9b0 0x559228065240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.876+0000 7f0f413bb740 1 -- 172.21.15.139:0/2579231489 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0f30142cf0 con 0x559228041cc0 2024-09-16T01:55:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.876+0000 7f0f413bb740 1 -- 172.21.15.139:0/2579231489 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0f30141640 con 0x5592280425e0 2024-09-16T01:55:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.880+0000 7f0f377fe640 1 -- 172.21.15.139:0/2579231489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559228041cc0 msgr2=0x559228058cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.880+0000 7f0f377fe640 1 --2- 172.21.15.139:0/2579231489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559228041cc0 0x559228058cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.880+0000 7f0f377fe640 1 -- 172.21.15.139:0/2579231489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55922804a9b0 msgr2=0x559228065240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.880+0000 7f0f377fe640 1 --2- 172.21.15.139:0/2579231489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55922804a9b0 0x559228065240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.880+0000 7f0f377fe640 1 -- 172.21.15.139:0/2579231489 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559228041ae0 con 0x5592280425e0 2024-09-16T01:55:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.884+0000 7f0f377fe640 1 --2- 172.21.15.139:0/2579231489 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x55922803d150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x559228042c80 tx=0x7f0f380506b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a85ad58925898e7b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.886+0000 7f0f21ffb640 1 -- 172.21.15.139:0/2579231489 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3806d0d0 con 0x5592280425e0 2024-09-16T01:55:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.886+0000 7f0f21ffb640 1 -- 172.21.15.139:0/2579231489 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f3806d270 con 0x5592280425e0 2024-09-16T01:55:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.886+0000 7f0f21ffb640 1 -- 172.21.15.139:0/2579231489 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3807dcf0 con 0x5592280425e0 2024-09-16T01:55:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.887+0000 7f0f413bb740 1 -- 172.21.15.139:0/2579231489 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 msgr2=0x55922803d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.887+0000 7f0f413bb740 1 --2- 172.21.15.139:0/2579231489 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x55922803d150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x559228042c80 tx=0x7f0f380506b0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.888+0000 7f0f413bb740 1 -- 172.21.15.139:0/2579231489 shutdown_connections 2024-09-16T01:55:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.888+0000 7f0f413bb740 1 --2- 172.21.15.139:0/2579231489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55922804a9b0 0x559228065240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.888+0000 7f0f413bb740 1 --2- 172.21.15.139:0/2579231489 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x55922803d150 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.888+0000 7f0f413bb740 1 --2- 172.21.15.139:0/2579231489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559228041cc0 0x559228058cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.888+0000 7f0f413bb740 1 -- 172.21.15.139:0/2579231489 >> 172.21.15.139:0/2579231489 conn(0x55922803e230 msgr2=0x55922803ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.888+0000 7f0f413bb740 1 -- 172.21.15.139:0/2579231489 shutdown_connections 2024-09-16T01:55:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.890+0000 7f0f413bb740 1 -- 172.21.15.139:0/2579231489 wait complete. 2024-09-16T01:55:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.891+0000 7f0f413bb740 1 -- start start 2024-09-16T01:55:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.891+0000 7f0f413bb740 1 Processor -- start 2024-09-16T01:55:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.891+0000 7f0f413bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559228041cc0 0x559228050400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.891+0000 7f0f413bb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x559228050940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.891+0000 7f0f413bb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55922804a9b0 0x5592280c6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.891+0000 7f0f413bb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0f30142560 con 0x55922804a9b0 2024-09-16T01:55:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.891+0000 7f0f413bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0f30141dd0 con 0x559228041cc0 2024-09-16T01:55:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.891+0000 7f0f413bb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0f30095e30 con 0x5592280425e0 2024-09-16T01:55:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.891+0000 7f0f413bb740 10 client.0 _ll_register_callbacks cb 0x7f0f2c01a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:55:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.892+0000 7f0f377fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x559228050940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.892+0000 7f0f377fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x559228050940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:59146/0 (socket says 172.21.15.139:59146) 2024-09-16T01:55:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.892+0000 7f0f377fe640 1 -- 172.21.15.139:0/1466614927 learned_addr learned my addr 172.21.15.139:0/1466614927 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.893+0000 7f0f3cae6640 1 --2- 172.21.15.139:0/1466614927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55922804a9b0 0x5592280c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.893+0000 7f0f377fe640 1 -- 172.21.15.139:0/1466614927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559228041cc0 msgr2=0x559228050400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:55:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.893+0000 7f0f377fe640 1 --2- 172.21.15.139:0/1466614927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559228041cc0 0x559228050400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.893+0000 7f0f377fe640 1 -- 172.21.15.139:0/1466614927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55922804a9b0 msgr2=0x5592280c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:55:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.893+0000 7f0f377fe640 1 --2- 172.21.15.139:0/1466614927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55922804a9b0 0x5592280c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.893+0000 7f0f377fe640 1 -- 172.21.15.139:0/1466614927 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592280c6740 con 0x5592280425e0 2024-09-16T01:55:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.894+0000 7f0f377fe640 1 --2- 172.21.15.139:0/1466614927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x559228050940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0f38051790 tx=0x7f0f38065460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d30d1e013828d9e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.894+0000 7f0f177fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:55:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.894+0000 7f0f177fe640 1 -- 172.21.15.139:0/1466614927 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38097020 con 0x5592280425e0 2024-09-16T01:55:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.894+0000 7f0f177fe640 1 -- 172.21.15.139:0/1466614927 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9678+0+0 (secure 0 0 0) 0x7f0f3806a990 con 0x5592280425e0 2024-09-16T01:55:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.894+0000 7f0f413bb740 1 -- 172.21.15.139:0/1466614927 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f3806d270 con 0x5592280425e0 2024-09-16T01:55:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.894+0000 7f0f177fe640 1 -- 172.21.15.139:0/1466614927 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f0f3806d8b0 con 0x5592280425e0 2024-09-16T01:55:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.894+0000 7f0f413bb740 10 client.26464 fetch_fsmap learned FSMap version 71 2024-09-16T01:55:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.894+0000 7f0f413bb740 1 -- 172.21.15.139:0/1466614927 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5592280c7a50 con 0x5592280425e0 2024-09-16T01:55:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.895+0000 7f0f177fe640 1 -- 172.21.15.139:0/1466614927 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f3806d270 con 0x5592280425e0 2024-09-16T01:55:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.895+0000 7f0f177fe640 1 -- 172.21.15.139:0/1466614927 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f0f38087040 con 0x5592280425e0 2024-09-16T01:55:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.895+0000 7f0f413bb740 10 client.26464 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:55:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.896+0000 7f0f413bb740 10 client.26464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:55:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.896+0000 7f0f413bb740 20 client.26464 populate_metadata read hostname 'smithi139' 2024-09-16T01:55:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.896+0000 7f0f413bb740 1 --2- 172.21.15.139:0/1466614927 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5592280c8110 0x5592280ca530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.896+0000 7f0f413bb740 4 client.26464 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:55:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.896+0000 7f0f413bb740 1 -- 172.21.15.139:0/1466614927 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559228045a70 con 0x5592280c8110 2024-09-16T01:55:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.896+0000 7f0f37fff640 1 --2- 172.21.15.139:0/1466614927 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5592280c8110 0x5592280ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.897+0000 7f0f177fe640 10 client.26464 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.898+0000 7f0f37fff640 1 --2- 172.21.15.139:0/1466614927 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5592280c8110 0x5592280ca530 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d719936e43f37d1 server_cookie=e0b030569badf51d in_seq=0 out_seq=0 2024-09-16T01:55:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.988+0000 7f0f177fe640 1 -- 172.21.15.139:0/1466614927 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559228045a70 con 0x5592280c8110 2024-09-16T01:55:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.989+0000 7f0f177fe640 10 client.26464 handle_command_reply: tid=0 2024-09-16T01:55:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.990+0000 7f0f413bb740 1 client.26464 shutdown 2024-09-16T01:55:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.990+0000 7f0f413bb740 1 -- 172.21.15.139:0/1466614927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 msgr2=0x559228050940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.990+0000 7f0f413bb740 1 --2- 172.21.15.139:0/1466614927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x559228050940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0f38051790 tx=0x7f0f38065460 comp rx=0 tx=0).stop 2024-09-16T01:55:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.990+0000 7f0f413bb740 1 -- 172.21.15.139:0/1466614927 shutdown_connections 2024-09-16T01:55:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.990+0000 7f0f413bb740 1 --2- 172.21.15.139:0/1466614927 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5592280c8110 0x5592280ca530 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.990+0000 7f0f413bb740 1 --2- 172.21.15.139:0/1466614927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55922804a9b0 0x5592280c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.990+0000 7f0f413bb740 1 --2- 172.21.15.139:0/1466614927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5592280425e0 0x559228050940 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.990+0000 7f0f177fe640 0 client.26464 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:55:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.990+0000 7f0f413bb740 1 --2- 172.21.15.139:0/1466614927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559228041cc0 0x559228050400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.990+0000 7f0f413bb740 1 -- 172.21.15.139:0/1466614927 >> 172.21.15.139:0/1466614927 conn(0x55922803e230 msgr2=0x559228043f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.990+0000 7f0f413bb740 1 -- 172.21.15.139:0/1466614927 shutdown_connections 2024-09-16T01:55:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.990+0000 7f0f413bb740 1 -- 172.21.15.139:0/1466614927 wait complete. 2024-09-16T01:55:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.990+0000 7f0f413bb740 20 client.26464 trim_cache size 0 max 16384 2024-09-16T01:55:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:55:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:55:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:55:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:55:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.991+0000 7f0f3d2e7640 1 -- 172.21.15.139:0/974936276 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0f280a3590 msgr2=0x7f0f280a5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.991+0000 7f0f3d2e7640 1 --2- 172.21.15.139:0/974936276 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0f280a3590 0x7f0f280a5a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0f30093e50 tx=0x7f0f240046c0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.991+0000 7f0f3d2e7640 1 -- 172.21.15.139:0/974936276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f300959f0 msgr2=0x7f0f300a9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.991+0000 7f0f3d2e7640 1 --2- 172.21.15.139:0/974936276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f300959f0 0x7f0f300a9df0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c006010 tx=0x7f0f2c0038a0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.991+0000 7f0f3d2e7640 1 -- 172.21.15.139:0/974936276 shutdown_connections 2024-09-16T01:55:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.991+0000 7f0f3d2e7640 1 --2- 172.21.15.139:0/974936276 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0f280a3590 0x7f0f280a5a50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.991+0000 7f0f3d2e7640 1 --2- 172.21.15.139:0/974936276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f3009b690 0x7f0f300afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.991+0000 7f0f3d2e7640 1 --2- 172.21.15.139:0/974936276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f30096370 0x7f0f300aa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.991+0000 7f0f3d2e7640 1 --2- 172.21.15.139:0/974936276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f300959f0 0x7f0f300a9df0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.991+0000 7f0f3d2e7640 1 -- 172.21.15.139:0/974936276 >> 172.21.15.139:0/974936276 conn(0x7f0f3000b350 msgr2=0x7f0f30094770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.992+0000 7f0f3d2e7640 1 -- 172.21.15.139:0/974936276 shutdown_connections 2024-09-16T01:55:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:55:52.992+0000 7f0f3d2e7640 1 -- 172.21.15.139:0/974936276 wait complete. 2024-09-16T01:55:53.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:53 smithi139 ceph-mon[31175]: pgmap v489: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 982 KiB/s wr, 279 op/s 2024-09-16T01:55:53.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:53 smithi176 ceph-mon[35324]: pgmap v489: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 982 KiB/s wr, 279 op/s 2024-09-16T01:55:53.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:53 smithi164 ceph-mon[35121]: pgmap v489: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 982 KiB/s wr, 279 op/s 2024-09-16T01:55:54.412 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T01:55:54.413 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-09-16T01:55:54.413 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:55:54.952 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:55:55.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:55 smithi139 ceph-mon[31175]: pgmap v490: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 267 op/s 2024-09-16T01:55:55.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:55 smithi176 ceph-mon[35324]: pgmap v490: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 267 op/s 2024-09-16T01:55:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:55 smithi164 ceph-mon[35121]: pgmap v490: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 267 op/s 2024-09-16T01:55:55.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.695+0000 7f1fef40d640 1 -- 172.21.15.139:0/2439554984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fe81008c0 msgr2=0x7f1fe8100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:55.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.695+0000 7f1fef40d640 1 --2- 172.21.15.139:0/2439554984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fe81008c0 0x7f1fe8100ca0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1fe401a910 tx=0x7f1fe403b470 comp rx=0 tx=0).stop 2024-09-16T01:55:55.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.695+0000 7f1fef40d640 1 -- 172.21.15.139:0/2439554984 shutdown_connections 2024-09-16T01:55:55.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.695+0000 7f1fef40d640 1 --2- 172.21.15.139:0/2439554984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fe8105ec0 0x7f1fe8106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:55.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.695+0000 7f1fef40d640 1 --2- 172.21.15.139:0/2439554984 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe81011e0 0x7f1fe8105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:55.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.695+0000 7f1fef40d640 1 --2- 172.21.15.139:0/2439554984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fe81008c0 0x7f1fe8100ca0 secure :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1fe401a910 tx=0x7f1fe403b470 comp rx=0 tx=0).stop 2024-09-16T01:55:55.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.695+0000 7f1fef40d640 1 -- 172.21.15.139:0/2439554984 >> 172.21.15.139:0/2439554984 conn(0x7f1fe80fc7d0 msgr2=0x7f1fe80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:55.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.696+0000 7f1fef40d640 1 -- 172.21.15.139:0/2439554984 shutdown_connections 2024-09-16T01:55:55.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.697+0000 7f1fef40d640 1 -- 172.21.15.139:0/2439554984 wait complete. 2024-09-16T01:55:55.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.698+0000 7f1fef40d640 1 Processor -- start 2024-09-16T01:55:55.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.698+0000 7f1fef40d640 1 -- start start 2024-09-16T01:55:55.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.698+0000 7f1fef40d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fe81011e0 0x7f1fe8114f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:55.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.698+0000 7f1fef40d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe8105ec0 0x7f1fe8110fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:55.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.698+0000 7f1fef40d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fe8111510 0x7f1fe81119c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:55.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.698+0000 7f1fef40d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81098c0 con 0x7f1fe8111510 2024-09-16T01:55:55.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.698+0000 7f1fef40d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8109740 con 0x7f1fe81011e0 2024-09-16T01:55:55.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.698+0000 7f1fef40d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8109a40 con 0x7f1fe8105ec0 2024-09-16T01:55:55.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.699+0000 7f1fec9a8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe8105ec0 0x7f1fe8110fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:55.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.699+0000 7f1fec9a8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe8105ec0 0x7f1fe8110fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:59160/0 (socket says 172.21.15.139:59160) 2024-09-16T01:55:55.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.699+0000 7f1fec9a8640 1 -- 172.21.15.139:0/3741919343 learned_addr learned my addr 172.21.15.139:0/3741919343 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:55:55.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.700+0000 7f1fed1a9640 1 --2- 172.21.15.139:0/3741919343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fe81011e0 0x7f1fe8114f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:55.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.701+0000 7f1fec9a8640 1 -- 172.21.15.139:0/3741919343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fe81011e0 msgr2=0x7f1fe8114f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:55.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.701+0000 7f1fec9a8640 1 --2- 172.21.15.139:0/3741919343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fe81011e0 0x7f1fe8114f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:55.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.701+0000 7f1fec9a8640 1 -- 172.21.15.139:0/3741919343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fe8111510 msgr2=0x7f1fe81119c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:55.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.701+0000 7f1fec9a8640 1 --2- 172.21.15.139:0/3741919343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fe8111510 0x7f1fe81119c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:55.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.701+0000 7f1fec9a8640 1 -- 172.21.15.139:0/3741919343 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe81a2b30 con 0x7f1fe8105ec0 2024-09-16T01:55:55.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.701+0000 7f1fec9a8640 1 --2- 172.21.15.139:0/3741919343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe8105ec0 0x7f1fe8110fd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800d9b0 tx=0x7f1fd800de70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:55.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.702+0000 7f1fde7fc640 1 -- 172.21.15.139:0/3741919343 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8017070 con 0x7f1fe8105ec0 2024-09-16T01:55:55.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.702+0000 7f1fef40d640 1 -- 172.21.15.139:0/3741919343 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe81a2e20 con 0x7f1fe8105ec0 2024-09-16T01:55:55.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.702+0000 7f1fef40d640 1 -- 172.21.15.139:0/3741919343 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe81a32d0 con 0x7f1fe8105ec0 2024-09-16T01:55:55.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.703+0000 7f1fde7fc640 1 -- 172.21.15.139:0/3741919343 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fd8009bd0 con 0x7f1fe8105ec0 2024-09-16T01:55:55.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.703+0000 7f1fde7fc640 1 -- 172.21.15.139:0/3741919343 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd800f6b0 con 0x7f1fe8105ec0 2024-09-16T01:55:55.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.705+0000 7f1fde7fc640 1 -- 172.21.15.139:0/3741919343 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1fd800f890 con 0x7f1fe8105ec0 2024-09-16T01:55:55.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.706+0000 7f1fef40d640 1 -- 172.21.15.139:0/3741919343 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fcc005290 con 0x7f1fe8105ec0 2024-09-16T01:55:55.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.706+0000 7f1fde7fc640 1 --2- 172.21.15.139:0/3741919343 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1fd40a35c0 0x7f1fd40a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:55:55.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.706+0000 7f1fde7fc640 1 -- 172.21.15.139:0/3741919343 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9678+0+0 (secure 0 0 0) 0x7f1fd80cfa30 con 0x7f1fe8105ec0 2024-09-16T01:55:55.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.709+0000 7f1fed1a9640 1 --2- 172.21.15.139:0/3741919343 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1fd40a35c0 0x7f1fd40a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:55:55.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.709+0000 7f1fed1a9640 1 --2- 172.21.15.139:0/3741919343 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1fd40a35c0 0x7f1fd40a5a80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1fe403b9e0 tx=0x7f1fe4005fd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:55:55.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:55.717+0000 7f1fde7fc640 1 -- 172.21.15.139:0/3741919343 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fd809bf30 con 0x7f1fe8105ec0 2024-09-16T01:55:56.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:56.112+0000 7f1fef40d640 1 -- 172.21.15.139:0/3741919343 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fcc005850 con 0x7f1fe8105ec0 2024-09-16T01:55:56.117 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:55:56.117 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":71,"btime":"2024-09-16T01:54:19:332763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:54:19.051272+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:55:56.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:56.114+0000 7f1fde7fc640 1 -- 172.21.15.139:0/3741919343 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+12031 (secure 0 0 0) 0x7f1fd809b830 con 0x7f1fe8105ec0 2024-09-16T01:55:56.117 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 71 2024-09-16T01:55:56.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:56.118+0000 7f1fef40d640 1 -- 172.21.15.139:0/3741919343 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1fd40a35c0 msgr2=0x7f1fd40a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:56.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:56.118+0000 7f1fef40d640 1 --2- 172.21.15.139:0/3741919343 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1fd40a35c0 0x7f1fd40a5a80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1fe403b9e0 tx=0x7f1fe4005fd0 comp rx=0 tx=0).stop 2024-09-16T01:55:56.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:56.118+0000 7f1fef40d640 1 -- 172.21.15.139:0/3741919343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe8105ec0 msgr2=0x7f1fe8110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:55:56.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:56.118+0000 7f1fef40d640 1 --2- 172.21.15.139:0/3741919343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe8105ec0 0x7f1fe8110fd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800d9b0 tx=0x7f1fd800de70 comp rx=0 tx=0).stop 2024-09-16T01:55:56.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:56.120+0000 7f1fef40d640 1 -- 172.21.15.139:0/3741919343 shutdown_connections 2024-09-16T01:55:56.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:56.120+0000 7f1fef40d640 1 --2- 172.21.15.139:0/3741919343 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1fd40a35c0 0x7f1fd40a5a80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:56.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:56.120+0000 7f1fef40d640 1 --2- 172.21.15.139:0/3741919343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fe8111510 0x7f1fe81119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:56.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:56.120+0000 7f1fef40d640 1 --2- 172.21.15.139:0/3741919343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe8105ec0 0x7f1fe8110fd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:56.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:56.120+0000 7f1fef40d640 1 --2- 172.21.15.139:0/3741919343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fe81011e0 0x7f1fe8114f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:55:56.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:56.120+0000 7f1fef40d640 1 -- 172.21.15.139:0/3741919343 >> 172.21.15.139:0/3741919343 conn(0x7f1fe80fc7d0 msgr2=0x7f1fe8103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:55:56.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:56.122+0000 7f1fef40d640 1 -- 172.21.15.139:0/3741919343 shutdown_connections 2024-09-16T01:55:56.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:56.122+0000 7f1fef40d640 1 -- 172.21.15.139:0/3741919343 wait complete. 2024-09-16T01:55:56.389 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:56 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3741919343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:55:56.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:56 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3741919343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:55:56.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:56 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3741919343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:55:57.026 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0' 2024-09-16T01:55:57.026 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0 2024-09-16T01:55:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:57 smithi139 ceph-mon[31175]: pgmap v491: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 249 op/s 2024-09-16T01:55:57.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:57 smithi176 ceph-mon[35324]: pgmap v491: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 249 op/s 2024-09-16T01:55:57.457 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:55:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:57 smithi164 ceph-mon[35121]: pgmap v491: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 249 op/s 2024-09-16T01:55:58.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:58 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:58.609 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:58 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:58.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:58 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:55:59.539 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:55:59 smithi139 ceph-mon[31175]: pgmap v492: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 228 op/s 2024-09-16T01:55:59.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:55:59 smithi164 ceph-mon[35121]: pgmap v492: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 228 op/s 2024-09-16T01:55:59.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:55:59 smithi176 ceph-mon[35324]: pgmap v492: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 228 op/s 2024-09-16T01:56:00.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.783+0000 7fabad3e2640 1 -- 172.21.15.139:0/2840172055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8108e00 msgr2=0x7faba810ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:00.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.783+0000 7fabad3e2640 1 --2- 172.21.15.139:0/2840172055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8108e00 0x7faba810ce00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fab90009900 tx=0x7fab9002a440 comp rx=0 tx=0).stop 2024-09-16T01:56:00.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.786+0000 7fabad3e2640 1 -- 172.21.15.139:0/2840172055 shutdown_connections 2024-09-16T01:56:00.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.786+0000 7fabad3e2640 1 --2- 172.21.15.139:0/2840172055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faba810d530 0x7faba81150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.786+0000 7fabad3e2640 1 --2- 172.21.15.139:0/2840172055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8108e00 0x7faba810ce00 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.786+0000 7fabad3e2640 1 --2- 172.21.15.139:0/2840172055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba8108450 0x7faba8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.786+0000 7fabad3e2640 1 -- 172.21.15.139:0/2840172055 >> 172.21.15.139:0/2840172055 conn(0x7faba8077ba0 msgr2=0x7faba8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:00.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.786+0000 7fabad3e2640 1 -- 172.21.15.139:0/2840172055 shutdown_connections 2024-09-16T01:56:00.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.786+0000 7fabad3e2640 1 -- 172.21.15.139:0/2840172055 wait complete. 2024-09-16T01:56:00.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.787+0000 7fabad3e2640 1 Processor -- start 2024-09-16T01:56:00.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.788+0000 7fabad3e2640 1 -- start start 2024-09-16T01:56:00.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.788+0000 7fabad3e2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faba8108450 0x7faba80828b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.788+0000 7fabad3e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8108e00 0x7faba807b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.788+0000 7fabad3e2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba810d530 0x7faba807bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.788+0000 7fabad3e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faba81186c0 con 0x7faba8108e00 2024-09-16T01:56:00.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.788+0000 7fabad3e2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faba8118540 con 0x7faba810d530 2024-09-16T01:56:00.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.788+0000 7fabad3e2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7faba8118840 con 0x7faba8108450 2024-09-16T01:56:00.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.788+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faba8108450 0x7faba80828b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.788+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faba8108450 0x7faba80828b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:59174/0 (socket says 172.21.15.139:59174) 2024-09-16T01:56:00.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.788+0000 7faba6ffd640 1 -- 172.21.15.139:0/2569236534 learned_addr learned my addr 172.21.15.139:0/2569236534 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:00.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.788+0000 7faba67fc640 1 --2- 172.21.15.139:0/2569236534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8108e00 0x7faba807b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.789+0000 7faba67fc640 1 -- 172.21.15.139:0/2569236534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faba8108450 msgr2=0x7faba80828b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:00.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.789+0000 7faba67fc640 1 --2- 172.21.15.139:0/2569236534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faba8108450 0x7faba80828b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.789+0000 7faba67fc640 1 -- 172.21.15.139:0/2569236534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba810d530 msgr2=0x7faba807bed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:56:00.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.789+0000 7faba67fc640 1 --2- 172.21.15.139:0/2569236534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba810d530 0x7faba807bed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.789+0000 7faba67fc640 1 -- 172.21.15.139:0/2569236534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba807c510 con 0x7faba8108e00 2024-09-16T01:56:00.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.789+0000 7faba67fc640 1 --2- 172.21.15.139:0/2569236534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8108e00 0x7faba807b990 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fab9002aa30 tx=0x7fab90003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:00.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.790+0000 7fab9ffff640 1 -- 172.21.15.139:0/2569236534 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9003f070 con 0x7faba8108e00 2024-09-16T01:56:00.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.790+0000 7fab9ffff640 1 -- 172.21.15.139:0/2569236534 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab9003b460 con 0x7faba8108e00 2024-09-16T01:56:00.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.790+0000 7fab9ffff640 1 -- 172.21.15.139:0/2569236534 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90035030 con 0x7faba8108e00 2024-09-16T01:56:00.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.790+0000 7fabad3e2640 1 -- 172.21.15.139:0/2569236534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba807c7a0 con 0x7faba8108e00 2024-09-16T01:56:00.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.790+0000 7fabad3e2640 1 -- 172.21.15.139:0/2569236534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba80805b0 con 0x7faba8108e00 2024-09-16T01:56:00.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.793+0000 7fab9ffff640 1 -- 172.21.15.139:0/2569236534 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fab90042070 con 0x7faba8108e00 2024-09-16T01:56:00.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.794+0000 7fabaf7bb740 1 Processor -- start 2024-09-16T01:56:00.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.794+0000 7fabaf7bb740 1 -- start start 2024-09-16T01:56:00.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.794+0000 7fab9ffff640 1 --2- 172.21.15.139:0/2569236534 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fab940a3540 0x7fab940a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.794+0000 7faba77fe640 1 --2- 172.21.15.139:0/2569236534 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fab940a3540 0x7fab940a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.794+0000 7fab9ffff640 1 -- 172.21.15.139:0/2569236534 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9678+0+0 (secure 0 0 0) 0x7fab900f09a0 con 0x7faba8108e00 2024-09-16T01:56:00.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.795+0000 7fabaf7bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757962760 0x55b757962b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.795+0000 7faba77fe640 1 --2- 172.21.15.139:0/2569236534 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fab940a3540 0x7fab940a5a00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fab98009690 tx=0x7fab98006c70 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:00.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.795+0000 7fabaf7bb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757963080 0x55b75796d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.795+0000 7fabaf7bb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b75796d940 0x55b75796fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.795+0000 7fabaf7bb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b7578c82a0 con 0x55b757963080 2024-09-16T01:56:00.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.795+0000 7fabaf7bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b75786fb10 con 0x55b757962760 2024-09-16T01:56:00.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.795+0000 7fabaf7bb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b7578c4260 con 0x55b75796d940 2024-09-16T01:56:00.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.795+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757962760 0x55b757962b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.795+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757962760 0x55b757962b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:41740/0 (socket says 172.21.15.139:41740) 2024-09-16T01:56:00.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.795+0000 7faba6ffd640 1 -- 172.21.15.139:0/563805273 learned_addr learned my addr 172.21.15.139:0/563805273 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:00.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.795+0000 7faba77fe640 1 --2- 172.21.15.139:0/563805273 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b75796d940 0x55b75796fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.796+0000 7faba67fc640 1 --2- 172.21.15.139:0/563805273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757963080 0x55b75796d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.796+0000 7faba6ffd640 1 -- 172.21.15.139:0/563805273 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b75796d940 msgr2=0x55b75796fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:00.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.796+0000 7faba6ffd640 1 --2- 172.21.15.139:0/563805273 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b75796d940 0x55b75796fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.796+0000 7faba6ffd640 1 -- 172.21.15.139:0/563805273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757963080 msgr2=0x55b75796d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:00.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.796+0000 7faba6ffd640 1 --2- 172.21.15.139:0/563805273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757963080 0x55b75796d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.796+0000 7faba6ffd640 1 -- 172.21.15.139:0/563805273 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b757639b90 con 0x55b757962760 2024-09-16T01:56:00.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.796+0000 7faba6ffd640 1 --2- 172.21.15.139:0/563805273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757962760 0x55b757962b40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faba000e2b0 tx=0x7faba0009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=204914e73ad2e430 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:00.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.796+0000 7fab9d7fa640 1 -- 172.21.15.139:0/563805273 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0018020 con 0x55b757962760 2024-09-16T01:56:00.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.796+0000 7fab9d7fa640 1 -- 172.21.15.139:0/563805273 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba0014d70 con 0x55b757962760 2024-09-16T01:56:00.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.796+0000 7fab9d7fa640 1 -- 172.21.15.139:0/563805273 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0013dc0 con 0x55b757962760 2024-09-16T01:56:00.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.797+0000 7fabaf7bb740 1 -- 172.21.15.139:0/563805273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757962760 msgr2=0x55b757962b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:00.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.797+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/563805273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757962760 0x55b757962b40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faba000e2b0 tx=0x7faba0009770 comp rx=0 tx=0).stop 2024-09-16T01:56:00.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.797+0000 7fabaf7bb740 1 -- 172.21.15.139:0/563805273 shutdown_connections 2024-09-16T01:56:00.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.797+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/563805273 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b75796d940 0x55b75796fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.797+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/563805273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757963080 0x55b75796d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.797+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/563805273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757962760 0x55b757962b40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.797+0000 7fabaf7bb740 1 -- 172.21.15.139:0/563805273 >> 172.21.15.139:0/563805273 conn(0x55b75795ecf0 msgr2=0x55b75795f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:00.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.797+0000 7fabaf7bb740 1 -- 172.21.15.139:0/563805273 shutdown_connections 2024-09-16T01:56:00.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.797+0000 7fabaf7bb740 1 -- 172.21.15.139:0/563805273 wait complete. 2024-09-16T01:56:00.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7fabaf7bb740 1 -- start start 2024-09-16T01:56:00.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7fabaf7bb740 1 Processor -- start 2024-09-16T01:56:00.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7fabaf7bb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b757962760 0x55b75796a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7fabaf7bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757963080 0x55b75796ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7fabaf7bb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b75796d940 0x55b75796b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7fabaf7bb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b757970510 con 0x55b75796d940 2024-09-16T01:56:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7fabaf7bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b7578506e0 con 0x55b757963080 2024-09-16T01:56:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7fabaf7bb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b757970690 con 0x55b757962760 2024-09-16T01:56:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7fabaf7bb740 10 client.0 _ll_register_callbacks cb 0x55b757874a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7faba77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b75796d940 0x55b75796b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7faba77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b75796d940 0x55b75796b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33936/0 (socket says 172.21.15.139:33936) 2024-09-16T01:56:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7faba77fe640 1 -- 172.21.15.139:0/3884045864 learned_addr learned my addr 172.21.15.139:0/3884045864 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7faba67fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757963080 0x55b75796ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7faba77fe640 1 -- 172.21.15.139:0/3884045864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b757962760 msgr2=0x55b75796a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7faba77fe640 1 --2- 172.21.15.139:0/3884045864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b757962760 0x55b75796a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7faba77fe640 1 -- 172.21.15.139:0/3884045864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757963080 msgr2=0x55b75796ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7faba77fe640 1 --2- 172.21.15.139:0/3884045864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757963080 0x55b75796ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.798+0000 7faba77fe640 1 -- 172.21.15.139:0/3884045864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b757979980 con 0x55b75796d940 2024-09-16T01:56:00.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7faba77fe640 1 --2- 172.21.15.139:0/3884045864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b75796d940 0x55b75796b1d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fab9800b470 tx=0x7fab980047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3772e20fd0e7dbf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:00.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fab8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:56:00.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fab8e7fc640 1 -- 172.21.15.139:0/3884045864 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98023020 con 0x55b75796d940 2024-09-16T01:56:00.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fabaf7bb740 1 -- 172.21.15.139:0/3884045864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba0014d70 con 0x55b75796d940 2024-09-16T01:56:00.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fab8e7fc640 1 -- 172.21.15.139:0/3884045864 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9678+0+0 (secure 0 0 0) 0x7fab9801f850 con 0x55b75796d940 2024-09-16T01:56:00.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fab8e7fc640 1 -- 172.21.15.139:0/3884045864 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fab9802e050 con 0x55b75796d940 2024-09-16T01:56:00.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fabaf7bb740 10 client.17085 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:00.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fabaf7bb740 1 -- 172.21.15.139:0/3884045864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55b75797c360 con 0x55b75796d940 2024-09-16T01:56:00.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fab8e7fc640 1 -- 172.21.15.139:0/3884045864 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba0014d70 con 0x55b75796d940 2024-09-16T01:56:00.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fab8e7fc640 1 -- 172.21.15.139:0/3884045864 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7fab98012070 con 0x55b75796d940 2024-09-16T01:56:00.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fabaf7bb740 10 client.17085 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:00.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fabaf7bb740 10 client.17085 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:56:00.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fabaf7bb740 20 client.17085 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:00.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/3884045864 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b75797d880 0x55b75797fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fabaf7bb740 4 client.17085 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:56:00.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.799+0000 7fabaf7bb740 1 -- 172.21.15.139:0/3884045864 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7578d59d0 con 0x55b75797d880 2024-09-16T01:56:00.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.800+0000 7faba6ffd640 1 --2- 172.21.15.139:0/3884045864 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b75797d880 0x55b75797fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.800+0000 7fab8e7fc640 10 client.17085 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:00.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.800+0000 7faba6ffd640 1 --2- 172.21.15.139:0/3884045864 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b75797d880 0x55b75797fd20 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c23bbc64b6af3a46 server_cookie=755b31f916f09d99 in_seq=0 out_seq=0 2024-09-16T01:56:00.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fab8e7fc640 1 -- 172.21.15.139:0/3884045864 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7578d59d0 con 0x55b75797d880 2024-09-16T01:56:00.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fab8e7fc640 10 client.17085 handle_command_reply: tid=0 2024-09-16T01:56:00.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fabaf7bb740 1 client.17085 shutdown 2024-09-16T01:56:00.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fabaf7bb740 1 -- 172.21.15.139:0/3884045864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b75796d940 msgr2=0x55b75796b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:00.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/3884045864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b75796d940 0x55b75796b1d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fab9800b470 tx=0x7fab980047a0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fabaf7bb740 1 -- 172.21.15.139:0/3884045864 shutdown_connections 2024-09-16T01:56:00.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/3884045864 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b75797d880 0x55b75797fd20 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/3884045864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b75796d940 0x55b75796b1d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fab8e7fc640 0 client.17085 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:00.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/3884045864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757963080 0x55b75796ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/3884045864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b757962760 0x55b75796a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fabaf7bb740 1 -- 172.21.15.139:0/3884045864 >> 172.21.15.139:0/3884045864 conn(0x55b75795ecf0 msgr2=0x55b75795de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:00.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fabaf7bb740 1 -- 172.21.15.139:0/3884045864 shutdown_connections 2024-09-16T01:56:00.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fabaf7bb740 1 -- 172.21.15.139:0/3884045864 wait complete. 2024-09-16T01:56:00.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.801+0000 7fabaf7bb740 20 client.17085 trim_cache size 0 max 16384 2024-09-16T01:56:00.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.824+0000 7fabaf7bb740 1 Processor -- start 2024-09-16T01:56:00.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.824+0000 7fabaf7bb740 1 -- start start 2024-09-16T01:56:00.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7fabaf7bb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757962760 0x55b7579651c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7fabaf7bb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b757963080 0x55b7579855f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7fabaf7bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b75796d940 0x55b757985b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7fabaf7bb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faba81be0d0 con 0x55b757962760 2024-09-16T01:56:00.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7fabaf7bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faba81bff10 con 0x55b75796d940 2024-09-16T01:56:00.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7fabaf7bb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7faba81be860 con 0x55b757963080 2024-09-16T01:56:00.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757962760 0x55b7579651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757962760 0x55b7579651c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33950/0 (socket says 172.21.15.139:33950) 2024-09-16T01:56:00.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7faba6ffd640 1 -- 172.21.15.139:0/426853087 learned_addr learned my addr 172.21.15.139:0/426853087 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:00.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7faba67fc640 1 --2- 172.21.15.139:0/426853087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b757963080 0x55b7579855f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7faba77fe640 1 --2- 172.21.15.139:0/426853087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b75796d940 0x55b757985b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7faba6ffd640 1 -- 172.21.15.139:0/426853087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b757963080 msgr2=0x55b7579855f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:00.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7faba6ffd640 1 --2- 172.21.15.139:0/426853087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b757963080 0x55b7579855f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7faba6ffd640 1 -- 172.21.15.139:0/426853087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b75796d940 msgr2=0x55b757985b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:00.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7faba6ffd640 1 --2- 172.21.15.139:0/426853087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b75796d940 0x55b757985b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.825+0000 7faba6ffd640 1 -- 172.21.15.139:0/426853087 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b75795a260 con 0x55b757962760 2024-09-16T01:56:00.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.826+0000 7faba6ffd640 1 --2- 172.21.15.139:0/426853087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757962760 0x55b7579651c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7faba0013930 tx=0x7faba0009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1d8169f9c26e32 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:00.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.826+0000 7fab9cff9640 1 -- 172.21.15.139:0/426853087 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0018040 con 0x55b757962760 2024-09-16T01:56:00.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.826+0000 7fab9cff9640 1 -- 172.21.15.139:0/426853087 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba0029070 con 0x55b757962760 2024-09-16T01:56:00.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.826+0000 7fab9cff9640 1 -- 172.21.15.139:0/426853087 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0014690 con 0x55b757962760 2024-09-16T01:56:00.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.827+0000 7fabaf7bb740 1 -- 172.21.15.139:0/426853087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757962760 msgr2=0x55b7579651c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:00.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.827+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/426853087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757962760 0x55b7579651c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7faba0013930 tx=0x7faba0009430 comp rx=0 tx=0).stop 2024-09-16T01:56:00.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.827+0000 7fabaf7bb740 1 -- 172.21.15.139:0/426853087 shutdown_connections 2024-09-16T01:56:00.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.827+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/426853087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b75796d940 0x55b757985b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.827+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/426853087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b757963080 0x55b7579855f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.827+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/426853087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757962760 0x55b7579651c0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.827+0000 7fabaf7bb740 1 -- 172.21.15.139:0/426853087 >> 172.21.15.139:0/426853087 conn(0x55b75795ecf0 msgr2=0x55b757969020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:00.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.827+0000 7fabaf7bb740 1 -- 172.21.15.139:0/426853087 shutdown_connections 2024-09-16T01:56:00.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.827+0000 7fabaf7bb740 1 -- 172.21.15.139:0/426853087 wait complete. 2024-09-16T01:56:00.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.827+0000 7fabaf7bb740 1 -- start start 2024-09-16T01:56:00.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.827+0000 7fabaf7bb740 1 Processor -- start 2024-09-16T01:56:00.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.827+0000 7fabaf7bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757962760 0x55b757971790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7fabaf7bb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757963080 0x55b757971cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7fabaf7bb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b75796d940 0x55b757972210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7fabaf7bb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faba81bf780 con 0x55b757963080 2024-09-16T01:56:00.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7fabaf7bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faba81beff0 con 0x55b757962760 2024-09-16T01:56:00.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7fabaf7bb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7faba81bc0b0 con 0x55b75796d940 2024-09-16T01:56:00.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7fabaf7bb740 10 client.0 _ll_register_callbacks cb 0x7fab98013a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:00.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757962760 0x55b757971790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757962760 0x55b757971790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:41778/0 (socket says 172.21.15.139:41778) 2024-09-16T01:56:00.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7faba6ffd640 1 -- 172.21.15.139:0/1094594671 learned_addr learned my addr 172.21.15.139:0/1094594671 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:00.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7faba67fc640 1 --2- 172.21.15.139:0/1094594671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757963080 0x55b757971cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7faba77fe640 1 --2- 172.21.15.139:0/1094594671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b75796d940 0x55b757972210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7faba67fc640 1 -- 172.21.15.139:0/1094594671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b75796d940 msgr2=0x55b757972210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:00.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7faba67fc640 1 --2- 172.21.15.139:0/1094594671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b75796d940 0x55b757972210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7faba67fc640 1 -- 172.21.15.139:0/1094594671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757962760 msgr2=0x55b757971790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:00.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7faba67fc640 1 --2- 172.21.15.139:0/1094594671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757962760 0x55b757971790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:00.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7faba67fc640 1 -- 172.21.15.139:0/1094594671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7579e7720 con 0x55b757963080 2024-09-16T01:56:00.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7faba6ffd640 1 --2- 172.21.15.139:0/1094594671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757962760 0x55b757971790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:56:00.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7faba67fc640 1 --2- 172.21.15.139:0/1094594671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757963080 0x55b757971cd0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fab900edc70 tx=0x7fab900bc400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbee92d2681af64f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:00.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7fab8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:56:00.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7fab8e7fc640 1 -- 172.21.15.139:0/1094594671 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90049040 con 0x55b757963080 2024-09-16T01:56:00.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7fab8e7fc640 1 -- 172.21.15.139:0/1094594671 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9678+0+0 (secure 0 0 0) 0x7fab9003b610 con 0x55b757963080 2024-09-16T01:56:00.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.828+0000 7fabaf7bb740 1 -- 172.21.15.139:0/1094594671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba0029070 con 0x55b757963080 2024-09-16T01:56:00.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.829+0000 7fab8e7fc640 1 -- 172.21.15.139:0/1094594671 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fab900c0bb0 con 0x55b757963080 2024-09-16T01:56:00.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.829+0000 7fabaf7bb740 10 client.17091 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:00.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.829+0000 7fabaf7bb740 1 -- 172.21.15.139:0/1094594671 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55b7579e8a20 con 0x55b757963080 2024-09-16T01:56:00.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.829+0000 7fab8e7fc640 1 -- 172.21.15.139:0/1094594671 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba0029070 con 0x55b757963080 2024-09-16T01:56:00.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.829+0000 7fab8e7fc640 1 -- 172.21.15.139:0/1094594671 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7fab9003b9a0 con 0x55b757963080 2024-09-16T01:56:00.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.829+0000 7fabaf7bb740 10 client.17091 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:00.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.829+0000 7fabaf7bb740 10 client.17091 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:56:00.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.829+0000 7fabaf7bb740 20 client.17091 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:00.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.829+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/1094594671 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b7579e9370 0x55b7579eb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:00.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.829+0000 7fabaf7bb740 4 client.17091 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}] 2024-09-16T01:56:00.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.829+0000 7fabaf7bb740 1 -- 172.21.15.139:0/1094594671 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}) -- 0x55b7578cfd20 con 0x55b7579e9370 2024-09-16T01:56:00.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.829+0000 7faba6ffd640 1 --2- 172.21.15.139:0/1094594671 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b7579e9370 0x55b7579eb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:00.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.830+0000 7fab8e7fc640 10 client.17091 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:00.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:55:59.830+0000 7faba6ffd640 1 --2- 172.21.15.139:0/1094594671 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b7579e9370 0x55b7579eb790 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc23a38e9f03491b server_cookie=66b3b1eaa3fe1ee0 in_seq=0 out_seq=0 2024-09-16T01:56:00.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:00 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:00.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:00 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:00.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:00 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:56:00.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:00 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:00.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:00 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:00.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:00 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:56:00.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:00 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:56:00.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:00 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:56:00.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:00 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:56:02.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:01 smithi139 ceph-mon[31175]: pgmap v493: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2024-09-16T01:56:02.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:01 smithi176 ceph-mon[35324]: pgmap v493: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2024-09-16T01:56:02.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:01 smithi164 ceph-mon[35121]: pgmap v493: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2024-09-16T01:56:02.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.494+0000 7fab8e7fc640 1 -- 172.21.15.139:0/1094594671 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b7578cfd20 con 0x55b7579e9370 2024-09-16T01:56:02.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.495+0000 7fab8e7fc640 10 client.17091 handle_command_reply: tid=0 2024-09-16T01:56:02.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.495+0000 7fabaf7bb740 1 client.17091 shutdown 2024-09-16T01:56:02.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.495+0000 7fabaf7bb740 1 -- 172.21.15.139:0/1094594671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757963080 msgr2=0x55b757971cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:02.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.495+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/1094594671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757963080 0x55b757971cd0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fab900edc70 tx=0x7fab900bc400 comp rx=0 tx=0).stop 2024-09-16T01:56:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.495+0000 7fabaf7bb740 1 -- 172.21.15.139:0/1094594671 shutdown_connections 2024-09-16T01:56:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.495+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/1094594671 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55b7579e9370 0x55b7579eb790 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.495+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/1094594671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b75796d940 0x55b757972210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.495+0000 7fab8e7fc640 0 client.17091 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.496+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/1094594671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b757963080 0x55b757971cd0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.496+0000 7fabaf7bb740 1 --2- 172.21.15.139:0/1094594671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b757962760 0x55b757971790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.496+0000 7fabaf7bb740 1 -- 172.21.15.139:0/1094594671 >> 172.21.15.139:0/1094594671 conn(0x55b75795ecf0 msgr2=0x55b75795d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.496+0000 7fabaf7bb740 1 -- 172.21.15.139:0/1094594671 shutdown_connections 2024-09-16T01:56:02.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.496+0000 7fabaf7bb740 1 -- 172.21.15.139:0/1094594671 wait complete. 2024-09-16T01:56:02.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.496+0000 7fabaf7bb740 20 client.17091 trim_cache size 0 max 16384 2024-09-16T01:56:02.498 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:56:02.498 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 71, 2024-09-16T01:56:02.498 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:56:02.498 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 54, 2024-09-16T01:56:02.498 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:56:02.498 INFO:teuthology.orchestra.run.smithi139.stdout: "e4a5e579": { 2024-09-16T01:56:02.498 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 54, 2024-09-16T01:56:02.498 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:56:02.499 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:56:02.499 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:56:02.499 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:56:02.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:56:02.499 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:56:02.499 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 137.7, 2024-09-16T01:56:02.499 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:56:02.499 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:56:02.499 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:56:02.499 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:56:02.499 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:56:02.499 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:56:02.499 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:02.499 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:02.500 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:02.500 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:02.500 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:02.500 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:56:02.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.497+0000 7fabad3e2640 1 -- 172.21.15.139:0/2569236534 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fab940a3540 msgr2=0x7fab940a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:02.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.497+0000 7fabad3e2640 1 --2- 172.21.15.139:0/2569236534 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fab940a3540 0x7fab940a5a00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fab98009690 tx=0x7fab98006c70 comp rx=0 tx=0).stop 2024-09-16T01:56:02.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.497+0000 7fabad3e2640 1 -- 172.21.15.139:0/2569236534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8108e00 msgr2=0x7faba807b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:02.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.497+0000 7fabad3e2640 1 --2- 172.21.15.139:0/2569236534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8108e00 0x7faba807b990 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fab9002aa30 tx=0x7fab90003cd0 comp rx=0 tx=0).stop 2024-09-16T01:56:02.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.501+0000 7fabad3e2640 1 -- 172.21.15.139:0/2569236534 shutdown_connections 2024-09-16T01:56:02.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.501+0000 7fabad3e2640 1 --2- 172.21.15.139:0/2569236534 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fab940a3540 0x7fab940a5a00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:02.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.501+0000 7fabad3e2640 1 --2- 172.21.15.139:0/2569236534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faba810d530 0x7faba807bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:02.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.501+0000 7fabad3e2640 1 --2- 172.21.15.139:0/2569236534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba8108e00 0x7faba807b990 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:02.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.501+0000 7fabad3e2640 1 --2- 172.21.15.139:0/2569236534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faba8108450 0x7faba80828b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:02.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.501+0000 7fabad3e2640 1 -- 172.21.15.139:0/2569236534 >> 172.21.15.139:0/2569236534 conn(0x7faba8077ba0 msgr2=0x7faba8078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:02.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.502+0000 7fabad3e2640 1 -- 172.21.15.139:0/2569236534 shutdown_connections 2024-09-16T01:56:02.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:02.502+0000 7fabad3e2640 1 -- 172.21.15.139:0/2569236534 wait complete. 2024-09-16T01:56:03.275 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e4a5e579', quiesce took 8.9 seconds. Will release after: 8.799999999999999 2024-09-16T01:56:04.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:03 smithi139 ceph-mon[31175]: pgmap v494: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 201 op/s 2024-09-16T01:56:04.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:03 smithi176 ceph-mon[35324]: pgmap v494: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 201 op/s 2024-09-16T01:56:04.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:03 smithi164 ceph-mon[35121]: pgmap v494: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 201 op/s 2024-09-16T01:56:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.142+0000 7fb09f25e640 1 -- 172.21.15.139:0/2450274565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb09810d4b0 msgr2=0x7fb098115000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.142+0000 7fb09f25e640 1 --2- 172.21.15.139:0/2450274565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb09810d4b0 0x7fb098115000 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb08801a910 tx=0x7fb08803b470 comp rx=0 tx=0).stop 2024-09-16T01:56:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.142+0000 7fb09f25e640 1 -- 172.21.15.139:0/2450274565 shutdown_connections 2024-09-16T01:56:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.142+0000 7fb09f25e640 1 --2- 172.21.15.139:0/2450274565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb09810d4b0 0x7fb098115000 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.142+0000 7fb09f25e640 1 --2- 172.21.15.139:0/2450274565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb098108d10 0x7fb09810cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.142+0000 7fb09f25e640 1 --2- 172.21.15.139:0/2450274565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb098108360 0x7fb098108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.142+0000 7fb09f25e640 1 -- 172.21.15.139:0/2450274565 >> 172.21.15.139:0/2450274565 conn(0x7fb098077ad0 msgr2=0x7fb098079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.143+0000 7fb09f25e640 1 -- 172.21.15.139:0/2450274565 shutdown_connections 2024-09-16T01:56:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.147+0000 7fb09f25e640 1 -- 172.21.15.139:0/2450274565 wait complete. 2024-09-16T01:56:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.147+0000 7fb09f25e640 1 Processor -- start 2024-09-16T01:56:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.148+0000 7fb09f25e640 1 -- start start 2024-09-16T01:56:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.148+0000 7fb09f25e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb098108360 0x7fb098114b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.148+0000 7fb09f25e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb09810d4b0 0x7fb098110bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.148+0000 7fb09f25e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb098111100 0x7fb0981115b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.148+0000 7fb09f25e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb09807d3d0 con 0x7fb09810d4b0 2024-09-16T01:56:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.148+0000 7fb09f25e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb09807d250 con 0x7fb098108360 2024-09-16T01:56:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.148+0000 7fb09f25e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb09807d550 con 0x7fb098111100 2024-09-16T01:56:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.149+0000 7fb09cffa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb098108360 0x7fb098114b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.149+0000 7fb09cffa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb098108360 0x7fb098114b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56488/0 (socket says 172.21.15.139:56488) 2024-09-16T01:56:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.149+0000 7fb09cffa640 1 -- 172.21.15.139:0/1234041552 learned_addr learned my addr 172.21.15.139:0/1234041552 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.150+0000 7fb09d7fb640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb098111100 0x7fb0981115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.153+0000 7fb09cffa640 1 -- 172.21.15.139:0/1234041552 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb098111100 msgr2=0x7fb0981115b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.153+0000 7fb097fff640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb09810d4b0 0x7fb098110bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.153+0000 7fb09cffa640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb098111100 0x7fb0981115b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.153+0000 7fb09cffa640 1 -- 172.21.15.139:0/1234041552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb09810d4b0 msgr2=0x7fb098110bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.153+0000 7fb09cffa640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb09810d4b0 0x7fb098110bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.153+0000 7fb09cffa640 1 -- 172.21.15.139:0/1234041552 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb098073c60 con 0x7fb098108360 2024-09-16T01:56:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.153+0000 7fb097fff640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb09810d4b0 0x7fb098110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:56:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.158+0000 7fb09cffa640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb098108360 0x7fb098114b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb09000cc60 tx=0x7fb09000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.161+0000 7fb095ffb640 1 -- 172.21.15.139:0/1234041552 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090004b20 con 0x7fb098108360 2024-09-16T01:56:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.161+0000 7fb095ffb640 1 -- 172.21.15.139:0/1234041552 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb090004cc0 con 0x7fb098108360 2024-09-16T01:56:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.161+0000 7fb095ffb640 1 -- 172.21.15.139:0/1234041552 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09000f780 con 0x7fb098108360 2024-09-16T01:56:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.162+0000 7fb09f25e640 1 -- 172.21.15.139:0/1234041552 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb098073f50 con 0x7fb098108360 2024-09-16T01:56:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.163+0000 7fb09f25e640 1 -- 172.21.15.139:0/1234041552 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0981aba80 con 0x7fb098108360 2024-09-16T01:56:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.167+0000 7fb0a1ffb740 1 Processor -- start 2024-09-16T01:56:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.167+0000 7fb0a1ffb740 1 -- start start 2024-09-16T01:56:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.167+0000 7fb0a1ffb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068363af0 0x560068363ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.167+0000 7fb0a1ffb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560068364410 0x56006836e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.167+0000 7fb0a1ffb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56006836ecb0 0x5600683710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.167+0000 7fb0a1ffb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5600682c82f0 con 0x560068364410 2024-09-16T01:56:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.167+0000 7fb0a1ffb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560068270270 con 0x560068363af0 2024-09-16T01:56:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.167+0000 7fb0a1ffb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5600682c42b0 con 0x56006836ecb0 2024-09-16T01:56:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.168+0000 7fb09d7fb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068363af0 0x560068363ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.169+0000 7fb097fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56006836ecb0 0x5600683710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.169+0000 7fb097fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56006836ecb0 0x5600683710a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:52528/0 (socket says 172.21.15.139:52528) 2024-09-16T01:56:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.169+0000 7fb097fff640 1 -- 172.21.15.139:0/1749532800 learned_addr learned my addr 172.21.15.139:0/1749532800 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.171+0000 7fb097fff640 1 -- 172.21.15.139:0/1749532800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068363af0 msgr2=0x560068363ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.171+0000 7fb097fff640 1 --2- 172.21.15.139:0/1749532800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068363af0 0x560068363ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.171+0000 7fb097fff640 1 -- 172.21.15.139:0/1749532800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560068364410 msgr2=0x56006836e770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:56:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.171+0000 7fb097fff640 1 --2- 172.21.15.139:0/1749532800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560068364410 0x56006836e770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.171+0000 7fb097fff640 1 -- 172.21.15.139:0/1749532800 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56006826f340 con 0x56006836ecb0 2024-09-16T01:56:06.174 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:05 smithi139 ceph-mon[31175]: pgmap v495: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-09-16T01:56:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.173+0000 7fb097fff640 1 --2- 172.21.15.139:0/1749532800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56006836ecb0 0x5600683710a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009b30 tx=0x7fb08c00e2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b749091d5ac98439 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.173+0000 7fb095ffb640 1 -- 172.21.15.139:0/1234041552 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb09000f920 con 0x7fb098108360 2024-09-16T01:56:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.174+0000 7fb095ffb640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0800a3690 0x7fb0800a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.174+0000 7fb095ffb640 1 -- 172.21.15.139:0/1234041552 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9678+0+0 (secure 0 0 0) 0x7fb0900d0400 con 0x7fb098108360 2024-09-16T01:56:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.181+0000 7fb09d7fb640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0800a3690 0x7fb0800a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.181+0000 7fb07effd640 1 -- 172.21.15.139:0/1749532800 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c016070 con 0x56006836ecb0 2024-09-16T01:56:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.181+0000 7fb07effd640 1 -- 172.21.15.139:0/1749532800 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb08c00ed20 con 0x56006836ecb0 2024-09-16T01:56:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.182+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/1749532800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56006836ecb0 msgr2=0x5600683710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.182+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/1749532800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56006836ecb0 0x5600683710a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009b30 tx=0x7fb08c00e2f0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:05 smithi176 ceph-mon[35324]: pgmap v495: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-09-16T01:56:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.186+0000 7fb09d7fb640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0800a3690 0x7fb0800a5b50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb088004fd0 tx=0x7fb088005000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.189+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/1749532800 shutdown_connections 2024-09-16T01:56:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.189+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/1749532800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56006836ecb0 0x5600683710a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.189+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/1749532800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560068364410 0x56006836e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.189+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/1749532800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068363af0 0x560068363ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.189+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/1749532800 >> 172.21.15.139:0/1749532800 conn(0x560068360060 msgr2=0x560068360440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.191+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/1749532800 shutdown_connections 2024-09-16T01:56:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.200+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/1749532800 wait complete. 2024-09-16T01:56:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.200+0000 7fb0a1ffb740 1 -- start start 2024-09-16T01:56:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.200+0000 7fb0a1ffb740 1 Processor -- start 2024-09-16T01:56:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.203+0000 7fb0a1ffb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560068363af0 0x56006836bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.203+0000 7fb0a1ffb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068364410 0x56006836c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.203+0000 7fb0a1ffb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56006836ecb0 0x56006836c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.203+0000 7fb0a1ffb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560068371920 con 0x560068363af0 2024-09-16T01:56:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.203+0000 7fb0a1ffb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560068251290 con 0x56006836ecb0 2024-09-16T01:56:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.203+0000 7fb0a1ffb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560068371aa0 con 0x560068364410 2024-09-16T01:56:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.203+0000 7fb0a1ffb740 10 client.0 _ll_register_callbacks cb 0x560068282140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.204+0000 7fb09d7fb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56006836ecb0 0x56006836c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.205+0000 7fb09d7fb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56006836ecb0 0x56006836c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56498/0 (socket says 172.21.15.139:56498) 2024-09-16T01:56:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.205+0000 7fb09d7fb640 1 -- 172.21.15.139:0/3271837344 learned_addr learned my addr 172.21.15.139:0/3271837344 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.206+0000 7fb097fff640 1 --2- 172.21.15.139:0/3271837344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068364410 0x56006836c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.208+0000 7fb097fff640 1 -- 172.21.15.139:0/3271837344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56006836ecb0 msgr2=0x56006836c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.208+0000 7fb097fff640 1 --2- 172.21.15.139:0/3271837344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56006836ecb0 0x56006836c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.208+0000 7fb097fff640 1 -- 172.21.15.139:0/3271837344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560068363af0 msgr2=0x56006836bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.208+0000 7fb097fff640 1 --2- 172.21.15.139:0/3271837344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560068363af0 0x56006836bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.208+0000 7fb097fff640 1 -- 172.21.15.139:0/3271837344 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560067fdb210 con 0x560068364410 2024-09-16T01:56:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.212+0000 7fb097fff640 1 --2- 172.21.15.139:0/3271837344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068364410 0x56006836c040 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00e7f0 tx=0x7fb08c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bc073dba0ec86b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.212+0000 7fb077fff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:56:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.214+0000 7fb077fff640 1 -- 172.21.15.139:0/3271837344 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c027020 con 0x560068364410 2024-09-16T01:56:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.214+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3271837344 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb08c00ed20 con 0x560068364410 2024-09-16T01:56:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.214+0000 7fb077fff640 1 -- 172.21.15.139:0/3271837344 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9678+0+0 (secure 0 0 0) 0x7fb08c004020 con 0x560068364410 2024-09-16T01:56:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.214+0000 7fb077fff640 1 -- 172.21.15.139:0/3271837344 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fb08c02f050 con 0x560068364410 2024-09-16T01:56:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.215+0000 7fb0a1ffb740 10 client.26491 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.215+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3271837344 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5600680a7620 con 0x560068364410 2024-09-16T01:56:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.222+0000 7fb077fff640 1 -- 172.21.15.139:0/3271837344 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb08c00ed20 con 0x560068364410 2024-09-16T01:56:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.222+0000 7fb077fff640 1 -- 172.21.15.139:0/3271837344 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7fb08c00f040 con 0x560068364410 2024-09-16T01:56:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.223+0000 7fb0a1ffb740 10 client.26491 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.223+0000 7fb0a1ffb740 10 client.26491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:56:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.223+0000 7fb0a1ffb740 20 client.26491 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.223+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/3271837344 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56006837ecc0 0x5600683810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.223+0000 7fb0a1ffb740 4 client.26491 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:56:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.223+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3271837344 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600682d5a20 con 0x56006837ecc0 2024-09-16T01:56:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.224+0000 7fb09cffa640 1 --2- 172.21.15.139:0/3271837344 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56006837ecc0 0x5600683810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.229+0000 7fb09cffa640 1 --2- 172.21.15.139:0/3271837344 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56006837ecc0 0x5600683810b0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=786de5640c283eae server_cookie=e0c577e98574deb1 in_seq=0 out_seq=0 2024-09-16T01:56:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.229+0000 7fb077fff640 10 client.26491 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.230+0000 7fb077fff640 1 -- 172.21.15.139:0/3271837344 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600682d5a20 con 0x56006837ecc0 2024-09-16T01:56:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.231+0000 7fb077fff640 10 client.26491 handle_command_reply: tid=0 2024-09-16T01:56:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.231+0000 7fb0a1ffb740 1 client.26491 shutdown 2024-09-16T01:56:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.235+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3271837344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068364410 msgr2=0x56006836c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.235+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/3271837344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068364410 0x56006836c040 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00e7f0 tx=0x7fb08c005270 comp rx=0 tx=0).stop 2024-09-16T01:56:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.238+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3271837344 shutdown_connections 2024-09-16T01:56:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.238+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/3271837344 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x56006837ecc0 0x5600683810b0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.238+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/3271837344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56006836ecb0 0x56006836c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.238+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/3271837344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068364410 0x56006836c040 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.238+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/3271837344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560068363af0 0x56006836bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.238+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3271837344 >> 172.21.15.139:0/3271837344 conn(0x560068360060 msgr2=0x560068365dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.238+0000 7fb077fff640 0 client.26491 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.241+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3271837344 shutdown_connections 2024-09-16T01:56:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.242+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3271837344 wait complete. 2024-09-16T01:56:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.242+0000 7fb0a1ffb740 20 client.26491 trim_cache size 0 max 16384 2024-09-16T01:56:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.272+0000 7fb0a1ffb740 1 Processor -- start 2024-09-16T01:56:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.275+0000 7fb0a1ffb740 1 -- start start 2024-09-16T01:56:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.275+0000 7fb0a1ffb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068363af0 0x5600683665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.275+0000 7fb0a1ffb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068364410 0x56006836c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.275+0000 7fb0a1ffb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56006836ecb0 0x5600683867a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.275+0000 7fb0a1ffb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981b58f0 con 0x56006836ecb0 2024-09-16T01:56:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.275+0000 7fb0a1ffb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981b7730 con 0x560068364410 2024-09-16T01:56:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.275+0000 7fb0a1ffb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981b6080 con 0x560068363af0 2024-09-16T01:56:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.275+0000 7fb09d7fb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56006836ecb0 0x5600683867a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.275+0000 7fb09d7fb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56006836ecb0 0x5600683867a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34704/0 (socket says 172.21.15.139:34704) 2024-09-16T01:56:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.275+0000 7fb09d7fb640 1 -- 172.21.15.139:0/1942156840 learned_addr learned my addr 172.21.15.139:0/1942156840 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.276+0000 7fb097fff640 1 --2- 172.21.15.139:0/1942156840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068364410 0x56006836c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.276+0000 7fb09d7fb640 1 -- 172.21.15.139:0/1942156840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068363af0 msgr2=0x5600683665b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:56:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.276+0000 7fb09d7fb640 1 --2- 172.21.15.139:0/1942156840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068363af0 0x5600683665b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.276+0000 7fb09d7fb640 1 -- 172.21.15.139:0/1942156840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068364410 msgr2=0x56006836c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.276+0000 7fb09d7fb640 1 --2- 172.21.15.139:0/1942156840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068364410 0x56006836c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.276+0000 7fb09d7fb640 1 -- 172.21.15.139:0/1942156840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56006835fe90 con 0x56006836ecb0 2024-09-16T01:56:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.276+0000 7fb09d7fb640 1 --2- 172.21.15.139:0/1942156840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56006836ecb0 0x5600683867a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb088005230 tx=0x7fb088050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151224155fb5c5fb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.277+0000 7fb07e7fc640 1 -- 172.21.15.139:0/1942156840 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088040510 con 0x56006836ecb0 2024-09-16T01:56:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.277+0000 7fb07e7fc640 1 -- 172.21.15.139:0/1942156840 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb088040b90 con 0x56006836ecb0 2024-09-16T01:56:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.277+0000 7fb07e7fc640 1 -- 172.21.15.139:0/1942156840 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08804ec00 con 0x56006836ecb0 2024-09-16T01:56:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.278+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/1942156840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56006836ecb0 msgr2=0x5600683867a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.278+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/1942156840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56006836ecb0 0x5600683867a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb088005230 tx=0x7fb088050000 comp rx=0 tx=0).stop 2024-09-16T01:56:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.280+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/1942156840 shutdown_connections 2024-09-16T01:56:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.280+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/1942156840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56006836ecb0 0x5600683867a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.280+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/1942156840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068364410 0x56006836c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.280+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/1942156840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068363af0 0x5600683665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.280+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/1942156840 >> 172.21.15.139:0/1942156840 conn(0x560068360060 msgr2=0x56006836db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.280+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/1942156840 shutdown_connections 2024-09-16T01:56:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.280+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/1942156840 wait complete. 2024-09-16T01:56:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.281+0000 7fb0a1ffb740 1 -- start start 2024-09-16T01:56:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.281+0000 7fb0a1ffb740 1 Processor -- start 2024-09-16T01:56:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.281+0000 7fb0a1ffb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068363af0 0x5600683773d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.281+0000 7fb0a1ffb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068364410 0x5600683726f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.281+0000 7fb0a1ffb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56006836ecb0 0x560068372c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.281+0000 7fb0a1ffb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981b6fa0 con 0x56006836ecb0 2024-09-16T01:56:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.281+0000 7fb0a1ffb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981b6810 con 0x560068363af0 2024-09-16T01:56:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.281+0000 7fb0a1ffb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981b38d0 con 0x560068364410 2024-09-16T01:56:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.281+0000 7fb0a1ffb740 10 client.0 _ll_register_callbacks cb 0x7fb08c0099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.282+0000 7fb09cffa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068363af0 0x5600683773d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.282+0000 7fb09cffa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068363af0 0x5600683773d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56524/0 (socket says 172.21.15.139:56524) 2024-09-16T01:56:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.282+0000 7fb09cffa640 1 -- 172.21.15.139:0/3381581580 learned_addr learned my addr 172.21.15.139:0/3381581580 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.282+0000 7fb097fff640 1 --2- 172.21.15.139:0/3381581580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068364410 0x5600683726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.284+0000 7fb09d7fb640 1 --2- 172.21.15.139:0/3381581580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56006836ecb0 0x560068372c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.287+0000 7fb09cffa640 1 -- 172.21.15.139:0/3381581580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068364410 msgr2=0x5600683726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.287+0000 7fb09cffa640 1 --2- 172.21.15.139:0/3381581580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068364410 0x5600683726f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.287+0000 7fb09cffa640 1 -- 172.21.15.139:0/3381581580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56006836ecb0 msgr2=0x560068372c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.287+0000 7fb09cffa640 1 --2- 172.21.15.139:0/3381581580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56006836ecb0 0x560068372c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.287+0000 7fb09cffa640 1 -- 172.21.15.139:0/3381581580 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600683732d0 con 0x560068363af0 2024-09-16T01:56:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.293+0000 7fb09cffa640 1 --2- 172.21.15.139:0/3381581580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068363af0 0x5600683773d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb0900cd6d0 tx=0x7fb09009bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bf9fe3e035e9cdd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.293+0000 7fb077fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:56:06.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:05 smithi164 ceph-mon[35121]: pgmap v495: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-09-16T01:56:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.294+0000 7fb077fff640 1 -- 172.21.15.139:0/3381581580 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0900a0630 con 0x560068363af0 2024-09-16T01:56:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.294+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3381581580 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb088040b90 con 0x560068363af0 2024-09-16T01:56:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.294+0000 7fb077fff640 1 -- 172.21.15.139:0/3381581580 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9678+0+0 (secure 0 0 0) 0x7fb0900b5180 con 0x560068363af0 2024-09-16T01:56:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.294+0000 7fb077fff640 1 -- 172.21.15.139:0/3381581580 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fb0900a4ef0 con 0x560068363af0 2024-09-16T01:56:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.295+0000 7fb0a1ffb740 10 client.26540 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.295+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3381581580 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5600683e9670 con 0x560068363af0 2024-09-16T01:56:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.298+0000 7fb077fff640 1 -- 172.21.15.139:0/3381581580 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb088040b90 con 0x560068363af0 2024-09-16T01:56:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.298+0000 7fb077fff640 1 -- 172.21.15.139:0/3381581580 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7fb0900cd8f0 con 0x560068363af0 2024-09-16T01:56:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.298+0000 7fb0a1ffb740 10 client.26540 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.298+0000 7fb0a1ffb740 10 client.26540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:56:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.298+0000 7fb0a1ffb740 20 client.26540 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.298+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/3381581580 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5600683e9ef0 0x5600683ec310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.298+0000 7fb0a1ffb740 4 client.26540 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:56:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.298+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3381581580 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600682cfd70 con 0x5600683e9ef0 2024-09-16T01:56:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.298+0000 7fb097fff640 1 --2- 172.21.15.139:0/3381581580 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5600683e9ef0 0x5600683ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.302+0000 7fb097fff640 1 --2- 172.21.15.139:0/3381581580 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5600683e9ef0 0x5600683ec310 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=491854b99464bbb2 server_cookie=f9a6e15c0f930acf in_seq=0 out_seq=0 2024-09-16T01:56:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:06.302+0000 7fb077fff640 10 client.26540 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.130+0000 7fb077fff640 1 -- 172.21.15.139:0/3381581580 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600682cfd70 con 0x5600683e9ef0 2024-09-16T01:56:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.130+0000 7fb077fff640 10 client.26540 handle_command_reply: tid=0 2024-09-16T01:56:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.130+0000 7fb0a1ffb740 1 client.26540 shutdown 2024-09-16T01:56:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.130+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3381581580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068363af0 msgr2=0x5600683773d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.130+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/3381581580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068363af0 0x5600683773d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb0900cd6d0 tx=0x7fb09009bef0 comp rx=0 tx=0).stop 2024-09-16T01:56:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.130+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3381581580 shutdown_connections 2024-09-16T01:56:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.130+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/3381581580 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5600683e9ef0 0x5600683ec310 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.130+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/3381581580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56006836ecb0 0x560068372c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.130+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/3381581580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560068364410 0x5600683726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.130+0000 7fb0a1ffb740 1 --2- 172.21.15.139:0/3381581580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560068363af0 0x5600683773d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.130+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3381581580 >> 172.21.15.139:0/3381581580 conn(0x560068360060 msgr2=0x56006836db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.131+0000 7fb077fff640 0 client.26540 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.131+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3381581580 shutdown_connections 2024-09-16T01:56:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.131+0000 7fb0a1ffb740 1 -- 172.21.15.139:0/3381581580 wait complete. 2024-09-16T01:56:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.131+0000 7fb0a1ffb740 20 client.26540 trim_cache size 0 max 16384 2024-09-16T01:56:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:56:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:56:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:56:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:56:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.132+0000 7fb09f25e640 1 -- 172.21.15.139:0/1234041552 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0800a3690 msgr2=0x7fb0800a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.132+0000 7fb09f25e640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0800a3690 0x7fb0800a5b50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb088004fd0 tx=0x7fb088005000 comp rx=0 tx=0).stop 2024-09-16T01:56:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.132+0000 7fb09f25e640 1 -- 172.21.15.139:0/1234041552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb098108360 msgr2=0x7fb098114b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.132+0000 7fb09f25e640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb098108360 0x7fb098114b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb09000cc60 tx=0x7fb09000a620 comp rx=0 tx=0).stop 2024-09-16T01:56:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.132+0000 7fb09f25e640 1 -- 172.21.15.139:0/1234041552 shutdown_connections 2024-09-16T01:56:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.132+0000 7fb09f25e640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fb0800a3690 0x7fb0800a5b50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.132+0000 7fb09f25e640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb098111100 0x7fb0981115b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.132+0000 7fb09f25e640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb09810d4b0 0x7fb098110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.132+0000 7fb09f25e640 1 --2- 172.21.15.139:0/1234041552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb098108360 0x7fb098114b30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.132+0000 7fb09f25e640 1 -- 172.21.15.139:0/1234041552 >> 172.21.15.139:0/1234041552 conn(0x7fb098077ad0 msgr2=0x7fb098078590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.132+0000 7fb09f25e640 1 -- 172.21.15.139:0/1234041552 shutdown_connections 2024-09-16T01:56:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:07.132+0000 7fb09f25e640 1 -- 172.21.15.139:0/1234041552 wait complete. 2024-09-16T01:56:08.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:07 smithi139 ceph-mon[31175]: pgmap v496: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 222 op/s 2024-09-16T01:56:08.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:07 smithi176 ceph-mon[35324]: pgmap v496: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 222 op/s 2024-09-16T01:56:08.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:07 smithi164 ceph-mon[35121]: pgmap v496: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 222 op/s 2024-09-16T01:56:10.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:09 smithi139 ceph-mon[31175]: pgmap v497: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 223 op/s 2024-09-16T01:56:10.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:09 smithi139 ceph-mon[31175]: osdmap e109: 12 total, 12 up, 12 in 2024-09-16T01:56:10.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:09 smithi176 ceph-mon[35324]: pgmap v497: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 223 op/s 2024-09-16T01:56:10.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:09 smithi176 ceph-mon[35324]: osdmap e109: 12 total, 12 up, 12 in 2024-09-16T01:56:10.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:09 smithi164 ceph-mon[35121]: pgmap v497: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 223 op/s 2024-09-16T01:56:10.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:09 smithi164 ceph-mon[35121]: osdmap e109: 12 total, 12 up, 12 in 2024-09-16T01:56:12.076 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:56:12.126 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:11 smithi139 ceph-mon[31175]: pgmap v499: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 497 KiB/s wr, 225 op/s 2024-09-16T01:56:12.126 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:11 smithi139 ceph-mon[31175]: osdmap e110: 12 total, 12 up, 12 in 2024-09-16T01:56:12.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:11 smithi176 ceph-mon[35324]: pgmap v499: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 497 KiB/s wr, 225 op/s 2024-09-16T01:56:12.188 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:11 smithi176 ceph-mon[35324]: osdmap e110: 12 total, 12 up, 12 in 2024-09-16T01:56:12.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:11 smithi164 ceph-mon[35121]: pgmap v499: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 497 KiB/s wr, 225 op/s 2024-09-16T01:56:12.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:11 smithi164 ceph-mon[35121]: osdmap e110: 12 total, 12 up, 12 in 2024-09-16T01:56:12.677 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:56:14.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:13 smithi139 ceph-mon[31175]: pgmap v501: 129 pgs: 20 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 49 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 75 KiB/s wr, 237 op/s 2024-09-16T01:56:14.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:13 smithi176 ceph-mon[35324]: pgmap v501: 129 pgs: 20 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 49 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 75 KiB/s wr, 237 op/s 2024-09-16T01:56:14.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:13 smithi164 ceph-mon[35121]: pgmap v501: 129 pgs: 20 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 49 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 75 KiB/s wr, 237 op/s 2024-09-16T01:56:14.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.701+0000 7fe3ab08d640 1 -- 172.21.15.139:0/2924562679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3a4109e00 msgr2=0x7fe3a410e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:14.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.701+0000 7fe3ab08d640 1 --2- 172.21.15.139:0/2924562679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3a4109e00 0x7fe3a410e2b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe394009ca0 tx=0x7fe39402b6b0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.701+0000 7fe3ab08d640 1 -- 172.21.15.139:0/2924562679 shutdown_connections 2024-09-16T01:56:14.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.701+0000 7fe3ab08d640 1 --2- 172.21.15.139:0/2924562679 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3a410ea10 0x7fe3a4115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.701+0000 7fe3ab08d640 1 --2- 172.21.15.139:0/2924562679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3a4109e00 0x7fe3a410e2b0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.701+0000 7fe3ab08d640 1 --2- 172.21.15.139:0/2924562679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3a4109450 0x7fe3a4109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.701+0000 7fe3ab08d640 1 -- 172.21.15.139:0/2924562679 >> 172.21.15.139:0/2924562679 conn(0x7fe3a4073660 msgr2=0x7fe3a4073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:14.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.705+0000 7fe3ab08d640 1 -- 172.21.15.139:0/2924562679 shutdown_connections 2024-09-16T01:56:14.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.705+0000 7fe3ab08d640 1 -- 172.21.15.139:0/2924562679 wait complete. 2024-09-16T01:56:14.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.705+0000 7fe3ab08d640 1 Processor -- start 2024-09-16T01:56:14.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.706+0000 7fe3ab08d640 1 -- start start 2024-09-16T01:56:14.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.706+0000 7fe3ab08d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3a4109450 0x7fe3a41a7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:14.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.706+0000 7fe3ab08d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3a4109e00 0x7fe3a41a7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:14.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.706+0000 7fe3ab08d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3a410ea10 0x7fe3a41ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:14.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.706+0000 7fe3ab08d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4118330 con 0x7fe3a4109450 2024-09-16T01:56:14.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.706+0000 7fe3ab08d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41181b0 con 0x7fe3a410ea10 2024-09-16T01:56:14.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.706+0000 7fe3ab08d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a41184b0 con 0x7fe3a4109e00 2024-09-16T01:56:14.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.712+0000 7fe3aa88c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3a410ea10 0x7fe3a41ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:14.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.712+0000 7fe3aa88c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3a410ea10 0x7fe3a41ab8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56916/0 (socket says 172.21.15.139:56916) 2024-09-16T01:56:14.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.712+0000 7fe3aa88c640 1 -- 172.21.15.139:0/168681777 learned_addr learned my addr 172.21.15.139:0/168681777 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:14.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.715+0000 7fe3aa08b640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3a4109450 0x7fe3a41a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:14.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.717+0000 7fe3aa88c640 1 -- 172.21.15.139:0/168681777 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3a4109e00 msgr2=0x7fe3a41a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:14.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.715+0000 7fe3a988a640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3a4109e00 0x7fe3a41a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:14.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.719+0000 7fe3aa88c640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3a4109e00 0x7fe3a41a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.719+0000 7fe3aa88c640 1 -- 172.21.15.139:0/168681777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3a4109450 msgr2=0x7fe3a41a7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:14.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.719+0000 7fe3aa88c640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3a4109450 0x7fe3a41a7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:14.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.719+0000 7fe3aa88c640 1 -- 172.21.15.139:0/168681777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a41abfd0 con 0x7fe3a410ea10 2024-09-16T01:56:14.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.720+0000 7fe3a988a640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3a4109e00 0x7fe3a41a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:56:14.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.724+0000 7fe3aa88c640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3a410ea10 0x7fe3a41ab8f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe39800e970 tx=0x7fe39800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:14.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.728+0000 7fe3a37fe640 1 -- 172.21.15.139:0/168681777 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39800ccc0 con 0x7fe3a410ea10 2024-09-16T01:56:14.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.728+0000 7fe3a37fe640 1 -- 172.21.15.139:0/168681777 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe39800ce60 con 0x7fe3a410ea10 2024-09-16T01:56:14.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.728+0000 7fe3ab08d640 1 -- 172.21.15.139:0/168681777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a41ac2c0 con 0x7fe3a410ea10 2024-09-16T01:56:14.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.728+0000 7fe3ab08d640 1 -- 172.21.15.139:0/168681777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a41b3ba0 con 0x7fe3a410ea10 2024-09-16T01:56:14.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.728+0000 7fe3a37fe640 1 -- 172.21.15.139:0/168681777 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398010430 con 0x7fe3a410ea10 2024-09-16T01:56:14.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.733+0000 7fe3ab08d640 1 -- 172.21.15.139:0/168681777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe390005290 con 0x7fe3a410ea10 2024-09-16T01:56:14.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.735+0000 7fe3a37fe640 1 -- 172.21.15.139:0/168681777 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe3980046c0 con 0x7fe3a410ea10 2024-09-16T01:56:14.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.736+0000 7fe3a37fe640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe38c0a35c0 0x7fe38c0a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:14.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.737+0000 7fe3a37fe640 1 -- 172.21.15.139:0/168681777 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7fe3980d0340 con 0x7fe3a410ea10 2024-09-16T01:56:14.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.743+0000 7fe3a37fe640 1 -- 172.21.15.139:0/168681777 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe39809c810 con 0x7fe3a410ea10 2024-09-16T01:56:14.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.743+0000 7fe3aa08b640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe38c0a35c0 0x7fe38c0a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:14.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:14.745+0000 7fe3aa08b640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe38c0a35c0 0x7fe38c0a5a80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00bbe0 tx=0x7fe39c00b390 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:15.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:15.145+0000 7fe3ab08d640 1 -- 172.21.15.139:0/168681777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe390005850 con 0x7fe3a410ea10 2024-09-16T01:56:15.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:15.147+0000 7fe3a37fe640 1 -- 172.21.15.139:0/168681777 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+12031 (secure 0 0 0) 0x7fe39809c110 con 0x7fe3a410ea10 2024-09-16T01:56:15.152 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:56:15.152 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":71,"btime":"2024-09-16T01:54:19:332763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:54:19.051272+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:56:15.152 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 71 2024-09-16T01:56:15.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:15.154+0000 7fe3ab08d640 1 -- 172.21.15.139:0/168681777 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe38c0a35c0 msgr2=0x7fe38c0a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:15.154+0000 7fe3ab08d640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe38c0a35c0 0x7fe38c0a5a80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00bbe0 tx=0x7fe39c00b390 comp rx=0 tx=0).stop 2024-09-16T01:56:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:15.154+0000 7fe3ab08d640 1 -- 172.21.15.139:0/168681777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3a410ea10 msgr2=0x7fe3a41ab8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:15.154+0000 7fe3ab08d640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3a410ea10 0x7fe3a41ab8f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe39800e970 tx=0x7fe39800ee30 comp rx=0 tx=0).stop 2024-09-16T01:56:15.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:15.158+0000 7fe3ab08d640 1 -- 172.21.15.139:0/168681777 shutdown_connections 2024-09-16T01:56:15.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:15.158+0000 7fe3ab08d640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe38c0a35c0 0x7fe38c0a5a80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:15.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:15.158+0000 7fe3ab08d640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3a410ea10 0x7fe3a41ab8f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:15.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:15.158+0000 7fe3ab08d640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3a4109e00 0x7fe3a41a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:15.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:15.158+0000 7fe3ab08d640 1 --2- 172.21.15.139:0/168681777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3a4109450 0x7fe3a41a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:15.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:15.158+0000 7fe3ab08d640 1 -- 172.21.15.139:0/168681777 >> 172.21.15.139:0/168681777 conn(0x7fe3a4073660 msgr2=0x7fe3a406cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:15.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:15.162+0000 7fe3ab08d640 1 -- 172.21.15.139:0/168681777 shutdown_connections 2024-09-16T01:56:15.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:15.165+0000 7fe3ab08d640 1 -- 172.21.15.139:0/168681777 wait complete. 2024-09-16T01:56:15.909 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id e4a5e579 --release --await' 2024-09-16T01:56:15.910 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id e4a5e579 --release --await 2024-09-16T01:56:16.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:15 smithi139 ceph-mon[31175]: pgmap v502: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 37 KiB/s wr, 196 op/s 2024-09-16T01:56:16.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:15 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/168681777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:56:16.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:15 smithi176 ceph-mon[35324]: pgmap v502: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 37 KiB/s wr, 196 op/s 2024-09-16T01:56:16.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:15 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/168681777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:56:16.296 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:15 smithi164 ceph-mon[35121]: pgmap v502: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 37 KiB/s wr, 196 op/s 2024-09-16T01:56:16.297 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:15 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/168681777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:56:16.386 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:56:17.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.065+0000 7f2267558640 1 -- 172.21.15.139:0/4107325326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f226011b140 msgr2=0x7f226011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:17.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.065+0000 7f2267558640 1 --2- 172.21.15.139:0/4107325326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f226011b140 0x7f226011d530 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2258009ca0 tx=0x7f225802a8f0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.070+0000 7f2267558640 1 -- 172.21.15.139:0/4107325326 shutdown_connections 2024-09-16T01:56:17.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.070+0000 7f2267558640 1 --2- 172.21.15.139:0/4107325326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f226011b140 0x7f226011d530 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.070+0000 7f2267558640 1 --2- 172.21.15.139:0/4107325326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f226006ea30 0x7f226006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.070+0000 7f2267558640 1 --2- 172.21.15.139:0/4107325326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2260110190 0x7f226006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.070+0000 7f2267558640 1 -- 172.21.15.139:0/4107325326 >> 172.21.15.139:0/4107325326 conn(0x7f2260073180 msgr2=0x7f2260073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:17.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.078+0000 7f2267558640 1 -- 172.21.15.139:0/4107325326 shutdown_connections 2024-09-16T01:56:17.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.080+0000 7f2267558640 1 -- 172.21.15.139:0/4107325326 wait complete. 2024-09-16T01:56:17.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.081+0000 7f2267558640 1 Processor -- start 2024-09-16T01:56:17.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.081+0000 7f2267558640 1 -- start start 2024-09-16T01:56:17.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.081+0000 7f2267558640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f226006ea30 0x7f2260116d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.081+0000 7f2267558640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2260110190 0x7f2260117280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.081+0000 7f2267558640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f226011b610 0x7f226011bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.081+0000 7f2267558640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2260110f30 con 0x7f226011b610 2024-09-16T01:56:17.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.081+0000 7f2267558640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2260110db0 con 0x7f226006ea30 2024-09-16T01:56:17.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.081+0000 7f2267558640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f22601110b0 con 0x7f2260110190 2024-09-16T01:56:17.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.082+0000 7f2265d55640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2260110190 0x7f2260117280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.082+0000 7f2266556640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f226006ea30 0x7f2260116d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.082+0000 7f2266556640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f226006ea30 0x7f2260116d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56936/0 (socket says 172.21.15.139:56936) 2024-09-16T01:56:17.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.082+0000 7f2265d55640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2260110190 0x7f2260117280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:36970/0 (socket says 172.21.15.139:36970) 2024-09-16T01:56:17.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.082+0000 7f2265d55640 1 -- 172.21.15.139:0/525673984 learned_addr learned my addr 172.21.15.139:0/525673984 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:17.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.083+0000 7f2266556640 1 -- 172.21.15.139:0/525673984 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2260110190 msgr2=0x7f2260117280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:17.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.083+0000 7f2266556640 1 --2- 172.21.15.139:0/525673984 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2260110190 0x7f2260117280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.083+0000 7f2266556640 1 -- 172.21.15.139:0/525673984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f226011b610 msgr2=0x7f226011bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:17.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.082+0000 7f2266d57640 1 --2- 172.21.15.139:0/525673984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f226011b610 0x7f226011bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.083+0000 7f2266556640 1 --2- 172.21.15.139:0/525673984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f226011b610 0x7f226011bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.083+0000 7f2266556640 1 -- 172.21.15.139:0/525673984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226011c030 con 0x7f226006ea30 2024-09-16T01:56:17.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.083+0000 7f2266d57640 1 --2- 172.21.15.139:0/525673984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f226011b610 0x7f226011bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:56:17.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.087+0000 7f2266556640 1 --2- 172.21.15.139:0/525673984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f226006ea30 0x7f2260116d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f225c009fd0 tx=0x7f225c00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:17.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.087+0000 7f22577fe640 1 -- 172.21.15.139:0/525673984 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c015020 con 0x7f226006ea30 2024-09-16T01:56:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.087+0000 7f22577fe640 1 -- 172.21.15.139:0/525673984 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f225c011d10 con 0x7f226006ea30 2024-09-16T01:56:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.087+0000 7f22577fe640 1 -- 172.21.15.139:0/525673984 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c015020 con 0x7f226006ea30 2024-09-16T01:56:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.088+0000 7f2267558640 1 -- 172.21.15.139:0/525673984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226011c320 con 0x7f226006ea30 2024-09-16T01:56:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.088+0000 7f2267558640 1 -- 172.21.15.139:0/525673984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22601bc650 con 0x7f226006ea30 2024-09-16T01:56:17.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.091+0000 7f226ab93740 1 Processor -- start 2024-09-16T01:56:17.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.092+0000 7f226ab93740 1 -- start start 2024-09-16T01:56:17.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.092+0000 7f226ab93740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3210 0x561cb9bd35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.093+0000 7f226ab93740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bd3b30 0x561cb9bddeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.093+0000 7f226ab93740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bde3f0 0x561cb9be07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.093+0000 7f226ab93740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561cb9b38620 con 0x561cb9bde3f0 2024-09-16T01:56:17.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.093+0000 7f226ab93740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561cb9adfe50 con 0x561cb9bd3b30 2024-09-16T01:56:17.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.093+0000 7f226ab93740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561cb9b345e0 con 0x561cb9bd3210 2024-09-16T01:56:17.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.093+0000 7f2265d55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bde3f0 0x561cb9be07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.093+0000 7f2266d57640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3210 0x561cb9bd35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.093+0000 7f2266d57640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3210 0x561cb9bd35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:36978/0 (socket says 172.21.15.139:36978) 2024-09-16T01:56:17.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.093+0000 7f2266d57640 1 -- 172.21.15.139:0/1946817080 learned_addr learned my addr 172.21.15.139:0/1946817080 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:17.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.094+0000 7f2265d55640 1 -- 172.21.15.139:0/1946817080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3210 msgr2=0x561cb9bd35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:17.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.094+0000 7f2265d55640 1 --2- 172.21.15.139:0/1946817080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3210 0x561cb9bd35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.094+0000 7f2265d55640 1 -- 172.21.15.139:0/1946817080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bd3b30 msgr2=0x561cb9bddeb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:56:17.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.094+0000 7f2265d55640 1 --2- 172.21.15.139:0/1946817080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bd3b30 0x561cb9bddeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.094+0000 7f2265d55640 1 -- 172.21.15.139:0/1946817080 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb9ad0d40 con 0x561cb9bde3f0 2024-09-16T01:56:17.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.094+0000 7f2265d55640 1 --2- 172.21.15.139:0/1946817080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bde3f0 0x561cb9be07e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f22601183c0 tx=0x7f2250009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65dd2efda0fcfe82 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:17.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.095+0000 7f22577fe640 1 -- 172.21.15.139:0/525673984 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f225c018020 con 0x7f226006ea30 2024-09-16T01:56:17.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.096+0000 7f22577fe640 1 --2- 172.21.15.139:0/525673984 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f224c0a37f0 0x7f224c0a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.096+0000 7f22577fe640 1 -- 172.21.15.139:0/525673984 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7f225c011630 con 0x7f226006ea30 2024-09-16T01:56:17.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.099+0000 7f2254ff9640 1 -- 172.21.15.139:0/1946817080 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250017070 con 0x561cb9bde3f0 2024-09-16T01:56:17.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.099+0000 7f2254ff9640 1 -- 172.21.15.139:0/1946817080 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2250013470 con 0x561cb9bde3f0 2024-09-16T01:56:17.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.100+0000 7f226ab93740 1 -- 172.21.15.139:0/1946817080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bde3f0 msgr2=0x561cb9be07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:17.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.100+0000 7f226ab93740 1 --2- 172.21.15.139:0/1946817080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bde3f0 0x561cb9be07e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f22601183c0 tx=0x7f2250009ea0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.101+0000 7f2266d57640 1 --2- 172.21.15.139:0/525673984 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f224c0a37f0 0x7f224c0a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.101+0000 7f226ab93740 1 -- 172.21.15.139:0/1946817080 shutdown_connections 2024-09-16T01:56:17.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.101+0000 7f226ab93740 1 --2- 172.21.15.139:0/1946817080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bde3f0 0x561cb9be07e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.101+0000 7f226ab93740 1 --2- 172.21.15.139:0/1946817080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bd3b30 0x561cb9bddeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.101+0000 7f226ab93740 1 --2- 172.21.15.139:0/1946817080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3210 0x561cb9bd35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.101+0000 7f226ab93740 1 -- 172.21.15.139:0/1946817080 >> 172.21.15.139:0/1946817080 conn(0x561cb9bcf7a0 msgr2=0x561cb9bcfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:17.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.102+0000 7f2266d57640 1 --2- 172.21.15.139:0/525673984 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f224c0a37f0 0x7f224c0a5cb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2258009bf0 tx=0x7f2258005ef0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:17.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.106+0000 7f226ab93740 1 -- 172.21.15.139:0/1946817080 shutdown_connections 2024-09-16T01:56:17.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.109+0000 7f226ab93740 1 -- 172.21.15.139:0/1946817080 wait complete. 2024-09-16T01:56:17.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.109+0000 7f226ab93740 1 -- start start 2024-09-16T01:56:17.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.110+0000 7f226ab93740 1 Processor -- start 2024-09-16T01:56:17.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.114+0000 7f226ab93740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bd3210 0x561cb9bdb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.114+0000 7f226ab93740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3b30 0x561cb9bdb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.114+0000 7f226ab93740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bde3f0 0x561cb9bdbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.114+0000 7f226ab93740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561cb9be1010 con 0x561cb9bd3210 2024-09-16T01:56:17.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.114+0000 7f226ab93740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561cb9ac0f10 con 0x561cb9bde3f0 2024-09-16T01:56:17.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.114+0000 7f226ab93740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561cb9be1190 con 0x561cb9bd3b30 2024-09-16T01:56:17.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.115+0000 7f226ab93740 10 client.0 _ll_register_callbacks cb 0x561cb9ae4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:17.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.117+0000 7f2265d55640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3b30 0x561cb9bdb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.117+0000 7f2265d55640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3b30 0x561cb9bdb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:36988/0 (socket says 172.21.15.139:36988) 2024-09-16T01:56:17.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.117+0000 7f2265d55640 1 -- 172.21.15.139:0/1452848357 learned_addr learned my addr 172.21.15.139:0/1452848357 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:17.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.117+0000 7f2266d57640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bde3f0 0x561cb9bdbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.121+0000 7f2266556640 1 --2- 172.21.15.139:0/1452848357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bd3210 0x561cb9bdb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.123+0000 7f2266556640 1 -- 172.21.15.139:0/1452848357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3b30 msgr2=0x561cb9bdb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:17.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.123+0000 7f2266556640 1 --2- 172.21.15.139:0/1452848357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3b30 0x561cb9bdb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.123+0000 7f2266556640 1 -- 172.21.15.139:0/1452848357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bde3f0 msgr2=0x561cb9bdbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:17.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.123+0000 7f2266556640 1 --2- 172.21.15.139:0/1452848357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bde3f0 0x561cb9bdbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.123+0000 7f2266556640 1 -- 172.21.15.139:0/1452848357 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb9847580 con 0x561cb9bd3210 2024-09-16T01:56:17.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.123+0000 7f2265d55640 1 --2- 172.21.15.139:0/1452848357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3b30 0x561cb9bdb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:56:17.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.123+0000 7f2266d57640 1 --2- 172.21.15.139:0/1452848357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bde3f0 0x561cb9bdbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:56:17.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.123+0000 7f2266556640 1 --2- 172.21.15.139:0/1452848357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bd3210 0x561cb9bdb200 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f225c0cc780 tx=0x7f225c011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e4eb7c0eafe04c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:17.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.123+0000 7f2249ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:56:17.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.123+0000 7f2249ffb640 1 -- 172.21.15.139:0/1452848357 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c09ba10 con 0x561cb9bd3210 2024-09-16T01:56:17.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.124+0000 7f2249ffb640 1 -- 172.21.15.139:0/1452848357 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7f225c0978f0 con 0x561cb9bd3210 2024-09-16T01:56:17.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.126+0000 7f226ab93740 1 -- 172.21.15.139:0/1452848357 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2250013470 con 0x561cb9bd3210 2024-09-16T01:56:17.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.130+0000 7f2249ffb640 1 -- 172.21.15.139:0/1452848357 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f225c09b310 con 0x561cb9bd3210 2024-09-16T01:56:17.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.131+0000 7f226ab93740 10 client.17115 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:17.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.131+0000 7f226ab93740 1 -- 172.21.15.139:0/1452848357 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x561cb9911960 con 0x561cb9bd3210 2024-09-16T01:56:17.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.133+0000 7f2249ffb640 1 -- 172.21.15.139:0/1452848357 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2250013470 con 0x561cb9bd3210 2024-09-16T01:56:17.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.135+0000 7f2249ffb640 1 -- 172.21.15.139:0/1452848357 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f225c09b570 con 0x561cb9bd3210 2024-09-16T01:56:17.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.135+0000 7f226ab93740 10 client.17115 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:17.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.135+0000 7f226ab93740 10 client.17115 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:56:17.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.135+0000 7f226ab93740 20 client.17115 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:17.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.135+0000 7f226ab93740 1 --2- 172.21.15.139:0/1452848357 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561cb9bee4e0 0x561cb9bf0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.135+0000 7f226ab93740 4 client.17115 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:56:17.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.135+0000 7f226ab93740 1 -- 172.21.15.139:0/1452848357 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cb9b45b90 con 0x561cb9bee4e0 2024-09-16T01:56:17.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.136+0000 7f2265d55640 1 --2- 172.21.15.139:0/1452848357 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561cb9bee4e0 0x561cb9bf0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.139+0000 7f2265d55640 1 --2- 172.21.15.139:0/1452848357 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561cb9bee4e0 0x561cb9bf0920 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a08e60514486f65 server_cookie=e08645851c66dfe3 in_seq=0 out_seq=0 2024-09-16T01:56:17.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.142+0000 7f2249ffb640 10 client.17115 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:17.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.142+0000 7f2249ffb640 1 -- 172.21.15.139:0/1452848357 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cb9b45b90 con 0x561cb9bee4e0 2024-09-16T01:56:17.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.142+0000 7f2249ffb640 10 client.17115 handle_command_reply: tid=0 2024-09-16T01:56:17.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.142+0000 7f226ab93740 1 client.17115 shutdown 2024-09-16T01:56:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.148+0000 7f226ab93740 1 -- 172.21.15.139:0/1452848357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bd3210 msgr2=0x561cb9bdb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.148+0000 7f226ab93740 1 --2- 172.21.15.139:0/1452848357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bd3210 0x561cb9bdb200 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f225c0cc780 tx=0x7f225c011ae0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.150+0000 7f226ab93740 1 -- 172.21.15.139:0/1452848357 shutdown_connections 2024-09-16T01:56:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.150+0000 7f226ab93740 1 --2- 172.21.15.139:0/1452848357 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561cb9bee4e0 0x561cb9bf0920 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.150+0000 7f226ab93740 1 --2- 172.21.15.139:0/1452848357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bde3f0 0x561cb9bdbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.150+0000 7f226ab93740 1 --2- 172.21.15.139:0/1452848357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3b30 0x561cb9bdb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.150+0000 7f226ab93740 1 --2- 172.21.15.139:0/1452848357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bd3210 0x561cb9bdb200 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.150+0000 7f226ab93740 1 -- 172.21.15.139:0/1452848357 >> 172.21.15.139:0/1452848357 conn(0x561cb9bcf7a0 msgr2=0x561cb9bd5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.150+0000 7f2249ffb640 0 client.17115 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.152+0000 7f226ab93740 1 -- 172.21.15.139:0/1452848357 shutdown_connections 2024-09-16T01:56:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.152+0000 7f226ab93740 1 -- 172.21.15.139:0/1452848357 wait complete. 2024-09-16T01:56:17.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.152+0000 7f226ab93740 20 client.17115 trim_cache size 0 max 16384 2024-09-16T01:56:17.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.188+0000 7f226ab93740 1 Processor -- start 2024-09-16T01:56:17.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.188+0000 7f226ab93740 1 -- start start 2024-09-16T01:56:17.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.188+0000 7f226ab93740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bd3210 0x561cb9bce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.188+0000 7f226ab93740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bd3b30 0x561cb9bf6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.188+0000 7f226ab93740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bde3f0 0x561cb9bf6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.188+0000 7f226ab93740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22601c6a60 con 0x561cb9bd3210 2024-09-16T01:56:17.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.188+0000 7f226ab93740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22601c88a0 con 0x561cb9bd3b30 2024-09-16T01:56:17.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.188+0000 7f226ab93740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f22601c71f0 con 0x561cb9bde3f0 2024-09-16T01:56:17.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.194+0000 7f2266d57640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bde3f0 0x561cb9bf6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.196+0000 7f2266556640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bd3210 0x561cb9bce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.196+0000 7f2266556640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bd3210 0x561cb9bce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55022/0 (socket says 172.21.15.139:55022) 2024-09-16T01:56:17.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.196+0000 7f2266556640 1 -- 172.21.15.139:0/768536571 learned_addr learned my addr 172.21.15.139:0/768536571 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:17.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.196+0000 7f2265d55640 1 --2- 172.21.15.139:0/768536571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bd3b30 0x561cb9bf6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.199+0000 7f2266d57640 1 -- 172.21.15.139:0/768536571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bd3b30 msgr2=0x561cb9bf6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:17.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.199+0000 7f2266d57640 1 --2- 172.21.15.139:0/768536571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bd3b30 0x561cb9bf6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.199+0000 7f2266d57640 1 -- 172.21.15.139:0/768536571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bd3210 msgr2=0x561cb9bce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:17.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.199+0000 7f2266d57640 1 --2- 172.21.15.139:0/768536571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bd3210 0x561cb9bce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.199+0000 7f2266d57640 1 -- 172.21.15.139:0/768536571 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb9bcf5d0 con 0x561cb9bde3f0 2024-09-16T01:56:17.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.200+0000 7f2266d57640 1 --2- 172.21.15.139:0/768536571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bde3f0 0x561cb9bf6770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2258009ca0 tx=0x7f225803a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25bf822568f537b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:17.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.205+0000 7f224bfff640 1 -- 172.21.15.139:0/768536571 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258055070 con 0x561cb9bde3f0 2024-09-16T01:56:17.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.205+0000 7f224bfff640 1 -- 172.21.15.139:0/768536571 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2258036040 con 0x561cb9bde3f0 2024-09-16T01:56:17.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.205+0000 7f224bfff640 1 -- 172.21.15.139:0/768536571 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22580457e0 con 0x561cb9bde3f0 2024-09-16T01:56:17.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.206+0000 7f226ab93740 1 -- 172.21.15.139:0/768536571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bde3f0 msgr2=0x561cb9bf6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:17.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.206+0000 7f226ab93740 1 --2- 172.21.15.139:0/768536571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bde3f0 0x561cb9bf6770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2258009ca0 tx=0x7f225803a000 comp rx=0 tx=0).stop 2024-09-16T01:56:17.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.209+0000 7f226ab93740 1 -- 172.21.15.139:0/768536571 shutdown_connections 2024-09-16T01:56:17.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.209+0000 7f226ab93740 1 --2- 172.21.15.139:0/768536571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bde3f0 0x561cb9bf6770 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.209+0000 7f226ab93740 1 --2- 172.21.15.139:0/768536571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bd3b30 0x561cb9bf6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.209+0000 7f226ab93740 1 --2- 172.21.15.139:0/768536571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bd3210 0x561cb9bce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.209+0000 7f226ab93740 1 -- 172.21.15.139:0/768536571 >> 172.21.15.139:0/768536571 conn(0x561cb9bcf7a0 msgr2=0x561cb9bce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:17.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.212+0000 7f226ab93740 1 -- 172.21.15.139:0/768536571 shutdown_connections 2024-09-16T01:56:17.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.212+0000 7f226ab93740 1 -- 172.21.15.139:0/768536571 wait complete. 2024-09-16T01:56:17.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.212+0000 7f226ab93740 1 -- start start 2024-09-16T01:56:17.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.212+0000 7f226ab93740 1 Processor -- start 2024-09-16T01:56:17.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.217+0000 7f226ab93740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3210 0x561cb9be69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.217+0000 7f226ab93740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bd3b30 0x561cb9bd7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.217+0000 7f226ab93740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bde3f0 0x561cb9bd8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.217+0000 7f226ab93740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22601c8110 con 0x561cb9bde3f0 2024-09-16T01:56:17.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.217+0000 7f226ab93740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22601c7980 con 0x561cb9bd3b30 2024-09-16T01:56:17.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.217+0000 7f226ab93740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f22601c4a40 con 0x561cb9bd3210 2024-09-16T01:56:17.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.217+0000 7f226ab93740 10 client.0 _ll_register_callbacks cb 0x7f225c09b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:17.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.219+0000 7f2266556640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3210 0x561cb9be69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.219+0000 7f2266556640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3210 0x561cb9be69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:37008/0 (socket says 172.21.15.139:37008) 2024-09-16T01:56:17.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.219+0000 7f2266556640 1 -- 172.21.15.139:0/3850583806 learned_addr learned my addr 172.21.15.139:0/3850583806 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:17.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.219+0000 7f2266d57640 1 --2- 172.21.15.139:0/3850583806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bde3f0 0x561cb9bd8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.219+0000 7f2265d55640 1 --2- 172.21.15.139:0/3850583806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bd3b30 0x561cb9bd7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.220+0000 7f2266556640 1 -- 172.21.15.139:0/3850583806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bd3b30 msgr2=0x561cb9bd7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:17.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.220+0000 7f2266556640 1 --2- 172.21.15.139:0/3850583806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bd3b30 0x561cb9bd7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.220+0000 7f2266556640 1 -- 172.21.15.139:0/3850583806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bde3f0 msgr2=0x561cb9bd8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:17.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.220+0000 7f2266556640 1 --2- 172.21.15.139:0/3850583806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bde3f0 0x561cb9bd8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.220+0000 7f2266556640 1 -- 172.21.15.139:0/3850583806 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb9bd89d0 con 0x561cb9bd3210 2024-09-16T01:56:17.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.221+0000 7f2266556640 1 --2- 172.21.15.139:0/3850583806 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3210 0x561cb9be69e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f225c020000 tx=0x7f225c0092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f86758c4b073ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:17.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.222+0000 7f2249ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:56:17.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.227+0000 7f2249ffb640 1 -- 172.21.15.139:0/3850583806 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c09ba10 con 0x561cb9bd3210 2024-09-16T01:56:17.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.227+0000 7f2249ffb640 1 -- 172.21.15.139:0/3850583806 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7f225c0cc860 con 0x561cb9bd3210 2024-09-16T01:56:17.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.229+0000 7f226ab93740 1 -- 172.21.15.139:0/3850583806 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2258036040 con 0x561cb9bd3210 2024-09-16T01:56:17.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.232+0000 7f2249ffb640 1 -- 172.21.15.139:0/3850583806 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f225c0cca60 con 0x561cb9bd3210 2024-09-16T01:56:17.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.232+0000 7f226ab93740 10 client.26512 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:17.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.232+0000 7f226ab93740 1 -- 172.21.15.139:0/3850583806 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x561cb9c59200 con 0x561cb9bd3210 2024-09-16T01:56:17.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.235+0000 7f2249ffb640 1 -- 172.21.15.139:0/3850583806 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2258036040 con 0x561cb9bd3210 2024-09-16T01:56:17.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.235+0000 7f2249ffb640 1 -- 172.21.15.139:0/3850583806 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f225c096fd0 con 0x561cb9bd3210 2024-09-16T01:56:17.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.236+0000 7f226ab93740 10 client.26512 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:17.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.236+0000 7f226ab93740 10 client.26512 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:56:17.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.236+0000 7f226ab93740 20 client.26512 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:17.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.236+0000 7f226ab93740 1 --2- 172.21.15.139:0/3850583806 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561cb9c598a0 0x561cb9c5bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:17.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.236+0000 7f226ab93740 4 client.26512 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e4a5e579", "release": true, "await": true}] 2024-09-16T01:56:17.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.236+0000 7f226ab93740 1 -- 172.21.15.139:0/3850583806 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e4a5e579", "release": true, "await": true}) -- 0x561cb9b400a0 con 0x561cb9c598a0 2024-09-16T01:56:17.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.240+0000 7f2265d55640 1 --2- 172.21.15.139:0/3850583806 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561cb9c598a0 0x561cb9c5bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:17.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.247+0000 7f2265d55640 1 --2- 172.21.15.139:0/3850583806 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561cb9c598a0 0x561cb9c5bcc0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b670110c18d6b4 server_cookie=d83263b147981410 in_seq=0 out_seq=0 2024-09-16T01:56:17.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.247+0000 7f2249ffb640 10 client.26512 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:17.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.955+0000 7f2249ffb640 1 -- 172.21.15.139:0/3850583806 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561cb9b400a0 con 0x561cb9c598a0 2024-09-16T01:56:17.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.955+0000 7f2249ffb640 10 client.26512 handle_command_reply: tid=0 2024-09-16T01:56:17.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.955+0000 7f226ab93740 1 client.26512 shutdown 2024-09-16T01:56:17.959 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:56:17.959 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 71, 2024-09-16T01:56:17.959 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:56:17.959 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 56, 2024-09-16T01:56:17.959 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:56:17.959 INFO:teuthology.orchestra.run.smithi139.stdout: "e4a5e579": { 2024-09-16T01:56:17.960 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 56, 2024-09-16T01:56:17.960 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:56:17.960 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:56:17.960 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:56:17.960 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:56:17.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:56:17.960 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:56:17.960 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 137.7, 2024-09-16T01:56:17.960 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:56:17.960 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:56:17.960 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:56:17.960 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:56:17.961 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:56:17.961 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:56:17.961 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:17.961 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:17.961 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:17.961 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:17.961 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:17.961 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:56:17.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.956+0000 7f226ab93740 1 -- 172.21.15.139:0/3850583806 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3210 msgr2=0x561cb9be69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:17.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.956+0000 7f226ab93740 1 --2- 172.21.15.139:0/3850583806 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3210 0x561cb9be69e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f225c020000 tx=0x7f225c0092e0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.956+0000 7f226ab93740 1 -- 172.21.15.139:0/3850583806 shutdown_connections 2024-09-16T01:56:17.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.956+0000 7f226ab93740 1 --2- 172.21.15.139:0/3850583806 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561cb9c598a0 0x561cb9c5bcc0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.956+0000 7f226ab93740 1 --2- 172.21.15.139:0/3850583806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561cb9bde3f0 0x561cb9bd8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.956+0000 7f226ab93740 1 --2- 172.21.15.139:0/3850583806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb9bd3b30 0x561cb9bd7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.956+0000 7f226ab93740 1 --2- 172.21.15.139:0/3850583806 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561cb9bd3210 0x561cb9be69e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.956+0000 7f226ab93740 1 -- 172.21.15.139:0/3850583806 >> 172.21.15.139:0/3850583806 conn(0x561cb9bcf7a0 msgr2=0x561cb9be9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:17.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.956+0000 7f2249ffb640 0 client.26512 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:17.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.956+0000 7f226ab93740 1 -- 172.21.15.139:0/3850583806 shutdown_connections 2024-09-16T01:56:17.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.956+0000 7f226ab93740 1 -- 172.21.15.139:0/3850583806 wait complete. 2024-09-16T01:56:17.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.957+0000 7f226ab93740 20 client.26512 trim_cache size 0 max 16384 2024-09-16T01:56:17.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.957+0000 7f2267558640 1 -- 172.21.15.139:0/525673984 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f224c0a37f0 msgr2=0x7f224c0a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:17.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.957+0000 7f2267558640 1 --2- 172.21.15.139:0/525673984 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f224c0a37f0 0x7f224c0a5cb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2258009bf0 tx=0x7f2258005ef0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.957+0000 7f2267558640 1 -- 172.21.15.139:0/525673984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f226006ea30 msgr2=0x7f2260116d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:17.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.957+0000 7f2267558640 1 --2- 172.21.15.139:0/525673984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f226006ea30 0x7f2260116d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f225c009fd0 tx=0x7f225c00b870 comp rx=0 tx=0).stop 2024-09-16T01:56:17.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.958+0000 7f2267558640 1 -- 172.21.15.139:0/525673984 shutdown_connections 2024-09-16T01:56:17.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.958+0000 7f2267558640 1 --2- 172.21.15.139:0/525673984 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f224c0a37f0 0x7f224c0a5cb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.958+0000 7f2267558640 1 --2- 172.21.15.139:0/525673984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f226011b610 0x7f226011bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.958+0000 7f2267558640 1 --2- 172.21.15.139:0/525673984 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2260110190 0x7f2260117280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.958+0000 7f2267558640 1 --2- 172.21.15.139:0/525673984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f226006ea30 0x7f2260116d40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:17.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.958+0000 7f2267558640 1 -- 172.21.15.139:0/525673984 >> 172.21.15.139:0/525673984 conn(0x7f2260073180 msgr2=0x7f226006dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:17.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.958+0000 7f2267558640 1 -- 172.21.15.139:0/525673984 shutdown_connections 2024-09-16T01:56:17.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:17.958+0000 7f2267558640 1 -- 172.21.15.139:0/525673984 wait complete. 2024-09-16T01:56:18.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:17 smithi139 ceph-mon[31175]: pgmap v503: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 169 op/s 2024-09-16T01:56:18.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:17 smithi176 ceph-mon[35324]: pgmap v503: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 169 op/s 2024-09-16T01:56:18.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:17 smithi164 ceph-mon[35121]: pgmap v503: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 169 op/s 2024-09-16T01:56:18.695 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e4a5e579', total seconds elapsed: 24.3 2024-09-16T01:56:18.696 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-09-16T01:56:18.696 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:56:19.203 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:56:19.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.869+0000 7f36b993a640 1 -- 172.21.15.139:0/301928039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36b4109450 msgr2=0x7f36b4110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.869+0000 7f36b993a640 1 --2- 172.21.15.139:0/301928039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36b4109450 0x7f36b4110250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f369c009860 tx=0x7f369c02a340 comp rx=0 tx=0).stop 2024-09-16T01:56:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.871+0000 7f36b993a640 1 -- 172.21.15.139:0/301928039 shutdown_connections 2024-09-16T01:56:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.871+0000 7f36b993a640 1 --2- 172.21.15.139:0/301928039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36b411b140 0x7f36b411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.871+0000 7f36b993a640 1 --2- 172.21.15.139:0/301928039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36b4109830 0x7f36b4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.871+0000 7f36b993a640 1 --2- 172.21.15.139:0/301928039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36b4109450 0x7f36b4110250 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.871+0000 7f36b993a640 1 -- 172.21.15.139:0/301928039 >> 172.21.15.139:0/301928039 conn(0x7f36b406e140 msgr2=0x7f36b406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.871+0000 7f36b993a640 1 -- 172.21.15.139:0/301928039 shutdown_connections 2024-09-16T01:56:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.872+0000 7f36b993a640 1 -- 172.21.15.139:0/301928039 wait complete. 2024-09-16T01:56:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.872+0000 7f36b993a640 1 Processor -- start 2024-09-16T01:56:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.872+0000 7f36b993a640 1 -- start start 2024-09-16T01:56:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.873+0000 7f36b993a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36b4109450 0x7f36b411a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.873+0000 7f36b993a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36b4109830 0x7f36b41168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.873+0000 7f36b993a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36b411b140 0x7f36b4116e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.873+0000 7f36b993a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f36b4113320 con 0x7f36b4109830 2024-09-16T01:56:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.873+0000 7f36b993a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41131a0 con 0x7f36b411b140 2024-09-16T01:56:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.873+0000 7f36b993a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41134a0 con 0x7f36b4109450 2024-09-16T01:56:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.875+0000 7f36aa7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36b4109830 0x7f36b41168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.875+0000 7f36aa7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36b4109830 0x7f36b41168c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55052/0 (socket says 172.21.15.139:55052) 2024-09-16T01:56:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.875+0000 7f36aa7fc640 1 -- 172.21.15.139:0/4030077366 learned_addr learned my addr 172.21.15.139:0/4030077366 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.875+0000 7f36b37fe640 1 --2- 172.21.15.139:0/4030077366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36b411b140 0x7f36b4116e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.875+0000 7f36b37fe640 1 -- 172.21.15.139:0/4030077366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36b4109450 msgr2=0x7f36b411a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.875+0000 7f36b37fe640 1 --2- 172.21.15.139:0/4030077366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36b4109450 0x7f36b411a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.875+0000 7f36b37fe640 1 -- 172.21.15.139:0/4030077366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36b4109830 msgr2=0x7f36b41168c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.875+0000 7f36b37fe640 1 --2- 172.21.15.139:0/4030077366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36b4109830 0x7f36b41168c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.875+0000 7f36b37fe640 1 -- 172.21.15.139:0/4030077366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b4117530 con 0x7f36b411b140 2024-09-16T01:56:19.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.875+0000 7f36b37fe640 1 --2- 172.21.15.139:0/4030077366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36b411b140 0x7f36b4116e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00caf0 tx=0x7f36ac0020e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:19.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.876+0000 7f36b8938640 1 -- 172.21.15.139:0/4030077366 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac016030 con 0x7f36b411b140 2024-09-16T01:56:19.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.876+0000 7f36b8938640 1 -- 172.21.15.139:0/4030077366 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ac003e50 con 0x7f36b411b140 2024-09-16T01:56:19.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.876+0000 7f36b8938640 1 -- 172.21.15.139:0/4030077366 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac004690 con 0x7f36b411b140 2024-09-16T01:56:19.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.876+0000 7f36b993a640 1 -- 172.21.15.139:0/4030077366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b41ab170 con 0x7f36b411b140 2024-09-16T01:56:19.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.876+0000 7f36b993a640 1 -- 172.21.15.139:0/4030077366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b41ab570 con 0x7f36b411b140 2024-09-16T01:56:19.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.879+0000 7f36b993a640 1 -- 172.21.15.139:0/4030077366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36940052c0 con 0x7f36b411b140 2024-09-16T01:56:19.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.881+0000 7f36b8938640 1 -- 172.21.15.139:0/4030077366 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f36ac019070 con 0x7f36b411b140 2024-09-16T01:56:19.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.882+0000 7f36b8938640 1 --2- 172.21.15.139:0/4030077366 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f36a00a35c0 0x7f36a00a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:19.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.883+0000 7f36b8938640 1 -- 172.21.15.139:0/4030077366 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7f36ac0cf9d0 con 0x7f36b411b140 2024-09-16T01:56:19.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.883+0000 7f36b2ffd640 1 --2- 172.21.15.139:0/4030077366 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f36a00a35c0 0x7f36a00a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:19.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.884+0000 7f36b2ffd640 1 --2- 172.21.15.139:0/4030077366 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f36a00a35c0 0x7f36a00a5a80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f369c0095f0 tx=0x7f369c005760 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:19.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:19.888+0000 7f36b8938640 1 -- 172.21.15.139:0/4030077366 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f36ac09bfb0 con 0x7f36b411b140 2024-09-16T01:56:20.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:19 smithi139 ceph-mon[31175]: pgmap v504: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 173 op/s 2024-09-16T01:56:20.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:19 smithi176 ceph-mon[35324]: pgmap v504: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 173 op/s 2024-09-16T01:56:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.290+0000 7fedb4852640 1 -- 172.21.15.139:0/2888389780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feda8094930 msgr2=0x7feda8094d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.290+0000 7fedb4852640 1 --2- 172.21.15.139:0/2888389780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feda8094930 0x7feda8094d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feda4009890 tx=0x7feda402a3b0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.291+0000 7fedb4852640 1 -- 172.21.15.139:0/2888389780 shutdown_connections 2024-09-16T01:56:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.291+0000 7fedb4852640 1 --2- 172.21.15.139:0/2888389780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda80952d0 0x7feda80a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.291+0000 7fedb4852640 1 --2- 172.21.15.139:0/2888389780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feda8094930 0x7feda8094d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.291+0000 7fedb4852640 1 --2- 172.21.15.139:0/2888389780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feda809a5b0 0x7feda809a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.291+0000 7fedb4852640 1 -- 172.21.15.139:0/2888389780 >> 172.21.15.139:0/2888389780 conn(0x7feda800a220 msgr2=0x7feda800a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:20.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:19 smithi164 ceph-mon[35121]: pgmap v504: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 173 op/s 2024-09-16T01:56:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.297+0000 7fedb4852640 1 -- 172.21.15.139:0/2888389780 shutdown_connections 2024-09-16T01:56:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.299+0000 7fedb4852640 1 -- 172.21.15.139:0/2888389780 wait complete. 2024-09-16T01:56:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.300+0000 7fedb4852640 1 Processor -- start 2024-09-16T01:56:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.300+0000 7fedb4852640 1 -- start start 2024-09-16T01:56:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.300+0000 7fedb4852640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feda8094930 0x7feda8131b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.300+0000 7fedb4852640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feda80952d0 0x7feda81320b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.300+0000 7fedb4852640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda809a5b0 0x7feda8129e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.300+0000 7fedb4852640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feda80a4940 con 0x7feda809a5b0 2024-09-16T01:56:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.300+0000 7fedb4852640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feda80a47c0 con 0x7feda80952d0 2024-09-16T01:56:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.300+0000 7fedb4852640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feda80a4ac0 con 0x7feda8094930 2024-09-16T01:56:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.301+0000 7fedaf59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda809a5b0 0x7feda8129e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.301+0000 7fedaf59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda809a5b0 0x7feda8129e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55078/0 (socket says 172.21.15.139:55078) 2024-09-16T01:56:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.301+0000 7fedaf59e640 1 -- 172.21.15.139:0/2308384235 learned_addr learned my addr 172.21.15.139:0/2308384235 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.304+0000 7fedae59c640 1 --2- 172.21.15.139:0/2308384235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feda80952d0 0x7feda81320b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.305+0000 7fedaf59e640 1 -- 172.21.15.139:0/2308384235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feda8094930 msgr2=0x7feda8131b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:56:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.305+0000 7fedaf59e640 1 --2- 172.21.15.139:0/2308384235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feda8094930 0x7feda8131b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.305+0000 7fedaf59e640 1 -- 172.21.15.139:0/2308384235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feda80952d0 msgr2=0x7feda81320b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.305+0000 7fedaf59e640 1 --2- 172.21.15.139:0/2308384235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feda80952d0 0x7feda81320b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.305+0000 7fedaf59e640 1 -- 172.21.15.139:0/2308384235 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda812a730 con 0x7feda809a5b0 2024-09-16T01:56:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.305+0000 7fedaf59e640 1 --2- 172.21.15.139:0/2308384235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda809a5b0 0x7feda8129e70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fedb0067960 tx=0x7fedb007a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.305+0000 7fed97fff640 1 -- 172.21.15.139:0/2308384235 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0081070 con 0x7feda809a5b0 2024-09-16T01:56:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.306+0000 7fed97fff640 1 -- 172.21.15.139:0/2308384235 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedb0068dc0 con 0x7feda809a5b0 2024-09-16T01:56:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.306+0000 7fedb4852640 1 -- 172.21.15.139:0/2308384235 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda812aa20 con 0x7feda809a5b0 2024-09-16T01:56:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.306+0000 7fedb4852640 1 -- 172.21.15.139:0/2308384235 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda8147830 con 0x7feda809a5b0 2024-09-16T01:56:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.306+0000 7fed97fff640 1 -- 172.21.15.139:0/2308384235 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb007d850 con 0x7feda809a5b0 2024-09-16T01:56:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.313+0000 7fed97fff640 1 -- 172.21.15.139:0/2308384235 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fedb007b040 con 0x7feda809a5b0 2024-09-16T01:56:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.313+0000 7fedb7df2740 1 Processor -- start 2024-09-16T01:56:20.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:20.313+0000 7f36b993a640 1 -- 172.21.15.139:0/4030077366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3694005880 con 0x7f36b411b140 2024-09-16T01:56:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.313+0000 7fedb7df2740 1 -- start start 2024-09-16T01:56:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.313+0000 7fedb7df2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd42cc0 0x55e73bd430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.313+0000 7fedb7df2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd435e0 0x55e73bd4d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.313+0000 7fedb7df2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd4de80 0x55e73bd50270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.313+0000 7fedb7df2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e73bca72f0 con 0x55e73bd435e0 2024-09-16T01:56:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.313+0000 7fedb7df2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e73bc4f270 con 0x55e73bd4de80 2024-09-16T01:56:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.313+0000 7fedb7df2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e73bca32b0 con 0x55e73bd42cc0 2024-09-16T01:56:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.314+0000 7fedae59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd42cc0 0x55e73bd430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.314+0000 7fedae59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd42cc0 0x55e73bd430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:37068/0 (socket says 172.21.15.139:37068) 2024-09-16T01:56:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.314+0000 7fedae59c640 1 -- 172.21.15.139:0/2621157627 learned_addr learned my addr 172.21.15.139:0/2621157627 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.314+0000 7fedae59c640 1 --2- 172.21.15.139:0/2621157627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd4de80 0x55e73bd50270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.315+0000 7fed97fff640 1 --2- 172.21.15.139:0/2308384235 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fed980a3590 0x7fed980a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.315+0000 7fedae59c640 1 -- 172.21.15.139:0/2621157627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd42cc0 msgr2=0x55e73bd430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.315+0000 7fedae59c640 1 --2- 172.21.15.139:0/2621157627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd42cc0 0x55e73bd430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.315+0000 7fedae59c640 1 -- 172.21.15.139:0/2621157627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd435e0 msgr2=0x55e73bd4d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.315+0000 7fedae59c640 1 --2- 172.21.15.139:0/2621157627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd435e0 0x55e73bd4d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.315+0000 7fedae59c640 1 -- 172.21.15.139:0/2621157627 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e73bc4e340 con 0x55e73bd4de80 2024-09-16T01:56:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.315+0000 7fed97fff640 1 -- 172.21.15.139:0/2308384235 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7fedb013c050 con 0x7feda809a5b0 2024-09-16T01:56:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.315+0000 7fedae59c640 1 --2- 172.21.15.139:0/2621157627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd4de80 0x55e73bd50270 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feda402a8d0 tx=0x7feda402a3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12e8fc9f309eb802 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.315+0000 7fedaf59e640 1 --2- 172.21.15.139:0/2308384235 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fed980a3590 0x7fed980a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.316+0000 7fedaf59e640 1 --2- 172.21.15.139:0/2308384235 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fed980a3590 0x7fed980a5a50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fedb0108c10 tx=0x7fedb0067e40 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:20.320 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:56:20.320 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":71,"btime":"2024-09-16T01:54:19:332763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:54:19.051272+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:56:20.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:20.315+0000 7f36b8938640 1 -- 172.21.15.139:0/4030077366 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+12031 (secure 0 0 0) 0x7f36ac09b8b0 con 0x7f36b411b140 2024-09-16T01:56:20.321 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 71 2024-09-16T01:56:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.316+0000 7fed957fa640 1 -- 172.21.15.139:0/2621157627 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4047070 con 0x55e73bd4de80 2024-09-16T01:56:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.316+0000 7fed957fa640 1 -- 172.21.15.139:0/2621157627 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda4005040 con 0x55e73bd4de80 2024-09-16T01:56:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.317+0000 7fedb7df2740 1 -- 172.21.15.139:0/2621157627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd4de80 msgr2=0x55e73bd50270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.317+0000 7fedb7df2740 1 --2- 172.21.15.139:0/2621157627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd4de80 0x55e73bd50270 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feda402a8d0 tx=0x7feda402a3b0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.319+0000 7fedb7df2740 1 -- 172.21.15.139:0/2621157627 shutdown_connections 2024-09-16T01:56:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.319+0000 7fedb7df2740 1 --2- 172.21.15.139:0/2621157627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd4de80 0x55e73bd50270 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.319+0000 7fedb7df2740 1 --2- 172.21.15.139:0/2621157627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd435e0 0x55e73bd4d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.319+0000 7fedb7df2740 1 --2- 172.21.15.139:0/2621157627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd42cc0 0x55e73bd430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.319+0000 7fedb7df2740 1 -- 172.21.15.139:0/2621157627 >> 172.21.15.139:0/2621157627 conn(0x55e73bd3f230 msgr2=0x55e73bd3f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:20.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:20.321+0000 7f36b993a640 1 -- 172.21.15.139:0/4030077366 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f36a00a35c0 msgr2=0x7f36a00a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:20.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:20.321+0000 7f36b993a640 1 --2- 172.21.15.139:0/4030077366 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f36a00a35c0 0x7f36a00a5a80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f369c0095f0 tx=0x7f369c005760 comp rx=0 tx=0).stop 2024-09-16T01:56:20.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:20.321+0000 7f36b993a640 1 -- 172.21.15.139:0/4030077366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36b411b140 msgr2=0x7f36b4116e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:20.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:20.321+0000 7f36b993a640 1 --2- 172.21.15.139:0/4030077366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36b411b140 0x7f36b4116e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00caf0 tx=0x7f36ac0020e0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.327+0000 7fedb7df2740 1 -- 172.21.15.139:0/2621157627 shutdown_connections 2024-09-16T01:56:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.327+0000 7fedb7df2740 1 -- 172.21.15.139:0/2621157627 wait complete. 2024-09-16T01:56:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.327+0000 7fedb7df2740 1 -- start start 2024-09-16T01:56:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.327+0000 7fedb7df2740 1 Processor -- start 2024-09-16T01:56:20.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:20.327+0000 7f36b993a640 1 -- 172.21.15.139:0/4030077366 shutdown_connections 2024-09-16T01:56:20.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:20.327+0000 7f36b993a640 1 --2- 172.21.15.139:0/4030077366 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f36a00a35c0 0x7f36a00a5a80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:20.327+0000 7f36b993a640 1 --2- 172.21.15.139:0/4030077366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36b411b140 0x7f36b4116e00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:20.327+0000 7f36b993a640 1 --2- 172.21.15.139:0/4030077366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36b4109830 0x7f36b41168c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:20.327+0000 7f36b993a640 1 --2- 172.21.15.139:0/4030077366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36b4109450 0x7f36b411a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:20.327+0000 7f36b993a640 1 -- 172.21.15.139:0/4030077366 >> 172.21.15.139:0/4030077366 conn(0x7f36b406e140 msgr2=0x7f36b40722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.328+0000 7fedb7df2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd42cc0 0x55e73bd4ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.328+0000 7fedb7df2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd435e0 0x55e73bd4b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.328+0000 7fedb7df2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd4de80 0x55e73bd4b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.328+0000 7fedb7df2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e73bd50aa0 con 0x55e73bd42cc0 2024-09-16T01:56:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.328+0000 7fedb7df2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e73bc30290 con 0x55e73bd435e0 2024-09-16T01:56:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.328+0000 7fedb7df2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e73bd50c20 con 0x55e73bd4de80 2024-09-16T01:56:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.328+0000 7fedae59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd42cc0 0x55e73bd4ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.328+0000 7fedb7df2740 10 client.0 _ll_register_callbacks cb 0x55e73bc61140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.328+0000 7fedae59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd42cc0 0x55e73bd4ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55100/0 (socket says 172.21.15.139:55100) 2024-09-16T01:56:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.328+0000 7fedae59c640 1 -- 172.21.15.139:0/2016967963 learned_addr learned my addr 172.21.15.139:0/2016967963 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.329+0000 7fedae59c640 1 --2- 172.21.15.139:0/2016967963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd4de80 0x55e73bd4b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.330+0000 7fedae59c640 1 -- 172.21.15.139:0/2016967963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd4de80 msgr2=0x55e73bd4b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.330+0000 7fedae59c640 1 --2- 172.21.15.139:0/2016967963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd4de80 0x55e73bd4b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.330+0000 7fedae59c640 1 -- 172.21.15.139:0/2016967963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd435e0 msgr2=0x55e73bd4b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.330+0000 7fedae59c640 1 --2- 172.21.15.139:0/2016967963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd435e0 0x55e73bd4b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.330+0000 7fedae59c640 1 -- 172.21.15.139:0/2016967963 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e73b9ba210 con 0x55e73bd42cc0 2024-09-16T01:56:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.330+0000 7fedae59c640 1 --2- 172.21.15.139:0/2016967963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd42cc0 0x55e73bd4ad80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7feda4045040 tx=0x7feda4030eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ca39fafe2297b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.330+0000 7fed92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:56:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.331+0000 7fed92ff5640 1 -- 172.21.15.139:0/2016967963 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4047030 con 0x55e73bd42cc0 2024-09-16T01:56:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.331+0000 7fedb7df2740 1 -- 172.21.15.139:0/2016967963 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feda4005040 con 0x55e73bd42cc0 2024-09-16T01:56:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.331+0000 7fed92ff5640 1 -- 172.21.15.139:0/2016967963 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7feda40051f0 con 0x55e73bd42cc0 2024-09-16T01:56:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.331+0000 7fed92ff5640 1 -- 172.21.15.139:0/2016967963 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7feda4059050 con 0x55e73bd42cc0 2024-09-16T01:56:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.332+0000 7fedb7df2740 10 client.17130 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.332+0000 7fedb7df2740 1 -- 172.21.15.139:0/2016967963 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55e73ba86620 con 0x55e73bd42cc0 2024-09-16T01:56:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.333+0000 7fed92ff5640 1 -- 172.21.15.139:0/2016967963 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda4005040 con 0x55e73bd42cc0 2024-09-16T01:56:20.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:20.329+0000 7f36b993a640 1 -- 172.21.15.139:0/4030077366 shutdown_connections 2024-09-16T01:56:20.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:20.330+0000 7f36b993a640 1 -- 172.21.15.139:0/4030077366 wait complete. 2024-09-16T01:56:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.335+0000 7fed92ff5640 1 -- 172.21.15.139:0/2016967963 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7feda40035f0 con 0x55e73bd42cc0 2024-09-16T01:56:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.335+0000 7fedb7df2740 10 client.17130 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.335+0000 7fedb7df2740 10 client.17130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:56:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.335+0000 7fedb7df2740 20 client.17130 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.335+0000 7fedb7df2740 1 --2- 172.21.15.139:0/2016967963 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e73bd5dfc0 0x55e73bd603e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.335+0000 7fedb7df2740 4 client.17130 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:56:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.335+0000 7fedb7df2740 1 -- 172.21.15.139:0/2016967963 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e73bcb4a20 con 0x55e73bd5dfc0 2024-09-16T01:56:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.335+0000 7fedaed9d640 1 --2- 172.21.15.139:0/2016967963 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e73bd5dfc0 0x55e73bd603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.337+0000 7fedaed9d640 1 --2- 172.21.15.139:0/2016967963 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e73bd5dfc0 0x55e73bd603e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=289d070f1801644b server_cookie=7886aa75c0296bfc in_seq=0 out_seq=0 2024-09-16T01:56:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.337+0000 7fed92ff5640 10 client.17130 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.340+0000 7fed92ff5640 1 -- 172.21.15.139:0/2016967963 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e73bcb4a20 con 0x55e73bd5dfc0 2024-09-16T01:56:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.340+0000 7fed92ff5640 10 client.17130 handle_command_reply: tid=0 2024-09-16T01:56:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.340+0000 7fedb7df2740 1 client.17130 shutdown 2024-09-16T01:56:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.343+0000 7fedb7df2740 1 -- 172.21.15.139:0/2016967963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd42cc0 msgr2=0x55e73bd4ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.343+0000 7fedb7df2740 1 --2- 172.21.15.139:0/2016967963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd42cc0 0x55e73bd4ad80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7feda4045040 tx=0x7feda4030eb0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.346+0000 7fedb7df2740 1 -- 172.21.15.139:0/2016967963 shutdown_connections 2024-09-16T01:56:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.346+0000 7fedb7df2740 1 --2- 172.21.15.139:0/2016967963 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e73bd5dfc0 0x55e73bd603e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.346+0000 7fedb7df2740 1 --2- 172.21.15.139:0/2016967963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd4de80 0x55e73bd4b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.347+0000 7fedb7df2740 1 --2- 172.21.15.139:0/2016967963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd435e0 0x55e73bd4b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.347+0000 7fedb7df2740 1 --2- 172.21.15.139:0/2016967963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd42cc0 0x55e73bd4ad80 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.347+0000 7fedb7df2740 1 -- 172.21.15.139:0/2016967963 >> 172.21.15.139:0/2016967963 conn(0x55e73bd3f230 msgr2=0x55e73bd44f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.347+0000 7fed92ff5640 0 client.17130 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.351+0000 7fedb7df2740 1 -- 172.21.15.139:0/2016967963 shutdown_connections 2024-09-16T01:56:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.353+0000 7fedb7df2740 1 -- 172.21.15.139:0/2016967963 wait complete. 2024-09-16T01:56:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.354+0000 7fedb7df2740 20 client.17130 trim_cache size 0 max 16384 2024-09-16T01:56:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.383+0000 7fedb7df2740 1 Processor -- start 2024-09-16T01:56:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.383+0000 7fedb7df2740 1 -- start start 2024-09-16T01:56:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.383+0000 7fedb7df2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd42cc0 0x55e73bd45780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.383+0000 7fedb7df2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd435e0 0x55e73bd49630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.383+0000 7fedb7df2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd4de80 0x55e73bd4b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.383+0000 7fedae59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd42cc0 0x55e73bd45780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.383+0000 7fedb7df2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c010180 con 0x55e73bd4de80 2024-09-16T01:56:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.383+0000 7fedb7df2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c011fc0 con 0x55e73bd435e0 2024-09-16T01:56:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.383+0000 7fedb7df2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c010910 con 0x55e73bd42cc0 2024-09-16T01:56:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.383+0000 7fedae59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd42cc0 0x55e73bd45780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:37092/0 (socket says 172.21.15.139:37092) 2024-09-16T01:56:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.383+0000 7fedae59c640 1 -- 172.21.15.139:0/295534606 learned_addr learned my addr 172.21.15.139:0/295534606 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.383+0000 7fedae59c640 1 --2- 172.21.15.139:0/295534606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd4de80 0x55e73bd4b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.384+0000 7fedae59c640 1 -- 172.21.15.139:0/295534606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd435e0 msgr2=0x55e73bd49630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:56:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.384+0000 7fedae59c640 1 --2- 172.21.15.139:0/295534606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd435e0 0x55e73bd49630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.384+0000 7fedae59c640 1 -- 172.21.15.139:0/295534606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd4de80 msgr2=0x55e73bd4b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.384+0000 7fedae59c640 1 --2- 172.21.15.139:0/295534606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd4de80 0x55e73bd4b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.384+0000 7fedae59c640 1 -- 172.21.15.139:0/295534606 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e73bd49d10 con 0x55e73bd42cc0 2024-09-16T01:56:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.386+0000 7fedae59c640 1 --2- 172.21.15.139:0/295534606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd42cc0 0x55e73bd45780 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feda4004340 tx=0x7feda402a3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=238f4ed49cc5770 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.391+0000 7fed94ff9640 1 -- 172.21.15.139:0/295534606 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4085070 con 0x55e73bd42cc0 2024-09-16T01:56:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.391+0000 7fed94ff9640 1 -- 172.21.15.139:0/295534606 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda4003f30 con 0x55e73bd42cc0 2024-09-16T01:56:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.391+0000 7fed94ff9640 1 -- 172.21.15.139:0/295534606 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4047040 con 0x55e73bd42cc0 2024-09-16T01:56:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.392+0000 7fedb7df2740 1 -- 172.21.15.139:0/295534606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd42cc0 msgr2=0x55e73bd45780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.392+0000 7fedb7df2740 1 --2- 172.21.15.139:0/295534606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd42cc0 0x55e73bd45780 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feda4004340 tx=0x7feda402a3b0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.392+0000 7fedb7df2740 1 -- 172.21.15.139:0/295534606 shutdown_connections 2024-09-16T01:56:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.392+0000 7fedb7df2740 1 --2- 172.21.15.139:0/295534606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd4de80 0x55e73bd4b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.392+0000 7fedb7df2740 1 --2- 172.21.15.139:0/295534606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd435e0 0x55e73bd49630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.392+0000 7fedb7df2740 1 --2- 172.21.15.139:0/295534606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd42cc0 0x55e73bd45780 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.392+0000 7fedb7df2740 1 -- 172.21.15.139:0/295534606 >> 172.21.15.139:0/295534606 conn(0x55e73bd3f230 msgr2=0x55e73bd4ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.393+0000 7fedb7df2740 1 -- 172.21.15.139:0/295534606 shutdown_connections 2024-09-16T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.395+0000 7fedb7df2740 1 -- 172.21.15.139:0/295534606 wait complete. 2024-09-16T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.396+0000 7fedb7df2740 1 -- start start 2024-09-16T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.396+0000 7fedb7df2740 1 Processor -- start 2024-09-16T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.396+0000 7fedb7df2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd42cc0 0x55e73bd56040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.396+0000 7fedb7df2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd435e0 0x55e73bd56580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.396+0000 7fedb7df2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd4de80 0x55e73bd51560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.396+0000 7fedb7df2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c011830 con 0x55e73bd435e0 2024-09-16T01:56:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.396+0000 7fedb7df2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c0110a0 con 0x55e73bd42cc0 2024-09-16T01:56:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.396+0000 7fedb7df2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c00e210 con 0x55e73bd4de80 2024-09-16T01:56:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.396+0000 7fedb7df2740 10 client.0 _ll_register_callbacks cb 0x7feda4045070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.397+0000 7fedae59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd42cc0 0x55e73bd56040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.397+0000 7fedae59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd4de80 0x55e73bd51560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.397+0000 7fedae59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd42cc0 0x55e73bd56040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:57050/0 (socket says 172.21.15.139:57050) 2024-09-16T01:56:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.397+0000 7fedae59c640 1 -- 172.21.15.139:0/3501169251 learned_addr learned my addr 172.21.15.139:0/3501169251 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.398+0000 7fedae59c640 1 -- 172.21.15.139:0/3501169251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd4de80 msgr2=0x55e73bd51560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.398+0000 7fedae59c640 1 --2- 172.21.15.139:0/3501169251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd4de80 0x55e73bd51560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.398+0000 7fedae59c640 1 -- 172.21.15.139:0/3501169251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd435e0 msgr2=0x55e73bd56580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.398+0000 7fedae59c640 1 --2- 172.21.15.139:0/3501169251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd435e0 0x55e73bd56580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.398+0000 7fedae59c640 1 -- 172.21.15.139:0/3501169251 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e73bd51d10 con 0x55e73bd42cc0 2024-09-16T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.400+0000 7fedae59c640 1 --2- 172.21.15.139:0/3501169251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd42cc0 0x55e73bd56040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feda4030f80 tx=0x7feda4030420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d2b228781393bc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.400+0000 7fed92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:56:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.400+0000 7fed92ff5640 1 -- 172.21.15.139:0/3501169251 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4085040 con 0x55e73bd42cc0 2024-09-16T01:56:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.400+0000 7fedb7df2740 1 -- 172.21.15.139:0/3501169251 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feda4003f30 con 0x55e73bd42cc0 2024-09-16T01:56:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.400+0000 7fed92ff5640 1 -- 172.21.15.139:0/3501169251 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7feda4052470 con 0x55e73bd42cc0 2024-09-16T01:56:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.400+0000 7fed92ff5640 1 -- 172.21.15.139:0/3501169251 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7feda4095050 con 0x55e73bd42cc0 2024-09-16T01:56:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.401+0000 7fedb7df2740 10 client.26573 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.401+0000 7fedb7df2740 1 -- 172.21.15.139:0/3501169251 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55e73bdd96e0 con 0x55e73bd42cc0 2024-09-16T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.401+0000 7fed92ff5640 1 -- 172.21.15.139:0/3501169251 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda4003f30 con 0x55e73bd42cc0 2024-09-16T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.401+0000 7fed92ff5640 1 -- 172.21.15.139:0/3501169251 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7feda408b040 con 0x55e73bd42cc0 2024-09-16T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.401+0000 7fedb7df2740 10 client.26573 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.401+0000 7fedb7df2740 10 client.26573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.401+0000 7fedb7df2740 20 client.26573 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.401+0000 7fedb7df2740 1 --2- 172.21.15.139:0/3501169251 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e73bdd9d40 0x55e73bddc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.401+0000 7fedb7df2740 4 client.26573 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.401+0000 7fedb7df2740 1 -- 172.21.15.139:0/3501169251 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e73bcaed70 con 0x55e73bdd9d40 2024-09-16T01:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.402+0000 7fedaed9d640 1 --2- 172.21.15.139:0/3501169251 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e73bdd9d40 0x55e73bddc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.405+0000 7fedaed9d640 1 --2- 172.21.15.139:0/3501169251 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e73bdd9d40 0x55e73bddc160 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a1f544ad55a856 server_cookie=4be4e946138b94ae in_seq=0 out_seq=0 2024-09-16T01:56:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.407+0000 7fed92ff5640 10 client.26573 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.577+0000 7fed92ff5640 1 -- 172.21.15.139:0/3501169251 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e73bcaed70 con 0x55e73bdd9d40 2024-09-16T01:56:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.578+0000 7fed92ff5640 10 client.26573 handle_command_reply: tid=0 2024-09-16T01:56:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.578+0000 7fedb7df2740 1 client.26573 shutdown 2024-09-16T01:56:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.578+0000 7fedb7df2740 1 -- 172.21.15.139:0/3501169251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd42cc0 msgr2=0x55e73bd56040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.578+0000 7fedb7df2740 1 --2- 172.21.15.139:0/3501169251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd42cc0 0x55e73bd56040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feda4030f80 tx=0x7feda4030420 comp rx=0 tx=0).stop 2024-09-16T01:56:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.578+0000 7fedb7df2740 1 -- 172.21.15.139:0/3501169251 shutdown_connections 2024-09-16T01:56:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.578+0000 7fedb7df2740 1 --2- 172.21.15.139:0/3501169251 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e73bdd9d40 0x55e73bddc160 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.578+0000 7fedb7df2740 1 --2- 172.21.15.139:0/3501169251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e73bd4de80 0x55e73bd51560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.578+0000 7fedb7df2740 1 --2- 172.21.15.139:0/3501169251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e73bd435e0 0x55e73bd56580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.578+0000 7fedb7df2740 1 --2- 172.21.15.139:0/3501169251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e73bd42cc0 0x55e73bd56040 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.578+0000 7fedb7df2740 1 -- 172.21.15.139:0/3501169251 >> 172.21.15.139:0/3501169251 conn(0x55e73bd3f230 msgr2=0x55e73bd3e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.578+0000 7fed92ff5640 0 client.26573 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.578+0000 7fedb7df2740 1 -- 172.21.15.139:0/3501169251 shutdown_connections 2024-09-16T01:56:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.578+0000 7fedb7df2740 1 -- 172.21.15.139:0/3501169251 wait complete. 2024-09-16T01:56:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.578+0000 7fedb7df2740 20 client.26573 trim_cache size 0 max 16384 2024-09-16T01:56:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:56:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:56:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:56:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:56:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.579+0000 7fedb4852640 1 -- 172.21.15.139:0/2308384235 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fed980a3590 msgr2=0x7fed980a5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.579+0000 7fedb4852640 1 --2- 172.21.15.139:0/2308384235 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fed980a3590 0x7fed980a5a50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fedb0108c10 tx=0x7fedb0067e40 comp rx=0 tx=0).stop 2024-09-16T01:56:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.579+0000 7fedb4852640 1 -- 172.21.15.139:0/2308384235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda809a5b0 msgr2=0x7feda8129e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.579+0000 7fedb4852640 1 --2- 172.21.15.139:0/2308384235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda809a5b0 0x7feda8129e70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fedb0067960 tx=0x7fedb007a690 comp rx=0 tx=0).stop 2024-09-16T01:56:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.579+0000 7fedb4852640 1 -- 172.21.15.139:0/2308384235 shutdown_connections 2024-09-16T01:56:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.579+0000 7fedb4852640 1 --2- 172.21.15.139:0/2308384235 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fed980a3590 0x7fed980a5a50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.580+0000 7fedb4852640 1 --2- 172.21.15.139:0/2308384235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda809a5b0 0x7feda8129e70 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.580+0000 7fedb4852640 1 --2- 172.21.15.139:0/2308384235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feda80952d0 0x7feda81320b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.580+0000 7fedb4852640 1 --2- 172.21.15.139:0/2308384235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feda8094930 0x7feda8131b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.580+0000 7fedb4852640 1 -- 172.21.15.139:0/2308384235 >> 172.21.15.139:0/2308384235 conn(0x7feda800a220 msgr2=0x7feda8004dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.581+0000 7fedb4852640 1 -- 172.21.15.139:0/2308384235 shutdown_connections 2024-09-16T01:56:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:20.581+0000 7fedb4852640 1 -- 172.21.15.139:0/2308384235 wait complete. 2024-09-16T01:56:21.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:20 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/4030077366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:56:21.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:20 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/4030077366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:56:21.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:20 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/4030077366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:56:21.836 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0' 2024-09-16T01:56:21.836 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0 2024-09-16T01:56:22.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:21 smithi139 ceph-mon[31175]: pgmap v505: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 81 KiB/s wr, 166 op/s 2024-09-16T01:56:22.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:21 smithi176 ceph-mon[35324]: pgmap v505: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 81 KiB/s wr, 166 op/s 2024-09-16T01:56:22.294 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:21 smithi164 ceph-mon[35121]: pgmap v505: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 81 KiB/s wr, 166 op/s 2024-09-16T01:56:22.325 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:56:23.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.030+0000 7fd761f86640 1 -- 172.21.15.139:0/591906495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd75c10ef30 msgr2=0x7fd75c10d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:23.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.030+0000 7fd761f86640 1 --2- 172.21.15.139:0/591906495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd75c10ef30 0x7fd75c10d2a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0073b0 tx=0x7fd74c02aff0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.031+0000 7fd761f86640 1 -- 172.21.15.139:0/591906495 shutdown_connections 2024-09-16T01:56:23.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.031+0000 7fd761f86640 1 --2- 172.21.15.139:0/591906495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd75c10d9b0 0x7fd75c115510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.031+0000 7fd761f86640 1 --2- 172.21.15.139:0/591906495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd75c10ef30 0x7fd75c10d2a0 secure :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0073b0 tx=0x7fd74c02aff0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.031+0000 7fd761f86640 1 --2- 172.21.15.139:0/591906495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd75c10e580 0x7fd75c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.031+0000 7fd761f86640 1 -- 172.21.15.139:0/591906495 >> 172.21.15.139:0/591906495 conn(0x7fd75c06e140 msgr2=0x7fd75c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:23.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.032+0000 7fd761f86640 1 -- 172.21.15.139:0/591906495 shutdown_connections 2024-09-16T01:56:23.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.033+0000 7fd761f86640 1 -- 172.21.15.139:0/591906495 wait complete. 2024-09-16T01:56:23.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.034+0000 7fd761f86640 1 Processor -- start 2024-09-16T01:56:23.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.037+0000 7fd761f86640 1 -- start start 2024-09-16T01:56:23.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.038+0000 7fd761f86640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd75c10d9b0 0x7fd75c11e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.038+0000 7fd761f86640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd75c10e580 0x7fd75c118a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.038+0000 7fd761f86640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd75c118fb0 0x7fd75c119460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.038+0000 7fd761f86640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c117290 con 0x7fd75c118fb0 2024-09-16T01:56:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.038+0000 7fd761f86640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c117110 con 0x7fd75c10e580 2024-09-16T01:56:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.038+0000 7fd761f86640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c117410 con 0x7fd75c10d9b0 2024-09-16T01:56:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.040+0000 7fd75bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd75c118fb0 0x7fd75c119460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.040+0000 7fd75bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd75c118fb0 0x7fd75c119460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59942/0 (socket says 172.21.15.139:59942) 2024-09-16T01:56:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.040+0000 7fd75bfff640 1 -- 172.21.15.139:0/2808381626 learned_addr learned my addr 172.21.15.139:0/2808381626 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.040+0000 7fd75b7fe640 1 --2- 172.21.15.139:0/2808381626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd75c10d9b0 0x7fd75c11e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.041+0000 7fd75b7fe640 1 -- 172.21.15.139:0/2808381626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd75c10e580 msgr2=0x7fd75c118a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:56:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.041+0000 7fd75b7fe640 1 --2- 172.21.15.139:0/2808381626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd75c10e580 0x7fd75c118a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.041+0000 7fd75b7fe640 1 -- 172.21.15.139:0/2808381626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd75c118fb0 msgr2=0x7fd75c119460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.041+0000 7fd75b7fe640 1 --2- 172.21.15.139:0/2808381626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd75c118fb0 0x7fd75c119460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.041+0000 7fd75b7fe640 1 -- 172.21.15.139:0/2808381626 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75c1bc2d0 con 0x7fd75c10d9b0 2024-09-16T01:56:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.042+0000 7fd75b7fe640 1 --2- 172.21.15.139:0/2808381626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd75c10d9b0 0x7fd75c11e9f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd75000c970 tx=0x7fd75000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.042+0000 7fd758ff9640 1 -- 172.21.15.139:0/2808381626 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750007bb0 con 0x7fd75c10d9b0 2024-09-16T01:56:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.042+0000 7fd761f86640 1 -- 172.21.15.139:0/2808381626 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75c1bc5c0 con 0x7fd75c10d9b0 2024-09-16T01:56:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.042+0000 7fd761f86640 1 -- 172.21.15.139:0/2808381626 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75c1bcb20 con 0x7fd75c10d9b0 2024-09-16T01:56:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.043+0000 7fd758ff9640 1 -- 172.21.15.139:0/2808381626 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd750007d50 con 0x7fd75c10d9b0 2024-09-16T01:56:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.043+0000 7fd758ff9640 1 -- 172.21.15.139:0/2808381626 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750018860 con 0x7fd75c10d9b0 2024-09-16T01:56:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.046+0000 7fd76435f740 1 Processor -- start 2024-09-16T01:56:23.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.053+0000 7fd76435f740 1 -- start start 2024-09-16T01:56:23.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.053+0000 7fd758ff9640 1 -- 172.21.15.139:0/2808381626 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd750018a00 con 0x7fd75c10d9b0 2024-09-16T01:56:23.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.053+0000 7fd76435f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3e6760 0x557c0c3e6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.053+0000 7fd76435f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e7080 0x557c0c3f1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.053+0000 7fd76435f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3f1940 0x557c0c3f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.053+0000 7fd76435f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557c0c34c2a0 con 0x557c0c3e6760 2024-09-16T01:56:23.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.053+0000 7fd76435f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557c0c2f3b10 con 0x557c0c3f1940 2024-09-16T01:56:23.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.053+0000 7fd76435f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557c0c348260 con 0x557c0c3e7080 2024-09-16T01:56:23.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.054+0000 7fd758ff9640 1 --2- 172.21.15.139:0/2808381626 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd7440a3520 0x7fd7440a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.054+0000 7fd758ff9640 1 -- 172.21.15.139:0/2808381626 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7fd7500d0150 con 0x7fd75c10d9b0 2024-09-16T01:56:23.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.055+0000 7fd75affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3f1940 0x557c0c3f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.055+0000 7fd75affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3f1940 0x557c0c3f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47568/0 (socket says 172.21.15.139:47568) 2024-09-16T01:56:23.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.055+0000 7fd75affd640 1 -- 172.21.15.139:0/1528286796 learned_addr learned my addr 172.21.15.139:0/1528286796 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:23.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.055+0000 7fd75bfff640 1 --2- 172.21.15.139:0/1528286796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3e6760 0x557c0c3e6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.056+0000 7fd75b7fe640 1 --2- 172.21.15.139:0/1528286796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e7080 0x557c0c3f1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.056+0000 7fd75affd640 1 -- 172.21.15.139:0/1528286796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e7080 msgr2=0x557c0c3f1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:23.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.056+0000 7fd75affd640 1 --2- 172.21.15.139:0/1528286796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e7080 0x557c0c3f1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.056+0000 7fd75affd640 1 -- 172.21.15.139:0/1528286796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3e6760 msgr2=0x557c0c3e6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:23.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.056+0000 7fd75affd640 1 --2- 172.21.15.139:0/1528286796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3e6760 0x557c0c3e6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.056+0000 7fd75affd640 1 -- 172.21.15.139:0/1528286796 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0c0bdb90 con 0x557c0c3f1940 2024-09-16T01:56:23.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.057+0000 7fd75affd640 1 --2- 172.21.15.139:0/1528286796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3f1940 0x557c0c3f3d30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd74c035040 tx=0x7fd74c008af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ef341e9cd0e29d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:23.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.057+0000 7fd75bfff640 1 --2- 172.21.15.139:0/2808381626 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd7440a3520 0x7fd7440a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.057+0000 7fd741ffb640 1 -- 172.21.15.139:0/1528286796 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c00adf0 con 0x557c0c3f1940 2024-09-16T01:56:23.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.057+0000 7fd741ffb640 1 -- 172.21.15.139:0/1528286796 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd74c005040 con 0x557c0c3f1940 2024-09-16T01:56:23.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.057+0000 7fd741ffb640 1 -- 172.21.15.139:0/1528286796 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c00adf0 con 0x557c0c3f1940 2024-09-16T01:56:23.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.058+0000 7fd75bfff640 1 --2- 172.21.15.139:0/2808381626 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd7440a3520 0x7fd7440a59e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd754009fd0 tx=0x7fd754004a40 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:23.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.063+0000 7fd76435f740 1 -- 172.21.15.139:0/1528286796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3f1940 msgr2=0x557c0c3f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:23.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.064+0000 7fd76435f740 1 --2- 172.21.15.139:0/1528286796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3f1940 0x557c0c3f3d30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd74c035040 tx=0x7fd74c008af0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.064+0000 7fd76435f740 1 -- 172.21.15.139:0/1528286796 shutdown_connections 2024-09-16T01:56:23.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.064+0000 7fd76435f740 1 --2- 172.21.15.139:0/1528286796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3f1940 0x557c0c3f3d30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.064+0000 7fd76435f740 1 --2- 172.21.15.139:0/1528286796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e7080 0x557c0c3f1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.064+0000 7fd76435f740 1 --2- 172.21.15.139:0/1528286796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3e6760 0x557c0c3e6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.064+0000 7fd76435f740 1 -- 172.21.15.139:0/1528286796 >> 172.21.15.139:0/1528286796 conn(0x557c0c3e2cf0 msgr2=0x557c0c3e30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:23.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.066+0000 7fd76435f740 1 -- 172.21.15.139:0/1528286796 shutdown_connections 2024-09-16T01:56:23.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.069+0000 7fd76435f740 1 -- 172.21.15.139:0/1528286796 wait complete. 2024-09-16T01:56:23.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.069+0000 7fd76435f740 1 -- start start 2024-09-16T01:56:23.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.070+0000 7fd76435f740 1 Processor -- start 2024-09-16T01:56:23.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.071+0000 7fd76435f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e6760 0x557c0c3ee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.071+0000 7fd76435f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 0x557c0c3eec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.071+0000 7fd76435f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3f1940 0x557c0c3ef160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.071+0000 7fd76435f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557c0c3f4510 con 0x557c0c3f1940 2024-09-16T01:56:23.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.071+0000 7fd76435f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557c0c2d46e0 con 0x557c0c3e7080 2024-09-16T01:56:23.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.071+0000 7fd76435f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557c0c3f4690 con 0x557c0c3e6760 2024-09-16T01:56:23.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.071+0000 7fd76435f740 10 client.0 _ll_register_callbacks cb 0x557c0c2f8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:23.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.073+0000 7fd75bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3f1940 0x557c0c3ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.077+0000 7fd75bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3f1940 0x557c0c3ef160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59966/0 (socket says 172.21.15.139:59966) 2024-09-16T01:56:23.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.077+0000 7fd75bfff640 1 -- 172.21.15.139:0/1210998644 learned_addr learned my addr 172.21.15.139:0/1210998644 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:23.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.078+0000 7fd75b7fe640 1 --2- 172.21.15.139:0/1210998644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e6760 0x557c0c3ee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.078+0000 7fd75affd640 1 --2- 172.21.15.139:0/1210998644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 0x557c0c3eec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.078+0000 7fd75bfff640 1 -- 172.21.15.139:0/1210998644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e6760 msgr2=0x557c0c3ee6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:23.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.078+0000 7fd75bfff640 1 --2- 172.21.15.139:0/1210998644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e6760 0x557c0c3ee6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.078+0000 7fd75bfff640 1 -- 172.21.15.139:0/1210998644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 msgr2=0x557c0c3eec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:23.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.078+0000 7fd75bfff640 1 --2- 172.21.15.139:0/1210998644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 0x557c0c3eec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.078+0000 7fd75bfff640 1 -- 172.21.15.139:0/1210998644 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c0c3fda10 con 0x557c0c3f1940 2024-09-16T01:56:23.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.078+0000 7fd75bfff640 1 --2- 172.21.15.139:0/1210998644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3f1940 0x557c0c3ef160 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd75401be00 tx=0x7fd754004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b5084d0b962f49 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:23.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.078+0000 7fd73affd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:56:23.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.079+0000 7fd73affd640 1 -- 172.21.15.139:0/1210998644 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754029060 con 0x557c0c3f1940 2024-09-16T01:56:23.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.079+0000 7fd73affd640 1 -- 172.21.15.139:0/1210998644 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7fd754016030 con 0x557c0c3f1940 2024-09-16T01:56:23.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.079+0000 7fd76435f740 1 -- 172.21.15.139:0/1210998644 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd74c005040 con 0x557c0c3f1940 2024-09-16T01:56:23.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.082+0000 7fd73affd640 1 -- 172.21.15.139:0/1210998644 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fd754024410 con 0x557c0c3f1940 2024-09-16T01:56:23.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.083+0000 7fd76435f740 10 client.17145 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:23.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.083+0000 7fd76435f740 1 -- 172.21.15.139:0/1210998644 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x557c0c400370 con 0x557c0c3f1940 2024-09-16T01:56:23.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.083+0000 7fd75affd640 1 --2- 172.21.15.139:0/1210998644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 0x557c0c3eec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:56:23.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.084+0000 7fd73affd640 1 -- 172.21.15.139:0/1210998644 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd74c005040 con 0x557c0c3f1940 2024-09-16T01:56:23.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.084+0000 7fd73affd640 1 -- 172.21.15.139:0/1210998644 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7fd75400b070 con 0x557c0c3f1940 2024-09-16T01:56:23.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.084+0000 7fd76435f740 10 client.17145 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:23.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.084+0000 7fd76435f740 10 client.17145 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:56:23.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.084+0000 7fd76435f740 20 client.17145 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:23.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.084+0000 7fd76435f740 1 --2- 172.21.15.139:0/1210998644 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557c0c401c80 0x557c0c404140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.085+0000 7fd76435f740 4 client.17145 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:56:23.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.085+0000 7fd76435f740 1 -- 172.21.15.139:0/1210998644 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c0c3599d0 con 0x557c0c401c80 2024-09-16T01:56:23.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.085+0000 7fd75b7fe640 1 --2- 172.21.15.139:0/1210998644 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557c0c401c80 0x557c0c404140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.088+0000 7fd75b7fe640 1 --2- 172.21.15.139:0/1210998644 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557c0c401c80 0x557c0c404140 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cb02c4c719852b1 server_cookie=9d96b4bb2c0e9de4 in_seq=0 out_seq=0 2024-09-16T01:56:23.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.088+0000 7fd73affd640 10 client.17145 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:23.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.091+0000 7fd73affd640 1 -- 172.21.15.139:0/1210998644 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c0c3599d0 con 0x557c0c401c80 2024-09-16T01:56:23.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.091+0000 7fd73affd640 10 client.17145 handle_command_reply: tid=0 2024-09-16T01:56:23.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.092+0000 7fd76435f740 1 client.17145 shutdown 2024-09-16T01:56:23.098 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:22 smithi139 ceph-mon[31175]: pgmap v506: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 69 KiB/s wr, 147 op/s 2024-09-16T01:56:23.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.096+0000 7fd76435f740 1 -- 172.21.15.139:0/1210998644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3f1940 msgr2=0x557c0c3ef160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:23.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.096+0000 7fd76435f740 1 --2- 172.21.15.139:0/1210998644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3f1940 0x557c0c3ef160 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd75401be00 tx=0x7fd754004ab0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.097+0000 7fd76435f740 1 -- 172.21.15.139:0/1210998644 shutdown_connections 2024-09-16T01:56:23.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.097+0000 7fd76435f740 1 --2- 172.21.15.139:0/1210998644 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557c0c401c80 0x557c0c404140 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.097+0000 7fd76435f740 1 --2- 172.21.15.139:0/1210998644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3f1940 0x557c0c3ef160 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.097+0000 7fd76435f740 1 --2- 172.21.15.139:0/1210998644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 0x557c0c3eec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.097+0000 7fd76435f740 1 --2- 172.21.15.139:0/1210998644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e6760 0x557c0c3ee6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.097+0000 7fd76435f740 1 -- 172.21.15.139:0/1210998644 >> 172.21.15.139:0/1210998644 conn(0x557c0c3e2cf0 msgr2=0x557c0c3e89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:23.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.097+0000 7fd73affd640 0 client.17145 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:23.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.097+0000 7fd76435f740 1 -- 172.21.15.139:0/1210998644 shutdown_connections 2024-09-16T01:56:23.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.097+0000 7fd76435f740 1 -- 172.21.15.139:0/1210998644 wait complete. 2024-09-16T01:56:23.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.098+0000 7fd76435f740 20 client.17145 trim_cache size 0 max 16384 2024-09-16T01:56:23.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.129+0000 7fd76435f740 1 Processor -- start 2024-09-16T01:56:23.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.129+0000 7fd76435f740 1 -- start start 2024-09-16T01:56:23.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.130+0000 7fd76435f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3e6760 0x557c0c3e1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.130+0000 7fd76435f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 0x557c0c409a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.130+0000 7fd76435f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3f1940 0x557c0c409f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.130+0000 7fd75b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3e6760 0x557c0c3e1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.130+0000 7fd75b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3e6760 0x557c0c3e1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59970/0 (socket says 172.21.15.139:59970) 2024-09-16T01:56:23.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.130+0000 7fd76435f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd7480102e0 con 0x557c0c3e6760 2024-09-16T01:56:23.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.130+0000 7fd76435f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd748012120 con 0x557c0c3e7080 2024-09-16T01:56:23.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.131+0000 7fd76435f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd748010a70 con 0x557c0c3f1940 2024-09-16T01:56:23.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.132+0000 7fd75affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 0x557c0c409a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.132+0000 7fd75bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3f1940 0x557c0c409f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.132+0000 7fd75bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3f1940 0x557c0c409f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:53584/0 (socket says 172.21.15.139:53584) 2024-09-16T01:56:23.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.132+0000 7fd75bfff640 1 -- 172.21.15.139:0/1883685845 learned_addr learned my addr 172.21.15.139:0/1883685845 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:23.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.133+0000 7fd75bfff640 1 -- 172.21.15.139:0/1883685845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 msgr2=0x557c0c409a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:23.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.133+0000 7fd75bfff640 1 --2- 172.21.15.139:0/1883685845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 0x557c0c409a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.133+0000 7fd75bfff640 1 -- 172.21.15.139:0/1883685845 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3e6760 msgr2=0x557c0c3e1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:23.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.133+0000 7fd75bfff640 1 --2- 172.21.15.139:0/1883685845 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3e6760 0x557c0c3e1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.133+0000 7fd75bfff640 1 -- 172.21.15.139:0/1883685845 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0c3e2b20 con 0x557c0c3f1940 2024-09-16T01:56:23.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.134+0000 7fd75b7fe640 1 --2- 172.21.15.139:0/1883685845 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3e6760 0x557c0c3e1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-16T01:56:23.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.140+0000 7fd75bfff640 1 --2- 172.21.15.139:0/1883685845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3f1940 0x557c0c409f90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd754017040 tx=0x7fd754005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e68580ec73940875 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:23.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.140+0000 7fd7417fa640 1 -- 172.21.15.139:0/1883685845 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754033080 con 0x557c0c3f1940 2024-09-16T01:56:23.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.146+0000 7fd7417fa640 1 -- 172.21.15.139:0/1883685845 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7540193e0 con 0x557c0c3f1940 2024-09-16T01:56:23.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.146+0000 7fd7417fa640 1 -- 172.21.15.139:0/1883685845 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754024e30 con 0x557c0c3f1940 2024-09-16T01:56:23.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.147+0000 7fd76435f740 1 -- 172.21.15.139:0/1883685845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3f1940 msgr2=0x557c0c409f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:23.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.147+0000 7fd76435f740 1 --2- 172.21.15.139:0/1883685845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3f1940 0x557c0c409f90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd754017040 tx=0x7fd754005b50 comp rx=0 tx=0).stop 2024-09-16T01:56:23.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.148+0000 7fd76435f740 1 -- 172.21.15.139:0/1883685845 shutdown_connections 2024-09-16T01:56:23.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.148+0000 7fd76435f740 1 --2- 172.21.15.139:0/1883685845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3f1940 0x557c0c409f90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.148+0000 7fd76435f740 1 --2- 172.21.15.139:0/1883685845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 0x557c0c409a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.148+0000 7fd76435f740 1 --2- 172.21.15.139:0/1883685845 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3e6760 0x557c0c3e1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.148+0000 7fd76435f740 1 -- 172.21.15.139:0/1883685845 >> 172.21.15.139:0/1883685845 conn(0x557c0c3e2cf0 msgr2=0x557c0c3e1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:23.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.149+0000 7fd76435f740 1 -- 172.21.15.139:0/1883685845 shutdown_connections 2024-09-16T01:56:23.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.150+0000 7fd76435f740 1 -- 172.21.15.139:0/1883685845 wait complete. 2024-09-16T01:56:23.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.150+0000 7fd76435f740 1 -- start start 2024-09-16T01:56:23.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.151+0000 7fd76435f740 1 Processor -- start 2024-09-16T01:56:23.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.151+0000 7fd76435f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e6760 0x557c0c3f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.151+0000 7fd76435f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 0x557c0c3f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.151+0000 7fd76435f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3f1940 0x557c0c3f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.151+0000 7fd76435f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd748011990 con 0x557c0c3f1940 2024-09-16T01:56:23.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.151+0000 7fd76435f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd748011200 con 0x557c0c3e7080 2024-09-16T01:56:23.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.151+0000 7fd76435f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd74800e2c0 con 0x557c0c3e6760 2024-09-16T01:56:23.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.151+0000 7fd76435f740 10 client.0 _ll_register_callbacks cb 0x7fd75401ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:23.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.155+0000 7fd75bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3f1940 0x557c0c3f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.155+0000 7fd75bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3f1940 0x557c0c3f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59978/0 (socket says 172.21.15.139:59978) 2024-09-16T01:56:23.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.155+0000 7fd75bfff640 1 -- 172.21.15.139:0/4150265523 learned_addr learned my addr 172.21.15.139:0/4150265523 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:23.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.157+0000 7fd75affd640 1 --2- 172.21.15.139:0/4150265523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 0x557c0c3f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.159+0000 7fd75b7fe640 1 --2- 172.21.15.139:0/4150265523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e6760 0x557c0c3f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.161+0000 7fd75affd640 1 -- 172.21.15.139:0/4150265523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e6760 msgr2=0x557c0c3f29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:23.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.161+0000 7fd75affd640 1 --2- 172.21.15.139:0/4150265523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e6760 0x557c0c3f29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.161+0000 7fd75affd640 1 -- 172.21.15.139:0/4150265523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3f1940 msgr2=0x557c0c3f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:23.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.161+0000 7fd75affd640 1 --2- 172.21.15.139:0/4150265523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3f1940 0x557c0c3f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:23.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.161+0000 7fd75affd640 1 -- 172.21.15.139:0/4150265523 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c0c3de260 con 0x557c0c3e7080 2024-09-16T01:56:23.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.161+0000 7fd75affd640 1 --2- 172.21.15.139:0/4150265523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 0x557c0c3f2ef0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00ace0 tx=0x7fd74c02bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c62b7dc789f2cdee server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:23.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.164+0000 7fd73affd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:56:23.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.164+0000 7fd73affd640 1 -- 172.21.15.139:0/4150265523 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c049020 con 0x557c0c3e7080 2024-09-16T01:56:23.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.164+0000 7fd73affd640 1 -- 172.21.15.139:0/4150265523 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7fd74c008d70 con 0x557c0c3e7080 2024-09-16T01:56:23.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.164+0000 7fd76435f740 1 -- 172.21.15.139:0/4150265523 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7540193e0 con 0x557c0c3e7080 2024-09-16T01:56:23.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.165+0000 7fd73affd640 1 -- 172.21.15.139:0/4150265523 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fd74c00adf0 con 0x557c0c3e7080 2024-09-16T01:56:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.170+0000 7fd76435f740 10 client.26582 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.170+0000 7fd76435f740 1 -- 172.21.15.139:0/4150265523 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x557c0c4001a0 con 0x557c0c3e7080 2024-09-16T01:56:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.172+0000 7fd73affd640 1 -- 172.21.15.139:0/4150265523 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd74c005730 con 0x557c0c3e7080 2024-09-16T01:56:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.172+0000 7fd73affd640 1 -- 172.21.15.139:0/4150265523 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7fd74c044070 con 0x557c0c3e7080 2024-09-16T01:56:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.172+0000 7fd76435f740 10 client.26582 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.172+0000 7fd76435f740 10 client.26582 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:56:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.172+0000 7fd76435f740 20 client.26582 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.172+0000 7fd76435f740 1 --2- 172.21.15.139:0/4150265523 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557c0c46cba0 0x557c0c46ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.172+0000 7fd76435f740 4 client.26582 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}] 2024-09-16T01:56:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.172+0000 7fd76435f740 1 -- 172.21.15.139:0/4150265523 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}) -- 0x7fd7540193e0 con 0x557c0c46cba0 2024-09-16T01:56:23.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.174+0000 7fd75b7fe640 1 --2- 172.21.15.139:0/4150265523 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557c0c46cba0 0x557c0c46ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:23.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.184+0000 7fd75b7fe640 1 --2- 172.21.15.139:0/4150265523 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557c0c46cba0 0x557c0c46ef90 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd1951ad1e23caf server_cookie=a1cbb3b601dac7c3 in_seq=0 out_seq=0 2024-09-16T01:56:23.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:23.185+0000 7fd73affd640 10 client.26582 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:23.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:22 smithi164 ceph-mon[35121]: pgmap v506: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 69 KiB/s wr, 147 op/s 2024-09-16T01:56:23.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:22 smithi176 ceph-mon[35324]: pgmap v506: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 69 KiB/s wr, 147 op/s 2024-09-16T01:56:24.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:56:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:56:24.241+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:56:24.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:56:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:56:24.241+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:56:25.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:25 smithi139 ceph-mon[31175]: pgmap v507: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 340 KiB/s wr, 159 op/s 2024-09-16T01:56:25.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:25 smithi176 ceph-mon[35324]: pgmap v507: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 340 KiB/s wr, 159 op/s 2024-09-16T01:56:25.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:25 smithi164 ceph-mon[35121]: pgmap v507: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 340 KiB/s wr, 159 op/s 2024-09-16T01:56:27.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.049+0000 7fd73affd640 1 -- 172.21.15.139:0/4150265523 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fd7540193e0 con 0x557c0c46cba0 2024-09-16T01:56:27.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.049+0000 7fd73affd640 10 client.26582 handle_command_reply: tid=0 2024-09-16T01:56:27.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.049+0000 7fd76435f740 1 client.26582 shutdown 2024-09-16T01:56:27.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.049+0000 7fd76435f740 1 -- 172.21.15.139:0/4150265523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 msgr2=0x557c0c3f2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:27.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.049+0000 7fd76435f740 1 --2- 172.21.15.139:0/4150265523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 0x557c0c3f2ef0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00ace0 tx=0x7fd74c02bd60 comp rx=0 tx=0).stop 2024-09-16T01:56:27.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.050+0000 7fd76435f740 1 -- 172.21.15.139:0/4150265523 shutdown_connections 2024-09-16T01:56:27.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.050+0000 7fd76435f740 1 --2- 172.21.15.139:0/4150265523 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557c0c46cba0 0x557c0c46ef90 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:27.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.050+0000 7fd76435f740 1 --2- 172.21.15.139:0/4150265523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557c0c3f1940 0x557c0c3f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:27.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.050+0000 7fd73affd640 0 client.26582 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:27.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.050+0000 7fd76435f740 1 --2- 172.21.15.139:0/4150265523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0c3e7080 0x557c0c3f2ef0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:27.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.050+0000 7fd76435f740 1 --2- 172.21.15.139:0/4150265523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c0c3e6760 0x557c0c3f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:27.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.050+0000 7fd76435f740 1 -- 172.21.15.139:0/4150265523 >> 172.21.15.139:0/4150265523 conn(0x557c0c3e2cf0 msgr2=0x557c0c3e1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:27.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.050+0000 7fd76435f740 1 -- 172.21.15.139:0/4150265523 shutdown_connections 2024-09-16T01:56:27.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.050+0000 7fd76435f740 1 -- 172.21.15.139:0/4150265523 wait complete. 2024-09-16T01:56:27.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.050+0000 7fd76435f740 20 client.26582 trim_cache size 0 max 16384 2024-09-16T01:56:27.053 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:56:27.053 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 71, 2024-09-16T01:56:27.053 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:56:27.054 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 58, 2024-09-16T01:56:27.054 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:56:27.054 INFO:teuthology.orchestra.run.smithi139.stdout: "cf44c288": { 2024-09-16T01:56:27.054 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 58, 2024-09-16T01:56:27.054 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:56:27.054 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:56:27.054 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:56:27.054 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:56:27.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:56:27.054 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:56:27.054 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 137.7, 2024-09-16T01:56:27.054 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:56:27.054 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:56:27.054 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:56:27.055 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:56:27.055 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:56:27.055 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:56:27.055 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:27.055 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:27.055 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:27.055 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:27.055 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:27.055 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:56:27.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.051+0000 7fd761f86640 1 -- 172.21.15.139:0/2808381626 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd7440a3520 msgr2=0x7fd7440a59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:27.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.051+0000 7fd761f86640 1 --2- 172.21.15.139:0/2808381626 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd7440a3520 0x7fd7440a59e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd754009fd0 tx=0x7fd754004a40 comp rx=0 tx=0).stop 2024-09-16T01:56:27.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.051+0000 7fd761f86640 1 -- 172.21.15.139:0/2808381626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd75c10d9b0 msgr2=0x7fd75c11e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:27.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.051+0000 7fd761f86640 1 --2- 172.21.15.139:0/2808381626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd75c10d9b0 0x7fd75c11e9f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd75000c970 tx=0x7fd75000ce30 comp rx=0 tx=0).stop 2024-09-16T01:56:27.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.051+0000 7fd761f86640 1 -- 172.21.15.139:0/2808381626 shutdown_connections 2024-09-16T01:56:27.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.051+0000 7fd761f86640 1 --2- 172.21.15.139:0/2808381626 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd7440a3520 0x7fd7440a59e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:27.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.051+0000 7fd761f86640 1 --2- 172.21.15.139:0/2808381626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd75c118fb0 0x7fd75c119460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:27.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.051+0000 7fd761f86640 1 --2- 172.21.15.139:0/2808381626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd75c10e580 0x7fd75c118a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:27.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.051+0000 7fd761f86640 1 --2- 172.21.15.139:0/2808381626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd75c10d9b0 0x7fd75c11e9f0 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd75000c970 tx=0x7fd75000ce30 comp rx=0 tx=0).stop 2024-09-16T01:56:27.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.051+0000 7fd761f86640 1 -- 172.21.15.139:0/2808381626 >> 172.21.15.139:0/2808381626 conn(0x7fd75c06e140 msgr2=0x7fd75c0726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:27.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.052+0000 7fd761f86640 1 -- 172.21.15.139:0/2808381626 shutdown_connections 2024-09-16T01:56:27.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:27.052+0000 7fd761f86640 1 -- 172.21.15.139:0/2808381626 wait complete. 2024-09-16T01:56:27.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:27 smithi139 ceph-mon[31175]: pgmap v508: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 433 KiB/s wr, 146 op/s 2024-09-16T01:56:27.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:27 smithi176 ceph-mon[35324]: pgmap v508: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 433 KiB/s wr, 146 op/s 2024-09-16T01:56:27.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:27 smithi164 ceph-mon[35121]: pgmap v508: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 433 KiB/s wr, 146 op/s 2024-09-16T01:56:27.835 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cf44c288', quiesce took 9.1 seconds. Will release after: 8.6 2024-09-16T01:56:29.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:29 smithi139 ceph-mon[31175]: pgmap v509: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 437 KiB/s wr, 155 op/s 2024-09-16T01:56:29.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:29 smithi176 ceph-mon[35324]: pgmap v509: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 437 KiB/s wr, 155 op/s 2024-09-16T01:56:29.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:29 smithi164 ceph-mon[35121]: pgmap v509: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 437 KiB/s wr, 155 op/s 2024-09-16T01:56:31.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:31 smithi139 ceph-mon[31175]: pgmap v510: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 554 KiB/s wr, 143 op/s 2024-09-16T01:56:31.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:31 smithi176 ceph-mon[35324]: pgmap v510: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 554 KiB/s wr, 143 op/s 2024-09-16T01:56:31.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:31 smithi164 ceph-mon[35121]: pgmap v510: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 554 KiB/s wr, 143 op/s 2024-09-16T01:56:33.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:33 smithi139 ceph-mon[31175]: pgmap v511: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 501 KiB/s wr, 162 op/s 2024-09-16T01:56:33.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:33 smithi176 ceph-mon[35324]: pgmap v511: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 501 KiB/s wr, 162 op/s 2024-09-16T01:56:33.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:33 smithi164 ceph-mon[35121]: pgmap v511: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 501 KiB/s wr, 162 op/s 2024-09-16T01:56:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.742+0000 7f2c6f2dd640 1 -- 172.21.15.139:0/2364702220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c68101ac0 msgr2=0x7f2c68101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.742+0000 7f2c6f2dd640 1 --2- 172.21.15.139:0/2364702220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c68101ac0 0x7f2c68101f20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009860 tx=0x7f2c5802a360 comp rx=0 tx=0).stop 2024-09-16T01:56:33.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.749+0000 7f2c6f2dd640 1 -- 172.21.15.139:0/2364702220 shutdown_connections 2024-09-16T01:56:33.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.749+0000 7f2c6f2dd640 1 --2- 172.21.15.139:0/2364702220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c68102460 0x7f2c6810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.749+0000 7f2c6f2dd640 1 --2- 172.21.15.139:0/2364702220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c68101ac0 0x7f2c68101f20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.749+0000 7f2c6f2dd640 1 --2- 172.21.15.139:0/2364702220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c68107ad0 0x7f2c68107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.749+0000 7f2c6f2dd640 1 -- 172.21.15.139:0/2364702220 >> 172.21.15.139:0/2364702220 conn(0x7f2c680fd820 msgr2=0x7f2c680ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.753+0000 7f2c6f2dd640 1 -- 172.21.15.139:0/2364702220 shutdown_connections 2024-09-16T01:56:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.755+0000 7f2c6f2dd640 1 -- 172.21.15.139:0/2364702220 wait complete. 2024-09-16T01:56:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.756+0000 7f2c6f2dd640 1 Processor -- start 2024-09-16T01:56:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.756+0000 7f2c6f2dd640 1 -- start start 2024-09-16T01:56:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.756+0000 7f2c6f2dd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c68101ac0 0x7f2c68117230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.756+0000 7f2c6f2dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c68102460 0x7f2c68117770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.757+0000 7f2c6f2dd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c68107ad0 0x7f2c68111380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.757+0000 7f2c6f2dd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6810fd50 con 0x7f2c68102460 2024-09-16T01:56:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.757+0000 7f2c6f2dd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6810fbd0 con 0x7f2c68107ad0 2024-09-16T01:56:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.757+0000 7f2c6f2dd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6810fed0 con 0x7f2c68101ac0 2024-09-16T01:56:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.757+0000 7f2c6d87a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c68107ad0 0x7f2c68111380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.757+0000 7f2c6d87a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c68107ad0 0x7f2c68111380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:54012/0 (socket says 172.21.15.139:54012) 2024-09-16T01:56:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.757+0000 7f2c6d87a640 1 -- 172.21.15.139:0/2850139657 learned_addr learned my addr 172.21.15.139:0/2850139657 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.759+0000 7f2c6d079640 1 --2- 172.21.15.139:0/2850139657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c68101ac0 0x7f2c68117230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.762+0000 7f2c6d87a640 1 -- 172.21.15.139:0/2850139657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c68101ac0 msgr2=0x7f2c68117230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.762+0000 7f2c6d87a640 1 --2- 172.21.15.139:0/2850139657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c68101ac0 0x7f2c68117230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.762+0000 7f2c6d87a640 1 -- 172.21.15.139:0/2850139657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c68102460 msgr2=0x7f2c68117770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.762+0000 7f2c6d87a640 1 --2- 172.21.15.139:0/2850139657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c68102460 0x7f2c68117770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.762+0000 7f2c6d87a640 1 -- 172.21.15.139:0/2850139657 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c68111c40 con 0x7f2c68107ad0 2024-09-16T01:56:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.764+0000 7f2c6d87a640 1 --2- 172.21.15.139:0/2850139657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c68107ad0 0x7f2c68111380 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000cad0 tx=0x7f2c6000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.764+0000 7f2c5e7fc640 1 -- 172.21.15.139:0/2850139657 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000ae20 con 0x7f2c68107ad0 2024-09-16T01:56:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.764+0000 7f2c6f2dd640 1 -- 172.21.15.139:0/2850139657 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c68111f30 con 0x7f2c68107ad0 2024-09-16T01:56:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.764+0000 7f2c6f2dd640 1 -- 172.21.15.139:0/2850139657 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c68071e20 con 0x7f2c68107ad0 2024-09-16T01:56:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.765+0000 7f2c5e7fc640 1 -- 172.21.15.139:0/2850139657 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c60005020 con 0x7f2c68107ad0 2024-09-16T01:56:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.765+0000 7f2c5e7fc640 1 -- 172.21.15.139:0/2850139657 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000f6f0 con 0x7f2c68107ad0 2024-09-16T01:56:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.771+0000 7f2c7207c740 1 Processor -- start 2024-09-16T01:56:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.773+0000 7f2c5e7fc640 1 -- 172.21.15.139:0/2850139657 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f2c60004a40 con 0x7f2c68107ad0 2024-09-16T01:56:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.778+0000 7f2c5e7fc640 1 --2- 172.21.15.139:0/2850139657 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2c540a35d0 0x7f2c540a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.778+0000 7f2c5e7fc640 1 -- 172.21.15.139:0/2850139657 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7f2c600d00f0 con 0x7f2c68107ad0 2024-09-16T01:56:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.780+0000 7f2c7207c740 1 -- start start 2024-09-16T01:56:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.781+0000 7f2c7207c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737bdaf0 0x55ce737bded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.781+0000 7f2c7207c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737be410 0x55ce737c8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.781+0000 7f2c7207c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737c8cb0 0x55ce737cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.781+0000 7f2c7207c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ce737222f0 con 0x55ce737c8cb0 2024-09-16T01:56:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.781+0000 7f2c7207c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ce736ca270 con 0x55ce737bdaf0 2024-09-16T01:56:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.781+0000 7f2c7207c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ce7371e2b0 con 0x55ce737be410 2024-09-16T01:56:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.782+0000 7f2c6c878640 1 --2- 172.21.15.139:0/2850139657 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2c540a35d0 0x7f2c540a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.783+0000 7f2c6c878640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737be410 0x55ce737c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.783+0000 7f2c6c878640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737be410 0x55ce737c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:45338/0 (socket says 172.21.15.139:45338) 2024-09-16T01:56:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.783+0000 7f2c6c878640 1 -- 172.21.15.139:0/1756320070 learned_addr learned my addr 172.21.15.139:0/1756320070 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.785+0000 7f2c6c878640 1 -- 172.21.15.139:0/1756320070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737bdaf0 msgr2=0x55ce737bded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.785+0000 7f2c6c878640 1 --2- 172.21.15.139:0/1756320070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737bdaf0 0x55ce737bded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.785+0000 7f2c6c878640 1 -- 172.21.15.139:0/1756320070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737c8cb0 msgr2=0x55ce737cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.785+0000 7f2c6c878640 1 --2- 172.21.15.139:0/1756320070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737c8cb0 0x55ce737cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.785+0000 7f2c6c878640 1 -- 172.21.15.139:0/1756320070 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce736c9340 con 0x55ce737be410 2024-09-16T01:56:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.785+0000 7f2c6c878640 1 --2- 172.21.15.139:0/1756320070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737be410 0x55ce737c8770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2c580041b0 tx=0x7f2c580041e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feb973b807d6b441 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.786+0000 7f2c6c878640 1 --2- 172.21.15.139:0/2850139657 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2c540a35d0 0x7f2c540a5a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2c58004250 tx=0x7f2c5803a000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.794+0000 7f2c537fe640 1 -- 172.21.15.139:0/1756320070 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58045070 con 0x55ce737be410 2024-09-16T01:56:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.794+0000 7f2c537fe640 1 -- 172.21.15.139:0/1756320070 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c58003cd0 con 0x55ce737be410 2024-09-16T01:56:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.794+0000 7f2c537fe640 1 -- 172.21.15.139:0/1756320070 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c580328e0 con 0x55ce737be410 2024-09-16T01:56:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.795+0000 7f2c7207c740 1 -- 172.21.15.139:0/1756320070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737be410 msgr2=0x55ce737c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.795+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1756320070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737be410 0x55ce737c8770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2c580041b0 tx=0x7f2c580041e0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.800+0000 7f2c7207c740 1 -- 172.21.15.139:0/1756320070 shutdown_connections 2024-09-16T01:56:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.800+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1756320070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737c8cb0 0x55ce737cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.800+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1756320070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737be410 0x55ce737c8770 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.800+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1756320070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737bdaf0 0x55ce737bded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.800+0000 7f2c7207c740 1 -- 172.21.15.139:0/1756320070 >> 172.21.15.139:0/1756320070 conn(0x55ce737ba060 msgr2=0x55ce737ba440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.801+0000 7f2c7207c740 1 -- 172.21.15.139:0/1756320070 shutdown_connections 2024-09-16T01:56:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.807+0000 7f2c7207c740 1 -- 172.21.15.139:0/1756320070 wait complete. 2024-09-16T01:56:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.807+0000 7f2c7207c740 1 -- start start 2024-09-16T01:56:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.807+0000 7f2c7207c740 1 Processor -- start 2024-09-16T01:56:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.815+0000 7f2c7207c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737bdaf0 0x55ce737c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.815+0000 7f2c7207c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737be410 0x55ce737c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.815+0000 7f2c7207c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737c8cb0 0x55ce737c6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.815+0000 7f2c7207c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ce737cb820 con 0x55ce737bdaf0 2024-09-16T01:56:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.815+0000 7f2c7207c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ce736ab290 con 0x55ce737be410 2024-09-16T01:56:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.815+0000 7f2c7207c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ce737cb9a0 con 0x55ce737c8cb0 2024-09-16T01:56:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.815+0000 7f2c7207c740 10 client.0 _ll_register_callbacks cb 0x55ce736dc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.819+0000 7f2c6d87a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737c8cb0 0x55ce737c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.819+0000 7f2c6d079640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737bdaf0 0x55ce737c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.819+0000 7f2c6d079640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737bdaf0 0x55ce737c5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48040/0 (socket says 172.21.15.139:48040) 2024-09-16T01:56:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.819+0000 7f2c6d079640 1 -- 172.21.15.139:0/1844135841 learned_addr learned my addr 172.21.15.139:0/1844135841 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.820+0000 7f2c6c878640 1 --2- 172.21.15.139:0/1844135841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737be410 0x55ce737c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.824+0000 7f2c6d079640 1 -- 172.21.15.139:0/1844135841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737c8cb0 msgr2=0x55ce737c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.824+0000 7f2c6d079640 1 --2- 172.21.15.139:0/1844135841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737c8cb0 0x55ce737c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.824+0000 7f2c6d079640 1 -- 172.21.15.139:0/1844135841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737be410 msgr2=0x55ce737c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.824+0000 7f2c6d079640 1 --2- 172.21.15.139:0/1844135841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737be410 0x55ce737c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.824+0000 7f2c6d079640 1 -- 172.21.15.139:0/1844135841 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce73435210 con 0x55ce737bdaf0 2024-09-16T01:56:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.826+0000 7f2c6c878640 1 --2- 172.21.15.139:0/1844135841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737be410 0x55ce737c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:56:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.832+0000 7f2c6d079640 1 --2- 172.21.15.139:0/1844135841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737bdaf0 0x55ce737c5c90 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400b940 tx=0x7f2c6400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b45fed5f3b4d8d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.832+0000 7f2c50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:56:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.834+0000 7f2c50ff9640 1 -- 172.21.15.139:0/1844135841 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6401a470 con 0x55ce737bdaf0 2024-09-16T01:56:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.834+0000 7f2c7207c740 1 -- 172.21.15.139:0/1844135841 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c58003cd0 con 0x55ce737bdaf0 2024-09-16T01:56:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.834+0000 7f2c50ff9640 1 -- 172.21.15.139:0/1844135841 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7f2c6401a610 con 0x55ce737bdaf0 2024-09-16T01:56:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.836+0000 7f2c50ff9640 1 -- 172.21.15.139:0/1844135841 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f2c64023050 con 0x55ce737bdaf0 2024-09-16T01:56:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.837+0000 7f2c7207c740 10 client.17154 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.837+0000 7f2c7207c740 1 -- 172.21.15.139:0/1844135841 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55ce73501620 con 0x55ce737bdaf0 2024-09-16T01:56:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.841+0000 7f2c50ff9640 1 -- 172.21.15.139:0/1844135841 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c58003cd0 con 0x55ce737bdaf0 2024-09-16T01:56:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.841+0000 7f2c50ff9640 1 -- 172.21.15.139:0/1844135841 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f2c64011b50 con 0x55ce737bdaf0 2024-09-16T01:56:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.843+0000 7f2c7207c740 10 client.17154 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.843+0000 7f2c7207c740 10 client.17154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:56:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.843+0000 7f2c7207c740 20 client.17154 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.843+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1844135841 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ce737d8d50 0x55ce737db190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.843+0000 7f2c7207c740 4 client.17154 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:56:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.843+0000 7f2c7207c740 1 -- 172.21.15.139:0/1844135841 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce7372fa20 con 0x55ce737d8d50 2024-09-16T01:56:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.843+0000 7f2c6c878640 1 --2- 172.21.15.139:0/1844135841 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ce737d8d50 0x55ce737db190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.851+0000 7f2c6c878640 1 --2- 172.21.15.139:0/1844135841 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ce737d8d50 0x55ce737db190 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63ac6dd20c0be330 server_cookie=f6a34325d691aa1c in_seq=0 out_seq=0 2024-09-16T01:56:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.854+0000 7f2c50ff9640 10 client.17154 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.854+0000 7f2c50ff9640 1 -- 172.21.15.139:0/1844135841 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce7372fa20 con 0x55ce737d8d50 2024-09-16T01:56:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.854+0000 7f2c50ff9640 10 client.17154 handle_command_reply: tid=0 2024-09-16T01:56:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.854+0000 7f2c7207c740 1 client.17154 shutdown 2024-09-16T01:56:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.856+0000 7f2c7207c740 1 -- 172.21.15.139:0/1844135841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737bdaf0 msgr2=0x55ce737c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.856+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1844135841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737bdaf0 0x55ce737c5c90 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400b940 tx=0x7f2c6400be00 comp rx=0 tx=0).stop 2024-09-16T01:56:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.859+0000 7f2c7207c740 1 -- 172.21.15.139:0/1844135841 shutdown_connections 2024-09-16T01:56:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.859+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1844135841 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ce737d8d50 0x55ce737db190 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.859+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1844135841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737c8cb0 0x55ce737c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.859+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1844135841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737be410 0x55ce737c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.859+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1844135841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737bdaf0 0x55ce737c5c90 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.859+0000 7f2c7207c740 1 -- 172.21.15.139:0/1844135841 >> 172.21.15.139:0/1844135841 conn(0x55ce737ba060 msgr2=0x55ce737bfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.861+0000 7f2c50ff9640 0 client.17154 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.864+0000 7f2c7207c740 1 -- 172.21.15.139:0/1844135841 shutdown_connections 2024-09-16T01:56:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.867+0000 7f2c7207c740 1 -- 172.21.15.139:0/1844135841 wait complete. 2024-09-16T01:56:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.869+0000 7f2c7207c740 20 client.17154 trim_cache size 0 max 16384 2024-09-16T01:56:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.897+0000 7f2c7207c740 1 Processor -- start 2024-09-16T01:56:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.899+0000 7f2c7207c740 1 -- start start 2024-09-16T01:56:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.900+0000 7f2c7207c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737bdaf0 0x55ce737c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.900+0000 7f2c7207c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737be410 0x55ce737e0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.900+0000 7f2c7207c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737c8cb0 0x55ce737e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.900+0000 7f2c7207c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c0102e0 con 0x55ce737c8cb0 2024-09-16T01:56:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.900+0000 7f2c7207c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c012120 con 0x55ce737bdaf0 2024-09-16T01:56:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.900+0000 7f2c7207c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c010a70 con 0x55ce737be410 2024-09-16T01:56:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.900+0000 7f2c6c878640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737be410 0x55ce737e0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.900+0000 7f2c6c878640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737be410 0x55ce737e0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:45358/0 (socket says 172.21.15.139:45358) 2024-09-16T01:56:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.900+0000 7f2c6c878640 1 -- 172.21.15.139:0/2192529946 learned_addr learned my addr 172.21.15.139:0/2192529946 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.902+0000 7f2c6d079640 1 --2- 172.21.15.139:0/2192529946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737bdaf0 0x55ce737c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.904+0000 7f2c6d079640 1 -- 172.21.15.139:0/2192529946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737be410 msgr2=0x55ce737e0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.904+0000 7f2c6d079640 1 --2- 172.21.15.139:0/2192529946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737be410 0x55ce737e0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.904+0000 7f2c6d079640 1 -- 172.21.15.139:0/2192529946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737c8cb0 msgr2=0x55ce737e0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.904+0000 7f2c6d079640 1 --2- 172.21.15.139:0/2192529946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737c8cb0 0x55ce737e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.904+0000 7f2c6d079640 1 -- 172.21.15.139:0/2192529946 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce737bd920 con 0x55ce737bdaf0 2024-09-16T01:56:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.906+0000 7f2c6d079640 1 --2- 172.21.15.139:0/2192529946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737bdaf0 0x55ce737c6540 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400b940 tx=0x7f2c64005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71098dde905d9a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.906+0000 7f2c52ffd640 1 -- 172.21.15.139:0/2192529946 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64005c50 con 0x55ce737bdaf0 2024-09-16T01:56:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.906+0000 7f2c52ffd640 1 -- 172.21.15.139:0/2192529946 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c6402c070 con 0x55ce737bdaf0 2024-09-16T01:56:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.907+0000 7f2c7207c740 1 -- 172.21.15.139:0/2192529946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737bdaf0 msgr2=0x55ce737c6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.907+0000 7f2c7207c740 1 --2- 172.21.15.139:0/2192529946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737bdaf0 0x55ce737c6540 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400b940 tx=0x7f2c64005850 comp rx=0 tx=0).stop 2024-09-16T01:56:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.914+0000 7f2c7207c740 1 -- 172.21.15.139:0/2192529946 shutdown_connections 2024-09-16T01:56:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.914+0000 7f2c7207c740 1 --2- 172.21.15.139:0/2192529946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737c8cb0 0x55ce737e0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.914+0000 7f2c7207c740 1 --2- 172.21.15.139:0/2192529946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737be410 0x55ce737e0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.914+0000 7f2c7207c740 1 --2- 172.21.15.139:0/2192529946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737bdaf0 0x55ce737c6540 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.914+0000 7f2c7207c740 1 -- 172.21.15.139:0/2192529946 >> 172.21.15.139:0/2192529946 conn(0x55ce737ba060 msgr2=0x55ce737c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.917+0000 7f2c7207c740 1 -- 172.21.15.139:0/2192529946 shutdown_connections 2024-09-16T01:56:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.921+0000 7f2c7207c740 1 -- 172.21.15.139:0/2192529946 wait complete. 2024-09-16T01:56:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.922+0000 7f2c7207c740 1 -- start start 2024-09-16T01:56:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.922+0000 7f2c7207c740 1 Processor -- start 2024-09-16T01:56:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.926+0000 7f2c7207c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737bdaf0 0x55ce737d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.926+0000 7f2c7207c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737be410 0x55ce737cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.926+0000 7f2c7207c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737c8cb0 0x55ce737ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.926+0000 7f2c7207c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c011990 con 0x55ce737be410 2024-09-16T01:56:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.926+0000 7f2c7207c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c011200 con 0x55ce737bdaf0 2024-09-16T01:56:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.926+0000 7f2c7207c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c00e2c0 con 0x55ce737c8cb0 2024-09-16T01:56:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.926+0000 7f2c7207c740 10 client.0 _ll_register_callbacks cb 0x7f2c64004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.926+0000 7f2c6c878640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737be410 0x55ce737cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.926+0000 7f2c6c878640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737be410 0x55ce737cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48052/0 (socket says 172.21.15.139:48052) 2024-09-16T01:56:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.926+0000 7f2c6c878640 1 -- 172.21.15.139:0/1484156647 learned_addr learned my addr 172.21.15.139:0/1484156647 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.927+0000 7f2c6c878640 1 -- 172.21.15.139:0/1484156647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737c8cb0 msgr2=0x55ce737ccae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:56:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.927+0000 7f2c6c878640 1 --2- 172.21.15.139:0/1484156647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737c8cb0 0x55ce737ccae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.927+0000 7f2c6c878640 1 -- 172.21.15.139:0/1484156647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737bdaf0 msgr2=0x55ce737d1280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:56:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.927+0000 7f2c6c878640 1 --2- 172.21.15.139:0/1484156647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737bdaf0 0x55ce737d1280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.927+0000 7f2c6c878640 1 -- 172.21.15.139:0/1484156647 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce737c6a80 con 0x55ce737be410 2024-09-16T01:56:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.927+0000 7f2c6c878640 1 --2- 172.21.15.139:0/1484156647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737be410 0x55ce737cc5a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f2c58032460 tx=0x7f2c58030de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b57a09c3eada69 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.927+0000 7f2c50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:56:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.928+0000 7f2c50ff9640 1 -- 172.21.15.139:0/1484156647 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5805e060 con 0x55ce737be410 2024-09-16T01:56:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.928+0000 7f2c50ff9640 1 -- 172.21.15.139:0/1484156647 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7f2c58003b00 con 0x55ce737be410 2024-09-16T01:56:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.929+0000 7f2c7207c740 1 -- 172.21.15.139:0/1484156647 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c6402c070 con 0x55ce737be410 2024-09-16T01:56:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.931+0000 7f2c50ff9640 1 -- 172.21.15.139:0/1484156647 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f2c58045040 con 0x55ce737be410 2024-09-16T01:56:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.934+0000 7f2c7207c740 10 client.17157 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.934+0000 7f2c7207c740 1 -- 172.21.15.139:0/1484156647 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55ce737bb1d0 con 0x55ce737be410 2024-09-16T01:56:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.940+0000 7f2c50ff9640 1 -- 172.21.15.139:0/1484156647 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c6402c070 con 0x55ce737be410 2024-09-16T01:56:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.940+0000 7f2c50ff9640 1 -- 172.21.15.139:0/1484156647 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f2c58049410 con 0x55ce737be410 2024-09-16T01:56:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.940+0000 7f2c7207c740 10 client.17157 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.940+0000 7f2c7207c740 10 client.17157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:56:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.940+0000 7f2c7207c740 20 client.17157 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.940+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1484156647 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ce73843940 0x55ce73845d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.940+0000 7f2c7207c740 4 client.17157 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:56:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.940+0000 7f2c7207c740 1 -- 172.21.15.139:0/1484156647 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce73729d70 con 0x55ce73843940 2024-09-16T01:56:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.942+0000 7f2c6d079640 1 --2- 172.21.15.139:0/1484156647 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ce73843940 0x55ce73845d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.945+0000 7f2c6d079640 1 --2- 172.21.15.139:0/1484156647 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ce73843940 0x55ce73845d60 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dacce6d0196d931 server_cookie=cffa06b28fb6aceb in_seq=0 out_seq=0 2024-09-16T01:56:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:33.946+0000 7f2c50ff9640 10 client.17157 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.103+0000 7f2c50ff9640 1 -- 172.21.15.139:0/1484156647 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ce73729d70 con 0x55ce73843940 2024-09-16T01:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.103+0000 7f2c50ff9640 10 client.17157 handle_command_reply: tid=0 2024-09-16T01:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.103+0000 7f2c7207c740 1 client.17157 shutdown 2024-09-16T01:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.103+0000 7f2c7207c740 1 -- 172.21.15.139:0/1484156647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737be410 msgr2=0x55ce737cc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.103+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1484156647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737be410 0x55ce737cc5a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f2c58032460 tx=0x7f2c58030de0 comp rx=0 tx=0).stop 2024-09-16T01:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.103+0000 7f2c7207c740 1 -- 172.21.15.139:0/1484156647 shutdown_connections 2024-09-16T01:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.103+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1484156647 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55ce73843940 0x55ce73845d60 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.103+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1484156647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ce737c8cb0 0x55ce737ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.103+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1484156647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ce737be410 0x55ce737cc5a0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.103+0000 7f2c7207c740 1 --2- 172.21.15.139:0/1484156647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce737bdaf0 0x55ce737d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.103+0000 7f2c7207c740 1 -- 172.21.15.139:0/1484156647 >> 172.21.15.139:0/1484156647 conn(0x55ce737ba060 msgr2=0x55ce737bf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.103+0000 7f2c50ff9640 0 client.17157 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.103+0000 7f2c7207c740 1 -- 172.21.15.139:0/1484156647 shutdown_connections 2024-09-16T01:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.104+0000 7f2c7207c740 1 -- 172.21.15.139:0/1484156647 wait complete. 2024-09-16T01:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.104+0000 7f2c7207c740 20 client.17157 trim_cache size 0 max 16384 2024-09-16T01:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:56:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.104+0000 7f2c6f2dd640 1 -- 172.21.15.139:0/2850139657 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2c540a35d0 msgr2=0x7f2c540a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.104+0000 7f2c6f2dd640 1 --2- 172.21.15.139:0/2850139657 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2c540a35d0 0x7f2c540a5a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2c58004250 tx=0x7f2c5803a000 comp rx=0 tx=0).stop 2024-09-16T01:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.104+0000 7f2c6f2dd640 1 -- 172.21.15.139:0/2850139657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c68107ad0 msgr2=0x7f2c68111380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.104+0000 7f2c6f2dd640 1 --2- 172.21.15.139:0/2850139657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c68107ad0 0x7f2c68111380 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000cad0 tx=0x7f2c6000cf90 comp rx=0 tx=0).stop 2024-09-16T01:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.104+0000 7f2c6f2dd640 1 -- 172.21.15.139:0/2850139657 shutdown_connections 2024-09-16T01:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.104+0000 7f2c6f2dd640 1 --2- 172.21.15.139:0/2850139657 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f2c540a35d0 0x7f2c540a5a90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.104+0000 7f2c6f2dd640 1 --2- 172.21.15.139:0/2850139657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c68107ad0 0x7f2c68111380 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.104+0000 7f2c6f2dd640 1 --2- 172.21.15.139:0/2850139657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c68102460 0x7f2c68117770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.104+0000 7f2c6f2dd640 1 --2- 172.21.15.139:0/2850139657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c68101ac0 0x7f2c68117230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.104+0000 7f2c6f2dd640 1 -- 172.21.15.139:0/2850139657 >> 172.21.15.139:0/2850139657 conn(0x7f2c680fd820 msgr2=0x7f2c680fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.105+0000 7f2c6f2dd640 1 -- 172.21.15.139:0/2850139657 shutdown_connections 2024-09-16T01:56:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:34.105+0000 7f2c6f2dd640 1 -- 172.21.15.139:0/2850139657 wait complete. 2024-09-16T01:56:35.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:35 smithi139 ceph-mon[31175]: pgmap v512: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 498 KiB/s wr, 165 op/s 2024-09-16T01:56:35.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:35 smithi176 ceph-mon[35324]: pgmap v512: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 498 KiB/s wr, 165 op/s 2024-09-16T01:56:35.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:35 smithi164 ceph-mon[35121]: pgmap v512: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 498 KiB/s wr, 165 op/s 2024-09-16T01:56:36.436 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:56:36.966 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:56:37.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:37 smithi139 ceph-mon[31175]: pgmap v513: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 228 KiB/s wr, 155 op/s 2024-09-16T01:56:37.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:37 smithi176 ceph-mon[35324]: pgmap v513: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 228 KiB/s wr, 155 op/s 2024-09-16T01:56:37.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:37 smithi164 ceph-mon[35121]: pgmap v513: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 228 KiB/s wr, 155 op/s 2024-09-16T01:56:37.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.608+0000 7f883b59e640 1 -- 172.21.15.139:0/1469959006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f883c109450 msgr2=0x7f883c109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:37.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.608+0000 7f883b59e640 1 --2- 172.21.15.139:0/1469959006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f883c109450 0x7f883c109830 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f882c009930 tx=0x7f882c02a480 comp rx=0 tx=0).stop 2024-09-16T01:56:37.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.608+0000 7f883b59e640 1 -- 172.21.15.139:0/1469959006 shutdown_connections 2024-09-16T01:56:37.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.608+0000 7f883b59e640 1 --2- 172.21.15.139:0/1469959006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f883c10ea10 0x7f883c115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:37.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.608+0000 7f883b59e640 1 --2- 172.21.15.139:0/1469959006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f883c109e00 0x7f883c10e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:37.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.608+0000 7f883b59e640 1 --2- 172.21.15.139:0/1469959006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f883c109450 0x7f883c109830 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:37.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.608+0000 7f883b59e640 1 -- 172.21.15.139:0/1469959006 >> 172.21.15.139:0/1469959006 conn(0x7f883c073660 msgr2=0x7f883c073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:37.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.608+0000 7f883b59e640 1 -- 172.21.15.139:0/1469959006 shutdown_connections 2024-09-16T01:56:37.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.608+0000 7f883b59e640 1 -- 172.21.15.139:0/1469959006 wait complete. 2024-09-16T01:56:37.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.609+0000 7f883b59e640 1 Processor -- start 2024-09-16T01:56:37.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.609+0000 7f883b59e640 1 -- start start 2024-09-16T01:56:37.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.611+0000 7f883b59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f883c109450 0x7f883c1a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:37.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.611+0000 7f883b59e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f883c109e00 0x7f883c1a3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:37.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.611+0000 7f883b59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f883c10ea10 0x7f883c1a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:37.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.611+0000 7f883b59e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f883c118940 con 0x7f883c10ea10 2024-09-16T01:56:37.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.611+0000 7f883b59e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1187c0 con 0x7f883c109450 2024-09-16T01:56:37.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.611+0000 7f883b59e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f883c118ac0 con 0x7f883c109e00 2024-09-16T01:56:37.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.611+0000 7f883ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f883c10ea10 0x7f883c1a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:37.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.611+0000 7f883ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f883c10ea10 0x7f883c1a3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48076/0 (socket says 172.21.15.139:48076) 2024-09-16T01:56:37.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.611+0000 7f883ad9d640 1 -- 172.21.15.139:0/1688067242 learned_addr learned my addr 172.21.15.139:0/1688067242 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:37.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.612+0000 7f883ad9d640 1 -- 172.21.15.139:0/1688067242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f883c109e00 msgr2=0x7f883c1a3320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:56:37.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.612+0000 7f883ad9d640 1 --2- 172.21.15.139:0/1688067242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f883c109e00 0x7f883c1a3320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:37.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.612+0000 7f883ad9d640 1 -- 172.21.15.139:0/1688067242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f883c109450 msgr2=0x7f883c1a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:56:37.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.612+0000 7f883ad9d640 1 --2- 172.21.15.139:0/1688067242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f883c109450 0x7f883c1a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:37.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.612+0000 7f883ad9d640 1 -- 172.21.15.139:0/1688067242 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883c1a4120 con 0x7f883c10ea10 2024-09-16T01:56:37.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.612+0000 7f883ad9d640 1 --2- 172.21.15.139:0/1688067242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f883c10ea10 0x7f883c1a3860 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f883400b9e0 tx=0x7f883400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:37.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.613+0000 7f882b7fe640 1 -- 172.21.15.139:0/1688067242 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834019070 con 0x7f883c10ea10 2024-09-16T01:56:37.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.613+0000 7f882b7fe640 1 -- 172.21.15.139:0/1688067242 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8834014440 con 0x7f883c10ea10 2024-09-16T01:56:37.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.613+0000 7f883b59e640 1 -- 172.21.15.139:0/1688067242 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883c119a50 con 0x7f883c10ea10 2024-09-16T01:56:37.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.613+0000 7f882b7fe640 1 -- 172.21.15.139:0/1688067242 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88340075b0 con 0x7f883c10ea10 2024-09-16T01:56:37.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.613+0000 7f883b59e640 1 -- 172.21.15.139:0/1688067242 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883c119f90 con 0x7f883c10ea10 2024-09-16T01:56:37.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.616+0000 7f883b59e640 1 -- 172.21.15.139:0/1688067242 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f883c06eef0 con 0x7f883c10ea10 2024-09-16T01:56:37.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.617+0000 7f882b7fe640 1 -- 172.21.15.139:0/1688067242 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f883401c070 con 0x7f883c10ea10 2024-09-16T01:56:37.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.618+0000 7f882b7fe640 1 --2- 172.21.15.139:0/1688067242 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f88240a3500 0x7f88240a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:37.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.618+0000 7f882b7fe640 1 -- 172.21.15.139:0/1688067242 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9726+0+0 (secure 0 0 0) 0x7f88340cf830 con 0x7f883c10ea10 2024-09-16T01:56:37.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.623+0000 7f883a59c640 1 --2- 172.21.15.139:0/1688067242 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f88240a3500 0x7f88240a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:37.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.628+0000 7f882b7fe640 1 -- 172.21.15.139:0/1688067242 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f883409bd90 con 0x7f883c10ea10 2024-09-16T01:56:37.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:37.639+0000 7f883a59c640 1 --2- 172.21.15.139:0/1688067242 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f88240a3500 0x7f88240a59c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f882c005c20 tx=0x7f882c005fd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:38.034 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:56:38.034 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":71,"btime":"2024-09-16T01:54:19:332763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:54:19.051272+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:56:38.034 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 71 2024-09-16T01:56:38.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:38.030+0000 7f883b59e640 1 -- 172.21.15.139:0/1688067242 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f883c11a2b0 con 0x7f883c10ea10 2024-09-16T01:56:38.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:38.032+0000 7f882b7fe640 1 -- 172.21.15.139:0/1688067242 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+12031 (secure 0 0 0) 0x7f883409b690 con 0x7f883c10ea10 2024-09-16T01:56:38.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:38.036+0000 7f883b59e640 1 -- 172.21.15.139:0/1688067242 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f88240a3500 msgr2=0x7f88240a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:38.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:38.036+0000 7f883b59e640 1 --2- 172.21.15.139:0/1688067242 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f88240a3500 0x7f88240a59c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f882c005c20 tx=0x7f882c005fd0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:38.036+0000 7f883b59e640 1 -- 172.21.15.139:0/1688067242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f883c10ea10 msgr2=0x7f883c1a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:38.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:38.036+0000 7f883b59e640 1 --2- 172.21.15.139:0/1688067242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f883c10ea10 0x7f883c1a3860 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f883400b9e0 tx=0x7f883400bea0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:38.039+0000 7f883b59e640 1 -- 172.21.15.139:0/1688067242 shutdown_connections 2024-09-16T01:56:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:38.039+0000 7f883b59e640 1 --2- 172.21.15.139:0/1688067242 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f88240a3500 0x7f88240a59c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:38.039+0000 7f883b59e640 1 --2- 172.21.15.139:0/1688067242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f883c10ea10 0x7f883c1a3860 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:38.039+0000 7f883b59e640 1 --2- 172.21.15.139:0/1688067242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f883c109e00 0x7f883c1a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:38.039+0000 7f883b59e640 1 --2- 172.21.15.139:0/1688067242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f883c109450 0x7f883c1a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:38.039+0000 7f883b59e640 1 -- 172.21.15.139:0/1688067242 >> 172.21.15.139:0/1688067242 conn(0x7f883c073660 msgr2=0x7f883c10c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:38.039+0000 7f883b59e640 1 -- 172.21.15.139:0/1688067242 shutdown_connections 2024-09-16T01:56:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:38.039+0000 7f883b59e640 1 -- 172.21.15.139:0/1688067242 wait complete. 2024-09-16T01:56:38.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:38 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1688067242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:56:38.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:38 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1688067242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:56:38.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:38 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1688067242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:56:39.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:39 smithi139 ceph-mon[31175]: pgmap v514: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 178 op/s 2024-09-16T01:56:39.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:39 smithi176 ceph-mon[35324]: pgmap v514: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 178 op/s 2024-09-16T01:56:39.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:39 smithi164 ceph-mon[35121]: pgmap v514: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 178 op/s 2024-09-16T01:56:40.490 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id cf44c288 --release --await' 2024-09-16T01:56:40.491 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id cf44c288 --release --await 2024-09-16T01:56:40.995 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:56:41.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:41 smithi139 ceph-mon[31175]: pgmap v515: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 142 KiB/s wr, 161 op/s 2024-09-16T01:56:41.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:41 smithi139 ceph-mon[31175]: osdmap e111: 12 total, 12 up, 12 in 2024-09-16T01:56:41.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:41 smithi176 ceph-mon[35324]: pgmap v515: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 142 KiB/s wr, 161 op/s 2024-09-16T01:56:41.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:41 smithi176 ceph-mon[35324]: osdmap e111: 12 total, 12 up, 12 in 2024-09-16T01:56:41.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:41 smithi164 ceph-mon[35121]: pgmap v515: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 142 KiB/s wr, 161 op/s 2024-09-16T01:56:41.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:41 smithi164 ceph-mon[35121]: osdmap e111: 12 total, 12 up, 12 in 2024-09-16T01:56:41.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.680+0000 7fc81a3b0640 1 -- 172.21.15.139:0/109448939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc814110190 msgr2=0x7fc81406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:41.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.680+0000 7fc81a3b0640 1 --2- 172.21.15.139:0/109448939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc814110190 0x7fc81406e4f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009900 tx=0x7fc80c02a3f0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.680+0000 7fc81a3b0640 1 -- 172.21.15.139:0/109448939 shutdown_connections 2024-09-16T01:56:41.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.680+0000 7fc81a3b0640 1 --2- 172.21.15.139:0/109448939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc81411b140 0x7fc81411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.680+0000 7fc81a3b0640 1 --2- 172.21.15.139:0/109448939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc81406ea30 0x7fc81406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.680+0000 7fc81a3b0640 1 --2- 172.21.15.139:0/109448939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc814110190 0x7fc81406e4f0 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009900 tx=0x7fc80c02a3f0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.680+0000 7fc81a3b0640 1 -- 172.21.15.139:0/109448939 >> 172.21.15.139:0/109448939 conn(0x7fc814073180 msgr2=0x7fc814073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:41.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.681+0000 7fc81a3b0640 1 -- 172.21.15.139:0/109448939 shutdown_connections 2024-09-16T01:56:41.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.681+0000 7fc81a3b0640 1 -- 172.21.15.139:0/109448939 wait complete. 2024-09-16T01:56:41.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.682+0000 7fc81a3b0640 1 Processor -- start 2024-09-16T01:56:41.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.682+0000 7fc81a3b0640 1 -- start start 2024-09-16T01:56:41.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.682+0000 7fc81a3b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc81406ea30 0x7fc8141198b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.683+0000 7fc81a3b0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc81411b140 0x7fc814112990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.683+0000 7fc81a3b0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc814112ed0 0x7fc814113380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.683+0000 7fc81a3b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc814120c40 con 0x7fc81406ea30 2024-09-16T01:56:41.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.683+0000 7fc81a3b0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc814120ac0 con 0x7fc81411b140 2024-09-16T01:56:41.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.683+0000 7fc81a3b0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc814120dc0 con 0x7fc814112ed0 2024-09-16T01:56:41.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.683+0000 7fc819baf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc814112ed0 0x7fc814113380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.683+0000 7fc818bad640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc81411b140 0x7fc814112990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.683+0000 7fc818bad640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc81411b140 0x7fc814112990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:54104/0 (socket says 172.21.15.139:54104) 2024-09-16T01:56:41.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.683+0000 7fc818bad640 1 -- 172.21.15.139:0/2707861596 learned_addr learned my addr 172.21.15.139:0/2707861596 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:41.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.687+0000 7fc818bad640 1 -- 172.21.15.139:0/2707861596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc814112ed0 msgr2=0x7fc814113380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:41.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.687+0000 7fc818bad640 1 --2- 172.21.15.139:0/2707861596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc814112ed0 0x7fc814113380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.687+0000 7fc818bad640 1 -- 172.21.15.139:0/2707861596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc81406ea30 msgr2=0x7fc8141198b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:41.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.687+0000 7fc818bad640 1 --2- 172.21.15.139:0/2707861596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc81406ea30 0x7fc8141198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.687+0000 7fc818bad640 1 -- 172.21.15.139:0/2707861596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc814117460 con 0x7fc81411b140 2024-09-16T01:56:41.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.695+0000 7fc818bad640 1 --2- 172.21.15.139:0/2707861596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc81411b140 0x7fc814112990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc80800e9f0 tx=0x7fc80800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:41.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.698+0000 7fc8127fc640 1 -- 172.21.15.139:0/2707861596 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80800cd80 con 0x7fc81411b140 2024-09-16T01:56:41.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.698+0000 7fc8127fc640 1 -- 172.21.15.139:0/2707861596 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc808010910 con 0x7fc81411b140 2024-09-16T01:56:41.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.698+0000 7fc8127fc640 1 -- 172.21.15.139:0/2707861596 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc808018980 con 0x7fc81411b140 2024-09-16T01:56:41.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.699+0000 7fc81a3b0640 1 -- 172.21.15.139:0/2707861596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc814117750 con 0x7fc81411b140 2024-09-16T01:56:41.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.699+0000 7fc81a3b0640 1 -- 172.21.15.139:0/2707861596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc814117c40 con 0x7fc81411b140 2024-09-16T01:56:41.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.703+0000 7fc81d9eb740 1 Processor -- start 2024-09-16T01:56:41.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.703+0000 7fc81d9eb740 1 -- start start 2024-09-16T01:56:41.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.703+0000 7fc81d9eb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561798865210 0x5617988655f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.703+0000 7fc81d9eb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561798865b30 0x56179886feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.703+0000 7fc81d9eb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5617988703f0 0x5617988727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.703+0000 7fc81d9eb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5617987ca620 con 0x5617988703f0 2024-09-16T01:56:41.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.703+0000 7fc81d9eb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561798771e50 con 0x561798865b30 2024-09-16T01:56:41.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.703+0000 7fc81d9eb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5617987c65e0 con 0x561798865210 2024-09-16T01:56:41.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.704+0000 7fc819baf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561798865210 0x5617988655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.705+0000 7fc819baf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561798865210 0x5617988655f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:45390/0 (socket says 172.21.15.139:45390) 2024-09-16T01:56:41.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.705+0000 7fc819baf640 1 -- 172.21.15.139:0/3758647436 learned_addr learned my addr 172.21.15.139:0/3758647436 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:41.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.706+0000 7fc8127fc640 1 -- 172.21.15.139:0/2707861596 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc808010ac0 con 0x7fc81411b140 2024-09-16T01:56:41.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.706+0000 7fc818bad640 1 --2- 172.21.15.139:0/3758647436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5617988703f0 0x5617988727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.706+0000 7fc8193ae640 1 --2- 172.21.15.139:0/3758647436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561798865b30 0x56179886feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.707+0000 7fc8127fc640 1 --2- 172.21.15.139:0/2707861596 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc7fc0a35c0 0x7fc7fc0a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.708+0000 7fc819baf640 1 --2- 172.21.15.139:0/2707861596 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc7fc0a35c0 0x7fc7fc0a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.708+0000 7fc819baf640 1 -- 172.21.15.139:0/3758647436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561798865b30 msgr2=0x56179886feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:41.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.708+0000 7fc819baf640 1 --2- 172.21.15.139:0/3758647436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561798865b30 0x56179886feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.708+0000 7fc819baf640 1 -- 172.21.15.139:0/3758647436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5617988703f0 msgr2=0x5617988727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:41.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.708+0000 7fc819baf640 1 --2- 172.21.15.139:0/3758647436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5617988703f0 0x5617988727e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.708+0000 7fc819baf640 1 -- 172.21.15.139:0/3758647436 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561798762d40 con 0x561798865210 2024-09-16T01:56:41.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.708+0000 7fc8127fc640 1 -- 172.21.15.139:0/2707861596 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9710+0+0 (secure 0 0 0) 0x7fc8080d06a0 con 0x7fc81411b140 2024-09-16T01:56:41.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.708+0000 7fc819baf640 1 --2- 172.21.15.139:0/3758647436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561798865210 0x5617988655f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc80400aa80 tx=0x7fc80400af40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d703b9730efce148 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:41.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.710+0000 7fc819baf640 1 --2- 172.21.15.139:0/2707861596 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc7fc0a35c0 0x7fc7fc0a5a80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc80400afd0 tx=0x7fc804018740 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:41.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.712+0000 7fc7fb7fe640 1 -- 172.21.15.139:0/3758647436 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804014040 con 0x561798865210 2024-09-16T01:56:41.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.712+0000 7fc7fb7fe640 1 -- 172.21.15.139:0/3758647436 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8040218e0 con 0x561798865210 2024-09-16T01:56:41.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.713+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3758647436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561798865210 msgr2=0x5617988655f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:41.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.713+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/3758647436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561798865210 0x5617988655f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc80400aa80 tx=0x7fc80400af40 comp rx=0 tx=0).stop 2024-09-16T01:56:41.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.718+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3758647436 shutdown_connections 2024-09-16T01:56:41.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.718+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/3758647436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5617988703f0 0x5617988727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.718+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/3758647436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561798865b30 0x56179886feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.718+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/3758647436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561798865210 0x5617988655f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.718+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3758647436 >> 172.21.15.139:0/3758647436 conn(0x5617988617a0 msgr2=0x561798861b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:41.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.721+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3758647436 shutdown_connections 2024-09-16T01:56:41.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.723+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3758647436 wait complete. 2024-09-16T01:56:41.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.723+0000 7fc81d9eb740 1 -- start start 2024-09-16T01:56:41.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.723+0000 7fc81d9eb740 1 Processor -- start 2024-09-16T01:56:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.724+0000 7fc81d9eb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561798865210 0x56179886d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.724+0000 7fc81d9eb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865b30 0x56179886d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.724+0000 7fc81d9eb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617988703f0 0x56179886dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.724+0000 7fc81d9eb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561798873010 con 0x561798865b30 2024-09-16T01:56:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.724+0000 7fc81d9eb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561798752f10 con 0x561798865210 2024-09-16T01:56:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.724+0000 7fc81d9eb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561798873190 con 0x5617988703f0 2024-09-16T01:56:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.724+0000 7fc81d9eb740 10 client.0 _ll_register_callbacks cb 0x561798776db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.724+0000 7fc818bad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865b30 0x56179886d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.724+0000 7fc818bad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865b30 0x56179886d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48120/0 (socket says 172.21.15.139:48120) 2024-09-16T01:56:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.724+0000 7fc818bad640 1 -- 172.21.15.139:0/3879709201 learned_addr learned my addr 172.21.15.139:0/3879709201 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.725+0000 7fc819baf640 1 --2- 172.21.15.139:0/3879709201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617988703f0 0x56179886dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.725+0000 7fc818bad640 1 -- 172.21.15.139:0/3879709201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617988703f0 msgr2=0x56179886dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.725+0000 7fc818bad640 1 --2- 172.21.15.139:0/3879709201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617988703f0 0x56179886dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.725+0000 7fc818bad640 1 -- 172.21.15.139:0/3879709201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561798865210 msgr2=0x56179886d220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:56:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.725+0000 7fc818bad640 1 --2- 172.21.15.139:0/3879709201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561798865210 0x56179886d220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.725+0000 7fc818bad640 1 -- 172.21.15.139:0/3879709201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617984d9580 con 0x561798865b30 2024-09-16T01:56:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.725+0000 7fc818bad640 1 --2- 172.21.15.139:0/3879709201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865b30 0x56179886d760 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc80800e9c0 tx=0x7fc80800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3fadec6c6b1819 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.726+0000 7fc7f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:56:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.727+0000 7fc7f8ff9640 1 -- 172.21.15.139:0/3879709201 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80809c480 con 0x561798865b30 2024-09-16T01:56:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.727+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3879709201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8040218e0 con 0x561798865b30 2024-09-16T01:56:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.727+0000 7fc7f8ff9640 1 -- 172.21.15.139:0/3879709201 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9710+0+0 (secure 0 0 0) 0x7fc8080a4c60 con 0x561798865b30 2024-09-16T01:56:41.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.729+0000 7fc7f8ff9640 1 -- 172.21.15.139:0/3879709201 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fc8080a0830 con 0x561798865b30 2024-09-16T01:56:41.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.729+0000 7fc81d9eb740 10 client.17169 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:41.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.729+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3879709201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5617985a3960 con 0x561798865b30 2024-09-16T01:56:41.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.732+0000 7fc7f8ff9640 1 -- 172.21.15.139:0/3879709201 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8040218e0 con 0x561798865b30 2024-09-16T01:56:41.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.733+0000 7fc7f8ff9640 1 -- 172.21.15.139:0/3879709201 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7fc8080981d0 con 0x561798865b30 2024-09-16T01:56:41.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.733+0000 7fc81d9eb740 10 client.17169 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:41.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.733+0000 7fc81d9eb740 10 client.17169 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:56:41.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.733+0000 7fc81d9eb740 20 client.17169 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:41.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.734+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/3879709201 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561798880490 0x561798882950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.734+0000 7fc81d9eb740 4 client.17169 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:56:41.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.734+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3879709201 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617987d7b90 con 0x561798880490 2024-09-16T01:56:41.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.736+0000 7fc8193ae640 1 --2- 172.21.15.139:0/3879709201 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561798880490 0x561798882950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.740+0000 7fc8193ae640 1 --2- 172.21.15.139:0/3879709201 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561798880490 0x561798882950 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9bb9348ac99b9cf server_cookie=81c570e00bd79168 in_seq=0 out_seq=0 2024-09-16T01:56:41.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.740+0000 7fc7f8ff9640 10 client.17169 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:41.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.741+0000 7fc7f8ff9640 1 -- 172.21.15.139:0/3879709201 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617987d7b90 con 0x561798880490 2024-09-16T01:56:41.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.741+0000 7fc7f8ff9640 10 client.17169 handle_command_reply: tid=0 2024-09-16T01:56:41.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.742+0000 7fc81d9eb740 1 client.17169 shutdown 2024-09-16T01:56:41.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.745+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3879709201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865b30 msgr2=0x56179886d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:41.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.745+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/3879709201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865b30 0x56179886d760 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc80800e9c0 tx=0x7fc80800cf20 comp rx=0 tx=0).stop 2024-09-16T01:56:41.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.748+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3879709201 shutdown_connections 2024-09-16T01:56:41.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.748+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/3879709201 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561798880490 0x561798882950 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.748+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/3879709201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617988703f0 0x56179886dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.748+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/3879709201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865b30 0x56179886d760 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.748+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/3879709201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561798865210 0x56179886d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.748+0000 7fc7f8ff9640 0 client.17169 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:41.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.748+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3879709201 >> 172.21.15.139:0/3879709201 conn(0x5617988617a0 msgr2=0x561798867480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:41.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.750+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3879709201 shutdown_connections 2024-09-16T01:56:41.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.751+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3879709201 wait complete. 2024-09-16T01:56:41.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.752+0000 7fc81d9eb740 20 client.17169 trim_cache size 0 max 16384 2024-09-16T01:56:41.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.783+0000 7fc81d9eb740 1 Processor -- start 2024-09-16T01:56:41.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.784+0000 7fc81d9eb740 1 -- start start 2024-09-16T01:56:41.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.784+0000 7fc81d9eb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865210 0x56179887bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.784+0000 7fc81d9eb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561798865b30 0x56179886a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.784+0000 7fc81d9eb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617988703f0 0x56179886a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.785+0000 7fc81d9eb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141c6d30 con 0x561798865210 2024-09-16T01:56:41.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.785+0000 7fc81d9eb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141c8b70 con 0x5617988703f0 2024-09-16T01:56:41.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.785+0000 7fc81d9eb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141c74c0 con 0x561798865b30 2024-09-16T01:56:41.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.785+0000 7fc819baf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617988703f0 0x56179886a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.785+0000 7fc819baf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617988703f0 0x56179886a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:54126/0 (socket says 172.21.15.139:54126) 2024-09-16T01:56:41.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.785+0000 7fc819baf640 1 -- 172.21.15.139:0/3816886031 learned_addr learned my addr 172.21.15.139:0/3816886031 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:41.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.785+0000 7fc818bad640 1 --2- 172.21.15.139:0/3816886031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561798865b30 0x56179886a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.786+0000 7fc818bad640 1 -- 172.21.15.139:0/3816886031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617988703f0 msgr2=0x56179886a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:41.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.786+0000 7fc818bad640 1 --2- 172.21.15.139:0/3816886031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617988703f0 0x56179886a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.786+0000 7fc818bad640 1 -- 172.21.15.139:0/3816886031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865210 msgr2=0x56179887bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:41.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.786+0000 7fc818bad640 1 --2- 172.21.15.139:0/3816886031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865210 0x56179887bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.786+0000 7fc818bad640 1 -- 172.21.15.139:0/3816886031 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56179886fb90 con 0x561798865b30 2024-09-16T01:56:41.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.790+0000 7fc818bad640 1 --2- 172.21.15.139:0/3816886031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561798865b30 0x56179886a480 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x56179886e7d0 tx=0x7fc808018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c649983dfa3117ce server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:41.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.793+0000 7fc7faffd640 1 -- 172.21.15.139:0/3816886031 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80809c480 con 0x561798865b30 2024-09-16T01:56:41.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.793+0000 7fc7faffd640 1 -- 172.21.15.139:0/3816886031 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8080a50f0 con 0x561798865b30 2024-09-16T01:56:41.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.793+0000 7fc7faffd640 1 -- 172.21.15.139:0/3816886031 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8080d0180 con 0x561798865b30 2024-09-16T01:56:41.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.794+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3816886031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561798865b30 msgr2=0x56179886a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:41.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.794+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/3816886031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561798865b30 0x56179886a480 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x56179886e7d0 tx=0x7fc808018420 comp rx=0 tx=0).stop 2024-09-16T01:56:41.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.802+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3816886031 shutdown_connections 2024-09-16T01:56:41.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.802+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/3816886031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617988703f0 0x56179886a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.802+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/3816886031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561798865b30 0x56179886a480 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.802+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/3816886031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865210 0x56179887bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.802+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3816886031 >> 172.21.15.139:0/3816886031 conn(0x5617988617a0 msgr2=0x56179887b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:41.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.804+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3816886031 shutdown_connections 2024-09-16T01:56:41.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.806+0000 7fc81d9eb740 1 -- 172.21.15.139:0/3816886031 wait complete. 2024-09-16T01:56:41.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.807+0000 7fc81d9eb740 1 -- start start 2024-09-16T01:56:41.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.807+0000 7fc81d9eb740 1 Processor -- start 2024-09-16T01:56:41.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.811+0000 7fc81d9eb740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865210 0x561798888800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.812+0000 7fc81d9eb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561798865b30 0x561798883b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.812+0000 7fc81d9eb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617988703f0 0x5617988840c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.812+0000 7fc81d9eb740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141c83e0 con 0x561798865210 2024-09-16T01:56:41.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.812+0000 7fc81d9eb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141c7c50 con 0x561798865b30 2024-09-16T01:56:41.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.812+0000 7fc81d9eb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141c4dc0 con 0x5617988703f0 2024-09-16T01:56:41.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.812+0000 7fc81d9eb740 10 client.0 _ll_register_callbacks cb 0x7fc808009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:41.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.816+0000 7fc8193ae640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865210 0x561798888800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.816+0000 7fc8193ae640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865210 0x561798888800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48138/0 (socket says 172.21.15.139:48138) 2024-09-16T01:56:41.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.816+0000 7fc8193ae640 1 -- 172.21.15.139:0/2605016420 learned_addr learned my addr 172.21.15.139:0/2605016420 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:41.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.817+0000 7fc819baf640 1 --2- 172.21.15.139:0/2605016420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617988703f0 0x5617988840c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.817+0000 7fc818bad640 1 --2- 172.21.15.139:0/2605016420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561798865b30 0x561798883b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.826+0000 7fc8193ae640 1 -- 172.21.15.139:0/2605016420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617988703f0 msgr2=0x5617988840c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:41.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.826+0000 7fc8193ae640 1 --2- 172.21.15.139:0/2605016420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617988703f0 0x5617988840c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.826+0000 7fc8193ae640 1 -- 172.21.15.139:0/2605016420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561798865b30 msgr2=0x561798883b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:41.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.826+0000 7fc8193ae640 1 --2- 172.21.15.139:0/2605016420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561798865b30 0x561798883b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:41.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.826+0000 7fc8193ae640 1 -- 172.21.15.139:0/2605016420 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617988848c0 con 0x561798865210 2024-09-16T01:56:41.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.832+0000 7fc8193ae640 1 --2- 172.21.15.139:0/2605016420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865210 0x561798888800 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc80c005ab0 tx=0x7fc80c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de73a09af89f3279 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:41.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.832+0000 7fc7f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:56:41.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.833+0000 7fc7f8ff9640 1 -- 172.21.15.139:0/2605016420 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c047020 con 0x561798865210 2024-09-16T01:56:41.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.833+0000 7fc7f8ff9640 1 -- 172.21.15.139:0/2605016420 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9710+0+0 (secure 0 0 0) 0x7fc80c004350 con 0x561798865210 2024-09-16T01:56:41.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.834+0000 7fc81d9eb740 1 -- 172.21.15.139:0/2605016420 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8080a50f0 con 0x561798865210 2024-09-16T01:56:41.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.837+0000 7fc7f8ff9640 1 -- 172.21.15.139:0/2605016420 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fc80c035040 con 0x561798865210 2024-09-16T01:56:41.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.838+0000 7fc81d9eb740 10 client.17172 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:41.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.838+0000 7fc81d9eb740 1 -- 172.21.15.139:0/2605016420 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5617988eb180 con 0x561798865210 2024-09-16T01:56:41.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.840+0000 7fc7f8ff9640 1 -- 172.21.15.139:0/2605016420 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8080a50f0 con 0x561798865210 2024-09-16T01:56:41.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.840+0000 7fc7f8ff9640 1 -- 172.21.15.139:0/2605016420 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7fc80c003800 con 0x561798865210 2024-09-16T01:56:41.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.843+0000 7fc81d9eb740 10 client.17172 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:41.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.843+0000 7fc81d9eb740 10 client.17172 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:56:41.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.843+0000 7fc81d9eb740 20 client.17172 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:41.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.843+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/2605016420 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5617988eb8c0 0x5617988edcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:41.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.843+0000 7fc81d9eb740 4 client.17172 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cf44c288", "release": true, "await": true}] 2024-09-16T01:56:41.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.843+0000 7fc81d9eb740 1 -- 172.21.15.139:0/2605016420 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cf44c288", "release": true, "await": true}) -- 0x5617987d20a0 con 0x5617988eb8c0 2024-09-16T01:56:41.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.846+0000 7fc818bad640 1 --2- 172.21.15.139:0/2605016420 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5617988eb8c0 0x5617988edcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:41.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.854+0000 7fc818bad640 1 --2- 172.21.15.139:0/2605016420 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5617988eb8c0 0x5617988edcb0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e624b9fe51ce7ba0 server_cookie=4ce45705128a0bea in_seq=0 out_seq=0 2024-09-16T01:56:41.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:41.854+0000 7fc7f8ff9640 10 client.17172 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:42.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.523+0000 7fc7f8ff9640 1 -- 172.21.15.139:0/2605016420 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5617987d20a0 con 0x5617988eb8c0 2024-09-16T01:56:42.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.523+0000 7fc7f8ff9640 10 client.17172 handle_command_reply: tid=0 2024-09-16T01:56:42.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.523+0000 7fc81d9eb740 1 client.17172 shutdown 2024-09-16T01:56:42.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.523+0000 7fc81d9eb740 1 -- 172.21.15.139:0/2605016420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865210 msgr2=0x561798888800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:42.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.523+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/2605016420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865210 0x561798888800 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc80c005ab0 tx=0x7fc80c005760 comp rx=0 tx=0).stop 2024-09-16T01:56:42.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.523+0000 7fc81d9eb740 1 -- 172.21.15.139:0/2605016420 shutdown_connections 2024-09-16T01:56:42.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.523+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/2605016420 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5617988eb8c0 0x5617988edcb0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:42.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.523+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/2605016420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617988703f0 0x5617988840c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:42.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.523+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/2605016420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561798865b30 0x561798883b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.523+0000 7fc81d9eb740 1 --2- 172.21.15.139:0/2605016420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561798865210 0x561798888800 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.523+0000 7fc81d9eb740 1 -- 172.21.15.139:0/2605016420 >> 172.21.15.139:0/2605016420 conn(0x5617988617a0 msgr2=0x56179886da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.523+0000 7fc7f8ff9640 0 client.17172 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.524+0000 7fc81d9eb740 1 -- 172.21.15.139:0/2605016420 shutdown_connections 2024-09-16T01:56:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.524+0000 7fc81d9eb740 1 -- 172.21.15.139:0/2605016420 wait complete. 2024-09-16T01:56:42.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.524+0000 7fc81d9eb740 20 client.17172 trim_cache size 0 max 16384 2024-09-16T01:56:42.527 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:56:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 71, 2024-09-16T01:56:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:56:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 60, 2024-09-16T01:56:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:56:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "cf44c288": { 2024-09-16T01:56:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 60, 2024-09-16T01:56:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:56:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:56:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:56:42.527 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:56:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:56:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:56:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 137.7, 2024-09-16T01:56:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:56:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:56:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:56:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:56:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:56:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:56:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:42.528 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:56:42.529 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:56:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.525+0000 7fc81a3b0640 1 -- 172.21.15.139:0/2707861596 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc7fc0a35c0 msgr2=0x7fc7fc0a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.525+0000 7fc81a3b0640 1 --2- 172.21.15.139:0/2707861596 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc7fc0a35c0 0x7fc7fc0a5a80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc80400afd0 tx=0x7fc804018740 comp rx=0 tx=0).stop 2024-09-16T01:56:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.525+0000 7fc81a3b0640 1 -- 172.21.15.139:0/2707861596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc81411b140 msgr2=0x7fc814112990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.525+0000 7fc81a3b0640 1 --2- 172.21.15.139:0/2707861596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc81411b140 0x7fc814112990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc80800e9f0 tx=0x7fc80800eeb0 comp rx=0 tx=0).stop 2024-09-16T01:56:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.525+0000 7fc81a3b0640 1 -- 172.21.15.139:0/2707861596 shutdown_connections 2024-09-16T01:56:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.525+0000 7fc81a3b0640 1 --2- 172.21.15.139:0/2707861596 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fc7fc0a35c0 0x7fc7fc0a5a80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.525+0000 7fc81a3b0640 1 --2- 172.21.15.139:0/2707861596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc814112ed0 0x7fc814113380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:42.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.525+0000 7fc81a3b0640 1 --2- 172.21.15.139:0/2707861596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc81411b140 0x7fc814112990 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc80800e9f0 tx=0x7fc80800eeb0 comp rx=0 tx=0).stop 2024-09-16T01:56:42.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.525+0000 7fc81a3b0640 1 --2- 172.21.15.139:0/2707861596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc81406ea30 0x7fc8141198b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:42.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.525+0000 7fc81a3b0640 1 -- 172.21.15.139:0/2707861596 >> 172.21.15.139:0/2707861596 conn(0x7fc814073180 msgr2=0x7fc81406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:42.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.525+0000 7fc81a3b0640 1 -- 172.21.15.139:0/2707861596 shutdown_connections 2024-09-16T01:56:42.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:56:42.526+0000 7fc81a3b0640 1 -- 172.21.15.139:0/2707861596 wait complete. 2024-09-16T01:56:43.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:43 smithi139 ceph-mon[31175]: pgmap v517: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 171 op/s 2024-09-16T01:56:43.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:43 smithi176 ceph-mon[35324]: pgmap v517: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 171 op/s 2024-09-16T01:56:43.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:43 smithi164 ceph-mon[35121]: pgmap v517: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 171 op/s 2024-09-16T01:56:45.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:45 smithi139 ceph-mon[31175]: pgmap v518: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 164 op/s 2024-09-16T01:56:45.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:45 smithi176 ceph-mon[35324]: pgmap v518: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 164 op/s 2024-09-16T01:56:45.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:45 smithi164 ceph-mon[35121]: pgmap v518: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 164 op/s 2024-09-16T01:56:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.251+0000 7f538edc4640 1 -- 172.21.15.139:0/147533814 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5388074eb0 msgr2=0x7f53881125a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.251+0000 7f538edc4640 1 --2- 172.21.15.139:0/147533814 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5388074eb0 0x7f53881125a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f537c009860 tx=0x7f537c02a410 comp rx=0 tx=0).stop 2024-09-16T01:56:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.252+0000 7f538edc4640 1 -- 172.21.15.139:0/147533814 shutdown_connections 2024-09-16T01:56:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.252+0000 7f538edc4640 1 --2- 172.21.15.139:0/147533814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5388112ae0 0x7f5388114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.252+0000 7f538edc4640 1 --2- 172.21.15.139:0/147533814 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5388074eb0 0x7f53881125a0 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f537c009860 tx=0x7f537c02a410 comp rx=0 tx=0).stop 2024-09-16T01:56:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.252+0000 7f538edc4640 1 --2- 172.21.15.139:0/147533814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5388074590 0x7f5388074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.252+0000 7f538edc4640 1 -- 172.21.15.139:0/147533814 >> 172.21.15.139:0/147533814 conn(0x7f53880fd7d0 msgr2=0x7f53880ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.256+0000 7f538edc4640 1 -- 172.21.15.139:0/147533814 shutdown_connections 2024-09-16T01:56:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.258+0000 7f538edc4640 1 -- 172.21.15.139:0/147533814 wait complete. 2024-09-16T01:56:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.262+0000 7f538edc4640 1 Processor -- start 2024-09-16T01:56:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.263+0000 7f538edc4640 1 -- start start 2024-09-16T01:56:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.263+0000 7f538edc4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5388074590 0x7f53881a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.263+0000 7f538edc4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5388112ae0 0x7f53881a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.263+0000 7f538edc4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f538819ea30 0x7f538819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.263+0000 7f538edc4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5388118580 con 0x7f5388112ae0 2024-09-16T01:56:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.263+0000 7f538edc4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5388118400 con 0x7f5388074590 2024-09-16T01:56:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.263+0000 7f538edc4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5388118700 con 0x7f538819ea30 2024-09-16T01:56:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.263+0000 7f5387fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5388112ae0 0x7f53881a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.263+0000 7f5387fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5388112ae0 0x7f53881a4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41894/0 (socket says 172.21.15.139:41894) 2024-09-16T01:56:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.263+0000 7f5387fff640 1 -- 172.21.15.139:0/610111913 learned_addr learned my addr 172.21.15.139:0/610111913 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.263+0000 7f538d361640 1 --2- 172.21.15.139:0/610111913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f538819ea30 0x7f538819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.264+0000 7f538d361640 1 -- 172.21.15.139:0/610111913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5388074590 msgr2=0x7f53881a4880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:56:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.264+0000 7f538d361640 1 --2- 172.21.15.139:0/610111913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5388074590 0x7f53881a4880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.264+0000 7f538d361640 1 -- 172.21.15.139:0/610111913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5388112ae0 msgr2=0x7f53881a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.264+0000 7f538d361640 1 --2- 172.21.15.139:0/610111913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5388112ae0 0x7f53881a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.264+0000 7f538d361640 1 -- 172.21.15.139:0/610111913 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538819f550 con 0x7f538819ea30 2024-09-16T01:56:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.264+0000 7f538d361640 1 --2- 172.21.15.139:0/610111913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f538819ea30 0x7f538819eee0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f537800d680 tx=0x7f537800db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.269+0000 7f5385ffb640 1 -- 172.21.15.139:0/610111913 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378017070 con 0x7f538819ea30 2024-09-16T01:56:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.269+0000 7f5385ffb640 1 -- 172.21.15.139:0/610111913 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5378009820 con 0x7f538819ea30 2024-09-16T01:56:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.269+0000 7f5385ffb640 1 -- 172.21.15.139:0/610111913 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378004ce0 con 0x7f538819ea30 2024-09-16T01:56:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.269+0000 7f538edc4640 1 -- 172.21.15.139:0/610111913 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538819f840 con 0x7f538819ea30 2024-09-16T01:56:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.269+0000 7f538edc4640 1 -- 172.21.15.139:0/610111913 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f538806bf60 con 0x7f538819ea30 2024-09-16T01:56:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.279+0000 7f5385ffb640 1 -- 172.21.15.139:0/610111913 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f5378010460 con 0x7f538819ea30 2024-09-16T01:56:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.280+0000 7f5391c3f740 1 Processor -- start 2024-09-16T01:56:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.281+0000 7f5391c3f740 1 -- start start 2024-09-16T01:56:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.284+0000 7f5391c3f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.284+0000 7f5391c3f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd4800770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.284+0000 7f5391c3f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd4800cb0 0x555bd48030a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.284+0000 7f5391c3f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555bd475a2f0 con 0x555bd4800cb0 2024-09-16T01:56:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.284+0000 7f5391c3f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555bd4702270 con 0x555bd47f6410 2024-09-16T01:56:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.284+0000 7f5391c3f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555bd47562b0 con 0x555bd47f5af0 2024-09-16T01:56:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.285+0000 7f5385ffb640 1 --2- 172.21.15.139:0/610111913 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f53700a3690 0x7f53700a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.285+0000 7f5387fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.285+0000 7f5387fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47f5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:46828/0 (socket says 172.21.15.139:46828) 2024-09-16T01:56:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.285+0000 7f5387fff640 1 -- 172.21.15.139:0/2463279763 learned_addr learned my addr 172.21.15.139:0/2463279763 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.285+0000 7f538d361640 1 --2- 172.21.15.139:0/610111913 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f53700a3690 0x7f53700a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.285+0000 7f5387fff640 1 -- 172.21.15.139:0/2463279763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 msgr2=0x555bd4800770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:56:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.285+0000 7f5387fff640 1 --2- 172.21.15.139:0/2463279763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd4800770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.285+0000 7f5387fff640 1 -- 172.21.15.139:0/2463279763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd4800cb0 msgr2=0x555bd48030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.285+0000 7f5387fff640 1 --2- 172.21.15.139:0/2463279763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd4800cb0 0x555bd48030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.285+0000 7f5387fff640 1 -- 172.21.15.139:0/2463279763 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bd4701340 con 0x555bd47f5af0 2024-09-16T01:56:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.285+0000 7f5385ffb640 1 -- 172.21.15.139:0/610111913 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9710+0+0 (secure 0 0 0) 0x7f53780d0b30 con 0x7f538819ea30 2024-09-16T01:56:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.285+0000 7f538d361640 1 --2- 172.21.15.139:0/610111913 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f53700a3690 0x7f53700a5b50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5378016000 tx=0x7f5378014000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.286+0000 7f5387fff640 1 --2- 172.21.15.139:0/2463279763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47f5ed0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f537c009780 tx=0x7f537c009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f2b5881d0d2cf43 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.286+0000 7f536effd640 1 -- 172.21.15.139:0/2463279763 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c0305a0 con 0x555bd47f5af0 2024-09-16T01:56:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.286+0000 7f536effd640 1 -- 172.21.15.139:0/2463279763 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537c042030 con 0x555bd47f5af0 2024-09-16T01:56:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.286+0000 7f536effd640 1 -- 172.21.15.139:0/2463279763 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c03e440 con 0x555bd47f5af0 2024-09-16T01:56:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.287+0000 7f5391c3f740 1 -- 172.21.15.139:0/2463279763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 msgr2=0x555bd47f5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.287+0000 7f5391c3f740 1 --2- 172.21.15.139:0/2463279763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47f5ed0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f537c009780 tx=0x7f537c009550 comp rx=0 tx=0).stop 2024-09-16T01:56:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.291+0000 7f5391c3f740 1 -- 172.21.15.139:0/2463279763 shutdown_connections 2024-09-16T01:56:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.292+0000 7f5391c3f740 1 --2- 172.21.15.139:0/2463279763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd4800cb0 0x555bd48030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.292+0000 7f5391c3f740 1 --2- 172.21.15.139:0/2463279763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd4800770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.292+0000 7f5391c3f740 1 --2- 172.21.15.139:0/2463279763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47f5ed0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.292+0000 7f5391c3f740 1 -- 172.21.15.139:0/2463279763 >> 172.21.15.139:0/2463279763 conn(0x555bd47f2060 msgr2=0x555bd47f2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.293+0000 7f5391c3f740 1 -- 172.21.15.139:0/2463279763 shutdown_connections 2024-09-16T01:56:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.294+0000 7f5391c3f740 1 -- 172.21.15.139:0/2463279763 wait complete. 2024-09-16T01:56:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.294+0000 7f5391c3f740 1 -- start start 2024-09-16T01:56:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.294+0000 7f5391c3f740 1 Processor -- start 2024-09-16T01:56:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.295+0000 7f5391c3f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47fdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.295+0000 7f5391c3f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd47fe010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.295+0000 7f5391c3f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd4800cb0 0x555bd47fe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.295+0000 7f5391c3f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555bd48038d0 con 0x555bd4800cb0 2024-09-16T01:56:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.295+0000 7f5391c3f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555bd46e3290 con 0x555bd47f6410 2024-09-16T01:56:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.295+0000 7f5391c3f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555bd4803a50 con 0x555bd47f5af0 2024-09-16T01:56:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.295+0000 7f5391c3f740 10 client.0 _ll_register_callbacks cb 0x555bd4714140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.296+0000 7f5387fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47fdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.296+0000 7f5387fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47fdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:46842/0 (socket says 172.21.15.139:46842) 2024-09-16T01:56:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.296+0000 7f5387fff640 1 -- 172.21.15.139:0/858232352 learned_addr learned my addr 172.21.15.139:0/858232352 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.296+0000 7f5387fff640 1 --2- 172.21.15.139:0/858232352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd4800cb0 0x555bd47fe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.299+0000 7f538cb60640 1 --2- 172.21.15.139:0/858232352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd47fe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.300+0000 7f538cb60640 1 -- 172.21.15.139:0/858232352 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 msgr2=0x555bd47fdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.300+0000 7f538cb60640 1 --2- 172.21.15.139:0/858232352 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47fdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.300+0000 7f538cb60640 1 -- 172.21.15.139:0/858232352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd4800cb0 msgr2=0x555bd47fe550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.300+0000 7f538cb60640 1 --2- 172.21.15.139:0/858232352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd4800cb0 0x555bd47fe550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.300+0000 7f538cb60640 1 -- 172.21.15.139:0/858232352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bd446d210 con 0x555bd47f6410 2024-09-16T01:56:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.301+0000 7f538cb60640 1 --2- 172.21.15.139:0/858232352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd47fe010 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f538000b7d0 tx=0x7f538000bc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f5bd9fd911a5f3e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.301+0000 7f536c7f8640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:56:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.301+0000 7f536c7f8640 1 -- 172.21.15.139:0/858232352 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380019060 con 0x555bd47f6410 2024-09-16T01:56:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.301+0000 7f536c7f8640 1 -- 172.21.15.139:0/858232352 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9710+0+0 (secure 0 0 0) 0x7f538000c940 con 0x555bd47f6410 2024-09-16T01:56:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.301+0000 7f5391c3f740 1 -- 172.21.15.139:0/858232352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f537c042030 con 0x555bd47f6410 2024-09-16T01:56:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.301+0000 7f536c7f8640 1 -- 172.21.15.139:0/858232352 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f5380020050 con 0x555bd47f6410 2024-09-16T01:56:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.302+0000 7f5391c3f740 10 client.26612 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.302+0000 7f5391c3f740 1 -- 172.21.15.139:0/858232352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x555bd4539620 con 0x555bd47f6410 2024-09-16T01:56:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.303+0000 7f536c7f8640 1 -- 172.21.15.139:0/858232352 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537c042030 con 0x555bd47f6410 2024-09-16T01:56:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.303+0000 7f536c7f8640 1 -- 172.21.15.139:0/858232352 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f5380012340 con 0x555bd47f6410 2024-09-16T01:56:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.303+0000 7f5391c3f740 10 client.26612 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.303+0000 7f5391c3f740 10 client.26612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:56:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.303+0000 7f5391c3f740 20 client.26612 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.303+0000 7f5391c3f740 1 --2- 172.21.15.139:0/858232352 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555bd4821d90 0x555bd48241d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.303+0000 7f5391c3f740 4 client.26612 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:56:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.303+0000 7f5391c3f740 1 -- 172.21.15.139:0/858232352 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bd4767a20 con 0x555bd4821d90 2024-09-16T01:56:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.303+0000 7f5387fff640 1 --2- 172.21.15.139:0/858232352 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555bd4821d90 0x555bd48241d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.305+0000 7f5387fff640 1 --2- 172.21.15.139:0/858232352 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555bd4821d90 0x555bd48241d0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3fbf47074ecffd4 server_cookie=6821107b04ff679c in_seq=0 out_seq=0 2024-09-16T01:56:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.305+0000 7f536c7f8640 10 client.26612 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.306+0000 7f536c7f8640 1 -- 172.21.15.139:0/858232352 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555bd4767a20 con 0x555bd4821d90 2024-09-16T01:56:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.306+0000 7f536c7f8640 10 client.26612 handle_command_reply: tid=0 2024-09-16T01:56:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.306+0000 7f5391c3f740 1 client.26612 shutdown 2024-09-16T01:56:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.309+0000 7f5391c3f740 1 -- 172.21.15.139:0/858232352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 msgr2=0x555bd47fe010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.309+0000 7f5391c3f740 1 --2- 172.21.15.139:0/858232352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd47fe010 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f538000b7d0 tx=0x7f538000bc90 comp rx=0 tx=0).stop 2024-09-16T01:56:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.311+0000 7f5391c3f740 1 -- 172.21.15.139:0/858232352 shutdown_connections 2024-09-16T01:56:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.311+0000 7f5391c3f740 1 --2- 172.21.15.139:0/858232352 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555bd4821d90 0x555bd48241d0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.311+0000 7f5391c3f740 1 --2- 172.21.15.139:0/858232352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd4800cb0 0x555bd47fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.311+0000 7f5391c3f740 1 --2- 172.21.15.139:0/858232352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd47fe010 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.311+0000 7f5391c3f740 1 --2- 172.21.15.139:0/858232352 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47fdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.311+0000 7f5391c3f740 1 -- 172.21.15.139:0/858232352 >> 172.21.15.139:0/858232352 conn(0x555bd47f2060 msgr2=0x555bd47f11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.311+0000 7f536c7f8640 0 client.26612 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.314+0000 7f5391c3f740 1 -- 172.21.15.139:0/858232352 shutdown_connections 2024-09-16T01:56:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.314+0000 7f5391c3f740 1 -- 172.21.15.139:0/858232352 wait complete. 2024-09-16T01:56:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.314+0000 7f5391c3f740 20 client.26612 trim_cache size 0 max 16384 2024-09-16T01:56:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.345+0000 7f5391c3f740 1 Processor -- start 2024-09-16T01:56:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.345+0000 7f5391c3f740 1 -- start start 2024-09-16T01:56:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.346+0000 7f5391c3f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47fe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.346+0000 7f5391c3f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd4829ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.346+0000 7f5391c3f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd4800cb0 0x555bd482a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.346+0000 7f5391c3f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53740101b0 con 0x555bd4800cb0 2024-09-16T01:56:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.346+0000 7f5391c3f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5374011ff0 con 0x555bd47f6410 2024-09-16T01:56:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.346+0000 7f5391c3f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5374010940 con 0x555bd47f5af0 2024-09-16T01:56:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.346+0000 7f538cb60640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd4829ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.347+0000 7f5387fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.347+0000 7f5387fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:46854/0 (socket says 172.21.15.139:46854) 2024-09-16T01:56:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.347+0000 7f5387fff640 1 -- 172.21.15.139:0/3306056719 learned_addr learned my addr 172.21.15.139:0/3306056719 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.347+0000 7f5387fff640 1 -- 172.21.15.139:0/3306056719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 msgr2=0x555bd4829ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.347+0000 7f5387fff640 1 --2- 172.21.15.139:0/3306056719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd4829ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.347+0000 7f5387fff640 1 -- 172.21.15.139:0/3306056719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd4800cb0 msgr2=0x555bd482a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.347+0000 7f5387fff640 1 --2- 172.21.15.139:0/3306056719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd4800cb0 0x555bd482a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.347+0000 7f5387fff640 1 -- 172.21.15.139:0/3306056719 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bd47f6240 con 0x555bd47f5af0 2024-09-16T01:56:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.348+0000 7f5387fff640 1 --2- 172.21.15.139:0/3306056719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47fe380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f537c009780 tx=0x7f537c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8fa7b1ea56c457d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.350+0000 7f536e7fc640 1 -- 172.21.15.139:0/3306056719 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c004050 con 0x555bd47f5af0 2024-09-16T01:56:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.350+0000 7f536e7fc640 1 -- 172.21.15.139:0/3306056719 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537c042040 con 0x555bd47f5af0 2024-09-16T01:56:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.351+0000 7f536e7fc640 1 -- 172.21.15.139:0/3306056719 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c03d6a0 con 0x555bd47f5af0 2024-09-16T01:56:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.351+0000 7f5391c3f740 1 -- 172.21.15.139:0/3306056719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 msgr2=0x555bd47fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.351+0000 7f5391c3f740 1 --2- 172.21.15.139:0/3306056719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47fe380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f537c009780 tx=0x7f537c02a410 comp rx=0 tx=0).stop 2024-09-16T01:56:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.352+0000 7f5391c3f740 1 -- 172.21.15.139:0/3306056719 shutdown_connections 2024-09-16T01:56:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.352+0000 7f5391c3f740 1 --2- 172.21.15.139:0/3306056719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd4800cb0 0x555bd482a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.352+0000 7f5391c3f740 1 --2- 172.21.15.139:0/3306056719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd4829ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.352+0000 7f5391c3f740 1 --2- 172.21.15.139:0/3306056719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd47f5af0 0x555bd47fe380 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.352+0000 7f5391c3f740 1 -- 172.21.15.139:0/3306056719 >> 172.21.15.139:0/3306056719 conn(0x555bd47f2060 msgr2=0x555bd47fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.356+0000 7f5391c3f740 1 -- 172.21.15.139:0/3306056719 shutdown_connections 2024-09-16T01:56:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.358+0000 7f5391c3f740 1 -- 172.21.15.139:0/3306056719 wait complete. 2024-09-16T01:56:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.358+0000 7f5391c3f740 1 -- start start 2024-09-16T01:56:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.358+0000 7f5391c3f740 1 Processor -- start 2024-09-16T01:56:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.358+0000 7f5391c3f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd47f5af0 0x555bd4809070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.359+0000 7f5391c3f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd48095b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.359+0000 7f5391c3f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd4800cb0 0x555bd47fa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.359+0000 7f5391c3f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5374011860 con 0x555bd47f5af0 2024-09-16T01:56:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.359+0000 7f5391c3f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53740110d0 con 0x555bd47f6410 2024-09-16T01:56:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.359+0000 7f5391c3f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f537400e190 con 0x555bd4800cb0 2024-09-16T01:56:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.359+0000 7f5391c3f740 10 client.0 _ll_register_callbacks cb 0x7f5380009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:56:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.360+0000 7f5387fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd4800cb0 0x555bd47fa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.360+0000 7f5387fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd47f5af0 0x555bd4809070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.360+0000 7f5387fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd47f5af0 0x555bd4809070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41920/0 (socket says 172.21.15.139:41920) 2024-09-16T01:56:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.360+0000 7f5387fff640 1 -- 172.21.15.139:0/2288814220 learned_addr learned my addr 172.21.15.139:0/2288814220 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:56:47.362 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.361+0000 7f5387fff640 1 -- 172.21.15.139:0/2288814220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd4800cb0 msgr2=0x555bd47fa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:47.362 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.361+0000 7f5387fff640 1 --2- 172.21.15.139:0/2288814220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd4800cb0 0x555bd47fa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.361+0000 7f5387fff640 1 -- 172.21.15.139:0/2288814220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 msgr2=0x555bd48095b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:56:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.361+0000 7f5387fff640 1 --2- 172.21.15.139:0/2288814220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd48095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.361+0000 7f5387fff640 1 -- 172.21.15.139:0/2288814220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bd47fab80 con 0x555bd47f5af0 2024-09-16T01:56:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.361+0000 7f5387fff640 1 --2- 172.21.15.139:0/2288814220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd47f5af0 0x555bd4809070 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f537c02aa00 tx=0x7f537c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef5a0ab6439fe8c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:56:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.361+0000 7f536c7f8640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:56:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.362+0000 7f536c7f8640 1 -- 172.21.15.139:0/2288814220 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c05c070 con 0x555bd47f5af0 2024-09-16T01:56:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.362+0000 7f5391c3f740 1 -- 172.21.15.139:0/2288814220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f537c042040 con 0x555bd47f5af0 2024-09-16T01:56:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.362+0000 7f536c7f8640 1 -- 172.21.15.139:0/2288814220 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9710+0+0 (secure 0 0 0) 0x7f537c03d460 con 0x555bd47f5af0 2024-09-16T01:56:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.362+0000 7f536c7f8640 1 -- 172.21.15.139:0/2288814220 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f537c03d6a0 con 0x555bd47f5af0 2024-09-16T01:56:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.365+0000 7f5391c3f740 10 client.17184 fetch_fsmap learned FSMap version 71 2024-09-16T01:56:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.365+0000 7f5391c3f740 1 -- 172.21.15.139:0/2288814220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x555bd4808930 con 0x555bd47f5af0 2024-09-16T01:56:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.368+0000 7f536c7f8640 1 -- 172.21.15.139:0/2288814220 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537c042040 con 0x555bd47f5af0 2024-09-16T01:56:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.369+0000 7f536c7f8640 1 -- 172.21.15.139:0/2288814220 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f537c060060 con 0x555bd47f5af0 2024-09-16T01:56:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.369+0000 7f5391c3f740 10 client.17184 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:56:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.369+0000 7f5391c3f740 10 client.17184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:56:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.369+0000 7f5391c3f740 20 client.17184 populate_metadata read hostname 'smithi139' 2024-09-16T01:56:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.369+0000 7f5391c3f740 1 --2- 172.21.15.139:0/2288814220 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555bd488cc70 0x555bd488f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:56:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.369+0000 7f5391c3f740 4 client.17184 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:56:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.369+0000 7f5391c3f740 1 -- 172.21.15.139:0/2288814220 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555bd4761d70 con 0x555bd488cc70 2024-09-16T01:56:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.373+0000 7f538cb60640 1 --2- 172.21.15.139:0/2288814220 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555bd488cc70 0x555bd488f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:56:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.383+0000 7f538cb60640 1 --2- 172.21.15.139:0/2288814220 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555bd488cc70 0x555bd488f090 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e51aa978df302e server_cookie=f91398e806ecbcb0 in_seq=0 out_seq=0 2024-09-16T01:56:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.384+0000 7f536c7f8640 10 client.17184 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:47.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:47 smithi176 ceph-mon[35324]: pgmap v519: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 413 KiB/s wr, 157 op/s 2024-09-16T01:56:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.482+0000 7f536c7f8640 1 -- 172.21.15.139:0/2288814220 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555bd4761d70 con 0x555bd488cc70 2024-09-16T01:56:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.482+0000 7f536c7f8640 10 client.17184 handle_command_reply: tid=0 2024-09-16T01:56:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.482+0000 7f5391c3f740 1 client.17184 shutdown 2024-09-16T01:56:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.482+0000 7f5391c3f740 1 -- 172.21.15.139:0/2288814220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd47f5af0 msgr2=0x555bd4809070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.482+0000 7f5391c3f740 1 --2- 172.21.15.139:0/2288814220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd47f5af0 0x555bd4809070 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f537c02aa00 tx=0x7f537c0058b0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.483+0000 7f5391c3f740 1 -- 172.21.15.139:0/2288814220 shutdown_connections 2024-09-16T01:56:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.483+0000 7f5391c3f740 1 --2- 172.21.15.139:0/2288814220 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x555bd488cc70 0x555bd488f090 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.483+0000 7f5391c3f740 1 --2- 172.21.15.139:0/2288814220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555bd4800cb0 0x555bd47fa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.483+0000 7f5391c3f740 1 --2- 172.21.15.139:0/2288814220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd47f6410 0x555bd48095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.483+0000 7f5391c3f740 1 --2- 172.21.15.139:0/2288814220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555bd47f5af0 0x555bd4809070 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.483+0000 7f5391c3f740 1 -- 172.21.15.139:0/2288814220 >> 172.21.15.139:0/2288814220 conn(0x555bd47f2060 msgr2=0x555bd47f7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.483+0000 7f536c7f8640 0 client.17184 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:56:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.483+0000 7f5391c3f740 1 -- 172.21.15.139:0/2288814220 shutdown_connections 2024-09-16T01:56:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.483+0000 7f5391c3f740 1 -- 172.21.15.139:0/2288814220 wait complete. 2024-09-16T01:56:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.483+0000 7f5391c3f740 20 client.17184 trim_cache size 0 max 16384 2024-09-16T01:56:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:56:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:56:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:56:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:56:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.484+0000 7f538edc4640 1 -- 172.21.15.139:0/610111913 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f53700a3690 msgr2=0x7f53700a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.484+0000 7f538edc4640 1 --2- 172.21.15.139:0/610111913 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f53700a3690 0x7f53700a5b50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5378016000 tx=0x7f5378014000 comp rx=0 tx=0).stop 2024-09-16T01:56:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.484+0000 7f538edc4640 1 -- 172.21.15.139:0/610111913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f538819ea30 msgr2=0x7f538819eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:56:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.484+0000 7f538edc4640 1 --2- 172.21.15.139:0/610111913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f538819ea30 0x7f538819eee0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f537800d680 tx=0x7f537800db40 comp rx=0 tx=0).stop 2024-09-16T01:56:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.484+0000 7f538edc4640 1 -- 172.21.15.139:0/610111913 shutdown_connections 2024-09-16T01:56:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.484+0000 7f538edc4640 1 --2- 172.21.15.139:0/610111913 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f53700a3690 0x7f53700a5b50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.484+0000 7f538edc4640 1 --2- 172.21.15.139:0/610111913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f538819ea30 0x7f538819eee0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.484+0000 7f538edc4640 1 --2- 172.21.15.139:0/610111913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5388112ae0 0x7f53881a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.484+0000 7f538edc4640 1 --2- 172.21.15.139:0/610111913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5388074590 0x7f53881a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:56:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.484+0000 7f538edc4640 1 -- 172.21.15.139:0/610111913 >> 172.21.15.139:0/610111913 conn(0x7f53880fd7d0 msgr2=0x7f53880fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:56:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.484+0000 7f538edc4640 1 -- 172.21.15.139:0/610111913 shutdown_connections 2024-09-16T01:56:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:56:47.485+0000 7f538edc4640 1 -- 172.21.15.139:0/610111913 wait complete. 2024-09-16T01:56:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:47 smithi164 ceph-mon[35121]: pgmap v519: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 413 KiB/s wr, 157 op/s 2024-09-16T01:56:47.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:47 smithi139 ceph-mon[31175]: pgmap v519: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 413 KiB/s wr, 157 op/s 2024-09-16T01:56:48.134 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cf44c288', total seconds elapsed: 29.4 2024-09-16T01:56:49.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:49 smithi164 ceph-mon[35121]: pgmap v520: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 415 KiB/s wr, 143 op/s 2024-09-16T01:56:49.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:49 smithi139 ceph-mon[31175]: pgmap v520: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 415 KiB/s wr, 143 op/s 2024-09-16T01:56:49.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:49 smithi176 ceph-mon[35324]: pgmap v520: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 415 KiB/s wr, 143 op/s 2024-09-16T01:56:51.488 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:51 smithi176 ceph-mon[35324]: pgmap v521: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 406 KiB/s wr, 142 op/s 2024-09-16T01:56:51.488 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:51 smithi176 ceph-mon[35324]: osdmap e112: 12 total, 12 up, 12 in 2024-09-16T01:56:51.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:51 smithi164 ceph-mon[35121]: pgmap v521: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 406 KiB/s wr, 142 op/s 2024-09-16T01:56:51.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:51 smithi164 ceph-mon[35121]: osdmap e112: 12 total, 12 up, 12 in 2024-09-16T01:56:51.598 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:51 smithi139 ceph-mon[31175]: pgmap v521: 129 pgs: 20 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 53 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 406 KiB/s wr, 142 op/s 2024-09-16T01:56:51.599 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:51 smithi139 ceph-mon[31175]: osdmap e112: 12 total, 12 up, 12 in 2024-09-16T01:56:53.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:53 smithi164 ceph-mon[35121]: pgmap v523: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 998 KiB/s wr, 143 op/s 2024-09-16T01:56:53.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:53 smithi139 ceph-mon[31175]: pgmap v523: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 998 KiB/s wr, 143 op/s 2024-09-16T01:56:53.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:53 smithi176 ceph-mon[35324]: pgmap v523: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 998 KiB/s wr, 143 op/s 2024-09-16T01:56:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:55 smithi164 ceph-mon[35121]: pgmap v524: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-09-16T01:56:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:55 smithi139 ceph-mon[31175]: pgmap v524: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-09-16T01:56:55.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:55 smithi176 ceph-mon[35324]: pgmap v524: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-09-16T01:56:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:57 smithi164 ceph-mon[35121]: pgmap v525: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-09-16T01:56:57.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:57 smithi139 ceph-mon[31175]: pgmap v525: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-09-16T01:56:57.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:57 smithi176 ceph-mon[35324]: pgmap v525: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-09-16T01:56:59.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:56:59 smithi164 ceph-mon[35121]: pgmap v526: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 211 op/s 2024-09-16T01:56:59.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:56:59 smithi139 ceph-mon[31175]: pgmap v526: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 211 op/s 2024-09-16T01:56:59.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:56:59 smithi176 ceph-mon[35324]: pgmap v526: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 211 op/s 2024-09-16T01:57:00.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:00 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.646+0000 7f1e1872c640 1 -- 172.21.15.139:0/811701505 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e101009b0 msgr2=0x7f1e10100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.646+0000 7f1e1872c640 1 --2- 172.21.15.139:0/811701505 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e101009b0 0x7f1e10100e10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00d940 tx=0x7f1e0c02e600 comp rx=0 tx=0).stop 2024-09-16T01:57:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.651+0000 7f1e1872c640 1 -- 172.21.15.139:0/811701505 shutdown_connections 2024-09-16T01:57:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.651+0000 7f1e1872c640 1 --2- 172.21.15.139:0/811701505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e10101350 0x7f1e1010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.651+0000 7f1e1872c640 1 --2- 172.21.15.139:0/811701505 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e101009b0 0x7f1e10100e10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.651+0000 7f1e1872c640 1 --2- 172.21.15.139:0/811701505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e101069c0 0x7f1e10106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.651+0000 7f1e1872c640 1 -- 172.21.15.139:0/811701505 >> 172.21.15.139:0/811701505 conn(0x7f1e10077990 msgr2=0x7f1e100fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:00.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.654+0000 7f1e1872c640 1 -- 172.21.15.139:0/811701505 shutdown_connections 2024-09-16T01:57:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.658+0000 7f1e1872c640 1 -- 172.21.15.139:0/811701505 wait complete. 2024-09-16T01:57:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.658+0000 7f1e1872c640 1 Processor -- start 2024-09-16T01:57:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.659+0000 7f1e1872c640 1 -- start start 2024-09-16T01:57:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.659+0000 7f1e1872c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e101009b0 0x7f1e10073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.659+0000 7f1e1872c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e10101350 0x7f1e1006cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.659+0000 7f1e1872c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e101069c0 0x7f1e1006d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.659+0000 7f1e1872c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1e101185d0 con 0x7f1e101069c0 2024-09-16T01:57:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.659+0000 7f1e1872c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e10118450 con 0x7f1e10101350 2024-09-16T01:57:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.659+0000 7f1e1872c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1e10118750 con 0x7f1e101009b0 2024-09-16T01:57:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.660+0000 7f1e15cc7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e10101350 0x7f1e1006cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.660+0000 7f1e15cc7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e10101350 0x7f1e1006cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:42612/0 (socket says 172.21.15.139:42612) 2024-09-16T01:57:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.660+0000 7f1e15cc7640 1 -- 172.21.15.139:0/883196628 learned_addr learned my addr 172.21.15.139:0/883196628 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.663+0000 7f1e164c8640 1 --2- 172.21.15.139:0/883196628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e101009b0 0x7f1e10073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.666+0000 7f1e16cc9640 1 --2- 172.21.15.139:0/883196628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e101069c0 0x7f1e1006d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.668+0000 7f1e15cc7640 1 -- 172.21.15.139:0/883196628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e101009b0 msgr2=0x7f1e10073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.668+0000 7f1e15cc7640 1 --2- 172.21.15.139:0/883196628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e101009b0 0x7f1e10073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.668+0000 7f1e15cc7640 1 -- 172.21.15.139:0/883196628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e101069c0 msgr2=0x7f1e1006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.668+0000 7f1e15cc7640 1 --2- 172.21.15.139:0/883196628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e101069c0 0x7f1e1006d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.668+0000 7f1e15cc7640 1 -- 172.21.15.139:0/883196628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1006d640 con 0x7f1e10101350 2024-09-16T01:57:00.674 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:00 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.674+0000 7f1e15cc7640 1 --2- 172.21.15.139:0/883196628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e10101350 0x7f1e1006cac0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c02ebf0 tx=0x7f1e0c0053c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.676+0000 7f1e077fe640 1 -- 172.21.15.139:0/883196628 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c043070 con 0x7f1e10101350 2024-09-16T01:57:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.677+0000 7f1e077fe640 1 -- 172.21.15.139:0/883196628 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0c003cd0 con 0x7f1e10101350 2024-09-16T01:57:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.677+0000 7f1e077fe640 1 -- 172.21.15.139:0/883196628 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c03ed70 con 0x7f1e10101350 2024-09-16T01:57:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.679+0000 7f1e1872c640 1 -- 172.21.15.139:0/883196628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e1006d8d0 con 0x7f1e10101350 2024-09-16T01:57:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.679+0000 7f1e1872c640 1 -- 172.21.15.139:0/883196628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e10071860 con 0x7f1e10101350 2024-09-16T01:57:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.682+0000 7f1e1b4bf740 1 Processor -- start 2024-09-16T01:57:00.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:00 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:57:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.692+0000 7f1e077fe640 1 -- 172.21.15.139:0/883196628 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1e0c03e470 con 0x7f1e10101350 2024-09-16T01:57:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.693+0000 7f1e1b4bf740 1 -- start start 2024-09-16T01:57:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.698+0000 7f1e077fe640 1 --2- 172.21.15.139:0/883196628 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1dfc0a3690 0x7f1dfc0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.698+0000 7f1e077fe640 1 -- 172.21.15.139:0/883196628 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9694+0+0 (secure 0 0 0) 0x7f1e0c0ef8c0 con 0x7f1e10101350 2024-09-16T01:57:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.699+0000 7f1e1b4bf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d38af0 0x557cf1d38ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.699+0000 7f1e1b4bf740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d39410 0x557cf1d43770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.699+0000 7f1e1b4bf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d43cb0 0x557cf1d460a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.699+0000 7f1e1b4bf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557cf1c9d2f0 con 0x557cf1d43cb0 2024-09-16T01:57:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.699+0000 7f1e1b4bf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557cf1c45270 con 0x557cf1d38af0 2024-09-16T01:57:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.699+0000 7f1e1b4bf740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557cf1c992b0 con 0x557cf1d39410 2024-09-16T01:57:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.700+0000 7f1e16cc9640 1 --2- 172.21.15.139:0/883196628 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1dfc0a3690 0x7f1dfc0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.700+0000 7f1e16cc9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d43cb0 0x557cf1d460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.700+0000 7f1e16cc9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d43cb0 0x557cf1d460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47366/0 (socket says 172.21.15.139:47366) 2024-09-16T01:57:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.700+0000 7f1e16cc9640 1 -- 172.21.15.139:0/2002180839 learned_addr learned my addr 172.21.15.139:0/2002180839 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.702+0000 7f1e15cc7640 1 --2- 172.21.15.139:0/2002180839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d39410 0x557cf1d43770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.707+0000 7f1e164c8640 1 --2- 172.21.15.139:0/2002180839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d38af0 0x557cf1d38ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.708+0000 7f1e16cc9640 1 -- 172.21.15.139:0/2002180839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d39410 msgr2=0x557cf1d43770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.708+0000 7f1e16cc9640 1 --2- 172.21.15.139:0/2002180839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d39410 0x557cf1d43770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.708+0000 7f1e16cc9640 1 -- 172.21.15.139:0/2002180839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d38af0 msgr2=0x557cf1d38ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.708+0000 7f1e16cc9640 1 --2- 172.21.15.139:0/2002180839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d38af0 0x557cf1d38ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.708+0000 7f1e16cc9640 1 -- 172.21.15.139:0/2002180839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cf1c44340 con 0x557cf1d43cb0 2024-09-16T01:57:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.716+0000 7f1e16cc9640 1 --2- 172.21.15.139:0/883196628 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1dfc0a3690 0x7f1dfc0a5b50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e000145f0 tx=0x7f1e0000a8c0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.716+0000 7f1e16cc9640 1 --2- 172.21.15.139:0/2002180839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d43cb0 0x557cf1d460a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1e00014fd0 tx=0x7f1e00019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a2110ef195071b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.722+0000 7f1e04ff9640 1 -- 172.21.15.139:0/2002180839 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00025070 con 0x557cf1d43cb0 2024-09-16T01:57:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.722+0000 7f1e04ff9640 1 -- 172.21.15.139:0/2002180839 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0001c070 con 0x557cf1d43cb0 2024-09-16T01:57:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.722+0000 7f1e04ff9640 1 -- 172.21.15.139:0/2002180839 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00020680 con 0x557cf1d43cb0 2024-09-16T01:57:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.724+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/2002180839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d43cb0 msgr2=0x557cf1d460a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.724+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/2002180839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d43cb0 0x557cf1d460a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1e00014fd0 tx=0x7f1e00019000 comp rx=0 tx=0).stop 2024-09-16T01:57:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.731+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/2002180839 shutdown_connections 2024-09-16T01:57:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.731+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/2002180839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d43cb0 0x557cf1d460a0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.731+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/2002180839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d39410 0x557cf1d43770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.731+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/2002180839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d38af0 0x557cf1d38ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.731+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/2002180839 >> 172.21.15.139:0/2002180839 conn(0x557cf1d35060 msgr2=0x557cf1d35440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.733+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/2002180839 shutdown_connections 2024-09-16T01:57:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.735+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/2002180839 wait complete. 2024-09-16T01:57:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.735+0000 7f1e1b4bf740 1 -- start start 2024-09-16T01:57:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.735+0000 7f1e1b4bf740 1 Processor -- start 2024-09-16T01:57:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.736+0000 7f1e1b4bf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d38af0 0x557cf1d40c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.736+0000 7f1e1b4bf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d39410 0x557cf1d41160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.736+0000 7f1e1b4bf740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d43cb0 0x557cf1d416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.736+0000 7f1e1b4bf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557cf1d46880 con 0x557cf1d38af0 2024-09-16T01:57:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.736+0000 7f1e1b4bf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557cf1c26290 con 0x557cf1d39410 2024-09-16T01:57:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.736+0000 7f1e1b4bf740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557cf1d46a00 con 0x557cf1d43cb0 2024-09-16T01:57:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.736+0000 7f1e1b4bf740 10 client.0 _ll_register_callbacks cb 0x557cf1c57140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:57:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.737+0000 7f1e16cc9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d43cb0 0x557cf1d416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.737+0000 7f1e16cc9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d43cb0 0x557cf1d416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:47654/0 (socket says 172.21.15.139:47654) 2024-09-16T01:57:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.737+0000 7f1e16cc9640 1 -- 172.21.15.139:0/1408920414 learned_addr learned my addr 172.21.15.139:0/1408920414 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.737+0000 7f1e15cc7640 1 --2- 172.21.15.139:0/1408920414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d39410 0x557cf1d41160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.737+0000 7f1e164c8640 1 --2- 172.21.15.139:0/1408920414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d38af0 0x557cf1d40c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.737+0000 7f1e164c8640 1 -- 172.21.15.139:0/1408920414 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d43cb0 msgr2=0x557cf1d416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.737+0000 7f1e164c8640 1 --2- 172.21.15.139:0/1408920414 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d43cb0 0x557cf1d416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.737+0000 7f1e164c8640 1 -- 172.21.15.139:0/1408920414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d39410 msgr2=0x557cf1d41160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.737+0000 7f1e164c8640 1 --2- 172.21.15.139:0/1408920414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d39410 0x557cf1d41160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.737+0000 7f1e164c8640 1 -- 172.21.15.139:0/1408920414 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cf19b0210 con 0x557cf1d38af0 2024-09-16T01:57:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.738+0000 7f1e164c8640 1 --2- 172.21.15.139:0/1408920414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d38af0 0x557cf1d40c20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800ab40 tx=0x7f1e080072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fa8e25492c5690 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.738+0000 7f1df9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:57:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.738+0000 7f1df9ffb640 1 -- 172.21.15.139:0/1408920414 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e080153f0 con 0x557cf1d38af0 2024-09-16T01:57:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.738+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1408920414 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e0001c070 con 0x557cf1d38af0 2024-09-16T01:57:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.739+0000 7f1df9ffb640 1 -- 172.21.15.139:0/1408920414 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9694+0+0 (secure 0 0 0) 0x7f1e08015590 con 0x557cf1d38af0 2024-09-16T01:57:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.739+0000 7f1df9ffb640 1 -- 172.21.15.139:0/1408920414 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f1e08015790 con 0x557cf1d38af0 2024-09-16T01:57:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.739+0000 7f1e1b4bf740 10 client.17193 fetch_fsmap learned FSMap version 71 2024-09-16T01:57:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.739+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1408920414 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x557cf1a7c620 con 0x557cf1d38af0 2024-09-16T01:57:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.740+0000 7f1df9ffb640 1 -- 172.21.15.139:0/1408920414 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0001c070 con 0x557cf1d38af0 2024-09-16T01:57:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.740+0000 7f1df9ffb640 1 -- 172.21.15.139:0/1408920414 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f1e0801ea70 con 0x557cf1d38af0 2024-09-16T01:57:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.741+0000 7f1e1b4bf740 10 client.17193 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:57:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.741+0000 7f1e1b4bf740 10 client.17193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:57:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.741+0000 7f1e1b4bf740 20 client.17193 populate_metadata read hostname 'smithi139' 2024-09-16T01:57:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.741+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/1408920414 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557cf1d540e0 0x557cf1d565a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.741+0000 7f1e1b4bf740 4 client.17193 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:57:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.741+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1408920414 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cf1caaa20 con 0x557cf1d540e0 2024-09-16T01:57:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.743+0000 7f1e15cc7640 1 --2- 172.21.15.139:0/1408920414 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557cf1d540e0 0x557cf1d565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.753+0000 7f1e15cc7640 1 --2- 172.21.15.139:0/1408920414 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557cf1d540e0 0x557cf1d565a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c5532732972c323 server_cookie=de6e16eaa1905594 in_seq=0 out_seq=0 2024-09-16T01:57:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.754+0000 7f1df9ffb640 10 client.17193 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.758+0000 7f1df9ffb640 1 -- 172.21.15.139:0/1408920414 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cf1caaa20 con 0x557cf1d540e0 2024-09-16T01:57:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.758+0000 7f1df9ffb640 10 client.17193 handle_command_reply: tid=0 2024-09-16T01:57:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.758+0000 7f1e1b4bf740 1 client.17193 shutdown 2024-09-16T01:57:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.764+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1408920414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d38af0 msgr2=0x557cf1d40c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.764+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/1408920414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d38af0 0x557cf1d40c20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800ab40 tx=0x7f1e080072b0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.768+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1408920414 shutdown_connections 2024-09-16T01:57:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.768+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/1408920414 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557cf1d540e0 0x557cf1d565a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.768+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/1408920414 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d43cb0 0x557cf1d416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.768+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/1408920414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d39410 0x557cf1d41160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.768+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/1408920414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d38af0 0x557cf1d40c20 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.768+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1408920414 >> 172.21.15.139:0/1408920414 conn(0x557cf1d35060 msgr2=0x557cf1d3adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.768+0000 7f1df9ffb640 0 client.17193 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.769+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1408920414 shutdown_connections 2024-09-16T01:57:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.770+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1408920414 wait complete. 2024-09-16T01:57:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.774+0000 7f1e1b4bf740 20 client.17193 trim_cache size 0 max 16384 2024-09-16T01:57:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.804+0000 7f1e1b4bf740 1 Processor -- start 2024-09-16T01:57:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.804+0000 7f1e1b4bf740 1 -- start start 2024-09-16T01:57:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.804+0000 7f1e1b4bf740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d38af0 0x557cf1d42b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.804+0000 7f1e1b4bf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d39410 0x557cf1d414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.804+0000 7f1e1b4bf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d43cb0 0x557cf1d5beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.804+0000 7f1e1b4bf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1e101c6980 con 0x557cf1d39410 2024-09-16T01:57:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.804+0000 7f1e1b4bf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e101c87c0 con 0x557cf1d43cb0 2024-09-16T01:57:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.804+0000 7f1e1b4bf740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1e101c7110 con 0x557cf1d38af0 2024-09-16T01:57:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.805+0000 7f1e16cc9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d43cb0 0x557cf1d5beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.805+0000 7f1e16cc9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d43cb0 0x557cf1d5beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:42628/0 (socket says 172.21.15.139:42628) 2024-09-16T01:57:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.805+0000 7f1e16cc9640 1 -- 172.21.15.139:0/3986629363 learned_addr learned my addr 172.21.15.139:0/3986629363 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.807+0000 7f1e16cc9640 1 -- 172.21.15.139:0/3986629363 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d38af0 msgr2=0x557cf1d42b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.807+0000 7f1e16cc9640 1 --2- 172.21.15.139:0/3986629363 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d38af0 0x557cf1d42b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.807+0000 7f1e16cc9640 1 -- 172.21.15.139:0/3986629363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d39410 msgr2=0x557cf1d414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.807+0000 7f1e16cc9640 1 --2- 172.21.15.139:0/3986629363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d39410 0x557cf1d414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.807+0000 7f1e16cc9640 1 -- 172.21.15.139:0/3986629363 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cf1d39240 con 0x557cf1d43cb0 2024-09-16T01:57:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.808+0000 7f1e16cc9640 1 --2- 172.21.15.139:0/3986629363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d43cb0 0x557cf1d5beb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1e00006f60 tx=0x7f1e0000a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c90e8f7491c1820 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.809+0000 7f1dfbfff640 1 -- 172.21.15.139:0/3986629363 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00025040 con 0x557cf1d43cb0 2024-09-16T01:57:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.809+0000 7f1dfbfff640 1 -- 172.21.15.139:0/3986629363 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0001c050 con 0x557cf1d43cb0 2024-09-16T01:57:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.810+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/3986629363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d43cb0 msgr2=0x557cf1d5beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.810+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/3986629363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d43cb0 0x557cf1d5beb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1e00006f60 tx=0x7f1e0000a1d0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.817+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/3986629363 shutdown_connections 2024-09-16T01:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.817+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/3986629363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d43cb0 0x557cf1d5beb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.817+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/3986629363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d39410 0x557cf1d414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.817+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/3986629363 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d38af0 0x557cf1d42b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.817+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/3986629363 >> 172.21.15.139:0/3986629363 conn(0x557cf1d35060 msgr2=0x557cf1d42480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.818+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/3986629363 shutdown_connections 2024-09-16T01:57:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.818+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/3986629363 wait complete. 2024-09-16T01:57:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.819+0000 7f1e1b4bf740 1 -- start start 2024-09-16T01:57:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.819+0000 7f1e1b4bf740 1 Processor -- start 2024-09-16T01:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.819+0000 7f1e1b4bf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d38af0 0x557cf1d4bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.819+0000 7f1e1b4bf740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d39410 0x557cf1d4c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.819+0000 7f1e1b4bf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d43cb0 0x557cf1d3d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.819+0000 7f1e1b4bf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1e101c8030 con 0x557cf1d43cb0 2024-09-16T01:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.819+0000 7f1e1b4bf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e101c78a0 con 0x557cf1d38af0 2024-09-16T01:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.819+0000 7f1e1b4bf740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1e101020f0 con 0x557cf1d39410 2024-09-16T01:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.819+0000 7f1e1b4bf740 10 client.0 _ll_register_callbacks cb 0x7f1e08009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.820+0000 7f1e16cc9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d43cb0 0x557cf1d3d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.820+0000 7f1e16cc9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d43cb0 0x557cf1d3d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47384/0 (socket says 172.21.15.139:47384) 2024-09-16T01:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.820+0000 7f1e16cc9640 1 -- 172.21.15.139:0/1250641413 learned_addr learned my addr 172.21.15.139:0/1250641413 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.820+0000 7f1e16cc9640 1 -- 172.21.15.139:0/1250641413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d39410 msgr2=0x557cf1d4c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.820+0000 7f1e16cc9640 1 --2- 172.21.15.139:0/1250641413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d39410 0x557cf1d4c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.820+0000 7f1e16cc9640 1 -- 172.21.15.139:0/1250641413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d38af0 msgr2=0x557cf1d4bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.820+0000 7f1e16cc9640 1 --2- 172.21.15.139:0/1250641413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d38af0 0x557cf1d4bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.820+0000 7f1e16cc9640 1 -- 172.21.15.139:0/1250641413 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cf1d3dce0 con 0x557cf1d43cb0 2024-09-16T01:57:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.820+0000 7f1e16cc9640 1 --2- 172.21.15.139:0/1250641413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d43cb0 0x557cf1d3d4b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000a730 tx=0x7f1e00003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2418eb312fb9cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.821+0000 7f1df9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:57:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.821+0000 7f1df9ffb640 1 -- 172.21.15.139:0/1250641413 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e000041e0 con 0x557cf1d43cb0 2024-09-16T01:57:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.822+0000 7f1df9ffb640 1 -- 172.21.15.139:0/1250641413 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9694+0+0 (secure 0 0 0) 0x7f1e00004380 con 0x557cf1d43cb0 2024-09-16T01:57:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.825+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1250641413 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e0001c050 con 0x557cf1d43cb0 2024-09-16T01:57:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.827+0000 7f1df9ffb640 1 -- 172.21.15.139:0/1250641413 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f1e00004580 con 0x557cf1d43cb0 2024-09-16T01:57:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.829+0000 7f1e1b4bf740 10 client.17196 fetch_fsmap learned FSMap version 71 2024-09-16T01:57:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.829+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1250641413 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x557cf1dbe9d0 con 0x557cf1d43cb0 2024-09-16T01:57:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.834+0000 7f1df9ffb640 1 -- 172.21.15.139:0/1250641413 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0001c050 con 0x557cf1d43cb0 2024-09-16T01:57:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.834+0000 7f1df9ffb640 1 -- 172.21.15.139:0/1250641413 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7f1e00015040 con 0x557cf1d43cb0 2024-09-16T01:57:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.835+0000 7f1e1b4bf740 10 client.17196 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:57:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.835+0000 7f1e1b4bf740 10 client.17196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:57:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.835+0000 7f1e1b4bf740 20 client.17196 populate_metadata read hostname 'smithi139' 2024-09-16T01:57:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.835+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/1250641413 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557cf1dbf040 0x557cf1dc1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.835+0000 7f1e1b4bf740 4 client.17196 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:57:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.835+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1250641413 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557cf1ca4d70 con 0x557cf1dbf040 2024-09-16T01:57:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.836+0000 7f1e164c8640 1 --2- 172.21.15.139:0/1250641413 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557cf1dbf040 0x557cf1dc1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.838+0000 7f1df9ffb640 10 client.17196 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:00.839+0000 7f1e164c8640 1 --2- 172.21.15.139:0/1250641413 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557cf1dbf040 0x557cf1dc1480 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=760fa4aca1cba0b5 server_cookie=905c7062883d4398 in_seq=0 out_seq=0 2024-09-16T01:57:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:01 smithi164 ceph-mon[35121]: pgmap v527: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-16T01:57:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:01 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:01 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:01 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:57:01.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:01 smithi139 ceph-mon[31175]: pgmap v527: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-16T01:57:01.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:01 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:01.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:01 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:01.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:01 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:57:01.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:01 smithi176 ceph-mon[35324]: pgmap v527: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-16T01:57:01.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:01 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:57:01.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:01 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:57:01.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:01 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:57:03.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.122+0000 7f1df9ffb640 1 -- 172.21.15.139:0/1250641413 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557cf1ca4d70 con 0x557cf1dbf040 2024-09-16T01:57:03.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.122+0000 7f1df9ffb640 10 client.17196 handle_command_reply: tid=0 2024-09-16T01:57:03.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.122+0000 7f1e1b4bf740 1 client.17196 shutdown 2024-09-16T01:57:03.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.122+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1250641413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d43cb0 msgr2=0x557cf1d3d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:03.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.122+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/1250641413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d43cb0 0x557cf1d3d4b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000a730 tx=0x7f1e00003b30 comp rx=0 tx=0).stop 2024-09-16T01:57:03.125 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.123+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1250641413 shutdown_connections 2024-09-16T01:57:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.123+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/1250641413 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557cf1dbf040 0x557cf1dc1480 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.123+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/1250641413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557cf1d43cb0 0x557cf1d3d4b0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.123+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/1250641413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557cf1d39410 0x557cf1d4c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.123+0000 7f1df9ffb640 0 client.17196 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.123+0000 7f1e1b4bf740 1 --2- 172.21.15.139:0/1250641413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cf1d38af0 0x557cf1d4bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.123+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1250641413 >> 172.21.15.139:0/1250641413 conn(0x557cf1d35060 msgr2=0x557cf1d341c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.123+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1250641413 shutdown_connections 2024-09-16T01:57:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.123+0000 7f1e1b4bf740 1 -- 172.21.15.139:0/1250641413 wait complete. 2024-09-16T01:57:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.123+0000 7f1e1b4bf740 20 client.17196 trim_cache size 0 max 16384 2024-09-16T01:57:03.126 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:57:03.127 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:57:03.127 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:57:03.127 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:57:03.127 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.124+0000 7f1e1872c640 1 -- 172.21.15.139:0/883196628 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1dfc0a3690 msgr2=0x7f1dfc0a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:03.127 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.124+0000 7f1e1872c640 1 --2- 172.21.15.139:0/883196628 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1dfc0a3690 0x7f1dfc0a5b50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e000145f0 tx=0x7f1e0000a8c0 comp rx=0 tx=0).stop 2024-09-16T01:57:03.127 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.124+0000 7f1e1872c640 1 -- 172.21.15.139:0/883196628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e10101350 msgr2=0x7f1e1006cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:03.128 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.124+0000 7f1e1872c640 1 --2- 172.21.15.139:0/883196628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e10101350 0x7f1e1006cac0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c02ebf0 tx=0x7f1e0c0053c0 comp rx=0 tx=0).stop 2024-09-16T01:57:03.128 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.124+0000 7f1e1872c640 1 -- 172.21.15.139:0/883196628 shutdown_connections 2024-09-16T01:57:03.128 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.124+0000 7f1e1872c640 1 --2- 172.21.15.139:0/883196628 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1dfc0a3690 0x7f1dfc0a5b50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:03.128 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.124+0000 7f1e1872c640 1 --2- 172.21.15.139:0/883196628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e101069c0 0x7f1e1006d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:03.128 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.124+0000 7f1e1872c640 1 --2- 172.21.15.139:0/883196628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e10101350 0x7f1e1006cac0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:03.128 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.124+0000 7f1e1872c640 1 --2- 172.21.15.139:0/883196628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e101009b0 0x7f1e10073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:03.128 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.124+0000 7f1e1872c640 1 -- 172.21.15.139:0/883196628 >> 172.21.15.139:0/883196628 conn(0x7f1e10077990 msgr2=0x7f1e10103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:03.128 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.124+0000 7f1e1872c640 1 -- 172.21.15.139:0/883196628 shutdown_connections 2024-09-16T01:57:03.128 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:03.124+0000 7f1e1872c640 1 -- 172.21.15.139:0/883196628 wait complete. 2024-09-16T01:57:03.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:03 smithi164 ceph-mon[35121]: pgmap v528: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-09-16T01:57:03.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:03 smithi139 ceph-mon[31175]: pgmap v528: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-09-16T01:57:03.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:03 smithi176 ceph-mon[35324]: pgmap v528: 129 pgs: 17 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 59 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-09-16T01:57:05.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:05 smithi164 ceph-mon[35121]: pgmap v529: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 296 op/s 2024-09-16T01:57:05.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:05 smithi139 ceph-mon[31175]: pgmap v529: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 296 op/s 2024-09-16T01:57:05.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:05 smithi176 ceph-mon[35324]: pgmap v529: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 296 op/s 2024-09-16T01:57:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:07 smithi164 ceph-mon[35121]: pgmap v530: 129 pgs: 16 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 64 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 992 KiB/s wr, 285 op/s 2024-09-16T01:57:07.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:07 smithi139 ceph-mon[31175]: pgmap v530: 129 pgs: 16 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 64 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 992 KiB/s wr, 285 op/s 2024-09-16T01:57:07.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:07 smithi176 ceph-mon[35324]: pgmap v530: 129 pgs: 16 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 64 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 992 KiB/s wr, 285 op/s 2024-09-16T01:57:09.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:09 smithi164 ceph-mon[35121]: pgmap v531: 129 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 65 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 339 op/s 2024-09-16T01:57:09.614 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:09 smithi139 ceph-mon[31175]: pgmap v531: 129 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 65 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 339 op/s 2024-09-16T01:57:09.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:09 smithi176 ceph-mon[35324]: pgmap v531: 129 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 65 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 339 op/s 2024-09-16T01:57:11.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:11 smithi164 ceph-mon[35121]: pgmap v532: 129 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 65 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 300 op/s 2024-09-16T01:57:11.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:11 smithi139 ceph-mon[31175]: pgmap v532: 129 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 65 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 300 op/s 2024-09-16T01:57:11.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:11 smithi176 ceph-mon[35324]: pgmap v532: 129 pgs: 15 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 65 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 300 op/s 2024-09-16T01:57:13.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:13 smithi164 ceph-mon[35121]: pgmap v533: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 932 KiB/s wr, 312 op/s 2024-09-16T01:57:13.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:13 smithi139 ceph-mon[31175]: pgmap v533: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 932 KiB/s wr, 312 op/s 2024-09-16T01:57:13.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:13 smithi176 ceph-mon[35324]: pgmap v533: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 932 KiB/s wr, 312 op/s 2024-09-16T01:57:14.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:14 smithi164 ceph-mon[35121]: osdmap e113: 12 total, 12 up, 12 in 2024-09-16T01:57:14.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:14 smithi139 ceph-mon[31175]: osdmap e113: 12 total, 12 up, 12 in 2024-09-16T01:57:14.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:14 smithi176 ceph-mon[35324]: osdmap e113: 12 total, 12 up, 12 in 2024-09-16T01:57:15.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:15 smithi164 ceph-mon[35121]: pgmap v535: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 272 op/s 2024-09-16T01:57:15.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:15 smithi139 ceph-mon[31175]: pgmap v535: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 272 op/s 2024-09-16T01:57:15.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:15 smithi176 ceph-mon[35324]: pgmap v535: 129 pgs: 14 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 73 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 272 op/s 2024-09-16T01:57:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.287+0000 7fec23b91640 1 -- 172.21.15.139:0/4058077881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec1c1095d0 msgr2=0x7fec1c109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.287+0000 7fec23b91640 1 --2- 172.21.15.139:0/4058077881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec1c1095d0 0x7fec1c109a10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009930 tx=0x7fec0c02a460 comp rx=0 tx=0).stop 2024-09-16T01:57:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.295+0000 7fec23b91640 1 -- 172.21.15.139:0/4058077881 shutdown_connections 2024-09-16T01:57:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.295+0000 7fec23b91640 1 --2- 172.21.15.139:0/4058077881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec1c11b140 0x7fec1c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.295+0000 7fec23b91640 1 --2- 172.21.15.139:0/4058077881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec1c1095d0 0x7fec1c109a10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.295+0000 7fec23b91640 1 --2- 172.21.15.139:0/4058077881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec1c1091f0 0x7fec1c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.295+0000 7fec23b91640 1 -- 172.21.15.139:0/4058077881 >> 172.21.15.139:0/4058077881 conn(0x7fec1c06ce00 msgr2=0x7fec1c06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.300+0000 7fec23b91640 1 -- 172.21.15.139:0/4058077881 shutdown_connections 2024-09-16T01:57:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.304+0000 7fec23b91640 1 -- 172.21.15.139:0/4058077881 wait complete. 2024-09-16T01:57:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.304+0000 7fec23b91640 1 Processor -- start 2024-09-16T01:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.310+0000 7fec23b91640 1 -- start start 2024-09-16T01:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.310+0000 7fec23b91640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec1c1091f0 0x7fec1c1a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.310+0000 7fec23b91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec1c1095d0 0x7fec1c1a7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.310+0000 7fec23b91640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec1c11b140 0x7fec1c1abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.310+0000 7fec23b91640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c120dd0 con 0x7fec1c1095d0 2024-09-16T01:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.310+0000 7fec23b91640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c120c50 con 0x7fec1c1091f0 2024-09-16T01:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.310+0000 7fec23b91640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c120f50 con 0x7fec1c11b140 2024-09-16T01:57:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.312+0000 7fec2192d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec1c1091f0 0x7fec1c1a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.314+0000 7fec2112c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec1c1095d0 0x7fec1c1a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.314+0000 7fec2112c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec1c1095d0 0x7fec1c1a7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41790/0 (socket says 172.21.15.139:41790) 2024-09-16T01:57:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.314+0000 7fec2112c640 1 -- 172.21.15.139:0/3074043125 learned_addr learned my addr 172.21.15.139:0/3074043125 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.315+0000 7fec2212e640 1 --2- 172.21.15.139:0/3074043125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec1c11b140 0x7fec1c1abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.315+0000 7fec2212e640 1 -- 172.21.15.139:0/3074043125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec1c1091f0 msgr2=0x7fec1c1a72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.315+0000 7fec2212e640 1 --2- 172.21.15.139:0/3074043125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec1c1091f0 0x7fec1c1a72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.315+0000 7fec2212e640 1 -- 172.21.15.139:0/3074043125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec1c1095d0 msgr2=0x7fec1c1a7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.315+0000 7fec2212e640 1 --2- 172.21.15.139:0/3074043125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec1c1095d0 0x7fec1c1a7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.315+0000 7fec2212e640 1 -- 172.21.15.139:0/3074043125 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c1ac280 con 0x7fec1c11b140 2024-09-16T01:57:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.323+0000 7fec2212e640 1 --2- 172.21.15.139:0/3074043125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec1c11b140 0x7fec1c1abb80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fec1800b440 tx=0x7fec1800b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.326+0000 7fec12ffd640 1 -- 172.21.15.139:0/3074043125 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18015020 con 0x7fec1c11b140 2024-09-16T01:57:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.326+0000 7fec12ffd640 1 -- 172.21.15.139:0/3074043125 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec18011d10 con 0x7fec1c11b140 2024-09-16T01:57:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.326+0000 7fec12ffd640 1 -- 172.21.15.139:0/3074043125 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1800f2e0 con 0x7fec1c11b140 2024-09-16T01:57:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.326+0000 7fec23b91640 1 -- 172.21.15.139:0/3074043125 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1c1ac570 con 0x7fec1c11b140 2024-09-16T01:57:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.326+0000 7fec23b91640 1 -- 172.21.15.139:0/3074043125 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1c110920 con 0x7fec1c11b140 2024-09-16T01:57:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.330+0000 7fec26a0d740 1 Processor -- start 2024-09-16T01:57:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.331+0000 7fec26a0d740 1 -- start start 2024-09-16T01:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.332+0000 7fec26a0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b84eaf0 0x564b6b84eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.332+0000 7fec26a0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84f410 0x564b6b859770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.332+0000 7fec26a0d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b859cb0 0x564b6b85c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.332+0000 7fec26a0d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564b6b7b32f0 con 0x564b6b84f410 2024-09-16T01:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.332+0000 7fec26a0d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564b6b75b270 con 0x564b6b84eaf0 2024-09-16T01:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.332+0000 7fec26a0d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564b6b7af2b0 con 0x564b6b859cb0 2024-09-16T01:57:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.334+0000 7fec2112c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b84eaf0 0x564b6b84eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.334+0000 7fec2112c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b84eaf0 0x564b6b84eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:33626/0 (socket says 172.21.15.139:33626) 2024-09-16T01:57:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.334+0000 7fec2112c640 1 -- 172.21.15.139:0/1152329757 learned_addr learned my addr 172.21.15.139:0/1152329757 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.339+0000 7fec2112c640 1 --2- 172.21.15.139:0/1152329757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b859cb0 0x564b6b85c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.339+0000 7fec12ffd640 1 -- 172.21.15.139:0/3074043125 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fec18018020 con 0x7fec1c11b140 2024-09-16T01:57:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.341+0000 7fec2192d640 1 --2- 172.21.15.139:0/1152329757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84f410 0x564b6b859770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.344+0000 7fec12ffd640 1 --2- 172.21.15.139:0/3074043125 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fec080a3690 0x7fec080a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.344+0000 7fec2112c640 1 -- 172.21.15.139:0/1152329757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b859cb0 msgr2=0x564b6b85c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.344+0000 7fec2112c640 1 --2- 172.21.15.139:0/1152329757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b859cb0 0x564b6b85c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.344+0000 7fec2112c640 1 -- 172.21.15.139:0/1152329757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84f410 msgr2=0x564b6b859770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.344+0000 7fec2112c640 1 --2- 172.21.15.139:0/1152329757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84f410 0x564b6b859770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.344+0000 7fec2112c640 1 -- 172.21.15.139:0/1152329757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b6b75a340 con 0x564b6b84eaf0 2024-09-16T01:57:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.344+0000 7fec2212e640 1 --2- 172.21.15.139:0/3074043125 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fec080a3690 0x7fec080a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.344+0000 7fec2112c640 1 --2- 172.21.15.139:0/1152329757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b84eaf0 0x564b6b84eed0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fec0c004650 tx=0x7fec0c031420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ba7be96b5056f34 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:16.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.345+0000 7fec2212e640 1 --2- 172.21.15.139:0/3074043125 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fec080a3690 0x7fec080a5b50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fec180a3d30 tx=0x7fec180d5240 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.345+0000 7fec12ffd640 1 -- 172.21.15.139:0/3074043125 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9818+0+0 (secure 0 0 0) 0x7fec180cf770 con 0x7fec1c11b140 2024-09-16T01:57:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.347+0000 7fec07fff640 1 -- 172.21.15.139:0/1152329757 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c035030 con 0x564b6b84eaf0 2024-09-16T01:57:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.347+0000 7fec07fff640 1 -- 172.21.15.139:0/1152329757 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec0c0315b0 con 0x564b6b84eaf0 2024-09-16T01:57:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.347+0000 7fec07fff640 1 -- 172.21.15.139:0/1152329757 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c03d6b0 con 0x564b6b84eaf0 2024-09-16T01:57:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.348+0000 7fec26a0d740 1 -- 172.21.15.139:0/1152329757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b84eaf0 msgr2=0x564b6b84eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.348+0000 7fec26a0d740 1 --2- 172.21.15.139:0/1152329757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b84eaf0 0x564b6b84eed0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fec0c004650 tx=0x7fec0c031420 comp rx=0 tx=0).stop 2024-09-16T01:57:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.354+0000 7fec26a0d740 1 -- 172.21.15.139:0/1152329757 shutdown_connections 2024-09-16T01:57:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.354+0000 7fec26a0d740 1 --2- 172.21.15.139:0/1152329757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b859cb0 0x564b6b85c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.354+0000 7fec26a0d740 1 --2- 172.21.15.139:0/1152329757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84f410 0x564b6b859770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.354+0000 7fec26a0d740 1 --2- 172.21.15.139:0/1152329757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b84eaf0 0x564b6b84eed0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.354+0000 7fec26a0d740 1 -- 172.21.15.139:0/1152329757 >> 172.21.15.139:0/1152329757 conn(0x564b6b84b060 msgr2=0x564b6b84b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.363+0000 7fec26a0d740 1 -- 172.21.15.139:0/1152329757 shutdown_connections 2024-09-16T01:57:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.364+0000 7fec26a0d740 1 -- 172.21.15.139:0/1152329757 wait complete. 2024-09-16T01:57:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.364+0000 7fec26a0d740 1 -- start start 2024-09-16T01:57:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.365+0000 7fec26a0d740 1 Processor -- start 2024-09-16T01:57:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.366+0000 7fec26a0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 0x564b6b856bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.366+0000 7fec26a0d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b84f410 0x564b6b857100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.366+0000 7fec26a0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b859cb0 0x564b6b857640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.366+0000 7fec26a0d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564b6b85c880 con 0x564b6b84eaf0 2024-09-16T01:57:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.366+0000 7fec26a0d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564b6b73c290 con 0x564b6b859cb0 2024-09-16T01:57:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.366+0000 7fec26a0d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564b6b85ca00 con 0x564b6b84f410 2024-09-16T01:57:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.366+0000 7fec26a0d740 10 client.0 _ll_register_callbacks cb 0x564b6b76d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:57:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.367+0000 7fec2112c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b859cb0 0x564b6b857640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.367+0000 7fec2112c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b859cb0 0x564b6b857640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:33638/0 (socket says 172.21.15.139:33638) 2024-09-16T01:57:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.367+0000 7fec2112c640 1 -- 172.21.15.139:0/3199836872 learned_addr learned my addr 172.21.15.139:0/3199836872 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.367+0000 7fec2112c640 1 --2- 172.21.15.139:0/3199836872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 0x564b6b856bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.371+0000 7fec2192d640 1 --2- 172.21.15.139:0/3199836872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b84f410 0x564b6b857100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.372+0000 7fec2112c640 1 -- 172.21.15.139:0/3199836872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b84f410 msgr2=0x564b6b857100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.372+0000 7fec2112c640 1 --2- 172.21.15.139:0/3199836872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b84f410 0x564b6b857100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.372+0000 7fec2112c640 1 -- 172.21.15.139:0/3199836872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b859cb0 msgr2=0x564b6b857640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.372+0000 7fec2112c640 1 --2- 172.21.15.139:0/3199836872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b859cb0 0x564b6b857640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.372+0000 7fec2112c640 1 -- 172.21.15.139:0/3199836872 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b6b4c6210 con 0x564b6b84eaf0 2024-09-16T01:57:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.374+0000 7fec2112c640 1 --2- 172.21.15.139:0/3199836872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 0x564b6b856bc0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0314b0 tx=0x7fec0c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36248832586c40f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.375+0000 7fec057fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:57:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.375+0000 7fec057fa640 1 -- 172.21.15.139:0/3199836872 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c035040 con 0x564b6b84eaf0 2024-09-16T01:57:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.375+0000 7fec26a0d740 1 -- 172.21.15.139:0/3199836872 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec0c0315b0 con 0x564b6b84eaf0 2024-09-16T01:57:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.375+0000 7fec057fa640 1 -- 172.21.15.139:0/3199836872 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9818+0+0 (secure 0 0 0) 0x7fec0c031a20 con 0x564b6b84eaf0 2024-09-16T01:57:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.375+0000 7fec057fa640 1 -- 172.21.15.139:0/3199836872 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fec0c057050 con 0x564b6b84eaf0 2024-09-16T01:57:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.377+0000 7fec26a0d740 10 client.17205 fetch_fsmap learned FSMap version 71 2024-09-16T01:57:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.377+0000 7fec26a0d740 1 -- 172.21.15.139:0/3199836872 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x564b6b592620 con 0x564b6b84eaf0 2024-09-16T01:57:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.379+0000 7fec057fa640 1 -- 172.21.15.139:0/3199836872 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec0c0315b0 con 0x564b6b84eaf0 2024-09-16T01:57:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.379+0000 7fec057fa640 1 -- 172.21.15.139:0/3199836872 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7fec0c004a40 con 0x564b6b84eaf0 2024-09-16T01:57:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.379+0000 7fec26a0d740 10 client.17205 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:57:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.379+0000 7fec26a0d740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:57:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.379+0000 7fec26a0d740 20 client.17205 populate_metadata read hostname 'smithi139' 2024-09-16T01:57:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.379+0000 7fec26a0d740 1 --2- 172.21.15.139:0/3199836872 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564b6b869ed0 0x564b6b86c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.379+0000 7fec26a0d740 4 client.17205 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:57:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.379+0000 7fec26a0d740 1 -- 172.21.15.139:0/3199836872 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b6b7c0a20 con 0x564b6b869ed0 2024-09-16T01:57:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.382+0000 7fec2192d640 1 --2- 172.21.15.139:0/3199836872 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564b6b869ed0 0x564b6b86c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.386+0000 7fec2192d640 1 --2- 172.21.15.139:0/3199836872 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564b6b869ed0 0x564b6b86c350 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b17dbf76a755bf01 server_cookie=3fde49db360d70c3 in_seq=0 out_seq=0 2024-09-16T01:57:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.388+0000 7fec057fa640 10 client.17205 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.391+0000 7fec057fa640 1 -- 172.21.15.139:0/3199836872 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b6b7c0a20 con 0x564b6b869ed0 2024-09-16T01:57:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.391+0000 7fec057fa640 10 client.17205 handle_command_reply: tid=0 2024-09-16T01:57:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.391+0000 7fec26a0d740 1 client.17205 shutdown 2024-09-16T01:57:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.397+0000 7fec26a0d740 1 -- 172.21.15.139:0/3199836872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 msgr2=0x564b6b856bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.397+0000 7fec26a0d740 1 --2- 172.21.15.139:0/3199836872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 0x564b6b856bc0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0314b0 tx=0x7fec0c005fd0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.401+0000 7fec26a0d740 1 -- 172.21.15.139:0/3199836872 shutdown_connections 2024-09-16T01:57:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.401+0000 7fec26a0d740 1 --2- 172.21.15.139:0/3199836872 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564b6b869ed0 0x564b6b86c350 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.401+0000 7fec26a0d740 1 --2- 172.21.15.139:0/3199836872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b859cb0 0x564b6b857640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.401+0000 7fec26a0d740 1 --2- 172.21.15.139:0/3199836872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b84f410 0x564b6b857100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.401+0000 7fec26a0d740 1 --2- 172.21.15.139:0/3199836872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 0x564b6b856bc0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.401+0000 7fec26a0d740 1 -- 172.21.15.139:0/3199836872 >> 172.21.15.139:0/3199836872 conn(0x564b6b84b060 msgr2=0x564b6b84a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.401+0000 7fec057fa640 0 client.17205 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.406+0000 7fec26a0d740 1 -- 172.21.15.139:0/3199836872 shutdown_connections 2024-09-16T01:57:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.409+0000 7fec26a0d740 1 -- 172.21.15.139:0/3199836872 wait complete. 2024-09-16T01:57:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.409+0000 7fec26a0d740 20 client.17205 trim_cache size 0 max 16384 2024-09-16T01:57:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.442+0000 7fec26a0d740 1 Processor -- start 2024-09-16T01:57:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.444+0000 7fec26a0d740 1 -- start start 2024-09-16T01:57:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.445+0000 7fec26a0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 0x564b6b857470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.445+0000 7fec26a0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b84f410 0x564b6b871c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.445+0000 7fec26a0d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b859cb0 0x564b6b872160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.445+0000 7fec26a0d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fec00010360 con 0x564b6b84eaf0 2024-09-16T01:57:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.445+0000 7fec26a0d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fec000121a0 con 0x564b6b84f410 2024-09-16T01:57:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.445+0000 7fec26a0d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec00010af0 con 0x564b6b859cb0 2024-09-16T01:57:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.447+0000 7fec2192d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b84f410 0x564b6b871c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.447+0000 7fec2192d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b84f410 0x564b6b871c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:33654/0 (socket says 172.21.15.139:33654) 2024-09-16T01:57:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.447+0000 7fec2192d640 1 -- 172.21.15.139:0/2422139629 learned_addr learned my addr 172.21.15.139:0/2422139629 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.449+0000 7fec2112c640 1 --2- 172.21.15.139:0/2422139629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b859cb0 0x564b6b872160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.450+0000 7fec2112c640 1 --2- 172.21.15.139:0/2422139629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 0x564b6b857470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.451+0000 7fec2192d640 1 -- 172.21.15.139:0/2422139629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b859cb0 msgr2=0x564b6b872160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.451+0000 7fec2192d640 1 --2- 172.21.15.139:0/2422139629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b859cb0 0x564b6b872160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.451+0000 7fec2192d640 1 -- 172.21.15.139:0/2422139629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 msgr2=0x564b6b857470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.451+0000 7fec2192d640 1 --2- 172.21.15.139:0/2422139629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 0x564b6b857470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.451+0000 7fec2192d640 1 -- 172.21.15.139:0/2422139629 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b6b84e920 con 0x564b6b84f410 2024-09-16T01:57:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.455+0000 7fec2112c640 1 --2- 172.21.15.139:0/2422139629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 0x564b6b857470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:57:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.456+0000 7fec2192d640 1 --2- 172.21.15.139:0/2422139629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b84f410 0x564b6b871c20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fec14000f80 tx=0x7fec140147c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4907ebd929fb893c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.459+0000 7fec077fe640 1 -- 172.21.15.139:0/2422139629 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14021020 con 0x564b6b84f410 2024-09-16T01:57:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.459+0000 7fec077fe640 1 -- 172.21.15.139:0/2422139629 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec1400c050 con 0x564b6b84f410 2024-09-16T01:57:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.460+0000 7fec26a0d740 1 -- 172.21.15.139:0/2422139629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b84f410 msgr2=0x564b6b871c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.460+0000 7fec26a0d740 1 --2- 172.21.15.139:0/2422139629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b84f410 0x564b6b871c20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fec14000f80 tx=0x7fec140147c0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.462+0000 7fec26a0d740 1 -- 172.21.15.139:0/2422139629 shutdown_connections 2024-09-16T01:57:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.462+0000 7fec26a0d740 1 --2- 172.21.15.139:0/2422139629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b859cb0 0x564b6b872160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.462+0000 7fec26a0d740 1 --2- 172.21.15.139:0/2422139629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b84f410 0x564b6b871c20 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fec14000f80 tx=0x7fec140147c0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.462+0000 7fec26a0d740 1 --2- 172.21.15.139:0/2422139629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 0x564b6b857470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.462+0000 7fec26a0d740 1 -- 172.21.15.139:0/2422139629 >> 172.21.15.139:0/2422139629 conn(0x564b6b84b060 msgr2=0x564b6b855c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.464+0000 7fec26a0d740 1 -- 172.21.15.139:0/2422139629 shutdown_connections 2024-09-16T01:57:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.464+0000 7fec26a0d740 1 -- 172.21.15.139:0/2422139629 wait complete. 2024-09-16T01:57:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.464+0000 7fec26a0d740 1 -- start start 2024-09-16T01:57:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.464+0000 7fec26a0d740 1 Processor -- start 2024-09-16T01:57:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.465+0000 7fec26a0d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 0x564b6b85d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.465+0000 7fec26a0d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b859cb0 0x564b6b85d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.465+0000 7fec26a0d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b8d0bd0 0x564b6b8d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.465+0000 7fec26a0d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fec00011a10 con 0x564b6b84eaf0 2024-09-16T01:57:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.465+0000 7fec26a0d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fec00011280 con 0x564b6b859cb0 2024-09-16T01:57:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.465+0000 7fec26a0d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec0000e360 con 0x564b6b8d0bd0 2024-09-16T01:57:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.465+0000 7fec26a0d740 10 client.0 _ll_register_callbacks cb 0x7fec0c003810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:57:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.469+0000 7fec2112c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b8d0bd0 0x564b6b8d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.469+0000 7fec2112c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b8d0bd0 0x564b6b8d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:34446/0 (socket says 172.21.15.139:34446) 2024-09-16T01:57:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.469+0000 7fec2112c640 1 -- 172.21.15.139:0/1669363417 learned_addr learned my addr 172.21.15.139:0/1669363417 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.469+0000 7fec2112c640 1 --2- 172.21.15.139:0/1669363417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 0x564b6b85d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.471+0000 7fec2192d640 1 --2- 172.21.15.139:0/1669363417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b859cb0 0x564b6b85d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.472+0000 7fec2112c640 1 -- 172.21.15.139:0/1669363417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b859cb0 msgr2=0x564b6b85d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.472+0000 7fec2112c640 1 --2- 172.21.15.139:0/1669363417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b859cb0 0x564b6b85d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.472+0000 7fec2112c640 1 -- 172.21.15.139:0/1669363417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 msgr2=0x564b6b85d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.472+0000 7fec2112c640 1 --2- 172.21.15.139:0/1669363417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 0x564b6b85d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.472+0000 7fec2112c640 1 -- 172.21.15.139:0/1669363417 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b6b8d3500 con 0x564b6b8d0bd0 2024-09-16T01:57:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.479+0000 7fec2112c640 1 --2- 172.21.15.139:0/1669363417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b8d0bd0 0x564b6b8d2fc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fec0c001eb0 tx=0x7fec0c04b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96f3fb46fc1e5de8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.480+0000 7fec057fa640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:57:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.483+0000 7fec057fa640 1 -- 172.21.15.139:0/1669363417 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c035050 con 0x564b6b8d0bd0 2024-09-16T01:57:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.483+0000 7fec057fa640 1 -- 172.21.15.139:0/1669363417 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9818+0+0 (secure 0 0 0) 0x7fec0c004bb0 con 0x564b6b8d0bd0 2024-09-16T01:57:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.484+0000 7fec26a0d740 1 -- 172.21.15.139:0/1669363417 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564b6b7bad70 con 0x564b6b8d0bd0 2024-09-16T01:57:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.490+0000 7fec057fa640 1 -- 172.21.15.139:0/1669363417 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fec0c009520 con 0x564b6b8d0bd0 2024-09-16T01:57:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.490+0000 7fec26a0d740 10 client.26608 fetch_fsmap learned FSMap version 71 2024-09-16T01:57:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.490+0000 7fec26a0d740 1 -- 172.21.15.139:0/1669363417 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x564b6b8d47c0 con 0x564b6b8d0bd0 2024-09-16T01:57:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.491+0000 7fec057fa640 1 -- 172.21.15.139:0/1669363417 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564b6b7bad70 con 0x564b6b8d0bd0 2024-09-16T01:57:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.491+0000 7fec057fa640 1 -- 172.21.15.139:0/1669363417 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 71) ==== 8450+0+0 (secure 0 0 0) 0x7fec0c04ca40 con 0x564b6b8d0bd0 2024-09-16T01:57:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.491+0000 7fec26a0d740 10 client.26608 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T01:57:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.491+0000 7fec26a0d740 10 client.26608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:57:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.491+0000 7fec26a0d740 20 client.26608 populate_metadata read hostname 'smithi139' 2024-09-16T01:57:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.491+0000 7fec26a0d740 1 --2- 172.21.15.139:0/1669363417 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564b6b8d4db0 0x564b6b8d71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.491+0000 7fec26a0d740 4 client.26608 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:57:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.491+0000 7fec26a0d740 1 -- 172.21.15.139:0/1669363417 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b6b737900 con 0x564b6b8d4db0 2024-09-16T01:57:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.499+0000 7fec2192d640 1 --2- 172.21.15.139:0/1669363417 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564b6b8d4db0 0x564b6b8d71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.507+0000 7fec2192d640 1 --2- 172.21.15.139:0/1669363417 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564b6b8d4db0 0x564b6b8d71d0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eae9cd720bba7c0d server_cookie=29ee9f59ba858aae in_seq=0 out_seq=0 2024-09-16T01:57:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.508+0000 7fec057fa640 10 client.26608 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.653+0000 7fec057fa640 1 -- 172.21.15.139:0/1669363417 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b6b737900 con 0x564b6b8d4db0 2024-09-16T01:57:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.653+0000 7fec057fa640 10 client.26608 handle_command_reply: tid=0 2024-09-16T01:57:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.653+0000 7fec26a0d740 1 client.26608 shutdown 2024-09-16T01:57:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.654+0000 7fec26a0d740 1 -- 172.21.15.139:0/1669363417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b8d0bd0 msgr2=0x564b6b8d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.654+0000 7fec26a0d740 1 --2- 172.21.15.139:0/1669363417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b8d0bd0 0x564b6b8d2fc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fec0c001eb0 tx=0x7fec0c04b040 comp rx=0 tx=0).stop 2024-09-16T01:57:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.654+0000 7fec26a0d740 1 -- 172.21.15.139:0/1669363417 shutdown_connections 2024-09-16T01:57:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.654+0000 7fec26a0d740 1 --2- 172.21.15.139:0/1669363417 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564b6b8d4db0 0x564b6b8d71d0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.654+0000 7fec26a0d740 1 --2- 172.21.15.139:0/1669363417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564b6b8d0bd0 0x564b6b8d2fc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.654+0000 7fec26a0d740 1 --2- 172.21.15.139:0/1669363417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b6b859cb0 0x564b6b85d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.654+0000 7fec26a0d740 1 --2- 172.21.15.139:0/1669363417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564b6b84eaf0 0x564b6b85d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.654+0000 7fec26a0d740 1 -- 172.21.15.139:0/1669363417 >> 172.21.15.139:0/1669363417 conn(0x564b6b84b060 msgr2=0x564b6b8499a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.654+0000 7fec057fa640 0 client.26608 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.654+0000 7fec26a0d740 1 -- 172.21.15.139:0/1669363417 shutdown_connections 2024-09-16T01:57:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.654+0000 7fec26a0d740 1 -- 172.21.15.139:0/1669363417 wait complete. 2024-09-16T01:57:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.654+0000 7fec26a0d740 20 client.26608 trim_cache size 0 max 16384 2024-09-16T01:57:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:57:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:57:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:57:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:57:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.655+0000 7fec23b91640 1 -- 172.21.15.139:0/3074043125 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fec080a3690 msgr2=0x7fec080a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.655+0000 7fec23b91640 1 --2- 172.21.15.139:0/3074043125 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fec080a3690 0x7fec080a5b50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fec180a3d30 tx=0x7fec180d5240 comp rx=0 tx=0).stop 2024-09-16T01:57:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.655+0000 7fec23b91640 1 -- 172.21.15.139:0/3074043125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec1c11b140 msgr2=0x7fec1c1abb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.655+0000 7fec23b91640 1 --2- 172.21.15.139:0/3074043125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec1c11b140 0x7fec1c1abb80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fec1800b440 tx=0x7fec1800b900 comp rx=0 tx=0).stop 2024-09-16T01:57:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.655+0000 7fec23b91640 1 -- 172.21.15.139:0/3074043125 shutdown_connections 2024-09-16T01:57:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.655+0000 7fec23b91640 1 --2- 172.21.15.139:0/3074043125 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fec080a3690 0x7fec080a5b50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.655+0000 7fec23b91640 1 --2- 172.21.15.139:0/3074043125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec1c11b140 0x7fec1c1abb80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.655+0000 7fec23b91640 1 --2- 172.21.15.139:0/3074043125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec1c1095d0 0x7fec1c1a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.655+0000 7fec23b91640 1 --2- 172.21.15.139:0/3074043125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec1c1091f0 0x7fec1c1a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.655+0000 7fec23b91640 1 -- 172.21.15.139:0/3074043125 >> 172.21.15.139:0/3074043125 conn(0x7fec1c06ce00 msgr2=0x7fec1c11b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.655+0000 7fec23b91640 1 -- 172.21.15.139:0/3074043125 shutdown_connections 2024-09-16T01:57:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:16.655+0000 7fec23b91640 1 -- 172.21.15.139:0/3074043125 wait complete. 2024-09-16T01:57:17.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:17 smithi164 ceph-mon[35121]: pgmap v536: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 278 op/s 2024-09-16T01:57:17.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:17 smithi139 ceph-mon[31175]: pgmap v536: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 278 op/s 2024-09-16T01:57:17.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:17 smithi176 ceph-mon[35324]: pgmap v536: 129 pgs: 16 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 68 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 278 op/s 2024-09-16T01:57:19.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:19 smithi139 ceph-mon[31175]: pgmap v537: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 262 op/s 2024-09-16T01:57:19.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:19 smithi176 ceph-mon[35324]: pgmap v537: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 262 op/s 2024-09-16T01:57:19.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:19 smithi164 ceph-mon[35121]: pgmap v537: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 262 op/s 2024-09-16T01:57:20.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:20 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:57:20.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:20 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:57:20.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:20 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:57:21.587 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:21 smithi176 ceph-mon[35324]: pgmap v538: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 255 op/s 2024-09-16T01:57:21.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:21 smithi139 ceph-mon[31175]: pgmap v538: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 255 op/s 2024-09-16T01:57:21.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:21 smithi164 ceph-mon[35121]: pgmap v538: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 255 op/s 2024-09-16T01:57:23.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:23 smithi139 ceph-mon[31175]: pgmap v539: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 233 op/s 2024-09-16T01:57:23.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:23 smithi176 ceph-mon[35324]: pgmap v539: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 233 op/s 2024-09-16T01:57:23.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:23 smithi164 ceph-mon[35121]: pgmap v539: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 233 op/s 2024-09-16T01:57:24.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:57:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:57:24.268+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:57:24.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:57:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:57:24.269+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:57:25.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:25 smithi139 ceph-mon[31175]: pgmap v540: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-09-16T01:57:25.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:25 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:57:25.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:25 smithi176 ceph-mon[35324]: pgmap v540: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-09-16T01:57:25.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:25 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:57:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:25 smithi164 ceph-mon[35121]: pgmap v540: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-09-16T01:57:25.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:25 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:57:27.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:27 smithi139 ceph-mon[31175]: pgmap v541: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-09-16T01:57:27.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:27 smithi176 ceph-mon[35324]: pgmap v541: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-09-16T01:57:27.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:27 smithi164 ceph-mon[35121]: pgmap v541: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-09-16T01:57:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:29 smithi139 ceph-mon[31175]: pgmap v542: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1007 KiB/s wr, 197 op/s 2024-09-16T01:57:29.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:29 smithi176 ceph-mon[35324]: pgmap v542: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1007 KiB/s wr, 197 op/s 2024-09-16T01:57:29.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:29 smithi164 ceph-mon[35121]: pgmap v542: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1007 KiB/s wr, 197 op/s 2024-09-16T01:57:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.813+0000 7f013f386640 1 -- 172.21.15.139:0/2212500314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01381069c0 msgr2=0x7f0138106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.814+0000 7f013f386640 1 --2- 172.21.15.139:0/2212500314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01381069c0 0x7f0138106da0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0128009930 tx=0x7f012802a480 comp rx=0 tx=0).stop 2024-09-16T01:57:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.819+0000 7f013f386640 1 -- 172.21.15.139:0/2212500314 shutdown_connections 2024-09-16T01:57:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.819+0000 7f013f386640 1 --2- 172.21.15.139:0/2212500314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0138101350 0x7f013810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.819+0000 7f013f386640 1 --2- 172.21.15.139:0/2212500314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01381009b0 0x7f0138100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.819+0000 7f013f386640 1 --2- 172.21.15.139:0/2212500314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01381069c0 0x7f0138106da0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.819+0000 7f013f386640 1 -- 172.21.15.139:0/2212500314 >> 172.21.15.139:0/2212500314 conn(0x7f0138077990 msgr2=0x7f01380fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.823+0000 7f013f386640 1 -- 172.21.15.139:0/2212500314 shutdown_connections 2024-09-16T01:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.824+0000 7f013f386640 1 -- 172.21.15.139:0/2212500314 wait complete. 2024-09-16T01:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.824+0000 7f013f386640 1 Processor -- start 2024-09-16T01:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.828+0000 7f013f386640 1 -- start start 2024-09-16T01:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.828+0000 7f013f386640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01381009b0 0x7f013810d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.828+0000 7f013f386640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0138101350 0x7f013810d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.828+0000 7f013f386640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01381069c0 0x7f0138073e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.828+0000 7f013f386640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f013810f2f0 con 0x7f01381009b0 2024-09-16T01:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.828+0000 7f013f386640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f013810f170 con 0x7f0138101350 2024-09-16T01:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.828+0000 7f013f386640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f013810f470 con 0x7f01381069c0 2024-09-16T01:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.829+0000 7f013c921640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0138101350 0x7f013810d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.832+0000 7f013d923640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01381069c0 0x7f0138073e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.832+0000 7f013d122640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01381009b0 0x7f013810d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.832+0000 7f013d122640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01381009b0 0x7f013810d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47642/0 (socket says 172.21.15.139:47642) 2024-09-16T01:57:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.832+0000 7f013d122640 1 -- 172.21.15.139:0/751117239 learned_addr learned my addr 172.21.15.139:0/751117239 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.832+0000 7f013c921640 1 -- 172.21.15.139:0/751117239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01381069c0 msgr2=0x7f0138073e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.832+0000 7f013c921640 1 --2- 172.21.15.139:0/751117239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01381069c0 0x7f0138073e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.833+0000 7f013c921640 1 -- 172.21.15.139:0/751117239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01381009b0 msgr2=0x7f013810d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.833+0000 7f013c921640 1 --2- 172.21.15.139:0/751117239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01381009b0 0x7f013810d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.833+0000 7f013c921640 1 -- 172.21.15.139:0/751117239 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013810b670 con 0x7f0138101350 2024-09-16T01:57:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.833+0000 7f013c921640 1 --2- 172.21.15.139:0/751117239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0138101350 0x7f013810d7f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f01340097d0 tx=0x7f0134009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.835+0000 7f012e7fc640 1 -- 172.21.15.139:0/751117239 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134018030 con 0x7f0138101350 2024-09-16T01:57:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.835+0000 7f012e7fc640 1 -- 172.21.15.139:0/751117239 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f013400eb80 con 0x7f0138101350 2024-09-16T01:57:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.835+0000 7f012e7fc640 1 -- 172.21.15.139:0/751117239 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134014dc0 con 0x7f0138101350 2024-09-16T01:57:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.837+0000 7f013f386640 1 -- 172.21.15.139:0/751117239 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013810b960 con 0x7f0138101350 2024-09-16T01:57:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.837+0000 7f013f386640 1 -- 172.21.15.139:0/751117239 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013810be30 con 0x7f0138101350 2024-09-16T01:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.843+0000 7f01421f7740 1 Processor -- start 2024-09-16T01:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.846+0000 7f012e7fc640 1 -- 172.21.15.139:0/751117239 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f013401b070 con 0x7f0138101350 2024-09-16T01:57:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.847+0000 7f012e7fc640 1 --2- 172.21.15.139:0/751117239 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f01240a35c0 0x7f01240a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.847+0000 7f01421f7740 1 -- start start 2024-09-16T01:57:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.847+0000 7f01421f7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bff1af0 0x564d4bff1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.847+0000 7f01421f7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bff2410 0x564d4bffc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.847+0000 7f01421f7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bffccb0 0x564d4bfff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.847+0000 7f01421f7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564d4bf562f0 con 0x564d4bff1af0 2024-09-16T01:57:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.847+0000 7f01421f7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564d4befe270 con 0x564d4bff2410 2024-09-16T01:57:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.847+0000 7f01421f7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564d4bf522b0 con 0x564d4bffccb0 2024-09-16T01:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.848+0000 7f012e7fc640 1 -- 172.21.15.139:0/751117239 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9818+0+0 (secure 0 0 0) 0x7f01340991b0 con 0x7f0138101350 2024-09-16T01:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.849+0000 7f013d923640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bffccb0 0x564d4bfff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.849+0000 7f013d923640 1 --2- 172.21.15.139:0/751117239 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f01240a35c0 0x7f01240a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.849+0000 7f013d923640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bffccb0 0x564d4bfff0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:53176/0 (socket says 172.21.15.139:53176) 2024-09-16T01:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.849+0000 7f013d923640 1 -- 172.21.15.139:0/2689344597 learned_addr learned my addr 172.21.15.139:0/2689344597 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.849+0000 7f013c921640 1 --2- 172.21.15.139:0/2689344597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bff2410 0x564d4bffc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.850+0000 7f013d923640 1 -- 172.21.15.139:0/2689344597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bff2410 msgr2=0x564d4bffc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.850+0000 7f013d923640 1 --2- 172.21.15.139:0/2689344597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bff2410 0x564d4bffc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.850+0000 7f013d923640 1 -- 172.21.15.139:0/2689344597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bff1af0 msgr2=0x564d4bff1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.850+0000 7f013d923640 1 --2- 172.21.15.139:0/2689344597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bff1af0 0x564d4bff1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.850+0000 7f013d923640 1 -- 172.21.15.139:0/2689344597 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4befd340 con 0x564d4bffccb0 2024-09-16T01:57:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.852+0000 7f013d923640 1 --2- 172.21.15.139:0/2689344597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bffccb0 0x564d4bfff0a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01300147e0 tx=0x7f0130014ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17300d2d4aa27117 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.853+0000 7f01237fe640 1 -- 172.21.15.139:0/2689344597 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130023070 con 0x564d4bffccb0 2024-09-16T01:57:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.853+0000 7f01237fe640 1 -- 172.21.15.139:0/2689344597 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01300141b0 con 0x564d4bffccb0 2024-09-16T01:57:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.853+0000 7f013d923640 1 --2- 172.21.15.139:0/751117239 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f01240a35c0 0x7f01240a5a80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0130014000 tx=0x7f013001b000 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.855+0000 7f01421f7740 1 -- 172.21.15.139:0/2689344597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bffccb0 msgr2=0x564d4bfff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.855+0000 7f01421f7740 1 --2- 172.21.15.139:0/2689344597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bffccb0 0x564d4bfff0a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01300147e0 tx=0x7f0130014ca0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.862+0000 7f01421f7740 1 -- 172.21.15.139:0/2689344597 shutdown_connections 2024-09-16T01:57:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.862+0000 7f01421f7740 1 --2- 172.21.15.139:0/2689344597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bffccb0 0x564d4bfff0a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.862+0000 7f01421f7740 1 --2- 172.21.15.139:0/2689344597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bff2410 0x564d4bffc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.862+0000 7f01421f7740 1 --2- 172.21.15.139:0/2689344597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bff1af0 0x564d4bff1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.862+0000 7f01421f7740 1 -- 172.21.15.139:0/2689344597 >> 172.21.15.139:0/2689344597 conn(0x564d4bfee060 msgr2=0x564d4bfee440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.865+0000 7f01421f7740 1 -- 172.21.15.139:0/2689344597 shutdown_connections 2024-09-16T01:57:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.865+0000 7f01421f7740 1 -- 172.21.15.139:0/2689344597 wait complete. 2024-09-16T01:57:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.866+0000 7f01421f7740 1 -- start start 2024-09-16T01:57:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.866+0000 7f01421f7740 1 Processor -- start 2024-09-16T01:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.867+0000 7f01421f7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bff1af0 0x564d4bff9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.867+0000 7f01421f7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4bffa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.867+0000 7f01421f7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bffccb0 0x564d4bffa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.867+0000 7f01421f7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564d4bfff8c0 con 0x564d4bffccb0 2024-09-16T01:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.867+0000 7f01421f7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564d4bedf290 con 0x564d4bff1af0 2024-09-16T01:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.867+0000 7f01421f7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564d4bfffa40 con 0x564d4bff2410 2024-09-16T01:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.867+0000 7f01421f7740 10 client.0 _ll_register_callbacks cb 0x564d4bf10140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.867+0000 7f013c921640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4bffa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.867+0000 7f013c921640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4bffa140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:53186/0 (socket says 172.21.15.139:53186) 2024-09-16T01:57:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.867+0000 7f013c921640 1 -- 172.21.15.139:0/1425205225 learned_addr learned my addr 172.21.15.139:0/1425205225 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.867+0000 7f013d923640 1 --2- 172.21.15.139:0/1425205225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bffccb0 0x564d4bffa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.868+0000 7f013c921640 1 -- 172.21.15.139:0/1425205225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bff1af0 msgr2=0x564d4bff9c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:57:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.868+0000 7f013c921640 1 --2- 172.21.15.139:0/1425205225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bff1af0 0x564d4bff9c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.868+0000 7f013c921640 1 -- 172.21.15.139:0/1425205225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bffccb0 msgr2=0x564d4bffa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.868+0000 7f013c921640 1 --2- 172.21.15.139:0/1425205225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bffccb0 0x564d4bffa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.868+0000 7f013c921640 1 -- 172.21.15.139:0/1425205225 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4bc69210 con 0x564d4bff2410 2024-09-16T01:57:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.869+0000 7f013c921640 1 --2- 172.21.15.139:0/1425205225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4bffa140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f013409ae10 tx=0x7f01340146c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=584880a39eca3aea server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.872+0000 7f0120ff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:57:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.876+0000 7f0120ff9640 1 -- 172.21.15.139:0/1425205225 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01340a3d80 con 0x564d4bff2410 2024-09-16T01:57:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.876+0000 7f0120ff9640 1 -- 172.21.15.139:0/1425205225 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9818+0+0 (secure 0 0 0) 0x7f0134019d50 con 0x564d4bff2410 2024-09-16T01:57:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.880+0000 7f01421f7740 1 -- 172.21.15.139:0/1425205225 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01300141b0 con 0x564d4bff2410 2024-09-16T01:57:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.883+0000 7f0120ff9640 1 -- 172.21.15.139:0/1425205225 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f013409f150 con 0x564d4bff2410 2024-09-16T01:57:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.885+0000 7f01421f7740 10 client.26617 fetch_fsmap learned FSMap version 74 2024-09-16T01:57:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.885+0000 7f01421f7740 1 -- 172.21.15.139:0/1425205225 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x564d4bd35620 con 0x564d4bff2410 2024-09-16T01:57:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.886+0000 7f0120ff9640 1 -- 172.21.15.139:0/1425205225 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01300141b0 con 0x564d4bff2410 2024-09-16T01:57:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.887+0000 7f0120ff9640 1 -- 172.21.15.139:0/1425205225 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 74) ==== 8446+0+0 (secure 0 0 0) 0x7f013409a560 con 0x564d4bff2410 2024-09-16T01:57:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.888+0000 7f01421f7740 10 client.26617 fetch_fsmap finished waiting for FSMap version 74 2024-09-16T01:57:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.888+0000 7f01421f7740 10 client.26617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:57:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.888+0000 7f01421f7740 20 client.26617 populate_metadata read hostname 'smithi139' 2024-09-16T01:57:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.888+0000 7f01421f7740 1 --2- 172.21.15.139:0/1425205225 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d4c00cd10 0x564d4c00f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.888+0000 7f01421f7740 4 client.26617 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:57:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.888+0000 7f01421f7740 1 -- 172.21.15.139:0/1425205225 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d4bf63a20 con 0x564d4c00cd10 2024-09-16T01:57:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.892+0000 7f013d122640 1 --2- 172.21.15.139:0/1425205225 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d4c00cd10 0x564d4c00f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.898+0000 7f013d122640 1 --2- 172.21.15.139:0/1425205225 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d4c00cd10 0x564d4c00f170 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e082accb44358c1b server_cookie=d7b684fb40d290f5 in_seq=0 out_seq=0 2024-09-16T01:57:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.899+0000 7f0120ff9640 10 client.26617 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.902+0000 7f0120ff9640 1 -- 172.21.15.139:0/1425205225 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d4bf63a20 con 0x564d4c00cd10 2024-09-16T01:57:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.902+0000 7f0120ff9640 10 client.26617 handle_command_reply: tid=0 2024-09-16T01:57:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.902+0000 7f01421f7740 1 client.26617 shutdown 2024-09-16T01:57:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.910+0000 7f01421f7740 1 -- 172.21.15.139:0/1425205225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 msgr2=0x564d4bffa140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.910+0000 7f01421f7740 1 --2- 172.21.15.139:0/1425205225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4bffa140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f013409ae10 tx=0x7f01340146c0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.911+0000 7f01421f7740 1 -- 172.21.15.139:0/1425205225 shutdown_connections 2024-09-16T01:57:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.911+0000 7f01421f7740 1 --2- 172.21.15.139:0/1425205225 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d4c00cd10 0x564d4c00f170 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.911+0000 7f01421f7740 1 --2- 172.21.15.139:0/1425205225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bffccb0 0x564d4bffa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.911+0000 7f01421f7740 1 --2- 172.21.15.139:0/1425205225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4bffa140 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.911+0000 7f01421f7740 1 --2- 172.21.15.139:0/1425205225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bff1af0 0x564d4bff9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.911+0000 7f01421f7740 1 -- 172.21.15.139:0/1425205225 >> 172.21.15.139:0/1425205225 conn(0x564d4bfee060 msgr2=0x564d4bff3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.911+0000 7f0120ff9640 0 client.26617 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.915+0000 7f01421f7740 1 -- 172.21.15.139:0/1425205225 shutdown_connections 2024-09-16T01:57:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.919+0000 7f01421f7740 1 -- 172.21.15.139:0/1425205225 wait complete. 2024-09-16T01:57:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.920+0000 7f01421f7740 20 client.26617 trim_cache size 0 max 16384 2024-09-16T01:57:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.950+0000 7f01421f7740 1 Processor -- start 2024-09-16T01:57:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.950+0000 7f01421f7740 1 -- start start 2024-09-16T01:57:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.951+0000 7f01421f7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bff1af0 0x564d4bffa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.951+0000 7f01421f7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4bff3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.951+0000 7f013d122640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bff1af0 0x564d4bffa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.951+0000 7f013d122640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bff1af0 0x564d4bffa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47678/0 (socket says 172.21.15.139:47678) 2024-09-16T01:57:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.951+0000 7f013d122640 1 -- 172.21.15.139:0/3546988691 learned_addr learned my addr 172.21.15.139:0/3546988691 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.951+0000 7f01421f7740 1 --2- 172.21.15.139:0/3546988691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bffccb0 0x564d4c014a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.952+0000 7f01421f7740 1 -- 172.21.15.139:0/3546988691 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f01381b5ae0 con 0x564d4bff1af0 2024-09-16T01:57:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.952+0000 7f01421f7740 1 -- 172.21.15.139:0/3546988691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01381b78f0 con 0x564d4bffccb0 2024-09-16T01:57:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.952+0000 7f01421f7740 1 -- 172.21.15.139:0/3546988691 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f01381b6240 con 0x564d4bff2410 2024-09-16T01:57:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.952+0000 7f013d923640 1 --2- 172.21.15.139:0/3546988691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bffccb0 0x564d4c014a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.953+0000 7f013c921640 1 --2- 172.21.15.139:0/3546988691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4bff3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.953+0000 7f013d122640 1 -- 172.21.15.139:0/3546988691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 msgr2=0x564d4bff3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.953+0000 7f013d122640 1 --2- 172.21.15.139:0/3546988691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4bff3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.953+0000 7f013d122640 1 -- 172.21.15.139:0/3546988691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bffccb0 msgr2=0x564d4c014a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.953+0000 7f013d122640 1 --2- 172.21.15.139:0/3546988691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bffccb0 0x564d4c014a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.953+0000 7f013d122640 1 -- 172.21.15.139:0/3546988691 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4bfede90 con 0x564d4bff1af0 2024-09-16T01:57:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.954+0000 7f013d122640 1 --2- 172.21.15.139:0/3546988691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bff1af0 0x564d4bffa4b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0128009900 tx=0x7f0128005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794c2b8c0a4eff22 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.955+0000 7f0122ffd640 1 -- 172.21.15.139:0/3546988691 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128046020 con 0x564d4bff1af0 2024-09-16T01:57:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.955+0000 7f0122ffd640 1 -- 172.21.15.139:0/3546988691 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01280044b0 con 0x564d4bff1af0 2024-09-16T01:57:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.956+0000 7f01421f7740 1 -- 172.21.15.139:0/3546988691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bff1af0 msgr2=0x564d4bffa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.956+0000 7f01421f7740 1 --2- 172.21.15.139:0/3546988691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bff1af0 0x564d4bffa4b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0128009900 tx=0x7f0128005a00 comp rx=0 tx=0).stop 2024-09-16T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.963+0000 7f01421f7740 1 -- 172.21.15.139:0/3546988691 shutdown_connections 2024-09-16T01:57:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.963+0000 7f01421f7740 1 --2- 172.21.15.139:0/3546988691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bffccb0 0x564d4c014a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.963+0000 7f01421f7740 1 --2- 172.21.15.139:0/3546988691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4bff3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.963+0000 7f01421f7740 1 --2- 172.21.15.139:0/3546988691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bff1af0 0x564d4bffa4b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.963+0000 7f01421f7740 1 -- 172.21.15.139:0/3546988691 >> 172.21.15.139:0/3546988691 conn(0x564d4bfee060 msgr2=0x564d4bff35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.963+0000 7f01421f7740 1 -- 172.21.15.139:0/3546988691 shutdown_connections 2024-09-16T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.966+0000 7f01421f7740 1 -- 172.21.15.139:0/3546988691 wait complete. 2024-09-16T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.967+0000 7f01421f7740 1 -- start start 2024-09-16T01:57:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.967+0000 7f01421f7740 1 Processor -- start 2024-09-16T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.968+0000 7f01421f7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bff1af0 0x564d4c005010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.968+0000 7f01421f7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4c005550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.968+0000 7f01421f7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bffccb0 0x564d4c000430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.968+0000 7f01421f7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f01381b7160 con 0x564d4bffccb0 2024-09-16T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.968+0000 7f01421f7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01381b69d0 con 0x564d4bff1af0 2024-09-16T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.968+0000 7f01421f7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0138100e10 con 0x564d4bff2410 2024-09-16T01:57:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.968+0000 7f01421f7740 10 client.0 _ll_register_callbacks cb 0x7f0134097430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.969+0000 7f013c921640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4c005550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.969+0000 7f013c921640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4c005550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:53216/0 (socket says 172.21.15.139:53216) 2024-09-16T01:57:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.969+0000 7f013c921640 1 -- 172.21.15.139:0/754012262 learned_addr learned my addr 172.21.15.139:0/754012262 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.969+0000 7f013d923640 1 --2- 172.21.15.139:0/754012262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bffccb0 0x564d4c000430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.969+0000 7f013d122640 1 --2- 172.21.15.139:0/754012262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bff1af0 0x564d4c005010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.972+0000 7f013c921640 1 -- 172.21.15.139:0/754012262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bff1af0 msgr2=0x564d4c005010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.972+0000 7f013c921640 1 --2- 172.21.15.139:0/754012262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bff1af0 0x564d4c005010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.972+0000 7f013c921640 1 -- 172.21.15.139:0/754012262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bffccb0 msgr2=0x564d4c000430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.972+0000 7f013c921640 1 --2- 172.21.15.139:0/754012262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bffccb0 0x564d4c000430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.972+0000 7f013c921640 1 -- 172.21.15.139:0/754012262 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4bfef1d0 con 0x564d4bff2410 2024-09-16T01:57:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.978+0000 7f013c921640 1 --2- 172.21.15.139:0/754012262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4c005550 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f013409f680 tx=0x7f0134014400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9adbcf393e96bd83 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.980+0000 7f0120ff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:57:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.984+0000 7f0120ff9640 1 -- 172.21.15.139:0/754012262 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013409ac50 con 0x564d4bff2410 2024-09-16T01:57:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.984+0000 7f0120ff9640 1 -- 172.21.15.139:0/754012262 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9818+0+0 (secure 0 0 0) 0x7f013409a560 con 0x564d4bff2410 2024-09-16T01:57:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.984+0000 7f01421f7740 1 -- 172.21.15.139:0/754012262 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01280044b0 con 0x564d4bff2410 2024-09-16T01:57:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.990+0000 7f0120ff9640 1 -- 172.21.15.139:0/754012262 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f01340a3d80 con 0x564d4bff2410 2024-09-16T01:57:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.991+0000 7f01421f7740 10 client.26623 fetch_fsmap learned FSMap version 74 2024-09-16T01:57:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.991+0000 7f01421f7740 1 -- 172.21.15.139:0/754012262 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x564d4c077420 con 0x564d4bff2410 2024-09-16T01:57:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.996+0000 7f0120ff9640 1 -- 172.21.15.139:0/754012262 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01280044b0 con 0x564d4bff2410 2024-09-16T01:57:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.996+0000 7f0120ff9640 1 -- 172.21.15.139:0/754012262 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 74) ==== 8446+0+0 (secure 0 0 0) 0x7f01340af0d0 con 0x564d4bff2410 2024-09-16T01:57:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.997+0000 7f01421f7740 10 client.26623 fetch_fsmap finished waiting for FSMap version 74 2024-09-16T01:57:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.997+0000 7f01421f7740 10 client.26623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:57:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.997+0000 7f01421f7740 20 client.26623 populate_metadata read hostname 'smithi139' 2024-09-16T01:57:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.997+0000 7f01421f7740 1 --2- 172.21.15.139:0/754012262 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d4c077a40 0x564d4c079e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.997+0000 7f01421f7740 4 client.26623 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:57:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:29.997+0000 7f01421f7740 1 -- 172.21.15.139:0/754012262 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d4bf5dd70 con 0x564d4c077a40 2024-09-16T01:57:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.001+0000 7f013d122640 1 --2- 172.21.15.139:0/754012262 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d4c077a40 0x564d4c079e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.005+0000 7f013d122640 1 --2- 172.21.15.139:0/754012262 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d4c077a40 0x564d4c079e60 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d595924d4736c1b server_cookie=65eeeb80b9948572 in_seq=0 out_seq=0 2024-09-16T01:57:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.005+0000 7f0120ff9640 10 client.26623 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.179+0000 7f0120ff9640 1 -- 172.21.15.139:0/754012262 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d4bf5dd70 con 0x564d4c077a40 2024-09-16T01:57:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.179+0000 7f0120ff9640 10 client.26623 handle_command_reply: tid=0 2024-09-16T01:57:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.179+0000 7f01421f7740 1 client.26623 shutdown 2024-09-16T01:57:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.179+0000 7f01421f7740 1 -- 172.21.15.139:0/754012262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 msgr2=0x564d4c005550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.179+0000 7f01421f7740 1 --2- 172.21.15.139:0/754012262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4c005550 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f013409f680 tx=0x7f0134014400 comp rx=0 tx=0).stop 2024-09-16T01:57:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.179+0000 7f01421f7740 1 -- 172.21.15.139:0/754012262 shutdown_connections 2024-09-16T01:57:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.179+0000 7f01421f7740 1 --2- 172.21.15.139:0/754012262 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x564d4c077a40 0x564d4c079e60 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.179+0000 7f01421f7740 1 --2- 172.21.15.139:0/754012262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d4bffccb0 0x564d4c000430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.179+0000 7f01421f7740 1 --2- 172.21.15.139:0/754012262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564d4bff2410 0x564d4c005550 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.179+0000 7f0120ff9640 0 client.26623 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.179+0000 7f01421f7740 1 --2- 172.21.15.139:0/754012262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564d4bff1af0 0x564d4c005010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.179+0000 7f01421f7740 1 -- 172.21.15.139:0/754012262 >> 172.21.15.139:0/754012262 conn(0x564d4bfee060 msgr2=0x564d4bff3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.179+0000 7f01421f7740 1 -- 172.21.15.139:0/754012262 shutdown_connections 2024-09-16T01:57:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.179+0000 7f01421f7740 1 -- 172.21.15.139:0/754012262 wait complete. 2024-09-16T01:57:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.180+0000 7f01421f7740 20 client.26623 trim_cache size 0 max 16384 2024-09-16T01:57:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:57:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:57:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:57:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:57:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.180+0000 7f013f386640 1 -- 172.21.15.139:0/751117239 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f01240a35c0 msgr2=0x7f01240a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.181+0000 7f013f386640 1 --2- 172.21.15.139:0/751117239 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f01240a35c0 0x7f01240a5a80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0130014000 tx=0x7f013001b000 comp rx=0 tx=0).stop 2024-09-16T01:57:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.181+0000 7f013f386640 1 -- 172.21.15.139:0/751117239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0138101350 msgr2=0x7f013810d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.181+0000 7f013f386640 1 --2- 172.21.15.139:0/751117239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0138101350 0x7f013810d7f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f01340097d0 tx=0x7f0134009c90 comp rx=0 tx=0).stop 2024-09-16T01:57:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.181+0000 7f013f386640 1 -- 172.21.15.139:0/751117239 shutdown_connections 2024-09-16T01:57:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.181+0000 7f013f386640 1 --2- 172.21.15.139:0/751117239 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f01240a35c0 0x7f01240a5a80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.181+0000 7f013f386640 1 --2- 172.21.15.139:0/751117239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01381069c0 0x7f0138073e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.181+0000 7f013f386640 1 --2- 172.21.15.139:0/751117239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0138101350 0x7f013810d7f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.181+0000 7f013f386640 1 --2- 172.21.15.139:0/751117239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01381009b0 0x7f013810d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.181+0000 7f013f386640 1 -- 172.21.15.139:0/751117239 >> 172.21.15.139:0/751117239 conn(0x7f0138077990 msgr2=0x7f01381037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.181+0000 7f013f386640 1 -- 172.21.15.139:0/751117239 shutdown_connections 2024-09-16T01:57:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:30.181+0000 7f013f386640 1 -- 172.21.15.139:0/751117239 wait complete. 2024-09-16T01:57:30.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:30 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:57:30.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:30 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:57:30.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:30 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:57:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:31 smithi139 ceph-mon[31175]: pgmap v543: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 994 KiB/s wr, 150 op/s 2024-09-16T01:57:31.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:31 smithi176 ceph-mon[35324]: pgmap v543: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 994 KiB/s wr, 150 op/s 2024-09-16T01:57:31.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:31 smithi164 ceph-mon[35121]: pgmap v543: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 994 KiB/s wr, 150 op/s 2024-09-16T01:57:33.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:33 smithi139 ceph-mon[31175]: pgmap v544: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 701 KiB/s wr, 160 op/s 2024-09-16T01:57:33.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:33 smithi176 ceph-mon[35324]: pgmap v544: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 701 KiB/s wr, 160 op/s 2024-09-16T01:57:33.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:33 smithi164 ceph-mon[35121]: pgmap v544: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 701 KiB/s wr, 160 op/s 2024-09-16T01:57:35.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:35 smithi139 ceph-mon[31175]: pgmap v545: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-09-16T01:57:35.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:35 smithi176 ceph-mon[35324]: pgmap v545: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-09-16T01:57:35.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:35 smithi164 ceph-mon[35121]: pgmap v545: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-09-16T01:57:37.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:37 smithi139 ceph-mon[31175]: pgmap v546: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 525 KiB/s wr, 155 op/s 2024-09-16T01:57:37.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:37 smithi139 ceph-mon[31175]: osdmap e114: 12 total, 12 up, 12 in 2024-09-16T01:57:37.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:37 smithi176 ceph-mon[35324]: pgmap v546: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 525 KiB/s wr, 155 op/s 2024-09-16T01:57:37.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:37 smithi176 ceph-mon[35324]: osdmap e114: 12 total, 12 up, 12 in 2024-09-16T01:57:37.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:37 smithi164 ceph-mon[35121]: pgmap v546: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 525 KiB/s wr, 155 op/s 2024-09-16T01:57:37.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:37 smithi164 ceph-mon[35121]: osdmap e114: 12 total, 12 up, 12 in 2024-09-16T01:57:39.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:39 smithi139 ceph-mon[31175]: pgmap v548: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2024-09-16T01:57:39.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:39 smithi176 ceph-mon[35324]: pgmap v548: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2024-09-16T01:57:39.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:39 smithi164 ceph-mon[35121]: pgmap v548: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2024-09-16T01:57:41.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:41 smithi139 ceph-mon[31175]: pgmap v549: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-09-16T01:57:41.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:41 smithi176 ceph-mon[35324]: pgmap v549: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-09-16T01:57:41.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:41 smithi164 ceph-mon[35121]: pgmap v549: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-09-16T01:57:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.341+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1675915898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f449c11b140 msgr2=0x7f449c11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.341+0000 7f44a1bbd640 1 --2- 172.21.15.139:0/1675915898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f449c11b140 0x7f449c11d530 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4494009c10 tx=0x7f449402a720 comp rx=0 tx=0).stop 2024-09-16T01:57:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.346+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1675915898 shutdown_connections 2024-09-16T01:57:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.346+0000 7f44a1bbd640 1 --2- 172.21.15.139:0/1675915898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f449c11b140 0x7f449c11d530 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.346+0000 7f44a1bbd640 1 --2- 172.21.15.139:0/1675915898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f449c1095d0 0x7f449c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.346+0000 7f44a1bbd640 1 --2- 172.21.15.139:0/1675915898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f449c1091f0 0x7f449c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.346+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1675915898 >> 172.21.15.139:0/1675915898 conn(0x7f449c06ce00 msgr2=0x7f449c06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.347+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1675915898 shutdown_connections 2024-09-16T01:57:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.347+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1675915898 wait complete. 2024-09-16T01:57:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.348+0000 7f44a1bbd640 1 Processor -- start 2024-09-16T01:57:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.348+0000 7f44a1bbd640 1 -- start start 2024-09-16T01:57:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.349+0000 7f44a1bbd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f449c1091f0 0x7f449c11a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.349+0000 7f44a1bbd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f449c1095d0 0x7f449c1148f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.349+0000 7f44a1bbd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f449c11b140 0x7f449c114e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.349+0000 7f44a1bbd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f449c111370 con 0x7f449c11b140 2024-09-16T01:57:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.349+0000 7f449affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f449c1095d0 0x7f449c1148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.349+0000 7f449affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f449c1095d0 0x7f449c1148f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:40808/0 (socket says 172.21.15.139:40808) 2024-09-16T01:57:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.349+0000 7f449affd640 1 -- 172.21.15.139:0/1125314130 learned_addr learned my addr 172.21.15.139:0/1125314130 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.349+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1125314130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f449c1111f0 con 0x7f449c1095d0 2024-09-16T01:57:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.349+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1125314130 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f449c1114f0 con 0x7f449c1091f0 2024-09-16T01:57:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.350+0000 7f449affd640 1 -- 172.21.15.139:0/1125314130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f449c1091f0 msgr2=0x7f449c11a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.350+0000 7f449affd640 1 --2- 172.21.15.139:0/1125314130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f449c1091f0 0x7f449c11a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.350+0000 7f449affd640 1 -- 172.21.15.139:0/1125314130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f449c11b140 msgr2=0x7f449c114e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.350+0000 7f449affd640 1 --2- 172.21.15.139:0/1125314130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f449c11b140 0x7f449c114e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.350+0000 7f449affd640 1 -- 172.21.15.139:0/1125314130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449c115560 con 0x7f449c1095d0 2024-09-16T01:57:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.353+0000 7f449affd640 1 --2- 172.21.15.139:0/1125314130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f449c1095d0 0x7f449c1148f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f449000b450 tx=0x7f449000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.355+0000 7f4498ff9640 1 -- 172.21.15.139:0/1125314130 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490015020 con 0x7f449c1095d0 2024-09-16T01:57:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.355+0000 7f4498ff9640 1 -- 172.21.15.139:0/1125314130 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4490011d70 con 0x7f449c1095d0 2024-09-16T01:57:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.355+0000 7f4498ff9640 1 -- 172.21.15.139:0/1125314130 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449000f340 con 0x7f449c1095d0 2024-09-16T01:57:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.357+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1125314130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f449c1bc000 con 0x7f449c1095d0 2024-09-16T01:57:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.357+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1125314130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f449c1bc470 con 0x7f449c1095d0 2024-09-16T01:57:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.363+0000 7f44a4a32740 1 Processor -- start 2024-09-16T01:57:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.364+0000 7f44a4a32740 1 -- start start 2024-09-16T01:57:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.364+0000 7f44a4a32740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a77af0 0x5623e8a77ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.364+0000 7f44a4a32740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a82770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.364+0000 7f44a4a32740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a82cb0 0x5623e8a850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.364+0000 7f44a4a32740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5623e89dc2f0 con 0x5623e8a77af0 2024-09-16T01:57:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.364+0000 7f44a4a32740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5623e8984270 con 0x5623e8a78410 2024-09-16T01:57:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.364+0000 7f44a4a32740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5623e89d82b0 con 0x5623e8a82cb0 2024-09-16T01:57:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.365+0000 7f449b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a82770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.365+0000 7f449b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a82770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:40812/0 (socket says 172.21.15.139:40812) 2024-09-16T01:57:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.365+0000 7f449b7fe640 1 -- 172.21.15.139:0/1670126253 learned_addr learned my addr 172.21.15.139:0/1670126253 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.365+0000 7f449bfff640 1 --2- 172.21.15.139:0/1670126253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a77af0 0x5623e8a77ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.365+0000 7f449bfff640 1 -- 172.21.15.139:0/1670126253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a82cb0 msgr2=0x5623e8a850a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:57:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.365+0000 7f449bfff640 1 --2- 172.21.15.139:0/1670126253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a82cb0 0x5623e8a850a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.365+0000 7f449bfff640 1 -- 172.21.15.139:0/1670126253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 msgr2=0x5623e8a82770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.365+0000 7f449bfff640 1 --2- 172.21.15.139:0/1670126253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a82770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.365+0000 7f449bfff640 1 -- 172.21.15.139:0/1670126253 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e8983340 con 0x5623e8a77af0 2024-09-16T01:57:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.365+0000 7f449b7fe640 1 --2- 172.21.15.139:0/1670126253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a82770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:57:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.366+0000 7f449bfff640 1 --2- 172.21.15.139:0/1670126253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a77af0 0x5623e8a77ed0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4494009be0 tx=0x7f44940303a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9f3f98c1946160 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.369+0000 7f4498ff9640 1 -- 172.21.15.139:0/1125314130 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4490018020 con 0x7f449c1095d0 2024-09-16T01:57:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.370+0000 7f4498ff9640 1 --2- 172.21.15.139:0/1125314130 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f44840a3640 0x7f44840a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.370+0000 7f4498ff9640 1 -- 172.21.15.139:0/1125314130 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9742+0+0 (secure 0 0 0) 0x7f44900cfa50 con 0x7f449c1095d0 2024-09-16T01:57:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.371+0000 7f4481ffb640 1 -- 172.21.15.139:0/1670126253 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494041020 con 0x5623e8a77af0 2024-09-16T01:57:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.371+0000 7f4481ffb640 1 -- 172.21.15.139:0/1670126253 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4494030e60 con 0x5623e8a77af0 2024-09-16T01:57:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.371+0000 7f4481ffb640 1 -- 172.21.15.139:0/1670126253 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494036040 con 0x5623e8a77af0 2024-09-16T01:57:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.371+0000 7f449b7fe640 1 --2- 172.21.15.139:0/1125314130 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f44840a3640 0x7f44840a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.372+0000 7f44a4a32740 1 -- 172.21.15.139:0/1670126253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a77af0 msgr2=0x5623e8a77ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.372+0000 7f44a4a32740 1 --2- 172.21.15.139:0/1670126253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a77af0 0x5623e8a77ed0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4494009be0 tx=0x7f44940303a0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.373+0000 7f44a4a32740 1 -- 172.21.15.139:0/1670126253 shutdown_connections 2024-09-16T01:57:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.373+0000 7f44a4a32740 1 --2- 172.21.15.139:0/1670126253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a82cb0 0x5623e8a850a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.373+0000 7f44a4a32740 1 --2- 172.21.15.139:0/1670126253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a82770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.373+0000 7f44a4a32740 1 --2- 172.21.15.139:0/1670126253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a77af0 0x5623e8a77ed0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.373+0000 7f44a4a32740 1 -- 172.21.15.139:0/1670126253 >> 172.21.15.139:0/1670126253 conn(0x5623e8a74060 msgr2=0x5623e8a74440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.374+0000 7f449b7fe640 1 --2- 172.21.15.139:0/1125314130 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f44840a3640 0x7f44840a5b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x5623e8a74ec0 tx=0x7f448c005a40 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.374+0000 7f44a4a32740 1 -- 172.21.15.139:0/1670126253 shutdown_connections 2024-09-16T01:57:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.374+0000 7f44a4a32740 1 -- 172.21.15.139:0/1670126253 wait complete. 2024-09-16T01:57:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.375+0000 7f44a4a32740 1 -- start start 2024-09-16T01:57:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.375+0000 7f44a4a32740 1 Processor -- start 2024-09-16T01:57:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.376+0000 7f44a4a32740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a77af0 0x5623e8a7fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.376+0000 7f44a4a32740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a80160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.377+0000 7f449b7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a77af0 0x5623e8a7fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.377+0000 7f449affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a80160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.377+0000 7f449affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a80160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:40818/0 (socket says 172.21.15.139:40818) 2024-09-16T01:57:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.377+0000 7f449b7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a77af0 0x5623e8a7fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:51824/0 (socket says 172.21.15.139:51824) 2024-09-16T01:57:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.377+0000 7f44a4a32740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a82cb0 0x5623e8a806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.377+0000 7f449affd640 1 -- 172.21.15.139:0/2657795807 learned_addr learned my addr 172.21.15.139:0/2657795807 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.377+0000 7f44a4a32740 1 -- 172.21.15.139:0/2657795807 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5623e8a85880 con 0x5623e8a82cb0 2024-09-16T01:57:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.377+0000 7f44a4a32740 1 -- 172.21.15.139:0/2657795807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5623e8965290 con 0x5623e8a78410 2024-09-16T01:57:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.377+0000 7f44a4a32740 1 -- 172.21.15.139:0/2657795807 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5623e8a85a00 con 0x5623e8a77af0 2024-09-16T01:57:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.377+0000 7f44a4a32740 10 client.0 _ll_register_callbacks cb 0x5623e8996140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:57:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.377+0000 7f449affd640 1 -- 172.21.15.139:0/2657795807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a77af0 msgr2=0x5623e8a7fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.377+0000 7f449affd640 1 --2- 172.21.15.139:0/2657795807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a77af0 0x5623e8a7fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.377+0000 7f449affd640 1 -- 172.21.15.139:0/2657795807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a82cb0 msgr2=0x5623e8a806a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:57:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.377+0000 7f449affd640 1 --2- 172.21.15.139:0/2657795807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a82cb0 0x5623e8a806a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.378+0000 7f449affd640 1 -- 172.21.15.139:0/2657795807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e86ef210 con 0x5623e8a78410 2024-09-16T01:57:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.378+0000 7f449affd640 1 --2- 172.21.15.139:0/2657795807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a80160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f449009b620 tx=0x7f449009c150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=196efb3e24a5f74f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.378+0000 7f44a4a32740 1 -- 172.21.15.139:0/2657795807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4494030e60 con 0x5623e8a78410 2024-09-16T01:57:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.381+0000 7f447affd640 10 client.26660 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:57:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.381+0000 7f447affd640 1 -- 172.21.15.139:0/2657795807 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44900b3020 con 0x5623e8a78410 2024-09-16T01:57:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.382+0000 7f447affd640 1 -- 172.21.15.139:0/2657795807 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9742+0+0 (secure 0 0 0) 0x7f44900ccf00 con 0x5623e8a78410 2024-09-16T01:57:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.382+0000 7f447affd640 1 -- 172.21.15.139:0/2657795807 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f44900a4590 con 0x5623e8a78410 2024-09-16T01:57:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.382+0000 7f44a4a32740 10 client.26660 fetch_fsmap learned FSMap version 74 2024-09-16T01:57:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.382+0000 7f44a4a32740 1 -- 172.21.15.139:0/2657795807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=74}) -- 0x5623e87bb620 con 0x5623e8a78410 2024-09-16T01:57:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.383+0000 7f447affd640 1 -- 172.21.15.139:0/2657795807 <== mon.2 v2:172.21.15.164:3300/0 4 ==== fsmap(e 74) ==== 8446+0+0 (secure 0 0 0) 0x7f4490022060 con 0x5623e8a78410 2024-09-16T01:57:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.383+0000 7f44a4a32740 10 client.26660 fetch_fsmap finished waiting for FSMap version 74 2024-09-16T01:57:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.383+0000 7f44a4a32740 10 client.26660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:57:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.383+0000 7f44a4a32740 20 client.26660 populate_metadata read hostname 'smithi139' 2024-09-16T01:57:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.383+0000 7f44a4a32740 1 --2- 172.21.15.139:0/2657795807 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5623e8a92fd0 0x5623e8a95490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.383+0000 7f44a4a32740 4 client.26660 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:57:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.383+0000 7f44a4a32740 1 -- 172.21.15.139:0/2657795807 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623e89e9a20 con 0x5623e8a92fd0 2024-09-16T01:57:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.385+0000 7f449b7fe640 1 --2- 172.21.15.139:0/2657795807 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5623e8a92fd0 0x5623e8a95490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.388+0000 7f449b7fe640 1 --2- 172.21.15.139:0/2657795807 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5623e8a92fd0 0x5623e8a95490 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcf93ed46b365a14 server_cookie=240fc23bfcf381a2 in_seq=0 out_seq=0 2024-09-16T01:57:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.388+0000 7f447affd640 10 client.26660 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.390+0000 7f447affd640 1 -- 172.21.15.139:0/2657795807 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623e89e9a20 con 0x5623e8a92fd0 2024-09-16T01:57:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.390+0000 7f447affd640 10 client.26660 handle_command_reply: tid=0 2024-09-16T01:57:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.390+0000 7f44a4a32740 1 client.26660 shutdown 2024-09-16T01:57:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.391+0000 7f44a4a32740 1 -- 172.21.15.139:0/2657795807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 msgr2=0x5623e8a80160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.391+0000 7f44a4a32740 1 --2- 172.21.15.139:0/2657795807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a80160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f449009b620 tx=0x7f449009c150 comp rx=0 tx=0).stop 2024-09-16T01:57:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.395+0000 7f44a4a32740 1 -- 172.21.15.139:0/2657795807 shutdown_connections 2024-09-16T01:57:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.395+0000 7f44a4a32740 1 --2- 172.21.15.139:0/2657795807 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5623e8a92fd0 0x5623e8a95490 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.395+0000 7f44a4a32740 1 --2- 172.21.15.139:0/2657795807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a82cb0 0x5623e8a806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.395+0000 7f44a4a32740 1 --2- 172.21.15.139:0/2657795807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a80160 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.395+0000 7f44a4a32740 1 --2- 172.21.15.139:0/2657795807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a77af0 0x5623e8a7fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.395+0000 7f44a4a32740 1 -- 172.21.15.139:0/2657795807 >> 172.21.15.139:0/2657795807 conn(0x5623e8a74060 msgr2=0x5623e8a79dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.395+0000 7f447affd640 0 client.26660 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.398+0000 7f44a4a32740 1 -- 172.21.15.139:0/2657795807 shutdown_connections 2024-09-16T01:57:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.398+0000 7f44a4a32740 1 -- 172.21.15.139:0/2657795807 wait complete. 2024-09-16T01:57:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.399+0000 7f44a4a32740 20 client.26660 trim_cache size 0 max 16384 2024-09-16T01:57:43.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:43 smithi139 ceph-mon[31175]: pgmap v550: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-09-16T01:57:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.431+0000 7f44a4a32740 1 Processor -- start 2024-09-16T01:57:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.431+0000 7f44a4a32740 1 -- start start 2024-09-16T01:57:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.431+0000 7f44a4a32740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a77af0 0x5623e8a7a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.431+0000 7f44a4a32740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.431+0000 7f44a4a32740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a82cb0 0x5623e8a9ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.432+0000 7f449b7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a77af0 0x5623e8a7a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.432+0000 7f449b7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a77af0 0x5623e8a7a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:51832/0 (socket says 172.21.15.139:51832) 2024-09-16T01:57:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.432+0000 7f449affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.432+0000 7f449affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a804d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:40828/0 (socket says 172.21.15.139:40828) 2024-09-16T01:57:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.432+0000 7f449b7fe640 1 -- 172.21.15.139:0/1643221857 learned_addr learned my addr 172.21.15.139:0/1643221857 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.432+0000 7f44a4a32740 1 -- 172.21.15.139:0/1643221857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f449c1c6930 con 0x5623e8a82cb0 2024-09-16T01:57:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.432+0000 7f44a4a32740 1 -- 172.21.15.139:0/1643221857 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f449c1c8740 con 0x5623e8a78410 2024-09-16T01:57:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.432+0000 7f44a4a32740 1 -- 172.21.15.139:0/1643221857 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f449c1c7090 con 0x5623e8a77af0 2024-09-16T01:57:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.432+0000 7f449bfff640 1 --2- 172.21.15.139:0/1643221857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a82cb0 0x5623e8a9ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.433+0000 7f449bfff640 1 -- 172.21.15.139:0/1643221857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a77af0 msgr2=0x5623e8a7a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.433+0000 7f449bfff640 1 --2- 172.21.15.139:0/1643221857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a77af0 0x5623e8a7a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.433+0000 7f449bfff640 1 -- 172.21.15.139:0/1643221857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 msgr2=0x5623e8a804d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.433+0000 7f449bfff640 1 --2- 172.21.15.139:0/1643221857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a804d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.433+0000 7f449bfff640 1 -- 172.21.15.139:0/1643221857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e8a77920 con 0x5623e8a82cb0 2024-09-16T01:57:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.433+0000 7f449bfff640 1 --2- 172.21.15.139:0/1643221857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a82cb0 0x5623e8a9ad60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4494040040 tx=0x7f4494005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10f9c0feb62b38f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.433+0000 7f449b7fe640 1 --2- 172.21.15.139:0/1643221857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a77af0 0x5623e8a7a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:57:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.433+0000 7f449affd640 1 --2- 172.21.15.139:0/1643221857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a804d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:57:43.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:43 smithi176 ceph-mon[35324]: pgmap v550: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-09-16T01:57:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.436+0000 7f44817fa640 1 -- 172.21.15.139:0/1643221857 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449404a030 con 0x5623e8a82cb0 2024-09-16T01:57:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.436+0000 7f44817fa640 1 -- 172.21.15.139:0/1643221857 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4494030510 con 0x5623e8a82cb0 2024-09-16T01:57:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.436+0000 7f44817fa640 1 -- 172.21.15.139:0/1643221857 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494041070 con 0x5623e8a82cb0 2024-09-16T01:57:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.437+0000 7f44a4a32740 1 -- 172.21.15.139:0/1643221857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a82cb0 msgr2=0x5623e8a9ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.437+0000 7f44a4a32740 1 --2- 172.21.15.139:0/1643221857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a82cb0 0x5623e8a9ad60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4494040040 tx=0x7f4494005fd0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.441+0000 7f44a4a32740 1 -- 172.21.15.139:0/1643221857 shutdown_connections 2024-09-16T01:57:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.441+0000 7f44a4a32740 1 --2- 172.21.15.139:0/1643221857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a82cb0 0x5623e8a9ad60 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.441+0000 7f44a4a32740 1 --2- 172.21.15.139:0/1643221857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a804d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.441+0000 7f44a4a32740 1 --2- 172.21.15.139:0/1643221857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a77af0 0x5623e8a7a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.441+0000 7f44a4a32740 1 -- 172.21.15.139:0/1643221857 >> 172.21.15.139:0/1643221857 conn(0x5623e8a74060 msgr2=0x5623e8a81ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.442+0000 7f44a4a32740 1 -- 172.21.15.139:0/1643221857 shutdown_connections 2024-09-16T01:57:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.445+0000 7f44a4a32740 1 -- 172.21.15.139:0/1643221857 wait complete. 2024-09-16T01:57:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.446+0000 7f44a4a32740 1 -- start start 2024-09-16T01:57:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.446+0000 7f44a4a32740 1 Processor -- start 2024-09-16T01:57:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.447+0000 7f44a4a32740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a77af0 0x5623e8a8ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.447+0000 7f44a4a32740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a8b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.447+0000 7f44a4a32740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a82cb0 0x5623e8a7c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.447+0000 7f44a4a32740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f449c1c7fb0 con 0x5623e8a77af0 2024-09-16T01:57:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.447+0000 7f44a4a32740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f449c1c7820 con 0x5623e8a78410 2024-09-16T01:57:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.447+0000 7f44a4a32740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f449c1c4910 con 0x5623e8a82cb0 2024-09-16T01:57:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.447+0000 7f44a4a32740 10 client.0 _ll_register_callbacks cb 0x7f44900108d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:57:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.448+0000 7f449affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a8b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.448+0000 7f449affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a8b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:40834/0 (socket says 172.21.15.139:40834) 2024-09-16T01:57:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.448+0000 7f449affd640 1 -- 172.21.15.139:0/1972204326 learned_addr learned my addr 172.21.15.139:0/1972204326 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.448+0000 7f449b7fe640 1 --2- 172.21.15.139:0/1972204326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a77af0 0x5623e8a8ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.449+0000 7f449affd640 1 -- 172.21.15.139:0/1972204326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a82cb0 msgr2=0x5623e8a7c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.449+0000 7f449affd640 1 --2- 172.21.15.139:0/1972204326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a82cb0 0x5623e8a7c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.449+0000 7f449affd640 1 -- 172.21.15.139:0/1972204326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a77af0 msgr2=0x5623e8a8ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.449+0000 7f449affd640 1 --2- 172.21.15.139:0/1972204326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a77af0 0x5623e8a8ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.449+0000 7f449affd640 1 -- 172.21.15.139:0/1972204326 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e8a7c7a0 con 0x5623e8a78410 2024-09-16T01:57:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.449+0000 7f449affd640 1 --2- 172.21.15.139:0/1972204326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a8b1d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f449009b8b0 tx=0x7f44900ce000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adb392d51a0e2fef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.451+0000 7f447affd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:57:43.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.451+0000 7f447affd640 1 -- 172.21.15.139:0/1972204326 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44900b3070 con 0x5623e8a78410 2024-09-16T01:57:43.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.451+0000 7f44a4a32740 1 -- 172.21.15.139:0/1972204326 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4494030510 con 0x5623e8a78410 2024-09-16T01:57:43.452 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.451+0000 7f447affd640 1 -- 172.21.15.139:0/1972204326 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9742+0+0 (secure 0 0 0) 0x7f449000eb70 con 0x5623e8a78410 2024-09-16T01:57:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.458+0000 7f447affd640 1 -- 172.21.15.139:0/1972204326 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f449000ed70 con 0x5623e8a78410 2024-09-16T01:57:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.458+0000 7f44a4a32740 10 client.26663 fetch_fsmap learned FSMap version 74 2024-09-16T01:57:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.458+0000 7f44a4a32740 1 -- 172.21.15.139:0/1972204326 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x5623e8afd450 con 0x5623e8a78410 2024-09-16T01:57:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.459+0000 7f447affd640 1 -- 172.21.15.139:0/1972204326 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4494030510 con 0x5623e8a78410 2024-09-16T01:57:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.459+0000 7f447affd640 1 -- 172.21.15.139:0/1972204326 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 74) ==== 8446+0+0 (secure 0 0 0) 0x7f4490011a30 con 0x5623e8a78410 2024-09-16T01:57:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.459+0000 7f44a4a32740 10 client.26663 fetch_fsmap finished waiting for FSMap version 74 2024-09-16T01:57:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.459+0000 7f44a4a32740 10 client.26663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:57:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.459+0000 7f44a4a32740 20 client.26663 populate_metadata read hostname 'smithi139' 2024-09-16T01:57:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.459+0000 7f44a4a32740 1 --2- 172.21.15.139:0/1972204326 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5623e8afd9e0 0x5623e8affe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.459+0000 7f44a4a32740 4 client.26663 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:57:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.459+0000 7f44a4a32740 1 -- 172.21.15.139:0/1972204326 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623e89e3d70 con 0x5623e8afd9e0 2024-09-16T01:57:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.462+0000 7f449b7fe640 1 --2- 172.21.15.139:0/1972204326 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5623e8afd9e0 0x5623e8affe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.464+0000 7f449b7fe640 1 --2- 172.21.15.139:0/1972204326 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5623e8afd9e0 0x5623e8affe40 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62145eafc658da14 server_cookie=88bd9ee18f1fa97d in_seq=0 out_seq=0 2024-09-16T01:57:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.464+0000 7f447affd640 10 client.26663 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:43.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:43 smithi164 ceph-mon[35121]: pgmap v550: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-09-16T01:57:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.613+0000 7f447affd640 1 -- 172.21.15.139:0/1972204326 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5623e89e3d70 con 0x5623e8afd9e0 2024-09-16T01:57:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.613+0000 7f447affd640 10 client.26663 handle_command_reply: tid=0 2024-09-16T01:57:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.613+0000 7f44a4a32740 1 client.26663 shutdown 2024-09-16T01:57:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.613+0000 7f44a4a32740 1 -- 172.21.15.139:0/1972204326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 msgr2=0x5623e8a8b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.613+0000 7f44a4a32740 1 --2- 172.21.15.139:0/1972204326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a8b1d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f449009b8b0 tx=0x7f44900ce000 comp rx=0 tx=0).stop 2024-09-16T01:57:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.614+0000 7f44a4a32740 1 -- 172.21.15.139:0/1972204326 shutdown_connections 2024-09-16T01:57:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.614+0000 7f44a4a32740 1 --2- 172.21.15.139:0/1972204326 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5623e8afd9e0 0x5623e8affe40 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.614+0000 7f44a4a32740 1 --2- 172.21.15.139:0/1972204326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623e8a82cb0 0x5623e8a7c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.614+0000 7f44a4a32740 1 --2- 172.21.15.139:0/1972204326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623e8a78410 0x5623e8a8b1d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.614+0000 7f44a4a32740 1 --2- 172.21.15.139:0/1972204326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623e8a77af0 0x5623e8a8ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.614+0000 7f44a4a32740 1 -- 172.21.15.139:0/1972204326 >> 172.21.15.139:0/1972204326 conn(0x5623e8a74060 msgr2=0x5623e8a81ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.614+0000 7f447affd640 0 client.26663 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.614+0000 7f44a4a32740 1 -- 172.21.15.139:0/1972204326 shutdown_connections 2024-09-16T01:57:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.614+0000 7f44a4a32740 1 -- 172.21.15.139:0/1972204326 wait complete. 2024-09-16T01:57:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.614+0000 7f44a4a32740 20 client.26663 trim_cache size 0 max 16384 2024-09-16T01:57:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:57:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:57:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:57:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:57:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.615+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1125314130 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f44840a3640 msgr2=0x7f44840a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.615+0000 7f44a1bbd640 1 --2- 172.21.15.139:0/1125314130 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f44840a3640 0x7f44840a5b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x5623e8a74ec0 tx=0x7f448c005a40 comp rx=0 tx=0).stop 2024-09-16T01:57:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.615+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1125314130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f449c1095d0 msgr2=0x7f449c1148f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.615+0000 7f44a1bbd640 1 --2- 172.21.15.139:0/1125314130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f449c1095d0 0x7f449c1148f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f449000b450 tx=0x7f449000b910 comp rx=0 tx=0).stop 2024-09-16T01:57:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.615+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1125314130 shutdown_connections 2024-09-16T01:57:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.615+0000 7f44a1bbd640 1 --2- 172.21.15.139:0/1125314130 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f44840a3640 0x7f44840a5b00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.615+0000 7f44a1bbd640 1 --2- 172.21.15.139:0/1125314130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f449c11b140 0x7f449c114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.615+0000 7f44a1bbd640 1 --2- 172.21.15.139:0/1125314130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f449c1095d0 0x7f449c1148f0 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f449000b450 tx=0x7f449000b910 comp rx=0 tx=0).stop 2024-09-16T01:57:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.615+0000 7f44a1bbd640 1 --2- 172.21.15.139:0/1125314130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f449c1091f0 0x7f449c11a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.615+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1125314130 >> 172.21.15.139:0/1125314130 conn(0x7f449c06ce00 msgr2=0x7f449c0707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.615+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1125314130 shutdown_connections 2024-09-16T01:57:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:43.615+0000 7f44a1bbd640 1 -- 172.21.15.139:0/1125314130 wait complete. 2024-09-16T01:57:45.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:45 smithi139 ceph-mon[31175]: pgmap v551: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-09-16T01:57:45.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:45 smithi176 ceph-mon[35324]: pgmap v551: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-09-16T01:57:45.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:45 smithi164 ceph-mon[35121]: pgmap v551: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-09-16T01:57:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:47 smithi139 ceph-mon[31175]: pgmap v552: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 184 op/s 2024-09-16T01:57:47.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:47 smithi176 ceph-mon[35324]: pgmap v552: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 184 op/s 2024-09-16T01:57:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:47 smithi164 ceph-mon[35121]: pgmap v552: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 184 op/s 2024-09-16T01:57:49.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:49 smithi139 ceph-mon[31175]: pgmap v553: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2024-09-16T01:57:49.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:49 smithi176 ceph-mon[35324]: pgmap v553: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2024-09-16T01:57:49.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:49 smithi164 ceph-mon[35121]: pgmap v553: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 204 op/s 2024-09-16T01:57:50.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:50 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:57:50.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:50 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:57:50.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:50 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:57:51.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:51 smithi139 ceph-mon[31175]: pgmap v554: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-09-16T01:57:51.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:51 smithi176 ceph-mon[35324]: pgmap v554: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-09-16T01:57:51.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:51 smithi164 ceph-mon[35121]: pgmap v554: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-09-16T01:57:53.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:53 smithi139 ceph-mon[31175]: pgmap v555: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 173 op/s 2024-09-16T01:57:53.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:53 smithi176 ceph-mon[35324]: pgmap v555: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 173 op/s 2024-09-16T01:57:53.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:53 smithi164 ceph-mon[35121]: pgmap v555: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 173 op/s 2024-09-16T01:57:53.878 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T01:57:53.879 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.95 2024-09-16T01:57:53.879 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:57:54.364 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:57:55.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:55 smithi176 ceph-mon[35324]: pgmap v556: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-09-16T01:57:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:55 smithi164 ceph-mon[35121]: pgmap v556: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-09-16T01:57:55.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:55 smithi139 ceph-mon[31175]: pgmap v556: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-09-16T01:57:55.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.706+0000 7ffb85e14640 1 -- 172.21.15.139:0/1434909670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb80109830 msgr2=0x7ffb80109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:55.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.706+0000 7ffb85e14640 1 --2- 172.21.15.139:0/1434909670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb80109830 0x7ffb80109c90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009860 tx=0x7ffb7402a380 comp rx=0 tx=0).stop 2024-09-16T01:57:55.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.707+0000 7ffb85e14640 1 -- 172.21.15.139:0/1434909670 shutdown_connections 2024-09-16T01:57:55.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.707+0000 7ffb85e14640 1 --2- 172.21.15.139:0/1434909670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb8011b140 0x7ffb8011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:55.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.707+0000 7ffb85e14640 1 --2- 172.21.15.139:0/1434909670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb80109830 0x7ffb80109c90 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009860 tx=0x7ffb7402a380 comp rx=0 tx=0).stop 2024-09-16T01:57:55.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.707+0000 7ffb85e14640 1 --2- 172.21.15.139:0/1434909670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb80109450 0x7ffb80110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:55.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.707+0000 7ffb85e14640 1 -- 172.21.15.139:0/1434909670 >> 172.21.15.139:0/1434909670 conn(0x7ffb8006e140 msgr2=0x7ffb8006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:55.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.708+0000 7ffb85e14640 1 -- 172.21.15.139:0/1434909670 shutdown_connections 2024-09-16T01:57:55.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.712+0000 7ffb85e14640 1 -- 172.21.15.139:0/1434909670 wait complete. 2024-09-16T01:57:55.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.712+0000 7ffb85e14640 1 Processor -- start 2024-09-16T01:57:55.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.718+0000 7ffb85e14640 1 -- start start 2024-09-16T01:57:55.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.718+0000 7ffb85e14640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb80109450 0x7ffb8011a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:55.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.720+0000 7ffb7f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb80109450 0x7ffb8011a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:55.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.720+0000 7ffb85e14640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb8011b140 0x7ffb801139e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:55.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.720+0000 7ffb85e14640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb80113f20 0x7ffb801143d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:55.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.720+0000 7ffb85e14640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80120df0 con 0x7ffb8011b140 2024-09-16T01:57:55.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.720+0000 7ffb85e14640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80120c70 con 0x7ffb80113f20 2024-09-16T01:57:55.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.720+0000 7ffb85e14640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80120f70 con 0x7ffb80109450 2024-09-16T01:57:55.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.720+0000 7ffb7f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb80109450 0x7ffb8011a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:42334/0 (socket says 172.21.15.139:42334) 2024-09-16T01:57:55.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.720+0000 7ffb7f7fe640 1 -- 172.21.15.139:0/2549146788 learned_addr learned my addr 172.21.15.139:0/2549146788 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:55.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.722+0000 7ffb7effd640 1 --2- 172.21.15.139:0/2549146788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb8011b140 0x7ffb801139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:55.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.722+0000 7ffb7f7fe640 1 -- 172.21.15.139:0/2549146788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb80113f20 msgr2=0x7ffb801143d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:57:55.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.722+0000 7ffb7f7fe640 1 --2- 172.21.15.139:0/2549146788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb80113f20 0x7ffb801143d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:55.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.722+0000 7ffb7f7fe640 1 -- 172.21.15.139:0/2549146788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb8011b140 msgr2=0x7ffb801139e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:55.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.722+0000 7ffb7f7fe640 1 --2- 172.21.15.139:0/2549146788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb8011b140 0x7ffb801139e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:55.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.722+0000 7ffb7f7fe640 1 -- 172.21.15.139:0/2549146788 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb801185b0 con 0x7ffb80109450 2024-09-16T01:57:55.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.723+0000 7ffb7f7fe640 1 --2- 172.21.15.139:0/2549146788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb80109450 0x7ffb8011a950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000d950 tx=0x7ffb7000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:55.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.723+0000 7ffb7cff9640 1 -- 172.21.15.139:0/2549146788 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb70017070 con 0x7ffb80109450 2024-09-16T01:57:55.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.723+0000 7ffb85e14640 1 -- 172.21.15.139:0/2549146788 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb801188a0 con 0x7ffb80109450 2024-09-16T01:57:55.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.723+0000 7ffb85e14640 1 -- 172.21.15.139:0/2549146788 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb80118e00 con 0x7ffb80109450 2024-09-16T01:57:55.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.724+0000 7ffb7cff9640 1 -- 172.21.15.139:0/2549146788 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb70009c30 con 0x7ffb80109450 2024-09-16T01:57:55.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.724+0000 7ffb7cff9640 1 -- 172.21.15.139:0/2549146788 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7000f690 con 0x7ffb80109450 2024-09-16T01:57:55.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.726+0000 7ffb85e14640 1 -- 172.21.15.139:0/2549146788 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb80073b00 con 0x7ffb80109450 2024-09-16T01:57:55.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.731+0000 7ffb7cff9640 1 -- 172.21.15.139:0/2549146788 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ffb700109d0 con 0x7ffb80109450 2024-09-16T01:57:55.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.732+0000 7ffb7cff9640 1 --2- 172.21.15.139:0/2549146788 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ffb680a34a0 0x7ffb680a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:55.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.733+0000 7ffb7cff9640 1 -- 172.21.15.139:0/2549146788 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9742+0+0 (secure 0 0 0) 0x7ffb700cfde0 con 0x7ffb80109450 2024-09-16T01:57:55.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.737+0000 7ffb7effd640 1 --2- 172.21.15.139:0/2549146788 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ffb680a34a0 0x7ffb680a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:55.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.738+0000 7ffb7cff9640 1 -- 172.21.15.139:0/2549146788 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb7009c300 con 0x7ffb80109450 2024-09-16T01:57:55.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:55.739+0000 7ffb7effd640 1 --2- 172.21.15.139:0/2549146788 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ffb680a34a0 0x7ffb680a5960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffb7402a8f0 tx=0x7ffb74000bc0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:56.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:56.143+0000 7ffb85e14640 1 -- 172.21.15.139:0/2549146788 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb80114f10 con 0x7ffb80109450 2024-09-16T01:57:56.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:56.148+0000 7ffb7cff9640 1 -- 172.21.15.139:0/2549146788 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+12031 (secure 0 0 0) 0x7ffb7009bc00 con 0x7ffb80109450 2024-09-16T01:57:56.150 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:57:56.150 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":75,"btime":"2024-09-16T01:57:49:138339+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":75,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:57:49.054365+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:57:56.151 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 75 2024-09-16T01:57:56.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:56.151+0000 7ffb85e14640 1 -- 172.21.15.139:0/2549146788 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ffb680a34a0 msgr2=0x7ffb680a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:56.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:56.151+0000 7ffb85e14640 1 --2- 172.21.15.139:0/2549146788 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ffb680a34a0 0x7ffb680a5960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffb7402a8f0 tx=0x7ffb74000bc0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:56.151+0000 7ffb85e14640 1 -- 172.21.15.139:0/2549146788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb80109450 msgr2=0x7ffb8011a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:56.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:56.152+0000 7ffb85e14640 1 --2- 172.21.15.139:0/2549146788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb80109450 0x7ffb8011a950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000d950 tx=0x7ffb7000de10 comp rx=0 tx=0).stop 2024-09-16T01:57:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:56.153+0000 7ffb85e14640 1 -- 172.21.15.139:0/2549146788 shutdown_connections 2024-09-16T01:57:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:56.153+0000 7ffb85e14640 1 --2- 172.21.15.139:0/2549146788 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7ffb680a34a0 0x7ffb680a5960 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:56.153+0000 7ffb85e14640 1 --2- 172.21.15.139:0/2549146788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb80113f20 0x7ffb801143d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:56.153+0000 7ffb85e14640 1 --2- 172.21.15.139:0/2549146788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb8011b140 0x7ffb801139e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:56.153+0000 7ffb85e14640 1 --2- 172.21.15.139:0/2549146788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb80109450 0x7ffb8011a950 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:56.153+0000 7ffb85e14640 1 -- 172.21.15.139:0/2549146788 >> 172.21.15.139:0/2549146788 conn(0x7ffb8006e140 msgr2=0x7ffb8010ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:56.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:56.155+0000 7ffb85e14640 1 -- 172.21.15.139:0/2549146788 shutdown_connections 2024-09-16T01:57:56.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:56.155+0000 7ffb85e14640 1 -- 172.21.15.139:0/2549146788 wait complete. 2024-09-16T01:57:56.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:56 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2549146788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:57:56.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:56 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2549146788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:57:56.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:56 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2549146788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:57:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.728+0000 7fd178786640 1 -- 172.21.15.139:0/43214524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd170101350 msgr2=0x7fd17010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.728+0000 7fd178786640 1 --2- 172.21.15.139:0/43214524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd170101350 0x7fd17010d880 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd168009cd0 tx=0x7fd16802a8e0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.728+0000 7fd178786640 1 -- 172.21.15.139:0/43214524 shutdown_connections 2024-09-16T01:57:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.728+0000 7fd178786640 1 --2- 172.21.15.139:0/43214524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd170101350 0x7fd17010d880 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.728+0000 7fd178786640 1 --2- 172.21.15.139:0/43214524 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1701009b0 0x7fd170100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.728+0000 7fd178786640 1 --2- 172.21.15.139:0/43214524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1701069c0 0x7fd170106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.728+0000 7fd178786640 1 -- 172.21.15.139:0/43214524 >> 172.21.15.139:0/43214524 conn(0x7fd170077990 msgr2=0x7fd1700fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.728+0000 7fd178786640 1 -- 172.21.15.139:0/43214524 shutdown_connections 2024-09-16T01:57:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.728+0000 7fd178786640 1 -- 172.21.15.139:0/43214524 wait complete. 2024-09-16T01:57:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd178786640 1 Processor -- start 2024-09-16T01:57:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd178786640 1 -- start start 2024-09-16T01:57:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd178786640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1701009b0 0x7fd1701982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd178786640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd170101350 0x7fd170198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd178786640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1701069c0 0x7fd17019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd178786640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd170110e00 con 0x7fd1701069c0 2024-09-16T01:57:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd178786640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd170110c80 con 0x7fd170101350 2024-09-16T01:57:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd178786640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd170110f80 con 0x7fd1701009b0 2024-09-16T01:57:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd176d23640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1701069c0 0x7fd17019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd176d23640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1701069c0 0x7fd17019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56066/0 (socket says 172.21.15.139:56066) 2024-09-16T01:57:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd176d23640 1 -- 172.21.15.139:0/3333964910 learned_addr learned my addr 172.21.15.139:0/3333964910 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd176522640 1 --2- 172.21.15.139:0/3333964910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1701009b0 0x7fd1701982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd176d23640 1 -- 172.21.15.139:0/3333964910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1701009b0 msgr2=0x7fd1701982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd176d23640 1 --2- 172.21.15.139:0/3333964910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1701009b0 0x7fd1701982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd176d23640 1 -- 172.21.15.139:0/3333964910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd170101350 msgr2=0x7fd170198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd176d23640 1 --2- 172.21.15.139:0/3333964910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd170101350 0x7fd170198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd176d23640 1 -- 172.21.15.139:0/3333964910 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17019d2c0 con 0x7fd1701069c0 2024-09-16T01:57:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd176d23640 1 --2- 172.21.15.139:0/3333964910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1701069c0 0x7fd17019cb90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd168005cc0 tx=0x7fd1680093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.729+0000 7fd1677fe640 1 -- 172.21.15.139:0/3333964910 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd168003e00 con 0x7fd1701069c0 2024-09-16T01:57:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.730+0000 7fd1677fe640 1 -- 172.21.15.139:0/3333964910 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd168004520 con 0x7fd1701069c0 2024-09-16T01:57:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.730+0000 7fd178786640 1 -- 172.21.15.139:0/3333964910 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17019d550 con 0x7fd1701069c0 2024-09-16T01:57:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.730+0000 7fd1677fe640 1 -- 172.21.15.139:0/3333964910 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd168030db0 con 0x7fd1701069c0 2024-09-16T01:57:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.730+0000 7fd178786640 1 -- 172.21.15.139:0/3333964910 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1701a4e40 con 0x7fd1701069c0 2024-09-16T01:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.732+0000 7fd1677fe640 1 -- 172.21.15.139:0/3333964910 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd1680046d0 con 0x7fd1701069c0 2024-09-16T01:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.732+0000 7fd17b5fc740 1 Processor -- start 2024-09-16T01:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.732+0000 7fd17b5fc740 1 -- start start 2024-09-16T01:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd17b5fc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b594059f0 0x557b59405dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd17b5fc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b59406310 0x557b59414b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd17b5fc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b594174b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd17b5fc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557b5936a2f0 con 0x557b59406310 2024-09-16T01:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd17b5fc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557b59312270 con 0x557b594059f0 2024-09-16T01:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd17b5fc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557b593662b0 con 0x557b594150c0 2024-09-16T01:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd1677fe640 1 --2- 172.21.15.139:0/3333964910 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd15c0a34e0 0x7fd15c0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd175d21640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b594059f0 0x557b59405dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd175d21640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b594059f0 0x557b59405dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:55930/0 (socket says 172.21.15.139:55930) 2024-09-16T01:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd175d21640 1 -- 172.21.15.139:0/2572066187 learned_addr learned my addr 172.21.15.139:0/2572066187 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd175d21640 1 --2- 172.21.15.139:0/2572066187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b594174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd176522640 1 --2- 172.21.15.139:0/2572066187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b59406310 0x557b59414b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd176d23640 1 --2- 172.21.15.139:0/3333964910 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd15c0a34e0 0x7fd15c0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd1677fe640 1 -- 172.21.15.139:0/3333964910 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9742+0+0 (secure 0 0 0) 0x7fd16802e030 con 0x7fd1701069c0 2024-09-16T01:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd175d21640 1 -- 172.21.15.139:0/2572066187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 msgr2=0x557b594174b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd175d21640 1 --2- 172.21.15.139:0/2572066187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b594174b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd175d21640 1 -- 172.21.15.139:0/2572066187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b59406310 msgr2=0x557b59414b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd175d21640 1 --2- 172.21.15.139:0/2572066187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b59406310 0x557b59414b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd175d21640 1 -- 172.21.15.139:0/2572066187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b59311340 con 0x557b594059f0 2024-09-16T01:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.733+0000 7fd176522640 1 --2- 172.21.15.139:0/2572066187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b59406310 0x557b59414b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T01:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.734+0000 7fd176d23640 1 --2- 172.21.15.139:0/3333964910 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd15c0a34e0 0x7fd15c0a59a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd1680ee9a0 tx=0x7fd16808c170 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.734+0000 7fd175d21640 1 --2- 172.21.15.139:0/2572066187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b594059f0 0x557b59405dd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x557b59407570 tx=0x7fd160009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ff619d2ac5ad26e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.734+0000 7fd164ff9640 1 -- 172.21.15.139:0/2572066187 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1600130d0 con 0x557b594059f0 2024-09-16T01:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.734+0000 7fd164ff9640 1 -- 172.21.15.139:0/2572066187 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd160013270 con 0x557b594059f0 2024-09-16T01:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.734+0000 7fd164ff9640 1 -- 172.21.15.139:0/2572066187 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1600035f0 con 0x557b594059f0 2024-09-16T01:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.735+0000 7fd17b5fc740 1 -- 172.21.15.139:0/2572066187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b594059f0 msgr2=0x557b59405dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.735+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/2572066187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b594059f0 0x557b59405dd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x557b59407570 tx=0x7fd160009650 comp rx=0 tx=0).stop 2024-09-16T01:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.735+0000 7fd17b5fc740 1 -- 172.21.15.139:0/2572066187 shutdown_connections 2024-09-16T01:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.735+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/2572066187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b594174b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.735+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/2572066187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b59406310 0x557b59414b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.735+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/2572066187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b594059f0 0x557b59405dd0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.735+0000 7fd17b5fc740 1 -- 172.21.15.139:0/2572066187 >> 172.21.15.139:0/2572066187 conn(0x557b59402060 msgr2=0x557b59402440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.735+0000 7fd17b5fc740 1 -- 172.21.15.139:0/2572066187 shutdown_connections 2024-09-16T01:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.735+0000 7fd17b5fc740 1 -- 172.21.15.139:0/2572066187 wait complete. 2024-09-16T01:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.735+0000 7fd17b5fc740 1 -- start start 2024-09-16T01:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.735+0000 7fd17b5fc740 1 Processor -- start 2024-09-16T01:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd17b5fc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b594059f0 0x557b59411f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd17b5fc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b59406310 0x557b59412490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd17b5fc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b594129d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd17b5fc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x557b59417cc0 con 0x557b594059f0 2024-09-16T01:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd17b5fc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557b592f3290 con 0x557b59406310 2024-09-16T01:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd17b5fc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557b59417e40 con 0x557b594150c0 2024-09-16T01:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd17b5fc740 10 client.0 _ll_register_callbacks cb 0x557b59324140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd175d21640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b594059f0 0x557b59411f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd175d21640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b594059f0 0x557b59411f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56086/0 (socket says 172.21.15.139:56086) 2024-09-16T01:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd175d21640 1 -- 172.21.15.139:0/3534432380 learned_addr learned my addr 172.21.15.139:0/3534432380 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd176522640 1 --2- 172.21.15.139:0/3534432380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b59406310 0x557b59412490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd175d21640 1 --2- 172.21.15.139:0/3534432380 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b594129d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd175d21640 1 -- 172.21.15.139:0/3534432380 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 msgr2=0x557b594129d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd175d21640 1 --2- 172.21.15.139:0/3534432380 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b594129d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd175d21640 1 -- 172.21.15.139:0/3534432380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b59406310 msgr2=0x557b59412490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd175d21640 1 --2- 172.21.15.139:0/3534432380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b59406310 0x557b59412490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd175d21640 1 -- 172.21.15.139:0/3534432380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b5907d210 con 0x557b594059f0 2024-09-16T01:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd175d21640 1 --2- 172.21.15.139:0/3534432380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b594059f0 0x557b59411f50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd160004700 tx=0x7fd160009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e36d40a2a5bbfb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd159ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd159ffb640 1 -- 172.21.15.139:0/3534432380 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160026030 con 0x557b594059f0 2024-09-16T01:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd159ffb640 1 -- 172.21.15.139:0/3534432380 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9742+0+0 (secure 0 0 0) 0x7fd16000ee00 con 0x557b594059f0 2024-09-16T01:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.736+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3534432380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd160013270 con 0x557b594059f0 2024-09-16T01:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.737+0000 7fd159ffb640 1 -- 172.21.15.139:0/3534432380 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fd160033050 con 0x557b594059f0 2024-09-16T01:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.737+0000 7fd17b5fc740 10 client.17256 fetch_fsmap learned FSMap version 75 2024-09-16T01:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.737+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3534432380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x557b59149620 con 0x557b594059f0 2024-09-16T01:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.737+0000 7fd159ffb640 1 -- 172.21.15.139:0/3534432380 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd160013270 con 0x557b594059f0 2024-09-16T01:57:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.737+0000 7fd159ffb640 1 -- 172.21.15.139:0/3534432380 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 75) ==== 8450+0+0 (secure 0 0 0) 0x7fd16002d530 con 0x557b594059f0 2024-09-16T01:57:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.737+0000 7fd17b5fc740 10 client.17256 fetch_fsmap finished waiting for FSMap version 75 2024-09-16T01:57:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.737+0000 7fd17b5fc740 10 client.17256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:57:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.737+0000 7fd17b5fc740 20 client.17256 populate_metadata read hostname 'smithi139' 2024-09-16T01:57:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.737+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/3534432380 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b59424ed0 0x557b594272c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.737+0000 7fd17b5fc740 4 client.17256 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:57:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.737+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3534432380 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b59377a20 con 0x557b59424ed0 2024-09-16T01:57:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.737+0000 7fd176522640 1 --2- 172.21.15.139:0/3534432380 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b59424ed0 0x557b594272c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.738+0000 7fd176522640 1 --2- 172.21.15.139:0/3534432380 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b59424ed0 0x557b594272c0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9362ba881fffe609 server_cookie=ac82fb03e452594d in_seq=0 out_seq=0 2024-09-16T01:57:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.738+0000 7fd159ffb640 10 client.17256 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.738+0000 7fd159ffb640 1 -- 172.21.15.139:0/3534432380 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b59377a20 con 0x557b59424ed0 2024-09-16T01:57:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.738+0000 7fd159ffb640 10 client.17256 handle_command_reply: tid=0 2024-09-16T01:57:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.739+0000 7fd17b5fc740 1 client.17256 shutdown 2024-09-16T01:57:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.739+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3534432380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b594059f0 msgr2=0x557b59411f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.739+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/3534432380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b594059f0 0x557b59411f50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd160004700 tx=0x7fd160009f70 comp rx=0 tx=0).stop 2024-09-16T01:57:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.739+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3534432380 shutdown_connections 2024-09-16T01:57:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.739+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/3534432380 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b59424ed0 0x557b594272c0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.739+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/3534432380 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b594129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.739+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/3534432380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b59406310 0x557b59412490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.739+0000 7fd159ffb640 0 client.17256 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.739+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/3534432380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b594059f0 0x557b59411f50 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.739+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3534432380 >> 172.21.15.139:0/3534432380 conn(0x557b59402060 msgr2=0x557b59407dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.739+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3534432380 shutdown_connections 2024-09-16T01:57:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.739+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3534432380 wait complete. 2024-09-16T01:57:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.739+0000 7fd17b5fc740 20 client.17256 trim_cache size 0 max 16384 2024-09-16T01:57:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.753+0000 7fd17b5fc740 1 Processor -- start 2024-09-16T01:57:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.753+0000 7fd17b5fc740 1 -- start start 2024-09-16T01:57:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.753+0000 7fd17b5fc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b594059f0 0x557b59421680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.753+0000 7fd17b5fc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b59406310 0x557b59413ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.753+0000 7fd17b5fc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b5942c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.753+0000 7fd17b5fc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701acfc0 con 0x557b59406310 2024-09-16T01:57:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.753+0000 7fd17b5fc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701aee00 con 0x557b594059f0 2024-09-16T01:57:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.753+0000 7fd17b5fc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701ad750 con 0x557b594150c0 2024-09-16T01:57:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd176522640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b59406310 0x557b59413ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd176522640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b59406310 0x557b59413ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56090/0 (socket says 172.21.15.139:56090) 2024-09-16T01:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd176522640 1 -- 172.21.15.139:0/4197900111 learned_addr learned my addr 172.21.15.139:0/4197900111 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd175d21640 1 --2- 172.21.15.139:0/4197900111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b594059f0 0x557b59421680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd175d21640 1 --2- 172.21.15.139:0/4197900111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b5942c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd176522640 1 -- 172.21.15.139:0/4197900111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 msgr2=0x557b5942c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd176522640 1 --2- 172.21.15.139:0/4197900111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b5942c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd176522640 1 -- 172.21.15.139:0/4197900111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b594059f0 msgr2=0x557b59421680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd176522640 1 --2- 172.21.15.139:0/4197900111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b594059f0 0x557b59421680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd176522640 1 -- 172.21.15.139:0/4197900111 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b59401e90 con 0x557b59406310 2024-09-16T01:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd175d21640 1 --2- 172.21.15.139:0/4197900111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b5942c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd176522640 1 --2- 172.21.15.139:0/4197900111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b59406310 0x557b59413ec0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd16c000c00 tx=0x7fd16c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545b8d5fbbd05ae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd15bfff640 1 -- 172.21.15.139:0/4197900111 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c004050 con 0x557b59406310 2024-09-16T01:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd15bfff640 1 -- 172.21.15.139:0/4197900111 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd16c0041f0 con 0x557b59406310 2024-09-16T01:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.754+0000 7fd15bfff640 1 -- 172.21.15.139:0/4197900111 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c011860 con 0x557b59406310 2024-09-16T01:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.755+0000 7fd17b5fc740 1 -- 172.21.15.139:0/4197900111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b59406310 msgr2=0x557b59413ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.755+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/4197900111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b59406310 0x557b59413ec0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd16c000c00 tx=0x7fd16c005f90 comp rx=0 tx=0).stop 2024-09-16T01:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.755+0000 7fd17b5fc740 1 -- 172.21.15.139:0/4197900111 shutdown_connections 2024-09-16T01:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.755+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/4197900111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b5942c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.755+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/4197900111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b59406310 0x557b59413ec0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.755+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/4197900111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b594059f0 0x557b59421680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.755+0000 7fd17b5fc740 1 -- 172.21.15.139:0/4197900111 >> 172.21.15.139:0/4197900111 conn(0x557b59402060 msgr2=0x557b594137b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.755+0000 7fd17b5fc740 1 -- 172.21.15.139:0/4197900111 shutdown_connections 2024-09-16T01:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.755+0000 7fd17b5fc740 1 -- 172.21.15.139:0/4197900111 wait complete. 2024-09-16T01:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.755+0000 7fd17b5fc740 1 -- start start 2024-09-16T01:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.755+0000 7fd17b5fc740 1 Processor -- start 2024-09-16T01:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.755+0000 7fd17b5fc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b594059f0 0x557b5941d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd17b5fc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b59406310 0x557b5940ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd17b5fc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b5940f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd17b5fc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701ae670 con 0x557b594059f0 2024-09-16T01:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd17b5fc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701adee0 con 0x557b59406310 2024-09-16T01:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd17b5fc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701aafa0 con 0x557b594150c0 2024-09-16T01:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd17b5fc740 10 client.0 _ll_register_callbacks cb 0x7fd16c009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd175d21640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b594059f0 0x557b5941d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd175d21640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b594059f0 0x557b5941d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56100/0 (socket says 172.21.15.139:56100) 2024-09-16T01:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd175d21640 1 -- 172.21.15.139:0/3159822081 learned_addr learned my addr 172.21.15.139:0/3159822081 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd176522640 1 --2- 172.21.15.139:0/3159822081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b59406310 0x557b5940ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd175d21640 1 --2- 172.21.15.139:0/3159822081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b5940f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd175d21640 1 -- 172.21.15.139:0/3159822081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 msgr2=0x557b5940f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd175d21640 1 --2- 172.21.15.139:0/3159822081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b5940f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd175d21640 1 -- 172.21.15.139:0/3159822081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b59406310 msgr2=0x557b5940ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd175d21640 1 --2- 172.21.15.139:0/3159822081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b59406310 0x557b5940ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd175d21640 1 -- 172.21.15.139:0/3159822081 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b5940cd50 con 0x557b594059f0 2024-09-16T01:57:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd175d21640 1 --2- 172.21.15.139:0/3159822081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b594059f0 0x557b5941d7c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd160001eb0 tx=0x7fd160009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617bd2f40abf82c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd159ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:57:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd159ffb640 1 -- 172.21.15.139:0/3159822081 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160026040 con 0x557b594059f0 2024-09-16T01:57:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd159ffb640 1 -- 172.21.15.139:0/3159822081 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9742+0+0 (secure 0 0 0) 0x7fd16000eca0 con 0x557b594059f0 2024-09-16T01:57:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3159822081 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd16c0041f0 con 0x557b594059f0 2024-09-16T01:57:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd159ffb640 1 -- 172.21.15.139:0/3159822081 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fd160040050 con 0x557b594059f0 2024-09-16T01:57:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd17b5fc740 10 client.17262 fetch_fsmap learned FSMap version 75 2024-09-16T01:57:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.756+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3159822081 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x557b5948f720 con 0x557b594059f0 2024-09-16T01:57:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.757+0000 7fd159ffb640 1 -- 172.21.15.139:0/3159822081 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd16c0041f0 con 0x557b594059f0 2024-09-16T01:57:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.757+0000 7fd159ffb640 1 -- 172.21.15.139:0/3159822081 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 75) ==== 8450+0+0 (secure 0 0 0) 0x7fd16002d110 con 0x557b594059f0 2024-09-16T01:57:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.757+0000 7fd17b5fc740 10 client.17262 fetch_fsmap finished waiting for FSMap version 75 2024-09-16T01:57:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.757+0000 7fd17b5fc740 10 client.17262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:57:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.757+0000 7fd17b5fc740 20 client.17262 populate_metadata read hostname 'smithi139' 2024-09-16T01:57:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.757+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/3159822081 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b5948fe20 0x557b59492240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.757+0000 7fd17b5fc740 4 client.17262 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:57:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.757+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3159822081 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b59371d70 con 0x557b5948fe20 2024-09-16T01:57:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.757+0000 7fd176522640 1 --2- 172.21.15.139:0/3159822081 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b5948fe20 0x557b59492240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.757+0000 7fd176522640 1 --2- 172.21.15.139:0/3159822081 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b5948fe20 0x557b59492240 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b073f42e596638 server_cookie=30c1f244d70a0c42 in_seq=0 out_seq=0 2024-09-16T01:57:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.757+0000 7fd159ffb640 10 client.17262 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.876+0000 7fd159ffb640 1 -- 172.21.15.139:0/3159822081 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b59371d70 con 0x557b5948fe20 2024-09-16T01:57:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.876+0000 7fd159ffb640 10 client.17262 handle_command_reply: tid=0 2024-09-16T01:57:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.876+0000 7fd17b5fc740 1 client.17262 shutdown 2024-09-16T01:57:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.876+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3159822081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b594059f0 msgr2=0x557b5941d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.876+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/3159822081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b594059f0 0x557b5941d7c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd160001eb0 tx=0x7fd160009b20 comp rx=0 tx=0).stop 2024-09-16T01:57:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.876+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3159822081 shutdown_connections 2024-09-16T01:57:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.876+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/3159822081 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x557b5948fe20 0x557b59492240 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.876+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/3159822081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557b594150c0 0x557b5940f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.876+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/3159822081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557b59406310 0x557b5940ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.876+0000 7fd17b5fc740 1 --2- 172.21.15.139:0/3159822081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557b594059f0 0x557b5941d7c0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.876+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3159822081 >> 172.21.15.139:0/3159822081 conn(0x557b59402060 msgr2=0x557b594074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.876+0000 7fd159ffb640 0 client.17262 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.877+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3159822081 shutdown_connections 2024-09-16T01:57:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.877+0000 7fd17b5fc740 1 -- 172.21.15.139:0/3159822081 wait complete. 2024-09-16T01:57:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.877+0000 7fd17b5fc740 20 client.17262 trim_cache size 0 max 16384 2024-09-16T01:57:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:57:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:57:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:57:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:57:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.878+0000 7fd178786640 1 -- 172.21.15.139:0/3333964910 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd15c0a34e0 msgr2=0x7fd15c0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.878+0000 7fd178786640 1 --2- 172.21.15.139:0/3333964910 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd15c0a34e0 0x7fd15c0a59a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd1680ee9a0 tx=0x7fd16808c170 comp rx=0 tx=0).stop 2024-09-16T01:57:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.878+0000 7fd178786640 1 -- 172.21.15.139:0/3333964910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1701069c0 msgr2=0x7fd17019cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.878+0000 7fd178786640 1 --2- 172.21.15.139:0/3333964910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1701069c0 0x7fd17019cb90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd168005cc0 tx=0x7fd1680093d0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.878+0000 7fd178786640 1 -- 172.21.15.139:0/3333964910 shutdown_connections 2024-09-16T01:57:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.878+0000 7fd178786640 1 --2- 172.21.15.139:0/3333964910 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd15c0a34e0 0x7fd15c0a59a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.878+0000 7fd178786640 1 --2- 172.21.15.139:0/3333964910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1701069c0 0x7fd17019cb90 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.878+0000 7fd178786640 1 --2- 172.21.15.139:0/3333964910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd170101350 0x7fd170198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.878+0000 7fd178786640 1 --2- 172.21.15.139:0/3333964910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1701009b0 0x7fd1701982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.878+0000 7fd178786640 1 -- 172.21.15.139:0/3333964910 >> 172.21.15.139:0/3333964910 conn(0x7fd170077990 msgr2=0x7fd170069a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.878+0000 7fd178786640 1 -- 172.21.15.139:0/3333964910 shutdown_connections 2024-09-16T01:57:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:57:56.878+0000 7fd178786640 1 -- 172.21.15.139:0/3333964910 wait complete. 2024-09-16T01:57:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:57 smithi164 ceph-mon[35121]: pgmap v557: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2024-09-16T01:57:57.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:57 smithi139 ceph-mon[31175]: pgmap v557: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2024-09-16T01:57:57.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:57 smithi176 ceph-mon[35324]: pgmap v557: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2024-09-16T01:57:58.515 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 142.95 --await-for 45.0' 2024-09-16T01:57:58.516 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 142.95 --await-for 45.0 2024-09-16T01:57:59.018 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:57:59.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:57:59 smithi139 ceph-mon[31175]: pgmap v558: 129 pgs: 22 active+clean+snaptrim, 68 active+clean+snaptrim_wait, 39 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-09-16T01:57:59.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:57:59 smithi164 ceph-mon[35121]: pgmap v558: 129 pgs: 22 active+clean+snaptrim, 68 active+clean+snaptrim_wait, 39 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-09-16T01:57:59.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:57:59 smithi176 ceph-mon[35324]: pgmap v558: 129 pgs: 22 active+clean+snaptrim, 68 active+clean+snaptrim_wait, 39 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-09-16T01:57:59.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.686+0000 7f6d8332a640 1 -- 172.21.15.139:0/790810887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c104e80 msgr2=0x7f6d7c105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:59.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.686+0000 7f6d8332a640 1 --2- 172.21.15.139:0/790810887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c104e80 0x7f6d7c105330 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009ca0 tx=0x7f6d7802a8f0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.687+0000 7f6d8332a640 1 -- 172.21.15.139:0/790810887 shutdown_connections 2024-09-16T01:57:59.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.687+0000 7f6d8332a640 1 --2- 172.21.15.139:0/790810887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c104e80 0x7f6d7c105330 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.687+0000 7f6d8332a640 1 --2- 172.21.15.139:0/790810887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d7c06a370 0x7f6d7c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.687+0000 7f6d8332a640 1 --2- 172.21.15.139:0/790810887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c069f00 0x7f6d7c100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.687+0000 7f6d8332a640 1 -- 172.21.15.139:0/790810887 >> 172.21.15.139:0/790810887 conn(0x7f6d7c077210 msgr2=0x7f6d7c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:59.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.687+0000 7f6d8332a640 1 -- 172.21.15.139:0/790810887 shutdown_connections 2024-09-16T01:57:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.687+0000 7f6d8332a640 1 -- 172.21.15.139:0/790810887 wait complete. 2024-09-16T01:57:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.688+0000 7f6d8332a640 1 Processor -- start 2024-09-16T01:57:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.688+0000 7f6d8332a640 1 -- start start 2024-09-16T01:57:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.688+0000 7f6d8332a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d7c069f00 0x7f6d7c1a7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.688+0000 7f6d8332a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c06a370 0x7f6d7c1a7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.688+0000 7f6d8332a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c104e80 0x7f6d7c1abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.688+0000 7f6d8332a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c10ddb0 con 0x7f6d7c06a370 2024-09-16T01:57:59.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.688+0000 7f6d8332a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c10dc30 con 0x7f6d7c104e80 2024-09-16T01:57:59.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.688+0000 7f6d8332a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c10df30 con 0x7f6d7c069f00 2024-09-16T01:57:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.689+0000 7f6d808c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c06a370 0x7f6d7c1a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.689+0000 7f6d808c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c06a370 0x7f6d7c1a7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56108/0 (socket says 172.21.15.139:56108) 2024-09-16T01:57:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.689+0000 7f6d808c5640 1 -- 172.21.15.139:0/1193494900 learned_addr learned my addr 172.21.15.139:0/1193494900 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.689+0000 7f6d810c6640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d7c069f00 0x7f6d7c1a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.689+0000 7f6d818c7640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c104e80 0x7f6d7c1abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.689+0000 7f6d808c5640 1 -- 172.21.15.139:0/1193494900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d7c069f00 msgr2=0x7f6d7c1a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.689+0000 7f6d808c5640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d7c069f00 0x7f6d7c1a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.689+0000 7f6d808c5640 1 -- 172.21.15.139:0/1193494900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c104e80 msgr2=0x7f6d7c1abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:59.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.689+0000 7f6d808c5640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c104e80 0x7f6d7c1abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.689+0000 7f6d808c5640 1 -- 172.21.15.139:0/1193494900 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7c1ac1f0 con 0x7f6d7c06a370 2024-09-16T01:57:59.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.690+0000 7f6d818c7640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c104e80 0x7f6d7c1abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:57:59.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.690+0000 7f6d808c5640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c06a370 0x7f6d7c1a7760 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00b450 tx=0x7f6d6c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:59.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.691+0000 7f6d727fc640 1 -- 172.21.15.139:0/1193494900 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c015020 con 0x7f6d7c06a370 2024-09-16T01:57:59.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.691+0000 7f6d8332a640 1 -- 172.21.15.139:0/1193494900 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7c1ac4e0 con 0x7f6d7c06a370 2024-09-16T01:57:59.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.692+0000 7f6d8332a640 1 -- 172.21.15.139:0/1193494900 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d7c06bc10 con 0x7f6d7c06a370 2024-09-16T01:57:59.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.693+0000 7f6d727fc640 1 -- 172.21.15.139:0/1193494900 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c011d70 con 0x7f6d7c06a370 2024-09-16T01:57:59.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.693+0000 7f6d727fc640 1 -- 172.21.15.139:0/1193494900 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c00f340 con 0x7f6d7c06a370 2024-09-16T01:57:59.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.696+0000 7f6d727fc640 1 -- 172.21.15.139:0/1193494900 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6d6c018020 con 0x7f6d7c06a370 2024-09-16T01:57:59.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.697+0000 7f6d727fc640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6d680a35c0 0x7f6d680a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.698+0000 7f6d727fc640 1 -- 172.21.15.139:0/1193494900 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9742+0+0 (secure 0 0 0) 0x7f6d6c0d0b70 con 0x7f6d7c06a370 2024-09-16T01:57:59.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.699+0000 7f6d810c6640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6d680a35c0 0x7f6d680a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.716+0000 7f6d810c6640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6d680a35c0 0x7f6d680a5a80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400abd0 tx=0x7f6d7400a380 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:59.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.716+0000 7f6d85703740 1 Processor -- start 2024-09-16T01:57:59.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.717+0000 7f6d85703740 1 -- start start 2024-09-16T01:57:59.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.717+0000 7f6d85703740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711de6850 0x55e711de6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.717+0000 7f6d85703740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711df14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.717+0000 7f6d85703740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711df1a30 0x55e711df3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.717+0000 7f6d85703740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e711d4b160 con 0x55e711de7170 2024-09-16T01:57:59.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.717+0000 7f6d85703740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e711cf2720 con 0x55e711de6850 2024-09-16T01:57:59.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.717+0000 7f6d85703740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e711d47120 con 0x55e711df1a30 2024-09-16T01:57:59.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.721+0000 7f6d818c7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711df1a30 0x55e711df3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.721+0000 7f6d808c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711df14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.721+0000 7f6d808c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711df14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56122/0 (socket says 172.21.15.139:56122) 2024-09-16T01:57:59.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.721+0000 7f6d808c5640 1 -- 172.21.15.139:0/732374479 learned_addr learned my addr 172.21.15.139:0/732374479 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:59.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.721+0000 7f6d808c5640 1 -- 172.21.15.139:0/732374479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711df1a30 msgr2=0x55e711df3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:59.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.721+0000 7f6d808c5640 1 --2- 172.21.15.139:0/732374479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711df1a30 0x55e711df3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.721+0000 7f6d808c5640 1 -- 172.21.15.139:0/732374479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711de6850 msgr2=0x55e711de6c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:57:59.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.721+0000 7f6d808c5640 1 --2- 172.21.15.139:0/732374479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711de6850 0x55e711de6c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.721+0000 7f6d808c5640 1 -- 172.21.15.139:0/732374479 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e711abcba0 con 0x55e711de7170 2024-09-16T01:57:59.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.721+0000 7f6d808c5640 1 --2- 172.21.15.139:0/732374479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711df14f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c010b40 tx=0x7f6d6c09d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d11a8b250707758 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:59.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.722+0000 7f6d677fe640 1 -- 172.21.15.139:0/732374479 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0a56b0 con 0x55e711de7170 2024-09-16T01:57:59.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.722+0000 7f6d677fe640 1 -- 172.21.15.139:0/732374479 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c09ad40 con 0x55e711de7170 2024-09-16T01:57:59.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.722+0000 7f6d677fe640 1 -- 172.21.15.139:0/732374479 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0a5480 con 0x55e711de7170 2024-09-16T01:57:59.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.723+0000 7f6d85703740 1 -- 172.21.15.139:0/732374479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 msgr2=0x55e711df14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:59.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.723+0000 7f6d85703740 1 --2- 172.21.15.139:0/732374479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711df14f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c010b40 tx=0x7f6d6c09d280 comp rx=0 tx=0).stop 2024-09-16T01:57:59.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.725+0000 7f6d85703740 1 -- 172.21.15.139:0/732374479 shutdown_connections 2024-09-16T01:57:59.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.725+0000 7f6d85703740 1 --2- 172.21.15.139:0/732374479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711df1a30 0x55e711df3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.725+0000 7f6d85703740 1 --2- 172.21.15.139:0/732374479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711df14f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.725+0000 7f6d85703740 1 --2- 172.21.15.139:0/732374479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711de6850 0x55e711de6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.725+0000 7f6d85703740 1 -- 172.21.15.139:0/732374479 >> 172.21.15.139:0/732374479 conn(0x55e711de2de0 msgr2=0x55e711de31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:59.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.730+0000 7f6d85703740 1 -- 172.21.15.139:0/732374479 shutdown_connections 2024-09-16T01:57:59.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.730+0000 7f6d85703740 1 -- 172.21.15.139:0/732374479 wait complete. 2024-09-16T01:57:59.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.730+0000 7f6d85703740 1 -- start start 2024-09-16T01:57:59.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.730+0000 7f6d85703740 1 Processor -- start 2024-09-16T01:57:59.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.731+0000 7f6d85703740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711de6850 0x55e711deea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.731+0000 7f6d85703740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711deef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.731+0000 7f6d85703740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711df1a30 0x55e711def490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.731+0000 7f6d85703740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e711df4600 con 0x55e711de7170 2024-09-16T01:57:59.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.731+0000 7f6d85703740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e711cd3900 con 0x55e711de6850 2024-09-16T01:57:59.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.731+0000 7f6d85703740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e711df4780 con 0x55e711df1a30 2024-09-16T01:57:59.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.731+0000 7f6d85703740 10 client.0 _ll_register_callbacks cb 0x55e711cd1780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:57:59.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.732+0000 7f6d808c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711deef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.732+0000 7f6d808c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711deef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56134/0 (socket says 172.21.15.139:56134) 2024-09-16T01:57:59.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.732+0000 7f6d808c5640 1 -- 172.21.15.139:0/2151848746 learned_addr learned my addr 172.21.15.139:0/2151848746 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:59.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.732+0000 7f6d818c7640 1 --2- 172.21.15.139:0/2151848746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711df1a30 0x55e711def490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.732+0000 7f6d810c6640 1 --2- 172.21.15.139:0/2151848746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711de6850 0x55e711deea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.732+0000 7f6d808c5640 1 -- 172.21.15.139:0/2151848746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711df1a30 msgr2=0x55e711def490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:59.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.732+0000 7f6d808c5640 1 --2- 172.21.15.139:0/2151848746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711df1a30 0x55e711def490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.732+0000 7f6d808c5640 1 -- 172.21.15.139:0/2151848746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711de6850 msgr2=0x55e711deea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:59.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.733+0000 7f6d808c5640 1 --2- 172.21.15.139:0/2151848746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711de6850 0x55e711deea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.733+0000 7f6d808c5640 1 -- 172.21.15.139:0/2151848746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e711d13c50 con 0x55e711de7170 2024-09-16T01:57:59.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.733+0000 7f6d808c5640 1 --2- 172.21.15.139:0/2151848746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711deef50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0a0fb0 tx=0x7f6d6c010a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9e3502e6dfaf13 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:59.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.734+0000 7f6d64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:57:59.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.736+0000 7f6d64ff9640 1 -- 172.21.15.139:0/2151848746 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0a56b0 con 0x55e711de7170 2024-09-16T01:57:59.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.736+0000 7f6d85703740 1 -- 172.21.15.139:0/2151848746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6c09ad40 con 0x55e711de7170 2024-09-16T01:57:59.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.736+0000 7f6d64ff9640 1 -- 172.21.15.139:0/2151848746 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9742+0+0 (secure 0 0 0) 0x7f6d6c0a5430 con 0x55e711de7170 2024-09-16T01:57:59.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.737+0000 7f6d64ff9640 1 -- 172.21.15.139:0/2151848746 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c0bb2b0 con 0x55e711de7170 2024-09-16T01:57:59.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.737+0000 7f6d85703740 10 client.17274 fetch_fsmap learned FSMap version 76 2024-09-16T01:57:59.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.737+0000 7f6d85703740 1 -- 172.21.15.139:0/2151848746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55e711e003f0 con 0x55e711de7170 2024-09-16T01:57:59.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.739+0000 7f6d64ff9640 1 -- 172.21.15.139:0/2151848746 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c09ad40 con 0x55e711de7170 2024-09-16T01:57:59.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.740+0000 7f6d64ff9640 1 -- 172.21.15.139:0/2151848746 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 76) ==== 8446+0+0 (secure 0 0 0) 0x7f6d6c098990 con 0x55e711de7170 2024-09-16T01:57:59.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.740+0000 7f6d85703740 10 client.17274 fetch_fsmap finished waiting for FSMap version 76 2024-09-16T01:57:59.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.740+0000 7f6d85703740 10 client.17274 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:57:59.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.740+0000 7f6d85703740 20 client.17274 populate_metadata read hostname 'smithi139' 2024-09-16T01:57:59.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.740+0000 7f6d85703740 1 --2- 172.21.15.139:0/2151848746 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e711e019b0 0x55e711e03dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.740+0000 7f6d85703740 4 client.17274 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:57:59.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.740+0000 7f6d85703740 1 -- 172.21.15.139:0/2151848746 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e711d58890 con 0x55e711e019b0 2024-09-16T01:57:59.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.743+0000 7f6d810c6640 1 --2- 172.21.15.139:0/2151848746 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e711e019b0 0x55e711e03dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.745+0000 7f6d810c6640 1 --2- 172.21.15.139:0/2151848746 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e711e019b0 0x55e711e03dd0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16e6de68428c1cda server_cookie=380ed228c9d477a7 in_seq=0 out_seq=0 2024-09-16T01:57:59.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.748+0000 7f6d64ff9640 10 client.17274 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:59.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.748+0000 7f6d64ff9640 1 -- 172.21.15.139:0/2151848746 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e711d58890 con 0x55e711e019b0 2024-09-16T01:57:59.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.748+0000 7f6d64ff9640 10 client.17274 handle_command_reply: tid=0 2024-09-16T01:57:59.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.748+0000 7f6d85703740 1 client.17274 shutdown 2024-09-16T01:57:59.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.757+0000 7f6d85703740 1 -- 172.21.15.139:0/2151848746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 msgr2=0x55e711deef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:59.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.757+0000 7f6d85703740 1 --2- 172.21.15.139:0/2151848746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711deef50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0a0fb0 tx=0x7f6d6c010a20 comp rx=0 tx=0).stop 2024-09-16T01:57:59.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.762+0000 7f6d85703740 1 -- 172.21.15.139:0/2151848746 shutdown_connections 2024-09-16T01:57:59.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.762+0000 7f6d85703740 1 --2- 172.21.15.139:0/2151848746 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e711e019b0 0x55e711e03dd0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.762+0000 7f6d85703740 1 --2- 172.21.15.139:0/2151848746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711df1a30 0x55e711def490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.762+0000 7f6d85703740 1 --2- 172.21.15.139:0/2151848746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711deef50 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.762+0000 7f6d85703740 1 --2- 172.21.15.139:0/2151848746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711de6850 0x55e711deea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.762+0000 7f6d85703740 1 -- 172.21.15.139:0/2151848746 >> 172.21.15.139:0/2151848746 conn(0x55e711de2de0 msgr2=0x55e711de8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:59.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.762+0000 7f6d64ff9640 0 client.17274 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:57:59.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.762+0000 7f6d85703740 1 -- 172.21.15.139:0/2151848746 shutdown_connections 2024-09-16T01:57:59.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.762+0000 7f6d85703740 1 -- 172.21.15.139:0/2151848746 wait complete. 2024-09-16T01:57:59.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.763+0000 7f6d85703740 20 client.17274 trim_cache size 0 max 16384 2024-09-16T01:57:59.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.795+0000 7f6d85703740 1 Processor -- start 2024-09-16T01:57:59.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.798+0000 7f6d85703740 1 -- start start 2024-09-16T01:57:59.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.798+0000 7f6d85703740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711de6850 0x55e711df0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.798+0000 7f6d85703740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711e09660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.798+0000 7f6d85703740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711df1a30 0x55e711e09ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.798+0000 7f6d85703740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6d600101a0 con 0x55e711de7170 2024-09-16T01:57:59.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.798+0000 7f6d85703740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60011fb0 con 0x55e711df1a30 2024-09-16T01:57:59.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.798+0000 7f6d85703740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60010900 con 0x55e711de6850 2024-09-16T01:57:59.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.798+0000 7f6d810c6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711de6850 0x55e711df0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.798+0000 7f6d810c6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711de6850 0x55e711df0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:42422/0 (socket says 172.21.15.139:42422) 2024-09-16T01:57:59.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.798+0000 7f6d810c6640 1 -- 172.21.15.139:0/3226972753 learned_addr learned my addr 172.21.15.139:0/3226972753 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:59.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.798+0000 7f6d818c7640 1 --2- 172.21.15.139:0/3226972753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711df1a30 0x55e711e09ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.799+0000 7f6d810c6640 1 -- 172.21.15.139:0/3226972753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711df1a30 msgr2=0x55e711e09ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:59.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.799+0000 7f6d810c6640 1 --2- 172.21.15.139:0/3226972753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711df1a30 0x55e711e09ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.799+0000 7f6d810c6640 1 -- 172.21.15.139:0/3226972753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 msgr2=0x55e711e09660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:57:59.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.799+0000 7f6d810c6640 1 --2- 172.21.15.139:0/3226972753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711e09660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.799+0000 7f6d810c6640 1 -- 172.21.15.139:0/3226972753 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e711e04140 con 0x55e711de6850 2024-09-16T01:57:59.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.799+0000 7f6d810c6640 1 --2- 172.21.15.139:0/3226972753 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711de6850 0x55e711df0980 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400fe80 tx=0x7f6d74013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f53ce69f0da1d4a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.803+0000 7f6d66ffd640 1 -- 172.21.15.139:0/3226972753 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7402b070 con 0x55e711de6850 2024-09-16T01:57:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.803+0000 7f6d66ffd640 1 -- 172.21.15.139:0/3226972753 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d74005510 con 0x55e711de6850 2024-09-16T01:57:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.803+0000 7f6d66ffd640 1 -- 172.21.15.139:0/3226972753 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d74017070 con 0x55e711de6850 2024-09-16T01:57:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.806+0000 7f6d85703740 1 -- 172.21.15.139:0/3226972753 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711de6850 msgr2=0x55e711df0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:57:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.807+0000 7f6d85703740 1 --2- 172.21.15.139:0/3226972753 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711de6850 0x55e711df0980 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400fe80 tx=0x7f6d74013000 comp rx=0 tx=0).stop 2024-09-16T01:57:59.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.810+0000 7f6d85703740 1 -- 172.21.15.139:0/3226972753 shutdown_connections 2024-09-16T01:57:59.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.810+0000 7f6d85703740 1 --2- 172.21.15.139:0/3226972753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711df1a30 0x55e711e09ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.810+0000 7f6d85703740 1 --2- 172.21.15.139:0/3226972753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711de7170 0x55e711e09660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.810+0000 7f6d85703740 1 --2- 172.21.15.139:0/3226972753 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711de6850 0x55e711df0980 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.810+0000 7f6d85703740 1 -- 172.21.15.139:0/3226972753 >> 172.21.15.139:0/3226972753 conn(0x55e711de2de0 msgr2=0x55e711df0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:57:59.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.812+0000 7f6d85703740 1 -- 172.21.15.139:0/3226972753 shutdown_connections 2024-09-16T01:57:59.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.812+0000 7f6d85703740 1 -- 172.21.15.139:0/3226972753 wait complete. 2024-09-16T01:57:59.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.812+0000 7f6d85703740 1 -- start start 2024-09-16T01:57:59.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.812+0000 7f6d85703740 1 Processor -- start 2024-09-16T01:57:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.813+0000 7f6d85703740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711de6850 0x55e711df9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.813+0000 7f6d85703740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711de7170 0x55e711dfa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.814+0000 7f6d85703740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711df1a30 0x55e711df6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.814+0000 7f6d85703740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60011820 con 0x55e711df1a30 2024-09-16T01:57:59.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.814+0000 7f6d85703740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60011090 con 0x55e711de7170 2024-09-16T01:57:59.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.814+0000 7f6d85703740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6000e230 con 0x55e711de6850 2024-09-16T01:57:59.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.814+0000 7f6d85703740 10 client.0 _ll_register_callbacks cb 0x7f6d6c09c430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:57:59.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.815+0000 7f6d808c5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711de7170 0x55e711dfa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.815+0000 7f6d810c6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711de6850 0x55e711df9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.816+0000 7f6d818c7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711df1a30 0x55e711df6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.816+0000 7f6d818c7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711df1a30 0x55e711df6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56148/0 (socket says 172.21.15.139:56148) 2024-09-16T01:57:59.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.816+0000 7f6d818c7640 1 -- 172.21.15.139:0/454249582 learned_addr learned my addr 172.21.15.139:0/454249582 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:57:59.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.820+0000 7f6d810c6640 1 -- 172.21.15.139:0/454249582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711de7170 msgr2=0x55e711dfa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:59.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.820+0000 7f6d810c6640 1 --2- 172.21.15.139:0/454249582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711de7170 0x55e711dfa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.820+0000 7f6d810c6640 1 -- 172.21.15.139:0/454249582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711df1a30 msgr2=0x55e711df6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:57:59.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.820+0000 7f6d810c6640 1 --2- 172.21.15.139:0/454249582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711df1a30 0x55e711df6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:57:59.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.820+0000 7f6d810c6640 1 -- 172.21.15.139:0/454249582 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e711de6fa0 con 0x55e711de6850 2024-09-16T01:57:59.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.821+0000 7f6d810c6640 1 --2- 172.21.15.139:0/454249582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711de6850 0x55e711df9c10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6d74010010 tx=0x7f6d740048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf89b9f777fccd6a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:57:59.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.821+0000 7f6d64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:57:59.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.824+0000 7f6d64ff9640 1 -- 172.21.15.139:0/454249582 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7402b040 con 0x55e711de6850 2024-09-16T01:57:59.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.824+0000 7f6d64ff9640 1 -- 172.21.15.139:0/454249582 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9742+0+0 (secure 0 0 0) 0x7f6d740109a0 con 0x55e711de6850 2024-09-16T01:57:59.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.825+0000 7f6d85703740 1 -- 172.21.15.139:0/454249582 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d74005510 con 0x55e711de6850 2024-09-16T01:57:59.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.829+0000 7f6d64ff9640 1 -- 172.21.15.139:0/454249582 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f6d74010ba0 con 0x55e711de6850 2024-09-16T01:57:59.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.830+0000 7f6d85703740 10 client.26662 fetch_fsmap learned FSMap version 76 2024-09-16T01:57:59.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.830+0000 7f6d85703740 1 -- 172.21.15.139:0/454249582 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55e711e6ce60 con 0x55e711de6850 2024-09-16T01:57:59.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.830+0000 7f6d64ff9640 1 -- 172.21.15.139:0/454249582 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d74010d40 con 0x55e711de6850 2024-09-16T01:57:59.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.831+0000 7f6d64ff9640 1 -- 172.21.15.139:0/454249582 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 76) ==== 8446+0+0 (secure 0 0 0) 0x7f6d7402e030 con 0x55e711de6850 2024-09-16T01:57:59.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.831+0000 7f6d85703740 10 client.26662 fetch_fsmap finished waiting for FSMap version 76 2024-09-16T01:57:59.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.831+0000 7f6d85703740 10 client.26662 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:57:59.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.831+0000 7f6d85703740 20 client.26662 populate_metadata read hostname 'smithi139' 2024-09-16T01:57:59.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.831+0000 7f6d85703740 1 --2- 172.21.15.139:0/454249582 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e711e6d500 0x55e711e6f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:57:59.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.831+0000 7f6d85703740 4 client.26662 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.95, "await_for": 45.0}] 2024-09-16T01:57:59.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.831+0000 7f6d85703740 1 -- 172.21.15.139:0/454249582 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.95, "await_for": 45.0}) -- 0x7f6d74005510 con 0x55e711e6d500 2024-09-16T01:57:59.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.832+0000 7f6d808c5640 1 --2- 172.21.15.139:0/454249582 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e711e6d500 0x55e711e6f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:57:59.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.838+0000 7f6d808c5640 1 --2- 172.21.15.139:0/454249582 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e711e6d500 0x55e711e6f920 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27d5b304de61b8f5 server_cookie=8faa6500b8f2d28d in_seq=0 out_seq=0 2024-09-16T01:57:59.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:57:59.838+0000 7f6d64ff9640 10 client.26662 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:00.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:00 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:58:00.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:00 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:58:00.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:00 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:58:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.868+0000 7f6d64ff9640 1 -- 172.21.15.139:0/454249582 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f6d74005510 con 0x55e711e6d500 2024-09-16T01:58:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.868+0000 7f6d64ff9640 10 client.26662 handle_command_reply: tid=0 2024-09-16T01:58:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.868+0000 7f6d85703740 1 client.26662 shutdown 2024-09-16T01:58:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.869+0000 7f6d85703740 1 -- 172.21.15.139:0/454249582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711de6850 msgr2=0x55e711df9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:00.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.869+0000 7f6d85703740 1 --2- 172.21.15.139:0/454249582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711de6850 0x55e711df9c10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6d74010010 tx=0x7f6d740048f0 comp rx=0 tx=0).stop 2024-09-16T01:58:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.869+0000 7f6d85703740 1 -- 172.21.15.139:0/454249582 shutdown_connections 2024-09-16T01:58:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.869+0000 7f6d85703740 1 --2- 172.21.15.139:0/454249582 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e711e6d500 0x55e711e6f920 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.869+0000 7f6d64ff9640 0 client.26662 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.869+0000 7f6d85703740 1 --2- 172.21.15.139:0/454249582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e711df1a30 0x55e711df6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.869+0000 7f6d85703740 1 --2- 172.21.15.139:0/454249582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e711de7170 0x55e711dfa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.869+0000 7f6d85703740 1 --2- 172.21.15.139:0/454249582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e711de6850 0x55e711df9c10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.869+0000 7f6d85703740 1 -- 172.21.15.139:0/454249582 >> 172.21.15.139:0/454249582 conn(0x55e711de2de0 msgr2=0x55e711de1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.869+0000 7f6d85703740 1 -- 172.21.15.139:0/454249582 shutdown_connections 2024-09-16T01:58:00.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.870+0000 7f6d85703740 1 -- 172.21.15.139:0/454249582 wait complete. 2024-09-16T01:58:00.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.870+0000 7f6d85703740 20 client.26662 trim_cache size 0 max 16384 2024-09-16T01:58:00.873 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:58:00.873 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 76, 2024-09-16T01:58:00.873 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:58:00.873 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 62, 2024-09-16T01:58:00.873 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:58:00.873 INFO:teuthology.orchestra.run.smithi139.stdout: "7235a3e6": { 2024-09-16T01:58:00.873 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 62, 2024-09-16T01:58:00.873 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:58:00.874 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:58:00.874 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:58:00.874 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:58:00.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:58:00.874 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:58:00.874 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 142.9, 2024-09-16T01:58:00.874 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:58:00.874 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:58:00.874 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:58:00.874 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:58:00.874 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:58:00.874 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:58:00.874 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:00.874 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:00.875 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:00.875 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:00.875 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:00.875 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:58:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.871+0000 7f6d8332a640 1 -- 172.21.15.139:0/1193494900 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6d680a35c0 msgr2=0x7f6d680a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.871+0000 7f6d8332a640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6d680a35c0 0x7f6d680a5a80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400abd0 tx=0x7f6d7400a380 comp rx=0 tx=0).stop 2024-09-16T01:58:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.871+0000 7f6d8332a640 1 -- 172.21.15.139:0/1193494900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c06a370 msgr2=0x7f6d7c1a7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.871+0000 7f6d8332a640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c06a370 0x7f6d7c1a7760 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00b450 tx=0x7f6d6c00b910 comp rx=0 tx=0).stop 2024-09-16T01:58:00.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.871+0000 7f6d8332a640 1 -- 172.21.15.139:0/1193494900 shutdown_connections 2024-09-16T01:58:00.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.871+0000 7f6d8332a640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6d680a35c0 0x7f6d680a5a80 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400abd0 tx=0x7f6d7400a380 comp rx=0 tx=0).stop 2024-09-16T01:58:00.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.871+0000 7f6d8332a640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c104e80 0x7f6d7c1abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:00.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.871+0000 7f6d8332a640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c06a370 0x7f6d7c1a7760 secure :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00b450 tx=0x7f6d6c00b910 comp rx=0 tx=0).stop 2024-09-16T01:58:00.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.871+0000 7f6d8332a640 1 --2- 172.21.15.139:0/1193494900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d7c069f00 0x7f6d7c1a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:00.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.871+0000 7f6d8332a640 1 -- 172.21.15.139:0/1193494900 >> 172.21.15.139:0/1193494900 conn(0x7f6d7c077210 msgr2=0x7f6d7c10f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:00.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.871+0000 7f6d8332a640 1 -- 172.21.15.139:0/1193494900 shutdown_connections 2024-09-16T01:58:00.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:00.871+0000 7f6d8332a640 1 -- 172.21.15.139:0/1193494900 wait complete. 2024-09-16T01:58:01.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:01 smithi139 ceph-mon[31175]: pgmap v559: 129 pgs: 21 active+clean+snaptrim, 63 active+clean+snaptrim_wait, 45 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-16T01:58:01.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:01 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:01 smithi164 ceph-mon[35121]: pgmap v559: 129 pgs: 21 active+clean+snaptrim, 63 active+clean+snaptrim_wait, 45 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-16T01:58:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:01 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:01.661 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7235a3e6', quiesce took 7.8 seconds. Will release after: 15.149999999999999 2024-09-16T01:58:01.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:01 smithi176 ceph-mon[35324]: pgmap v559: 129 pgs: 21 active+clean+snaptrim, 63 active+clean+snaptrim_wait, 45 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-16T01:58:01.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:01 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:58:02.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:02 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:02.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:02 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:02.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:02 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:58:02.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:02 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:02.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:02 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:02.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:02 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:58:02.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:02 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:58:02.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:02 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:58:02.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:02 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:58:04.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:03 smithi139 ceph-mon[31175]: pgmap v560: 129 pgs: 20 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 49 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.0 MiB/s wr, 150 op/s 2024-09-16T01:58:04.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:03 smithi176 ceph-mon[35324]: pgmap v560: 129 pgs: 20 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 49 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.0 MiB/s wr, 150 op/s 2024-09-16T01:58:04.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:03 smithi164 ceph-mon[35121]: pgmap v560: 129 pgs: 20 active+clean+snaptrim, 60 active+clean+snaptrim_wait, 49 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.0 MiB/s wr, 150 op/s 2024-09-16T01:58:06.173 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:05 smithi139 ceph-mon[31175]: pgmap v561: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-09-16T01:58:06.186 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:05 smithi176 ceph-mon[35324]: pgmap v561: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-09-16T01:58:06.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:05 smithi164 ceph-mon[35121]: pgmap v561: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-09-16T01:58:07.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:07 smithi139 ceph-mon[31175]: pgmap v562: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 925 KiB/s wr, 140 op/s 2024-09-16T01:58:07.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:07 smithi139 ceph-mon[31175]: osdmap e115: 12 total, 12 up, 12 in 2024-09-16T01:58:07.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:07 smithi176 ceph-mon[35324]: pgmap v562: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 925 KiB/s wr, 140 op/s 2024-09-16T01:58:07.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:07 smithi176 ceph-mon[35324]: osdmap e115: 12 total, 12 up, 12 in 2024-09-16T01:58:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:07 smithi164 ceph-mon[35121]: pgmap v562: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 925 KiB/s wr, 140 op/s 2024-09-16T01:58:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:07 smithi164 ceph-mon[35121]: osdmap e115: 12 total, 12 up, 12 in 2024-09-16T01:58:08.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:08 smithi139 ceph-mon[31175]: osdmap e116: 12 total, 12 up, 12 in 2024-09-16T01:58:08.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:08 smithi176 ceph-mon[35324]: osdmap e116: 12 total, 12 up, 12 in 2024-09-16T01:58:08.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:08 smithi164 ceph-mon[35121]: osdmap e116: 12 total, 12 up, 12 in 2024-09-16T01:58:09.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:09 smithi139 ceph-mon[31175]: pgmap v565: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 359 KiB/s wr, 156 op/s 2024-09-16T01:58:09.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:09 smithi176 ceph-mon[35324]: pgmap v565: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 359 KiB/s wr, 156 op/s 2024-09-16T01:58:09.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:09 smithi164 ceph-mon[35121]: pgmap v565: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 359 KiB/s wr, 156 op/s 2024-09-16T01:58:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.034+0000 7f6d83453640 1 -- 172.21.15.139:0/4074173620 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d7c101170 msgr2=0x7f6d7c1015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.034+0000 7f6d83453640 1 --2- 172.21.15.139:0/4074173620 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d7c101170 0x7f6d7c1015d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009900 tx=0x7f6d6c02a420 comp rx=0 tx=0).stop 2024-09-16T01:58:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.034+0000 7f6d83453640 1 -- 172.21.15.139:0/4074173620 shutdown_connections 2024-09-16T01:58:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.034+0000 7f6d83453640 1 --2- 172.21.15.139:0/4074173620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c101b10 0x7f6d7c10d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.034+0000 7f6d83453640 1 --2- 172.21.15.139:0/4074173620 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d7c101170 0x7f6d7c1015d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.034+0000 7f6d83453640 1 --2- 172.21.15.139:0/4074173620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c1029a0 0x7f6d7c102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.034+0000 7f6d83453640 1 -- 172.21.15.139:0/4074173620 >> 172.21.15.139:0/4074173620 conn(0x7f6d7c0770f0 msgr2=0x7f6d7c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.035+0000 7f6d83453640 1 -- 172.21.15.139:0/4074173620 shutdown_connections 2024-09-16T01:58:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.035+0000 7f6d83453640 1 -- 172.21.15.139:0/4074173620 wait complete. 2024-09-16T01:58:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.036+0000 7f6d83453640 1 Processor -- start 2024-09-16T01:58:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.036+0000 7f6d83453640 1 -- start start 2024-09-16T01:58:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.036+0000 7f6d83453640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d7c101170 0x7f6d7c073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.036+0000 7f6d83453640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c101b10 0x7f6d7c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.036+0000 7f6d83453640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c1029a0 0x7f6d7c06c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.036+0000 7f6d83453640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c110f20 con 0x7f6d7c101b10 2024-09-16T01:58:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.036+0000 7f6d83453640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c110da0 con 0x7f6d7c1029a0 2024-09-16T01:58:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.036+0000 7f6d83453640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c1110a0 con 0x7f6d7c101170 2024-09-16T01:58:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.038+0000 7f6d811ef640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d7c101170 0x7f6d7c073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.037+0000 7f6d809ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c101b10 0x7f6d7c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.039+0000 7f6d809ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c101b10 0x7f6d7c06bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34660/0 (socket says 172.21.15.139:34660) 2024-09-16T01:58:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.039+0000 7f6d809ee640 1 -- 172.21.15.139:0/3187993242 learned_addr learned my addr 172.21.15.139:0/3187993242 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.038+0000 7f6d819f0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c1029a0 0x7f6d7c06c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.042+0000 7f6d819f0640 1 -- 172.21.15.139:0/3187993242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d7c101170 msgr2=0x7f6d7c073d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.042+0000 7f6d819f0640 1 --2- 172.21.15.139:0/3187993242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d7c101170 0x7f6d7c073d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.042+0000 7f6d819f0640 1 -- 172.21.15.139:0/3187993242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c101b10 msgr2=0x7f6d7c06bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.042+0000 7f6d819f0640 1 --2- 172.21.15.139:0/3187993242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c101b10 0x7f6d7c06bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.042+0000 7f6d819f0640 1 -- 172.21.15.139:0/3187993242 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7c06ca40 con 0x7f6d7c1029a0 2024-09-16T01:58:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.042+0000 7f6d819f0640 1 --2- 172.21.15.139:0/3187993242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c1029a0 0x7f6d7c06c220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009fd0 tx=0x7f6d7400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.045+0000 7f6d727fc640 1 -- 172.21.15.139:0/3187993242 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7400ce20 con 0x7f6d7c1029a0 2024-09-16T01:58:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.045+0000 7f6d727fc640 1 -- 172.21.15.139:0/3187993242 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d74005020 con 0x7f6d7c1029a0 2024-09-16T01:58:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.045+0000 7f6d727fc640 1 -- 172.21.15.139:0/3187993242 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d74010720 con 0x7f6d7c1029a0 2024-09-16T01:58:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.046+0000 7f6d83453640 1 -- 172.21.15.139:0/3187993242 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7c0fc4d0 con 0x7f6d7c1029a0 2024-09-16T01:58:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.046+0000 7f6d83453640 1 -- 172.21.15.139:0/3187993242 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d7c0fc970 con 0x7f6d7c1029a0 2024-09-16T01:58:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.053+0000 7f6d727fc640 1 -- 172.21.15.139:0/3187993242 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6d74010900 con 0x7f6d7c1029a0 2024-09-16T01:58:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.054+0000 7f6d861ef740 1 Processor -- start 2024-09-16T01:58:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.055+0000 7f6d727fc640 1 --2- 172.21.15.139:0/3187993242 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6d680a3640 0x7f6d680a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.056+0000 7f6d861ef740 1 -- start start 2024-09-16T01:58:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.056+0000 7f6d727fc640 1 -- 172.21.15.139:0/3187993242 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7f6d740d0870 con 0x7f6d7c1029a0 2024-09-16T01:58:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.056+0000 7f6d809ee640 1 --2- 172.21.15.139:0/3187993242 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6d680a3640 0x7f6d680a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.056+0000 7f6d861ef740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a1af0 0x55c3e99a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.056+0000 7f6d861ef740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99a2410 0x55c3e99ac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.056+0000 7f6d861ef740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99accb0 0x55c3e99af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.056+0000 7f6d861ef740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c3e99062f0 con 0x55c3e99accb0 2024-09-16T01:58:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.056+0000 7f6d861ef740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c3e98ae270 con 0x55c3e99a1af0 2024-09-16T01:58:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.056+0000 7f6d861ef740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c3e99022b0 con 0x55c3e99a2410 2024-09-16T01:58:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.058+0000 7f6d809ee640 1 --2- 172.21.15.139:0/3187993242 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6d680a3640 0x7f6d680a5b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c005940 tx=0x7f6d6c032df0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.058+0000 7f6d809ee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99a2410 0x55c3e99ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.058+0000 7f6d809ee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99a2410 0x55c3e99ac770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:34816/0 (socket says 172.21.15.139:34816) 2024-09-16T01:58:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.058+0000 7f6d809ee640 1 -- 172.21.15.139:0/2081410821 learned_addr learned my addr 172.21.15.139:0/2081410821 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.062+0000 7f6d811ef640 1 --2- 172.21.15.139:0/2081410821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a1af0 0x55c3e99a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.062+0000 7f6d809ee640 1 -- 172.21.15.139:0/2081410821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a1af0 msgr2=0x55c3e99a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.062+0000 7f6d809ee640 1 --2- 172.21.15.139:0/2081410821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a1af0 0x55c3e99a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.062+0000 7f6d809ee640 1 -- 172.21.15.139:0/2081410821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99accb0 msgr2=0x55c3e99af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.062+0000 7f6d809ee640 1 --2- 172.21.15.139:0/2081410821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99accb0 0x55c3e99af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.062+0000 7f6d809ee640 1 -- 172.21.15.139:0/2081410821 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3e98ad340 con 0x55c3e99a2410 2024-09-16T01:58:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.062+0000 7f6d809ee640 1 --2- 172.21.15.139:0/2081410821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99a2410 0x55c3e99ac770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c000c00 tx=0x7f6d6c041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a97815716fd41da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.063+0000 7f6d677fe640 1 -- 172.21.15.139:0/2081410821 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c035050 con 0x55c3e99a2410 2024-09-16T01:58:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.064+0000 7f6d677fe640 1 -- 172.21.15.139:0/2081410821 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c004480 con 0x55c3e99a2410 2024-09-16T01:58:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.064+0000 7f6d677fe640 1 -- 172.21.15.139:0/2081410821 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0035f0 con 0x55c3e99a2410 2024-09-16T01:58:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.065+0000 7f6d861ef740 1 -- 172.21.15.139:0/2081410821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99a2410 msgr2=0x55c3e99ac770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.065+0000 7f6d861ef740 1 --2- 172.21.15.139:0/2081410821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99a2410 0x55c3e99ac770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c000c00 tx=0x7f6d6c041000 comp rx=0 tx=0).stop 2024-09-16T01:58:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.066+0000 7f6d811ef640 1 --2- 172.21.15.139:0/2081410821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a1af0 0x55c3e99a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:58:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.069+0000 7f6d861ef740 1 -- 172.21.15.139:0/2081410821 shutdown_connections 2024-09-16T01:58:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.069+0000 7f6d861ef740 1 --2- 172.21.15.139:0/2081410821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99accb0 0x55c3e99af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.069+0000 7f6d861ef740 1 --2- 172.21.15.139:0/2081410821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99a2410 0x55c3e99ac770 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.069+0000 7f6d861ef740 1 --2- 172.21.15.139:0/2081410821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a1af0 0x55c3e99a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.069+0000 7f6d861ef740 1 -- 172.21.15.139:0/2081410821 >> 172.21.15.139:0/2081410821 conn(0x55c3e999e060 msgr2=0x55c3e999e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.077+0000 7f6d861ef740 1 -- 172.21.15.139:0/2081410821 shutdown_connections 2024-09-16T01:58:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.080+0000 7f6d861ef740 1 -- 172.21.15.139:0/2081410821 wait complete. 2024-09-16T01:58:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.081+0000 7f6d861ef740 1 -- start start 2024-09-16T01:58:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.081+0000 7f6d861ef740 1 Processor -- start 2024-09-16T01:58:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.082+0000 7f6d861ef740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a1af0 0x55c3e99a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.082+0000 7f6d861ef740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99a2410 0x55c3e99aa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.082+0000 7f6d861ef740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99accb0 0x55c3e99aa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.082+0000 7f6d861ef740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c3e99af880 con 0x55c3e99a2410 2024-09-16T01:58:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.082+0000 7f6d861ef740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c3e988f290 con 0x55c3e99a1af0 2024-09-16T01:58:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.082+0000 7f6d861ef740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c3e99afa00 con 0x55c3e99accb0 2024-09-16T01:58:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.082+0000 7f6d861ef740 10 client.0 _ll_register_callbacks cb 0x55c3e98c0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.083+0000 7f6d819f0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99accb0 0x55c3e99aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.083+0000 7f6d819f0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99accb0 0x55c3e99aa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:34828/0 (socket says 172.21.15.139:34828) 2024-09-16T01:58:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.083+0000 7f6d819f0640 1 -- 172.21.15.139:0/3244252921 learned_addr learned my addr 172.21.15.139:0/3244252921 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.083+0000 7f6d809ee640 1 --2- 172.21.15.139:0/3244252921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99a2410 0x55c3e99aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.084+0000 7f6d809ee640 1 -- 172.21.15.139:0/3244252921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99accb0 msgr2=0x55c3e99aa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.084+0000 7f6d809ee640 1 --2- 172.21.15.139:0/3244252921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99accb0 0x55c3e99aa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.084+0000 7f6d809ee640 1 -- 172.21.15.139:0/3244252921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a1af0 msgr2=0x55c3e99a9c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:58:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.084+0000 7f6d809ee640 1 --2- 172.21.15.139:0/3244252921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a1af0 0x55c3e99a9c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.084+0000 7f6d809ee640 1 -- 172.21.15.139:0/3244252921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3e9619210 con 0x55c3e99a2410 2024-09-16T01:58:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.084+0000 7f6d809ee640 1 --2- 172.21.15.139:0/3244252921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99a2410 0x55c3e99aa140 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c031550 tx=0x7f6d6c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fbaffbefcf2c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.084+0000 7f6d64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:58:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.085+0000 7f6d64ff9640 1 -- 172.21.15.139:0/3244252921 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c005610 con 0x55c3e99a2410 2024-09-16T01:58:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.085+0000 7f6d861ef740 1 -- 172.21.15.139:0/3244252921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6c004480 con 0x55c3e99a2410 2024-09-16T01:58:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.085+0000 7f6d64ff9640 1 -- 172.21.15.139:0/3244252921 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7f6d6c0046c0 con 0x55c3e99a2410 2024-09-16T01:58:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.086+0000 7f6d64ff9640 1 -- 172.21.15.139:0/3244252921 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c0035f0 con 0x55c3e99a2410 2024-09-16T01:58:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.086+0000 7f6d861ef740 10 client.17286 fetch_fsmap learned FSMap version 76 2024-09-16T01:58:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.086+0000 7f6d861ef740 1 -- 172.21.15.139:0/3244252921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55c3e96e5620 con 0x55c3e99a2410 2024-09-16T01:58:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.086+0000 7f6d64ff9640 1 -- 172.21.15.139:0/3244252921 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c004480 con 0x55c3e99a2410 2024-09-16T01:58:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.087+0000 7f6d64ff9640 1 -- 172.21.15.139:0/3244252921 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 76) ==== 8446+0+0 (secure 0 0 0) 0x7f6d6c0488b0 con 0x55c3e99a2410 2024-09-16T01:58:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.087+0000 7f6d861ef740 10 client.17286 fetch_fsmap finished waiting for FSMap version 76 2024-09-16T01:58:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.087+0000 7f6d861ef740 10 client.17286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:58:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.087+0000 7f6d861ef740 20 client.17286 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.087+0000 7f6d861ef740 1 --2- 172.21.15.139:0/3244252921 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c3e99b3df0 0x55c3e99b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.088+0000 7f6d861ef740 4 client.17286 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:58:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.088+0000 7f6d861ef740 1 -- 172.21.15.139:0/3244252921 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3e9913a20 con 0x55c3e99b3df0 2024-09-16T01:58:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.091+0000 7f6d811ef640 1 --2- 172.21.15.139:0/3244252921 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c3e99b3df0 0x55c3e99b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.098+0000 7f6d811ef640 1 --2- 172.21.15.139:0/3244252921 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c3e99b3df0 0x55c3e99b6210 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4c33ba0c817d38 server_cookie=a5b843bfb1f14b08 in_seq=0 out_seq=0 2024-09-16T01:58:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.099+0000 7f6d64ff9640 10 client.17286 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.100+0000 7f6d64ff9640 1 -- 172.21.15.139:0/3244252921 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3e9913a20 con 0x55c3e99b3df0 2024-09-16T01:58:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.100+0000 7f6d64ff9640 10 client.17286 handle_command_reply: tid=0 2024-09-16T01:58:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.100+0000 7f6d861ef740 1 client.17286 shutdown 2024-09-16T01:58:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.104+0000 7f6d861ef740 1 -- 172.21.15.139:0/3244252921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99a2410 msgr2=0x55c3e99aa140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.104+0000 7f6d861ef740 1 --2- 172.21.15.139:0/3244252921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99a2410 0x55c3e99aa140 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c031550 tx=0x7f6d6c0058b0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.106+0000 7f6d861ef740 1 -- 172.21.15.139:0/3244252921 shutdown_connections 2024-09-16T01:58:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.106+0000 7f6d861ef740 1 --2- 172.21.15.139:0/3244252921 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c3e99b3df0 0x55c3e99b6210 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.106+0000 7f6d861ef740 1 --2- 172.21.15.139:0/3244252921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99accb0 0x55c3e99aa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.106+0000 7f6d861ef740 1 --2- 172.21.15.139:0/3244252921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99a2410 0x55c3e99aa140 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.106+0000 7f6d861ef740 1 --2- 172.21.15.139:0/3244252921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a1af0 0x55c3e99a9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.106+0000 7f6d861ef740 1 -- 172.21.15.139:0/3244252921 >> 172.21.15.139:0/3244252921 conn(0x55c3e999e060 msgr2=0x55c3e99a3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.108+0000 7f6d64ff9640 0 client.17286 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.109+0000 7f6d861ef740 1 -- 172.21.15.139:0/3244252921 shutdown_connections 2024-09-16T01:58:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.112+0000 7f6d861ef740 1 -- 172.21.15.139:0/3244252921 wait complete. 2024-09-16T01:58:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.114+0000 7f6d861ef740 20 client.17286 trim_cache size 0 max 16384 2024-09-16T01:58:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.146+0000 7f6d861ef740 1 Processor -- start 2024-09-16T01:58:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.147+0000 7f6d861ef740 1 -- start start 2024-09-16T01:58:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.147+0000 7f6d861ef740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99a1af0 0x55c3e99a4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.147+0000 7f6d861ef740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a2410 0x55c3e99a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.147+0000 7f6d861ef740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99accb0 0x55c3e99bbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.147+0000 7f6d861ef740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60010240 con 0x55c3e99accb0 2024-09-16T01:58:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.148+0000 7f6d861ef740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60012050 con 0x55c3e99a2410 2024-09-16T01:58:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.148+0000 7f6d861ef740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d600109a0 con 0x55c3e99a1af0 2024-09-16T01:58:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.149+0000 7f6d819f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99accb0 0x55c3e99bbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.149+0000 7f6d811ef640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99a1af0 0x55c3e99a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.149+0000 7f6d819f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99accb0 0x55c3e99bbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34698/0 (socket says 172.21.15.139:34698) 2024-09-16T01:58:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.149+0000 7f6d819f0640 1 -- 172.21.15.139:0/2262760146 learned_addr learned my addr 172.21.15.139:0/2262760146 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.149+0000 7f6d809ee640 1 --2- 172.21.15.139:0/2262760146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a2410 0x55c3e99a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.152+0000 7f6d819f0640 1 -- 172.21.15.139:0/2262760146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99a1af0 msgr2=0x55c3e99a4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.152+0000 7f6d819f0640 1 --2- 172.21.15.139:0/2262760146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99a1af0 0x55c3e99a4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.152+0000 7f6d819f0640 1 -- 172.21.15.139:0/2262760146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a2410 msgr2=0x55c3e99a8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.152+0000 7f6d819f0640 1 --2- 172.21.15.139:0/2262760146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a2410 0x55c3e99a8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.152+0000 7f6d819f0640 1 -- 172.21.15.139:0/2262760146 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3e99a1870 con 0x55c3e99accb0 2024-09-16T01:58:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.153+0000 7f6d809ee640 1 --2- 172.21.15.139:0/2262760146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a2410 0x55c3e99a8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:58:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.155+0000 7f6d819f0640 1 --2- 172.21.15.139:0/2262760146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99accb0 0x55c3e99bbaa0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6d74098f90 tx=0x7f6d7409cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec36314802fdb49 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.160+0000 7f6d66ffd640 1 -- 172.21.15.139:0/2262760146 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d740ba070 con 0x55c3e99accb0 2024-09-16T01:58:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.160+0000 7f6d66ffd640 1 -- 172.21.15.139:0/2262760146 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d740cf040 con 0x55c3e99accb0 2024-09-16T01:58:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.160+0000 7f6d66ffd640 1 -- 172.21.15.139:0/2262760146 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d740b4030 con 0x55c3e99accb0 2024-09-16T01:58:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.161+0000 7f6d861ef740 1 -- 172.21.15.139:0/2262760146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99accb0 msgr2=0x55c3e99bbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.161+0000 7f6d861ef740 1 --2- 172.21.15.139:0/2262760146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99accb0 0x55c3e99bbaa0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6d74098f90 tx=0x7f6d7409cdc0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.164+0000 7f6d861ef740 1 -- 172.21.15.139:0/2262760146 shutdown_connections 2024-09-16T01:58:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.164+0000 7f6d861ef740 1 --2- 172.21.15.139:0/2262760146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99accb0 0x55c3e99bbaa0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.164+0000 7f6d861ef740 1 --2- 172.21.15.139:0/2262760146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a2410 0x55c3e99a8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.164+0000 7f6d861ef740 1 --2- 172.21.15.139:0/2262760146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99a1af0 0x55c3e99a4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.164+0000 7f6d861ef740 1 -- 172.21.15.139:0/2262760146 >> 172.21.15.139:0/2262760146 conn(0x55c3e999e060 msgr2=0x55c3e99a84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.166+0000 7f6d861ef740 1 -- 172.21.15.139:0/2262760146 shutdown_connections 2024-09-16T01:58:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.171+0000 7f6d861ef740 1 -- 172.21.15.139:0/2262760146 wait complete. 2024-09-16T01:58:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.171+0000 7f6d861ef740 1 -- start start 2024-09-16T01:58:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.171+0000 7f6d861ef740 1 Processor -- start 2024-09-16T01:58:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.176+0000 7f6d861ef740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99a1af0 0x55c3e99bbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.176+0000 7f6d861ef740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a2410 0x55c3e99bc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.176+0000 7f6d861ef740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99accb0 0x55c3e99b9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.176+0000 7f6d861ef740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6d600118c0 con 0x55c3e99a1af0 2024-09-16T01:58:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.176+0000 7f6d861ef740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60011130 con 0x55c3e99a2410 2024-09-16T01:58:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.176+0000 7f6d861ef740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6000e2d0 con 0x55c3e99accb0 2024-09-16T01:58:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.176+0000 7f6d861ef740 10 client.0 _ll_register_callbacks cb 0x7f6d740093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.180+0000 7f6d811ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99a1af0 0x55c3e99bbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.180+0000 7f6d809ee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a2410 0x55c3e99bc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.180+0000 7f6d811ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99a1af0 0x55c3e99bbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34702/0 (socket says 172.21.15.139:34702) 2024-09-16T01:58:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.180+0000 7f6d811ef640 1 -- 172.21.15.139:0/1315027088 learned_addr learned my addr 172.21.15.139:0/1315027088 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.182+0000 7f6d819f0640 1 --2- 172.21.15.139:0/1315027088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99accb0 0x55c3e99b9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.183+0000 7f6d809ee640 1 -- 172.21.15.139:0/1315027088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99accb0 msgr2=0x55c3e99b9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.183+0000 7f6d809ee640 1 --2- 172.21.15.139:0/1315027088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99accb0 0x55c3e99b9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.183+0000 7f6d809ee640 1 -- 172.21.15.139:0/1315027088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99a1af0 msgr2=0x55c3e99bbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.183+0000 7f6d809ee640 1 --2- 172.21.15.139:0/1315027088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99a1af0 0x55c3e99bbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.183+0000 7f6d809ee640 1 -- 172.21.15.139:0/1315027088 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3e99b9830 con 0x55c3e99a2410 2024-09-16T01:58:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.185+0000 7f6d809ee640 1 --2- 172.21.15.139:0/1315027088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a2410 0x55c3e99bc130 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c004ba0 tx=0x7f6d6c032eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b71cbd38a348037 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.186+0000 7f6d64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:58:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.188+0000 7f6d64ff9640 1 -- 172.21.15.139:0/1315027088 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c041040 con 0x55c3e99a2410 2024-09-16T01:58:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.189+0000 7f6d64ff9640 1 -- 172.21.15.139:0/1315027088 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7f6d6c004480 con 0x55c3e99a2410 2024-09-16T01:58:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.189+0000 7f6d861ef740 1 -- 172.21.15.139:0/1315027088 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d740cf040 con 0x55c3e99a2410 2024-09-16T01:58:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.194+0000 7f6d64ff9640 1 -- 172.21.15.139:0/1315027088 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c0046c0 con 0x55c3e99a2410 2024-09-16T01:58:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.196+0000 7f6d861ef740 10 client.26702 fetch_fsmap learned FSMap version 76 2024-09-16T01:58:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.196+0000 7f6d861ef740 1 -- 172.21.15.139:0/1315027088 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55c3e9a270f0 con 0x55c3e99a2410 2024-09-16T01:58:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.201+0000 7f6d64ff9640 1 -- 172.21.15.139:0/1315027088 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d740cf040 con 0x55c3e99a2410 2024-09-16T01:58:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.202+0000 7f6d64ff9640 1 -- 172.21.15.139:0/1315027088 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 76) ==== 8446+0+0 (secure 0 0 0) 0x7f6d6c003ad0 con 0x55c3e99a2410 2024-09-16T01:58:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.203+0000 7f6d861ef740 10 client.26702 fetch_fsmap finished waiting for FSMap version 76 2024-09-16T01:58:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.203+0000 7f6d861ef740 10 client.26702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:58:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.203+0000 7f6d861ef740 20 client.26702 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.203+0000 7f6d861ef740 1 --2- 172.21.15.139:0/1315027088 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c3e9a27900 0x55c3e9a29d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.203+0000 7f6d861ef740 4 client.26702 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:58:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.203+0000 7f6d861ef740 1 -- 172.21.15.139:0/1315027088 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3e990dd70 con 0x55c3e9a27900 2024-09-16T01:58:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.208+0000 7f6d811ef640 1 --2- 172.21.15.139:0/1315027088 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c3e9a27900 0x55c3e9a29d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.215+0000 7f6d811ef640 1 --2- 172.21.15.139:0/1315027088 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c3e9a27900 0x55c3e9a29d20 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b98e2ca763f56a71 server_cookie=5eb8ebe4822b854f in_seq=0 out_seq=0 2024-09-16T01:58:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.216+0000 7f6d64ff9640 10 client.26702 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.404+0000 7f6d64ff9640 1 -- 172.21.15.139:0/1315027088 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c3e990dd70 con 0x55c3e9a27900 2024-09-16T01:58:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.404+0000 7f6d64ff9640 10 client.26702 handle_command_reply: tid=0 2024-09-16T01:58:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.404+0000 7f6d861ef740 1 client.26702 shutdown 2024-09-16T01:58:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.404+0000 7f6d861ef740 1 -- 172.21.15.139:0/1315027088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a2410 msgr2=0x55c3e99bc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.404+0000 7f6d861ef740 1 --2- 172.21.15.139:0/1315027088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a2410 0x55c3e99bc130 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c004ba0 tx=0x7f6d6c032eb0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.405+0000 7f6d861ef740 1 -- 172.21.15.139:0/1315027088 shutdown_connections 2024-09-16T01:58:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.405+0000 7f6d861ef740 1 --2- 172.21.15.139:0/1315027088 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55c3e9a27900 0x55c3e9a29d20 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.405+0000 7f6d861ef740 1 --2- 172.21.15.139:0/1315027088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3e99accb0 0x55c3e99b9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.405+0000 7f6d861ef740 1 --2- 172.21.15.139:0/1315027088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3e99a2410 0x55c3e99bc130 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.405+0000 7f6d861ef740 1 --2- 172.21.15.139:0/1315027088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c3e99a1af0 0x55c3e99bbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.405+0000 7f6d861ef740 1 -- 172.21.15.139:0/1315027088 >> 172.21.15.139:0/1315027088 conn(0x55c3e999e060 msgr2=0x55c3e99a8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.405+0000 7f6d64ff9640 0 client.26702 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.405+0000 7f6d861ef740 1 -- 172.21.15.139:0/1315027088 shutdown_connections 2024-09-16T01:58:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.405+0000 7f6d861ef740 1 -- 172.21.15.139:0/1315027088 wait complete. 2024-09-16T01:58:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.405+0000 7f6d861ef740 20 client.26702 trim_cache size 0 max 16384 2024-09-16T01:58:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:58:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:58:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:58:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:58:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.406+0000 7f6d83453640 1 -- 172.21.15.139:0/3187993242 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6d680a3640 msgr2=0x7f6d680a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.406+0000 7f6d83453640 1 --2- 172.21.15.139:0/3187993242 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6d680a3640 0x7f6d680a5b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c005940 tx=0x7f6d6c032df0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.406+0000 7f6d83453640 1 -- 172.21.15.139:0/3187993242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c1029a0 msgr2=0x7f6d7c06c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.406+0000 7f6d83453640 1 --2- 172.21.15.139:0/3187993242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c1029a0 0x7f6d7c06c220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009fd0 tx=0x7f6d7400ef10 comp rx=0 tx=0).stop 2024-09-16T01:58:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.406+0000 7f6d83453640 1 -- 172.21.15.139:0/3187993242 shutdown_connections 2024-09-16T01:58:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.406+0000 7f6d83453640 1 --2- 172.21.15.139:0/3187993242 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f6d680a3640 0x7f6d680a5b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.406+0000 7f6d83453640 1 --2- 172.21.15.139:0/3187993242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d7c1029a0 0x7f6d7c06c220 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.406+0000 7f6d83453640 1 --2- 172.21.15.139:0/3187993242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d7c101b10 0x7f6d7c06bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.406+0000 7f6d83453640 1 --2- 172.21.15.139:0/3187993242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d7c101170 0x7f6d7c073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.406+0000 7f6d83453640 1 -- 172.21.15.139:0/3187993242 >> 172.21.15.139:0/3187993242 conn(0x7f6d7c0770f0 msgr2=0x7f6d7c100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.406+0000 7f6d83453640 1 -- 172.21.15.139:0/3187993242 shutdown_connections 2024-09-16T01:58:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:10.406+0000 7f6d83453640 1 -- 172.21.15.139:0/3187993242 wait complete. 2024-09-16T01:58:11.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:11 smithi139 ceph-mon[31175]: pgmap v566: 129 pgs: 23 active+clean+snaptrim, 68 active+clean+snaptrim_wait, 38 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 336 KiB/s wr, 159 op/s 2024-09-16T01:58:11.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:11 smithi176 ceph-mon[35324]: pgmap v566: 129 pgs: 23 active+clean+snaptrim, 68 active+clean+snaptrim_wait, 38 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 336 KiB/s wr, 159 op/s 2024-09-16T01:58:11.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:11 smithi164 ceph-mon[35121]: pgmap v566: 129 pgs: 23 active+clean+snaptrim, 68 active+clean+snaptrim_wait, 38 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 336 KiB/s wr, 159 op/s 2024-09-16T01:58:13.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:13 smithi139 ceph-mon[31175]: pgmap v567: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 146 op/s 2024-09-16T01:58:13.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:13 smithi176 ceph-mon[35324]: pgmap v567: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 146 op/s 2024-09-16T01:58:13.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:13 smithi164 ceph-mon[35121]: pgmap v567: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 146 op/s 2024-09-16T01:58:15.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:15 smithi139 ceph-mon[31175]: pgmap v568: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 181 op/s 2024-09-16T01:58:15.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:15 smithi176 ceph-mon[35324]: pgmap v568: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 181 op/s 2024-09-16T01:58:15.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:15 smithi164 ceph-mon[35121]: pgmap v568: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 181 op/s 2024-09-16T01:58:16.812 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:58:17.333 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:58:17.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:17 smithi139 ceph-mon[31175]: pgmap v569: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 159 op/s 2024-09-16T01:58:17.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:17 smithi176 ceph-mon[35324]: pgmap v569: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 159 op/s 2024-09-16T01:58:17.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:17 smithi164 ceph-mon[35121]: pgmap v569: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 159 op/s 2024-09-16T01:58:17.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.978+0000 7feec204f640 1 -- 172.21.15.139:0/1049330252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feebc11b140 msgr2=0x7feebc11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:17.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.978+0000 7feec204f640 1 --2- 172.21.15.139:0/1049330252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feebc11b140 0x7feebc11d530 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feea4009860 tx=0x7feea402a380 comp rx=0 tx=0).stop 2024-09-16T01:58:17.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.979+0000 7feec204f640 1 -- 172.21.15.139:0/1049330252 shutdown_connections 2024-09-16T01:58:17.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.979+0000 7feec204f640 1 --2- 172.21.15.139:0/1049330252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feebc11b140 0x7feebc11d530 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:17.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.979+0000 7feec204f640 1 --2- 172.21.15.139:0/1049330252 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feebc109830 0x7feebc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:17.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.979+0000 7feec204f640 1 --2- 172.21.15.139:0/1049330252 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc109450 0x7feebc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:17.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.979+0000 7feec204f640 1 -- 172.21.15.139:0/1049330252 >> 172.21.15.139:0/1049330252 conn(0x7feebc06e140 msgr2=0x7feebc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:17.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.984+0000 7feec204f640 1 -- 172.21.15.139:0/1049330252 shutdown_connections 2024-09-16T01:58:17.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.985+0000 7feec204f640 1 -- 172.21.15.139:0/1049330252 wait complete. 2024-09-16T01:58:17.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.985+0000 7feec204f640 1 Processor -- start 2024-09-16T01:58:17.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.986+0000 7feec204f640 1 -- start start 2024-09-16T01:58:17.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.986+0000 7feec204f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc109450 0x7feebc1186d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.986+0000 7feec204f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feebc109830 0x7feebc111760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.986+0000 7feec204f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feebc11b140 0x7feebc111ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.986+0000 7feec204f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feebc120da0 con 0x7feebc109450 2024-09-16T01:58:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.986+0000 7feec204f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feebc120c20 con 0x7feebc11b140 2024-09-16T01:58:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.986+0000 7feec204f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feebc120f20 con 0x7feebc109830 2024-09-16T01:58:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.986+0000 7feebb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc109450 0x7feebc1186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.986+0000 7feebb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc109450 0x7feebc1186d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:33008/0 (socket says 172.21.15.139:33008) 2024-09-16T01:58:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.986+0000 7feebb7fe640 1 -- 172.21.15.139:0/350862292 learned_addr learned my addr 172.21.15.139:0/350862292 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.987+0000 7feebbfff640 1 --2- 172.21.15.139:0/350862292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feebc11b140 0x7feebc111ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:17.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.989+0000 7feebb7fe640 1 -- 172.21.15.139:0/350862292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feebc109830 msgr2=0x7feebc111760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:58:17.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.989+0000 7feebb7fe640 1 --2- 172.21.15.139:0/350862292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feebc109830 0x7feebc111760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:17.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.989+0000 7feebb7fe640 1 -- 172.21.15.139:0/350862292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feebc11b140 msgr2=0x7feebc111ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:17.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.989+0000 7feebb7fe640 1 --2- 172.21.15.139:0/350862292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feebc11b140 0x7feebc111ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:17.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.989+0000 7feebb7fe640 1 -- 172.21.15.139:0/350862292 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc112260 con 0x7feebc109450 2024-09-16T01:58:17.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.990+0000 7feebb7fe640 1 --2- 172.21.15.139:0/350862292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc109450 0x7feebc1186d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feeb400e9c0 tx=0x7feeb400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:17.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.994+0000 7feeb8ff9640 1 -- 172.21.15.139:0/350862292 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb400ccc0 con 0x7feebc109450 2024-09-16T01:58:17.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.994+0000 7feeb8ff9640 1 -- 172.21.15.139:0/350862292 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb4005500 con 0x7feebc109450 2024-09-16T01:58:17.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.994+0000 7feeb8ff9640 1 -- 172.21.15.139:0/350862292 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4010430 con 0x7feebc109450 2024-09-16T01:58:17.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.994+0000 7feec204f640 1 -- 172.21.15.139:0/350862292 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc112550 con 0x7feebc109450 2024-09-16T01:58:17.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.994+0000 7feec204f640 1 -- 172.21.15.139:0/350862292 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc116520 con 0x7feebc109450 2024-09-16T01:58:18.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:17.997+0000 7feec204f640 1 -- 172.21.15.139:0/350862292 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feebc073b00 con 0x7feebc109450 2024-09-16T01:58:18.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.001+0000 7feeb8ff9640 1 -- 172.21.15.139:0/350862292 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7feeb40046c0 con 0x7feebc109450 2024-09-16T01:58:18.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.002+0000 7feeb8ff9640 1 --2- 172.21.15.139:0/350862292 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feea80a35c0 0x7feea80a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:18.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.002+0000 7feeb8ff9640 1 -- 172.21.15.139:0/350862292 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7feeb40d0600 con 0x7feebc109450 2024-09-16T01:58:18.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.004+0000 7feebaffd640 1 --2- 172.21.15.139:0/350862292 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feea80a35c0 0x7feea80a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:18.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.014+0000 7feeb8ff9640 1 -- 172.21.15.139:0/350862292 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feeb409ca50 con 0x7feebc109450 2024-09-16T01:58:18.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.019+0000 7feebaffd640 1 --2- 172.21.15.139:0/350862292 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feea80a35c0 0x7feea80a5a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feeac005fd0 tx=0x7feeac005e80 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:18.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.418+0000 7feec204f640 1 -- 172.21.15.139:0/350862292 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feebc10ae40 con 0x7feebc109450 2024-09-16T01:58:18.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.420+0000 7feeb8ff9640 1 -- 172.21.15.139:0/350862292 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+12029 (secure 0 0 0) 0x7feeb409c350 con 0x7feebc109450 2024-09-16T01:58:18.422 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:58:18.423 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":76,"btime":"2024-09-16T01:57:59:198741+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:57:59.054865+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24454,24443,24442,24433]},"id":1}]} 2024-09-16T01:58:18.423 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 76 2024-09-16T01:58:18.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.424+0000 7feec204f640 1 -- 172.21.15.139:0/350862292 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feea80a35c0 msgr2=0x7feea80a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:18.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.424+0000 7feec204f640 1 --2- 172.21.15.139:0/350862292 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feea80a35c0 0x7feea80a5a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feeac005fd0 tx=0x7feeac005e80 comp rx=0 tx=0).stop 2024-09-16T01:58:18.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.424+0000 7feec204f640 1 -- 172.21.15.139:0/350862292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc109450 msgr2=0x7feebc1186d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:18.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.424+0000 7feec204f640 1 --2- 172.21.15.139:0/350862292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc109450 0x7feebc1186d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feeb400e9c0 tx=0x7feeb400ee80 comp rx=0 tx=0).stop 2024-09-16T01:58:18.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.428+0000 7feec204f640 1 -- 172.21.15.139:0/350862292 shutdown_connections 2024-09-16T01:58:18.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.428+0000 7feec204f640 1 --2- 172.21.15.139:0/350862292 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7feea80a35c0 0x7feea80a5a80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:18.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.428+0000 7feec204f640 1 --2- 172.21.15.139:0/350862292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feebc11b140 0x7feebc111ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:18.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.428+0000 7feec204f640 1 --2- 172.21.15.139:0/350862292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feebc109830 0x7feebc111760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:18.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.428+0000 7feec204f640 1 --2- 172.21.15.139:0/350862292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc109450 0x7feebc1186d0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:18.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.428+0000 7feec204f640 1 -- 172.21.15.139:0/350862292 >> 172.21.15.139:0/350862292 conn(0x7feebc06e140 msgr2=0x7feebc11c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:18.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.433+0000 7feec204f640 1 -- 172.21.15.139:0/350862292 shutdown_connections 2024-09-16T01:58:18.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:18.434+0000 7feec204f640 1 -- 172.21.15.139:0/350862292 wait complete. 2024-09-16T01:58:19.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:19 smithi139 ceph-mon[31175]: pgmap v570: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 158 op/s 2024-09-16T01:58:19.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:19 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/350862292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:58:19.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:19 smithi176 ceph-mon[35324]: pgmap v570: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 158 op/s 2024-09-16T01:58:19.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:19 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/350862292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:58:19.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:19 smithi164 ceph-mon[35121]: pgmap v570: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 158 op/s 2024-09-16T01:58:19.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:19 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/350862292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:58:21.271 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:21 smithi139 ceph-mon[31175]: pgmap v571: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 159 op/s 2024-09-16T01:58:21.322 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 7235a3e6 --release --await' 2024-09-16T01:58:21.322 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 7235a3e6 --release --await 2024-09-16T01:58:21.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:21 smithi176 ceph-mon[35324]: pgmap v571: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 159 op/s 2024-09-16T01:58:21.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:21 smithi164 ceph-mon[35121]: pgmap v571: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 159 op/s 2024-09-16T01:58:21.855 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:58:22.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.503+0000 7fa345d9b640 1 -- 172.21.15.139:0/1473586910 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338004230 con 0x7fa348101450 2024-09-16T01:58:22.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.504+0000 7fa34cbdb640 1 -- 172.21.15.139:0/1473586910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348101450 msgr2=0x7fa348105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:22.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.504+0000 7fa34cbdb640 1 --2- 172.21.15.139:0/1473586910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348101450 0x7fa348105920 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa338009900 tx=0x7fa33802a4b0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.512+0000 7fa34cbdb640 1 -- 172.21.15.139:0/1473586910 shutdown_connections 2024-09-16T01:58:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.512+0000 7fa34cbdb640 1 --2- 172.21.15.139:0/1473586910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa348105f40 0x7fa3481063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.512+0000 7fa34cbdb640 1 --2- 172.21.15.139:0/1473586910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348101450 0x7fa348105920 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.512+0000 7fa34cbdb640 1 --2- 172.21.15.139:0/1473586910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa348100b30 0x7fa348100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.512+0000 7fa34cbdb640 1 -- 172.21.15.139:0/1473586910 >> 172.21.15.139:0/1473586910 conn(0x7fa3480fc7a0 msgr2=0x7fa3480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.512+0000 7fa34cbdb640 1 -- 172.21.15.139:0/1473586910 shutdown_connections 2024-09-16T01:58:22.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.512+0000 7fa34cbdb640 1 -- 172.21.15.139:0/1473586910 wait complete. 2024-09-16T01:58:22.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.513+0000 7fa34cbdb640 1 Processor -- start 2024-09-16T01:58:22.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.513+0000 7fa34cbdb640 1 -- start start 2024-09-16T01:58:22.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.514+0000 7fa34cbdb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa348100b30 0x7fa348073ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.514+0000 7fa34cbdb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa348101450 0x7fa34806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.514+0000 7fa34cbdb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348105f40 0x7fa34806e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.514+0000 7fa34cbdb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa3481185e0 con 0x7fa348105f40 2024-09-16T01:58:22.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.514+0000 7fa34cbdb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa348118460 con 0x7fa348101450 2024-09-16T01:58:22.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.514+0000 7fa34cbdb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa348118760 con 0x7fa348100b30 2024-09-16T01:58:22.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.515+0000 7fa346d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348105f40 0x7fa34806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:22.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.515+0000 7fa346d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348105f40 0x7fa34806e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38770/0 (socket says 172.21.15.139:38770) 2024-09-16T01:58:22.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.515+0000 7fa346d9d640 1 -- 172.21.15.139:0/814543769 learned_addr learned my addr 172.21.15.139:0/814543769 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:22.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.517+0000 7fa33ffff640 1 --2- 172.21.15.139:0/814543769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa348101450 0x7fa34806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:22.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.517+0000 7fa34659c640 1 --2- 172.21.15.139:0/814543769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa348100b30 0x7fa348073ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:22.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.517+0000 7fa33ffff640 1 -- 172.21.15.139:0/814543769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa348100b30 msgr2=0x7fa348073ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:22.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.517+0000 7fa33ffff640 1 --2- 172.21.15.139:0/814543769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa348100b30 0x7fa348073ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.517+0000 7fa33ffff640 1 -- 172.21.15.139:0/814543769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348105f40 msgr2=0x7fa34806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:22.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.517+0000 7fa33ffff640 1 --2- 172.21.15.139:0/814543769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348105f40 0x7fa34806e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.517+0000 7fa33ffff640 1 -- 172.21.15.139:0/814543769 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34806e7b0 con 0x7fa348101450 2024-09-16T01:58:22.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.518+0000 7fa33ffff640 1 --2- 172.21.15.139:0/814543769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa348101450 0x7fa34806db40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa338006010 tx=0x7fa3380054c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:22.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.519+0000 7fa33f7fe640 1 -- 172.21.15.139:0/814543769 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338040030 con 0x7fa348101450 2024-09-16T01:58:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.519+0000 7fa34cbdb640 1 -- 172.21.15.139:0/814543769 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34810cf20 con 0x7fa348101450 2024-09-16T01:58:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.519+0000 7fa34cbdb640 1 -- 172.21.15.139:0/814543769 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa34810d400 con 0x7fa348101450 2024-09-16T01:58:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.523+0000 7fa34efb4740 1 Processor -- start 2024-09-16T01:58:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.523+0000 7fa33f7fe640 1 -- 172.21.15.139:0/814543769 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa338005d10 con 0x7fa348101450 2024-09-16T01:58:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.523+0000 7fa33f7fe640 1 -- 172.21.15.139:0/814543769 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338004230 con 0x7fa348101450 2024-09-16T01:58:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.526+0000 7fa34efb4740 1 -- start start 2024-09-16T01:58:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.526+0000 7fa34efb4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x560831427440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.526+0000 7fa34efb4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x560831431d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.526+0000 7fa34efb4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x560831434630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.526+0000 7fa34efb4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56083138c620 con 0x560831432240 2024-09-16T01:58:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.526+0000 7fa34efb4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560831333e50 con 0x560831427980 2024-09-16T01:58:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.526+0000 7fa34efb4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5608313885e0 con 0x560831427060 2024-09-16T01:58:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.529+0000 7fa346d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x560831427440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.529+0000 7fa346d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x560831427440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43344/0 (socket says 172.21.15.139:43344) 2024-09-16T01:58:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.529+0000 7fa34659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x560831431d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.529+0000 7fa346d9d640 1 -- 172.21.15.139:0/655430130 learned_addr learned my addr 172.21.15.139:0/655430130 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.532+0000 7fa34659c640 1 -- 172.21.15.139:0/655430130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 msgr2=0x560831427440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.532+0000 7fa34659c640 1 --2- 172.21.15.139:0/655430130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x560831427440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.532+0000 7fa34659c640 1 -- 172.21.15.139:0/655430130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 msgr2=0x560831434630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:58:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.532+0000 7fa34659c640 1 --2- 172.21.15.139:0/655430130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x560831434630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.532+0000 7fa34659c640 1 -- 172.21.15.139:0/655430130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560831324d40 con 0x560831427980 2024-09-16T01:58:22.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.534+0000 7fa34659c640 1 --2- 172.21.15.139:0/655430130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x560831431d00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa33000bbf0 tx=0x7fa3300107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=544e5a1376bf6136 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:22.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.539+0000 7fa33cff9640 1 -- 172.21.15.139:0/655430130 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330004030 con 0x560831427980 2024-09-16T01:58:22.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.542+0000 7fa33cff9640 1 -- 172.21.15.139:0/655430130 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3300041d0 con 0x560831427980 2024-09-16T01:58:22.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.542+0000 7fa33cff9640 1 -- 172.21.15.139:0/655430130 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3300116c0 con 0x560831427980 2024-09-16T01:58:22.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.544+0000 7fa33f7fe640 1 -- 172.21.15.139:0/814543769 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa338043030 con 0x7fa348101450 2024-09-16T01:58:22.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.544+0000 7fa34efb4740 1 -- 172.21.15.139:0/655430130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 msgr2=0x560831431d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:22.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.544+0000 7fa34efb4740 1 --2- 172.21.15.139:0/655430130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x560831431d00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa33000bbf0 tx=0x7fa3300107a0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.544+0000 7fa34efb4740 1 -- 172.21.15.139:0/655430130 shutdown_connections 2024-09-16T01:58:22.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.544+0000 7fa34efb4740 1 --2- 172.21.15.139:0/655430130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x560831434630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.545+0000 7fa34efb4740 1 --2- 172.21.15.139:0/655430130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x560831431d00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.545+0000 7fa34efb4740 1 --2- 172.21.15.139:0/655430130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x560831427440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.545+0000 7fa34efb4740 1 -- 172.21.15.139:0/655430130 >> 172.21.15.139:0/655430130 conn(0x5608314235f0 msgr2=0x5608314239d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:22.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.545+0000 7fa33f7fe640 1 --2- 172.21.15.139:0/814543769 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa3340a3580 0x7fa3340a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.545+0000 7fa33f7fe640 1 -- 172.21.15.139:0/814543769 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7fa3380f0780 con 0x7fa348101450 2024-09-16T01:58:22.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.546+0000 7fa34659c640 1 --2- 172.21.15.139:0/814543769 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa3340a3580 0x7fa3340a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:22.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.549+0000 7fa34efb4740 1 -- 172.21.15.139:0/655430130 shutdown_connections 2024-09-16T01:58:22.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.549+0000 7fa34659c640 1 --2- 172.21.15.139:0/814543769 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa3340a3580 0x7fa3340a5a40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa330010ca0 tx=0x7fa330010fd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:22.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.551+0000 7fa34efb4740 1 -- 172.21.15.139:0/655430130 wait complete. 2024-09-16T01:58:22.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.552+0000 7fa34efb4740 1 -- start start 2024-09-16T01:58:22.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.552+0000 7fa34efb4740 1 Processor -- start 2024-09-16T01:58:22.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.552+0000 7fa34efb4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x56083142f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.553+0000 7fa34efb4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x56083142f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.553+0000 7fa34efb4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x56083142fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.553+0000 7fa34efb4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560831434e10 con 0x560831432240 2024-09-16T01:58:22.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.553+0000 7fa34efb4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560831314f10 con 0x560831427980 2024-09-16T01:58:22.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.553+0000 7fa34efb4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560831434f90 con 0x560831427060 2024-09-16T01:58:22.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.553+0000 7fa34efb4740 10 client.0 _ll_register_callbacks cb 0x560831338db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:22.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.553+0000 7fa34659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x56083142f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:22.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.553+0000 7fa34659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x56083142f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43360/0 (socket says 172.21.15.139:43360) 2024-09-16T01:58:22.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.553+0000 7fa34659c640 1 -- 172.21.15.139:0/4244518626 learned_addr learned my addr 172.21.15.139:0/4244518626 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:22.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.554+0000 7fa34659c640 1 -- 172.21.15.139:0/4244518626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 msgr2=0x56083142f720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:58:22.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.554+0000 7fa34659c640 1 --2- 172.21.15.139:0/4244518626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x56083142f720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.554+0000 7fa34659c640 1 -- 172.21.15.139:0/4244518626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 msgr2=0x56083142fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:22.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.554+0000 7fa34659c640 1 --2- 172.21.15.139:0/4244518626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x56083142fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.554+0000 7fa34659c640 1 -- 172.21.15.139:0/4244518626 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56083109b580 con 0x560831427060 2024-09-16T01:58:22.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.559+0000 7fa34659c640 1 --2- 172.21.15.139:0/4244518626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x56083142f1e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa33001a000 tx=0x7fa330005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae5f221895f524a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:22.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.559+0000 7fa32dffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:58:22.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.559+0000 7fa32dffb640 1 -- 172.21.15.139:0/4244518626 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330036060 con 0x560831427060 2024-09-16T01:58:22.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.560+0000 7fa34efb4740 1 -- 172.21.15.139:0/4244518626 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3300041d0 con 0x560831427060 2024-09-16T01:58:22.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.560+0000 7fa32dffb640 1 -- 172.21.15.139:0/4244518626 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7fa3300046c0 con 0x560831427060 2024-09-16T01:58:22.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.560+0000 7fa32dffb640 1 -- 172.21.15.139:0/4244518626 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa33003d050 con 0x560831427060 2024-09-16T01:58:22.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.560+0000 7fa34efb4740 10 client.26689 fetch_fsmap learned FSMap version 76 2024-09-16T01:58:22.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.560+0000 7fa34efb4740 1 -- 172.21.15.139:0/4244518626 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x560831165960 con 0x560831427060 2024-09-16T01:58:22.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.565+0000 7fa32dffb640 1 -- 172.21.15.139:0/4244518626 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa33000be70 con 0x560831427060 2024-09-16T01:58:22.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.565+0000 7fa32dffb640 1 -- 172.21.15.139:0/4244518626 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 76) ==== 8446+0+0 (secure 0 0 0) 0x7fa330011c20 con 0x560831427060 2024-09-16T01:58:22.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.566+0000 7fa34efb4740 10 client.26689 fetch_fsmap finished waiting for FSMap version 76 2024-09-16T01:58:22.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.566+0000 7fa34efb4740 10 client.26689 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:58:22.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.566+0000 7fa34efb4740 20 client.26689 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:22.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.566+0000 7fa34efb4740 1 --2- 172.21.15.139:0/4244518626 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5608314642c0 0x560831466780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.566+0000 7fa34efb4740 4 client.26689 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:58:22.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.566+0000 7fa34efb4740 1 -- 172.21.15.139:0/4244518626 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa3300041d0 con 0x5608314642c0 2024-09-16T01:58:22.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.571+0000 7fa33ffff640 1 --2- 172.21.15.139:0/4244518626 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5608314642c0 0x560831466780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:22.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.582+0000 7fa33ffff640 1 --2- 172.21.15.139:0/4244518626 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5608314642c0 0x560831466780 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d9a60cbee4004a server_cookie=9e1f18719e81d4b1 in_seq=0 out_seq=0 2024-09-16T01:58:22.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.583+0000 7fa32dffb640 10 client.26689 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:22.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.585+0000 7fa32dffb640 1 -- 172.21.15.139:0/4244518626 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa3300041d0 con 0x5608314642c0 2024-09-16T01:58:22.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.585+0000 7fa32dffb640 10 client.26689 handle_command_reply: tid=0 2024-09-16T01:58:22.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.585+0000 7fa34efb4740 1 client.26689 shutdown 2024-09-16T01:58:22.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.589+0000 7fa34efb4740 1 -- 172.21.15.139:0/4244518626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 msgr2=0x56083142f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:22.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.589+0000 7fa34efb4740 1 --2- 172.21.15.139:0/4244518626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x56083142f1e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa33001a000 tx=0x7fa330005850 comp rx=0 tx=0).stop 2024-09-16T01:58:22.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.594+0000 7fa34efb4740 1 -- 172.21.15.139:0/4244518626 shutdown_connections 2024-09-16T01:58:22.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.594+0000 7fa34efb4740 1 --2- 172.21.15.139:0/4244518626 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5608314642c0 0x560831466780 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.594+0000 7fa34efb4740 1 --2- 172.21.15.139:0/4244518626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x56083142fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.594+0000 7fa34efb4740 1 --2- 172.21.15.139:0/4244518626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x56083142f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.594+0000 7fa34efb4740 1 --2- 172.21.15.139:0/4244518626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x56083142f1e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.594+0000 7fa34efb4740 1 -- 172.21.15.139:0/4244518626 >> 172.21.15.139:0/4244518626 conn(0x5608314235f0 msgr2=0x5608314292d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:22.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.594+0000 7fa32dffb640 0 client.26689 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:22.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.596+0000 7fa34efb4740 1 -- 172.21.15.139:0/4244518626 shutdown_connections 2024-09-16T01:58:22.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.597+0000 7fa34efb4740 1 -- 172.21.15.139:0/4244518626 wait complete. 2024-09-16T01:58:22.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.597+0000 7fa34efb4740 20 client.26689 trim_cache size 0 max 16384 2024-09-16T01:58:22.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.628+0000 7fa34efb4740 1 Processor -- start 2024-09-16T01:58:22.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.628+0000 7fa34efb4740 1 -- start start 2024-09-16T01:58:22.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.628+0000 7fa34efb4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x5608314225a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.628+0000 7fa34efb4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x560831429ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.628+0000 7fa34efb4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x56083146c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.629+0000 7fa34efb4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa3280102e0 con 0x560831432240 2024-09-16T01:58:22.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.629+0000 7fa34efb4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa3280120f0 con 0x560831427980 2024-09-16T01:58:22.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.629+0000 7fa34efb4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa328010a40 con 0x560831427060 2024-09-16T01:58:22.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.629+0000 7fa34659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x5608314225a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:22.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.634+0000 7fa346d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x56083146c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:22.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.634+0000 7fa346d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x56083146c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38786/0 (socket says 172.21.15.139:38786) 2024-09-16T01:58:22.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.634+0000 7fa346d9d640 1 -- 172.21.15.139:0/1920464286 learned_addr learned my addr 172.21.15.139:0/1920464286 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:22.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.637+0000 7fa33ffff640 1 --2- 172.21.15.139:0/1920464286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x560831429ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:22.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.637+0000 7fa34659c640 1 -- 172.21.15.139:0/1920464286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 msgr2=0x560831429ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:22.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.637+0000 7fa34659c640 1 --2- 172.21.15.139:0/1920464286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x560831429ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.637+0000 7fa34659c640 1 -- 172.21.15.139:0/1920464286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 msgr2=0x56083146c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:22.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.637+0000 7fa34659c640 1 --2- 172.21.15.139:0/1920464286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x56083146c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.637+0000 7fa34659c640 1 -- 172.21.15.139:0/1920464286 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560831424760 con 0x560831427060 2024-09-16T01:58:22.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.642+0000 7fa34659c640 1 --2- 172.21.15.139:0/1920464286 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x5608314225a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa330034040 tx=0x7fa33000b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e381139cd05f901 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:22.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.649+0000 7fa32ffff640 1 -- 172.21.15.139:0/1920464286 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33000be50 con 0x560831427060 2024-09-16T01:58:22.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.649+0000 7fa32ffff640 1 -- 172.21.15.139:0/1920464286 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa330005b90 con 0x560831427060 2024-09-16T01:58:22.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.649+0000 7fa32ffff640 1 -- 172.21.15.139:0/1920464286 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330004eb0 con 0x560831427060 2024-09-16T01:58:22.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.651+0000 7fa34efb4740 1 -- 172.21.15.139:0/1920464286 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 msgr2=0x5608314225a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:22.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.651+0000 7fa34efb4740 1 --2- 172.21.15.139:0/1920464286 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x5608314225a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa330034040 tx=0x7fa33000b390 comp rx=0 tx=0).stop 2024-09-16T01:58:22.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.653+0000 7fa34efb4740 1 -- 172.21.15.139:0/1920464286 shutdown_connections 2024-09-16T01:58:22.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.653+0000 7fa34efb4740 1 --2- 172.21.15.139:0/1920464286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x56083146c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.653+0000 7fa34efb4740 1 --2- 172.21.15.139:0/1920464286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x560831429ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.653+0000 7fa34efb4740 1 --2- 172.21.15.139:0/1920464286 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x5608314225a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.653+0000 7fa34efb4740 1 -- 172.21.15.139:0/1920464286 >> 172.21.15.139:0/1920464286 conn(0x5608314235f0 msgr2=0x56083142f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:22.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.653+0000 7fa34efb4740 1 -- 172.21.15.139:0/1920464286 shutdown_connections 2024-09-16T01:58:22.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.653+0000 7fa34efb4740 1 -- 172.21.15.139:0/1920464286 wait complete. 2024-09-16T01:58:22.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.654+0000 7fa34efb4740 1 -- start start 2024-09-16T01:58:22.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.654+0000 7fa34efb4740 1 Processor -- start 2024-09-16T01:58:22.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.655+0000 7fa34efb4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x56083143a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.655+0000 7fa34efb4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x56083143a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.655+0000 7fa34efb4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x560831437240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.655+0000 7fa34efb4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa328011960 con 0x560831432240 2024-09-16T01:58:22.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.655+0000 7fa34efb4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa3280111d0 con 0x560831427980 2024-09-16T01:58:22.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.655+0000 7fa34efb4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa32800e2e0 con 0x560831427060 2024-09-16T01:58:22.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.656+0000 7fa34efb4740 10 client.0 _ll_register_callbacks cb 0x7fa33000bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:22.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.659+0000 7fa34659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x56083143a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:22.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.659+0000 7fa34659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x56083143a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43380/0 (socket says 172.21.15.139:43380) 2024-09-16T01:58:22.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.659+0000 7fa34659c640 1 -- 172.21.15.139:0/162937740 learned_addr learned my addr 172.21.15.139:0/162937740 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:22.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.659+0000 7fa346d9d640 1 --2- 172.21.15.139:0/162937740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x560831437240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:22.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.659+0000 7fa34659c640 1 -- 172.21.15.139:0/162937740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 msgr2=0x56083143a7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:58:22.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.659+0000 7fa34659c640 1 --2- 172.21.15.139:0/162937740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x56083143a7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.659+0000 7fa34659c640 1 -- 172.21.15.139:0/162937740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 msgr2=0x560831437240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:22.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.659+0000 7fa34659c640 1 --2- 172.21.15.139:0/162937740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x560831437240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.659+0000 7fa34659c640 1 -- 172.21.15.139:0/162937740 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608314377b0 con 0x560831427060 2024-09-16T01:58:22.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.660+0000 7fa346d9d640 1 --2- 172.21.15.139:0/162937740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x560831437240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:58:22.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.663+0000 7fa34659c640 1 --2- 172.21.15.139:0/162937740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x56083143a270 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa330004910 tx=0x7fa330004940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaf54d5ff18440e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:22.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.666+0000 7fa32dffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:58:22.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.666+0000 7fa32dffb640 1 -- 172.21.15.139:0/162937740 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330003bd0 con 0x560831427060 2024-09-16T01:58:22.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.666+0000 7fa34efb4740 1 -- 172.21.15.139:0/162937740 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa330005b90 con 0x560831427060 2024-09-16T01:58:22.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.666+0000 7fa32dffb640 1 -- 172.21.15.139:0/162937740 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7fa330004bf0 con 0x560831427060 2024-09-16T01:58:22.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.668+0000 7fa32dffb640 1 -- 172.21.15.139:0/162937740 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa330058050 con 0x560831427060 2024-09-16T01:58:22.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.669+0000 7fa34efb4740 10 client.26695 fetch_fsmap learned FSMap version 76 2024-09-16T01:58:22.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.669+0000 7fa34efb4740 1 -- 172.21.15.139:0/162937740 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5608314cf910 con 0x560831427060 2024-09-16T01:58:22.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.675+0000 7fa32dffb640 1 -- 172.21.15.139:0/162937740 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa330005b90 con 0x560831427060 2024-09-16T01:58:22.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.675+0000 7fa32dffb640 1 -- 172.21.15.139:0/162937740 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 76) ==== 8446+0+0 (secure 0 0 0) 0x7fa330011700 con 0x560831427060 2024-09-16T01:58:22.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.676+0000 7fa34efb4740 10 client.26695 fetch_fsmap finished waiting for FSMap version 76 2024-09-16T01:58:22.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.676+0000 7fa34efb4740 10 client.26695 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:58:22.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.676+0000 7fa34efb4740 20 client.26695 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:22.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.676+0000 7fa34efb4740 1 --2- 172.21.15.139:0/162937740 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5608314cff80 0x5608314d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:22.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.676+0000 7fa34efb4740 4 client.26695 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7235a3e6", "release": true, "await": true}] 2024-09-16T01:58:22.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.676+0000 7fa34efb4740 1 -- 172.21.15.139:0/162937740 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7235a3e6", "release": true, "await": true}) -- 0x560831399b90 con 0x5608314cff80 2024-09-16T01:58:22.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.678+0000 7fa33ffff640 1 --2- 172.21.15.139:0/162937740 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5608314cff80 0x5608314d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:22.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.684+0000 7fa33ffff640 1 --2- 172.21.15.139:0/162937740 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5608314cff80 0x5608314d23a0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d42d140a664c94aa server_cookie=304365ebf0a3ef06 in_seq=0 out_seq=0 2024-09-16T01:58:22.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.686+0000 7fa32dffb640 10 client.26695 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:22.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa32dffb640 1 -- 172.21.15.139:0/162937740 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560831399b90 con 0x5608314cff80 2024-09-16T01:58:22.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa32dffb640 10 client.26695 handle_command_reply: tid=0 2024-09-16T01:58:22.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa34efb4740 1 client.26695 shutdown 2024-09-16T01:58:22.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa34efb4740 1 -- 172.21.15.139:0/162937740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 msgr2=0x56083143a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:22.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa34efb4740 1 --2- 172.21.15.139:0/162937740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x56083143a270 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa330004910 tx=0x7fa330004940 comp rx=0 tx=0).stop 2024-09-16T01:58:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa34efb4740 1 -- 172.21.15.139:0/162937740 shutdown_connections 2024-09-16T01:58:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa34efb4740 1 --2- 172.21.15.139:0/162937740 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5608314cff80 0x5608314d23a0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa34efb4740 1 --2- 172.21.15.139:0/162937740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560831432240 0x560831437240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa34efb4740 1 --2- 172.21.15.139:0/162937740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560831427980 0x56083143a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa34efb4740 1 --2- 172.21.15.139:0/162937740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560831427060 0x56083143a270 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa34efb4740 1 -- 172.21.15.139:0/162937740 >> 172.21.15.139:0/162937740 conn(0x5608314235f0 msgr2=0x56083143d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa32dffb640 0 client.26695 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa34efb4740 1 -- 172.21.15.139:0/162937740 shutdown_connections 2024-09-16T01:58:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa34efb4740 1 -- 172.21.15.139:0/162937740 wait complete. 2024-09-16T01:58:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.742+0000 7fa34efb4740 20 client.26695 trim_cache size 0 max 16384 2024-09-16T01:58:22.745 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:58:22.745 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 76, 2024-09-16T01:58:22.745 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:58:22.745 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 64, 2024-09-16T01:58:22.745 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:58:22.745 INFO:teuthology.orchestra.run.smithi139.stdout: "7235a3e6": { 2024-09-16T01:58:22.745 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 64, 2024-09-16T01:58:22.745 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 142.9, 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:22.746 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:22.747 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:22.747 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:22.747 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:58:22.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.744+0000 7fa34cbdb640 1 -- 172.21.15.139:0/814543769 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa3340a3580 msgr2=0x7fa3340a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.744+0000 7fa34cbdb640 1 --2- 172.21.15.139:0/814543769 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa3340a3580 0x7fa3340a5a40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa330010ca0 tx=0x7fa330010fd0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.744+0000 7fa34cbdb640 1 -- 172.21.15.139:0/814543769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa348101450 msgr2=0x7fa34806db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.744+0000 7fa34cbdb640 1 --2- 172.21.15.139:0/814543769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa348101450 0x7fa34806db40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa338006010 tx=0x7fa3380054c0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.745+0000 7fa34cbdb640 1 -- 172.21.15.139:0/814543769 shutdown_connections 2024-09-16T01:58:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.745+0000 7fa34cbdb640 1 --2- 172.21.15.139:0/814543769 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa3340a3580 0x7fa3340a5a40 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa330010ca0 tx=0x7fa330010fd0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.745+0000 7fa34cbdb640 1 --2- 172.21.15.139:0/814543769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348105f40 0x7fa34806e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.745+0000 7fa34cbdb640 1 --2- 172.21.15.139:0/814543769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa348101450 0x7fa34806db40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.745+0000 7fa34cbdb640 1 --2- 172.21.15.139:0/814543769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa348100b30 0x7fa348073ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.745+0000 7fa34cbdb640 1 -- 172.21.15.139:0/814543769 >> 172.21.15.139:0/814543769 conn(0x7fa3480fc7a0 msgr2=0x7fa3481038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.746+0000 7fa34cbdb640 1 -- 172.21.15.139:0/814543769 shutdown_connections 2024-09-16T01:58:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:22.746+0000 7fa34cbdb640 1 -- 172.21.15.139:0/814543769 wait complete. 2024-09-16T01:58:23.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:23 smithi139 ceph-mon[31175]: pgmap v572: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 153 op/s 2024-09-16T01:58:23.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:23 smithi176 ceph-mon[35324]: pgmap v572: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 153 op/s 2024-09-16T01:58:23.530 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7235a3e6', total seconds elapsed: 29.7 2024-09-16T01:58:23.530 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.95 2024-09-16T01:58:23.530 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:58:23.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:23 smithi164 ceph-mon[35121]: pgmap v572: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 153 op/s 2024-09-16T01:58:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.568+0000 7fe57df2c640 1 -- 172.21.15.139:0/2687203309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe57810ea10 msgr2=0x7fe578115430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.568+0000 7fe57df2c640 1 --2- 172.21.15.139:0/2687203309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe57810ea10 0x7fe578115430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe570005fd0 tx=0x7fe57002b430 comp rx=0 tx=0).stop 2024-09-16T01:58:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.574+0000 7fe57df2c640 1 -- 172.21.15.139:0/2687203309 shutdown_connections 2024-09-16T01:58:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.574+0000 7fe57df2c640 1 --2- 172.21.15.139:0/2687203309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe57810ea10 0x7fe578115430 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.574+0000 7fe57df2c640 1 --2- 172.21.15.139:0/2687203309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe578109e10 0x7fe57810e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.574+0000 7fe57df2c640 1 --2- 172.21.15.139:0/2687203309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe578109460 0x7fe578109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.574+0000 7fe57df2c640 1 -- 172.21.15.139:0/2687203309 >> 172.21.15.139:0/2687203309 conn(0x7fe57806d210 msgr2=0x7fe57806d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:23.582 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.581+0000 7fe57df2c640 1 -- 172.21.15.139:0/2687203309 shutdown_connections 2024-09-16T01:58:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.582+0000 7fe57df2c640 1 -- 172.21.15.139:0/2687203309 wait complete. 2024-09-16T01:58:23.585 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.583+0000 7fe57df2c640 1 Processor -- start 2024-09-16T01:58:23.585 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.584+0000 7fe57df2c640 1 -- start start 2024-09-16T01:58:23.585 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.584+0000 7fe57df2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe578109460 0x7fe578114c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.585 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.584+0000 7fe57df2c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe57810ea10 0x7fe5781151c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.584+0000 7fe57df2c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5781246b0 0x7fe578124b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.584+0000 7fe57df2c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe578118200 con 0x7fe578109460 2024-09-16T01:58:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.584+0000 7fe57df2c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe578118080 con 0x7fe5781246b0 2024-09-16T01:58:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.584+0000 7fe57df2c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe578118380 con 0x7fe57810ea10 2024-09-16T01:58:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.585+0000 7fe577fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5781246b0 0x7fe578124b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.585+0000 7fe577fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5781246b0 0x7fe578124b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37110/0 (socket says 172.21.15.139:37110) 2024-09-16T01:58:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.585+0000 7fe577fff640 1 -- 172.21.15.139:0/1694033645 learned_addr learned my addr 172.21.15.139:0/1694033645 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.585+0000 7fe576ffd640 1 --2- 172.21.15.139:0/1694033645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe57810ea10 0x7fe5781151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.585+0000 7fe577fff640 1 -- 172.21.15.139:0/1694033645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe57810ea10 msgr2=0x7fe5781151c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.585+0000 7fe577fff640 1 --2- 172.21.15.139:0/1694033645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe57810ea10 0x7fe5781151c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.585+0000 7fe577fff640 1 -- 172.21.15.139:0/1694033645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe578109460 msgr2=0x7fe578114c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.585+0000 7fe577fff640 1 --2- 172.21.15.139:0/1694033645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe578109460 0x7fe578114c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.585+0000 7fe577fff640 1 -- 172.21.15.139:0/1694033645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe578125420 con 0x7fe5781246b0 2024-09-16T01:58:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.585+0000 7fe577fff640 1 --2- 172.21.15.139:0/1694033645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5781246b0 0x7fe578124b60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe57002b9a0 tx=0x7fe57000a990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.588+0000 7fe574ff9640 1 -- 172.21.15.139:0/1694033645 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe570037040 con 0x7fe5781246b0 2024-09-16T01:58:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.588+0000 7fe57df2c640 1 -- 172.21.15.139:0/1694033645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe578112e20 con 0x7fe5781246b0 2024-09-16T01:58:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.588+0000 7fe57df2c640 1 -- 172.21.15.139:0/1694033645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe578113370 con 0x7fe5781246b0 2024-09-16T01:58:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.588+0000 7fe574ff9640 1 -- 172.21.15.139:0/1694033645 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe57000a2d0 con 0x7fe5781246b0 2024-09-16T01:58:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.588+0000 7fe574ff9640 1 -- 172.21.15.139:0/1694033645 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5700042b0 con 0x7fe5781246b0 2024-09-16T01:58:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.593+0000 7fe580cc2740 1 Processor -- start 2024-09-16T01:58:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.595+0000 7fe574ff9640 1 -- 172.21.15.139:0/1694033645 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe570004450 con 0x7fe5781246b0 2024-09-16T01:58:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.596+0000 7fe574ff9640 1 --2- 172.21.15.139:0/1694033645 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe5600a3690 0x7fe5600a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.597+0000 7fe574ff9640 1 -- 172.21.15.139:0/1694033645 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7fe5700f0220 con 0x7fe5781246b0 2024-09-16T01:58:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.598+0000 7fe580cc2740 1 -- start start 2024-09-16T01:58:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.598+0000 7fe580cc2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3665af0 0x55e1c3665ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.598+0000 7fe580cc2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3666410 0x55e1c3670770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.598+0000 7fe580cc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3670cb0 0x55e1c36730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.598+0000 7fe580cc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e1c35ca2f0 con 0x55e1c3670cb0 2024-09-16T01:58:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.598+0000 7fe580cc2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e1c3572270 con 0x55e1c3665af0 2024-09-16T01:58:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.598+0000 7fe580cc2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e1c35c62b0 con 0x55e1c3666410 2024-09-16T01:58:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.600+0000 7fe576ffd640 1 --2- 172.21.15.139:0/1694033645 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe5600a3690 0x7fe5600a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.602+0000 7fe576ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3666410 0x55e1c3670770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.602+0000 7fe576ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3666410 0x55e1c3670770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43406/0 (socket says 172.21.15.139:43406) 2024-09-16T01:58:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.602+0000 7fe576ffd640 1 -- 172.21.15.139:0/1562323507 learned_addr learned my addr 172.21.15.139:0/1562323507 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.602+0000 7fe5777fe640 1 --2- 172.21.15.139:0/1562323507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3665af0 0x55e1c3665ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.602+0000 7fe577fff640 1 --2- 172.21.15.139:0/1562323507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3670cb0 0x55e1c36730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.604+0000 7fe5777fe640 1 -- 172.21.15.139:0/1562323507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3666410 msgr2=0x55e1c3670770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.604+0000 7fe5777fe640 1 --2- 172.21.15.139:0/1562323507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3666410 0x55e1c3670770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.604+0000 7fe5777fe640 1 -- 172.21.15.139:0/1562323507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3670cb0 msgr2=0x55e1c36730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.604+0000 7fe5777fe640 1 --2- 172.21.15.139:0/1562323507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3670cb0 0x55e1c36730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.605+0000 7fe5777fe640 1 -- 172.21.15.139:0/1562323507 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c3571340 con 0x55e1c3665af0 2024-09-16T01:58:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.608+0000 7fe5777fe640 1 --2- 172.21.15.139:0/1562323507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3665af0 0x55e1c3665ed0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00ba40 tx=0x7fe56c00bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecf20edcdb972811 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.611+0000 7fe55dffb640 1 -- 172.21.15.139:0/1562323507 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c016070 con 0x55e1c3665af0 2024-09-16T01:58:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.611+0000 7fe55dffb640 1 -- 172.21.15.139:0/1562323507 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56c00dc40 con 0x55e1c3665af0 2024-09-16T01:58:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.611+0000 7fe576ffd640 1 --2- 172.21.15.139:0/1694033645 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe5600a3690 0x7fe5600a5b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe578073990 tx=0x7fe568009580 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.612+0000 7fe580cc2740 1 -- 172.21.15.139:0/1562323507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3665af0 msgr2=0x55e1c3665ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.612+0000 7fe580cc2740 1 --2- 172.21.15.139:0/1562323507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3665af0 0x55e1c3665ed0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00ba40 tx=0x7fe56c00bf00 comp rx=0 tx=0).stop 2024-09-16T01:58:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.622+0000 7fe580cc2740 1 -- 172.21.15.139:0/1562323507 shutdown_connections 2024-09-16T01:58:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.622+0000 7fe580cc2740 1 --2- 172.21.15.139:0/1562323507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3670cb0 0x55e1c36730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.622+0000 7fe580cc2740 1 --2- 172.21.15.139:0/1562323507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3666410 0x55e1c3670770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.622+0000 7fe580cc2740 1 --2- 172.21.15.139:0/1562323507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3665af0 0x55e1c3665ed0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.622+0000 7fe580cc2740 1 -- 172.21.15.139:0/1562323507 >> 172.21.15.139:0/1562323507 conn(0x55e1c3662060 msgr2=0x55e1c3662440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.626+0000 7fe580cc2740 1 -- 172.21.15.139:0/1562323507 shutdown_connections 2024-09-16T01:58:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.628+0000 7fe580cc2740 1 -- 172.21.15.139:0/1562323507 wait complete. 2024-09-16T01:58:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.628+0000 7fe580cc2740 1 -- start start 2024-09-16T01:58:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.632+0000 7fe580cc2740 1 Processor -- start 2024-09-16T01:58:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.633+0000 7fe580cc2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3665af0 0x55e1c366da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.633+0000 7fe580cc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3666410 0x55e1c366dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.633+0000 7fe580cc2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3670cb0 0x55e1c366e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.633+0000 7fe580cc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e1c36738d0 con 0x55e1c3666410 2024-09-16T01:58:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.633+0000 7fe580cc2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e1c3553290 con 0x55e1c3665af0 2024-09-16T01:58:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.633+0000 7fe580cc2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e1c3673a50 con 0x55e1c3670cb0 2024-09-16T01:58:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.633+0000 7fe580cc2740 10 client.0 _ll_register_callbacks cb 0x55e1c3584140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.635+0000 7fe577fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3670cb0 0x55e1c366e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.635+0000 7fe577fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3670cb0 0x55e1c366e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43420/0 (socket says 172.21.15.139:43420) 2024-09-16T01:58:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.635+0000 7fe577fff640 1 -- 172.21.15.139:0/1267814216 learned_addr learned my addr 172.21.15.139:0/1267814216 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.635+0000 7fe5777fe640 1 --2- 172.21.15.139:0/1267814216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3665af0 0x55e1c366da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.635+0000 7fe5777fe640 1 -- 172.21.15.139:0/1267814216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3670cb0 msgr2=0x55e1c366e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.635+0000 7fe5777fe640 1 --2- 172.21.15.139:0/1267814216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3670cb0 0x55e1c366e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.635+0000 7fe5777fe640 1 -- 172.21.15.139:0/1267814216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3666410 msgr2=0x55e1c366dfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:58:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.635+0000 7fe5777fe640 1 --2- 172.21.15.139:0/1267814216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3666410 0x55e1c366dfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.635+0000 7fe5777fe640 1 -- 172.21.15.139:0/1267814216 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c32dd210 con 0x55e1c3665af0 2024-09-16T01:58:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.636+0000 7fe5777fe640 1 --2- 172.21.15.139:0/1267814216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3665af0 0x55e1c366da70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe56c004700 tx=0x7fe56c0056f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90f92588663c1335 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.636+0000 7fe556ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:58:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.636+0000 7fe556ffd640 1 -- 172.21.15.139:0/1267814216 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c016070 con 0x55e1c3665af0 2024-09-16T01:58:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.636+0000 7fe580cc2740 1 -- 172.21.15.139:0/1267814216 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe56c00dc40 con 0x55e1c3665af0 2024-09-16T01:58:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.637+0000 7fe556ffd640 1 -- 172.21.15.139:0/1267814216 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7fe56c004e20 con 0x55e1c3665af0 2024-09-16T01:58:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.637+0000 7fe556ffd640 1 -- 172.21.15.139:0/1267814216 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fe56c02c050 con 0x55e1c3665af0 2024-09-16T01:58:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.638+0000 7fe580cc2740 10 client.26726 fetch_fsmap learned FSMap version 76 2024-09-16T01:58:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.638+0000 7fe580cc2740 1 -- 172.21.15.139:0/1267814216 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55e1c33a9620 con 0x55e1c3665af0 2024-09-16T01:58:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.639+0000 7fe556ffd640 1 -- 172.21.15.139:0/1267814216 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56c00dc40 con 0x55e1c3665af0 2024-09-16T01:58:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.639+0000 7fe556ffd640 1 -- 172.21.15.139:0/1267814216 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 76) ==== 8446+0+0 (secure 0 0 0) 0x7fe56c003b90 con 0x55e1c3665af0 2024-09-16T01:58:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.639+0000 7fe580cc2740 10 client.26726 fetch_fsmap finished waiting for FSMap version 76 2024-09-16T01:58:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.639+0000 7fe580cc2740 10 client.26726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:58:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.639+0000 7fe580cc2740 20 client.26726 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.639+0000 7fe580cc2740 1 --2- 172.21.15.139:0/1267814216 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e1c36810c0 0x55e1c3683580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.639+0000 7fe580cc2740 4 client.26726 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:58:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.639+0000 7fe580cc2740 1 -- 172.21.15.139:0/1267814216 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1c35d7a20 con 0x55e1c36810c0 2024-09-16T01:58:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.643+0000 7fe576ffd640 1 --2- 172.21.15.139:0/1267814216 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e1c36810c0 0x55e1c3683580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.653+0000 7fe576ffd640 1 --2- 172.21.15.139:0/1267814216 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e1c36810c0 0x55e1c3683580 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3de5e4c68886b96 server_cookie=c029ba78667ea449 in_seq=0 out_seq=0 2024-09-16T01:58:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.654+0000 7fe556ffd640 10 client.26726 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.655+0000 7fe556ffd640 1 -- 172.21.15.139:0/1267814216 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1c35d7a20 con 0x55e1c36810c0 2024-09-16T01:58:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.655+0000 7fe556ffd640 10 client.26726 handle_command_reply: tid=0 2024-09-16T01:58:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.655+0000 7fe580cc2740 1 client.26726 shutdown 2024-09-16T01:58:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.668+0000 7fe580cc2740 1 -- 172.21.15.139:0/1267814216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3665af0 msgr2=0x55e1c366da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.668+0000 7fe580cc2740 1 --2- 172.21.15.139:0/1267814216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3665af0 0x55e1c366da70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe56c004700 tx=0x7fe56c0056f0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.671+0000 7fe580cc2740 1 -- 172.21.15.139:0/1267814216 shutdown_connections 2024-09-16T01:58:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.671+0000 7fe580cc2740 1 --2- 172.21.15.139:0/1267814216 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e1c36810c0 0x55e1c3683580 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.671+0000 7fe580cc2740 1 --2- 172.21.15.139:0/1267814216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3670cb0 0x55e1c366e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.671+0000 7fe580cc2740 1 --2- 172.21.15.139:0/1267814216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3666410 0x55e1c366dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.671+0000 7fe580cc2740 1 --2- 172.21.15.139:0/1267814216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3665af0 0x55e1c366da70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.671+0000 7fe580cc2740 1 -- 172.21.15.139:0/1267814216 >> 172.21.15.139:0/1267814216 conn(0x55e1c3662060 msgr2=0x55e1c36611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.671+0000 7fe556ffd640 0 client.26726 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.680+0000 7fe580cc2740 1 -- 172.21.15.139:0/1267814216 shutdown_connections 2024-09-16T01:58:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.682+0000 7fe580cc2740 1 -- 172.21.15.139:0/1267814216 wait complete. 2024-09-16T01:58:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.683+0000 7fe580cc2740 20 client.26726 trim_cache size 0 max 16384 2024-09-16T01:58:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.717+0000 7fe580cc2740 1 Processor -- start 2024-09-16T01:58:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.717+0000 7fe580cc2740 1 -- start start 2024-09-16T01:58:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.717+0000 7fe580cc2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3665af0 0x55e1c366e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.717+0000 7fe580cc2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3666410 0x55e1c366f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.717+0000 7fe580cc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3670cb0 0x55e1c3688e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.717+0000 7fe5777fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3665af0 0x55e1c366e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.717+0000 7fe580cc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe564010350 con 0x55e1c3670cb0 2024-09-16T01:58:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.717+0000 7fe580cc2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe564012160 con 0x55e1c3666410 2024-09-16T01:58:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.717+0000 7fe5777fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3665af0 0x55e1c366e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43428/0 (socket says 172.21.15.139:43428) 2024-09-16T01:58:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.717+0000 7fe5777fe640 1 -- 172.21.15.139:0/3187340563 learned_addr learned my addr 172.21.15.139:0/3187340563 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.717+0000 7fe580cc2740 1 -- 172.21.15.139:0/3187340563 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe564010ab0 con 0x55e1c3665af0 2024-09-16T01:58:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.717+0000 7fe576ffd640 1 --2- 172.21.15.139:0/3187340563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3666410 0x55e1c366f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.718+0000 7fe576ffd640 1 -- 172.21.15.139:0/3187340563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3665af0 msgr2=0x55e1c366e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.718+0000 7fe576ffd640 1 --2- 172.21.15.139:0/3187340563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3665af0 0x55e1c366e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.718+0000 7fe577fff640 1 --2- 172.21.15.139:0/3187340563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3670cb0 0x55e1c3688e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.718+0000 7fe576ffd640 1 -- 172.21.15.139:0/3187340563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3670cb0 msgr2=0x55e1c3688e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.718+0000 7fe576ffd640 1 --2- 172.21.15.139:0/3187340563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3670cb0 0x55e1c3688e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.718+0000 7fe576ffd640 1 -- 172.21.15.139:0/3187340563 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c3666240 con 0x55e1c3666410 2024-09-16T01:58:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.718+0000 7fe577fff640 1 --2- 172.21.15.139:0/3187340563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3670cb0 0x55e1c3688e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T01:58:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.718+0000 7fe576ffd640 1 --2- 172.21.15.139:0/3187340563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3666410 0x55e1c366f9e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55e1c366f020 tx=0x7fe56801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76aaaa6aa28b8440 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.720+0000 7fe55d7fa640 1 -- 172.21.15.139:0/3187340563 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568018da0 con 0x55e1c3666410 2024-09-16T01:58:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.720+0000 7fe55d7fa640 1 -- 172.21.15.139:0/3187340563 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56802e920 con 0x55e1c3666410 2024-09-16T01:58:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.721+0000 7fe580cc2740 1 -- 172.21.15.139:0/3187340563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3666410 msgr2=0x55e1c366f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.721+0000 7fe580cc2740 1 --2- 172.21.15.139:0/3187340563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3666410 0x55e1c366f9e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55e1c366f020 tx=0x7fe56801c000 comp rx=0 tx=0).stop 2024-09-16T01:58:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.733+0000 7fe580cc2740 1 -- 172.21.15.139:0/3187340563 shutdown_connections 2024-09-16T01:58:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.733+0000 7fe580cc2740 1 --2- 172.21.15.139:0/3187340563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3670cb0 0x55e1c3688e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.733+0000 7fe580cc2740 1 --2- 172.21.15.139:0/3187340563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3666410 0x55e1c366f9e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.733+0000 7fe580cc2740 1 --2- 172.21.15.139:0/3187340563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3665af0 0x55e1c366e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.733+0000 7fe580cc2740 1 -- 172.21.15.139:0/3187340563 >> 172.21.15.139:0/3187340563 conn(0x55e1c3662060 msgr2=0x55e1c366f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:23.736 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.736+0000 7fe580cc2740 1 -- 172.21.15.139:0/3187340563 shutdown_connections 2024-09-16T01:58:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.736+0000 7fe580cc2740 1 -- 172.21.15.139:0/3187340563 wait complete. 2024-09-16T01:58:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.736+0000 7fe580cc2740 1 -- start start 2024-09-16T01:58:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.736+0000 7fe580cc2740 1 Processor -- start 2024-09-16T01:58:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.737+0000 7fe580cc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3665af0 0x55e1c366b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.737+0000 7fe580cc2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3666410 0x55e1c3669ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.737+0000 7fe580cc2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3670cb0 0x55e1c366a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.737+0000 7fe580cc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe5640119d0 con 0x55e1c3665af0 2024-09-16T01:58:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.737+0000 7fe580cc2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe564011240 con 0x55e1c3666410 2024-09-16T01:58:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.737+0000 7fe580cc2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe56400e350 con 0x55e1c3670cb0 2024-09-16T01:58:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.737+0000 7fe580cc2740 10 client.0 _ll_register_callbacks cb 0x7fe56c00b890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.737+0000 7fe577fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3670cb0 0x55e1c366a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.737+0000 7fe577fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3670cb0 0x55e1c366a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43436/0 (socket says 172.21.15.139:43436) 2024-09-16T01:58:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.737+0000 7fe577fff640 1 -- 172.21.15.139:0/2064222382 learned_addr learned my addr 172.21.15.139:0/2064222382 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.737+0000 7fe576ffd640 1 --2- 172.21.15.139:0/2064222382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3666410 0x55e1c3669ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.738+0000 7fe577fff640 1 -- 172.21.15.139:0/2064222382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3666410 msgr2=0x55e1c3669ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.738+0000 7fe577fff640 1 --2- 172.21.15.139:0/2064222382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3666410 0x55e1c3669ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.738+0000 7fe577fff640 1 -- 172.21.15.139:0/2064222382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3665af0 msgr2=0x55e1c366b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.738+0000 7fe577fff640 1 --2- 172.21.15.139:0/2064222382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3665af0 0x55e1c366b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.738+0000 7fe577fff640 1 -- 172.21.15.139:0/2064222382 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c366a850 con 0x55e1c3670cb0 2024-09-16T01:58:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.739+0000 7fe577fff640 1 --2- 172.21.15.139:0/2064222382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3670cb0 0x55e1c366a1e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe5700bbc70 tx=0x7fe57000af00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f30b9582aedecaa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.739+0000 7fe556ffd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:58:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.739+0000 7fe556ffd640 1 -- 172.21.15.139:0/2064222382 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5700d6060 con 0x55e1c3670cb0 2024-09-16T01:58:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.739+0000 7fe580cc2740 1 -- 172.21.15.139:0/2064222382 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe56802e920 con 0x55e1c3670cb0 2024-09-16T01:58:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.745+0000 7fe556ffd640 1 -- 172.21.15.139:0/2064222382 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7fe570004740 con 0x55e1c3670cb0 2024-09-16T01:58:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.745+0000 7fe556ffd640 1 -- 172.21.15.139:0/2064222382 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fe5700bb8d0 con 0x55e1c3670cb0 2024-09-16T01:58:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.746+0000 7fe580cc2740 10 client.26713 fetch_fsmap learned FSMap version 76 2024-09-16T01:58:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.746+0000 7fe580cc2740 1 -- 172.21.15.139:0/2064222382 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55e1c3678be0 con 0x55e1c3670cb0 2024-09-16T01:58:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.752+0000 7fe556ffd640 1 -- 172.21.15.139:0/2064222382 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56802e920 con 0x55e1c3670cb0 2024-09-16T01:58:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.753+0000 7fe556ffd640 1 -- 172.21.15.139:0/2064222382 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 76) ==== 8446+0+0 (secure 0 0 0) 0x7fe5700b85e0 con 0x55e1c3670cb0 2024-09-16T01:58:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.754+0000 7fe580cc2740 10 client.26713 fetch_fsmap finished waiting for FSMap version 76 2024-09-16T01:58:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.754+0000 7fe580cc2740 10 client.26713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:58:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.754+0000 7fe580cc2740 20 client.26713 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.754+0000 7fe580cc2740 1 --2- 172.21.15.139:0/2064222382 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e1c36794e0 0x55e1c36ede30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.754+0000 7fe580cc2740 4 client.26713 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:58:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.754+0000 7fe580cc2740 1 -- 172.21.15.139:0/2064222382 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1c35d1d70 con 0x55e1c36794e0 2024-09-16T01:58:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.757+0000 7fe5777fe640 1 --2- 172.21.15.139:0/2064222382 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e1c36794e0 0x55e1c36ede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.763+0000 7fe5777fe640 1 --2- 172.21.15.139:0/2064222382 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e1c36794e0 0x55e1c36ede30 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda4d109a89f5a9c server_cookie=ed391af2cb4fe8b6 in_seq=0 out_seq=0 2024-09-16T01:58:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.763+0000 7fe556ffd640 10 client.26713 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.935+0000 7fe556ffd640 1 -- 172.21.15.139:0/2064222382 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1c35d1d70 con 0x55e1c36794e0 2024-09-16T01:58:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.935+0000 7fe556ffd640 10 client.26713 handle_command_reply: tid=0 2024-09-16T01:58:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.940+0000 7fe580cc2740 1 client.26713 shutdown 2024-09-16T01:58:23.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.941+0000 7fe580cc2740 1 -- 172.21.15.139:0/2064222382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3670cb0 msgr2=0x55e1c366a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:23.942 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.941+0000 7fe580cc2740 1 --2- 172.21.15.139:0/2064222382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3670cb0 0x55e1c366a1e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe5700bbc70 tx=0x7fe57000af00 comp rx=0 tx=0).stop 2024-09-16T01:58:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.944+0000 7fe580cc2740 1 -- 172.21.15.139:0/2064222382 shutdown_connections 2024-09-16T01:58:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.944+0000 7fe580cc2740 1 --2- 172.21.15.139:0/2064222382 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55e1c36794e0 0x55e1c36ede30 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.944+0000 7fe580cc2740 1 --2- 172.21.15.139:0/2064222382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1c3670cb0 0x55e1c366a1e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.944+0000 7fe580cc2740 1 --2- 172.21.15.139:0/2064222382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1c3666410 0x55e1c3669ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.944+0000 7fe580cc2740 1 --2- 172.21.15.139:0/2064222382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1c3665af0 0x55e1c366b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.944+0000 7fe580cc2740 1 -- 172.21.15.139:0/2064222382 >> 172.21.15.139:0/2064222382 conn(0x55e1c3662060 msgr2=0x55e1c36611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.946+0000 7fe556ffd640 0 client.26713 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:23.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.954+0000 7fe580cc2740 1 -- 172.21.15.139:0/2064222382 shutdown_connections 2024-09-16T01:58:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.956+0000 7fe580cc2740 1 -- 172.21.15.139:0/2064222382 wait complete. 2024-09-16T01:58:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.956+0000 7fe580cc2740 20 client.26713 trim_cache size 0 max 16384 2024-09-16T01:58:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:58:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:58:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:58:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:58:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.959+0000 7fe57df2c640 1 -- 172.21.15.139:0/1694033645 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe5600a3690 msgr2=0x7fe5600a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.959+0000 7fe57df2c640 1 --2- 172.21.15.139:0/1694033645 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe5600a3690 0x7fe5600a5b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe578073990 tx=0x7fe568009580 comp rx=0 tx=0).stop 2024-09-16T01:58:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.959+0000 7fe57df2c640 1 -- 172.21.15.139:0/1694033645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5781246b0 msgr2=0x7fe578124b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.959+0000 7fe57df2c640 1 --2- 172.21.15.139:0/1694033645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5781246b0 0x7fe578124b60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe57002b9a0 tx=0x7fe57000a990 comp rx=0 tx=0).stop 2024-09-16T01:58:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.963+0000 7fe57df2c640 1 -- 172.21.15.139:0/1694033645 shutdown_connections 2024-09-16T01:58:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.963+0000 7fe57df2c640 1 --2- 172.21.15.139:0/1694033645 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe5600a3690 0x7fe5600a5b50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.963+0000 7fe57df2c640 1 --2- 172.21.15.139:0/1694033645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5781246b0 0x7fe578124b60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.963+0000 7fe57df2c640 1 --2- 172.21.15.139:0/1694033645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe57810ea10 0x7fe5781151c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.963+0000 7fe57df2c640 1 --2- 172.21.15.139:0/1694033645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe578109460 0x7fe578114c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.963+0000 7fe57df2c640 1 -- 172.21.15.139:0/1694033645 >> 172.21.15.139:0/1694033645 conn(0x7fe57806d210 msgr2=0x7fe578072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.968+0000 7fe57df2c640 1 -- 172.21.15.139:0/1694033645 shutdown_connections 2024-09-16T01:58:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:23.973+0000 7fe57df2c640 1 -- 172.21.15.139:0/1694033645 wait complete. 2024-09-16T01:58:24.087 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:58:24.519 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:58:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:58:24.313+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:58:24.519 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:58:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:58:24.313+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:58:24.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.807+0000 7faf102cb640 1 -- 172.21.15.139:0/1911368092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf08105560 msgr2=0x7faf0810d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:24.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.807+0000 7faf102cb640 1 --2- 172.21.15.139:0/1911368092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf08105560 0x7faf0810d0e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7faef800b920 tx=0x7faef802e360 comp rx=0 tx=0).stop 2024-09-16T01:58:24.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.808+0000 7faf102cb640 1 -- 172.21.15.139:0/1911368092 shutdown_connections 2024-09-16T01:58:24.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.808+0000 7faf102cb640 1 --2- 172.21.15.139:0/1911368092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf08105560 0x7faf0810d0e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:24.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.808+0000 7faf102cb640 1 --2- 172.21.15.139:0/1911368092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faf08100de0 0x7faf08104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:24.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.808+0000 7faf102cb640 1 --2- 172.21.15.139:0/1911368092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf08100430 0x7faf08100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:24.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.808+0000 7faf102cb640 1 -- 172.21.15.139:0/1911368092 >> 172.21.15.139:0/1911368092 conn(0x7faf08077230 msgr2=0x7faf080fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:24.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.811+0000 7faf102cb640 1 -- 172.21.15.139:0/1911368092 shutdown_connections 2024-09-16T01:58:24.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.811+0000 7faf102cb640 1 -- 172.21.15.139:0/1911368092 wait complete. 2024-09-16T01:58:24.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.812+0000 7faf102cb640 1 Processor -- start 2024-09-16T01:58:24.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.812+0000 7faf102cb640 1 -- start start 2024-09-16T01:58:24.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.812+0000 7faf102cb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf08100430 0x7faf08073970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:24.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.813+0000 7faf102cb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faf08100de0 0x7faf08073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:24.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.813+0000 7faf102cb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf08105560 0x7faf0806cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:24.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.813+0000 7faf102cb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faf0810fe30 con 0x7faf08105560 2024-09-16T01:58:24.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.813+0000 7faf102cb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf0810fcb0 con 0x7faf08100430 2024-09-16T01:58:24.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.813+0000 7faf102cb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7faf0810ffb0 con 0x7faf08100de0 2024-09-16T01:58:24.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.813+0000 7faf0e868640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf08105560 0x7faf0806cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:24.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.813+0000 7faf0e868640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf08105560 0x7faf0806cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38860/0 (socket says 172.21.15.139:38860) 2024-09-16T01:58:24.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.813+0000 7faf0e868640 1 -- 172.21.15.139:0/1454506506 learned_addr learned my addr 172.21.15.139:0/1454506506 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:24.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.813+0000 7faf0d866640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faf08100de0 0x7faf08073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:24.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.813+0000 7faf0e067640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf08100430 0x7faf08073970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:24.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.818+0000 7faf0d866640 1 -- 172.21.15.139:0/1454506506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf08100430 msgr2=0x7faf08073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:24.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.818+0000 7faf0d866640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf08100430 0x7faf08073970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:24.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.818+0000 7faf0d866640 1 -- 172.21.15.139:0/1454506506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf08105560 msgr2=0x7faf0806cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:24.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.818+0000 7faf0d866640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf08105560 0x7faf0806cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:24.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.818+0000 7faf0d866640 1 -- 172.21.15.139:0/1454506506 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0806d470 con 0x7faf08100de0 2024-09-16T01:58:24.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.823+0000 7faf0e067640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf08100430 0x7faf08073970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:58:24.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.825+0000 7faf0d866640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faf08100de0 0x7faf08073eb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf0400b3f0 tx=0x7faf0400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:24.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.826+0000 7faeff7fe640 1 -- 172.21.15.139:0/1454506506 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04015020 con 0x7faf08100de0 2024-09-16T01:58:24.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.826+0000 7faf102cb640 1 -- 172.21.15.139:0/1454506506 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0806d760 con 0x7faf08100de0 2024-09-16T01:58:24.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.826+0000 7faf102cb640 1 -- 172.21.15.139:0/1454506506 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf08071730 con 0x7faf08100de0 2024-09-16T01:58:24.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.827+0000 7faeff7fe640 1 -- 172.21.15.139:0/1454506506 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf04011d10 con 0x7faf08100de0 2024-09-16T01:58:24.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.827+0000 7faeff7fe640 1 -- 172.21.15.139:0/1454506506 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0400f2e0 con 0x7faf08100de0 2024-09-16T01:58:24.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.832+0000 7faf102cb640 1 -- 172.21.15.139:0/1454506506 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeec005290 con 0x7faf08100de0 2024-09-16T01:58:24.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.833+0000 7faeff7fe640 1 -- 172.21.15.139:0/1454506506 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7faf04018020 con 0x7faf08100de0 2024-09-16T01:58:24.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.834+0000 7faeff7fe640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faef40a35c0 0x7faef40a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:24.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.834+0000 7faeff7fe640 1 -- 172.21.15.139:0/1454506506 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7faf040cf800 con 0x7faf08100de0 2024-09-16T01:58:24.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.838+0000 7faf0e067640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faef40a35c0 0x7faef40a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:24.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.846+0000 7faeff7fe640 1 -- 172.21.15.139:0/1454506506 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf0401f020 con 0x7faf08100de0 2024-09-16T01:58:24.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:24.861+0000 7faf0e067640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faef40a35c0 0x7faef40a5a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf00009fa0 tx=0x7faf00009570 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:25.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:25.260+0000 7faf102cb640 1 -- 172.21.15.139:0/1454506506 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faeec005850 con 0x7faf08100de0 2024-09-16T01:58:25.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:25.263+0000 7faeff7fe640 1 -- 172.21.15.139:0/1454506506 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+12031 (secure 0 0 0) 0x7faf0409bd60 con 0x7faf08100de0 2024-09-16T01:58:25.265 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:58:25.265 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":77,"btime":"2024-09-16T01:58:24:183207+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":77,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:58:24.058642+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:58:25.265 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 77 2024-09-16T01:58:25.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:25.271+0000 7faf102cb640 1 -- 172.21.15.139:0/1454506506 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faef40a35c0 msgr2=0x7faef40a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:25.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:25.271+0000 7faf102cb640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faef40a35c0 0x7faef40a5a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf00009fa0 tx=0x7faf00009570 comp rx=0 tx=0).stop 2024-09-16T01:58:25.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:25.271+0000 7faf102cb640 1 -- 172.21.15.139:0/1454506506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faf08100de0 msgr2=0x7faf08073eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:25.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:25.271+0000 7faf102cb640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faf08100de0 0x7faf08073eb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf0400b3f0 tx=0x7faf0400b8b0 comp rx=0 tx=0).stop 2024-09-16T01:58:25.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:25.274+0000 7faf102cb640 1 -- 172.21.15.139:0/1454506506 shutdown_connections 2024-09-16T01:58:25.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:25.274+0000 7faf102cb640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7faef40a35c0 0x7faef40a5a80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:25.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:25.274+0000 7faf102cb640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf08105560 0x7faf0806cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:25.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:25.274+0000 7faf102cb640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faf08100de0 0x7faf08073eb0 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf0400b3f0 tx=0x7faf0400b8b0 comp rx=0 tx=0).stop 2024-09-16T01:58:25.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:25.274+0000 7faf102cb640 1 --2- 172.21.15.139:0/1454506506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf08100430 0x7faf08073970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:25.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:25.274+0000 7faf102cb640 1 -- 172.21.15.139:0/1454506506 >> 172.21.15.139:0/1454506506 conn(0x7faf08077230 msgr2=0x7faf08103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:25.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:25.277+0000 7faf102cb640 1 -- 172.21.15.139:0/1454506506 shutdown_connections 2024-09-16T01:58:25.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:25.277+0000 7faf102cb640 1 -- 172.21.15.139:0/1454506506 wait complete. 2024-09-16T01:58:25.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:25 smithi164 ceph-mon[35121]: pgmap v573: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 65 KiB/s wr, 149 op/s 2024-09-16T01:58:25.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:25 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:58:25.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:25 smithi139 ceph-mon[31175]: pgmap v573: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 65 KiB/s wr, 149 op/s 2024-09-16T01:58:25.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:25 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:58:25.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:25 smithi176 ceph-mon[35324]: pgmap v573: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 65 KiB/s wr, 149 op/s 2024-09-16T01:58:25.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:25 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:58:26.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:26 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1454506506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:58:26.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:26 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1454506506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:58:26.570 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 142.95 --await-for 45.0' 2024-09-16T01:58:26.571 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 142.95 --await-for 45.0 2024-09-16T01:58:26.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:26 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1454506506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:58:27.145 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:58:27.502 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:27 smithi139 ceph-mon[31175]: pgmap v574: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 74 KiB/s wr, 132 op/s 2024-09-16T01:58:27.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:27 smithi164 ceph-mon[35121]: pgmap v574: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 74 KiB/s wr, 132 op/s 2024-09-16T01:58:27.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:27 smithi176 ceph-mon[35324]: pgmap v574: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 74 KiB/s wr, 132 op/s 2024-09-16T01:58:27.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.858+0000 7f5cb3121640 1 -- 172.21.15.139:0/997534467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5cac100980 msgr2=0x7f5cac100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:27.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.858+0000 7f5cb3121640 1 --2- 172.21.15.139:0/997534467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5cac100980 0x7f5cac100d60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009900 tx=0x7f5c9c02a490 comp rx=0 tx=0).stop 2024-09-16T01:58:27.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.859+0000 7f5cb3121640 1 -- 172.21.15.139:0/997534467 shutdown_connections 2024-09-16T01:58:27.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.859+0000 7f5cb3121640 1 --2- 172.21.15.139:0/997534467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cac105d90 0x7f5cac106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.859+0000 7f5cb3121640 1 --2- 172.21.15.139:0/997534467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac1012a0 0x7f5cac105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.859+0000 7f5cb3121640 1 --2- 172.21.15.139:0/997534467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5cac100980 0x7f5cac100d60 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.859+0000 7f5cb3121640 1 -- 172.21.15.139:0/997534467 >> 172.21.15.139:0/997534467 conn(0x7f5cac0fc610 msgr2=0x7f5cac0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:27.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.862+0000 7f5cb3121640 1 -- 172.21.15.139:0/997534467 shutdown_connections 2024-09-16T01:58:27.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.863+0000 7f5cb3121640 1 -- 172.21.15.139:0/997534467 wait complete. 2024-09-16T01:58:27.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.864+0000 7f5cb3121640 1 Processor -- start 2024-09-16T01:58:27.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.864+0000 7f5cb3121640 1 -- start start 2024-09-16T01:58:27.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.864+0000 7f5cb3121640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5cac100980 0x7f5cac073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:27.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.864+0000 7f5cb3121640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac1012a0 0x7f5cac06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:27.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.864+0000 7f5cb3121640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cac105d90 0x7f5cac06c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:27.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.864+0000 7f5cb3121640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac1183f0 con 0x7f5cac100980 2024-09-16T01:58:27.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.864+0000 7f5cb3121640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac118270 con 0x7f5cac1012a0 2024-09-16T01:58:27.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.864+0000 7f5cb3121640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac118570 con 0x7f5cac105d90 2024-09-16T01:58:27.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.865+0000 7f5cb191e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac1012a0 0x7f5cac06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:27.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.865+0000 7f5cb191e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac1012a0 0x7f5cac06bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37188/0 (socket says 172.21.15.139:37188) 2024-09-16T01:58:27.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.865+0000 7f5cb191e640 1 -- 172.21.15.139:0/1775915510 learned_addr learned my addr 172.21.15.139:0/1775915510 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:27.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.865+0000 7f5cb211f640 1 --2- 172.21.15.139:0/1775915510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5cac100980 0x7f5cac073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:27.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.866+0000 7f5cb211f640 1 -- 172.21.15.139:0/1775915510 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cac105d90 msgr2=0x7f5cac06c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:27.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.866+0000 7f5cb211f640 1 --2- 172.21.15.139:0/1775915510 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cac105d90 0x7f5cac06c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.866+0000 7f5cb211f640 1 -- 172.21.15.139:0/1775915510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac1012a0 msgr2=0x7f5cac06bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:27.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.866+0000 7f5cb211f640 1 --2- 172.21.15.139:0/1775915510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac1012a0 0x7f5cac06bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.866+0000 7f5cb211f640 1 -- 172.21.15.139:0/1775915510 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac06ca40 con 0x7f5cac100980 2024-09-16T01:58:27.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.867+0000 7f5cb211f640 1 --2- 172.21.15.139:0/1775915510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5cac100980 0x7f5cac073d10 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c02aa80 tx=0x7f5c9c003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:27.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.869+0000 7f5ca37fe640 1 -- 172.21.15.139:0/1775915510 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c03f070 con 0x7f5cac100980 2024-09-16T01:58:27.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.869+0000 7f5ca37fe640 1 -- 172.21.15.139:0/1775915510 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c9c031c90 con 0x7f5cac100980 2024-09-16T01:58:27.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.869+0000 7f5cb3121640 1 -- 172.21.15.139:0/1775915510 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cac1b3620 con 0x7f5cac100980 2024-09-16T01:58:27.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.869+0000 7f5cb3121640 1 -- 172.21.15.139:0/1775915510 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cac1b3ad0 con 0x7f5cac100980 2024-09-16T01:58:27.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.870+0000 7f5ca37fe640 1 -- 172.21.15.139:0/1775915510 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c035030 con 0x7f5cac100980 2024-09-16T01:58:27.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.874+0000 7f5cb675c740 1 Processor -- start 2024-09-16T01:58:27.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.874+0000 7f5cb675c740 1 -- start start 2024-09-16T01:58:27.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.875+0000 7f5cb675c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1bfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:27.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.875+0000 7f5cb675c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1c0340 0x55cf3f1ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:27.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.875+0000 7f5cb675c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1cac00 0x55cf3f1ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:27.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.875+0000 7f5cb675c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cf3f124160 con 0x55cf3f1cac00 2024-09-16T01:58:27.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.875+0000 7f5cb675c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cf3f0cb720 con 0x55cf3f1c0340 2024-09-16T01:58:27.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.875+0000 7f5cb675c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55cf3f120120 con 0x55cf3f1bfa20 2024-09-16T01:58:27.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.876+0000 7f5cb211f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1c0340 0x55cf3f1ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:27.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.876+0000 7f5ca37fe640 1 -- 172.21.15.139:0/1775915510 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f5c9c042070 con 0x7f5cac100980 2024-09-16T01:58:27.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.877+0000 7f5cb211f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1c0340 0x55cf3f1ca6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:37202/0 (socket says 172.21.15.139:37202) 2024-09-16T01:58:27.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.877+0000 7f5cb211f640 1 -- 172.21.15.139:0/3178168355 learned_addr learned my addr 172.21.15.139:0/3178168355 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:27.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.877+0000 7f5ca37fe640 1 --2- 172.21.15.139:0/1775915510 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5c980a35c0 0x7f5c980a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:27.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.878+0000 7f5ca37fe640 1 -- 172.21.15.139:0/1775915510 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7f5c9c0f0830 con 0x7f5cac100980 2024-09-16T01:58:27.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.878+0000 7f5cb191e640 1 --2- 172.21.15.139:0/3178168355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1cac00 0x55cf3f1ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:27.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.879+0000 7f5cb211f640 1 -- 172.21.15.139:0/3178168355 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1bfa20 msgr2=0x55cf3f1bfe00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:58:27.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.879+0000 7f5cb211f640 1 --2- 172.21.15.139:0/3178168355 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1bfe00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.879+0000 7f5cb211f640 1 -- 172.21.15.139:0/3178168355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1cac00 msgr2=0x55cf3f1ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:27.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.879+0000 7f5cb211f640 1 --2- 172.21.15.139:0/3178168355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1cac00 0x55cf3f1ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.879+0000 7f5cb211f640 1 -- 172.21.15.139:0/3178168355 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf3ee95ba0 con 0x55cf3f1c0340 2024-09-16T01:58:27.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.880+0000 7f5cb2920640 1 --2- 172.21.15.139:0/1775915510 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5c980a35c0 0x7f5c980a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:27.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.885+0000 7f5cb211f640 1 --2- 172.21.15.139:0/3178168355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1c0340 0x55cf3f1ca6c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c02aa00 tx=0x7f5c9c04a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=327548879a0504b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:27.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.888+0000 7f5cb2920640 1 --2- 172.21.15.139:0/1775915510 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5c980a35c0 0x7f5c980a5a80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4005fd0 tx=0x7f5ca4009710 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:27.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.892+0000 7f5ca0dea640 1 -- 172.21.15.139:0/3178168355 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0bc350 con 0x55cf3f1c0340 2024-09-16T01:58:27.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.892+0000 7f5ca0dea640 1 -- 172.21.15.139:0/3178168355 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c9c0f3e50 con 0x55cf3f1c0340 2024-09-16T01:58:27.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.893+0000 7f5cb675c740 1 -- 172.21.15.139:0/3178168355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1c0340 msgr2=0x55cf3f1ca6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:27.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.893+0000 7f5cb675c740 1 --2- 172.21.15.139:0/3178168355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1c0340 0x55cf3f1ca6c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c02aa00 tx=0x7f5c9c04a000 comp rx=0 tx=0).stop 2024-09-16T01:58:27.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.897+0000 7f5cb675c740 1 -- 172.21.15.139:0/3178168355 shutdown_connections 2024-09-16T01:58:27.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.897+0000 7f5cb675c740 1 --2- 172.21.15.139:0/3178168355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1cac00 0x55cf3f1ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.897+0000 7f5cb675c740 1 --2- 172.21.15.139:0/3178168355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1c0340 0x55cf3f1ca6c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.897+0000 7f5cb675c740 1 --2- 172.21.15.139:0/3178168355 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1bfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.897+0000 7f5cb675c740 1 -- 172.21.15.139:0/3178168355 >> 172.21.15.139:0/3178168355 conn(0x55cf3f1bbfb0 msgr2=0x55cf3f1bc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:27.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.903+0000 7f5cb675c740 1 -- 172.21.15.139:0/3178168355 shutdown_connections 2024-09-16T01:58:27.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.904+0000 7f5cb675c740 1 -- 172.21.15.139:0/3178168355 wait complete. 2024-09-16T01:58:27.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.904+0000 7f5cb675c740 1 -- start start 2024-09-16T01:58:27.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.904+0000 7f5cb675c740 1 Processor -- start 2024-09-16T01:58:27.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.908+0000 7f5cb675c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:27.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.908+0000 7f5cb675c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1c0340 0x55cf3f1c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:27.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.908+0000 7f5cb675c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1cac00 0x55cf3f1c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:27.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.908+0000 7f5cb675c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cf3f1cd590 con 0x55cf3f1c0340 2024-09-16T01:58:27.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.908+0000 7f5cb675c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cf3f0ac900 con 0x55cf3f1bfa20 2024-09-16T01:58:27.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.908+0000 7f5cb675c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55cf3f1cd710 con 0x55cf3f1cac00 2024-09-16T01:58:27.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.908+0000 7f5cb675c740 10 client.0 _ll_register_callbacks cb 0x55cf3f0aa780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:27.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.909+0000 7f5cb2920640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1cac00 0x55cf3f1c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:27.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.909+0000 7f5cb2920640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1cac00 0x55cf3f1c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43498/0 (socket says 172.21.15.139:43498) 2024-09-16T01:58:27.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.909+0000 7f5cb2920640 1 -- 172.21.15.139:0/2917620890 learned_addr learned my addr 172.21.15.139:0/2917620890 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.909+0000 7f5cb2920640 1 -- 172.21.15.139:0/2917620890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1bfa20 msgr2=0x55cf3f1c7aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:58:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.909+0000 7f5cb2920640 1 --2- 172.21.15.139:0/2917620890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1c7aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.909+0000 7f5cb2920640 1 -- 172.21.15.139:0/2917620890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1c0340 msgr2=0x55cf3f1c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.909+0000 7f5cb2920640 1 --2- 172.21.15.139:0/2917620890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1c0340 0x55cf3f1c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.909+0000 7f5cb2920640 1 -- 172.21.15.139:0/2917620890 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf3f0ecc50 con 0x55cf3f1cac00 2024-09-16T01:58:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.910+0000 7f5cb2920640 1 --2- 172.21.15.139:0/2917620890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1cac00 0x55cf3f1c8520 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009910 tx=0x7f5ca4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=846ed6ccfff751a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:27.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.910+0000 7f5c95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:58:27.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.912+0000 7f5c95ffb640 1 -- 172.21.15.139:0/2917620890 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4022020 con 0x55cf3f1cac00 2024-09-16T01:58:27.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.912+0000 7f5cb675c740 1 -- 172.21.15.139:0/2917620890 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c9c0f3e50 con 0x55cf3f1cac00 2024-09-16T01:58:27.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.912+0000 7f5c95ffb640 1 -- 172.21.15.139:0/2917620890 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7f5ca4010950 con 0x55cf3f1cac00 2024-09-16T01:58:27.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.921+0000 7f5c95ffb640 1 -- 172.21.15.139:0/2917620890 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f5ca402d050 con 0x55cf3f1cac00 2024-09-16T01:58:27.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.925+0000 7f5cb675c740 10 client.26722 fetch_fsmap learned FSMap version 77 2024-09-16T01:58:27.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.925+0000 7f5cb675c740 1 -- 172.21.15.139:0/2917620890 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55cf3f1d92f0 con 0x55cf3f1cac00 2024-09-16T01:58:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.926+0000 7f5c95ffb640 1 -- 172.21.15.139:0/2917620890 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c9c0f3e50 con 0x55cf3f1cac00 2024-09-16T01:58:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.926+0000 7f5c95ffb640 1 -- 172.21.15.139:0/2917620890 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 77) ==== 8450+0+0 (secure 0 0 0) 0x7f5ca400f040 con 0x55cf3f1cac00 2024-09-16T01:58:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.926+0000 7f5cb675c740 10 client.26722 fetch_fsmap finished waiting for FSMap version 77 2024-09-16T01:58:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.926+0000 7f5cb675c740 10 client.26722 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:58:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.927+0000 7f5cb675c740 20 client.26722 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.927+0000 7f5cb675c740 1 --2- 172.21.15.139:0/2917620890 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cf3f1dad30 0x55cf3f1dd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.927+0000 7f5cb675c740 4 client.26722 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:58:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.927+0000 7f5cb675c740 1 -- 172.21.15.139:0/2917620890 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf3f131890 con 0x55cf3f1dad30 2024-09-16T01:58:27.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.928+0000 7f5cb211f640 1 --2- 172.21.15.139:0/2917620890 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cf3f1dad30 0x55cf3f1dd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:27.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.936+0000 7f5cb211f640 1 --2- 172.21.15.139:0/2917620890 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cf3f1dad30 0x55cf3f1dd1f0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1da0feb9f2e6020 server_cookie=b33a4ff5a5c8827f in_seq=0 out_seq=0 2024-09-16T01:58:27.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.936+0000 7f5c95ffb640 10 client.26722 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:27.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.940+0000 7f5c95ffb640 1 -- 172.21.15.139:0/2917620890 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf3f131890 con 0x55cf3f1dad30 2024-09-16T01:58:27.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.940+0000 7f5c95ffb640 10 client.26722 handle_command_reply: tid=0 2024-09-16T01:58:27.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.940+0000 7f5cb675c740 1 client.26722 shutdown 2024-09-16T01:58:27.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.945+0000 7f5cb675c740 1 -- 172.21.15.139:0/2917620890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1cac00 msgr2=0x55cf3f1c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:27.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.945+0000 7f5cb675c740 1 --2- 172.21.15.139:0/2917620890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1cac00 0x55cf3f1c8520 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009910 tx=0x7f5ca4004a40 comp rx=0 tx=0).stop 2024-09-16T01:58:27.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.946+0000 7f5cb675c740 1 -- 172.21.15.139:0/2917620890 shutdown_connections 2024-09-16T01:58:27.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.946+0000 7f5cb675c740 1 --2- 172.21.15.139:0/2917620890 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cf3f1dad30 0x55cf3f1dd1f0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.946+0000 7f5cb675c740 1 --2- 172.21.15.139:0/2917620890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1cac00 0x55cf3f1c8520 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.946+0000 7f5cb675c740 1 --2- 172.21.15.139:0/2917620890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1c0340 0x55cf3f1c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.946+0000 7f5cb675c740 1 --2- 172.21.15.139:0/2917620890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.946+0000 7f5cb675c740 1 -- 172.21.15.139:0/2917620890 >> 172.21.15.139:0/2917620890 conn(0x55cf3f1bbfb0 msgr2=0x55cf3f1c1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:27.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.946+0000 7f5c95ffb640 0 client.26722 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:27.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.948+0000 7f5cb675c740 1 -- 172.21.15.139:0/2917620890 shutdown_connections 2024-09-16T01:58:27.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.951+0000 7f5cb675c740 1 -- 172.21.15.139:0/2917620890 wait complete. 2024-09-16T01:58:27.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.951+0000 7f5cb675c740 20 client.26722 trim_cache size 0 max 16384 2024-09-16T01:58:27.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.984+0000 7f5cb675c740 1 Processor -- start 2024-09-16T01:58:27.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.984+0000 7f5cb675c740 1 -- start start 2024-09-16T01:58:27.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.984+0000 7f5cb675c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1c2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:27.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.984+0000 7f5cb675c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1c0340 0x55cf3f1e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:27.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.984+0000 7f5cb675c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1cac00 0x55cf3f1e3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:27.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.985+0000 7f5cb675c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5c900103f0 con 0x55cf3f1bfa20 2024-09-16T01:58:27.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.985+0000 7f5cb675c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90012230 con 0x55cf3f1cac00 2024-09-16T01:58:27.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.985+0000 7f5cb675c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90010b80 con 0x55cf3f1c0340 2024-09-16T01:58:27.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.985+0000 7f5cb191e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1c0340 0x55cf3f1e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:27.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.985+0000 7f5cb191e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1c0340 0x55cf3f1e2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43512/0 (socket says 172.21.15.139:43512) 2024-09-16T01:58:27.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.985+0000 7f5cb191e640 1 -- 172.21.15.139:0/3407126734 learned_addr learned my addr 172.21.15.139:0/3407126734 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:27.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.985+0000 7f5cb2920640 1 --2- 172.21.15.139:0/3407126734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1cac00 0x55cf3f1e3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:27.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.985+0000 7f5cb211f640 1 --2- 172.21.15.139:0/3407126734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:27.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.985+0000 7f5cb191e640 1 -- 172.21.15.139:0/3407126734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1cac00 msgr2=0x55cf3f1e3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:27.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.985+0000 7f5cb191e640 1 --2- 172.21.15.139:0/3407126734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1cac00 0x55cf3f1e3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.985+0000 7f5cb191e640 1 -- 172.21.15.139:0/3407126734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1bfa20 msgr2=0x55cf3f1c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:27.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.985+0000 7f5cb191e640 1 --2- 172.21.15.139:0/3407126734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1c2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:27.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.985+0000 7f5cb191e640 1 -- 172.21.15.139:0/3407126734 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf3f1c0170 con 0x55cf3f1c0340 2024-09-16T01:58:27.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.989+0000 7f5cb191e640 1 --2- 172.21.15.139:0/3407126734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1c0340 0x55cf3f1e2b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8005fd0 tx=0x7f5ca8003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14e3519e5c7a8dc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:27.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.996+0000 7f5c97fff640 1 -- 172.21.15.139:0/3407126734 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8011b90 con 0x55cf3f1c0340 2024-09-16T01:58:27.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.996+0000 7f5c97fff640 1 -- 172.21.15.139:0/3407126734 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ca8003e50 con 0x55cf3f1c0340 2024-09-16T01:58:27.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.996+0000 7f5c97fff640 1 -- 172.21.15.139:0/3407126734 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8012780 con 0x55cf3f1c0340 2024-09-16T01:58:28.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.998+0000 7f5cb675c740 1 -- 172.21.15.139:0/3407126734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1c0340 msgr2=0x55cf3f1e2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:28.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.998+0000 7f5cb675c740 1 --2- 172.21.15.139:0/3407126734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1c0340 0x55cf3f1e2b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8005fd0 tx=0x7f5ca8003c00 comp rx=0 tx=0).stop 2024-09-16T01:58:28.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.999+0000 7f5cb675c740 1 -- 172.21.15.139:0/3407126734 shutdown_connections 2024-09-16T01:58:28.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.999+0000 7f5cb675c740 1 --2- 172.21.15.139:0/3407126734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1cac00 0x55cf3f1e3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:28.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.999+0000 7f5cb675c740 1 --2- 172.21.15.139:0/3407126734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1c0340 0x55cf3f1e2b00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:28.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.999+0000 7f5cb675c740 1 --2- 172.21.15.139:0/3407126734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:28.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.999+0000 7f5cb675c740 1 -- 172.21.15.139:0/3407126734 >> 172.21.15.139:0/3407126734 conn(0x55cf3f1bbfb0 msgr2=0x55cf3f1c9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:28.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.999+0000 7f5cb675c740 1 -- 172.21.15.139:0/3407126734 shutdown_connections 2024-09-16T01:58:28.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:27.999+0000 7f5cb675c740 1 -- 172.21.15.139:0/3407126734 wait complete. 2024-09-16T01:58:28.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.000+0000 7f5cb675c740 1 -- start start 2024-09-16T01:58:28.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.000+0000 7f5cb675c740 1 Processor -- start 2024-09-16T01:58:28.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.001+0000 7f5cb675c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:28.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.001+0000 7f5cb675c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1c0340 0x55cf3f1cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:28.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.001+0000 7f5cb675c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1cac00 0x55cf3f1cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:28.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.001+0000 7f5cb211f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:28.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.001+0000 7f5cb211f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:43516/0 (socket says 172.21.15.139:43516) 2024-09-16T01:58:28.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.001+0000 7f5cb211f640 1 -- 172.21.15.139:0/2870252143 learned_addr learned my addr 172.21.15.139:0/2870252143 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:28.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.001+0000 7f5cb675c740 1 -- 172.21.15.139:0/2870252143 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90011aa0 con 0x55cf3f1c0340 2024-09-16T01:58:28.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.001+0000 7f5cb675c740 1 -- 172.21.15.139:0/2870252143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90011310 con 0x55cf3f1cac00 2024-09-16T01:58:28.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.002+0000 7f5cb2920640 1 --2- 172.21.15.139:0/2870252143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1cac00 0x55cf3f1cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:28.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.002+0000 7f5cb675c740 1 -- 172.21.15.139:0/2870252143 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9000e3f0 con 0x55cf3f1bfa20 2024-09-16T01:58:28.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.002+0000 7f5cb675c740 10 client.0 _ll_register_callbacks cb 0x7f5cac074370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:28.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.002+0000 7f5cb211f640 1 -- 172.21.15.139:0/2870252143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1cac00 msgr2=0x55cf3f1cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:28.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.002+0000 7f5cb211f640 1 --2- 172.21.15.139:0/2870252143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1cac00 0x55cf3f1cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:28.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.002+0000 7f5cb211f640 1 -- 172.21.15.139:0/2870252143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1c0340 msgr2=0x55cf3f1cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:28.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.002+0000 7f5cb211f640 1 --2- 172.21.15.139:0/2870252143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1c0340 0x55cf3f1cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:28.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.002+0000 7f5cb211f640 1 -- 172.21.15.139:0/2870252143 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf3f244ad0 con 0x55cf3f1bfa20 2024-09-16T01:58:28.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.003+0000 7f5cb211f640 1 --2- 172.21.15.139:0/2870252143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1ce6e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c006010 tx=0x7f5c9c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8848eb1bb5655fc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:28.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.005+0000 7f5c95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:58:28.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.005+0000 7f5c95ffb640 1 -- 172.21.15.139:0/2870252143 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0da060 con 0x55cf3f1bfa20 2024-09-16T01:58:28.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.005+0000 7f5c95ffb640 1 -- 172.21.15.139:0/2870252143 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7f5c9c0edc70 con 0x55cf3f1bfa20 2024-09-16T01:58:28.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.006+0000 7f5cb675c740 1 -- 172.21.15.139:0/2870252143 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ca8003e50 con 0x55cf3f1bfa20 2024-09-16T01:58:28.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.007+0000 7f5c95ffb640 1 -- 172.21.15.139:0/2870252143 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f5c9c0ca050 con 0x55cf3f1bfa20 2024-09-16T01:58:28.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.008+0000 7f5cb675c740 10 client.26728 fetch_fsmap learned FSMap version 77 2024-09-16T01:58:28.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.008+0000 7f5cb675c740 1 -- 172.21.15.139:0/2870252143 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55cf3f245d50 con 0x55cf3f1bfa20 2024-09-16T01:58:28.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.011+0000 7f5c95ffb640 1 -- 172.21.15.139:0/2870252143 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c9c0f3e50 con 0x55cf3f1bfa20 2024-09-16T01:58:28.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.011+0000 7f5c95ffb640 1 -- 172.21.15.139:0/2870252143 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 77) ==== 8450+0+0 (secure 0 0 0) 0x7f5c9c0c4d60 con 0x55cf3f1bfa20 2024-09-16T01:58:28.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.012+0000 7f5cb675c740 10 client.26728 fetch_fsmap finished waiting for FSMap version 77 2024-09-16T01:58:28.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.012+0000 7f5cb675c740 10 client.26728 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:58:28.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.013+0000 7f5cb675c740 20 client.26728 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:28.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.013+0000 7f5cb675c740 1 --2- 172.21.15.139:0/2870252143 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cf3f246360 0x55cf3f2487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:28.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.013+0000 7f5cb675c740 4 client.26728 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.95, "await_for": 45.0}] 2024-09-16T01:58:28.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.013+0000 7f5cb675c740 1 -- 172.21.15.139:0/2870252143 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.95, "await_for": 45.0}) -- 0x7f5ca8003e50 con 0x55cf3f246360 2024-09-16T01:58:28.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.014+0000 7f5cb191e640 1 --2- 172.21.15.139:0/2870252143 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cf3f246360 0x55cf3f2487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:28.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.025+0000 7f5cb191e640 1 --2- 172.21.15.139:0/2870252143 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cf3f246360 0x55cf3f2487c0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64589b36590b2ec8 server_cookie=27060ef561685d25 in_seq=0 out_seq=0 2024-09-16T01:58:28.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.025+0000 7f5c95ffb640 10 client.26728 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:28.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.061+0000 7f5c95ffb640 1 -- 172.21.15.139:0/2870252143 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f5ca8003e50 con 0x55cf3f246360 2024-09-16T01:58:28.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.061+0000 7f5c95ffb640 10 client.26728 handle_command_reply: tid=0 2024-09-16T01:58:28.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.061+0000 7f5cb675c740 1 client.26728 shutdown 2024-09-16T01:58:28.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.061+0000 7f5cb675c740 1 -- 172.21.15.139:0/2870252143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1bfa20 msgr2=0x55cf3f1ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:28.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.061+0000 7f5cb675c740 1 --2- 172.21.15.139:0/2870252143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1ce6e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c006010 tx=0x7f5c9c0045e0 comp rx=0 tx=0).stop 2024-09-16T01:58:28.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.061+0000 7f5cb675c740 1 -- 172.21.15.139:0/2870252143 shutdown_connections 2024-09-16T01:58:28.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.061+0000 7f5cb675c740 1 --2- 172.21.15.139:0/2870252143 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55cf3f246360 0x55cf3f2487c0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:28.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.062+0000 7f5cb675c740 1 --2- 172.21.15.139:0/2870252143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf3f1cac00 0x55cf3f1cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:28.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.062+0000 7f5c95ffb640 0 client.26728 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:28.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.062+0000 7f5cb675c740 1 --2- 172.21.15.139:0/2870252143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf3f1c0340 0x55cf3f1cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:28.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.062+0000 7f5cb675c740 1 --2- 172.21.15.139:0/2870252143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf3f1bfa20 0x55cf3f1ce6e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:28.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.062+0000 7f5cb675c740 1 -- 172.21.15.139:0/2870252143 >> 172.21.15.139:0/2870252143 conn(0x55cf3f1bbfb0 msgr2=0x55cf3f1c7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:28.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.062+0000 7f5cb675c740 1 -- 172.21.15.139:0/2870252143 shutdown_connections 2024-09-16T01:58:28.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.062+0000 7f5cb675c740 1 -- 172.21.15.139:0/2870252143 wait complete. 2024-09-16T01:58:28.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.062+0000 7f5cb675c740 20 client.26728 trim_cache size 0 max 16384 2024-09-16T01:58:28.065 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:58:28.065 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 77, 2024-09-16T01:58:28.065 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:58:28.065 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 66, 2024-09-16T01:58:28.065 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:58:28.065 INFO:teuthology.orchestra.run.smithi139.stdout: "72a1fc77": { 2024-09-16T01:58:28.065 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 66, 2024-09-16T01:58:28.065 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:58:28.065 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:58:28.065 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:58:28.066 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:58:28.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:58:28.066 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:58:28.066 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 142.9, 2024-09-16T01:58:28.066 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:58:28.066 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:58:28.066 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:58:28.066 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:58:28.066 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "QUIESCED", 2024-09-16T01:58:28.066 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:58:28.066 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:28.066 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:28.066 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:28.066 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:28.067 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:28.067 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:58:28.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.063+0000 7f5cb3121640 1 -- 172.21.15.139:0/1775915510 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5c980a35c0 msgr2=0x7f5c980a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:28.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.063+0000 7f5cb3121640 1 --2- 172.21.15.139:0/1775915510 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5c980a35c0 0x7f5c980a5a80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4005fd0 tx=0x7f5ca4009710 comp rx=0 tx=0).stop 2024-09-16T01:58:28.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.063+0000 7f5cb3121640 1 -- 172.21.15.139:0/1775915510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5cac100980 msgr2=0x7f5cac073d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:28.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.063+0000 7f5cb3121640 1 --2- 172.21.15.139:0/1775915510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5cac100980 0x7f5cac073d10 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c02aa80 tx=0x7f5c9c003b70 comp rx=0 tx=0).stop 2024-09-16T01:58:28.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.063+0000 7f5cb3121640 1 -- 172.21.15.139:0/1775915510 shutdown_connections 2024-09-16T01:58:28.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.063+0000 7f5cb3121640 1 --2- 172.21.15.139:0/1775915510 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f5c980a35c0 0x7f5c980a5a80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:28.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.063+0000 7f5cb3121640 1 --2- 172.21.15.139:0/1775915510 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cac105d90 0x7f5cac06c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:28.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.063+0000 7f5cb3121640 1 --2- 172.21.15.139:0/1775915510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac1012a0 0x7f5cac06bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:28.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.063+0000 7f5cb3121640 1 --2- 172.21.15.139:0/1775915510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5cac100980 0x7f5cac073d10 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:28.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.063+0000 7f5cb3121640 1 -- 172.21.15.139:0/1775915510 >> 172.21.15.139:0/1775915510 conn(0x7f5cac0fc610 msgr2=0x7f5cac103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:28.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.064+0000 7f5cb3121640 1 -- 172.21.15.139:0/1775915510 shutdown_connections 2024-09-16T01:58:28.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:28.064+0000 7f5cb3121640 1 -- 172.21.15.139:0/1775915510 wait complete. 2024-09-16T01:58:28.849 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '72a1fc77', quiesce took 5.3 seconds. Will release after: 17.65 2024-09-16T01:58:29.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:29 smithi164 ceph-mon[35121]: pgmap v575: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 290 KiB/s wr, 158 op/s 2024-09-16T01:58:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:29 smithi139 ceph-mon[31175]: pgmap v575: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 290 KiB/s wr, 158 op/s 2024-09-16T01:58:29.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:29 smithi176 ceph-mon[35324]: pgmap v575: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 290 KiB/s wr, 158 op/s 2024-09-16T01:58:31.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:31 smithi164 ceph-mon[35121]: pgmap v576: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 413 KiB/s wr, 156 op/s 2024-09-16T01:58:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:31 smithi139 ceph-mon[31175]: pgmap v576: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 413 KiB/s wr, 156 op/s 2024-09-16T01:58:31.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:31 smithi176 ceph-mon[35324]: pgmap v576: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 413 KiB/s wr, 156 op/s 2024-09-16T01:58:33.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:33 smithi164 ceph-mon[35121]: pgmap v577: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 413 KiB/s wr, 137 op/s 2024-09-16T01:58:33.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:33 smithi139 ceph-mon[31175]: pgmap v577: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 413 KiB/s wr, 137 op/s 2024-09-16T01:58:33.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:33 smithi176 ceph-mon[35324]: pgmap v577: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 413 KiB/s wr, 137 op/s 2024-09-16T01:58:35.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:35 smithi164 ceph-mon[35121]: pgmap v578: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 534 KiB/s wr, 153 op/s 2024-09-16T01:58:35.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:35 smithi139 ceph-mon[31175]: pgmap v578: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 534 KiB/s wr, 153 op/s 2024-09-16T01:58:35.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:35 smithi176 ceph-mon[35324]: pgmap v578: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 534 KiB/s wr, 153 op/s 2024-09-16T01:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.147+0000 7f84c26f1640 1 -- 172.21.15.139:0/964859701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84bc1095d0 msgr2=0x7f84bc109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:37.150 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.147+0000 7f84c26f1640 1 --2- 172.21.15.139:0/964859701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84bc1095d0 0x7f84bc109a10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009ca0 tx=0x7f84ac02a8e0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.149+0000 7f84c26f1640 1 -- 172.21.15.139:0/964859701 shutdown_connections 2024-09-16T01:58:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.149+0000 7f84c26f1640 1 --2- 172.21.15.139:0/964859701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84bc11b140 0x7f84bc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.149+0000 7f84c26f1640 1 --2- 172.21.15.139:0/964859701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84bc1095d0 0x7f84bc109a10 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.149+0000 7f84c26f1640 1 --2- 172.21.15.139:0/964859701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84bc1091f0 0x7f84bc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.149+0000 7f84c26f1640 1 -- 172.21.15.139:0/964859701 >> 172.21.15.139:0/964859701 conn(0x7f84bc06ce00 msgr2=0x7f84bc06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.160+0000 7f84c26f1640 1 -- 172.21.15.139:0/964859701 shutdown_connections 2024-09-16T01:58:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.160+0000 7f84c26f1640 1 -- 172.21.15.139:0/964859701 wait complete. 2024-09-16T01:58:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.161+0000 7f84c26f1640 1 Processor -- start 2024-09-16T01:58:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.165+0000 7f84c26f1640 1 -- start start 2024-09-16T01:58:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.165+0000 7f84c26f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84bc1091f0 0x7f84bc1a7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.165+0000 7f84c26f1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84bc1095d0 0x7f84bc1a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.165+0000 7f84c26f1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84bc11b140 0x7f84bc1aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.165+0000 7f84c26f1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc120cf0 con 0x7f84bc11b140 2024-09-16T01:58:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.165+0000 7f84c26f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc120b70 con 0x7f84bc1091f0 2024-09-16T01:58:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.165+0000 7f84c26f1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc120e70 con 0x7f84bc1095d0 2024-09-16T01:58:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.166+0000 7f84bb7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84bc1095d0 0x7f84bc1a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.167+0000 7f84bb7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84bc1095d0 0x7f84bc1a76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:60084/0 (socket says 172.21.15.139:60084) 2024-09-16T01:58:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.167+0000 7f84bb7fe640 1 -- 172.21.15.139:0/2008863325 learned_addr learned my addr 172.21.15.139:0/2008863325 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.168+0000 7f84bb7fe640 1 -- 172.21.15.139:0/2008863325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84bc1091f0 msgr2=0x7f84bc1a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.168+0000 7f84bb7fe640 1 --2- 172.21.15.139:0/2008863325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84bc1091f0 0x7f84bc1a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.168+0000 7f84bb7fe640 1 -- 172.21.15.139:0/2008863325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84bc11b140 msgr2=0x7f84bc1aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.168+0000 7f84bb7fe640 1 --2- 172.21.15.139:0/2008863325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84bc11b140 0x7f84bc1aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.168+0000 7f84bb7fe640 1 -- 172.21.15.139:0/2008863325 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84bc1ac120 con 0x7f84bc1095d0 2024-09-16T01:58:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.169+0000 7f84bb7fe640 1 --2- 172.21.15.139:0/2008863325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84bc1095d0 0x7f84bc1a76a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f84ac032fe0 tx=0x7f84ac003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.172+0000 7f84b97fa640 1 -- 172.21.15.139:0/2008863325 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac003d10 con 0x7f84bc1095d0 2024-09-16T01:58:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.172+0000 7f84b97fa640 1 -- 172.21.15.139:0/2008863325 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84ac004520 con 0x7f84bc1095d0 2024-09-16T01:58:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.172+0000 7f84b97fa640 1 -- 172.21.15.139:0/2008863325 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac030d90 con 0x7f84bc1095d0 2024-09-16T01:58:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.172+0000 7f84c26f1640 1 -- 172.21.15.139:0/2008863325 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84bc1ac3b0 con 0x7f84bc1095d0 2024-09-16T01:58:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.172+0000 7f84c26f1640 1 -- 172.21.15.139:0/2008863325 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84bc110920 con 0x7f84bc1095d0 2024-09-16T01:58:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.177+0000 7f84b97fa640 1 -- 172.21.15.139:0/2008863325 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f84ac004040 con 0x7f84bc1095d0 2024-09-16T01:58:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.178+0000 7f84b97fa640 1 --2- 172.21.15.139:0/2008863325 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f84a80a3690 0x7f84a80a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.178+0000 7f84b97fa640 1 -- 172.21.15.139:0/2008863325 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7f84ac0f15e0 con 0x7f84bc1095d0 2024-09-16T01:58:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.181+0000 7f84c5493740 1 Processor -- start 2024-09-16T01:58:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.183+0000 7f84bbfff640 1 --2- 172.21.15.139:0/2008863325 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f84a80a3690 0x7f84a80a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.184+0000 7f84c5493740 1 -- start start 2024-09-16T01:58:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.184+0000 7f84c5493740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafcaf0 0x5559ffafced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.184+0000 7f84c5493740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffafd410 0x5559ffb07770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.184+0000 7f84c5493740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffb07cb0 0x5559ffb0a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.184+0000 7f84c5493740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5559ffa612f0 con 0x5559ffafd410 2024-09-16T01:58:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.184+0000 7f84c5493740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5559ffa09270 con 0x5559ffb07cb0 2024-09-16T01:58:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.184+0000 7f84c5493740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5559ffa5d2b0 con 0x5559ffafcaf0 2024-09-16T01:58:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.187+0000 7f84bbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafcaf0 0x5559ffafced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.187+0000 7f84bbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafcaf0 0x5559ffafced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:60094/0 (socket says 172.21.15.139:60094) 2024-09-16T01:58:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.187+0000 7f84bbfff640 1 -- 172.21.15.139:0/2487771460 learned_addr learned my addr 172.21.15.139:0/2487771460 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.188+0000 7f84bb7fe640 1 --2- 172.21.15.139:0/2487771460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffafd410 0x5559ffb07770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.188+0000 7f84bb7fe640 1 -- 172.21.15.139:0/2487771460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafcaf0 msgr2=0x5559ffafced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.188+0000 7f84bb7fe640 1 --2- 172.21.15.139:0/2487771460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafcaf0 0x5559ffafced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.188+0000 7f84bb7fe640 1 -- 172.21.15.139:0/2487771460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffb07cb0 msgr2=0x5559ffb0a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.188+0000 7f84bb7fe640 1 --2- 172.21.15.139:0/2487771460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffb07cb0 0x5559ffb0a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.188+0000 7f84bb7fe640 1 -- 172.21.15.139:0/2487771460 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ffa08340 con 0x5559ffafd410 2024-09-16T01:58:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.189+0000 7f84bb7fe640 1 --2- 172.21.15.139:0/2487771460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffafd410 0x5559ffb07770 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0ee8b0 tx=0x7f84ac02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba279dde156a1d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.189+0000 7f84b27fc640 1 -- 172.21.15.139:0/2487771460 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac0bdb40 con 0x5559ffafd410 2024-09-16T01:58:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.189+0000 7f84b27fc640 1 -- 172.21.15.139:0/2487771460 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84ac036030 con 0x5559ffafd410 2024-09-16T01:58:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.190+0000 7f84c5493740 1 -- 172.21.15.139:0/2487771460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffafd410 msgr2=0x5559ffb07770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.190+0000 7f84c5493740 1 --2- 172.21.15.139:0/2487771460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffafd410 0x5559ffb07770 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0ee8b0 tx=0x7f84ac02a8e0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.192+0000 7f84c5493740 1 -- 172.21.15.139:0/2487771460 shutdown_connections 2024-09-16T01:58:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.192+0000 7f84c5493740 1 --2- 172.21.15.139:0/2487771460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffb07cb0 0x5559ffb0a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.192+0000 7f84c5493740 1 --2- 172.21.15.139:0/2487771460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffafd410 0x5559ffb07770 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.192+0000 7f84c5493740 1 --2- 172.21.15.139:0/2487771460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafcaf0 0x5559ffafced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.192+0000 7f84c5493740 1 -- 172.21.15.139:0/2487771460 >> 172.21.15.139:0/2487771460 conn(0x5559ffaf9060 msgr2=0x5559ffaf9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.197+0000 7f84bbfff640 1 --2- 172.21.15.139:0/2008863325 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f84a80a3690 0x7f84a80a5b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f84b40058c0 tx=0x7f84b40123f0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.197+0000 7f84c5493740 1 -- 172.21.15.139:0/2487771460 shutdown_connections 2024-09-16T01:58:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.198+0000 7f84c5493740 1 -- 172.21.15.139:0/2487771460 wait complete. 2024-09-16T01:58:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.198+0000 7f84c5493740 1 -- start start 2024-09-16T01:58:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.199+0000 7f84c5493740 1 Processor -- start 2024-09-16T01:58:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.203+0000 7f84c5493740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafcaf0 0x5559ffb04af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.203+0000 7f84c5493740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafd410 0x5559ffb05030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.203+0000 7f84c5493740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffb07cb0 0x5559ffb05570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.203+0000 7f84c5493740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5559ffb0a640 con 0x5559ffb07cb0 2024-09-16T01:58:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.203+0000 7f84c5493740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5559ff9ea290 con 0x5559ffafcaf0 2024-09-16T01:58:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.203+0000 7f84c5493740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5559ffb0a7c0 con 0x5559ffafd410 2024-09-16T01:58:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.203+0000 7f84c5493740 10 client.0 _ll_register_callbacks cb 0x5559ffa1b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.204+0000 7f84bbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafcaf0 0x5559ffb04af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.204+0000 7f84c0c8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffb07cb0 0x5559ffb05570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.204+0000 7f84bb7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafd410 0x5559ffb05030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.204+0000 7f84c0c8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffb07cb0 0x5559ffb05570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58410/0 (socket says 172.21.15.139:58410) 2024-09-16T01:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.204+0000 7f84c0c8e640 1 -- 172.21.15.139:0/530059919 learned_addr learned my addr 172.21.15.139:0/530059919 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.206+0000 7f84bb7fe640 1 -- 172.21.15.139:0/530059919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafcaf0 msgr2=0x5559ffb04af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.206+0000 7f84bb7fe640 1 --2- 172.21.15.139:0/530059919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafcaf0 0x5559ffb04af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.206+0000 7f84bb7fe640 1 -- 172.21.15.139:0/530059919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffb07cb0 msgr2=0x5559ffb05570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.206+0000 7f84bb7fe640 1 --2- 172.21.15.139:0/530059919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffb07cb0 0x5559ffb05570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.206+0000 7f84bb7fe640 1 -- 172.21.15.139:0/530059919 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559ff774210 con 0x5559ffafd410 2024-09-16T01:58:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.208+0000 7f84bbfff640 1 --2- 172.21.15.139:0/530059919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafcaf0 0x5559ffb04af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:58:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.208+0000 7f84bb7fe640 1 --2- 172.21.15.139:0/530059919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafd410 0x5559ffb05030 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x5559ffafd9b0 tx=0x7f84ac031000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef1b43861172de22 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.208+0000 7f849b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:58:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.213+0000 7f849b7fe640 1 -- 172.21.15.139:0/530059919 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac0da070 con 0x5559ffafd410 2024-09-16T01:58:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.213+0000 7f84c5493740 1 -- 172.21.15.139:0/530059919 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84ac036030 con 0x5559ffafd410 2024-09-16T01:58:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.213+0000 7f849b7fe640 1 -- 172.21.15.139:0/530059919 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7f84ac0f1180 con 0x5559ffafd410 2024-09-16T01:58:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.216+0000 7f849b7fe640 1 -- 172.21.15.139:0/530059919 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f84ac0d6440 con 0x5559ffafd410 2024-09-16T01:58:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.216+0000 7f84c5493740 10 client.26740 fetch_fsmap learned FSMap version 77 2024-09-16T01:58:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.216+0000 7f84c5493740 1 -- 172.21.15.139:0/530059919 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x5559ff840620 con 0x5559ffafd410 2024-09-16T01:58:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.217+0000 7f849b7fe640 1 -- 172.21.15.139:0/530059919 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84ac036030 con 0x5559ffafd410 2024-09-16T01:58:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.218+0000 7f849b7fe640 1 -- 172.21.15.139:0/530059919 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 77) ==== 8450+0+0 (secure 0 0 0) 0x7f84ac0ee990 con 0x5559ffafd410 2024-09-16T01:58:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.220+0000 7f84c5493740 10 client.26740 fetch_fsmap finished waiting for FSMap version 77 2024-09-16T01:58:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.220+0000 7f84c5493740 10 client.26740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:58:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.220+0000 7f84c5493740 20 client.26740 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.220+0000 7f84c5493740 1 --2- 172.21.15.139:0/530059919 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5559ffb177b0 0x5559ffb19bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.220+0000 7f84c5493740 4 client.26740 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:58:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.220+0000 7f84c5493740 1 -- 172.21.15.139:0/530059919 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559ffa6ea20 con 0x5559ffb177b0 2024-09-16T01:58:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.222+0000 7f84bbfff640 1 --2- 172.21.15.139:0/530059919 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5559ffb177b0 0x5559ffb19bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.223+0000 7f84bbfff640 1 --2- 172.21.15.139:0/530059919 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5559ffb177b0 0x5559ffb19bd0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdba20ce969b3f3c server_cookie=35868118ccf1d66e in_seq=0 out_seq=0 2024-09-16T01:58:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.223+0000 7f849b7fe640 10 client.26740 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.227+0000 7f849b7fe640 1 -- 172.21.15.139:0/530059919 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559ffa6ea20 con 0x5559ffb177b0 2024-09-16T01:58:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.227+0000 7f849b7fe640 10 client.26740 handle_command_reply: tid=0 2024-09-16T01:58:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.229+0000 7f84c5493740 1 client.26740 shutdown 2024-09-16T01:58:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.231+0000 7f84c5493740 1 -- 172.21.15.139:0/530059919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafd410 msgr2=0x5559ffb05030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.231+0000 7f84c5493740 1 --2- 172.21.15.139:0/530059919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafd410 0x5559ffb05030 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x5559ffafd9b0 tx=0x7f84ac031000 comp rx=0 tx=0).stop 2024-09-16T01:58:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.231+0000 7f84c5493740 1 -- 172.21.15.139:0/530059919 shutdown_connections 2024-09-16T01:58:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.231+0000 7f84c5493740 1 --2- 172.21.15.139:0/530059919 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5559ffb177b0 0x5559ffb19bd0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.232+0000 7f84c5493740 1 --2- 172.21.15.139:0/530059919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffb07cb0 0x5559ffb05570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.232+0000 7f84c5493740 1 --2- 172.21.15.139:0/530059919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafd410 0x5559ffb05030 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.232+0000 7f84c5493740 1 --2- 172.21.15.139:0/530059919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafcaf0 0x5559ffb04af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.232+0000 7f84c5493740 1 -- 172.21.15.139:0/530059919 >> 172.21.15.139:0/530059919 conn(0x5559ffaf9060 msgr2=0x5559ffafedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.235+0000 7f849b7fe640 0 client.26740 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.239+0000 7f84c5493740 1 -- 172.21.15.139:0/530059919 shutdown_connections 2024-09-16T01:58:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.241+0000 7f84c5493740 1 -- 172.21.15.139:0/530059919 wait complete. 2024-09-16T01:58:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.243+0000 7f84c5493740 20 client.26740 trim_cache size 0 max 16384 2024-09-16T01:58:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.281+0000 7f84c5493740 1 Processor -- start 2024-09-16T01:58:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.282+0000 7f84c5493740 1 -- start start 2024-09-16T01:58:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.282+0000 7f84c5493740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafcaf0 0x5559ffb053a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.282+0000 7f84c5493740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafd410 0x5559ffb06a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.282+0000 7f84c5493740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffb07cb0 0x5559ffb1f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.282+0000 7f84c5493740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc1c6970 con 0x5559ffb07cb0 2024-09-16T01:58:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.282+0000 7f84c5493740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc1c87b0 con 0x5559ffafd410 2024-09-16T01:58:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.282+0000 7f84c5493740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc1c7100 con 0x5559ffafcaf0 2024-09-16T01:58:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.283+0000 7f84c0c8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffb07cb0 0x5559ffb1f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.283+0000 7f84c0c8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffb07cb0 0x5559ffb1f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58416/0 (socket says 172.21.15.139:58416) 2024-09-16T01:58:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.283+0000 7f84c0c8e640 1 -- 172.21.15.139:0/789109132 learned_addr learned my addr 172.21.15.139:0/789109132 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.283+0000 7f84c0c8e640 1 -- 172.21.15.139:0/789109132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafcaf0 msgr2=0x5559ffb053a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:58:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.283+0000 7f84c0c8e640 1 --2- 172.21.15.139:0/789109132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafcaf0 0x5559ffb053a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.283+0000 7f84c0c8e640 1 -- 172.21.15.139:0/789109132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafd410 msgr2=0x5559ffb06a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.283+0000 7f84c0c8e640 1 --2- 172.21.15.139:0/789109132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafd410 0x5559ffb06a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.283+0000 7f84c0c8e640 1 -- 172.21.15.139:0/789109132 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ffaf8e90 con 0x5559ffb07cb0 2024-09-16T01:58:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.284+0000 7f84c0c8e640 1 --2- 172.21.15.139:0/789109132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffb07cb0 0x5559ffb1f4e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f84a400aa50 tx=0x7f84a400af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734e534ce94639c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.284+0000 7f84b1ffb640 1 -- 172.21.15.139:0/789109132 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a401d070 con 0x5559ffb07cb0 2024-09-16T01:58:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.284+0000 7f84b1ffb640 1 -- 172.21.15.139:0/789109132 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a400fe90 con 0x5559ffb07cb0 2024-09-16T01:58:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.285+0000 7f84c5493740 1 -- 172.21.15.139:0/789109132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffb07cb0 msgr2=0x5559ffb1f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.285+0000 7f84c5493740 1 --2- 172.21.15.139:0/789109132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffb07cb0 0x5559ffb1f4e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f84a400aa50 tx=0x7f84a400af10 comp rx=0 tx=0).stop 2024-09-16T01:58:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.288+0000 7f84c5493740 1 -- 172.21.15.139:0/789109132 shutdown_connections 2024-09-16T01:58:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.288+0000 7f84c5493740 1 --2- 172.21.15.139:0/789109132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffb07cb0 0x5559ffb1f4e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.288+0000 7f84c5493740 1 --2- 172.21.15.139:0/789109132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafd410 0x5559ffb06a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.288+0000 7f84c5493740 1 --2- 172.21.15.139:0/789109132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffafcaf0 0x5559ffb053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.288+0000 7f84c5493740 1 -- 172.21.15.139:0/789109132 >> 172.21.15.139:0/789109132 conn(0x5559ffaf9060 msgr2=0x5559ffb06350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.293+0000 7f84c5493740 1 -- 172.21.15.139:0/789109132 shutdown_connections 2024-09-16T01:58:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.295+0000 7f84c5493740 1 -- 172.21.15.139:0/789109132 wait complete. 2024-09-16T01:58:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.295+0000 7f84c5493740 1 -- start start 2024-09-16T01:58:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.295+0000 7f84c5493740 1 Processor -- start 2024-09-16T01:58:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.296+0000 7f84c5493740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffafcaf0 0x5559ffb100f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.296+0000 7f84c5493740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafd410 0x5559ffb0b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.296+0000 7f84c5493740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffb07cb0 0x5559ffb0b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.296+0000 7f84c5493740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc1c8020 con 0x5559ffafcaf0 2024-09-16T01:58:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.296+0000 7f84c5493740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc1c7890 con 0x5559ffafd410 2024-09-16T01:58:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.296+0000 7f84c5493740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc073ab0 con 0x5559ffb07cb0 2024-09-16T01:58:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.296+0000 7f84c5493740 10 client.0 _ll_register_callbacks cb 0x7f84ac0b96e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.296+0000 7f84c0c8e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffb07cb0 0x5559ffb0b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.296+0000 7f84c0c8e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffb07cb0 0x5559ffb0b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:60126/0 (socket says 172.21.15.139:60126) 2024-09-16T01:58:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.296+0000 7f84c0c8e640 1 -- 172.21.15.139:0/2429781338 learned_addr learned my addr 172.21.15.139:0/2429781338 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.297+0000 7f84bb7fe640 1 --2- 172.21.15.139:0/2429781338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafd410 0x5559ffb0b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.297+0000 7f84bbfff640 1 --2- 172.21.15.139:0/2429781338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffafcaf0 0x5559ffb100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.297+0000 7f84c0c8e640 1 -- 172.21.15.139:0/2429781338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafd410 msgr2=0x5559ffb0b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.297+0000 7f84c0c8e640 1 --2- 172.21.15.139:0/2429781338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafd410 0x5559ffb0b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.297+0000 7f84c0c8e640 1 -- 172.21.15.139:0/2429781338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffafcaf0 msgr2=0x5559ffb100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.297+0000 7f84c0c8e640 1 --2- 172.21.15.139:0/2429781338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffafcaf0 0x5559ffb100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.297+0000 7f84c0c8e640 1 -- 172.21.15.139:0/2429781338 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559ffb0c180 con 0x5559ffb07cb0 2024-09-16T01:58:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.298+0000 7f84c0c8e640 1 --2- 172.21.15.139:0/2429781338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffb07cb0 0x5559ffb0b950 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f84a4010470 tx=0x7f84a4018670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e710de916cc889b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.298+0000 7f849b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:58:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.304+0000 7f849b7fe640 1 -- 172.21.15.139:0/2429781338 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a401d050 con 0x5559ffb07cb0 2024-09-16T01:58:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.304+0000 7f849b7fe640 1 -- 172.21.15.139:0/2429781338 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9850+0+0 (secure 0 0 0) 0x7f84a40189f0 con 0x5559ffb07cb0 2024-09-16T01:58:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.304+0000 7f84c5493740 1 -- 172.21.15.139:0/2429781338 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84a400fe90 con 0x5559ffb07cb0 2024-09-16T01:58:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.307+0000 7f849b7fe640 1 -- 172.21.15.139:0/2429781338 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f84a400a6c0 con 0x5559ffb07cb0 2024-09-16T01:58:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.311+0000 7f84c5493740 10 client.26743 fetch_fsmap learned FSMap version 77 2024-09-16T01:58:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.311+0000 7f84c5493740 1 -- 172.21.15.139:0/2429781338 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x5559ffb82450 con 0x5559ffb07cb0 2024-09-16T01:58:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.317+0000 7f849b7fe640 1 -- 172.21.15.139:0/2429781338 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a400fe90 con 0x5559ffb07cb0 2024-09-16T01:58:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.317+0000 7f849b7fe640 1 -- 172.21.15.139:0/2429781338 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 77) ==== 8450+0+0 (secure 0 0 0) 0x7f84a402c030 con 0x5559ffb07cb0 2024-09-16T01:58:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.317+0000 7f84c5493740 10 client.26743 fetch_fsmap finished waiting for FSMap version 77 2024-09-16T01:58:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.317+0000 7f84c5493740 10 client.26743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:58:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.317+0000 7f84c5493740 20 client.26743 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.317+0000 7f84c5493740 1 --2- 172.21.15.139:0/2429781338 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5559ffb82ad0 0x5559ffb84ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.317+0000 7f84c5493740 4 client.26743 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:58:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.317+0000 7f84c5493740 1 -- 172.21.15.139:0/2429781338 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5559ffa68d70 con 0x5559ffb82ad0 2024-09-16T01:58:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.317+0000 7f84bbfff640 1 --2- 172.21.15.139:0/2429781338 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5559ffb82ad0 0x5559ffb84ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.326+0000 7f84bbfff640 1 --2- 172.21.15.139:0/2429781338 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5559ffb82ad0 0x5559ffb84ef0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8873c9783192fc32 server_cookie=efc8966a74f7c261 in_seq=0 out_seq=0 2024-09-16T01:58:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.327+0000 7f849b7fe640 10 client.26743 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.467+0000 7f849b7fe640 1 -- 172.21.15.139:0/2429781338 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5559ffa68d70 con 0x5559ffb82ad0 2024-09-16T01:58:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.467+0000 7f849b7fe640 10 client.26743 handle_command_reply: tid=0 2024-09-16T01:58:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.467+0000 7f84c5493740 1 client.26743 shutdown 2024-09-16T01:58:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.467+0000 7f84c5493740 1 -- 172.21.15.139:0/2429781338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffb07cb0 msgr2=0x5559ffb0b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.467+0000 7f84c5493740 1 --2- 172.21.15.139:0/2429781338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffb07cb0 0x5559ffb0b950 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f84a4010470 tx=0x7f84a4018670 comp rx=0 tx=0).stop 2024-09-16T01:58:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.467+0000 7f84c5493740 1 -- 172.21.15.139:0/2429781338 shutdown_connections 2024-09-16T01:58:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.467+0000 7f84c5493740 1 --2- 172.21.15.139:0/2429781338 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5559ffb82ad0 0x5559ffb84ef0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.467+0000 7f84c5493740 1 --2- 172.21.15.139:0/2429781338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559ffb07cb0 0x5559ffb0b950 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.467+0000 7f849b7fe640 0 client.26743 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.467+0000 7f84c5493740 1 --2- 172.21.15.139:0/2429781338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559ffafd410 0x5559ffb0b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.467+0000 7f84c5493740 1 --2- 172.21.15.139:0/2429781338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559ffafcaf0 0x5559ffb100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.467+0000 7f84c5493740 1 -- 172.21.15.139:0/2429781338 >> 172.21.15.139:0/2429781338 conn(0x5559ffaf9060 msgr2=0x5559ffafed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.467+0000 7f84c5493740 1 -- 172.21.15.139:0/2429781338 shutdown_connections 2024-09-16T01:58:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.467+0000 7f84c5493740 1 -- 172.21.15.139:0/2429781338 wait complete. 2024-09-16T01:58:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.468+0000 7f84c5493740 20 client.26743 trim_cache size 0 max 16384 2024-09-16T01:58:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:58:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:58:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:58:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:58:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.468+0000 7f84c26f1640 1 -- 172.21.15.139:0/2008863325 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f84a80a3690 msgr2=0x7f84a80a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.468+0000 7f84c26f1640 1 --2- 172.21.15.139:0/2008863325 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f84a80a3690 0x7f84a80a5b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f84b40058c0 tx=0x7f84b40123f0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.468+0000 7f84c26f1640 1 -- 172.21.15.139:0/2008863325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84bc1095d0 msgr2=0x7f84bc1a76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.468+0000 7f84c26f1640 1 --2- 172.21.15.139:0/2008863325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84bc1095d0 0x7f84bc1a76a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f84ac032fe0 tx=0x7f84ac003970 comp rx=0 tx=0).stop 2024-09-16T01:58:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.469+0000 7f84c26f1640 1 -- 172.21.15.139:0/2008863325 shutdown_connections 2024-09-16T01:58:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.469+0000 7f84c26f1640 1 --2- 172.21.15.139:0/2008863325 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f84a80a3690 0x7f84a80a5b50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.469+0000 7f84c26f1640 1 --2- 172.21.15.139:0/2008863325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84bc11b140 0x7f84bc1aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.469+0000 7f84c26f1640 1 --2- 172.21.15.139:0/2008863325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84bc1095d0 0x7f84bc1a76a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.469+0000 7f84c26f1640 1 --2- 172.21.15.139:0/2008863325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84bc1091f0 0x7f84bc1a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.469+0000 7f84c26f1640 1 -- 172.21.15.139:0/2008863325 >> 172.21.15.139:0/2008863325 conn(0x7f84bc06ce00 msgr2=0x7f84bc11b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.469+0000 7f84c26f1640 1 -- 172.21.15.139:0/2008863325 shutdown_connections 2024-09-16T01:58:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:37.469+0000 7f84c26f1640 1 -- 172.21.15.139:0/2008863325 wait complete. 2024-09-16T01:58:37.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:37 smithi164 ceph-mon[35121]: pgmap v579: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 472 KiB/s wr, 139 op/s 2024-09-16T01:58:37.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:37 smithi139 ceph-mon[31175]: pgmap v579: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 472 KiB/s wr, 139 op/s 2024-09-16T01:58:37.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:37 smithi176 ceph-mon[35324]: pgmap v579: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 472 KiB/s wr, 139 op/s 2024-09-16T01:58:39.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:39 smithi164 ceph-mon[35121]: pgmap v580: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 475 KiB/s wr, 155 op/s 2024-09-16T01:58:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:39 smithi139 ceph-mon[31175]: pgmap v580: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 475 KiB/s wr, 155 op/s 2024-09-16T01:58:39.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:39 smithi176 ceph-mon[35324]: pgmap v580: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 475 KiB/s wr, 155 op/s 2024-09-16T01:58:41.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:41 smithi164 ceph-mon[35121]: pgmap v581: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 260 KiB/s wr, 147 op/s 2024-09-16T01:58:41.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:41 smithi139 ceph-mon[31175]: pgmap v581: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 260 KiB/s wr, 147 op/s 2024-09-16T01:58:41.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:41 smithi176 ceph-mon[35324]: pgmap v581: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 260 KiB/s wr, 147 op/s 2024-09-16T01:58:43.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:43 smithi164 ceph-mon[35121]: pgmap v582: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 138 KiB/s wr, 123 op/s 2024-09-16T01:58:43.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:43 smithi139 ceph-mon[31175]: pgmap v582: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 138 KiB/s wr, 123 op/s 2024-09-16T01:58:43.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:43 smithi176 ceph-mon[35324]: pgmap v582: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 138 KiB/s wr, 123 op/s 2024-09-16T01:58:45.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:45 smithi164 ceph-mon[35121]: pgmap v583: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 139 KiB/s wr, 157 op/s 2024-09-16T01:58:45.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:45 smithi139 ceph-mon[31175]: pgmap v583: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 139 KiB/s wr, 157 op/s 2024-09-16T01:58:45.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:45 smithi176 ceph-mon[35324]: pgmap v583: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 139 KiB/s wr, 157 op/s 2024-09-16T01:58:46.500 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:58:46.982 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:58:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:47 smithi139 ceph-mon[31175]: pgmap v584: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 138 op/s 2024-09-16T01:58:47.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:47 smithi139 ceph-mon[31175]: osdmap e117: 12 total, 12 up, 12 in 2024-09-16T01:58:47.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:47 smithi176 ceph-mon[35324]: pgmap v584: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 138 op/s 2024-09-16T01:58:47.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:47 smithi176 ceph-mon[35324]: osdmap e117: 12 total, 12 up, 12 in 2024-09-16T01:58:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:47 smithi164 ceph-mon[35121]: pgmap v584: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 138 op/s 2024-09-16T01:58:47.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:47 smithi164 ceph-mon[35121]: osdmap e117: 12 total, 12 up, 12 in 2024-09-16T01:58:48.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.624+0000 7f6229230640 1 -- 172.21.15.139:0/3542421659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6224109830 msgr2=0x7f6224109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:48.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.624+0000 7f6229230640 1 --2- 172.21.15.139:0/3542421659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6224109830 0x7f6224109c90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f621401a910 tx=0x7f621403b450 comp rx=0 tx=0).stop 2024-09-16T01:58:48.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.625+0000 7f6229230640 1 -- 172.21.15.139:0/3542421659 shutdown_connections 2024-09-16T01:58:48.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.625+0000 7f6229230640 1 --2- 172.21.15.139:0/3542421659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622411b140 0x7f622411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:48.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.625+0000 7f6229230640 1 --2- 172.21.15.139:0/3542421659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6224109830 0x7f6224109c90 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:48.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.625+0000 7f6229230640 1 --2- 172.21.15.139:0/3542421659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6224109450 0x7f6224110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:48.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.625+0000 7f6229230640 1 -- 172.21.15.139:0/3542421659 >> 172.21.15.139:0/3542421659 conn(0x7f622406e140 msgr2=0x7f622406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:48.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.627+0000 7f6229230640 1 -- 172.21.15.139:0/3542421659 shutdown_connections 2024-09-16T01:58:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.628+0000 7f6229230640 1 -- 172.21.15.139:0/3542421659 wait complete. 2024-09-16T01:58:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.629+0000 7f6229230640 1 Processor -- start 2024-09-16T01:58:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.629+0000 7f6229230640 1 -- start start 2024-09-16T01:58:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.629+0000 7f6229230640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6224109450 0x7f6224116850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.629+0000 7f6229230640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6224109830 0x7f6224116d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.629+0000 7f6229230640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622411b140 0x7f622411a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.629+0000 7f6229230640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6224113300 con 0x7f6224109830 2024-09-16T01:58:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.629+0000 7f6229230640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6224113180 con 0x7f622411b140 2024-09-16T01:58:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.629+0000 7f6229230640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6224113480 con 0x7f6224109450 2024-09-16T01:58:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.630+0000 7f622359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622411b140 0x7f622411a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.630+0000 7f622359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622411b140 0x7f622411a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:58222/0 (socket says 172.21.15.139:58222) 2024-09-16T01:58:48.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.630+0000 7f622359e640 1 -- 172.21.15.139:0/3672458756 learned_addr learned my addr 172.21.15.139:0/3672458756 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:48.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.630+0000 7f6222d9d640 1 --2- 172.21.15.139:0/3672458756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6224109450 0x7f6224116850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:48.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.630+0000 7f622259c640 1 --2- 172.21.15.139:0/3672458756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6224109830 0x7f6224116d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:48.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.631+0000 7f622359e640 1 -- 172.21.15.139:0/3672458756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6224109450 msgr2=0x7f6224116850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:48.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.631+0000 7f622359e640 1 --2- 172.21.15.139:0/3672458756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6224109450 0x7f6224116850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:48.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.631+0000 7f622359e640 1 -- 172.21.15.139:0/3672458756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6224109830 msgr2=0x7f6224116d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:48.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.631+0000 7f622359e640 1 --2- 172.21.15.139:0/3672458756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6224109830 0x7f6224116d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:48.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.631+0000 7f622359e640 1 -- 172.21.15.139:0/3672458756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622411a7f0 con 0x7f622411b140 2024-09-16T01:58:48.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.632+0000 7f622359e640 1 --2- 172.21.15.139:0/3672458756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622411b140 0x7f622411a110 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f621800e970 tx=0x7f621800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:48.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.632+0000 7f620bfff640 1 -- 172.21.15.139:0/3672458756 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621800ccc0 con 0x7f622411b140 2024-09-16T01:58:48.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.632+0000 7f620bfff640 1 -- 172.21.15.139:0/3672458756 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621800ce60 con 0x7f622411b140 2024-09-16T01:58:48.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.632+0000 7f620bfff640 1 -- 172.21.15.139:0/3672458756 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621800ccc0 con 0x7f622411b140 2024-09-16T01:58:48.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.632+0000 7f6229230640 1 -- 172.21.15.139:0/3672458756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622411aa80 con 0x7f622411b140 2024-09-16T01:58:48.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.632+0000 7f6229230640 1 -- 172.21.15.139:0/3672458756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62241ab420 con 0x7f622411b140 2024-09-16T01:58:48.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.637+0000 7f6229230640 1 -- 172.21.15.139:0/3672458756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6210005290 con 0x7f622411b140 2024-09-16T01:58:48.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.641+0000 7f620bfff640 1 -- 172.21.15.139:0/3672458756 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6218005020 con 0x7f622411b140 2024-09-16T01:58:48.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.642+0000 7f620bfff640 1 --2- 172.21.15.139:0/3672458756 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f620c0a3870 0x7f620c0a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:48.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.642+0000 7f620bfff640 1 -- 172.21.15.139:0/3672458756 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9774+0+0 (secure 0 0 0) 0x7f621800cac0 con 0x7f622411b140 2024-09-16T01:58:48.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.642+0000 7f6222d9d640 1 --2- 172.21.15.139:0/3672458756 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f620c0a3870 0x7f620c0a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:48.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.643+0000 7f6222d9d640 1 --2- 172.21.15.139:0/3672458756 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f620c0a3870 0x7f620c0a5d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f62241177c0 tx=0x7f621c009500 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:48.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:48.648+0000 7f620bfff640 1 -- 172.21.15.139:0/3672458756 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f621809c460 con 0x7f622411b140 2024-09-16T01:58:49.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:49.042+0000 7f6229230640 1 -- 172.21.15.139:0/3672458756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6210005580 con 0x7f622411b140 2024-09-16T01:58:49.052 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:58:49.052 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 77 2024-09-16T01:58:49.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:49.046+0000 7f620bfff640 1 -- 172.21.15.139:0/3672458756 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+12031 (secure 0 0 0) 0x7f621809c460 con 0x7f622411b140 2024-09-16T01:58:49.052 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":77,"btime":"2024-09-16T01:58:24:183207+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":77,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:58:24.058642+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:58:49.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:49.053+0000 7f6229230640 1 -- 172.21.15.139:0/3672458756 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f620c0a3870 msgr2=0x7f620c0a5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:49.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:49.053+0000 7f6229230640 1 --2- 172.21.15.139:0/3672458756 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f620c0a3870 0x7f620c0a5d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f62241177c0 tx=0x7f621c009500 comp rx=0 tx=0).stop 2024-09-16T01:58:49.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:49.053+0000 7f6229230640 1 -- 172.21.15.139:0/3672458756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622411b140 msgr2=0x7f622411a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:49.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:49.053+0000 7f6229230640 1 --2- 172.21.15.139:0/3672458756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622411b140 0x7f622411a110 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f621800e970 tx=0x7f621800ee30 comp rx=0 tx=0).stop 2024-09-16T01:58:49.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:49.053+0000 7f6229230640 1 -- 172.21.15.139:0/3672458756 shutdown_connections 2024-09-16T01:58:49.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:49.053+0000 7f6229230640 1 --2- 172.21.15.139:0/3672458756 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f620c0a3870 0x7f620c0a5d30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:49.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:49.053+0000 7f6229230640 1 --2- 172.21.15.139:0/3672458756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622411b140 0x7f622411a110 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:49.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:49.053+0000 7f6229230640 1 --2- 172.21.15.139:0/3672458756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6224109830 0x7f6224116d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:49.053+0000 7f6229230640 1 --2- 172.21.15.139:0/3672458756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6224109450 0x7f6224116850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:49.053+0000 7f6229230640 1 -- 172.21.15.139:0/3672458756 >> 172.21.15.139:0/3672458756 conn(0x7f622406e140 msgr2=0x7f6224070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:49.054+0000 7f6229230640 1 -- 172.21.15.139:0/3672458756 shutdown_connections 2024-09-16T01:58:49.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:49.054+0000 7f6229230640 1 -- 172.21.15.139:0/3672458756 wait complete. 2024-09-16T01:58:49.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:49 smithi139 ceph-mon[31175]: pgmap v586: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.7 KiB/s wr, 158 op/s 2024-09-16T01:58:49.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:49 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3672458756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:58:49.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:49 smithi176 ceph-mon[35324]: pgmap v586: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.7 KiB/s wr, 158 op/s 2024-09-16T01:58:49.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:49 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3672458756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:58:49.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:49 smithi164 ceph-mon[35121]: pgmap v586: 129 pgs: 22 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 38 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.7 KiB/s wr, 158 op/s 2024-09-16T01:58:49.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:49 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3672458756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:58:49.929 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 72a1fc77 --release --await' 2024-09-16T01:58:49.930 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 72a1fc77 --release --await 2024-09-16T01:58:50.490 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:58:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.640+0000 7fe22a1e4640 1 -- 172.21.15.139:0/3801704476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2241095d0 msgr2=0x7fe224109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.640+0000 7fe22a1e4640 1 --2- 172.21.15.139:0/3801704476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2241095d0 0x7fe224109a10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009900 tx=0x7fe21c02a440 comp rx=0 tx=0).stop 2024-09-16T01:58:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.646+0000 7fe22a1e4640 1 -- 172.21.15.139:0/3801704476 shutdown_connections 2024-09-16T01:58:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.646+0000 7fe22a1e4640 1 --2- 172.21.15.139:0/3801704476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe22411b140 0x7fe22411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.646+0000 7fe22a1e4640 1 --2- 172.21.15.139:0/3801704476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2241095d0 0x7fe224109a10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.646+0000 7fe22a1e4640 1 --2- 172.21.15.139:0/3801704476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2241091f0 0x7fe224110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.646+0000 7fe22a1e4640 1 -- 172.21.15.139:0/3801704476 >> 172.21.15.139:0/3801704476 conn(0x7fe22406cc60 msgr2=0x7fe22406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.650+0000 7fe22a1e4640 1 -- 172.21.15.139:0/3801704476 shutdown_connections 2024-09-16T01:58:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.651+0000 7fe22a1e4640 1 -- 172.21.15.139:0/3801704476 wait complete. 2024-09-16T01:58:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.651+0000 7fe22a1e4640 1 Processor -- start 2024-09-16T01:58:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.656+0000 7fe22a1e4640 1 -- start start 2024-09-16T01:58:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.656+0000 7fe22a1e4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2241091f0 0x7fe2241a7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.657+0000 7fe22a1e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2241095d0 0x7fe2241a7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.657+0000 7fe22a1e4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe22411b140 0x7fe2241ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.657+0000 7fe22a1e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe224120ba0 con 0x7fe2241095d0 2024-09-16T01:58:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.657+0000 7fe22a1e4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe224120a20 con 0x7fe22411b140 2024-09-16T01:58:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.657+0000 7fe22a1e4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe224120d20 con 0x7fe2241091f0 2024-09-16T01:58:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.662+0000 7fe2237fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2241091f0 0x7fe2241a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.662+0000 7fe2237fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2241091f0 0x7fe2241a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:56288/0 (socket says 172.21.15.139:56288) 2024-09-16T01:58:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.662+0000 7fe2237fe640 1 -- 172.21.15.139:0/930799715 learned_addr learned my addr 172.21.15.139:0/930799715 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.662+0000 7fe222ffd640 1 --2- 172.21.15.139:0/930799715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2241095d0 0x7fe2241a7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.663+0000 7fe223fff640 1 --2- 172.21.15.139:0/930799715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe22411b140 0x7fe2241ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.664+0000 7fe2237fe640 1 -- 172.21.15.139:0/930799715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe22411b140 msgr2=0x7fe2241ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.664+0000 7fe2237fe640 1 --2- 172.21.15.139:0/930799715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe22411b140 0x7fe2241ab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.664+0000 7fe2237fe640 1 -- 172.21.15.139:0/930799715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2241095d0 msgr2=0x7fe2241a7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.664+0000 7fe2237fe640 1 --2- 172.21.15.139:0/930799715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2241095d0 0x7fe2241a7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.664+0000 7fe2237fe640 1 -- 172.21.15.139:0/930799715 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2241ac100 con 0x7fe2241091f0 2024-09-16T01:58:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.666+0000 7fe2237fe640 1 --2- 172.21.15.139:0/930799715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2241091f0 0x7fe2241a7110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe21400ebc0 tx=0x7fe21400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.669+0000 7fe220ff9640 1 -- 172.21.15.139:0/930799715 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214010630 con 0x7fe2241091f0 2024-09-16T01:58:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.669+0000 7fe22a1e4640 1 -- 172.21.15.139:0/930799715 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2241ac3f0 con 0x7fe2241091f0 2024-09-16T01:58:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.669+0000 7fe22a1e4640 1 -- 172.21.15.139:0/930799715 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2241b3c80 con 0x7fe2241091f0 2024-09-16T01:58:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.669+0000 7fe220ff9640 1 -- 172.21.15.139:0/930799715 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe214010cb0 con 0x7fe2241091f0 2024-09-16T01:58:50.671 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.669+0000 7fe220ff9640 1 -- 172.21.15.139:0/930799715 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214018c40 con 0x7fe2241091f0 2024-09-16T01:58:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.673+0000 7fe22d058740 1 Processor -- start 2024-09-16T01:58:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.673+0000 7fe22d058740 1 -- start start 2024-09-16T01:58:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.673+0000 7fe22d058740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf8cc0 0x561a0bdf90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.673+0000 7fe22d058740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0bdf95e0 0x561a0be03940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.673+0000 7fe22d058740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0be03e80 0x561a0be06270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.673+0000 7fe22d058740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561a0bd5d2f0 con 0x561a0bdf8cc0 2024-09-16T01:58:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.673+0000 7fe22d058740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561a0bd05270 con 0x561a0bdf95e0 2024-09-16T01:58:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.673+0000 7fe22d058740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561a0bd592b0 con 0x561a0be03e80 2024-09-16T01:58:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.674+0000 7fe223fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf8cc0 0x561a0bdf90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.674+0000 7fe223fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf8cc0 0x561a0bdf90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35446/0 (socket says 172.21.15.139:35446) 2024-09-16T01:58:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.674+0000 7fe223fff640 1 -- 172.21.15.139:0/997714546 learned_addr learned my addr 172.21.15.139:0/997714546 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.675+0000 7fe223fff640 1 -- 172.21.15.139:0/997714546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0be03e80 msgr2=0x561a0be06270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:58:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.675+0000 7fe223fff640 1 --2- 172.21.15.139:0/997714546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0be03e80 0x561a0be06270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.675+0000 7fe223fff640 1 -- 172.21.15.139:0/997714546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0bdf95e0 msgr2=0x561a0be03940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:58:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.675+0000 7fe223fff640 1 --2- 172.21.15.139:0/997714546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0bdf95e0 0x561a0be03940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.675+0000 7fe223fff640 1 -- 172.21.15.139:0/997714546 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0bd04340 con 0x561a0bdf8cc0 2024-09-16T01:58:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.675+0000 7fe223fff640 1 --2- 172.21.15.139:0/997714546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf8cc0 0x561a0bdf90a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009820 tx=0x7fe20c009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bfacd0beda0119 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.677+0000 7fe219ffb640 1 -- 172.21.15.139:0/997714546 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c039070 con 0x561a0bdf8cc0 2024-09-16T01:58:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.677+0000 7fe219ffb640 1 -- 172.21.15.139:0/997714546 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20c035480 con 0x561a0bdf8cc0 2024-09-16T01:58:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.678+0000 7fe22d058740 1 -- 172.21.15.139:0/997714546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf8cc0 msgr2=0x561a0bdf90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.678+0000 7fe22d058740 1 --2- 172.21.15.139:0/997714546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf8cc0 0x561a0bdf90a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009820 tx=0x7fe20c009ce0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.680+0000 7fe220ff9640 1 -- 172.21.15.139:0/930799715 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe214018420 con 0x7fe2241091f0 2024-09-16T01:58:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.681+0000 7fe220ff9640 1 --2- 172.21.15.139:0/930799715 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe2100a3580 0x7fe2100a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.682+0000 7fe220ff9640 1 -- 172.21.15.139:0/930799715 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9774+0+0 (secure 0 0 0) 0x7fe2140d0750 con 0x7fe2241091f0 2024-09-16T01:58:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.684+0000 7fe223fff640 1 --2- 172.21.15.139:0/930799715 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe2100a3580 0x7fe2100a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.688+0000 7fe22d058740 1 -- 172.21.15.139:0/997714546 shutdown_connections 2024-09-16T01:58:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.688+0000 7fe22d058740 1 --2- 172.21.15.139:0/997714546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0be03e80 0x561a0be06270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.688+0000 7fe22d058740 1 --2- 172.21.15.139:0/997714546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0bdf95e0 0x561a0be03940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.688+0000 7fe22d058740 1 --2- 172.21.15.139:0/997714546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf8cc0 0x561a0bdf90a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.688+0000 7fe22d058740 1 -- 172.21.15.139:0/997714546 >> 172.21.15.139:0/997714546 conn(0x561a0bdf5230 msgr2=0x561a0bdf5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.696+0000 7fe22d058740 1 -- 172.21.15.139:0/997714546 shutdown_connections 2024-09-16T01:58:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.704+0000 7fe223fff640 1 --2- 172.21.15.139:0/930799715 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe2100a3580 0x7fe2100a5a40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe20c035f40 tx=0x7fe20c030040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.704+0000 7fe22d058740 1 -- 172.21.15.139:0/997714546 wait complete. 2024-09-16T01:58:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.705+0000 7fe22d058740 1 -- start start 2024-09-16T01:58:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.705+0000 7fe22d058740 1 Processor -- start 2024-09-16T01:58:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.706+0000 7fe22d058740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0bdf8cc0 0x561a0be00c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.706+0000 7fe22d058740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf95e0 0x561a0be01180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.706+0000 7fe22d058740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0be03e80 0x561a0be016c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.706+0000 7fe22d058740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561a0be06aa0 con 0x561a0bdf95e0 2024-09-16T01:58:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.706+0000 7fe22d058740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561a0bce6290 con 0x561a0be03e80 2024-09-16T01:58:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.706+0000 7fe22d058740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561a0be06c20 con 0x561a0bdf8cc0 2024-09-16T01:58:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.706+0000 7fe22d058740 10 client.0 _ll_register_callbacks cb 0x561a0bd17140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.707+0000 7fe222ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf95e0 0x561a0be01180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.707+0000 7fe222ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf95e0 0x561a0be01180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35452/0 (socket says 172.21.15.139:35452) 2024-09-16T01:58:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.707+0000 7fe222ffd640 1 -- 172.21.15.139:0/3173531277 learned_addr learned my addr 172.21.15.139:0/3173531277 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.707+0000 7fe2237fe640 1 --2- 172.21.15.139:0/3173531277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0bdf8cc0 0x561a0be00c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.707+0000 7fe222ffd640 1 -- 172.21.15.139:0/3173531277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0bdf8cc0 msgr2=0x561a0be00c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.707+0000 7fe222ffd640 1 --2- 172.21.15.139:0/3173531277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0bdf8cc0 0x561a0be00c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.707+0000 7fe222ffd640 1 -- 172.21.15.139:0/3173531277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0be03e80 msgr2=0x561a0be016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.707+0000 7fe222ffd640 1 --2- 172.21.15.139:0/3173531277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0be03e80 0x561a0be016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.707+0000 7fe222ffd640 1 -- 172.21.15.139:0/3173531277 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0ba70210 con 0x561a0bdf95e0 2024-09-16T01:58:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.709+0000 7fe222ffd640 1 --2- 172.21.15.139:0/3173531277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf95e0 0x561a0be01180 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe21c004fd0 tx=0x7fe21c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300133eafdeaf778 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.709+0000 7fe202ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:58:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.710+0000 7fe202ffd640 1 -- 172.21.15.139:0/3173531277 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c041070 con 0x561a0bdf95e0 2024-09-16T01:58:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.710+0000 7fe202ffd640 1 -- 172.21.15.139:0/3173531277 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9774+0+0 (secure 0 0 0) 0x7fe21c005200 con 0x561a0bdf95e0 2024-09-16T01:58:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.710+0000 7fe22d058740 1 -- 172.21.15.139:0/3173531277 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe20c035480 con 0x561a0bdf95e0 2024-09-16T01:58:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.720+0000 7fe202ffd640 1 -- 172.21.15.139:0/3173531277 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fe21c004a10 con 0x561a0bdf95e0 2024-09-16T01:58:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.732+0000 7fe22d058740 10 client.17358 fetch_fsmap learned FSMap version 77 2024-09-16T01:58:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.732+0000 7fe22d058740 1 -- 172.21.15.139:0/3173531277 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x561a0bb3c620 con 0x561a0bdf95e0 2024-09-16T01:58:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.735+0000 7fe202ffd640 1 -- 172.21.15.139:0/3173531277 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20c035480 con 0x561a0bdf95e0 2024-09-16T01:58:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.735+0000 7fe202ffd640 1 -- 172.21.15.139:0/3173531277 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 77) ==== 8450+0+0 (secure 0 0 0) 0x7fe21c031440 con 0x561a0bdf95e0 2024-09-16T01:58:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.736+0000 7fe22d058740 10 client.17358 fetch_fsmap finished waiting for FSMap version 77 2024-09-16T01:58:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.736+0000 7fe22d058740 10 client.17358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:58:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.736+0000 7fe22d058740 20 client.17358 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.736+0000 7fe22d058740 1 --2- 172.21.15.139:0/3173531277 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a0be13df0 0x561a0be16240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.736+0000 7fe22d058740 4 client.17358 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:58:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.736+0000 7fe22d058740 1 -- 172.21.15.139:0/3173531277 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a0bd6aa20 con 0x561a0be13df0 2024-09-16T01:58:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.737+0000 7fe2237fe640 1 --2- 172.21.15.139:0/3173531277 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a0be13df0 0x561a0be16240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.737+0000 7fe2237fe640 1 --2- 172.21.15.139:0/3173531277 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a0be13df0 0x561a0be16240 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14ebb7e8ab6b121 server_cookie=45594a563cdacc97 in_seq=0 out_seq=0 2024-09-16T01:58:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.738+0000 7fe202ffd640 10 client.17358 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.738+0000 7fe202ffd640 1 -- 172.21.15.139:0/3173531277 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a0bd6aa20 con 0x561a0be13df0 2024-09-16T01:58:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.738+0000 7fe202ffd640 10 client.17358 handle_command_reply: tid=0 2024-09-16T01:58:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.738+0000 7fe22d058740 1 client.17358 shutdown 2024-09-16T01:58:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.740+0000 7fe22d058740 1 -- 172.21.15.139:0/3173531277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf95e0 msgr2=0x561a0be01180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.740+0000 7fe22d058740 1 --2- 172.21.15.139:0/3173531277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf95e0 0x561a0be01180 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe21c004fd0 tx=0x7fe21c003820 comp rx=0 tx=0).stop 2024-09-16T01:58:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.741+0000 7fe22d058740 1 -- 172.21.15.139:0/3173531277 shutdown_connections 2024-09-16T01:58:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.741+0000 7fe22d058740 1 --2- 172.21.15.139:0/3173531277 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a0be13df0 0x561a0be16240 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.741+0000 7fe22d058740 1 --2- 172.21.15.139:0/3173531277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0be03e80 0x561a0be016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.741+0000 7fe22d058740 1 --2- 172.21.15.139:0/3173531277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf95e0 0x561a0be01180 secure :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe21c004fd0 tx=0x7fe21c003820 comp rx=0 tx=0).stop 2024-09-16T01:58:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.741+0000 7fe22d058740 1 --2- 172.21.15.139:0/3173531277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0bdf8cc0 0x561a0be00c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.741+0000 7fe22d058740 1 -- 172.21.15.139:0/3173531277 >> 172.21.15.139:0/3173531277 conn(0x561a0bdf5230 msgr2=0x561a0bdfaf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.741+0000 7fe202ffd640 0 client.17358 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.744+0000 7fe22d058740 1 -- 172.21.15.139:0/3173531277 shutdown_connections 2024-09-16T01:58:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.749+0000 7fe22d058740 1 -- 172.21.15.139:0/3173531277 wait complete. 2024-09-16T01:58:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.750+0000 7fe22d058740 20 client.17358 trim_cache size 0 max 16384 2024-09-16T01:58:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.781+0000 7fe22d058740 1 Processor -- start 2024-09-16T01:58:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.782+0000 7fe22d058740 1 -- start start 2024-09-16T01:58:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.782+0000 7fe22d058740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf8cc0 0x561a0be014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.782+0000 7fe22d058740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0be03e80 0x561a0bdffad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.782+0000 7fe22d058740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0be00010 0x561a0be1bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.782+0000 7fe22d058740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe2040102f0 con 0x561a0bdf8cc0 2024-09-16T01:58:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.782+0000 7fe22d058740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe204012130 con 0x561a0be03e80 2024-09-16T01:58:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.782+0000 7fe22d058740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe204010a80 con 0x561a0be00010 2024-09-16T01:58:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.786+0000 7fe222ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0be03e80 0x561a0bdffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.786+0000 7fe222ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0be03e80 0x561a0bdffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:58260/0 (socket says 172.21.15.139:58260) 2024-09-16T01:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.786+0000 7fe222ffd640 1 -- 172.21.15.139:0/1593307670 learned_addr learned my addr 172.21.15.139:0/1593307670 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.787+0000 7fe223fff640 1 --2- 172.21.15.139:0/1593307670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0be00010 0x561a0be1bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.787+0000 7fe222ffd640 1 -- 172.21.15.139:0/1593307670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0be00010 msgr2=0x561a0be1bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.787+0000 7fe222ffd640 1 --2- 172.21.15.139:0/1593307670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0be00010 0x561a0be1bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.787+0000 7fe222ffd640 1 -- 172.21.15.139:0/1593307670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf8cc0 msgr2=0x561a0be014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.787+0000 7fe222ffd640 1 --2- 172.21.15.139:0/1593307670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf8cc0 0x561a0be014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.787+0000 7fe222ffd640 1 -- 172.21.15.139:0/1593307670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0bdff170 con 0x561a0be03e80 2024-09-16T01:58:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.789+0000 7fe222ffd640 1 --2- 172.21.15.139:0/1593307670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0be03e80 0x561a0bdffad0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe21c0319b0 tx=0x7fe21c009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfd0c3e5e69ab824 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.794+0000 7fe2197fa640 1 -- 172.21.15.139:0/1593307670 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c041040 con 0x561a0be03e80 2024-09-16T01:58:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.795+0000 7fe2197fa640 1 -- 172.21.15.139:0/1593307670 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe21c009780 con 0x561a0be03e80 2024-09-16T01:58:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.796+0000 7fe22d058740 1 -- 172.21.15.139:0/1593307670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0be03e80 msgr2=0x561a0bdffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.796+0000 7fe22d058740 1 --2- 172.21.15.139:0/1593307670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0be03e80 0x561a0bdffad0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe21c0319b0 tx=0x7fe21c009520 comp rx=0 tx=0).stop 2024-09-16T01:58:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.803+0000 7fe22d058740 1 -- 172.21.15.139:0/1593307670 shutdown_connections 2024-09-16T01:58:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.803+0000 7fe22d058740 1 --2- 172.21.15.139:0/1593307670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0be00010 0x561a0be1bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.804+0000 7fe22d058740 1 --2- 172.21.15.139:0/1593307670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0be03e80 0x561a0bdffad0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.804+0000 7fe22d058740 1 --2- 172.21.15.139:0/1593307670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0bdf8cc0 0x561a0be014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.804+0000 7fe22d058740 1 -- 172.21.15.139:0/1593307670 >> 172.21.15.139:0/1593307670 conn(0x561a0bdf5230 msgr2=0x561a0bdff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.808+0000 7fe22d058740 1 -- 172.21.15.139:0/1593307670 shutdown_connections 2024-09-16T01:58:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.811+0000 7fe22d058740 1 -- 172.21.15.139:0/1593307670 wait complete. 2024-09-16T01:58:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.811+0000 7fe22d058740 1 -- start start 2024-09-16T01:58:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.811+0000 7fe22d058740 1 Processor -- start 2024-09-16T01:58:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.814+0000 7fe22d058740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0bdf8cc0 0x561a0be050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.814+0000 7fe22d058740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0be00010 0x561a0be05600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.814+0000 7fe22d058740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0be03e80 0x561a0be05b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.814+0000 7fe22d058740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe2040119a0 con 0x561a0be03e80 2024-09-16T01:58:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.814+0000 7fe22d058740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe204011210 con 0x561a0bdf8cc0 2024-09-16T01:58:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.814+0000 7fe22d058740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe20400e2d0 con 0x561a0be00010 2024-09-16T01:58:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.814+0000 7fe22d058740 10 client.0 _ll_register_callbacks cb 0x7fe2140093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.815+0000 7fe2237fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0bdf8cc0 0x561a0be050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.815+0000 7fe2237fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0bdf8cc0 0x561a0be050c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:58272/0 (socket says 172.21.15.139:58272) 2024-09-16T01:58:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.815+0000 7fe2237fe640 1 -- 172.21.15.139:0/1696757881 learned_addr learned my addr 172.21.15.139:0/1696757881 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.816+0000 7fe2237fe640 1 -- 172.21.15.139:0/1696757881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0be00010 msgr2=0x561a0be05600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:58:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.816+0000 7fe2237fe640 1 --2- 172.21.15.139:0/1696757881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0be00010 0x561a0be05600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.816+0000 7fe2237fe640 1 -- 172.21.15.139:0/1696757881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0be03e80 msgr2=0x561a0be05b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:58:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.816+0000 7fe2237fe640 1 --2- 172.21.15.139:0/1696757881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0be03e80 0x561a0be05b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.816+0000 7fe2237fe640 1 -- 172.21.15.139:0/1696757881 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0be0a720 con 0x561a0bdf8cc0 2024-09-16T01:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.816+0000 7fe2237fe640 1 --2- 172.21.15.139:0/1696757881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0bdf8cc0 0x561a0be050c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe2140989d0 tx=0x7fe21409c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bd04ca8e1716481 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.820+0000 7fe202ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.820+0000 7fe202ffd640 1 -- 172.21.15.139:0/1696757881 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214010e60 con 0x561a0bdf8cc0 2024-09-16T01:58:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.820+0000 7fe202ffd640 1 -- 172.21.15.139:0/1696757881 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9774+0+0 (secure 0 0 0) 0x7fe214098be0 con 0x561a0bdf8cc0 2024-09-16T01:58:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.821+0000 7fe22d058740 1 -- 172.21.15.139:0/1696757881 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe21c009780 con 0x561a0bdf8cc0 2024-09-16T01:58:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.821+0000 7fe202ffd640 1 -- 172.21.15.139:0/1696757881 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fe2140a0920 con 0x561a0bdf8cc0 2024-09-16T01:58:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.822+0000 7fe22d058740 10 client.26774 fetch_fsmap learned FSMap version 77 2024-09-16T01:58:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.822+0000 7fe22d058740 1 -- 172.21.15.139:0/1696757881 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x561a0be7ef40 con 0x561a0bdf8cc0 2024-09-16T01:58:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.825+0000 7fe202ffd640 1 -- 172.21.15.139:0/1696757881 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe21c009780 con 0x561a0bdf8cc0 2024-09-16T01:58:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.825+0000 7fe202ffd640 1 -- 172.21.15.139:0/1696757881 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 77) ==== 8450+0+0 (secure 0 0 0) 0x7fe2140bc430 con 0x561a0bdf8cc0 2024-09-16T01:58:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.826+0000 7fe22d058740 10 client.26774 fetch_fsmap finished waiting for FSMap version 77 2024-09-16T01:58:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.826+0000 7fe22d058740 10 client.26774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-16T01:58:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.826+0000 7fe22d058740 20 client.26774 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.826+0000 7fe22d058740 1 --2- 172.21.15.139:0/1696757881 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a0be7f700 0x561a0be81b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.826+0000 7fe22d058740 4 client.26774 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-16T01:58:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.826+0000 7fe22d058740 1 -- 172.21.15.139:0/1696757881 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a0bd64d70 con 0x561a0be7f700 2024-09-16T01:58:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.828+0000 7fe222ffd640 1 --2- 172.21.15.139:0/1696757881 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a0be7f700 0x561a0be81b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.831+0000 7fe222ffd640 1 --2- 172.21.15.139:0/1696757881 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a0be7f700 0x561a0be81b20 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=359bbe3bb15e6ba2 server_cookie=dff19fa1608b3f62 in_seq=0 out_seq=0 2024-09-16T01:58:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.832+0000 7fe202ffd640 10 client.26774 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.949+0000 7fe202ffd640 1 -- 172.21.15.139:0/1696757881 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a0bd64d70 con 0x561a0be7f700 2024-09-16T01:58:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.949+0000 7fe202ffd640 10 client.26774 handle_command_reply: tid=0 2024-09-16T01:58:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.949+0000 7fe22d058740 1 client.26774 shutdown 2024-09-16T01:58:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.950+0000 7fe22d058740 1 -- 172.21.15.139:0/1696757881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0bdf8cc0 msgr2=0x561a0be050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.950+0000 7fe22d058740 1 --2- 172.21.15.139:0/1696757881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0bdf8cc0 0x561a0be050c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe2140989d0 tx=0x7fe21409c410 comp rx=0 tx=0).stop 2024-09-16T01:58:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.950+0000 7fe22d058740 1 -- 172.21.15.139:0/1696757881 shutdown_connections 2024-09-16T01:58:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.950+0000 7fe22d058740 1 --2- 172.21.15.139:0/1696757881 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x561a0be7f700 0x561a0be81b20 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.950+0000 7fe22d058740 1 --2- 172.21.15.139:0/1696757881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a0be03e80 0x561a0be05b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.950+0000 7fe22d058740 1 --2- 172.21.15.139:0/1696757881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561a0be00010 0x561a0be05600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.950+0000 7fe202ffd640 0 client.26774 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.950+0000 7fe22d058740 1 --2- 172.21.15.139:0/1696757881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a0bdf8cc0 0x561a0be050c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.950+0000 7fe22d058740 1 -- 172.21.15.139:0/1696757881 >> 172.21.15.139:0/1696757881 conn(0x561a0bdf5230 msgr2=0x561a0bdf3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.950+0000 7fe22d058740 1 -- 172.21.15.139:0/1696757881 shutdown_connections 2024-09-16T01:58:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.950+0000 7fe22d058740 1 -- 172.21.15.139:0/1696757881 wait complete. 2024-09-16T01:58:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.950+0000 7fe22d058740 20 client.26774 trim_cache size 0 max 16384 2024-09-16T01:58:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:{ 2024-09-16T01:58:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "message": "", 2024-09-16T01:58:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout: "return_code": 0 2024-09-16T01:58:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stdout:} 2024-09-16T01:58:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.951+0000 7fe22a1e4640 1 -- 172.21.15.139:0/930799715 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe2100a3580 msgr2=0x7fe2100a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.951+0000 7fe22a1e4640 1 --2- 172.21.15.139:0/930799715 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe2100a3580 0x7fe2100a5a40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe20c035f40 tx=0x7fe20c030040 comp rx=0 tx=0).stop 2024-09-16T01:58:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.951+0000 7fe22a1e4640 1 -- 172.21.15.139:0/930799715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2241091f0 msgr2=0x7fe2241a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.951+0000 7fe22a1e4640 1 --2- 172.21.15.139:0/930799715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2241091f0 0x7fe2241a7110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe21400ebc0 tx=0x7fe21400c6a0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.951+0000 7fe22a1e4640 1 -- 172.21.15.139:0/930799715 shutdown_connections 2024-09-16T01:58:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.951+0000 7fe22a1e4640 1 --2- 172.21.15.139:0/930799715 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe2100a3580 0x7fe2100a5a40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.951+0000 7fe22a1e4640 1 --2- 172.21.15.139:0/930799715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe22411b140 0x7fe2241ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.951+0000 7fe22a1e4640 1 --2- 172.21.15.139:0/930799715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2241095d0 0x7fe2241a7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.951+0000 7fe22a1e4640 1 --2- 172.21.15.139:0/930799715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2241091f0 0x7fe2241a7110 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.951+0000 7fe22a1e4640 1 -- 172.21.15.139:0/930799715 >> 172.21.15.139:0/930799715 conn(0x7fe22406cc60 msgr2=0x7fe22411b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.951+0000 7fe22a1e4640 1 -- 172.21.15.139:0/930799715 shutdown_connections 2024-09-16T01:58:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi139.front.sepia.ceph.com.smithi139.stderr:2024-09-16T01:58:50.951+0000 7fe22a1e4640 1 -- 172.21.15.139:0/930799715 wait complete. 2024-09-16T01:58:51.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.162+0000 7fd66fc57640 1 -- 172.21.15.139:0/2742871309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd668109830 msgr2=0x7fd668109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:51.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.162+0000 7fd66fc57640 1 --2- 172.21.15.139:0/2742871309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd668109830 0x7fd668109c90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd66401a910 tx=0x7fd66403b470 comp rx=0 tx=0).stop 2024-09-16T01:58:51.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.162+0000 7fd66fc57640 1 -- 172.21.15.139:0/2742871309 shutdown_connections 2024-09-16T01:58:51.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.162+0000 7fd66fc57640 1 --2- 172.21.15.139:0/2742871309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd66811b140 0x7fd66811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.162+0000 7fd66fc57640 1 --2- 172.21.15.139:0/2742871309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd668109830 0x7fd668109c90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.162+0000 7fd66fc57640 1 --2- 172.21.15.139:0/2742871309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd668109450 0x7fd668110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.162+0000 7fd66fc57640 1 -- 172.21.15.139:0/2742871309 >> 172.21.15.139:0/2742871309 conn(0x7fd66806e140 msgr2=0x7fd66806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:51.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.162+0000 7fd66fc57640 1 -- 172.21.15.139:0/2742871309 shutdown_connections 2024-09-16T01:58:51.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.166+0000 7fd66fc57640 1 -- 172.21.15.139:0/2742871309 wait complete. 2024-09-16T01:58:51.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.166+0000 7fd66fc57640 1 Processor -- start 2024-09-16T01:58:51.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.167+0000 7fd66fc57640 1 -- start start 2024-09-16T01:58:51.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.167+0000 7fd66fc57640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd668109450 0x7fd66811a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.167+0000 7fd66fc57640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd668109830 0x7fd6681168f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.167+0000 7fd66fc57640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd66811b140 0x7fd668116e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.167+0000 7fd66fc57640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd668113340 con 0x7fd66811b140 2024-09-16T01:58:51.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.167+0000 7fd66fc57640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd6681131c0 con 0x7fd668109830 2024-09-16T01:58:51.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.167+0000 7fd66d1f2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd668109830 0x7fd6681168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:51.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.167+0000 7fd66d1f2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd668109830 0x7fd6681168f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:58290/0 (socket says 172.21.15.139:58290) 2024-09-16T01:58:51.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.167+0000 7fd66d1f2640 1 -- 172.21.15.139:0/1332308144 learned_addr learned my addr 172.21.15.139:0/1332308144 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:51.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.167+0000 7fd66fc57640 1 -- 172.21.15.139:0/1332308144 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd6681134c0 con 0x7fd668109450 2024-09-16T01:58:51.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.167+0000 7fd66d9f3640 1 --2- 172.21.15.139:0/1332308144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd668109450 0x7fd66811a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:51.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.168+0000 7fd66e1f4640 1 --2- 172.21.15.139:0/1332308144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd66811b140 0x7fd668116e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:51.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.168+0000 7fd66d9f3640 1 -- 172.21.15.139:0/1332308144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd668109830 msgr2=0x7fd6681168f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:51.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.168+0000 7fd66d9f3640 1 --2- 172.21.15.139:0/1332308144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd668109830 0x7fd6681168f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.168+0000 7fd66d9f3640 1 -- 172.21.15.139:0/1332308144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd66811b140 msgr2=0x7fd668116e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:51.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.168+0000 7fd66d9f3640 1 --2- 172.21.15.139:0/1332308144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd66811b140 0x7fd668116e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.168+0000 7fd66d9f3640 1 -- 172.21.15.139:0/1332308144 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd668117560 con 0x7fd668109450 2024-09-16T01:58:51.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.168+0000 7fd66d9f3640 1 --2- 172.21.15.139:0/1332308144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd668109450 0x7fd66811a850 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd658009fd0 tx=0x7fd65800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:51.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.174+0000 7fd65effd640 1 -- 172.21.15.139:0/1332308144 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658015020 con 0x7fd668109450 2024-09-16T01:58:51.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.174+0000 7fd65effd640 1 -- 172.21.15.139:0/1332308144 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd658011d10 con 0x7fd668109450 2024-09-16T01:58:51.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.174+0000 7fd65effd640 1 -- 172.21.15.139:0/1332308144 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65800f2e0 con 0x7fd668109450 2024-09-16T01:58:51.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.175+0000 7fd66fc57640 1 -- 172.21.15.139:0/1332308144 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6681ab170 con 0x7fd668109450 2024-09-16T01:58:51.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.175+0000 7fd66fc57640 1 -- 172.21.15.139:0/1332308144 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6681ab570 con 0x7fd668109450 2024-09-16T01:58:51.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.179+0000 7fd672030740 1 Processor -- start 2024-09-16T01:58:51.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.179+0000 7fd672030740 1 -- start start 2024-09-16T01:58:51.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.180+0000 7fd672030740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78a330 0x55afcb78a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.180+0000 7fd672030740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78ac50 0x55afcb794fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.180+0000 7fd672030740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb795510 0x55afcb797900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.180+0000 7fd672030740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55afcb6ef620 con 0x55afcb795510 2024-09-16T01:58:51.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.180+0000 7fd672030740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55afcb696e50 con 0x55afcb78ac50 2024-09-16T01:58:51.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.180+0000 7fd672030740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55afcb6eb5e0 con 0x55afcb78a330 2024-09-16T01:58:51.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.181+0000 7fd66e1f4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78a330 0x55afcb78a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:51.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.181+0000 7fd66e1f4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78a330 0x55afcb78a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:56342/0 (socket says 172.21.15.139:56342) 2024-09-16T01:58:51.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.182+0000 7fd66e1f4640 1 -- 172.21.15.139:0/3979181734 learned_addr learned my addr 172.21.15.139:0/3979181734 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:51.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.182+0000 7fd66d9f3640 1 --2- 172.21.15.139:0/3979181734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78ac50 0x55afcb794fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:51.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.182+0000 7fd66e1f4640 1 -- 172.21.15.139:0/3979181734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78ac50 msgr2=0x55afcb794fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:51.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.182+0000 7fd66e1f4640 1 --2- 172.21.15.139:0/3979181734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78ac50 0x55afcb794fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.182+0000 7fd66e1f4640 1 -- 172.21.15.139:0/3979181734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb795510 msgr2=0x55afcb797900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:51.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.182+0000 7fd66e1f4640 1 --2- 172.21.15.139:0/3979181734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb795510 0x55afcb797900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.182+0000 7fd66e1f4640 1 -- 172.21.15.139:0/3979181734 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afcb687d40 con 0x55afcb78a330 2024-09-16T01:58:51.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.185+0000 7fd66e1f4640 1 --2- 172.21.15.139:0/3979181734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78a330 0x55afcb78a710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd660009fb0 tx=0x7fd660032870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f7a091f2bf33a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:51.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.186+0000 7fd65effd640 1 -- 172.21.15.139:0/1332308144 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd658018020 con 0x7fd668109450 2024-09-16T01:58:51.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.186+0000 7fd6537fe640 1 -- 172.21.15.139:0/3979181734 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660003b70 con 0x55afcb78a330 2024-09-16T01:58:51.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.186+0000 7fd6537fe640 1 -- 172.21.15.139:0/3979181734 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd660003d10 con 0x55afcb78a330 2024-09-16T01:58:51.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.186+0000 7fd6537fe640 1 -- 172.21.15.139:0/3979181734 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660033a70 con 0x55afcb78a330 2024-09-16T01:58:51.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.187+0000 7fd65effd640 1 --2- 172.21.15.139:0/1332308144 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd6540a35c0 0x7fd6540a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.187+0000 7fd65effd640 1 -- 172.21.15.139:0/1332308144 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7fd6580cf900 con 0x7fd668109450 2024-09-16T01:58:51.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.189+0000 7fd672030740 1 -- 172.21.15.139:0/3979181734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78a330 msgr2=0x55afcb78a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:51.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.189+0000 7fd672030740 1 --2- 172.21.15.139:0/3979181734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78a330 0x55afcb78a710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd660009fb0 tx=0x7fd660032870 comp rx=0 tx=0).stop 2024-09-16T01:58:51.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.189+0000 7fd672030740 1 -- 172.21.15.139:0/3979181734 shutdown_connections 2024-09-16T01:58:51.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.189+0000 7fd672030740 1 --2- 172.21.15.139:0/3979181734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb795510 0x55afcb797900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.189+0000 7fd672030740 1 --2- 172.21.15.139:0/3979181734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78ac50 0x55afcb794fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.189+0000 7fd672030740 1 --2- 172.21.15.139:0/3979181734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78a330 0x55afcb78a710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.189+0000 7fd672030740 1 -- 172.21.15.139:0/3979181734 >> 172.21.15.139:0/3979181734 conn(0x55afcb7868c0 msgr2=0x55afcb786ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:51.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.190+0000 7fd66d9f3640 1 --2- 172.21.15.139:0/1332308144 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd6540a35c0 0x7fd6540a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:51.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.190+0000 7fd66d9f3640 1 --2- 172.21.15.139:0/1332308144 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd6540a35c0 0x7fd6540a5a80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55afcb787720 tx=0x7fd658010990 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:51.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.202+0000 7fd672030740 1 -- 172.21.15.139:0/3979181734 shutdown_connections 2024-09-16T01:58:51.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.205+0000 7fd672030740 1 -- 172.21.15.139:0/3979181734 wait complete. 2024-09-16T01:58:51.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.206+0000 7fd672030740 1 -- start start 2024-09-16T01:58:51.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.206+0000 7fd672030740 1 Processor -- start 2024-09-16T01:58:51.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.211+0000 7fd672030740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78a330 0x55afcb794fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.211+0000 7fd672030740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb78ac50 0x55afcb792280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.211+0000 7fd672030740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb795510 0x55afcb792a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.211+0000 7fd672030740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55afcb797ff0 con 0x55afcb78ac50 2024-09-16T01:58:51.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.211+0000 7fd672030740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55afcb677f10 con 0x55afcb78a330 2024-09-16T01:58:51.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.211+0000 7fd672030740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55afcb798170 con 0x55afcb795510 2024-09-16T01:58:51.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.211+0000 7fd672030740 10 client.0 _ll_register_callbacks cb 0x55afcb69bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:51.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.212+0000 7fd66e1f4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78a330 0x55afcb794fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:51.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.212+0000 7fd66e1f4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb795510 0x55afcb792a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:51.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.212+0000 7fd66e1f4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78a330 0x55afcb794fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:58320/0 (socket says 172.21.15.139:58320) 2024-09-16T01:58:51.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.212+0000 7fd66e1f4640 1 -- 172.21.15.139:0/4037274549 learned_addr learned my addr 172.21.15.139:0/4037274549 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:51.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.212+0000 7fd66e1f4640 1 -- 172.21.15.139:0/4037274549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb795510 msgr2=0x55afcb792a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:51.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.212+0000 7fd66e1f4640 1 --2- 172.21.15.139:0/4037274549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb795510 0x55afcb792a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.212+0000 7fd66e1f4640 1 -- 172.21.15.139:0/4037274549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb78ac50 msgr2=0x55afcb792280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:51.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.212+0000 7fd66e1f4640 1 --2- 172.21.15.139:0/4037274549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb78ac50 0x55afcb792280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.212+0000 7fd66e1f4640 1 -- 172.21.15.139:0/4037274549 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afcb3fe580 con 0x55afcb78a330 2024-09-16T01:58:51.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.213+0000 7fd66e1f4640 1 --2- 172.21.15.139:0/4037274549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78a330 0x55afcb794fd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd660032d70 tx=0x7fd660033650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d82ef7ad8654cee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:51.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.213+0000 7fd6517fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:58:51.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.213+0000 7fd6517fa640 1 -- 172.21.15.139:0/4037274549 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660045030 con 0x55afcb78a330 2024-09-16T01:58:51.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.213+0000 7fd6517fa640 1 -- 172.21.15.139:0/4037274549 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7fd660033c20 con 0x55afcb78a330 2024-09-16T01:58:51.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.220+0000 7fd672030740 1 -- 172.21.15.139:0/4037274549 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd660003d10 con 0x55afcb78a330 2024-09-16T01:58:51.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.224+0000 7fd6517fa640 1 -- 172.21.15.139:0/4037274549 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fd660053050 con 0x55afcb78a330 2024-09-16T01:58:51.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.224+0000 7fd672030740 10 client.26786 fetch_fsmap learned FSMap version 77 2024-09-16T01:58:51.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.224+0000 7fd672030740 1 -- 172.21.15.139:0/4037274549 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55afcb4c8960 con 0x55afcb78a330 2024-09-16T01:58:51.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.225+0000 7fd6517fa640 1 -- 172.21.15.139:0/4037274549 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd660003d10 con 0x55afcb78a330 2024-09-16T01:58:51.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.225+0000 7fd6517fa640 1 -- 172.21.15.139:0/4037274549 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 77) ==== 8450+0+0 (secure 0 0 0) 0x7fd66004e440 con 0x55afcb78a330 2024-09-16T01:58:51.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.225+0000 7fd672030740 10 client.26786 fetch_fsmap finished waiting for FSMap version 77 2024-09-16T01:58:51.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.225+0000 7fd672030740 10 client.26786 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:58:51.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.225+0000 7fd672030740 20 client.26786 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:51.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.225+0000 7fd672030740 1 --2- 172.21.15.139:0/4037274549 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55afcb7a4d00 0x55afcb7a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.225+0000 7fd672030740 4 client.26786 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:58:51.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.225+0000 7fd672030740 1 -- 172.21.15.139:0/4037274549 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afcb6fcb90 con 0x55afcb7a4d00 2024-09-16T01:58:51.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.228+0000 7fd66d1f2640 1 --2- 172.21.15.139:0/4037274549 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55afcb7a4d00 0x55afcb7a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:51.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.229+0000 7fd66d1f2640 1 --2- 172.21.15.139:0/4037274549 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55afcb7a4d00 0x55afcb7a70f0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=359b98ab1e1a5cb7 server_cookie=245043ba11dc6a50 in_seq=0 out_seq=0 2024-09-16T01:58:51.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.229+0000 7fd6517fa640 10 client.26786 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:51.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.229+0000 7fd6517fa640 1 -- 172.21.15.139:0/4037274549 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55afcb6fcb90 con 0x55afcb7a4d00 2024-09-16T01:58:51.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.229+0000 7fd6517fa640 10 client.26786 handle_command_reply: tid=0 2024-09-16T01:58:51.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.230+0000 7fd672030740 1 client.26786 shutdown 2024-09-16T01:58:51.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.234+0000 7fd672030740 1 -- 172.21.15.139:0/4037274549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78a330 msgr2=0x55afcb794fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:51.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.234+0000 7fd672030740 1 --2- 172.21.15.139:0/4037274549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78a330 0x55afcb794fd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd660032d70 tx=0x7fd660033650 comp rx=0 tx=0).stop 2024-09-16T01:58:51.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.237+0000 7fd672030740 1 -- 172.21.15.139:0/4037274549 shutdown_connections 2024-09-16T01:58:51.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.237+0000 7fd672030740 1 --2- 172.21.15.139:0/4037274549 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55afcb7a4d00 0x55afcb7a70f0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.237+0000 7fd672030740 1 --2- 172.21.15.139:0/4037274549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb795510 0x55afcb792a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.237+0000 7fd672030740 1 --2- 172.21.15.139:0/4037274549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb78ac50 0x55afcb792280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.237+0000 7fd672030740 1 --2- 172.21.15.139:0/4037274549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78a330 0x55afcb794fd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.237+0000 7fd672030740 1 -- 172.21.15.139:0/4037274549 >> 172.21.15.139:0/4037274549 conn(0x55afcb7868c0 msgr2=0x55afcb785b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:51.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.237+0000 7fd6517fa640 0 client.26786 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:51.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.237+0000 7fd672030740 1 -- 172.21.15.139:0/4037274549 shutdown_connections 2024-09-16T01:58:51.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.237+0000 7fd672030740 1 -- 172.21.15.139:0/4037274549 wait complete. 2024-09-16T01:58:51.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.237+0000 7fd672030740 20 client.26786 trim_cache size 0 max 16384 2024-09-16T01:58:51.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.269+0000 7fd672030740 1 Processor -- start 2024-09-16T01:58:51.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.269+0000 7fd672030740 1 -- start start 2024-09-16T01:58:51.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.270+0000 7fd672030740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78a330 0x55afcb78cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.270+0000 7fd672030740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78ac50 0x55afcb785a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.270+0000 7fd672030740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb795510 0x55afcb7ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.270+0000 7fd672030740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd66807e7a0 con 0x55afcb795510 2024-09-16T01:58:51.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.270+0000 7fd672030740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd6680805e0 con 0x55afcb78a330 2024-09-16T01:58:51.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.270+0000 7fd672030740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd66807ef30 con 0x55afcb78ac50 2024-09-16T01:58:51.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.270+0000 7fd66e1f4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78a330 0x55afcb78cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:51.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.270+0000 7fd66e1f4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78a330 0x55afcb78cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:58330/0 (socket says 172.21.15.139:58330) 2024-09-16T01:58:51.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.270+0000 7fd66e1f4640 1 -- 172.21.15.139:0/2834009685 learned_addr learned my addr 172.21.15.139:0/2834009685 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:51.274 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:51 smithi139 ceph-mon[31175]: pgmap v587: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.4 KiB/s wr, 155 op/s 2024-09-16T01:58:51.274 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:51 smithi139 ceph-mon[31175]: osdmap e118: 12 total, 12 up, 12 in 2024-09-16T01:58:51.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.271+0000 7fd66e1f4640 1 --2- 172.21.15.139:0/2834009685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb795510 0x55afcb7ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:51.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.271+0000 7fd66e1f4640 1 -- 172.21.15.139:0/2834009685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78ac50 msgr2=0x55afcb785a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T01:58:51.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.271+0000 7fd66e1f4640 1 --2- 172.21.15.139:0/2834009685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78ac50 0x55afcb785a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.271+0000 7fd66e1f4640 1 -- 172.21.15.139:0/2834009685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb795510 msgr2=0x55afcb7ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:51.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.271+0000 7fd66e1f4640 1 --2- 172.21.15.139:0/2834009685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb795510 0x55afcb7ac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.271+0000 7fd66e1f4640 1 -- 172.21.15.139:0/2834009685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afcb792d80 con 0x55afcb78a330 2024-09-16T01:58:51.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.272+0000 7fd66e1f4640 1 --2- 172.21.15.139:0/2834009685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78a330 0x55afcb78cd90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd660004df0 tx=0x7fd660005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bdecef31250d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:51.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.272+0000 7fd652ffd640 1 -- 172.21.15.139:0/2834009685 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660045040 con 0x55afcb78a330 2024-09-16T01:58:51.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.277+0000 7fd652ffd640 1 -- 172.21.15.139:0/2834009685 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd66004ebb0 con 0x55afcb78a330 2024-09-16T01:58:51.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.278+0000 7fd672030740 1 -- 172.21.15.139:0/2834009685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78a330 msgr2=0x55afcb78cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:51.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.278+0000 7fd672030740 1 --2- 172.21.15.139:0/2834009685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78a330 0x55afcb78cd90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd660004df0 tx=0x7fd660005bd0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.278+0000 7fd672030740 1 -- 172.21.15.139:0/2834009685 shutdown_connections 2024-09-16T01:58:51.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.278+0000 7fd672030740 1 --2- 172.21.15.139:0/2834009685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb795510 0x55afcb7ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.278+0000 7fd672030740 1 --2- 172.21.15.139:0/2834009685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78ac50 0x55afcb785a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.278+0000 7fd672030740 1 --2- 172.21.15.139:0/2834009685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78a330 0x55afcb78cd90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.278+0000 7fd672030740 1 -- 172.21.15.139:0/2834009685 >> 172.21.15.139:0/2834009685 conn(0x55afcb7868c0 msgr2=0x55afcb785360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:51.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.282+0000 7fd672030740 1 -- 172.21.15.139:0/2834009685 shutdown_connections 2024-09-16T01:58:51.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.282+0000 7fd672030740 1 -- 172.21.15.139:0/2834009685 wait complete. 2024-09-16T01:58:51.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.282+0000 7fd672030740 1 -- start start 2024-09-16T01:58:51.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.282+0000 7fd672030740 1 Processor -- start 2024-09-16T01:58:51.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.283+0000 7fd672030740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78a330 0x55afcb7990c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.283+0000 7fd672030740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78ac50 0x55afcb799600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.283+0000 7fd672030740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb795510 0x55afcb799b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.283+0000 7fd672030740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd66807fe50 con 0x55afcb795510 2024-09-16T01:58:51.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.283+0000 7fd672030740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd66807f6c0 con 0x55afcb78ac50 2024-09-16T01:58:51.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.283+0000 7fd672030740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd66807c780 con 0x55afcb78a330 2024-09-16T01:58:51.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.283+0000 7fd672030740 10 client.0 _ll_register_callbacks cb 0x7fd660004e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:51.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.283+0000 7fd66e1f4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78a330 0x55afcb7990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:51.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.283+0000 7fd66e1f4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78a330 0x55afcb7990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:56364/0 (socket says 172.21.15.139:56364) 2024-09-16T01:58:51.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.283+0000 7fd66e1f4640 1 -- 172.21.15.139:0/1298449565 learned_addr learned my addr 172.21.15.139:0/1298449565 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:51.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.284+0000 7fd66e1f4640 1 -- 172.21.15.139:0/1298449565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78ac50 msgr2=0x55afcb799600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:51.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.284+0000 7fd66e1f4640 1 --2- 172.21.15.139:0/1298449565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78ac50 0x55afcb799600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.284+0000 7fd66e1f4640 1 -- 172.21.15.139:0/1298449565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb795510 msgr2=0x55afcb799b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:51.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.284+0000 7fd66e1f4640 1 --2- 172.21.15.139:0/1298449565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb795510 0x55afcb799b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.284+0000 7fd66e1f4640 1 -- 172.21.15.139:0/1298449565 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afcb760080 con 0x55afcb78a330 2024-09-16T01:58:51.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.284+0000 7fd66e1f4640 1 --2- 172.21.15.139:0/1298449565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78a330 0x55afcb7990c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd660009fb0 tx=0x7fd66004ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd62cfaf8eda8c74 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:51.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.284+0000 7fd650ff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-16T01:58:51.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.284+0000 7fd650ff9640 1 -- 172.21.15.139:0/1298449565 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660033580 con 0x55afcb78a330 2024-09-16T01:58:51.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.284+0000 7fd650ff9640 1 -- 172.21.15.139:0/1298449565 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7fd66003d630 con 0x55afcb78a330 2024-09-16T01:58:51.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.285+0000 7fd672030740 1 -- 172.21.15.139:0/1298449565 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd66004ebb0 con 0x55afcb78a330 2024-09-16T01:58:51.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.285+0000 7fd650ff9640 1 -- 172.21.15.139:0/1298449565 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fd66004e440 con 0x55afcb78a330 2024-09-16T01:58:51.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.285+0000 7fd672030740 10 client.26776 fetch_fsmap learned FSMap version 77 2024-09-16T01:58:51.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.285+0000 7fd672030740 1 -- 172.21.15.139:0/1298449565 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55afcb80f870 con 0x55afcb78a330 2024-09-16T01:58:51.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.287+0000 7fd650ff9640 1 -- 172.21.15.139:0/1298449565 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd66004ebb0 con 0x55afcb78a330 2024-09-16T01:58:51.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.287+0000 7fd650ff9640 1 -- 172.21.15.139:0/1298449565 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 77) ==== 8450+0+0 (secure 0 0 0) 0x7fd66007ab00 con 0x55afcb78a330 2024-09-16T01:58:51.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.289+0000 7fd672030740 10 client.26776 fetch_fsmap finished waiting for FSMap version 77 2024-09-16T01:58:51.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.289+0000 7fd672030740 10 client.26776 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:58:51.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.289+0000 7fd672030740 20 client.26776 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:51.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.289+0000 7fd672030740 1 --2- 172.21.15.139:0/1298449565 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55afcb80fe60 0x55afcb8122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:51.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.289+0000 7fd672030740 4 client.26776 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "72a1fc77", "release": true, "await": true}] 2024-09-16T01:58:51.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.289+0000 7fd672030740 1 -- 172.21.15.139:0/1298449565 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "set_id": "72a1fc77", "release": true, "await": true}) -- 0x55afcb6f70a0 con 0x55afcb80fe60 2024-09-16T01:58:51.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.290+0000 7fd66d1f2640 1 --2- 172.21.15.139:0/1298449565 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55afcb80fe60 0x55afcb8122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:51.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.302+0000 7fd66d1f2640 1 --2- 172.21.15.139:0/1298449565 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55afcb80fe60 0x55afcb8122a0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86ed9921160c7c7c server_cookie=dba85c12a5cbc84c in_seq=0 out_seq=0 2024-09-16T01:58:51.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.302+0000 7fd650ff9640 10 client.26776 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:51.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.310+0000 7fd650ff9640 1 -- 172.21.15.139:0/1298449565 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55afcb6f70a0 con 0x55afcb80fe60 2024-09-16T01:58:51.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.310+0000 7fd650ff9640 10 client.26776 handle_command_reply: tid=0 2024-09-16T01:58:51.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.310+0000 7fd672030740 1 client.26776 shutdown 2024-09-16T01:58:51.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.312+0000 7fd672030740 1 -- 172.21.15.139:0/1298449565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78a330 msgr2=0x55afcb7990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:51.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.312+0000 7fd672030740 1 --2- 172.21.15.139:0/1298449565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78a330 0x55afcb7990c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd660009fb0 tx=0x7fd66004ed60 comp rx=0 tx=0).stop 2024-09-16T01:58:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.312+0000 7fd672030740 1 -- 172.21.15.139:0/1298449565 shutdown_connections 2024-09-16T01:58:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.312+0000 7fd672030740 1 --2- 172.21.15.139:0/1298449565 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x55afcb80fe60 0x55afcb8122a0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.312+0000 7fd672030740 1 --2- 172.21.15.139:0/1298449565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55afcb795510 0x55afcb799b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.312+0000 7fd672030740 1 --2- 172.21.15.139:0/1298449565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55afcb78ac50 0x55afcb799600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.312+0000 7fd672030740 1 --2- 172.21.15.139:0/1298449565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55afcb78a330 0x55afcb7990c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.312+0000 7fd672030740 1 -- 172.21.15.139:0/1298449565 >> 172.21.15.139:0/1298449565 conn(0x55afcb7868c0 msgr2=0x55afcb785b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.313+0000 7fd650ff9640 0 client.26776 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:51.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.315+0000 7fd672030740 1 -- 172.21.15.139:0/1298449565 shutdown_connections 2024-09-16T01:58:51.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.317+0000 7fd672030740 1 -- 172.21.15.139:0/1298449565 wait complete. 2024-09-16T01:58:51.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.317+0000 7fd672030740 20 client.26776 trim_cache size 0 max 16384 2024-09-16T01:58:51.319 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:58:51.319 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 77, 2024-09-16T01:58:51.319 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:58:51.319 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 68, 2024-09-16T01:58:51.319 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": { 2024-09-16T01:58:51.319 INFO:teuthology.orchestra.run.smithi139.stdout: "72a1fc77": { 2024-09-16T01:58:51.320 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 68, 2024-09-16T01:58:51.320 INFO:teuthology.orchestra.run.smithi139.stdout: "age_ref": 0.0, 2024-09-16T01:58:51.320 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:58:51.320 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:58:51.320 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:58:51.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:58:51.320 INFO:teuthology.orchestra.run.smithi139.stdout: "timeout": 90.0, 2024-09-16T01:58:51.320 INFO:teuthology.orchestra.run.smithi139.stdout: "expiration": 142.9, 2024-09-16T01:58:51.320 INFO:teuthology.orchestra.run.smithi139.stdout: "members": { 2024-09-16T01:58:51.320 INFO:teuthology.orchestra.run.smithi139.stdout: "file:/": { 2024-09-16T01:58:51.320 INFO:teuthology.orchestra.run.smithi139.stdout: "excluded": false, 2024-09-16T01:58:51.320 INFO:teuthology.orchestra.run.smithi139.stdout: "state": { 2024-09-16T01:58:51.320 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RELEASED", 2024-09-16T01:58:51.321 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0 2024-09-16T01:58:51.321 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:51.321 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:51.321 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:51.321 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:51.321 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:58:51.321 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:58:51.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.318+0000 7fd65cdea640 1 -- 172.21.15.139:0/1332308144 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd6540a35c0 msgr2=0x7fd6540a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:51.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.318+0000 7fd65cdea640 1 --2- 172.21.15.139:0/1332308144 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd6540a35c0 0x7fd6540a5a80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55afcb787720 tx=0x7fd658010990 comp rx=0 tx=0).stop 2024-09-16T01:58:51.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.318+0000 7fd65cdea640 1 -- 172.21.15.139:0/1332308144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd668109450 msgr2=0x7fd66811a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:51.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.318+0000 7fd65cdea640 1 --2- 172.21.15.139:0/1332308144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd668109450 0x7fd66811a850 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd658009fd0 tx=0x7fd65800b870 comp rx=0 tx=0).stop 2024-09-16T01:58:51.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.318+0000 7fd65cdea640 1 -- 172.21.15.139:0/1332308144 shutdown_connections 2024-09-16T01:58:51.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.318+0000 7fd65cdea640 1 --2- 172.21.15.139:0/1332308144 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd6540a35c0 0x7fd6540a5a80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.318+0000 7fd65cdea640 1 --2- 172.21.15.139:0/1332308144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd66811b140 0x7fd668116e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.318+0000 7fd65cdea640 1 --2- 172.21.15.139:0/1332308144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd668109830 0x7fd6681168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.318+0000 7fd65cdea640 1 --2- 172.21.15.139:0/1332308144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd668109450 0x7fd66811a850 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:51.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.318+0000 7fd65cdea640 1 -- 172.21.15.139:0/1332308144 >> 172.21.15.139:0/1332308144 conn(0x7fd66806e140 msgr2=0x7fd668070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:51.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.318+0000 7fd65cdea640 1 -- 172.21.15.139:0/1332308144 shutdown_connections 2024-09-16T01:58:51.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:51.318+0000 7fd65cdea640 1 -- 172.21.15.139:0/1332308144 wait complete. 2024-09-16T01:58:51.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:51 smithi176 ceph-mon[35324]: pgmap v587: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.4 KiB/s wr, 155 op/s 2024-09-16T01:58:51.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:51 smithi176 ceph-mon[35324]: osdmap e118: 12 total, 12 up, 12 in 2024-09-16T01:58:51.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:51 smithi164 ceph-mon[35121]: pgmap v587: 129 pgs: 21 active+clean+snaptrim, 64 active+clean+snaptrim_wait, 44 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.4 KiB/s wr, 155 op/s 2024-09-16T01:58:51.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:51 smithi164 ceph-mon[35121]: osdmap e118: 12 total, 12 up, 12 in 2024-09-16T01:58:51.684 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-09-16T01:58:51.684 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-16T01:58:52.241 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '72a1fc77', total seconds elapsed: 28.7 2024-09-16T01:58:52.575 DEBUG:teuthology.parallel:result is None 2024-09-16T01:58:52.576 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T01:58:52.703 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T01:58:52.703 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-16T01:58:52.713 INFO:tasks.quiescer:joining Quiescers 2024-09-16T01:58:52.713 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-16T01:58:52.714 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:58:53.252 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:58:53.409 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:53 smithi139 ceph-mon[31175]: pgmap v589: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 131 op/s 2024-09-16T01:58:53.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:53 smithi176 ceph-mon[35324]: pgmap v589: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 131 op/s 2024-09-16T01:58:53.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:53 smithi164 ceph-mon[35121]: pgmap v589: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 131 op/s 2024-09-16T01:58:54.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.324+0000 7f1611d9b640 1 --2- 172.21.15.139:0/3887401049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1614109830 0x7f1614109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:58:54.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.324+0000 7f1618c05640 1 -- 172.21.15.139:0/3887401049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1614109450 msgr2=0x7f1614110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:54.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.324+0000 7f1618c05640 1 --2- 172.21.15.139:0/3887401049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1614109450 0x7f1614110250 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f160c009900 tx=0x7f160c02a440 comp rx=0 tx=0).stop 2024-09-16T01:58:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.326+0000 7f1618c05640 1 -- 172.21.15.139:0/3887401049 shutdown_connections 2024-09-16T01:58:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.326+0000 7f1618c05640 1 --2- 172.21.15.139:0/3887401049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f161411b140 0x7f161411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.326+0000 7f1618c05640 1 --2- 172.21.15.139:0/3887401049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1614109830 0x7f1614109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.326+0000 7f1618c05640 1 --2- 172.21.15.139:0/3887401049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1614109450 0x7f1614110250 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.326+0000 7f1618c05640 1 -- 172.21.15.139:0/3887401049 >> 172.21.15.139:0/3887401049 conn(0x7f161406e140 msgr2=0x7f161406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.327+0000 7f1618c05640 1 -- 172.21.15.139:0/3887401049 shutdown_connections 2024-09-16T01:58:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.327+0000 7f1618c05640 1 -- 172.21.15.139:0/3887401049 wait complete. 2024-09-16T01:58:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.328+0000 7f1618c05640 1 Processor -- start 2024-09-16T01:58:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.328+0000 7f1618c05640 1 -- start start 2024-09-16T01:58:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.328+0000 7f1618c05640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1614109450 0x7f16141a7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.329+0000 7f1618c05640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1614109830 0x7f16141a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.329+0000 7f1618c05640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f161411b140 0x7f16141abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.329+0000 7f1618c05640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1614120eb0 con 0x7f1614109830 2024-09-16T01:58:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.329+0000 7f1618c05640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1614120d30 con 0x7f1614109450 2024-09-16T01:58:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.329+0000 7f1618c05640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1614121030 con 0x7f161411b140 2024-09-16T01:58:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.332+0000 7f161259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1614109450 0x7f16141a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.332+0000 7f1612d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f161411b140 0x7f16141abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.332+0000 7f1612d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f161411b140 0x7f16141abb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:44842/0 (socket says 172.21.15.139:44842) 2024-09-16T01:58:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.332+0000 7f1612d9d640 1 -- 172.21.15.139:0/1564353937 learned_addr learned my addr 172.21.15.139:0/1564353937 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.332+0000 7f1611d9b640 1 --2- 172.21.15.139:0/1564353937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1614109830 0x7f16141a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.337+0000 7f1612d9d640 1 -- 172.21.15.139:0/1564353937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1614109450 msgr2=0x7f16141a7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.337+0000 7f1612d9d640 1 --2- 172.21.15.139:0/1564353937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1614109450 0x7f16141a7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:54.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.337+0000 7f1612d9d640 1 -- 172.21.15.139:0/1564353937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1614109830 msgr2=0x7f16141a77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.337+0000 7f1612d9d640 1 --2- 172.21.15.139:0/1564353937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1614109830 0x7f16141a77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.337+0000 7f1612d9d640 1 -- 172.21.15.139:0/1564353937 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16141ac240 con 0x7f161411b140 2024-09-16T01:58:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.337+0000 7f1612d9d640 1 --2- 172.21.15.139:0/1564353937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f161411b140 0x7f16141abb60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f160400caf0 tx=0x7f160400cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.338+0000 7f160b7fe640 1 -- 172.21.15.139:0/1564353937 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1604016030 con 0x7f161411b140 2024-09-16T01:58:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.338+0000 7f1618c05640 1 -- 172.21.15.139:0/1564353937 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16141ac530 con 0x7f161411b140 2024-09-16T01:58:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.338+0000 7f1618c05640 1 -- 172.21.15.139:0/1564353937 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16141b3e10 con 0x7f161411b140 2024-09-16T01:58:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.338+0000 7f160b7fe640 1 -- 172.21.15.139:0/1564353937 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1604003970 con 0x7f161411b140 2024-09-16T01:58:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.338+0000 7f160b7fe640 1 -- 172.21.15.139:0/1564353937 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16040046c0 con 0x7f161411b140 2024-09-16T01:58:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.345+0000 7f160b7fe640 1 -- 172.21.15.139:0/1564353937 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1604019070 con 0x7f161411b140 2024-09-16T01:58:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.346+0000 7f160b7fe640 1 --2- 172.21.15.139:0/1564353937 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f16000a3500 0x7f16000a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.346+0000 7f161259c640 1 --2- 172.21.15.139:0/1564353937 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f16000a3500 0x7f16000a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:54.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.346+0000 7f160b7fe640 1 -- 172.21.15.139:0/1564353937 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7f16040cfa70 con 0x7f161411b140 2024-09-16T01:58:54.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.349+0000 7f1618c05640 1 -- 172.21.15.139:0/1564353937 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f4005290 con 0x7f161411b140 2024-09-16T01:58:54.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.353+0000 7f161259c640 1 --2- 172.21.15.139:0/1564353937 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f16000a3500 0x7f16000a59c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f160c02aa30 tx=0x7f160c02a940 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:54.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.358+0000 7f160b7fe640 1 -- 172.21.15.139:0/1564353937 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f160409c030 con 0x7f161411b140 2024-09-16T01:58:54.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.756+0000 7f1618c05640 1 -- 172.21.15.139:0/1564353937 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15f4005850 con 0x7f161411b140 2024-09-16T01:58:54.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.759+0000 7f160b7fe640 1 -- 172.21.15.139:0/1564353937 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+12031 (secure 0 0 0) 0x7f160409b930 con 0x7f161411b140 2024-09-16T01:58:54.761 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:58:54.761 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":77,"btime":"2024-09-16T01:58:24:183207+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":77,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:58:24.058642+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:58:54.761 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 77 2024-09-16T01:58:54.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.763+0000 7f1618c05640 1 -- 172.21.15.139:0/1564353937 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f16000a3500 msgr2=0x7f16000a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:54.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.763+0000 7f1618c05640 1 --2- 172.21.15.139:0/1564353937 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f16000a3500 0x7f16000a59c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f160c02aa30 tx=0x7f160c02a940 comp rx=0 tx=0).stop 2024-09-16T01:58:54.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.763+0000 7f1618c05640 1 -- 172.21.15.139:0/1564353937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f161411b140 msgr2=0x7f16141abb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:54.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.763+0000 7f1618c05640 1 --2- 172.21.15.139:0/1564353937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f161411b140 0x7f16141abb60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f160400caf0 tx=0x7f160400cef0 comp rx=0 tx=0).stop 2024-09-16T01:58:54.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.763+0000 7f1618c05640 1 -- 172.21.15.139:0/1564353937 shutdown_connections 2024-09-16T01:58:54.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.763+0000 7f1618c05640 1 --2- 172.21.15.139:0/1564353937 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f16000a3500 0x7f16000a59c0 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f160c02aa30 tx=0x7f160c02a940 comp rx=0 tx=0).stop 2024-09-16T01:58:54.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.763+0000 7f1618c05640 1 --2- 172.21.15.139:0/1564353937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f161411b140 0x7f16141abb60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:54.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.763+0000 7f1618c05640 1 --2- 172.21.15.139:0/1564353937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1614109830 0x7f16141a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:54.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.763+0000 7f1618c05640 1 --2- 172.21.15.139:0/1564353937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1614109450 0x7f16141a7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:54.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.763+0000 7f1618c05640 1 -- 172.21.15.139:0/1564353937 >> 172.21.15.139:0/1564353937 conn(0x7f161406e140 msgr2=0x7f161411bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:54.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.764+0000 7f1618c05640 1 -- 172.21.15.139:0/1564353937 shutdown_connections 2024-09-16T01:58:54.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:54.766+0000 7f1618c05640 1 -- 172.21.15.139:0/1564353937 wait complete. 2024-09-16T01:58:55.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:55 smithi139 ceph-mon[31175]: pgmap v590: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 KiB/s wr, 155 op/s 2024-09-16T01:58:55.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:55 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1564353937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:58:55.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:55 smithi176 ceph-mon[35324]: pgmap v590: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 KiB/s wr, 155 op/s 2024-09-16T01:58:55.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:55 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1564353937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:58:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:55 smithi164 ceph-mon[35121]: pgmap v590: 129 pgs: 21 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 46 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 KiB/s wr, 155 op/s 2024-09-16T01:58:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:55 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1564353937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:58:55.552 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --cancel --all' 2024-09-16T01:58:55.552 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --cancel --all 2024-09-16T01:58:56.032 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:58:56.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.684+0000 7f9f5359e640 1 -- 172.21.15.139:0/1873305343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f54109bb0 msgr2=0x7f9f541120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:56.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.684+0000 7f9f5359e640 1 --2- 172.21.15.139:0/1873305343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f54109bb0 0x7f9f541120a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009900 tx=0x7f9f3c02a440 comp rx=0 tx=0).stop 2024-09-16T01:58:56.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.686+0000 7f9f5359e640 1 -- 172.21.15.139:0/1873305343 shutdown_connections 2024-09-16T01:58:56.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.686+0000 7f9f5359e640 1 --2- 172.21.15.139:0/1873305343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f54112670 0x7f9f54114aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.686+0000 7f9f5359e640 1 --2- 172.21.15.139:0/1873305343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f54109bb0 0x7f9f541120a0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.686+0000 7f9f5359e640 1 --2- 172.21.15.139:0/1873305343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f54109290 0x7f9f54109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.686+0000 7f9f5359e640 1 -- 172.21.15.139:0/1873305343 >> 172.21.15.139:0/1873305343 conn(0x7f9f54077960 msgr2=0x7f9f54079d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:56.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.688+0000 7f9f5359e640 1 -- 172.21.15.139:0/1873305343 shutdown_connections 2024-09-16T01:58:56.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.688+0000 7f9f5359e640 1 -- 172.21.15.139:0/1873305343 wait complete. 2024-09-16T01:58:56.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.689+0000 7f9f5359e640 1 Processor -- start 2024-09-16T01:58:56.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.689+0000 7f9f5359e640 1 -- start start 2024-09-16T01:58:56.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.689+0000 7f9f5359e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f54109290 0x7f9f54073940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.689+0000 7f9f5359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f54109bb0 0x7f9f54073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.689+0000 7f9f5359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f54112670 0x7f9f5406cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.689+0000 7f9f5359e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54118490 con 0x7f9f54112670 2024-09-16T01:58:56.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.689+0000 7f9f5359e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54118310 con 0x7f9f54109bb0 2024-09-16T01:58:56.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.689+0000 7f9f5359e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54118610 con 0x7f9f54109290 2024-09-16T01:58:56.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.689+0000 7f9f5259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f54109290 0x7f9f54073940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:56.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.689+0000 7f9f5259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f54109290 0x7f9f54073940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:44866/0 (socket says 172.21.15.139:44866) 2024-09-16T01:58:56.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.689+0000 7f9f5259c640 1 -- 172.21.15.139:0/4151327269 learned_addr learned my addr 172.21.15.139:0/4151327269 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:56.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.691+0000 7f9f51d9b640 1 --2- 172.21.15.139:0/4151327269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f54109bb0 0x7f9f54073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:56.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.692+0000 7f9f51d9b640 1 -- 172.21.15.139:0/4151327269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f54109290 msgr2=0x7f9f54073940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:56.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.692+0000 7f9f51d9b640 1 --2- 172.21.15.139:0/4151327269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f54109290 0x7f9f54073940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.692+0000 7f9f51d9b640 1 -- 172.21.15.139:0/4151327269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f54112670 msgr2=0x7f9f5406cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:56.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.692+0000 7f9f51d9b640 1 --2- 172.21.15.139:0/4151327269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f54112670 0x7f9f5406cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.692+0000 7f9f51d9b640 1 -- 172.21.15.139:0/4151327269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5406d470 con 0x7f9f54109bb0 2024-09-16T01:58:56.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.693+0000 7f9f51d9b640 1 --2- 172.21.15.139:0/4151327269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f54109bb0 0x7f9f54073e80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009850 tx=0x7f9f3c005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:56.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.693+0000 7f9f4b7fe640 1 -- 172.21.15.139:0/4151327269 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c040070 con 0x7f9f54109bb0 2024-09-16T01:58:56.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.693+0000 7f9f4b7fe640 1 -- 172.21.15.139:0/4151327269 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c005630 con 0x7f9f54109bb0 2024-09-16T01:58:56.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.693+0000 7f9f4b7fe640 1 -- 172.21.15.139:0/4151327269 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c03cb20 con 0x7f9f54109bb0 2024-09-16T01:58:56.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.693+0000 7f9f5359e640 1 -- 172.21.15.139:0/4151327269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5406d6a0 con 0x7f9f54109bb0 2024-09-16T01:58:56.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.693+0000 7f9f5359e640 1 -- 172.21.15.139:0/4151327269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f54071640 con 0x7f9f54109bb0 2024-09-16T01:58:56.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.698+0000 7f9f5b34b740 1 Processor -- start 2024-09-16T01:58:56.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.699+0000 7f9f4b7fe640 1 -- 172.21.15.139:0/4151327269 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9f3c03a040 con 0x7f9f54109bb0 2024-09-16T01:58:56.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.700+0000 7f9f5b34b740 1 -- start start 2024-09-16T01:58:56.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.700+0000 7f9f5b34b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d013ebb0 0x5583d013ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.700+0000 7f9f5b34b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013f4d0 0x5583d0149850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.700+0000 7f9f5b34b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d0149d90 0x5583d014c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.700+0000 7f9f5b34b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5583d00a3200 con 0x5583d013ebb0 2024-09-16T01:58:56.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.700+0000 7f9f5b34b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5583d0049dc0 con 0x5583d0149d90 2024-09-16T01:58:56.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.700+0000 7f9f5b34b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5583d009f1c0 con 0x5583d013f4d0 2024-09-16T01:58:56.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.700+0000 7f9f5259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013f4d0 0x5583d0149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:56.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.700+0000 7f9f5259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013f4d0 0x5583d0149850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:44874/0 (socket says 172.21.15.139:44874) 2024-09-16T01:58:56.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.700+0000 7f9f5259c640 1 -- 172.21.15.139:0/412671966 learned_addr learned my addr 172.21.15.139:0/412671966 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:56.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.701+0000 7f9f5259c640 1 -- 172.21.15.139:0/412671966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d0149d90 msgr2=0x5583d014c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:56.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.701+0000 7f9f5259c640 1 --2- 172.21.15.139:0/412671966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d0149d90 0x5583d014c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.701+0000 7f9f5259c640 1 -- 172.21.15.139:0/412671966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d013ebb0 msgr2=0x5583d013ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:56.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.701+0000 7f9f5259c640 1 --2- 172.21.15.139:0/412671966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d013ebb0 0x5583d013ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.701+0000 7f9f5259c640 1 -- 172.21.15.139:0/412671966 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583cfe14620 con 0x5583d013f4d0 2024-09-16T01:58:56.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.701+0000 7f9f4b7fe640 1 --2- 172.21.15.139:0/4151327269 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9f400a3540 0x7f9f400a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.701+0000 7f9f5259c640 1 --2- 172.21.15.139:0/412671966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013f4d0 0x5583d0149850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00bda0 tx=0x7f9f4c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b03b0d9d2911aca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:56.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.702+0000 7f9f4b7fe640 1 -- 172.21.15.139:0/4151327269 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7f9f3c0f0740 con 0x7f9f54109bb0 2024-09-16T01:58:56.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.702+0000 7f9f48ff9640 1 -- 172.21.15.139:0/412671966 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c01c680 con 0x5583d013f4d0 2024-09-16T01:58:56.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.702+0000 7f9f48ff9640 1 -- 172.21.15.139:0/412671966 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c01cd00 con 0x5583d013f4d0 2024-09-16T01:58:56.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.702+0000 7f9f48ff9640 1 -- 172.21.15.139:0/412671966 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c011d80 con 0x5583d013f4d0 2024-09-16T01:58:56.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.703+0000 7f9f5b34b740 1 -- 172.21.15.139:0/412671966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013f4d0 msgr2=0x5583d0149850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:56.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.703+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/412671966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013f4d0 0x5583d0149850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00bda0 tx=0x7f9f4c0107a0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.706+0000 7f9f52d9d640 1 --2- 172.21.15.139:0/4151327269 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9f400a3540 0x7f9f400a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:56.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.706+0000 7f9f5b34b740 1 -- 172.21.15.139:0/412671966 shutdown_connections 2024-09-16T01:58:56.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.706+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/412671966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d0149d90 0x5583d014c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.706+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/412671966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013f4d0 0x5583d0149850 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.706+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/412671966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d013ebb0 0x5583d013ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.706+0000 7f9f5b34b740 1 -- 172.21.15.139:0/412671966 >> 172.21.15.139:0/412671966 conn(0x5583d013b140 msgr2=0x5583d013b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:56.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.709+0000 7f9f5b34b740 1 -- 172.21.15.139:0/412671966 shutdown_connections 2024-09-16T01:58:56.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.709+0000 7f9f52d9d640 1 --2- 172.21.15.139:0/4151327269 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9f400a3540 0x7f9f400a5a00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400aa30 tx=0x7f9f44009230 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:56.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.711+0000 7f9f5b34b740 1 -- 172.21.15.139:0/412671966 wait complete. 2024-09-16T01:58:56.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.711+0000 7f9f5b34b740 1 -- start start 2024-09-16T01:58:56.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.711+0000 7f9f5b34b740 1 Processor -- start 2024-09-16T01:58:56.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.711+0000 7f9f5b34b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013ebb0 0x5583d0149800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.712+0000 7f9f5b34b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d013f4d0 0x5583d0146b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.712+0000 7f9f5b34b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d0149d90 0x5583d0147290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.712+0000 7f9f5b34b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5583d014c960 con 0x5583d0149d90 2024-09-16T01:58:56.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.712+0000 7f9f5b34b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5583d003dc50 con 0x5583d013f4d0 2024-09-16T01:58:56.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.712+0000 7f9f5b34b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5583d014cae0 con 0x5583d013ebb0 2024-09-16T01:58:56.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.712+0000 7f9f5b34b740 10 client.0 _ll_register_callbacks cb 0x5583d004fba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:56.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.712+0000 7f9f52d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d0149d90 0x5583d0147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:56.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.712+0000 7f9f5259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013ebb0 0x5583d0149800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:56.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.712+0000 7f9f52d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d0149d90 0x5583d0147290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54534/0 (socket says 172.21.15.139:54534) 2024-09-16T01:58:56.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.712+0000 7f9f52d9d640 1 -- 172.21.15.139:0/1048752614 learned_addr learned my addr 172.21.15.139:0/1048752614 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:56.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.713+0000 7f9f52d9d640 1 -- 172.21.15.139:0/1048752614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013ebb0 msgr2=0x5583d0149800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:56.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.713+0000 7f9f52d9d640 1 --2- 172.21.15.139:0/1048752614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013ebb0 0x5583d0149800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.713+0000 7f9f52d9d640 1 -- 172.21.15.139:0/1048752614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d013f4d0 msgr2=0x5583d0146b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T01:58:56.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.713+0000 7f9f52d9d640 1 --2- 172.21.15.139:0/1048752614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d013f4d0 0x5583d0146b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.713+0000 7f9f52d9d640 1 -- 172.21.15.139:0/1048752614 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d0155e10 con 0x5583d0149d90 2024-09-16T01:58:56.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.718+0000 7f9f52d9d640 1 --2- 172.21.15.139:0/1048752614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d0149d90 0x5583d0147290 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400d980 tx=0x7f9f44004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1eacb2533da198 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:56.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.720+0000 7f9f39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T01:58:56.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.720+0000 7f9f39ffb640 1 -- 172.21.15.139:0/1048752614 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44018cd0 con 0x5583d0149d90 2024-09-16T01:58:56.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.720+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1048752614 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4c01cd00 con 0x5583d0149d90 2024-09-16T01:58:56.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.720+0000 7f9f39ffb640 1 -- 172.21.15.139:0/1048752614 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7f9f44023480 con 0x5583d0149d90 2024-09-16T01:58:56.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.732+0000 7f9f39ffb640 1 -- 172.21.15.139:0/1048752614 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f9f4402c050 con 0x5583d0149d90 2024-09-16T01:58:56.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.733+0000 7f9f5b34b740 10 client.17376 fetch_fsmap learned FSMap version 77 2024-09-16T01:58:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.733+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1048752614 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x5583d0158750 con 0x5583d0149d90 2024-09-16T01:58:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.735+0000 7f9f39ffb640 1 -- 172.21.15.139:0/1048752614 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c01cd00 con 0x5583d0149d90 2024-09-16T01:58:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.735+0000 7f9f39ffb640 1 -- 172.21.15.139:0/1048752614 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 77) ==== 8450+0+0 (secure 0 0 0) 0x7f9f440194b0 con 0x5583d0149d90 2024-09-16T01:58:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.735+0000 7f9f5b34b740 10 client.17376 fetch_fsmap finished waiting for FSMap version 77 2024-09-16T01:58:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.735+0000 7f9f5b34b740 10 client.17376 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:58:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.735+0000 7f9f5b34b740 20 client.17376 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.735+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/1048752614 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5583d0159c10 0x5583d015c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.735+0000 7f9f5b34b740 4 client.17376 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T01:58:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.735+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1048752614 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583d00b0770 con 0x5583d0159c10 2024-09-16T01:58:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.736+0000 7f9f5259c640 1 --2- 172.21.15.139:0/1048752614 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5583d0159c10 0x5583d015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:56.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.740+0000 7f9f5259c640 1 --2- 172.21.15.139:0/1048752614 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5583d0159c10 0x5583d015c040 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6891a0d8bf9cc8a server_cookie=ab7a8bd146fca47 in_seq=0 out_seq=0 2024-09-16T01:58:56.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.741+0000 7f9f39ffb640 10 client.17376 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:56.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.744+0000 7f9f39ffb640 1 -- 172.21.15.139:0/1048752614 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5583d00b0770 con 0x5583d0159c10 2024-09-16T01:58:56.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.744+0000 7f9f39ffb640 10 client.17376 handle_command_reply: tid=0 2024-09-16T01:58:56.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.744+0000 7f9f5b34b740 1 client.17376 shutdown 2024-09-16T01:58:56.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.746+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1048752614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d0149d90 msgr2=0x5583d0147290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:56.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.746+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/1048752614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d0149d90 0x5583d0147290 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400d980 tx=0x7f9f44004cd0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.746+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1048752614 shutdown_connections 2024-09-16T01:58:56.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.746+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/1048752614 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5583d0159c10 0x5583d015c040 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.746+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/1048752614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d0149d90 0x5583d0147290 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.746+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/1048752614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d013f4d0 0x5583d0146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.746+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/1048752614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013ebb0 0x5583d0149800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.746+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1048752614 >> 172.21.15.139:0/1048752614 conn(0x5583d013b140 msgr2=0x5583d013a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:56.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.746+0000 7f9f39ffb640 0 client.17376 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:56.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.748+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1048752614 shutdown_connections 2024-09-16T01:58:56.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.752+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1048752614 wait complete. 2024-09-16T01:58:56.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.756+0000 7f9f5b34b740 20 client.17376 trim_cache size 0 max 16384 2024-09-16T01:58:56.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.789+0000 7f9f5b34b740 1 Processor -- start 2024-09-16T01:58:56.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.789+0000 7f9f5b34b740 1 -- start start 2024-09-16T01:58:56.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.789+0000 7f9f5b34b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013ebb0 0x5583d0140d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.789+0000 7f9f5b34b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d013f4d0 0x5583d0161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.789+0000 7f9f5b34b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d0149d90 0x5583d0161e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.789+0000 7f9f5b34b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541b5870 con 0x5583d013f4d0 2024-09-16T01:58:56.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.789+0000 7f9f5b34b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541b76b0 con 0x5583d0149d90 2024-09-16T01:58:56.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.789+0000 7f9f5b34b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541b6000 con 0x5583d013ebb0 2024-09-16T01:58:56.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.790+0000 7f9f52d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d0149d90 0x5583d0161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:56.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.790+0000 7f9f5259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013ebb0 0x5583d0140d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:56.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.790+0000 7f9f5259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013ebb0 0x5583d0140d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:44880/0 (socket says 172.21.15.139:44880) 2024-09-16T01:58:56.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.790+0000 7f9f5259c640 1 -- 172.21.15.139:0/3354151909 learned_addr learned my addr 172.21.15.139:0/3354151909 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:56.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.791+0000 7f9f5259c640 1 -- 172.21.15.139:0/3354151909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d0149d90 msgr2=0x5583d0161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:56.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.791+0000 7f9f5259c640 1 --2- 172.21.15.139:0/3354151909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d0149d90 0x5583d0161e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.791+0000 7f9f5259c640 1 -- 172.21.15.139:0/3354151909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d013f4d0 msgr2=0x5583d0161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:56.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.791+0000 7f9f5259c640 1 --2- 172.21.15.139:0/3354151909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d013f4d0 0x5583d0161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.791+0000 7f9f5259c640 1 -- 172.21.15.139:0/3354151909 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d013f300 con 0x5583d013ebb0 2024-09-16T01:58:56.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.795+0000 7f9f5259c640 1 --2- 172.21.15.139:0/3354151909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013ebb0 0x5583d0140d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c010ca0 tx=0x7f9f4c011840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a937bee7f0d0a1d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:56.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.795+0000 7f9f3bfff640 1 -- 172.21.15.139:0/3354151909 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c012d80 con 0x5583d013ebb0 2024-09-16T01:58:56.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.795+0000 7f9f3bfff640 1 -- 172.21.15.139:0/3354151909 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c005520 con 0x5583d013ebb0 2024-09-16T01:58:56.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.796+0000 7f9f5b34b740 1 -- 172.21.15.139:0/3354151909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013ebb0 msgr2=0x5583d0140d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:56.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.796+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/3354151909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013ebb0 0x5583d0140d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c010ca0 tx=0x7f9f4c011840 comp rx=0 tx=0).stop 2024-09-16T01:58:56.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.797+0000 7f9f5b34b740 1 -- 172.21.15.139:0/3354151909 shutdown_connections 2024-09-16T01:58:56.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.797+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/3354151909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d0149d90 0x5583d0161e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.797+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/3354151909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d013f4d0 0x5583d0161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.797+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/3354151909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d013ebb0 0x5583d0140d10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.797+0000 7f9f5b34b740 1 -- 172.21.15.139:0/3354151909 >> 172.21.15.139:0/3354151909 conn(0x5583d013b140 msgr2=0x5583d0140600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:56.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.807+0000 7f9f5b34b740 1 -- 172.21.15.139:0/3354151909 shutdown_connections 2024-09-16T01:58:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.807+0000 7f9f5b34b740 1 -- 172.21.15.139:0/3354151909 wait complete. 2024-09-16T01:58:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.807+0000 7f9f5b34b740 1 -- start start 2024-09-16T01:58:56.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.807+0000 7f9f5b34b740 1 Processor -- start 2024-09-16T01:58:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.808+0000 7f9f5b34b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d013ebb0 0x5583d014e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.808+0000 7f9f5b34b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d013f4d0 0x5583d014e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.808+0000 7f9f5b34b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d0149d90 0x5583d01c3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.808+0000 7f9f5b34b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541b6f20 con 0x5583d013ebb0 2024-09-16T01:58:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.808+0000 7f9f5b34b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541b6790 con 0x5583d013f4d0 2024-09-16T01:58:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.808+0000 7f9f5b34b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541b3850 con 0x5583d0149d90 2024-09-16T01:58:56.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.808+0000 7f9f5b34b740 10 client.0 _ll_register_callbacks cb 0x7f9f4400d630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T01:58:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.808+0000 7f9f52d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d0149d90 0x5583d01c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.808+0000 7f9f51d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d013f4d0 0x5583d014e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.808+0000 7f9f52d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d0149d90 0x5583d01c3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:44894/0 (socket says 172.21.15.139:44894) 2024-09-16T01:58:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.808+0000 7f9f52d9d640 1 -- 172.21.15.139:0/1244192385 learned_addr learned my addr 172.21.15.139:0/1244192385 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.808+0000 7f9f5259c640 1 --2- 172.21.15.139:0/1244192385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d013ebb0 0x5583d014e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.809+0000 7f9f51d9b640 1 -- 172.21.15.139:0/1244192385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d0149d90 msgr2=0x5583d01c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.809+0000 7f9f51d9b640 1 --2- 172.21.15.139:0/1244192385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d0149d90 0x5583d01c3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.809+0000 7f9f51d9b640 1 -- 172.21.15.139:0/1244192385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d013ebb0 msgr2=0x5583d014e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T01:58:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.809+0000 7f9f51d9b640 1 --2- 172.21.15.139:0/1244192385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d013ebb0 0x5583d014e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.809+0000 7f9f51d9b640 1 -- 172.21.15.139:0/1244192385 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d01c3e60 con 0x5583d013f4d0 2024-09-16T01:58:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.809+0000 7f9f51d9b640 1 --2- 172.21.15.139:0/1244192385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d013f4d0 0x5583d014e710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0ed9c0 tx=0x7f9f3c0bc090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e75961f91cfc7c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.809+0000 7f9f39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-16T01:58:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.813+0000 7f9f39ffb640 1 -- 172.21.15.139:0/1244192385 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c049040 con 0x5583d013f4d0 2024-09-16T01:58:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.813+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1244192385 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4c005520 con 0x5583d013f4d0 2024-09-16T01:58:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.813+0000 7f9f39ffb640 1 -- 172.21.15.139:0/1244192385 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7f9f3c004210 con 0x5583d013f4d0 2024-09-16T01:58:56.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.816+0000 7f9f39ffb640 1 -- 172.21.15.139:0/1244192385 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f9f3c0bcc50 con 0x5583d013f4d0 2024-09-16T01:58:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.820+0000 7f9f5b34b740 10 client.26807 fetch_fsmap learned FSMap version 77 2024-09-16T01:58:56.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.820+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1244192385 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x5583d01c50a0 con 0x5583d013f4d0 2024-09-16T01:58:56.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.821+0000 7f9f39ffb640 1 -- 172.21.15.139:0/1244192385 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c005520 con 0x5583d013f4d0 2024-09-16T01:58:56.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.821+0000 7f9f39ffb640 1 -- 172.21.15.139:0/1244192385 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 77) ==== 8450+0+0 (secure 0 0 0) 0x7f9f3c04c060 con 0x5583d013f4d0 2024-09-16T01:58:56.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.821+0000 7f9f5b34b740 10 client.26807 fetch_fsmap finished waiting for FSMap version 77 2024-09-16T01:58:56.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.821+0000 7f9f5b34b740 10 client.26807 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T01:58:56.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.821+0000 7f9f5b34b740 20 client.26807 populate_metadata read hostname 'smithi139' 2024-09-16T01:58:56.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.821+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/1244192385 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5583d01c5840 0x5583d01c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:56.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.821+0000 7f9f5b34b740 4 client.26807 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-16T01:58:56.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.821+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1244192385 --> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5583d00aac80 con 0x5583d01c5840 2024-09-16T01:58:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.821+0000 7f9f5259c640 1 --2- 172.21.15.139:0/1244192385 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5583d01c5840 0x5583d01c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.822+0000 7f9f39ffb640 10 client.26807 ms_handle_connect on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.823+0000 7f9f5259c640 1 --2- 172.21.15.139:0/1244192385 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5583d01c5840 0x5583d01c7c60 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f18076dc8c1de88 server_cookie=b29cfa36756cfba8 in_seq=0 out_seq=0 2024-09-16T01:58:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.823+0000 7f9f39ffb640 1 -- 172.21.15.139:0/1244192385 <== mds.0 v2:172.21.15.164:6838/2447189436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5583d00aac80 con 0x5583d01c5840 2024-09-16T01:58:56.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.823+0000 7f9f39ffb640 10 client.26807 handle_command_reply: tid=0 2024-09-16T01:58:56.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.824+0000 7f9f5b34b740 1 client.26807 shutdown 2024-09-16T01:58:56.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.832+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1244192385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d013f4d0 msgr2=0x5583d014e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:56.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.832+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/1244192385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d013f4d0 0x5583d014e710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0ed9c0 tx=0x7f9f3c0bc090 comp rx=0 tx=0).stop 2024-09-16T01:58:56.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.836+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1244192385 shutdown_connections 2024-09-16T01:58:56.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.836+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/1244192385 >> [v2:172.21.15.164:6838/2447189436,v1:172.21.15.164:6839/2447189436] conn(0x5583d01c5840 0x5583d01c7c60 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.836+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/1244192385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583d0149d90 0x5583d01c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.836+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/1244192385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5583d013f4d0 0x5583d014e710 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.836+0000 7f9f5b34b740 1 --2- 172.21.15.139:0/1244192385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583d013ebb0 0x5583d014e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.836+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1244192385 >> 172.21.15.139:0/1244192385 conn(0x5583d013b140 msgr2=0x5583d0148b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:56.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.836+0000 7f9f39ffb640 0 client.26807 ms_handle_reset on v2:172.21.15.164:6838/2447189436 2024-09-16T01:58:56.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.837+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1244192385 shutdown_connections 2024-09-16T01:58:56.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.837+0000 7f9f5b34b740 1 -- 172.21.15.139:0/1244192385 wait complete. 2024-09-16T01:58:56.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.837+0000 7f9f5b34b740 20 client.26807 trim_cache size 0 max 16384 2024-09-16T01:58:56.840 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:58:56.840 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 77, 2024-09-16T01:58:56.840 INFO:teuthology.orchestra.run.smithi139.stdout: "leader": 24470, 2024-09-16T01:58:56.840 INFO:teuthology.orchestra.run.smithi139.stdout: "set_version": 68, 2024-09-16T01:58:56.840 INFO:teuthology.orchestra.run.smithi139.stdout: "sets": {} 2024-09-16T01:58:56.840 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:58:56.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.843+0000 7f9f5359e640 1 -- 172.21.15.139:0/4151327269 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9f400a3540 msgr2=0x7f9f400a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:56.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.843+0000 7f9f5359e640 1 --2- 172.21.15.139:0/4151327269 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9f400a3540 0x7f9f400a5a00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400aa30 tx=0x7f9f44009230 comp rx=0 tx=0).stop 2024-09-16T01:58:56.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.843+0000 7f9f5359e640 1 -- 172.21.15.139:0/4151327269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f54109bb0 msgr2=0x7f9f54073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:56.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.843+0000 7f9f5359e640 1 --2- 172.21.15.139:0/4151327269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f54109bb0 0x7f9f54073e80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009850 tx=0x7f9f3c005230 comp rx=0 tx=0).stop 2024-09-16T01:58:56.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.843+0000 7f9f5359e640 1 -- 172.21.15.139:0/4151327269 shutdown_connections 2024-09-16T01:58:56.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.843+0000 7f9f5359e640 1 --2- 172.21.15.139:0/4151327269 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f9f400a3540 0x7f9f400a5a00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.843+0000 7f9f5359e640 1 --2- 172.21.15.139:0/4151327269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f54112670 0x7f9f5406cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.843+0000 7f9f5359e640 1 --2- 172.21.15.139:0/4151327269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f54109bb0 0x7f9f54073e80 secure :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009850 tx=0x7f9f3c005230 comp rx=0 tx=0).stop 2024-09-16T01:58:56.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.843+0000 7f9f5359e640 1 --2- 172.21.15.139:0/4151327269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f54109290 0x7f9f54073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:56.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.843+0000 7f9f5359e640 1 -- 172.21.15.139:0/4151327269 >> 172.21.15.139:0/4151327269 conn(0x7f9f54077960 msgr2=0x7f9f540780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:56.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.845+0000 7f9f5359e640 1 -- 172.21.15.139:0/4151327269 shutdown_connections 2024-09-16T01:58:56.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:56.845+0000 7f9f5359e640 1 -- 172.21.15.139:0/4151327269 wait complete. 2024-09-16T01:58:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:57 smithi139 ceph-mon[31175]: pgmap v591: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 KiB/s wr, 143 op/s 2024-09-16T01:58:57.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:57 smithi176 ceph-mon[35324]: pgmap v591: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 KiB/s wr, 143 op/s 2024-09-16T01:58:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:57 smithi164 ceph-mon[35121]: pgmap v591: 129 pgs: 21 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 52 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 KiB/s wr, 143 op/s 2024-09-16T01:58:57.810 INFO:tasks.quiescer:done joining Quiescers 2024-09-16T01:58:57.810 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-16T01:58:57.820 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T01:58:57.831 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-16T01:58:57.839 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-16T01:58:57.840 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T01:58:57.840 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T01:58:57.840 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-16T01:58:57.936 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-16T01:58:57.985 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-16T01:58:58.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.178+0000 7fecf218d640 1 -- 172.21.15.139:0/3607759218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecec1013b0 msgr2=0x7fecec105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:58.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.178+0000 7fecf218d640 1 --2- 172.21.15.139:0/3607759218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecec1013b0 0x7fecec105870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fece0009900 tx=0x7fece002a440 comp rx=0 tx=0).stop 2024-09-16T01:58:58.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.178+0000 7fecf218d640 1 -- 172.21.15.139:0/3607759218 shutdown_connections 2024-09-16T01:58:58.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.178+0000 7fecf218d640 1 --2- 172.21.15.139:0/3607759218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecec105f80 0x7fecec106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:58.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.178+0000 7fecf218d640 1 --2- 172.21.15.139:0/3607759218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecec1013b0 0x7fecec105870 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fece0009900 tx=0x7fece002a440 comp rx=0 tx=0).stop 2024-09-16T01:58:58.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.178+0000 7fecf218d640 1 --2- 172.21.15.139:0/3607759218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecec100a00 0x7fecec100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:58.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.178+0000 7fecf218d640 1 -- 172.21.15.139:0/3607759218 >> 172.21.15.139:0/3607759218 conn(0x7fecec0fc6f0 msgr2=0x7fecec0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:58.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.178+0000 7fecf218d640 1 -- 172.21.15.139:0/3607759218 shutdown_connections 2024-09-16T01:58:58.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.182+0000 7fecf218d640 1 -- 172.21.15.139:0/3607759218 wait complete. 2024-09-16T01:58:58.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.182+0000 7fecf218d640 1 Processor -- start 2024-09-16T01:58:58.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.183+0000 7fecf218d640 1 -- start start 2024-09-16T01:58:58.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.183+0000 7fecf218d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecec100a00 0x7fecec073870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:58.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.183+0000 7fecf218d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecec105f80 0x7fecec073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:58.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.184+0000 7fecf218d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecec06d8e0 0x7fecec06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:58.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.184+0000 7fecf218d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fecec06c220 con 0x7fecec06d8e0 2024-09-16T01:58:58.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.184+0000 7fecf218d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fecec06c0a0 con 0x7fecec105f80 2024-09-16T01:58:58.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.184+0000 7fecf218d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fecec06c3a0 con 0x7fecec100a00 2024-09-16T01:58:58.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.185+0000 7fecebfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecec06d8e0 0x7fecec06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:58.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.185+0000 7fecebfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecec06d8e0 0x7fecec06dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54556/0 (socket says 172.21.15.139:54556) 2024-09-16T01:58:58.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.185+0000 7fecebfff640 1 -- 172.21.15.139:0/2688063821 learned_addr learned my addr 172.21.15.139:0/2688063821 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:58.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.185+0000 7fecebfff640 1 -- 172.21.15.139:0/2688063821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecec100a00 msgr2=0x7fecec073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:58.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.185+0000 7fecebfff640 1 --2- 172.21.15.139:0/2688063821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecec100a00 0x7fecec073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:58.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.185+0000 7fecebfff640 1 -- 172.21.15.139:0/2688063821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecec105f80 msgr2=0x7fecec073db0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:58:58.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.185+0000 7fecebfff640 1 --2- 172.21.15.139:0/2688063821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecec105f80 0x7fecec073db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:58.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.185+0000 7fecebfff640 1 -- 172.21.15.139:0/2688063821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec06e610 con 0x7fecec06d8e0 2024-09-16T01:58:58.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.185+0000 7fecebfff640 1 --2- 172.21.15.139:0/2688063821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecec06d8e0 0x7fecec06dd50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fece400eaa0 tx=0x7fece400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:58.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.186+0000 7fece8ff9640 1 -- 172.21.15.139:0/2688063821 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece400cdb0 con 0x7fecec06d8e0 2024-09-16T01:58:58.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.186+0000 7fecf218d640 1 -- 172.21.15.139:0/2688063821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecec1b3c90 con 0x7fecec06d8e0 2024-09-16T01:58:58.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.186+0000 7fecf218d640 1 -- 172.21.15.139:0/2688063821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecec1b41b0 con 0x7fecec06d8e0 2024-09-16T01:58:58.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.186+0000 7fece8ff9640 1 -- 172.21.15.139:0/2688063821 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece4005020 con 0x7fecec06d8e0 2024-09-16T01:58:58.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.186+0000 7fece8ff9640 1 -- 172.21.15.139:0/2688063821 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece40116d0 con 0x7fecec06d8e0 2024-09-16T01:58:58.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.190+0000 7fecf218d640 1 -- 172.21.15.139:0/2688063821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd8005290 con 0x7fecec06d8e0 2024-09-16T01:58:58.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.190+0000 7fece8ff9640 1 -- 172.21.15.139:0/2688063821 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fece4004a40 con 0x7fecec06d8e0 2024-09-16T01:58:58.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.191+0000 7fece8ff9640 1 --2- 172.21.15.139:0/2688063821 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fecd40a3680 0x7fecd40a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:58.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.191+0000 7fece8ff9640 1 -- 172.21.15.139:0/2688063821 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7fece40d02e0 con 0x7fecec06d8e0 2024-09-16T01:58:58.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.191+0000 7feceb7fe640 1 --2- 172.21.15.139:0/2688063821 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fecd40a3680 0x7fecd40a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:58.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.192+0000 7feceb7fe640 1 --2- 172.21.15.139:0/2688063821 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fecd40a3680 0x7fecd40a5b40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0058c0 tx=0x7fecdc005850 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:58.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.201+0000 7fece8ff9640 1 -- 172.21.15.139:0/2688063821 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fece4016030 con 0x7fecec06d8e0 2024-09-16T01:58:58.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.587+0000 7fecf218d640 1 -- 172.21.15.139:0/2688063821 --> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fecd8000bc0 con 0x7fecd40a3680 2024-09-16T01:58:58.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.725+0000 7fece8ff9640 1 -- 172.21.15.139:0/2688063821 <== mgr.15324 v2:172.21.15.139:6800/4031776938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7fecd8000bc0 con 0x7fecd40a3680 2024-09-16T01:58:58.728 INFO:teuthology.orchestra.run.smithi139.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-16T00:00:00", "created": "2024-09-16T01:40:24", "first": "2024-09-16T01:41:00", "last": "2024-09-16T01:58:00", "last_pruned": "2024-09-16T01:58:01", "created_count": 18, "pruned_count": 12, "active": true} 2024-09-16T01:58:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.729+0000 7fecf218d640 1 -- 172.21.15.139:0/2688063821 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fecd40a3680 msgr2=0x7fecd40a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.729+0000 7fecf218d640 1 --2- 172.21.15.139:0/2688063821 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fecd40a3680 0x7fecd40a5b40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0058c0 tx=0x7fecdc005850 comp rx=0 tx=0).stop 2024-09-16T01:58:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.729+0000 7fecf218d640 1 -- 172.21.15.139:0/2688063821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecec06d8e0 msgr2=0x7fecec06dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.729+0000 7fecf218d640 1 --2- 172.21.15.139:0/2688063821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecec06d8e0 0x7fecec06dd50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fece400eaa0 tx=0x7fece400ef60 comp rx=0 tx=0).stop 2024-09-16T01:58:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.729+0000 7fecf218d640 1 -- 172.21.15.139:0/2688063821 shutdown_connections 2024-09-16T01:58:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.729+0000 7fecf218d640 1 --2- 172.21.15.139:0/2688063821 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fecd40a3680 0x7fecd40a5b40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.729+0000 7fecf218d640 1 --2- 172.21.15.139:0/2688063821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fecec06d8e0 0x7fecec06dd50 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.729+0000 7fecf218d640 1 --2- 172.21.15.139:0/2688063821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecec105f80 0x7fecec073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:58.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.729+0000 7fecf218d640 1 --2- 172.21.15.139:0/2688063821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecec100a00 0x7fecec073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:58.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.729+0000 7fecf218d640 1 -- 172.21.15.139:0/2688063821 >> 172.21.15.139:0/2688063821 conn(0x7fecec0fc6f0 msgr2=0x7fecec0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:58.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.729+0000 7fecf218d640 1 -- 172.21.15.139:0/2688063821 shutdown_connections 2024-09-16T01:58:58.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.729+0000 7fecf218d640 1 -- 172.21.15.139:0/2688063821 wait complete. 2024-09-16T01:58:58.749 DEBUG:teuthology.orchestra.run.smithi139:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-16T01:58:58.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.930+0000 7fa19399a640 1 -- 172.21.15.139:0/2720454932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa18c100490 msgr2=0x7fa18c100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:58.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.930+0000 7fa19399a640 1 --2- 172.21.15.139:0/2720454932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa18c100490 0x7fa18c100870 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009900 tx=0x7fa17c02a440 comp rx=0 tx=0).stop 2024-09-16T01:58:58.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.938+0000 7fa19399a640 1 -- 172.21.15.139:0/2720454932 shutdown_connections 2024-09-16T01:58:58.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.938+0000 7fa19399a640 1 --2- 172.21.15.139:0/2720454932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa18c1026f0 0x7fa18c10d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:58.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.938+0000 7fa19399a640 1 --2- 172.21.15.139:0/2720454932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa18c101d70 0x7fa18c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:58.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.938+0000 7fa19399a640 1 --2- 172.21.15.139:0/2720454932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa18c100490 0x7fa18c100870 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:58.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.938+0000 7fa19399a640 1 -- 172.21.15.139:0/2720454932 >> 172.21.15.139:0/2720454932 conn(0x7fa18c0770f0 msgr2=0x7fa18c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:58.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.941+0000 7fa19399a640 1 -- 172.21.15.139:0/2720454932 shutdown_connections 2024-09-16T01:58:58.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.942+0000 7fa19399a640 1 -- 172.21.15.139:0/2720454932 wait complete. 2024-09-16T01:58:58.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.943+0000 7fa19399a640 1 Processor -- start 2024-09-16T01:58:58.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.943+0000 7fa19399a640 1 -- start start 2024-09-16T01:58:58.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.944+0000 7fa19399a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa18c100490 0x7fa18c1a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:58.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.944+0000 7fa19399a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa18c101d70 0x7fa18c1a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:58.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.944+0000 7fa19399a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa18c1026f0 0x7fa18c10c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:58.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.944+0000 7fa191736640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa18c100490 0x7fa18c1a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:58.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.944+0000 7fa191736640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa18c100490 0x7fa18c1a35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:44920/0 (socket says 172.21.15.139:44920) 2024-09-16T01:58:58.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.944+0000 7fa191736640 1 -- 172.21.15.139:0/4079610264 learned_addr learned my addr 172.21.15.139:0/4079610264 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:58:58.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.944+0000 7fa19399a640 1 -- 172.21.15.139:0/4079610264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c1120e0 con 0x7fa18c1026f0 2024-09-16T01:58:58.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.944+0000 7fa19399a640 1 -- 172.21.15.139:0/4079610264 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c111f60 con 0x7fa18c101d70 2024-09-16T01:58:58.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.945+0000 7fa19399a640 1 -- 172.21.15.139:0/4079610264 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c112260 con 0x7fa18c100490 2024-09-16T01:58:58.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.946+0000 7fa191f37640 1 --2- 172.21.15.139:0/4079610264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa18c1026f0 0x7fa18c10c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.947+0000 7fa191736640 1 -- 172.21.15.139:0/4079610264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa18c101d70 msgr2=0x7fa18c1a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.947+0000 7fa190f35640 1 --2- 172.21.15.139:0/4079610264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa18c101d70 0x7fa18c1a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.947+0000 7fa191736640 1 --2- 172.21.15.139:0/4079610264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa18c101d70 0x7fa18c1a3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.947+0000 7fa191736640 1 -- 172.21.15.139:0/4079610264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa18c1026f0 msgr2=0x7fa18c10c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:58.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.947+0000 7fa191736640 1 --2- 172.21.15.139:0/4079610264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa18c1026f0 0x7fa18c10c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:58.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.947+0000 7fa191736640 1 -- 172.21.15.139:0/4079610264 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18c10c850 con 0x7fa18c100490 2024-09-16T01:58:58.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.950+0000 7fa191736640 1 --2- 172.21.15.139:0/4079610264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa18c100490 0x7fa18c1a35d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009a20 tx=0x7fa17c032c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:58.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.951+0000 7fa1827fc640 1 -- 172.21.15.139:0/4079610264 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c03f030 con 0x7fa18c100490 2024-09-16T01:58:58.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.951+0000 7fa1827fc640 1 -- 172.21.15.139:0/4079610264 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa17c003ce0 con 0x7fa18c100490 2024-09-16T01:58:58.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.951+0000 7fa1827fc640 1 -- 172.21.15.139:0/4079610264 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c035070 con 0x7fa18c100490 2024-09-16T01:58:58.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.951+0000 7fa19399a640 1 -- 172.21.15.139:0/4079610264 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa18c10cae0 con 0x7fa18c100490 2024-09-16T01:58:58.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.951+0000 7fa19399a640 1 -- 172.21.15.139:0/4079610264 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa18c1ab560 con 0x7fa18c100490 2024-09-16T01:58:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.953+0000 7fa177fff640 1 -- 172.21.15.139:0/4079610264 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa18c100870 con 0x7fa18c100490 2024-09-16T01:58:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.955+0000 7fa1827fc640 1 -- 172.21.15.139:0/4079610264 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa17c042030 con 0x7fa18c100490 2024-09-16T01:58:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.956+0000 7fa1827fc640 1 --2- 172.21.15.139:0/4079610264 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa1780a3640 0x7fa1780a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:58:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.956+0000 7fa1827fc640 1 -- 172.21.15.139:0/4079610264 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7fa17c0f0770 con 0x7fa18c100490 2024-09-16T01:58:58.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.963+0000 7fa190f35640 1 --2- 172.21.15.139:0/4079610264 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa1780a3640 0x7fa1780a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:58:58.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.963+0000 7fa1827fc640 1 -- 172.21.15.139:0/4079610264 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa17c0bcc20 con 0x7fa18c100490 2024-09-16T01:58:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:58.978+0000 7fa190f35640 1 --2- 172.21.15.139:0/4079610264 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa1780a3640 0x7fa1780a5b00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa18c10d620 tx=0x7fa188005c60 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:58:59.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:59.334+0000 7fa177fff640 1 -- 172.21.15.139:0/4079610264 --> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fa18c062630 con 0x7fa1780a3640 2024-09-16T01:58:59.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:59 smithi139 ceph-mon[31175]: pgmap v592: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 126 op/s 2024-09-16T01:58:59.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:58:59 smithi139 ceph-mon[31175]: from='client.17385 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:59.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:59 smithi176 ceph-mon[35324]: pgmap v592: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 126 op/s 2024-09-16T01:58:59.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:58:59 smithi176 ceph-mon[35324]: from='client.17385 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:59.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:59 smithi164 ceph-mon[35121]: pgmap v592: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 126 op/s 2024-09-16T01:58:59.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:58:59 smithi164 ceph-mon[35121]: from='client.17385 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:58:59.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:59.555+0000 7fa1827fc640 1 -- 172.21.15.139:0/4079610264 <== mgr.15324 v2:172.21.15.139:6800/4031776938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fa18c062630 con 0x7fa1780a3640 2024-09-16T01:58:59.557 INFO:teuthology.orchestra.run.smithi139.stdout:/ 1m 6m3h 2024-09-16T01:58:59.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:59.558+0000 7fa177fff640 1 -- 172.21.15.139:0/4079610264 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa1780a3640 msgr2=0x7fa1780a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:59.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:59.558+0000 7fa177fff640 1 --2- 172.21.15.139:0/4079610264 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa1780a3640 0x7fa1780a5b00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa18c10d620 tx=0x7fa188005c60 comp rx=0 tx=0).stop 2024-09-16T01:58:59.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:59.558+0000 7fa177fff640 1 -- 172.21.15.139:0/4079610264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa18c100490 msgr2=0x7fa18c1a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:58:59.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:59.558+0000 7fa177fff640 1 --2- 172.21.15.139:0/4079610264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa18c100490 0x7fa18c1a35d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009a20 tx=0x7fa17c032c80 comp rx=0 tx=0).stop 2024-09-16T01:58:59.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:59.558+0000 7fa177fff640 1 -- 172.21.15.139:0/4079610264 shutdown_connections 2024-09-16T01:58:59.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:59.558+0000 7fa177fff640 1 --2- 172.21.15.139:0/4079610264 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fa1780a3640 0x7fa1780a5b00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:59.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:59.558+0000 7fa177fff640 1 --2- 172.21.15.139:0/4079610264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa18c1026f0 0x7fa18c10c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:59.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:59.558+0000 7fa177fff640 1 --2- 172.21.15.139:0/4079610264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa18c101d70 0x7fa18c1a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:59.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:59.558+0000 7fa177fff640 1 --2- 172.21.15.139:0/4079610264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa18c100490 0x7fa18c1a35d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:58:59.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:59.558+0000 7fa177fff640 1 -- 172.21.15.139:0/4079610264 >> 172.21.15.139:0/4079610264 conn(0x7fa18c0770f0 msgr2=0x7fa18c103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:58:59.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:59.558+0000 7fa177fff640 1 -- 172.21.15.139:0/4079610264 shutdown_connections 2024-09-16T01:58:59.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:58:59.558+0000 7fa177fff640 1 -- 172.21.15.139:0/4079610264 wait complete. 2024-09-16T01:58:59.582 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-16T01:58:59.909 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T01:59:00.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.101+0000 7f0e48d2c640 1 -- 172.21.15.139:0/906676212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e441009a0 msgr2=0x7f0e44100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:00.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.101+0000 7f0e48d2c640 1 --2- 172.21.15.139:0/906676212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e441009a0 0x7f0e44100e00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0e3401a940 tx=0x7f0e3403b470 comp rx=0 tx=0).stop 2024-09-16T01:59:00.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.101+0000 7f0e48d2c640 1 -- 172.21.15.139:0/906676212 shutdown_connections 2024-09-16T01:59:00.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.101+0000 7f0e48d2c640 1 --2- 172.21.15.139:0/906676212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e44101340 0x7f0e4410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.101+0000 7f0e48d2c640 1 --2- 172.21.15.139:0/906676212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e441009a0 0x7f0e44100e00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.101+0000 7f0e48d2c640 1 --2- 172.21.15.139:0/906676212 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e441069b0 0x7f0e44106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.101+0000 7f0e48d2c640 1 -- 172.21.15.139:0/906676212 >> 172.21.15.139:0/906676212 conn(0x7f0e440fc6c0 msgr2=0x7f0e440feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:00.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.104+0000 7f0e48d2c640 1 -- 172.21.15.139:0/906676212 shutdown_connections 2024-09-16T01:59:00.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.104+0000 7f0e48d2c640 1 -- 172.21.15.139:0/906676212 wait complete. 2024-09-16T01:59:00.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.105+0000 7f0e48d2c640 1 Processor -- start 2024-09-16T01:59:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.108+0000 7f0e48d2c640 1 -- start start 2024-09-16T01:59:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.108+0000 7f0e48d2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e441009a0 0x7f0e44073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.108+0000 7f0e48d2c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e44101340 0x7f0e4406dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.108+0000 7f0e48d2c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e441069b0 0x7f0e4406e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.108+0000 7f0e48d2c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4406c620 con 0x7f0e441009a0 2024-09-16T01:59:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.108+0000 7f0e48d2c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4406c4a0 con 0x7f0e44101340 2024-09-16T01:59:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.108+0000 7f0e48d2c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4406c7a0 con 0x7f0e441069b0 2024-09-16T01:59:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.109+0000 7f0e4259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e441009a0 0x7f0e44073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.109+0000 7f0e42d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e441069b0 0x7f0e4406e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.109+0000 7f0e4259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e441009a0 0x7f0e44073c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54590/0 (socket says 172.21.15.139:54590) 2024-09-16T01:59:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.109+0000 7f0e42d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e441069b0 0x7f0e4406e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:44928/0 (socket says 172.21.15.139:44928) 2024-09-16T01:59:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.109+0000 7f0e4259c640 1 -- 172.21.15.139:0/1682814594 learned_addr learned my addr 172.21.15.139:0/1682814594 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:59:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.109+0000 7f0e42d9d640 1 -- 172.21.15.139:0/1682814594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e44101340 msgr2=0x7f0e4406dc80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:59:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.109+0000 7f0e42d9d640 1 --2- 172.21.15.139:0/1682814594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e44101340 0x7f0e4406dc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.109+0000 7f0e42d9d640 1 -- 172.21.15.139:0/1682814594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e441009a0 msgr2=0x7f0e44073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.109+0000 7f0e42d9d640 1 --2- 172.21.15.139:0/1682814594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e441009a0 0x7f0e44073c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.109+0000 7f0e42d9d640 1 -- 172.21.15.139:0/1682814594 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4406e990 con 0x7f0e441069b0 2024-09-16T01:59:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.110+0000 7f0e42d9d640 1 --2- 172.21.15.139:0/1682814594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e441069b0 0x7f0e4406e1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00ca30 tx=0x7f0e3c00cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.110+0000 7f0e3b7fe640 1 -- 172.21.15.139:0/1682814594 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c007d90 con 0x7f0e441069b0 2024-09-16T01:59:00.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.110+0000 7f0e3b7fe640 1 -- 172.21.15.139:0/1682814594 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e3c005020 con 0x7f0e441069b0 2024-09-16T01:59:00.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.111+0000 7f0e3b7fe640 1 -- 172.21.15.139:0/1682814594 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c00f740 con 0x7f0e441069b0 2024-09-16T01:59:00.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.113+0000 7f0e48d2c640 1 -- 172.21.15.139:0/1682814594 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e441ab030 con 0x7f0e441069b0 2024-09-16T01:59:00.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.113+0000 7f0e48d2c640 1 -- 172.21.15.139:0/1682814594 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e441ab500 con 0x7f0e441069b0 2024-09-16T01:59:00.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.114+0000 7f0e48d2c640 1 -- 172.21.15.139:0/1682814594 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e441020e0 con 0x7f0e441069b0 2024-09-16T01:59:00.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.118+0000 7f0e3b7fe640 1 -- 172.21.15.139:0/1682814594 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0e3c004a00 con 0x7f0e441069b0 2024-09-16T01:59:00.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.118+0000 7f0e3b7fe640 1 --2- 172.21.15.139:0/1682814594 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e300a33a0 0x7f0e300a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:00.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.119+0000 7f0e3b7fe640 1 -- 172.21.15.139:0/1682814594 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7f0e3c0d0da0 con 0x7f0e441069b0 2024-09-16T01:59:00.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.121+0000 7f0e4259c640 1 --2- 172.21.15.139:0/1682814594 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e300a33a0 0x7f0e300a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:00.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.122+0000 7f0e4259c640 1 --2- 172.21.15.139:0/1682814594 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e300a33a0 0x7f0e300a5860 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009ba0 tx=0x7f0e2c005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:00.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.123+0000 7f0e3b7fe640 1 -- 172.21.15.139:0/1682814594 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e3c01d020 con 0x7f0e441069b0 2024-09-16T01:59:00.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:00 smithi139 ceph-mon[31175]: from='client.26800 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:00.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:00 smithi176 ceph-mon[35324]: from='client.26800 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:00.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.456+0000 7f0e48d2c640 1 -- 172.21.15.139:0/1682814594 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e4406f1b0 con 0x7f0e441069b0 2024-09-16T01:59:00.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.459+0000 7f0e3b7fe640 1 -- 172.21.15.139:0/1682814594 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+25863 (secure 0 0 0) 0x7f0e3c014070 con 0x7f0e441069b0 2024-09-16T01:59:00.461 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:59:00.461 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":118,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","created":"2024-09-16T01:30:17.770313+0000","modified":"2024-09-16T01:58:51.120338+0000","last_up_change":"2024-09-16T01:37:08.611901+0000","last_in_change":"2024-09-16T01:36:45.223926+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:33:19.639647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T01:37:57.948214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T01:38:00.263797+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T01:38:06.773799+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b66696da-ef66-4f25-9527-bbcc9ded2cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6803","nonce":3190810133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6805","nonce":3190810133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6809","nonce":3190810133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6807","nonce":3190810133}]},"public_addr":"172.21.15.139:6803/3190810133","cluster_addr":"172.21.15.139:6805/3190810133","heartbeat_back_addr":"172.21.15.139:6809/3190810133","heartbeat_front_addr":"172.21.15.139:6807/3190810133","state":["exists","up"]},{"osd":1,"uuid":"ccad17f2-0750-425e-9810-49a13a929ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6801","nonce":2699270330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6803","nonce":2699270330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6807","nonce":2699270330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6805","nonce":2699270330}]},"public_addr":"172.21.15.164:6801/2699270330","cluster_addr":"172.21.15.164:6803/2699270330","heartbeat_back_addr":"172.21.15.164:6807/2699270330","heartbeat_front_addr":"172.21.15.164:6805/2699270330","state":["exists","up"]},{"osd":2,"uuid":"baf0ba73-8faf-4e51-a171-c47898800638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6801","nonce":2034471646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6803","nonce":2034471646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6807","nonce":2034471646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6805","nonce":2034471646}]},"public_addr":"172.21.15.176:6801/2034471646","cluster_addr":"172.21.15.176:6803/2034471646","heartbeat_back_addr":"172.21.15.176:6807/2034471646","heartbeat_front_addr":"172.21.15.176:6805/2034471646","state":["exists","up"]},{"osd":3,"uuid":"98957bf1-67bb-4092-9ea4-b263a5706f8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6811","nonce":797147071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6813","nonce":797147071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6817","nonce":797147071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6815","nonce":797147071}]},"public_addr":"172.21.15.139:6811/797147071","cluster_addr":"172.21.15.139:6813/797147071","heartbeat_back_addr":"172.21.15.139:6817/797147071","heartbeat_front_addr":"172.21.15.139:6815/797147071","state":["exists","up"]},{"osd":4,"uuid":"9cd76696-8161-4e82-a571-29e98c11d5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6809","nonce":1421304792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6811","nonce":1421304792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6815","nonce":1421304792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6813","nonce":1421304792}]},"public_addr":"172.21.15.164:6809/1421304792","cluster_addr":"172.21.15.164:6811/1421304792","heartbeat_back_addr":"172.21.15.164:6815/1421304792","heartbeat_front_addr":"172.21.15.164:6813/1421304792","state":["exists","up"]},{"osd":5,"uuid":"ccc17a29-9a58-40ec-b453-5c38bee86f25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6809","nonce":10600602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6811","nonce":10600602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6815","nonce":10600602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6813","nonce":10600602}]},"public_addr":"172.21.15.176:6809/10600602","cluster_addr":"172.21.15.176:6811/10600602","heartbeat_back_addr":"172.21.15.176:6815/10600602","heartbeat_front_addr":"172.21.15.176:6813/10600602","state":["exists","up"]},{"osd":6,"uuid":"e5ee7212-10c6-4a05-992e-622a14a5a6b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6819","nonce":2601111164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6821","nonce":2601111164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6825","nonce":2601111164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6823","nonce":2601111164}]},"public_addr":"172.21.15.139:6819/2601111164","cluster_addr":"172.21.15.139:6821/2601111164","heartbeat_back_addr":"172.21.15.139:6825/2601111164","heartbeat_front_addr":"172.21.15.139:6823/2601111164","state":["exists","up"]},{"osd":7,"uuid":"23143c15-0839-4fd5-9d0a-19ae891000f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6817","nonce":2074080693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6819","nonce":2074080693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6823","nonce":2074080693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6821","nonce":2074080693}]},"public_addr":"172.21.15.164:6817/2074080693","cluster_addr":"172.21.15.164:6819/2074080693","heartbeat_back_addr":"172.21.15.164:6823/2074080693","heartbeat_front_addr":"172.21.15.164:6821/2074080693","state":["exists","up"]},{"osd":8,"uuid":"1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6817","nonce":593133361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6819","nonce":593133361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6823","nonce":593133361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6821","nonce":593133361}]},"public_addr":"172.21.15.176:6817/593133361","cluster_addr":"172.21.15.176:6819/593133361","heartbeat_back_addr":"172.21.15.176:6823/593133361","heartbeat_front_addr":"172.21.15.176:6821/593133361","state":["exists","up"]},{"osd":9,"uuid":"09d25a3b-1e66-480e-9433-ceef7f0b9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6827","nonce":3047942108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6829","nonce":3047942108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6833","nonce":3047942108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6831","nonce":3047942108}]},"public_addr":"172.21.15.139:6827/3047942108","cluster_addr":"172.21.15.139:6829/3047942108","heartbeat_back_addr":"172.21.15.139:6833/3047942108","heartbeat_front_addr":"172.21.15.139:6831/3047942108","state":["exists","up"]},{"osd":10,"uuid":"32b4d8ce-b644-42fa-80cc-a15a1f5f3a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6825","nonce":4172533828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6827","nonce":4172533828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6831","nonce":4172533828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6829","nonce":4172533828}]},"public_addr":"172.21.15.164:6825/4172533828","cluster_addr":"172.21.15.164:6827/4172533828","heartbeat_back_addr":"172.21.15.164:6831/4172533828","heartbeat_front_addr":"172.21.15.164:6829/4172533828","state":["exists","up"]},{"osd":11,"uuid":"49098bdc-58a9-4a74-8045-d408af7711ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6825","nonce":1854547565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6827","nonce":1854547565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6831","nonce":1854547565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6829","nonce":1854547565}]},"public_addr":"172.21.15.176:6825/1854547565","cluster_addr":"172.21.15.176:6827/1854547565","heartbeat_back_addr":"172.21.15.176:6831/1854547565","heartbeat_front_addr":"172.21.15.176:6829/1854547565","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:27.374845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:52.611853+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:17.229567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:43.330503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:07.988876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:33.787864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:58.864910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:23.815006+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:49.551954+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:15.496773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:41.647557+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:37:07.089355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2901696849":"2024-09-17T01:40:23.674956+0000","172.21.15.139:0/2608207648":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/1830135963":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/677216867":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6800/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/3167151595":"2024-09-17T01:40:23.674956+0000","172.21.15.139:0/1329877785":"2024-09-17T01:40:23.674956+0000","172.21.15.139:6800/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6801/331514141":"2024-09-17T01:40:23.674956+0000","172.21.15.139:6801/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/1608930262":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/2362566775":"2024-09-17T01:40:23.674956+0000","172.21.15.139:6801/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6800/331514141":"2024-09-17T01:40:23.674956+0000","172.21.15.139:0/3997627612":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/3772232169":"2024-09-17T01:31:07.344357+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":1},{"begin":31,"length":1}]},{"pool":4,"snaps":[{"begin":16,"length":1},{"begin":18,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":1},{"begin":31,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:59:00.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.462+0000 7f0e48d2c640 1 -- 172.21.15.139:0/1682814594 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e300a33a0 msgr2=0x7f0e300a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:00.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.462+0000 7f0e48d2c640 1 --2- 172.21.15.139:0/1682814594 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e300a33a0 0x7f0e300a5860 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009ba0 tx=0x7f0e2c005f90 comp rx=0 tx=0).stop 2024-09-16T01:59:00.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.463+0000 7f0e48d2c640 1 -- 172.21.15.139:0/1682814594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e441069b0 msgr2=0x7f0e4406e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:00.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.463+0000 7f0e48d2c640 1 --2- 172.21.15.139:0/1682814594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e441069b0 0x7f0e4406e1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00ca30 tx=0x7f0e3c00cef0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.463+0000 7f0e48d2c640 1 -- 172.21.15.139:0/1682814594 shutdown_connections 2024-09-16T01:59:00.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.463+0000 7f0e48d2c640 1 --2- 172.21.15.139:0/1682814594 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f0e300a33a0 0x7f0e300a5860 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.463+0000 7f0e48d2c640 1 --2- 172.21.15.139:0/1682814594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e441069b0 0x7f0e4406e1c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.463+0000 7f0e48d2c640 1 --2- 172.21.15.139:0/1682814594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e44101340 0x7f0e4406dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.463+0000 7f0e48d2c640 1 --2- 172.21.15.139:0/1682814594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e441009a0 0x7f0e44073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.463+0000 7f0e48d2c640 1 -- 172.21.15.139:0/1682814594 >> 172.21.15.139:0/1682814594 conn(0x7f0e440fc6c0 msgr2=0x7f0e440fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:00.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.463+0000 7f0e48d2c640 1 -- 172.21.15.139:0/1682814594 shutdown_connections 2024-09-16T01:59:00.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.463+0000 7f0e48d2c640 1 -- 172.21.15.139:0/1682814594 wait complete. 2024-09-16T01:59:00.490 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T01:33:19.639647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T01:37:57.948214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T01:38:00.263797+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 31, 'snap_epoch': 116, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-16T01:38:06.773799+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 31, 'snap_epoch': 116, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-16T01:59:00.490 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T01:59:00.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:00 smithi164 ceph-mon[35121]: from='client.26800 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T01:59:00.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.682+0000 7fe95c25e640 1 -- 172.21.15.139:0/4038139143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe954107ab0 msgr2=0x7fe954107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:00.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.682+0000 7fe95c25e640 1 --2- 172.21.15.139:0/4038139143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe954107ab0 0x7fe954107e90 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009930 tx=0x7fe94c02e890 comp rx=0 tx=0).stop 2024-09-16T01:59:00.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.684+0000 7fe95c25e640 1 -- 172.21.15.139:0/4038139143 shutdown_connections 2024-09-16T01:59:00.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.684+0000 7fe95c25e640 1 --2- 172.21.15.139:0/4038139143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe954102440 0x7fe95410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.684+0000 7fe95c25e640 1 --2- 172.21.15.139:0/4038139143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe954101aa0 0x7fe954101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.684+0000 7fe95c25e640 1 --2- 172.21.15.139:0/4038139143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe954107ab0 0x7fe954107e90 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.684+0000 7fe95c25e640 1 -- 172.21.15.139:0/4038139143 >> 172.21.15.139:0/4038139143 conn(0x7fe9540fd7a0 msgr2=0x7fe9540ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:00.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.684+0000 7fe95c25e640 1 -- 172.21.15.139:0/4038139143 shutdown_connections 2024-09-16T01:59:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.689+0000 7fe95c25e640 1 -- 172.21.15.139:0/4038139143 wait complete. 2024-09-16T01:59:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.689+0000 7fe95c25e640 1 Processor -- start 2024-09-16T01:59:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.689+0000 7fe95c25e640 1 -- start start 2024-09-16T01:59:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe95c25e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe954101aa0 0x7fe954073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe95c25e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe954102440 0x7fe95406db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe95c25e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe954107ab0 0x7fe95406e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:00.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe95c25e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe95406c4c0 con 0x7fe954107ab0 2024-09-16T01:59:00.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe95c25e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe95406c340 con 0x7fe954102440 2024-09-16T01:59:00.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe95c25e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe95406c640 con 0x7fe954101aa0 2024-09-16T01:59:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe959ffa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe954101aa0 0x7fe954073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe959ffa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe954101aa0 0x7fe954073aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:44946/0 (socket says 172.21.15.139:44946) 2024-09-16T01:59:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe959ffa640 1 -- 172.21.15.139:0/4135308556 learned_addr learned my addr 172.21.15.139:0/4135308556 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:59:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe9597f9640 1 --2- 172.21.15.139:0/4135308556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe954102440 0x7fe95406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe959ffa640 1 -- 172.21.15.139:0/4135308556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe954102440 msgr2=0x7fe95406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe959ffa640 1 --2- 172.21.15.139:0/4135308556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe954102440 0x7fe95406db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe959ffa640 1 -- 172.21.15.139:0/4135308556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe954107ab0 msgr2=0x7fe95406e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe959ffa640 1 --2- 172.21.15.139:0/4135308556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe954107ab0 0x7fe95406e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.690+0000 7fe959ffa640 1 -- 172.21.15.139:0/4135308556 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95406e780 con 0x7fe954101aa0 2024-09-16T01:59:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.691+0000 7fe959ffa640 1 --2- 172.21.15.139:0/4135308556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe954101aa0 0x7fe954073aa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe94c02ee80 tx=0x7fe94c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.693+0000 7fe94affd640 1 -- 172.21.15.139:0/4135308556 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c03a030 con 0x7fe954101aa0 2024-09-16T01:59:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.693+0000 7fe95c25e640 1 -- 172.21.15.139:0/4135308556 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9541ab2b0 con 0x7fe954101aa0 2024-09-16T01:59:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.693+0000 7fe95c25e640 1 -- 172.21.15.139:0/4135308556 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9541ab7a0 con 0x7fe954101aa0 2024-09-16T01:59:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.693+0000 7fe94affd640 1 -- 172.21.15.139:0/4135308556 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe94c043070 con 0x7fe954101aa0 2024-09-16T01:59:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.693+0000 7fe94affd640 1 -- 172.21.15.139:0/4135308556 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c03e6a0 con 0x7fe954101aa0 2024-09-16T01:59:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.695+0000 7fe95c25e640 1 -- 172.21.15.139:0/4135308556 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe938005290 con 0x7fe954101aa0 2024-09-16T01:59:00.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.701+0000 7fe94affd640 1 -- 172.21.15.139:0/4135308556 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe94c004ab0 con 0x7fe954101aa0 2024-09-16T01:59:00.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.702+0000 7fe94affd640 1 --2- 172.21.15.139:0/4135308556 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe9400a35c0 0x7fe9400a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:00.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.702+0000 7fe94affd640 1 -- 172.21.15.139:0/4135308556 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7fe94c0f0f90 con 0x7fe954101aa0 2024-09-16T01:59:00.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.703+0000 7fe9597f9640 1 --2- 172.21.15.139:0/4135308556 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe9400a35c0 0x7fe9400a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:00.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.708+0000 7fe94affd640 1 -- 172.21.15.139:0/4135308556 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe94c046360 con 0x7fe954101aa0 2024-09-16T01:59:00.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:00.720+0000 7fe9597f9640 1 --2- 172.21.15.139:0/4135308556 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe9400a35c0 0x7fe9400a5a80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe944006fd0 tx=0x7fe944008040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:01.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.040+0000 7fe95c25e640 1 -- 172.21.15.139:0/4135308556 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe938005850 con 0x7fe954101aa0 2024-09-16T01:59:01.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.041+0000 7fe94affd640 1 -- 172.21.15.139:0/4135308556 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7fe94c0bd5d0 con 0x7fe954101aa0 2024-09-16T01:59:01.043 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-09-16T01:59:01.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.044+0000 7fe95c25e640 1 -- 172.21.15.139:0/4135308556 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe9400a35c0 msgr2=0x7fe9400a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:01.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.044+0000 7fe95c25e640 1 --2- 172.21.15.139:0/4135308556 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe9400a35c0 0x7fe9400a5a80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe944006fd0 tx=0x7fe944008040 comp rx=0 tx=0).stop 2024-09-16T01:59:01.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.044+0000 7fe95c25e640 1 -- 172.21.15.139:0/4135308556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe954101aa0 msgr2=0x7fe954073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:01.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.044+0000 7fe95c25e640 1 --2- 172.21.15.139:0/4135308556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe954101aa0 0x7fe954073aa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe94c02ee80 tx=0x7fe94c0046c0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.045+0000 7fe95c25e640 1 -- 172.21.15.139:0/4135308556 shutdown_connections 2024-09-16T01:59:01.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.045+0000 7fe95c25e640 1 --2- 172.21.15.139:0/4135308556 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fe9400a35c0 0x7fe9400a5a80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.045+0000 7fe95c25e640 1 --2- 172.21.15.139:0/4135308556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe954107ab0 0x7fe95406e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.045+0000 7fe95c25e640 1 --2- 172.21.15.139:0/4135308556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe954102440 0x7fe95406db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.045+0000 7fe95c25e640 1 --2- 172.21.15.139:0/4135308556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe954101aa0 0x7fe954073aa0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.045+0000 7fe95c25e640 1 -- 172.21.15.139:0/4135308556 >> 172.21.15.139:0/4135308556 conn(0x7fe9540fd7a0 msgr2=0x7fe9540fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:01.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.045+0000 7fe95c25e640 1 -- 172.21.15.139:0/4135308556 shutdown_connections 2024-09-16T01:59:01.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.045+0000 7fe95c25e640 1 -- 172.21.15.139:0/4135308556 wait complete. 2024-09-16T01:59:01.064 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T01:59:01.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.268+0000 7f709072c640 1 -- 172.21.15.139:0/3866891234 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f708811b140 msgr2=0x7f708811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:01.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.268+0000 7f709072c640 1 --2- 172.21.15.139:0/3866891234 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f708811b140 0x7f708811d530 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7080009860 tx=0x7f708002a340 comp rx=0 tx=0).stop 2024-09-16T01:59:01.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.270+0000 7f709072c640 1 -- 172.21.15.139:0/3866891234 shutdown_connections 2024-09-16T01:59:01.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.270+0000 7f709072c640 1 --2- 172.21.15.139:0/3866891234 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f708811b140 0x7f708811d530 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.270+0000 7f709072c640 1 --2- 172.21.15.139:0/3866891234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7088109840 0x7f7088109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.270+0000 7f709072c640 1 --2- 172.21.15.139:0/3866891234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7088109460 0x7f7088110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.270+0000 7f709072c640 1 -- 172.21.15.139:0/3866891234 >> 172.21.15.139:0/3866891234 conn(0x7f708806cdd0 msgr2=0x7f708806d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:01.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.272+0000 7f709072c640 1 -- 172.21.15.139:0/3866891234 shutdown_connections 2024-09-16T01:59:01.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.274+0000 7f709072c640 1 -- 172.21.15.139:0/3866891234 wait complete. 2024-09-16T01:59:01.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.274+0000 7f709072c640 1 Processor -- start 2024-09-16T01:59:01.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.275+0000 7f709072c640 1 -- start start 2024-09-16T01:59:01.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.275+0000 7f709072c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7088109460 0x7f7088118730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:01.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.275+0000 7f709072c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7088109840 0x7f70881117b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:01.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.275+0000 7f709072c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f708811b140 0x7f7088111cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:01.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.275+0000 7f709072c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7088120d50 con 0x7f7088109840 2024-09-16T01:59:01.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.275+0000 7f709072c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7088120bd0 con 0x7f708811b140 2024-09-16T01:59:01.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.275+0000 7f709072c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7088120ed0 con 0x7f7088109460 2024-09-16T01:59:01.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.276+0000 7f708e4c8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7088109460 0x7f7088118730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:01.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.276+0000 7f708e4c8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7088109460 0x7f7088118730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:44968/0 (socket says 172.21.15.139:44968) 2024-09-16T01:59:01.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.276+0000 7f708e4c8640 1 -- 172.21.15.139:0/3061538143 learned_addr learned my addr 172.21.15.139:0/3061538143 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:59:01.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.277+0000 7f708e4c8640 1 -- 172.21.15.139:0/3061538143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f708811b140 msgr2=0x7f7088111cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:59:01.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.277+0000 7f708e4c8640 1 --2- 172.21.15.139:0/3061538143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f708811b140 0x7f7088111cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.277+0000 7f708e4c8640 1 -- 172.21.15.139:0/3061538143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7088109840 msgr2=0x7f70881117b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:59:01.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.277+0000 7f708e4c8640 1 --2- 172.21.15.139:0/3061538143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7088109840 0x7f70881117b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.277+0000 7f708e4c8640 1 -- 172.21.15.139:0/3061538143 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70881122e0 con 0x7f7088109460 2024-09-16T01:59:01.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.277+0000 7f708e4c8640 1 --2- 172.21.15.139:0/3061538143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7088109460 0x7f7088118730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f707800e9c0 tx=0x7f707800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:01.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.281+0000 7f70877fe640 1 -- 172.21.15.139:0/3061538143 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707800cc60 con 0x7f7088109460 2024-09-16T01:59:01.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.281+0000 7f70877fe640 1 -- 172.21.15.139:0/3061538143 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f707800ce00 con 0x7f7088109460 2024-09-16T01:59:01.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.281+0000 7f70877fe640 1 -- 172.21.15.139:0/3061538143 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078010430 con 0x7f7088109460 2024-09-16T01:59:01.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.281+0000 7f709072c640 1 -- 172.21.15.139:0/3061538143 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70881125d0 con 0x7f7088109460 2024-09-16T01:59:01.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.282+0000 7f709072c640 1 -- 172.21.15.139:0/3061538143 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7088116580 con 0x7f7088109460 2024-09-16T01:59:01.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.289+0000 7f709072c640 1 -- 172.21.15.139:0/3061538143 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7088117060 con 0x7f7088109460 2024-09-16T01:59:01.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.293+0000 7f70877fe640 1 -- 172.21.15.139:0/3061538143 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7078005000 con 0x7f7088109460 2024-09-16T01:59:01.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.294+0000 7f70877fe640 1 --2- 172.21.15.139:0/3061538143 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f70740a3590 0x7f70740a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:01.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.295+0000 7f70877fe640 1 -- 172.21.15.139:0/3061538143 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7f70780d05e0 con 0x7f7088109460 2024-09-16T01:59:01.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.296+0000 7f708dcc7640 1 --2- 172.21.15.139:0/3061538143 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f70740a3590 0x7f70740a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:01.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.298+0000 7f708dcc7640 1 --2- 172.21.15.139:0/3061538143 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f70740a3590 0x7f70740a5a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7088112f70 tx=0x7f7070009500 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:01.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.304+0000 7f70877fe640 1 -- 172.21.15.139:0/3061538143 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f707809ca90 con 0x7f7088109460 2024-09-16T01:59:01.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:01 smithi139 ceph-mon[31175]: pgmap v593: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 36 KiB/s wr, 118 op/s 2024-09-16T01:59:01.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:01 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1682814594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:59:01.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:01 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/4135308556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T01:59:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:01 smithi164 ceph-mon[35121]: pgmap v593: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 36 KiB/s wr, 118 op/s 2024-09-16T01:59:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:01 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1682814594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:59:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:01 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/4135308556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T01:59:01.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.641+0000 7f709072c640 1 -- 172.21.15.139:0/3061538143 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f708810ae00 con 0x7f7088109460 2024-09-16T01:59:01.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.642+0000 7f70877fe640 1 -- 172.21.15.139:0/3061538143 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) ==== 104+0+11 (secure 0 0 0) 0x7f707809c390 con 0x7f7088109460 2024-09-16T01:59:01.644 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-09-16T01:59:01.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.645+0000 7f709072c640 1 -- 172.21.15.139:0/3061538143 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f70740a3590 msgr2=0x7f70740a5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:01.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.645+0000 7f709072c640 1 --2- 172.21.15.139:0/3061538143 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f70740a3590 0x7f70740a5a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7088112f70 tx=0x7f7070009500 comp rx=0 tx=0).stop 2024-09-16T01:59:01.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.645+0000 7f709072c640 1 -- 172.21.15.139:0/3061538143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7088109460 msgr2=0x7f7088118730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:01.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.645+0000 7f709072c640 1 --2- 172.21.15.139:0/3061538143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7088109460 0x7f7088118730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f707800e9c0 tx=0x7f707800ee80 comp rx=0 tx=0).stop 2024-09-16T01:59:01.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.646+0000 7f709072c640 1 -- 172.21.15.139:0/3061538143 shutdown_connections 2024-09-16T01:59:01.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.646+0000 7f709072c640 1 --2- 172.21.15.139:0/3061538143 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f70740a3590 0x7f70740a5a50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.646+0000 7f709072c640 1 --2- 172.21.15.139:0/3061538143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f708811b140 0x7f7088111cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.646+0000 7f709072c640 1 --2- 172.21.15.139:0/3061538143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7088109840 0x7f70881117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.646+0000 7f709072c640 1 --2- 172.21.15.139:0/3061538143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7088109460 0x7f7088118730 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.646+0000 7f709072c640 1 -- 172.21.15.139:0/3061538143 >> 172.21.15.139:0/3061538143 conn(0x7f708806cdd0 msgr2=0x7f708811bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:01.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.646+0000 7f709072c640 1 -- 172.21.15.139:0/3061538143 shutdown_connections 2024-09-16T01:59:01.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.646+0000 7f709072c640 1 -- 172.21.15.139:0/3061538143 wait complete. 2024-09-16T01:59:01.665 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T01:59:01.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:01 smithi176 ceph-mon[35324]: pgmap v593: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 36 KiB/s wr, 118 op/s 2024-09-16T01:59:01.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:01 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1682814594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:59:01.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:01 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/4135308556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T01:59:01.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.860+0000 7f1d64f11640 1 -- 172.21.15.139:0/2872170277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60105f10 msgr2=0x7f1d601063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:01.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.860+0000 7f1d64f11640 1 --2- 172.21.15.139:0/2872170277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60105f10 0x7f1d601063c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1d5801ad10 tx=0x7f1d5803ba20 comp rx=0 tx=0).stop 2024-09-16T01:59:01.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.867+0000 7f1d64f11640 1 -- 172.21.15.139:0/2872170277 shutdown_connections 2024-09-16T01:59:01.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.867+0000 7f1d64f11640 1 --2- 172.21.15.139:0/2872170277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60105f10 0x7f1d601063c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.867+0000 7f1d64f11640 1 --2- 172.21.15.139:0/2872170277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d60101400 0x7f1d601058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.867+0000 7f1d64f11640 1 --2- 172.21.15.139:0/2872170277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d60100ae0 0x7f1d60100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.867+0000 7f1d64f11640 1 -- 172.21.15.139:0/2872170277 >> 172.21.15.139:0/2872170277 conn(0x7f1d600779d0 msgr2=0x7f1d600fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:01.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.871+0000 7f1d64f11640 1 -- 172.21.15.139:0/2872170277 shutdown_connections 2024-09-16T01:59:01.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.874+0000 7f1d64f11640 1 -- 172.21.15.139:0/2872170277 wait complete. 2024-09-16T01:59:01.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.875+0000 7f1d64f11640 1 Processor -- start 2024-09-16T01:59:01.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.879+0000 7f1d64f11640 1 -- start start 2024-09-16T01:59:01.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.879+0000 7f1d64f11640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d60100ae0 0x7f1d6010d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:01.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.879+0000 7f1d64f11640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60101400 0x7f1d6010dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:01.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.879+0000 7f1d64f11640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d60105f10 0x7f1d60072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:01.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.879+0000 7f1d64f11640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1d60109950 con 0x7f1d60101400 2024-09-16T01:59:01.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.879+0000 7f1d64f11640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601097d0 con 0x7f1d60105f10 2024-09-16T01:59:01.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.879+0000 7f1d64f11640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1d60109ad0 con 0x7f1d60100ae0 2024-09-16T01:59:01.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.880+0000 7f1d5dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60101400 0x7f1d6010dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:01.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.880+0000 7f1d5dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60101400 0x7f1d6010dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54632/0 (socket says 172.21.15.139:54632) 2024-09-16T01:59:01.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.880+0000 7f1d5dd9b640 1 -- 172.21.15.139:0/2662717134 learned_addr learned my addr 172.21.15.139:0/2662717134 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:59:01.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.884+0000 7f1d5ed9d640 1 --2- 172.21.15.139:0/2662717134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d60105f10 0x7f1d60072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:01.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.887+0000 7f1d5dd9b640 1 -- 172.21.15.139:0/2662717134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d60100ae0 msgr2=0x7f1d6010d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:59:01.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.887+0000 7f1d5dd9b640 1 --2- 172.21.15.139:0/2662717134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d60100ae0 0x7f1d6010d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.887+0000 7f1d5dd9b640 1 -- 172.21.15.139:0/2662717134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d60105f10 msgr2=0x7f1d60072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:01.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.887+0000 7f1d5dd9b640 1 --2- 172.21.15.139:0/2662717134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d60105f10 0x7f1d60072ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:01.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.887+0000 7f1d5dd9b640 1 -- 172.21.15.139:0/2662717134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d60072ff0 con 0x7f1d60101400 2024-09-16T01:59:01.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.888+0000 7f1d5dd9b640 1 --2- 172.21.15.139:0/2662717134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60101400 0x7f1d6010dab0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400b450 tx=0x7f1d5400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:01.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.891+0000 7f1d4f7fe640 1 -- 172.21.15.139:0/2662717134 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54015020 con 0x7f1d60101400 2024-09-16T01:59:01.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.891+0000 7f1d64f11640 1 -- 172.21.15.139:0/2662717134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d600732e0 con 0x7f1d60101400 2024-09-16T01:59:01.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.891+0000 7f1d64f11640 1 -- 172.21.15.139:0/2662717134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d60073800 con 0x7f1d60101400 2024-09-16T01:59:01.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.891+0000 7f1d4f7fe640 1 -- 172.21.15.139:0/2662717134 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d54011d70 con 0x7f1d60101400 2024-09-16T01:59:01.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.891+0000 7f1d4f7fe640 1 -- 172.21.15.139:0/2662717134 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5400f340 con 0x7f1d60101400 2024-09-16T01:59:01.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.894+0000 7f1d64f11640 1 -- 172.21.15.139:0/2662717134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d40005290 con 0x7f1d60101400 2024-09-16T01:59:01.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.898+0000 7f1d4f7fe640 1 -- 172.21.15.139:0/2662717134 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1d54018020 con 0x7f1d60101400 2024-09-16T01:59:01.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.898+0000 7f1d4f7fe640 1 --2- 172.21.15.139:0/2662717134 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1d480a35d0 0x7f1d480a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:01.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.899+0000 7f1d4f7fe640 1 -- 172.21.15.139:0/2662717134 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7f1d540cfa90 con 0x7f1d60101400 2024-09-16T01:59:01.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.904+0000 7f1d5e59c640 1 --2- 172.21.15.139:0/2662717134 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1d480a35d0 0x7f1d480a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:01.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.909+0000 7f1d4f7fe640 1 -- 172.21.15.139:0/2662717134 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d5401f020 con 0x7f1d60101400 2024-09-16T01:59:01.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:01.912+0000 7f1d5e59c640 1 --2- 172.21.15.139:0/2662717134 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1d480a35d0 0x7f1d480a5a90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009670 tx=0x7f1d50006bd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:02.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.262+0000 7f1d64f11640 1 -- 172.21.15.139:0/2662717134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1d40005850 con 0x7f1d60101400 2024-09-16T01:59:02.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.266+0000 7f1d4f7fe640 1 -- 172.21.15.139:0/2662717134 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) ==== 100+0+11 (secure 0 0 0) 0x7f1d5400d040 con 0x7f1d60101400 2024-09-16T01:59:02.271 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-09-16T01:59:02.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.275+0000 7f1d64f11640 1 -- 172.21.15.139:0/2662717134 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1d480a35d0 msgr2=0x7f1d480a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:02.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.275+0000 7f1d64f11640 1 --2- 172.21.15.139:0/2662717134 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1d480a35d0 0x7f1d480a5a90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009670 tx=0x7f1d50006bd0 comp rx=0 tx=0).stop 2024-09-16T01:59:02.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.275+0000 7f1d64f11640 1 -- 172.21.15.139:0/2662717134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60101400 msgr2=0x7f1d6010dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:02.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.275+0000 7f1d64f11640 1 --2- 172.21.15.139:0/2662717134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60101400 0x7f1d6010dab0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400b450 tx=0x7f1d5400b910 comp rx=0 tx=0).stop 2024-09-16T01:59:02.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.275+0000 7f1d64f11640 1 -- 172.21.15.139:0/2662717134 shutdown_connections 2024-09-16T01:59:02.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.275+0000 7f1d64f11640 1 --2- 172.21.15.139:0/2662717134 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1d480a35d0 0x7f1d480a5a90 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009670 tx=0x7f1d50006bd0 comp rx=0 tx=0).stop 2024-09-16T01:59:02.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.275+0000 7f1d64f11640 1 --2- 172.21.15.139:0/2662717134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d60105f10 0x7f1d60072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:02.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.275+0000 7f1d64f11640 1 --2- 172.21.15.139:0/2662717134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60101400 0x7f1d6010dab0 secure :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400b450 tx=0x7f1d5400b910 comp rx=0 tx=0).stop 2024-09-16T01:59:02.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.275+0000 7f1d64f11640 1 --2- 172.21.15.139:0/2662717134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d60100ae0 0x7f1d6010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:02.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.275+0000 7f1d64f11640 1 -- 172.21.15.139:0/2662717134 >> 172.21.15.139:0/2662717134 conn(0x7f1d600779d0 msgr2=0x7f1d60103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:02.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.279+0000 7f1d64f11640 1 -- 172.21.15.139:0/2662717134 shutdown_connections 2024-09-16T01:59:02.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.279+0000 7f1d64f11640 1 -- 172.21.15.139:0/2662717134 wait complete. 2024-09-16T01:59:02.303 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-16T01:59:02.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.494+0000 7f14c9759640 1 -- 172.21.15.139:0/4273945634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14c4101d70 msgr2=0x7f14c4105680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.494+0000 7f14c9759640 1 --2- 172.21.15.139:0/4273945634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14c4101d70 0x7f14c4105680 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00b9f0 tx=0x7f14ac02c0a0 comp rx=0 tx=0).stop 2024-09-16T01:59:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.494+0000 7f14c9759640 1 -- 172.21.15.139:0/4273945634 shutdown_connections 2024-09-16T01:59:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.494+0000 7f14c9759640 1 --2- 172.21.15.139:0/4273945634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14c4105de0 0x7f14c410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.494+0000 7f14c9759640 1 --2- 172.21.15.139:0/4273945634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14c4101d70 0x7f14c4105680 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.494+0000 7f14c9759640 1 --2- 172.21.15.139:0/4273945634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c41013c0 0x7f14c41017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.494+0000 7f14c9759640 1 -- 172.21.15.139:0/4273945634 >> 172.21.15.139:0/4273945634 conn(0x7f14c4077170 msgr2=0x7f14c40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.494+0000 7f14c9759640 1 -- 172.21.15.139:0/4273945634 shutdown_connections 2024-09-16T01:59:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.494+0000 7f14c9759640 1 -- 172.21.15.139:0/4273945634 wait complete. 2024-09-16T01:59:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.495+0000 7f14c9759640 1 Processor -- start 2024-09-16T01:59:02.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.495+0000 7f14c9759640 1 -- start start 2024-09-16T01:59:02.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.495+0000 7f14c9759640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c41013c0 0x7f14c4196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:02.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.495+0000 7f14c9759640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14c4105de0 0x7f14c41967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:02.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.495+0000 7f14c9759640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14c419ab50 0x7f14c419b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:02.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.495+0000 7f14c9759640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f14c410fd40 con 0x7f14c4105de0 2024-09-16T01:59:02.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.495+0000 7f14c9759640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f14c410fbc0 con 0x7f14c419ab50 2024-09-16T01:59:02.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.495+0000 7f14c9759640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f14c410fec0 con 0x7f14c41013c0 2024-09-16T01:59:02.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.495+0000 7f14c27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14c4105de0 0x7f14c41967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:02.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.495+0000 7f14c27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14c4105de0 0x7f14c41967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39444/0 (socket says 172.21.15.139:39444) 2024-09-16T01:59:02.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.495+0000 7f14c27fc640 1 -- 172.21.15.139:0/3721461192 learned_addr learned my addr 172.21.15.139:0/3721461192 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:59:02.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.495+0000 7f14c2ffd640 1 --2- 172.21.15.139:0/3721461192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c41013c0 0x7f14c4196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:02.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.495+0000 7f14c37fe640 1 --2- 172.21.15.139:0/3721461192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14c419ab50 0x7f14c419b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:02.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.496+0000 7f14c27fc640 1 -- 172.21.15.139:0/3721461192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c41013c0 msgr2=0x7f14c4196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:02.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.496+0000 7f14c27fc640 1 --2- 172.21.15.139:0/3721461192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c41013c0 0x7f14c4196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:02.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.496+0000 7f14c27fc640 1 -- 172.21.15.139:0/3721461192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14c419ab50 msgr2=0x7f14c419b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:02.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.496+0000 7f14c27fc640 1 --2- 172.21.15.139:0/3721461192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14c419ab50 0x7f14c419b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:02.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.496+0000 7f14c27fc640 1 -- 172.21.15.139:0/3721461192 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c419b640 con 0x7f14c4105de0 2024-09-16T01:59:02.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.496+0000 7f14c27fc640 1 --2- 172.21.15.139:0/3721461192 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14c4105de0 0x7f14c41967d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009b20 tx=0x7f14ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:02.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.496+0000 7f14bbfff640 1 -- 172.21.15.139:0/3721461192 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac042070 con 0x7f14c4105de0 2024-09-16T01:59:02.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.496+0000 7f14c9759640 1 -- 172.21.15.139:0/3721461192 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c419b8d0 con 0x7f14c4105de0 2024-09-16T01:59:02.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.496+0000 7f14bbfff640 1 -- 172.21.15.139:0/3721461192 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ac004340 con 0x7f14c4105de0 2024-09-16T01:59:02.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.496+0000 7f14bbfff640 1 -- 172.21.15.139:0/3721461192 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac042070 con 0x7f14c4105de0 2024-09-16T01:59:02.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.496+0000 7f14c9759640 1 -- 172.21.15.139:0/3721461192 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c41a2e00 con 0x7f14c4105de0 2024-09-16T01:59:02.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.497+0000 7f14c9759640 1 -- 172.21.15.139:0/3721461192 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c41975e0 con 0x7f14c4105de0 2024-09-16T01:59:02.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.498+0000 7f14bbfff640 1 -- 172.21.15.139:0/3721461192 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f14ac045070 con 0x7f14c4105de0 2024-09-16T01:59:02.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.499+0000 7f14bbfff640 1 --2- 172.21.15.139:0/3721461192 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f14b00abfd0 0x7f14b00ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:02.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.499+0000 7f14c2ffd640 1 --2- 172.21.15.139:0/3721461192 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f14b00abfd0 0x7f14b00ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:02.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.499+0000 7f14bbfff640 1 -- 172.21.15.139:0/3721461192 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7f14ac004140 con 0x7f14c4105de0 2024-09-16T01:59:02.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.499+0000 7f14c2ffd640 1 --2- 172.21.15.139:0/3721461192 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f14b00abfd0 0x7f14b00ae490 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009530 tx=0x7f14b4009260 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:02.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.506+0000 7f14bbfff640 1 -- 172.21.15.139:0/3721461192 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f14ac04c020 con 0x7f14c4105de0 2024-09-16T01:59:02.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:02 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3061538143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T01:59:02.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:02 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:02.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:02 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3061538143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T01:59:02.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:02 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:02.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:02 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3061538143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T01:59:02.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:02 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T01:59:02.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.821+0000 7f14c9759640 1 -- 172.21.15.139:0/3721461192 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f14c402d980 con 0x7f14c4105de0 2024-09-16T01:59:02.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.825+0000 7f14bbfff640 1 -- 172.21.15.139:0/3721461192 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v118) ==== 103+0+11 (secure 0 0 0) 0x7f14ac04c020 con 0x7f14c4105de0 2024-09-16T01:59:02.828 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-09-16T01:59:02.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.832+0000 7f14c9759640 1 -- 172.21.15.139:0/3721461192 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f14b00abfd0 msgr2=0x7f14b00ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:02.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.833+0000 7f14c9759640 1 --2- 172.21.15.139:0/3721461192 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f14b00abfd0 0x7f14b00ae490 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009530 tx=0x7f14b4009260 comp rx=0 tx=0).stop 2024-09-16T01:59:02.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.833+0000 7f14c9759640 1 -- 172.21.15.139:0/3721461192 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14c4105de0 msgr2=0x7f14c41967d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:02.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.833+0000 7f14c9759640 1 --2- 172.21.15.139:0/3721461192 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14c4105de0 0x7f14c41967d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009b20 tx=0x7f14ac009b50 comp rx=0 tx=0).stop 2024-09-16T01:59:02.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.833+0000 7f14c9759640 1 -- 172.21.15.139:0/3721461192 shutdown_connections 2024-09-16T01:59:02.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.833+0000 7f14c9759640 1 --2- 172.21.15.139:0/3721461192 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f14b00abfd0 0x7f14b00ae490 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:02.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.833+0000 7f14c9759640 1 --2- 172.21.15.139:0/3721461192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14c419ab50 0x7f14c419b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:02.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.833+0000 7f14c9759640 1 --2- 172.21.15.139:0/3721461192 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14c4105de0 0x7f14c41967d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:02.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.834+0000 7f14c9759640 1 --2- 172.21.15.139:0/3721461192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c41013c0 0x7f14c4196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:02.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.834+0000 7f14c9759640 1 -- 172.21.15.139:0/3721461192 >> 172.21.15.139:0/3721461192 conn(0x7f14c4077170 msgr2=0x7f14c41040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:02.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.834+0000 7f14c9759640 1 -- 172.21.15.139:0/3721461192 shutdown_connections 2024-09-16T01:59:02.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:02.834+0000 7f14c9759640 1 -- 172.21.15.139:0/3721461192 wait complete. 2024-09-16T01:59:02.856 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-16T01:59:03.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.039+0000 7fbed359e640 1 -- 172.21.15.139:0/2195793231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed41091f0 msgr2=0x7fbed4110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:03.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.039+0000 7fbed359e640 1 --2- 172.21.15.139:0/2195793231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed41091f0 0x7fbed4110250 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fbecc01acb0 tx=0x7fbecc03b920 comp rx=0 tx=0).stop 2024-09-16T01:59:03.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.044+0000 7fbed359e640 1 -- 172.21.15.139:0/2195793231 shutdown_connections 2024-09-16T01:59:03.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.044+0000 7fbed359e640 1 --2- 172.21.15.139:0/2195793231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbed411b140 0x7fbed411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:03.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.044+0000 7fbed359e640 1 --2- 172.21.15.139:0/2195793231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed41095d0 0x7fbed4109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:03.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.044+0000 7fbed359e640 1 --2- 172.21.15.139:0/2195793231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed41091f0 0x7fbed4110250 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:03.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.044+0000 7fbed359e640 1 -- 172.21.15.139:0/2195793231 >> 172.21.15.139:0/2195793231 conn(0x7fbed406cba0 msgr2=0x7fbed406cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:03.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.050+0000 7fbed359e640 1 -- 172.21.15.139:0/2195793231 shutdown_connections 2024-09-16T01:59:03.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.054+0000 7fbed359e640 1 -- 172.21.15.139:0/2195793231 wait complete. 2024-09-16T01:59:03.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.055+0000 7fbed359e640 1 Processor -- start 2024-09-16T01:59:03.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.055+0000 7fbed359e640 1 -- start start 2024-09-16T01:59:03.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.056+0000 7fbed359e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbed41091f0 0x7fbed411a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:03.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.056+0000 7fbed359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed41095d0 0x7fbed4116940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:03.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.056+0000 7fbed359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed411b140 0x7fbed4116e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:03.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.056+0000 7fbed359e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbed4113360 con 0x7fbed411b140 2024-09-16T01:59:03.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.056+0000 7fbed359e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbed41131e0 con 0x7fbed41095d0 2024-09-16T01:59:03.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.056+0000 7fbed359e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbed41134e0 con 0x7fbed41091f0 2024-09-16T01:59:03.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.056+0000 7fbed2d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed411b140 0x7fbed4116e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:03.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.056+0000 7fbed2d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed411b140 0x7fbed4116e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39464/0 (socket says 172.21.15.139:39464) 2024-09-16T01:59:03.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.056+0000 7fbed2d9d640 1 -- 172.21.15.139:0/1109415943 learned_addr learned my addr 172.21.15.139:0/1109415943 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:59:03.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.056+0000 7fbed1d9b640 1 --2- 172.21.15.139:0/1109415943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed41095d0 0x7fbed4116940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:03.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.057+0000 7fbed1d9b640 1 -- 172.21.15.139:0/1109415943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbed41091f0 msgr2=0x7fbed411a8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:59:03.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.057+0000 7fbed1d9b640 1 --2- 172.21.15.139:0/1109415943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbed41091f0 0x7fbed411a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:03.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.057+0000 7fbed1d9b640 1 -- 172.21.15.139:0/1109415943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed411b140 msgr2=0x7fbed4116e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:03.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.057+0000 7fbed1d9b640 1 --2- 172.21.15.139:0/1109415943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed411b140 0x7fbed4116e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:03.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.057+0000 7fbed1d9b640 1 -- 172.21.15.139:0/1109415943 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed4117580 con 0x7fbed41095d0 2024-09-16T01:59:03.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.057+0000 7fbed1d9b640 1 --2- 172.21.15.139:0/1109415943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed41095d0 0x7fbed4116940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009fd0 tx=0x7fbec000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:03.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.061+0000 7fbecb7fe640 1 -- 172.21.15.139:0/1109415943 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0015020 con 0x7fbed41095d0 2024-09-16T01:59:03.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.061+0000 7fbecb7fe640 1 -- 172.21.15.139:0/1109415943 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbec0011d10 con 0x7fbed41095d0 2024-09-16T01:59:03.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.062+0000 7fbecb7fe640 1 -- 172.21.15.139:0/1109415943 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec000f2e0 con 0x7fbed41095d0 2024-09-16T01:59:03.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.062+0000 7fbed359e640 1 -- 172.21.15.139:0/1109415943 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed41aaf60 con 0x7fbed41095d0 2024-09-16T01:59:03.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.062+0000 7fbed359e640 1 -- 172.21.15.139:0/1109415943 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed41ab310 con 0x7fbed41095d0 2024-09-16T01:59:03.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.068+0000 7fbed359e640 1 -- 172.21.15.139:0/1109415943 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed4073830 con 0x7fbed41095d0 2024-09-16T01:59:03.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.068+0000 7fbecb7fe640 1 -- 172.21.15.139:0/1109415943 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fbec0018020 con 0x7fbed41095d0 2024-09-16T01:59:03.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.069+0000 7fbecb7fe640 1 --2- 172.21.15.139:0/1109415943 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbeb80a35c0 0x7fbeb80a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:03.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.070+0000 7fbecb7fe640 1 -- 172.21.15.139:0/1109415943 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9758+0+0 (secure 0 0 0) 0x7fbec00cf920 con 0x7fbed41095d0 2024-09-16T01:59:03.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.078+0000 7fbecb7fe640 1 -- 172.21.15.139:0/1109415943 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbec001f020 con 0x7fbed41095d0 2024-09-16T01:59:03.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.087+0000 7fbed259c640 1 --2- 172.21.15.139:0/1109415943 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbeb80a35c0 0x7fbeb80a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:03.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.089+0000 7fbed259c640 1 --2- 172.21.15.139:0/1109415943 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbeb80a35c0 0x7fbeb80a5a80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbecc006010 tx=0x7fbecc000f40 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:03.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.513+0000 7fbed359e640 1 -- 172.21.15.139:0/1109415943 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fbed41ab5a0 con 0x7fbed41095d0 2024-09-16T01:59:03.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.526+0000 7fbecb7fe640 1 -- 172.21.15.139:0/1109415943 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+742883 (secure 0 0 0) 0x7fbec009bee0 con 0x7fbed41095d0 2024-09-16T01:59:03.541 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:59:03.542 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:59:03.542 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 25, 2024-09-16T01:59:03.542 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.542 INFO:teuthology.orchestra.run.smithi139.stdout: "active_gid": 15324, 2024-09-16T01:59:03.542 INFO:teuthology.orchestra.run.smithi139.stdout: "active_name": "x", 2024-09-16T01:59:03.542 INFO:teuthology.orchestra.run.smithi139.stdout: "active_addrs": { 2024-09-16T01:59:03.542 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T01:59:03.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.542 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T01:59:03.542 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6800", 2024-09-16T01:59:03.542 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4031776938 2024-09-16T01:59:03.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.543 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T01:59:03.543 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6801", 2024-09-16T01:59:03.543 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4031776938 2024-09-16T01:59:03.543 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.543 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T01:59:03.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.543 INFO:teuthology.orchestra.run.smithi139.stdout: "active_addr": "172.21.15.139:6801/4031776938", 2024-09-16T01:59:03.543 INFO:teuthology.orchestra.run.smithi139.stdout: "active_change": "2024-09-16T01:40:23.675288+0000", 2024-09-16T01:59:03.543 INFO:teuthology.orchestra.run.smithi139.stdout: "active_mgr_features": 4540701547738038271, 2024-09-16T01:59:03.543 INFO:teuthology.orchestra.run.smithi139.stdout: "available": true, 2024-09-16T01:59:03.543 INFO:teuthology.orchestra.run.smithi139.stdout: "standbys": [ 2024-09-16T01:59:03.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.543 INFO:teuthology.orchestra.run.smithi139.stdout: "gid": 24865, 2024-09-16T01:59:03.543 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "z", 2024-09-16T01:59:03.544 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_features": 4540701547738038271, 2024-09-16T01:59:03.544 INFO:teuthology.orchestra.run.smithi139.stdout: "available_modules": [ 2024-09-16T01:59:03.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.544 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "alerts", 2024-09-16T01:59:03.544 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.544 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.544 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.544 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:03.544 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:03.545 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.545 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.545 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.545 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:03.545 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.545 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.545 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.545 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T01:59:03.545 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.546 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.546 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.546 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.546 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.546 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.546 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.546 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.546 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.547 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.547 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.547 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.547 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.547 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.547 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.547 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.547 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.547 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.547 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.547 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.547 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.547 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.547 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.548 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.548 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.548 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.548 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.548 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.548 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.548 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.548 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.549 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.549 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.549 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.549 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.549 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.549 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.549 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.549 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.549 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.550 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.550 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.550 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.550 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.550 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.550 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.550 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.550 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.550 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.550 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.550 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.551 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.551 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.551 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.551 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.551 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.551 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.551 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.551 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.551 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.551 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.551 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.551 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.552 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.552 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.552 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.552 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.552 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.552 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.552 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_destination": { 2024-09-16T01:59:03.552 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_destination", 2024-09-16T01:59:03.552 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.552 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.552 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.553 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.553 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.553 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.553 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.553 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Email address to send alerts to", 2024-09-16T01:59:03.553 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.553 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.553 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.553 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_from_name": { 2024-09-16T01:59:03.553 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_from_name", 2024-09-16T01:59:03.553 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.553 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.553 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.554 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "Ceph", 2024-09-16T01:59:03.554 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.554 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.554 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.554 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Email From: name", 2024-09-16T01:59:03.554 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.554 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.554 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.554 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_host": { 2024-09-16T01:59:03.554 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_host", 2024-09-16T01:59:03.554 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.554 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.554 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.555 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.555 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.555 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.555 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.555 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "SMTP server", 2024-09-16T01:59:03.555 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.555 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.555 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.555 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_password": { 2024-09-16T01:59:03.555 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_password", 2024-09-16T01:59:03.555 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.556 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.556 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.556 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.556 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.557 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.557 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.557 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Password to authenticate with", 2024-09-16T01:59:03.557 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.557 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.557 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.557 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_port": { 2024-09-16T01:59:03.557 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_port", 2024-09-16T01:59:03.557 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.557 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.557 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.557 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "465", 2024-09-16T01:59:03.558 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.558 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.558 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.558 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "SMTP port", 2024-09-16T01:59:03.558 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.558 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.558 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.558 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_sender": { 2024-09-16T01:59:03.558 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_sender", 2024-09-16T01:59:03.558 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.558 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.558 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.558 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.559 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.559 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.559 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.559 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "SMTP envelope sender", 2024-09-16T01:59:03.559 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.559 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.559 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.559 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_ssl": { 2024-09-16T01:59:03.559 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_ssl", 2024-09-16T01:59:03.559 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.559 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.560 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.560 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.560 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.560 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.560 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.560 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T01:59:03.560 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.560 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.560 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.560 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_user": { 2024-09-16T01:59:03.560 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_user", 2024-09-16T01:59:03.560 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.561 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.561 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.561 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.561 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.561 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.561 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.561 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "User to authenticate as", 2024-09-16T01:59:03.561 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.561 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.561 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.561 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.561 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "balancer", 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "active": { 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "active", 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T01:59:03.562 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.563 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.563 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.563 INFO:teuthology.orchestra.run.smithi139.stdout: "begin_time": { 2024-09-16T01:59:03.563 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "begin_time", 2024-09-16T01:59:03.563 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.563 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.563 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.563 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0000", 2024-09-16T01:59:03.563 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.563 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.563 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.563 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T01:59:03.564 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T01:59:03.564 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.564 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.564 INFO:teuthology.orchestra.run.smithi139.stdout: "begin_weekday": { 2024-09-16T01:59:03.564 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "begin_weekday", 2024-09-16T01:59:03.564 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:03.564 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.564 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.564 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:03.565 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "0", 2024-09-16T01:59:03.565 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "6", 2024-09-16T01:59:03.565 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.565 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T01:59:03.565 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T01:59:03.565 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.565 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.565 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_compat_max_iterations": { 2024-09-16T01:59:03.565 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crush_compat_max_iterations", 2024-09-16T01:59:03.565 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:03.565 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.565 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.565 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "25", 2024-09-16T01:59:03.566 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1", 2024-09-16T01:59:03.566 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "250", 2024-09-16T01:59:03.566 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.566 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T01:59:03.566 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.566 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.566 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.566 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_compat_metrics": { 2024-09-16T01:59:03.566 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crush_compat_metrics", 2024-09-16T01:59:03.566 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.566 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.567 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.567 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T01:59:03.567 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.567 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.567 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.567 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T01:59:03.567 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T01:59:03.567 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.567 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.568 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_compat_step": { 2024-09-16T01:59:03.568 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crush_compat_step", 2024-09-16T01:59:03.568 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:03.568 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.568 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.568 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0.5", 2024-09-16T01:59:03.568 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "0.001", 2024-09-16T01:59:03.568 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "0.999", 2024-09-16T01:59:03.568 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.568 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "aggressiveness of optimization", 2024-09-16T01:59:03.568 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T01:59:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: "end_time": { 2024-09-16T01:59:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "end_time", 2024-09-16T01:59:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "2359", 2024-09-16T01:59:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T01:59:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: "end_weekday": { 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "end_weekday", 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "0", 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "6", 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T01:59:03.570 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.571 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.571 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.571 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.571 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.571 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.571 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.571 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.571 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.571 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.571 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.571 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.571 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.571 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.572 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.572 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.572 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.572 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.572 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.572 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.572 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.572 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.572 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.572 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.572 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.572 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.572 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.573 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.573 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.573 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.573 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.573 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.573 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.573 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.573 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.573 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.573 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.574 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.574 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.574 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.574 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.574 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.574 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.574 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.574 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.574 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.574 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.574 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.574 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.574 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.574 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.575 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.575 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.575 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.575 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.575 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.575 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.575 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.575 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.575 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.575 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.575 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.575 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.575 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.576 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.576 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.576 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.576 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.576 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.576 INFO:teuthology.orchestra.run.smithi139.stdout: "min_score": { 2024-09-16T01:59:03.576 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "min_score", 2024-09-16T01:59:03.576 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:03.576 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.576 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.576 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:03.577 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.577 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.577 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.577 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T01:59:03.577 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.577 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.577 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.577 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": { 2024-09-16T01:59:03.577 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mode", 2024-09-16T01:59:03.577 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.577 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.577 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.577 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "upmap", 2024-09-16T01:59:03.578 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.578 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.578 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.578 INFO:teuthology.orchestra.run.smithi139.stdout: "crush-compat", 2024-09-16T01:59:03.578 INFO:teuthology.orchestra.run.smithi139.stdout: "none", 2024-09-16T01:59:03.578 INFO:teuthology.orchestra.run.smithi139.stdout: "read", 2024-09-16T01:59:03.578 INFO:teuthology.orchestra.run.smithi139.stdout: "upmap", 2024-09-16T01:59:03.578 INFO:teuthology.orchestra.run.smithi139.stdout: "upmap-read" 2024-09-16T01:59:03.578 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.578 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Balancer mode", 2024-09-16T01:59:03.578 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.578 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.578 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_ids": { 2024-09-16T01:59:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "pool_ids", 2024-09-16T01:59:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T01:59:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.580 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:03.580 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:03.580 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.580 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.580 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.580 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:03.580 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.580 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.580 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.580 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T01:59:03.580 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.580 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.580 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.581 INFO:teuthology.orchestra.run.smithi139.stdout: "upmap_max_deviation": { 2024-09-16T01:59:03.581 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "upmap_max_deviation", 2024-09-16T01:59:03.581 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.581 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.581 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.581 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:03.581 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1", 2024-09-16T01:59:03.581 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.581 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.581 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T01:59:03.581 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T01:59:03.581 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: "upmap_max_optimizations": { 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "upmap_max_optimizations", 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.582 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.583 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.583 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.583 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cephadm", 2024-09-16T01:59:03.583 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.583 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.583 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.583 INFO:teuthology.orchestra.run.smithi139.stdout: "agent_down_multiplier": { 2024-09-16T01:59:03.583 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "agent_down_multiplier", 2024-09-16T01:59:03.583 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:03.583 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.583 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.583 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "3.0", 2024-09-16T01:59:03.584 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.584 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.584 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.584 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T01:59:03.584 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.584 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.584 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.584 INFO:teuthology.orchestra.run.smithi139.stdout: "agent_refresh_rate": { 2024-09-16T01:59:03.584 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "agent_refresh_rate", 2024-09-16T01:59:03.584 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.584 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.584 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.584 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "20", 2024-09-16T01:59:03.585 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.585 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.585 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.585 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T01:59:03.585 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.585 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.585 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.585 INFO:teuthology.orchestra.run.smithi139.stdout: "agent_starting_port": { 2024-09-16T01:59:03.585 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "agent_starting_port", 2024-09-16T01:59:03.585 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.585 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.585 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.585 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "4721", 2024-09-16T01:59:03.586 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.586 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.586 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.586 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T01:59:03.586 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.586 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.586 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.586 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_ptrace": { 2024-09-16T01:59:03.586 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "allow_ptrace", 2024-09-16T01:59:03.586 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.586 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: "autotune_interval": { 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "autotune_interval", 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.587 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.588 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:03.588 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.588 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.588 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.588 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T01:59:03.588 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.588 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.588 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.588 INFO:teuthology.orchestra.run.smithi139.stdout: "autotune_memory_target_ratio": { 2024-09-16T01:59:03.588 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T01:59:03.588 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:03.588 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.589 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.589 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0.7", 2024-09-16T01:59:03.589 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.589 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.589 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.589 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T01:59:03.589 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.589 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.589 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.589 INFO:teuthology.orchestra.run.smithi139.stdout: "cephadm_log_destination": { 2024-09-16T01:59:03.589 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cephadm_log_destination", 2024-09-16T01:59:03.589 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.589 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.590 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.590 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.590 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.590 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.590 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.590 INFO:teuthology.orchestra.run.smithi139.stdout: "file", 2024-09-16T01:59:03.590 INFO:teuthology.orchestra.run.smithi139.stdout: "file,syslog", 2024-09-16T01:59:03.590 INFO:teuthology.orchestra.run.smithi139.stdout: "syslog" 2024-09-16T01:59:03.590 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.590 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T01:59:03.590 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.590 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.590 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.591 INFO:teuthology.orchestra.run.smithi139.stdout: "cgroups_split": { 2024-09-16T01:59:03.591 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cgroups_split", 2024-09-16T01:59:03.591 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.591 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.591 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.591 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.591 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.591 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.591 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.591 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T01:59:03.591 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.591 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.591 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.592 INFO:teuthology.orchestra.run.smithi139.stdout: "config_checks_enabled": { 2024-09-16T01:59:03.592 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "config_checks_enabled", 2024-09-16T01:59:03.592 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.592 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.592 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.592 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.592 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.592 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.592 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.593 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T01:59:03.593 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.593 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.593 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.593 INFO:teuthology.orchestra.run.smithi139.stdout: "config_dashboard": { 2024-09-16T01:59:03.594 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "config_dashboard", 2024-09-16T01:59:03.594 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.594 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.594 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.594 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.594 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.594 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.595 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.595 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T01:59:03.595 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.595 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.595 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.595 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_alertmanager": { 2024-09-16T01:59:03.596 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_alertmanager", 2024-09-16T01:59:03.596 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.596 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.596 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.596 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T01:59:03.596 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.596 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.597 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.597 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Prometheus container image", 2024-09-16T01:59:03.597 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.597 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.597 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.597 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_base": { 2024-09-16T01:59:03.597 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_base", 2024-09-16T01:59:03.597 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.598 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.598 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.598 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T01:59:03.598 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.598 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.598 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.599 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Container image name, without the tag", 2024-09-16T01:59:03.599 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.599 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.599 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.599 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_elasticsearch": { 2024-09-16T01:59:03.599 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_elasticsearch", 2024-09-16T01:59:03.600 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.600 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.600 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.600 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T01:59:03.600 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.600 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.600 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.601 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "elasticsearch container image", 2024-09-16T01:59:03.601 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.601 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.601 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.601 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_grafana": { 2024-09-16T01:59:03.601 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_grafana", 2024-09-16T01:59:03.601 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.601 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.601 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.601 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T01:59:03.602 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.602 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.602 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.602 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Prometheus container image", 2024-09-16T01:59:03.602 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.602 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.602 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.602 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_haproxy": { 2024-09-16T01:59:03.602 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_haproxy", 2024-09-16T01:59:03.602 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.603 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.603 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.603 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T01:59:03.603 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.603 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.603 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.603 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "HAproxy container image", 2024-09-16T01:59:03.603 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.603 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.603 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.603 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_jaeger_agent": { 2024-09-16T01:59:03.604 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_jaeger_agent", 2024-09-16T01:59:03.604 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.604 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.604 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.604 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T01:59:03.604 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.604 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.604 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.604 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Jaeger agent container image", 2024-09-16T01:59:03.604 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.604 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.604 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.605 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_jaeger_collector": { 2024-09-16T01:59:03.605 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_jaeger_collector", 2024-09-16T01:59:03.605 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.605 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.605 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.605 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T01:59:03.605 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.605 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.605 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.605 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Jaeger collector container image", 2024-09-16T01:59:03.605 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.605 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.605 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.606 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_jaeger_query": { 2024-09-16T01:59:03.606 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_jaeger_query", 2024-09-16T01:59:03.606 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.606 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.606 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.606 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T01:59:03.606 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.606 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.606 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.606 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Jaeger query container image", 2024-09-16T01:59:03.606 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.606 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.606 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.607 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_keepalived": { 2024-09-16T01:59:03.607 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_keepalived", 2024-09-16T01:59:03.607 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.607 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.607 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.607 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T01:59:03.607 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.607 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.607 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.607 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Keepalived container image", 2024-09-16T01:59:03.607 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.607 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.607 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.608 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_loki": { 2024-09-16T01:59:03.608 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_loki", 2024-09-16T01:59:03.608 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.608 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.608 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.608 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T01:59:03.608 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.608 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.608 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.608 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Loki container image", 2024-09-16T01:59:03.608 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.608 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.609 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.609 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_node_exporter": { 2024-09-16T01:59:03.609 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_node_exporter", 2024-09-16T01:59:03.609 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.609 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.609 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.609 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T01:59:03.609 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.609 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.609 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.609 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Prometheus container image", 2024-09-16T01:59:03.609 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.609 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.610 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.610 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_nvmeof": { 2024-09-16T01:59:03.610 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_nvmeof", 2024-09-16T01:59:03.610 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.610 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.610 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.610 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T01:59:03.610 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.610 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.610 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.610 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Nvme-of container image", 2024-09-16T01:59:03.610 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.610 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.611 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.611 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_prometheus": { 2024-09-16T01:59:03.611 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_prometheus", 2024-09-16T01:59:03.611 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.611 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.611 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.611 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T01:59:03.611 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.611 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.611 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.611 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Prometheus container image", 2024-09-16T01:59:03.611 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.611 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.612 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.612 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_promtail": { 2024-09-16T01:59:03.612 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_promtail", 2024-09-16T01:59:03.612 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.612 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.612 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.612 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T01:59:03.612 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.612 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.612 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.612 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Promtail container image", 2024-09-16T01:59:03.612 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.612 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.613 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.613 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_samba": { 2024-09-16T01:59:03.613 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_samba", 2024-09-16T01:59:03.613 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.613 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.613 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.613 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T01:59:03.613 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.613 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.613 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.613 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Samba/SMB container image", 2024-09-16T01:59:03.613 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.614 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.614 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.614 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_snmp_gateway": { 2024-09-16T01:59:03.614 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_snmp_gateway", 2024-09-16T01:59:03.614 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.614 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.614 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.614 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T01:59:03.614 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.615 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.615 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.615 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "SNMP Gateway container image", 2024-09-16T01:59:03.615 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.615 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.615 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.615 INFO:teuthology.orchestra.run.smithi139.stdout: "container_init": { 2024-09-16T01:59:03.615 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_init", 2024-09-16T01:59:03.616 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.616 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.616 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.616 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.616 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.616 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.616 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.617 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T01:59:03.617 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.617 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.617 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.617 INFO:teuthology.orchestra.run.smithi139.stdout: "daemon_cache_timeout": { 2024-09-16T01:59:03.617 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "daemon_cache_timeout", 2024-09-16T01:59:03.617 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.617 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.618 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.618 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:03.618 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.618 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.618 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.618 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T01:59:03.618 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.618 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.618 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.619 INFO:teuthology.orchestra.run.smithi139.stdout: "default_cephadm_command_timeout": { 2024-09-16T01:59:03.619 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T01:59:03.619 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.619 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.619 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.619 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "900", 2024-09-16T01:59:03.619 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.619 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.620 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.620 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T01:59:03.620 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.620 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.620 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.620 INFO:teuthology.orchestra.run.smithi139.stdout: "default_registry": { 2024-09-16T01:59:03.620 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "default_registry", 2024-09-16T01:59:03.620 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.621 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.621 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.621 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "docker.io", 2024-09-16T01:59:03.621 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.621 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.621 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.621 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T01:59:03.621 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.622 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.622 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.622 INFO:teuthology.orchestra.run.smithi139.stdout: "device_cache_timeout": { 2024-09-16T01:59:03.622 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "device_cache_timeout", 2024-09-16T01:59:03.622 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.622 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.622 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.622 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "1800", 2024-09-16T01:59:03.623 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.623 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.623 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.623 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "seconds to cache device inventory", 2024-09-16T01:59:03.623 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.623 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.623 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.624 INFO:teuthology.orchestra.run.smithi139.stdout: "device_enhanced_scan": { 2024-09-16T01:59:03.624 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "device_enhanced_scan", 2024-09-16T01:59:03.624 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.624 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.624 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.624 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.624 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.624 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.624 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.625 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T01:59:03.625 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.625 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.625 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.625 INFO:teuthology.orchestra.run.smithi139.stdout: "facts_cache_timeout": { 2024-09-16T01:59:03.625 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "facts_cache_timeout", 2024-09-16T01:59:03.625 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.625 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.626 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.626 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:03.626 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.626 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.626 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.626 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "seconds to cache host facts data", 2024-09-16T01:59:03.626 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.626 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.626 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.626 INFO:teuthology.orchestra.run.smithi139.stdout: "grafana_dashboards_path": { 2024-09-16T01:59:03.626 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "grafana_dashboards_path", 2024-09-16T01:59:03.627 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.627 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.627 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.627 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T01:59:03.627 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.627 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.627 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.627 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T01:59:03.627 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.627 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.627 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.628 INFO:teuthology.orchestra.run.smithi139.stdout: "host_check_interval": { 2024-09-16T01:59:03.628 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "host_check_interval", 2024-09-16T01:59:03.628 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.628 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.628 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.628 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:03.628 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.628 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.628 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.628 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to perform a host check", 2024-09-16T01:59:03.628 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.629 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.629 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.629 INFO:teuthology.orchestra.run.smithi139.stdout: "hw_monitoring": { 2024-09-16T01:59:03.629 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "hw_monitoring", 2024-09-16T01:59:03.629 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.629 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.629 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.629 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.629 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.629 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.629 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.629 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T01:59:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: "inventory_list_all": { 2024-09-16T01:59:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "inventory_list_all", 2024-09-16T01:59:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T01:59:03.631 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.631 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.631 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.631 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.631 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.631 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.631 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.631 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.631 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.631 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.631 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.631 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.631 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.632 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.632 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.632 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.632 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.632 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.632 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.632 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.632 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.632 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.632 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.632 INFO:teuthology.orchestra.run.smithi139.stdout: "log_refresh_metadata": { 2024-09-16T01:59:03.632 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_refresh_metadata", 2024-09-16T01:59:03.632 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.633 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.633 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.633 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.633 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.633 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.633 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.633 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T01:59:03.633 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.633 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.633 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T01:59:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.636 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.636 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.636 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.636 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.636 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.636 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.636 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.636 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.637 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.637 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.637 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.637 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.637 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.637 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.637 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.637 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.637 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.637 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.637 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.637 INFO:teuthology.orchestra.run.smithi139.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T01:59:03.638 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T01:59:03.638 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.638 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.638 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.638 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.638 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.638 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.638 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.638 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T01:59:03.638 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.638 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.638 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.638 INFO:teuthology.orchestra.run.smithi139.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T01:59:03.639 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T01:59:03.639 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.639 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.639 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.639 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "*", 2024-09-16T01:59:03.639 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.639 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.639 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.639 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T01:59:03.639 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.639 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.639 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.639 INFO:teuthology.orchestra.run.smithi139.stdout: "max_count_per_host": { 2024-09-16T01:59:03.640 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_count_per_host", 2024-09-16T01:59:03.640 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.640 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.640 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.640 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:03.640 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.640 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.640 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.640 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "max number of daemons per service per host", 2024-09-16T01:59:03.640 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.640 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.640 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.640 INFO:teuthology.orchestra.run.smithi139.stdout: "max_osd_draining_count": { 2024-09-16T01:59:03.641 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_osd_draining_count", 2024-09-16T01:59:03.641 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.641 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.641 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.641 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:03.641 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.641 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.641 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.641 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T01:59:03.641 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.641 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.641 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.641 INFO:teuthology.orchestra.run.smithi139.stdout: "migration_current": { 2024-09-16T01:59:03.642 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "migration_current", 2024-09-16T01:59:03.642 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.642 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.642 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.642 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.642 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.642 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.642 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.642 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "internal - do not modify", 2024-09-16T01:59:03.642 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.642 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.642 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.643 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": { 2024-09-16T01:59:03.643 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mode", 2024-09-16T01:59:03.643 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.643 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.643 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.643 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "root", 2024-09-16T01:59:03.643 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.643 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.643 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.643 INFO:teuthology.orchestra.run.smithi139.stdout: "cephadm-package", 2024-09-16T01:59:03.643 INFO:teuthology.orchestra.run.smithi139.stdout: "root" 2024-09-16T01:59:03.643 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.644 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T01:59:03.644 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.644 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.644 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.644 INFO:teuthology.orchestra.run.smithi139.stdout: "oob_default_addr": { 2024-09-16T01:59:03.644 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "oob_default_addr", 2024-09-16T01:59:03.644 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.644 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.644 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.644 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "169.254.1.1", 2024-09-16T01:59:03.644 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.644 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.645 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.645 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T01:59:03.645 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.645 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.645 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.645 INFO:teuthology.orchestra.run.smithi139.stdout: "prometheus_alerts_path": { 2024-09-16T01:59:03.645 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "prometheus_alerts_path", 2024-09-16T01:59:03.645 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.645 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.645 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.645 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T01:59:03.645 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.645 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.646 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.646 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T01:59:03.646 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.646 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.646 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.646 INFO:teuthology.orchestra.run.smithi139.stdout: "registry_insecure": { 2024-09-16T01:59:03.646 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "registry_insecure", 2024-09-16T01:59:03.646 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.646 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.647 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.647 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.647 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.647 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.647 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.647 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T01:59:03.647 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.647 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.647 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.647 INFO:teuthology.orchestra.run.smithi139.stdout: "registry_password": { 2024-09-16T01:59:03.647 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "registry_password", 2024-09-16T01:59:03.647 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.647 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.648 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.648 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.648 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.648 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.648 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.648 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T01:59:03.648 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.648 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.648 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.648 INFO:teuthology.orchestra.run.smithi139.stdout: "registry_url": { 2024-09-16T01:59:03.648 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "registry_url", 2024-09-16T01:59:03.648 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.649 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.649 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.649 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.649 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.649 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.649 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.649 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T01:59:03.649 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.649 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.649 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.649 INFO:teuthology.orchestra.run.smithi139.stdout: "registry_username": { 2024-09-16T01:59:03.649 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "registry_username", 2024-09-16T01:59:03.649 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.650 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.650 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.650 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.650 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.650 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.650 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.650 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T01:59:03.650 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.650 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.650 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.650 INFO:teuthology.orchestra.run.smithi139.stdout: "secure_monitoring_stack": { 2024-09-16T01:59:03.650 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "secure_monitoring_stack", 2024-09-16T01:59:03.650 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.651 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.651 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.651 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.651 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.651 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.651 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.651 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T01:59:03.651 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.651 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.651 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.651 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.651 INFO:teuthology.orchestra.run.smithi139.stdout: "service_discovery_port": { 2024-09-16T01:59:03.651 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "service_discovery_port", 2024-09-16T01:59:03.651 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.652 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.652 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.652 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8765", 2024-09-16T01:59:03.652 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.652 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.652 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.652 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "cephadm service discovery port", 2024-09-16T01:59:03.652 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.652 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.652 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.653 INFO:teuthology.orchestra.run.smithi139.stdout: "ssh_config_file": { 2024-09-16T01:59:03.653 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ssh_config_file", 2024-09-16T01:59:03.653 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.653 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.653 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.653 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.653 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.653 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.653 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.653 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T01:59:03.653 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.653 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.654 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.654 INFO:teuthology.orchestra.run.smithi139.stdout: "use_agent": { 2024-09-16T01:59:03.654 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "use_agent", 2024-09-16T01:59:03.654 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.654 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.654 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.654 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.654 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.654 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.654 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.654 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T01:59:03.654 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.654 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.655 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.655 INFO:teuthology.orchestra.run.smithi139.stdout: "use_repo_digest": { 2024-09-16T01:59:03.655 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "use_repo_digest", 2024-09-16T01:59:03.655 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.655 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.655 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.655 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.655 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.655 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.655 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.655 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T01:59:03.655 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.655 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_on_failed_host_check": { 2024-09-16T01:59:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_on_failed_host_check", 2024-09-16T01:59:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T01:59:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.657 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.657 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.657 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_on_stray_daemons": { 2024-09-16T01:59:03.657 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_on_stray_daemons", 2024-09-16T01:59:03.657 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.657 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.657 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.657 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.657 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.657 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.657 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.658 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T01:59:03.658 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.658 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.658 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.658 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_on_stray_hosts": { 2024-09-16T01:59:03.658 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_on_stray_hosts", 2024-09-16T01:59:03.658 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.658 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.658 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.658 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.658 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.658 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.658 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.659 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T01:59:03.659 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.659 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.659 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.659 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.659 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.659 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crash", 2024-09-16T01:59:03.659 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.659 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.659 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.659 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.660 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.660 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.660 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.660 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.660 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.660 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.660 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.660 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.661 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.661 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.661 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.661 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.661 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.661 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.661 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.661 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.661 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.661 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.661 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.661 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.661 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.662 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.662 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.662 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.662 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.662 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.662 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.662 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.662 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.662 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.662 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.662 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.662 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.663 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.663 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.663 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.663 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.663 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.663 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.663 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.663 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.663 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.663 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.663 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.663 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.663 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.663 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.664 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.664 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.664 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.664 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.664 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.664 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.664 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.664 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.664 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.665 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.665 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.665 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.665 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.665 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.665 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.665 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.665 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.665 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.665 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.665 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.665 INFO:teuthology.orchestra.run.smithi139.stdout: "retain_interval": { 2024-09-16T01:59:03.665 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "retain_interval", 2024-09-16T01:59:03.666 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.666 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.666 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.666 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "31536000", 2024-09-16T01:59:03.666 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.666 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.666 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.666 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T01:59:03.666 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.666 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.666 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.666 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_recent_interval": { 2024-09-16T01:59:03.667 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_recent_interval", 2024-09-16T01:59:03.667 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.667 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.667 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.667 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "1209600", 2024-09-16T01:59:03.667 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.667 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.667 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.667 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T01:59:03.667 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.667 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.667 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.667 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.667 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.668 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "dashboard", 2024-09-16T01:59:03.668 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.668 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.668 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.668 INFO:teuthology.orchestra.run.smithi139.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T01:59:03.668 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T01:59:03.668 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.668 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.668 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.669 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:03.669 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.669 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.669 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.669 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.669 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.669 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.669 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.669 INFO:teuthology.orchestra.run.smithi139.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T01:59:03.669 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T01:59:03.670 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.670 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.670 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.670 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.670 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.670 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.670 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.670 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.670 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.670 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.670 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.670 INFO:teuthology.orchestra.run.smithi139.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: "AUDIT_API_ENABLED": { 2024-09-16T01:59:03.671 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T01:59:03.672 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.672 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.672 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.672 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.672 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.672 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.672 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.672 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.672 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.672 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.672 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.672 INFO:teuthology.orchestra.run.smithi139.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T01:59:03.672 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T01:59:03.673 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.673 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.673 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.673 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.673 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.673 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.673 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.673 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.673 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.673 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.673 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.673 INFO:teuthology.orchestra.run.smithi139.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T01:59:03.673 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T01:59:03.674 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.674 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.674 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.674 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.674 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.674 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.674 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.674 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.674 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.674 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.674 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.675 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T01:59:03.675 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T01:59:03.675 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.675 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.675 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.675 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.675 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.675 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.675 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.675 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.675 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.675 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.675 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.675 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.676 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T01:59:03.676 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T01:59:03.676 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.676 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.676 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.676 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.676 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.676 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.676 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.676 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.676 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.676 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.676 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.677 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T01:59:03.677 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T01:59:03.677 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.677 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.677 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.677 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.677 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.677 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.677 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.678 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.678 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.678 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.678 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.678 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T01:59:03.678 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T01:59:03.678 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.678 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.678 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.678 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.678 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.678 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.679 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.679 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.679 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.679 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.679 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.679 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T01:59:03.679 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T01:59:03.679 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.680 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.680 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.680 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.680 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.680 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.680 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.680 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.680 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.681 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.681 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.681 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T01:59:03.681 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T01:59:03.681 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.681 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.681 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.682 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.682 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.682 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.682 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.682 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.682 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.682 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.682 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.683 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T01:59:03.683 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T01:59:03.683 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.683 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.683 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.683 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.683 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.684 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.684 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.684 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.684 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.684 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.684 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.684 INFO:teuthology.orchestra.run.smithi139.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T01:59:03.684 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T01:59:03.685 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.685 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.685 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.685 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.685 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.685 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.685 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.685 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.686 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.686 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.686 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.686 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T01:59:03.686 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T01:59:03.686 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.686 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.687 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.687 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "admin", 2024-09-16T01:59:03.687 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.687 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.687 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.687 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.687 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.687 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.688 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.688 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T01:59:03.688 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T01:59:03.688 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.688 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.688 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.688 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.689 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.689 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.689 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.689 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.689 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.689 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.689 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.689 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_API_URL": { 2024-09-16T01:59:03.690 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_API_URL", 2024-09-16T01:59:03.690 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.690 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.690 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.690 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.690 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.690 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.690 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.691 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.691 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.691 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.691 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.691 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T01:59:03.691 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T01:59:03.691 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.691 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.692 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.692 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "admin", 2024-09-16T01:59:03.692 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.692 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.692 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.692 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.692 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.692 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.692 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.693 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T01:59:03.693 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T01:59:03.693 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.693 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.693 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.693 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.693 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.693 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.694 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.694 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.694 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.694 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.694 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.694 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T01:59:03.694 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T01:59:03.694 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.695 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.695 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.695 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.695 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.695 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.695 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.695 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.695 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.695 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.695 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T01:59:03.696 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T01:59:03.696 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.696 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.696 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.696 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.696 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.696 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.696 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.696 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.696 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.697 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.697 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.697 INFO:teuthology.orchestra.run.smithi139.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T01:59:03.697 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T01:59:03.697 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.697 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.697 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.697 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.697 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.698 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.698 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.698 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.698 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.698 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.698 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.698 INFO:teuthology.orchestra.run.smithi139.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T01:59:03.698 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T01:59:03.698 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.698 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.698 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.698 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.699 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.699 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.699 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.699 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.699 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.699 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.699 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.699 INFO:teuthology.orchestra.run.smithi139.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T01:59:03.699 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T01:59:03.699 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.699 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.700 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.700 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.700 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.700 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.700 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.700 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.700 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.700 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.700 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.700 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T01:59:03.701 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T01:59:03.701 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.701 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.701 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.701 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.701 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.701 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.701 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.701 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.701 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.701 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.701 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.702 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T01:59:03.702 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T01:59:03.702 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.702 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.702 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.702 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.702 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.702 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.702 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.702 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.703 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.703 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.703 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.703 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T01:59:03.703 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T01:59:03.703 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.703 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.703 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.703 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.703 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.703 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.704 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.704 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.704 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.704 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.704 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.704 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T01:59:03.704 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T01:59:03.704 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.704 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.704 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.704 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.705 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.705 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.705 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.705 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.705 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.705 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.705 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.705 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T01:59:03.705 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T01:59:03.705 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.705 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.706 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.706 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.706 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.706 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.706 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.706 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.706 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.706 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.706 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.706 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T01:59:03.706 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T01:59:03.706 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.707 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.707 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.707 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.707 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.707 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.707 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.707 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.707 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.707 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.707 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.707 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T01:59:03.707 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T01:59:03.708 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.708 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.708 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.708 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.708 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.708 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.708 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.708 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.708 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.708 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.708 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.708 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T01:59:03.709 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T01:59:03.709 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.709 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.709 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.709 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.709 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.709 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.709 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.709 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.709 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.709 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.709 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.710 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T01:59:03.710 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T01:59:03.710 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.710 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.710 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.710 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T01:59:03.710 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.710 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.710 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.710 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.710 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.711 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.711 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.711 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T01:59:03.711 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T01:59:03.711 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.711 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.711 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.711 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:03.711 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.711 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.711 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.711 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.712 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.712 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.712 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.712 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T01:59:03.712 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T01:59:03.712 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.712 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.712 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.712 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8", 2024-09-16T01:59:03.712 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.713 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.713 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.713 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.713 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.713 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.713 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.713 INFO:teuthology.orchestra.run.smithi139.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T01:59:03.713 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T01:59:03.713 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.713 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.713 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.714 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "45", 2024-09-16T01:59:03.714 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.714 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.714 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.714 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.714 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.714 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.714 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.714 INFO:teuthology.orchestra.run.smithi139.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T01:59:03.714 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T01:59:03.714 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.715 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.716 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.716 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "admin", 2024-09-16T01:59:03.716 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.716 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.716 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.716 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.716 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.716 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.716 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.716 INFO:teuthology.orchestra.run.smithi139.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T01:59:03.716 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T01:59:03.716 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.717 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.717 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.717 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.717 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.717 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.717 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.717 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.717 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.717 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.717 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.717 INFO:teuthology.orchestra.run.smithi139.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T01:59:03.717 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T01:59:03.717 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.718 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.719 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.719 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:03.719 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.719 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.719 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.719 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.719 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.719 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.719 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.719 INFO:teuthology.orchestra.run.smithi139.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T01:59:03.720 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T01:59:03.720 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.720 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.720 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.720 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:03.720 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.720 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.720 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.720 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.720 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.720 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.720 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.720 INFO:teuthology.orchestra.run.smithi139.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T01:59:03.721 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T01:59:03.721 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.721 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.721 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.721 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:03.721 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.721 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.721 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.721 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.721 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.721 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.721 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.721 INFO:teuthology.orchestra.run.smithi139.stdout: "cross_origin_url": { 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cross_origin_url", 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: "crt_file": { 2024-09-16T01:59:03.722 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crt_file", 2024-09-16T01:59:03.723 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.723 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.723 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.723 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.723 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.723 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.723 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.723 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.723 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.723 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.723 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.723 INFO:teuthology.orchestra.run.smithi139.stdout: "debug": { 2024-09-16T01:59:03.723 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "debug", 2024-09-16T01:59:03.724 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.724 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.724 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.724 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.724 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.724 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.724 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.724 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Enable/disable debug options", 2024-09-16T01:59:03.724 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.725 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.725 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.725 INFO:teuthology.orchestra.run.smithi139.stdout: "jwt_token_ttl": { 2024-09-16T01:59:03.725 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "jwt_token_ttl", 2024-09-16T01:59:03.725 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.725 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.725 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.725 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "28800", 2024-09-16T01:59:03.725 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.725 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.725 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.725 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.725 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.726 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.726 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.726 INFO:teuthology.orchestra.run.smithi139.stdout: "key_file": { 2024-09-16T01:59:03.726 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "key_file", 2024-09-16T01:59:03.726 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.726 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.726 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.726 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.726 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.726 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.726 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.726 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.726 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.727 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.727 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.727 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.727 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.727 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.727 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.727 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.727 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.727 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.727 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.727 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.727 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.727 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.728 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.728 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.728 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.728 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.728 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.728 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.728 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.728 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.728 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.728 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.728 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.729 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.729 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.729 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.729 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.729 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.729 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.729 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.729 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.729 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.729 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.729 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.729 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.730 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.731 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.731 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.731 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.731 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.731 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.731 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.731 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.731 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.731 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.731 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.731 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.731 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.731 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.732 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.732 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.732 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.732 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.732 INFO:teuthology.orchestra.run.smithi139.stdout: "motd": { 2024-09-16T01:59:03.732 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "motd", 2024-09-16T01:59:03.732 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.732 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.732 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.732 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.732 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.732 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.732 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.733 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "The message of the day", 2024-09-16T01:59:03.733 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.733 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.733 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.733 INFO:teuthology.orchestra.run.smithi139.stdout: "redirect_resolve_ip_addr": { 2024-09-16T01:59:03.733 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T01:59:03.733 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.733 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.733 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.733 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.733 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.733 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.733 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: "server_addr": { 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_addr", 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "::", 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.734 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.735 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.735 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.735 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.735 INFO:teuthology.orchestra.run.smithi139.stdout: "server_port": { 2024-09-16T01:59:03.735 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_port", 2024-09-16T01:59:03.735 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.735 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.735 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.735 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8080", 2024-09-16T01:59:03.735 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.735 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.735 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.735 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.736 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.736 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.736 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.736 INFO:teuthology.orchestra.run.smithi139.stdout: "ssl": { 2024-09-16T01:59:03.736 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ssl", 2024-09-16T01:59:03.736 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.736 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.736 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.736 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.736 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.736 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.737 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.737 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.737 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.737 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.737 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.737 INFO:teuthology.orchestra.run.smithi139.stdout: "ssl_server_port": { 2024-09-16T01:59:03.737 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ssl_server_port", 2024-09-16T01:59:03.737 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.738 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.738 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.738 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8443", 2024-09-16T01:59:03.738 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.738 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.738 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.738 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.738 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.739 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.739 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.739 INFO:teuthology.orchestra.run.smithi139.stdout: "standby_behaviour": { 2024-09-16T01:59:03.739 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "standby_behaviour", 2024-09-16T01:59:03.739 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.739 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.739 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.739 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "redirect", 2024-09-16T01:59:03.740 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.740 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.740 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.740 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.740 INFO:teuthology.orchestra.run.smithi139.stdout: "redirect" 2024-09-16T01:59:03.740 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.740 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.740 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.740 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.740 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.740 INFO:teuthology.orchestra.run.smithi139.stdout: "standby_error_status_code": { 2024-09-16T01:59:03.740 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "standby_error_status_code", 2024-09-16T01:59:03.740 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.741 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.741 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.741 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "500", 2024-09-16T01:59:03.741 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "400", 2024-09-16T01:59:03.741 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "599", 2024-09-16T01:59:03.741 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.741 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.741 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.741 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.741 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.741 INFO:teuthology.orchestra.run.smithi139.stdout: "url_prefix": { 2024-09-16T01:59:03.741 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "url_prefix", 2024-09-16T01:59:03.741 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.742 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.742 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.742 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.742 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.742 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.742 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.742 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.742 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.742 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.742 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.742 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.742 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.743 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "devicehealth", 2024-09-16T01:59:03.743 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.743 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.743 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.743 INFO:teuthology.orchestra.run.smithi139.stdout: "enable_monitoring": { 2024-09-16T01:59:03.743 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "enable_monitoring", 2024-09-16T01:59:03.743 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.743 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.743 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.743 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.743 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.743 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.743 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.743 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "monitor device health metrics", 2024-09-16T01:59:03.744 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.744 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.744 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.744 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.744 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.744 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.744 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.744 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.744 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.744 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.745 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.745 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.745 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.745 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.745 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.745 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.745 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.745 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.745 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.746 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.746 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.746 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.746 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.746 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.746 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.747 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.747 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.747 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.747 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.747 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.747 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.747 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.748 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.748 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.748 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.748 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.748 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.748 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.749 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.749 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.749 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.749 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.749 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.749 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.749 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.749 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.749 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.750 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.750 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.750 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.750 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.750 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.750 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.750 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.750 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.750 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.750 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.750 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.750 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.750 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: "mark_out_threshold": { 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mark_out_threshold", 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.751 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.752 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "2419200", 2024-09-16T01:59:03.752 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.752 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.752 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.752 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T01:59:03.752 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.752 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.752 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.752 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": { 2024-09-16T01:59:03.752 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "pool_name", 2024-09-16T01:59:03.752 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.752 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.753 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.753 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "device_health_metrics", 2024-09-16T01:59:03.753 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.753 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.753 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.753 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T01:59:03.753 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.753 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.753 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.753 INFO:teuthology.orchestra.run.smithi139.stdout: "retention_period": { 2024-09-16T01:59:03.753 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "retention_period", 2024-09-16T01:59:03.753 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.753 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.754 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.754 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "15552000", 2024-09-16T01:59:03.754 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.754 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.754 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.754 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how long to retain device health metrics", 2024-09-16T01:59:03.754 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.754 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.754 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.754 INFO:teuthology.orchestra.run.smithi139.stdout: "scrape_frequency": { 2024-09-16T01:59:03.754 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "scrape_frequency", 2024-09-16T01:59:03.754 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.754 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.755 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.755 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "86400", 2024-09-16T01:59:03.755 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.755 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.755 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.755 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T01:59:03.755 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.755 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.755 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.756 INFO:teuthology.orchestra.run.smithi139.stdout: "self_heal": { 2024-09-16T01:59:03.756 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "self_heal", 2024-09-16T01:59:03.756 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.756 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.756 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.756 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.756 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.756 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.756 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.756 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T01:59:03.757 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.757 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.757 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.757 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:03.757 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:03.757 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.762 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.762 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.762 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:03.762 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.762 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.762 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.762 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T01:59:03.762 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.762 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.762 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.763 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_threshold": { 2024-09-16T01:59:03.763 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_threshold", 2024-09-16T01:59:03.763 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.763 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.763 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.763 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "7257600", 2024-09-16T01:59:03.763 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.763 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.763 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.763 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T01:59:03.763 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.763 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.763 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.763 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.764 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.764 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "diskprediction_local", 2024-09-16T01:59:03.764 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.764 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.764 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.764 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.764 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.764 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.764 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.764 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.764 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.765 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.765 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.765 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.765 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.765 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.765 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.765 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.765 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.765 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.765 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.765 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.766 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.766 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.766 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.766 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.766 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.766 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.766 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.766 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.766 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.766 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.766 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.766 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.767 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.767 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.767 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.767 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.767 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.767 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.767 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.767 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.767 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.767 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.767 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.767 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.767 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.768 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.768 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.768 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.768 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.768 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.768 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.768 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.768 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.768 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.768 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.769 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.769 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.769 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.769 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.769 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.769 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.769 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.770 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.770 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.770 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.770 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.770 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.770 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.770 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.770 INFO:teuthology.orchestra.run.smithi139.stdout: "predict_interval": { 2024-09-16T01:59:03.770 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "predict_interval", 2024-09-16T01:59:03.770 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.770 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.771 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.771 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "86400", 2024-09-16T01:59:03.771 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.771 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.771 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.771 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.771 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.771 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.771 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.771 INFO:teuthology.orchestra.run.smithi139.stdout: "predictor_model": { 2024-09-16T01:59:03.771 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "predictor_model", 2024-09-16T01:59:03.771 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.771 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.772 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.772 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "prophetstor", 2024-09-16T01:59:03.772 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.772 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.772 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.772 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.772 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.772 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.772 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.772 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:03.772 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:03.772 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.773 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.773 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.773 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:03.773 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.773 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.773 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.773 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.773 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.773 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.773 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.773 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.773 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.774 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "influx", 2024-09-16T01:59:03.774 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": false, 2024-09-16T01:59:03.774 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "influxdb python module not found", 2024-09-16T01:59:03.774 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.774 INFO:teuthology.orchestra.run.smithi139.stdout: "batch_size": { 2024-09-16T01:59:03.774 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "batch_size", 2024-09-16T01:59:03.774 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.774 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.774 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.774 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5000", 2024-09-16T01:59:03.774 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.775 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.775 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.775 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T01:59:03.775 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.775 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.775 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.775 INFO:teuthology.orchestra.run.smithi139.stdout: "database": { 2024-09-16T01:59:03.775 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "database", 2024-09-16T01:59:03.775 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.775 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.775 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.776 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "ceph", 2024-09-16T01:59:03.776 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.776 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.776 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.776 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T01:59:03.776 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.776 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.776 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.776 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": { 2024-09-16T01:59:03.776 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "hostname", 2024-09-16T01:59:03.776 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.776 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.777 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.777 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.777 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.777 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.777 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.777 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "InfluxDB server hostname", 2024-09-16T01:59:03.777 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.777 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.777 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.777 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:03.777 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:03.777 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.777 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.778 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.778 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "30", 2024-09-16T01:59:03.778 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "5", 2024-09-16T01:59:03.778 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.778 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.778 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T01:59:03.778 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.778 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.778 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.778 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.778 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.778 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.778 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.779 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.780 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.780 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.780 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.780 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.780 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.780 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.780 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.780 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.780 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.780 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.780 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.780 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.781 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.781 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.781 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.781 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.781 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.781 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.781 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.781 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.781 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.781 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.781 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.781 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.781 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.782 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.782 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.782 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.782 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.782 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.782 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.782 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.782 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.782 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.782 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.782 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.782 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.782 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: "password": { 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "password", 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.783 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.784 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.784 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.784 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.784 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.784 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "password of InfluxDB server user", 2024-09-16T01:59:03.784 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.784 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.784 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.784 INFO:teuthology.orchestra.run.smithi139.stdout: "port": { 2024-09-16T01:59:03.784 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "port", 2024-09-16T01:59:03.784 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.784 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8086", 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "InfluxDB server port", 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: "ssl": { 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ssl", 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.786 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "false", 2024-09-16T01:59:03.786 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.786 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.786 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.786 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T01:59:03.786 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.786 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.786 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.786 INFO:teuthology.orchestra.run.smithi139.stdout: "threads": { 2024-09-16T01:59:03.786 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "threads", 2024-09-16T01:59:03.786 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.786 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.786 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.787 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:03.787 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1", 2024-09-16T01:59:03.787 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "32", 2024-09-16T01:59:03.787 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.787 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T01:59:03.787 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.787 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.787 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.787 INFO:teuthology.orchestra.run.smithi139.stdout: "username": { 2024-09-16T01:59:03.787 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "username", 2024-09-16T01:59:03.787 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.787 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.787 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.788 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.788 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.788 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.788 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.788 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "username of InfluxDB server user", 2024-09-16T01:59:03.788 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.788 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.788 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.788 INFO:teuthology.orchestra.run.smithi139.stdout: "verify_ssl": { 2024-09-16T01:59:03.788 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "verify_ssl", 2024-09-16T01:59:03.788 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.789 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.789 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.789 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "true", 2024-09-16T01:59:03.789 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.789 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.789 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.789 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T01:59:03.789 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.789 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.789 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.789 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.789 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.790 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "insights", 2024-09-16T01:59:03.790 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.790 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.790 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.790 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.790 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.790 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.790 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.790 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.790 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.790 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.790 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.792 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.792 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.792 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.792 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.792 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.792 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.792 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.792 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.792 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.792 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.792 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.792 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.793 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.793 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.793 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.793 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.793 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.793 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.793 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.793 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.793 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.793 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.793 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.793 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.793 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.793 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.794 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.794 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.794 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.794 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.794 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.794 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.794 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.794 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.794 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.794 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.794 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.794 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.794 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.795 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.795 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.795 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.795 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.795 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.795 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.795 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.795 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "iostat", 2024-09-16T01:59:03.795 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.795 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.795 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.795 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.796 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.796 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.796 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.796 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.796 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.796 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.796 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.796 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.796 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.796 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.797 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.797 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.797 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.797 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.797 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.797 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.797 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.797 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.797 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.797 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.797 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.797 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.797 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.799 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.800 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.800 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.800 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.800 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.800 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.800 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.800 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.800 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.800 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.800 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.800 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.800 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.801 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.801 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.801 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.801 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.801 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.801 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "k8sevents", 2024-09-16T01:59:03.801 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.801 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.801 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.801 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_event_retention_days": { 2024-09-16T01:59:03.801 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ceph_event_retention_days", 2024-09-16T01:59:03.801 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.802 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.802 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.802 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "7", 2024-09-16T01:59:03.802 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.802 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.802 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.802 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T01:59:03.802 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.802 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.802 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.803 INFO:teuthology.orchestra.run.smithi139.stdout: "config_check_secs": { 2024-09-16T01:59:03.803 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "config_check_secs", 2024-09-16T01:59:03.803 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.803 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.803 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.803 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:03.803 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "10", 2024-09-16T01:59:03.803 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.803 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.803 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T01:59:03.803 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.804 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.804 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.804 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.804 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.804 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.804 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.804 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.804 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.804 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.804 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.804 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.804 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.805 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.805 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.805 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.805 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.805 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.805 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.805 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.805 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.805 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.805 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.805 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.807 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.807 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.807 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.807 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.807 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.807 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.807 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.807 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.807 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.807 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.807 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.807 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.807 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.807 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.808 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.808 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.808 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.808 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.808 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.808 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.808 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.808 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.808 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.808 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.808 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.808 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.808 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.809 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.809 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.809 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.809 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.809 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.809 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.809 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.809 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.809 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "localpool", 2024-09-16T01:59:03.809 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.809 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.810 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.810 INFO:teuthology.orchestra.run.smithi139.stdout: "failure_domain": { 2024-09-16T01:59:03.810 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "failure_domain", 2024-09-16T01:59:03.810 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.810 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.810 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.810 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "host", 2024-09-16T01:59:03.810 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.810 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.810 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.810 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "failure domain for any created local pool", 2024-09-16T01:59:03.810 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T01:59:03.810 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.810 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.811 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.811 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.811 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.811 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.811 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.811 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.811 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.811 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.811 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.811 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.811 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.811 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.811 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.813 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.813 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.813 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.813 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.813 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.813 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.813 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.813 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.813 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.813 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.813 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.813 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.814 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.814 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.814 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.814 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.814 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.814 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.814 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.814 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.814 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.814 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.814 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.814 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.814 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.814 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.815 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.815 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.815 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.815 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.815 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.815 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.815 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.815 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.815 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.815 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.815 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.815 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.815 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.816 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.816 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": { 2024-09-16T01:59:03.816 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "min_size", 2024-09-16T01:59:03.816 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.816 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.816 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.816 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.816 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.816 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.816 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.816 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "default min_size for any created local pool", 2024-09-16T01:59:03.816 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T01:59:03.817 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.817 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.817 INFO:teuthology.orchestra.run.smithi139.stdout: "num_rep": { 2024-09-16T01:59:03.817 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "num_rep", 2024-09-16T01:59:03.817 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.817 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.817 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.817 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "3", 2024-09-16T01:59:03.817 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.817 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.817 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.818 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "default replica count for any created local pool", 2024-09-16T01:59:03.818 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.818 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.818 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.818 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": { 2024-09-16T01:59:03.818 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "pg_num", 2024-09-16T01:59:03.818 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.818 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.818 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.818 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "128", 2024-09-16T01:59:03.818 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.818 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.818 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: "prefix": { 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "prefix", 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.819 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "name prefix for any created local pool", 2024-09-16T01:59:03.820 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.820 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.820 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.820 INFO:teuthology.orchestra.run.smithi139.stdout: "subtree": { 2024-09-16T01:59:03.820 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "subtree", 2024-09-16T01:59:03.820 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.820 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.820 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.820 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "rack", 2024-09-16T01:59:03.820 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.820 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.820 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.821 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T01:59:03.821 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T01:59:03.821 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.821 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.821 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.821 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.821 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mds_autoscaler", 2024-09-16T01:59:03.821 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.821 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.821 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.821 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.822 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.822 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.822 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.822 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.822 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.822 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.822 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.822 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.822 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.822 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.822 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.822 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.822 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.822 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.823 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.823 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.823 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.823 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.823 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.823 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.823 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.823 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.823 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.823 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.823 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.825 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.825 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.825 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.825 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.825 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.825 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.825 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.825 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.825 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.825 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.825 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.825 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.825 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.825 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.826 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.826 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.826 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.826 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.826 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.826 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.826 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.826 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.826 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.826 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.826 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.826 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.827 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.827 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.827 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.827 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.827 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mirroring", 2024-09-16T01:59:03.827 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.827 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.827 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.827 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.827 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.827 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.827 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.828 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.828 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.828 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.828 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.828 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.828 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.828 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.828 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.828 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.828 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.828 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.828 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.828 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.828 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.829 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.829 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.829 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.829 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.829 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.829 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.829 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.829 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.829 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.829 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.829 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.829 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.829 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.830 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.830 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.830 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.830 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.830 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.830 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.830 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.830 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.830 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.830 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.830 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.830 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.830 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.831 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.831 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.831 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.831 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.831 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.831 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.831 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.831 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.831 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.831 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.831 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.831 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.831 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.832 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.832 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.832 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.832 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.832 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.832 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.832 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.832 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.832 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.832 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.832 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.833 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "nfs", 2024-09-16T01:59:03.833 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.833 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.833 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.833 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.833 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.833 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.833 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.833 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.833 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.833 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.833 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.833 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.834 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.835 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.835 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.835 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.835 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.835 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.835 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.835 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.835 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.835 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.835 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.835 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.835 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.835 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.836 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.836 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.836 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.836 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.836 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.836 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.836 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.836 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.836 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.836 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.836 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.836 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.836 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.837 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.837 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.837 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.837 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.837 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.837 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.837 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.837 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.837 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.838 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.838 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.838 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.838 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.838 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.838 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.838 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.838 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.838 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.838 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.838 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "orchestrator", 2024-09-16T01:59:03.839 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.839 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.839 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.839 INFO:teuthology.orchestra.run.smithi139.stdout: "fail_fs": { 2024-09-16T01:59:03.839 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "fail_fs", 2024-09-16T01:59:03.839 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.839 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.839 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.839 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.839 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.839 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.839 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.839 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T01:59:03.839 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.840 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.840 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.840 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.840 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.840 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.840 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.840 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.840 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.840 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.840 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.840 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.840 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.841 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.841 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.841 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.841 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.841 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.841 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.841 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.841 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.841 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.841 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.841 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.841 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.841 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.842 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.842 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.842 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.842 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.842 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.842 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.842 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.842 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.842 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.842 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.842 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.842 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.842 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.843 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.844 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.844 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.844 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.844 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.844 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.844 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.844 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.844 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.844 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.844 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.844 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.844 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.845 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.845 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.845 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.845 INFO:teuthology.orchestra.run.smithi139.stdout: "orchestrator": { 2024-09-16T01:59:03.845 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "orchestrator", 2024-09-16T01:59:03.845 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.845 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.845 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.845 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.845 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.845 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.845 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.846 INFO:teuthology.orchestra.run.smithi139.stdout: "cephadm", 2024-09-16T01:59:03.846 INFO:teuthology.orchestra.run.smithi139.stdout: "rook", 2024-09-16T01:59:03.846 INFO:teuthology.orchestra.run.smithi139.stdout: "test_orchestrator" 2024-09-16T01:59:03.846 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.846 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Orchestrator backend", 2024-09-16T01:59:03.846 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.846 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.846 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.846 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.846 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.847 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "osd_perf_query", 2024-09-16T01:59:03.847 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.847 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.847 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.847 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.847 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.847 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.847 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.847 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.847 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.847 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.847 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.847 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.848 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.848 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.848 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.848 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.848 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.848 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.848 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.848 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.848 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.848 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.848 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.848 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.848 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.849 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.849 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.849 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.849 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.849 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.849 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.849 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.849 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.849 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.849 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.849 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.849 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.849 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.850 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.850 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.850 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.850 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.850 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.850 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.850 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.850 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.850 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.850 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.850 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.850 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.850 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.851 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.851 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.851 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.851 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.851 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.851 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.851 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.851 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.851 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.851 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.851 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.851 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.851 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.852 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.852 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.852 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.852 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.852 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.852 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.852 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.852 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "osd_support", 2024-09-16T01:59:03.852 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.852 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.852 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.852 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.853 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.853 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.853 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.853 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.853 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.853 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.853 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.853 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.853 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.853 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.853 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.853 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.853 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.854 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.854 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.854 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.854 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.854 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.854 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.854 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.854 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.854 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.854 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.854 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.854 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.855 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.855 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.855 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.855 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.855 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.855 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.855 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.855 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.855 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.855 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.855 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.855 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.855 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.856 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.856 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.856 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.856 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.856 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.856 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.856 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.856 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.856 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.856 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.856 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.856 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.856 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.856 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.857 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.857 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.857 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.857 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.857 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.857 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.857 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.857 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.857 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.857 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.857 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.857 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.857 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.858 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.858 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.858 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "pg_autoscaler", 2024-09-16T01:59:03.858 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.858 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.858 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.858 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.858 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.858 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.858 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.858 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.859 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.860 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.860 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.860 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.860 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.860 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.860 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.860 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.860 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.860 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.860 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.860 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.860 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.860 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.861 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.861 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.861 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.861 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.861 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.861 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.861 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.861 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.861 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.861 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.861 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.863 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.863 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.863 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.863 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.863 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.863 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.863 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.863 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.863 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.863 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:03.863 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:03.863 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.864 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.864 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.864 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:03.864 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.864 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.864 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.864 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.864 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.864 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.864 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.864 INFO:teuthology.orchestra.run.smithi139.stdout: "threshold": { 2024-09-16T01:59:03.864 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "threshold", 2024-09-16T01:59:03.865 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:03.865 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.865 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.865 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "3.0", 2024-09-16T01:59:03.865 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1.0", 2024-09-16T01:59:03.865 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.865 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.865 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "scaling threshold", 2024-09-16T01:59:03.865 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T01:59:03.865 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.865 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.865 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.865 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.866 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "progress", 2024-09-16T01:59:03.866 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.866 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.866 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.866 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_pg_recovery_event": { 2024-09-16T01:59:03.866 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "allow_pg_recovery_event", 2024-09-16T01:59:03.866 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.866 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.866 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.866 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.866 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.866 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.867 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.867 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T01:59:03.867 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.867 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.867 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.867 INFO:teuthology.orchestra.run.smithi139.stdout: "enabled": { 2024-09-16T01:59:03.867 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "enabled", 2024-09-16T01:59:03.867 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.867 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.867 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.867 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.867 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.867 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.868 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.868 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.868 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.868 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.868 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.868 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.868 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.868 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.868 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.868 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.868 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.868 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.868 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.869 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.869 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.869 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.869 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.869 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.869 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.869 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.869 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.869 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.869 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.869 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.869 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.869 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.870 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.870 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.870 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.870 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.870 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.870 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.870 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.870 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.870 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.870 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.870 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.870 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.870 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.871 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.871 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.871 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.871 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.871 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.871 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.871 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.871 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.871 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.871 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.871 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.871 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.871 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.871 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.872 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.872 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.872 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.872 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.872 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.872 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.872 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.872 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.872 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.872 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.872 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.872 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.873 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.873 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.873 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.873 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.873 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.873 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.873 INFO:teuthology.orchestra.run.smithi139.stdout: "max_completed_events": { 2024-09-16T01:59:03.873 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_completed_events", 2024-09-16T01:59:03.873 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.873 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.873 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.873 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "50", 2024-09-16T01:59:03.873 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.874 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.874 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.874 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "number of past completed events to remember", 2024-09-16T01:59:03.874 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.874 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.874 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.874 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:03.874 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:03.874 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.874 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.874 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.874 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:03.874 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.875 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.875 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.875 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how long the module is going to sleep", 2024-09-16T01:59:03.875 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.875 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.875 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.875 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.875 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.875 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "prometheus", 2024-09-16T01:59:03.875 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.875 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.876 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.876 INFO:teuthology.orchestra.run.smithi139.stdout: "cache": { 2024-09-16T01:59:03.876 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cache", 2024-09-16T01:59:03.876 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.876 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.876 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.876 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.876 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.876 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.876 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.876 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.876 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.876 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.876 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.877 INFO:teuthology.orchestra.run.smithi139.stdout: "exclude_perf_counters": { 2024-09-16T01:59:03.877 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "exclude_perf_counters", 2024-09-16T01:59:03.877 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.877 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.877 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.877 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.877 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.877 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.877 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.877 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T01:59:03.877 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T01:59:03.877 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.878 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.878 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.878 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.878 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.878 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.878 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.878 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.878 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.878 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.878 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.878 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.878 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.878 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.879 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.879 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.879 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.879 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.879 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.879 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.879 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.879 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.879 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.879 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.879 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.879 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.880 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.880 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.880 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.880 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.880 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.880 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.880 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.880 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.880 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.880 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.880 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.880 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.880 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.881 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.881 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.881 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.881 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.881 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.881 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.881 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.881 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.881 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.881 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.881 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.881 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.881 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.881 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.882 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.882 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.882 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.882 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.882 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.882 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.882 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.882 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.882 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.882 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.882 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.882 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.882 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.883 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.883 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.883 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_stats_pools": { 2024-09-16T01:59:03.883 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rbd_stats_pools", 2024-09-16T01:59:03.883 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.883 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.883 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.883 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.883 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.883 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.883 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.883 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.883 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.884 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.884 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.884 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T01:59:03.884 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T01:59:03.884 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.884 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.884 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.884 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "300", 2024-09-16T01:59:03.884 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.884 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.884 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.884 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.884 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.885 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.885 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.885 INFO:teuthology.orchestra.run.smithi139.stdout: "scrape_interval": { 2024-09-16T01:59:03.885 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "scrape_interval", 2024-09-16T01:59:03.885 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:03.885 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.885 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.885 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "15.0", 2024-09-16T01:59:03.885 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.885 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.885 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.885 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.885 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.886 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.886 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.886 INFO:teuthology.orchestra.run.smithi139.stdout: "server_addr": { 2024-09-16T01:59:03.886 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_addr", 2024-09-16T01:59:03.886 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.886 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.886 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.886 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "::", 2024-09-16T01:59:03.886 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.886 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.886 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.886 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T01:59:03.886 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.887 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.887 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.887 INFO:teuthology.orchestra.run.smithi139.stdout: "server_port": { 2024-09-16T01:59:03.887 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_port", 2024-09-16T01:59:03.887 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.887 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.887 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.887 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "9283", 2024-09-16T01:59:03.887 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.887 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.887 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.887 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T01:59:03.887 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.888 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.888 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.888 INFO:teuthology.orchestra.run.smithi139.stdout: "stale_cache_strategy": { 2024-09-16T01:59:03.888 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "stale_cache_strategy", 2024-09-16T01:59:03.888 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.888 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.888 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.888 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "log", 2024-09-16T01:59:03.888 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.888 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.888 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.889 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.889 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.889 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.889 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.889 INFO:teuthology.orchestra.run.smithi139.stdout: "standby_behaviour": { 2024-09-16T01:59:03.889 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "standby_behaviour", 2024-09-16T01:59:03.889 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.889 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.889 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.889 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "default", 2024-09-16T01:59:03.889 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.889 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.889 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.890 INFO:teuthology.orchestra.run.smithi139.stdout: "default", 2024-09-16T01:59:03.890 INFO:teuthology.orchestra.run.smithi139.stdout: "error" 2024-09-16T01:59:03.890 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.890 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.890 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.890 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.890 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.890 INFO:teuthology.orchestra.run.smithi139.stdout: "standby_error_status_code": { 2024-09-16T01:59:03.890 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "standby_error_status_code", 2024-09-16T01:59:03.890 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.890 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.890 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.890 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "500", 2024-09-16T01:59:03.891 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "400", 2024-09-16T01:59:03.891 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "599", 2024-09-16T01:59:03.891 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.891 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.891 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.891 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.891 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.891 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.891 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.891 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rbd_support", 2024-09-16T01:59:03.891 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.891 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.892 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.892 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.892 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.892 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.892 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.892 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.892 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.892 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.892 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.892 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.892 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.892 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.892 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.893 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.893 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.893 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.893 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.893 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.893 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.893 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.893 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.893 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.893 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.893 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.893 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.895 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.895 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.895 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.895 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.895 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.895 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.895 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.895 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.895 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.895 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.895 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.895 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.895 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.895 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.896 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.896 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.896 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.896 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.896 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.896 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.896 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.896 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.896 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.896 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.896 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.896 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.897 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.897 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.897 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.897 INFO:teuthology.orchestra.run.smithi139.stdout: "max_concurrent_snap_create": { 2024-09-16T01:59:03.897 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_concurrent_snap_create", 2024-09-16T01:59:03.897 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.897 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.897 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.897 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:03.897 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.897 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.897 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.897 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: "mirror_snapshot_schedule": { 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.898 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.899 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.899 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.899 INFO:teuthology.orchestra.run.smithi139.stdout: "trash_purge_schedule": { 2024-09-16T01:59:03.899 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "trash_purge_schedule", 2024-09-16T01:59:03.899 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.899 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.899 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.899 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.899 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.899 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.899 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.899 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.900 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.900 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.900 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.900 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.900 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.900 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "restful", 2024-09-16T01:59:03.900 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.900 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.900 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.900 INFO:teuthology.orchestra.run.smithi139.stdout: "enable_auth": { 2024-09-16T01:59:03.900 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "enable_auth", 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: "key_file": { 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "key_file", 2024-09-16T01:59:03.901 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.902 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.902 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.902 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.902 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.902 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.902 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.902 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.902 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.902 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.902 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.903 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.903 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.903 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.903 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.903 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.903 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.903 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.903 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.903 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.903 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.903 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.903 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: "server_addr": { 2024-09-16T01:59:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_addr", 2024-09-16T01:59:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: "server_port": { 2024-09-16T01:59:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_port", 2024-09-16T01:59:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rgw", 2024-09-16T01:59:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T01:59:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T01:59:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T01:59:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rook", 2024-09-16T01:59:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: "storage_class": { 2024-09-16T01:59:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "storage_class", 2024-09-16T01:59:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "local", 2024-09-16T01:59:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T01:59:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "selftest", 2024-09-16T01:59:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: "roption1": { 2024-09-16T01:59:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "roption1", 2024-09-16T01:59:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: "roption2": { 2024-09-16T01:59:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "roption2", 2024-09-16T01:59:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.931 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.931 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "xyz", 2024-09-16T01:59:03.931 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.931 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.931 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.931 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.931 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.931 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.931 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.931 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption1": { 2024-09-16T01:59:03.931 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption1", 2024-09-16T01:59:03.931 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.932 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.932 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.932 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.932 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.932 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.932 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.932 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.932 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.932 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.932 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.932 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption2": { 2024-09-16T01:59:03.932 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption2", 2024-09-16T01:59:03.932 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.933 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.933 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.933 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.933 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.933 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.933 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.933 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.933 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.933 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.933 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.933 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption3": { 2024-09-16T01:59:03.933 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption3", 2024-09-16T01:59:03.933 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:03.934 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.934 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.934 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.934 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.934 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.934 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.934 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.934 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.934 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.934 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.934 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption4": { 2024-09-16T01:59:03.934 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption4", 2024-09-16T01:59:03.935 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.935 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.935 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.935 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.935 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.935 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.935 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.935 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.935 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.935 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.935 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.935 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption5": { 2024-09-16T01:59:03.935 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption5", 2024-09-16T01:59:03.936 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.936 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.936 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.936 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.936 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.936 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.936 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.936 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.936 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.936 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.936 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.936 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption6": { 2024-09-16T01:59:03.937 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption6", 2024-09-16T01:59:03.937 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.937 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.937 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.937 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.937 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.937 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.937 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.937 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.937 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.937 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.938 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.938 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption7": { 2024-09-16T01:59:03.938 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption7", 2024-09-16T01:59:03.938 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.938 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.938 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.938 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.938 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1", 2024-09-16T01:59:03.938 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "42", 2024-09-16T01:59:03.938 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.938 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.938 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.939 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.939 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.939 INFO:teuthology.orchestra.run.smithi139.stdout: "testkey": { 2024-09-16T01:59:03.939 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "testkey", 2024-09-16T01:59:03.939 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.939 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.939 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.939 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.939 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.939 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.939 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.939 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.940 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.940 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.940 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.940 INFO:teuthology.orchestra.run.smithi139.stdout: "testlkey": { 2024-09-16T01:59:03.940 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "testlkey", 2024-09-16T01:59:03.940 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.940 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.940 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.940 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.940 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.940 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.940 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.941 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.941 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.941 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.941 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.941 INFO:teuthology.orchestra.run.smithi139.stdout: "testnewline": { 2024-09-16T01:59:03.941 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "testnewline", 2024-09-16T01:59:03.941 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.941 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.941 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.941 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.941 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.941 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.941 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.942 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.942 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.942 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.942 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.942 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.942 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.942 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "snap_schedule", 2024-09-16T01:59:03.942 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.942 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.942 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.942 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_m_granularity": { 2024-09-16T01:59:03.942 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "allow_m_granularity", 2024-09-16T01:59:03.943 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.943 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.943 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.943 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.943 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.943 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.943 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.943 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T01:59:03.943 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.943 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.943 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.943 INFO:teuthology.orchestra.run.smithi139.stdout: "dump_on_update": { 2024-09-16T01:59:03.943 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "dump_on_update", 2024-09-16T01:59:03.944 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.944 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.944 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.944 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.944 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.944 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.944 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.944 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "dump database to debug log on update", 2024-09-16T01:59:03.944 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.944 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.944 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.944 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.944 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.945 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.945 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.945 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.945 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.945 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.945 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.945 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.945 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.945 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.945 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.945 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.945 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.945 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.946 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.946 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.946 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.946 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.946 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.946 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.946 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.946 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.946 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.946 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.946 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.946 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.947 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.947 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.947 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.947 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.947 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.947 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.947 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.947 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.947 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.947 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.947 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.947 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.948 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.948 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.948 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.948 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.948 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.948 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.948 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.948 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.948 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.948 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.948 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.948 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.948 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.948 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.949 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.949 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.949 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.949 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.949 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.949 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.949 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.949 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.949 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.949 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.949 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.949 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.950 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.950 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.950 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.950 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "stats", 2024-09-16T01:59:03.950 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.950 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.950 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.950 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.950 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.950 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.950 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.951 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.951 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.951 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.951 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.951 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.951 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.951 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.951 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.951 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.951 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.951 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.951 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.951 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.951 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.952 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.952 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.952 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.952 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.952 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.952 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.952 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.952 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.952 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.952 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.952 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.952 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.953 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.953 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.953 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.953 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.953 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.953 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.953 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.953 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.953 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.953 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.953 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.953 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.953 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.954 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.954 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.954 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.954 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.954 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.954 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.954 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.954 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.954 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.954 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.954 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.954 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.954 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.955 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.955 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.955 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.955 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.955 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.955 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.955 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.955 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.955 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.955 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.955 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.955 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.956 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "status", 2024-09-16T01:59:03.956 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.956 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.956 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.956 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.956 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.956 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.956 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.956 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.956 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.956 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.956 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.956 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.956 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.957 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.957 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.957 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.957 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.957 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.957 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.957 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.957 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.957 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.957 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.957 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.957 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.957 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.958 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.958 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.958 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.958 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.958 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.958 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.958 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.958 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.958 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.958 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.958 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.958 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.958 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.959 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.959 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.959 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.959 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.959 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.959 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.959 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.959 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.959 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.959 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.959 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.959 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.959 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.959 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.960 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.960 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.960 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.960 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.960 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.960 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.960 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.960 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.960 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.960 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.960 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.960 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.961 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.961 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.961 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.961 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.961 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.961 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.961 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "telegraf", 2024-09-16T01:59:03.961 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.961 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.961 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.961 INFO:teuthology.orchestra.run.smithi139.stdout: "address": { 2024-09-16T01:59:03.962 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "address", 2024-09-16T01:59:03.962 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.962 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.962 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.962 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T01:59:03.962 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.962 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.962 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.962 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.962 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.962 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.962 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.962 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:03.963 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:03.963 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:03.963 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.963 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.963 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "15", 2024-09-16T01:59:03.963 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.963 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.963 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.963 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.963 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.963 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.963 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.966 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.966 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.966 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.966 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.966 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.966 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.966 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.966 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.966 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.966 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.966 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.966 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.966 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.967 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.967 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.967 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.967 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.967 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.967 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.967 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.967 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.967 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.967 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.967 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.967 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.967 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.968 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.968 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.968 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.968 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.968 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.968 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.968 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.968 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.968 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.968 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.968 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.968 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.968 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.969 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.969 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.969 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "telemetry", 2024-09-16T01:59:03.969 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.969 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.969 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.969 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_basic": { 2024-09-16T01:59:03.969 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_basic", 2024-09-16T01:59:03.969 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.969 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.969 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.969 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.970 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.970 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.970 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.970 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T01:59:03.970 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.970 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.970 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.970 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_crash": { 2024-09-16T01:59:03.970 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_crash", 2024-09-16T01:59:03.970 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.970 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.970 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.970 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.971 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.971 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.971 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.971 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T01:59:03.971 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.971 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.971 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.971 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_device": { 2024-09-16T01:59:03.971 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_device", 2024-09-16T01:59:03.971 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.971 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.971 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.971 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:03.972 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.972 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.972 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.972 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T01:59:03.972 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.972 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.972 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.972 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_ident": { 2024-09-16T01:59:03.972 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_ident", 2024-09-16T01:59:03.972 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.972 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.972 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T01:59:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_perf": { 2024-09-16T01:59:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_perf", 2024-09-16T01:59:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.974 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.974 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.974 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.974 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.974 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T01:59:03.974 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.974 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.974 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.974 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.974 INFO:teuthology.orchestra.run.smithi139.stdout: "contact": { 2024-09-16T01:59:03.974 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "contact", 2024-09-16T01:59:03.974 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.974 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.975 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.975 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.975 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.975 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.975 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.975 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.975 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.975 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.975 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.975 INFO:teuthology.orchestra.run.smithi139.stdout: "description": { 2024-09-16T01:59:03.975 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "description", 2024-09-16T01:59:03.975 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.975 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.976 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.976 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.976 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.976 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.976 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.976 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.976 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.976 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.976 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.976 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.976 INFO:teuthology.orchestra.run.smithi139.stdout: "device_url": { 2024-09-16T01:59:03.976 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "device_url", 2024-09-16T01:59:03.976 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.977 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.977 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.977 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T01:59:03.977 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.977 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.977 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.977 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.977 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.977 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.977 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.977 INFO:teuthology.orchestra.run.smithi139.stdout: "enabled": { 2024-09-16T01:59:03.977 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "enabled", 2024-09-16T01:59:03.977 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.978 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.978 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.978 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.978 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.978 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.978 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.978 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.978 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.978 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.978 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.978 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:03.978 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:03.978 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "24", 2024-09-16T01:59:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "8", 2024-09-16T01:59:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "last_opt_revision": { 2024-09-16T01:59:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "last_opt_revision", 2024-09-16T01:59:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:03.980 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.980 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.980 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "1", 2024-09-16T01:59:03.980 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.980 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.980 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.980 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.980 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.980 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.980 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.980 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.980 INFO:teuthology.orchestra.run.smithi139.stdout: "leaderboard": { 2024-09-16T01:59:03.980 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "leaderboard", 2024-09-16T01:59:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: "leaderboard_description": { 2024-09-16T01:59:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "leaderboard_description", 2024-09-16T01:59:03.982 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.982 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.982 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.982 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.982 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.982 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.982 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.982 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.982 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.982 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.982 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.982 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.982 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.983 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.983 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.983 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.983 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.983 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.983 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.983 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.983 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.983 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.983 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.983 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.983 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.983 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.983 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.984 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.984 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.984 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.984 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.984 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.984 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.984 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.984 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.984 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.984 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.984 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.984 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.985 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.985 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.985 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.985 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.985 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.985 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.985 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.985 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.985 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.985 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.985 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.985 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.985 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.987 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.987 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.987 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.987 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.987 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.987 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.987 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.987 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.987 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.987 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.987 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.987 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.987 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.987 INFO:teuthology.orchestra.run.smithi139.stdout: "organization": { 2024-09-16T01:59:03.988 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "organization", 2024-09-16T01:59:03.988 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.988 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.988 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.988 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.988 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.988 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.988 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.988 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.988 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.988 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.989 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.989 INFO:teuthology.orchestra.run.smithi139.stdout: "proxy": { 2024-09-16T01:59:03.989 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "proxy", 2024-09-16T01:59:03.989 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.989 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.989 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.989 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.989 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.989 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.989 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.989 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.989 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.990 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.990 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.990 INFO:teuthology.orchestra.run.smithi139.stdout: "url": { 2024-09-16T01:59:03.990 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "url", 2024-09-16T01:59:03.990 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.990 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.990 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:03.990 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T01:59:03.990 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.990 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.990 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.991 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.991 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.991 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.991 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.991 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.991 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.991 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "test_orchestrator", 2024-09-16T01:59:03.991 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.991 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.991 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.991 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.991 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.992 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.992 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.992 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.992 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.992 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.992 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.992 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.992 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.992 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.992 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.992 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.992 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.992 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.993 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.993 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.993 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.993 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.993 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.993 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.993 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.993 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.993 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.993 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.993 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.993 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.993 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.993 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.994 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.994 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.994 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.994 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.994 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.994 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:03.994 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:03.994 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.994 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.994 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.994 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:03.994 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.995 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.995 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.995 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.995 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.995 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.995 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.995 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.995 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.995 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.995 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.995 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.995 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.995 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.996 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.996 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:03.996 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:03.996 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.996 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.996 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.996 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.996 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.996 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.996 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.996 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.996 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.996 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.997 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.997 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.997 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:03.997 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:03.997 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "volumes", 2024-09-16T01:59:03.997 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:03.997 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:03.997 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:03.997 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:03.997 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:03.997 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:03.997 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.997 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.998 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:03.998 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.998 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.998 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:03.998 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:03.998 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:03.998 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:03.998 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:03.998 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:03.998 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:03.998 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:03.998 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.998 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.998 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:03.999 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:03.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:03.999 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:03.999 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:03.999 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:03.999 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:03.999 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:03.999 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:03.999 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:03.999 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:03.999 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:03.999 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:03.999 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:03.999 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.000 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.000 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.000 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.000 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.000 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.000 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.000 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.000 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.000 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.000 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.000 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.000 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.001 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.001 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.001 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.001 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.001 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.001 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.001 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.001 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.001 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.001 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.001 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.001 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.001 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.002 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.002 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.002 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.002 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.002 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.002 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.002 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.002 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.002 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.002 INFO:teuthology.orchestra.run.smithi139.stdout: "max_concurrent_clones": { 2024-09-16T01:59:04.002 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_concurrent_clones", 2024-09-16T01:59:04.002 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.002 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.003 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.003 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "4", 2024-09-16T01:59:04.003 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.003 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.003 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.003 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T01:59:04.003 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.003 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.003 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.003 INFO:teuthology.orchestra.run.smithi139.stdout: "periodic_async_work": { 2024-09-16T01:59:04.003 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "periodic_async_work", 2024-09-16T01:59:04.003 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.004 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.004 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.004 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.004 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.004 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.004 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.004 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Periodically check for async work", 2024-09-16T01:59:04.004 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.004 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.004 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.004 INFO:teuthology.orchestra.run.smithi139.stdout: "snapshot_clone_delay": { 2024-09-16T01:59:04.004 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "snapshot_clone_delay", 2024-09-16T01:59:04.005 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.005 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.005 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.005 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:04.005 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.005 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.005 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.005 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T01:59:04.005 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.005 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.005 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.005 INFO:teuthology.orchestra.run.smithi139.stdout: "snapshot_clone_no_wait": { 2024-09-16T01:59:04.005 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T01:59:04.006 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.006 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.006 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.006 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.006 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.006 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.006 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.006 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T01:59:04.006 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.006 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.006 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.006 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.006 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.007 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "zabbix", 2024-09-16T01:59:04.007 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.007 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.007 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.007 INFO:teuthology.orchestra.run.smithi139.stdout: "discovery_interval": { 2024-09-16T01:59:04.007 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "discovery_interval", 2024-09-16T01:59:04.007 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:04.007 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.007 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.007 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "100", 2024-09-16T01:59:04.007 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.007 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.008 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.008 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.008 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.008 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.008 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.008 INFO:teuthology.orchestra.run.smithi139.stdout: "identifier": { 2024-09-16T01:59:04.008 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "identifier", 2024-09-16T01:59:04.008 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.008 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.008 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.008 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.008 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.008 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.009 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.009 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.009 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.009 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.009 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.009 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:04.009 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:04.009 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.009 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.009 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.009 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:04.009 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.010 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.010 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.010 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.010 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.010 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.010 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.010 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.010 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.010 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.010 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.010 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.010 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.011 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.011 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.011 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.011 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.011 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.011 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.011 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.011 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.011 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.011 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.011 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.011 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.011 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.011 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.012 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.012 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.012 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.012 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.012 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.012 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.012 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.012 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.012 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.012 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.012 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.012 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.012 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.013 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.013 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.013 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.013 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.013 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.013 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.013 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.013 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.013 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.013 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.013 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.013 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.013 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.014 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.014 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.014 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.014 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.014 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.014 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.014 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.014 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.014 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.014 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.014 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.014 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.014 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: "zabbix_host": { 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "zabbix_host", 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.015 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.016 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.016 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.016 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.016 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.016 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.016 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.016 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.016 INFO:teuthology.orchestra.run.smithi139.stdout: "zabbix_port": { 2024-09-16T01:59:04.016 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "zabbix_port", 2024-09-16T01:59:04.016 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.016 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.016 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.016 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10051", 2024-09-16T01:59:04.017 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.017 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.017 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.017 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.017 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.017 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.017 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.017 INFO:teuthology.orchestra.run.smithi139.stdout: "zabbix_sender": { 2024-09-16T01:59:04.017 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "zabbix_sender", 2024-09-16T01:59:04.017 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.017 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.017 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.018 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T01:59:04.018 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.018 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.018 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.018 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.018 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.018 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.018 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.018 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.018 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.018 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.018 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T01:59:04.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.019 INFO:teuthology.orchestra.run.smithi139.stdout: "gid": 24923, 2024-09-16T01:59:04.019 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "y", 2024-09-16T01:59:04.019 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_features": 4540701547738038271, 2024-09-16T01:59:04.019 INFO:teuthology.orchestra.run.smithi139.stdout: "available_modules": [ 2024-09-16T01:59:04.019 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.019 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "alerts", 2024-09-16T01:59:04.019 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.019 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.019 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.019 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:04.019 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:04.019 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.019 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.020 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.020 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:04.020 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.020 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.020 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.020 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T01:59:04.020 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.020 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.020 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.020 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.020 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.020 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.021 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.021 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.021 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.021 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.021 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.021 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.021 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.021 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.021 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.021 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.021 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.021 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.021 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.022 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.022 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.022 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.022 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.022 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.022 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.022 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.022 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.022 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.022 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.022 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.022 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.022 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.023 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.023 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.023 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.023 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.023 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.023 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.023 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.023 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.023 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.023 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.023 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.023 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.023 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.023 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.024 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.024 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.024 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.024 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.024 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.024 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.024 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.024 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.024 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.024 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.024 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.024 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.025 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.025 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.025 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.025 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.025 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.025 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.025 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.025 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.025 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.025 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.025 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.025 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.025 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.026 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_destination": { 2024-09-16T01:59:04.026 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_destination", 2024-09-16T01:59:04.026 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.026 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.026 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.026 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.026 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.026 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.026 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.026 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Email address to send alerts to", 2024-09-16T01:59:04.026 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.026 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.026 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.027 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_from_name": { 2024-09-16T01:59:04.027 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_from_name", 2024-09-16T01:59:04.027 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.027 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.027 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.027 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "Ceph", 2024-09-16T01:59:04.027 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.027 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.027 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.027 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Email From: name", 2024-09-16T01:59:04.027 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.027 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.027 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.028 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_host": { 2024-09-16T01:59:04.028 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_host", 2024-09-16T01:59:04.028 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.028 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.028 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.028 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.028 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.028 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.028 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.028 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "SMTP server", 2024-09-16T01:59:04.028 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.028 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.028 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.029 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_password": { 2024-09-16T01:59:04.029 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_password", 2024-09-16T01:59:04.029 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.029 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.029 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.029 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.029 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.029 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.029 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.029 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Password to authenticate with", 2024-09-16T01:59:04.029 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.029 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.029 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.030 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_port": { 2024-09-16T01:59:04.030 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_port", 2024-09-16T01:59:04.030 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.030 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.030 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.030 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "465", 2024-09-16T01:59:04.030 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.030 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.030 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.030 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "SMTP port", 2024-09-16T01:59:04.030 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.030 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.030 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.031 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_sender": { 2024-09-16T01:59:04.031 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_sender", 2024-09-16T01:59:04.031 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.031 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.031 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.031 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.031 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.031 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.031 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.031 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "SMTP envelope sender", 2024-09-16T01:59:04.031 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.031 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.032 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.032 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.032 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_ssl": { 2024-09-16T01:59:04.032 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_ssl", 2024-09-16T01:59:04.032 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.032 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.032 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.032 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.032 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.032 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.032 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.032 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T01:59:04.032 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.033 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.033 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.033 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_user": { 2024-09-16T01:59:04.033 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_user", 2024-09-16T01:59:04.033 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.033 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.033 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.033 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.033 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.033 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.033 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.033 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "User to authenticate as", 2024-09-16T01:59:04.033 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.034 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.034 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.034 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.034 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.034 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "balancer", 2024-09-16T01:59:04.034 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.034 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.034 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.034 INFO:teuthology.orchestra.run.smithi139.stdout: "active": { 2024-09-16T01:59:04.034 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "active", 2024-09-16T01:59:04.034 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.034 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.035 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.035 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.035 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.035 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.035 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.035 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T01:59:04.035 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.035 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.035 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.035 INFO:teuthology.orchestra.run.smithi139.stdout: "begin_time": { 2024-09-16T01:59:04.035 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "begin_time", 2024-09-16T01:59:04.035 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.036 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.036 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.036 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0000", 2024-09-16T01:59:04.036 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.036 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.036 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.036 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T01:59:04.036 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T01:59:04.036 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.036 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.036 INFO:teuthology.orchestra.run.smithi139.stdout: "begin_weekday": { 2024-09-16T01:59:04.036 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "begin_weekday", 2024-09-16T01:59:04.037 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:04.037 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.037 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.037 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:04.037 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "0", 2024-09-16T01:59:04.037 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "6", 2024-09-16T01:59:04.037 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.037 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T01:59:04.037 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T01:59:04.037 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.037 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.037 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_compat_max_iterations": { 2024-09-16T01:59:04.037 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crush_compat_max_iterations", 2024-09-16T01:59:04.038 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:04.038 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.038 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.038 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "25", 2024-09-16T01:59:04.038 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1", 2024-09-16T01:59:04.038 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "250", 2024-09-16T01:59:04.038 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.038 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T01:59:04.038 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.038 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.038 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.038 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_compat_metrics": { 2024-09-16T01:59:04.039 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crush_compat_metrics", 2024-09-16T01:59:04.039 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.039 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.039 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.039 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T01:59:04.039 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.039 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.039 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.039 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T01:59:04.039 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T01:59:04.039 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.039 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.039 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_compat_step": { 2024-09-16T01:59:04.040 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crush_compat_step", 2024-09-16T01:59:04.040 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:04.040 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.040 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.040 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0.5", 2024-09-16T01:59:04.040 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "0.001", 2024-09-16T01:59:04.040 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "0.999", 2024-09-16T01:59:04.040 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.040 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "aggressiveness of optimization", 2024-09-16T01:59:04.040 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T01:59:04.040 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.040 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.040 INFO:teuthology.orchestra.run.smithi139.stdout: "end_time": { 2024-09-16T01:59:04.041 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "end_time", 2024-09-16T01:59:04.041 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.041 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.041 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.041 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "2359", 2024-09-16T01:59:04.041 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.041 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.041 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.041 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T01:59:04.041 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T01:59:04.041 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.041 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.041 INFO:teuthology.orchestra.run.smithi139.stdout: "end_weekday": { 2024-09-16T01:59:04.042 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "end_weekday", 2024-09-16T01:59:04.042 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:04.042 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.042 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.042 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:04.042 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "0", 2024-09-16T01:59:04.042 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "6", 2024-09-16T01:59:04.042 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.042 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T01:59:04.042 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T01:59:04.042 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.042 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.042 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.043 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.044 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.044 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.044 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.044 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.044 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.044 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.044 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.044 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.044 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.044 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.044 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.045 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.045 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.045 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.045 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.045 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.045 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.045 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.045 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.045 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.045 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.045 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.045 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.045 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.046 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.046 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.046 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.046 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.046 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.046 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.046 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.046 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.046 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.046 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.046 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.046 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.046 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.046 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.047 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.047 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.047 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.047 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.047 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.047 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.047 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.047 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.047 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.047 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.047 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.047 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.047 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.048 INFO:teuthology.orchestra.run.smithi139.stdout: "min_score": { 2024-09-16T01:59:04.048 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "min_score", 2024-09-16T01:59:04.048 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:04.048 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.048 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.048 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:04.048 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.048 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.048 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.048 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T01:59:04.048 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.048 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.049 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.049 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": { 2024-09-16T01:59:04.049 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mode", 2024-09-16T01:59:04.049 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.049 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.049 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.049 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "upmap", 2024-09-16T01:59:04.049 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.049 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.049 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.049 INFO:teuthology.orchestra.run.smithi139.stdout: "crush-compat", 2024-09-16T01:59:04.049 INFO:teuthology.orchestra.run.smithi139.stdout: "none", 2024-09-16T01:59:04.049 INFO:teuthology.orchestra.run.smithi139.stdout: "read", 2024-09-16T01:59:04.050 INFO:teuthology.orchestra.run.smithi139.stdout: "upmap", 2024-09-16T01:59:04.050 INFO:teuthology.orchestra.run.smithi139.stdout: "upmap-read" 2024-09-16T01:59:04.050 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.050 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Balancer mode", 2024-09-16T01:59:04.050 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.050 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.050 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.050 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_ids": { 2024-09-16T01:59:04.050 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "pool_ids", 2024-09-16T01:59:04.050 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.050 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.050 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.050 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.051 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.051 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.051 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.051 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T01:59:04.051 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.051 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.051 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.051 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:04.051 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:04.051 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.051 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.051 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.052 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:04.052 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.052 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.052 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.052 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T01:59:04.052 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.052 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.052 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.052 INFO:teuthology.orchestra.run.smithi139.stdout: "upmap_max_deviation": { 2024-09-16T01:59:04.052 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "upmap_max_deviation", 2024-09-16T01:59:04.052 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.052 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.052 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.053 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:04.053 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1", 2024-09-16T01:59:04.053 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.053 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.053 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T01:59:04.053 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T01:59:04.053 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.053 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.053 INFO:teuthology.orchestra.run.smithi139.stdout: "upmap_max_optimizations": { 2024-09-16T01:59:04.053 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "upmap_max_optimizations", 2024-09-16T01:59:04.053 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:04.053 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.054 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.054 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.054 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.054 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.054 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.054 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T01:59:04.054 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.054 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.054 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.054 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.054 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.054 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cephadm", 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "agent_down_multiplier": { 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "agent_down_multiplier", 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "3.0", 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.055 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.056 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.056 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.056 INFO:teuthology.orchestra.run.smithi139.stdout: "agent_refresh_rate": { 2024-09-16T01:59:04.056 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "agent_refresh_rate", 2024-09-16T01:59:04.056 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.056 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.056 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.056 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "20", 2024-09-16T01:59:04.056 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.056 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.056 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.056 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T01:59:04.056 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.056 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.057 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.057 INFO:teuthology.orchestra.run.smithi139.stdout: "agent_starting_port": { 2024-09-16T01:59:04.057 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "agent_starting_port", 2024-09-16T01:59:04.057 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.057 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.057 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.057 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "4721", 2024-09-16T01:59:04.057 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.057 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.057 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.057 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T01:59:04.058 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.058 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.058 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.058 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_ptrace": { 2024-09-16T01:59:04.058 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "allow_ptrace", 2024-09-16T01:59:04.058 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.058 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.058 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.058 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.058 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.058 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.058 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.058 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T01:59:04.059 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T01:59:04.059 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.059 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.059 INFO:teuthology.orchestra.run.smithi139.stdout: "autotune_interval": { 2024-09-16T01:59:04.059 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "autotune_interval", 2024-09-16T01:59:04.059 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.059 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.059 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.059 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:04.059 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.059 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.059 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.060 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T01:59:04.060 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.060 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.060 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.060 INFO:teuthology.orchestra.run.smithi139.stdout: "autotune_memory_target_ratio": { 2024-09-16T01:59:04.060 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T01:59:04.060 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:04.060 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.060 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.060 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0.7", 2024-09-16T01:59:04.060 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.060 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.060 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.061 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T01:59:04.061 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.061 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.061 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.061 INFO:teuthology.orchestra.run.smithi139.stdout: "cephadm_log_destination": { 2024-09-16T01:59:04.061 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cephadm_log_destination", 2024-09-16T01:59:04.061 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.061 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.061 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.061 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.061 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.061 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.061 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.062 INFO:teuthology.orchestra.run.smithi139.stdout: "file", 2024-09-16T01:59:04.062 INFO:teuthology.orchestra.run.smithi139.stdout: "file,syslog", 2024-09-16T01:59:04.062 INFO:teuthology.orchestra.run.smithi139.stdout: "syslog" 2024-09-16T01:59:04.062 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.062 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T01:59:04.062 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.062 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.062 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.062 INFO:teuthology.orchestra.run.smithi139.stdout: "cgroups_split": { 2024-09-16T01:59:04.062 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cgroups_split", 2024-09-16T01:59:04.062 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.062 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.063 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.063 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.063 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.063 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.063 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.063 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T01:59:04.063 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.063 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.063 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.063 INFO:teuthology.orchestra.run.smithi139.stdout: "config_checks_enabled": { 2024-09-16T01:59:04.063 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "config_checks_enabled", 2024-09-16T01:59:04.063 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.063 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.064 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.064 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.064 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.064 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.064 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.064 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T01:59:04.064 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.064 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.064 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.064 INFO:teuthology.orchestra.run.smithi139.stdout: "config_dashboard": { 2024-09-16T01:59:04.064 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "config_dashboard", 2024-09-16T01:59:04.064 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.064 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.065 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.065 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.065 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.065 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.065 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.065 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T01:59:04.065 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.065 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.065 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.065 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_alertmanager": { 2024-09-16T01:59:04.065 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_alertmanager", 2024-09-16T01:59:04.065 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.065 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Prometheus container image", 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_base": { 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_base", 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.066 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.067 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T01:59:04.067 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.067 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.067 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.067 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Container image name, without the tag", 2024-09-16T01:59:04.067 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.067 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.067 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.067 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_elasticsearch": { 2024-09-16T01:59:04.067 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_elasticsearch", 2024-09-16T01:59:04.067 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.067 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.067 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.068 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T01:59:04.068 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.068 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.068 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.068 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "elasticsearch container image", 2024-09-16T01:59:04.068 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.068 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.068 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.068 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.068 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_grafana": { 2024-09-16T01:59:04.068 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_grafana", 2024-09-16T01:59:04.068 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.068 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.068 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.069 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T01:59:04.069 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.069 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.069 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.069 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Prometheus container image", 2024-09-16T01:59:04.069 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.069 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.069 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.069 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_haproxy": { 2024-09-16T01:59:04.069 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_haproxy", 2024-09-16T01:59:04.069 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.069 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.070 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.070 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T01:59:04.070 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.070 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.070 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.070 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "HAproxy container image", 2024-09-16T01:59:04.070 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.070 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.070 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.070 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_jaeger_agent": { 2024-09-16T01:59:04.070 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_jaeger_agent", 2024-09-16T01:59:04.070 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.071 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.071 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.071 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T01:59:04.071 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.071 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.071 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.071 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Jaeger agent container image", 2024-09-16T01:59:04.071 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.071 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.071 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.071 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_jaeger_collector": { 2024-09-16T01:59:04.071 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_jaeger_collector", 2024-09-16T01:59:04.071 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.072 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.072 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.072 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T01:59:04.072 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.072 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.072 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.072 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Jaeger collector container image", 2024-09-16T01:59:04.072 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.072 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.072 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.072 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_jaeger_query": { 2024-09-16T01:59:04.072 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_jaeger_query", 2024-09-16T01:59:04.072 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.073 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.073 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.073 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T01:59:04.073 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.073 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.073 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.073 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Jaeger query container image", 2024-09-16T01:59:04.073 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.073 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.073 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.073 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_keepalived": { 2024-09-16T01:59:04.073 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_keepalived", 2024-09-16T01:59:04.074 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.074 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.074 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.074 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T01:59:04.074 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.074 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.074 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.074 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Keepalived container image", 2024-09-16T01:59:04.074 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.074 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.074 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.074 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_loki": { 2024-09-16T01:59:04.074 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_loki", 2024-09-16T01:59:04.075 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.075 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.075 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.075 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T01:59:04.075 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.075 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.075 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.075 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Loki container image", 2024-09-16T01:59:04.075 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.075 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.075 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.075 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_node_exporter": { 2024-09-16T01:59:04.076 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_node_exporter", 2024-09-16T01:59:04.076 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.076 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.076 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.076 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T01:59:04.076 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.076 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.076 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.076 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Prometheus container image", 2024-09-16T01:59:04.076 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.076 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.076 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.076 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_nvmeof": { 2024-09-16T01:59:04.077 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_nvmeof", 2024-09-16T01:59:04.077 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.077 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.077 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.077 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T01:59:04.077 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.077 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.077 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.077 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Nvme-of container image", 2024-09-16T01:59:04.077 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.077 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.077 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.077 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_prometheus": { 2024-09-16T01:59:04.078 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_prometheus", 2024-09-16T01:59:04.078 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.078 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.078 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.078 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T01:59:04.078 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.078 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.078 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.078 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Prometheus container image", 2024-09-16T01:59:04.078 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.078 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.078 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.078 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_promtail": { 2024-09-16T01:59:04.079 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_promtail", 2024-09-16T01:59:04.079 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.079 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.079 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.079 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T01:59:04.079 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.079 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.079 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.079 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Promtail container image", 2024-09-16T01:59:04.079 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.079 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.079 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.079 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_samba": { 2024-09-16T01:59:04.080 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_samba", 2024-09-16T01:59:04.080 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.080 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.080 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.080 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T01:59:04.080 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.080 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.080 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.080 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Samba/SMB container image", 2024-09-16T01:59:04.080 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.080 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.080 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.080 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_snmp_gateway": { 2024-09-16T01:59:04.081 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_snmp_gateway", 2024-09-16T01:59:04.081 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.081 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.081 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.081 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T01:59:04.081 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.081 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.081 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.081 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "SNMP Gateway container image", 2024-09-16T01:59:04.081 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.081 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.081 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.082 INFO:teuthology.orchestra.run.smithi139.stdout: "container_init": { 2024-09-16T01:59:04.082 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_init", 2024-09-16T01:59:04.082 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.082 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.082 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.082 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.082 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.082 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.082 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.082 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T01:59:04.082 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.082 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.083 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.083 INFO:teuthology.orchestra.run.smithi139.stdout: "daemon_cache_timeout": { 2024-09-16T01:59:04.083 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "daemon_cache_timeout", 2024-09-16T01:59:04.083 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.083 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.083 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.083 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:04.083 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.083 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.083 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.083 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T01:59:04.083 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "default_cephadm_command_timeout": { 2024-09-16T01:59:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T01:59:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "900", 2024-09-16T01:59:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T01:59:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.085 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.085 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.085 INFO:teuthology.orchestra.run.smithi139.stdout: "default_registry": { 2024-09-16T01:59:04.085 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "default_registry", 2024-09-16T01:59:04.085 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.085 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.085 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.085 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "docker.io", 2024-09-16T01:59:04.085 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.085 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.085 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.085 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T01:59:04.086 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.086 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.086 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.086 INFO:teuthology.orchestra.run.smithi139.stdout: "device_cache_timeout": { 2024-09-16T01:59:04.086 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "device_cache_timeout", 2024-09-16T01:59:04.086 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.086 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.086 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.086 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "1800", 2024-09-16T01:59:04.086 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.086 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.086 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.087 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "seconds to cache device inventory", 2024-09-16T01:59:04.087 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.087 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.087 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.087 INFO:teuthology.orchestra.run.smithi139.stdout: "device_enhanced_scan": { 2024-09-16T01:59:04.087 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "device_enhanced_scan", 2024-09-16T01:59:04.087 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.087 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.087 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.087 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.087 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.087 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.087 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.088 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T01:59:04.088 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.088 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.088 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.088 INFO:teuthology.orchestra.run.smithi139.stdout: "facts_cache_timeout": { 2024-09-16T01:59:04.088 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "facts_cache_timeout", 2024-09-16T01:59:04.088 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.088 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.088 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.088 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:04.088 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.088 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.089 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.089 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "seconds to cache host facts data", 2024-09-16T01:59:04.089 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.089 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.089 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.089 INFO:teuthology.orchestra.run.smithi139.stdout: "grafana_dashboards_path": { 2024-09-16T01:59:04.089 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "grafana_dashboards_path", 2024-09-16T01:59:04.089 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.089 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.089 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.089 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T01:59:04.089 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.089 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.090 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.090 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T01:59:04.090 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.090 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.090 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.090 INFO:teuthology.orchestra.run.smithi139.stdout: "host_check_interval": { 2024-09-16T01:59:04.090 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "host_check_interval", 2024-09-16T01:59:04.090 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.090 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.090 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.090 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:04.090 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.090 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.091 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.091 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to perform a host check", 2024-09-16T01:59:04.091 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.091 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.091 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.091 INFO:teuthology.orchestra.run.smithi139.stdout: "hw_monitoring": { 2024-09-16T01:59:04.091 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "hw_monitoring", 2024-09-16T01:59:04.091 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.091 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.091 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.091 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.091 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.091 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.092 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.092 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T01:59:04.092 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.092 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.092 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.092 INFO:teuthology.orchestra.run.smithi139.stdout: "inventory_list_all": { 2024-09-16T01:59:04.092 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "inventory_list_all", 2024-09-16T01:59:04.092 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.092 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.092 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.092 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.092 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.092 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.093 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.093 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T01:59:04.093 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.093 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.093 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.093 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.093 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.093 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.093 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.093 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.093 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.093 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.093 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.094 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.094 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.094 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.094 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.094 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.094 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.094 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.094 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.094 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.094 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.094 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.094 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.094 INFO:teuthology.orchestra.run.smithi139.stdout: "log_refresh_metadata": { 2024-09-16T01:59:04.095 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_refresh_metadata", 2024-09-16T01:59:04.095 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.095 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.095 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.095 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.095 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.095 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.095 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.095 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T01:59:04.095 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.095 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.095 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.095 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.096 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.096 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.096 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.096 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.096 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.096 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.096 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.096 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.096 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.096 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T01:59:04.096 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.096 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.096 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.097 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.097 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.097 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.097 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.097 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.097 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.097 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.097 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.097 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.097 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.097 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.097 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.097 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.097 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.098 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.098 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.098 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.098 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.098 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.098 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.098 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.098 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.098 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.098 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.098 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.098 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.098 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.099 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.099 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.099 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.099 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.099 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.099 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.099 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.099 INFO:teuthology.orchestra.run.smithi139.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T01:59:04.099 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T01:59:04.099 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.099 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.099 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.100 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.100 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.100 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.100 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.100 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T01:59:04.100 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.100 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.100 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.100 INFO:teuthology.orchestra.run.smithi139.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T01:59:04.100 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T01:59:04.100 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.100 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.101 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.101 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "*", 2024-09-16T01:59:04.101 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.101 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.101 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.101 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T01:59:04.101 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.101 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.101 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.101 INFO:teuthology.orchestra.run.smithi139.stdout: "max_count_per_host": { 2024-09-16T01:59:04.101 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_count_per_host", 2024-09-16T01:59:04.101 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.101 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.102 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.102 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.102 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.102 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.102 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.102 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "max number of daemons per service per host", 2024-09-16T01:59:04.102 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.102 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.102 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.102 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.102 INFO:teuthology.orchestra.run.smithi139.stdout: "max_osd_draining_count": { 2024-09-16T01:59:04.102 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_osd_draining_count", 2024-09-16T01:59:04.102 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.102 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: "migration_current": { 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "migration_current", 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.103 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.104 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.104 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.104 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.104 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.104 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "internal - do not modify", 2024-09-16T01:59:04.104 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.104 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.104 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.104 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": { 2024-09-16T01:59:04.104 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mode", 2024-09-16T01:59:04.104 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.104 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.105 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.105 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "root", 2024-09-16T01:59:04.105 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.105 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.105 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.105 INFO:teuthology.orchestra.run.smithi139.stdout: "cephadm-package", 2024-09-16T01:59:04.105 INFO:teuthology.orchestra.run.smithi139.stdout: "root" 2024-09-16T01:59:04.105 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.105 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T01:59:04.105 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.105 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.105 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.105 INFO:teuthology.orchestra.run.smithi139.stdout: "oob_default_addr": { 2024-09-16T01:59:04.106 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "oob_default_addr", 2024-09-16T01:59:04.106 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.106 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.106 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.106 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "169.254.1.1", 2024-09-16T01:59:04.106 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.106 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.106 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.106 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T01:59:04.106 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.106 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.106 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.106 INFO:teuthology.orchestra.run.smithi139.stdout: "prometheus_alerts_path": { 2024-09-16T01:59:04.107 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "prometheus_alerts_path", 2024-09-16T01:59:04.107 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.107 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.107 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.107 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T01:59:04.107 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.107 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.107 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.107 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T01:59:04.107 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.107 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.107 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.108 INFO:teuthology.orchestra.run.smithi139.stdout: "registry_insecure": { 2024-09-16T01:59:04.108 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "registry_insecure", 2024-09-16T01:59:04.108 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.108 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.108 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.108 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.108 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.108 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.108 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.108 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T01:59:04.108 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.108 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.108 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.109 INFO:teuthology.orchestra.run.smithi139.stdout: "registry_password": { 2024-09-16T01:59:04.109 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "registry_password", 2024-09-16T01:59:04.109 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.109 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.109 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.109 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.109 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.109 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.109 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.109 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T01:59:04.109 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.109 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.109 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.110 INFO:teuthology.orchestra.run.smithi139.stdout: "registry_url": { 2024-09-16T01:59:04.110 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "registry_url", 2024-09-16T01:59:04.110 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.110 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.110 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.110 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.110 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.110 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.110 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.110 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T01:59:04.110 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.110 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.110 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.111 INFO:teuthology.orchestra.run.smithi139.stdout: "registry_username": { 2024-09-16T01:59:04.111 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "registry_username", 2024-09-16T01:59:04.111 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.111 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.111 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.111 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.111 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.111 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.111 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.111 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T01:59:04.111 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.111 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.111 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.112 INFO:teuthology.orchestra.run.smithi139.stdout: "secure_monitoring_stack": { 2024-09-16T01:59:04.112 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "secure_monitoring_stack", 2024-09-16T01:59:04.112 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.112 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.112 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.112 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.112 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.112 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.112 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.112 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T01:59:04.112 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.112 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.113 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.113 INFO:teuthology.orchestra.run.smithi139.stdout: "service_discovery_port": { 2024-09-16T01:59:04.113 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "service_discovery_port", 2024-09-16T01:59:04.113 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.113 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.113 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.113 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8765", 2024-09-16T01:59:04.113 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.113 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.113 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.113 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "cephadm service discovery port", 2024-09-16T01:59:04.113 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.113 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.114 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.114 INFO:teuthology.orchestra.run.smithi139.stdout: "ssh_config_file": { 2024-09-16T01:59:04.114 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ssh_config_file", 2024-09-16T01:59:04.114 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.114 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.114 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.114 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.114 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.114 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.114 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.114 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T01:59:04.114 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.114 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.115 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.115 INFO:teuthology.orchestra.run.smithi139.stdout: "use_agent": { 2024-09-16T01:59:04.115 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "use_agent", 2024-09-16T01:59:04.115 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.115 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.115 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.115 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.115 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.115 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.115 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.115 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T01:59:04.115 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.115 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.116 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.116 INFO:teuthology.orchestra.run.smithi139.stdout: "use_repo_digest": { 2024-09-16T01:59:04.116 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "use_repo_digest", 2024-09-16T01:59:04.116 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.116 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.116 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.116 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.116 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.116 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.116 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.116 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T01:59:04.116 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.116 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.117 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.117 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_on_failed_host_check": { 2024-09-16T01:59:04.117 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_on_failed_host_check", 2024-09-16T01:59:04.117 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.117 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.117 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.117 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.117 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.117 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.117 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.117 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T01:59:04.117 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.117 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.118 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.118 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_on_stray_daemons": { 2024-09-16T01:59:04.118 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_on_stray_daemons", 2024-09-16T01:59:04.118 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.118 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.118 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.118 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.118 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.118 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.118 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.118 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T01:59:04.118 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.118 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.119 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.119 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_on_stray_hosts": { 2024-09-16T01:59:04.119 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_on_stray_hosts", 2024-09-16T01:59:04.119 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.119 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.119 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.119 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.119 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.119 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.119 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.119 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T01:59:04.119 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.119 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.120 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.120 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.120 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.120 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crash", 2024-09-16T01:59:04.120 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.120 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.120 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.120 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.120 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.120 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.120 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.121 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.121 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.121 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.121 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.121 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.121 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.121 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.121 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.121 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.121 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.121 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.121 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.121 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.121 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.122 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.122 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.122 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.122 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.122 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.122 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.122 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.122 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.122 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.122 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.122 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.122 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.123 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.123 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.123 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.123 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.123 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.123 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.123 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.123 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.123 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.123 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.123 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.123 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.123 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.124 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.124 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.124 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.124 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.124 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.124 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.124 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.124 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.124 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.124 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.124 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.124 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.124 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.125 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.125 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.125 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.125 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.125 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.125 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.125 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.125 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.125 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.125 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.125 INFO:teuthology.orchestra.run.smithi139.stdout: "retain_interval": { 2024-09-16T01:59:04.125 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "retain_interval", 2024-09-16T01:59:04.126 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.126 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.126 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.126 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "31536000", 2024-09-16T01:59:04.126 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.126 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.126 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.126 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T01:59:04.126 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.126 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.126 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.126 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_recent_interval": { 2024-09-16T01:59:04.126 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_recent_interval", 2024-09-16T01:59:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "1209600", 2024-09-16T01:59:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T01:59:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.128 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "dashboard", 2024-09-16T01:59:04.128 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.128 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.128 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.128 INFO:teuthology.orchestra.run.smithi139.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T01:59:04.128 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T01:59:04.128 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.128 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.128 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.128 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.128 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.128 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.128 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.129 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.129 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.129 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.129 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.129 INFO:teuthology.orchestra.run.smithi139.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T01:59:04.129 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T01:59:04.129 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.129 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.129 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.129 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.129 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.129 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.129 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.130 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.130 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.130 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.130 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.130 INFO:teuthology.orchestra.run.smithi139.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T01:59:04.130 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T01:59:04.130 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.130 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.130 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.130 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.130 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.130 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.130 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.131 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.131 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.131 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.131 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.131 INFO:teuthology.orchestra.run.smithi139.stdout: "AUDIT_API_ENABLED": { 2024-09-16T01:59:04.131 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T01:59:04.131 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.131 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.131 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.131 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.131 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.131 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.131 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.132 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.132 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.132 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.132 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.132 INFO:teuthology.orchestra.run.smithi139.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T01:59:04.132 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T01:59:04.132 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.132 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.132 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.132 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.132 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.132 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.132 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.133 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.133 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.133 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.133 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.133 INFO:teuthology.orchestra.run.smithi139.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T01:59:04.133 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T01:59:04.133 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.133 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.133 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.133 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.133 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.133 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.134 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.134 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.134 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.134 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.134 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.134 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T01:59:04.134 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T01:59:04.134 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.134 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.134 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.134 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.134 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.135 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.135 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.135 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.135 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.135 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.135 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.135 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T01:59:04.135 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T01:59:04.135 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.135 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.135 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.135 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.135 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.136 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.136 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.136 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.136 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.136 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.136 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.136 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T01:59:04.136 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T01:59:04.136 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.136 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.136 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.136 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.137 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.137 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.137 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.137 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.137 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.137 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.137 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.137 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T01:59:04.137 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T01:59:04.137 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.137 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.137 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.137 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.138 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.138 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.138 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.138 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.138 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.138 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.138 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.138 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T01:59:04.138 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T01:59:04.138 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.138 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.138 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.138 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.139 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.139 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.139 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.139 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.139 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.139 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.139 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.139 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T01:59:04.139 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T01:59:04.139 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.139 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.139 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.139 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.140 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.140 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.140 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.140 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.140 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.140 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.140 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.140 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T01:59:04.140 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T01:59:04.140 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.140 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.140 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.141 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.141 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.141 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.141 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.141 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.141 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.141 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.141 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.141 INFO:teuthology.orchestra.run.smithi139.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T01:59:04.141 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T01:59:04.141 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.141 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.141 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.142 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.142 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.142 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.142 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.142 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.142 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.142 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.142 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.142 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T01:59:04.142 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T01:59:04.142 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.142 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.142 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.143 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "admin", 2024-09-16T01:59:04.143 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.143 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.143 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.143 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.143 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.143 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.143 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.143 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T01:59:04.143 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T01:59:04.143 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.143 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.143 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.144 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.144 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.144 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.144 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.144 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.144 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.144 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.144 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.144 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_API_URL": { 2024-09-16T01:59:04.144 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_API_URL", 2024-09-16T01:59:04.144 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.144 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.144 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.145 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.145 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.145 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.145 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.145 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.145 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.145 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.145 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.145 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T01:59:04.145 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T01:59:04.145 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.145 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.145 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.146 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "admin", 2024-09-16T01:59:04.146 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.146 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.146 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.146 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.146 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.146 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.146 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.146 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T01:59:04.146 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T01:59:04.146 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.146 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.146 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.147 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.147 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.147 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.147 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.147 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.147 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.147 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.147 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.147 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T01:59:04.147 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T01:59:04.147 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.147 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.148 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.148 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.148 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.148 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.148 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.148 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.148 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.148 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.148 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.148 INFO:teuthology.orchestra.run.smithi139.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T01:59:04.148 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T01:59:04.148 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.149 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.149 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.149 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.149 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.149 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.149 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.149 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.149 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.149 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.149 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.149 INFO:teuthology.orchestra.run.smithi139.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T01:59:04.149 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T01:59:04.149 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.150 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.150 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.150 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.150 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.150 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.150 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.150 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.150 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.150 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.150 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.150 INFO:teuthology.orchestra.run.smithi139.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T01:59:04.150 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T01:59:04.150 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.151 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.151 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.151 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.151 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.151 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.151 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.151 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.151 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.151 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.151 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.151 INFO:teuthology.orchestra.run.smithi139.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T01:59:04.151 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T01:59:04.151 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.152 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.152 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.152 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.152 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.152 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.152 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.152 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.152 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.152 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.152 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.152 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T01:59:04.152 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T01:59:04.153 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.153 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.153 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.153 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.153 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.153 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.153 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.153 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.153 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.153 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.153 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.153 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T01:59:04.153 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T01:59:04.154 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.154 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.154 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.154 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.154 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.154 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.154 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.154 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.154 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.154 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.154 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.154 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T01:59:04.154 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T01:59:04.155 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.155 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.155 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.155 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.155 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.155 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.155 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.155 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.155 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.155 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.155 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.155 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T01:59:04.155 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T01:59:04.156 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.156 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.156 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.156 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.156 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.156 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.156 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.156 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.156 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.156 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.156 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.156 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T01:59:04.156 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T01:59:04.157 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.157 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.157 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.157 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.157 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.157 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.157 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.157 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.157 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.157 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.157 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.157 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T01:59:04.157 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T01:59:04.158 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.158 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.158 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.158 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.158 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.158 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.158 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.158 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.158 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.158 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.158 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.158 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T01:59:04.158 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T01:59:04.159 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.159 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.159 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.159 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.159 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.159 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.159 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.159 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.159 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.159 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.159 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.159 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T01:59:04.159 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T01:59:04.160 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.160 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.160 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.160 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.160 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.160 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.160 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.160 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.160 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.160 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.160 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.161 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T01:59:04.161 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T01:59:04.161 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.161 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.161 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.161 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T01:59:04.161 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.161 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.161 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.161 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.161 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.161 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.161 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.162 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T01:59:04.162 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T01:59:04.162 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.162 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.162 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.162 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.162 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.162 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.162 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.162 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.162 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.162 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.162 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.163 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T01:59:04.163 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T01:59:04.163 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.163 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.163 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.163 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8", 2024-09-16T01:59:04.163 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.163 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.163 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.163 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.163 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.163 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.163 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.164 INFO:teuthology.orchestra.run.smithi139.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T01:59:04.164 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T01:59:04.164 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.164 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.164 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.164 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "45", 2024-09-16T01:59:04.164 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.164 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.164 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.164 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.164 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.164 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.164 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.165 INFO:teuthology.orchestra.run.smithi139.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T01:59:04.165 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T01:59:04.165 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.165 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.165 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.165 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.165 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.165 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.165 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.165 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.165 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.165 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.165 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.166 INFO:teuthology.orchestra.run.smithi139.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T01:59:04.166 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T01:59:04.166 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.166 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.166 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.166 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "admin", 2024-09-16T01:59:04.166 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.166 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.166 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.166 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.166 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.166 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.167 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.168 INFO:teuthology.orchestra.run.smithi139.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T01:59:04.168 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T01:59:04.168 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.168 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.168 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.168 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.169 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.169 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.169 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.169 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.169 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.169 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.169 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.169 INFO:teuthology.orchestra.run.smithi139.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T01:59:04.169 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T01:59:04.169 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.169 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.169 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.169 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:04.170 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.171 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.171 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.171 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.171 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.171 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.171 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.171 INFO:teuthology.orchestra.run.smithi139.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T01:59:04.171 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T01:59:04.171 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.171 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.171 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.171 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.172 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.172 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.172 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.172 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.172 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.172 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.172 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.172 INFO:teuthology.orchestra.run.smithi139.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T01:59:04.172 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T01:59:04.172 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.172 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.172 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.172 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: "cross_origin_url": { 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cross_origin_url", 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.173 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.174 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.174 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.174 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.174 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.174 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.174 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.174 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.174 INFO:teuthology.orchestra.run.smithi139.stdout: "crt_file": { 2024-09-16T01:59:04.174 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crt_file", 2024-09-16T01:59:04.174 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.174 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.174 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.174 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.174 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.175 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.175 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.175 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.175 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.175 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.175 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.175 INFO:teuthology.orchestra.run.smithi139.stdout: "debug": { 2024-09-16T01:59:04.175 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "debug", 2024-09-16T01:59:04.175 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.175 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.175 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.175 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.176 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.176 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.176 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.176 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Enable/disable debug options", 2024-09-16T01:59:04.176 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.176 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.176 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.176 INFO:teuthology.orchestra.run.smithi139.stdout: "jwt_token_ttl": { 2024-09-16T01:59:04.176 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "jwt_token_ttl", 2024-09-16T01:59:04.176 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.176 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.176 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "28800", 2024-09-16T01:59:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: "key_file": { 2024-09-16T01:59:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "key_file", 2024-09-16T01:59:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.178 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.178 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.178 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.178 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.178 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.178 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.178 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.178 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.178 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.178 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.178 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.178 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.178 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.179 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.179 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.179 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.179 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.179 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.179 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.179 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.179 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.179 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.179 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.179 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.179 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.179 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.179 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.180 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.180 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.180 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.180 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.180 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.180 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.180 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.180 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.180 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.180 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.180 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.180 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.181 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.181 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.181 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.181 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.181 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.181 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.181 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.181 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.181 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.181 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.181 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.181 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.181 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.182 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.182 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.182 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.182 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.182 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.182 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.182 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.182 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.182 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.182 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.182 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.182 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.182 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.183 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.183 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.183 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.183 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.183 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.183 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.183 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.183 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.183 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.183 INFO:teuthology.orchestra.run.smithi139.stdout: "motd": { 2024-09-16T01:59:04.183 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "motd", 2024-09-16T01:59:04.183 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.183 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.184 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.184 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.184 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.184 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.184 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.184 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "The message of the day", 2024-09-16T01:59:04.184 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.184 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.184 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.184 INFO:teuthology.orchestra.run.smithi139.stdout: "redirect_resolve_ip_addr": { 2024-09-16T01:59:04.184 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T01:59:04.184 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.184 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.185 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.185 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.185 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.185 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.185 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.185 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.185 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.185 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.185 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.185 INFO:teuthology.orchestra.run.smithi139.stdout: "server_addr": { 2024-09-16T01:59:04.185 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_addr", 2024-09-16T01:59:04.185 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.185 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.186 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.186 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "::", 2024-09-16T01:59:04.186 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.186 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.186 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.186 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.186 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.186 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.186 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.186 INFO:teuthology.orchestra.run.smithi139.stdout: "server_port": { 2024-09-16T01:59:04.186 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_port", 2024-09-16T01:59:04.186 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.186 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8080", 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: "ssl": { 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ssl", 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.187 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.188 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.188 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.188 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.188 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.188 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.188 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.188 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.188 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.188 INFO:teuthology.orchestra.run.smithi139.stdout: "ssl_server_port": { 2024-09-16T01:59:04.188 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ssl_server_port", 2024-09-16T01:59:04.188 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.188 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.189 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.189 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8443", 2024-09-16T01:59:04.189 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.189 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.189 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.189 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.189 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.189 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.189 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.189 INFO:teuthology.orchestra.run.smithi139.stdout: "standby_behaviour": { 2024-09-16T01:59:04.189 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "standby_behaviour", 2024-09-16T01:59:04.189 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.190 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.190 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.190 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "redirect", 2024-09-16T01:59:04.190 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.190 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.190 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.190 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.190 INFO:teuthology.orchestra.run.smithi139.stdout: "redirect" 2024-09-16T01:59:04.190 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.190 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.190 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.190 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.190 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.191 INFO:teuthology.orchestra.run.smithi139.stdout: "standby_error_status_code": { 2024-09-16T01:59:04.191 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "standby_error_status_code", 2024-09-16T01:59:04.191 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.191 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.191 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.191 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "500", 2024-09-16T01:59:04.191 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "400", 2024-09-16T01:59:04.191 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "599", 2024-09-16T01:59:04.191 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.191 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.191 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.191 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.191 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.192 INFO:teuthology.orchestra.run.smithi139.stdout: "url_prefix": { 2024-09-16T01:59:04.192 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "url_prefix", 2024-09-16T01:59:04.192 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.192 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.192 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.192 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.192 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.192 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.192 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.192 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.192 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.192 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.192 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.193 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.193 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.193 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "devicehealth", 2024-09-16T01:59:04.193 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.193 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.193 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.193 INFO:teuthology.orchestra.run.smithi139.stdout: "enable_monitoring": { 2024-09-16T01:59:04.193 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "enable_monitoring", 2024-09-16T01:59:04.193 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.193 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.193 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.193 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.194 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.194 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.194 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.194 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "monitor device health metrics", 2024-09-16T01:59:04.194 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.194 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.194 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.194 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.194 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.194 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.194 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.194 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.194 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.195 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.195 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.195 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.195 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.195 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.195 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.195 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.195 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.195 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.195 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.195 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.195 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.195 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.196 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.196 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.196 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.196 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.196 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.196 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.196 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.196 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.196 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.196 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.196 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.196 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.196 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.197 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.197 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.197 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.197 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.197 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.197 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.197 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.197 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.197 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.197 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.197 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.197 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.197 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.198 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.198 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.198 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.198 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.198 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.198 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.198 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.198 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.198 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.198 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.198 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.198 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.198 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.199 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.199 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.199 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.199 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.199 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.199 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.199 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.199 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.199 INFO:teuthology.orchestra.run.smithi139.stdout: "mark_out_threshold": { 2024-09-16T01:59:04.199 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mark_out_threshold", 2024-09-16T01:59:04.199 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.199 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.199 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.200 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "2419200", 2024-09-16T01:59:04.200 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.200 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.200 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.200 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T01:59:04.200 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.200 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.200 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.200 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": { 2024-09-16T01:59:04.200 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "pool_name", 2024-09-16T01:59:04.200 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.200 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.200 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.201 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "device_health_metrics", 2024-09-16T01:59:04.201 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.201 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.201 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.201 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T01:59:04.201 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.201 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.201 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.201 INFO:teuthology.orchestra.run.smithi139.stdout: "retention_period": { 2024-09-16T01:59:04.201 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "retention_period", 2024-09-16T01:59:04.201 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.201 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.201 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.202 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "15552000", 2024-09-16T01:59:04.202 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.202 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.202 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.202 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how long to retain device health metrics", 2024-09-16T01:59:04.202 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.202 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.202 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.202 INFO:teuthology.orchestra.run.smithi139.stdout: "scrape_frequency": { 2024-09-16T01:59:04.202 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "scrape_frequency", 2024-09-16T01:59:04.202 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.202 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.203 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.203 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "86400", 2024-09-16T01:59:04.203 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.203 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.203 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.203 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T01:59:04.203 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.203 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.203 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.203 INFO:teuthology.orchestra.run.smithi139.stdout: "self_heal": { 2024-09-16T01:59:04.203 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "self_heal", 2024-09-16T01:59:04.203 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.203 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.204 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.204 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.204 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.204 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.204 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.204 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T01:59:04.204 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.204 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.204 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.204 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:04.204 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:04.204 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.205 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.205 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.205 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:04.205 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.205 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.205 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.205 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T01:59:04.205 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.205 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.205 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.205 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_threshold": { 2024-09-16T01:59:04.205 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_threshold", 2024-09-16T01:59:04.205 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.206 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.206 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.206 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "7257600", 2024-09-16T01:59:04.206 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.206 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.206 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.206 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T01:59:04.206 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.206 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.206 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.206 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.206 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.207 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "diskprediction_local", 2024-09-16T01:59:04.207 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.207 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.207 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.207 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.207 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.207 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.207 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.207 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.207 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.207 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.207 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.207 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.207 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.208 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.208 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.208 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.208 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.208 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.208 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.208 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.208 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.208 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.208 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.208 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.209 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.210 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.210 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.210 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.210 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.210 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.210 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.210 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.210 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.210 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.210 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.210 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.210 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.210 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.212 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.212 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.212 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.212 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.212 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.212 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.212 INFO:teuthology.orchestra.run.smithi139.stdout: "predict_interval": { 2024-09-16T01:59:04.212 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "predict_interval", 2024-09-16T01:59:04.212 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.212 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.212 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.212 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "86400", 2024-09-16T01:59:04.212 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.213 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.213 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.213 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.213 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.213 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.213 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.213 INFO:teuthology.orchestra.run.smithi139.stdout: "predictor_model": { 2024-09-16T01:59:04.213 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "predictor_model", 2024-09-16T01:59:04.213 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.213 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.213 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.213 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "prophetstor", 2024-09-16T01:59:04.213 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.214 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.214 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.214 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.214 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.214 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.214 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.214 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:04.214 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:04.214 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.214 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.214 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.214 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:04.215 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.215 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.215 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.215 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.215 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.215 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.215 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.215 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.215 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.215 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "influx", 2024-09-16T01:59:04.215 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": false, 2024-09-16T01:59:04.215 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "influxdb python module not found", 2024-09-16T01:59:04.216 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.216 INFO:teuthology.orchestra.run.smithi139.stdout: "batch_size": { 2024-09-16T01:59:04.216 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "batch_size", 2024-09-16T01:59:04.216 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.216 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.216 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.216 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5000", 2024-09-16T01:59:04.216 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.216 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.216 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.216 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T01:59:04.216 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.216 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.217 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.217 INFO:teuthology.orchestra.run.smithi139.stdout: "database": { 2024-09-16T01:59:04.217 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "database", 2024-09-16T01:59:04.217 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.217 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.217 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.217 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "ceph", 2024-09-16T01:59:04.217 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.217 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.217 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.217 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T01:59:04.217 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.218 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.218 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.218 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": { 2024-09-16T01:59:04.218 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "hostname", 2024-09-16T01:59:04.218 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.218 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.218 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.218 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.218 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.218 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.218 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.218 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "InfluxDB server hostname", 2024-09-16T01:59:04.218 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.219 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.219 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.219 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:04.219 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:04.219 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.219 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.219 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.219 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "30", 2024-09-16T01:59:04.219 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "5", 2024-09-16T01:59:04.219 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.219 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.219 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T01:59:04.219 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.220 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.220 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.220 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.220 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.220 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.220 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.220 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.220 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.220 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.220 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.220 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.220 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.221 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.221 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.221 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.221 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.221 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.221 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.221 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.221 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.221 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.221 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.221 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.221 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.221 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.222 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.222 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.222 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.222 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.222 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.222 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.222 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.222 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.222 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.222 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.222 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.222 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.223 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.223 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.223 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.223 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.223 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.223 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.223 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.223 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.223 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.223 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.223 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.223 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.223 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.223 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.224 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.224 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.224 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.224 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.224 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.224 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.224 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.224 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.224 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.224 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.224 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.224 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.224 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.225 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.225 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.225 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.225 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.225 INFO:teuthology.orchestra.run.smithi139.stdout: "password": { 2024-09-16T01:59:04.225 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "password", 2024-09-16T01:59:04.225 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.225 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.225 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.225 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.225 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.225 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.225 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.226 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "password of InfluxDB server user", 2024-09-16T01:59:04.226 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.226 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.226 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.226 INFO:teuthology.orchestra.run.smithi139.stdout: "port": { 2024-09-16T01:59:04.226 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "port", 2024-09-16T01:59:04.226 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.226 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.226 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.226 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8086", 2024-09-16T01:59:04.226 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.226 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.226 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.227 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "InfluxDB server port", 2024-09-16T01:59:04.227 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.227 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.227 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.227 INFO:teuthology.orchestra.run.smithi139.stdout: "ssl": { 2024-09-16T01:59:04.227 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ssl", 2024-09-16T01:59:04.227 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.227 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.227 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.227 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "false", 2024-09-16T01:59:04.227 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.227 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.227 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.228 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T01:59:04.228 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.228 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.228 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.228 INFO:teuthology.orchestra.run.smithi139.stdout: "threads": { 2024-09-16T01:59:04.228 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "threads", 2024-09-16T01:59:04.228 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.228 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.228 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.228 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:04.228 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1", 2024-09-16T01:59:04.228 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "32", 2024-09-16T01:59:04.228 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.229 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T01:59:04.229 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.229 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.229 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.229 INFO:teuthology.orchestra.run.smithi139.stdout: "username": { 2024-09-16T01:59:04.229 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "username", 2024-09-16T01:59:04.229 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.229 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.229 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.229 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.229 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.229 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.229 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.230 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "username of InfluxDB server user", 2024-09-16T01:59:04.230 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.230 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.230 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.230 INFO:teuthology.orchestra.run.smithi139.stdout: "verify_ssl": { 2024-09-16T01:59:04.230 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "verify_ssl", 2024-09-16T01:59:04.230 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.230 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.230 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.230 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "true", 2024-09-16T01:59:04.230 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.230 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.231 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.231 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T01:59:04.231 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.231 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.231 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.231 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.231 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.231 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "insights", 2024-09-16T01:59:04.231 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.231 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.231 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.231 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.232 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.232 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.232 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.232 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.232 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.232 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.232 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.232 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.232 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.232 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.232 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.232 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.232 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.233 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.233 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.233 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.233 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.233 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.233 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.233 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.233 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.233 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.233 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.233 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.233 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.233 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.234 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.234 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.234 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.234 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.234 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.234 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.234 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.234 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.234 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.234 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.234 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.234 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.234 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.236 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.236 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.236 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.236 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.236 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.236 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.236 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.236 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.236 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.236 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.236 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.236 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.236 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.237 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.237 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.237 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "iostat", 2024-09-16T01:59:04.237 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.237 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.237 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.237 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.237 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.237 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.237 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.237 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.237 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.239 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.239 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.239 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.239 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.239 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.239 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.239 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.239 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.239 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.239 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.239 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.239 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.239 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.240 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.240 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.240 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.240 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.240 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.240 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.240 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.240 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.240 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.240 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.240 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.240 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.240 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.241 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.242 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.242 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.242 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.242 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.242 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.242 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.242 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.242 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.242 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "k8sevents", 2024-09-16T01:59:04.242 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.242 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.242 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.243 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_event_retention_days": { 2024-09-16T01:59:04.243 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ceph_event_retention_days", 2024-09-16T01:59:04.243 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.243 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.243 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.243 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "7", 2024-09-16T01:59:04.243 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.243 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.243 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.243 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T01:59:04.243 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.243 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.243 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.244 INFO:teuthology.orchestra.run.smithi139.stdout: "config_check_secs": { 2024-09-16T01:59:04.244 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "config_check_secs", 2024-09-16T01:59:04.244 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.244 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.244 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.244 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.244 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "10", 2024-09-16T01:59:04.244 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.244 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.244 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T01:59:04.244 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.245 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.245 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.245 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.245 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.245 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.245 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.245 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.245 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.245 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.245 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.245 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.245 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.245 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.246 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.246 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.246 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.246 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.246 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.246 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.246 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.246 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.246 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.246 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.246 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.246 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.246 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.247 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.247 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.247 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.247 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.247 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.247 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.247 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.247 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.247 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.247 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.247 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.247 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.248 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.248 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.248 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.248 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.248 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.248 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.248 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.248 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.248 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.248 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.248 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.248 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.248 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.249 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.249 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.249 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.249 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.249 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.249 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.249 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.249 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.249 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.249 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.249 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.249 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.250 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.250 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.250 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.250 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.250 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.250 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.250 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.250 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "localpool", 2024-09-16T01:59:04.250 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.250 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.250 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.250 INFO:teuthology.orchestra.run.smithi139.stdout: "failure_domain": { 2024-09-16T01:59:04.251 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "failure_domain", 2024-09-16T01:59:04.251 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.251 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.251 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.251 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "host", 2024-09-16T01:59:04.251 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.251 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.251 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.251 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "failure domain for any created local pool", 2024-09-16T01:59:04.251 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T01:59:04.251 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.251 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.252 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.252 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.252 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.252 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.252 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.252 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.252 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.252 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.252 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.252 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.252 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.252 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.252 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.252 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.254 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.254 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.254 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.254 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.254 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.254 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.254 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.254 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.254 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.254 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.254 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.254 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.254 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.255 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.255 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.255 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.255 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.255 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.255 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.255 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.255 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.255 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.255 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.255 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.255 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.255 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.256 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.256 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.256 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.256 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.256 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.256 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.256 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.256 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.256 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.256 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.256 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.256 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.256 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.257 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": { 2024-09-16T01:59:04.257 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "min_size", 2024-09-16T01:59:04.257 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.257 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.257 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.257 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.257 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.257 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.257 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.257 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "default min_size for any created local pool", 2024-09-16T01:59:04.257 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T01:59:04.257 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.257 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.258 INFO:teuthology.orchestra.run.smithi139.stdout: "num_rep": { 2024-09-16T01:59:04.258 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "num_rep", 2024-09-16T01:59:04.258 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.258 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.258 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.258 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "3", 2024-09-16T01:59:04.258 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.258 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.258 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "default replica count for any created local pool", 2024-09-16T01:59:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": { 2024-09-16T01:59:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "pg_num", 2024-09-16T01:59:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "128", 2024-09-16T01:59:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.259 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.260 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T01:59:04.260 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.260 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.260 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.260 INFO:teuthology.orchestra.run.smithi139.stdout: "prefix": { 2024-09-16T01:59:04.260 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "prefix", 2024-09-16T01:59:04.260 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.260 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.260 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.260 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.260 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.260 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.261 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.261 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "name prefix for any created local pool", 2024-09-16T01:59:04.261 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.261 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.261 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.261 INFO:teuthology.orchestra.run.smithi139.stdout: "subtree": { 2024-09-16T01:59:04.261 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "subtree", 2024-09-16T01:59:04.261 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.261 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.261 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.261 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "rack", 2024-09-16T01:59:04.261 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.261 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.262 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.262 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T01:59:04.262 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T01:59:04.262 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.262 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.262 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.262 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.262 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mds_autoscaler", 2024-09-16T01:59:04.262 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.262 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.262 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.263 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.263 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.263 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.263 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.263 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.263 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.263 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.263 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.263 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.263 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.263 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.263 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.263 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.263 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.264 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.264 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.264 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.264 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.264 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.264 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.264 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.264 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.264 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.264 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.264 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.265 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.265 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.265 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.265 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.265 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.265 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.265 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.265 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.265 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.265 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.265 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.265 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.265 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.266 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.266 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.266 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.266 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.266 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.266 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.266 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.266 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.266 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.266 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.266 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.266 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.266 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.266 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.267 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.267 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.267 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.267 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.267 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.267 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.267 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.267 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.267 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.267 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.267 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.267 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.267 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.268 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.268 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.268 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.268 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mirroring", 2024-09-16T01:59:04.268 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.268 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.268 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.268 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.268 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.268 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.268 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.269 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.269 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.269 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.269 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.269 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.269 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.269 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.269 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.269 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.269 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.269 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.269 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.269 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.270 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.270 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.270 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.270 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.270 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.270 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.270 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.270 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.270 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.270 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.270 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.270 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.270 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.271 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.271 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.271 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.271 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.271 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.271 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.271 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.271 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.271 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.271 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.271 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.271 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.271 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.272 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.272 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.272 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.272 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.272 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.272 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.272 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.272 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.272 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.272 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.272 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.272 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.273 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.273 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.273 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.273 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.273 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.273 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.273 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.273 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.273 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.273 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.273 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.273 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.273 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.274 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "nfs", 2024-09-16T01:59:04.274 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.274 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.274 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.274 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.274 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.274 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.274 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.274 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.274 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.274 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.274 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.275 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.275 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.275 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.275 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.275 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.275 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.275 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.275 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.275 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.275 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.275 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.275 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.275 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.276 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.276 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.276 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.276 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.276 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.276 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.276 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.276 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.276 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.276 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.276 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.276 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.278 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.278 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.278 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.278 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.278 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.278 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.278 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.278 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.278 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.278 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.278 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.278 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.278 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.279 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.279 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.279 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.279 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.279 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.279 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.279 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.279 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.279 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "orchestrator", 2024-09-16T01:59:04.279 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.279 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.279 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.280 INFO:teuthology.orchestra.run.smithi139.stdout: "fail_fs": { 2024-09-16T01:59:04.280 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "fail_fs", 2024-09-16T01:59:04.280 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.280 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.280 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.280 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.280 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.280 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.280 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.280 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T01:59:04.280 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.280 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.280 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.281 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.281 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.281 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.281 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.281 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.281 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.281 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.281 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.281 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.281 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.281 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.281 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.281 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.282 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.283 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.283 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.283 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.283 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.283 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.283 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.283 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.283 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.283 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.283 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.283 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.283 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.284 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.284 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.284 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.284 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.284 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.284 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.284 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.284 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.284 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.284 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.284 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.284 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.284 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.284 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.286 INFO:teuthology.orchestra.run.smithi139.stdout: "orchestrator": { 2024-09-16T01:59:04.286 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "orchestrator", 2024-09-16T01:59:04.286 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.286 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.286 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.286 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.286 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.286 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.286 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.286 INFO:teuthology.orchestra.run.smithi139.stdout: "cephadm", 2024-09-16T01:59:04.286 INFO:teuthology.orchestra.run.smithi139.stdout: "rook", 2024-09-16T01:59:04.286 INFO:teuthology.orchestra.run.smithi139.stdout: "test_orchestrator" 2024-09-16T01:59:04.286 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.286 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Orchestrator backend", 2024-09-16T01:59:04.287 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.287 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.287 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.287 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.287 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.287 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "osd_perf_query", 2024-09-16T01:59:04.287 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.287 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.287 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.287 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.287 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.288 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.288 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.288 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.288 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.288 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.288 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.288 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.288 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.288 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.288 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.288 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.288 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.288 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.288 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.289 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.289 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.289 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.289 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.289 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.289 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.289 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.289 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.289 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.289 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.289 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.289 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.290 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.290 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.290 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.290 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.290 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.290 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.290 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.290 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.290 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.290 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.290 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.290 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.290 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.292 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.292 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.292 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.292 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.292 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.292 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.292 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.292 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.292 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.292 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.292 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.292 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.292 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.293 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.293 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "osd_support", 2024-09-16T01:59:04.293 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.293 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.293 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.293 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.293 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.293 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.293 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.293 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.293 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.293 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.294 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.294 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.294 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.294 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.294 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.294 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.294 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.294 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.294 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.294 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.294 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.294 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.294 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.295 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.295 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.295 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.295 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.295 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.295 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.295 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.295 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.295 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.295 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.295 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.295 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.295 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.296 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.296 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.296 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.296 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.296 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.296 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.296 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.296 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.296 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.296 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.296 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.296 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.296 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.296 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.297 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.297 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.297 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.297 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.297 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.297 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.297 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.297 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.297 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.297 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.297 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.297 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.297 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.298 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.298 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.298 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.298 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.298 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.298 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.298 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.298 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.298 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "pg_autoscaler", 2024-09-16T01:59:04.298 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.298 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.298 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.299 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.299 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.299 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.299 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.299 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.299 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.299 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.299 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.299 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.299 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.299 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.299 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.299 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.299 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.300 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.300 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.300 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.300 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.300 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.300 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.300 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.300 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.300 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.300 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.300 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.300 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.300 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.301 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.301 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.301 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.301 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.301 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.301 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.301 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.301 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.301 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.301 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.301 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.301 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.302 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.302 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.302 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.302 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.302 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.302 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.302 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.302 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.302 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.302 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.302 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.302 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.302 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.303 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.304 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.304 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:04.304 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:04.304 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.304 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.304 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.304 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:04.304 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.304 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.304 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.304 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.304 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.305 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.305 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.305 INFO:teuthology.orchestra.run.smithi139.stdout: "threshold": { 2024-09-16T01:59:04.305 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "threshold", 2024-09-16T01:59:04.305 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:04.305 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.305 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.305 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "3.0", 2024-09-16T01:59:04.305 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1.0", 2024-09-16T01:59:04.305 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.305 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.305 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "scaling threshold", 2024-09-16T01:59:04.305 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T01:59:04.306 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.306 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.306 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.306 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.306 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "progress", 2024-09-16T01:59:04.306 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.306 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.306 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.306 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_pg_recovery_event": { 2024-09-16T01:59:04.306 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "allow_pg_recovery_event", 2024-09-16T01:59:04.306 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.306 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.307 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.307 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.307 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.307 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.307 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.307 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T01:59:04.307 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.307 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.307 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.307 INFO:teuthology.orchestra.run.smithi139.stdout: "enabled": { 2024-09-16T01:59:04.307 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "enabled", 2024-09-16T01:59:04.307 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.308 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.308 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.308 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.308 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.308 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.308 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.308 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.308 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.308 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.308 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.308 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.308 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.308 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.309 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.309 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.309 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.309 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.309 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.309 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.309 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.309 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.309 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.309 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.309 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.309 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.309 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.310 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.310 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.310 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.310 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.310 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.310 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.310 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.310 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.310 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.310 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.310 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.310 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.310 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.311 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.311 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.311 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.311 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.311 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.311 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.311 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.311 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.311 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.311 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.311 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.311 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.311 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.312 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.312 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.312 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.312 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.312 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.312 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.312 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.312 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.312 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.312 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.312 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.312 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.312 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.313 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.313 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.313 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.313 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.313 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.313 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.313 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.313 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.313 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.313 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.313 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.313 INFO:teuthology.orchestra.run.smithi139.stdout: "max_completed_events": { 2024-09-16T01:59:04.313 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_completed_events", 2024-09-16T01:59:04.314 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.314 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.314 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.314 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "50", 2024-09-16T01:59:04.314 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.314 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.314 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.314 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "number of past completed events to remember", 2024-09-16T01:59:04.314 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.314 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.314 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.314 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:04.314 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:04.315 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.315 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.315 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.315 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:04.315 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.315 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.315 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.315 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how long the module is going to sleep", 2024-09-16T01:59:04.315 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.315 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.315 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.315 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.315 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.316 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "prometheus", 2024-09-16T01:59:04.316 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.316 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.316 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.316 INFO:teuthology.orchestra.run.smithi139.stdout: "cache": { 2024-09-16T01:59:04.316 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cache", 2024-09-16T01:59:04.316 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.316 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.316 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.316 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.316 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: "exclude_perf_counters": { 2024-09-16T01:59:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "exclude_perf_counters", 2024-09-16T01:59:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.318 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.318 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.318 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T01:59:04.318 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T01:59:04.318 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.318 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.318 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.318 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.318 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.318 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.318 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.318 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.318 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.319 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.319 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.319 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.319 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.319 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.319 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.319 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.319 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.319 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.319 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.319 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.319 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.319 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.320 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.320 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.320 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.320 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.320 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.320 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.320 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.320 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.320 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.320 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.320 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.320 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.320 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.321 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.321 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.321 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.321 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.321 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.321 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.321 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.321 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.321 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.321 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.321 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.321 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.321 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.322 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.322 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.322 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.322 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.322 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.322 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.322 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.322 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.322 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.322 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.322 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.322 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.322 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.323 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.323 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.323 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.323 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.323 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.323 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.323 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.323 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_stats_pools": { 2024-09-16T01:59:04.323 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rbd_stats_pools", 2024-09-16T01:59:04.323 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.323 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.323 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.323 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.324 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.324 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.324 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.324 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.324 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.324 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.324 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.324 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T01:59:04.324 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T01:59:04.324 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.324 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.324 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.325 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "300", 2024-09-16T01:59:04.325 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.325 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.325 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.325 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.325 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.325 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.325 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.325 INFO:teuthology.orchestra.run.smithi139.stdout: "scrape_interval": { 2024-09-16T01:59:04.325 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "scrape_interval", 2024-09-16T01:59:04.325 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:04.325 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.325 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "15.0", 2024-09-16T01:59:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: "server_addr": { 2024-09-16T01:59:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_addr", 2024-09-16T01:59:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.326 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.327 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "::", 2024-09-16T01:59:04.327 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.327 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.327 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.327 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T01:59:04.327 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.327 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.327 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.327 INFO:teuthology.orchestra.run.smithi139.stdout: "server_port": { 2024-09-16T01:59:04.327 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_port", 2024-09-16T01:59:04.327 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.327 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.327 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.328 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "9283", 2024-09-16T01:59:04.328 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.328 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.328 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.328 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T01:59:04.328 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.328 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.328 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.328 INFO:teuthology.orchestra.run.smithi139.stdout: "stale_cache_strategy": { 2024-09-16T01:59:04.328 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "stale_cache_strategy", 2024-09-16T01:59:04.328 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.329 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.329 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.329 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "log", 2024-09-16T01:59:04.329 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.329 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.329 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.329 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.329 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.329 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.329 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.329 INFO:teuthology.orchestra.run.smithi139.stdout: "standby_behaviour": { 2024-09-16T01:59:04.329 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "standby_behaviour", 2024-09-16T01:59:04.330 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.330 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.330 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.330 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "default", 2024-09-16T01:59:04.330 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.330 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.330 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.330 INFO:teuthology.orchestra.run.smithi139.stdout: "default", 2024-09-16T01:59:04.330 INFO:teuthology.orchestra.run.smithi139.stdout: "error" 2024-09-16T01:59:04.330 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.330 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.330 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.330 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.331 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.331 INFO:teuthology.orchestra.run.smithi139.stdout: "standby_error_status_code": { 2024-09-16T01:59:04.331 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "standby_error_status_code", 2024-09-16T01:59:04.331 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.331 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.331 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.331 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "500", 2024-09-16T01:59:04.331 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "400", 2024-09-16T01:59:04.331 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "599", 2024-09-16T01:59:04.331 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.331 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.331 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.331 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.332 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.332 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.332 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.332 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rbd_support", 2024-09-16T01:59:04.332 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.332 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.332 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.332 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.332 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.332 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.332 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.333 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.333 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.333 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.333 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.333 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.333 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.333 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.333 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.333 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.333 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.333 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.333 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.333 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.333 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.334 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.334 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.334 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.334 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.334 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.334 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.334 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.334 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.334 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.334 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.334 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.334 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.334 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.335 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.335 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.335 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.335 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.335 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.335 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.335 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.335 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.335 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.335 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.335 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.335 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.335 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.336 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.336 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.336 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.336 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.336 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.336 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.336 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.336 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.336 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.336 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.336 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.336 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.336 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.337 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.337 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.337 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.337 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.337 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.337 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.337 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.337 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.337 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.337 INFO:teuthology.orchestra.run.smithi139.stdout: "max_concurrent_snap_create": { 2024-09-16T01:59:04.337 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_concurrent_snap_create", 2024-09-16T01:59:04.337 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.337 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.338 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.338 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.338 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.338 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.338 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.338 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.338 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.338 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.338 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.338 INFO:teuthology.orchestra.run.smithi139.stdout: "mirror_snapshot_schedule": { 2024-09-16T01:59:04.338 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T01:59:04.338 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.338 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: "trash_purge_schedule": { 2024-09-16T01:59:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "trash_purge_schedule", 2024-09-16T01:59:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.340 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.340 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.340 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.340 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.340 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.340 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.340 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.340 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.340 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.340 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.340 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.340 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "restful", 2024-09-16T01:59:04.341 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.341 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.341 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.341 INFO:teuthology.orchestra.run.smithi139.stdout: "enable_auth": { 2024-09-16T01:59:04.341 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "enable_auth", 2024-09-16T01:59:04.341 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.341 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.341 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.341 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.341 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.341 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.341 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.341 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.341 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.342 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.342 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.342 INFO:teuthology.orchestra.run.smithi139.stdout: "key_file": { 2024-09-16T01:59:04.342 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "key_file", 2024-09-16T01:59:04.342 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.342 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.342 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.342 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.342 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.342 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.342 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.342 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.343 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.343 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.343 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.343 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.343 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.343 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.343 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.343 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.343 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.343 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.343 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.343 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.343 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.344 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.344 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.344 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.344 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.344 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.344 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.344 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.344 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.344 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.344 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.344 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.344 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.345 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.345 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.345 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.345 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.345 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.345 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.345 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.345 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.345 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.345 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.345 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.345 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.345 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.346 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.346 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.346 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.346 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.346 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.346 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.346 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.346 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.346 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.346 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.346 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.346 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.346 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.346 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.347 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.347 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.347 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.347 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.347 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.347 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.347 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.347 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.347 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.347 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.347 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.347 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.347 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.348 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.348 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.348 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.348 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.348 INFO:teuthology.orchestra.run.smithi139.stdout: "server_addr": { 2024-09-16T01:59:04.348 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_addr", 2024-09-16T01:59:04.348 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.348 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.348 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.348 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.348 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.348 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.349 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.349 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.349 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.349 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.349 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.349 INFO:teuthology.orchestra.run.smithi139.stdout: "server_port": { 2024-09-16T01:59:04.349 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_port", 2024-09-16T01:59:04.349 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.349 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.349 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.349 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.349 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.349 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rgw", 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.350 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.351 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.351 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.351 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.351 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.351 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.351 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.351 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.351 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.351 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.351 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.351 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.351 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.351 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.351 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.352 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.352 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.352 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.352 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.352 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.352 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.352 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.352 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.352 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.352 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.352 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.352 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.352 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.352 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.353 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.353 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.353 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.353 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.353 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.353 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.353 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.353 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.353 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.353 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.353 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.353 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.353 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.354 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.354 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.354 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.354 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.354 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.354 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.354 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.354 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.354 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.354 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.354 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.354 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.354 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.355 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.355 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.355 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.355 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.355 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.355 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.355 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.355 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.355 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.355 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.355 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.355 INFO:teuthology.orchestra.run.smithi139.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T01:59:04.355 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T01:59:04.356 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.356 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.356 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.356 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:04.356 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.356 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.356 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.356 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T01:59:04.356 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.356 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.356 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.356 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.356 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.357 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rook", 2024-09-16T01:59:04.357 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.357 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.357 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.357 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.357 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.357 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.357 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.357 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.357 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.357 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.358 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.358 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.358 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.358 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.358 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.358 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.358 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.358 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.358 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.358 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.358 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.358 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.358 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.359 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.359 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.359 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.359 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.359 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.359 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.359 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.359 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.359 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.359 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.359 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.359 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.360 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.360 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.360 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.360 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.360 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.360 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.360 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.360 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.360 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.360 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.360 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.360 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.360 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.361 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.361 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.361 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.361 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.361 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.361 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.361 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.361 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.361 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.361 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.361 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.361 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.362 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.362 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.362 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.362 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.362 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.362 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.362 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.362 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.362 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.362 INFO:teuthology.orchestra.run.smithi139.stdout: "storage_class": { 2024-09-16T01:59:04.362 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "storage_class", 2024-09-16T01:59:04.362 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.363 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.363 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.363 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "local", 2024-09-16T01:59:04.363 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.363 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.363 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.363 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T01:59:04.363 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.363 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.363 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.363 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.363 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.364 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "selftest", 2024-09-16T01:59:04.364 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.364 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.364 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.364 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.364 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.364 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.364 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.364 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.364 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.364 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.364 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.364 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.364 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.365 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.365 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.365 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.365 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.365 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.365 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.365 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.365 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.365 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.365 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.365 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.365 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.365 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.366 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.366 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.366 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.366 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.366 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.366 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.366 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.366 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.366 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.366 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.366 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.366 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.367 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.367 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.367 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.367 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.367 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.367 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.367 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.367 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.367 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.367 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.367 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.367 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.367 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.367 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.368 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.368 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.368 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.368 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.368 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.368 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.368 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.368 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.368 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.368 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.368 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.368 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.368 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.369 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.369 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.369 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.369 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.369 INFO:teuthology.orchestra.run.smithi139.stdout: "roption1": { 2024-09-16T01:59:04.369 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "roption1", 2024-09-16T01:59:04.369 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.369 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.369 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.369 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.369 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.369 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.369 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.370 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.370 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.370 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.370 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.370 INFO:teuthology.orchestra.run.smithi139.stdout: "roption2": { 2024-09-16T01:59:04.370 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "roption2", 2024-09-16T01:59:04.370 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.370 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.370 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.370 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "xyz", 2024-09-16T01:59:04.370 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.370 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.371 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.371 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.371 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.371 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.371 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.371 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption1": { 2024-09-16T01:59:04.371 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption1", 2024-09-16T01:59:04.371 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.371 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.371 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.371 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.371 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.372 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.372 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.372 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.372 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.372 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.372 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.372 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption2": { 2024-09-16T01:59:04.372 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption2", 2024-09-16T01:59:04.372 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.372 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.372 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.372 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.373 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.373 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.373 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.373 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.373 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.373 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.373 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.373 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption3": { 2024-09-16T01:59:04.376 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption3", 2024-09-16T01:59:04.376 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:04.376 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.377 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.377 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.377 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.377 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.377 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.377 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.377 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.377 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.377 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.377 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption4": { 2024-09-16T01:59:04.377 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption4", 2024-09-16T01:59:04.377 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.377 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption5": { 2024-09-16T01:59:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption5", 2024-09-16T01:59:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.378 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.379 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.379 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.379 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.379 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.379 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.379 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.379 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.379 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.379 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.379 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption6": { 2024-09-16T01:59:04.379 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption6", 2024-09-16T01:59:04.379 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.380 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.380 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.380 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.380 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.380 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.380 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.380 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.380 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.380 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.380 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.380 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption7": { 2024-09-16T01:59:04.380 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption7", 2024-09-16T01:59:04.381 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.381 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.381 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.381 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.381 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1", 2024-09-16T01:59:04.381 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "42", 2024-09-16T01:59:04.381 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.381 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.381 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.381 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.381 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.381 INFO:teuthology.orchestra.run.smithi139.stdout: "testkey": { 2024-09-16T01:59:04.382 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "testkey", 2024-09-16T01:59:04.382 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.382 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.382 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.382 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.382 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.382 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.382 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.382 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.382 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.382 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.382 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.383 INFO:teuthology.orchestra.run.smithi139.stdout: "testlkey": { 2024-09-16T01:59:04.383 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "testlkey", 2024-09-16T01:59:04.383 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.383 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.383 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.383 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.383 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.383 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.383 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.383 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.383 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.383 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.383 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.384 INFO:teuthology.orchestra.run.smithi139.stdout: "testnewline": { 2024-09-16T01:59:04.384 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "testnewline", 2024-09-16T01:59:04.384 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.384 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.384 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.384 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.384 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.384 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.384 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.384 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.384 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.384 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.384 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.385 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.385 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.385 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "snap_schedule", 2024-09-16T01:59:04.385 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.385 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.385 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.385 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_m_granularity": { 2024-09-16T01:59:04.385 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "allow_m_granularity", 2024-09-16T01:59:04.385 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.385 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.385 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.385 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.386 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.386 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.386 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.386 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T01:59:04.386 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.386 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.386 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.386 INFO:teuthology.orchestra.run.smithi139.stdout: "dump_on_update": { 2024-09-16T01:59:04.386 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "dump_on_update", 2024-09-16T01:59:04.386 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.386 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.386 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.387 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.387 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.387 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.387 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.387 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "dump database to debug log on update", 2024-09-16T01:59:04.387 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.387 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.387 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.387 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.387 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.387 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.387 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.387 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.388 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.388 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.388 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.388 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.388 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.388 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.388 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.388 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.388 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.388 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.388 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.388 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.388 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.389 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.389 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.389 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.389 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.389 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.389 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.389 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.389 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.389 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.389 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.389 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.389 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.389 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.390 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.390 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.390 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.390 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.390 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.390 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.390 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.390 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.390 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.390 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.390 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.390 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.390 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.391 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.391 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.391 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.391 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.391 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.391 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.391 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.391 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.391 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.391 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.391 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.391 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.391 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.392 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.392 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.392 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.392 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.392 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.392 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.392 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.392 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.392 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.392 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.392 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.392 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "stats", 2024-09-16T01:59:04.393 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.393 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.393 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.393 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.393 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.393 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.393 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.393 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.393 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.393 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.393 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.393 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.393 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.393 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.394 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.394 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.394 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.394 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.394 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.394 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.394 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.394 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.394 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.394 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.394 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.394 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.394 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.395 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.395 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.395 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.395 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.395 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.395 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.395 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.395 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.395 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.395 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.395 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.395 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.395 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.396 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.396 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.396 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.396 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.396 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.396 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.396 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.396 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.396 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.396 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.396 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.396 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.396 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.397 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.397 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.397 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.397 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.397 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.397 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.397 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.397 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.397 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.397 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.397 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.397 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.397 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "status", 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.398 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.399 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.399 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.399 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.399 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.399 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.399 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.399 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.399 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.399 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.399 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.399 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.399 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.399 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.399 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.400 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.400 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.400 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.400 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.400 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.400 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.400 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.400 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.400 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.400 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.400 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.400 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.400 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.401 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.401 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.401 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.401 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.401 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.401 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.401 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.401 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.401 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.401 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.401 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.401 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.401 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.402 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.403 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.403 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.403 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.403 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.403 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.403 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.403 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.403 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.403 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.403 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.403 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.404 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "telegraf", 2024-09-16T01:59:04.404 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.404 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.404 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.404 INFO:teuthology.orchestra.run.smithi139.stdout: "address": { 2024-09-16T01:59:04.404 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "address", 2024-09-16T01:59:04.404 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.404 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.404 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.404 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T01:59:04.404 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.404 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.404 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.405 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.405 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.405 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.405 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.405 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:04.405 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:04.405 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.405 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.405 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.405 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "15", 2024-09-16T01:59:04.405 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.405 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.405 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.406 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.406 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.406 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.406 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.406 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.406 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.406 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.406 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.406 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.406 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.406 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.406 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.407 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.407 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.407 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.407 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.407 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.407 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.407 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.407 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.407 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.407 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.407 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.407 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.407 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.408 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.408 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.408 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.408 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.408 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.408 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.408 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.408 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.408 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.408 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.408 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.408 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.409 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.409 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.409 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.409 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.409 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.409 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.409 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.409 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.409 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.409 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.409 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.409 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.409 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.409 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.410 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.410 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.410 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.410 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.410 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.410 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.410 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.410 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.410 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.410 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.410 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.410 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.410 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.411 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.411 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.411 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.411 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.411 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.411 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.411 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.411 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.411 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "telemetry", 2024-09-16T01:59:04.411 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.411 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.411 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.412 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_basic": { 2024-09-16T01:59:04.412 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_basic", 2024-09-16T01:59:04.412 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.412 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.412 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.412 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.412 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.412 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.412 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.412 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T01:59:04.412 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.412 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.412 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.413 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_crash": { 2024-09-16T01:59:04.413 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_crash", 2024-09-16T01:59:04.413 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.413 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.413 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.413 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.413 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.413 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.413 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.413 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T01:59:04.413 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.413 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.413 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.414 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_device": { 2024-09-16T01:59:04.414 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_device", 2024-09-16T01:59:04.414 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.414 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.414 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.414 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.414 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.414 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.414 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.414 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T01:59:04.414 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.414 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.414 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.415 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_ident": { 2024-09-16T01:59:04.415 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_ident", 2024-09-16T01:59:04.415 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.415 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.415 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.415 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.415 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.415 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.415 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.415 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T01:59:04.415 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.415 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.415 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.416 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_perf": { 2024-09-16T01:59:04.416 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_perf", 2024-09-16T01:59:04.416 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.416 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.416 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.416 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.416 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.416 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.416 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.416 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T01:59:04.416 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.416 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.416 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.417 INFO:teuthology.orchestra.run.smithi139.stdout: "contact": { 2024-09-16T01:59:04.417 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "contact", 2024-09-16T01:59:04.417 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.417 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.417 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.417 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.417 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.417 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.417 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.417 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.417 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.417 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.418 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.418 INFO:teuthology.orchestra.run.smithi139.stdout: "description": { 2024-09-16T01:59:04.418 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "description", 2024-09-16T01:59:04.418 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.418 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.418 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.418 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.418 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.418 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.418 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.418 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.418 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.418 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.419 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.419 INFO:teuthology.orchestra.run.smithi139.stdout: "device_url": { 2024-09-16T01:59:04.419 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "device_url", 2024-09-16T01:59:04.419 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.419 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.419 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.419 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T01:59:04.419 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.419 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.419 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.419 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.419 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.419 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.420 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.420 INFO:teuthology.orchestra.run.smithi139.stdout: "enabled": { 2024-09-16T01:59:04.420 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "enabled", 2024-09-16T01:59:04.420 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.420 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.420 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.420 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.420 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.420 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.420 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.420 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.420 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.421 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.421 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.421 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:04.421 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:04.421 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.421 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.421 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.421 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "24", 2024-09-16T01:59:04.421 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "8", 2024-09-16T01:59:04.421 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.421 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.421 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.421 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.422 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.422 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.422 INFO:teuthology.orchestra.run.smithi139.stdout: "last_opt_revision": { 2024-09-16T01:59:04.422 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "last_opt_revision", 2024-09-16T01:59:04.422 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.422 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.422 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.422 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "1", 2024-09-16T01:59:04.422 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.422 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.422 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.422 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.422 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.423 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.423 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.423 INFO:teuthology.orchestra.run.smithi139.stdout: "leaderboard": { 2024-09-16T01:59:04.423 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "leaderboard", 2024-09-16T01:59:04.423 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.423 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.423 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.423 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.423 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.423 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.423 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.423 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.423 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.424 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.424 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.424 INFO:teuthology.orchestra.run.smithi139.stdout: "leaderboard_description": { 2024-09-16T01:59:04.424 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "leaderboard_description", 2024-09-16T01:59:04.424 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.424 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.424 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.424 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.424 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.424 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.424 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.424 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.425 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.425 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.425 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.425 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.425 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.425 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.425 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.425 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.425 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.425 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.425 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.425 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.425 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.426 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.426 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.426 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.426 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.426 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.426 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.426 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.426 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.426 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.426 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.426 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.426 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.426 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.427 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.427 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.427 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.427 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.427 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.427 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.427 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.427 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.427 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.427 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.427 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.427 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.427 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.428 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.428 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.428 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.428 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.428 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.428 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.428 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.428 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.428 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.428 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.428 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.428 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.428 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.428 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.429 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.429 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.429 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.429 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.429 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.429 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.429 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.429 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.429 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.429 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.429 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.429 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.429 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.430 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.430 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.430 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.430 INFO:teuthology.orchestra.run.smithi139.stdout: "organization": { 2024-09-16T01:59:04.430 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "organization", 2024-09-16T01:59:04.430 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.430 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.430 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.430 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.430 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.430 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.430 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.431 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.431 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.431 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.431 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.431 INFO:teuthology.orchestra.run.smithi139.stdout: "proxy": { 2024-09-16T01:59:04.431 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "proxy", 2024-09-16T01:59:04.431 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.431 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.431 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.431 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.431 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.431 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.432 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.432 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.432 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.432 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.432 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.432 INFO:teuthology.orchestra.run.smithi139.stdout: "url": { 2024-09-16T01:59:04.432 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "url", 2024-09-16T01:59:04.432 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.432 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.432 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.432 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T01:59:04.432 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.432 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.433 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.433 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.433 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.433 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.433 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.433 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.433 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.433 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "test_orchestrator", 2024-09-16T01:59:04.433 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.433 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.433 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.433 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.434 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.434 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.434 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.434 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.434 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.434 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.434 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.434 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.434 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.434 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.434 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.434 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.434 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.434 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.435 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.435 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.435 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.435 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.435 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.435 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.435 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.435 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.435 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.435 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.435 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.435 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.435 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.436 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.436 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.436 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.436 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.436 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.436 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.436 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.436 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.436 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.436 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.436 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.436 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.437 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.437 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.437 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.437 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.437 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.437 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.437 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.437 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.437 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.437 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.437 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.437 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.437 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.438 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.438 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.438 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.438 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.438 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.438 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.438 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.438 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.438 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.438 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.438 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.438 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.438 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.438 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "volumes", 2024-09-16T01:59:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.440 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.440 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.440 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.440 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.440 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.440 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.440 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.440 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.440 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.440 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.440 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.440 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.440 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.441 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.441 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.441 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.441 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.441 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.441 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.441 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.441 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.441 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.441 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.441 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.441 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.441 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.442 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.442 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.442 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.442 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.442 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.442 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.442 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.442 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.442 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.442 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.442 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.442 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.442 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.442 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.444 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.444 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.444 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.444 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.444 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.444 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.444 INFO:teuthology.orchestra.run.smithi139.stdout: "max_concurrent_clones": { 2024-09-16T01:59:04.444 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_concurrent_clones", 2024-09-16T01:59:04.444 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.444 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.444 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.444 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "4", 2024-09-16T01:59:04.445 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.445 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.445 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.445 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T01:59:04.445 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.445 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.445 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.445 INFO:teuthology.orchestra.run.smithi139.stdout: "periodic_async_work": { 2024-09-16T01:59:04.445 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "periodic_async_work", 2024-09-16T01:59:04.445 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.445 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.445 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.446 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.446 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.446 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.446 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.446 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Periodically check for async work", 2024-09-16T01:59:04.446 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.446 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.446 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.446 INFO:teuthology.orchestra.run.smithi139.stdout: "snapshot_clone_delay": { 2024-09-16T01:59:04.446 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "snapshot_clone_delay", 2024-09-16T01:59:04.446 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.446 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.447 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.447 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:04.447 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.447 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.447 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.447 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T01:59:04.447 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.447 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.447 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.447 INFO:teuthology.orchestra.run.smithi139.stdout: "snapshot_clone_no_wait": { 2024-09-16T01:59:04.447 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T01:59:04.447 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.447 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.448 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.448 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.448 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.448 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.448 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.448 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T01:59:04.448 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.448 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.448 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.448 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.448 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.448 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "zabbix", 2024-09-16T01:59:04.449 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.449 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.449 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.449 INFO:teuthology.orchestra.run.smithi139.stdout: "discovery_interval": { 2024-09-16T01:59:04.449 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "discovery_interval", 2024-09-16T01:59:04.449 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:04.449 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.449 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.449 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "100", 2024-09-16T01:59:04.449 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.449 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.449 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.449 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.449 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.450 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.450 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.450 INFO:teuthology.orchestra.run.smithi139.stdout: "identifier": { 2024-09-16T01:59:04.450 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "identifier", 2024-09-16T01:59:04.450 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.450 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.450 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.450 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.450 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.450 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.450 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.450 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.451 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.451 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.451 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.451 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:04.451 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:04.451 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.451 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.451 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.451 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:04.451 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.451 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.451 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.451 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.452 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.452 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.452 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.452 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.452 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.452 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.452 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.452 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.452 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.452 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.452 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.452 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.452 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.453 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.453 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.453 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.453 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.453 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.453 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.453 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.453 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.453 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.453 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.453 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.453 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.453 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.454 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.454 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.454 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.454 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.454 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.454 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.454 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.454 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.454 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.454 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.454 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.454 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.454 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.456 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.456 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.456 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.456 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.456 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.456 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.456 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.456 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.456 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.456 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.456 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.456 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.456 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.457 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.457 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.457 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.457 INFO:teuthology.orchestra.run.smithi139.stdout: "zabbix_host": { 2024-09-16T01:59:04.457 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "zabbix_host", 2024-09-16T01:59:04.457 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.457 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.457 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.457 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.457 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.457 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.457 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.457 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.458 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.458 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.458 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.458 INFO:teuthology.orchestra.run.smithi139.stdout: "zabbix_port": { 2024-09-16T01:59:04.458 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "zabbix_port", 2024-09-16T01:59:04.458 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.458 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.458 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.458 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10051", 2024-09-16T01:59:04.458 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.458 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.458 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.459 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.459 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.459 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.459 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.459 INFO:teuthology.orchestra.run.smithi139.stdout: "zabbix_sender": { 2024-09-16T01:59:04.459 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "zabbix_sender", 2024-09-16T01:59:04.459 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.459 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.459 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.459 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T01:59:04.459 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.459 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.459 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.460 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.460 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.460 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.460 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.460 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.460 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.460 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.460 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T01:59:04.460 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.460 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.460 INFO:teuthology.orchestra.run.smithi139.stdout: "modules": [ 2024-09-16T01:59:04.460 INFO:teuthology.orchestra.run.smithi139.stdout: "cephadm", 2024-09-16T01:59:04.460 INFO:teuthology.orchestra.run.smithi139.stdout: "dashboard", 2024-09-16T01:59:04.461 INFO:teuthology.orchestra.run.smithi139.stdout: "iostat", 2024-09-16T01:59:04.461 INFO:teuthology.orchestra.run.smithi139.stdout: "nfs", 2024-09-16T01:59:04.461 INFO:teuthology.orchestra.run.smithi139.stdout: "restful", 2024-09-16T01:59:04.461 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_schedule" 2024-09-16T01:59:04.461 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.461 INFO:teuthology.orchestra.run.smithi139.stdout: "available_modules": [ 2024-09-16T01:59:04.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.461 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "alerts", 2024-09-16T01:59:04.461 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.461 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.461 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.461 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:04.461 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:04.462 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.462 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.462 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.462 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:04.462 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.462 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.462 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.462 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T01:59:04.462 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.462 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.462 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.462 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.462 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.463 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.463 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.463 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.463 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.463 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.463 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.463 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.463 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.463 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.463 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.463 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.463 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.463 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.464 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.464 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.464 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.464 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.464 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.464 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.464 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.464 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.464 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.464 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.464 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.464 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.464 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.465 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.465 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.465 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.465 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.465 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.465 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.465 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.465 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.465 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.465 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.465 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.465 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.466 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.466 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.466 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.466 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.466 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.466 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.466 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.466 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.466 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.466 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.466 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.466 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.466 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.467 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.467 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.467 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.467 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.467 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.467 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.467 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.467 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.467 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.467 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.467 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.467 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.467 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.468 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_destination": { 2024-09-16T01:59:04.468 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_destination", 2024-09-16T01:59:04.468 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.468 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.468 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.468 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.468 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.468 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.468 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.468 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Email address to send alerts to", 2024-09-16T01:59:04.468 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.468 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.468 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.469 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_from_name": { 2024-09-16T01:59:04.469 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_from_name", 2024-09-16T01:59:04.469 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.469 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.469 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.469 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "Ceph", 2024-09-16T01:59:04.469 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.469 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.469 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.469 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Email From: name", 2024-09-16T01:59:04.469 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.469 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.469 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_host": { 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_host", 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "SMTP server", 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.471 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_password": { 2024-09-16T01:59:04.471 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_password", 2024-09-16T01:59:04.471 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.471 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.471 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.471 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.471 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.471 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.471 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.471 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Password to authenticate with", 2024-09-16T01:59:04.471 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.471 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.471 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.472 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_port": { 2024-09-16T01:59:04.472 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_port", 2024-09-16T01:59:04.472 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.472 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.472 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.472 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "465", 2024-09-16T01:59:04.472 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.472 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.472 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.472 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "SMTP port", 2024-09-16T01:59:04.472 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.472 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.472 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.473 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_sender": { 2024-09-16T01:59:04.473 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_sender", 2024-09-16T01:59:04.473 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.473 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.473 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.473 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.473 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.473 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.473 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.473 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "SMTP envelope sender", 2024-09-16T01:59:04.473 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.473 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.473 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.474 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_ssl": { 2024-09-16T01:59:04.474 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_ssl", 2024-09-16T01:59:04.474 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.474 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.474 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.474 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.474 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.474 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.474 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.474 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T01:59:04.474 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.474 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.475 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.475 INFO:teuthology.orchestra.run.smithi139.stdout: "smtp_user": { 2024-09-16T01:59:04.475 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "smtp_user", 2024-09-16T01:59:04.475 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.475 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.475 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.475 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.475 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.475 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.475 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.475 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "User to authenticate as", 2024-09-16T01:59:04.475 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.475 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.476 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.476 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.476 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.476 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "balancer", 2024-09-16T01:59:04.476 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.476 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.476 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.476 INFO:teuthology.orchestra.run.smithi139.stdout: "active": { 2024-09-16T01:59:04.476 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "active", 2024-09-16T01:59:04.476 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.476 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.476 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.477 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.477 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.477 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.477 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.477 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T01:59:04.477 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.477 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.477 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.477 INFO:teuthology.orchestra.run.smithi139.stdout: "begin_time": { 2024-09-16T01:59:04.477 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "begin_time", 2024-09-16T01:59:04.477 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.477 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.477 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0000", 2024-09-16T01:59:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T01:59:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T01:59:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: "begin_weekday": { 2024-09-16T01:59:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "begin_weekday", 2024-09-16T01:59:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.479 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:04.479 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "0", 2024-09-16T01:59:04.479 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "6", 2024-09-16T01:59:04.479 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.479 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T01:59:04.479 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T01:59:04.479 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.479 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.479 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_compat_max_iterations": { 2024-09-16T01:59:04.479 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crush_compat_max_iterations", 2024-09-16T01:59:04.479 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:04.479 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "25", 2024-09-16T01:59:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1", 2024-09-16T01:59:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "250", 2024-09-16T01:59:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T01:59:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_compat_metrics": { 2024-09-16T01:59:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crush_compat_metrics", 2024-09-16T01:59:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.481 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.481 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T01:59:04.481 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.481 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.481 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.481 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T01:59:04.481 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T01:59:04.481 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.481 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.481 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_compat_step": { 2024-09-16T01:59:04.481 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crush_compat_step", 2024-09-16T01:59:04.481 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:04.482 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.482 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.482 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0.5", 2024-09-16T01:59:04.482 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "0.001", 2024-09-16T01:59:04.482 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "0.999", 2024-09-16T01:59:04.482 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.482 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "aggressiveness of optimization", 2024-09-16T01:59:04.482 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T01:59:04.482 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.482 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.482 INFO:teuthology.orchestra.run.smithi139.stdout: "end_time": { 2024-09-16T01:59:04.482 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "end_time", 2024-09-16T01:59:04.482 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.483 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.483 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.483 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "2359", 2024-09-16T01:59:04.483 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.483 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.483 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.483 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T01:59:04.483 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T01:59:04.483 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.483 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.483 INFO:teuthology.orchestra.run.smithi139.stdout: "end_weekday": { 2024-09-16T01:59:04.483 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "end_weekday", 2024-09-16T01:59:04.483 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:04.484 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.484 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.484 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:04.484 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "0", 2024-09-16T01:59:04.484 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "6", 2024-09-16T01:59:04.484 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.484 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T01:59:04.484 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T01:59:04.484 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.484 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.484 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.484 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.485 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.485 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.485 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.485 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.485 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.485 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.485 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.485 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.485 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.485 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.485 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.485 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.485 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.485 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.486 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.486 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.486 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.486 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.486 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.486 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.486 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.486 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.486 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.486 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.486 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.486 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.486 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.487 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.487 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.487 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.487 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.487 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.487 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.487 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.487 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.487 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.487 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.487 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.487 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.487 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.488 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.488 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.488 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.488 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.488 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.488 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.488 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.488 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.488 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.488 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.488 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.488 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.488 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.489 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.489 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.489 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.489 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.489 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.489 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.489 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.489 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.489 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.489 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.489 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.489 INFO:teuthology.orchestra.run.smithi139.stdout: "min_score": { 2024-09-16T01:59:04.490 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "min_score", 2024-09-16T01:59:04.490 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:04.490 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.490 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.490 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:04.490 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.490 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.490 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.490 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T01:59:04.490 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.490 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.490 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.490 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": { 2024-09-16T01:59:04.491 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mode", 2024-09-16T01:59:04.491 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.491 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.491 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.491 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "upmap", 2024-09-16T01:59:04.491 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.491 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.491 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.491 INFO:teuthology.orchestra.run.smithi139.stdout: "crush-compat", 2024-09-16T01:59:04.491 INFO:teuthology.orchestra.run.smithi139.stdout: "none", 2024-09-16T01:59:04.491 INFO:teuthology.orchestra.run.smithi139.stdout: "read", 2024-09-16T01:59:04.491 INFO:teuthology.orchestra.run.smithi139.stdout: "upmap", 2024-09-16T01:59:04.491 INFO:teuthology.orchestra.run.smithi139.stdout: "upmap-read" 2024-09-16T01:59:04.491 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.492 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Balancer mode", 2024-09-16T01:59:04.492 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.492 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.492 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.492 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_ids": { 2024-09-16T01:59:04.492 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "pool_ids", 2024-09-16T01:59:04.492 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.492 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.492 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.492 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.492 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.492 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.493 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.493 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T01:59:04.493 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.493 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.493 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.493 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:04.493 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:04.493 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.493 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.493 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.493 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:04.493 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.493 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.494 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.494 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T01:59:04.494 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.494 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.494 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.494 INFO:teuthology.orchestra.run.smithi139.stdout: "upmap_max_deviation": { 2024-09-16T01:59:04.494 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "upmap_max_deviation", 2024-09-16T01:59:04.494 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.494 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.494 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.494 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:04.494 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1", 2024-09-16T01:59:04.494 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.495 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.495 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T01:59:04.495 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T01:59:04.495 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.495 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.495 INFO:teuthology.orchestra.run.smithi139.stdout: "upmap_max_optimizations": { 2024-09-16T01:59:04.495 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "upmap_max_optimizations", 2024-09-16T01:59:04.495 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:04.495 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.495 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.495 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.495 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.495 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.496 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.496 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T01:59:04.496 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.496 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.496 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.496 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.496 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.496 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cephadm", 2024-09-16T01:59:04.496 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.496 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.496 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.497 INFO:teuthology.orchestra.run.smithi139.stdout: "agent_down_multiplier": { 2024-09-16T01:59:04.497 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "agent_down_multiplier", 2024-09-16T01:59:04.497 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:04.497 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.497 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.497 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "3.0", 2024-09-16T01:59:04.497 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.497 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.497 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.497 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T01:59:04.497 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.497 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.497 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.498 INFO:teuthology.orchestra.run.smithi139.stdout: "agent_refresh_rate": { 2024-09-16T01:59:04.498 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "agent_refresh_rate", 2024-09-16T01:59:04.498 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.498 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.498 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.498 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "20", 2024-09-16T01:59:04.498 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.498 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.498 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.498 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T01:59:04.498 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.498 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.498 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.499 INFO:teuthology.orchestra.run.smithi139.stdout: "agent_starting_port": { 2024-09-16T01:59:04.499 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "agent_starting_port", 2024-09-16T01:59:04.499 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.499 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.499 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.499 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "4721", 2024-09-16T01:59:04.499 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.499 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.499 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.499 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T01:59:04.499 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.499 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.499 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.500 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_ptrace": { 2024-09-16T01:59:04.500 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "allow_ptrace", 2024-09-16T01:59:04.500 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.500 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.500 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.500 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.500 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.500 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.500 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.500 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T01:59:04.500 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T01:59:04.500 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.500 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.501 INFO:teuthology.orchestra.run.smithi139.stdout: "autotune_interval": { 2024-09-16T01:59:04.501 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "autotune_interval", 2024-09-16T01:59:04.501 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.501 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.501 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.501 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:04.501 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.501 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.501 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.501 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T01:59:04.501 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.501 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.501 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.502 INFO:teuthology.orchestra.run.smithi139.stdout: "autotune_memory_target_ratio": { 2024-09-16T01:59:04.502 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T01:59:04.502 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:04.502 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.502 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.502 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0.7", 2024-09-16T01:59:04.502 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.502 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.502 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.502 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T01:59:04.502 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.502 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.502 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: "cephadm_log_destination": { 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cephadm_log_destination", 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: "file", 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: "file,syslog", 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: "syslog" 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.503 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T01:59:04.504 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.504 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.504 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.504 INFO:teuthology.orchestra.run.smithi139.stdout: "cgroups_split": { 2024-09-16T01:59:04.504 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cgroups_split", 2024-09-16T01:59:04.504 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.504 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.504 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.504 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.504 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.504 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.504 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.504 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T01:59:04.505 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.505 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.505 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.505 INFO:teuthology.orchestra.run.smithi139.stdout: "config_checks_enabled": { 2024-09-16T01:59:04.505 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "config_checks_enabled", 2024-09-16T01:59:04.505 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.505 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.505 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.505 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.505 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.505 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.505 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.505 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T01:59:04.506 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.506 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.506 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.506 INFO:teuthology.orchestra.run.smithi139.stdout: "config_dashboard": { 2024-09-16T01:59:04.506 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "config_dashboard", 2024-09-16T01:59:04.506 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.506 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.506 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.506 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.506 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.506 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.506 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.507 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T01:59:04.507 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.507 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.507 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.507 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_alertmanager": { 2024-09-16T01:59:04.507 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_alertmanager", 2024-09-16T01:59:04.507 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.507 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.507 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.507 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T01:59:04.507 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.507 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.507 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.508 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Prometheus container image", 2024-09-16T01:59:04.508 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.508 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.508 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.508 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_base": { 2024-09-16T01:59:04.508 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_base", 2024-09-16T01:59:04.508 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.508 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.508 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.508 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T01:59:04.508 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.508 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Container image name, without the tag", 2024-09-16T01:59:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_elasticsearch": { 2024-09-16T01:59:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_elasticsearch", 2024-09-16T01:59:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T01:59:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.510 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.510 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "elasticsearch container image", 2024-09-16T01:59:04.510 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.510 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.510 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.510 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_grafana": { 2024-09-16T01:59:04.510 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_grafana", 2024-09-16T01:59:04.510 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.510 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.510 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.510 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T01:59:04.510 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.510 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.511 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.511 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Prometheus container image", 2024-09-16T01:59:04.511 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.511 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.511 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.511 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_haproxy": { 2024-09-16T01:59:04.511 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_haproxy", 2024-09-16T01:59:04.511 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.511 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.511 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.511 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T01:59:04.511 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.511 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.512 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.512 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "HAproxy container image", 2024-09-16T01:59:04.512 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.512 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.512 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.512 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_jaeger_agent": { 2024-09-16T01:59:04.512 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_jaeger_agent", 2024-09-16T01:59:04.512 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.512 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.512 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.512 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T01:59:04.512 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.513 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.513 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.513 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Jaeger agent container image", 2024-09-16T01:59:04.513 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.513 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.513 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.513 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_jaeger_collector": { 2024-09-16T01:59:04.513 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_jaeger_collector", 2024-09-16T01:59:04.513 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.513 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.513 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.513 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T01:59:04.513 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.514 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.514 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.514 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Jaeger collector container image", 2024-09-16T01:59:04.514 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.514 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.514 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.514 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.514 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_jaeger_query": { 2024-09-16T01:59:04.514 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_jaeger_query", 2024-09-16T01:59:04.514 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.514 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.514 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.514 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T01:59:04.514 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.515 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.515 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.515 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Jaeger query container image", 2024-09-16T01:59:04.515 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.515 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.515 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.515 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.515 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_keepalived": { 2024-09-16T01:59:04.515 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_keepalived", 2024-09-16T01:59:04.515 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.515 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.515 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.515 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T01:59:04.515 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.516 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.516 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.516 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Keepalived container image", 2024-09-16T01:59:04.516 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.516 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.516 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.516 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.516 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_loki": { 2024-09-16T01:59:04.516 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_loki", 2024-09-16T01:59:04.516 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.516 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.516 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.516 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T01:59:04.516 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Loki container image", 2024-09-16T01:59:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_node_exporter": { 2024-09-16T01:59:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_node_exporter", 2024-09-16T01:59:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T01:59:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.518 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.518 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.518 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Prometheus container image", 2024-09-16T01:59:04.518 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.518 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.518 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.518 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_nvmeof": { 2024-09-16T01:59:04.518 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_nvmeof", 2024-09-16T01:59:04.518 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.518 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.518 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.518 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T01:59:04.518 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.519 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.519 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.519 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Nvme-of container image", 2024-09-16T01:59:04.519 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.519 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.519 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.519 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_prometheus": { 2024-09-16T01:59:04.519 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_prometheus", 2024-09-16T01:59:04.519 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.519 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.519 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.519 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T01:59:04.519 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.520 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.520 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.520 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Prometheus container image", 2024-09-16T01:59:04.520 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.520 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.520 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.520 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.520 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_promtail": { 2024-09-16T01:59:04.520 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_promtail", 2024-09-16T01:59:04.520 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.520 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.520 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.520 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T01:59:04.520 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.521 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.521 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.521 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Promtail container image", 2024-09-16T01:59:04.521 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.521 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.521 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.521 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.521 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_samba": { 2024-09-16T01:59:04.521 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_samba", 2024-09-16T01:59:04.521 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.521 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.521 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.521 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T01:59:04.521 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Samba/SMB container image", 2024-09-16T01:59:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image_snmp_gateway": { 2024-09-16T01:59:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_image_snmp_gateway", 2024-09-16T01:59:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T01:59:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "SNMP Gateway container image", 2024-09-16T01:59:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: "container_init": { 2024-09-16T01:59:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "container_init", 2024-09-16T01:59:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T01:59:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: "daemon_cache_timeout": { 2024-09-16T01:59:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "daemon_cache_timeout", 2024-09-16T01:59:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T01:59:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: "default_cephadm_command_timeout": { 2024-09-16T01:59:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T01:59:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "900", 2024-09-16T01:59:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T01:59:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: "default_registry": { 2024-09-16T01:59:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "default_registry", 2024-09-16T01:59:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.527 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "docker.io", 2024-09-16T01:59:04.527 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.527 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.527 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.527 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T01:59:04.527 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.527 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.527 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.527 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.527 INFO:teuthology.orchestra.run.smithi139.stdout: "device_cache_timeout": { 2024-09-16T01:59:04.527 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "device_cache_timeout", 2024-09-16T01:59:04.527 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.527 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.527 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.528 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "1800", 2024-09-16T01:59:04.528 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.528 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.528 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.528 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "seconds to cache device inventory", 2024-09-16T01:59:04.528 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.528 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.528 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.528 INFO:teuthology.orchestra.run.smithi139.stdout: "device_enhanced_scan": { 2024-09-16T01:59:04.528 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "device_enhanced_scan", 2024-09-16T01:59:04.528 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.528 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.529 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.529 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.529 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.529 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.529 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.529 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T01:59:04.529 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.529 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.529 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.529 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.529 INFO:teuthology.orchestra.run.smithi139.stdout: "facts_cache_timeout": { 2024-09-16T01:59:04.529 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "facts_cache_timeout", 2024-09-16T01:59:04.529 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.529 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.530 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.530 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:04.530 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.530 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.530 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.530 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "seconds to cache host facts data", 2024-09-16T01:59:04.530 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.530 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.530 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.530 INFO:teuthology.orchestra.run.smithi139.stdout: "grafana_dashboards_path": { 2024-09-16T01:59:04.530 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "grafana_dashboards_path", 2024-09-16T01:59:04.530 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.530 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: "host_check_interval": { 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "host_check_interval", 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.531 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to perform a host check", 2024-09-16T01:59:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: "hw_monitoring": { 2024-09-16T01:59:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "hw_monitoring", 2024-09-16T01:59:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.533 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.533 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.533 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.533 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.533 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.533 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T01:59:04.533 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.533 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.533 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.533 INFO:teuthology.orchestra.run.smithi139.stdout: "inventory_list_all": { 2024-09-16T01:59:04.533 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "inventory_list_all", 2024-09-16T01:59:04.533 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.533 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.534 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.536 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.536 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.536 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.536 INFO:teuthology.orchestra.run.smithi139.stdout: "log_refresh_metadata": { 2024-09-16T01:59:04.536 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_refresh_metadata", 2024-09-16T01:59:04.536 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.536 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.536 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.536 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.536 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.536 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.536 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.536 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T01:59:04.536 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.537 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.537 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.537 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.537 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.537 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.537 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.537 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.537 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.537 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.537 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.537 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.537 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T01:59:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.538 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.539 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T01:59:04.540 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T01:59:04.541 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.541 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.541 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.541 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.541 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.541 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.541 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.541 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T01:59:04.541 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.541 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.541 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.541 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.541 INFO:teuthology.orchestra.run.smithi139.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T01:59:04.541 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T01:59:04.542 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.542 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.542 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.542 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "*", 2024-09-16T01:59:04.542 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.542 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.542 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.542 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T01:59:04.542 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.542 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.542 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.542 INFO:teuthology.orchestra.run.smithi139.stdout: "max_count_per_host": { 2024-09-16T01:59:04.543 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_count_per_host", 2024-09-16T01:59:04.543 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.543 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.543 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.543 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.543 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.543 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.543 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.543 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "max number of daemons per service per host", 2024-09-16T01:59:04.543 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.543 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.543 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.543 INFO:teuthology.orchestra.run.smithi139.stdout: "max_osd_draining_count": { 2024-09-16T01:59:04.544 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_osd_draining_count", 2024-09-16T01:59:04.544 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.544 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.544 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.544 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.544 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.544 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.544 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.544 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T01:59:04.544 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.544 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.544 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.544 INFO:teuthology.orchestra.run.smithi139.stdout: "migration_current": { 2024-09-16T01:59:04.545 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "migration_current", 2024-09-16T01:59:04.545 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.545 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.545 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.545 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.545 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.545 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.545 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.545 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "internal - do not modify", 2024-09-16T01:59:04.545 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.545 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.545 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.545 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": { 2024-09-16T01:59:04.546 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mode", 2024-09-16T01:59:04.546 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.546 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.546 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.546 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "root", 2024-09-16T01:59:04.546 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.546 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.546 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.546 INFO:teuthology.orchestra.run.smithi139.stdout: "cephadm-package", 2024-09-16T01:59:04.546 INFO:teuthology.orchestra.run.smithi139.stdout: "root" 2024-09-16T01:59:04.546 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.546 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T01:59:04.546 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.546 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.547 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.547 INFO:teuthology.orchestra.run.smithi139.stdout: "oob_default_addr": { 2024-09-16T01:59:04.547 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "oob_default_addr", 2024-09-16T01:59:04.547 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.547 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.547 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.547 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "169.254.1.1", 2024-09-16T01:59:04.547 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.547 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.547 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.547 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T01:59:04.547 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.547 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.548 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.548 INFO:teuthology.orchestra.run.smithi139.stdout: "prometheus_alerts_path": { 2024-09-16T01:59:04.548 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "prometheus_alerts_path", 2024-09-16T01:59:04.548 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.548 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.548 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.548 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T01:59:04.548 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.548 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.548 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.548 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T01:59:04.548 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.548 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.549 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.549 INFO:teuthology.orchestra.run.smithi139.stdout: "registry_insecure": { 2024-09-16T01:59:04.549 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "registry_insecure", 2024-09-16T01:59:04.549 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.549 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.549 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.549 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.549 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.549 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.549 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.549 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T01:59:04.549 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.549 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.550 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.550 INFO:teuthology.orchestra.run.smithi139.stdout: "registry_password": { 2024-09-16T01:59:04.550 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "registry_password", 2024-09-16T01:59:04.550 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.550 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.550 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.550 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.550 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.550 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.550 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.550 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T01:59:04.550 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.550 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.551 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.551 INFO:teuthology.orchestra.run.smithi139.stdout: "registry_url": { 2024-09-16T01:59:04.551 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "registry_url", 2024-09-16T01:59:04.551 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.551 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.551 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.551 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.551 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.551 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.551 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.551 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T01:59:04.551 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.551 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.552 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.552 INFO:teuthology.orchestra.run.smithi139.stdout: "registry_username": { 2024-09-16T01:59:04.552 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "registry_username", 2024-09-16T01:59:04.552 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.552 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.552 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.552 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.552 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.552 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.552 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.552 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T01:59:04.552 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.553 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.553 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.553 INFO:teuthology.orchestra.run.smithi139.stdout: "secure_monitoring_stack": { 2024-09-16T01:59:04.553 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "secure_monitoring_stack", 2024-09-16T01:59:04.553 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.553 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.553 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.553 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.553 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.553 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.553 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.553 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T01:59:04.553 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.554 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.554 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.554 INFO:teuthology.orchestra.run.smithi139.stdout: "service_discovery_port": { 2024-09-16T01:59:04.554 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "service_discovery_port", 2024-09-16T01:59:04.554 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.554 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.554 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.554 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8765", 2024-09-16T01:59:04.554 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.554 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.554 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.554 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "cephadm service discovery port", 2024-09-16T01:59:04.554 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.555 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.555 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.555 INFO:teuthology.orchestra.run.smithi139.stdout: "ssh_config_file": { 2024-09-16T01:59:04.555 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ssh_config_file", 2024-09-16T01:59:04.555 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.555 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.555 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.555 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.555 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.555 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.555 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.555 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T01:59:04.555 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.556 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.556 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.556 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.556 INFO:teuthology.orchestra.run.smithi139.stdout: "use_agent": { 2024-09-16T01:59:04.556 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "use_agent", 2024-09-16T01:59:04.556 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.556 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.556 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.556 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.556 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.556 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.556 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.556 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T01:59:04.556 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.557 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.557 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.557 INFO:teuthology.orchestra.run.smithi139.stdout: "use_repo_digest": { 2024-09-16T01:59:04.557 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "use_repo_digest", 2024-09-16T01:59:04.557 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.557 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.557 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.557 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.557 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.557 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.557 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.557 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T01:59:04.558 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.558 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.558 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.558 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_on_failed_host_check": { 2024-09-16T01:59:04.558 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_on_failed_host_check", 2024-09-16T01:59:04.558 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.558 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.558 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.558 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.558 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.558 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.558 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.558 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T01:59:04.559 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.559 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.559 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.559 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_on_stray_daemons": { 2024-09-16T01:59:04.559 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_on_stray_daemons", 2024-09-16T01:59:04.559 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.559 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.559 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.559 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.559 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.559 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.559 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.559 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T01:59:04.560 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.560 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.560 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.560 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_on_stray_hosts": { 2024-09-16T01:59:04.560 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_on_stray_hosts", 2024-09-16T01:59:04.560 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.560 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.560 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.560 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.560 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.560 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.560 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.560 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T01:59:04.561 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.561 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.561 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.561 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.561 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.561 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crash", 2024-09-16T01:59:04.561 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.561 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.561 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.561 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.561 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.561 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.562 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.562 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.562 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.562 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.562 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.562 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.562 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.562 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.562 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.562 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.562 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.562 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.562 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.562 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "retain_interval": { 2024-09-16T01:59:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "retain_interval", 2024-09-16T01:59:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "31536000", 2024-09-16T01:59:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T01:59:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_recent_interval": { 2024-09-16T01:59:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_recent_interval", 2024-09-16T01:59:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "1209600", 2024-09-16T01:59:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T01:59:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "dashboard", 2024-09-16T01:59:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T01:59:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T01:59:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T01:59:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T01:59:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T01:59:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T01:59:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "AUDIT_API_ENABLED": { 2024-09-16T01:59:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T01:59:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.573 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.573 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.573 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.573 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.573 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.573 INFO:teuthology.orchestra.run.smithi139.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T01:59:04.573 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T01:59:04.573 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.573 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.573 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.573 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.573 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.573 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.573 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T01:59:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T01:59:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T01:59:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T01:59:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T01:59:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T01:59:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T01:59:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T01:59:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T01:59:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T01:59:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T01:59:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T01:59:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T01:59:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T01:59:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T01:59:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T01:59:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T01:59:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T01:59:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "admin", 2024-09-16T01:59:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T01:59:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T01:59:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_API_URL": { 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_API_URL", 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "admin", 2024-09-16T01:59:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T01:59:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T01:59:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T01:59:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T01:59:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T01:59:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T01:59:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T01:59:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T01:59:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T01:59:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T01:59:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T01:59:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T01:59:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T01:59:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T01:59:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T01:59:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T01:59:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T01:59:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T01:59:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T01:59:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T01:59:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T01:59:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T01:59:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T01:59:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T01:59:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8", 2024-09-16T01:59:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "45", 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T01:59:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T01:59:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T01:59:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T01:59:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "admin", 2024-09-16T01:59:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T01:59:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T01:59:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T01:59:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T01:59:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T01:59:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T01:59:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "cross_origin_url": { 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cross_origin_url", 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "crt_file": { 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "crt_file", 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "debug": { 2024-09-16T01:59:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "debug", 2024-09-16T01:59:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Enable/disable debug options", 2024-09-16T01:59:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "jwt_token_ttl": { 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "jwt_token_ttl", 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "28800", 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "key_file": { 2024-09-16T01:59:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "key_file", 2024-09-16T01:59:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "motd": { 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "motd", 2024-09-16T01:59:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "The message of the day", 2024-09-16T01:59:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "redirect_resolve_ip_addr": { 2024-09-16T01:59:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T01:59:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "server_addr": { 2024-09-16T01:59:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_addr", 2024-09-16T01:59:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "::", 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "server_port": { 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_port", 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8080", 2024-09-16T01:59:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "ssl": { 2024-09-16T01:59:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ssl", 2024-09-16T01:59:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "ssl_server_port": { 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ssl_server_port", 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8443", 2024-09-16T01:59:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "standby_behaviour": { 2024-09-16T01:59:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "standby_behaviour", 2024-09-16T01:59:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "redirect", 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "redirect" 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "standby_error_status_code": { 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "standby_error_status_code", 2024-09-16T01:59:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "500", 2024-09-16T01:59:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "400", 2024-09-16T01:59:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "599", 2024-09-16T01:59:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "url_prefix": { 2024-09-16T01:59:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "url_prefix", 2024-09-16T01:59:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "devicehealth", 2024-09-16T01:59:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "enable_monitoring": { 2024-09-16T01:59:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "enable_monitoring", 2024-09-16T01:59:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "monitor device health metrics", 2024-09-16T01:59:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "mark_out_threshold": { 2024-09-16T01:59:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mark_out_threshold", 2024-09-16T01:59:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "2419200", 2024-09-16T01:59:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T01:59:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": { 2024-09-16T01:59:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "pool_name", 2024-09-16T01:59:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "device_health_metrics", 2024-09-16T01:59:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T01:59:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "retention_period": { 2024-09-16T01:59:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "retention_period", 2024-09-16T01:59:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "15552000", 2024-09-16T01:59:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how long to retain device health metrics", 2024-09-16T01:59:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "scrape_frequency": { 2024-09-16T01:59:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "scrape_frequency", 2024-09-16T01:59:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "86400", 2024-09-16T01:59:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T01:59:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "self_heal": { 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "self_heal", 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "warn_threshold": { 2024-09-16T01:59:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "warn_threshold", 2024-09-16T01:59:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "7257600", 2024-09-16T01:59:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T01:59:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "diskprediction_local", 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "predict_interval": { 2024-09-16T01:59:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "predict_interval", 2024-09-16T01:59:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "86400", 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "predictor_model": { 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "predictor_model", 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "prophetstor", 2024-09-16T01:59:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "600", 2024-09-16T01:59:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "influx", 2024-09-16T01:59:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": false, 2024-09-16T01:59:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "influxdb python module not found", 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "batch_size": { 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "batch_size", 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5000", 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "database": { 2024-09-16T01:59:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "database", 2024-09-16T01:59:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "ceph", 2024-09-16T01:59:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T01:59:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": { 2024-09-16T01:59:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "hostname", 2024-09-16T01:59:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "InfluxDB server hostname", 2024-09-16T01:59:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "30", 2024-09-16T01:59:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "5", 2024-09-16T01:59:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T01:59:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: "password": { 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "password", 2024-09-16T01:59:04.662 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "password of InfluxDB server user", 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "port": { 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "port", 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.663 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.664 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "8086", 2024-09-16T01:59:04.664 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.664 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.664 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.664 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "InfluxDB server port", 2024-09-16T01:59:04.664 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.664 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.664 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.664 INFO:teuthology.orchestra.run.smithi139.stdout: "ssl": { 2024-09-16T01:59:04.664 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ssl", 2024-09-16T01:59:04.665 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.665 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.665 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.665 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "false", 2024-09-16T01:59:04.665 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.665 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.665 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.665 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T01:59:04.665 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.665 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.665 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.665 INFO:teuthology.orchestra.run.smithi139.stdout: "threads": { 2024-09-16T01:59:04.665 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "threads", 2024-09-16T01:59:04.666 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.666 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.666 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.666 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:04.666 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1", 2024-09-16T01:59:04.666 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "32", 2024-09-16T01:59:04.666 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.666 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T01:59:04.666 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.666 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.666 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.666 INFO:teuthology.orchestra.run.smithi139.stdout: "username": { 2024-09-16T01:59:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "username", 2024-09-16T01:59:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "username of InfluxDB server user", 2024-09-16T01:59:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: "verify_ssl": { 2024-09-16T01:59:04.668 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "verify_ssl", 2024-09-16T01:59:04.668 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.668 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.668 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.668 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "true", 2024-09-16T01:59:04.668 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.668 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.668 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.668 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T01:59:04.668 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.668 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.668 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.668 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.669 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.669 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "insights", 2024-09-16T01:59:04.669 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.669 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.669 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.669 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.669 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.669 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.669 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.669 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.669 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.670 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.670 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.670 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.670 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.670 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.670 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.670 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.670 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.670 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.670 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.670 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.670 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.670 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.670 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.671 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.671 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.671 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.671 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.671 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.671 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.671 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.671 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.671 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.671 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.671 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.671 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.671 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.672 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.672 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.672 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.672 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.672 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.672 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.672 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.672 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.672 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.672 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.672 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.672 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.672 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.673 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "iostat", 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.674 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.675 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.675 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.675 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.675 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.675 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.675 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.675 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.675 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.675 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.675 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.675 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.675 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.675 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.675 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.677 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.677 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.677 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.677 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.677 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.677 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.677 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.677 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.677 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.677 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.677 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.677 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.677 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.678 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.679 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.679 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.679 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.679 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.679 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.679 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.679 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.679 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.679 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.679 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.679 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.679 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.680 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "k8sevents", 2024-09-16T01:59:04.680 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.680 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.680 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.680 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_event_retention_days": { 2024-09-16T01:59:04.680 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "ceph_event_retention_days", 2024-09-16T01:59:04.680 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.680 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.680 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.680 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "7", 2024-09-16T01:59:04.680 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.680 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.680 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.680 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T01:59:04.681 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.681 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.681 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.681 INFO:teuthology.orchestra.run.smithi139.stdout: "config_check_secs": { 2024-09-16T01:59:04.681 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "config_check_secs", 2024-09-16T01:59:04.681 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.681 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.681 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.681 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.681 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "10", 2024-09-16T01:59:04.681 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.681 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.681 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.682 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.683 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.683 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.683 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.683 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.683 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.683 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.683 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.683 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.683 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.683 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.683 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.683 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.684 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.684 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.684 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.684 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.684 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.684 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.684 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.684 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.684 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.684 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.684 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.684 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.684 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.685 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.685 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.685 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.685 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.685 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.685 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.685 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.685 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.685 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.685 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.685 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.685 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.685 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.685 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.686 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.686 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.686 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.686 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.686 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.686 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.686 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.686 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.686 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.686 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.686 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.686 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.686 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.687 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.687 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.687 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.687 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.687 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.687 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "localpool", 2024-09-16T01:59:04.687 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.687 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.687 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.687 INFO:teuthology.orchestra.run.smithi139.stdout: "failure_domain": { 2024-09-16T01:59:04.687 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "failure_domain", 2024-09-16T01:59:04.687 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.688 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.688 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.688 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "host", 2024-09-16T01:59:04.688 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.688 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.688 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.688 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "failure domain for any created local pool", 2024-09-16T01:59:04.688 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T01:59:04.688 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.688 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.688 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.688 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.688 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.689 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.689 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.689 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.689 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.689 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.689 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.689 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.689 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.689 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.689 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.689 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.689 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.689 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.690 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.691 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.691 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.691 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.691 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.691 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.691 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.691 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.691 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.691 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.691 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.691 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.691 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.691 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.692 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.692 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.692 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.692 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.692 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.692 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.692 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.692 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.692 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.692 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.692 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.692 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.692 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.693 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.693 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.693 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.693 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.693 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.693 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.693 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.693 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.693 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.693 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.693 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": { 2024-09-16T01:59:04.693 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "min_size", 2024-09-16T01:59:04.693 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "default min_size for any created local pool", 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: "num_rep": { 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "num_rep", 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.694 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.695 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.695 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "3", 2024-09-16T01:59:04.695 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.695 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.695 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.695 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "default replica count for any created local pool", 2024-09-16T01:59:04.695 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.695 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.695 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.695 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": { 2024-09-16T01:59:04.695 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "pg_num", 2024-09-16T01:59:04.695 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.695 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "128", 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: "prefix": { 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "prefix", 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.696 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "name prefix for any created local pool", 2024-09-16T01:59:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: "subtree": { 2024-09-16T01:59:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "subtree", 2024-09-16T01:59:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.697 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "rack", 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mds_autoscaler", 2024-09-16T01:59:04.698 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.699 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.699 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.699 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.699 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.699 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.699 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.699 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.699 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.699 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.699 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.699 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.699 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.699 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.699 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.700 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.700 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.700 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.700 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.700 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.700 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.700 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.700 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.700 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.700 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.700 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.700 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.700 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.701 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.701 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.701 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.701 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.701 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.701 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.701 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.701 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.701 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.701 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.701 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.701 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.701 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.702 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.702 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.702 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.702 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.702 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.702 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.702 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.702 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.702 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.702 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.702 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.702 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.702 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.702 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.704 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.704 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.704 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mirroring", 2024-09-16T01:59:04.704 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.704 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.704 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.704 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.704 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.704 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.704 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.704 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.705 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.706 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.706 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.706 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.706 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.706 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.706 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.706 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.706 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.706 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.706 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.706 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.706 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.706 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.707 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.708 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.709 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.709 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.709 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.709 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.709 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.709 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.709 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.709 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "nfs", 2024-09-16T01:59:04.709 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.709 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.709 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.709 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.710 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.711 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.712 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.712 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.712 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.712 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.712 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.712 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.712 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.712 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.712 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.712 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.712 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.712 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.712 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.713 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.713 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.713 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.713 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.713 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.713 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.713 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.713 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.713 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.713 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.713 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.713 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.713 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.714 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.714 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.714 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.714 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.714 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.714 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.714 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.714 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.714 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.714 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.714 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.714 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "orchestrator", 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "fail_fs": { 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "fail_fs", 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T01:59:04.715 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.716 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.716 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.716 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.716 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.716 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.716 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.716 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.716 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.716 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.716 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.716 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.716 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.716 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.717 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.717 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.717 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.717 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.717 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.717 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.717 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.717 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.717 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.717 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.717 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.717 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.717 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.718 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.718 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.718 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.718 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.718 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.718 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.718 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.718 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.718 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.718 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.718 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.718 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.718 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.719 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.720 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.720 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.720 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.720 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.720 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.720 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.720 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.720 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.720 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.720 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.720 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.720 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: "orchestrator": { 2024-09-16T01:59:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "orchestrator", 2024-09-16T01:59:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: "cephadm", 2024-09-16T01:59:04.721 INFO:teuthology.orchestra.run.smithi139.stdout: "rook", 2024-09-16T01:59:04.722 INFO:teuthology.orchestra.run.smithi139.stdout: "test_orchestrator" 2024-09-16T01:59:04.722 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.722 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Orchestrator backend", 2024-09-16T01:59:04.722 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.722 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.722 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.722 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.722 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.722 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "osd_perf_query", 2024-09-16T01:59:04.722 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.722 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.722 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.723 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.724 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.724 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.724 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.724 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.724 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.724 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.724 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.724 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.724 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.724 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.724 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.724 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.724 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.725 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.726 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.726 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.726 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.726 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.726 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.726 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.726 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.726 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.726 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.726 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.726 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.726 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.726 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.728 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "osd_support", 2024-09-16T01:59:04.728 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.728 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.728 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.728 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.728 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.728 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.728 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.728 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.728 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.728 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.728 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.728 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.728 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.729 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.730 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.730 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.730 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.730 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.730 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.730 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.730 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.730 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.730 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.730 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.730 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.730 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.730 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.731 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.732 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.732 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.732 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.732 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.732 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.732 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.732 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.732 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.732 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.732 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.732 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.732 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.732 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "pg_autoscaler", 2024-09-16T01:59:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.734 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.734 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.734 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.734 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.734 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.734 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.734 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.734 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.734 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.734 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.734 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.734 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.734 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.734 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.735 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.735 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.735 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.735 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.735 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.735 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.735 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.735 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.735 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.735 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.735 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.735 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.735 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.736 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.736 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.736 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.736 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.736 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.736 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.736 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.736 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.736 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.736 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.736 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.736 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.736 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.737 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.737 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.737 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.737 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.737 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.737 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.737 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.737 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.737 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.737 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.737 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.737 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.737 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.738 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.738 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.738 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.738 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.738 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.738 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.738 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.738 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.738 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:04.738 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:04.738 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.738 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.738 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.739 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:04.739 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.739 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.739 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.739 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.739 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.739 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.739 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.739 INFO:teuthology.orchestra.run.smithi139.stdout: "threshold": { 2024-09-16T01:59:04.739 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "threshold", 2024-09-16T01:59:04.739 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:04.739 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.739 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.740 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "3.0", 2024-09-16T01:59:04.740 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1.0", 2024-09-16T01:59:04.740 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.740 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.740 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "scaling threshold", 2024-09-16T01:59:04.740 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T01:59:04.740 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.740 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.740 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.740 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.740 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "progress", 2024-09-16T01:59:04.740 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_pg_recovery_event": { 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "allow_pg_recovery_event", 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.741 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: "enabled": { 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "enabled", 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.743 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.743 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.743 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.743 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.743 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.743 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.743 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.743 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.743 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.743 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.743 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.743 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.743 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.743 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.744 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.744 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.744 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.744 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.744 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.744 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.744 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.744 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.744 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.744 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.744 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.744 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.744 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.745 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.746 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.746 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.746 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.746 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.746 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.746 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.746 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.746 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.746 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.746 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.746 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.746 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.746 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.748 INFO:teuthology.orchestra.run.smithi139.stdout: "max_completed_events": { 2024-09-16T01:59:04.748 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_completed_events", 2024-09-16T01:59:04.748 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.748 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.748 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.748 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "50", 2024-09-16T01:59:04.748 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.748 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.748 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.748 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "number of past completed events to remember", 2024-09-16T01:59:04.748 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.748 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.748 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: "sleep_interval": { 2024-09-16T01:59:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "sleep_interval", 2024-09-16T01:59:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "how long the module is going to sleep", 2024-09-16T01:59:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "prometheus", 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: "cache": { 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "cache", 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.750 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.751 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.751 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.751 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.751 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.751 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.751 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.751 INFO:teuthology.orchestra.run.smithi139.stdout: "exclude_perf_counters": { 2024-09-16T01:59:04.751 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "exclude_perf_counters", 2024-09-16T01:59:04.751 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.751 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.751 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.751 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.751 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.752 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.752 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.752 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T01:59:04.752 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T01:59:04.752 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.752 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.752 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.752 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.752 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.752 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.752 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.752 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.752 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.753 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.753 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.753 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.753 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.753 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.753 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.753 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.753 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.753 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.753 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.753 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.753 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.753 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.754 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.754 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.754 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.754 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.754 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.754 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.754 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.754 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.754 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.754 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.754 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.754 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.754 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.755 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.756 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.756 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.756 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.756 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.756 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.756 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.756 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.756 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.756 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.756 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.756 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.756 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.757 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.757 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.757 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.757 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.757 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.757 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.757 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_stats_pools": { 2024-09-16T01:59:04.757 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rbd_stats_pools", 2024-09-16T01:59:04.757 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.757 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.757 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.757 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "300", 2024-09-16T01:59:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.759 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.759 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.759 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.759 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.759 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.759 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.759 INFO:teuthology.orchestra.run.smithi139.stdout: "scrape_interval": { 2024-09-16T01:59:04.759 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "scrape_interval", 2024-09-16T01:59:04.759 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:04.759 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.759 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.759 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "15.0", 2024-09-16T01:59:04.759 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.760 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.760 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.760 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.760 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.760 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.760 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.760 INFO:teuthology.orchestra.run.smithi139.stdout: "server_addr": { 2024-09-16T01:59:04.760 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_addr", 2024-09-16T01:59:04.760 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.760 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.760 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.760 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "::", 2024-09-16T01:59:04.760 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: "server_port": { 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_port", 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "9283", 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.761 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.762 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.762 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T01:59:04.762 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.762 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.762 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.762 INFO:teuthology.orchestra.run.smithi139.stdout: "stale_cache_strategy": { 2024-09-16T01:59:04.762 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "stale_cache_strategy", 2024-09-16T01:59:04.762 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.762 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.762 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.762 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "log", 2024-09-16T01:59:04.762 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.762 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.763 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.763 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.763 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.763 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.763 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.763 INFO:teuthology.orchestra.run.smithi139.stdout: "standby_behaviour": { 2024-09-16T01:59:04.763 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "standby_behaviour", 2024-09-16T01:59:04.763 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.763 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.763 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.763 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "default", 2024-09-16T01:59:04.763 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.763 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.764 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.764 INFO:teuthology.orchestra.run.smithi139.stdout: "default", 2024-09-16T01:59:04.764 INFO:teuthology.orchestra.run.smithi139.stdout: "error" 2024-09-16T01:59:04.764 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.764 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.764 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.764 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.764 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.764 INFO:teuthology.orchestra.run.smithi139.stdout: "standby_error_status_code": { 2024-09-16T01:59:04.764 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "standby_error_status_code", 2024-09-16T01:59:04.764 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.764 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.764 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.765 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "500", 2024-09-16T01:59:04.765 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "400", 2024-09-16T01:59:04.765 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "599", 2024-09-16T01:59:04.765 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.765 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.765 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.765 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.765 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.765 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.765 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.765 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rbd_support", 2024-09-16T01:59:04.765 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.766 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.767 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.767 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.767 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.767 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.767 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.767 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.767 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.767 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.767 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.767 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.767 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.767 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.767 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.768 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.768 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.768 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.768 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.768 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.768 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.768 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.768 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.768 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.768 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.768 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.768 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.768 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.769 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.769 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.769 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.769 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.769 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.769 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.769 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.769 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.769 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.769 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.769 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.769 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.770 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.771 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.771 INFO:teuthology.orchestra.run.smithi139.stdout: "max_concurrent_snap_create": { 2024-09-16T01:59:04.771 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_concurrent_snap_create", 2024-09-16T01:59:04.771 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.771 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.771 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.771 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10", 2024-09-16T01:59:04.771 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.771 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.771 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.771 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.771 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.771 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.772 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.772 INFO:teuthology.orchestra.run.smithi139.stdout: "mirror_snapshot_schedule": { 2024-09-16T01:59:04.772 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T01:59:04.772 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.772 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.772 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.772 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.772 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.772 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.772 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.772 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.772 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.773 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.773 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.773 INFO:teuthology.orchestra.run.smithi139.stdout: "trash_purge_schedule": { 2024-09-16T01:59:04.773 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "trash_purge_schedule", 2024-09-16T01:59:04.773 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.773 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.773 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.773 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.773 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.773 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.773 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.773 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.773 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.774 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.774 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.774 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.774 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.774 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "restful", 2024-09-16T01:59:04.774 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.774 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.774 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.774 INFO:teuthology.orchestra.run.smithi139.stdout: "enable_auth": { 2024-09-16T01:59:04.774 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "enable_auth", 2024-09-16T01:59:04.774 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.774 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: "key_file": { 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "key_file", 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.775 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.776 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.776 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.776 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.776 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.776 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.776 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.776 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.776 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.776 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.776 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.776 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.776 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.776 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.777 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.777 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.777 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.777 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.777 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.777 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.777 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.777 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.777 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.777 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.777 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.777 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.777 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.777 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.778 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.779 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.779 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.779 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.779 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.779 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.779 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.779 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.779 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.779 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.779 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.779 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.779 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.779 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.780 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.780 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.780 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.780 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.780 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.780 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.780 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.780 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.780 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.780 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.780 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.780 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.780 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.781 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.781 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.781 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.781 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.785 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.785 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.785 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.785 INFO:teuthology.orchestra.run.smithi139.stdout: "server_addr": { 2024-09-16T01:59:04.785 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_addr", 2024-09-16T01:59:04.785 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.785 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.785 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.785 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.785 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.785 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.785 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.785 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: "server_port": { 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "server_port", 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.786 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.787 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.787 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.787 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.787 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.787 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rgw", 2024-09-16T01:59:04.787 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.787 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.787 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.787 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.787 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.787 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.787 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.788 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.788 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.788 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.788 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.788 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.788 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.788 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.788 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.788 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.788 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.788 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.788 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.788 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.789 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.790 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.790 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.790 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.790 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.790 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.790 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.790 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.790 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.790 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.790 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.790 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.791 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.791 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.791 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.791 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.791 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.791 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.791 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.791 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.791 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.791 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.791 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.791 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.792 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.792 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.792 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.792 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.792 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.792 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.792 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.792 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.792 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.792 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.792 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.792 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.792 INFO:teuthology.orchestra.run.smithi139.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T01:59:04.793 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T01:59:04.793 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.793 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.793 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.793 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "5", 2024-09-16T01:59:04.793 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.793 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.793 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.793 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T01:59:04.793 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.793 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.793 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.793 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.793 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.794 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rook", 2024-09-16T01:59:04.794 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.794 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.794 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.794 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.794 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.794 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.794 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.794 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.794 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.794 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.794 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.795 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.795 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.795 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.795 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.795 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.795 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.795 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.795 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.795 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.795 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.795 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.795 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.796 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.797 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.797 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.797 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.797 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.797 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.797 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.797 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.797 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.797 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.797 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.797 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.797 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.797 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.797 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.798 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.798 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.798 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.798 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.798 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.798 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.798 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.798 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.798 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.798 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.798 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.798 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.798 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: "storage_class": { 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "storage_class", 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "local", 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.799 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "selftest", 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.800 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.801 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.801 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.801 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.801 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.801 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.801 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.801 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.801 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.801 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.801 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.801 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.801 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.801 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.801 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.802 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.802 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.802 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.802 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.802 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.802 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.802 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.802 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.802 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.802 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.802 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.802 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.802 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.803 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.804 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.804 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.804 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.804 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.804 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.804 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.804 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.804 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.804 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.804 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.804 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.804 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: "roption1": { 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "roption1", 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.805 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.806 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.806 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.806 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.806 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.806 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.806 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.806 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.806 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.806 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.806 INFO:teuthology.orchestra.run.smithi139.stdout: "roption2": { 2024-09-16T01:59:04.806 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "roption2", 2024-09-16T01:59:04.806 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.806 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.807 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.807 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "xyz", 2024-09-16T01:59:04.807 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.807 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.807 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.807 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.807 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.807 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.807 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.807 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption1": { 2024-09-16T01:59:04.807 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption1", 2024-09-16T01:59:04.807 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.808 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.808 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.808 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.808 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.808 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.808 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.808 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.808 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.808 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.808 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.808 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption2": { 2024-09-16T01:59:04.808 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption2", 2024-09-16T01:59:04.808 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption3": { 2024-09-16T01:59:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption3", 2024-09-16T01:59:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "float", 2024-09-16T01:59:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption4": { 2024-09-16T01:59:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption4", 2024-09-16T01:59:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.811 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.811 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.811 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.811 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.811 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.811 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.811 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.811 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.811 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.811 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.811 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption5": { 2024-09-16T01:59:04.811 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption5", 2024-09-16T01:59:04.811 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.812 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.812 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.812 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.812 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.812 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.812 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.812 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.812 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.812 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.812 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.812 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption6": { 2024-09-16T01:59:04.812 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption6", 2024-09-16T01:59:04.813 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.813 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.813 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.813 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.813 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.813 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.813 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.813 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.813 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.813 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.813 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.813 INFO:teuthology.orchestra.run.smithi139.stdout: "rwoption7": { 2024-09-16T01:59:04.813 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rwoption7", 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "1", 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "42", 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: "testkey": { 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "testkey", 2024-09-16T01:59:04.814 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.815 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.815 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.815 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.815 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.815 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.815 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.815 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.815 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.815 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.815 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.815 INFO:teuthology.orchestra.run.smithi139.stdout: "testlkey": { 2024-09-16T01:59:04.815 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "testlkey", 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: "testnewline": { 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "testnewline", 2024-09-16T01:59:04.816 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.817 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.817 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.817 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.817 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.817 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.817 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.817 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.817 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.817 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.817 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.817 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.817 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.818 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "snap_schedule", 2024-09-16T01:59:04.818 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.818 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.818 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.818 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_m_granularity": { 2024-09-16T01:59:04.818 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "allow_m_granularity", 2024-09-16T01:59:04.818 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.818 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.818 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.818 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.818 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.818 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.818 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: "dump_on_update": { 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "dump_on_update", 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "dump database to debug log on update", 2024-09-16T01:59:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.821 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.821 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.821 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.821 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.821 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.821 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.821 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.821 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.821 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.821 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.821 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.821 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.822 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.822 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.822 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.822 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.822 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.822 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.822 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.822 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.822 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.822 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.822 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.823 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.823 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.823 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.823 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.823 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.823 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.823 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.823 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.823 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.823 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.823 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.823 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.824 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.824 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.824 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.824 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.824 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.824 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.824 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.824 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.824 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.824 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.824 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.824 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.825 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.825 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.825 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.825 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.825 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.825 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.825 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.825 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.825 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.825 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.825 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.826 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "stats", 2024-09-16T01:59:04.826 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.826 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.826 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.826 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.826 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.826 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.826 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.826 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.826 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.829 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.829 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.829 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.829 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.829 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.829 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.829 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.829 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.829 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.829 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.829 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.830 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.830 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.830 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.830 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.830 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.830 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.830 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.830 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.830 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.831 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.831 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.831 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.831 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.831 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.831 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.831 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.831 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.831 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.832 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.832 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.832 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.832 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.832 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.832 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.832 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.832 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.832 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.832 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.832 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.832 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.832 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.833 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.833 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.833 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.833 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.833 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.833 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.833 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.833 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.833 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.833 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.833 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.833 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.834 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.834 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.834 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.834 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.834 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.834 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.834 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.834 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.834 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "status", 2024-09-16T01:59:04.834 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.834 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.834 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.835 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.835 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.835 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.835 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.835 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.835 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.835 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.835 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.835 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.835 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.835 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.835 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.835 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.836 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.836 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.836 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.836 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.836 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.836 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.836 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.836 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.836 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.836 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.836 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.836 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.837 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.837 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.837 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.837 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.837 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.837 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.837 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.837 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.837 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.837 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.837 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.837 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.837 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.838 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.838 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.838 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.838 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.838 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.838 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.838 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.838 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.838 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.838 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.838 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.838 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.838 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.839 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.839 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.839 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.839 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.839 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.839 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.839 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.839 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.839 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.839 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.839 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.839 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.839 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.840 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.840 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.840 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.840 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.840 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "telegraf", 2024-09-16T01:59:04.840 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.840 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.840 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.840 INFO:teuthology.orchestra.run.smithi139.stdout: "address": { 2024-09-16T01:59:04.841 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "address", 2024-09-16T01:59:04.841 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.841 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.841 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.841 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T01:59:04.841 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.841 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.841 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.841 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.841 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.842 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.842 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.842 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:04.842 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:04.842 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.842 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.842 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.842 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "15", 2024-09-16T01:59:04.842 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.842 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.842 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.842 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.842 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.843 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.843 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.843 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.843 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.843 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.843 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.843 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.843 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.843 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.843 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.843 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.843 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.844 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.844 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.844 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.844 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.844 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.844 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.844 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.844 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.844 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.844 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.844 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.844 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.845 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.845 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.845 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.845 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.845 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.845 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.845 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.845 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.845 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.845 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.845 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.845 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.845 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.846 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.846 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.846 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.846 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.846 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.846 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.846 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.846 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.846 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.846 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.846 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.846 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.846 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.846 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.848 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.848 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.848 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.848 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.848 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.848 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.848 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "telemetry", 2024-09-16T01:59:04.848 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.848 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.848 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.848 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_basic": { 2024-09-16T01:59:04.849 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_basic", 2024-09-16T01:59:04.849 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.849 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.849 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.849 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.849 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.849 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.849 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.849 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T01:59:04.849 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.849 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.849 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_crash": { 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_crash", 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.850 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_device": { 2024-09-16T01:59:04.851 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_device", 2024-09-16T01:59:04.851 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.851 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.851 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.851 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.851 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.851 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.851 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.851 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T01:59:04.851 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.851 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.851 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.851 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_ident": { 2024-09-16T01:59:04.852 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_ident", 2024-09-16T01:59:04.852 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.852 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.852 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.852 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.852 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.852 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.852 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.852 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T01:59:04.852 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.852 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.852 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.852 INFO:teuthology.orchestra.run.smithi139.stdout: "channel_perf": { 2024-09-16T01:59:04.853 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "channel_perf", 2024-09-16T01:59:04.853 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.853 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.853 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.853 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.853 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.853 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.853 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.853 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T01:59:04.853 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.853 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.853 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.853 INFO:teuthology.orchestra.run.smithi139.stdout: "contact": { 2024-09-16T01:59:04.854 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "contact", 2024-09-16T01:59:04.854 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.854 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.854 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.854 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.854 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.854 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.854 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.854 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.854 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.854 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.854 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.854 INFO:teuthology.orchestra.run.smithi139.stdout: "description": { 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "description", 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: "device_url": { 2024-09-16T01:59:04.855 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "device_url", 2024-09-16T01:59:04.856 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.856 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.856 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.856 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T01:59:04.856 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.856 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.856 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.856 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.856 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.856 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.856 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.856 INFO:teuthology.orchestra.run.smithi139.stdout: "enabled": { 2024-09-16T01:59:04.856 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "enabled", 2024-09-16T01:59:04.857 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.857 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.857 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.857 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.857 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.857 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.857 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.857 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.857 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.857 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.857 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.857 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:04.858 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:04.858 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.858 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.858 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.858 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "24", 2024-09-16T01:59:04.858 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "8", 2024-09-16T01:59:04.858 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.858 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.858 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.858 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.858 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.858 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.858 INFO:teuthology.orchestra.run.smithi139.stdout: "last_opt_revision": { 2024-09-16T01:59:04.859 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "last_opt_revision", 2024-09-16T01:59:04.859 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.859 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.859 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.859 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "1", 2024-09-16T01:59:04.859 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.859 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.859 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.859 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.859 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.859 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.859 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.859 INFO:teuthology.orchestra.run.smithi139.stdout: "leaderboard": { 2024-09-16T01:59:04.860 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "leaderboard", 2024-09-16T01:59:04.860 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.860 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.860 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.860 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.860 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.860 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.860 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.860 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.860 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.860 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.860 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.860 INFO:teuthology.orchestra.run.smithi139.stdout: "leaderboard_description": { 2024-09-16T01:59:04.861 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "leaderboard_description", 2024-09-16T01:59:04.861 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.861 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.861 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.861 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.861 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.861 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.861 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.861 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.861 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.861 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.861 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.862 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.862 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.862 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.862 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.862 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.862 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.862 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.862 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.862 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.862 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.862 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.862 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.862 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.862 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.863 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.863 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.863 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.863 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.863 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.863 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.863 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.863 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.863 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.863 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.863 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.863 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.863 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.864 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.864 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.864 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.864 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.864 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.864 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.864 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.864 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.864 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.864 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.864 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.864 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.865 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.865 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.865 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.865 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.865 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.865 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.865 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.865 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.865 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.865 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.865 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.865 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.865 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.865 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.866 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.866 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.866 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.866 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.866 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.866 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.866 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.866 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.866 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.866 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.866 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.866 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.867 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.867 INFO:teuthology.orchestra.run.smithi139.stdout: "organization": { 2024-09-16T01:59:04.867 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "organization", 2024-09-16T01:59:04.867 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.867 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.867 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.867 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.867 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.867 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.867 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.867 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.867 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.867 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.868 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.868 INFO:teuthology.orchestra.run.smithi139.stdout: "proxy": { 2024-09-16T01:59:04.868 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "proxy", 2024-09-16T01:59:04.868 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.868 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.868 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.868 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.868 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.868 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.868 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.868 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.868 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.868 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.869 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.869 INFO:teuthology.orchestra.run.smithi139.stdout: "url": { 2024-09-16T01:59:04.869 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "url", 2024-09-16T01:59:04.869 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.869 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.869 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.869 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T01:59:04.869 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.869 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.869 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.869 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.869 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.869 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.870 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.870 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.870 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.870 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "test_orchestrator", 2024-09-16T01:59:04.870 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.870 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.870 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.870 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.870 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.870 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.870 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.870 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.871 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.871 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.871 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.871 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.871 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.871 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.871 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.871 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.871 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.871 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.871 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.871 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.871 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.871 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.872 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.872 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.872 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.872 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.872 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.872 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.872 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.872 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.872 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.872 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.872 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.872 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.872 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.873 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.873 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.873 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.873 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.873 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.873 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.873 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.873 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.873 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.873 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.873 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.873 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.873 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.874 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.874 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.874 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.874 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.874 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.874 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.874 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.874 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.874 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.874 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.874 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.874 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.875 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.875 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.875 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.875 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.875 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.875 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.875 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.875 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.875 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.875 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.875 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.875 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "volumes", 2024-09-16T01:59:04.876 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.876 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.876 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.876 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.876 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.876 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.876 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.876 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.876 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.876 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.876 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.876 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.876 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.877 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.877 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.877 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.877 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.877 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.877 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.877 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.877 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.877 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.877 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.877 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.877 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.877 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.878 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.878 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.878 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.878 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.878 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.878 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.878 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.878 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.878 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.878 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.878 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.878 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.878 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.879 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.879 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.879 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.879 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.879 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.879 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.879 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.879 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.879 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.879 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.879 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.879 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.879 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.880 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.880 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.880 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.880 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.880 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.880 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.880 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.880 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.880 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.880 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.880 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.880 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.880 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.881 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.881 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.881 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.881 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.881 INFO:teuthology.orchestra.run.smithi139.stdout: "max_concurrent_clones": { 2024-09-16T01:59:04.881 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "max_concurrent_clones", 2024-09-16T01:59:04.881 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.881 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.881 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.881 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "4", 2024-09-16T01:59:04.881 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.881 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.882 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.882 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T01:59:04.882 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.882 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.882 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.882 INFO:teuthology.orchestra.run.smithi139.stdout: "periodic_async_work": { 2024-09-16T01:59:04.882 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "periodic_async_work", 2024-09-16T01:59:04.882 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.882 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.882 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.882 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.882 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Periodically check for async work", 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: "snapshot_clone_delay": { 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "snapshot_clone_delay", 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "0", 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.883 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.884 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.884 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T01:59:04.884 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.884 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.884 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.884 INFO:teuthology.orchestra.run.smithi139.stdout: "snapshot_clone_no_wait": { 2024-09-16T01:59:04.884 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T01:59:04.884 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.884 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.884 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.884 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "True", 2024-09-16T01:59:04.884 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.885 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.885 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.885 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T01:59:04.885 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.885 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.885 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.885 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.885 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.885 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "zabbix", 2024-09-16T01:59:04.885 INFO:teuthology.orchestra.run.smithi139.stdout: "can_run": true, 2024-09-16T01:59:04.885 INFO:teuthology.orchestra.run.smithi139.stdout: "error_string": "", 2024-09-16T01:59:04.885 INFO:teuthology.orchestra.run.smithi139.stdout: "module_options": { 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: "discovery_interval": { 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "discovery_interval", 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "uint", 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "100", 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.886 INFO:teuthology.orchestra.run.smithi139.stdout: "identifier": { 2024-09-16T01:59:04.887 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "identifier", 2024-09-16T01:59:04.887 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.887 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.887 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.887 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.887 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.887 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.887 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.887 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.887 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.887 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.887 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.888 INFO:teuthology.orchestra.run.smithi139.stdout: "interval": { 2024-09-16T01:59:04.888 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "interval", 2024-09-16T01:59:04.888 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "secs", 2024-09-16T01:59:04.888 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.888 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.888 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "60", 2024-09-16T01:59:04.888 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.888 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.888 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.888 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.888 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.888 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.888 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "log_level": { 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_level", 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.889 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.890 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.890 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.890 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.890 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.890 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.890 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster": { 2024-09-16T01:59:04.890 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster", 2024-09-16T01:59:04.890 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.890 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.890 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.890 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.890 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.891 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.891 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.891 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.891 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.891 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.891 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.891 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_cluster_level": { 2024-09-16T01:59:04.891 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_cluster_level", 2024-09-16T01:59:04.891 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.891 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.891 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.891 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "info", 2024-09-16T01:59:04.891 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.892 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.892 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [ 2024-09-16T01:59:04.892 INFO:teuthology.orchestra.run.smithi139.stdout: "", 2024-09-16T01:59:04.892 INFO:teuthology.orchestra.run.smithi139.stdout: "critical", 2024-09-16T01:59:04.892 INFO:teuthology.orchestra.run.smithi139.stdout: "debug", 2024-09-16T01:59:04.892 INFO:teuthology.orchestra.run.smithi139.stdout: "error", 2024-09-16T01:59:04.892 INFO:teuthology.orchestra.run.smithi139.stdout: "info", 2024-09-16T01:59:04.892 INFO:teuthology.orchestra.run.smithi139.stdout: "warning" 2024-09-16T01:59:04.892 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.892 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.892 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.892 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.892 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.893 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": { 2024-09-16T01:59:04.893 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "log_to_file", 2024-09-16T01:59:04.893 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "bool", 2024-09-16T01:59:04.893 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.893 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T01:59:04.893 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "False", 2024-09-16T01:59:04.893 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.893 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.893 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.893 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.893 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.893 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.893 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.894 INFO:teuthology.orchestra.run.smithi139.stdout: "zabbix_host": { 2024-09-16T01:59:04.894 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "zabbix_host", 2024-09-16T01:59:04.894 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.894 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.894 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.894 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "", 2024-09-16T01:59:04.894 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.894 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.894 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.895 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.895 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.895 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.895 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.895 INFO:teuthology.orchestra.run.smithi139.stdout: "zabbix_port": { 2024-09-16T01:59:04.896 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "zabbix_port", 2024-09-16T01:59:04.896 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "int", 2024-09-16T01:59:04.896 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.896 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.896 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "10051", 2024-09-16T01:59:04.896 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.896 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.896 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.896 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.896 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.896 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.896 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.897 INFO:teuthology.orchestra.run.smithi139.stdout: "zabbix_sender": { 2024-09-16T01:59:04.897 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "zabbix_sender", 2024-09-16T01:59:04.897 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "str", 2024-09-16T01:59:04.897 INFO:teuthology.orchestra.run.smithi139.stdout: "level": "advanced", 2024-09-16T01:59:04.897 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 0, 2024-09-16T01:59:04.897 INFO:teuthology.orchestra.run.smithi139.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T01:59:04.897 INFO:teuthology.orchestra.run.smithi139.stdout: "min": "", 2024-09-16T01:59:04.897 INFO:teuthology.orchestra.run.smithi139.stdout: "max": "", 2024-09-16T01:59:04.897 INFO:teuthology.orchestra.run.smithi139.stdout: "enum_allowed": [], 2024-09-16T01:59:04.897 INFO:teuthology.orchestra.run.smithi139.stdout: "desc": "", 2024-09-16T01:59:04.897 INFO:teuthology.orchestra.run.smithi139.stdout: "long_desc": "", 2024-09-16T01:59:04.897 INFO:teuthology.orchestra.run.smithi139.stdout: "tags": [], 2024-09-16T01:59:04.897 INFO:teuthology.orchestra.run.smithi139.stdout: "see_also": [] 2024-09-16T01:59:04.897 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.898 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.898 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.898 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.898 INFO:teuthology.orchestra.run.smithi139.stdout: "services": { 2024-09-16T01:59:04.898 INFO:teuthology.orchestra.run.smithi139.stdout: "dashboard": "https://172.21.15.139:8443/" 2024-09-16T01:59:04.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.898 INFO:teuthology.orchestra.run.smithi139.stdout: "always_on_modules": { 2024-09-16T01:59:04.898 INFO:teuthology.orchestra.run.smithi139.stdout: "octopus": [ 2024-09-16T01:59:04.898 INFO:teuthology.orchestra.run.smithi139.stdout: "balancer", 2024-09-16T01:59:04.898 INFO:teuthology.orchestra.run.smithi139.stdout: "crash", 2024-09-16T01:59:04.906 INFO:teuthology.orchestra.run.smithi139.stdout: "devicehealth", 2024-09-16T01:59:04.906 INFO:teuthology.orchestra.run.smithi139.stdout: "orchestrator", 2024-09-16T01:59:04.906 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscaler", 2024-09-16T01:59:04.906 INFO:teuthology.orchestra.run.smithi139.stdout: "progress", 2024-09-16T01:59:04.906 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_support", 2024-09-16T01:59:04.906 INFO:teuthology.orchestra.run.smithi139.stdout: "status", 2024-09-16T01:59:04.906 INFO:teuthology.orchestra.run.smithi139.stdout: "telemetry", 2024-09-16T01:59:04.907 INFO:teuthology.orchestra.run.smithi139.stdout: "volumes" 2024-09-16T01:59:04.907 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.907 INFO:teuthology.orchestra.run.smithi139.stdout: "pacific": [ 2024-09-16T01:59:04.907 INFO:teuthology.orchestra.run.smithi139.stdout: "balancer", 2024-09-16T01:59:04.907 INFO:teuthology.orchestra.run.smithi139.stdout: "crash", 2024-09-16T01:59:04.908 INFO:teuthology.orchestra.run.smithi139.stdout: "devicehealth", 2024-09-16T01:59:04.908 INFO:teuthology.orchestra.run.smithi139.stdout: "orchestrator", 2024-09-16T01:59:04.908 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscaler", 2024-09-16T01:59:04.908 INFO:teuthology.orchestra.run.smithi139.stdout: "progress", 2024-09-16T01:59:04.908 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_support", 2024-09-16T01:59:04.908 INFO:teuthology.orchestra.run.smithi139.stdout: "status", 2024-09-16T01:59:04.908 INFO:teuthology.orchestra.run.smithi139.stdout: "telemetry", 2024-09-16T01:59:04.909 INFO:teuthology.orchestra.run.smithi139.stdout: "volumes" 2024-09-16T01:59:04.909 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.909 INFO:teuthology.orchestra.run.smithi139.stdout: "quincy": [ 2024-09-16T01:59:04.909 INFO:teuthology.orchestra.run.smithi139.stdout: "balancer", 2024-09-16T01:59:04.909 INFO:teuthology.orchestra.run.smithi139.stdout: "crash", 2024-09-16T01:59:04.909 INFO:teuthology.orchestra.run.smithi139.stdout: "devicehealth", 2024-09-16T01:59:04.909 INFO:teuthology.orchestra.run.smithi139.stdout: "orchestrator", 2024-09-16T01:59:04.910 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscaler", 2024-09-16T01:59:04.910 INFO:teuthology.orchestra.run.smithi139.stdout: "progress", 2024-09-16T01:59:04.910 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_support", 2024-09-16T01:59:04.910 INFO:teuthology.orchestra.run.smithi139.stdout: "status", 2024-09-16T01:59:04.910 INFO:teuthology.orchestra.run.smithi139.stdout: "telemetry", 2024-09-16T01:59:04.910 INFO:teuthology.orchestra.run.smithi139.stdout: "volumes" 2024-09-16T01:59:04.910 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.911 INFO:teuthology.orchestra.run.smithi139.stdout: "reef": [ 2024-09-16T01:59:04.911 INFO:teuthology.orchestra.run.smithi139.stdout: "balancer", 2024-09-16T01:59:04.911 INFO:teuthology.orchestra.run.smithi139.stdout: "crash", 2024-09-16T01:59:04.911 INFO:teuthology.orchestra.run.smithi139.stdout: "devicehealth", 2024-09-16T01:59:04.911 INFO:teuthology.orchestra.run.smithi139.stdout: "orchestrator", 2024-09-16T01:59:04.911 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscaler", 2024-09-16T01:59:04.911 INFO:teuthology.orchestra.run.smithi139.stdout: "progress", 2024-09-16T01:59:04.911 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_support", 2024-09-16T01:59:04.911 INFO:teuthology.orchestra.run.smithi139.stdout: "status", 2024-09-16T01:59:04.912 INFO:teuthology.orchestra.run.smithi139.stdout: "telemetry", 2024-09-16T01:59:04.912 INFO:teuthology.orchestra.run.smithi139.stdout: "volumes" 2024-09-16T01:59:04.912 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T01:59:04.912 INFO:teuthology.orchestra.run.smithi139.stdout: "squid": [ 2024-09-16T01:59:04.912 INFO:teuthology.orchestra.run.smithi139.stdout: "balancer", 2024-09-16T01:59:04.912 INFO:teuthology.orchestra.run.smithi139.stdout: "crash", 2024-09-16T01:59:04.912 INFO:teuthology.orchestra.run.smithi139.stdout: "devicehealth", 2024-09-16T01:59:04.912 INFO:teuthology.orchestra.run.smithi139.stdout: "orchestrator", 2024-09-16T01:59:04.912 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscaler", 2024-09-16T01:59:04.912 INFO:teuthology.orchestra.run.smithi139.stdout: "progress", 2024-09-16T01:59:04.912 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_support", 2024-09-16T01:59:04.912 INFO:teuthology.orchestra.run.smithi139.stdout: "status", 2024-09-16T01:59:04.913 INFO:teuthology.orchestra.run.smithi139.stdout: "telemetry", 2024-09-16T01:59:04.913 INFO:teuthology.orchestra.run.smithi139.stdout: "volumes" 2024-09-16T01:59:04.913 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T01:59:04.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.913 INFO:teuthology.orchestra.run.smithi139.stdout: "last_failure_osd_epoch": 86, 2024-09-16T01:59:04.913 INFO:teuthology.orchestra.run.smithi139.stdout: "active_clients": [ 2024-09-16T01:59:04.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.913 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "devicehealth", 2024-09-16T01:59:04.913 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T01:59:04.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.913 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T01:59:04.913 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:0", 2024-09-16T01:59:04.913 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2926084438 2024-09-16T01:59:04.914 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.914 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T01:59:04.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.914 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "libcephsqlite", 2024-09-16T01:59:04.914 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T01:59:04.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.914 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T01:59:04.914 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:0", 2024-09-16T01:59:04.914 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1611359795 2024-09-16T01:59:04.915 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.915 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T01:59:04.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.915 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "rbd_support", 2024-09-16T01:59:04.915 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T01:59:04.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.915 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T01:59:04.915 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:0", 2024-09-16T01:59:04.915 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4100719844 2024-09-16T01:59:04.915 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.915 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T01:59:04.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "snap_schedule", 2024-09-16T01:59:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T01:59:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T01:59:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:0", 2024-09-16T01:59:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2399395927 2024-09-16T01:59:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T01:59:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "snap_schedule", 2024-09-16T01:59:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T01:59:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T01:59:04.917 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:0", 2024-09-16T01:59:04.917 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4166355656 2024-09-16T01:59:04.917 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.917 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T01:59:04.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:04.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.917 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "volumes", 2024-09-16T01:59:04.917 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T01:59:04.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T01:59:04.917 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T01:59:04.917 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:0", 2024-09-16T01:59:04.917 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2651358007 2024-09-16T01:59:04.917 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.917 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T01:59:04.918 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:04.918 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T01:59:04.918 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:59:04.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.535+0000 7fbed359e640 1 -- 172.21.15.139:0/1109415943 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbeb80a35c0 msgr2=0x7fbeb80a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:04.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.535+0000 7fbed359e640 1 --2- 172.21.15.139:0/1109415943 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbeb80a35c0 0x7fbeb80a5a80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbecc006010 tx=0x7fbecc000f40 comp rx=0 tx=0).stop 2024-09-16T01:59:04.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.535+0000 7fbed359e640 1 -- 172.21.15.139:0/1109415943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed41095d0 msgr2=0x7fbed4116940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:04.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.535+0000 7fbed359e640 1 --2- 172.21.15.139:0/1109415943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed41095d0 0x7fbed4116940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009fd0 tx=0x7fbec000b870 comp rx=0 tx=0).stop 2024-09-16T01:59:04.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.536+0000 7fbed359e640 1 -- 172.21.15.139:0/1109415943 shutdown_connections 2024-09-16T01:59:04.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.536+0000 7fbed359e640 1 --2- 172.21.15.139:0/1109415943 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fbeb80a35c0 0x7fbeb80a5a80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:04.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.536+0000 7fbed359e640 1 --2- 172.21.15.139:0/1109415943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed411b140 0x7fbed4116e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:04.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.536+0000 7fbed359e640 1 --2- 172.21.15.139:0/1109415943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed41095d0 0x7fbed4116940 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:04.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.536+0000 7fbed359e640 1 --2- 172.21.15.139:0/1109415943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbed41091f0 0x7fbed411a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:04.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.536+0000 7fbed359e640 1 -- 172.21.15.139:0/1109415943 >> 172.21.15.139:0/1109415943 conn(0x7fbed406cba0 msgr2=0x7fbed40706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:04.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.540+0000 7fbed359e640 1 -- 172.21.15.139:0/1109415943 shutdown_connections 2024-09-16T01:59:04.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:03.540+0000 7fbed359e640 1 -- 172.21.15.139:0/1109415943 wait complete. 2024-09-16T01:59:04.920 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:03 smithi139 ceph-mon[31175]: pgmap v594: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 KiB/s wr, 116 op/s 2024-09-16T01:59:04.920 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:03 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2662717134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T01:59:04.920 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:03 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3721461192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T01:59:04.920 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:03 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:04.920 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:03 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:04.920 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:03 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:59:04.921 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:03 smithi176 ceph-mon[35324]: pgmap v594: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 KiB/s wr, 116 op/s 2024-09-16T01:59:04.921 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:03 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2662717134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T01:59:04.921 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:03 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3721461192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T01:59:04.921 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:03 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:04.921 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:03 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:04.921 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:03 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:59:04.922 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:04 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1109415943' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T01:59:04.922 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:03 smithi164 ceph-mon[35121]: pgmap v594: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 KiB/s wr, 116 op/s 2024-09-16T01:59:04.922 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:03 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2662717134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T01:59:04.922 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:03 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3721461192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T01:59:04.922 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:03 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T01:59:04.922 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:03 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T01:59:04.922 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:03 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T01:59:04.922 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:04 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1109415943' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T01:59:04.978 DEBUG:tasks.check_counter:Getting stats from a 2024-09-16T01:59:04.978 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-16T01:59:04.979 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:04 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1109415943' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T01:59:05.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:05 smithi139 ceph-mon[31175]: pgmap v595: 129 pgs: 16 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 59 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 58 KiB/s wr, 148 op/s 2024-09-16T01:59:05.475 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:05.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:05 smithi164 ceph-mon[35121]: pgmap v595: 129 pgs: 16 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 59 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 58 KiB/s wr, 148 op/s 2024-09-16T01:59:05.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:05 smithi176 ceph-mon[35324]: pgmap v595: 129 pgs: 16 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 59 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 58 KiB/s wr, 148 op/s 2024-09-16T01:59:06.198 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:59:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T01:59:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 1984, 2024-09-16T01:59:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 1986, 2024-09-16T01:59:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 84526302, 2024-09-16T01:59:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 630913, 2024-09-16T01:59:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 8, 2024-09-16T01:59:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 4, 2024-09-16T01:59:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.240103663, 2024-09-16T01:59:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.053940199, 2024-09-16T01:59:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.106225235, 2024-09-16T01:59:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.064743548, 2024-09-16T01:59:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1984, 2024-09-16T01:59:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.279267820, 2024-09-16T01:59:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000140759 2024-09-16T01:59:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T01:59:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T01:59:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T01:59:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 1589, 2024-09-16T01:59:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 1531, 2024-09-16T01:59:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 128005404, 2024-09-16T01:59:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 583096, 2024-09-16T01:59:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 6, 2024-09-16T01:59:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 5, 2024-09-16T01:59:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.246418144, 2024-09-16T01:59:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.049742716, 2024-09-16T01:59:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.167467988, 2024-09-16T01:59:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.036443292, 2024-09-16T01:59:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1530, 2024-09-16T01:59:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.130916812, 2024-09-16T01:59:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000085566 2024-09-16T01:59:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 1855280, 2024-09-16T01:59:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 217328 2024-09-16T01:59:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T01:59:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 965, 2024-09-16T01:59:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 1220, 2024-09-16T01:59:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 92103835, 2024-09-16T01:59:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 1010440, 2024-09-16T01:59:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 9, 2024-09-16T01:59:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 6, 2024-09-16T01:59:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.163219814, 2024-09-16T01:59:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.041114050, 2024-09-16T01:59:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.081484700, 2024-09-16T01:59:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.032100138, 2024-09-16T01:59:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1218, 2024-09-16T01:59:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.125164593, 2024-09-16T01:59:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000102762 2024-09-16T01:59:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-09-16T01:59:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 706432 2024-09-16T01:59:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-09-16T01:59:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-09-16T01:59:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-09-16T01:59:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-09-16T01:59:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-09-16T01:59:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-09-16T01:59:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 6678, 2024-09-16T01:59:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.176599767, 2024-09-16T01:59:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000026445 2024-09-16T01:59:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-09-16T01:59:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-09-16T01:59:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-09-16T01:59:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-09-16T01:59:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 0, 2024-09-16T01:59:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 0, 2024-09-16T01:59:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-09-16T01:59:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-09-16T01:59:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 0, 2024-09-16T01:59:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-09-16T01:59:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-09-16T01:59:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 0, 2024-09-16T01:59:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 0, 2024-09-16T01:59:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-09-16T01:59:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 58584, 2024-09-16T01:59:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 7252, 2024-09-16T01:59:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 51332, 2024-09-16T01:59:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 0, 2024-09-16T01:59:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 48223, 2024-09-16T01:59:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 0, 2024-09-16T01:59:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 0, 2024-09-16T01:59:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 0, 2024-09-16T01:59:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 1, 2024-09-16T01:59:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 0, 2024-09-16T01:59:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 0, 2024-09-16T01:59:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 0, 2024-09-16T01:59:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 0, 2024-09-16T01:59:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-09-16T01:59:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-09-16T01:59:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 0, 2024-09-16T01:59:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 0, 2024-09-16T01:59:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-09-16T01:59:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 0, 2024-09-16T01:59:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 0, 2024-09-16T01:59:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 0, 2024-09-16T01:59:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 0, 2024-09-16T01:59:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-09-16T01:59:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-09-16T01:59:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-09-16T01:59:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 0, 2024-09-16T01:59:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 0, 2024-09-16T01:59:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 0, 2024-09-16T01:59:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T01:59:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-09-16T01:59:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T01:59:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-09-16T01:59:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-09-16T01:59:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T01:59:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T01:59:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-09-16T01:59:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 0, 2024-09-16T01:59:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T01:59:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 0, 2024-09-16T01:59:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T01:59:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T01:59:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T01:59:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 0, 2024-09-16T01:59:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 0, 2024-09-16T01:59:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 0, 2024-09-16T01:59:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 0 2024-09-16T01:59:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-09-16T01:59:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 0, 2024-09-16T01:59:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 0, 2024-09-16T01:59:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 0, 2024-09-16T01:59:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 0, 2024-09-16T01:59:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 0, 2024-09-16T01:59:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 0, 2024-09-16T01:59:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-09-16T01:59:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-09-16T01:59:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 0, 2024-09-16T01:59:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 0, 2024-09-16T01:59:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-09-16T01:59:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-09-16T01:59:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-09-16T01:59:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-09-16T01:59:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-09-16T01:59:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-09-16T01:59:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-09-16T01:59:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 0, 2024-09-16T01:59:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 0, 2024-09-16T01:59:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T01:59:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 0, 2024-09-16T01:59:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-09-16T01:59:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 0, 2024-09-16T01:59:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-09-16T01:59:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-09-16T01:59:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-09-16T01:59:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-09-16T01:59:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 0, 2024-09-16T01:59:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 0, 2024-09-16T01:59:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 0, 2024-09-16T01:59:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 92480, 2024-09-16T01:59:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-09-16T01:59:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 0, 2024-09-16T01:59:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 0, 2024-09-16T01:59:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 0, 2024-09-16T01:59:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 0, 2024-09-16T01:59:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 128, 2024-09-16T01:59:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 9, 2024-09-16T01:59:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-09-16T01:59:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 0, 2024-09-16T01:59:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 4194304, 2024-09-16T01:59:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 305759786, 2024-09-16T01:59:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 305751549, 2024-09-16T01:59:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-09-16T01:59:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 92480 2024-09-16T01:59:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-09-16T01:59:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 19558, 2024-09-16T01:59:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 59509, 2024-09-16T01:59:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 39951, 2024-09-16T01:59:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 807, 2024-09-16T01:59:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 1842, 2024-09-16T01:59:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 1035, 2024-09-16T01:59:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 58584, 2024-09-16T01:59:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 98338, 2024-09-16T01:59:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 39754, 2024-09-16T01:59:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 0, 2024-09-16T01:59:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 0, 2024-09-16T01:59:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 0, 2024-09-16T01:59:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 155348, 2024-09-16T01:59:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 215328 2024-09-16T01:59:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-09-16T01:59:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 0, 2024-09-16T01:59:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 0, 2024-09-16T01:59:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 0, 2024-09-16T01:59:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 0, 2024-09-16T01:59:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 0, 2024-09-16T01:59:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-09-16T01:59:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-09-16T01:59:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-09-16T01:59:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-09-16T01:59:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-09-16T01:59:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-09-16T01:59:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-09-16T01:59:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-09-16T01:59:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-09-16T01:59:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-09-16T01:59:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-09-16T01:59:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.224 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-09-16T01:59:06.224 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.224 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-09-16T01:59:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-09-16T01:59:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-09-16T01:59:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-09-16T01:59:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-09-16T01:59:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-09-16T01:59:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-09-16T01:59:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-09-16T01:59:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-09-16T01:59:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-09-16T01:59:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-09-16T01:59:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-09-16T01:59:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-09-16T01:59:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-09-16T01:59:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-09-16T01:59:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-09-16T01:59:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-09-16T01:59:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-09-16T01:59:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-09-16T01:59:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-09-16T01:59:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-09-16T01:59:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 2, 2024-09-16T01:59:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 4, 2024-09-16T01:59:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 2, 2024-09-16T01:59:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 3, 2024-09-16T01:59:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-09-16T01:59:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 0, 2024-09-16T01:59:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 0, 2024-09-16T01:59:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 2655, 2024-09-16T01:59:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-09-16T01:59:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-09-16T01:59:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-09-16T01:59:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-09-16T01:59:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T01:59:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-09-16T01:59:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T01:59:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-09-16T01:59:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T01:59:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T01:59:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T01:59:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T01:59:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T01:59:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-09-16T01:59:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T01:59:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T01:59:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-09-16T01:59:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-09-16T01:59:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-09-16T01:59:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-09-16T01:59:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T01:59:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T01:59:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T01:59:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T01:59:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T01:59:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-09-16T01:59:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-09-16T01:59:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-09-16T01:59:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T01:59:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T01:59:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-09-16T01:59:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-09-16T01:59:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-09-16T01:59:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-09-16T01:59:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T01:59:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-09-16T01:59:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T01:59:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-09-16T01:59:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 976903, 2024-09-16T01:59:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 19972, 2024-09-16T01:59:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-09-16T01:59:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-09-16T01:59:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-09-16T01:59:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-09-16T01:59:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-09-16T01:59:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-09-16T01:59:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-09-16T01:59:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-09-16T01:59:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 10160, 2024-09-16T01:59:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 216, 2024-09-16T01:59:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T01:59:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-09-16T01:59:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-09-16T01:59:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-09-16T01:59:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 87966762, 2024-09-16T01:59:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 793497, 2024-09-16T01:59:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-09-16T01:59:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-09-16T01:59:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-09-16T01:59:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-09-16T01:59:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-09-16T01:59:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-09-16T01:59:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-09-16T01:59:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 4090, 2024-09-16T01:59:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 21, 2024-09-16T01:59:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 3, 2024-09-16T01:59:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 4090, 2024-09-16T01:59:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-09-16T01:59:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 4090, 2024-09-16T01:59:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 43.524099980, 2024-09-16T01:59:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.010641589 2024-09-16T01:59:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-09-16T01:59:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-09-16T01:59:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 4090, 2024-09-16T01:59:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 4091 2024-09-16T01:59:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 4090, 2024-09-16T01:59:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 4090, 2024-09-16T01:59:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 0, 2024-09-16T01:59:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-09-16T01:59:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-09-16T01:59:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 2485, 2024-09-16T01:59:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 0, 2024-09-16T01:59:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 1603, 2024-09-16T01:59:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 0, 2024-09-16T01:59:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 0, 2024-09-16T01:59:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-09-16T01:59:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-09-16T01:59:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-09-16T01:59:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 0, 2024-09-16T01:59:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 0, 2024-09-16T01:59:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-09-16T01:59:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-09-16T01:59:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-09-16T01:59:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 1, 2024-09-16T01:59:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 0, 2024-09-16T01:59:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-09-16T01:59:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-09-16T01:59:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-09-16T01:59:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-09-16T01:59:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-09-16T01:59:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-09-16T01:59:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T01:59:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-09-16T01:59:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-09-16T01:59:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-09-16T01:59:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-09-16T01:59:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-09-16T01:59:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-09-16T01:59:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-09-16T01:59:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-09-16T01:59:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-09-16T01:59:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-09-16T01:59:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-09-16T01:59:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-09-16T01:59:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-09-16T01:59:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-09-16T01:59:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-09-16T01:59:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-09-16T01:59:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-09-16T01:59:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-09-16T01:59:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-09-16T01:59:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 118, 2024-09-16T01:59:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-09-16T01:59:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 35, 2024-09-16T01:59:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 11, 2024-09-16T01:59:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 11, 2024-09-16T01:59:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-09-16T01:59:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-09-16T01:59:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 0, 2024-09-16T01:59:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 2, 2024-09-16T01:59:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 0 2024-09-16T01:59:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-09-16T01:59:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 0, 2024-09-16T01:59:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-09-16T01:59:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-09-16T01:59:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-09-16T01:59:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-09-16T01:59:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 0 2024-09-16T01:59:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-09-16T01:59:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-09-16T01:59:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T01:59:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-09-16T01:59:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 0, 2024-09-16T01:59:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 0, 2024-09-16T01:59:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 0, 2024-09-16T01:59:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-09-16T01:59:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T01:59:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-16T01:59:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 4534, 2024-09-16T01:59:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 304202178, 2024-09-16T01:59:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 4534, 2024-09-16T01:59:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-16T01:59:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-16T01:59:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 4534, 2024-09-16T01:59:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 304202178, 2024-09-16T01:59:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-09-16T01:59:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-16T01:59:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-16T01:59:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-16T01:59:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 4090, 2024-09-16T01:59:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 301637245, 2024-09-16T01:59:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 4090, 2024-09-16T01:59:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 301637245, 2024-09-16T01:59:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-09-16T01:59:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-09-16T01:59:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-16T01:59:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-16T01:59:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 4090, 2024-09-16T01:59:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 4090, 2024-09-16T01:59:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 4090, 2024-09-16T01:59:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 4090, 2024-09-16T01:59:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-09-16T01:59:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-09-16T01:59:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-16T01:59:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-16T01:59:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-16T01:59:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-16T01:59:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-09-16T01:59:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-09-16T01:59:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x55e312f10b40": { 2024-09-16T01:59:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-09-16T01:59:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-16T01:59:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-16T01:59:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-16T01:59:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-16T01:59:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-09-16T01:59:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-09-16T01:59:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:06.255 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:06.255 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:06.255 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:06.255 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:06.255 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:59:07.218 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-16T01:59:07.218 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-16T01:59:07.218 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-16T01:59:07.218 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-16T01:59:07.219 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-16T01:59:07.219 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-16T01:59:07.219 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T01:59:07.219 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-09-16T01:59:07.219 DEBUG:tasks.check_counter:Getting stats from d 2024-09-16T01:59:07.219 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-16T01:59:07.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:07 smithi139 ceph-mon[31175]: pgmap v596: 129 pgs: 14 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 64 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 KiB/s wr, 154 op/s 2024-09-16T01:59:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:07 smithi164 ceph-mon[35121]: pgmap v596: 129 pgs: 14 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 64 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 KiB/s wr, 154 op/s 2024-09-16T01:59:07.681 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:07.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:07 smithi176 ceph-mon[35324]: pgmap v596: 129 pgs: 14 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 64 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 KiB/s wr, 154 op/s 2024-09-16T01:59:08.431 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:59:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T01:59:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 621, 2024-09-16T01:59:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 623, 2024-09-16T01:59:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 186603, 2024-09-16T01:59:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 197488, 2024-09-16T01:59:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 7, 2024-09-16T01:59:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 3, 2024-09-16T01:59:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.063792832, 2024-09-16T01:59:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.016089348, 2024-09-16T01:59:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.017032348, 2024-09-16T01:59:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.024695488, 2024-09-16T01:59:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 621, 2024-09-16T01:59:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.091382482, 2024-09-16T01:59:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000147153 2024-09-16T01:59:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T01:59:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T01:59:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T01:59:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 485, 2024-09-16T01:59:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 748, 2024-09-16T01:59:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 11653714, 2024-09-16T01:59:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 475075, 2024-09-16T01:59:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 7, 2024-09-16T01:59:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 5, 2024-09-16T01:59:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.077900944, 2024-09-16T01:59:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.030180411, 2024-09-16T01:59:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.023415474, 2024-09-16T01:59:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.016753101, 2024-09-16T01:59:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 745, 2024-09-16T01:59:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.115762340, 2024-09-16T01:59:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000155385 2024-09-16T01:59:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T01:59:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 321424 2024-09-16T01:59:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T01:59:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 394, 2024-09-16T01:59:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 372, 2024-09-16T01:59:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 1588085, 2024-09-16T01:59:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 219044, 2024-09-16T01:59:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 4, 2024-09-16T01:59:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 3, 2024-09-16T01:59:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.058874012, 2024-09-16T01:59:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.022668485, 2024-09-16T01:59:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.049506422, 2024-09-16T01:59:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.000090260, 2024-09-16T01:59:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 372, 2024-09-16T01:59:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.072507849, 2024-09-16T01:59:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000194913 2024-09-16T01:59:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 1586192, 2024-09-16T01:59:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 218080 2024-09-16T01:59:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-09-16T01:59:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-09-16T01:59:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-09-16T01:59:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-09-16T01:59:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-09-16T01:59:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-09-16T01:59:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1838, 2024-09-16T01:59:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.092659294, 2024-09-16T01:59:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000050413 2024-09-16T01:59:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-09-16T01:59:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-09-16T01:59:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-09-16T01:59:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-09-16T01:59:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 0, 2024-09-16T01:59:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 0, 2024-09-16T01:59:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-09-16T01:59:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-09-16T01:59:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 0, 2024-09-16T01:59:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-09-16T01:59:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-09-16T01:59:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 0, 2024-09-16T01:59:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 0, 2024-09-16T01:59:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-09-16T01:59:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 704, 2024-09-16T01:59:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 410, 2024-09-16T01:59:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 294, 2024-09-16T01:59:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 0, 2024-09-16T01:59:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 118, 2024-09-16T01:59:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 0, 2024-09-16T01:59:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 0, 2024-09-16T01:59:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 0, 2024-09-16T01:59:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 2, 2024-09-16T01:59:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 0, 2024-09-16T01:59:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 0, 2024-09-16T01:59:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 0, 2024-09-16T01:59:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 0, 2024-09-16T01:59:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-09-16T01:59:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-09-16T01:59:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 0, 2024-09-16T01:59:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 0, 2024-09-16T01:59:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-09-16T01:59:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 0, 2024-09-16T01:59:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 0, 2024-09-16T01:59:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 0, 2024-09-16T01:59:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 0, 2024-09-16T01:59:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-09-16T01:59:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-09-16T01:59:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-09-16T01:59:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 4, 2024-09-16T01:59:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 234, 2024-09-16T01:59:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 6, 2024-09-16T01:59:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T01:59:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-09-16T01:59:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T01:59:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-09-16T01:59:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-09-16T01:59:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T01:59:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T01:59:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-09-16T01:59:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 0, 2024-09-16T01:59:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T01:59:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 0, 2024-09-16T01:59:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T01:59:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T01:59:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T01:59:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 0, 2024-09-16T01:59:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 0, 2024-09-16T01:59:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 0, 2024-09-16T01:59:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 0 2024-09-16T01:59:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-09-16T01:59:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 0, 2024-09-16T01:59:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 0, 2024-09-16T01:59:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 0, 2024-09-16T01:59:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 0, 2024-09-16T01:59:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 0, 2024-09-16T01:59:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 0, 2024-09-16T01:59:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-09-16T01:59:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-09-16T01:59:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 0, 2024-09-16T01:59:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 0, 2024-09-16T01:59:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-09-16T01:59:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-09-16T01:59:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-09-16T01:59:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-09-16T01:59:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-09-16T01:59:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-09-16T01:59:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-09-16T01:59:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 0, 2024-09-16T01:59:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 0, 2024-09-16T01:59:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T01:59:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 0, 2024-09-16T01:59:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-09-16T01:59:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 0, 2024-09-16T01:59:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-09-16T01:59:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-09-16T01:59:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-09-16T01:59:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-09-16T01:59:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 0, 2024-09-16T01:59:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 0, 2024-09-16T01:59:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 0, 2024-09-16T01:59:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 283, 2024-09-16T01:59:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-09-16T01:59:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 0, 2024-09-16T01:59:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 0, 2024-09-16T01:59:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 0, 2024-09-16T01:59:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 0, 2024-09-16T01:59:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 2, 2024-09-16T01:59:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 25, 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 0, 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 363483155, 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 363898235, 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 363894072, 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 283 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 47, 2024-09-16T01:59:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 11553, 2024-09-16T01:59:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 11506, 2024-09-16T01:59:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 24, 2024-09-16T01:59:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 492, 2024-09-16T01:59:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 468, 2024-09-16T01:59:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 704, 2024-09-16T01:59:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 20400, 2024-09-16T01:59:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 19696, 2024-09-16T01:59:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 0, 2024-09-16T01:59:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 0, 2024-09-16T01:59:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 0, 2024-09-16T01:59:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 60184, 2024-09-16T01:59:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 215328 2024-09-16T01:59:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-09-16T01:59:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 0, 2024-09-16T01:59:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 0, 2024-09-16T01:59:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 0, 2024-09-16T01:59:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 0, 2024-09-16T01:59:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 0, 2024-09-16T01:59:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-09-16T01:59:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-09-16T01:59:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-09-16T01:59:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-09-16T01:59:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-09-16T01:59:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-09-16T01:59:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-09-16T01:59:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-09-16T01:59:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-09-16T01:59:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-09-16T01:59:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-09-16T01:59:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-09-16T01:59:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-09-16T01:59:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-09-16T01:59:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-09-16T01:59:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-09-16T01:59:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-09-16T01:59:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-09-16T01:59:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-09-16T01:59:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-09-16T01:59:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-09-16T01:59:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-09-16T01:59:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-09-16T01:59:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-09-16T01:59:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-09-16T01:59:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-09-16T01:59:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-09-16T01:59:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 2, 2024-09-16T01:59:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 2, 2024-09-16T01:59:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 0, 2024-09-16T01:59:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 0, 2024-09-16T01:59:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-09-16T01:59:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 0, 2024-09-16T01:59:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 0, 2024-09-16T01:59:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 2148, 2024-09-16T01:59:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-09-16T01:59:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-09-16T01:59:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-09-16T01:59:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-09-16T01:59:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T01:59:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-09-16T01:59:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T01:59:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-09-16T01:59:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T01:59:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T01:59:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T01:59:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T01:59:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T01:59:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-09-16T01:59:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T01:59:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T01:59:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-09-16T01:59:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-09-16T01:59:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-09-16T01:59:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-09-16T01:59:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T01:59:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T01:59:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T01:59:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T01:59:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T01:59:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-09-16T01:59:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-09-16T01:59:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-09-16T01:59:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T01:59:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T01:59:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-09-16T01:59:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-09-16T01:59:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-09-16T01:59:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-09-16T01:59:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T01:59:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-09-16T01:59:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T01:59:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-09-16T01:59:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 208928, 2024-09-16T01:59:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 131, 2024-09-16T01:59:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-09-16T01:59:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-09-16T01:59:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-09-16T01:59:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-09-16T01:59:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-09-16T01:59:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-09-16T01:59:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-09-16T01:59:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-09-16T01:59:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 10160, 2024-09-16T01:59:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 216, 2024-09-16T01:59:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T01:59:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-09-16T01:59:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-09-16T01:59:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-09-16T01:59:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 683712, 2024-09-16T01:59:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 4315, 2024-09-16T01:59:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-09-16T01:59:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-09-16T01:59:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-09-16T01:59:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 1107, 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 26, 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 4, 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 1107, 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1107, 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 4.833901938, 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.004366668 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-09-16T01:59:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1107, 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 1108 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 1107, 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 1107, 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 0, 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 664, 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 0, 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 440, 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 0, 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 0, 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-09-16T01:59:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-09-16T01:59:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-09-16T01:59:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 0, 2024-09-16T01:59:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 0, 2024-09-16T01:59:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-09-16T01:59:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-09-16T01:59:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-09-16T01:59:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 2, 2024-09-16T01:59:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 0, 2024-09-16T01:59:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-09-16T01:59:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-09-16T01:59:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-09-16T01:59:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-09-16T01:59:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-09-16T01:59:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-09-16T01:59:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T01:59:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-09-16T01:59:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-09-16T01:59:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-09-16T01:59:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-09-16T01:59:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-09-16T01:59:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-09-16T01:59:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-09-16T01:59:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-09-16T01:59:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-09-16T01:59:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-09-16T01:59:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-09-16T01:59:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-09-16T01:59:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 118, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 14, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 7, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 7, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 0, 2024-09-16T01:59:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 2, 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 0 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 0, 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 0 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T01:59:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-09-16T01:59:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 0, 2024-09-16T01:59:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 0, 2024-09-16T01:59:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 0, 2024-09-16T01:59:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-09-16T01:59:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T01:59:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-16T01:59:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 1496, 2024-09-16T01:59:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 13270932, 2024-09-16T01:59:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 1496, 2024-09-16T01:59:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-16T01:59:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-16T01:59:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 1496, 2024-09-16T01:59:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 13270932, 2024-09-16T01:59:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-09-16T01:59:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-16T01:59:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-16T01:59:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-16T01:59:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 1107, 2024-09-16T01:59:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 11527763, 2024-09-16T01:59:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 1107, 2024-09-16T01:59:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 11527763, 2024-09-16T01:59:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 1107, 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 1107, 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 1107, 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 1107, 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-09-16T01:59:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-09-16T01:59:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-16T01:59:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-16T01:59:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-16T01:59:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-16T01:59:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-09-16T01:59:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-09-16T01:59:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x55dde0650b40": { 2024-09-16T01:59:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-09-16T01:59:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-16T01:59:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-16T01:59:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-16T01:59:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-16T01:59:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-09-16T01:59:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-09-16T01:59:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:08.477 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:59:09.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:09 smithi139 ceph-mon[31175]: pgmap v597: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 54 KiB/s wr, 168 op/s 2024-09-16T01:59:09.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:09 smithi164 ceph-mon[35121]: pgmap v597: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 54 KiB/s wr, 168 op/s 2024-09-16T01:59:09.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:09 smithi176 ceph-mon[35324]: pgmap v597: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 1.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 54 KiB/s wr, 168 op/s 2024-09-16T01:59:10.224 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-09-16T01:59:10.224 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-09-16T01:59:10.224 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2024-09-16T01:59:10.224 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2024-09-16T01:59:10.225 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=6 2024-09-16T01:59:10.225 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-09-16T01:59:10.225 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T01:59:10.225 INFO:tasks.check_counter:Daemon mds.d mds.dir_split=0 2024-09-16T01:59:10.225 DEBUG:tasks.check_counter:Getting stats from g 2024-09-16T01:59:10.225 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-16T01:59:10.718 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:11.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:11 smithi164 ceph-mon[35121]: pgmap v598: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 50 KiB/s wr, 157 op/s 2024-09-16T01:59:11.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:11 smithi139 ceph-mon[31175]: pgmap v598: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 50 KiB/s wr, 157 op/s 2024-09-16T01:59:11.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:11 smithi176 ceph-mon[35324]: pgmap v598: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 50 KiB/s wr, 157 op/s 2024-09-16T01:59:12.545 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:59:12.545 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T01:59:12.546 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 1622, 2024-09-16T01:59:12.546 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 1623, 2024-09-16T01:59:12.546 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 84418888, 2024-09-16T01:59:12.546 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 515479, 2024-09-16T01:59:12.546 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 8, 2024-09-16T01:59:12.546 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 4, 2024-09-16T01:59:12.546 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.249688787, 2024-09-16T01:59:12.546 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.060456275, 2024-09-16T01:59:12.546 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.094242833, 2024-09-16T01:59:12.546 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.080241610, 2024-09-16T01:59:12.546 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:12.546 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1621, 2024-09-16T01:59:12.546 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.131841224, 2024-09-16T01:59:12.547 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000081333 2024-09-16T01:59:12.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.547 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:12.547 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.547 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.547 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.547 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T01:59:12.547 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T01:59:12.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.547 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T01:59:12.547 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 1789, 2024-09-16T01:59:12.547 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 2047, 2024-09-16T01:59:12.547 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 74004114, 2024-09-16T01:59:12.548 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 1185323, 2024-09-16T01:59:12.548 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 9, 2024-09-16T01:59:12.548 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 7, 2024-09-16T01:59:12.548 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.250675843, 2024-09-16T01:59:12.548 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.047669045, 2024-09-16T01:59:12.548 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.099222729, 2024-09-16T01:59:12.548 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.092765966, 2024-09-16T01:59:12.548 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:12.548 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2045, 2024-09-16T01:59:12.548 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.134030438, 2024-09-16T01:59:12.548 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000065540 2024-09-16T01:59:12.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.549 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:12.549 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.549 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.549 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.549 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-16T01:59:12.549 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 618272 2024-09-16T01:59:12.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.549 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T01:59:12.549 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 889, 2024-09-16T01:59:12.549 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 831, 2024-09-16T01:59:12.549 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 73276105, 2024-09-16T01:59:12.550 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 361405, 2024-09-16T01:59:12.550 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 5, 2024-09-16T01:59:12.550 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 4, 2024-09-16T01:59:12.550 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.143782286, 2024-09-16T01:59:12.550 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.035246003, 2024-09-16T01:59:12.550 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.107589098, 2024-09-16T01:59:12.550 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.012409913, 2024-09-16T01:59:12.550 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:12.550 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 830, 2024-09-16T01:59:12.550 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.064569019, 2024-09-16T01:59:12.550 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000077793 2024-09-16T01:59:12.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.551 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:12.551 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.551 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.551 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.551 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 1856096, 2024-09-16T01:59:12.551 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 219248 2024-09-16T01:59:12.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.551 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-09-16T01:59:12.551 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-09-16T01:59:12.552 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-09-16T01:59:12.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.552 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-09-16T01:59:12.552 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-09-16T01:59:12.552 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-09-16T01:59:12.552 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 6375, 2024-09-16T01:59:12.552 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.163617596, 2024-09-16T01:59:12.552 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000025665 2024-09-16T01:59:12.552 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:12.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.552 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-09-16T01:59:12.553 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-09-16T01:59:12.553 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-09-16T01:59:12.553 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.553 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.553 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.553 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:12.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.553 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-09-16T01:59:12.553 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 0, 2024-09-16T01:59:12.553 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 0, 2024-09-16T01:59:12.553 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-09-16T01:59:12.553 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.553 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.553 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.554 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-09-16T01:59:12.554 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 0, 2024-09-16T01:59:12.554 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-09-16T01:59:12.554 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-09-16T01:59:12.554 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 0, 2024-09-16T01:59:12.554 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 0, 2024-09-16T01:59:12.554 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-09-16T01:59:12.554 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 33842, 2024-09-16T01:59:12.554 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 196, 2024-09-16T01:59:12.554 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 33646, 2024-09-16T01:59:12.554 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 0, 2024-09-16T01:59:12.554 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 33561, 2024-09-16T01:59:12.555 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 0, 2024-09-16T01:59:12.555 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 0, 2024-09-16T01:59:12.555 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 0, 2024-09-16T01:59:12.555 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 1, 2024-09-16T01:59:12.555 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 0, 2024-09-16T01:59:12.555 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 0, 2024-09-16T01:59:12.555 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 0, 2024-09-16T01:59:12.555 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 0, 2024-09-16T01:59:12.555 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-09-16T01:59:12.555 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-09-16T01:59:12.555 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 0, 2024-09-16T01:59:12.556 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 0, 2024-09-16T01:59:12.556 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-09-16T01:59:12.556 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 0, 2024-09-16T01:59:12.556 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 0, 2024-09-16T01:59:12.556 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 0, 2024-09-16T01:59:12.556 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 0, 2024-09-16T01:59:12.556 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-09-16T01:59:12.556 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-09-16T01:59:12.556 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-09-16T01:59:12.556 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 0, 2024-09-16T01:59:12.556 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 0, 2024-09-16T01:59:12.556 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 0, 2024-09-16T01:59:12.556 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T01:59:12.557 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-09-16T01:59:12.557 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T01:59:12.557 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-09-16T01:59:12.557 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-09-16T01:59:12.557 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T01:59:12.557 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T01:59:12.557 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-09-16T01:59:12.557 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 0, 2024-09-16T01:59:12.557 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T01:59:12.557 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 0, 2024-09-16T01:59:12.557 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T01:59:12.557 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T01:59:12.557 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T01:59:12.557 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 0, 2024-09-16T01:59:12.558 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 0, 2024-09-16T01:59:12.558 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 0, 2024-09-16T01:59:12.558 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 0 2024-09-16T01:59:12.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.558 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-09-16T01:59:12.558 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 0, 2024-09-16T01:59:12.558 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 0, 2024-09-16T01:59:12.558 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 0, 2024-09-16T01:59:12.558 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 0, 2024-09-16T01:59:12.558 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 0, 2024-09-16T01:59:12.558 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 0, 2024-09-16T01:59:12.558 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-09-16T01:59:12.558 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-09-16T01:59:12.559 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 0, 2024-09-16T01:59:12.559 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 0, 2024-09-16T01:59:12.559 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-09-16T01:59:12.559 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-09-16T01:59:12.559 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-09-16T01:59:12.559 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-09-16T01:59:12.559 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-09-16T01:59:12.559 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-09-16T01:59:12.559 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-09-16T01:59:12.559 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 0, 2024-09-16T01:59:12.559 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 0, 2024-09-16T01:59:12.559 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T01:59:12.559 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 0, 2024-09-16T01:59:12.559 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-09-16T01:59:12.560 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 0, 2024-09-16T01:59:12.560 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-09-16T01:59:12.560 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-09-16T01:59:12.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.560 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-09-16T01:59:12.560 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-09-16T01:59:12.560 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 0, 2024-09-16T01:59:12.560 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 0, 2024-09-16T01:59:12.560 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 0, 2024-09-16T01:59:12.560 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 83473, 2024-09-16T01:59:12.560 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-09-16T01:59:12.560 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 0, 2024-09-16T01:59:12.560 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 0, 2024-09-16T01:59:12.560 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 0, 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 0, 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 110, 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 8, 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 0, 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 4194304, 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 232914524, 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 232903948, 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 83473 2024-09-16T01:59:12.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.562 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-09-16T01:59:12.562 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 10752, 2024-09-16T01:59:12.562 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 33751, 2024-09-16T01:59:12.562 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 22999, 2024-09-16T01:59:12.562 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 398, 2024-09-16T01:59:12.562 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 999, 2024-09-16T01:59:12.562 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 601, 2024-09-16T01:59:12.562 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 33842, 2024-09-16T01:59:12.562 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 55303, 2024-09-16T01:59:12.562 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 21461, 2024-09-16T01:59:12.562 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 0, 2024-09-16T01:59:12.562 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 0, 2024-09-16T01:59:12.562 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 0, 2024-09-16T01:59:12.562 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 135952, 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 215328 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 0, 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 0, 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 0, 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 0, 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 0, 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-09-16T01:59:12.563 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.564 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.564 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.564 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-09-16T01:59:12.564 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.564 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.564 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.564 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-09-16T01:59:12.564 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.565 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.565 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.565 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-09-16T01:59:12.565 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.565 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.565 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.565 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-09-16T01:59:12.566 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.566 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.566 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.566 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-09-16T01:59:12.566 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.566 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.566 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.566 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-09-16T01:59:12.566 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.566 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.567 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.567 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-09-16T01:59:12.567 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.567 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.567 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.567 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-09-16T01:59:12.567 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.567 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.567 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.567 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-09-16T01:59:12.567 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.568 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.568 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.568 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-09-16T01:59:12.569 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.569 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.569 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.569 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-09-16T01:59:12.569 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.569 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.569 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.569 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-09-16T01:59:12.573 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.573 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.573 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.573 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.573 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-09-16T01:59:12.574 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.574 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.574 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.574 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-09-16T01:59:12.574 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.574 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.574 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.574 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-09-16T01:59:12.574 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.575 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.575 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.575 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-09-16T01:59:12.575 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.575 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.575 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.575 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-09-16T01:59:12.575 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.576 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.576 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.576 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-09-16T01:59:12.576 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.576 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.576 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.576 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-09-16T01:59:12.576 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.576 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.577 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.577 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-09-16T01:59:12.577 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.577 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.577 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.577 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-09-16T01:59:12.577 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.577 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.577 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.577 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-09-16T01:59:12.577 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-09-16T01:59:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-09-16T01:59:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-09-16T01:59:12.579 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.579 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.579 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.579 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-09-16T01:59:12.579 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.579 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.579 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.579 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-09-16T01:59:12.579 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.579 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.579 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.580 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-09-16T01:59:12.580 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-09-16T01:59:12.580 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-09-16T01:59:12.580 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.580 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.580 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.580 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:12.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.580 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-09-16T01:59:12.580 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 1, 2024-09-16T01:59:12.580 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 2, 2024-09-16T01:59:12.580 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 1, 2024-09-16T01:59:12.580 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 1, 2024-09-16T01:59:12.581 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-09-16T01:59:12.581 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 0, 2024-09-16T01:59:12.581 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 0, 2024-09-16T01:59:12.581 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 2149, 2024-09-16T01:59:12.581 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-09-16T01:59:12.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.581 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-09-16T01:59:12.581 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-09-16T01:59:12.581 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-09-16T01:59:12.581 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T01:59:12.581 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-09-16T01:59:12.581 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T01:59:12.581 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-09-16T01:59:12.581 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T01:59:12.582 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T01:59:12.582 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T01:59:12.582 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T01:59:12.582 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T01:59:12.582 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-09-16T01:59:12.582 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T01:59:12.582 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T01:59:12.582 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-09-16T01:59:12.582 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-09-16T01:59:12.582 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-09-16T01:59:12.582 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-09-16T01:59:12.582 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T01:59:12.582 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T01:59:12.582 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T01:59:12.583 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T01:59:12.583 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T01:59:12.583 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-09-16T01:59:12.583 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-09-16T01:59:12.583 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-09-16T01:59:12.583 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T01:59:12.583 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T01:59:12.583 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-09-16T01:59:12.583 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-09-16T01:59:12.583 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-09-16T01:59:12.583 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-09-16T01:59:12.583 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T01:59:12.583 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-09-16T01:59:12.583 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T01:59:12.584 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-09-16T01:59:12.584 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 642233, 2024-09-16T01:59:12.584 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 10985, 2024-09-16T01:59:12.584 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-09-16T01:59:12.584 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-09-16T01:59:12.584 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-09-16T01:59:12.584 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-09-16T01:59:12.584 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-09-16T01:59:12.584 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-09-16T01:59:12.585 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-09-16T01:59:12.585 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-09-16T01:59:12.585 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 10160, 2024-09-16T01:59:12.585 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 216, 2024-09-16T01:59:12.585 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T01:59:12.585 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-09-16T01:59:12.585 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-09-16T01:59:12.585 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-09-16T01:59:12.585 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 49686918, 2024-09-16T01:59:12.585 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 518731, 2024-09-16T01:59:12.585 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-09-16T01:59:12.585 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-09-16T01:59:12.585 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-09-16T01:59:12.586 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-09-16T01:59:12.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.586 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-09-16T01:59:12.586 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-09-16T01:59:12.586 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-09-16T01:59:12.586 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 3851, 2024-09-16T01:59:12.586 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 21, 2024-09-16T01:59:12.586 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 3, 2024-09-16T01:59:12.586 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 3851, 2024-09-16T01:59:12.586 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-09-16T01:59:12.586 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 3851, 2024-09-16T01:59:12.586 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 67.621596373, 2024-09-16T01:59:12.586 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.017559490 2024-09-16T01:59:12.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 3851, 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 3852 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 3851, 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 3851, 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 0, 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 2367, 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 0, 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 1482, 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 0, 2024-09-16T01:59:12.587 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 0, 2024-09-16T01:59:12.588 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-09-16T01:59:12.588 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-09-16T01:59:12.588 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-09-16T01:59:12.588 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 0, 2024-09-16T01:59:12.588 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 0, 2024-09-16T01:59:12.588 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-09-16T01:59:12.588 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-09-16T01:59:12.588 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-09-16T01:59:12.588 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 1, 2024-09-16T01:59:12.588 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 0, 2024-09-16T01:59:12.588 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-09-16T01:59:12.588 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-09-16T01:59:12.588 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-09-16T01:59:12.588 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-09-16T01:59:12.589 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-09-16T01:59:12.589 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-09-16T01:59:12.589 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T01:59:12.589 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-09-16T01:59:12.589 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-09-16T01:59:12.589 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-09-16T01:59:12.589 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-09-16T01:59:12.589 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-09-16T01:59:12.589 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-09-16T01:59:12.589 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-09-16T01:59:12.589 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-09-16T01:59:12.589 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-09-16T01:59:12.589 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-09-16T01:59:12.589 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-09-16T01:59:12.590 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-09-16T01:59:12.590 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-09-16T01:59:12.590 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-09-16T01:59:12.590 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-09-16T01:59:12.590 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-09-16T01:59:12.590 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-09-16T01:59:12.590 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-09-16T01:59:12.590 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-09-16T01:59:12.590 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 118, 2024-09-16T01:59:12.590 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-09-16T01:59:12.590 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 35, 2024-09-16T01:59:12.590 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 11, 2024-09-16T01:59:12.590 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 11, 2024-09-16T01:59:12.590 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-09-16T01:59:12.591 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-09-16T01:59:12.591 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 0, 2024-09-16T01:59:12.591 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 2, 2024-09-16T01:59:12.591 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 0 2024-09-16T01:59:12.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.591 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-09-16T01:59:12.591 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 0, 2024-09-16T01:59:12.591 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-09-16T01:59:12.591 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-09-16T01:59:12.591 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-09-16T01:59:12.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.591 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-09-16T01:59:12.591 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 0 2024-09-16T01:59:12.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.592 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-09-16T01:59:12.592 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-09-16T01:59:12.592 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T01:59:12.592 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-09-16T01:59:12.592 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 0, 2024-09-16T01:59:12.592 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 0, 2024-09-16T01:59:12.592 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 0, 2024-09-16T01:59:12.592 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-09-16T01:59:12.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.592 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T01:59:12.592 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:12.592 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-16T01:59:12.592 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:12.592 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 4296, 2024-09-16T01:59:12.593 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 231287051, 2024-09-16T01:59:12.593 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:12.593 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 4296, 2024-09-16T01:59:12.593 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-16T01:59:12.593 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-16T01:59:12.593 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 4296, 2024-09-16T01:59:12.593 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 231287051, 2024-09-16T01:59:12.593 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:12.593 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.593 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.593 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.593 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:12.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.593 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-09-16T01:59:12.594 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:12.594 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-16T01:59:12.594 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:12.594 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-16T01:59:12.594 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-16T01:59:12.594 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:12.594 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:12.594 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 3851, 2024-09-16T01:59:12.594 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 228763875, 2024-09-16T01:59:12.594 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 3851, 2024-09-16T01:59:12.595 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 228763875, 2024-09-16T01:59:12.595 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:12.595 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.595 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.595 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.595 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:12.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.595 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-09-16T01:59:12.595 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:12.595 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-09-16T01:59:12.595 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:12.595 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-16T01:59:12.595 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-16T01:59:12.595 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:12.596 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:12.596 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 3851, 2024-09-16T01:59:12.596 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 3851, 2024-09-16T01:59:12.596 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 3851, 2024-09-16T01:59:12.596 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 3851, 2024-09-16T01:59:12.596 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:12.596 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.596 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.596 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.596 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:12.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.597 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-09-16T01:59:12.597 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:12.597 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-09-16T01:59:12.597 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:12.597 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-16T01:59:12.597 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-16T01:59:12.597 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:12.597 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:12.597 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-16T01:59:12.597 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-16T01:59:12.597 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-09-16T01:59:12.597 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-09-16T01:59:12.597 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:12.597 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x5613a257cb40": { 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-16T01:59:12.598 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-09-16T01:59:12.599 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-09-16T01:59:12.599 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:12.599 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:12.599 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:12.599 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:12.599 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:12.599 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:12.599 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:59:13.308 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:13 smithi139 ceph-mon[31175]: pgmap v599: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 50 KiB/s wr, 136 op/s 2024-09-16T01:59:13.365 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-16T01:59:13.365 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-16T01:59:13.365 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-16T01:59:13.365 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-16T01:59:13.365 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-16T01:59:13.365 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-16T01:59:13.365 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T01:59:13.365 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-09-16T01:59:13.365 DEBUG:tasks.check_counter:Getting stats from j 2024-09-16T01:59:13.366 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-16T01:59:13.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:13 smithi164 ceph-mon[35121]: pgmap v599: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 50 KiB/s wr, 136 op/s 2024-09-16T01:59:13.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:13 smithi176 ceph-mon[35324]: pgmap v599: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 50 KiB/s wr, 136 op/s 2024-09-16T01:59:13.921 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:14.334 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:14 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:59:14.334 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:14 smithi139 ceph-mon[31175]: osdmap e119: 12 total, 12 up, 12 in 2024-09-16T01:59:14.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:14 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:59:14.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:14 smithi164 ceph-mon[35121]: osdmap e119: 12 total, 12 up, 12 in 2024-09-16T01:59:14.636 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T01:59:14.637 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T01:59:14.637 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 0, 2024-09-16T01:59:14.637 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 2, 2024-09-16T01:59:14.637 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 0, 2024-09-16T01:59:14.637 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 0, 2024-09-16T01:59:14.637 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 4, 2024-09-16T01:59:14.638 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 0, 2024-09-16T01:59:14.638 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.001053371, 2024-09-16T01:59:14.638 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T01:59:14.638 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.000191536, 2024-09-16T01:59:14.638 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T01:59:14.638 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:14.639 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:14.639 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:14.639 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:14.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:14.639 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:14.639 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:14.640 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:14.640 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:14.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:14.640 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T01:59:14.640 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T01:59:14.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:14.640 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T01:59:14.641 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 359, 2024-09-16T01:59:14.641 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 349, 2024-09-16T01:59:14.641 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 1487744, 2024-09-16T01:59:14.641 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 215456, 2024-09-16T01:59:14.641 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 2, 2024-09-16T01:59:14.641 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 1, 2024-09-16T01:59:14.641 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.059683489, 2024-09-16T01:59:14.641 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.023221519, 2024-09-16T01:59:14.641 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.050289307, 2024-09-16T01:59:14.641 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T01:59:14.641 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:14.642 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 348, 2024-09-16T01:59:14.642 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.015787336, 2024-09-16T01:59:14.642 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000045365 2024-09-16T01:59:14.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:14.642 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:14.642 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:14.642 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:14.642 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:14.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 1487744, 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 215456 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 7, 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 269, 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 3504, 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 283552, 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 6, 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 3, 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.022209822, 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.015051167, 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.004456152, 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T01:59:14.643 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:14.644 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 267, 2024-09-16T01:59:14.644 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.029587428, 2024-09-16T01:59:14.644 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000110814 2024-09-16T01:59:14.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:14.644 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:14.644 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:14.644 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:14.645 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:14.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:14.645 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-09-16T01:59:14.645 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 283552 2024-09-16T01:59:14.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:14.645 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-09-16T01:59:14.645 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 0, 2024-09-16T01:59:14.645 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-09-16T01:59:14.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:14.645 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-09-16T01:59:14.645 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-09-16T01:59:14.645 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-09-16T01:59:14.645 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T01:59:14.645 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-09-16T01:59:14.646 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T01:59:14.646 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-09-16T01:59:14.646 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T01:59:14.646 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T01:59:14.646 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T01:59:14.646 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T01:59:14.646 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T01:59:14.647 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-09-16T01:59:14.647 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T01:59:14.647 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T01:59:14.647 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-09-16T01:59:14.647 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-09-16T01:59:14.647 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-09-16T01:59:14.647 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-09-16T01:59:14.647 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T01:59:14.647 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T01:59:14.647 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T01:59:14.647 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T01:59:14.647 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T01:59:14.647 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-09-16T01:59:14.647 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-09-16T01:59:14.648 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-09-16T01:59:14.648 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T01:59:14.648 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T01:59:14.648 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-09-16T01:59:14.648 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-09-16T01:59:14.648 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-09-16T01:59:14.648 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-09-16T01:59:14.648 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T01:59:14.648 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-09-16T01:59:14.649 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T01:59:14.649 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-09-16T01:59:14.649 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 148692, 2024-09-16T01:59:14.649 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 30, 2024-09-16T01:59:14.649 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-09-16T01:59:14.649 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-09-16T01:59:14.649 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-09-16T01:59:14.649 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-09-16T01:59:14.649 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-09-16T01:59:14.649 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-09-16T01:59:14.649 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-09-16T01:59:14.649 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-09-16T01:59:14.650 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 0, 2024-09-16T01:59:14.650 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 0, 2024-09-16T01:59:14.650 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T01:59:14.650 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-09-16T01:59:14.650 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-09-16T01:59:14.650 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-09-16T01:59:14.650 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 928, 2024-09-16T01:59:14.650 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 2, 2024-09-16T01:59:14.650 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-09-16T01:59:14.651 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-09-16T01:59:14.651 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-09-16T01:59:14.651 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-09-16T01:59:14.651 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T01:59:14.651 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T01:59:14.651 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-09-16T01:59:14.651 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-09-16T01:59:14.651 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-09-16T01:59:14.651 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 362, 2024-09-16T01:59:14.651 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 1437050, 2024-09-16T01:59:14.651 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:14.651 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 362, 2024-09-16T01:59:14.652 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-09-16T01:59:14.652 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-09-16T01:59:14.652 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 362, 2024-09-16T01:59:14.652 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 1437050, 2024-09-16T01:59:14.652 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-09-16T01:59:14.652 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-09-16T01:59:14.652 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-09-16T01:59:14.652 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-09-16T01:59:14.652 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:14.652 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T01:59:14.653 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T01:59:14.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:14 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:59:14.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:14 smithi176 ceph-mon[35324]: osdmap e119: 12 total, 12 up, 12 in 2024-09-16T01:59:15.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:15 smithi139 ceph-mon[31175]: pgmap v601: 129 pgs: 17 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 57 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 98 op/s 2024-09-16T01:59:15.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:15 smithi164 ceph-mon[35121]: pgmap v601: 129 pgs: 17 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 57 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 98 op/s 2024-09-16T01:59:15.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:15 smithi176 ceph-mon[35324]: pgmap v601: 129 pgs: 17 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 57 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 98 op/s 2024-09-16T01:59:16.512 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-16T01:59:16.512 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-16T01:59:16.513 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-16T01:59:16.513 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-16T01:59:16.513 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-16T01:59:16.513 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-16T01:59:16.513 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-16T01:59:16.513 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-16T01:59:16.513 DEBUG:tasks.check_counter:Getting stats from b 2024-09-16T01:59:16.513 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-16T01:59:17.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:17 smithi164 ceph-mon[35121]: pgmap v602: 129 pgs: 23 active+clean+snaptrim, 68 active+clean+snaptrim_wait, 38 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1023 B/s wr, 55 op/s 2024-09-16T01:59:17.626 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:59:17.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:17 smithi139 ceph-mon[31175]: pgmap v602: 129 pgs: 23 active+clean+snaptrim, 68 active+clean+snaptrim_wait, 38 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1023 B/s wr, 55 op/s 2024-09-16T01:59:17.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:17 smithi176 ceph-mon[35324]: pgmap v602: 129 pgs: 23 active+clean+snaptrim, 68 active+clean+snaptrim_wait, 38 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1023 B/s wr, 55 op/s 2024-09-16T01:59:18.604 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T01:59:18.604 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T01:59:18.604 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 328685, 2024-09-16T01:59:18.604 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 459489, 2024-09-16T01:59:18.604 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 129485250, 2024-09-16T01:59:18.604 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 405024193, 2024-09-16T01:59:18.604 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 17, 2024-09-16T01:59:18.604 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 7, 2024-09-16T01:59:18.604 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 20.704185660, 2024-09-16T01:59:18.604 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 8.995467260, 2024-09-16T01:59:18.604 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 23.941892067, 2024-09-16T01:59:18.605 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 1.983550071, 2024-09-16T01:59:18.605 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:18.605 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 459481, 2024-09-16T01:59:18.605 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 111.471848784, 2024-09-16T01:59:18.605 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000242603 2024-09-16T01:59:18.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.605 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:18.605 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 611528, 2024-09-16T01:59:18.605 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.100198080, 2024-09-16T01:59:18.605 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000163 2024-09-16T01:59:18.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.605 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 97514432, 2024-09-16T01:59:18.605 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 221520704 2024-09-16T01:59:18.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.606 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T01:59:18.606 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 82318, 2024-09-16T01:59:18.606 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 81107, 2024-09-16T01:59:18.606 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 54946733, 2024-09-16T01:59:18.606 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 223821149, 2024-09-16T01:59:18.606 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 17, 2024-09-16T01:59:18.606 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 7, 2024-09-16T01:59:18.606 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 6.385898543, 2024-09-16T01:59:18.606 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 2.347349405, 2024-09-16T01:59:18.606 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 2.126994821, 2024-09-16T01:59:18.606 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 2.039479836, 2024-09-16T01:59:18.606 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:18.606 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 81110, 2024-09-16T01:59:18.607 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 68.758924423, 2024-09-16T01:59:18.607 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000847724 2024-09-16T01:59:18.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.607 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:18.607 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 62682, 2024-09-16T01:59:18.607 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.010759822, 2024-09-16T01:59:18.607 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000171 2024-09-16T01:59:18.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.607 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 2502688, 2024-09-16T01:59:18.607 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 431088 2024-09-16T01:59:18.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.607 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T01:59:18.607 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 91705, 2024-09-16T01:59:18.607 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 90078, 2024-09-16T01:59:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 64832636, 2024-09-16T01:59:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 177631368, 2024-09-16T01:59:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 232, 2024-09-16T01:59:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-09-16T01:59:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 5.717473555, 2024-09-16T01:59:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 1.996150509, 2024-09-16T01:59:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 3.361894403, 2024-09-16T01:59:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 1.198024302, 2024-09-16T01:59:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 90078, 2024-09-16T01:59:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 47.177804831, 2024-09-16T01:59:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000523743 2024-09-16T01:59:18.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 113472, 2024-09-16T01:59:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.018496764, 2024-09-16T01:59:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000163 2024-09-16T01:59:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 85232, 2024-09-16T01:59:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 139808 2024-09-16T01:59:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-09-16T01:59:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-09-16T01:59:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-09-16T01:59:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.609 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-09-16T01:59:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-09-16T01:59:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-09-16T01:59:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 14861, 2024-09-16T01:59:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 117.500226355, 2024-09-16T01:59:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.007906616 2024-09-16T01:59:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-09-16T01:59:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-09-16T01:59:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-09-16T01:59:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 15602, 2024-09-16T01:59:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 3.507490048, 2024-09-16T01:59:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000224810 2024-09-16T01:59:18.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:18.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.611 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-09-16T01:59:18.611 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 158885, 2024-09-16T01:59:18.611 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 139132, 2024-09-16T01:59:18.611 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-09-16T01:59:18.611 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 139132, 2024-09-16T01:59:18.611 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 2772.566005154, 2024-09-16T01:59:18.611 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.019927593 2024-09-16T01:59:18.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.611 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-09-16T01:59:18.611 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 19752, 2024-09-16T01:59:18.611 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 154, 2024-09-16T01:59:18.611 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-09-16T01:59:18.611 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 5917, 2024-09-16T01:59:18.612 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 109, 2024-09-16T01:59:18.612 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 3, 2024-09-16T01:59:18.612 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 2599, 2024-09-16T01:59:18.612 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 1740, 2024-09-16T01:59:18.612 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 833, 2024-09-16T01:59:18.612 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 26, 2024-09-16T01:59:18.612 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 110, 2024-09-16T01:59:18.612 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 160852, 2024-09-16T01:59:18.612 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 2, 2024-09-16T01:59:18.612 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 2, 2024-09-16T01:59:18.612 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 5, 2024-09-16T01:59:18.612 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 262430, 2024-09-16T01:59:18.612 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 204848, 2024-09-16T01:59:18.612 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 19275, 2024-09-16T01:59:18.613 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 1116, 2024-09-16T01:59:18.613 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 320, 2024-09-16T01:59:18.613 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-09-16T01:59:18.613 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 151, 2024-09-16T01:59:18.613 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 127, 2024-09-16T01:59:18.613 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-09-16T01:59:18.613 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 437, 2024-09-16T01:59:18.613 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 117803, 2024-09-16T01:59:18.613 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 219, 2024-09-16T01:59:18.613 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 102049, 2024-09-16T01:59:18.613 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-09-16T01:59:18.613 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-09-16T01:59:18.613 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-09-16T01:59:18.614 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 4, 2024-09-16T01:59:18.614 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 234, 2024-09-16T01:59:18.614 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 6, 2024-09-16T01:59:18.614 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T01:59:18.614 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-09-16T01:59:18.614 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T01:59:18.614 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-09-16T01:59:18.614 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-09-16T01:59:18.614 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T01:59:18.614 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T01:59:18.614 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-09-16T01:59:18.614 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 249, 2024-09-16T01:59:18.614 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 54653, 2024-09-16T01:59:18.614 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 51151, 2024-09-16T01:59:18.615 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 5, 2024-09-16T01:59:18.615 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T01:59:18.615 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T01:59:18.615 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 81661, 2024-09-16T01:59:18.615 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 25457, 2024-09-16T01:59:18.615 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 2942, 2024-09-16T01:59:18.615 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 35128 2024-09-16T01:59:18.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.615 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-09-16T01:59:18.615 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 3622, 2024-09-16T01:59:18.615 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 10226, 2024-09-16T01:59:18.615 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 5600, 2024-09-16T01:59:18.615 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 7033, 2024-09-16T01:59:18.615 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 8618, 2024-09-16T01:59:18.616 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 52, 2024-09-16T01:59:18.616 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-09-16T01:59:18.616 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-09-16T01:59:18.616 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 8038, 2024-09-16T01:59:18.616 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 7986, 2024-09-16T01:59:18.616 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-09-16T01:59:18.616 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-09-16T01:59:18.616 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-09-16T01:59:18.616 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-09-16T01:59:18.617 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-09-16T01:59:18.617 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-09-16T01:59:18.617 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-09-16T01:59:18.617 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 17, 2024-09-16T01:59:18.617 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 127874, 2024-09-16T01:59:18.617 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T01:59:18.617 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 437, 2024-09-16T01:59:18.617 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-09-16T01:59:18.617 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 114, 2024-09-16T01:59:18.617 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-09-16T01:59:18.617 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-09-16T01:59:18.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.617 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-09-16T01:59:18.617 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 6, 2024-09-16T01:59:18.618 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 114107, 2024-09-16T01:59:18.618 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 114022, 2024-09-16T01:59:18.618 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 114022, 2024-09-16T01:59:18.618 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 85, 2024-09-16T01:59:18.618 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-09-16T01:59:18.618 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 1, 2024-09-16T01:59:18.618 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 223, 2024-09-16T01:59:18.618 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 222, 2024-09-16T01:59:18.618 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 222, 2024-09-16T01:59:18.618 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 1, 2024-09-16T01:59:18.618 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 1, 2024-09-16T01:59:18.618 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-09-16T01:59:18.618 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 1, 2024-09-16T01:59:18.618 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 363483155, 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 363898515, 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 0, 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 10336, 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 3622.094829481, 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.350434871 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 0 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 105, 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 104877, 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 104772, 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 165, 2024-09-16T01:59:18.619 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 8362, 2024-09-16T01:59:18.620 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 8197, 2024-09-16T01:59:18.620 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 2599, 2024-09-16T01:59:18.620 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 177048, 2024-09-16T01:59:18.620 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 174449, 2024-09-16T01:59:18.620 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 2, 2024-09-16T01:59:18.620 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 44755, 2024-09-16T01:59:18.620 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 44753, 2024-09-16T01:59:18.620 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 171140, 2024-09-16T01:59:18.620 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 231712 2024-09-16T01:59:18.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.620 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-09-16T01:59:18.620 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 165585, 2024-09-16T01:59:18.620 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 6541, 2024-09-16T01:59:18.621 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 158885, 2024-09-16T01:59:18.621 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 528, 2024-09-16T01:59:18.621 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 13642, 2024-09-16T01:59:18.621 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-09-16T01:59:18.621 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 25476, 2024-09-16T01:59:18.621 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 156.026546597, 2024-09-16T01:59:18.621 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.006124452 2024-09-16T01:59:18.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.621 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-09-16T01:59:18.621 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1753, 2024-09-16T01:59:18.621 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 155.478027228, 2024-09-16T01:59:18.621 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.088692542 2024-09-16T01:59:18.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.621 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 67592, 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1889.896842098, 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.027960362 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.622 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-09-16T01:59:18.623 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.623 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.623 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.623 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-09-16T01:59:18.623 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.623 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.623 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.623 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-09-16T01:59:18.623 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.623 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.623 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.624 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-09-16T01:59:18.624 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.624 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.624 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.624 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-09-16T01:59:18.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.625 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.625 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-09-16T01:59:18.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 19, 2024-09-16T01:59:18.625 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.009684912, 2024-09-16T01:59:18.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000509732 2024-09-16T01:59:18.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.625 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-09-16T01:59:18.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 222, 2024-09-16T01:59:18.625 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 44.659058584, 2024-09-16T01:59:18.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.201166930 2024-09-16T01:59:18.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.626 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-09-16T01:59:18.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.626 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.626 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-09-16T01:59:18.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 19, 2024-09-16T01:59:18.626 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 12.971617821, 2024-09-16T01:59:18.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.682716727 2024-09-16T01:59:18.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.626 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-09-16T01:59:18.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 12194, 2024-09-16T01:59:18.626 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 84.446425266, 2024-09-16T01:59:18.627 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.006925243 2024-09-16T01:59:18.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.627 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-09-16T01:59:18.627 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 368, 2024-09-16T01:59:18.627 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 8.022534682, 2024-09-16T01:59:18.627 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.021800365 2024-09-16T01:59:18.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.627 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-09-16T01:59:18.627 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 25109, 2024-09-16T01:59:18.627 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 325.383601488, 2024-09-16T01:59:18.627 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.012958843 2024-09-16T01:59:18.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.627 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-09-16T01:59:18.627 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.628 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.628 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.628 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-09-16T01:59:18.628 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.628 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.628 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.628 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-09-16T01:59:18.628 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 56, 2024-09-16T01:59:18.628 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 10.828085193, 2024-09-16T01:59:18.628 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.193358664 2024-09-16T01:59:18.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.628 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-09-16T01:59:18.629 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 13, 2024-09-16T01:59:18.629 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 16.517155194, 2024-09-16T01:59:18.629 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 1.270550399 2024-09-16T01:59:18.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.629 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-09-16T01:59:18.629 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.629 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.629 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.629 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-09-16T01:59:18.629 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 7, 2024-09-16T01:59:18.629 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.032531816, 2024-09-16T01:59:18.629 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.004647402 2024-09-16T01:59:18.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.630 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-09-16T01:59:18.630 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.630 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.630 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.630 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-09-16T01:59:18.630 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.630 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.630 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.630 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-09-16T01:59:18.630 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.630 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.630 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.631 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-09-16T01:59:18.631 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 8, 2024-09-16T01:59:18.631 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.021025813, 2024-09-16T01:59:18.631 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.002628226 2024-09-16T01:59:18.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.631 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-09-16T01:59:18.631 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 3, 2024-09-16T01:59:18.631 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.001797229, 2024-09-16T01:59:18.631 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000599076 2024-09-16T01:59:18.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.631 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-09-16T01:59:18.631 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 6293, 2024-09-16T01:59:18.631 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 68.271071233, 2024-09-16T01:59:18.632 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.010848732 2024-09-16T01:59:18.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.632 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-09-16T01:59:18.632 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-09-16T01:59:18.632 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-09-16T01:59:18.632 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.632 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.632 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.632 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:18.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.632 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-09-16T01:59:18.632 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 2, 2024-09-16T01:59:18.632 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 6, 2024-09-16T01:59:18.632 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 4, 2024-09-16T01:59:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 2, 2024-09-16T01:59:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-09-16T01:59:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 179, 2024-09-16T01:59:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 89, 2024-09-16T01:59:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 1117, 2024-09-16T01:59:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-09-16T01:59:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-09-16T01:59:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 1560, 2024-09-16T01:59:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 1560, 2024-09-16T01:59:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T01:59:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-09-16T01:59:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T01:59:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T01:59:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T01:59:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T01:59:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T01:59:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-09-16T01:59:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-09-16T01:59:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-09-16T01:59:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T01:59:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T01:59:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-09-16T01:59:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-09-16T01:59:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-09-16T01:59:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-09-16T01:59:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T01:59:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-09-16T01:59:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T01:59:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-09-16T01:59:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 497501, 2024-09-16T01:59:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 211, 2024-09-16T01:59:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-09-16T01:59:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-09-16T01:59:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-09-16T01:59:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-09-16T01:59:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-09-16T01:59:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-09-16T01:59:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-09-16T01:59:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-09-16T01:59:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 10344, 2024-09-16T01:59:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 223, 2024-09-16T01:59:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T01:59:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-09-16T01:59:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-09-16T01:59:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-09-16T01:59:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 2651273, 2024-09-16T01:59:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 25248, 2024-09-16T01:59:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-09-16T01:59:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-09-16T01:59:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-09-16T01:59:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-09-16T01:59:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-09-16T01:59:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-09-16T01:59:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-09-16T01:59:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 120206, 2024-09-16T01:59:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 467350379, 2024-09-16T01:59:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 33, 2024-09-16T01:59:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 120206, 2024-09-16T01:59:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-09-16T01:59:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 120206, 2024-09-16T01:59:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 135971.627177234, 2024-09-16T01:59:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 1.131155076 2024-09-16T01:59:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-09-16T01:59:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-09-16T01:59:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 120206, 2024-09-16T01:59:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 261192 2024-09-16T01:59:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 120206, 2024-09-16T01:59:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 792, 2024-09-16T01:59:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 119414, 2024-09-16T01:59:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-09-16T01:59:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-09-16T01:59:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 4810, 2024-09-16T01:59:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 85098, 2024-09-16T01:59:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 275, 2024-09-16T01:59:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 10614, 2024-09-16T01:59:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 457, 2024-09-16T01:59:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-09-16T01:59:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-09-16T01:59:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-09-16T01:59:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 107, 2024-09-16T01:59:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 16663, 2024-09-16T01:59:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-09-16T01:59:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-09-16T01:59:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-09-16T01:59:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 8, 2024-09-16T01:59:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 128177, 2024-09-16T01:59:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-09-16T01:59:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-09-16T01:59:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-09-16T01:59:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-09-16T01:59:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-09-16T01:59:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-09-16T01:59:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T01:59:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-09-16T01:59:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-09-16T01:59:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 5, 2024-09-16T01:59:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-09-16T01:59:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-09-16T01:59:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-09-16T01:59:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-09-16T01:59:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-09-16T01:59:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-09-16T01:59:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-09-16T01:59:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-09-16T01:59:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-09-16T01:59:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-09-16T01:59:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-09-16T01:59:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-09-16T01:59:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-09-16T01:59:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-09-16T01:59:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-09-16T01:59:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-09-16T01:59:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 119, 2024-09-16T01:59:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-09-16T01:59:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 42, 2024-09-16T01:59:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 12, 2024-09-16T01:59:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 12, 2024-09-16T01:59:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-09-16T01:59:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-09-16T01:59:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 12033, 2024-09-16T01:59:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 671, 2024-09-16T01:59:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 2274 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 1, 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 119402 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 10, 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-09-16T01:59:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 6, 2024-09-16T01:59:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 15965, 2024-09-16T01:59:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 7986, 2024-09-16T01:59:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-09-16T01:59:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T01:59:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-09-16T01:59:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 502704, 2024-09-16T01:59:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 190811094, 2024-09-16T01:59:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 502704, 2024-09-16T01:59:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-16T01:59:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-16T01:59:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 502704, 2024-09-16T01:59:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 190811094, 2024-09-16T01:59:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-09-16T01:59:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-09-16T01:59:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-16T01:59:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-16T01:59:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 120206, 2024-09-16T01:59:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 468486509, 2024-09-16T01:59:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 120206, 2024-09-16T01:59:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 468486509, 2024-09-16T01:59:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-09-16T01:59:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-09-16T01:59:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-16T01:59:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-16T01:59:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 120206, 2024-09-16T01:59:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 120206, 2024-09-16T01:59:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 120206, 2024-09-16T01:59:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 120206, 2024-09-16T01:59:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-09-16T01:59:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-09-16T01:59:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 7986, 2024-09-16T01:59:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1150658, 2024-09-16T01:59:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 7986, 2024-09-16T01:59:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-16T01:59:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-16T01:59:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 276, 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1150658, 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x564373888960": { 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 114107, 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 359704211, 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 114107, 2024-09-16T01:59:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-16T01:59:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-16T01:59:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 10336, 2024-09-16T01:59:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 359704211, 2024-09-16T01:59:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:18.650 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T01:59:19.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:19 smithi164 ceph-mon[35121]: pgmap v603: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 21 op/s 2024-09-16T01:59:19.652 INFO:tasks.check_counter:Daemon mds.b mds.exported=437 2024-09-16T01:59:19.652 INFO:tasks.check_counter:Daemon mds.b mds.imported=219 2024-09-16T01:59:19.652 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=3622 2024-09-16T01:59:19.652 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=10226 2024-09-16T01:59:19.652 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-16T01:59:19.652 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=19 2024-09-16T01:59:19.652 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=13 2024-09-16T01:59:19.653 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=109 2024-09-16T01:59:19.653 DEBUG:tasks.check_counter:Getting stats from e 2024-09-16T01:59:19.653 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-16T01:59:19.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:19 smithi139 ceph-mon[31175]: pgmap v603: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 21 op/s 2024-09-16T01:59:19.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:19 smithi176 ceph-mon[35324]: pgmap v603: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 21 op/s 2024-09-16T01:59:20.163 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:59:20.806 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T01:59:20.806 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T01:59:20.806 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 0, 2024-09-16T01:59:20.806 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 2, 2024-09-16T01:59:20.806 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 0, 2024-09-16T01:59:20.806 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 0, 2024-09-16T01:59:20.806 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 4, 2024-09-16T01:59:20.806 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 0, 2024-09-16T01:59:20.806 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.001479680, 2024-09-16T01:59:20.806 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T01:59:20.806 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.000268253, 2024-09-16T01:59:20.807 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T01:59:20.807 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:20.807 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:20.807 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:20.807 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:20.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:20.807 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:20.807 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:20.808 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:20.808 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:20.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:20.808 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T01:59:20.808 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T01:59:20.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:20.808 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T01:59:20.808 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 97, 2024-09-16T01:59:20.808 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 96, 2024-09-16T01:59:20.808 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 168208, 2024-09-16T01:59:20.808 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 57952, 2024-09-16T01:59:20.808 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 2, 2024-09-16T01:59:20.809 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 1, 2024-09-16T01:59:20.809 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.013570965, 2024-09-16T01:59:20.809 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.005711755, 2024-09-16T01:59:20.809 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.009720668, 2024-09-16T01:59:20.809 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T01:59:20.809 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:20.809 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 95, 2024-09-16T01:59:20.809 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.004836998, 2024-09-16T01:59:20.809 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000050915 2024-09-16T01:59:20.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:20.809 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:20.809 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:20.809 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:20.810 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:20.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:20.810 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 168208, 2024-09-16T01:59:20.810 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 57952 2024-09-16T01:59:20.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:20.810 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T01:59:20.810 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 5, 2024-09-16T01:59:20.810 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 75, 2024-09-16T01:59:20.810 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 3184, 2024-09-16T01:59:20.810 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 91824, 2024-09-16T01:59:20.810 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 3, 2024-09-16T01:59:20.810 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2024-09-16T01:59:20.810 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.006756840, 2024-09-16T01:59:20.810 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.002696354, 2024-09-16T01:59:20.811 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.003431211, 2024-09-16T01:59:20.811 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T01:59:20.811 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:20.811 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 74, 2024-09-16T01:59:20.811 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.011691761, 2024-09-16T01:59:20.811 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000157996 2024-09-16T01:59:20.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:20.811 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:20.811 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:20.811 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:20.811 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:20.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:20.812 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-16T01:59:20.812 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 91824 2024-09-16T01:59:20.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:20.812 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-09-16T01:59:20.812 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 0, 2024-09-16T01:59:20.812 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-09-16T01:59:20.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:20.812 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-09-16T01:59:20.812 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-09-16T01:59:20.812 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-09-16T01:59:20.812 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T01:59:20.812 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-09-16T01:59:20.812 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T01:59:20.813 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-09-16T01:59:20.813 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T01:59:20.813 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T01:59:20.813 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T01:59:20.813 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T01:59:20.813 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T01:59:20.813 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-09-16T01:59:20.813 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T01:59:20.813 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T01:59:20.813 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-09-16T01:59:20.813 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-09-16T01:59:20.813 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-09-16T01:59:20.813 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-09-16T01:59:20.813 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T01:59:20.814 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T01:59:20.814 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T01:59:20.814 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T01:59:20.814 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T01:59:20.814 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-09-16T01:59:20.814 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-09-16T01:59:20.815 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-09-16T01:59:20.815 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T01:59:20.815 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T01:59:20.815 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-09-16T01:59:20.815 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-09-16T01:59:20.815 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-09-16T01:59:20.815 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-09-16T01:59:20.816 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T01:59:20.816 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-09-16T01:59:20.816 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T01:59:20.816 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-09-16T01:59:20.817 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 148769, 2024-09-16T01:59:20.817 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 26, 2024-09-16T01:59:20.817 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-09-16T01:59:20.817 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-09-16T01:59:20.817 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-09-16T01:59:20.817 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-09-16T01:59:20.817 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-09-16T01:59:20.817 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-09-16T01:59:20.818 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-09-16T01:59:20.818 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-09-16T01:59:20.818 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 0, 2024-09-16T01:59:20.818 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 0, 2024-09-16T01:59:20.818 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T01:59:20.818 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-09-16T01:59:20.818 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-09-16T01:59:20.818 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-09-16T01:59:20.819 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 928, 2024-09-16T01:59:20.819 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 2, 2024-09-16T01:59:20.819 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-09-16T01:59:20.819 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-09-16T01:59:20.819 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-09-16T01:59:20.819 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-09-16T01:59:20.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:20.819 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T01:59:20.820 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:20.820 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-09-16T01:59:20.820 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:20.820 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 98, 2024-09-16T01:59:20.820 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 154526, 2024-09-16T01:59:20.820 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:20.820 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 98, 2024-09-16T01:59:20.820 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-16T01:59:20.820 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-16T01:59:20.821 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 98, 2024-09-16T01:59:20.821 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 154526, 2024-09-16T01:59:20.821 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:20.821 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:20.821 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:20.821 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:20.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:20.822 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:20.822 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T01:59:21.502 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-09-16T01:59:21.502 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-09-16T01:59:21.502 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-09-16T01:59:21.502 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-09-16T01:59:21.502 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-09-16T01:59:21.503 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-09-16T01:59:21.503 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-09-16T01:59:21.503 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-09-16T01:59:21.503 DEBUG:tasks.check_counter:Getting stats from h 2024-09-16T01:59:21.503 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-16T01:59:21.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:21 smithi164 ceph-mon[35121]: pgmap v604: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-16T01:59:21.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:21 smithi139 ceph-mon[31175]: pgmap v604: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-16T01:59:21.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:21 smithi176 ceph-mon[35324]: pgmap v604: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-16T01:59:21.989 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:59:23.040 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T01:59:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T01:59:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 32942, 2024-09-16T01:59:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 33646, 2024-09-16T01:59:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 13001811, 2024-09-16T01:59:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 127740811, 2024-09-16T01:59:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 16, 2024-09-16T01:59:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 3, 2024-09-16T01:59:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 2.808487407, 2024-09-16T01:59:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.924284527, 2024-09-16T01:59:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 1.057239333, 2024-09-16T01:59:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.873503430, 2024-09-16T01:59:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 33644, 2024-09-16T01:59:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 4.635684244, 2024-09-16T01:59:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000137786 2024-09-16T01:59:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 35159, 2024-09-16T01:59:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.006036736, 2024-09-16T01:59:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000171 2024-09-16T01:59:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T01:59:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 496912 2024-09-16T01:59:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T01:59:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 281663, 2024-09-16T01:59:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 386399, 2024-09-16T01:59:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 111383203, 2024-09-16T01:59:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 380549166, 2024-09-16T01:59:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 13, 2024-09-16T01:59:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 8, 2024-09-16T01:59:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 17.981403247, 2024-09-16T01:59:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 7.886595262, 2024-09-16T01:59:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 18.489759238, 2024-09-16T01:59:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 1.088110495, 2024-09-16T01:59:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 386397, 2024-09-16T01:59:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 62.520895590, 2024-09-16T01:59:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000161804 2024-09-16T01:59:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 520538, 2024-09-16T01:59:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.107747192, 2024-09-16T01:59:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000206 2024-09-16T01:59:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 91876592, 2024-09-16T01:59:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 196437664 2024-09-16T01:59:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T01:59:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 61989, 2024-09-16T01:59:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 60141, 2024-09-16T01:59:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 50332130, 2024-09-16T01:59:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 100145042, 2024-09-16T01:59:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 10, 2024-09-16T01:59:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 7, 2024-09-16T01:59:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 4.476181650, 2024-09-16T01:59:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 1.974084478, 2024-09-16T01:59:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 2.384658212, 2024-09-16T01:59:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.578763276, 2024-09-16T01:59:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 60140, 2024-09-16T01:59:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 8.290762266, 2024-09-16T01:59:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000137857 2024-09-16T01:59:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 97876, 2024-09-16T01:59:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.015894275, 2024-09-16T01:59:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000162 2024-09-16T01:59:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1889424, 2024-09-16T01:59:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 229056 2024-09-16T01:59:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-09-16T01:59:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-09-16T01:59:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-09-16T01:59:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-09-16T01:59:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-09-16T01:59:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-09-16T01:59:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 9736, 2024-09-16T01:59:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 72.023261207, 2024-09-16T01:59:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.007397623 2024-09-16T01:59:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-09-16T01:59:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-09-16T01:59:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-09-16T01:59:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 31560, 2024-09-16T01:59:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 4.826336448, 2024-09-16T01:59:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000152925 2024-09-16T01:59:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-09-16T01:59:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 132833, 2024-09-16T01:59:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 116740, 2024-09-16T01:59:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-09-16T01:59:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 116740, 2024-09-16T01:59:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1485.915258179, 2024-09-16T01:59:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.012728415 2024-09-16T01:59:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-09-16T01:59:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 15987, 2024-09-16T01:59:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 453, 2024-09-16T01:59:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-09-16T01:59:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 810, 2024-09-16T01:59:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 85, 2024-09-16T01:59:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-09-16T01:59:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 35514, 2024-09-16T01:59:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 1, 2024-09-16T01:59:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 7, 2024-09-16T01:59:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 35506, 2024-09-16T01:59:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 35509, 2024-09-16T01:59:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 56996, 2024-09-16T01:59:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 1, 2024-09-16T01:59:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 1, 2024-09-16T01:59:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 4, 2024-09-16T01:59:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 220067, 2024-09-16T01:59:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 176102, 2024-09-16T01:59:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 15602, 2024-09-16T01:59:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 1089, 2024-09-16T01:59:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 77, 2024-09-16T01:59:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-09-16T01:59:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 311, 2024-09-16T01:59:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-09-16T01:59:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-09-16T01:59:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 98, 2024-09-16T01:59:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 43284, 2024-09-16T01:59:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 214, 2024-09-16T01:59:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 41781, 2024-09-16T01:59:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-09-16T01:59:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-09-16T01:59:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-09-16T01:59:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 60337, 2024-09-16T01:59:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 1323008661, 2024-09-16T01:59:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 6, 2024-09-16T01:59:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T01:59:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-09-16T01:59:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T01:59:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-09-16T01:59:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-09-16T01:59:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T01:59:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T01:59:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-09-16T01:59:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 318, 2024-09-16T01:59:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 66274, 2024-09-16T01:59:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 59211, 2024-09-16T01:59:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T01:59:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T01:59:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T01:59:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 83468, 2024-09-16T01:59:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 17064, 2024-09-16T01:59:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 2573, 2024-09-16T01:59:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 33822 2024-09-16T01:59:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-09-16T01:59:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 4911, 2024-09-16T01:59:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 11658, 2024-09-16T01:59:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 6565, 2024-09-16T01:59:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 8156, 2024-09-16T01:59:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 8439, 2024-09-16T01:59:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 135, 2024-09-16T01:59:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-09-16T01:59:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-09-16T01:59:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 16294, 2024-09-16T01:59:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 16160, 2024-09-16T01:59:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-09-16T01:59:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-09-16T01:59:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-09-16T01:59:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-09-16T01:59:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-09-16T01:59:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-09-16T01:59:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-09-16T01:59:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 17, 2024-09-16T01:59:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 54217, 2024-09-16T01:59:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T01:59:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 102, 2024-09-16T01:59:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-09-16T01:59:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 86, 2024-09-16T01:59:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-09-16T01:59:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-09-16T01:59:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-09-16T01:59:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 3, 2024-09-16T01:59:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 108012, 2024-09-16T01:59:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 12291, 2024-09-16T01:59:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 12291, 2024-09-16T01:59:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 95721, 2024-09-16T01:59:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-09-16T01:59:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 12290, 2024-09-16T01:59:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 144, 2024-09-16T01:59:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 16, 2024-09-16T01:59:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 16, 2024-09-16T01:59:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 128, 2024-09-16T01:59:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 8, 2024-09-16T01:59:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-09-16T01:59:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 15, 2024-09-16T01:59:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 32027907, 2024-09-16T01:59:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 340549470, 2024-09-16T01:59:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 0, 2024-09-16T01:59:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-09-16T01:59:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 6152, 2024-09-16T01:59:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1526.796265984, 2024-09-16T01:59:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.248178846 2024-09-16T01:59:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 0 2024-09-16T01:59:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-09-16T01:59:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 17836, 2024-09-16T01:59:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 69759, 2024-09-16T01:59:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 51923, 2024-09-16T01:59:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 849, 2024-09-16T01:59:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 7342, 2024-09-16T01:59:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 6493, 2024-09-16T01:59:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 35514, 2024-09-16T01:59:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 116144, 2024-09-16T01:59:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 80630, 2024-09-16T01:59:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 1, 2024-09-16T01:59:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 37556, 2024-09-16T01:59:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 37555, 2024-09-16T01:59:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 226288, 2024-09-16T01:59:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 223520 2024-09-16T01:59:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-09-16T01:59:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 139602, 2024-09-16T01:59:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 5006, 2024-09-16T01:59:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 132833, 2024-09-16T01:59:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 204, 2024-09-16T01:59:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 11281, 2024-09-16T01:59:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-09-16T01:59:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 13961, 2024-09-16T01:59:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 107.836926809, 2024-09-16T01:59:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.007724154 2024-09-16T01:59:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-09-16T01:59:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 55, 2024-09-16T01:59:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.860351382, 2024-09-16T01:59:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.015642752 2024-09-16T01:59:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-09-16T01:59:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-09-16T01:59:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-09-16T01:59:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 61144, 2024-09-16T01:59:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 868.108871189, 2024-09-16T01:59:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.014197776 2024-09-16T01:59:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-09-16T01:59:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-09-16T01:59:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-09-16T01:59:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000115570, 2024-09-16T01:59:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000115570 2024-09-16T01:59:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-09-16T01:59:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-09-16T01:59:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-09-16T01:59:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-09-16T01:59:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-09-16T01:59:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 106, 2024-09-16T01:59:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 43.048186593, 2024-09-16T01:59:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.406114967 2024-09-16T01:59:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-09-16T01:59:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-09-16T01:59:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-09-16T01:59:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 11447, 2024-09-16T01:59:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 49.624351803, 2024-09-16T01:59:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.004335140 2024-09-16T01:59:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-09-16T01:59:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 769, 2024-09-16T01:59:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 19.817702790, 2024-09-16T01:59:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.025770744 2024-09-16T01:59:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-09-16T01:59:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 13961, 2024-09-16T01:59:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 274.490521689, 2024-09-16T01:59:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.019661236 2024-09-16T01:59:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-09-16T01:59:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-09-16T01:59:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-09-16T01:59:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 141, 2024-09-16T01:59:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 39.203501796, 2024-09-16T01:59:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.278039019 2024-09-16T01:59:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-09-16T01:59:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-09-16T01:59:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-09-16T01:59:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-09-16T01:59:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-09-16T01:59:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-09-16T01:59:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-09-16T01:59:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-09-16T01:59:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-09-16T01:59:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 15155, 2024-09-16T01:59:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 82.924728558, 2024-09-16T01:59:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.005471773 2024-09-16T01:59:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-09-16T01:59:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-09-16T01:59:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-09-16T01:59:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-09-16T01:59:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2024-09-16T01:59:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 2, 2024-09-16T01:59:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 1, 2024-09-16T01:59:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 1, 2024-09-16T01:59:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-09-16T01:59:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 1252, 2024-09-16T01:59:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 1252, 2024-09-16T01:59:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 1120, 2024-09-16T01:59:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-09-16T01:59:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-09-16T01:59:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-09-16T01:59:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-09-16T01:59:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T01:59:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-09-16T01:59:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T01:59:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-09-16T01:59:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T01:59:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T01:59:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T01:59:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T01:59:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T01:59:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-09-16T01:59:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T01:59:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T01:59:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-09-16T01:59:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-09-16T01:59:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-09-16T01:59:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-09-16T01:59:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T01:59:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T01:59:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T01:59:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T01:59:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T01:59:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-09-16T01:59:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-09-16T01:59:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-09-16T01:59:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T01:59:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T01:59:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-09-16T01:59:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-09-16T01:59:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-09-16T01:59:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-09-16T01:59:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T01:59:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-09-16T01:59:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T01:59:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-09-16T01:59:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 2063659, 2024-09-16T01:59:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 16978, 2024-09-16T01:59:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-09-16T01:59:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-09-16T01:59:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-09-16T01:59:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-09-16T01:59:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-09-16T01:59:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-09-16T01:59:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-09-16T01:59:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-09-16T01:59:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 10344, 2024-09-16T01:59:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 223, 2024-09-16T01:59:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T01:59:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-09-16T01:59:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-09-16T01:59:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-09-16T01:59:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 68817150, 2024-09-16T01:59:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 743827, 2024-09-16T01:59:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-09-16T01:59:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-09-16T01:59:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-09-16T01:59:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-09-16T01:59:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-09-16T01:59:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-09-16T01:59:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-09-16T01:59:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 42283, 2024-09-16T01:59:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 356290804, 2024-09-16T01:59:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 33, 2024-09-16T01:59:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 42283, 2024-09-16T01:59:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-09-16T01:59:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 42283, 2024-09-16T01:59:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 2823.412406328, 2024-09-16T01:59:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.066774174 2024-09-16T01:59:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-09-16T01:59:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-09-16T01:59:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 42283, 2024-09-16T01:59:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 44413 2024-09-16T01:59:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 42283, 2024-09-16T01:59:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 1823, 2024-09-16T01:59:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 40460, 2024-09-16T01:59:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-09-16T01:59:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-09-16T01:59:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 40, 2024-09-16T01:59:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 18, 2024-09-16T01:59:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 328, 2024-09-16T01:59:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 6482, 2024-09-16T01:59:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 186, 2024-09-16T01:59:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-09-16T01:59:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-09-16T01:59:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 1, 2024-09-16T01:59:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 85, 2024-09-16T01:59:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 32433, 2024-09-16T01:59:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-09-16T01:59:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-09-16T01:59:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-09-16T01:59:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 17, 2024-09-16T01:59:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 36, 2024-09-16T01:59:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-09-16T01:59:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-09-16T01:59:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-09-16T01:59:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-09-16T01:59:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-09-16T01:59:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-09-16T01:59:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T01:59:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-09-16T01:59:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-09-16T01:59:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-09-16T01:59:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-09-16T01:59:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-09-16T01:59:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-09-16T01:59:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-09-16T01:59:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-09-16T01:59:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-09-16T01:59:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-09-16T01:59:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-09-16T01:59:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-09-16T01:59:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-09-16T01:59:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-09-16T01:59:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-09-16T01:59:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-09-16T01:59:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-09-16T01:59:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-09-16T01:59:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-09-16T01:59:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 119, 2024-09-16T01:59:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-09-16T01:59:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 36, 2024-09-16T01:59:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 12, 2024-09-16T01:59:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 12, 2024-09-16T01:59:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-09-16T01:59:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-09-16T01:59:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 2427, 2024-09-16T01:59:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 1948, 2024-09-16T01:59:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 412 2024-09-16T01:59:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-09-16T01:59:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 1, 2024-09-16T01:59:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-09-16T01:59:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-09-16T01:59:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-09-16T01:59:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-09-16T01:59:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 54100 2024-09-16T01:59:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-09-16T01:59:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-09-16T01:59:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 17, 2024-09-16T01:59:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-09-16T01:59:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 5, 2024-09-16T01:59:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 32336, 2024-09-16T01:59:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 16160, 2024-09-16T01:59:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-09-16T01:59:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T01:59:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-09-16T01:59:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 376590, 2024-09-16T01:59:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 128609317, 2024-09-16T01:59:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 376590, 2024-09-16T01:59:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-16T01:59:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-16T01:59:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 376590, 2024-09-16T01:59:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 128609317, 2024-09-16T01:59:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-09-16T01:59:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-09-16T01:59:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-16T01:59:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-16T01:59:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 42283, 2024-09-16T01:59:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 358635317, 2024-09-16T01:59:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 42283, 2024-09-16T01:59:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 358635317, 2024-09-16T01:59:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-09-16T01:59:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-09-16T01:59:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-16T01:59:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-16T01:59:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 42283, 2024-09-16T01:59:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 42283, 2024-09-16T01:59:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 42283, 2024-09-16T01:59:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 42283, 2024-09-16T01:59:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-09-16T01:59:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-09-16T01:59:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 16160, 2024-09-16T01:59:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 2387408, 2024-09-16T01:59:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 16160, 2024-09-16T01:59:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-16T01:59:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-16T01:59:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 329, 2024-09-16T01:59:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2387408, 2024-09-16T01:59:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x556ad1ac6960": { 2024-09-16T01:59:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-09-16T01:59:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 108012, 2024-09-16T01:59:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 336355166, 2024-09-16T01:59:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 108012, 2024-09-16T01:59:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-16T01:59:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-16T01:59:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 6152, 2024-09-16T01:59:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 336355166, 2024-09-16T01:59:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:23.103 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T01:59:23.604 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:23 smithi164 ceph-mon[35121]: pgmap v605: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-16T01:59:23.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:23 smithi139 ceph-mon[31175]: pgmap v605: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-16T01:59:23.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:23 smithi176 ceph-mon[35324]: pgmap v605: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-16T01:59:23.945 INFO:tasks.check_counter:Daemon mds.h mds.exported=98 2024-09-16T01:59:23.945 INFO:tasks.check_counter:Daemon mds.h mds.imported=214 2024-09-16T01:59:23.945 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=4911 2024-09-16T01:59:23.945 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=11658 2024-09-16T01:59:23.945 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-16T01:59:23.945 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-16T01:59:23.945 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T01:59:23.945 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=85 2024-09-16T01:59:23.945 DEBUG:tasks.check_counter:Getting stats from k 2024-09-16T01:59:23.946 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-16T01:59:24.417 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T01:59:24.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:59:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:59:24.346+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:59:24.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 01:59:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T01:59:24.346+0000 7fe1f46ec640 -1 calc_pg_upmaps abort due to max <= 0 2024-09-16T01:59:25.060 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T01:59:25.061 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T01:59:25.061 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1395, 2024-09-16T01:59:25.061 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1397, 2024-09-16T01:59:25.061 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 131222470, 2024-09-16T01:59:25.061 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 443674, 2024-09-16T01:59:25.061 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 8, 2024-09-16T01:59:25.061 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-09-16T01:59:25.061 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.186319637, 2024-09-16T01:59:25.061 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.034104160, 2024-09-16T01:59:25.061 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.102971548, 2024-09-16T01:59:25.061 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.038760508, 2024-09-16T01:59:25.061 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:25.061 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1395, 2024-09-16T01:59:25.062 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.070195544, 2024-09-16T01:59:25.062 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000050319 2024-09-16T01:59:25.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.062 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:25.062 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.062 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.062 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.062 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T01:59:25.062 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T01:59:25.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.062 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T01:59:25.062 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1453, 2024-09-16T01:59:25.063 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1394, 2024-09-16T01:59:25.063 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 119584866, 2024-09-16T01:59:25.063 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 542867, 2024-09-16T01:59:25.063 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 6, 2024-09-16T01:59:25.063 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2024-09-16T01:59:25.063 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.216556047, 2024-09-16T01:59:25.063 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.043164728, 2024-09-16T01:59:25.063 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.155982139, 2024-09-16T01:59:25.063 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.026425364, 2024-09-16T01:59:25.063 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:25.063 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1393, 2024-09-16T01:59:25.063 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.088737678, 2024-09-16T01:59:25.063 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000063702 2024-09-16T01:59:25.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.064 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:25.064 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.064 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.064 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.064 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1866400, 2024-09-16T01:59:25.064 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 224544 2024-09-16T01:59:25.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.064 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T01:59:25.064 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1846, 2024-09-16T01:59:25.064 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 2112, 2024-09-16T01:59:25.064 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 88656373, 2024-09-16T01:59:25.064 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 1211264, 2024-09-16T01:59:25.065 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 8, 2024-09-16T01:59:25.065 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 6, 2024-09-16T01:59:25.065 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.241094988, 2024-09-16T01:59:25.065 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.057529987, 2024-09-16T01:59:25.065 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.100426815, 2024-09-16T01:59:25.065 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.067191494, 2024-09-16T01:59:25.065 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:25.065 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2110, 2024-09-16T01:59:25.065 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.121415692, 2024-09-16T01:59:25.065 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000057542 2024-09-16T01:59:25.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.065 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:25.065 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.065 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.066 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.066 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-16T01:59:25.066 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 626144 2024-09-16T01:59:25.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.066 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-09-16T01:59:25.066 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-09-16T01:59:25.066 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-09-16T01:59:25.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.066 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-09-16T01:59:25.066 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-09-16T01:59:25.066 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-09-16T01:59:25.066 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 6841, 2024-09-16T01:59:25.067 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.187281212, 2024-09-16T01:59:25.067 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000027376 2024-09-16T01:59:25.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:25.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.067 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-09-16T01:59:25.067 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-09-16T01:59:25.067 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-09-16T01:59:25.067 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.067 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.067 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:25.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.067 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-09-16T01:59:25.067 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 0, 2024-09-16T01:59:25.068 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 0, 2024-09-16T01:59:25.068 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-09-16T01:59:25.068 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.068 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.068 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.068 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-09-16T01:59:25.068 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2024-09-16T01:59:25.068 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-09-16T01:59:25.068 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-09-16T01:59:25.068 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 0, 2024-09-16T01:59:25.068 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-09-16T01:59:25.068 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-09-16T01:59:25.068 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 60439, 2024-09-16T01:59:25.069 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 3903, 2024-09-16T01:59:25.069 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 56536, 2024-09-16T01:59:25.069 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-09-16T01:59:25.069 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 54863, 2024-09-16T01:59:25.069 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2024-09-16T01:59:25.069 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2024-09-16T01:59:25.069 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2024-09-16T01:59:25.069 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 1, 2024-09-16T01:59:25.069 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 0, 2024-09-16T01:59:25.069 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 0, 2024-09-16T01:59:25.069 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-09-16T01:59:25.069 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2024-09-16T01:59:25.069 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-09-16T01:59:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-09-16T01:59:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-09-16T01:59:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-09-16T01:59:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-09-16T01:59:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-09-16T01:59:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-09-16T01:59:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-09-16T01:59:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-09-16T01:59:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-09-16T01:59:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-09-16T01:59:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-09-16T01:59:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 0, 2024-09-16T01:59:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 0, 2024-09-16T01:59:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2024-09-16T01:59:25.071 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T01:59:25.071 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-09-16T01:59:25.071 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T01:59:25.071 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-09-16T01:59:25.071 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-09-16T01:59:25.071 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T01:59:25.071 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T01:59:25.071 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-09-16T01:59:25.071 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-09-16T01:59:25.071 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T01:59:25.071 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-09-16T01:59:25.071 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T01:59:25.071 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T01:59:25.071 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T01:59:25.072 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-09-16T01:59:25.072 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-09-16T01:59:25.072 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-09-16T01:59:25.072 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-09-16T01:59:25.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.072 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-09-16T01:59:25.072 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-09-16T01:59:25.072 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2024-09-16T01:59:25.072 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2024-09-16T01:59:25.072 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2024-09-16T01:59:25.072 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-09-16T01:59:25.072 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-09-16T01:59:25.072 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-09-16T01:59:25.072 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-09-16T01:59:25.073 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-09-16T01:59:25.073 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-09-16T01:59:25.073 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-09-16T01:59:25.073 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-09-16T01:59:25.073 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-09-16T01:59:25.073 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-09-16T01:59:25.073 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-09-16T01:59:25.073 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-09-16T01:59:25.073 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-09-16T01:59:25.073 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-09-16T01:59:25.073 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-09-16T01:59:25.073 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T01:59:25.073 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 0, 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 108012, 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 0, 2024-09-16T01:59:25.074 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-09-16T01:59:25.075 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-09-16T01:59:25.075 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 143, 2024-09-16T01:59:25.075 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 10, 2024-09-16T01:59:25.075 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-09-16T01:59:25.075 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-09-16T01:59:25.075 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 32027907, 2024-09-16T01:59:25.075 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 340549470, 2024-09-16T01:59:25.075 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 340543125, 2024-09-16T01:59:25.075 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-09-16T01:59:25.075 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.075 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.075 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.075 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 108012 2024-09-16T01:59:25.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.076 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-09-16T01:59:25.076 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 20322, 2024-09-16T01:59:25.076 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 63097, 2024-09-16T01:59:25.076 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 42775, 2024-09-16T01:59:25.076 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 838, 2024-09-16T01:59:25.076 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 2007, 2024-09-16T01:59:25.076 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 1169, 2024-09-16T01:59:25.076 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 60439, 2024-09-16T01:59:25.076 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 106481, 2024-09-16T01:59:25.076 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 46042, 2024-09-16T01:59:25.076 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2024-09-16T01:59:25.076 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 0, 2024-09-16T01:59:25.076 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 0, 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 172376, 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 215328 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 0, 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 0, 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 0, 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.077 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-09-16T01:59:25.078 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.078 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.078 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.078 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-09-16T01:59:25.078 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.078 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.078 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.078 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-09-16T01:59:25.078 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.078 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.079 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.079 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-09-16T01:59:25.079 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.079 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.079 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.079 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-09-16T01:59:25.079 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.079 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.079 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.079 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-09-16T01:59:25.079 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-09-16T01:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-09-16T01:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-09-16T01:59:25.081 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.081 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.081 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.081 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-09-16T01:59:25.081 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.081 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.081 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.081 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-09-16T01:59:25.081 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.081 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.081 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.082 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-09-16T01:59:25.082 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.082 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.082 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.082 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-09-16T01:59:25.082 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.082 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.082 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.082 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-09-16T01:59:25.082 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.082 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.082 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.083 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-09-16T01:59:25.083 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.083 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.083 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.083 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-09-16T01:59:25.083 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.083 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.083 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-09-16T01:59:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-09-16T01:59:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-09-16T01:59:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.086 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-09-16T01:59:25.086 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.086 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.086 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.086 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-09-16T01:59:25.086 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.086 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.086 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.086 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-09-16T01:59:25.086 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.086 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.087 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.087 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-09-16T01:59:25.087 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.087 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.087 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.087 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-09-16T01:59:25.087 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.087 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.087 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.087 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-09-16T01:59:25.087 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.088 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.088 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.088 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-09-16T01:59:25.088 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.088 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.088 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.088 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-09-16T01:59:25.088 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.088 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.088 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.089 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-09-16T01:59:25.089 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.089 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.089 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.089 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-09-16T01:59:25.089 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-09-16T01:59:25.089 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-09-16T01:59:25.089 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.089 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.089 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:25.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.090 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-09-16T01:59:25.090 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2024-09-16T01:59:25.090 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 2, 2024-09-16T01:59:25.090 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 1, 2024-09-16T01:59:25.090 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 2, 2024-09-16T01:59:25.090 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-09-16T01:59:25.090 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 0, 2024-09-16T01:59:25.090 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 0, 2024-09-16T01:59:25.090 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 3617, 2024-09-16T01:59:25.090 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-09-16T01:59:25.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.090 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-09-16T01:59:25.090 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-09-16T01:59:25.090 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-09-16T01:59:25.091 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T01:59:25.091 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-09-16T01:59:25.091 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T01:59:25.091 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-09-16T01:59:25.091 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T01:59:25.091 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T01:59:25.091 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T01:59:25.091 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T01:59:25.091 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T01:59:25.091 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-09-16T01:59:25.091 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T01:59:25.091 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T01:59:25.092 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-09-16T01:59:25.092 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-09-16T01:59:25.092 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-09-16T01:59:25.092 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-09-16T01:59:25.092 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T01:59:25.092 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T01:59:25.092 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T01:59:25.092 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T01:59:25.092 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T01:59:25.092 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-09-16T01:59:25.092 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-09-16T01:59:25.092 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-09-16T01:59:25.092 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T01:59:25.093 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T01:59:25.093 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-09-16T01:59:25.093 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-09-16T01:59:25.093 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-09-16T01:59:25.093 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-09-16T01:59:25.093 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T01:59:25.093 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-09-16T01:59:25.093 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T01:59:25.093 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-09-16T01:59:25.093 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 1006936, 2024-09-16T01:59:25.093 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 20783, 2024-09-16T01:59:25.093 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-09-16T01:59:25.093 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-09-16T01:59:25.094 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-09-16T01:59:25.094 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-09-16T01:59:25.094 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-09-16T01:59:25.094 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-09-16T01:59:25.094 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-09-16T01:59:25.094 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-09-16T01:59:25.094 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 10344, 2024-09-16T01:59:25.094 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 223, 2024-09-16T01:59:25.094 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T01:59:25.094 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-09-16T01:59:25.094 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-09-16T01:59:25.094 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-09-16T01:59:25.094 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 91318510, 2024-09-16T01:59:25.094 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 858224, 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 4236, 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 21, 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 3, 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 4236, 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 4236, 2024-09-16T01:59:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 40.129365461, 2024-09-16T01:59:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.009473410 2024-09-16T01:59:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-09-16T01:59:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-09-16T01:59:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 4236, 2024-09-16T01:59:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 4237 2024-09-16T01:59:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 4236, 2024-09-16T01:59:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 4236, 2024-09-16T01:59:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 0, 2024-09-16T01:59:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-09-16T01:59:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-09-16T01:59:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 2613, 2024-09-16T01:59:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 0, 2024-09-16T01:59:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 1621, 2024-09-16T01:59:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 0, 2024-09-16T01:59:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 0, 2024-09-16T01:59:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-09-16T01:59:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-09-16T01:59:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-09-16T01:59:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-09-16T01:59:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 0, 2024-09-16T01:59:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-09-16T01:59:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-09-16T01:59:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-09-16T01:59:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 1, 2024-09-16T01:59:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 0, 2024-09-16T01:59:25.098 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-09-16T01:59:25.098 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-09-16T01:59:25.098 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-09-16T01:59:25.098 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-09-16T01:59:25.098 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-09-16T01:59:25.098 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-09-16T01:59:25.098 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T01:59:25.098 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-09-16T01:59:25.098 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-09-16T01:59:25.098 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-09-16T01:59:25.098 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-09-16T01:59:25.098 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-09-16T01:59:25.098 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-09-16T01:59:25.098 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-09-16T01:59:25.099 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-09-16T01:59:25.099 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-09-16T01:59:25.099 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-09-16T01:59:25.099 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-09-16T01:59:25.099 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-09-16T01:59:25.099 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-09-16T01:59:25.099 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-09-16T01:59:25.099 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-09-16T01:59:25.099 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-09-16T01:59:25.099 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-09-16T01:59:25.099 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-09-16T01:59:25.099 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-09-16T01:59:25.099 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 119, 2024-09-16T01:59:25.099 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-09-16T01:59:25.100 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 36, 2024-09-16T01:59:25.100 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 11, 2024-09-16T01:59:25.100 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 11, 2024-09-16T01:59:25.100 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-09-16T01:59:25.100 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-09-16T01:59:25.100 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 0, 2024-09-16T01:59:25.100 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 2, 2024-09-16T01:59:25.100 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 0 2024-09-16T01:59:25.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.100 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-09-16T01:59:25.100 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 0, 2024-09-16T01:59:25.100 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-09-16T01:59:25.100 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-09-16T01:59:25.101 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-09-16T01:59:25.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.101 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-09-16T01:59:25.101 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-09-16T01:59:25.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.101 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-09-16T01:59:25.101 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-09-16T01:59:25.101 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T01:59:25.101 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-09-16T01:59:25.101 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-09-16T01:59:25.101 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-09-16T01:59:25.101 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-09-16T01:59:25.101 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-09-16T01:59:25.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T01:59:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-09-16T01:59:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 4690, 2024-09-16T01:59:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 339015674, 2024-09-16T01:59:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 4690, 2024-09-16T01:59:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-16T01:59:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-16T01:59:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 4690, 2024-09-16T01:59:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 339015674, 2024-09-16T01:59:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-09-16T01:59:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-09-16T01:59:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-16T01:59:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-16T01:59:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 4236, 2024-09-16T01:59:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 336415263, 2024-09-16T01:59:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 4236, 2024-09-16T01:59:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 336415263, 2024-09-16T01:59:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-09-16T01:59:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-09-16T01:59:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-16T01:59:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-16T01:59:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 4236, 2024-09-16T01:59:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 4236, 2024-09-16T01:59:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 4236, 2024-09-16T01:59:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 4236, 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x55f4cf1b6b40": { 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-16T01:59:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-16T01:59:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-16T01:59:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-16T01:59:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-09-16T01:59:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-09-16T01:59:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-16T01:59:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-16T01:59:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-16T01:59:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-16T01:59:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T01:59:25.108 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T01:59:25.606 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:25 smithi164 ceph-mon[35121]: pgmap v606: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 188 B/s wr, 10 op/s 2024-09-16T01:59:25.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:25 smithi139 ceph-mon[31175]: pgmap v606: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 188 B/s wr, 10 op/s 2024-09-16T01:59:25.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:25 smithi176 ceph-mon[35324]: pgmap v606: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 188 B/s wr, 10 op/s 2024-09-16T01:59:26.137 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-16T01:59:26.137 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-16T01:59:26.137 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-16T01:59:26.138 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-16T01:59:26.138 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-16T01:59:26.138 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-16T01:59:26.138 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T01:59:26.138 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-09-16T01:59:26.138 DEBUG:tasks.check_counter:Getting stats from c 2024-09-16T01:59:26.138 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-16T01:59:26.682 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:59:27.433 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:27 smithi176 ceph-mon[35324]: pgmap v607: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-16T01:59:27.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:27 smithi139 ceph-mon[31175]: pgmap v607: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-16T01:59:27.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:27 smithi164 ceph-mon[35121]: pgmap v607: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-16T01:59:28.605 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-16T01:59:28.605 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T01:59:28.605 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 339483, 2024-09-16T01:59:28.605 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 426037, 2024-09-16T01:59:28.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 150987290, 2024-09-16T01:59:28.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 333082256, 2024-09-16T01:59:28.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 10, 2024-09-16T01:59:28.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 4, 2024-09-16T01:59:28.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 21.193312520, 2024-09-16T01:59:28.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 9.544298593, 2024-09-16T01:59:28.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 20.811676554, 2024-09-16T01:59:28.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.539942127, 2024-09-16T01:59:28.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:28.606 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 426036, 2024-09-16T01:59:28.606 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 85.122239556, 2024-09-16T01:59:28.606 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000199800 2024-09-16T01:59:28.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.607 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:28.607 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 632871, 2024-09-16T01:59:28.607 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.118327431, 2024-09-16T01:59:28.607 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000186 2024-09-16T01:59:28.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.607 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 102165792, 2024-09-16T01:59:28.607 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 221978624 2024-09-16T01:59:28.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.607 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T01:59:28.607 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 19897, 2024-09-16T01:59:28.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 20220, 2024-09-16T01:59:28.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 10862042, 2024-09-16T01:59:28.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 138512698, 2024-09-16T01:59:28.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 20, 2024-09-16T01:59:28.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 6, 2024-09-16T01:59:28.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 2.763952866, 2024-09-16T01:59:28.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.839271141, 2024-09-16T01:59:28.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.440196524, 2024-09-16T01:59:28.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 1.351398906, 2024-09-16T01:59:28.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:28.609 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 20216, 2024-09-16T01:59:28.609 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 3.390551605, 2024-09-16T01:59:28.609 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000167716 2024-09-16T01:59:28.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.609 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:28.609 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 125, 2024-09-16T01:59:28.609 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000024693, 2024-09-16T01:59:28.609 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000197 2024-09-16T01:59:28.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.609 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 12080, 2024-09-16T01:59:28.609 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 653072 2024-09-16T01:59:28.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.610 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T01:59:28.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 56055, 2024-09-16T01:59:28.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 59085, 2024-09-16T01:59:28.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 24714885, 2024-09-16T01:59:28.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 132561470, 2024-09-16T01:59:28.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 11, 2024-09-16T01:59:28.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 6, 2024-09-16T01:59:28.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 4.811078760, 2024-09-16T01:59:28.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 1.865610857, 2024-09-16T01:59:28.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 2.073324385, 2024-09-16T01:59:28.611 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 1.148212732, 2024-09-16T01:59:28.611 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:28.611 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 59085, 2024-09-16T01:59:28.611 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 11.138577078, 2024-09-16T01:59:28.611 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000188517 2024-09-16T01:59:28.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.611 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:28.611 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 76785, 2024-09-16T01:59:28.611 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.013392432, 2024-09-16T01:59:28.612 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000174 2024-09-16T01:59:28.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.612 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T01:59:28.612 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T01:59:28.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.612 INFO:teuthology.orchestra.run.smithi176.stdout: "cct": { 2024-09-16T01:59:28.612 INFO:teuthology.orchestra.run.smithi176.stdout: "total_workers": 1, 2024-09-16T01:59:28.612 INFO:teuthology.orchestra.run.smithi176.stdout: "unhealthy_workers": 0 2024-09-16T01:59:28.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.613 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-MDSRank": { 2024-09-16T01:59:28.613 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2024-09-16T01:59:28.613 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2024-09-16T01:59:28.613 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 10750, 2024-09-16T01:59:28.613 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 71.656154602, 2024-09-16T01:59:28.613 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.006665688 2024-09-16T01:59:28.613 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:28.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.613 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-PurgeQueue": { 2024-09-16T01:59:28.613 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2024-09-16T01:59:28.613 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2024-09-16T01:59:28.613 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 31390, 2024-09-16T01:59:28.613 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 5.934956051, 2024-09-16T01:59:28.614 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000189071 2024-09-16T01:59:28.614 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:28.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.614 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-09-16T01:59:28.614 INFO:teuthology.orchestra.run.smithi176.stdout: "request": 159645, 2024-09-16T01:59:28.614 INFO:teuthology.orchestra.run.smithi176.stdout: "reply": 144145, 2024-09-16T01:59:28.614 INFO:teuthology.orchestra.run.smithi176.stdout: "reply_latency": { 2024-09-16T01:59:28.614 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 144145, 2024-09-16T01:59:28.614 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 3810.112815148, 2024-09-16T01:59:28.614 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.026432500 2024-09-16T01:59:28.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.614 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_reply": 0, 2024-09-16T01:59:28.614 INFO:teuthology.orchestra.run.smithi176.stdout: "forward": 15489, 2024-09-16T01:59:28.615 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_complete": 494, 2024-09-16T01:59:28.615 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_keys": 0, 2024-09-16T01:59:28.615 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_commit": 611, 2024-09-16T01:59:28.615 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_split": 67, 2024-09-16T01:59:28.615 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_merge": 0, 2024-09-16T01:59:28.615 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes": 34162, 2024-09-16T01:59:28.615 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_top": 1426, 2024-09-16T01:59:28.615 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_bottom": 28855, 2024-09-16T01:59:28.615 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pin_tail": 3881, 2024-09-16T01:59:28.615 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pinned": 32121, 2024-09-16T01:59:28.615 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_expired": 46757, 2024-09-16T01:59:28.615 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_with_caps": 1, 2024-09-16T01:59:28.615 INFO:teuthology.orchestra.run.smithi176.stdout: "caps": 1, 2024-09-16T01:59:28.615 INFO:teuthology.orchestra.run.smithi176.stdout: "subtrees": 6, 2024-09-16T01:59:28.616 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse": 282250, 2024-09-16T01:59:28.616 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_hit": 224941, 2024-09-16T01:59:28.616 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_forward": 15465, 2024-09-16T01:59:28.616 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_discover": 1236, 2024-09-16T01:59:28.616 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_dir_fetch": 73, 2024-09-16T01:59:28.616 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_remote_ino": 0, 2024-09-16T01:59:28.616 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_lock": 494, 2024-09-16T01:59:28.616 INFO:teuthology.orchestra.run.smithi176.stdout: "load_cent": 1000, 2024-09-16T01:59:28.616 INFO:teuthology.orchestra.run.smithi176.stdout: "q": 1, 2024-09-16T01:59:28.616 INFO:teuthology.orchestra.run.smithi176.stdout: "exported": 158, 2024-09-16T01:59:28.616 INFO:teuthology.orchestra.run.smithi176.stdout: "exported_inodes": 34199, 2024-09-16T01:59:28.616 INFO:teuthology.orchestra.run.smithi176.stdout: "imported": 311, 2024-09-16T01:59:28.616 INFO:teuthology.orchestra.run.smithi176.stdout: "imported_inodes": 38296, 2024-09-16T01:59:28.617 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_dir_fetch": 0, 2024-09-16T01:59:28.617 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_backtrace_fetch": 0, 2024-09-16T01:59:28.617 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_peer_discover": 0, 2024-09-16T01:59:28.617 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rfiles": 60337, 2024-09-16T01:59:28.617 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rbytes": 1323008661, 2024-09-16T01:59:28.617 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rsnaps": 6, 2024-09-16T01:59:28.617 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T01:59:28.617 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_set_tag": 0, 2024-09-16T01:59:28.617 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T01:59:28.617 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_inotable_repaired": 0, 2024-09-16T01:59:28.617 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_inodes": 0, 2024-09-16T01:59:28.617 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T01:59:28.617 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T01:59:28.617 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_file_inodes": 0, 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_inode_file_caps": 288, 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_revoke": 59270, 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_grant": 46159, 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps": 73312, 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps_dirty": 14032, 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_cap_release": 2758, 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: "process_request_cap_release": 31967 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache": { 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update": 5317, 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update_receipt": 11343, 2024-09-16T01:59:28.618 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_try_discover": 6388, 2024-09-16T01:59:28.619 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_send_discover": 7867, 2024-09-16T01:59:28.619 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_handle_discover": 8671, 2024-09-16T01:59:28.619 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays": 124, 2024-09-16T01:59:28.619 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_delayed": 0, 2024-09-16T01:59:28.619 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_enqueuing": 0, 2024-09-16T01:59:28.619 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_created": 16229, 2024-09-16T01:59:28.619 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_enqueued": 16105, 2024-09-16T01:59:28.619 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_reintegrated": 0, 2024-09-16T01:59:28.619 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_migrated": 0, 2024-09-16T01:59:28.619 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_processing": 0, 2024-09-16T01:59:28.619 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_enqueued": 0, 2024-09-16T01:59:28.619 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_prioritized": 0, 2024-09-16T01:59:28.619 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_started": 0, 2024-09-16T01:59:28.620 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_completed": 0, 2024-09-16T01:59:28.620 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_quiesce_path": 17, 2024-09-16T01:59:28.620 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_quiesce_inode": 94875, 2024-09-16T01:59:28.620 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T01:59:28.620 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_exportdir": 160, 2024-09-16T01:59:28.620 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_flush": 0, 2024-09-16T01:59:28.620 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragmentdir": 67, 2024-09-16T01:59:28.620 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragstats": 0, 2024-09-16T01:59:28.620 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_inodestats": 0 2024-09-16T01:59:28.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.620 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log": { 2024-09-16T01:59:28.620 INFO:teuthology.orchestra.run.smithi176.stdout: "evlrg": 3, 2024-09-16T01:59:28.620 INFO:teuthology.orchestra.run.smithi176.stdout: "evadd": 92480, 2024-09-16T01:59:28.620 INFO:teuthology.orchestra.run.smithi176.stdout: "evex": 0, 2024-09-16T01:59:28.621 INFO:teuthology.orchestra.run.smithi176.stdout: "evtrm": 0, 2024-09-16T01:59:28.621 INFO:teuthology.orchestra.run.smithi176.stdout: "ev": 92480, 2024-09-16T01:59:28.621 INFO:teuthology.orchestra.run.smithi176.stdout: "evexg": 0, 2024-09-16T01:59:28.621 INFO:teuthology.orchestra.run.smithi176.stdout: "evexd": 0, 2024-09-16T01:59:28.621 INFO:teuthology.orchestra.run.smithi176.stdout: "segadd": 128, 2024-09-16T01:59:28.621 INFO:teuthology.orchestra.run.smithi176.stdout: "segex": 0, 2024-09-16T01:59:28.621 INFO:teuthology.orchestra.run.smithi176.stdout: "segtrm": 0, 2024-09-16T01:59:28.621 INFO:teuthology.orchestra.run.smithi176.stdout: "seg": 128, 2024-09-16T01:59:28.621 INFO:teuthology.orchestra.run.smithi176.stdout: "segmjr": 9, 2024-09-16T01:59:28.621 INFO:teuthology.orchestra.run.smithi176.stdout: "segexg": 0, 2024-09-16T01:59:28.621 INFO:teuthology.orchestra.run.smithi176.stdout: "segexd": 0, 2024-09-16T01:59:28.621 INFO:teuthology.orchestra.run.smithi176.stdout: "expos": 4194304, 2024-09-16T01:59:28.621 INFO:teuthology.orchestra.run.smithi176.stdout: "wrpos": 305759786, 2024-09-16T01:59:28.621 INFO:teuthology.orchestra.run.smithi176.stdout: "rdpos": 0, 2024-09-16T01:59:28.622 INFO:teuthology.orchestra.run.smithi176.stdout: "jlat": { 2024-09-16T01:59:28.622 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 8255, 2024-09-16T01:59:28.622 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 1253.596389846, 2024-09-16T01:59:28.622 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.151859041 2024-09-16T01:59:28.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.622 INFO:teuthology.orchestra.run.smithi176.stdout: "replayed": 0 2024-09-16T01:59:28.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.622 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_mem": { 2024-09-16T01:59:28.622 INFO:teuthology.orchestra.run.smithi176.stdout: "ino": 18527, 2024-09-16T01:59:28.622 INFO:teuthology.orchestra.run.smithi176.stdout: "ino+": 61703, 2024-09-16T01:59:28.622 INFO:teuthology.orchestra.run.smithi176.stdout: "ino-": 43176, 2024-09-16T01:59:28.622 INFO:teuthology.orchestra.run.smithi176.stdout: "dir": 847, 2024-09-16T01:59:28.622 INFO:teuthology.orchestra.run.smithi176.stdout: "dir+": 6647, 2024-09-16T01:59:28.622 INFO:teuthology.orchestra.run.smithi176.stdout: "dir-": 5800, 2024-09-16T01:59:28.623 INFO:teuthology.orchestra.run.smithi176.stdout: "dn": 34162, 2024-09-16T01:59:28.623 INFO:teuthology.orchestra.run.smithi176.stdout: "dn+": 103286, 2024-09-16T01:59:28.623 INFO:teuthology.orchestra.run.smithi176.stdout: "dn-": 69124, 2024-09-16T01:59:28.623 INFO:teuthology.orchestra.run.smithi176.stdout: "cap": 1, 2024-09-16T01:59:28.623 INFO:teuthology.orchestra.run.smithi176.stdout: "cap+": 40350, 2024-09-16T01:59:28.623 INFO:teuthology.orchestra.run.smithi176.stdout: "cap-": 40349, 2024-09-16T01:59:28.623 INFO:teuthology.orchestra.run.smithi176.stdout: "rss": 203064, 2024-09-16T01:59:28.623 INFO:teuthology.orchestra.run.smithi176.stdout: "heap": 223520 2024-09-16T01:59:28.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.623 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_server": { 2024-09-16T01:59:28.623 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_client_request": 166442, 2024-09-16T01:59:28.623 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_server_request": 3382, 2024-09-16T01:59:28.623 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_request": 159645, 2024-09-16T01:59:28.623 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_session": 255, 2024-09-16T01:59:28.624 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_peer_request": 7962, 2024-09-16T01:59:28.624 INFO:teuthology.orchestra.run.smithi176.stdout: "req_create_latency": { 2024-09-16T01:59:28.624 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 10664, 2024-09-16T01:59:28.624 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 205.769357253, 2024-09-16T01:59:28.624 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.019295701 2024-09-16T01:59:28.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.624 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getattr_latency": { 2024-09-16T01:59:28.624 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 231, 2024-09-16T01:59:28.624 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 189.975700142, 2024-09-16T01:59:28.624 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.822405628 2024-09-16T01:59:28.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.624 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getfilelock_latency": { 2024-09-16T01:59:28.624 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: "req_link_latency": { 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookup_latency": { 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 99177, 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 2744.645461982, 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.027674213 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookuphash_latency": { 2024-09-16T01:59:28.625 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.626 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.626 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.626 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupino_latency": { 2024-09-16T01:59:28.626 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.626 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.626 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.626 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupname_latency": { 2024-09-16T01:59:28.626 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.626 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.626 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.626 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupparent_latency": { 2024-09-16T01:59:28.627 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.627 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.627 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.627 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupsnap_latency": { 2024-09-16T01:59:28.627 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.627 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.628 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.628 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lssnap_latency": { 2024-09-16T01:59:28.628 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.629 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.629 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.629 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mkdir_latency": { 2024-09-16T01:59:28.629 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 71, 2024-09-16T01:59:28.629 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 6.166717916, 2024-09-16T01:59:28.629 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.086855181 2024-09-16T01:59:28.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.629 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mknod_latency": { 2024-09-16T01:59:28.629 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.629 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.629 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.630 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mksnap_latency": { 2024-09-16T01:59:28.630 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.630 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.630 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.630 INFO:teuthology.orchestra.run.smithi176.stdout: "req_open_latency": { 2024-09-16T01:59:28.630 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 7535, 2024-09-16T01:59:28.630 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 194.236998190, 2024-09-16T01:59:28.630 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.025777969 2024-09-16T01:59:28.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.630 INFO:teuthology.orchestra.run.smithi176.stdout: "req_readdir_latency": { 2024-09-16T01:59:28.630 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 768, 2024-09-16T01:59:28.630 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 11.764044673, 2024-09-16T01:59:28.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.015317766 2024-09-16T01:59:28.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.631 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rename_latency": { 2024-09-16T01:59:28.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 10671, 2024-09-16T01:59:28.631 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 344.674820491, 2024-09-16T01:59:28.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.032300142 2024-09-16T01:59:28.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.631 INFO:teuthology.orchestra.run.smithi176.stdout: "req_renamesnap_latency": { 2024-09-16T01:59:28.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.631 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.631 INFO:teuthology.orchestra.run.smithi176.stdout: "req_snapdiff_latency": { 2024-09-16T01:59:28.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmdir_latency": { 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 124, 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 61.875676919, 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.498997394 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmsnap_latency": { 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmxattr_latency": { 2024-09-16T01:59:28.632 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.633 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.633 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.633 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setattr_latency": { 2024-09-16T01:59:28.633 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.633 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.633 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.633 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setdirlayout_latency": { 2024-09-16T01:59:28.633 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.633 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.633 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.633 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setfilelock_latency": { 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setlayout_latency": { 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setxattr_latency": { 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.634 INFO:teuthology.orchestra.run.smithi176.stdout: "req_symlink_latency": { 2024-09-16T01:59:28.635 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.635 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.635 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.635 INFO:teuthology.orchestra.run.smithi176.stdout: "req_unlink_latency": { 2024-09-16T01:59:28.635 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 14904, 2024-09-16T01:59:28.635 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 51.004037582, 2024-09-16T01:59:28.635 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.003422171 2024-09-16T01:59:28.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.635 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_revoke_eviction": 0, 2024-09-16T01:59:28.635 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_acquisition_throttle": 0, 2024-09-16T01:59:28.635 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getvxattr_latency": { 2024-09-16T01:59:28.635 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.635 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.636 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.636 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:28.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.636 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_sessions": { 2024-09-16T01:59:28.636 INFO:teuthology.orchestra.run.smithi176.stdout: "session_count": 2, 2024-09-16T01:59:28.636 INFO:teuthology.orchestra.run.smithi176.stdout: "session_add": 4, 2024-09-16T01:59:28.636 INFO:teuthology.orchestra.run.smithi176.stdout: "session_remove": 2, 2024-09-16T01:59:28.636 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_open": 2, 2024-09-16T01:59:28.636 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_stale": 0, 2024-09-16T01:59:28.636 INFO:teuthology.orchestra.run.smithi176.stdout: "total_load": 3416, 2024-09-16T01:59:28.636 INFO:teuthology.orchestra.run.smithi176.stdout: "average_load": 1708, 2024-09-16T01:59:28.636 INFO:teuthology.orchestra.run.smithi176.stdout: "avg_session_uptime": 1086, 2024-09-16T01:59:28.636 INFO:teuthology.orchestra.run.smithi176.stdout: "mdthresh_evicted": 0 2024-09-16T01:59:28.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.637 INFO:teuthology.orchestra.run.smithi176.stdout: "mempool": { 2024-09-16T01:59:28.637 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_bytes": 0, 2024-09-16T01:59:28.637 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_items": 0, 2024-09-16T01:59:28.637 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T01:59:28.637 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_items": 0, 2024-09-16T01:59:28.637 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T01:59:28.637 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_items": 0, 2024-09-16T01:59:28.637 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T01:59:28.637 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T01:59:28.637 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T01:59:28.637 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T01:59:28.637 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T01:59:28.637 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_items": 0, 2024-09-16T01:59:28.637 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_bytes": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_items": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_bytes": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_items": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_items": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_bytes": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_items": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T01:59:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T01:59:28.639 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_bytes": 0, 2024-09-16T01:59:28.639 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_items": 0, 2024-09-16T01:59:28.639 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_bytes": 0, 2024-09-16T01:59:28.639 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_items": 0, 2024-09-16T01:59:28.639 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T01:59:28.639 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_items": 0, 2024-09-16T01:59:28.639 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T01:59:28.639 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_items": 0, 2024-09-16T01:59:28.639 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_bytes": 2012962, 2024-09-16T01:59:28.639 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_items": 18619, 2024-09-16T01:59:28.639 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_bytes": 0, 2024-09-16T01:59:28.639 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_items": 0, 2024-09-16T01:59:28.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bytes": 0, 2024-09-16T01:59:28.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_items": 0, 2024-09-16T01:59:28.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_bytes": 0, 2024-09-16T01:59:28.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_items": 0, 2024-09-16T01:59:28.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_bytes": 0, 2024-09-16T01:59:28.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_items": 0, 2024-09-16T01:59:28.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_bytes": 10344, 2024-09-16T01:59:28.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_items": 223, 2024-09-16T01:59:28.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T01:59:28.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_items": 0, 2024-09-16T01:59:28.640 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_bytes": 0, 2024-09-16T01:59:28.640 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_items": 0, 2024-09-16T01:59:28.640 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_bytes": 69746875, 2024-09-16T01:59:28.640 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_items": 741626, 2024-09-16T01:59:28.641 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_bytes": 0, 2024-09-16T01:59:28.641 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_items": 0, 2024-09-16T01:59:28.641 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_bytes": 0, 2024-09-16T01:59:28.641 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_items": 0 2024-09-16T01:59:28.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.641 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter": { 2024-09-16T01:59:28.641 INFO:teuthology.orchestra.run.smithi176.stdout: "op_active": 0, 2024-09-16T01:59:28.641 INFO:teuthology.orchestra.run.smithi176.stdout: "op_laggy": 0, 2024-09-16T01:59:28.641 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send": 44228, 2024-09-16T01:59:28.642 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send_bytes": 317726525, 2024-09-16T01:59:28.642 INFO:teuthology.orchestra.run.smithi176.stdout: "op_resend": 33, 2024-09-16T01:59:28.642 INFO:teuthology.orchestra.run.smithi176.stdout: "op_reply": 44228, 2024-09-16T01:59:28.642 INFO:teuthology.orchestra.run.smithi176.stdout: "op_latency": { 2024-09-16T01:59:28.642 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 44228, 2024-09-16T01:59:28.642 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 2602.265705367, 2024-09-16T01:59:28.642 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.058837517 2024-09-16T01:59:28.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.642 INFO:teuthology.orchestra.run.smithi176.stdout: "op_inflight": 0, 2024-09-16T01:59:28.642 INFO:teuthology.orchestra.run.smithi176.stdout: "oplen_avg": { 2024-09-16T01:59:28.642 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 44228, 2024-09-16T01:59:28.642 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 46080 2024-09-16T01:59:28.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "op": 44228, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "op_r": 2018, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "op_w": 42210, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "op_rmw": 0, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "op_pg": 0, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_stat": 0, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_create": 10, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_read": 340, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_write": 8595, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writefull": 184, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writesame": 0, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_append": 0, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_zero": 0, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_truncate": 67, 2024-09-16T01:59:28.643 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_delete": 32293, 2024-09-16T01:59:28.644 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_mapext": 0, 2024-09-16T01:59:28.644 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_sparse_read": 0, 2024-09-16T01:59:28.644 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_clonerange": 0, 2024-09-16T01:59:28.644 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_getxattr": 11, 2024-09-16T01:59:28.644 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_setxattr": 20, 2024-09-16T01:59:28.644 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_cmpxattr": 0, 2024-09-16T01:59:28.644 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_rmxattr": 0, 2024-09-16T01:59:28.644 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_resetxattrs": 0, 2024-09-16T01:59:28.644 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_call": 0, 2024-09-16T01:59:28.644 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_watch": 0, 2024-09-16T01:59:28.644 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_notify": 0, 2024-09-16T01:59:28.644 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T01:59:28.644 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls": 0, 2024-09-16T01:59:28.644 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls_filter": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_other": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_active": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_send": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_resend": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_ping": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_active": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_send": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_resend": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_active": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_send": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_resend": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_active": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_send": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_resend": 0, 2024-09-16T01:59:28.645 INFO:teuthology.orchestra.run.smithi176.stdout: "command_active": 0, 2024-09-16T01:59:28.646 INFO:teuthology.orchestra.run.smithi176.stdout: "command_send": 0, 2024-09-16T01:59:28.646 INFO:teuthology.orchestra.run.smithi176.stdout: "command_resend": 0, 2024-09-16T01:59:28.646 INFO:teuthology.orchestra.run.smithi176.stdout: "map_epoch": 119, 2024-09-16T01:59:28.646 INFO:teuthology.orchestra.run.smithi176.stdout: "map_full": 0, 2024-09-16T01:59:28.646 INFO:teuthology.orchestra.run.smithi176.stdout: "map_inc": 36, 2024-09-16T01:59:28.646 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_sessions": 12, 2024-09-16T01:59:28.646 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_open": 12, 2024-09-16T01:59:28.646 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_close": 0, 2024-09-16T01:59:28.646 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_laggy": 0, 2024-09-16T01:59:28.647 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_wr": 2035, 2024-09-16T01:59:28.647 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_rd": 2172, 2024-09-16T01:59:28.647 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_del": 353 2024-09-16T01:59:28.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.647 INFO:teuthology.orchestra.run.smithi176.stdout: "oft": { 2024-09-16T01:59:28.647 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_objs": 1, 2024-09-16T01:59:28.647 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_kv_pairs": 0, 2024-09-16T01:59:28.647 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_updates": 0, 2024-09-16T01:59:28.647 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_removes": 0 2024-09-16T01:59:28.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.647 INFO:teuthology.orchestra.run.smithi176.stdout: "osd-slow-ops": { 2024-09-16T01:59:28.647 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_ops_count": 94560 2024-09-16T01:59:28.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.647 INFO:teuthology.orchestra.run.smithi176.stdout: "purge_queue": { 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops": 0, 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops_high_water": 10, 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing": 0, 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_high_water": 5, 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed_ops": 32210, 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed": 16105, 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_item_in_journal": 0 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 415431, 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 135645186, 2024-09-16T01:59:28.648 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:28.649 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 415431, 2024-09-16T01:59:28.649 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-09-16T01:59:28.649 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-09-16T01:59:28.649 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 415431, 2024-09-16T01:59:28.649 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 135645186, 2024-09-16T01:59:28.649 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:28.649 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.649 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.649 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.649 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:28.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.649 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_bytes": { 2024-09-16T01:59:28.649 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:28.650 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-09-16T01:59:28.650 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:28.650 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-09-16T01:59:28.650 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-09-16T01:59:28.650 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:28.650 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:28.650 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 44228, 2024-09-16T01:59:28.650 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 320145508, 2024-09-16T01:59:28.650 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 44228, 2024-09-16T01:59:28.650 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 320145508, 2024-09-16T01:59:28.650 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:28.650 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.650 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.650 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.651 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:28.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.651 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_ops": { 2024-09-16T01:59:28.651 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:28.651 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 1024, 2024-09-16T01:59:28.651 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:28.651 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-09-16T01:59:28.651 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-09-16T01:59:28.651 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:28.651 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:28.651 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 44228, 2024-09-16T01:59:28.651 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 44228, 2024-09-16T01:59:28.651 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 44228, 2024-09-16T01:59:28.652 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 44228, 2024-09-16T01:59:28.652 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:28.652 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.652 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.652 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.652 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:28.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.652 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle": { 2024-09-16T01:59:28.652 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:28.652 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2024-09-16T01:59:28.652 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:28.652 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 16105, 2024-09-16T01:59:28.652 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 2467261, 2024-09-16T01:59:28.652 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:28.653 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 16105, 2024-09-16T01:59:28.653 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-09-16T01:59:28.653 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-09-16T01:59:28.653 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 340, 2024-09-16T01:59:28.653 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 2467261, 2024-09-16T01:59:28.653 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:28.653 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.653 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.653 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.653 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:28.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:28.653 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle-0x55cb86568960": { 2024-09-16T01:59:28.653 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:28.653 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 92480, 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 301565482, 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 92480, 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 8255, 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 301565482, 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:28.654 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:28.655 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-16T01:59:29.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:29 smithi139 ceph-mon[31175]: pgmap v608: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-16T01:59:29.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:29 smithi176 ceph-mon[35324]: pgmap v608: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-16T01:59:29.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:29 smithi164 ceph-mon[35121]: pgmap v608: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-16T01:59:29.939 INFO:tasks.check_counter:Daemon mds.c mds.exported=158 2024-09-16T01:59:29.939 INFO:tasks.check_counter:Daemon mds.c mds.imported=311 2024-09-16T01:59:29.939 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=5317 2024-09-16T01:59:29.939 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=11343 2024-09-16T01:59:29.939 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-16T01:59:29.939 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-16T01:59:29.939 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T01:59:29.939 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=67 2024-09-16T01:59:29.939 DEBUG:tasks.check_counter:Getting stats from f 2024-09-16T01:59:29.939 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-16T01:59:30.463 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:59:31.162 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-16T01:59:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T01:59:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 28252, 2024-09-16T01:59:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 27773, 2024-09-16T01:59:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 12777120, 2024-09-16T01:59:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 104178977, 2024-09-16T01:59:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 11, 2024-09-16T01:59:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 5, 2024-09-16T01:59:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 2.314863343, 2024-09-16T01:59:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.856204362, 2024-09-16T01:59:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 1.059962521, 2024-09-16T01:59:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.534824114, 2024-09-16T01:59:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 27772, 2024-09-16T01:59:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 4.571130531, 2024-09-16T01:59:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000164594 2024-09-16T01:59:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 35337, 2024-09-16T01:59:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.006108784, 2024-09-16T01:59:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000172 2024-09-16T01:59:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-16T01:59:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T01:59:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T01:59:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 252450, 2024-09-16T01:59:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 336735, 2024-09-16T01:59:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 103003862, 2024-09-16T01:59:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 232269550, 2024-09-16T01:59:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 12, 2024-09-16T01:59:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 9, 2024-09-16T01:59:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 15.315270931, 2024-09-16T01:59:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 6.879009004, 2024-09-16T01:59:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 18.169475947, 2024-09-16T01:59:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.595576270, 2024-09-16T01:59:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 336732, 2024-09-16T01:59:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 72.038192195, 2024-09-16T01:59:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000213933 2024-09-16T01:59:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 462972, 2024-09-16T01:59:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.103473219, 2024-09-16T01:59:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000223 2024-09-16T01:59:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 85115376, 2024-09-16T01:59:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 170232544 2024-09-16T01:59:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T01:59:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 49403, 2024-09-16T01:59:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 44665, 2024-09-16T01:59:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 21911037, 2024-09-16T01:59:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 110428238, 2024-09-16T01:59:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 15, 2024-09-16T01:59:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 6, 2024-09-16T01:59:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 4.028170731, 2024-09-16T01:59:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 1.926314085, 2024-09-16T01:59:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 1.950831487, 2024-09-16T01:59:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.474655198, 2024-09-16T01:59:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 44664, 2024-09-16T01:59:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 6.937323460, 2024-09-16T01:59:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000155322 2024-09-16T01:59:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 73000, 2024-09-16T01:59:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.018691687, 2024-09-16T01:59:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000256 2024-09-16T01:59:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-16T01:59:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 627952 2024-09-16T01:59:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "cct": { 2024-09-16T01:59:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "total_workers": 1, 2024-09-16T01:59:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "unhealthy_workers": 0 2024-09-16T01:59:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-MDSRank": { 2024-09-16T01:59:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2024-09-16T01:59:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2024-09-16T01:59:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 8219, 2024-09-16T01:59:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 60.714133151, 2024-09-16T01:59:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.007387046 2024-09-16T01:59:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-PurgeQueue": { 2024-09-16T01:59:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2024-09-16T01:59:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2024-09-16T01:59:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 17175, 2024-09-16T01:59:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 3.884571671, 2024-09-16T01:59:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000226175 2024-09-16T01:59:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-09-16T01:59:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "request": 99222, 2024-09-16T01:59:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "reply": 98134, 2024-09-16T01:59:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "reply_latency": { 2024-09-16T01:59:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 98134, 2024-09-16T01:59:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 1887.674624304, 2024-09-16T01:59:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.019235684 2024-09-16T01:59:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_reply": 0, 2024-09-16T01:59:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "forward": 1088, 2024-09-16T01:59:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_complete": 80, 2024-09-16T01:59:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_keys": 0, 2024-09-16T01:59:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_commit": 619, 2024-09-16T01:59:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_split": 68, 2024-09-16T01:59:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_merge": 0, 2024-09-16T01:59:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes": 19877, 2024-09-16T01:59:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_top": 2, 2024-09-16T01:59:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_bottom": 17, 2024-09-16T01:59:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pin_tail": 19858, 2024-09-16T01:59:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pinned": 19872, 2024-09-16T01:59:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_expired": 26327, 2024-09-16T01:59:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_with_caps": 0, 2024-09-16T01:59:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "caps": 0, 2024-09-16T01:59:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "subtrees": 4, 2024-09-16T01:59:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse": 182908, 2024-09-16T01:59:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_hit": 155160, 2024-09-16T01:59:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_forward": 755, 2024-09-16T01:59:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_discover": 1295, 2024-09-16T01:59:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_dir_fetch": 81, 2024-09-16T01:59:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_remote_ino": 0, 2024-09-16T01:59:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_lock": 551, 2024-09-16T01:59:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "load_cent": 0, 2024-09-16T01:59:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "q": 0, 2024-09-16T01:59:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "exported": 169, 2024-09-16T01:59:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "exported_inodes": 20144, 2024-09-16T01:59:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "imported": 206, 2024-09-16T01:59:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "imported_inodes": 11003, 2024-09-16T01:59:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_dir_fetch": 0, 2024-09-16T01:59:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_backtrace_fetch": 0, 2024-09-16T01:59:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_peer_discover": 0, 2024-09-16T01:59:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rfiles": 60337, 2024-09-16T01:59:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rbytes": 1323008661, 2024-09-16T01:59:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rsnaps": 6, 2024-09-16T01:59:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T01:59:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_set_tag": 0, 2024-09-16T01:59:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T01:59:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_inotable_repaired": 0, 2024-09-16T01:59:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_inodes": 0, 2024-09-16T01:59:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T01:59:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T01:59:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_file_inodes": 0, 2024-09-16T01:59:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_inode_file_caps": 336, 2024-09-16T01:59:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_revoke": 66627, 2024-09-16T01:59:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_grant": 49321, 2024-09-16T01:59:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T01:59:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T01:59:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T01:59:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps": 81895, 2024-09-16T01:59:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps_dirty": 15136, 2024-09-16T01:59:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_cap_release": 2496, 2024-09-16T01:59:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "process_request_cap_release": 24436 2024-09-16T01:59:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache": { 2024-09-16T01:59:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update": 4286, 2024-09-16T01:59:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update_receipt": 11899, 2024-09-16T01:59:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_try_discover": 6797, 2024-09-16T01:59:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_send_discover": 8276, 2024-09-16T01:59:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_handle_discover": 6402, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays": 65, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_delayed": 0, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_enqueuing": 0, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_created": 8825, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_enqueued": 8760, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_reintegrated": 0, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_migrated": 0, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_processing": 0, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_enqueued": 0, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_prioritized": 0, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_started": 0, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_completed": 0, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_quiesce_path": 17, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_quiesce_inode": 78798, 2024-09-16T01:59:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T01:59:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_exportdir": 169, 2024-09-16T01:59:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_flush": 0, 2024-09-16T01:59:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragmentdir": 68, 2024-09-16T01:59:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragstats": 0, 2024-09-16T01:59:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_inodestats": 0 2024-09-16T01:59:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log": { 2024-09-16T01:59:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "evlrg": 0, 2024-09-16T01:59:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "evadd": 83473, 2024-09-16T01:59:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "evex": 0, 2024-09-16T01:59:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "evtrm": 0, 2024-09-16T01:59:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "ev": 83473, 2024-09-16T01:59:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "evexg": 0, 2024-09-16T01:59:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "evexd": 0, 2024-09-16T01:59:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "segadd": 110, 2024-09-16T01:59:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "segex": 0, 2024-09-16T01:59:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "segtrm": 0, 2024-09-16T01:59:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "seg": 110, 2024-09-16T01:59:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "segmjr": 8, 2024-09-16T01:59:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "segexg": 0, 2024-09-16T01:59:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "segexd": 0, 2024-09-16T01:59:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "expos": 4194304, 2024-09-16T01:59:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "wrpos": 232914524, 2024-09-16T01:59:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "rdpos": 0, 2024-09-16T01:59:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "jlat": { 2024-09-16T01:59:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 6356, 2024-09-16T01:59:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 1024.284822710, 2024-09-16T01:59:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.161152426 2024-09-16T01:59:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "replayed": 0 2024-09-16T01:59:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_mem": { 2024-09-16T01:59:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "ino": 10004, 2024-09-16T01:59:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "ino+": 35206, 2024-09-16T01:59:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "ino-": 25202, 2024-09-16T01:59:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "dir": 409, 2024-09-16T01:59:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "dir+": 6254, 2024-09-16T01:59:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "dir-": 5845, 2024-09-16T01:59:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "dn": 19877, 2024-09-16T01:59:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "dn+": 59468, 2024-09-16T01:59:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "dn-": 39591, 2024-09-16T01:59:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "cap": 0, 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "cap+": 30345, 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "cap-": 30345, 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "rss": 194888, 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "heap": 223520 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_server": { 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_client_request": 106276, 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_server_request": 4895, 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_request": 99222, 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_session": 299, 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_peer_request": 10071, 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "req_create_latency": { 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 11823, 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 65.269658011, 2024-09-16T01:59:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.005520566 2024-09-16T01:59:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getattr_latency": { 2024-09-16T01:59:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 13, 2024-09-16T01:59:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.149604050, 2024-09-16T01:59:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.011508003 2024-09-16T01:59:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getfilelock_latency": { 2024-09-16T01:59:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "req_link_latency": { 2024-09-16T01:59:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookup_latency": { 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 56768, 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 1275.612702140, 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.022470629 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookuphash_latency": { 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupino_latency": { 2024-09-16T01:59:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupname_latency": { 2024-09-16T01:59:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupparent_latency": { 2024-09-16T01:59:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupsnap_latency": { 2024-09-16T01:59:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lssnap_latency": { 2024-09-16T01:59:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mkdir_latency": { 2024-09-16T01:59:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 82, 2024-09-16T01:59:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.960948085, 2024-09-16T01:59:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.011718879 2024-09-16T01:59:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mknod_latency": { 2024-09-16T01:59:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mksnap_latency": { 2024-09-16T01:59:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "req_open_latency": { 2024-09-16T01:59:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 9593, 2024-09-16T01:59:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 62.470850799, 2024-09-16T01:59:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.006512128 2024-09-16T01:59:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "req_readdir_latency": { 2024-09-16T01:59:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 372, 2024-09-16T01:59:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 1.928680870, 2024-09-16T01:59:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.005184625 2024-09-16T01:59:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rename_latency": { 2024-09-16T01:59:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 11806, 2024-09-16T01:59:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 412.995893973, 2024-09-16T01:59:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.034981864 2024-09-16T01:59:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "req_renamesnap_latency": { 2024-09-16T01:59:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "req_snapdiff_latency": { 2024-09-16T01:59:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmdir_latency": { 2024-09-16T01:59:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 65, 2024-09-16T01:59:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 13.185325152, 2024-09-16T01:59:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.202851156 2024-09-16T01:59:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmsnap_latency": { 2024-09-16T01:59:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmxattr_latency": { 2024-09-16T01:59:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setattr_latency": { 2024-09-16T01:59:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setdirlayout_latency": { 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setfilelock_latency": { 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setlayout_latency": { 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setxattr_latency": { 2024-09-16T01:59:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "req_symlink_latency": { 2024-09-16T01:59:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "req_unlink_latency": { 2024-09-16T01:59:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 7612, 2024-09-16T01:59:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 55.100961224, 2024-09-16T01:59:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.007238696 2024-09-16T01:59:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_revoke_eviction": 0, 2024-09-16T01:59:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_acquisition_throttle": 0, 2024-09-16T01:59:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getvxattr_latency": { 2024-09-16T01:59:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_sessions": { 2024-09-16T01:59:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "session_count": 1, 2024-09-16T01:59:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "session_add": 2, 2024-09-16T01:59:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "session_remove": 1, 2024-09-16T01:59:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_open": 1, 2024-09-16T01:59:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_stale": 0, 2024-09-16T01:59:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "total_load": 11889, 2024-09-16T01:59:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "average_load": 11889, 2024-09-16T01:59:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "avg_session_uptime": 1042, 2024-09-16T01:59:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "mdthresh_evicted": 0 2024-09-16T01:59:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "mempool": { 2024-09-16T01:59:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_bytes": 0, 2024-09-16T01:59:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_items": 0, 2024-09-16T01:59:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T01:59:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_items": 0, 2024-09-16T01:59:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T01:59:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_items": 0, 2024-09-16T01:59:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T01:59:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T01:59:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T01:59:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T01:59:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T01:59:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_items": 0, 2024-09-16T01:59:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T01:59:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T01:59:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_bytes": 0, 2024-09-16T01:59:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_items": 0, 2024-09-16T01:59:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_bytes": 0, 2024-09-16T01:59:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_items": 0, 2024-09-16T01:59:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T01:59:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T01:59:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T01:59:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T01:59:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T01:59:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_items": 0, 2024-09-16T01:59:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_bytes": 0, 2024-09-16T01:59:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_items": 0, 2024-09-16T01:59:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T01:59:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T01:59:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_bytes": 0, 2024-09-16T01:59:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_items": 0, 2024-09-16T01:59:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_bytes": 0, 2024-09-16T01:59:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_items": 0, 2024-09-16T01:59:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T01:59:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_items": 0, 2024-09-16T01:59:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T01:59:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_items": 0, 2024-09-16T01:59:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_bytes": 1135293, 2024-09-16T01:59:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_items": 8415, 2024-09-16T01:59:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_bytes": 0, 2024-09-16T01:59:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_items": 0, 2024-09-16T01:59:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bytes": 0, 2024-09-16T01:59:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_items": 0, 2024-09-16T01:59:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_bytes": 0, 2024-09-16T01:59:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_items": 0, 2024-09-16T01:59:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_bytes": 0, 2024-09-16T01:59:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_items": 0, 2024-09-16T01:59:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_bytes": 10344, 2024-09-16T01:59:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_items": 223, 2024-09-16T01:59:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T01:59:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_items": 0, 2024-09-16T01:59:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_bytes": 0, 2024-09-16T01:59:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_items": 0, 2024-09-16T01:59:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_bytes": 38207137, 2024-09-16T01:59:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_items": 393772, 2024-09-16T01:59:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_bytes": 0, 2024-09-16T01:59:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_items": 0, 2024-09-16T01:59:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_bytes": 0, 2024-09-16T01:59:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_items": 0 2024-09-16T01:59:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter": { 2024-09-16T01:59:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "op_active": 0, 2024-09-16T01:59:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "op_laggy": 0, 2024-09-16T01:59:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send": 26082, 2024-09-16T01:59:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send_bytes": 243600293, 2024-09-16T01:59:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "op_resend": 33, 2024-09-16T01:59:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "op_reply": 26082, 2024-09-16T01:59:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "op_latency": { 2024-09-16T01:59:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 26082, 2024-09-16T01:59:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 1394.803825490, 2024-09-16T01:59:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.053477640 2024-09-16T01:59:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "op_inflight": 0, 2024-09-16T01:59:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "oplen_avg": { 2024-09-16T01:59:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 26082, 2024-09-16T01:59:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 27452 2024-09-16T01:59:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "op": 26082, 2024-09-16T01:59:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "op_r": 550, 2024-09-16T01:59:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "op_w": 25532, 2024-09-16T01:59:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "op_rmw": 0, 2024-09-16T01:59:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "op_pg": 0, 2024-09-16T01:59:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_stat": 0, 2024-09-16T01:59:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_create": 10, 2024-09-16T01:59:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_read": 319, 2024-09-16T01:59:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_write": 6675, 2024-09-16T01:59:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writefull": 196, 2024-09-16T01:59:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writesame": 0, 2024-09-16T01:59:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_append": 0, 2024-09-16T01:59:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_zero": 0, 2024-09-16T01:59:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_truncate": 68, 2024-09-16T01:59:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_delete": 17586, 2024-09-16T01:59:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_mapext": 0, 2024-09-16T01:59:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_sparse_read": 0, 2024-09-16T01:59:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_clonerange": 0, 2024-09-16T01:59:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_getxattr": 0, 2024-09-16T01:59:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_setxattr": 20, 2024-09-16T01:59:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_cmpxattr": 0, 2024-09-16T01:59:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_rmxattr": 0, 2024-09-16T01:59:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_resetxattrs": 0, 2024-09-16T01:59:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_call": 0, 2024-09-16T01:59:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_watch": 0, 2024-09-16T01:59:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_notify": 0, 2024-09-16T01:59:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T01:59:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls": 0, 2024-09-16T01:59:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls_filter": 0, 2024-09-16T01:59:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_other": 0, 2024-09-16T01:59:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_active": 0, 2024-09-16T01:59:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_send": 0, 2024-09-16T01:59:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_resend": 0, 2024-09-16T01:59:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_ping": 0, 2024-09-16T01:59:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_active": 0, 2024-09-16T01:59:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_send": 0, 2024-09-16T01:59:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_resend": 0, 2024-09-16T01:59:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_active": 0, 2024-09-16T01:59:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_send": 0, 2024-09-16T01:59:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_resend": 0, 2024-09-16T01:59:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_active": 0, 2024-09-16T01:59:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_send": 0, 2024-09-16T01:59:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_resend": 0, 2024-09-16T01:59:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "command_active": 0, 2024-09-16T01:59:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "command_send": 0, 2024-09-16T01:59:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "command_resend": 0, 2024-09-16T01:59:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "map_epoch": 119, 2024-09-16T01:59:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "map_full": 0, 2024-09-16T01:59:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "map_inc": 36, 2024-09-16T01:59:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_sessions": 12, 2024-09-16T01:59:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_open": 12, 2024-09-16T01:59:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_close": 0, 2024-09-16T01:59:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_laggy": 0, 2024-09-16T01:59:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_wr": 1924, 2024-09-16T01:59:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_rd": 311, 2024-09-16T01:59:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_del": 343 2024-09-16T01:59:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "oft": { 2024-09-16T01:59:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_objs": 1, 2024-09-16T01:59:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_kv_pairs": 0, 2024-09-16T01:59:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_updates": 0, 2024-09-16T01:59:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_removes": 0 2024-09-16T01:59:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "osd-slow-ops": { 2024-09-16T01:59:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_ops_count": 78331 2024-09-16T01:59:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "purge_queue": { 2024-09-16T01:59:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops": 0, 2024-09-16T01:59:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops_high_water": 10, 2024-09-16T01:59:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing": 0, 2024-09-16T01:59:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_high_water": 5, 2024-09-16T01:59:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed_ops": 17520, 2024-09-16T01:59:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed": 8760, 2024-09-16T01:59:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_item_in_journal": 0 2024-09-16T01:59:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T01:59:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-09-16T01:59:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 330101, 2024-09-16T01:59:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 97102709, 2024-09-16T01:59:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 330101, 2024-09-16T01:59:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-09-16T01:59:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-09-16T01:59:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 330101, 2024-09-16T01:59:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 97102709, 2024-09-16T01:59:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_bytes": { 2024-09-16T01:59:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-09-16T01:59:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-09-16T01:59:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-09-16T01:59:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 26082, 2024-09-16T01:59:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 244861498, 2024-09-16T01:59:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 26082, 2024-09-16T01:59:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 244861498, 2024-09-16T01:59:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_ops": { 2024-09-16T01:59:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 1024, 2024-09-16T01:59:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-09-16T01:59:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-09-16T01:59:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 26082, 2024-09-16T01:59:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 26082, 2024-09-16T01:59:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 26082, 2024-09-16T01:59:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 26082, 2024-09-16T01:59:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle": { 2024-09-16T01:59:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2024-09-16T01:59:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 8760, 2024-09-16T01:59:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 1267680, 2024-09-16T01:59:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 8760, 2024-09-16T01:59:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-09-16T01:59:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-09-16T01:59:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 319, 2024-09-16T01:59:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 1267680, 2024-09-16T01:59:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle-0x559322c96960": { 2024-09-16T01:59:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2024-09-16T01:59:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 83473, 2024-09-16T01:59:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 228720220, 2024-09-16T01:59:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 83473, 2024-09-16T01:59:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-09-16T01:59:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-09-16T01:59:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 6356, 2024-09-16T01:59:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 228720220, 2024-09-16T01:59:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:31.212 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-16T01:59:31.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:31 smithi139 ceph-mon[31175]: pgmap v609: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T01:59:31.685 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:31 smithi176 ceph-mon[35324]: pgmap v609: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T01:59:31.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:31 smithi164 ceph-mon[35121]: pgmap v609: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T01:59:31.989 INFO:tasks.check_counter:Daemon mds.f mds.exported=169 2024-09-16T01:59:31.989 INFO:tasks.check_counter:Daemon mds.f mds.imported=206 2024-09-16T01:59:31.989 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=4286 2024-09-16T01:59:31.989 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=11899 2024-09-16T01:59:31.989 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-16T01:59:31.989 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-16T01:59:31.989 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T01:59:31.989 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=68 2024-09-16T01:59:31.989 DEBUG:tasks.check_counter:Getting stats from i 2024-09-16T01:59:31.989 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-16T01:59:32.503 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:59:33.383 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-16T01:59:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T01:59:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 29690, 2024-09-16T01:59:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 33538, 2024-09-16T01:59:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 15974453, 2024-09-16T01:59:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 88832502, 2024-09-16T01:59:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 12, 2024-09-16T01:59:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 7, 2024-09-16T01:59:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 3.336843378, 2024-09-16T01:59:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 1.367953422, 2024-09-16T01:59:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 1.087945671, 2024-09-16T01:59:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.920598592, 2024-09-16T01:59:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 33538, 2024-09-16T01:59:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 7.326401341, 2024-09-16T01:59:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000218450 2024-09-16T01:59:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 32297, 2024-09-16T01:59:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.007901687, 2024-09-16T01:59:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000244 2024-09-16T01:59:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 1898704, 2024-09-16T01:59:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 239568 2024-09-16T01:59:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T01:59:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 56219, 2024-09-16T01:59:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 62242, 2024-09-16T01:59:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 23260979, 2024-09-16T01:59:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 150539314, 2024-09-16T01:59:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 17, 2024-09-16T01:59:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 8, 2024-09-16T01:59:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 5.268215516, 2024-09-16T01:59:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 2.240648362, 2024-09-16T01:59:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 2.271621652, 2024-09-16T01:59:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.999763160, 2024-09-16T01:59:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 62239, 2024-09-16T01:59:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 14.011256684, 2024-09-16T01:59:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000225120 2024-09-16T01:59:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 78537, 2024-09-16T01:59:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.018661296, 2024-09-16T01:59:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000237 2024-09-16T01:59:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T01:59:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 626784 2024-09-16T01:59:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T01:59:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 257437, 2024-09-16T01:59:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 345431, 2024-09-16T01:59:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 121045635, 2024-09-16T01:59:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 279461256, 2024-09-16T01:59:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 10, 2024-09-16T01:59:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 7, 2024-09-16T01:59:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 17.789063925, 2024-09-16T01:59:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 7.551590011, 2024-09-16T01:59:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 16.018276246, 2024-09-16T01:59:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 1.088407223, 2024-09-16T01:59:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 345429, 2024-09-16T01:59:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 80.706093824, 2024-09-16T01:59:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000233640 2024-09-16T01:59:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 473488, 2024-09-16T01:59:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.099010521, 2024-09-16T01:59:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000209 2024-09-16T01:59:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 81056672, 2024-09-16T01:59:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 175476336 2024-09-16T01:59:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "cct": { 2024-09-16T01:59:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "total_workers": 1, 2024-09-16T01:59:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "unhealthy_workers": 0 2024-09-16T01:59:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-MDSRank": { 2024-09-16T01:59:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2024-09-16T01:59:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2024-09-16T01:59:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 6757, 2024-09-16T01:59:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 58.553759772, 2024-09-16T01:59:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.008665644 2024-09-16T01:59:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-PurgeQueue": { 2024-09-16T01:59:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2024-09-16T01:59:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2024-09-16T01:59:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 37332, 2024-09-16T01:59:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 5.911020062, 2024-09-16T01:59:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000158336 2024-09-16T01:59:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-09-16T01:59:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "request": 127293, 2024-09-16T01:59:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "reply": 109883, 2024-09-16T01:59:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "reply_latency": { 2024-09-16T01:59:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 109883, 2024-09-16T01:59:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 1736.183604556, 2024-09-16T01:59:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.015800293 2024-09-16T01:59:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_reply": 0, 2024-09-16T01:59:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "forward": 17344, 2024-09-16T01:59:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_complete": 127, 2024-09-16T01:59:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_keys": 0, 2024-09-16T01:59:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_commit": 506, 2024-09-16T01:59:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_split": 56, 2024-09-16T01:59:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_merge": 0, 2024-09-16T01:59:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes": 40846, 2024-09-16T01:59:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_top": 1768, 2024-09-16T01:59:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_bottom": 18134, 2024-09-16T01:59:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pin_tail": 20944, 2024-09-16T01:59:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pinned": 38316, 2024-09-16T01:59:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_expired": 32484, 2024-09-16T01:59:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_with_caps": 1, 2024-09-16T01:59:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "caps": 1, 2024-09-16T01:59:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "subtrees": 5, 2024-09-16T01:59:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse": 217911, 2024-09-16T01:59:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_hit": 178396, 2024-09-16T01:59:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_forward": 17324, 2024-09-16T01:59:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_discover": 790, 2024-09-16T01:59:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_dir_fetch": 7, 2024-09-16T01:59:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_remote_ino": 0, 2024-09-16T01:59:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_lock": 169, 2024-09-16T01:59:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "load_cent": 0, 2024-09-16T01:59:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "q": 0, 2024-09-16T01:59:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "exported": 219, 2024-09-16T01:59:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "exported_inodes": 24377, 2024-09-16T01:59:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "imported": 131, 2024-09-16T01:59:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "imported_inodes": 46678, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_dir_fetch": 0, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_backtrace_fetch": 0, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_peer_discover": 0, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rfiles": 60337, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rbytes": 1323008661, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rsnaps": 6, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_set_tag": 0, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_inotable_repaired": 0, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_inodes": 0, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_file_inodes": 0, 2024-09-16T01:59:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_inode_file_caps": 216, 2024-09-16T01:59:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_revoke": 47596, 2024-09-16T01:59:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_grant": 45548, 2024-09-16T01:59:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T01:59:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T01:59:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T01:59:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps": 58656, 2024-09-16T01:59:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps_dirty": 10383, 2024-09-16T01:59:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_cap_release": 1644, 2024-09-16T01:59:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "process_request_cap_release": 31706 2024-09-16T01:59:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache": { 2024-09-16T01:59:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update": 6173, 2024-09-16T01:59:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update_receipt": 10700, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_try_discover": 6170, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_send_discover": 7610, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_handle_discover": 7353, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays": 140, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_delayed": 0, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_enqueuing": 0, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_created": 19436, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_enqueued": 19297, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_reintegrated": 0, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_migrated": 0, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_processing": 0, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_enqueued": 0, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_prioritized": 0, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_started": 0, 2024-09-16T01:59:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_completed": 0, 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_quiesce_path": 17, 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_quiesce_inode": 127886, 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_exportdir": 220, 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_flush": 0, 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragmentdir": 61, 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragstats": 0, 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_inodestats": 0 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log": { 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "evlrg": 3, 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "evadd": 83524, 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "evex": 0, 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "evtrm": 0, 2024-09-16T01:59:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "ev": 83524, 2024-09-16T01:59:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "evexg": 0, 2024-09-16T01:59:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "evexd": 0, 2024-09-16T01:59:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "segadd": 108, 2024-09-16T01:59:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "segex": 0, 2024-09-16T01:59:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "segtrm": 0, 2024-09-16T01:59:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "seg": 108, 2024-09-16T01:59:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "segmjr": 8, 2024-09-16T01:59:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "segexg": 0, 2024-09-16T01:59:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "segexd": 0, 2024-09-16T01:59:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "expos": 4194304, 2024-09-16T01:59:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "wrpos": 288554176, 2024-09-16T01:59:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "rdpos": 0, 2024-09-16T01:59:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "jlat": { 2024-09-16T01:59:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 5164, 2024-09-16T01:59:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 773.821129709, 2024-09-16T01:59:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.149849173 2024-09-16T01:59:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "replayed": 0 2024-09-16T01:59:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_mem": { 2024-09-16T01:59:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "ino": 22121, 2024-09-16T01:59:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "ino+": 60253, 2024-09-16T01:59:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "ino-": 38132, 2024-09-16T01:59:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "dir": 894, 2024-09-16T01:59:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "dir+": 6009, 2024-09-16T01:59:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "dir-": 5115, 2024-09-16T01:59:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "dn": 40846, 2024-09-16T01:59:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "dn+": 103699, 2024-09-16T01:59:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "dn-": 62853, 2024-09-16T01:59:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "cap": 1, 2024-09-16T01:59:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "cap+": 34570, 2024-09-16T01:59:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "cap-": 34569, 2024-09-16T01:59:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "rss": 217724, 2024-09-16T01:59:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "heap": 223520 2024-09-16T01:59:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_server": { 2024-09-16T01:59:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_client_request": 134601, 2024-09-16T01:59:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_server_request": 1616, 2024-09-16T01:59:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_request": 127293, 2024-09-16T01:59:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_session": 83, 2024-09-16T01:59:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_peer_request": 7208, 2024-09-16T01:59:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "req_create_latency": { 2024-09-16T01:59:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 6374, 2024-09-16T01:59:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 114.694304174, 2024-09-16T01:59:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.017994086 2024-09-16T01:59:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getattr_latency": { 2024-09-16T01:59:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 4, 2024-09-16T01:59:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.293183899, 2024-09-16T01:59:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.073295974 2024-09-16T01:59:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getfilelock_latency": { 2024-09-16T01:59:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "req_link_latency": { 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookup_latency": { 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 71580, 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 1365.402343512, 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.019075193 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookuphash_latency": { 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupino_latency": { 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 1, 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000147177, 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000147177 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupname_latency": { 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupparent_latency": { 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupsnap_latency": { 2024-09-16T01:59:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lssnap_latency": { 2024-09-16T01:59:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mkdir_latency": { 2024-09-16T01:59:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 50, 2024-09-16T01:59:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.745340251, 2024-09-16T01:59:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.014906805 2024-09-16T01:59:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mknod_latency": { 2024-09-16T01:59:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mksnap_latency": { 2024-09-16T01:59:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "req_open_latency": { 2024-09-16T01:59:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 5561, 2024-09-16T01:59:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 22.701754450, 2024-09-16T01:59:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.004082315 2024-09-16T01:59:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "req_readdir_latency": { 2024-09-16T01:59:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 1004, 2024-09-16T01:59:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 4.097808083, 2024-09-16T01:59:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.004081482 2024-09-16T01:59:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rename_latency": { 2024-09-16T01:59:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 6373, 2024-09-16T01:59:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 126.316493806, 2024-09-16T01:59:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.019820570 2024-09-16T01:59:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "req_renamesnap_latency": { 2024-09-16T01:59:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "req_snapdiff_latency": { 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmdir_latency": { 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 139, 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 35.730253497, 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.257052183 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmsnap_latency": { 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmxattr_latency": { 2024-09-16T01:59:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setattr_latency": { 2024-09-16T01:59:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setdirlayout_latency": { 2024-09-16T01:59:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setfilelock_latency": { 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setlayout_latency": { 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setxattr_latency": { 2024-09-16T01:59:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "req_symlink_latency": { 2024-09-16T01:59:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "req_unlink_latency": { 2024-09-16T01:59:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 18797, 2024-09-16T01:59:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 66.201975707, 2024-09-16T01:59:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.003521943 2024-09-16T01:59:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_revoke_eviction": 0, 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_acquisition_throttle": 0, 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getvxattr_latency": { 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_sessions": { 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "session_count": 2, 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "session_add": 2, 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "session_remove": 0, 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_open": 2, 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_stale": 0, 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "total_load": 9674, 2024-09-16T01:59:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "average_load": 4837, 2024-09-16T01:59:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "avg_session_uptime": 1123, 2024-09-16T01:59:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "mdthresh_evicted": 0 2024-09-16T01:59:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "mempool": { 2024-09-16T01:59:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_bytes": 0, 2024-09-16T01:59:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_items": 0, 2024-09-16T01:59:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T01:59:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_items": 0, 2024-09-16T01:59:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T01:59:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_items": 0, 2024-09-16T01:59:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T01:59:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T01:59:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T01:59:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T01:59:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T01:59:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_items": 0, 2024-09-16T01:59:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T01:59:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T01:59:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_bytes": 0, 2024-09-16T01:59:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_items": 0, 2024-09-16T01:59:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_bytes": 0, 2024-09-16T01:59:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_items": 0, 2024-09-16T01:59:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T01:59:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T01:59:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T01:59:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T01:59:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T01:59:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_items": 0, 2024-09-16T01:59:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_bytes": 0, 2024-09-16T01:59:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_items": 0, 2024-09-16T01:59:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T01:59:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T01:59:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_bytes": 0, 2024-09-16T01:59:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_items": 0, 2024-09-16T01:59:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_bytes": 0, 2024-09-16T01:59:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_items": 0, 2024-09-16T01:59:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T01:59:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_items": 0, 2024-09-16T01:59:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T01:59:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_items": 0, 2024-09-16T01:59:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_bytes": 2181715, 2024-09-16T01:59:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_items": 20380, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_bytes": 0, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_items": 0, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bytes": 0, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_items": 0, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_bytes": 0, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_items": 0, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_bytes": 0, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_items": 0, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_bytes": 10344, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_items": 223, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_items": 0, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_bytes": 0, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_items": 0, 2024-09-16T01:59:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_bytes": 82359441, 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_items": 827098, 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_bytes": 0, 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_items": 0, 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_bytes": 0, 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_items": 0 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter": { 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: "op_active": 0, 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: "op_laggy": 0, 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send": 45953, 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send_bytes": 297380076, 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: "op_resend": 28, 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: "op_reply": 45953, 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: "op_latency": { 2024-09-16T01:59:33.415 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 45953, 2024-09-16T01:59:33.416 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 2389.145190309, 2024-09-16T01:59:33.416 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.051991060 2024-09-16T01:59:33.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.416 INFO:teuthology.orchestra.run.smithi176.stdout: "op_inflight": 0, 2024-09-16T01:59:33.416 INFO:teuthology.orchestra.run.smithi176.stdout: "oplen_avg": { 2024-09-16T01:59:33.416 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 45953, 2024-09-16T01:59:33.416 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 47254 2024-09-16T01:59:33.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.416 INFO:teuthology.orchestra.run.smithi176.stdout: "op": 45953, 2024-09-16T01:59:33.416 INFO:teuthology.orchestra.run.smithi176.stdout: "op_r": 643, 2024-09-16T01:59:33.416 INFO:teuthology.orchestra.run.smithi176.stdout: "op_w": 45310, 2024-09-16T01:59:33.416 INFO:teuthology.orchestra.run.smithi176.stdout: "op_rmw": 0, 2024-09-16T01:59:33.416 INFO:teuthology.orchestra.run.smithi176.stdout: "op_pg": 0, 2024-09-16T01:59:33.416 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_stat": 0, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_create": 10, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_read": 303, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_write": 5468, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writefull": 171, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writesame": 0, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_append": 0, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_zero": 1, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_truncate": 56, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_delete": 38671, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_mapext": 0, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_sparse_read": 0, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_clonerange": 0, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_getxattr": 0, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_setxattr": 20, 2024-09-16T01:59:33.417 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_cmpxattr": 0, 2024-09-16T01:59:33.418 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_rmxattr": 0, 2024-09-16T01:59:33.418 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_resetxattrs": 0, 2024-09-16T01:59:33.418 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_call": 0, 2024-09-16T01:59:33.418 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_watch": 0, 2024-09-16T01:59:33.418 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_notify": 0, 2024-09-16T01:59:33.418 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T01:59:33.418 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls": 0, 2024-09-16T01:59:33.418 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls_filter": 0, 2024-09-16T01:59:33.419 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_other": 0, 2024-09-16T01:59:33.419 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_active": 0, 2024-09-16T01:59:33.419 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_send": 0, 2024-09-16T01:59:33.419 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_resend": 0, 2024-09-16T01:59:33.419 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_ping": 0, 2024-09-16T01:59:33.419 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_active": 0, 2024-09-16T01:59:33.419 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_send": 0, 2024-09-16T01:59:33.419 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_resend": 0, 2024-09-16T01:59:33.419 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_active": 0, 2024-09-16T01:59:33.419 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_send": 0, 2024-09-16T01:59:33.419 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_resend": 0, 2024-09-16T01:59:33.419 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_active": 0, 2024-09-16T01:59:33.419 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_send": 0, 2024-09-16T01:59:33.420 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_resend": 0, 2024-09-16T01:59:33.420 INFO:teuthology.orchestra.run.smithi176.stdout: "command_active": 0, 2024-09-16T01:59:33.420 INFO:teuthology.orchestra.run.smithi176.stdout: "command_send": 0, 2024-09-16T01:59:33.420 INFO:teuthology.orchestra.run.smithi176.stdout: "command_resend": 0, 2024-09-16T01:59:33.420 INFO:teuthology.orchestra.run.smithi176.stdout: "map_epoch": 119, 2024-09-16T01:59:33.420 INFO:teuthology.orchestra.run.smithi176.stdout: "map_full": 0, 2024-09-16T01:59:33.420 INFO:teuthology.orchestra.run.smithi176.stdout: "map_inc": 36, 2024-09-16T01:59:33.420 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_sessions": 12, 2024-09-16T01:59:33.421 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_open": 12, 2024-09-16T01:59:33.421 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_close": 0, 2024-09-16T01:59:33.421 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_laggy": 0, 2024-09-16T01:59:33.421 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_wr": 1779, 2024-09-16T01:59:33.421 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_rd": 467, 2024-09-16T01:59:33.421 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_del": 308 2024-09-16T01:59:33.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.421 INFO:teuthology.orchestra.run.smithi176.stdout: "oft": { 2024-09-16T01:59:33.421 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_objs": 1, 2024-09-16T01:59:33.421 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_kv_pairs": 0, 2024-09-16T01:59:33.421 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_updates": 0, 2024-09-16T01:59:33.421 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_removes": 0 2024-09-16T01:59:33.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.421 INFO:teuthology.orchestra.run.smithi176.stdout: "osd-slow-ops": { 2024-09-16T01:59:33.422 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_ops_count": 124489 2024-09-16T01:59:33.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.422 INFO:teuthology.orchestra.run.smithi176.stdout: "purge_queue": { 2024-09-16T01:59:33.422 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops": 0, 2024-09-16T01:59:33.422 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops_high_water": 10, 2024-09-16T01:59:33.422 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing": 0, 2024-09-16T01:59:33.422 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_high_water": 5, 2024-09-16T01:59:33.422 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed_ops": 38593, 2024-09-16T01:59:33.422 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed": 19297, 2024-09-16T01:59:33.422 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_item_in_journal": 0 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 343342, 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 118547464, 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 343342, 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 343342, 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 118547464, 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:33.423 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.424 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.424 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.424 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:33.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.424 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_bytes": { 2024-09-16T01:59:33.424 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:33.424 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-09-16T01:59:33.424 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:33.424 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-09-16T01:59:33.424 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-09-16T01:59:33.424 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:33.424 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:33.424 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 45953, 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 300369122, 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 45953, 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 300369122, 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_ops": { 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 1024, 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-09-16T01:59:33.425 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 45953, 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 45953, 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 45953, 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 45953, 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle": { 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:33.426 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2024-09-16T01:59:33.427 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:33.427 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 19297, 2024-09-16T01:59:33.427 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 2998349, 2024-09-16T01:59:33.427 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:33.427 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 19297, 2024-09-16T01:59:33.427 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-09-16T01:59:33.427 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-09-16T01:59:33.427 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 303, 2024-09-16T01:59:33.427 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 2998349, 2024-09-16T01:59:33.427 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:33.427 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.427 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.427 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.427 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:33.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:33.428 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle-0x5612ca5f4960": { 2024-09-16T01:59:33.428 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:33.428 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2024-09-16T01:59:33.428 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:33.428 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 83524, 2024-09-16T01:59:33.428 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 284359872, 2024-09-16T01:59:33.428 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:33.428 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 83524, 2024-09-16T01:59:33.428 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-09-16T01:59:33.428 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-09-16T01:59:33.428 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 5164, 2024-09-16T01:59:33.428 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 284359872, 2024-09-16T01:59:33.428 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:33.429 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:33.429 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:33.429 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:33.429 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:33.429 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:33.429 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-16T01:59:33.429 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:33 smithi176 ceph-mon[35324]: pgmap v610: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T01:59:33.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:33 smithi139 ceph-mon[31175]: pgmap v610: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T01:59:33.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:33 smithi164 ceph-mon[35121]: pgmap v610: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T01:59:34.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:34 smithi139 ceph-mon[31175]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:59:34.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:34 smithi176 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:59:34.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:34 smithi164 ceph-mon[35121]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T01:59:35.687 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:35 smithi139 ceph-mon[31175]: pgmap v611: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-16T01:59:35.692 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:35 smithi176 ceph-mon[35324]: pgmap v611: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-16T01:59:35.801 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:35 smithi164 ceph-mon[35121]: pgmap v611: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-16T01:59:37.281 INFO:tasks.check_counter:Daemon mds.i mds.exported=219 2024-09-16T01:59:37.281 INFO:tasks.check_counter:Daemon mds.i mds.imported=131 2024-09-16T01:59:37.281 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=6173 2024-09-16T01:59:37.282 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=10700 2024-09-16T01:59:37.282 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-16T01:59:37.282 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-16T01:59:37.282 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T01:59:37.282 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=56 2024-09-16T01:59:37.282 DEBUG:tasks.check_counter:Getting stats from l 2024-09-16T01:59:37.283 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-16T01:59:37.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:37 smithi139 ceph-mon[31175]: pgmap v612: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-16T01:59:37.688 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:37 smithi176 ceph-mon[35324]: pgmap v612: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-16T01:59:37.782 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T01:59:37.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:37 smithi164 ceph-mon[35121]: pgmap v612: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-16T01:59:38.424 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-16T01:59:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T01:59:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 3135, 2024-09-16T01:59:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 3137, 2024-09-16T01:59:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 117506351, 2024-09-16T01:59:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 996937, 2024-09-16T01:59:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 8, 2024-09-16T01:59:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 4, 2024-09-16T01:59:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.352669019, 2024-09-16T01:59:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.075758009, 2024-09-16T01:59:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.144074241, 2024-09-16T01:59:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.113323194, 2024-09-16T01:59:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 3135, 2024-09-16T01:59:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.155471648, 2024-09-16T01:59:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000049592 2024-09-16T01:59:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T01:59:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T01:59:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T01:59:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 968, 2024-09-16T01:59:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 908, 2024-09-16T01:59:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 85909652, 2024-09-16T01:59:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 389664, 2024-09-16T01:59:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 6, 2024-09-16T01:59:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 5, 2024-09-16T01:59:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.157384272, 2024-09-16T01:59:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.032176822, 2024-09-16T01:59:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.124415311, 2024-09-16T01:59:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.014390541, 2024-09-16T01:59:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 907, 2024-09-16T01:59:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.088223759, 2024-09-16T01:59:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000097269 2024-09-16T01:59:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.428 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 1874288, 2024-09-16T01:59:38.428 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 227104 2024-09-16T01:59:38.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.428 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T01:59:38.428 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 601, 2024-09-16T01:59:38.428 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 869, 2024-09-16T01:59:38.428 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 84068177, 2024-09-16T01:59:38.428 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 905340, 2024-09-16T01:59:38.428 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 9, 2024-09-16T01:59:38.428 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 6, 2024-09-16T01:59:38.428 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.125224750, 2024-09-16T01:59:38.428 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.022394490, 2024-09-16T01:59:38.428 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.071832429, 2024-09-16T01:59:38.429 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.024666591, 2024-09-16T01:59:38.429 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T01:59:38.429 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 867, 2024-09-16T01:59:38.429 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.080468607, 2024-09-16T01:59:38.429 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000092812 2024-09-16T01:59:38.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.429 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-09-16T01:59:38.429 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.429 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.429 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.429 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-09-16T01:59:38.429 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 716448 2024-09-16T01:59:38.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.430 INFO:teuthology.orchestra.run.smithi176.stdout: "cct": { 2024-09-16T01:59:38.430 INFO:teuthology.orchestra.run.smithi176.stdout: "total_workers": 1, 2024-09-16T01:59:38.430 INFO:teuthology.orchestra.run.smithi176.stdout: "unhealthy_workers": 0 2024-09-16T01:59:38.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.430 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-MDSRank": { 2024-09-16T01:59:38.430 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2024-09-16T01:59:38.430 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2024-09-16T01:59:38.430 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 6991, 2024-09-16T01:59:38.430 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.182314516, 2024-09-16T01:59:38.430 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000026078 2024-09-16T01:59:38.430 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:38.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.431 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-PurgeQueue": { 2024-09-16T01:59:38.431 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2024-09-16T01:59:38.431 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2024-09-16T01:59:38.431 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.431 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.431 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.431 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:38.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.431 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-09-16T01:59:38.431 INFO:teuthology.orchestra.run.smithi176.stdout: "request": 0, 2024-09-16T01:59:38.431 INFO:teuthology.orchestra.run.smithi176.stdout: "reply": 0, 2024-09-16T01:59:38.431 INFO:teuthology.orchestra.run.smithi176.stdout: "reply_latency": { 2024-09-16T01:59:38.431 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.431 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_reply": 0, 2024-09-16T01:59:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: "forward": 0, 2024-09-16T01:59:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_complete": 0, 2024-09-16T01:59:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_keys": 0, 2024-09-16T01:59:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_commit": 0, 2024-09-16T01:59:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_split": 0, 2024-09-16T01:59:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_merge": 0, 2024-09-16T01:59:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes": 73585, 2024-09-16T01:59:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_top": 8182, 2024-09-16T01:59:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_bottom": 65403, 2024-09-16T01:59:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pin_tail": 0, 2024-09-16T01:59:38.432 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pinned": 61895, 2024-09-16T01:59:38.433 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_expired": 2, 2024-09-16T01:59:38.433 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_with_caps": 0, 2024-09-16T01:59:38.433 INFO:teuthology.orchestra.run.smithi176.stdout: "caps": 0, 2024-09-16T01:59:38.433 INFO:teuthology.orchestra.run.smithi176.stdout: "subtrees": 1, 2024-09-16T01:59:38.433 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse": 0, 2024-09-16T01:59:38.433 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_hit": 0, 2024-09-16T01:59:38.433 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_forward": 0, 2024-09-16T01:59:38.433 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_discover": 0, 2024-09-16T01:59:38.433 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_dir_fetch": 0, 2024-09-16T01:59:38.433 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_remote_ino": 0, 2024-09-16T01:59:38.433 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_lock": 0, 2024-09-16T01:59:38.433 INFO:teuthology.orchestra.run.smithi176.stdout: "load_cent": 0, 2024-09-16T01:59:38.433 INFO:teuthology.orchestra.run.smithi176.stdout: "q": 0, 2024-09-16T01:59:38.433 INFO:teuthology.orchestra.run.smithi176.stdout: "exported": 0, 2024-09-16T01:59:38.434 INFO:teuthology.orchestra.run.smithi176.stdout: "exported_inodes": 0, 2024-09-16T01:59:38.434 INFO:teuthology.orchestra.run.smithi176.stdout: "imported": 0, 2024-09-16T01:59:38.434 INFO:teuthology.orchestra.run.smithi176.stdout: "imported_inodes": 0, 2024-09-16T01:59:38.434 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_dir_fetch": 0, 2024-09-16T01:59:38.434 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_backtrace_fetch": 0, 2024-09-16T01:59:38.434 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_peer_discover": 0, 2024-09-16T01:59:38.434 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rfiles": 0, 2024-09-16T01:59:38.434 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rbytes": 0, 2024-09-16T01:59:38.434 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rsnaps": 0, 2024-09-16T01:59:38.434 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T01:59:38.434 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_set_tag": 0, 2024-09-16T01:59:38.434 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T01:59:38.434 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_inotable_repaired": 0, 2024-09-16T01:59:38.434 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_inodes": 0, 2024-09-16T01:59:38.435 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T01:59:38.435 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T01:59:38.435 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_file_inodes": 0, 2024-09-16T01:59:38.435 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_inode_file_caps": 0, 2024-09-16T01:59:38.435 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T01:59:38.435 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_grant": 0, 2024-09-16T01:59:38.435 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T01:59:38.435 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T01:59:38.435 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T01:59:38.435 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps": 0, 2024-09-16T01:59:38.435 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps_dirty": 0, 2024-09-16T01:59:38.435 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_cap_release": 0, 2024-09-16T01:59:38.435 INFO:teuthology.orchestra.run.smithi176.stdout: "process_request_cap_release": 0 2024-09-16T01:59:38.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.436 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache": { 2024-09-16T01:59:38.436 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update": 0, 2024-09-16T01:59:38.436 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update_receipt": 0, 2024-09-16T01:59:38.436 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_try_discover": 0, 2024-09-16T01:59:38.436 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_send_discover": 0, 2024-09-16T01:59:38.436 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_handle_discover": 0, 2024-09-16T01:59:38.436 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays": 0, 2024-09-16T01:59:38.436 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_delayed": 0, 2024-09-16T01:59:38.436 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_enqueuing": 0, 2024-09-16T01:59:38.436 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_created": 0, 2024-09-16T01:59:38.436 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_enqueued": 0, 2024-09-16T01:59:38.436 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_reintegrated": 0, 2024-09-16T01:59:38.436 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_migrated": 0, 2024-09-16T01:59:38.436 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_processing": 0, 2024-09-16T01:59:38.437 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_enqueued": 0, 2024-09-16T01:59:38.437 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_prioritized": 0, 2024-09-16T01:59:38.437 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_started": 0, 2024-09-16T01:59:38.437 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_completed": 0, 2024-09-16T01:59:38.437 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_quiesce_path": 0, 2024-09-16T01:59:38.437 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_quiesce_inode": 0, 2024-09-16T01:59:38.437 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T01:59:38.437 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_exportdir": 0, 2024-09-16T01:59:38.437 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_flush": 0, 2024-09-16T01:59:38.437 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragmentdir": 0, 2024-09-16T01:59:38.437 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragstats": 0, 2024-09-16T01:59:38.437 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_inodestats": 0 2024-09-16T01:59:38.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.437 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log": { 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "evlrg": 0, 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "evadd": 0, 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "evex": 0, 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "evtrm": 0, 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "ev": 83524, 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "evexg": 0, 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "evexd": 0, 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "segadd": 0, 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "segex": 0, 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "segtrm": 0, 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "seg": 108, 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "segmjr": 8, 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "segexg": 0, 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "segexd": 0, 2024-09-16T01:59:38.438 INFO:teuthology.orchestra.run.smithi176.stdout: "expos": 4194304, 2024-09-16T01:59:38.439 INFO:teuthology.orchestra.run.smithi176.stdout: "wrpos": 288554176, 2024-09-16T01:59:38.439 INFO:teuthology.orchestra.run.smithi176.stdout: "rdpos": 288551702, 2024-09-16T01:59:38.439 INFO:teuthology.orchestra.run.smithi176.stdout: "jlat": { 2024-09-16T01:59:38.439 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.439 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.439 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.439 INFO:teuthology.orchestra.run.smithi176.stdout: "replayed": 83524 2024-09-16T01:59:38.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.439 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_mem": { 2024-09-16T01:59:38.439 INFO:teuthology.orchestra.run.smithi176.stdout: "ino": 22981, 2024-09-16T01:59:38.439 INFO:teuthology.orchestra.run.smithi176.stdout: "ino+": 59566, 2024-09-16T01:59:38.439 INFO:teuthology.orchestra.run.smithi176.stdout: "ino-": 36585, 2024-09-16T01:59:38.439 INFO:teuthology.orchestra.run.smithi176.stdout: "dir": 864, 2024-09-16T01:59:38.440 INFO:teuthology.orchestra.run.smithi176.stdout: "dir+": 1620, 2024-09-16T01:59:38.440 INFO:teuthology.orchestra.run.smithi176.stdout: "dir-": 756, 2024-09-16T01:59:38.440 INFO:teuthology.orchestra.run.smithi176.stdout: "dn": 73585, 2024-09-16T01:59:38.440 INFO:teuthology.orchestra.run.smithi176.stdout: "dn+": 101409, 2024-09-16T01:59:38.440 INFO:teuthology.orchestra.run.smithi176.stdout: "dn-": 27824, 2024-09-16T01:59:38.440 INFO:teuthology.orchestra.run.smithi176.stdout: "cap": 0, 2024-09-16T01:59:38.440 INFO:teuthology.orchestra.run.smithi176.stdout: "cap+": 0, 2024-09-16T01:59:38.440 INFO:teuthology.orchestra.run.smithi176.stdout: "cap-": 0, 2024-09-16T01:59:38.440 INFO:teuthology.orchestra.run.smithi176.stdout: "rss": 174812, 2024-09-16T01:59:38.440 INFO:teuthology.orchestra.run.smithi176.stdout: "heap": 215328 2024-09-16T01:59:38.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.440 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_server": { 2024-09-16T01:59:38.440 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_client_request": 0, 2024-09-16T01:59:38.440 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_server_request": 0, 2024-09-16T01:59:38.441 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_request": 0, 2024-09-16T01:59:38.441 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_session": 0, 2024-09-16T01:59:38.441 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_peer_request": 0, 2024-09-16T01:59:38.441 INFO:teuthology.orchestra.run.smithi176.stdout: "req_create_latency": { 2024-09-16T01:59:38.441 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.441 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.441 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.441 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getattr_latency": { 2024-09-16T01:59:38.441 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.441 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.441 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.441 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getfilelock_latency": { 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: "req_link_latency": { 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookup_latency": { 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.442 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookuphash_latency": { 2024-09-16T01:59:38.443 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.443 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.443 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.443 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupino_latency": { 2024-09-16T01:59:38.443 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.443 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.443 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.443 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupname_latency": { 2024-09-16T01:59:38.443 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.443 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.443 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.444 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupparent_latency": { 2024-09-16T01:59:38.444 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.444 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.444 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.444 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupsnap_latency": { 2024-09-16T01:59:38.444 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.444 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.444 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.444 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lssnap_latency": { 2024-09-16T01:59:38.444 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.444 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.444 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.445 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mkdir_latency": { 2024-09-16T01:59:38.445 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.445 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.445 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.445 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mknod_latency": { 2024-09-16T01:59:38.445 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.446 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.446 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.446 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mksnap_latency": { 2024-09-16T01:59:38.446 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.446 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.446 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.446 INFO:teuthology.orchestra.run.smithi176.stdout: "req_open_latency": { 2024-09-16T01:59:38.446 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.446 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.446 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.446 INFO:teuthology.orchestra.run.smithi176.stdout: "req_readdir_latency": { 2024-09-16T01:59:38.447 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.447 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.447 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.447 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rename_latency": { 2024-09-16T01:59:38.447 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.447 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.447 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.447 INFO:teuthology.orchestra.run.smithi176.stdout: "req_renamesnap_latency": { 2024-09-16T01:59:38.447 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.447 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.447 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.448 INFO:teuthology.orchestra.run.smithi176.stdout: "req_snapdiff_latency": { 2024-09-16T01:59:38.448 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.448 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.448 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.448 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmdir_latency": { 2024-09-16T01:59:38.448 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.448 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.448 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.448 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmsnap_latency": { 2024-09-16T01:59:38.448 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.448 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.448 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.449 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmxattr_latency": { 2024-09-16T01:59:38.449 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.449 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.449 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.449 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setattr_latency": { 2024-09-16T01:59:38.449 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.449 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.449 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.449 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setdirlayout_latency": { 2024-09-16T01:59:38.449 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.449 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.450 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.450 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setfilelock_latency": { 2024-09-16T01:59:38.450 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.450 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.450 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.450 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setlayout_latency": { 2024-09-16T01:59:38.450 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.450 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.450 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.450 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setxattr_latency": { 2024-09-16T01:59:38.450 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.451 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.451 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.451 INFO:teuthology.orchestra.run.smithi176.stdout: "req_symlink_latency": { 2024-09-16T01:59:38.451 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.451 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.451 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.451 INFO:teuthology.orchestra.run.smithi176.stdout: "req_unlink_latency": { 2024-09-16T01:59:38.451 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.451 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.451 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.451 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_revoke_eviction": 0, 2024-09-16T01:59:38.452 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_acquisition_throttle": 0, 2024-09-16T01:59:38.452 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getvxattr_latency": { 2024-09-16T01:59:38.452 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.452 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.452 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.452 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:38.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.452 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_sessions": { 2024-09-16T01:59:38.452 INFO:teuthology.orchestra.run.smithi176.stdout: "session_count": 2, 2024-09-16T01:59:38.452 INFO:teuthology.orchestra.run.smithi176.stdout: "session_add": 2, 2024-09-16T01:59:38.452 INFO:teuthology.orchestra.run.smithi176.stdout: "session_remove": 0, 2024-09-16T01:59:38.452 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_open": 2, 2024-09-16T01:59:38.452 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_stale": 0, 2024-09-16T01:59:38.453 INFO:teuthology.orchestra.run.smithi176.stdout: "total_load": 0, 2024-09-16T01:59:38.453 INFO:teuthology.orchestra.run.smithi176.stdout: "average_load": 0, 2024-09-16T01:59:38.453 INFO:teuthology.orchestra.run.smithi176.stdout: "avg_session_uptime": 2407, 2024-09-16T01:59:38.453 INFO:teuthology.orchestra.run.smithi176.stdout: "mdthresh_evicted": 0 2024-09-16T01:59:38.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.453 INFO:teuthology.orchestra.run.smithi176.stdout: "mempool": { 2024-09-16T01:59:38.453 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_bytes": 0, 2024-09-16T01:59:38.453 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_items": 0, 2024-09-16T01:59:38.453 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T01:59:38.453 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_items": 0, 2024-09-16T01:59:38.453 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T01:59:38.453 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_items": 0, 2024-09-16T01:59:38.453 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T01:59:38.453 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T01:59:38.454 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T01:59:38.454 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T01:59:38.454 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T01:59:38.454 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_items": 0, 2024-09-16T01:59:38.454 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T01:59:38.454 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T01:59:38.454 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_bytes": 0, 2024-09-16T01:59:38.454 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_items": 0, 2024-09-16T01:59:38.454 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_bytes": 0, 2024-09-16T01:59:38.454 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_items": 0, 2024-09-16T01:59:38.454 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T01:59:38.454 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T01:59:38.454 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T01:59:38.454 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T01:59:38.455 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T01:59:38.455 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_items": 0, 2024-09-16T01:59:38.455 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_bytes": 0, 2024-09-16T01:59:38.455 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_items": 0, 2024-09-16T01:59:38.455 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T01:59:38.455 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T01:59:38.455 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_bytes": 0, 2024-09-16T01:59:38.455 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_items": 0, 2024-09-16T01:59:38.455 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_bytes": 0, 2024-09-16T01:59:38.455 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_items": 0, 2024-09-16T01:59:38.455 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T01:59:38.455 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_items": 0, 2024-09-16T01:59:38.455 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T01:59:38.455 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_items": 0, 2024-09-16T01:59:38.456 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_bytes": 1105320, 2024-09-16T01:59:38.456 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_items": 23442, 2024-09-16T01:59:38.456 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_bytes": 0, 2024-09-16T01:59:38.456 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_items": 0, 2024-09-16T01:59:38.456 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bytes": 0, 2024-09-16T01:59:38.456 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_items": 0, 2024-09-16T01:59:38.456 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_bytes": 0, 2024-09-16T01:59:38.456 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_items": 0, 2024-09-16T01:59:38.456 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_bytes": 0, 2024-09-16T01:59:38.456 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_items": 0, 2024-09-16T01:59:38.456 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_bytes": 10344, 2024-09-16T01:59:38.456 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_items": 223, 2024-09-16T01:59:38.456 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T01:59:38.456 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_items": 0, 2024-09-16T01:59:38.457 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_bytes": 0, 2024-09-16T01:59:38.457 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_items": 0, 2024-09-16T01:59:38.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_bytes": 106806141, 2024-09-16T01:59:38.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_items": 1073357, 2024-09-16T01:59:38.457 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_bytes": 0, 2024-09-16T01:59:38.457 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_items": 0, 2024-09-16T01:59:38.457 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_bytes": 0, 2024-09-16T01:59:38.457 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_items": 0 2024-09-16T01:59:38.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.457 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter": { 2024-09-16T01:59:38.457 INFO:teuthology.orchestra.run.smithi176.stdout: "op_active": 0, 2024-09-16T01:59:38.457 INFO:teuthology.orchestra.run.smithi176.stdout: "op_laggy": 0, 2024-09-16T01:59:38.457 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send": 4240, 2024-09-16T01:59:38.457 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send_bytes": 21, 2024-09-16T01:59:38.458 INFO:teuthology.orchestra.run.smithi176.stdout: "op_resend": 3, 2024-09-16T01:59:38.458 INFO:teuthology.orchestra.run.smithi176.stdout: "op_reply": 4240, 2024-09-16T01:59:38.458 INFO:teuthology.orchestra.run.smithi176.stdout: "op_latency": { 2024-09-16T01:59:38.458 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 4240, 2024-09-16T01:59:38.458 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 39.534666977, 2024-09-16T01:59:38.458 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.009324213 2024-09-16T01:59:38.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.458 INFO:teuthology.orchestra.run.smithi176.stdout: "op_inflight": 0, 2024-09-16T01:59:38.458 INFO:teuthology.orchestra.run.smithi176.stdout: "oplen_avg": { 2024-09-16T01:59:38.458 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 4240, 2024-09-16T01:59:38.458 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 4241 2024-09-16T01:59:38.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.458 INFO:teuthology.orchestra.run.smithi176.stdout: "op": 4240, 2024-09-16T01:59:38.458 INFO:teuthology.orchestra.run.smithi176.stdout: "op_r": 4240, 2024-09-16T01:59:38.459 INFO:teuthology.orchestra.run.smithi176.stdout: "op_w": 0, 2024-09-16T01:59:38.459 INFO:teuthology.orchestra.run.smithi176.stdout: "op_rmw": 0, 2024-09-16T01:59:38.459 INFO:teuthology.orchestra.run.smithi176.stdout: "op_pg": 0, 2024-09-16T01:59:38.459 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_stat": 2569, 2024-09-16T01:59:38.459 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_create": 0, 2024-09-16T01:59:38.459 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_read": 1669, 2024-09-16T01:59:38.459 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_write": 0, 2024-09-16T01:59:38.459 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writefull": 0, 2024-09-16T01:59:38.459 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writesame": 0, 2024-09-16T01:59:38.459 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_append": 0, 2024-09-16T01:59:38.459 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_zero": 0, 2024-09-16T01:59:38.459 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_truncate": 0, 2024-09-16T01:59:38.459 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_delete": 0, 2024-09-16T01:59:38.459 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_mapext": 0, 2024-09-16T01:59:38.460 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_sparse_read": 0, 2024-09-16T01:59:38.460 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_clonerange": 0, 2024-09-16T01:59:38.460 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_getxattr": 1, 2024-09-16T01:59:38.460 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_setxattr": 0, 2024-09-16T01:59:38.460 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_cmpxattr": 0, 2024-09-16T01:59:38.460 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_rmxattr": 0, 2024-09-16T01:59:38.460 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_resetxattrs": 0, 2024-09-16T01:59:38.460 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_call": 0, 2024-09-16T01:59:38.460 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_watch": 0, 2024-09-16T01:59:38.460 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_notify": 0, 2024-09-16T01:59:38.460 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T01:59:38.460 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls": 0, 2024-09-16T01:59:38.460 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls_filter": 0, 2024-09-16T01:59:38.461 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_other": 0, 2024-09-16T01:59:38.461 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_active": 0, 2024-09-16T01:59:38.461 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_send": 0, 2024-09-16T01:59:38.461 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_resend": 0, 2024-09-16T01:59:38.461 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_ping": 0, 2024-09-16T01:59:38.461 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_active": 0, 2024-09-16T01:59:38.461 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_send": 0, 2024-09-16T01:59:38.461 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_resend": 0, 2024-09-16T01:59:38.461 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_active": 0, 2024-09-16T01:59:38.461 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_send": 0, 2024-09-16T01:59:38.461 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_resend": 0, 2024-09-16T01:59:38.461 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_active": 0, 2024-09-16T01:59:38.461 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_send": 0, 2024-09-16T01:59:38.461 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_resend": 0, 2024-09-16T01:59:38.462 INFO:teuthology.orchestra.run.smithi176.stdout: "command_active": 0, 2024-09-16T01:59:38.462 INFO:teuthology.orchestra.run.smithi176.stdout: "command_send": 0, 2024-09-16T01:59:38.462 INFO:teuthology.orchestra.run.smithi176.stdout: "command_resend": 0, 2024-09-16T01:59:38.462 INFO:teuthology.orchestra.run.smithi176.stdout: "map_epoch": 119, 2024-09-16T01:59:38.462 INFO:teuthology.orchestra.run.smithi176.stdout: "map_full": 0, 2024-09-16T01:59:38.462 INFO:teuthology.orchestra.run.smithi176.stdout: "map_inc": 36, 2024-09-16T01:59:38.462 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_sessions": 11, 2024-09-16T01:59:38.462 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_open": 11, 2024-09-16T01:59:38.462 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_close": 0, 2024-09-16T01:59:38.462 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_laggy": 0, 2024-09-16T01:59:38.462 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_wr": 0, 2024-09-16T01:59:38.462 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_rd": 2, 2024-09-16T01:59:38.462 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_del": 0 2024-09-16T01:59:38.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.463 INFO:teuthology.orchestra.run.smithi176.stdout: "oft": { 2024-09-16T01:59:38.463 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_objs": 0, 2024-09-16T01:59:38.463 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_kv_pairs": 0, 2024-09-16T01:59:38.463 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_updates": 0, 2024-09-16T01:59:38.463 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_removes": 0 2024-09-16T01:59:38.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.463 INFO:teuthology.orchestra.run.smithi176.stdout: "osd-slow-ops": { 2024-09-16T01:59:38.463 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_ops_count": 0 2024-09-16T01:59:38.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.463 INFO:teuthology.orchestra.run.smithi176.stdout: "purge_queue": { 2024-09-16T01:59:38.463 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops": 0, 2024-09-16T01:59:38.463 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T01:59:38.463 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing": 0, 2024-09-16T01:59:38.464 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_high_water": 0, 2024-09-16T01:59:38.464 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed_ops": 0, 2024-09-16T01:59:38.464 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed": 0, 2024-09-16T01:59:38.464 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_item_in_journal": 0 2024-09-16T01:59:38.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.464 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T01:59:38.464 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:38.464 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-09-16T01:59:38.464 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:38.464 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 4700, 2024-09-16T01:59:38.464 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 287034953, 2024-09-16T01:59:38.464 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:38.464 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 4700, 2024-09-16T01:59:38.464 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-09-16T01:59:38.465 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-09-16T01:59:38.465 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 4700, 2024-09-16T01:59:38.465 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 287034953, 2024-09-16T01:59:38.465 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:38.465 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.465 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.465 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:38.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.465 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_bytes": { 2024-09-16T01:59:38.465 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:38.465 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-09-16T01:59:38.465 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:38.465 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-09-16T01:59:38.466 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-09-16T01:59:38.466 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:38.466 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:38.466 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 4240, 2024-09-16T01:59:38.466 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 284423496, 2024-09-16T01:59:38.466 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 4240, 2024-09-16T01:59:38.466 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 284423496, 2024-09-16T01:59:38.466 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:38.466 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.466 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.466 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.466 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:38.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.466 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_ops": { 2024-09-16T01:59:38.467 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:38.467 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 1024, 2024-09-16T01:59:38.467 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:38.467 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-09-16T01:59:38.467 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-09-16T01:59:38.467 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:38.467 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:38.467 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 4240, 2024-09-16T01:59:38.467 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 4240, 2024-09-16T01:59:38.467 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 4240, 2024-09-16T01:59:38.467 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 4240, 2024-09-16T01:59:38.467 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:38.467 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.467 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle": { 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 0, 2024-09-16T01:59:38.468 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 0, 2024-09-16T01:59:38.469 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:38.469 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.469 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.469 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.469 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:38.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-16T01:59:38.469 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle-0x564fe828cb40": { 2024-09-16T01:59:38.469 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-09-16T01:59:38.469 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2024-09-16T01:59:38.469 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-09-16T01:59:38.469 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-09-16T01:59:38.469 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-09-16T01:59:38.469 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-09-16T01:59:38.469 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-09-16T01:59:38.470 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-09-16T01:59:38.470 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-09-16T01:59:38.470 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 0, 2024-09-16T01:59:38.470 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 0, 2024-09-16T01:59:38.470 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-09-16T01:59:38.470 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-09-16T01:59:38.470 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-09-16T01:59:38.470 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-09-16T01:59:38.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:38.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-16T01:59:38.470 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-16T01:59:39.266 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-16T01:59:39.266 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-16T01:59:39.266 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-16T01:59:39.267 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-16T01:59:39.267 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-16T01:59:39.267 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-16T01:59:39.267 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T01:59:39.267 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-09-16T01:59:39.267 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-16T01:59:39.277 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-16T01:59:39.277 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:59:39.277 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T01:59:39.330 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T01:59:39.330 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T01:59:39.412 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T01:59:39.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:39 smithi139 ceph-mon[31175]: pgmap v613: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-16T01:59:39.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:39 smithi176 ceph-mon[35324]: pgmap v613: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-16T01:59:39.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:39 smithi164 ceph-mon[35121]: pgmap v613: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-16T01:59:39.946 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:41.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:41 smithi139 ceph-mon[31175]: pgmap v614: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-16T01:59:41.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:41 smithi139 ceph-mon[31175]: osdmap e120: 12 total, 12 up, 12 in 2024-09-16T01:59:41.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:41 smithi176 ceph-mon[35324]: pgmap v614: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-16T01:59:41.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:41 smithi176 ceph-mon[35324]: osdmap e120: 12 total, 12 up, 12 in 2024-09-16T01:59:41.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:41 smithi164 ceph-mon[35121]: pgmap v614: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-16T01:59:41.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:41 smithi164 ceph-mon[35121]: osdmap e120: 12 total, 12 up, 12 in 2024-09-16T01:59:42.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.078+0000 7fd199dd0640 1 -- 172.21.15.139:0/1068154458 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c0b7570 msgr2=0x7fd18c0b9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:42.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.078+0000 7fd199dd0640 1 --2- 172.21.15.139:0/1068154458 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c0b7570 0x7fd18c0b9960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd194067960 tx=0x7fd19408e420 comp rx=0 tx=0).stop 2024-09-16T01:59:42.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.081+0000 7fd199dd0640 1 -- 172.21.15.139:0/1068154458 shutdown_connections 2024-09-16T01:59:42.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.081+0000 7fd199dd0640 1 --2- 172.21.15.139:0/1068154458 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c0b7570 0x7fd18c0b9960 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.081+0000 7fd199dd0640 1 --2- 172.21.15.139:0/1068154458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd18c0a5170 0x7fd18c0b7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.081+0000 7fd199dd0640 1 --2- 172.21.15.139:0/1068154458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd18c0a4850 0x7fd18c0a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.081+0000 7fd199dd0640 1 -- 172.21.15.139:0/1068154458 >> 172.21.15.139:0/1068154458 conn(0x7fd18c01a110 msgr2=0x7fd18c01a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:42.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.083+0000 7fd199dd0640 1 -- 172.21.15.139:0/1068154458 shutdown_connections 2024-09-16T01:59:42.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.083+0000 7fd199dd0640 1 -- 172.21.15.139:0/1068154458 wait complete. 2024-09-16T01:59:42.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.083+0000 7fd199dd0640 1 Processor -- start 2024-09-16T01:59:42.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.084+0000 7fd199dd0640 1 -- start start 2024-09-16T01:59:42.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.087+0000 7fd199dd0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c0a4850 0x7fd18c0b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:42.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.088+0000 7fd199dd0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd18c0a5170 0x7fd18c0b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:42.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.088+0000 7fd199dd0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd18c0b7570 0x7fd18c0b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:42.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.088+0000 7fd199dd0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c0ad260 con 0x7fd18c0b7570 2024-09-16T01:59:42.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.088+0000 7fd199dd0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c0ad0e0 con 0x7fd18c0a5170 2024-09-16T01:59:42.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.088+0000 7fd199dd0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c0ad3e0 con 0x7fd18c0a4850 2024-09-16T01:59:42.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.088+0000 7fd1937fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c0a4850 0x7fd18c0b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:42.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.088+0000 7fd1937fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c0a4850 0x7fd18c0b6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:58074/0 (socket says 172.21.15.139:58074) 2024-09-16T01:59:42.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.088+0000 7fd1937fe640 1 -- 172.21.15.139:0/3940842672 learned_addr learned my addr 172.21.15.139:0/3940842672 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:59:42.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.088+0000 7fd1937fe640 1 -- 172.21.15.139:0/3940842672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd18c0a5170 msgr2=0x7fd18c0b0e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:59:42.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.088+0000 7fd1937fe640 1 --2- 172.21.15.139:0/3940842672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd18c0a5170 0x7fd18c0b0e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.088+0000 7fd1937fe640 1 -- 172.21.15.139:0/3940842672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd18c0b7570 msgr2=0x7fd18c0b13c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:59:42.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.088+0000 7fd1937fe640 1 --2- 172.21.15.139:0/3940842672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd18c0b7570 0x7fd18c0b13c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.088+0000 7fd1937fe640 1 -- 172.21.15.139:0/3940842672 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c0b1c80 con 0x7fd18c0a4850 2024-09-16T01:59:42.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.090+0000 7fd1937fe640 1 --2- 172.21.15.139:0/3940842672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c0a4850 0x7fd18c0b6e00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd18800d950 tx=0x7fd18800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:42.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.093+0000 7fd190ff9640 1 -- 172.21.15.139:0/3940842672 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188017070 con 0x7fd18c0a4850 2024-09-16T01:59:42.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.093+0000 7fd190ff9640 1 -- 172.21.15.139:0/3940842672 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd188009c30 con 0x7fd18c0a4850 2024-09-16T01:59:42.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.093+0000 7fd190ff9640 1 -- 172.21.15.139:0/3940842672 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18800f600 con 0x7fd18c0a4850 2024-09-16T01:59:42.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.093+0000 7fd199dd0640 1 -- 172.21.15.139:0/3940842672 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18c1582f0 con 0x7fd18c0a4850 2024-09-16T01:59:42.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.093+0000 7fd199dd0640 1 -- 172.21.15.139:0/3940842672 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18c1587b0 con 0x7fd18c0a4850 2024-09-16T01:59:42.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.095+0000 7fd17e7fc640 1 -- 172.21.15.139:0/3940842672 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd18c0a5af0 con 0x7fd18c0a4850 2024-09-16T01:59:42.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.099+0000 7fd190ff9640 1 -- 172.21.15.139:0/3940842672 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd188010460 con 0x7fd18c0a4850 2024-09-16T01:59:42.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.100+0000 7fd190ff9640 1 --2- 172.21.15.139:0/3940842672 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd1840a35c0 0x7fd1840a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:42.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.100+0000 7fd190ff9640 1 -- 172.21.15.139:0/3940842672 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9786+0+0 (secure 0 0 0) 0x7fd1880d02d0 con 0x7fd18c0a4850 2024-09-16T01:59:42.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.100+0000 7fd192ffd640 1 --2- 172.21.15.139:0/3940842672 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd1840a35c0 0x7fd1840a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:42.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.102+0000 7fd192ffd640 1 --2- 172.21.15.139:0/3940842672 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd1840a35c0 0x7fd1840a5a80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0b2640 tx=0x7fd180006bd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:42.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.105+0000 7fd190ff9640 1 -- 172.21.15.139:0/3940842672 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd188020070 con 0x7fd18c0a4850 2024-09-16T01:59:42.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.456+0000 7fd17e7fc640 1 -- 172.21.15.139:0/3940842672 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd18c0b23b0 con 0x7fd18c0a4850 2024-09-16T01:59:42.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.457+0000 7fd190ff9640 1 -- 172.21.15.139:0/3940842672 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v120) ==== 81+0+959 (secure 0 0 0) 0x7fd18809c860 con 0x7fd18c0a4850 2024-09-16T01:59:42.460 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/2901696849 2024-09-17T01:40:23.674956+0000 2024-09-16T01:59:42.460 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/2608207648 2024-09-17T01:30:44.751168+0000 2024-09-16T01:59:42.460 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/1830135963 2024-09-17T01:30:44.751168+0000 2024-09-16T01:59:42.460 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/677216867 2024-09-17T01:30:44.751168+0000 2024-09-16T01:59:42.460 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6800/2853057507 2024-09-17T01:30:44.751168+0000 2024-09-16T01:59:42.461 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/3167151595 2024-09-17T01:40:23.674956+0000 2024-09-16T01:59:42.461 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/1329877785 2024-09-17T01:40:23.674956+0000 2024-09-16T01:59:42.461 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6800/1485008061 2024-09-17T01:31:07.344357+0000 2024-09-16T01:59:42.461 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6801/331514141 2024-09-17T01:40:23.674956+0000 2024-09-16T01:59:42.461 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6801/1485008061 2024-09-17T01:31:07.344357+0000 2024-09-16T01:59:42.461 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/1608930262 2024-09-17T01:31:07.344357+0000 2024-09-16T01:59:42.461 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/2362566775 2024-09-17T01:40:23.674956+0000 2024-09-16T01:59:42.461 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6801/2853057507 2024-09-17T01:30:44.751168+0000 2024-09-16T01:59:42.461 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6800/331514141 2024-09-17T01:40:23.674956+0000 2024-09-16T01:59:42.461 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/3997627612 2024-09-17T01:31:07.344357+0000 2024-09-16T01:59:42.461 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:0/3772232169 2024-09-17T01:31:07.344357+0000 2024-09-16T01:59:42.461 INFO:teuthology.orchestra.run.smithi139.stderr:listed 16 entries 2024-09-16T01:59:42.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.468+0000 7fd17e7fc640 1 -- 172.21.15.139:0/3940842672 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd1840a35c0 msgr2=0x7fd1840a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:42.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.468+0000 7fd17e7fc640 1 --2- 172.21.15.139:0/3940842672 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd1840a35c0 0x7fd1840a5a80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0b2640 tx=0x7fd180006bd0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.468+0000 7fd17e7fc640 1 -- 172.21.15.139:0/3940842672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c0a4850 msgr2=0x7fd18c0b6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:42.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.468+0000 7fd17e7fc640 1 --2- 172.21.15.139:0/3940842672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c0a4850 0x7fd18c0b6e00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd18800d950 tx=0x7fd18800de10 comp rx=0 tx=0).stop 2024-09-16T01:59:42.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.469+0000 7fd17e7fc640 1 -- 172.21.15.139:0/3940842672 shutdown_connections 2024-09-16T01:59:42.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.469+0000 7fd17e7fc640 1 --2- 172.21.15.139:0/3940842672 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd1840a35c0 0x7fd1840a5a80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.469+0000 7fd17e7fc640 1 --2- 172.21.15.139:0/3940842672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd18c0b7570 0x7fd18c0b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.469+0000 7fd17e7fc640 1 --2- 172.21.15.139:0/3940842672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd18c0a5170 0x7fd18c0b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.469+0000 7fd17e7fc640 1 --2- 172.21.15.139:0/3940842672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c0a4850 0x7fd18c0b6e00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:42.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.469+0000 7fd17e7fc640 1 -- 172.21.15.139:0/3940842672 >> 172.21.15.139:0/3940842672 conn(0x7fd18c01a110 msgr2=0x7fd18c0b7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:42.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.473+0000 7fd17e7fc640 1 -- 172.21.15.139:0/3940842672 shutdown_connections 2024-09-16T01:59:42.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:42.475+0000 7fd17e7fc640 1 -- 172.21.15.139:0/3940842672 wait complete. 2024-09-16T01:59:43.357 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T01:59:43.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:59:43.357 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T01:59:43.531 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T01:59:43.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:59:43.531 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T01:59:43.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:59:43.593 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-16T01:59:43.657 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T01:59:43.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:59:43.658 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T01:59:43.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:43 smithi139 ceph-mon[31175]: pgmap v616: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-16T01:59:43.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:43 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3940842672' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T01:59:43.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:43 smithi176 ceph-mon[35324]: pgmap v616: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-16T01:59:43.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:43 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3940842672' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T01:59:43.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T01:59:43.731 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link delete ceph-brx 2024-09-16T01:59:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:43 smithi164 ceph-mon[35121]: pgmap v616: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-16T01:59:43.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:43 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3940842672' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T01:59:43.862 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T01:59:43.876 INFO:tasks.cephadm:Teardown begin 2024-09-16T01:59:43.876 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T01:59:43.876 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:59:44.524 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.205+0000 7fd43e125640 1 -- 172.21.15.139:0/1667894780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4381027a0 msgr2=0x7fd4381067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:45.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.206+0000 7fd43e125640 1 --2- 172.21.15.139:0/1667894780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4381027a0 0x7fd4381067f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009860 tx=0x7fd42c02a380 comp rx=0 tx=0).stop 2024-09-16T01:59:45.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.217+0000 7fd43e125640 1 -- 172.21.15.139:0/1667894780 shutdown_connections 2024-09-16T01:59:45.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.217+0000 7fd43e125640 1 --2- 172.21.15.139:0/1667894780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd438106f20 0x7fd43810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:45.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.217+0000 7fd43e125640 1 --2- 172.21.15.139:0/1667894780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4381027a0 0x7fd4381067f0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:45.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.217+0000 7fd43e125640 1 --2- 172.21.15.139:0/1667894780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd438101df0 0x7fd4381021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:45.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.217+0000 7fd43e125640 1 -- 172.21.15.139:0/1667894780 >> 172.21.15.139:0/1667894780 conn(0x7fd4380fd8b0 msgr2=0x7fd4380ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:45.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.217+0000 7fd43e125640 1 -- 172.21.15.139:0/1667894780 shutdown_connections 2024-09-16T01:59:45.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.217+0000 7fd43e125640 1 -- 172.21.15.139:0/1667894780 wait complete. 2024-09-16T01:59:45.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.218+0000 7fd43e125640 1 Processor -- start 2024-09-16T01:59:45.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.219+0000 7fd43e125640 1 -- start start 2024-09-16T01:59:45.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.220+0000 7fd43e125640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd438101df0 0x7fd4380737c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:45.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.220+0000 7fd43e125640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4381027a0 0x7fd438073d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:45.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.220+0000 7fd43e125640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd438106f20 0x7fd43806c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:45.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.220+0000 7fd43e125640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd4381120b0 con 0x7fd4381027a0 2024-09-16T01:59:45.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.220+0000 7fd43e125640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd438111f30 con 0x7fd438106f20 2024-09-16T01:59:45.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.220+0000 7fd43e125640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd438112230 con 0x7fd438101df0 2024-09-16T01:59:45.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.220+0000 7fd437fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd438106f20 0x7fd43806c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:45.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.220+0000 7fd4377fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd438101df0 0x7fd4380737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:45.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.220+0000 7fd437fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd438106f20 0x7fd43806c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:35374/0 (socket says 172.21.15.139:35374) 2024-09-16T01:59:45.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.220+0000 7fd437fff640 1 -- 172.21.15.139:0/2401108806 learned_addr learned my addr 172.21.15.139:0/2401108806 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:59:45.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.221+0000 7fd436ffd640 1 --2- 172.21.15.139:0/2401108806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4381027a0 0x7fd438073d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:45.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.224+0000 7fd437fff640 1 -- 172.21.15.139:0/2401108806 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd438101df0 msgr2=0x7fd4380737c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:45.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.224+0000 7fd437fff640 1 --2- 172.21.15.139:0/2401108806 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd438101df0 0x7fd4380737c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:45.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.224+0000 7fd437fff640 1 -- 172.21.15.139:0/2401108806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4381027a0 msgr2=0x7fd438073d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:45.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.224+0000 7fd437fff640 1 --2- 172.21.15.139:0/2401108806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4381027a0 0x7fd438073d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:45.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.224+0000 7fd437fff640 1 -- 172.21.15.139:0/2401108806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43806d110 con 0x7fd438106f20 2024-09-16T01:59:45.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.224+0000 7fd437fff640 1 --2- 172.21.15.139:0/2401108806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd438106f20 0x7fd43806c850 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd4300075b0 tx=0x7fd430007a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:45.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.230+0000 7fd434ff9640 1 -- 172.21.15.139:0/2401108806 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430013070 con 0x7fd438106f20 2024-09-16T01:59:45.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.230+0000 7fd43e125640 1 -- 172.21.15.139:0/2401108806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43806d400 con 0x7fd438106f20 2024-09-16T01:59:45.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.230+0000 7fd43e125640 1 -- 172.21.15.139:0/2401108806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4380714c0 con 0x7fd438106f20 2024-09-16T01:59:45.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.230+0000 7fd434ff9640 1 -- 172.21.15.139:0/2401108806 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd43000bbd0 con 0x7fd438106f20 2024-09-16T01:59:45.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.230+0000 7fd434ff9640 1 -- 172.21.15.139:0/2401108806 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4300147b0 con 0x7fd438106f20 2024-09-16T01:59:45.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.233+0000 7fd43e125640 1 -- 172.21.15.139:0/2401108806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd424005290 con 0x7fd438106f20 2024-09-16T01:59:45.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.240+0000 7fd434ff9640 1 -- 172.21.15.139:0/2401108806 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd43000bd80 con 0x7fd438106f20 2024-09-16T01:59:45.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.241+0000 7fd434ff9640 1 --2- 172.21.15.139:0/2401108806 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd4200a35c0 0x7fd4200a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:45.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.241+0000 7fd434ff9640 1 -- 172.21.15.139:0/2401108806 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9786+0+0 (secure 0 0 0) 0x7fd4300cf970 con 0x7fd438106f20 2024-09-16T01:59:45.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.246+0000 7fd4377fe640 1 --2- 172.21.15.139:0/2401108806 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd4200a35c0 0x7fd4200a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:45.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.247+0000 7fd4377fe640 1 --2- 172.21.15.139:0/2401108806 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd4200a35c0 0x7fd4200a5a80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd42800bbe0 tx=0x7fd42800b390 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:45.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.248+0000 7fd434ff9640 1 -- 172.21.15.139:0/2401108806 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd430023020 con 0x7fd438106f20 2024-09-16T01:59:45.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.656+0000 7fd43e125640 1 -- 172.21.15.139:0/2401108806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd424005580 con 0x7fd438106f20 2024-09-16T01:59:45.660 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:45 smithi139 ceph-mon[31175]: pgmap v617: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T01:59:45.668 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:59:45.668 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":79,"btime":"2024-09-16T01:59:33:322828+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":79,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:59:33.074705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:59:45.668 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 79 2024-09-16T01:59:45.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.666+0000 7fd434ff9640 1 -- 172.21.15.139:0/2401108806 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+12024 (secure 0 0 0) 0x7fd43001a070 con 0x7fd438106f20 2024-09-16T01:59:45.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.674+0000 7fd43e125640 1 -- 172.21.15.139:0/2401108806 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd4200a35c0 msgr2=0x7fd4200a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:45.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.674+0000 7fd43e125640 1 --2- 172.21.15.139:0/2401108806 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd4200a35c0 0x7fd4200a5a80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd42800bbe0 tx=0x7fd42800b390 comp rx=0 tx=0).stop 2024-09-16T01:59:45.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.674+0000 7fd43e125640 1 -- 172.21.15.139:0/2401108806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd438106f20 msgr2=0x7fd43806c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:45.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.674+0000 7fd43e125640 1 --2- 172.21.15.139:0/2401108806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd438106f20 0x7fd43806c850 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd4300075b0 tx=0x7fd430007a70 comp rx=0 tx=0).stop 2024-09-16T01:59:45.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.675+0000 7fd43e125640 1 -- 172.21.15.139:0/2401108806 shutdown_connections 2024-09-16T01:59:45.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.675+0000 7fd43e125640 1 --2- 172.21.15.139:0/2401108806 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7fd4200a35c0 0x7fd4200a5a80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:45.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.675+0000 7fd43e125640 1 --2- 172.21.15.139:0/2401108806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd438106f20 0x7fd43806c850 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:45.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.675+0000 7fd43e125640 1 --2- 172.21.15.139:0/2401108806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4381027a0 0x7fd438073d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:45.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.675+0000 7fd43e125640 1 --2- 172.21.15.139:0/2401108806 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd438101df0 0x7fd4380737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:45.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.675+0000 7fd43e125640 1 -- 172.21.15.139:0/2401108806 >> 172.21.15.139:0/2401108806 conn(0x7fd4380fd8b0 msgr2=0x7fd4380fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:45.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.675+0000 7fd43e125640 1 -- 172.21.15.139:0/2401108806 shutdown_connections 2024-09-16T01:59:45.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:45.675+0000 7fd43e125640 1 -- 172.21.15.139:0/2401108806 wait complete. 2024-09-16T01:59:45.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:45 smithi176 ceph-mon[35324]: pgmap v617: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T01:59:45.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:45 smithi164 ceph-mon[35121]: pgmap v617: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T01:59:46.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:46 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2401108806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:59:46.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:46 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2401108806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:59:46.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:46 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2401108806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:59:46.921 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T01:59:47.406 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:47.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:47 smithi139 ceph-mon[31175]: pgmap v618: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 614 B/s wr, 11 op/s 2024-09-16T01:59:47.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:47 smithi176 ceph-mon[35324]: pgmap v618: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 614 B/s wr, 11 op/s 2024-09-16T01:59:47.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:47 smithi164 ceph-mon[35121]: pgmap v618: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 614 B/s wr, 11 op/s 2024-09-16T01:59:48.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.067+0000 7f26b5ce0640 1 -- 172.21.15.139:0/4024336144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26b0105ec0 msgr2=0x7f26b0106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:48.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.067+0000 7f26b5ce0640 1 --2- 172.21.15.139:0/4024336144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26b0105ec0 0x7f26b0106340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009930 tx=0x7f26a802a480 comp rx=0 tx=0).stop 2024-09-16T01:59:48.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.068+0000 7f26b5ce0640 1 -- 172.21.15.139:0/4024336144 shutdown_connections 2024-09-16T01:59:48.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.068+0000 7f26b5ce0640 1 --2- 172.21.15.139:0/4024336144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26b0105ec0 0x7f26b0106340 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:48.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.068+0000 7f26b5ce0640 1 --2- 172.21.15.139:0/4024336144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26b01011e0 0x7f26b0105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:48.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.068+0000 7f26b5ce0640 1 --2- 172.21.15.139:0/4024336144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26b01008c0 0x7f26b0100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:48.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.068+0000 7f26b5ce0640 1 -- 172.21.15.139:0/4024336144 >> 172.21.15.139:0/4024336144 conn(0x7f26b00fc7d0 msgr2=0x7f26b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:48.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.075+0000 7f26b5ce0640 1 -- 172.21.15.139:0/4024336144 shutdown_connections 2024-09-16T01:59:48.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.076+0000 7f26b5ce0640 1 -- 172.21.15.139:0/4024336144 wait complete. 2024-09-16T01:59:48.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.076+0000 7f26b5ce0640 1 Processor -- start 2024-09-16T01:59:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.079+0000 7f26b5ce0640 1 -- start start 2024-09-16T01:59:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.079+0000 7f26b5ce0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26b01008c0 0x7f26b0114cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.079+0000 7f26b5ce0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26b01011e0 0x7f26b010dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.079+0000 7f26b5ce0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26b0105ec0 0x7f26b010e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.079+0000 7f26b5ce0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0118660 con 0x7f26b0105ec0 2024-09-16T01:59:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.079+0000 7f26b5ce0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01184e0 con 0x7f26b01011e0 2024-09-16T01:59:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.079+0000 7f26b5ce0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01187e0 con 0x7f26b01008c0 2024-09-16T01:59:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.080+0000 7f26affff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26b0105ec0 0x7f26b010e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.080+0000 7f26affff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26b0105ec0 0x7f26b010e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44498/0 (socket says 172.21.15.139:44498) 2024-09-16T01:59:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.080+0000 7f26affff640 1 -- 172.21.15.139:0/2950981420 learned_addr learned my addr 172.21.15.139:0/2950981420 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:59:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.081+0000 7f26affff640 1 -- 172.21.15.139:0/2950981420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26b01008c0 msgr2=0x7f26b0114cc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:59:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.081+0000 7f26affff640 1 --2- 172.21.15.139:0/2950981420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26b01008c0 0x7f26b0114cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.081+0000 7f26affff640 1 -- 172.21.15.139:0/2950981420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26b01011e0 msgr2=0x7f26b010dda0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T01:59:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.081+0000 7f26affff640 1 --2- 172.21.15.139:0/2950981420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26b01011e0 0x7f26b010dda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.081+0000 7f26affff640 1 -- 172.21.15.139:0/2950981420 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b010e8a0 con 0x7f26b0105ec0 2024-09-16T01:59:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.082+0000 7f26affff640 1 --2- 172.21.15.139:0/2950981420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26b0105ec0 0x7f26b010e2e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f26a8005ab0 tx=0x7f26a8005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:48.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.082+0000 7f26acff9640 1 -- 172.21.15.139:0/2950981420 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8040030 con 0x7f26b0105ec0 2024-09-16T01:59:48.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.082+0000 7f26b5ce0640 1 -- 172.21.15.139:0/2950981420 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b010eb30 con 0x7f26b0105ec0 2024-09-16T01:59:48.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.082+0000 7f26b5ce0640 1 -- 172.21.15.139:0/2950981420 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b0112a70 con 0x7f26b0105ec0 2024-09-16T01:59:48.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.083+0000 7f26acff9640 1 -- 172.21.15.139:0/2950981420 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a8005400 con 0x7f26b0105ec0 2024-09-16T01:59:48.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.083+0000 7f26acff9640 1 -- 172.21.15.139:0/2950981420 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a803c830 con 0x7f26b0105ec0 2024-09-16T01:59:48.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.085+0000 7f26b5ce0640 1 -- 172.21.15.139:0/2950981420 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2690005290 con 0x7f26b0105ec0 2024-09-16T01:59:48.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.087+0000 7f26acff9640 1 -- 172.21.15.139:0/2950981420 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f26a8043030 con 0x7f26b0105ec0 2024-09-16T01:59:48.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.088+0000 7f26acff9640 1 --2- 172.21.15.139:0/2950981420 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f269c0a3580 0x7f269c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:48.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.088+0000 7f26acff9640 1 -- 172.21.15.139:0/2950981420 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9786+0+0 (secure 0 0 0) 0x7f26a80f0750 con 0x7f26b0105ec0 2024-09-16T01:59:48.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.094+0000 7f26af7fe640 1 --2- 172.21.15.139:0/2950981420 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f269c0a3580 0x7f269c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:48.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.096+0000 7f26acff9640 1 -- 172.21.15.139:0/2950981420 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f26a80bcbe0 con 0x7f26b0105ec0 2024-09-16T01:59:48.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.099+0000 7f26af7fe640 1 --2- 172.21.15.139:0/2950981420 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f269c0a3580 0x7f269c0a5a40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f26a0005c40 tx=0x7f26a0005bd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:48.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.493+0000 7f26b5ce0640 1 -- 172.21.15.139:0/2950981420 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2690005850 con 0x7f26b0105ec0 2024-09-16T01:59:48.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.498+0000 7f26acff9640 1 -- 172.21.15.139:0/2950981420 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+12024 (secure 0 0 0) 0x7f26a80bc4e0 con 0x7f26b0105ec0 2024-09-16T01:59:48.500 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:59:48.500 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":79,"btime":"2024-09-16T01:59:33:322828+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/129107170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":129107170},{"type":"v1","addr":"172.21.15.139:6835","nonce":129107170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":26309,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2728173014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2728173014},{"type":"v1","addr":"172.21.15.164:6835","nonce":2728173014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":79,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T01:38:02.129081+0000","modified":"2024-09-16T01:59:33.074705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24454,"mds_2":24443,"mds_3":24442,"mds_4":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/465602995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":465602995},{"type":"v1","addr":"172.21.15.139:6837","nonce":465602995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6839/1075273450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":1075273450},{"type":"v1","addr":"172.21.15.139:6839","nonce":1075273450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6841/1887551794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6840","nonce":1887551794},{"type":"v1","addr":"172.21.15.139:6841","nonce":1887551794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/416925170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":416925170},{"type":"v1","addr":"172.21.15.164:6833","nonce":416925170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6833/1026776933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1026776933},{"type":"v1","addr":"172.21.15.176:6833","nonce":1026776933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.176:6835/3060423566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":3060423566},{"type":"v1","addr":"172.21.15.176:6835","nonce":3060423566}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.176:6837/1942543693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1942543693},{"type":"v1","addr":"172.21.15.176:6837","nonce":1942543693}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.164:6837/1270226509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1270226509},{"type":"v1","addr":"172.21.15.164:6837","nonce":1270226509}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.176:6839/3912805068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3912805068},{"type":"v1","addr":"172.21.15.176:6839","nonce":3912805068}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.164:6839/2447189436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2447189436},{"type":"v1","addr":"172.21.15.164:6839","nonce":2447189436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24433,24442,24443,24454,24470]},"id":1}]} 2024-09-16T01:59:48.500 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 79 2024-09-16T01:59:48.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.504+0000 7f26b5ce0640 1 -- 172.21.15.139:0/2950981420 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f269c0a3580 msgr2=0x7f269c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:48.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.504+0000 7f26b5ce0640 1 --2- 172.21.15.139:0/2950981420 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f269c0a3580 0x7f269c0a5a40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f26a0005c40 tx=0x7f26a0005bd0 comp rx=0 tx=0).stop 2024-09-16T01:59:48.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.504+0000 7f26b5ce0640 1 -- 172.21.15.139:0/2950981420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26b0105ec0 msgr2=0x7f26b010e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:48.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.504+0000 7f26b5ce0640 1 --2- 172.21.15.139:0/2950981420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26b0105ec0 0x7f26b010e2e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f26a8005ab0 tx=0x7f26a8005000 comp rx=0 tx=0).stop 2024-09-16T01:59:48.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.506+0000 7f26b5ce0640 1 -- 172.21.15.139:0/2950981420 shutdown_connections 2024-09-16T01:59:48.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.506+0000 7f26b5ce0640 1 --2- 172.21.15.139:0/2950981420 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f269c0a3580 0x7f269c0a5a40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:48.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.506+0000 7f26b5ce0640 1 --2- 172.21.15.139:0/2950981420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26b0105ec0 0x7f26b010e2e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:48.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.506+0000 7f26b5ce0640 1 --2- 172.21.15.139:0/2950981420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26b01011e0 0x7f26b010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:48.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.506+0000 7f26b5ce0640 1 --2- 172.21.15.139:0/2950981420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26b01008c0 0x7f26b0114cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:48.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.506+0000 7f26b5ce0640 1 -- 172.21.15.139:0/2950981420 >> 172.21.15.139:0/2950981420 conn(0x7f26b00fc7d0 msgr2=0x7f26b0103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:48.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.509+0000 7f26b5ce0640 1 -- 172.21.15.139:0/2950981420 shutdown_connections 2024-09-16T01:59:48.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:48.509+0000 7f26b5ce0640 1 -- 172.21.15.139:0/2950981420 wait complete. 2024-09-16T01:59:49.256 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T01:59:49.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:49 smithi139 ceph-mon[31175]: pgmap v619: 129 pgs: 22 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 42 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 716 B/s wr, 10 op/s 2024-09-16T01:59:49.675 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:49 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/2950981420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:59:49.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:49 smithi176 ceph-mon[35324]: pgmap v619: 129 pgs: 22 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 42 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 716 B/s wr, 10 op/s 2024-09-16T01:59:49.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:49 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/2950981420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:59:49.766 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:49.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:49 smithi164 ceph-mon[35121]: pgmap v619: 129 pgs: 22 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 42 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 716 B/s wr, 10 op/s 2024-09-16T01:59:49.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:49 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/2950981420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T01:59:50.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.390+0000 7f90efefd640 1 -- 172.21.15.139:0/3177179765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e8114ab0 msgr2=0x7f90e8116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:50.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.390+0000 7f90efefd640 1 --2- 172.21.15.139:0/3177179765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e8114ab0 0x7f90e8116ea0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009ca0 tx=0x7f90e002a980 comp rx=0 tx=0).stop 2024-09-16T01:59:50.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.391+0000 7f90efefd640 1 -- 172.21.15.139:0/3177179765 shutdown_connections 2024-09-16T01:59:50.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.391+0000 7f90efefd640 1 --2- 172.21.15.139:0/3177179765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e8114ab0 0x7f90e8116ea0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:50.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.391+0000 7f90efefd640 1 --2- 172.21.15.139:0/3177179765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90e8109b10 0x7f90e8114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:50.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.391+0000 7f90efefd640 1 --2- 172.21.15.139:0/3177179765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90e81091f0 0x7f90e81095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:50.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.391+0000 7f90efefd640 1 -- 172.21.15.139:0/3177179765 >> 172.21.15.139:0/3177179765 conn(0x7f90e806e140 msgr2=0x7f90e806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:50.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.400+0000 7f90efefd640 1 -- 172.21.15.139:0/3177179765 shutdown_connections 2024-09-16T01:59:50.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.400+0000 7f90efefd640 1 -- 172.21.15.139:0/3177179765 wait complete. 2024-09-16T01:59:50.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.401+0000 7f90efefd640 1 Processor -- start 2024-09-16T01:59:50.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.401+0000 7f90efefd640 1 -- start start 2024-09-16T01:59:50.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.401+0000 7f90efefd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e81091f0 0x7f90e81135e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:50.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.401+0000 7f90efefd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90e8109b10 0x7f90e8113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:50.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.401+0000 7f90efefd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90e8114ab0 0x7f90e811f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:50.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.401+0000 7f90efefd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f90e8118630 con 0x7f90e81091f0 2024-09-16T01:59:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.401+0000 7f90efefd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81184b0 con 0x7f90e8114ab0 2024-09-16T01:59:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.401+0000 7f90efefd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81187b0 con 0x7f90e8109b10 2024-09-16T01:59:50.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.405+0000 7f90edc99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e81091f0 0x7f90e81135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:50.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.405+0000 7f90edc99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e81091f0 0x7f90e81135e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44516/0 (socket says 172.21.15.139:44516) 2024-09-16T01:59:50.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.405+0000 7f90edc99640 1 -- 172.21.15.139:0/1439426348 learned_addr learned my addr 172.21.15.139:0/1439426348 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:59:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.410+0000 7f90ee49a640 1 --2- 172.21.15.139:0/1439426348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90e8114ab0 0x7f90e811f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.410+0000 7f90ed498640 1 --2- 172.21.15.139:0/1439426348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90e8109b10 0x7f90e8113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.410+0000 7f90ee49a640 1 -- 172.21.15.139:0/1439426348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90e8109b10 msgr2=0x7f90e8113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.410+0000 7f90ee49a640 1 --2- 172.21.15.139:0/1439426348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90e8109b10 0x7f90e8113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.410+0000 7f90ee49a640 1 -- 172.21.15.139:0/1439426348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e81091f0 msgr2=0x7f90e81135e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.410+0000 7f90ee49a640 1 --2- 172.21.15.139:0/1439426348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e81091f0 0x7f90e81135e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.410+0000 7f90ee49a640 1 -- 172.21.15.139:0/1439426348 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e811ffa0 con 0x7f90e8114ab0 2024-09-16T01:59:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.411+0000 7f90ee49a640 1 --2- 172.21.15.139:0/1439426348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90e8114ab0 0x7f90e811f870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f90e002af70 tx=0x7f90e0030cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:50.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.414+0000 7f90daffd640 1 -- 172.21.15.139:0/1439426348 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0047070 con 0x7f90e8114ab0 2024-09-16T01:59:50.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.414+0000 7f90efefd640 1 -- 172.21.15.139:0/1439426348 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e8120230 con 0x7f90e8114ab0 2024-09-16T01:59:50.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.414+0000 7f90efefd640 1 -- 172.21.15.139:0/1439426348 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e8122940 con 0x7f90e8114ab0 2024-09-16T01:59:50.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.414+0000 7f90daffd640 1 -- 172.21.15.139:0/1439426348 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e002f6e0 con 0x7f90e8114ab0 2024-09-16T01:59:50.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.414+0000 7f90daffd640 1 -- 172.21.15.139:0/1439426348 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0034040 con 0x7f90e8114ab0 2024-09-16T01:59:50.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.416+0000 7f90efefd640 1 -- 172.21.15.139:0/1439426348 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90cc005290 con 0x7f90e8114ab0 2024-09-16T01:59:50.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.420+0000 7f90daffd640 1 -- 172.21.15.139:0/1439426348 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f90e0037020 con 0x7f90e8114ab0 2024-09-16T01:59:50.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.421+0000 7f90daffd640 1 --2- 172.21.15.139:0/1439426348 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90d40a3580 0x7f90d40a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:50.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.422+0000 7f90daffd640 1 -- 172.21.15.139:0/1439426348 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9786+0+0 (secure 0 0 0) 0x7f90e00f0770 con 0x7f90e8114ab0 2024-09-16T01:59:50.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.424+0000 7f90edc99640 1 --2- 172.21.15.139:0/1439426348 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90d40a3580 0x7f90d40a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:50.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.425+0000 7f90daffd640 1 -- 172.21.15.139:0/1439426348 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90e00bc7d0 con 0x7f90e8114ab0 2024-09-16T01:59:50.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.435+0000 7f90edc99640 1 --2- 172.21.15.139:0/1439426348 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90d40a3580 0x7f90d40a5a40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009a30 tx=0x7f90e4009190 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:50.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.758+0000 7f90efefd640 1 -- 172.21.15.139:0/1439426348 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90cc005850 con 0x7f90e8114ab0 2024-09-16T01:59:50.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.763+0000 7f90daffd640 1 -- 172.21.15.139:0/1439426348 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+25908 (secure 0 0 0) 0x7f90e00c0ad0 con 0x7f90e8114ab0 2024-09-16T01:59:50.765 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T01:59:50.766 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":120,"fsid":"1d90dd3a-73cb-11ef-bceb-c7b262605968","created":"2024-09-16T01:30:17.770313+0000","modified":"2024-09-16T01:59:41.137051+0000","last_up_change":"2024-09-16T01:37:08.611901+0000","last_in_change":"2024-09-16T01:36:45.223926+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T01:33:19.639647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T01:37:57.948214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T01:38:00.263797+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T01:38:06.773799+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b66696da-ef66-4f25-9527-bbcc9ded2cee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6803","nonce":3190810133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6805","nonce":3190810133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6809","nonce":3190810133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3190810133},{"type":"v1","addr":"172.21.15.139:6807","nonce":3190810133}]},"public_addr":"172.21.15.139:6803/3190810133","cluster_addr":"172.21.15.139:6805/3190810133","heartbeat_back_addr":"172.21.15.139:6809/3190810133","heartbeat_front_addr":"172.21.15.139:6807/3190810133","state":["exists","up"]},{"osd":1,"uuid":"ccad17f2-0750-425e-9810-49a13a929ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6801","nonce":2699270330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6803","nonce":2699270330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6807","nonce":2699270330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2699270330},{"type":"v1","addr":"172.21.15.164:6805","nonce":2699270330}]},"public_addr":"172.21.15.164:6801/2699270330","cluster_addr":"172.21.15.164:6803/2699270330","heartbeat_back_addr":"172.21.15.164:6807/2699270330","heartbeat_front_addr":"172.21.15.164:6805/2699270330","state":["exists","up"]},{"osd":2,"uuid":"baf0ba73-8faf-4e51-a171-c47898800638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6801","nonce":2034471646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6803","nonce":2034471646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6807","nonce":2034471646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2034471646},{"type":"v1","addr":"172.21.15.176:6805","nonce":2034471646}]},"public_addr":"172.21.15.176:6801/2034471646","cluster_addr":"172.21.15.176:6803/2034471646","heartbeat_back_addr":"172.21.15.176:6807/2034471646","heartbeat_front_addr":"172.21.15.176:6805/2034471646","state":["exists","up"]},{"osd":3,"uuid":"98957bf1-67bb-4092-9ea4-b263a5706f8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6811","nonce":797147071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6813","nonce":797147071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6817","nonce":797147071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":797147071},{"type":"v1","addr":"172.21.15.139:6815","nonce":797147071}]},"public_addr":"172.21.15.139:6811/797147071","cluster_addr":"172.21.15.139:6813/797147071","heartbeat_back_addr":"172.21.15.139:6817/797147071","heartbeat_front_addr":"172.21.15.139:6815/797147071","state":["exists","up"]},{"osd":4,"uuid":"9cd76696-8161-4e82-a571-29e98c11d5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6809","nonce":1421304792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6811","nonce":1421304792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6815","nonce":1421304792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1421304792},{"type":"v1","addr":"172.21.15.164:6813","nonce":1421304792}]},"public_addr":"172.21.15.164:6809/1421304792","cluster_addr":"172.21.15.164:6811/1421304792","heartbeat_back_addr":"172.21.15.164:6815/1421304792","heartbeat_front_addr":"172.21.15.164:6813/1421304792","state":["exists","up"]},{"osd":5,"uuid":"ccc17a29-9a58-40ec-b453-5c38bee86f25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6809","nonce":10600602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6811","nonce":10600602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6815","nonce":10600602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":10600602},{"type":"v1","addr":"172.21.15.176:6813","nonce":10600602}]},"public_addr":"172.21.15.176:6809/10600602","cluster_addr":"172.21.15.176:6811/10600602","heartbeat_back_addr":"172.21.15.176:6815/10600602","heartbeat_front_addr":"172.21.15.176:6813/10600602","state":["exists","up"]},{"osd":6,"uuid":"e5ee7212-10c6-4a05-992e-622a14a5a6b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6819","nonce":2601111164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6821","nonce":2601111164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6825","nonce":2601111164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2601111164},{"type":"v1","addr":"172.21.15.139:6823","nonce":2601111164}]},"public_addr":"172.21.15.139:6819/2601111164","cluster_addr":"172.21.15.139:6821/2601111164","heartbeat_back_addr":"172.21.15.139:6825/2601111164","heartbeat_front_addr":"172.21.15.139:6823/2601111164","state":["exists","up"]},{"osd":7,"uuid":"23143c15-0839-4fd5-9d0a-19ae891000f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6817","nonce":2074080693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6819","nonce":2074080693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6823","nonce":2074080693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2074080693},{"type":"v1","addr":"172.21.15.164:6821","nonce":2074080693}]},"public_addr":"172.21.15.164:6817/2074080693","cluster_addr":"172.21.15.164:6819/2074080693","heartbeat_back_addr":"172.21.15.164:6823/2074080693","heartbeat_front_addr":"172.21.15.164:6821/2074080693","state":["exists","up"]},{"osd":8,"uuid":"1c14f1e6-724e-4e3d-a9c3-fde25cdba0a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6817","nonce":593133361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6819","nonce":593133361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6823","nonce":593133361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":593133361},{"type":"v1","addr":"172.21.15.176:6821","nonce":593133361}]},"public_addr":"172.21.15.176:6817/593133361","cluster_addr":"172.21.15.176:6819/593133361","heartbeat_back_addr":"172.21.15.176:6823/593133361","heartbeat_front_addr":"172.21.15.176:6821/593133361","state":["exists","up"]},{"osd":9,"uuid":"09d25a3b-1e66-480e-9433-ceef7f0b9279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6827","nonce":3047942108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6829","nonce":3047942108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6833","nonce":3047942108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3047942108},{"type":"v1","addr":"172.21.15.139:6831","nonce":3047942108}]},"public_addr":"172.21.15.139:6827/3047942108","cluster_addr":"172.21.15.139:6829/3047942108","heartbeat_back_addr":"172.21.15.139:6833/3047942108","heartbeat_front_addr":"172.21.15.139:6831/3047942108","state":["exists","up"]},{"osd":10,"uuid":"32b4d8ce-b644-42fa-80cc-a15a1f5f3a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6825","nonce":4172533828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6827","nonce":4172533828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6831","nonce":4172533828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":4172533828},{"type":"v1","addr":"172.21.15.164:6829","nonce":4172533828}]},"public_addr":"172.21.15.164:6825/4172533828","cluster_addr":"172.21.15.164:6827/4172533828","heartbeat_back_addr":"172.21.15.164:6831/4172533828","heartbeat_front_addr":"172.21.15.164:6829/4172533828","state":["exists","up"]},{"osd":11,"uuid":"49098bdc-58a9-4a74-8045-d408af7711ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6825","nonce":1854547565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6827","nonce":1854547565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6831","nonce":1854547565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":1854547565},{"type":"v1","addr":"172.21.15.176:6829","nonce":1854547565}]},"public_addr":"172.21.15.176:6825/1854547565","cluster_addr":"172.21.15.176:6827/1854547565","heartbeat_back_addr":"172.21.15.176:6831/1854547565","heartbeat_front_addr":"172.21.15.176:6829/1854547565","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:27.374845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:32:52.611853+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:17.229567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:33:43.330503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:07.988876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:33.787864+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:34:58.864910+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:23.815006+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:35:49.551954+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:15.496773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:36:41.647557+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T01:37:07.089355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2901696849":"2024-09-17T01:40:23.674956+0000","172.21.15.139:0/2608207648":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/1830135963":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/677216867":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6800/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:0/3167151595":"2024-09-17T01:40:23.674956+0000","172.21.15.139:0/1329877785":"2024-09-17T01:40:23.674956+0000","172.21.15.139:6800/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:6801/331514141":"2024-09-17T01:40:23.674956+0000","172.21.15.139:6801/1485008061":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/1608930262":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/2362566775":"2024-09-17T01:40:23.674956+0000","172.21.15.139:6801/2853057507":"2024-09-17T01:30:44.751168+0000","172.21.15.139:6800/331514141":"2024-09-17T01:40:23.674956+0000","172.21.15.139:0/3997627612":"2024-09-17T01:31:07.344357+0000","172.21.15.139:0/3772232169":"2024-09-17T01:31:07.344357+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":20,"length":2},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":1},{"begin":31,"length":1},{"begin":33,"length":1}]},{"pool":4,"snaps":[{"begin":18,"length":1},{"begin":20,"length":2},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":29,"length":1},{"begin":31,"length":1},{"begin":33,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1}]},{"pool":4,"snaps":[{"begin":16,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T01:59:50.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.767+0000 7f90efefd640 1 -- 172.21.15.139:0/1439426348 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90d40a3580 msgr2=0x7f90d40a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:50.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.767+0000 7f90efefd640 1 --2- 172.21.15.139:0/1439426348 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90d40a3580 0x7f90d40a5a40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009a30 tx=0x7f90e4009190 comp rx=0 tx=0).stop 2024-09-16T01:59:50.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.767+0000 7f90efefd640 1 -- 172.21.15.139:0/1439426348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90e8114ab0 msgr2=0x7f90e811f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:50.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.767+0000 7f90efefd640 1 --2- 172.21.15.139:0/1439426348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90e8114ab0 0x7f90e811f870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f90e002af70 tx=0x7f90e0030cb0 comp rx=0 tx=0).stop 2024-09-16T01:59:50.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.770+0000 7f90efefd640 1 -- 172.21.15.139:0/1439426348 shutdown_connections 2024-09-16T01:59:50.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.770+0000 7f90efefd640 1 --2- 172.21.15.139:0/1439426348 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90d40a3580 0x7f90d40a5a40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:50.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.770+0000 7f90efefd640 1 --2- 172.21.15.139:0/1439426348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90e8114ab0 0x7f90e811f870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:50.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.770+0000 7f90efefd640 1 --2- 172.21.15.139:0/1439426348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90e8109b10 0x7f90e8113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:50.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.770+0000 7f90efefd640 1 --2- 172.21.15.139:0/1439426348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e81091f0 0x7f90e81135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:50.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.770+0000 7f90efefd640 1 -- 172.21.15.139:0/1439426348 >> 172.21.15.139:0/1439426348 conn(0x7f90e806e140 msgr2=0x7f90e80727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:50.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.773+0000 7f90efefd640 1 -- 172.21.15.139:0/1439426348 shutdown_connections 2024-09-16T01:59:50.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:50.775+0000 7f90efefd640 1 -- 172.21.15.139:0/1439426348 wait complete. 2024-09-16T01:59:51.535 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-16T01:59:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:51 smithi164 ceph-mon[35121]: pgmap v620: 129 pgs: 22 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 42 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-16T01:59:51.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:51 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1439426348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:59:51.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:51 smithi139 ceph-mon[31175]: pgmap v620: 129 pgs: 22 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 42 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-16T01:59:51.924 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:51 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1439426348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:59:51.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:51 smithi176 ceph-mon[35324]: pgmap v620: 129 pgs: 22 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 42 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-16T01:59:51.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:51 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1439426348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T01:59:52.019 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:52.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.676+0000 7f98498d8640 1 -- 172.21.15.139:0/3627806501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f984410d690 msgr2=0x7f984410db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:52.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.676+0000 7f98498d8640 1 --2- 172.21.15.139:0/3627806501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f984410d690 0x7f984410db40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f983c009900 tx=0x7f983c02a440 comp rx=0 tx=0).stop 2024-09-16T01:59:52.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.677+0000 7f98498d8640 1 -- 172.21.15.139:0/3627806501 shutdown_connections 2024-09-16T01:59:52.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.677+0000 7f98498d8640 1 --2- 172.21.15.139:0/3627806501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f984410d690 0x7f984410db40 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:52.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.677+0000 7f98498d8640 1 --2- 172.21.15.139:0/3627806501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9844073ca0 0x7f9844109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:52.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.677+0000 7f98498d8640 1 --2- 172.21.15.139:0/3627806501 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9844073830 0x7f9844109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:52.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.677+0000 7f98498d8640 1 -- 172.21.15.139:0/3627806501 >> 172.21.15.139:0/3627806501 conn(0x7f984406e140 msgr2=0x7f984406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:52.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.681+0000 7f98498d8640 1 -- 172.21.15.139:0/3627806501 shutdown_connections 2024-09-16T01:59:52.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.681+0000 7f98498d8640 1 -- 172.21.15.139:0/3627806501 wait complete. 2024-09-16T01:59:52.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.683+0000 7f98498d8640 1 Processor -- start 2024-09-16T01:59:52.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.683+0000 7f98498d8640 1 -- start start 2024-09-16T01:59:52.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.683+0000 7f98498d8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9844073830 0x7f984411ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:52.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.683+0000 7f98498d8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9844073ca0 0x7f9844119b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:52.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.683+0000 7f98498d8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f984410d690 0x7f984411a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:52.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.684+0000 7f9842ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9844073830 0x7f984411ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:52.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.684+0000 7f9842ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9844073830 0x7f984411ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:59438/0 (socket says 172.21.15.139:59438) 2024-09-16T01:59:52.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.684+0000 7f9842ffd640 1 -- 172.21.15.139:0/1584161296 learned_addr learned my addr 172.21.15.139:0/1584161296 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:59:52.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.684+0000 7f98498d8640 1 -- 172.21.15.139:0/1584161296 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9844116660 con 0x7f9844073ca0 2024-09-16T01:59:52.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.684+0000 7f98498d8640 1 -- 172.21.15.139:0/1584161296 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f98441164e0 con 0x7f9844073830 2024-09-16T01:59:52.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.684+0000 7f98498d8640 1 -- 172.21.15.139:0/1584161296 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f98441167e0 con 0x7f984410d690 2024-09-16T01:59:52.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.687+0000 7f98427fc640 1 --2- 172.21.15.139:0/1584161296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9844073ca0 0x7f9844119b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:52.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.688+0000 7f9842ffd640 1 -- 172.21.15.139:0/1584161296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f984410d690 msgr2=0x7f984411a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:52.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.688+0000 7f9842ffd640 1 --2- 172.21.15.139:0/1584161296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f984410d690 0x7f984411a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:52.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.688+0000 7f9842ffd640 1 -- 172.21.15.139:0/1584161296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9844073ca0 msgr2=0x7f9844119b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:52.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.688+0000 7f9842ffd640 1 --2- 172.21.15.139:0/1584161296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9844073ca0 0x7f9844119b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:52.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.688+0000 7f9842ffd640 1 -- 172.21.15.139:0/1584161296 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984411a920 con 0x7f9844073830 2024-09-16T01:59:52.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.693+0000 7f9842ffd640 1 --2- 172.21.15.139:0/1584161296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9844073830 0x7f984411ca00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f982c00b450 tx=0x7f982c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:52.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.694+0000 7f98488d6640 1 -- 172.21.15.139:0/1584161296 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c015020 con 0x7f9844073830 2024-09-16T01:59:52.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.694+0000 7f98488d6640 1 -- 172.21.15.139:0/1584161296 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f982c011d70 con 0x7f9844073830 2024-09-16T01:59:52.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.694+0000 7f98498d8640 1 -- 172.21.15.139:0/1584161296 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98441b3710 con 0x7f9844073830 2024-09-16T01:59:52.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.694+0000 7f98498d8640 1 -- 172.21.15.139:0/1584161296 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98441b3c50 con 0x7f9844073830 2024-09-16T01:59:52.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.695+0000 7f98488d6640 1 -- 172.21.15.139:0/1584161296 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c00f340 con 0x7f9844073830 2024-09-16T01:59:52.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.697+0000 7f98498d8640 1 -- 172.21.15.139:0/1584161296 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9824005290 con 0x7f9844073830 2024-09-16T01:59:52.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.701+0000 7f98488d6640 1 -- 172.21.15.139:0/1584161296 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f982c018020 con 0x7f9844073830 2024-09-16T01:59:52.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.702+0000 7f98488d6640 1 --2- 172.21.15.139:0/1584161296 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f98300a34c0 0x7f98300a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:52.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.702+0000 7f98488d6640 1 -- 172.21.15.139:0/1584161296 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9786+0+0 (secure 0 0 0) 0x7f982c009200 con 0x7f9844073830 2024-09-16T01:59:52.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.704+0000 7f98427fc640 1 --2- 172.21.15.139:0/1584161296 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f98300a34c0 0x7f98300a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:52.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.705+0000 7f98427fc640 1 --2- 172.21.15.139:0/1584161296 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f98300a34c0 0x7f98300a5980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f984411b270 tx=0x7f98340094f0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:52.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:52.707+0000 7f98488d6640 1 -- 172.21.15.139:0/1584161296 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f982c01f020 con 0x7f9844073830 2024-09-16T01:59:53.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:53.106+0000 7f98498d8640 1 -- 172.21.15.139:0/1584161296 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9824005850 con 0x7f9844073830 2024-09-16T01:59:53.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:53.127+0000 7f98488d6640 1 -- 172.21.15.139:0/1584161296 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) ==== 161+0+0 (secure 0 0 0) 0x7f982c00d040 con 0x7f9844073830 2024-09-16T01:59:53.129 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T01:59:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:53.131+0000 7f983a7fc640 1 -- 172.21.15.139:0/1584161296 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f98300a34c0 msgr2=0x7f98300a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:53.131+0000 7f983a7fc640 1 --2- 172.21.15.139:0/1584161296 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f98300a34c0 0x7f98300a5980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f984411b270 tx=0x7f98340094f0 comp rx=0 tx=0).stop 2024-09-16T01:59:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:53.131+0000 7f983a7fc640 1 -- 172.21.15.139:0/1584161296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9844073830 msgr2=0x7f984411ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:53.131+0000 7f983a7fc640 1 --2- 172.21.15.139:0/1584161296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9844073830 0x7f984411ca00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f982c00b450 tx=0x7f982c00b910 comp rx=0 tx=0).stop 2024-09-16T01:59:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:53.131+0000 7f983a7fc640 1 -- 172.21.15.139:0/1584161296 shutdown_connections 2024-09-16T01:59:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:53.131+0000 7f983a7fc640 1 --2- 172.21.15.139:0/1584161296 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f98300a34c0 0x7f98300a5980 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:53.131+0000 7f983a7fc640 1 --2- 172.21.15.139:0/1584161296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f984410d690 0x7f984411a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:53.131+0000 7f983a7fc640 1 --2- 172.21.15.139:0/1584161296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9844073ca0 0x7f9844119b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:53.131+0000 7f983a7fc640 1 --2- 172.21.15.139:0/1584161296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9844073830 0x7f984411ca00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:53.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:53.131+0000 7f983a7fc640 1 -- 172.21.15.139:0/1584161296 >> 172.21.15.139:0/1584161296 conn(0x7f984406e140 msgr2=0x7f984410c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:53.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:53.131+0000 7f983a7fc640 1 -- 172.21.15.139:0/1584161296 shutdown_connections 2024-09-16T01:59:53.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:53.133+0000 7f983a7fc640 1 -- 172.21.15.139:0/1584161296 wait complete. 2024-09-16T01:59:53.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a[31171]: 2024-09-16T01:59:53.108+0000 7f5cf3bd9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T01:59:53.423 INFO:journalctl@ceph.mds.a.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-a[68838]: ignoring --setuser ceph since I am not root 2024-09-16T01:59:53.423 INFO:journalctl@ceph.mds.a.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-a[68838]: ignoring --setgroup ceph since I am not root 2024-09-16T01:59:53.423 INFO:journalctl@ceph.mds.a.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-mds[68844]: -- 172.21.15.139:0/961282231 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a5b35ac680 con 0x55a5b3581000 2024-09-16T01:59:53.423 INFO:journalctl@ceph.mds.a.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-mds[68844]: -- 172.21.15.139:0/961282231 <== mon.0 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a5b35a3860 con 0x55a5b3581000 2024-09-16T01:59:53.424 INFO:journalctl@ceph.mds.a.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-a[68838]: starting mds.a at 2024-09-16T01:59:53.424 INFO:journalctl@ceph.mds.d.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-d[67655]: ignoring --setuser ceph since I am not root 2024-09-16T01:59:53.424 INFO:journalctl@ceph.mds.d.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-d[67655]: ignoring --setgroup ceph since I am not root 2024-09-16T01:59:53.424 INFO:journalctl@ceph.mds.d.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-d[67655]: starting mds.d at 2024-09-16T01:59:53.424 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-g[68430]: ignoring --setuser ceph since I am not root 2024-09-16T01:59:53.424 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-g[68430]: ignoring --setgroup ceph since I am not root 2024-09-16T01:59:53.424 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-mds[68434]: -- 172.21.15.139:0/2050162211 <== mon.0 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c15de3860 con 0x562c15dc1000 2024-09-16T01:59:53.424 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-g[68430]: starting mds.g at 2024-09-16T01:59:53.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-mon[35121]: pgmap v621: 129 pgs: 21 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 43 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-16T01:59:53.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1584161296' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T01:59:53.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-mon[35121]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T01:59:53.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-mon[35121]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T01:59:53.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-mon[35121]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T01:59:53.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-mon[35121]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T01:59:53.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-mon[35121]: osdmap e121: 12 total, 12 up, 12 in 2024-09-16T01:59:53.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-mon[35121]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-16T01:59:53.546 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-h[64337]: ignoring --setuser ceph since I am not root 2024-09-16T01:59:53.546 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-h[64337]: ignoring --setgroup ceph since I am not root 2024-09-16T01:59:53.547 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-mds[64341]: -- 172.21.15.164:0/2090109455 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56318f502680 con 0x56318f4d6800 2024-09-16T01:59:53.547 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-h[64337]: starting mds.h at 2024-09-16T01:59:53.547 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-k[63720]: ignoring --setuser ceph since I am not root 2024-09-16T01:59:53.547 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-k[63720]: ignoring --setgroup ceph since I am not root 2024-09-16T01:59:53.547 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-k[63720]: starting mds.k at 2024-09-16T01:59:53.547 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b[64648]: ignoring --setuser ceph since I am not root 2024-09-16T01:59:53.547 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b[64648]: ignoring --setgroup ceph since I am not root 2024-09-16T01:59:53.548 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-mds[64652]: -- 172.21.15.164:0/1099600259 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c8979e680 con 0x556c89772800 2024-09-16T01:59:53.548 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-mds[64652]: -- 172.21.15.164:0/1099600259 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c89795860 con 0x556c89772800 2024-09-16T01:59:53.548 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 01:59:53 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b[64648]: starting mds.b at 2024-09-16T01:59:53.901 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-mon[31175]: pgmap v621: 129 pgs: 21 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 43 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-16T01:59:53.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1584161296' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T01:59:53.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-mon[31175]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T01:59:53.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-mon[31175]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T01:59:53.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-mon[31175]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T01:59:53.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-mon[31175]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T01:59:53.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-mon[31175]: osdmap e121: 12 total, 12 up, 12 in 2024-09-16T01:59:53.902 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:53 smithi139 ceph-mon[31175]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-16T01:59:53.937 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-f[64955]: ignoring --setuser ceph since I am not root 2024-09-16T01:59:53.937 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-f[64955]: ignoring --setgroup ceph since I am not root 2024-09-16T01:59:53.937 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-f[64955]: starting mds.f at 2024-09-16T01:59:53.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-mon[35324]: pgmap v621: 129 pgs: 21 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 43 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-16T01:59:53.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1584161296' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T01:59:53.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-mon[35324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T01:59:53.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-mon[35324]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T01:59:53.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-mon[35324]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T01:59:53.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-mon[35324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T01:59:53.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-mon[35324]: osdmap e121: 12 total, 12 up, 12 in 2024-09-16T01:59:53.938 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-mon[35324]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-16T01:59:53.938 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-c[65254]: ignoring --setuser ceph since I am not root 2024-09-16T01:59:53.938 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-c[65254]: ignoring --setgroup ceph since I am not root 2024-09-16T01:59:53.939 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-mds[65258]: -- 172.21.15.176:0/676875772 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562ec484f860 con 0x562ec482d000 2024-09-16T01:59:53.939 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-c[65254]: starting mds.c at 2024-09-16T01:59:53.939 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-i[65555]: ignoring --setuser ceph since I am not root 2024-09-16T01:59:53.939 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-i[65555]: ignoring --setgroup ceph since I am not root 2024-09-16T01:59:53.939 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-mds[65559]: -- 172.21.15.176:0/1139472189 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c82dbb4680 con 0x55c82db88800 2024-09-16T01:59:53.939 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-i[65555]: starting mds.i at 2024-09-16T01:59:53.939 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-l[64652]: ignoring --setuser ceph since I am not root 2024-09-16T01:59:53.939 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-l[64652]: ignoring --setgroup ceph since I am not root 2024-09-16T01:59:53.940 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-mds[64656]: -- 172.21.15.176:0/3485198225 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ddb3689860 con 0x55ddb3666800 2024-09-16T01:59:53.940 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 01:59:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-l[64652]: starting mds.l at 2024-09-16T01:59:53.952 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T01:59:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:55.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.028+0000 7f173b8d1640 1 -- 172.21.15.139:0/3734377767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1734109830 msgr2=0x7f1734109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:55.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.028+0000 7f173b8d1640 1 --2- 172.21.15.139:0/3734377767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1734109830 0x7f1734109c90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1724009860 tx=0x7f172402a360 comp rx=0 tx=0).stop 2024-09-16T01:59:55.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.029+0000 7f173b8d1640 1 -- 172.21.15.139:0/3734377767 shutdown_connections 2024-09-16T01:59:55.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.029+0000 7f173b8d1640 1 --2- 172.21.15.139:0/3734377767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f173411b140 0x7f173411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.029+0000 7f173b8d1640 1 --2- 172.21.15.139:0/3734377767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1734109830 0x7f1734109c90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.029+0000 7f173b8d1640 1 --2- 172.21.15.139:0/3734377767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1734109450 0x7f1734110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.029+0000 7f173b8d1640 1 -- 172.21.15.139:0/3734377767 >> 172.21.15.139:0/3734377767 conn(0x7f173406e140 msgr2=0x7f173406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:55.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.030+0000 7f173b8d1640 1 -- 172.21.15.139:0/3734377767 shutdown_connections 2024-09-16T01:59:55.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.030+0000 7f173b8d1640 1 -- 172.21.15.139:0/3734377767 wait complete. 2024-09-16T01:59:55.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.031+0000 7f173b8d1640 1 Processor -- start 2024-09-16T01:59:55.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.031+0000 7f173b8d1640 1 -- start start 2024-09-16T01:59:55.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.032+0000 7f173b8d1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1734109450 0x7f17341a7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:55.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.032+0000 7f173b8d1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1734109830 0x7f17341a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:55.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.032+0000 7f173b8d1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f173411b140 0x7f17341aba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:55.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.032+0000 7f173966d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1734109450 0x7f17341a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:55.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.032+0000 7f173966d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1734109450 0x7f17341a7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:59522/0 (socket says 172.21.15.139:59522) 2024-09-16T01:59:55.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.032+0000 7f1738e6c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1734109830 0x7f17341a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:55.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.032+0000 7f1738e6c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1734109830 0x7f17341a76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:44836/0 (socket says 172.21.15.139:44836) 2024-09-16T01:59:55.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.032+0000 7f173b8d1640 1 -- 172.21.15.139:0/1213731468 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1734120e30 con 0x7f173411b140 2024-09-16T01:59:55.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.032+0000 7f173966d640 1 -- 172.21.15.139:0/1213731468 learned_addr learned my addr 172.21.15.139:0/1213731468 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:59:55.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.033+0000 7f173b8d1640 1 -- 172.21.15.139:0/1213731468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1734120cb0 con 0x7f1734109450 2024-09-16T01:59:55.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.034+0000 7f173b8d1640 1 -- 172.21.15.139:0/1213731468 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1734120fb0 con 0x7f1734109830 2024-09-16T01:59:55.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.034+0000 7f1739e6e640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f173411b140 0x7f17341aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:55.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.034+0000 7f1739e6e640 1 -- 172.21.15.139:0/1213731468 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1734109830 msgr2=0x7f17341a76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:55.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.035+0000 7f1739e6e640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1734109830 0x7f17341a76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.035+0000 7f1739e6e640 1 -- 172.21.15.139:0/1213731468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1734109450 msgr2=0x7f17341a7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:55.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.035+0000 7f1739e6e640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1734109450 0x7f17341a7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:55.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.035+0000 7f1739e6e640 1 -- 172.21.15.139:0/1213731468 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17341ac120 con 0x7f173411b140 2024-09-16T01:59:55.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.035+0000 7f1738e6c640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1734109830 0x7f17341a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:59:55.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.035+0000 7f173966d640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1734109450 0x7f17341a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T01:59:55.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.036+0000 7f1739e6e640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f173411b140 0x7f17341aba40 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f172c00be00 tx=0x7f172c00bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:55.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.037+0000 7f172a7fc640 1 -- 172.21.15.139:0/1213731468 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c019070 con 0x7f173411b140 2024-09-16T01:59:55.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.037+0000 7f172a7fc640 1 -- 172.21.15.139:0/1213731468 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172c00cd90 con 0x7f173411b140 2024-09-16T01:59:55.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.037+0000 7f173b8d1640 1 -- 172.21.15.139:0/1213731468 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17341ac410 con 0x7f173411b140 2024-09-16T01:59:55.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.037+0000 7f173b8d1640 1 -- 172.21.15.139:0/1213731468 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17341b3cf0 con 0x7f173411b140 2024-09-16T01:59:55.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.037+0000 7f172a7fc640 1 -- 172.21.15.139:0/1213731468 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c0128b0 con 0x7f173411b140 2024-09-16T01:59:55.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.040+0000 7f173b8d1640 1 -- 172.21.15.139:0/1213731468 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1718005290 con 0x7f173411b140 2024-09-16T01:59:55.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.042+0000 7f172a7fc640 1 -- 172.21.15.139:0/1213731468 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f172c01c070 con 0x7f173411b140 2024-09-16T01:59:55.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.043+0000 7f172a7fc640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17200a35c0 0x7f17200a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:55.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.043+0000 7f172a7fc640 1 -- 172.21.15.139:0/1213731468 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 10160+0+0 (secure 0 0 0) 0x7f172c0cf8a0 con 0x7f173411b140 2024-09-16T01:59:55.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.045+0000 7f173966d640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17200a35c0 0x7f17200a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:55.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.045+0000 7f173966d640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17200a35c0 0x7f17200a5a80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f17300058c0 tx=0x7f1730009260 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:55.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.051+0000 7f172a7fc640 1 -- 172.21.15.139:0/1213731468 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f172c006e20 con 0x7f173411b140 2024-09-16T01:59:55.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.164:6836/878240227,v1:172.21.15.164:6837/878240227] up:boot 2024-09-16T01:59:55.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.164:6832/2892246482,v1:172.21.15.164:6833/2892246482] up:boot 2024-09-16T01:59:55.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.139:6836/267437683,v1:172.21.15.139:6837/267437683] up:boot 2024-09-16T01:59:55.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.139:6838/3041399881,v1:172.21.15.139:6839/3041399881] up:boot 2024-09-16T01:59:55.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.139:6840/3637830231,v1:172.21.15.139:6841/3637830231] up:boot 2024-09-16T01:59:55.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.164:6838/4034956529,v1:172.21.15.164:6839/4034956529] up:boot 2024-09-16T01:59:55.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.176:6832/3462437449,v1:172.21.15.176:6833/3462437449] up:boot 2024-09-16T01:59:55.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.176:6834/4274381850,v1:172.21.15.176:6835/4274381850] up:boot 2024-09-16T01:59:55.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.176:6838/1260362635,v1:172.21.15.176:6839/1260362635] up:boot 2024-09-16T01:59:55.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: mds.? [v2:172.21.15.176:6836/3634104163,v1:172.21.15.176:6837/3634104163] up:boot 2024-09-16T01:59:55.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-16T01:59:55.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T01:59:55.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T01:59:55.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T01:59:55.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T01:59:55.425 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T01:59:55.425 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T01:59:55.425 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T01:59:55.425 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T01:59:55.425 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T01:59:55.425 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T01:59:55.425 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:55 smithi139 ceph-mon[31175]: pgmap v623: 129 pgs: 21 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 43 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-16T01:59:55.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.164:6836/878240227,v1:172.21.15.164:6837/878240227] up:boot 2024-09-16T01:59:55.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.164:6832/2892246482,v1:172.21.15.164:6833/2892246482] up:boot 2024-09-16T01:59:55.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.139:6836/267437683,v1:172.21.15.139:6837/267437683] up:boot 2024-09-16T01:59:55.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.139:6838/3041399881,v1:172.21.15.139:6839/3041399881] up:boot 2024-09-16T01:59:55.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.139:6840/3637830231,v1:172.21.15.139:6841/3637830231] up:boot 2024-09-16T01:59:55.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.164:6838/4034956529,v1:172.21.15.164:6839/4034956529] up:boot 2024-09-16T01:59:55.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.176:6832/3462437449,v1:172.21.15.176:6833/3462437449] up:boot 2024-09-16T01:59:55.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.176:6834/4274381850,v1:172.21.15.176:6835/4274381850] up:boot 2024-09-16T01:59:55.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.176:6838/1260362635,v1:172.21.15.176:6839/1260362635] up:boot 2024-09-16T01:59:55.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: mds.? [v2:172.21.15.176:6836/3634104163,v1:172.21.15.176:6837/3634104163] up:boot 2024-09-16T01:59:55.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-16T01:59:55.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T01:59:55.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T01:59:55.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T01:59:55.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T01:59:55.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T01:59:55.438 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T01:59:55.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T01:59:55.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T01:59:55.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T01:59:55.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T01:59:55.439 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:55 smithi176 ceph-mon[35324]: pgmap v623: 129 pgs: 21 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 43 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-16T01:59:55.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:55.451+0000 7f173b8d1640 1 -- 172.21.15.139:0/1213731468 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1718005850 con 0x7f173411b140 2024-09-16T01:59:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.164:6836/878240227,v1:172.21.15.164:6837/878240227] up:boot 2024-09-16T01:59:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.164:6832/2892246482,v1:172.21.15.164:6833/2892246482] up:boot 2024-09-16T01:59:55.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.139:6836/267437683,v1:172.21.15.139:6837/267437683] up:boot 2024-09-16T01:59:55.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.139:6838/3041399881,v1:172.21.15.139:6839/3041399881] up:boot 2024-09-16T01:59:55.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.139:6840/3637830231,v1:172.21.15.139:6841/3637830231] up:boot 2024-09-16T01:59:55.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.164:6838/4034956529,v1:172.21.15.164:6839/4034956529] up:boot 2024-09-16T01:59:55.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.176:6832/3462437449,v1:172.21.15.176:6833/3462437449] up:boot 2024-09-16T01:59:55.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.176:6834/4274381850,v1:172.21.15.176:6835/4274381850] up:boot 2024-09-16T01:59:55.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.176:6838/1260362635,v1:172.21.15.176:6839/1260362635] up:boot 2024-09-16T01:59:55.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: mds.? [v2:172.21.15.176:6836/3634104163,v1:172.21.15.176:6837/3634104163] up:boot 2024-09-16T01:59:55.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-16T01:59:55.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T01:59:55.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T01:59:55.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T01:59:55.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T01:59:55.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T01:59:55.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T01:59:55.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T01:59:55.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T01:59:55.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T01:59:55.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T01:59:55.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:55 smithi164 ceph-mon[35121]: pgmap v623: 129 pgs: 21 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 43 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-16T01:59:56.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:56.151+0000 7f172a7fc640 1 -- 172.21.15.139:0/1213731468 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) ==== 104+0+0 (secure 0 0 0) 0x7f172c09bcf0 con 0x7f173411b140 2024-09-16T01:59:56.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:56.159+0000 7f173b8d1640 1 -- 172.21.15.139:0/1213731468 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17200a35c0 msgr2=0x7f17200a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:56.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:56.159+0000 7f173b8d1640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17200a35c0 0x7f17200a5a80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f17300058c0 tx=0x7f1730009260 comp rx=0 tx=0).stop 2024-09-16T01:59:56.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:56.159+0000 7f173b8d1640 1 -- 172.21.15.139:0/1213731468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f173411b140 msgr2=0x7f17341aba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:56.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:56.159+0000 7f173b8d1640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f173411b140 0x7f17341aba40 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f172c00be00 tx=0x7f172c00bef0 comp rx=0 tx=0).stop 2024-09-16T01:59:56.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:56.159+0000 7f173b8d1640 1 -- 172.21.15.139:0/1213731468 shutdown_connections 2024-09-16T01:59:56.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:56.159+0000 7f173b8d1640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17200a35c0 0x7f17200a5a80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:56.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:56.159+0000 7f173b8d1640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f173411b140 0x7f17341aba40 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:56.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:56.159+0000 7f173b8d1640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1734109830 0x7f17341a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:56.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:56.159+0000 7f173b8d1640 1 --2- 172.21.15.139:0/1213731468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1734109450 0x7f17341a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:56.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:56.159+0000 7f173b8d1640 1 -- 172.21.15.139:0/1213731468 >> 172.21.15.139:0/1213731468 conn(0x7f173406e140 msgr2=0x7f173411c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:56.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:56.161+0000 7f173b8d1640 1 -- 172.21.15.139:0/1213731468 shutdown_connections 2024-09-16T01:59:56.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:56.163+0000 7f173b8d1640 1 -- 172.21.15.139:0/1213731468 wait complete. 2024-09-16T01:59:56.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:56 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1213731468' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:59:56.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:56 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1213731468' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:59:56.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:56 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1213731468' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T01:59:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:57 smithi139 ceph-mon[31175]: pgmap v624: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 921 B/s wr, 10 op/s 2024-09-16T01:59:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:57 smithi139 ceph-mon[31175]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T01:59:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:57 smithi139 ceph-mon[31175]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T01:59:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:57 smithi139 ceph-mon[31175]: Cluster is now healthy 2024-09-16T01:59:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:57 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/1213731468' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T01:59:57.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:57 smithi139 ceph-mon[31175]: fsmap 12 up:standby 2024-09-16T01:59:57.430 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:57 smithi176 ceph-mon[35324]: pgmap v624: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 921 B/s wr, 10 op/s 2024-09-16T01:59:57.430 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:57 smithi176 ceph-mon[35324]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T01:59:57.430 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:57 smithi176 ceph-mon[35324]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T01:59:57.431 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:57 smithi176 ceph-mon[35324]: Cluster is now healthy 2024-09-16T01:59:57.431 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:57 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/1213731468' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T01:59:57.431 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:57 smithi176 ceph-mon[35324]: fsmap 12 up:standby 2024-09-16T01:59:57.479 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T01:59:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:57 smithi164 ceph-mon[35121]: pgmap v624: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 921 B/s wr, 10 op/s 2024-09-16T01:59:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:57 smithi164 ceph-mon[35121]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T01:59:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:57 smithi164 ceph-mon[35121]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T01:59:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:57 smithi164 ceph-mon[35121]: Cluster is now healthy 2024-09-16T01:59:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:57 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/1213731468' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T01:59:57.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:57 smithi164 ceph-mon[35121]: fsmap 12 up:standby 2024-09-16T01:59:57.944 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T01:59:58.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.540+0000 7f1e7559a640 1 -- 172.21.15.139:0/3536013059 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68046020 con 0x7f1e78109450 2024-09-16T01:59:58.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.541+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3536013059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e78109450 msgr2=0x7f1e78110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:58.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.541+0000 7f1e7cc12640 1 --2- 172.21.15.139:0/3536013059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e78109450 0x7f1e78110250 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1e6801a910 tx=0x7f1e6803b450 comp rx=0 tx=0).stop 2024-09-16T01:59:58.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.541+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3536013059 shutdown_connections 2024-09-16T01:59:58.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.541+0000 7f1e7cc12640 1 --2- 172.21.15.139:0/3536013059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e7811b140 0x7f1e7811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:58.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.541+0000 7f1e7cc12640 1 --2- 172.21.15.139:0/3536013059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e78109830 0x7f1e78109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:58.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.541+0000 7f1e7cc12640 1 --2- 172.21.15.139:0/3536013059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e78109450 0x7f1e78110250 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:58.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.541+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3536013059 >> 172.21.15.139:0/3536013059 conn(0x7f1e7806e140 msgr2=0x7f1e7806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:58.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.545+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3536013059 shutdown_connections 2024-09-16T01:59:58.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.545+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3536013059 wait complete. 2024-09-16T01:59:58.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.546+0000 7f1e7cc12640 1 Processor -- start 2024-09-16T01:59:58.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.546+0000 7f1e7cc12640 1 -- start start 2024-09-16T01:59:58.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.546+0000 7f1e7cc12640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e78109450 0x7f1e7811a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:58.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.546+0000 7f1e7cc12640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e78109830 0x7f1e781168a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:58.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.546+0000 7f1e7cc12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e7811b140 0x7f1e78116de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:58.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.546+0000 7f1e7cc12640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78113300 con 0x7f1e7811b140 2024-09-16T01:59:58.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.546+0000 7f1e7cc12640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78113180 con 0x7f1e78109830 2024-09-16T01:59:58.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.546+0000 7f1e7cc12640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78113480 con 0x7f1e78109450 2024-09-16T01:59:58.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.546+0000 7f1e76d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e7811b140 0x7f1e78116de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:58.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.546+0000 7f1e76d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e7811b140 0x7f1e78116de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60298/0 (socket says 172.21.15.139:60298) 2024-09-16T01:59:58.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.546+0000 7f1e76d9d640 1 -- 172.21.15.139:0/3049172295 learned_addr learned my addr 172.21.15.139:0/3049172295 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T01:59:58.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.546+0000 7f1e7659c640 1 --2- 172.21.15.139:0/3049172295 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e78109450 0x7f1e7811a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:58.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.547+0000 7f1e76d9d640 1 -- 172.21.15.139:0/3049172295 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e78109450 msgr2=0x7f1e7811a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:58.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.547+0000 7f1e76d9d640 1 --2- 172.21.15.139:0/3049172295 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e78109450 0x7f1e7811a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:58.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.547+0000 7f1e76d9d640 1 -- 172.21.15.139:0/3049172295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e78109830 msgr2=0x7f1e781168a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T01:59:58.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.547+0000 7f1e76d9d640 1 --2- 172.21.15.139:0/3049172295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e78109830 0x7f1e781168a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:58.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.547+0000 7f1e76d9d640 1 -- 172.21.15.139:0/3049172295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e78117510 con 0x7f1e7811b140 2024-09-16T01:59:58.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.547+0000 7f1e76d9d640 1 --2- 172.21.15.139:0/3049172295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e7811b140 0x7f1e78116de0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000cab0 tx=0x7f1e7000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:58.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.547+0000 7f1e677fe640 1 -- 172.21.15.139:0/3049172295 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000ae20 con 0x7f1e7811b140 2024-09-16T01:59:58.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.547+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3049172295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7807a450 con 0x7f1e7811b140 2024-09-16T01:59:58.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.547+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3049172295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e7807a990 con 0x7f1e7811b140 2024-09-16T01:59:58.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.547+0000 7f1e677fe640 1 -- 172.21.15.139:0/3049172295 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e7000f930 con 0x7f1e7811b140 2024-09-16T01:59:58.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.547+0000 7f1e677fe640 1 -- 172.21.15.139:0/3049172295 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70018bf0 con 0x7f1e7811b140 2024-09-16T01:59:58.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.549+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3049172295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e58005290 con 0x7f1e7811b140 2024-09-16T01:59:58.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.552+0000 7f1e677fe640 1 -- 172.21.15.139:0/3049172295 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1e7000f450 con 0x7f1e7811b140 2024-09-16T01:59:58.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.553+0000 7f1e677fe640 1 --2- 172.21.15.139:0/3049172295 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1e600a3580 0x7f1e600a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T01:59:58.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.554+0000 7f1e677fe640 1 -- 172.21.15.139:0/3049172295 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 10160+0+0 (secure 0 0 0) 0x7f1e700d07c0 con 0x7f1e7811b140 2024-09-16T01:59:58.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.554+0000 7f1e7659c640 1 --2- 172.21.15.139:0/3049172295 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1e600a3580 0x7f1e600a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T01:59:58.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.555+0000 7f1e7659c640 1 --2- 172.21.15.139:0/3049172295 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1e600a3580 0x7f1e600a5a40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1e680057c0 tx=0x7f1e6801a570 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T01:59:58.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.558+0000 7f1e677fe640 1 -- 172.21.15.139:0/3049172295 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1e70014030 con 0x7f1e7811b140 2024-09-16T01:59:58.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:58.893+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3049172295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1e58005580 con 0x7f1e7811b140 2024-09-16T01:59:59.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:59.161+0000 7f1e677fe640 1 -- 172.21.15.139:0/3049172295 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v122) ==== 181+0+0 (secure 0 0 0) 0x7f1e7009cc90 con 0x7f1e7811b140 2024-09-16T01:59:59.166 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' removed 2024-09-16T01:59:59.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:59.165+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3049172295 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1e600a3580 msgr2=0x7f1e600a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:59.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:59.165+0000 7f1e7cc12640 1 --2- 172.21.15.139:0/3049172295 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1e600a3580 0x7f1e600a5a40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1e680057c0 tx=0x7f1e6801a570 comp rx=0 tx=0).stop 2024-09-16T01:59:59.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:59.165+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3049172295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e7811b140 msgr2=0x7f1e78116de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T01:59:59.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:59.165+0000 7f1e7cc12640 1 --2- 172.21.15.139:0/3049172295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e7811b140 0x7f1e78116de0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000cab0 tx=0x7f1e7000cf70 comp rx=0 tx=0).stop 2024-09-16T01:59:59.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:59.165+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3049172295 shutdown_connections 2024-09-16T01:59:59.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:59.165+0000 7f1e7cc12640 1 --2- 172.21.15.139:0/3049172295 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f1e600a3580 0x7f1e600a5a40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:59.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:59.165+0000 7f1e7cc12640 1 --2- 172.21.15.139:0/3049172295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e7811b140 0x7f1e78116de0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:59.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:59.165+0000 7f1e7cc12640 1 --2- 172.21.15.139:0/3049172295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e78109830 0x7f1e781168a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:59.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:59.165+0000 7f1e7cc12640 1 --2- 172.21.15.139:0/3049172295 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e78109450 0x7f1e7811a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T01:59:59.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:59.165+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3049172295 >> 172.21.15.139:0/3049172295 conn(0x7f1e7806e140 msgr2=0x7f1e78070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T01:59:59.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:59.165+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3049172295 shutdown_connections 2024-09-16T01:59:59.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T01:59:59.165+0000 7f1e7cc12640 1 -- 172.21.15.139:0/3049172295 wait complete. 2024-09-16T01:59:59.425 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:59 smithi139 ceph-mon[31175]: pgmap v625: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 818 B/s wr, 9 op/s 2024-09-16T01:59:59.425 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 01:59:59 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3049172295' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T01:59:59.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:59 smithi176 ceph-mon[35324]: pgmap v625: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 818 B/s wr, 9 op/s 2024-09-16T01:59:59.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 01:59:59 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3049172295' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T01:59:59.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:59 smithi164 ceph-mon[35121]: pgmap v625: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 1.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 818 B/s wr, 9 op/s 2024-09-16T01:59:59.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 01:59:59 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3049172295' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T01:59:59.888 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T02:00:00.397 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:00.423 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T02:00:00.274+0000 7fe19a871640 -1 client.15366: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-16T02:00:00.423 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: 2024-09-16T02:00:00.277+0000 7fe19a871640 -1 client.15366: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-16T02:00:00.423 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: Exception in thread Thread-24: 2024-09-16T02:00:00.423 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: Traceback (most recent call last): 2024-09-16T02:00:00.424 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2024-09-16T02:00:00.424 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self.run() 2024-09-16T02:00:00.424 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/lib64/python3.9/threading.py", line 1306, in run 2024-09-16T02:00:00.424 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self.function(*self.args, **self.kwargs) 2024-09-16T02:00:00.424 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line 355, in create_scheduled_snapshot 2024-09-16T02:00:00.424 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: self.prune_snapshots(sched) 2024-09-16T02:00:00.424 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x[31403]: UnboundLocalError: local variable 'sched' referenced before assignment 2024-09-16T02:00:00.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:00 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3049172295' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:00:00.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:00 smithi139 ceph-mon[31175]: osdmap e122: 12 total, 12 up, 12 in 2024-09-16T02:00:00.424 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:00 smithi139 ceph-mon[31175]: overall HEALTH_OK 2024-09-16T02:00:00.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:00 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3049172295' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:00:00.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:00 smithi176 ceph-mon[35324]: osdmap e122: 12 total, 12 up, 12 in 2024-09-16T02:00:00.437 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:00 smithi176 ceph-mon[35324]: overall HEALTH_OK 2024-09-16T02:00:00.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:00 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3049172295' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:00:00.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:00 smithi164 ceph-mon[35121]: osdmap e122: 12 total, 12 up, 12 in 2024-09-16T02:00:00.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:00 smithi164 ceph-mon[35121]: overall HEALTH_OK 2024-09-16T02:00:01.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:01 smithi139 ceph-mon[31175]: pgmap v627: 97 pgs: 17 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 23 active+clean; 564 MiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:01.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:01 smithi176 ceph-mon[35324]: pgmap v627: 97 pgs: 17 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 23 active+clean; 564 MiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:01.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:01 smithi164 ceph-mon[35121]: pgmap v627: 97 pgs: 17 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 23 active+clean; 564 MiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:03.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:03 smithi139 ceph-mon[31175]: pgmap v628: 97 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 31 active+clean; 563 MiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:03.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:03 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3130112427' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:00:03.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:03 smithi176 ceph-mon[35324]: pgmap v628: 97 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 31 active+clean; 563 MiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:03.436 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:03 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3130112427' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:00:03.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:03 smithi164 ceph-mon[35121]: pgmap v628: 97 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 31 active+clean; 563 MiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:03.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:03 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3130112427' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:00:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.739+0000 7f90d8a5b640 1 -- 172.21.15.139:0/4055772086 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90d4100a90 msgr2=0x7f90d4100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.739+0000 7f90d8a5b640 1 --2- 172.21.15.139:0/4055772086 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90d4100a90 0x7f90d4100ef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009930 tx=0x7f90c802a480 comp rx=0 tx=0).stop 2024-09-16T02:00:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.740+0000 7f90d8a5b640 1 -- 172.21.15.139:0/4055772086 shutdown_connections 2024-09-16T02:00:04.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.740+0000 7f90d8a5b640 1 --2- 172.21.15.139:0/4055772086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90d4101430 0x7f90d410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.740+0000 7f90d8a5b640 1 --2- 172.21.15.139:0/4055772086 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90d4100a90 0x7f90d4100ef0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.740+0000 7f90d8a5b640 1 --2- 172.21.15.139:0/4055772086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90d4106a90 0x7f90d4106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.741+0000 7f90d8a5b640 1 -- 172.21.15.139:0/4055772086 >> 172.21.15.139:0/4055772086 conn(0x7f90d40fc7d0 msgr2=0x7f90d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:04.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.745+0000 7f90d8a5b640 1 -- 172.21.15.139:0/4055772086 shutdown_connections 2024-09-16T02:00:04.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.746+0000 7f90d8a5b640 1 -- 172.21.15.139:0/4055772086 wait complete. 2024-09-16T02:00:04.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.747+0000 7f90d8a5b640 1 Processor -- start 2024-09-16T02:00:04.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.752+0000 7f90d8a5b640 1 -- start start 2024-09-16T02:00:04.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.753+0000 7f90d8a5b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90d4100a90 0x7f90d4197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:04.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.753+0000 7f90d8a5b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90d4101430 0x7f90d4197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:04.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.753+0000 7f90d8a5b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90d4106a90 0x7f90d4197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:04.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.753+0000 7f90d8a5b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4110fa0 con 0x7f90d4101430 2024-09-16T02:00:04.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.753+0000 7f90d8a5b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4110e20 con 0x7f90d4106a90 2024-09-16T02:00:04.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.753+0000 7f90d1d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90d4101430 0x7f90d4197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:04.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.753+0000 7f90d1d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90d4101430 0x7f90d4197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52116/0 (socket says 172.21.15.139:52116) 2024-09-16T02:00:04.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.753+0000 7f90d1d9b640 1 -- 172.21.15.139:0/3130112427 learned_addr learned my addr 172.21.15.139:0/3130112427 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T02:00:04.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.753+0000 7f90d8a5b640 1 -- 172.21.15.139:0/3130112427 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4111120 con 0x7f90d4100a90 2024-09-16T02:00:04.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.754+0000 7f90d1d9b640 1 -- 172.21.15.139:0/3130112427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90d4100a90 msgr2=0x7f90d4197220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T02:00:04.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.754+0000 7f90d1d9b640 1 --2- 172.21.15.139:0/3130112427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90d4100a90 0x7f90d4197220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.754+0000 7f90d1d9b640 1 -- 172.21.15.139:0/3130112427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90d4106a90 msgr2=0x7f90d4197ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T02:00:04.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.754+0000 7f90d1d9b640 1 --2- 172.21.15.139:0/3130112427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90d4106a90 0x7f90d4197ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.754+0000 7f90d1d9b640 1 -- 172.21.15.139:0/3130112427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d41a6a90 con 0x7f90d4101430 2024-09-16T02:00:04.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.755+0000 7f90d1d9b640 1 --2- 172.21.15.139:0/3130112427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90d4101430 0x7f90d4197760 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f90c802a9f0 tx=0x7f90c8003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:04.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.760+0000 7f90c37fe640 1 -- 172.21.15.139:0/3130112427 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c803f070 con 0x7f90d4101430 2024-09-16T02:00:04.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.760+0000 7f90d8a5b640 1 -- 172.21.15.139:0/3130112427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d41a6d20 con 0x7f90d4101430 2024-09-16T02:00:04.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.760+0000 7f90d8a5b640 1 -- 172.21.15.139:0/3130112427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d41a7140 con 0x7f90d4101430 2024-09-16T02:00:04.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.760+0000 7f90c37fe640 1 -- 172.21.15.139:0/3130112427 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c8031c20 con 0x7f90d4101430 2024-09-16T02:00:04.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.762+0000 7f90c37fe640 1 -- 172.21.15.139:0/3130112427 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8035030 con 0x7f90d4101430 2024-09-16T02:00:04.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.767+0000 7f90d8a5b640 1 -- 172.21.15.139:0/3130112427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b4005290 con 0x7f90d4101430 2024-09-16T02:00:04.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.770+0000 7f90c37fe640 1 -- 172.21.15.139:0/3130112427 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f90c8042070 con 0x7f90d4101430 2024-09-16T02:00:04.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.771+0000 7f90c37fe640 1 --2- 172.21.15.139:0/3130112427 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90bc0a3420 0x7f90bc0a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:04.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.771+0000 7f90c37fe640 1 -- 172.21.15.139:0/3130112427 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9577+0+0 (secure 0 0 0) 0x7f90c80f0a00 con 0x7f90d4101430 2024-09-16T02:00:04.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.777+0000 7f90d259c640 1 --2- 172.21.15.139:0/3130112427 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90bc0a3420 0x7f90bc0a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:04.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.779+0000 7f90c37fe640 1 -- 172.21.15.139:0/3130112427 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90c80bcf60 con 0x7f90d4101430 2024-09-16T02:00:04.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:02.785+0000 7f90d259c640 1 --2- 172.21.15.139:0/3130112427 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90bc0a3420 0x7f90bc0a58e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00abb0 tx=0x7f90cc009250 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:04.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:03.039+0000 7f90d8a5b640 1 -- 172.21.15.139:0/3130112427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f90b4005580 con 0x7f90d4101430 2024-09-16T02:00:04.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:03.182+0000 7f90c37fe640 1 -- 172.21.15.139:0/3130112427 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v123) ==== 169+0+0 (secure 0 0 0) 0x7f90c80bc860 con 0x7f90d4101430 2024-09-16T02:00:04.134 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' removed 2024-09-16T02:00:04.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:03.186+0000 7f90c17fa640 1 -- 172.21.15.139:0/3130112427 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90bc0a3420 msgr2=0x7f90bc0a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:04.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:03.186+0000 7f90c17fa640 1 --2- 172.21.15.139:0/3130112427 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90bc0a3420 0x7f90bc0a58e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00abb0 tx=0x7f90cc009250 comp rx=0 tx=0).stop 2024-09-16T02:00:04.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:03.186+0000 7f90c17fa640 1 -- 172.21.15.139:0/3130112427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90d4101430 msgr2=0x7f90d4197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:04.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:03.186+0000 7f90c17fa640 1 --2- 172.21.15.139:0/3130112427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90d4101430 0x7f90d4197760 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f90c802a9f0 tx=0x7f90c8003b70 comp rx=0 tx=0).stop 2024-09-16T02:00:04.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:03.186+0000 7f90c17fa640 1 -- 172.21.15.139:0/3130112427 shutdown_connections 2024-09-16T02:00:04.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:03.186+0000 7f90c17fa640 1 --2- 172.21.15.139:0/3130112427 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f90bc0a3420 0x7f90bc0a58e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:03.186+0000 7f90c17fa640 1 --2- 172.21.15.139:0/3130112427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90d4106a90 0x7f90d4197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:03.186+0000 7f90c17fa640 1 --2- 172.21.15.139:0/3130112427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90d4101430 0x7f90d4197760 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:03.186+0000 7f90c17fa640 1 --2- 172.21.15.139:0/3130112427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90d4100a90 0x7f90d4197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:04.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:03.186+0000 7f90c17fa640 1 -- 172.21.15.139:0/3130112427 >> 172.21.15.139:0/3130112427 conn(0x7f90d40fc7d0 msgr2=0x7f90d40fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:04.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:03.186+0000 7f90c17fa640 1 -- 172.21.15.139:0/3130112427 shutdown_connections 2024-09-16T02:00:04.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:03.186+0000 7f90c17fa640 1 -- 172.21.15.139:0/3130112427 wait complete. 2024-09-16T02:00:04.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:04 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/3130112427' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:00:04.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:04 smithi164 ceph-mon[35121]: osdmap e123: 12 total, 12 up, 12 in 2024-09-16T02:00:04.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:04 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:04.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:04 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/3130112427' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:00:04.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:04 smithi139 ceph-mon[31175]: osdmap e123: 12 total, 12 up, 12 in 2024-09-16T02:00:04.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:04 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:04.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:04 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/3130112427' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:00:04.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:04 smithi176 ceph-mon[35324]: osdmap e123: 12 total, 12 up, 12 in 2024-09-16T02:00:04.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:04 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T02:00:05.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:05 smithi164 ceph-mon[35121]: pgmap v630: 33 pgs: 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 7 active+clean; 559 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:05.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:05 smithi139 ceph-mon[31175]: pgmap v630: 33 pgs: 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 7 active+clean; 559 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:05.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:05 smithi176 ceph-mon[35324]: pgmap v630: 33 pgs: 11 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 7 active+clean; 559 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:07.476 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:07 smithi139 ceph-mon[31175]: pgmap v631: 33 pgs: 12 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 7 active+clean; 555 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:07.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:07 smithi164 ceph-mon[35121]: pgmap v631: 33 pgs: 12 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 7 active+clean; 555 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:07.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:07 smithi176 ceph-mon[35324]: pgmap v631: 33 pgs: 12 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 7 active+clean; 555 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:08.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:08 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:08.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:08 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:08.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:08 smithi164 ceph-mon[35121]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T02:00:08.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:08 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:08.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:08 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:08.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:08 smithi139 ceph-mon[31175]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T02:00:08.686 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:08 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T02:00:08.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:08 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T02:00:08.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:08 smithi176 ceph-mon[35324]: from='mgr.15324 172.21.15.139:0/1041859609' entity='mgr.x' 2024-09-16T02:00:09.368 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-16T02:00:09.673 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:09 smithi139 ceph-mon[31175]: pgmap v632: 33 pgs: 13 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 8 active+clean; 553 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:09.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:09 smithi164 ceph-mon[35121]: pgmap v632: 33 pgs: 13 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 8 active+clean; 553 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:09.898 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T02:00:09.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:09 smithi176 ceph-mon[35324]: pgmap v632: 33 pgs: 13 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 8 active+clean; 553 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:10.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.526+0000 7f180f218640 1 -- 172.21.15.139:0/818702511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1808109830 msgr2=0x7f1808109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:10.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.526+0000 7f180f218640 1 --2- 172.21.15.139:0/818702511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1808109830 0x7f1808109c90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f17f801a910 tx=0x7f17f803b470 comp rx=0 tx=0).stop 2024-09-16T02:00:10.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.532+0000 7f180f218640 1 -- 172.21.15.139:0/818702511 shutdown_connections 2024-09-16T02:00:10.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.532+0000 7f180f218640 1 --2- 172.21.15.139:0/818702511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f180811b140 0x7f180811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:10.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.532+0000 7f180f218640 1 --2- 172.21.15.139:0/818702511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1808109830 0x7f1808109c90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:10.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.532+0000 7f180f218640 1 --2- 172.21.15.139:0/818702511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1808109450 0x7f1808110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:10.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.532+0000 7f180f218640 1 -- 172.21.15.139:0/818702511 >> 172.21.15.139:0/818702511 conn(0x7f180806e140 msgr2=0x7f180806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:10.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.535+0000 7f180f218640 1 -- 172.21.15.139:0/818702511 shutdown_connections 2024-09-16T02:00:10.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.538+0000 7f180f218640 1 -- 172.21.15.139:0/818702511 wait complete. 2024-09-16T02:00:10.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.538+0000 7f180f218640 1 Processor -- start 2024-09-16T02:00:10.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.539+0000 7f180f218640 1 -- start start 2024-09-16T02:00:10.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.539+0000 7f180f218640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1808109450 0x7f180811a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:10.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.539+0000 7f180f218640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1808109830 0x7f1808116a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:10.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.539+0000 7f180f218640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f180811b140 0x7f1808116f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:10.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.539+0000 7f180f218640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f18081133c0 con 0x7f1808109830 2024-09-16T02:00:10.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.539+0000 7f180f218640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1808113240 con 0x7f1808109450 2024-09-16T02:00:10.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.539+0000 7f180f218640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1808113540 con 0x7f180811b140 2024-09-16T02:00:10.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.545+0000 7f180d7b5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f180811b140 0x7f1808116f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:10.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.545+0000 7f180cfb4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1808109450 0x7f180811a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:10.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.545+0000 7f180cfb4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1808109450 0x7f180811a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:49220/0 (socket says 172.21.15.139:49220) 2024-09-16T02:00:10.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.545+0000 7f180cfb4640 1 -- 172.21.15.139:0/59191582 learned_addr learned my addr 172.21.15.139:0/59191582 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T02:00:10.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.545+0000 7f180d7b5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f180811b140 0x7f1808116f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.139:47036/0 (socket says 172.21.15.139:47036) 2024-09-16T02:00:10.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.548+0000 7f180d7b5640 1 -- 172.21.15.139:0/59191582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1808109450 msgr2=0x7f180811a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:10.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.548+0000 7f180d7b5640 1 --2- 172.21.15.139:0/59191582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1808109450 0x7f180811a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:10.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.548+0000 7f180d7b5640 1 -- 172.21.15.139:0/59191582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1808109830 msgr2=0x7f1808116a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:10.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.548+0000 7f180d7b5640 1 --2- 172.21.15.139:0/59191582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1808109830 0x7f1808116a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:10.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.548+0000 7f180d7b5640 1 -- 172.21.15.139:0/59191582 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18081177d0 con 0x7f180811b140 2024-09-16T02:00:10.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.548+0000 7f180cfb4640 1 --2- 172.21.15.139:0/59191582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1808109450 0x7f180811a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T02:00:10.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.548+0000 7f180d7b5640 1 --2- 172.21.15.139:0/59191582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f180811b140 0x7f1808116f60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f17f000b410 tx=0x7f17f000b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:10.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.548+0000 7f1805ffb640 1 -- 172.21.15.139:0/59191582 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0015020 con 0x7f180811b140 2024-09-16T02:00:10.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.549+0000 7f180f218640 1 -- 172.21.15.139:0/59191582 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18081ab170 con 0x7f180811b140 2024-09-16T02:00:10.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.549+0000 7f180f218640 1 -- 172.21.15.139:0/59191582 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18081ab680 con 0x7f180811b140 2024-09-16T02:00:10.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.549+0000 7f1805ffb640 1 -- 172.21.15.139:0/59191582 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f0011d10 con 0x7f180811b140 2024-09-16T02:00:10.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.549+0000 7f1805ffb640 1 -- 172.21.15.139:0/59191582 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f000f2e0 con 0x7f180811b140 2024-09-16T02:00:10.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.552+0000 7f1805ffb640 1 -- 172.21.15.139:0/59191582 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f17f0018020 con 0x7f180811b140 2024-09-16T02:00:10.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.553+0000 7f1805ffb640 1 --2- 172.21.15.139:0/59191582 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17f40a3500 0x7f17f40a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T02:00:10.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.553+0000 7f180f218640 1 -- 172.21.15.139:0/59191582 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e8005290 con 0x7f180811b140 2024-09-16T02:00:10.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.553+0000 7f1805ffb640 1 -- 172.21.15.139:0/59191582 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9018+0+0 (secure 0 0 0) 0x7f17f00cf480 con 0x7f180811b140 2024-09-16T02:00:10.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.560+0000 7f1805ffb640 1 -- 172.21.15.139:0/59191582 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17f001f020 con 0x7f180811b140 2024-09-16T02:00:10.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.563+0000 7f180cfb4640 1 --2- 172.21.15.139:0/59191582 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17f40a3500 0x7f17f40a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T02:00:10.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.584+0000 7f180cfb4640 1 --2- 172.21.15.139:0/59191582 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17f40a3500 0x7f17f40a59c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f180000c530 tx=0x7f180000b040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T02:00:10.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:10.910+0000 7f180f218640 1 -- 172.21.15.139:0/59191582 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f17e8005580 con 0x7f180811b140 2024-09-16T02:00:11.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:11.480+0000 7f1805ffb640 1 -- 172.21.15.139:0/59191582 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v124) ==== 178+0+0 (secure 0 0 0) 0x7f17f009c280 con 0x7f180811b140 2024-09-16T02:00:11.482 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data_ec' removed 2024-09-16T02:00:11.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:11.484+0000 7f180f218640 1 -- 172.21.15.139:0/59191582 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17f40a3500 msgr2=0x7f17f40a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:11.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:11.484+0000 7f180f218640 1 --2- 172.21.15.139:0/59191582 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17f40a3500 0x7f17f40a59c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f180000c530 tx=0x7f180000b040 comp rx=0 tx=0).stop 2024-09-16T02:00:11.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:11.484+0000 7f180f218640 1 -- 172.21.15.139:0/59191582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f180811b140 msgr2=0x7f1808116f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T02:00:11.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:11.484+0000 7f180f218640 1 --2- 172.21.15.139:0/59191582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f180811b140 0x7f1808116f60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f17f000b410 tx=0x7f17f000b8d0 comp rx=0 tx=0).stop 2024-09-16T02:00:11.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:11.485+0000 7f180f218640 1 -- 172.21.15.139:0/59191582 shutdown_connections 2024-09-16T02:00:11.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:11.485+0000 7f180f218640 1 --2- 172.21.15.139:0/59191582 >> [v2:172.21.15.139:6800/4031776938,v1:172.21.15.139:6801/4031776938] conn(0x7f17f40a3500 0x7f17f40a59c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:11.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:11.485+0000 7f180f218640 1 --2- 172.21.15.139:0/59191582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f180811b140 0x7f1808116f60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:11.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:11.485+0000 7f180f218640 1 --2- 172.21.15.139:0/59191582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1808109830 0x7f1808116a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:11.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:11.485+0000 7f180f218640 1 --2- 172.21.15.139:0/59191582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1808109450 0x7f180811a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T02:00:11.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:11.485+0000 7f180f218640 1 -- 172.21.15.139:0/59191582 >> 172.21.15.139:0/59191582 conn(0x7f180806e140 msgr2=0x7f1808070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T02:00:11.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:11.485+0000 7f180f218640 1 -- 172.21.15.139:0/59191582 shutdown_connections 2024-09-16T02:00:11.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T02:00:11.486+0000 7f180f218640 1 -- 172.21.15.139:0/59191582 wait complete. 2024-09-16T02:00:11.506 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:11 smithi139 ceph-mon[31175]: pgmap v633: 33 pgs: 14 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 8 active+clean; 545 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:11.507 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:11 smithi139 ceph-mon[31175]: from='client.? 172.21.15.139:0/59191582' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:00:11.507 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:11 smithi139 ceph-mon[31175]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:00:11.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:11 smithi164 ceph-mon[35121]: pgmap v633: 33 pgs: 14 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 8 active+clean; 545 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:11.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:11 smithi164 ceph-mon[35121]: from='client.? 172.21.15.139:0/59191582' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:00:11.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:11 smithi164 ceph-mon[35121]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:00:11.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:11 smithi176 ceph-mon[35324]: pgmap v633: 33 pgs: 14 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 8 active+clean; 545 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:11.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:11 smithi176 ceph-mon[35324]: from='client.? 172.21.15.139:0/59191582' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:00:11.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:11 smithi176 ceph-mon[35324]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T02:00:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:12 smithi164 ceph-mon[35121]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:00:12.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:12 smithi164 ceph-mon[35121]: osdmap e124: 12 total, 12 up, 12 in 2024-09-16T02:00:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:12 smithi139 ceph-mon[31175]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:00:12.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:12 smithi139 ceph-mon[31175]: osdmap e124: 12 total, 12 up, 12 in 2024-09-16T02:00:12.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:12 smithi176 ceph-mon[35324]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T02:00:12.937 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:12 smithi176 ceph-mon[35324]: osdmap e124: 12 total, 12 up, 12 in 2024-09-16T02:00:13.795 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:13 smithi164 ceph-mon[35121]: pgmap v635: 1 pgs: 1 active+clean; 577 KiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:13.850 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:13 smithi139 ceph-mon[31175]: pgmap v635: 1 pgs: 1 active+clean; 577 KiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:13.879 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:00:13.932 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:00:13.936 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:13 smithi176 ceph-mon[35324]: pgmap v635: 1 pgs: 1 active+clean; 577 KiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T02:00:14.019 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:00:14.070 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T02:00:14.070 DEBUG:teuthology.orchestra.run.smithi139:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T02:00:14.098 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T02:00:14.117 DEBUG:teuthology.orchestra.run.smithi176:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T02:00:14.135 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T02:00:14.135 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-16T02:00:14.135 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.a 2024-09-16T02:00:14.423 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:14 smithi139 systemd[1]: Stopping Ceph mon.a for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:00:14.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:14 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a[31171]: 2024-09-16T02:00:14.470+0000 7f5cf93e4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:00:14.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:14 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a[31171]: 2024-09-16T02:00:14.470+0000 7f5cf93e4640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T02:00:14.923 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:14 smithi139 podman[106842]: 2024-09-16 02:00:14.643300366 +0000 UTC m=+0.306645826 container died 9a9750ef87b6c31e7f3a3ff1954a7c7ff21f7c5016735b7b1e79246d0e55a621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-16T02:00:15.331 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:15 smithi139 podman[106842]: 2024-09-16 02:00:15.087372564 +0000 UTC m=+0.750718023 container cleanup 9a9750ef87b6c31e7f3a3ff1954a7c7ff21f7c5016735b7b1e79246d0e55a621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:00:15.331 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:15 smithi139 bash[106842]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a 2024-09-16T02:00:15.555 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.a.service' 2024-09-16T02:00:15.582 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:15 smithi139 podman[106855]: 2024-09-16 02:00:15.330564473 +0000 UTC m=+0.682520279 container remove 9a9750ef87b6c31e7f3a3ff1954a7c7ff21f7c5016735b7b1e79246d0e55a621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-a, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-09-16T02:00:15.582 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:15 smithi139 systemd[1]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T02:00:15.582 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:15 smithi139 systemd[1]: Stopped Ceph mon.a for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T02:00:15.582 INFO:journalctl@ceph.mon.a.smithi139.stdout:Sep 16 02:00:15 smithi139 systemd[1]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.a.service: Consumed 27.546s CPU time. 2024-09-16T02:00:15.612 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:00:15.613 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-16T02:00:15.613 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-16T02:00:15.613 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.b 2024-09-16T02:00:15.999 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:15 smithi164 systemd[1]: Stopping Ceph mon.b for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:00:16.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:15 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-b[35117]: 2024-09-16T02:00:15.997+0000 7ff87c074640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:00:16.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:15 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-b[35117]: 2024-09-16T02:00:15.997+0000 7ff87c074640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-16T02:00:16.295 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:16 smithi164 podman[72236]: 2024-09-16 02:00:16.02886672 +0000 UTC m=+0.166125701 container died 05e60772d0f771f6f268869d9f840cb4bdd749bab5418a5d5fb45e0aad263ff2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-16T02:00:16.700 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:16 smithi164 podman[72236]: 2024-09-16 02:00:16.448156633 +0000 UTC m=+0.585415613 container cleanup 05e60772d0f771f6f268869d9f840cb4bdd749bab5418a5d5fb45e0aad263ff2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:00:16.701 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 16 02:00:16 smithi164 bash[72236]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-b 2024-09-16T02:00:16.925 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.b.service' 2024-09-16T02:00:16.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:00:16.969 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-16T02:00:16.969 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-16T02:00:16.969 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.c 2024-09-16T02:00:17.350 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:17 smithi176 systemd[1]: Stopping Ceph mon.c for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:00:17.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:17 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-c[35320]: 2024-09-16T02:00:17.347+0000 7efd51c81640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:00:17.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:17 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-c[35320]: 2024-09-16T02:00:17.347+0000 7efd51c81640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T02:00:17.687 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:17 smithi176 podman[72807]: 2024-09-16 02:00:17.50176051 +0000 UTC m=+0.274423484 container died 33fe964f1df3ecf74e6c3c2cd5d41f7661626c9f27a3eea570c7198aae766926 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-c, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:00:18.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:17 smithi176 podman[72807]: 2024-09-16 02:00:17.916030372 +0000 UTC m=+0.688693352 container cleanup 33fe964f1df3ecf74e6c3c2cd5d41f7661626c9f27a3eea570c7198aae766926 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T02:00:18.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:17 smithi176 bash[72807]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-c 2024-09-16T02:00:20.061 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.c.service' 2024-09-16T02:00:20.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:19 smithi176 podman[72819]: 2024-09-16 02:00:19.923588087 +0000 UTC m=+2.416211632 container remove 33fe964f1df3ecf74e6c3c2cd5d41f7661626c9f27a3eea570c7198aae766926 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T02:00:20.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:20 smithi176 systemd[1]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-16T02:00:20.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:20 smithi176 systemd[1]: Stopped Ceph mon.c for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T02:00:20.187 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 16 02:00:20 smithi176 systemd[1]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mon.c.service: Consumed 18.366s CPU time. 2024-09-16T02:00:20.395 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:00:20.395 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-16T02:00:20.395 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-16T02:00:20.396 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.x 2024-09-16T02:00:20.673 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:20 smithi139 systemd[1]: Stopping Ceph mgr.x for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:00:21.056 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:20 smithi139 podman[106967]: 2024-09-16 02:00:20.766410984 +0000 UTC m=+0.172829129 container died 6c955bee72d84911e2eb34d0b00c6092197a9e04c81d0312472d14fddfd8685d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-16T02:00:21.340 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:21 smithi139 podman[106967]: 2024-09-16 02:00:21.159482971 +0000 UTC m=+0.565901107 container cleanup 6c955bee72d84911e2eb34d0b00c6092197a9e04c81d0312472d14fddfd8685d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T02:00:21.340 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:21 smithi139 bash[106967]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x 2024-09-16T02:00:21.580 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-16T02:00:21.603 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:21 smithi139 podman[106979]: 2024-09-16 02:00:21.453433958 +0000 UTC m=+0.690077838 container remove 6c955bee72d84911e2eb34d0b00c6092197a9e04c81d0312472d14fddfd8685d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T02:00:21.603 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:21 smithi139 systemd[1]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-16T02:00:21.603 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:21 smithi139 systemd[1]: Stopped Ceph mgr.x for 1d90dd3a-73cb-11ef-bceb-c7b262605968. 2024-09-16T02:00:21.603 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Sep 16 02:00:21 smithi139 systemd[1]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.x.service: Consumed 56.814s CPU time. 2024-09-16T02:00:21.618 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:00:21.618 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-16T02:00:21.618 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-16T02:00:21.618 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.y 2024-09-16T02:00:21.909 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 02:00:21 smithi164 systemd[1]: Stopping Ceph mgr.y for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:00:22.255 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 02:00:21 smithi164 podman[72362]: 2024-09-16 02:00:21.967219063 +0000 UTC m=+0.156580434 container died bc3233dc290976ce0651d10712423cc1d83908ecf1e13ca1d04f3dff44a5ce38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:00:22.545 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 02:00:22 smithi164 podman[72362]: 2024-09-16 02:00:22.370367906 +0000 UTC m=+0.559729285 container cleanup bc3233dc290976ce0651d10712423cc1d83908ecf1e13ca1d04f3dff44a5ce38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-09-16T02:00:22.545 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 02:00:22 smithi164 bash[72362]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-y 2024-09-16T02:00:22.743 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-16T02:00:22.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:00:22.780 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-16T02:00:22.781 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-16T02:00:22.781 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.z 2024-09-16T02:00:23.112 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 02:00:22 smithi176 systemd[1]: Stopping Ceph mgr.z for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:00:23.436 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 02:00:23 smithi176 podman[72941]: 2024-09-16 02:00:23.111522741 +0000 UTC m=+0.136209832 container died 48120aef7454b98b17b09007388a9ce1c933390176259b14108b578fc47b8e3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-16T02:00:23.883 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 02:00:23 smithi176 podman[72941]: 2024-09-16 02:00:23.614449533 +0000 UTC m=+0.639136615 container cleanup 48120aef7454b98b17b09007388a9ce1c933390176259b14108b578fc47b8e3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T02:00:23.884 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 16 02:00:23 smithi176 bash[72941]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mgr-z 2024-09-16T02:00:24.012 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-16T02:00:24.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:00:24.050 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-16T02:00:24.050 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T02:00:24.050 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.0 2024-09-16T02:00:24.357 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 02:00:24 smithi139 systemd[1]: Stopping Ceph osd.0 for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:00:24.673 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 02:00:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0[42159]: 2024-09-16T02:00:24.355+0000 7fa1bf441640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:00:24.673 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 02:00:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0[42159]: 2024-09-16T02:00:24.355+0000 7fa1bf441640 -1 osd.0 124 *** Got signal Terminated *** 2024-09-16T02:00:24.673 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 02:00:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0[42159]: 2024-09-16T02:00:24.355+0000 7fa1bf441640 -1 osd.0 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:00:30.141 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 02:00:29 smithi139 podman[107089]: 2024-09-16 02:00:29.831188271 +0000 UTC m=+5.594555140 container died 1773d18d125084e818297e02d9bd23ff2c28072cde9494409ebb217c09bf9b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:00:30.423 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 02:00:30 smithi139 podman[107089]: 2024-09-16 02:00:30.247645009 +0000 UTC m=+6.011011865 container cleanup 1773d18d125084e818297e02d9bd23ff2c28072cde9494409ebb217c09bf9b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T02:00:30.423 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 02:00:30 smithi139 bash[107089]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0 2024-09-16T02:00:30.923 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 02:00:30 smithi139 podman[107107]: 2024-09-16 02:00:30.49935155 +0000 UTC m=+0.666885260 container remove 1773d18d125084e818297e02d9bd23ff2c28072cde9494409ebb217c09bf9b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:00:31.204 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 02:00:30 smithi139 podman[107194]: 2024-09-16 02:00:30.825607662 +0000 UTC m=+0.016399868 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:00:31.204 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 02:00:30 smithi139 podman[107194]: 2024-09-16 02:00:30.958463401 +0000 UTC m=+0.149255596 container create e62e07dfa83e242a323a8948eae583945e7e0c6a28f0db21aa0963e4a25aff60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T02:00:31.204 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 02:00:31 smithi139 podman[107194]: 2024-09-16 02:00:31.203209487 +0000 UTC m=+0.394001678 container init e62e07dfa83e242a323a8948eae583945e7e0c6a28f0db21aa0963e4a25aff60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T02:00:31.600 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 02:00:31 smithi139 podman[107194]: 2024-09-16 02:00:31.207142126 +0000 UTC m=+0.397934320 container start e62e07dfa83e242a323a8948eae583945e7e0c6a28f0db21aa0963e4a25aff60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-16T02:00:31.600 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 02:00:31 smithi139 podman[107194]: 2024-09-16 02:00:31.320019421 +0000 UTC m=+0.510811630 container attach e62e07dfa83e242a323a8948eae583945e7e0c6a28f0db21aa0963e4a25aff60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-16T02:00:31.923 INFO:journalctl@ceph.osd.0.smithi139.stdout:Sep 16 02:00:31 smithi139 podman[107194]: 2024-09-16 02:00:31.599385984 +0000 UTC m=+0.790178184 container died e62e07dfa83e242a323a8948eae583945e7e0c6a28f0db21aa0963e4a25aff60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-16T02:00:32.288 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T02:00:32.330 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:00:32.331 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T02:00:32.331 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T02:00:32.332 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.1 2024-09-16T02:00:32.651 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 02:00:32 smithi164 systemd[1]: Stopping Ceph osd.1 for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:00:33.016 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 02:00:32 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1[39585]: 2024-09-16T02:00:32.648+0000 7f6fa8ec3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:00:33.016 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 02:00:32 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1[39585]: 2024-09-16T02:00:32.648+0000 7f6fa8ec3640 -1 osd.1 124 *** Got signal Terminated *** 2024-09-16T02:00:33.016 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 02:00:32 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1[39585]: 2024-09-16T02:00:32.648+0000 7f6fa8ec3640 -1 osd.1 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:00:38.511 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 02:00:38 smithi164 podman[72498]: 2024-09-16 02:00:38.198489129 +0000 UTC m=+5.664847012 container died a8ebe4ceeb63a5e063ae028434e306e2a33b83b66750f289e48a574770468b36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-16T02:00:38.795 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 02:00:38 smithi164 podman[72498]: 2024-09-16 02:00:38.600091984 +0000 UTC m=+6.066449867 container cleanup a8ebe4ceeb63a5e063ae028434e306e2a33b83b66750f289e48a574770468b36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, ceph=True, org.label-schema.vendor=CentOS) 2024-09-16T02:00:38.795 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 02:00:38 smithi164 bash[72498]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1 2024-09-16T02:00:39.211 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 02:00:38 smithi164 podman[72513]: 2024-09-16 02:00:38.876605365 +0000 UTC m=+0.669036759 container remove a8ebe4ceeb63a5e063ae028434e306e2a33b83b66750f289e48a574770468b36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-16T02:00:39.545 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 02:00:39 smithi164 podman[72604]: 2024-09-16 02:00:39.210798527 +0000 UTC m=+0.108835967 container create 6e99e7c1f3e149acc532e0a3a16e72e322102d92cd75137c9ed5b51126d28d7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-16T02:00:39.545 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 02:00:39 smithi164 podman[72604]: 2024-09-16 02:00:39.118081572 +0000 UTC m=+0.016119022 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:00:39.545 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 02:00:39 smithi164 podman[72604]: 2024-09-16 02:00:39.455524279 +0000 UTC m=+0.353561716 container init 6e99e7c1f3e149acc532e0a3a16e72e322102d92cd75137c9ed5b51126d28d7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T02:00:39.545 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 02:00:39 smithi164 podman[72604]: 2024-09-16 02:00:39.459223357 +0000 UTC m=+0.357260794 container start 6e99e7c1f3e149acc532e0a3a16e72e322102d92cd75137c9ed5b51126d28d7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-16T02:00:39.876 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 02:00:39 smithi164 podman[72604]: 2024-09-16 02:00:39.581031023 +0000 UTC m=+0.479068474 container attach 6e99e7c1f3e149acc532e0a3a16e72e322102d92cd75137c9ed5b51126d28d7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T02:00:40.226 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 02:00:39 smithi164 podman[72604]: 2024-09-16 02:00:39.875664015 +0000 UTC m=+0.773701462 container died 6e99e7c1f3e149acc532e0a3a16e72e322102d92cd75137c9ed5b51126d28d7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T02:00:40.561 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T02:00:40.602 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:00:40.602 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T02:00:40.602 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T02:00:40.602 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.2 2024-09-16T02:00:40.920 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 02:00:40 smithi176 systemd[1]: Stopping Ceph osd.2 for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:00:41.187 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 02:00:40 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2[40288]: 2024-09-16T02:00:40.917+0000 7fe3b8f78640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:00:41.187 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 02:00:40 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2[40288]: 2024-09-16T02:00:40.917+0000 7fe3b8f78640 -1 osd.2 124 *** Got signal Terminated *** 2024-09-16T02:00:41.187 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 02:00:40 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2[40288]: 2024-09-16T02:00:40.917+0000 7fe3b8f78640 -1 osd.2 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:00:46.879 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 02:00:46 smithi176 podman[73068]: 2024-09-16 02:00:46.575461316 +0000 UTC m=+5.776842973 container died 4dc399cdb899c1ad13353a47a86ee6014924ca4ddc8503558b49afddfa343184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T02:00:47.187 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 02:00:46 smithi176 podman[73068]: 2024-09-16 02:00:46.951978274 +0000 UTC m=+6.153359975 container cleanup 4dc399cdb899c1ad13353a47a86ee6014924ca4ddc8503558b49afddfa343184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-16T02:00:47.187 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 02:00:46 smithi176 bash[73068]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2 2024-09-16T02:00:47.507 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 02:00:47 smithi176 podman[73082]: 2024-09-16 02:00:47.194885944 +0000 UTC m=+0.620525401 container remove 4dc399cdb899c1ad13353a47a86ee6014924ca4ddc8503558b49afddfa343184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-09-16T02:00:47.774 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 02:00:47 smithi176 podman[73173]: 2024-09-16 02:00:47.408809716 +0000 UTC m=+0.016007343 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:00:47.774 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 02:00:47 smithi176 podman[73173]: 2024-09-16 02:00:47.520605191 +0000 UTC m=+0.127802818 container create 7a4f60f28fbc8910d18f08aa51647b3f7ac2784d2a29e1752cd159967132cff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T02:00:48.187 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 02:00:47 smithi176 podman[73173]: 2024-09-16 02:00:47.773744282 +0000 UTC m=+0.380941910 container init 7a4f60f28fbc8910d18f08aa51647b3f7ac2784d2a29e1752cd159967132cff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=squid-8921c8e) 2024-09-16T02:00:48.187 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 02:00:47 smithi176 podman[73173]: 2024-09-16 02:00:47.777294489 +0000 UTC m=+0.384492119 container start 7a4f60f28fbc8910d18f08aa51647b3f7ac2784d2a29e1752cd159967132cff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-16T02:00:48.187 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 02:00:47 smithi176 podman[73173]: 2024-09-16 02:00:47.898739265 +0000 UTC m=+0.505936904 container attach 7a4f60f28fbc8910d18f08aa51647b3f7ac2784d2a29e1752cd159967132cff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:00:48.462 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 16 02:00:48 smithi176 podman[73173]: 2024-09-16 02:00:48.189497037 +0000 UTC m=+0.796694671 container died 7a4f60f28fbc8910d18f08aa51647b3f7ac2784d2a29e1752cd159967132cff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T02:00:48.836 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T02:00:48.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:00:48.876 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T02:00:48.876 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T02:00:48.876 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.3 2024-09-16T02:00:49.173 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 02:00:48 smithi139 systemd[1]: Stopping Ceph osd.3 for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:00:49.673 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 02:00:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3[48588]: 2024-09-16T02:00:49.192+0000 7f69b71e3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:00:49.673 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 02:00:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3[48588]: 2024-09-16T02:00:49.192+0000 7f69b71e3640 -1 osd.3 124 *** Got signal Terminated *** 2024-09-16T02:00:49.673 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 02:00:49 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3[48588]: 2024-09-16T02:00:49.192+0000 7f69b71e3640 -1 osd.3 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:00:50.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:00:50 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:00:50.628+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:00:51.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:00:51 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:00:51.277+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:00:51.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:00:51 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:00:51.623+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:00:52.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:00:52 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:00:52.231+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:00:52.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:00:52 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:00:52.609+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:00:53.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:00:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:00:53.248+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:00:53.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:00:53 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:00:53.563+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:00:54.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:00:54 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:00:54.270+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:00:54.687 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:00:54 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:00:54.290+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:00:54.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:00:54 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:00:54.321+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:00:54.824 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:00:54 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:00:54.566+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:00:55.135 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 02:00:54 smithi139 podman[107312]: 2024-09-16 02:00:54.823196977 +0000 UTC m=+5.755092803 container died 0a9d58b5157c7c554d13b867f30e4d901cf029e31b94ddacb306d5f1bf2ef7af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T02:00:55.423 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 02:00:55 smithi139 podman[107312]: 2024-09-16 02:00:55.234595167 +0000 UTC m=+6.166490990 container cleanup 0a9d58b5157c7c554d13b867f30e4d901cf029e31b94ddacb306d5f1bf2ef7af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T02:00:55.423 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 02:00:55 smithi139 bash[107312]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3 2024-09-16T02:00:55.687 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:00:55 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:00:55.274+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:00:55.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:00:55 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:00:55.243+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:00:55.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:00:55 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:00:55.319+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:00:55.845 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:00:55 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:00:55.591+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:00:55.845 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 02:00:55 smithi139 podman[107327]: 2024-09-16 02:00:55.511301348 +0000 UTC m=+0.671860386 container remove 0a9d58b5157c7c554d13b867f30e4d901cf029e31b94ddacb306d5f1bf2ef7af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T02:00:56.115 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 02:00:55 smithi139 podman[107414]: 2024-09-16 02:00:55.747151275 +0000 UTC m=+0.017312023 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:00:56.115 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 02:00:55 smithi139 podman[107414]: 2024-09-16 02:00:55.862008627 +0000 UTC m=+0.132169362 container create 29bb7c575aa123c1ebbcef3d59ba0b7d0d9cd3fa57676d3bd46212bead44c927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:00:56.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:00:55 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:00:55.720+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:00:56.423 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 02:00:56 smithi139 podman[107414]: 2024-09-16 02:00:56.114779492 +0000 UTC m=+0.384940227 container init 29bb7c575aa123c1ebbcef3d59ba0b7d0d9cd3fa57676d3bd46212bead44c927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:00:56.423 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 02:00:56 smithi139 podman[107414]: 2024-09-16 02:00:56.118159807 +0000 UTC m=+0.388320542 container start 29bb7c575aa123c1ebbcef3d59ba0b7d0d9cd3fa57676d3bd46212bead44c927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T02:00:56.423 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 02:00:56 smithi139 podman[107414]: 2024-09-16 02:00:56.22326485 +0000 UTC m=+0.493425595 container attach 29bb7c575aa123c1ebbcef3d59ba0b7d0d9cd3fa57676d3bd46212bead44c927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T02:00:56.423 INFO:journalctl@ceph.osd.3.smithi139.stdout:Sep 16 02:00:56 smithi139 podman[107414]: 2024-09-16 02:00:56.303187426 +0000 UTC m=+0.573348172 container died 29bb7c575aa123c1ebbcef3d59ba0b7d0d9cd3fa57676d3bd46212bead44c927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2) 2024-09-16T02:00:56.687 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:00:56 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:00:56.256+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:00:56.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:00:56 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:00:56.243+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:00:56.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:00:56 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:00:56.352+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:00:56.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:00:56 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:00:56.606+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:00:57.111 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T02:00:57.150 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:00:57.150 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T02:00:57.150 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T02:00:57.150 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.4 2024-09-16T02:00:57.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:00:56 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:00:56.704+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:00:57.513 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:00:57 smithi164 systemd[1]: Stopping Ceph osd.4 for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:00:57.513 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:00:57 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:00:57.373+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:00:57.687 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:00:57 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:00:57.275+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:00:57.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:00:57 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:00:57.208+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:00:57.795 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:00:57 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4[46048]: 2024-09-16T02:00:57.512+0000 7f56cc9dc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:00:57.795 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:00:57 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4[46048]: 2024-09-16T02:00:57.512+0000 7f56cc9dc640 -1 osd.4 124 *** Got signal Terminated *** 2024-09-16T02:00:57.795 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:00:57 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4[46048]: 2024-09-16T02:00:57.512+0000 7f56cc9dc640 -1 osd.4 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:00:57.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:00:57 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:00:57.623+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:00:58.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:00:57 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:00:57.742+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:00:58.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:00:58 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:00:58.188+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:00:58.687 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:00:58 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:00:58.272+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:00:58.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:00:58 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:00:58.329+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:00:58.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:00:58 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:00:58.607+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:00:59.163 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:00:58 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:00:58.756+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:00:59.436 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:00:59 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:00:59.227+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:00:59.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:00:59 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:00:59.162+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:00:59.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:00:59 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:00:59.326+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:00:59.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:00:59 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:00:59.585+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:00:59.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:00:59 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:00:59.585+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:00.177 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:00:59 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:00:59.803+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:00.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:00 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:00.175+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:00.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:00 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:00.204+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:01:00.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:00 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:00.204+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.560886+0000 front 2024-09-16T02:00:37.561051+0000 (oldest deadline 2024-09-16T02:00:59.260907+0000) 2024-09-16T02:01:00.545 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:01:00 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4[46048]: 2024-09-16T02:01:00.166+0000 7f56c4fea640 -1 osd.4 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.632317+0000 front 2024-09-16T02:00:36.632356+0000 (oldest deadline 2024-09-16T02:00:59.532442+0000) 2024-09-16T02:01:00.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:00 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:00.250+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:00.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:00 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:00.349+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:00.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:00 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:00.349+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:00.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:00.578+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:00.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:00 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:00.578+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:01.179 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:00 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:00.800+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:01.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:01 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:01.181+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:01.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:01 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:01.177+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:01:01.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:01 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:01.177+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.560886+0000 front 2024-09-16T02:00:37.561051+0000 (oldest deadline 2024-09-16T02:00:59.260907+0000) 2024-09-16T02:01:01.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:01 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:01.229+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:01.545 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:01:01 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4[46048]: 2024-09-16T02:01:01.211+0000 7f56c4fea640 -1 osd.4 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.632317+0000 front 2024-09-16T02:00:36.632356+0000 (oldest deadline 2024-09-16T02:00:59.532442+0000) 2024-09-16T02:01:01.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:01 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:01.343+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:01.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:01 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:01.343+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:01.794 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:01 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:01.567+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:01.794 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:01 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:01.567+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:01.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:01 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:01.775+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:02.173 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:01 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:01.793+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:02.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:02 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:02.169+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:02.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:02 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:02.128+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:01:02.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:02 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:02.128+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.560886+0000 front 2024-09-16T02:00:37.561051+0000 (oldest deadline 2024-09-16T02:00:59.260907+0000) 2024-09-16T02:01:02.545 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:01:02 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4[46048]: 2024-09-16T02:01:02.218+0000 7f56c4fea640 -1 osd.4 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.632317+0000 front 2024-09-16T02:00:36.632356+0000 (oldest deadline 2024-09-16T02:00:59.532442+0000) 2024-09-16T02:01:02.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:02 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:02.263+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:02.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:02 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:02.305+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:02.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:02 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:02.305+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:02.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:02 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:02.747+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:02.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:02 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:02.526+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:02.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:02 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:02.526+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:03.168 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:02 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:02.771+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:03.256 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:01:02 smithi164 podman[72719]: 2024-09-16 02:01:02.971536006 +0000 UTC m=+5.611103773 container died 5211890e6a69aca05dc8a17a3972eca5064e23fdfdc540f57b5ae846d0bfe407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:01:03.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:03 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:03.166+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:01:03.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:03 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:03.166+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.560886+0000 front 2024-09-16T02:00:37.561051+0000 (oldest deadline 2024-09-16T02:00:59.260907+0000) 2024-09-16T02:01:03.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:03 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:03.200+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:03.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:03 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:03.274+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:03.545 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:01:03 smithi164 podman[72719]: 2024-09-16 02:01:03.370798435 +0000 UTC m=+6.010366187 container cleanup 5211890e6a69aca05dc8a17a3972eca5064e23fdfdc540f57b5ae846d0bfe407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True) 2024-09-16T02:01:03.545 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:01:03 smithi164 bash[72719]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4 2024-09-16T02:01:03.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:03 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:03.255+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:03.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:03 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:03.255+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:03.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:03 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:03.780+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:03.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:03 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:03.573+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:03.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:03 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:03.573+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:03.965 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:01:03 smithi164 podman[72750]: 2024-09-16 02:01:03.638742351 +0000 UTC m=+0.661594943 container remove 5211890e6a69aca05dc8a17a3972eca5064e23fdfdc540f57b5ae846d0bfe407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:01:04.149 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:03 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:03.742+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:04.218 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:01:03 smithi164 podman[72841]: 2024-09-16 02:01:03.96462623 +0000 UTC m=+0.093759533 container create c5f14211df2b08219028c26e15f4045320b531dff03fa71a0266ab8f51482444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T02:01:04.218 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:01:03 smithi164 podman[72841]: 2024-09-16 02:01:03.88726086 +0000 UTC m=+0.016394119 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:01:04.218 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:01:04 smithi164 podman[72841]: 2024-09-16 02:01:04.21728047 +0000 UTC m=+0.346413728 container init c5f14211df2b08219028c26e15f4045320b531dff03fa71a0266ab8f51482444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-16T02:01:04.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:04 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:04.199+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:04.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:04 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:04.147+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:01:04.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:04 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:04.147+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.560886+0000 front 2024-09-16T02:00:37.561051+0000 (oldest deadline 2024-09-16T02:00:59.260907+0000) 2024-09-16T02:01:04.545 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:01:04 smithi164 podman[72841]: 2024-09-16 02:01:04.220804727 +0000 UTC m=+0.349937986 container start c5f14211df2b08219028c26e15f4045320b531dff03fa71a0266ab8f51482444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e) 2024-09-16T02:01:04.545 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:01:04 smithi164 podman[72841]: 2024-09-16 02:01:04.317406806 +0000 UTC m=+0.446540082 container attach c5f14211df2b08219028c26e15f4045320b531dff03fa71a0266ab8f51482444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-16T02:01:04.545 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 16 02:01:04 smithi164 podman[72864]: 2024-09-16 02:01:04.443260954 +0000 UTC m=+0.017541900 container died c5f14211df2b08219028c26e15f4045320b531dff03fa71a0266ab8f51482444 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-16T02:01:04.546 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:04 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:04.290+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:04.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:04 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:04.256+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:04.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:04 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:04.256+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:04.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:04 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:04.745+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:04.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:04 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:04.622+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:04.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:04 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:04.622+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:05.015 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T02:01:05.055 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:01:05.056 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T02:01:05.056 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T02:01:05.056 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.5 2024-09-16T02:01:05.085 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:04 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:04.715+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:05.373 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:05 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:05.191+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:05.373 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:05 smithi176 systemd[1]: Stopping Ceph osd.5 for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:01:05.373 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:05 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:05.134+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:01:05.373 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:05 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:05.134+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.560886+0000 front 2024-09-16T02:00:37.561051+0000 (oldest deadline 2024-09-16T02:00:59.260907+0000) 2024-09-16T02:01:05.530 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:05 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:05.264+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:05.530 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:05 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:05.256+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:05.530 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:05 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:05.256+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:05.687 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:05 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:05.371+0000 7f4939570640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:01:05.687 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:05 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:05.371+0000 7f4939570640 -1 osd.5 124 *** Got signal Terminated *** 2024-09-16T02:01:05.687 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:05 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:05.371+0000 7f4939570640 -1 osd.5 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:01:05.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:05 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:05.723+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:05.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:05 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:05.657+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:05.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:05 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:05.657+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:06.086 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:05 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:05.702+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:06.436 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:06 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:06.218+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:06.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:06 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:06.084+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:01:06.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:06 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:06.084+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.560886+0000 front 2024-09-16T02:00:37.561051+0000 (oldest deadline 2024-09-16T02:00:59.260907+0000) 2024-09-16T02:01:06.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:06 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:06.231+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:06.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:06 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:06.241+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:06.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:06 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:06.241+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:07.053 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:06 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:06.713+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:07.173 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:06 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:06.751+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:07.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:06 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:06.705+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:07.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:06 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:06.705+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:07.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:07 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:07.243+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:07.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:07 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:07.243+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:07.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:07 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:07.051+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:01:07.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:07 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:07.051+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.560886+0000 front 2024-09-16T02:00:37.561051+0000 (oldest deadline 2024-09-16T02:00:59.260907+0000) 2024-09-16T02:01:07.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:07 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:07.257+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:07.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:07 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:07.257+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:07.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:07 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:07.282+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:07.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:07 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:07.282+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:07.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:07 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:07.764+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:07.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:07 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:07.666+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:07.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:07 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:07.666+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:08.059 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:07 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:07.685+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:08.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:08 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:08.221+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:08.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:08 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:08.221+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:08.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:08 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:08.058+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:01:08.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:08 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:08.058+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.560886+0000 front 2024-09-16T02:00:37.561051+0000 (oldest deadline 2024-09-16T02:00:59.260907+0000) 2024-09-16T02:01:08.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:08 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:08.058+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.262522+0000 front 2024-09-16T02:00:43.262544+0000 (oldest deadline 2024-09-16T02:01:07.362651+0000) 2024-09-16T02:01:08.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:08 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:08.253+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:08.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:08 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:08.253+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:08.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:08 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:08.287+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:08.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:08 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:08.287+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:08.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:08 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:08.287+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:08.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:08 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:08.646+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:09.173 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:08.764+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:09.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:08.708+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:09.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:08 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:08.708+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:09.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:09 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:09.184+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:09.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:09 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:09.184+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:09.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:09 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:09.021+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:01:09.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:09 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:09.021+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.560886+0000 front 2024-09-16T02:00:37.561051+0000 (oldest deadline 2024-09-16T02:00:59.260907+0000) 2024-09-16T02:01:09.438 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:09 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:09.021+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.262522+0000 front 2024-09-16T02:00:43.262544+0000 (oldest deadline 2024-09-16T02:01:07.362651+0000) 2024-09-16T02:01:09.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:09 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:09.269+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:09.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:09 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:09.269+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:09.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:09 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:09.275+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:09.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:09 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:09.275+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:09.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:09 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:09.275+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:09.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:09 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:09.739+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:09.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:09 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:09.669+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:09.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:09 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:09.669+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:09.936 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:09 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:09.647+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:09.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:09 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:09.647+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:10.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:10 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:10.178+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:10.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:10 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:10.178+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:10.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:10 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:10.059+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:28.159735+0000 front 2024-09-16T02:00:28.159797+0000 (oldest deadline 2024-09-16T02:00:53.459844+0000) 2024-09-16T02:01:10.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:10 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:10.059+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.560886+0000 front 2024-09-16T02:00:37.561051+0000 (oldest deadline 2024-09-16T02:00:59.260907+0000) 2024-09-16T02:01:10.437 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:10 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5[46757]: 2024-09-16T02:01:10.059+0000 7f4935b86640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.262522+0000 front 2024-09-16T02:00:43.262544+0000 (oldest deadline 2024-09-16T02:01:07.362651+0000) 2024-09-16T02:01:10.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:10 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:10.275+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:10.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:10 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:10.275+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:10.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:10 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:10.245+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:10.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:10 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:10.245+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:10.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:10 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:10.245+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:10.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:10 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:10.725+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:10.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:10 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:10.725+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:41.093089+0000 front 2024-09-16T02:00:41.093037+0000 (oldest deadline 2024-09-16T02:01:09.893494+0000) 2024-09-16T02:01:10.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:10 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:10.627+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:10.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:10 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:10.627+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:10.936 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:10 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:10.681+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:10.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:10 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:10.681+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:11.281 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:10 smithi176 podman[73304]: 2024-09-16 02:01:10.977741591 +0000 UTC m=+5.723180061 container died 23589ee4366740fe9e4e90035a31e569bd55743d2a7e8da593d056f4b225a290 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:01:11.281 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:11 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:11.159+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:11.282 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:11 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:11.159+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:11.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:11 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:11.304+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:11.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:11 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:11.304+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:11.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:11 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:11.211+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:11.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:11 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:11.211+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:11.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:11 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:11.211+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:11.623 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:11 smithi176 podman[73304]: 2024-09-16 02:01:11.36329825 +0000 UTC m=+6.108736719 container cleanup 23589ee4366740fe9e4e90035a31e569bd55743d2a7e8da593d056f4b225a290 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-16T02:01:11.623 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:11 smithi176 bash[73304]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5 2024-09-16T02:01:11.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:11 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:11.746+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:11.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:11 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:11.746+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:41.093089+0000 front 2024-09-16T02:00:41.093037+0000 (oldest deadline 2024-09-16T02:01:09.893494+0000) 2024-09-16T02:01:11.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:11 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:11.578+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:11.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:11 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:11.578+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:11.924 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:11 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:11.578+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:11.937 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:11 smithi176 podman[73320]: 2024-09-16 02:01:11.622734554 +0000 UTC m=+0.623286099 container remove 23589ee4366740fe9e4e90035a31e569bd55743d2a7e8da593d056f4b225a290 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-16T02:01:11.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:11 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:11.689+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:11.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:11 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:11.689+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:12.236 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:11 smithi176 podman[73411]: 2024-09-16 02:01:11.973461658 +0000 UTC m=+0.109962379 container create b1556044450a17d05e5982e5b52eb64ef867673e3f6c06e6e0d18209d3ce8f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-16T02:01:12.236 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:11 smithi176 podman[73411]: 2024-09-16 02:01:11.879495788 +0000 UTC m=+0.015996509 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:01:12.237 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:12 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:12.160+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:12.237 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:12 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:12.160+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:12.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:12 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:12.326+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:12.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:12 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:12.326+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:12.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:12 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:12.247+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:12.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:12 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:12.247+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:12.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:12 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:12.247+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:12.671 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:12 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:12.670+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:12.672 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:12 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:12.670+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:12.672 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:12 smithi176 podman[73411]: 2024-09-16 02:01:12.234702371 +0000 UTC m=+0.371203113 container init b1556044450a17d05e5982e5b52eb64ef867673e3f6c06e6e0d18209d3ce8f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T02:01:12.672 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:12 smithi176 podman[73411]: 2024-09-16 02:01:12.238399727 +0000 UTC m=+0.374900454 container start b1556044450a17d05e5982e5b52eb64ef867673e3f6c06e6e0d18209d3ce8f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T02:01:12.672 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:12 smithi176 podman[73411]: 2024-09-16 02:01:12.359616149 +0000 UTC m=+0.496116885 container attach b1556044450a17d05e5982e5b52eb64ef867673e3f6c06e6e0d18209d3ce8f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:01:12.672 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 16 02:01:12 smithi176 podman[73411]: 2024-09-16 02:01:12.428199112 +0000 UTC m=+0.564699841 container died b1556044450a17d05e5982e5b52eb64ef867673e3f6c06e6e0d18209d3ce8f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-16T02:01:12.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:12 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:12.774+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:12.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:12 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:12.774+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:41.093089+0000 front 2024-09-16T02:00:41.093037+0000 (oldest deadline 2024-09-16T02:01:09.893494+0000) 2024-09-16T02:01:12.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:12 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:12.539+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:12.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:12 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:12.539+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:12.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:12 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:12.539+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:13.111 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T02:01:13.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:01:13.194 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T02:01:13.195 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-16T02:01:13.195 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.6 2024-09-16T02:01:13.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:13 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:13.198+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:13.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:13 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:13.198+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:13.531 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:13 smithi139 systemd[1]: Stopping Ceph osd.6 for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:01:13.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:13 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:13.328+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:13.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:13 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:13.328+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:13.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:13 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:13.215+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:13.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:13 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:13.215+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:13.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:13 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:13.215+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:13.799 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:13 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:13.529+0000 7f3731164640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:01:13.799 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:13 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:13.529+0000 7f3731164640 -1 osd.6 124 *** Got signal Terminated *** 2024-09-16T02:01:13.799 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:13 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:13.529+0000 7f3731164640 -1 osd.6 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:01:13.800 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:13 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:13.552+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:13.800 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:13 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:13.552+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:13.800 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:13 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:13.552+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:14.167 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:13 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:13.718+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:14.167 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:13 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:13.718+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:14.172 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:13 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:13.806+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:14.173 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:13 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:13.806+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:41.093089+0000 front 2024-09-16T02:00:41.093037+0000 (oldest deadline 2024-09-16T02:01:09.893494+0000) 2024-09-16T02:01:14.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:14 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:14.166+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:14.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:14 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:14.166+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:14.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:14 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:14.292+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:14.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:14 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:14.292+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:14.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:14 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:14.188+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:14.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:14 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:14.188+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:14.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:14 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:14.188+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:14.835 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:14 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:14.579+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:14.835 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:14 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:14.579+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:14.836 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:14 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:14.579+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:14.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:14 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:14.685+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:14.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:14 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:14.685+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:15.173 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:14 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:14.833+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:15.173 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:14 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:14.833+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:41.093089+0000 front 2024-09-16T02:00:41.093037+0000 (oldest deadline 2024-09-16T02:01:09.893494+0000) 2024-09-16T02:01:15.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:15 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:15.132+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:15.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:15 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:15.132+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:15.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:15 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:15.299+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:15.546 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:15 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:15.299+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:15.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:15 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:15.226+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:15.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:15 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:15.226+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:15.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:15 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:15.226+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:15.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:15 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:15.226+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:15.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:15 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:15.817+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:15.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:15 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:15.817+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:41.093089+0000 front 2024-09-16T02:00:41.093037+0000 (oldest deadline 2024-09-16T02:01:09.893494+0000) 2024-09-16T02:01:15.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:15 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:15.619+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:15.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:15 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:15.619+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:15.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:15 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:15.619+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:15.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:15 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:15.619+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:16.095 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:15 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:15.733+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:16.095 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:15 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:15.733+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:16.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:16 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:16.093+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:16.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:16 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:16.093+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:16.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:16 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:16.252+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:16.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:16 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:16.252+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:16.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:16 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:16.248+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:16.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:16 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:16.248+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:16.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:16 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:16.248+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:16.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:16 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:16.248+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:16.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:16 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:16.830+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:16.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:16 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:16.830+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:41.093089+0000 front 2024-09-16T02:00:41.093037+0000 (oldest deadline 2024-09-16T02:01:09.893494+0000) 2024-09-16T02:01:16.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:16 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:16.659+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:16.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:16 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:16.659+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:16.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:16 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:16.659+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:16.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:16 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:16.659+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:17.086 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:16 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:16.689+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:17.086 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:16 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:16.689+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:17.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:17 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:17.085+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:17.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:17 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:17.085+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:17.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:17 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:17.222+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:17.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:17 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:17.222+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:17.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:17 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:17.255+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:17.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:17 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:17.255+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:17.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:17 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:17.255+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:17.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:17 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:17.255+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:18.089 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:17 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:17.720+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:18.089 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:17 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:17.720+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:18.089 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:17 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:17.720+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:18.173 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:17 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:17.790+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:36.992342+0000 front 2024-09-16T02:00:36.992307+0000 (oldest deadline 2024-09-16T02:01:01.092476+0000) 2024-09-16T02:01:18.173 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:17 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:17.790+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:41.093089+0000 front 2024-09-16T02:00:41.093037+0000 (oldest deadline 2024-09-16T02:01:09.893494+0000) 2024-09-16T02:01:18.173 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:17 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6[55082]: 2024-09-16T02:01:17.790+0000 7f3729772640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.794443+0000 front 2024-09-16T02:00:52.794505+0000 (oldest deadline 2024-09-16T02:01:17.494511+0000) 2024-09-16T02:01:18.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:17 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:17.683+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:18.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:17 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:17.683+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:18.174 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:17 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:17.683+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:18.174 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:17 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:17.683+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:18.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:18 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:18.087+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:18.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:18 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:18.087+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:18.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:18 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:18.188+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:18.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:18 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:18.188+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:18.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:18 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:18.299+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:18.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:18 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:18.299+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:18.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:18 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:18.299+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:18.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:18 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:18.299+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:18.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:18 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:18.641+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:18.924 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:18 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:18.641+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:18.924 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:18 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:18.641+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:18.924 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:18 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:18.641+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:19.139 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:18 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:18.707+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:19.139 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:18 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:18.707+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:19.139 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:18 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:18.707+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:19.398 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:19 smithi139 podman[107531]: 2024-09-16 02:01:19.125097399 +0000 UTC m=+5.730044224 container died 87c1d82250ac535c7cedb37b56133a3d051c734c879ec42cbe287c065a46489c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:01:19.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:19 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:19.137+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:19.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:19 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:19.137+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:19.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:19 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:19.137+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:19.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:19 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:19.192+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:19.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:19 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:19.192+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:19.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:19 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:19.304+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:19.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:19 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:19.304+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:19.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:19 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:19.304+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:19.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:19 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:19.304+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:19.665 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:19 smithi139 podman[107531]: 2024-09-16 02:01:19.579988025 +0000 UTC m=+6.184934839 container cleanup 87c1d82250ac535c7cedb37b56133a3d051c734c879ec42cbe287c065a46489c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:01:19.666 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:19 smithi139 bash[107531]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6 2024-09-16T02:01:19.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:19 smithi139 podman[107560]: 2024-09-16 02:01:19.8981727 +0000 UTC m=+0.773592038 container remove 87c1d82250ac535c7cedb37b56133a3d051c734c879ec42cbe287c065a46489c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T02:01:19.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:19 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:19.663+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:19.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:19 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:19.663+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:19.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:19 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:19.663+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:19.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:19 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:19.663+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:19.936 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:19 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:19.683+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:19.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:19 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:19.683+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:19.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:19 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:19.683+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:20.436 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:20 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:20.111+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:20.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:20 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:20.111+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:20.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:20 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:20.111+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:20.494 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:20 smithi139 podman[107649]: 2024-09-16 02:01:20.109749591 +0000 UTC m=+0.015675818 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:01:20.494 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:20 smithi139 podman[107649]: 2024-09-16 02:01:20.240987701 +0000 UTC m=+0.146913963 container create a9a52420ec63c37e115119ea1cd3aee6cb73c0ab952ad1fc25034e433f5de56e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T02:01:20.494 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:20 smithi139 podman[107649]: 2024-09-16 02:01:20.493382541 +0000 UTC m=+0.399308760 container init a9a52420ec63c37e115119ea1cd3aee6cb73c0ab952ad1fc25034e433f5de56e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T02:01:20.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:20 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:20.230+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:20.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:20 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:20.230+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:20.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:20 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:20.345+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:20.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:20 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:20.345+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:20.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:20 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:20.345+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:20.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:20 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:20.345+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:20.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:20 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:20.680+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:20.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:20 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:20.680+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:20.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:20 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:20.680+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:20.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:20 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:20.680+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:20.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:20 smithi139 podman[107649]: 2024-09-16 02:01:20.496975413 +0000 UTC m=+0.402901630 container start a9a52420ec63c37e115119ea1cd3aee6cb73c0ab952ad1fc25034e433f5de56e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T02:01:20.923 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:20 smithi139 podman[107649]: 2024-09-16 02:01:20.727306717 +0000 UTC m=+0.633233016 container attach a9a52420ec63c37e115119ea1cd3aee6cb73c0ab952ad1fc25034e433f5de56e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T02:01:20.924 INFO:journalctl@ceph.osd.6.smithi139.stdout:Sep 16 02:01:20 smithi139 podman[107649]: 2024-09-16 02:01:20.727607094 +0000 UTC m=+0.633533316 container died a9a52420ec63c37e115119ea1cd3aee6cb73c0ab952ad1fc25034e433f5de56e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:01:21.098 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:20 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:20.706+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:21.099 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:20 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:20.706+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:21.099 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:20 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:20.706+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:21.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:21 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:21.097+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:21.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:21 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:21.097+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:21.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:21 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:21.097+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:21.444 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.6.service' 2024-09-16T02:01:21.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:01:21.484 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-16T02:01:21.484 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-16T02:01:21.484 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.7 2024-09-16T02:01:21.511 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:21.221+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:21.511 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:21.221+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:21.511 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:21.221+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:49.742683+0000 front 2024-09-16T02:00:49.742699+0000 (oldest deadline 2024-09-16T02:01:20.343148+0000) 2024-09-16T02:01:21.512 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:21.356+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:21.512 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:21.356+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:21.512 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:21.356+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:21.512 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:21.356+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:21.791 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:21 smithi164 systemd[1]: Stopping Ceph osd.7 for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:01:21.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:21 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:21.668+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:21.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:21 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:21.668+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:21.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:21 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:21.668+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:21.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:21 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:21.668+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:21.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:21 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:21.674+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:21.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:21 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:21.674+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:21.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:21 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:21.674+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:22.045 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:21.790+0000 7f2e2c266640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:01:22.045 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:21.790+0000 7f2e2c266640 -1 osd.7 124 *** Got signal Terminated *** 2024-09-16T02:01:22.045 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:21 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:21.790+0000 7f2e2c266640 -1 osd.7 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:01:22.373 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:22 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:22.138+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:22.373 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:22 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:22.138+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:22.373 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:22 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:22.138+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:22.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:22.223+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:22.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:22.223+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:22.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:22.223+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:49.742683+0000 front 2024-09-16T02:00:49.742699+0000 (oldest deadline 2024-09-16T02:01:20.343148+0000) 2024-09-16T02:01:22.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:22.335+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:22.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:22.335+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:22.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:22.335+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:22.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:22.335+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:22.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:22 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:22.641+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:22.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:22 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:22.641+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:22.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:22 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:22.641+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:22.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:22 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:22.641+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:23.186 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:22 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:22.713+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:23.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:22 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:22.713+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:23.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:22 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:22.713+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:23.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:23 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:23.250+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:23.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:23 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:23.250+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:23.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:23 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:23.250+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:49.742683+0000 front 2024-09-16T02:00:49.742699+0000 (oldest deadline 2024-09-16T02:01:20.343148+0000) 2024-09-16T02:01:23.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:23 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:23.362+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:23.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:23 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:23.362+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:23.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:23 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:23.362+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:23.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:23 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:23.362+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:23.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:23 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:23.362+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:23.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:23 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:23.186+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:23.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:23 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:23.186+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:23.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:23 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:23.186+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:24.168 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:23 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:23.738+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:24.168 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:23 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:23.738+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:24.168 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:23 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:23.738+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:24.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:23.674+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:24.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:23.674+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:24.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:23.674+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:24.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:23 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:23.674+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:24.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:24 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:24.167+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:24.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:24 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:24.167+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:24.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:24 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:24.167+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:24.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:24 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:24.281+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:24.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:24 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:24.281+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:24.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:24 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:24.281+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:49.742683+0000 front 2024-09-16T02:00:49.742699+0000 (oldest deadline 2024-09-16T02:01:20.343148+0000) 2024-09-16T02:01:24.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:24 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:24.337+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:24.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:24 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:24.337+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:24.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:24 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:24.337+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:24.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:24 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:24.337+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:24.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:24 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:24.337+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:25.146 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:24 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:24.695+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:25.146 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:24 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:24.695+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:25.146 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:24 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:24.695+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:25.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:24.705+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:25.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:24.705+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:25.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:24.705+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:25.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:24.705+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:25.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:24 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:24.705+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:25.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:25 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:25.145+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:25.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:25 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:25.145+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:25.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:25 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:25.145+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:25.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:25 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:25.145+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:25.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:25 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:25.289+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:25.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:25 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:25.289+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:25.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:25 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:25.289+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:49.742683+0000 front 2024-09-16T02:00:49.742699+0000 (oldest deadline 2024-09-16T02:01:20.343148+0000) 2024-09-16T02:01:25.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:25 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:25.289+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:00.844118+0000 front 2024-09-16T02:01:00.844161+0000 (oldest deadline 2024-09-16T02:01:24.944168+0000) 2024-09-16T02:01:25.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:25 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:25.387+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:25.545 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:25 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:25.387+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:25.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:25 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:25.387+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:25.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:25 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:25.387+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:25.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:25 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:25.387+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:25.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:25 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:25.661+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:25.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:25 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:25.661+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:25.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:25 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:25.661+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:25.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:25 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:25.661+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:25.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:25 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:25.661+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:26.124 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:25 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:25.714+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:26.124 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:25 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:25.714+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:26.124 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:25 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:25.714+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:26.125 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:25 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:25.714+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:01.910051+0000 front 2024-09-16T02:01:01.909966+0000 (oldest deadline 2024-09-16T02:01:24.809973+0000) 2024-09-16T02:01:26.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:26 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:26.123+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:26.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:26 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:26.123+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:26.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:26 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:26.123+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:26.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:26 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:26.123+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:26.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:26 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:26.277+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:37.039939+0000 front 2024-09-16T02:00:37.039894+0000 (oldest deadline 2024-09-16T02:00:59.940048+0000) 2024-09-16T02:01:26.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:26 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:26.277+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:42.741472+0000 front 2024-09-16T02:00:42.741520+0000 (oldest deadline 2024-09-16T02:01:06.841406+0000) 2024-09-16T02:01:26.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:26 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:26.277+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:49.742683+0000 front 2024-09-16T02:00:49.742699+0000 (oldest deadline 2024-09-16T02:01:20.343148+0000) 2024-09-16T02:01:26.545 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:26 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7[52556]: 2024-09-16T02:01:26.277+0000 7f2e24874640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:00.844118+0000 front 2024-09-16T02:01:00.844161+0000 (oldest deadline 2024-09-16T02:01:24.944168+0000) 2024-09-16T02:01:26.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:26 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:26.396+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:26.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:26 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:26.396+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:26.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:26 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:26.396+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:26.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:26 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:26.396+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:26.546 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:26 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:26.396+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:27.105 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:26 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:26.721+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:27.106 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:26 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:26.721+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:27.106 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:26 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:26.721+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:27.106 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:26 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:26.721+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:01.910051+0000 front 2024-09-16T02:01:01.909966+0000 (oldest deadline 2024-09-16T02:01:24.809973+0000) 2024-09-16T02:01:27.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:26 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:26.683+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:27.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:26 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:26.683+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:27.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:26 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:26.683+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:27.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:26 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:26.683+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:27.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:26 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:26.683+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:27.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:27 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:27.104+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:27.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:27 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:27.104+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:27.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:27 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:27.104+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:27.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:27 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:27.104+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:27.716 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:27 smithi164 podman[72944]: 2024-09-16 02:01:27.37743428 +0000 UTC m=+5.693363332 container died 87604f5cf10e26ea22338e09050a11f671bcbe34a43f32a33a587fdbdd6bb067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-16T02:01:27.717 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:27 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:27.397+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:27.717 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:27 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:27.397+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:27.717 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:27 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:27.397+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:27.717 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:27 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:27.397+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:27.717 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:27 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:27.397+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:27.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:27 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:27.667+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:27.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:27 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:27.667+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:27.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:27 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:27.667+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:27.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:27 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:27.667+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:27.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:27 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:27.667+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:27.992 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:27 smithi164 podman[72944]: 2024-09-16 02:01:27.815096818 +0000 UTC m=+6.131025867 container cleanup 87604f5cf10e26ea22338e09050a11f671bcbe34a43f32a33a587fdbdd6bb067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS) 2024-09-16T02:01:27.992 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:27 smithi164 bash[72944]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7 2024-09-16T02:01:28.142 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:27 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:27.767+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:28.142 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:27 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:27.767+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:28.142 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:27 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:27.767+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:28.142 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:27 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:27.767+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:01.910051+0000 front 2024-09-16T02:01:01.909966+0000 (oldest deadline 2024-09-16T02:01:24.809973+0000) 2024-09-16T02:01:28.295 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:27 smithi164 podman[72971]: 2024-09-16 02:01:27.991548509 +0000 UTC m=+0.610023712 container remove 87604f5cf10e26ea22338e09050a11f671bcbe34a43f32a33a587fdbdd6bb067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T02:01:28.436 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:28 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:28.140+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:28.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:28 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:28.140+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:28.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:28 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:28.140+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:28.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:28 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:28.140+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:28.687 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:28 smithi164 podman[73059]: 2024-09-16 02:01:28.330287624 +0000 UTC m=+0.015808889 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:01:28.687 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:28 smithi164 podman[73059]: 2024-09-16 02:01:28.450790498 +0000 UTC m=+0.136311804 container create 07262a427ca9f7adab69d9e3cc6d6179e5943fe039c0e3743afdbac12349a07e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:01:28.687 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:28 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:28.447+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:28.688 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:28 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:28.447+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:28.688 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:28 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:28.447+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:28.688 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:28 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:28.447+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:28.688 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:28 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:28.447+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:28.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:28 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:28.629+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:28.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:28 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:28.629+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:28.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:28 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:28.629+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:28.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:28 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:28.629+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:28.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:28 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:28.629+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:29.045 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:28 smithi164 podman[73059]: 2024-09-16 02:01:28.686599099 +0000 UTC m=+0.372120359 container init 07262a427ca9f7adab69d9e3cc6d6179e5943fe039c0e3743afdbac12349a07e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T02:01:29.045 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:28 smithi164 podman[73059]: 2024-09-16 02:01:28.690310654 +0000 UTC m=+0.375831914 container start 07262a427ca9f7adab69d9e3cc6d6179e5943fe039c0e3743afdbac12349a07e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-16T02:01:29.046 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:28 smithi164 podman[73059]: 2024-09-16 02:01:28.786439244 +0000 UTC m=+0.471960513 container attach 07262a427ca9f7adab69d9e3cc6d6179e5943fe039c0e3743afdbac12349a07e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-16T02:01:29.046 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 16 02:01:28 smithi164 podman[73059]: 2024-09-16 02:01:28.877836603 +0000 UTC m=+0.563357871 container died 07262a427ca9f7adab69d9e3cc6d6179e5943fe039c0e3743afdbac12349a07e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:01:29.175 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:28 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:28.730+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:29.176 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:28 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:28.730+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:29.176 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:28 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:28.730+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:29.176 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:28 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:28.730+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:01.910051+0000 front 2024-09-16T02:01:01.909966+0000 (oldest deadline 2024-09-16T02:01:24.809973+0000) 2024-09-16T02:01:29.436 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:29 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:29.174+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:29.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:29 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:29.174+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:29.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:29 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:29.174+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:29.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:29 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:29.174+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:29.482 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:29 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:29.422+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:29.482 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:29 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:29.422+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:29.483 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:29 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:29.422+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:29.483 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:29 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:29.422+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:29.483 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:29 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:29.422+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:29.516 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.7.service' 2024-09-16T02:01:29.598 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:01:29.599 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-16T02:01:29.599 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-16T02:01:29.599 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.8 2024-09-16T02:01:29.919 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:29 smithi176 systemd[1]: Stopping Ceph osd.8 for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:01:29.920 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:29 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:29.701+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:29.920 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:29 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:29.701+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:29.920 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:29 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:29.701+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:29.920 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:29 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:29.701+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:01.910051+0000 front 2024-09-16T02:01:01.909966+0000 (oldest deadline 2024-09-16T02:01:24.809973+0000) 2024-09-16T02:01:29.920 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:29 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:29.918+0000 7fc28afa0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:01:29.920 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:29 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:29.918+0000 7fc28afa0640 -1 osd.8 124 *** Got signal Terminated *** 2024-09-16T02:01:29.920 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:29 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:29.918+0000 7fc28afa0640 -1 osd.8 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:01:29.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:29 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:29.613+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:29.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:29 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:29.613+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:29.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:29 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:29.613+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:29.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:29 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:29.613+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:29.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:29 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:29.613+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:30.169 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:30 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:30.168+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:30.169 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:30 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:30.168+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:30.170 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:30 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:30.168+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:30.170 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:30 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:30.168+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:30.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:30 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:30.437+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:30.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:30 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:30.437+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:30.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:30 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:30.437+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:30.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:30 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:30.437+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:30.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:30 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:30.437+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:30.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:30 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:30.587+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:30.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:30 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:30.587+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:30.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:30 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:30.587+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:30.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:30 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:30.587+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:30.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:30 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:30.587+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:30.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:30 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:30.675+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:30.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:30 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:30.675+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:30.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:30 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:30.675+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:30.937 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:30 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:30.675+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:01.910051+0000 front 2024-09-16T02:01:01.909966+0000 (oldest deadline 2024-09-16T02:01:24.809973+0000) 2024-09-16T02:01:31.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:31 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:31.213+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:31.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:31 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:31.213+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:31.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:31 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:31.213+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:31.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:31 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:31.213+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:31.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:31 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:31.481+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:31.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:31 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:31.481+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:31.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:31 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:31.481+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:31.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:31 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:31.481+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:31.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:31 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:31.481+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:31.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:31 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:31.543+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:31.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:31 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:31.543+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:31.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:31 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:31.543+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:31.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:31 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:31.543+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:31.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:31 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:31.543+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:32.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:31 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:31.716+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:32.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:31 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:31.716+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:32.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:31 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:31.716+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:32.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:31 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:31.716+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:01.910051+0000 front 2024-09-16T02:01:01.909966+0000 (oldest deadline 2024-09-16T02:01:24.809973+0000) 2024-09-16T02:01:32.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:32 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:32.190+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:32.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:32 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:32.190+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:32.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:32 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:32.190+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:32.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:32 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:32.190+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:32.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:32 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:32.443+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:32.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:32 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:32.443+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:32.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:32 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:32.443+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:32.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:32 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:32.443+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:32.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:32 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:32.443+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:32.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:32 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:32.579+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:32.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:32 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:32.579+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:32.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:32 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:32.579+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:32.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:32 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:32.579+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:32.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:32 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:32.579+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:33.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:32 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:32.709+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:33.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:32 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:32.709+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:33.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:32 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:32.709+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:33.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:32 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:32.709+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:01.910051+0000 front 2024-09-16T02:01:01.909966+0000 (oldest deadline 2024-09-16T02:01:24.809973+0000) 2024-09-16T02:01:33.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:33 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:33.235+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:33.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:33 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:33.235+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:33.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:33 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:33.235+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:33.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:33 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:33.235+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:33.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:33 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:33.470+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:33.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:33 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:33.470+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:33.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:33 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:33.470+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:33.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:33 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:33.470+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:33.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:33 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:33.470+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:33.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:33 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:33.545+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:33.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:33 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:33.545+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:33.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:33 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:33.545+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:33.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:33 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:33.545+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:33.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:33 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:33.545+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:34.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:33 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:33.746+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:34.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:33 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:33.746+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:34.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:33 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:33.746+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:34.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:33 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:33.746+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:01.910051+0000 front 2024-09-16T02:01:01.909966+0000 (oldest deadline 2024-09-16T02:01:24.809973+0000) 2024-09-16T02:01:34.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:34 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:34.228+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:34.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:34 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:34.228+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:34.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:34 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:34.228+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:34.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:34 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:34.228+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:34.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:34 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:34.228+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:34.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:34 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:34.438+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:34.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:34 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:34.438+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:34.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:34 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:34.438+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:34.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:34 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:34.438+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:34.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:34 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:34.438+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:34.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:34 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:34.438+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:34.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:34 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:34.546+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:34.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:34 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:34.546+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:34.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:34 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:34.546+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:34.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:34 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:34.546+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:34.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:34 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:34.546+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:34.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:34 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:34.546+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:09.835791+0000 front 2024-09-16T02:01:09.835949+0000 (oldest deadline 2024-09-16T02:01:34.535677+0000) 2024-09-16T02:01:35.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:34 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:34.746+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.405085+0000 front 2024-09-16T02:00:27.405074+0000 (oldest deadline 2024-09-16T02:00:55.005678+0000) 2024-09-16T02:01:35.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:34 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:34.746+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.607354+0000 front 2024-09-16T02:00:45.607310+0000 (oldest deadline 2024-09-16T02:01:09.107488+0000) 2024-09-16T02:01:35.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:34 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:34.746+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:52.508989+0000 front 2024-09-16T02:00:52.509058+0000 (oldest deadline 2024-09-16T02:01:17.208898+0000) 2024-09-16T02:01:35.187 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:34 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8[53256]: 2024-09-16T02:01:34.746+0000 7fc2875b6640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:01.910051+0000 front 2024-09-16T02:01:01.909966+0000 (oldest deadline 2024-09-16T02:01:24.809973+0000) 2024-09-16T02:01:35.537 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:35 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:35.227+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:35.537 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:35 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:35.227+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:35.537 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:35 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:35.227+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:35.538 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:35 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:35.227+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:35.538 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:35 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:35.227+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:35.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:35 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:35.458+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:35.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:35 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:35.458+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:35.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:35 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:35.458+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:35.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:35 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:35.458+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:35.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:35 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:35.458+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:35.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:35 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:35.458+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:35.881 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:35 smithi176 podman[73525]: 2024-09-16 02:01:35.536563117 +0000 UTC m=+5.738274846 container died a26a3d4316f9c6d7b689b7b69de1305a36b9bbb576f58c5a763fe0494360ccb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T02:01:35.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:35 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:35.549+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:35.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:35 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:35.549+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:35.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:35 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:35.549+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:35.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:35 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:35.549+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:35.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:35 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:35.549+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:35.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:35 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:35.549+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:09.835791+0000 front 2024-09-16T02:01:09.835949+0000 (oldest deadline 2024-09-16T02:01:34.535677+0000) 2024-09-16T02:01:36.149 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:36 smithi176 podman[73525]: 2024-09-16 02:01:36.126497994 +0000 UTC m=+6.328209713 container cleanup a26a3d4316f9c6d7b689b7b69de1305a36b9bbb576f58c5a763fe0494360ccb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-16T02:01:36.149 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:36 smithi176 bash[73525]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8 2024-09-16T02:01:36.687 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:36 smithi176 podman[73540]: 2024-09-16 02:01:36.39413751 +0000 UTC m=+0.854481827 container remove a26a3d4316f9c6d7b689b7b69de1305a36b9bbb576f58c5a763fe0494360ccb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T02:01:36.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:36 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:36.269+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:36.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:36 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:36.269+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:36.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:36 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:36.269+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:36.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:36 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:36.269+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:36.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:36 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:36.269+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:36.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:36 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:36.476+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:36.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:36 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:36.476+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:36.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:36 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:36.476+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:36.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:36 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:36.476+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:36.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:36 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:36.476+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:36.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:36 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:36.476+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:36.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:36 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:36.544+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:36.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:36 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:36.544+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:36.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:36 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:36.544+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:36.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:36 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:36.544+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:36.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:36 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:36.544+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:36.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:36 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:36.544+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:09.835791+0000 front 2024-09-16T02:01:09.835949+0000 (oldest deadline 2024-09-16T02:01:34.535677+0000) 2024-09-16T02:01:37.048 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:36 smithi176 podman[73631]: 2024-09-16 02:01:36.606646108 +0000 UTC m=+0.015746311 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:01:37.048 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:36 smithi176 podman[73631]: 2024-09-16 02:01:36.722722206 +0000 UTC m=+0.131822403 container create daa133b5b88f7ed9e19edf56e0c9150130fd751e663b5c3ff726558c6e1f501e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T02:01:37.048 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:36 smithi176 podman[73631]: 2024-09-16 02:01:36.947207199 +0000 UTC m=+0.356307392 container init daa133b5b88f7ed9e19edf56e0c9150130fd751e663b5c3ff726558c6e1f501e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-09-16T02:01:37.048 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:36 smithi176 podman[73631]: 2024-09-16 02:01:36.950406803 +0000 UTC m=+0.359507005 container start daa133b5b88f7ed9e19edf56e0c9150130fd751e663b5c3ff726558c6e1f501e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True) 2024-09-16T02:01:37.048 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:37 smithi176 podman[73631]: 2024-09-16 02:01:37.047225207 +0000 UTC m=+0.456325419 container attach daa133b5b88f7ed9e19edf56e0c9150130fd751e663b5c3ff726558c6e1f501e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T02:01:37.319 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 16 02:01:37 smithi176 podman[73631]: 2024-09-16 02:01:37.150294516 +0000 UTC m=+0.559394713 container died daa133b5b88f7ed9e19edf56e0c9150130fd751e663b5c3ff726558c6e1f501e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T02:01:37.319 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:37 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:37.269+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:37.319 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:37 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:37.269+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:37.319 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:37 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:37.269+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:37.319 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:37 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:37.269+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:37.319 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:37 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:37.269+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:37.749 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.8.service' 2024-09-16T02:01:37.788 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:01:37.788 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-16T02:01:37.788 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-16T02:01:37.789 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.9 2024-09-16T02:01:37.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:37 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:37.478+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:37.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:37 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:37.478+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:37.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:37 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:37.478+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:37.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:37 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:37.478+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:37.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:37 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:37.478+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:37.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:37 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:37.478+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:37.815 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:37 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:37.528+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:37.816 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:37 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:37.528+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:37.816 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:37 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:37.528+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:37.816 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:37 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:37.528+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:37.816 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:37 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:37.528+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:37.816 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:37 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:37.528+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:09.835791+0000 front 2024-09-16T02:01:09.835949+0000 (oldest deadline 2024-09-16T02:01:34.535677+0000) 2024-09-16T02:01:38.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:37 smithi139 systemd[1]: Stopping Ceph osd.9 for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:01:38.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:38 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:38.331+0000 7fcf5693a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:01:38.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:38 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:38.331+0000 7fcf5693a640 -1 osd.9 124 *** Got signal Terminated *** 2024-09-16T02:01:38.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:38 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:38.331+0000 7fcf5693a640 -1 osd.9 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:01:38.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:38 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:38.498+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:38.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:38 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:38.498+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:38.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:38 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:38.498+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:38.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:38 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:38.498+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:38.674 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:38 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:38.498+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:38.674 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:38 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:38.498+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:09.835791+0000 front 2024-09-16T02:01:09.835949+0000 (oldest deadline 2024-09-16T02:01:34.535677+0000) 2024-09-16T02:01:38.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:38 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:38.254+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:38.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:38 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:38.254+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:38.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:38 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:38.254+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:38.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:38 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:38.254+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:38.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:38 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:38.254+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:38.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:38 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:38.453+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:38.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:38 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:38.453+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:38.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:38 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:38.453+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:38.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:38 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:38.453+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:38.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:38 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:38.453+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:38.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:38 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:38.453+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:39.686 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:39 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:39.277+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:39.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:39 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:39.277+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:39.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:39 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:39.277+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:39.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:39 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:39.277+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:39.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:39 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:39.277+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:39.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:39 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:39.488+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:39.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:39 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:39.488+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:39.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:39 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:39.488+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:39.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:39 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:39.488+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:39.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:39 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:39.488+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:39.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:39 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:39.488+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:39.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:39 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:39.488+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:16.919832+0000 front 2024-09-16T02:01:16.919770+0000 (oldest deadline 2024-09-16T02:01:39.219549+0000) 2024-09-16T02:01:39.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:39 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:39.468+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:39.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:39 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:39.468+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:39.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:39 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:39.468+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:39.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:39 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:39.468+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:39.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:39 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:39.468+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:39.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:39 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:39.468+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:09.835791+0000 front 2024-09-16T02:01:09.835949+0000 (oldest deadline 2024-09-16T02:01:34.535677+0000) 2024-09-16T02:01:40.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:40 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:40.286+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:40.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:40 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:40.286+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:40.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:40 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:40.286+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:40.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:40 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:40.286+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:40.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:40 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:40.286+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:40.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:40 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:40.534+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:40.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:40 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:40.534+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:40.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:40 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:40.534+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:40.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:40 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:40.534+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:40.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:40 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:40.534+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:40.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:40 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:40.534+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:40.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:40 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:40.534+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:16.919832+0000 front 2024-09-16T02:01:16.919770+0000 (oldest deadline 2024-09-16T02:01:39.219549+0000) 2024-09-16T02:01:40.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:40 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:40.432+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:40.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:40 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:40.432+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:40.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:40 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:40.432+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:40.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:40 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:40.432+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:40.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:40 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:40.432+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:40.923 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:40 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:40.432+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:09.835791+0000 front 2024-09-16T02:01:09.835949+0000 (oldest deadline 2024-09-16T02:01:34.535677+0000) 2024-09-16T02:01:41.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:41 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:41.401+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:41.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:41 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:41.401+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:41.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:41 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:41.401+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:41.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:41 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:41.401+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:41.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:41 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:41.401+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:41.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:41 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:41.401+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:09.835791+0000 front 2024-09-16T02:01:09.835949+0000 (oldest deadline 2024-09-16T02:01:34.535677+0000) 2024-09-16T02:01:41.686 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:41 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:41.302+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:41.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:41 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:41.302+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:41.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:41 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:41.302+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:41.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:41 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:41.302+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:41.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:41 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:41.302+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:41.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:41 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:41.502+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:41.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:41 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:41.502+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:41.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:41 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:41.502+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:41.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:41 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:41.502+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:41.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:41 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:41.502+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:41.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:41 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:41.502+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:41.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:41 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:41.502+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:16.919832+0000 front 2024-09-16T02:01:16.919770+0000 (oldest deadline 2024-09-16T02:01:39.219549+0000) 2024-09-16T02:01:42.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:42.354+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:42.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:42.354+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:42.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:42.354+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:42.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:42.354+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:42.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:42.354+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:42.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:42 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:42.354+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:09.835791+0000 front 2024-09-16T02:01:09.835949+0000 (oldest deadline 2024-09-16T02:01:34.535677+0000) 2024-09-16T02:01:42.686 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:42 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:42.314+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:42.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:42 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:42.314+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:42.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:42 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:42.314+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:42.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:42 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:42.314+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:42.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:42 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:42.314+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:42.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:42 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:42.314+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:42.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:42 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:42.464+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:42.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:42 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:42.464+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:42.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:42 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:42.464+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:42.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:42 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:42.464+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:42.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:42 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:42.464+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:42.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:42 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:42.464+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:42.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:42 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:42.464+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:16.919832+0000 front 2024-09-16T02:01:16.919770+0000 (oldest deadline 2024-09-16T02:01:39.219549+0000) 2024-09-16T02:01:43.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:43.326+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.830010+0000 front 2024-09-16T02:00:27.829921+0000 (oldest deadline 2024-09-16T02:00:50.130137+0000) 2024-09-16T02:01:43.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:43.326+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.031217+0000 front 2024-09-16T02:00:33.031412+0000 (oldest deadline 2024-09-16T02:00:58.931202+0000) 2024-09-16T02:01:43.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:43.326+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:43.632166+0000 front 2024-09-16T02:00:43.632239+0000 (oldest deadline 2024-09-16T02:01:10.632501+0000) 2024-09-16T02:01:43.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:43.326+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:54.133468+0000 front 2024-09-16T02:00:54.133440+0000 (oldest deadline 2024-09-16T02:01:15.233458+0000) 2024-09-16T02:01:43.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:43.326+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.634998+0000 front 2024-09-16T02:00:58.635106+0000 (oldest deadline 2024-09-16T02:01:23.934781+0000) 2024-09-16T02:01:43.673 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:43 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9[61594]: 2024-09-16T02:01:43.326+0000 7fcf52f50640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:09.835791+0000 front 2024-09-16T02:01:09.835949+0000 (oldest deadline 2024-09-16T02:01:34.535677+0000) 2024-09-16T02:01:43.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:43 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:43.332+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:43.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:43 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:43.332+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:43.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:43 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:43.332+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:43.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:43 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:43.332+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:43.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:43 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:43.332+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:43.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:43 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:43.332+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:43.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:43 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:43.452+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:43.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:43 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:43.452+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:43.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:43 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:43.452+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:43.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:43 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:43.452+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:43.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:43 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:43.452+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:43.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:43 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:43.452+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:43.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:43 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:43.452+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:16.919832+0000 front 2024-09-16T02:01:16.919770+0000 (oldest deadline 2024-09-16T02:01:39.219549+0000) 2024-09-16T02:01:44.134 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:43 smithi139 podman[107752]: 2024-09-16 02:01:43.836587845 +0000 UTC m=+5.857101751 container died 3879fd37f96566a1ee7cc2b3d62298492e2ab24dcd286544bceac58add25deac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T02:01:44.423 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:44 smithi139 podman[107752]: 2024-09-16 02:01:44.282141618 +0000 UTC m=+6.302655523 container cleanup 3879fd37f96566a1ee7cc2b3d62298492e2ab24dcd286544bceac58add25deac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-16T02:01:44.423 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:44 smithi139 bash[107752]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9 2024-09-16T02:01:44.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:44 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:44.335+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:44.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:44 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:44.335+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:44.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:44 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:44.335+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:44.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:44 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:44.335+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:44.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:44 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:44.335+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:44.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:44 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:44.335+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:44.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:44 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:44.469+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:44.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:44 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:44.469+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:44.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:44 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:44.469+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:44.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:44 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:44.469+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:44.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:44 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:44.469+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:44.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:44 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:44.469+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:44.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:44 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:44.469+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:16.919832+0000 front 2024-09-16T02:01:16.919770+0000 (oldest deadline 2024-09-16T02:01:39.219549+0000) 2024-09-16T02:01:44.878 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:44 smithi139 podman[107767]: 2024-09-16 02:01:44.558520724 +0000 UTC m=+0.715298752 container remove 3879fd37f96566a1ee7cc2b3d62298492e2ab24dcd286544bceac58add25deac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:01:45.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:44 smithi139 podman[107864]: 2024-09-16 02:01:44.781435916 +0000 UTC m=+0.018248451 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:01:45.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:44 smithi139 podman[107864]: 2024-09-16 02:01:44.892549136 +0000 UTC m=+0.129361657 container create 4e52b908a0014c5a6f3a41fd8428ffebdce854379119830cf792c8214becc1fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T02:01:45.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:45 smithi139 podman[107864]: 2024-09-16 02:01:45.120103585 +0000 UTC m=+0.356916121 container init 4e52b908a0014c5a6f3a41fd8428ffebdce854379119830cf792c8214becc1fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T02:01:45.173 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:45 smithi139 podman[107864]: 2024-09-16 02:01:45.123427187 +0000 UTC m=+0.360239717 container start 4e52b908a0014c5a6f3a41fd8428ffebdce854379119830cf792c8214becc1fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T02:01:45.623 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:45 smithi139 podman[107864]: 2024-09-16 02:01:45.207708505 +0000 UTC m=+0.444521031 container attach 4e52b908a0014c5a6f3a41fd8428ffebdce854379119830cf792c8214becc1fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:01:45.624 INFO:journalctl@ceph.osd.9.smithi139.stdout:Sep 16 02:01:45 smithi139 podman[107864]: 2024-09-16 02:01:45.323000376 +0000 UTC m=+0.559812909 container died 4e52b908a0014c5a6f3a41fd8428ffebdce854379119830cf792c8214becc1fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-16T02:01:45.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:45 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:45.310+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:45.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:45 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:45.310+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:45.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:45 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:45.310+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:45.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:45 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:45.310+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:45.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:45 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:45.310+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:45.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:45 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:45.310+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:45.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:45 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:45.475+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:45.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:45 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:45.475+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:45.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:45 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:45.475+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:45.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:45 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:45.475+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:45.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:45 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:45.475+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:45.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:45 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:45.475+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:45.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:45 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:45.475+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:16.919832+0000 front 2024-09-16T02:01:16.919770+0000 (oldest deadline 2024-09-16T02:01:39.219549+0000) 2024-09-16T02:01:45.927 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.9.service' 2024-09-16T02:01:46.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:01:46.010 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-16T02:01:46.011 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-16T02:01:46.011 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.10 2024-09-16T02:01:46.295 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:46 smithi164 systemd[1]: Stopping Ceph osd.10 for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:01:46.689 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:46 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:46.277+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:46.689 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:46 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:46.277+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:46.689 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:46 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:46.277+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:46.689 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:46 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:46.277+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:46.689 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:46 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:46.277+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:46.689 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:46 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:46.277+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:46.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:46 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:46.334+0000 7f781c816640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:01:46.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:46 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:46.334+0000 7f781c816640 -1 osd.10 124 *** Got signal Terminated *** 2024-09-16T02:01:46.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:46 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:46.334+0000 7f781c816640 -1 osd.10 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:01:46.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:46 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:46.427+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:46.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:46 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:46.427+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:46.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:46 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:46.427+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:46.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:46 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:46.427+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:46.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:46 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:46.427+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:46.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:46 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:46.427+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:46.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:46 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:46.427+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:16.919832+0000 front 2024-09-16T02:01:16.919770+0000 (oldest deadline 2024-09-16T02:01:39.219549+0000) 2024-09-16T02:01:47.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:47 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:47.322+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:47.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:47 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:47.322+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:47.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:47 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:47.322+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:47.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:47 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:47.322+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:47.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:47 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:47.322+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:47.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:47 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:47.322+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:47.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:47 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:47.415+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:47.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:47 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:47.415+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:47.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:47 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:47.415+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:47.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:47 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:47.415+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:47.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:47 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:47.415+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:47.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:47 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:47.415+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:47.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:47 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:47.415+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:16.919832+0000 front 2024-09-16T02:01:16.919770+0000 (oldest deadline 2024-09-16T02:01:39.219549+0000) 2024-09-16T02:01:48.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:48 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:48.273+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:48.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:48 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:48.273+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:48.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:48 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:48.273+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:48.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:48 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:48.273+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:48.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:48 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:48.273+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:48.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:48 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:48.273+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:48.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:48 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:48.273+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-09-16T02:01:25.995038+0000 front 2024-09-16T02:01:25.995056+0000 (oldest deadline 2024-09-16T02:01:47.695024+0000) 2024-09-16T02:01:48.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:48 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:48.432+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:48.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:48 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:48.433+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:48.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:48 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:48.433+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:48.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:48 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:48.433+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:48.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:48 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:48.433+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:48.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:48 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:48.433+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:48.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:48 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:48.433+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:16.919832+0000 front 2024-09-16T02:01:16.919770+0000 (oldest deadline 2024-09-16T02:01:39.219549+0000) 2024-09-16T02:01:49.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:49 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:49.246+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:49.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:49 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:49.246+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:49.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:49 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:49.246+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:49.688 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:49 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:49.246+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:49.688 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:49 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:49.246+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:49.688 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:49 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:49.246+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:49.688 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:49 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:49.246+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-09-16T02:01:25.995038+0000 front 2024-09-16T02:01:25.995056+0000 (oldest deadline 2024-09-16T02:01:47.695024+0000) 2024-09-16T02:01:49.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:49 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:49.394+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:49.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:49 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:49.394+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:49.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:49 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:49.394+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:49.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:49 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:49.394+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:49.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:49 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:49.394+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:49.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:49 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:49.394+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:49.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:49 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:49.394+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:16.919832+0000 front 2024-09-16T02:01:16.919770+0000 (oldest deadline 2024-09-16T02:01:39.219549+0000) 2024-09-16T02:01:50.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:50 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:50.280+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:50.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:50 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:50.280+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:50.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:50 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:50.280+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:50.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:50 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:50.280+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:50.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:50 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:50.280+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:50.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:50 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:50.280+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:50.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:50 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:50.280+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-09-16T02:01:25.995038+0000 front 2024-09-16T02:01:25.995056+0000 (oldest deadline 2024-09-16T02:01:47.695024+0000) 2024-09-16T02:01:50.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:50 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:50.358+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:29.110282+0000 front 2024-09-16T02:00:29.110313+0000 (oldest deadline 2024-09-16T02:00:53.810266+0000) 2024-09-16T02:01:50.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:50 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:50.358+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-09-16T02:00:33.810760+0000 front 2024-09-16T02:00:33.810787+0000 (oldest deadline 2024-09-16T02:01:00.211457+0000) 2024-09-16T02:01:50.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:50 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:50.358+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:44.713361+0000 front 2024-09-16T02:00:44.713475+0000 (oldest deadline 2024-09-16T02:01:08.213305+0000) 2024-09-16T02:01:50.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:50 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:50.358+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.114223+0000 front 2024-09-16T02:00:51.114126+0000 (oldest deadline 2024-09-16T02:01:15.214253+0000) 2024-09-16T02:01:50.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:50 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:50.358+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:01:02.115965+0000 front 2024-09-16T02:01:02.115995+0000 (oldest deadline 2024-09-16T02:01:23.116474+0000) 2024-09-16T02:01:50.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:50 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:50.358+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.917864+0000 front 2024-09-16T02:01:08.917931+0000 (oldest deadline 2024-09-16T02:01:33.617832+0000) 2024-09-16T02:01:50.796 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:50 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10[59128]: 2024-09-16T02:01:50.358+0000 7f7814e24640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:16.919832+0000 front 2024-09-16T02:01:16.919770+0000 (oldest deadline 2024-09-16T02:01:39.219549+0000) 2024-09-16T02:01:51.589 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:51 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:51.292+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:51.589 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:51 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:51.292+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:51.589 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:51 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:51.292+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:51.589 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:51 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:51.292+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:51.589 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:51 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:51.292+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:51.589 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:51 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:51.292+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:51.589 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:51 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:51.292+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-09-16T02:01:25.995038+0000 front 2024-09-16T02:01:25.995056+0000 (oldest deadline 2024-09-16T02:01:47.695024+0000) 2024-09-16T02:01:52.159 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:51 smithi164 podman[73163]: 2024-09-16 02:01:51.882946441 +0000 UTC m=+5.681810888 container died ef6ac774dcea62dcc351357960d22ae91a77d43ea93a465438eb3c593e9032e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:01:52.485 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:52 smithi164 podman[73163]: 2024-09-16 02:01:52.258445144 +0000 UTC m=+6.057309592 container cleanup ef6ac774dcea62dcc351357960d22ae91a77d43ea93a465438eb3c593e9032e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-16T02:01:52.485 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:52 smithi164 bash[73163]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10 2024-09-16T02:01:52.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:52 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:52.279+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:52.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:52 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:52.279+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:52.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:52 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:52.279+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:52.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:52 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:52.279+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:52.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:52 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:52.279+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:52.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:52 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:52.279+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:52.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:52 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:52.279+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-09-16T02:01:25.995038+0000 front 2024-09-16T02:01:25.995056+0000 (oldest deadline 2024-09-16T02:01:47.695024+0000) 2024-09-16T02:01:52.795 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:52 smithi164 podman[73180]: 2024-09-16 02:01:52.484679784 +0000 UTC m=+0.599013562 container remove ef6ac774dcea62dcc351357960d22ae91a77d43ea93a465438eb3c593e9032e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-8921c8e) 2024-09-16T02:01:53.082 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:52 smithi164 podman[73267]: 2024-09-16 02:01:52.711498176 +0000 UTC m=+0.015629216 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:01:53.083 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:52 smithi164 podman[73267]: 2024-09-16 02:01:52.812795468 +0000 UTC m=+0.116926500 container create bacfd834a6684d620ea5b7432fb64284fc97b3ba3776a5d7ff45fe75f2727bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T02:01:53.083 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:53 smithi164 podman[73267]: 2024-09-16 02:01:53.0066321 +0000 UTC m=+0.310763131 container init bacfd834a6684d620ea5b7432fb64284fc97b3ba3776a5d7ff45fe75f2727bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T02:01:53.083 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:53 smithi164 podman[73267]: 2024-09-16 02:01:53.009783273 +0000 UTC m=+0.313914313 container start bacfd834a6684d620ea5b7432fb64284fc97b3ba3776a5d7ff45fe75f2727bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-16T02:01:53.083 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:53 smithi164 podman[73267]: 2024-09-16 02:01:53.081573178 +0000 UTC m=+0.385704238 container attach bacfd834a6684d620ea5b7432fb64284fc97b3ba3776a5d7ff45fe75f2727bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:01:53.485 INFO:journalctl@ceph.osd.10.smithi164.stdout:Sep 16 02:01:53 smithi164 podman[73267]: 2024-09-16 02:01:53.193229505 +0000 UTC m=+0.497360548 container died bacfd834a6684d620ea5b7432fb64284fc97b3ba3776a5d7ff45fe75f2727bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T02:01:53.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:53.238+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:53.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:53.238+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:53.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:53.238+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:53.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:53.238+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:53.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:53.238+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:53.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:53.238+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:53.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:53 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:53.238+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-09-16T02:01:25.995038+0000 front 2024-09-16T02:01:25.995056+0000 (oldest deadline 2024-09-16T02:01:47.695024+0000) 2024-09-16T02:01:53.772 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.10.service' 2024-09-16T02:01:53.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:01:53.809 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-16T02:01:53.809 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-16T02:01:53.809 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.11 2024-09-16T02:01:54.113 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:53 smithi176 systemd[1]: Stopping Ceph osd.11 for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:01:54.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:54 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:54.111+0000 7fefd4de5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:01:54.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:54 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:54.111+0000 7fefd4de5640 -1 osd.11 124 *** Got signal Terminated *** 2024-09-16T02:01:54.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:54 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:54.111+0000 7fefd4de5640 -1 osd.11 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T02:01:54.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:54 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:54.211+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:54.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:54 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:54.211+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:54.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:54 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:54.211+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:54.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:54 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:54.211+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:54.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:54 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:54.211+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:54.438 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:54 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:54.211+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:54.438 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:54 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:54.211+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-09-16T02:01:25.995038+0000 front 2024-09-16T02:01:25.995056+0000 (oldest deadline 2024-09-16T02:01:47.695024+0000) 2024-09-16T02:01:55.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:55 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:55.251+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:55.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:55 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:55.251+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:55.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:55 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:55.251+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:55.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:55 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:55.251+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:55.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:55 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:55.251+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:55.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:55 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:55.251+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:55.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:55 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:55.251+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-09-16T02:01:25.995038+0000 front 2024-09-16T02:01:25.995056+0000 (oldest deadline 2024-09-16T02:01:47.695024+0000) 2024-09-16T02:01:56.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:56 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:56.245+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:56.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:56 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:56.245+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:56.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:56 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:56.245+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:56.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:56 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:56.245+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:56.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:56 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:56.245+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:56.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:56 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:56.245+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:56.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:56 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:56.245+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-09-16T02:01:25.995038+0000 front 2024-09-16T02:01:25.995056+0000 (oldest deadline 2024-09-16T02:01:47.695024+0000) 2024-09-16T02:01:57.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:57 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:57.216+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:57.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:57 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:57.216+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:57.688 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:57 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:57.216+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:57.688 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:57 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:57.216+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:57.688 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:57 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:57.216+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:57.688 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:57 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:57.216+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:57.688 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:57 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:57.216+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-09-16T02:01:25.995038+0000 front 2024-09-16T02:01:25.995056+0000 (oldest deadline 2024-09-16T02:01:47.695024+0000) 2024-09-16T02:01:57.688 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:57 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:57.216+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6820 osd.8 since back 2024-09-16T02:01:34.596985+0000 front 2024-09-16T02:01:34.597026+0000 (oldest deadline 2024-09-16T02:01:56.297097+0000) 2024-09-16T02:01:58.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:58 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:58.259+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6806 osd.0 since back 2024-09-16T02:00:27.984240+0000 front 2024-09-16T02:00:27.984250+0000 (oldest deadline 2024-09-16T02:00:50.884397+0000) 2024-09-16T02:01:58.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:58 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:58.259+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-16T02:00:45.387244+0000 front 2024-09-16T02:00:45.387308+0000 (oldest deadline 2024-09-16T02:01:06.987830+0000) 2024-09-16T02:01:58.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:58 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:58.259+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6814 osd.3 since back 2024-09-16T02:00:51.589691+0000 front 2024-09-16T02:00:51.589701+0000 (oldest deadline 2024-09-16T02:01:18.590053+0000) 2024-09-16T02:01:58.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:58 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:58.259+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-09-16T02:00:58.590670+0000 front 2024-09-16T02:00:58.590644+0000 (oldest deadline 2024-09-16T02:01:24.490548+0000) 2024-09-16T02:01:58.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:58 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:58.259+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-16T02:01:08.492215+0000 front 2024-09-16T02:01:08.492563+0000 (oldest deadline 2024-09-16T02:01:33.792225+0000) 2024-09-16T02:01:58.687 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:58 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:58.259+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.139:6822 osd.6 since back 2024-09-16T02:01:17.893415+0000 front 2024-09-16T02:01:17.893304+0000 (oldest deadline 2024-09-16T02:01:41.993162+0000) 2024-09-16T02:01:58.688 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:58 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:58.259+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-09-16T02:01:25.995038+0000 front 2024-09-16T02:01:25.995056+0000 (oldest deadline 2024-09-16T02:01:47.695024+0000) 2024-09-16T02:01:58.688 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:58 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11[60253]: 2024-09-16T02:01:58.259+0000 7fefcd3f3640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.176:6820 osd.8 since back 2024-09-16T02:01:34.596985+0000 front 2024-09-16T02:01:34.597026+0000 (oldest deadline 2024-09-16T02:01:56.297097+0000) 2024-09-16T02:02:00.006 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:01:59 smithi176 podman[73759]: 2024-09-16 02:01:59.743483717 +0000 UTC m=+5.740081111 container died 1f9d6dc4cc16624607d25778191e571a19494b4f517713eda6dab3f551f6a4c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-16T02:02:01.420 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:02:00 smithi176 podman[73759]: 2024-09-16 02:02:00.09470492 +0000 UTC m=+6.091302315 container cleanup 1f9d6dc4cc16624607d25778191e571a19494b4f517713eda6dab3f551f6a4c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T02:02:01.420 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:02:00 smithi176 bash[73759]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11 2024-09-16T02:02:01.420 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:02:00 smithi176 podman[73775]: 2024-09-16 02:02:00.412978076 +0000 UTC m=+0.670992920 container remove 1f9d6dc4cc16624607d25778191e571a19494b4f517713eda6dab3f551f6a4c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-16T02:02:01.420 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:02:00 smithi176 podman[73861]: 2024-09-16 02:02:00.662889553 +0000 UTC m=+0.015854825 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T02:02:01.420 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:02:00 smithi176 podman[73861]: 2024-09-16 02:02:00.771510064 +0000 UTC m=+0.124475364 container create f5cbfea2440bf0983a45f9c0fef99e375ccdc356b7bd46a45d4353a873465c87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T02:02:01.420 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:02:00 smithi176 podman[73861]: 2024-09-16 02:02:00.965531897 +0000 UTC m=+0.318497155 container init f5cbfea2440bf0983a45f9c0fef99e375ccdc356b7bd46a45d4353a873465c87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T02:02:01.421 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:02:00 smithi176 podman[73861]: 2024-09-16 02:02:00.969007647 +0000 UTC m=+0.321972908 container start f5cbfea2440bf0983a45f9c0fef99e375ccdc356b7bd46a45d4353a873465c87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T02:02:01.421 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:02:01 smithi176 podman[73861]: 2024-09-16 02:02:01.048826566 +0000 UTC m=+0.401791836 container attach f5cbfea2440bf0983a45f9c0fef99e375ccdc356b7bd46a45d4353a873465c87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2) 2024-09-16T02:02:01.437 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 16 02:02:01 smithi176 podman[73861]: 2024-09-16 02:02:01.156502807 +0000 UTC m=+0.509468076 container died f5cbfea2440bf0983a45f9c0fef99e375ccdc356b7bd46a45d4353a873465c87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e) 2024-09-16T02:02:01.777 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@osd.11.service' 2024-09-16T02:02:01.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:02:01.818 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-16T02:02:01.818 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-16T02:02:01.819 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.a 2024-09-16T02:02:02.117 INFO:journalctl@ceph.mds.a.smithi139.stdout:Sep 16 02:02:01 smithi139 systemd[1]: Stopping Ceph mds.a for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:02:02.422 INFO:journalctl@ceph.mds.a.smithi139.stdout:Sep 16 02:02:02 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-a[68838]: 2024-09-16T02:02:02.115+0000 7f14505d3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:02:02.423 INFO:journalctl@ceph.mds.a.smithi139.stdout:Sep 16 02:02:02 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-a[68838]: 2024-09-16T02:02:02.115+0000 7f14505d3640 -1 mds.a *** got signal Terminated *** 2024-09-16T02:02:05.592 INFO:journalctl@ceph.mds.a.smithi139.stdout:Sep 16 02:02:05 smithi139 podman[107969]: 2024-09-16 02:02:05.260394265 +0000 UTC m=+3.248128943 container died 73bf12d5392b95f0e54603f40bd6045882e5e5d670f2d5442aeda5d5b6e34d09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:02:05.917 INFO:journalctl@ceph.mds.a.smithi139.stdout:Sep 16 02:02:05 smithi139 podman[107969]: 2024-09-16 02:02:05.67357482 +0000 UTC m=+3.661309458 container cleanup 73bf12d5392b95f0e54603f40bd6045882e5e5d670f2d5442aeda5d5b6e34d09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-8921c8e) 2024-09-16T02:02:05.917 INFO:journalctl@ceph.mds.a.smithi139.stdout:Sep 16 02:02:05 smithi139 bash[107969]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-a 2024-09-16T02:02:06.040 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.a.service' 2024-09-16T02:02:06.076 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:02:06.077 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-16T02:02:06.077 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-16T02:02:06.077 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.d 2024-09-16T02:02:06.173 INFO:journalctl@ceph.mds.d.smithi139.stdout:Sep 16 02:02:06 smithi139 systemd[1]: Stopping Ceph mds.d for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:02:06.673 INFO:journalctl@ceph.mds.d.smithi139.stdout:Sep 16 02:02:06 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-d[67655]: 2024-09-16T02:02:06.393+0000 7facee9a0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:02:06.673 INFO:journalctl@ceph.mds.d.smithi139.stdout:Sep 16 02:02:06 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-d[67655]: 2024-09-16T02:02:06.393+0000 7facee9a0640 -1 mds.d *** got signal Terminated *** 2024-09-16T02:02:09.579 INFO:journalctl@ceph.mds.d.smithi139.stdout:Sep 16 02:02:09 smithi139 podman[108090]: 2024-09-16 02:02:09.276967935 +0000 UTC m=+2.983748806 container died ef25e7c09c08911f31dff7f4aac7b28a0c4c52625768b74212ac38b14af188cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:02:09.904 INFO:journalctl@ceph.mds.d.smithi139.stdout:Sep 16 02:02:09 smithi139 podman[108090]: 2024-09-16 02:02:09.660804737 +0000 UTC m=+3.367585618 container cleanup ef25e7c09c08911f31dff7f4aac7b28a0c4c52625768b74212ac38b14af188cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-16T02:02:09.904 INFO:journalctl@ceph.mds.d.smithi139.stdout:Sep 16 02:02:09 smithi139 bash[108090]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-d 2024-09-16T02:02:10.042 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.d.service' 2024-09-16T02:02:10.075 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:02:10.075 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-16T02:02:10.075 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-16T02:02:10.075 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.g 2024-09-16T02:02:10.423 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 02:02:10 smithi139 systemd[1]: Stopping Ceph mds.g for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:02:10.423 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 02:02:10 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-g[68430]: 2024-09-16T02:02:10.397+0000 7f27f8994640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:02:10.423 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 02:02:10 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-g[68430]: 2024-09-16T02:02:10.397+0000 7f27f8994640 -1 mds.g *** got signal Terminated *** 2024-09-16T02:02:13.566 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 02:02:13 smithi139 podman[108209]: 2024-09-16 02:02:13.277016363 +0000 UTC m=+2.976027614 container died 3e1b0cfb7c3c78852427252697e5ea1e507eff53fe5a29b645b9aff6b7c601c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_CLEAN=True) 2024-09-16T02:02:13.923 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 02:02:13 smithi139 podman[108209]: 2024-09-16 02:02:13.664243786 +0000 UTC m=+3.363255048 container cleanup 3e1b0cfb7c3c78852427252697e5ea1e507eff53fe5a29b645b9aff6b7c601c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-g, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T02:02:13.923 INFO:journalctl@ceph.mds.g.smithi139.stdout:Sep 16 02:02:13 smithi139 bash[108209]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-g 2024-09-16T02:02:14.095 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.g.service' 2024-09-16T02:02:14.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:02:14.128 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-16T02:02:14.128 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-16T02:02:14.128 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.j 2024-09-16T02:02:14.423 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 02:02:14 smithi139 systemd[1]: Stopping Ceph mds.j for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:02:14.923 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 02:02:14 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-j[67241]: 2024-09-16T02:02:14.451+0000 7fa40aab4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:02:14.923 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 02:02:14 smithi139 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-j[67241]: 2024-09-16T02:02:14.452+0000 7fa40aab4640 -1 mds.j *** got signal Terminated *** 2024-09-16T02:02:16.673 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 02:02:16 smithi139 podman[108330]: 2024-09-16 02:02:16.533594433 +0000 UTC m=+2.192771996 container died 2f7c0438797e951ada286950164ef9252360d602ae1342bf6566eebcb0406d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-j, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:02:17.078 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 02:02:16 smithi139 podman[108330]: 2024-09-16 02:02:16.893294195 +0000 UTC m=+2.552471715 container cleanup 2f7c0438797e951ada286950164ef9252360d602ae1342bf6566eebcb0406d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-j, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T02:02:17.078 INFO:journalctl@ceph.mds.j.smithi139.stdout:Sep 16 02:02:16 smithi139 bash[108330]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-j 2024-09-16T02:02:17.302 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.j.service' 2024-09-16T02:02:17.337 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:02:17.337 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-16T02:02:17.337 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-16T02:02:17.337 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.b 2024-09-16T02:02:17.637 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 02:02:17 smithi164 systemd[1]: Stopping Ceph mds.b for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:02:18.045 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 02:02:17 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b[64648]: 2024-09-16T02:02:17.635+0000 7fd708286640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:02:18.045 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 02:02:17 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b[64648]: 2024-09-16T02:02:17.635+0000 7fd708286640 -1 mds.b *** got signal Terminated *** 2024-09-16T02:02:21.545 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 02:02:21 smithi164 podman[73380]: 2024-09-16 02:02:21.25187566 +0000 UTC m=+3.717127410 container died f7cd4f522d4f14f2dc04e2b7686d6fd8bd8777bed913e368c895bb91da88c6b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T02:02:21.879 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 02:02:21 smithi164 podman[73380]: 2024-09-16 02:02:21.677196458 +0000 UTC m=+4.142448195 container cleanup f7cd4f522d4f14f2dc04e2b7686d6fd8bd8777bed913e368c895bb91da88c6b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-16T02:02:21.879 INFO:journalctl@ceph.mds.b.smithi164.stdout:Sep 16 02:02:21 smithi164 bash[73380]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-b 2024-09-16T02:02:22.097 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.b.service' 2024-09-16T02:02:22.132 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:02:22.133 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-16T02:02:22.133 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-16T02:02:22.133 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.e 2024-09-16T02:02:22.449 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 02:02:22 smithi164 systemd[1]: Stopping Ceph mds.e for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:02:22.795 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 02:02:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e[70744]: 2024-09-16T02:02:22.447+0000 7f02a54fb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:02:22.795 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 02:02:22 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e[70744]: 2024-09-16T02:02:22.447+0000 7f02a54fb640 -1 mds.e *** got signal Terminated *** 2024-09-16T02:02:26.474 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 02:02:26 smithi164 podman[73502]: 2024-09-16 02:02:26.165000766 +0000 UTC m=+3.819181931 container died 6f8b4444de136d03541dcaef3d4044b6c9a11cdfcce97d4c22540b477d5f5998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T02:02:26.795 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 02:02:26 smithi164 podman[73502]: 2024-09-16 02:02:26.572793824 +0000 UTC m=+4.226974979 container cleanup 6f8b4444de136d03541dcaef3d4044b6c9a11cdfcce97d4c22540b477d5f5998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-16T02:02:26.795 INFO:journalctl@ceph.mds.e.smithi164.stdout:Sep 16 02:02:26 smithi164 bash[73502]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-e 2024-09-16T02:02:26.997 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.e.service' 2024-09-16T02:02:27.030 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:02:27.030 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-16T02:02:27.031 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-16T02:02:27.031 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.h 2024-09-16T02:02:27.295 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 02:02:27 smithi164 systemd[1]: Stopping Ceph mds.h for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:02:27.795 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 02:02:27 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-h[64337]: 2024-09-16T02:02:27.351+0000 7fc702b80640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:02:27.795 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 02:02:27 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-h[64337]: 2024-09-16T02:02:27.351+0000 7fc702b80640 -1 mds.h *** got signal Terminated *** 2024-09-16T02:02:29.545 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 02:02:29 smithi164 podman[73621]: 2024-09-16 02:02:29.280171008 +0000 UTC m=+2.035535993 container died c0b5c2be9e172a10d157f6ffa3e039d6b5d0558327215ff86390f4d037763b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-h, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-16T02:02:29.913 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 02:02:29 smithi164 podman[73621]: 2024-09-16 02:02:29.627170119 +0000 UTC m=+2.382535102 container cleanup c0b5c2be9e172a10d157f6ffa3e039d6b5d0558327215ff86390f4d037763b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T02:02:29.913 INFO:journalctl@ceph.mds.h.smithi164.stdout:Sep 16 02:02:29 smithi164 bash[73621]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-h 2024-09-16T02:02:30.045 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.h.service' 2024-09-16T02:02:30.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:02:30.081 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-16T02:02:30.081 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-16T02:02:30.081 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.k 2024-09-16T02:02:30.545 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 02:02:30 smithi164 systemd[1]: Stopping Ceph mds.k for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:02:30.545 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 02:02:30 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-k[63720]: 2024-09-16T02:02:30.398+0000 7f56c0fb7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:02:30.545 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 02:02:30 smithi164 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-k[63720]: 2024-09-16T02:02:30.398+0000 7f56c0fb7640 -1 mds.k *** got signal Terminated *** 2024-09-16T02:02:33.516 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 02:02:33 smithi164 podman[73754]: 2024-09-16 02:02:33.256309517 +0000 UTC m=+2.948299780 container died e666239c623e88cb12b436f83b2732eb19512aac7ae5a29032849026c5d25c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True) 2024-09-16T02:02:33.795 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 02:02:33 smithi164 podman[73754]: 2024-09-16 02:02:33.61450534 +0000 UTC m=+3.306495611 container cleanup e666239c623e88cb12b436f83b2732eb19512aac7ae5a29032849026c5d25c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-k, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-16T02:02:33.795 INFO:journalctl@ceph.mds.k.smithi164.stdout:Sep 16 02:02:33 smithi164 bash[73754]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-k 2024-09-16T02:02:34.054 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.k.service' 2024-09-16T02:02:34.087 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:02:34.087 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-16T02:02:34.087 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-16T02:02:34.088 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.c 2024-09-16T02:02:34.436 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 02:02:34 smithi176 systemd[1]: Stopping Ceph mds.c for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:02:34.687 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 02:02:34 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-c[65254]: 2024-09-16T02:02:34.433+0000 7f4a3649a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:02:34.687 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 02:02:34 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-c[65254]: 2024-09-16T02:02:34.433+0000 7f4a3649a640 -1 mds.c *** got signal Terminated *** 2024-09-16T02:02:37.753 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 02:02:37 smithi176 podman[73963]: 2024-09-16 02:02:37.488421102 +0000 UTC m=+3.200422667 container died e845a934580cc5cbb4c758fff469ca2359f4ba077ef2ff76d2d3de49e76b7f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-c, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T02:02:38.187 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 02:02:37 smithi176 podman[73963]: 2024-09-16 02:02:37.834843638 +0000 UTC m=+3.546845193 container cleanup e845a934580cc5cbb4c758fff469ca2359f4ba077ef2ff76d2d3de49e76b7f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-c, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-16T02:02:38.187 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 02:02:37 smithi176 bash[73963]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-c 2024-09-16T02:02:38.187 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 16 02:02:37 smithi176 podman[73986]: 2024-09-16 02:02:37.99414014 +0000 UTC m=+0.498608566 container remove e845a934580cc5cbb4c758fff469ca2359f4ba077ef2ff76d2d3de49e76b7f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T02:02:38.213 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.c.service' 2024-09-16T02:02:38.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:02:38.250 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-16T02:02:38.251 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-16T02:02:38.251 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.f 2024-09-16T02:02:38.581 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 02:02:38 smithi176 systemd[1]: Stopping Ceph mds.f for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:02:38.937 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 02:02:38 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-f[64955]: 2024-09-16T02:02:38.578+0000 7f0263dcf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:02:38.937 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 02:02:38 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-f[64955]: 2024-09-16T02:02:38.578+0000 7f0263dcf640 -1 mds.f *** got signal Terminated *** 2024-09-16T02:02:41.781 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 02:02:41 smithi176 podman[74096]: 2024-09-16 02:02:41.477226988 +0000 UTC m=+3.011951785 container died 5b773f3b098191d535648a85cec01a270bef4dd38595ce8bf884223090a3be09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T02:02:42.131 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 02:02:41 smithi176 podman[74096]: 2024-09-16 02:02:41.854555435 +0000 UTC m=+3.389280233 container cleanup 5b773f3b098191d535648a85cec01a270bef4dd38595ce8bf884223090a3be09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T02:02:42.131 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 16 02:02:41 smithi176 bash[74096]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-f 2024-09-16T02:02:42.288 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.f.service' 2024-09-16T02:02:42.323 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:02:42.323 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-16T02:02:42.323 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-16T02:02:42.323 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.i 2024-09-16T02:02:42.687 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 02:02:42 smithi176 systemd[1]: Stopping Ceph mds.i for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:02:43.186 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 02:02:42 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-i[65555]: 2024-09-16T02:02:42.732+0000 7f0437201640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:02:43.187 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 02:02:42 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-i[65555]: 2024-09-16T02:02:42.732+0000 7f0437201640 -1 mds.i *** got signal Terminated *** 2024-09-16T02:02:45.777 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 02:02:45 smithi176 podman[74216]: 2024-09-16 02:02:45.488209723 +0000 UTC m=+2.951283925 container died 5acde05ee65539e973ac7a634164c21d86aa14cd05d219bd07d5d9144ca9b923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-09-16T02:02:46.135 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 02:02:45 smithi176 podman[74216]: 2024-09-16 02:02:45.875058836 +0000 UTC m=+3.338133041 container cleanup 5acde05ee65539e973ac7a634164c21d86aa14cd05d219bd07d5d9144ca9b923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-i, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T02:02:46.135 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 16 02:02:45 smithi176 bash[74216]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-i 2024-09-16T02:02:46.293 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.i.service' 2024-09-16T02:02:46.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:02:46.327 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-16T02:02:46.327 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-16T02:02:46.327 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.l 2024-09-16T02:02:46.654 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 02:02:46 smithi176 systemd[1]: Stopping Ceph mds.l for 1d90dd3a-73cb-11ef-bceb-c7b262605968... 2024-09-16T02:02:46.936 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 02:02:46 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-l[64652]: 2024-09-16T02:02:46.652+0000 7f540741c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T02:02:46.937 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 02:02:46 smithi176 ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-l[64652]: 2024-09-16T02:02:46.652+0000 7f540741c640 -1 mds.l *** got signal Terminated *** 2024-09-16T02:02:49.791 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 02:02:49 smithi176 podman[74338]: 2024-09-16 02:02:49.511095022 +0000 UTC m=+2.965215690 container died 7225b47e96d71bae99fe5193ab0e18048824905586422417b152993c247836b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T02:02:50.138 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 02:02:49 smithi176 podman[74338]: 2024-09-16 02:02:49.878379979 +0000 UTC m=+3.332500643 container cleanup 7225b47e96d71bae99fe5193ab0e18048824905586422417b152993c247836b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-l, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e) 2024-09-16T02:02:50.138 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 16 02:02:49 smithi176 bash[74338]: ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968-mds-l 2024-09-16T02:02:50.253 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1d90dd3a-73cb-11ef-bceb-c7b262605968@mds.l.service' 2024-09-16T02:02:50.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T02:02:50.285 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-16T02:02:50.286 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T02:02:50.494 INFO:teuthology.orchestra.run.smithi139.stdout:Deleting cluster with fsid: 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T02:02:53.151 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T02:02:53.354 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T02:02:55.778 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T02:02:55.981 INFO:teuthology.orchestra.run.smithi176.stdout:Deleting cluster with fsid: 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T02:02:58.372 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:02:58.404 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:02:58.433 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T02:02:58.459 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T02:02:58.461 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906474/remote/smithi139/crash 2024-09-16T02:02:58.461 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/crash -- . 2024-09-16T02:02:58.491 INFO:teuthology.orchestra.run.smithi139.stderr:tar: /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T02:02:58.491 INFO:teuthology.orchestra.run.smithi139.stderr:tar: Error is not recoverable: exiting now 2024-09-16T02:02:58.492 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906474/remote/smithi164/crash 2024-09-16T02:02:58.493 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/crash -- . 2024-09-16T02:02:58.545 INFO:teuthology.orchestra.run.smithi164.stderr:tar: /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T02:02:58.545 INFO:teuthology.orchestra.run.smithi164.stderr:tar: Error is not recoverable: exiting now 2024-09-16T02:02:58.547 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906474/remote/smithi176/crash 2024-09-16T02:02:58.548 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/crash -- . 2024-09-16T02:02:58.613 INFO:teuthology.orchestra.run.smithi176.stderr:tar: /var/lib/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T02:02:58.613 INFO:teuthology.orchestra.run.smithi176.stderr:tar: Error is not recoverable: exiting now 2024-09-16T02:02:58.614 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T02:02:58.615 DEBUG:teuthology.orchestra.run.smithi139:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-16T02:02:58.704 INFO:tasks.cephadm:Compressing logs... 2024-09-16T02:02:58.704 DEBUG:teuthology.orchestra.run.smithi139:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T02:02:58.707 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T02:02:58.710 DEBUG:teuthology.orchestra.run.smithi176:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T02:02:58.737 INFO:teuthology.orchestra.run.smithi139.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T02:02:58.738 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-16T02:02:58.738 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-16T02:02:58.739 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-16T02:02:58.739 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T02:02:58.741 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T02:02:58.741 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-16T02:02:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T02:02:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-16T02:02:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-16T02:02:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-16T02:02:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.log 2024-09-16T02:02:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-16T02:02:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-16T02:02:58.742 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-16T02:02:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.d.log: /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-16T02:02:58.743 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T02:02:58.744 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 87.3% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T02:02:58.744 INFO:teuthology.orchestra.run.smithi176.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T02:02:58.746 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T02:02:58.746 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T02:02:58.748 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-16T02:02:58.748 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-16T02:02:58.748 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T02:02:58.748 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-16T02:02:58.748 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-16T02:02:58.748 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-16T02:02:58.748 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.log 2024-09-16T02:02:58.748 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-16T02:02:58.750 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-16T02:02:58.751 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-16T02:02:58.751 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-16T02:02:58.751 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-16T02:02:58.751 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T02:02:58.751 INFO:teuthology.orchestra.run.smithi139.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-16T02:02:58.781 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 82.3% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T02:02:58.788 INFO:teuthology.orchestra.run.smithi164.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T02:02:58.789 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-16T02:02:58.789 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-16T02:02:58.790 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-16T02:02:58.790 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T02:02:58.790 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-16T02:02:58.790 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-16T02:02:58.791 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-16T02:02:58.791 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T02:02:58.791 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T02:02:58.791 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-16T02:02:58.792 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.log 2024-09-16T02:02:58.792 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-16T02:02:58.793 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-16T02:02:58.793 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-16T02:02:58.793 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T02:02:58.844 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T02:02:58.867 INFO:teuthology.orchestra.run.smithi139.stderr: 90.4% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T02:02:58.873 INFO:teuthology.orchestra.run.smithi176.stderr: 92.4% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-16T02:02:58.915 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.10.log: /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.1.log: /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.7.log: /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.b.log: /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.k.log: /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.4.log: /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.cephadm.log: /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mgr.y.log: /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.log: /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.h.log: /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mon.b.log: /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.e.log: 82.3% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T02:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr: 94.5% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-16T02:02:58.938 INFO:teuthology.orchestra.run.smithi139.stderr: 90.9% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T02:02:58.997 INFO:teuthology.orchestra.run.smithi139.stderr: 94.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T02:02:59.011 INFO:teuthology.orchestra.run.smithi139.stderr: 84.7% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T02:02:59.035 INFO:teuthology.orchestra.run.smithi176.stderr: 90.9% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T02:02:59.068 INFO:teuthology.orchestra.run.smithi164.stderr: 92.4% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-16T02:02:59.080 INFO:teuthology.orchestra.run.smithi164.stderr: 90.9% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T02:02:59.297 INFO:teuthology.orchestra.run.smithi164.stderr: 90.5% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T02:02:59.562 INFO:teuthology.orchestra.run.smithi176.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T02:02:59.565 INFO:teuthology.orchestra.run.smithi176.stderr: 87.4% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T02:02:59.568 INFO:teuthology.orchestra.run.smithi176.stderr: 90.5% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T02:03:00.424 INFO:teuthology.orchestra.run.smithi164.stderr: 87.4% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T02:03:04.304 INFO:teuthology.orchestra.run.smithi139.stderr: 90.4% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-16T02:03:06.174 INFO:teuthology.orchestra.run.smithi176.stderr: 91.7% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-16T02:03:06.519 INFO:teuthology.orchestra.run.smithi139.stderr: 90.2% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-16T02:03:13.609 INFO:teuthology.orchestra.run.smithi164.stderr: 91.7% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-16T02:03:15.200 INFO:teuthology.orchestra.run.smithi139.stderr: 94.1% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-16T02:03:45.560 INFO:teuthology.orchestra.run.smithi139.stderr: 95.5% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-16T02:03:56.871 INFO:teuthology.orchestra.run.smithi139.stderr: 95.3% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-16T02:06:12.557 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-16T02:06:29.489 INFO:teuthology.orchestra.run.smithi164.stderr: 95.2% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-16T02:06:57.136 INFO:teuthology.orchestra.run.smithi164.stderr: 95.5% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-16T02:08:04.328 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-16T02:08:44.185 INFO:teuthology.orchestra.run.smithi176.stderr: 95.3% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-16T02:08:46.892 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-16T02:08:55.164 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-16T02:08:55.167 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-09-16T02:08:55.167 INFO:teuthology.orchestra.run.smithi139.stderr:real 5m56.446s 2024-09-16T02:08:55.167 INFO:teuthology.orchestra.run.smithi139.stderr:user 5m45.592s 2024-09-16T02:08:55.167 INFO:teuthology.orchestra.run.smithi139.stderr:sys 0m13.928s 2024-09-16T02:11:29.597 INFO:teuthology.orchestra.run.smithi164.stderr: 91.4% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-16T02:17:55.284 INFO:teuthology.orchestra.run.smithi164.stderr: 91.3% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-16T02:25:09.899 INFO:teuthology.orchestra.run.smithi176.stderr: 91.4% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-16T02:25:46.501 INFO:teuthology.orchestra.run.smithi164.stderr: 91.2% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-16T02:28:27.183 INFO:teuthology.orchestra.run.smithi176.stderr: 94.4% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-16T02:28:27.936 INFO:teuthology.orchestra.run.smithi164.stderr: 91.4% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-16T02:28:39.155 INFO:teuthology.orchestra.run.smithi176.stderr: 91.4% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-16T02:32:16.620 INFO:teuthology.orchestra.run.smithi164.stderr: 94.1% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-16T02:32:21.912 INFO:teuthology.orchestra.run.smithi176.stderr: 91.4% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-16T02:32:41.840 INFO:teuthology.orchestra.run.smithi176.stderr: 94.3% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-16T02:33:01.684 INFO:teuthology.orchestra.run.smithi164.stderr: 94.4% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-16T02:33:01.731 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-09-16T02:33:01.731 INFO:teuthology.orchestra.run.smithi164.stderr:real 30m3.008s 2024-09-16T02:33:01.731 INFO:teuthology.orchestra.run.smithi164.stderr:user 10m12.183s 2024-09-16T02:33:01.732 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m30.932s 2024-09-16T02:34:40.484 INFO:teuthology.orchestra.run.smithi176.stderr: 91.1% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-16T02:34:55.094 INFO:teuthology.orchestra.run.smithi176.stderr: 94.6% -- replaced with /var/log/ceph/1d90dd3a-73cb-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-16T02:34:55.119 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-16T02:34:55.120 INFO:teuthology.orchestra.run.smithi176.stderr:real 31m56.394s 2024-09-16T02:34:55.120 INFO:teuthology.orchestra.run.smithi176.stderr:user 11m23.236s 2024-09-16T02:34:55.120 INFO:teuthology.orchestra.run.smithi176.stderr:sys 0m33.561s 2024-09-16T02:34:55.120 INFO:tasks.cephadm:Archiving logs... 2024-09-16T02:34:55.121 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906474/remote/smithi139/log 2024-09-16T02:34:55.123 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T02:35:56.184 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906474/remote/smithi164/log 2024-09-16T02:35:56.185 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T02:37:31.958 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906474/remote/smithi176/log 2024-09-16T02:37:32.186 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T02:39:22.492 INFO:tasks.cephadm:Removing cluster... 2024-09-16T02:39:22.493 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 --force 2024-09-16T02:39:22.696 INFO:teuthology.orchestra.run.smithi139.stdout:Deleting cluster with fsid: 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T02:39:23.438 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 --force 2024-09-16T02:39:24.165 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T02:39:25.075 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1d90dd3a-73cb-11ef-bceb-c7b262605968 --force 2024-09-16T02:39:25.579 INFO:teuthology.orchestra.run.smithi176.stdout:Deleting cluster with fsid: 1d90dd3a-73cb-11ef-bceb-c7b262605968 2024-09-16T02:39:26.497 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T02:39:26.497 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T02:39:26.518 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T02:39:26.647 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T02:39:26.664 INFO:tasks.cephadm:Teardown complete 2024-09-16T02:39:26.664 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T02:39:26.677 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T02:39:26.678 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T02:39:26.680 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T02:39:26.683 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T02:39:26.851 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T02:39:26.862 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T02:39:26.863 DEBUG:teuthology.orchestra.run.smithi139:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T02:39:26.865 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T02:39:26.868 DEBUG:teuthology.orchestra.run.smithi176:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T02:39:26.880 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-09-16T02:39:26.882 INFO:teuthology.orchestra.run.smithi176.stderr:bash: line 1: ntpq: command not found 2024-09-16T02:39:26.883 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-09-16T02:39:26.923 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T02:39:26.924 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-09-16T02:39:26.924 INFO:teuthology.orchestra.run.smithi139.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 -431us[ -420us] +/- 40ms 2024-09-16T02:39:26.924 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 20 +746us[ +746us] +/- 68ms 2024-09-16T02:39:26.924 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +66us[ +66us] +/- 69ms 2024-09-16T02:39:26.924 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T02:39:26.956 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T02:39:26.956 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-09-16T02:39:26.956 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 -458us[ -439us] +/- 40ms 2024-09-16T02:39:26.956 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 +724us[ +743us] +/- 68ms 2024-09-16T02:39:26.957 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +37us[ +56us] +/- 69ms 2024-09-16T02:39:26.957 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T02:39:26.964 INFO:teuthology.orchestra.run.smithi176.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T02:39:26.964 INFO:teuthology.orchestra.run.smithi176.stdout:=============================================================================== 2024-09-16T02:39:26.964 INFO:teuthology.orchestra.run.smithi176.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 -432us[ -411us] +/- 40ms 2024-09-16T02:39:26.965 INFO:teuthology.orchestra.run.smithi176.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 +741us[ +762us] +/- 68ms 2024-09-16T02:39:26.965 INFO:teuthology.orchestra.run.smithi176.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +64us[ +64us] +/- 69ms 2024-09-16T02:39:26.965 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T02:39:26.979 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T02:39:26.990 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T02:39:26.990 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T02:39:27.015 DEBUG:teuthology.orchestra.run.smithi139:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T02:39:27.160 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T02:39:27.301 DEBUG:teuthology.orchestra.run.smithi176:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T02:39:27.452 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T02:39:27.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:39:27.486 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T02:39:27.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:39:27.518 DEBUG:teuthology.orchestra.run.smithi176:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T02:39:27.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:39:27.547 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T02:39:27.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T02:39:27.567 INFO:teuthology.task.internal:Duration was 4540.967300 seconds 2024-09-16T02:39:27.568 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T02:39:27.578 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T02:39:27.578 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T02:39:27.581 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T02:39:27.583 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T02:39:27.640 INFO:teuthology.orchestra.run.smithi176.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T02:39:27.641 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T02:39:27.656 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T02:39:28.016 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T02:39:28.016 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T02:39:28.016 DEBUG:teuthology.orchestra.run.smithi139:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T02:39:28.039 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T02:39:28.040 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T02:39:28.070 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi176.front.sepia.ceph.com 2024-09-16T02:39:28.070 DEBUG:teuthology.orchestra.run.smithi176:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T02:39:28.107 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T02:39:28.107 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T02:39:28.110 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T02:39:28.113 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T02:39:28.174 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T02:39:28.174 DEBUG:teuthology.orchestra.run.smithi139:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T02:39:28.180 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T02:39:28.194 DEBUG:teuthology.orchestra.run.smithi176:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T02:39:29.028 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T02:39:29.073 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T02:39:29.073 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T02:39:29.099 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T02:39:29.129 DEBUG:teuthology.orchestra.run.smithi176:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T02:39:29.158 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T02:39:29.171 DEBUG:teuthology.orchestra.run.smithi139:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:39:29.173 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:39:29.176 DEBUG:teuthology.orchestra.run.smithi176:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:39:29.208 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-09-16T02:39:29.210 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = core 2024-09-16T02:39:29.243 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = core 2024-09-16T02:39:29.289 DEBUG:teuthology.orchestra.run.smithi139:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:39:29.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:39:29.304 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:39:29.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:39:29.321 DEBUG:teuthology.orchestra.run.smithi176:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T02:39:29.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T02:39:29.345 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T02:39:29.356 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T02:39:29.357 DEBUG:teuthology.misc:Transferring archived files from smithi139:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906474/remote/smithi139 2024-09-16T02:39:29.358 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T02:39:29.422 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906474/remote/smithi164 2024-09-16T02:39:29.422 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T02:39:29.461 DEBUG:teuthology.misc:Transferring archived files from smithi176:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906474/remote/smithi176 2024-09-16T02:39:29.461 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T02:39:29.504 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T02:39:29.504 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T02:39:29.507 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T02:39:29.509 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T02:39:29.552 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T02:39:29.566 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T02:39:29.567 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T02:39:29.580 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T02:39:29.581 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T02:39:29.583 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T02:39:29.586 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T02:39:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: 265885 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 02:39 /home/ubuntu/cephtest 2024-09-16T02:39:29.600 INFO:teuthology.orchestra.run.smithi164.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 02:39 /home/ubuntu/cephtest 2024-09-16T02:39:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 02:39 /home/ubuntu/cephtest 2024-09-16T02:39:29.610 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T02:39:29.621 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T02:39:29.648 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 4540.967299699783 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-16T02:39:29.648 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T02:39:29.732 INFO:teuthology.run:pass